In our project I have several JUnit tests that e.g. take every file from a directory and run a test on it. If I implement a testEveryFileInDirectory method in the TestCase this shows up as only one test that may fail or succeed. But I am interested in the results on each individual file. How can I write a TestCase / TestSuite such that each file shows up as a separate test e.g. in the graphical TestRunner of Eclipse? (Coding an explicit test method for each file is not an option.)
Compare also the question ParameterizedTest with a name in Eclipse Testrunner.
Take a look at Parameterized Tests in JUnit 4.
Actually I did this a few days ago. I'll try to explain ...
First build your test class normally, as you where just testing with one input file.
Decorate your class with:
#RunWith(Parameterized.class)
Build one constructor that takes the input that will change in every test call (in this case it may be the file itself)
Then, build a static method that will return a Collection of arrays. Each array in the collection will contain the input arguments for your class constructor e.g. the file. Decorate this method with:
#Parameters
Here's a sample class.
#RunWith(Parameterized.class)
public class ParameterizedTest {
private File file;
public ParameterizedTest(File file) {
this.file = file;
}
#Test
public void test1() throws Exception { }
#Test
public void test2() throws Exception { }
#Parameters
public static Collection<Object[]> data() {
// load the files as you want
Object[] fileArg1 = new Object[] { new File("path1") };
Object[] fileArg2 = new Object[] { new File("path2") };
Collection<Object[]> data = new ArrayList<Object[]>();
data.add(fileArg1);
data.add(fileArg2);
return data;
}
}
Also check this example
JUnit 3
public class XTest extends TestCase {
public File file;
public XTest(File file) {
super(file.toString());
this.file = file;
}
public void testX() {
fail("Failed: " + file);
}
}
public class XTestSuite extends TestSuite {
public static Test suite() {
TestSuite suite = new TestSuite("XTestSuite");
File[] files = new File(".").listFiles();
for (File file : files) {
suite.addTest(new XTest(file));
}
return suite;
}
}
JUnit 4
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.Parameterized;
import org.junit.runners.Parameterized.Parameters;
#RunWith(Parameterized.class)
public class TestY {
#Parameters
public static Collection<Object[]> getFiles() {
Collection<Object[]> params = new ArrayList<Object[]>();
for (File f : new File(".").listFiles()) {
Object[] arr = new Object[] { f };
params.add(arr);
}
return params;
}
private File file;
public TestY(File file) {
this.file = file;
}
#Test
public void testY() {
fail(file.toString());
}
}
Junit 5 Parameterized Tests
JUnit 5 parameterized tests support this by allowing the use of a method as data source:
#ParameterizedTest
#MethodSource("fileProvider")
void testFile(File f) {
// Your test comes here
}
static Stream<File> fileProvider() {
return Arrays.asList(new File(".").list()).stream();
}
JUnit 5 DynamicTests
JUnit 5 also supports this through the notion of a DynamicTest, which is to be generated in a #TestFactory, by means of the static method dynamicTest.
import org.junit.jupiter.api.DynamicTest;
import org.junit.jupiter.api.TestFactory;
import static org.junit.jupiter.api.DynamicTest.dynamicTest;
import java.util.stream.Stream;
#TestFactory
public Stream<DynamicTest> testFiles() {
return Arrays.asList(new File(".").list())
.stream()
.map((file) -> dynamicTest(
"Test for file: " + file,
() -> { /* Your test comes here */ }));
}
The tests run in your IDE (IntelliJ here) will be displayed like this:
Should be possible in JUnit 3 by inheriting from TestSuite and overriding the tests() method to list the files and for each return an instance of a subclass of TestCase that takes the filename as constructor parameter and has a test method that tests the file given in the constructor.
In JUnit 4 it might be even easier.
You could consider using JUnitParams library, so you would have a few more (cleaner) options:
#org.junit.runner.RunWith(junitparams.JUnitParamsRunner.class)
public class ParameterizedTest {
#org.junit.Test
#junitparams.Parameters(method = "data")
public void test1(File file) throws Exception { }
#org.junit.Test
#junitparams.Parameters(method = "data")
public void test2(File file) throws Exception { }
public static File[] data() {
return new File[] { new File("path1"), new File("path2") };
}
}
#org.junit.runner.RunWith(junitparams.JUnitParamsRunner.class)
public class ParameterizedTest {
#org.junit.Test
#junitparams.Parameters(value = { "path1", "path2" })
public void test1(String path) throws Exception {
File file = new File(path);
}
#org.junit.Test
#junitparams.Parameters(value = { "path1", "path2" })
public void test2(String path) throws Exception {
File file = new File(path);
}
}
You can see more samples of usage here.
In addition about JUnitParams, why writting parameterized tests with it is easier and more readable:
JUnitParams project adds a new runner to JUnit and provides much
easier and readable parametrised tests for JUnit >=4.6.
Main differences to standard JUnit Parametrised runner:
more explicit - params are in test method params, not class fields
less code - you don't need a constructor to set up parameters
you can mix parametrised with non-parametrised methods in one class
params can be passed as a CSV string or from a parameters provider class
parameters provider class can have as many parameters providing methods as you want, so that you can group different cases
you can have a test method that provides parameters (no external classes or statics anymore)
you can see actual parameter values in your IDE (in JUnit's Parametrised it's only consecutive numbers of parameters)
If TestNG is an option, you could use Parameters with DataProviders.
Each individual file's test will have its result shown in the text-based report or Eclipse's TestNG plugin UI. The number of total tests run will count each of your files individually.
This behavior differs from JUnit Theories, in which all results are lumped under one "theory" entry and only count as 1 test. If you want separate result reporting in JUnit, you can try Parameterized Tests.
Test and inputs
public class FileTest {
#DataProvider(name="files")
public File[][] getFiles(){
return new File[][] {
{ new File("file1") },
{ new File("file2") }
};
// or scan a directory
}
#Test(dataProvider="files")
public void testFile(File file){
//run tests on file
}
}
Example output
PASSED: testFile(file1)
PASSED: testFile(file2)
===============================================
Default test
Tests run: 2, Failures: 0, Skips: 0
===============================================
I had a similar problem and ended up writing a simple JUnit 4 runner that allows med to dynamically generate tests.
https://github.com/kimble/junit-test-factory
Related
I have a jUnit 4/5-project and I run my test classes via test suites.
I use the com.github.peterwippermann.junit4.parameterizedsuite.ParameterizedSuite.class runner for the annotation "RunWith()" at the test suites but my test classes are use with same annotation the JUnitPlatform.class runner.
For this reason there it's not possible to read the - in the test suite - declared parameters (#Parameters) in my test classes. It works fine, if I use the #RunWith(Parameterized.class) runner at my test classes, but my goal is to use the junit 5 JUnitPlatform.class runner at my test classes and the jUnit 4 ParameterizedSuite.class runner to parameterize my test suites.
E.g. my test suite:
package TestSuite.Dummy;
//...some imports...
#RunWith(ParameterizedSuite.class)
#SuiteClasses({
T5.class
})
public class TestSuite_Dummy_01
{
public static Object[] parameters = new Object[] {"A", "B", "C"};
#Parameters(name = "Parameter of suite are {0}")
public static Object[] params() throws NoSuchMethodException, SecurityException
{
System.out.println(parameters.length);
System.out.println(parameters.getClass().getTypeName());
return parameters;
}
#Parameter(0)
public static String myStringParameter;
public static XYZQA QAE = null;
public static XYZQA getXYZQA()
{
// Create a new XYZQA object
try
{
QAE = new XYZQA();
System.out.println("QAE created");
return QAE;
}
catch(MalformedURLException e)
{
e.printStackTrace();
return null;
}
}
public static String getParameter()
{
System.out.println("Parameter available?" + ParameterContext.isParameterSet());
return myStringParameter;
}
}
And my example test class:
package Main.Demo;
//...some imports...
#RunWith(JUnitPlatform.class) //is necessary to run jUnit 5 tests with non-junit5-implemented IDEs
#TestInstance(TestInstance.Lifecycle.PER_CLASS)
#ExtendWith(ClientDescriptionParameterResolver.class)
public class T5
{
public static XYZQA QAE;
public static String parameters;
public T5()
{
//...
}
#BeforeAll
static void setUpBeforeClass() throws Exception
{
QAE = TestSuite_Dummy_01.getXYZQA();
parameters = TestSuite_Dummy_01.getParameter();
System.out.println("Parameters from Suite: " + parameters);
}
#Test
public void test01(ClientDescription client)
{
//...
}
//...
}
Question:
The following code line:
QAE = TestSuite_Dummy_01.getXYZQA(); //to import/use the QAE object of the test suite class into the current test class
Blockquote
works fine for me, but it's fixed defined to the one test suite class "TestSuite_Dummy_01". But all of my test suites should worked with my test classes and not only this test suite. My idea is to use the current instance of a running test suite class (runner) of a test class instead of the fixed TestSuite_Dummy_01.class.
HOW I can get the current instance of a test class run and get the reference to the respective test suite class wihtin the test class?
This question already has answers here:
Excluding a non param test in parameterized test class
(9 answers)
Closed 6 years ago.
I use #Parameterized in many cases to run tests on a number of permutations. This works very well and keeps the test-code itself simple and clean.
However sometimes I would like to have some of the test-methods still run only once as they do not make use of the parameters, is there a way with JUnit to mark the test-method as "singleton" or "run-once"?
Note: This does not concern running single tests in Eclipse, I know how to do that :)
You could structure your test with the Enclosed runner.
#RunWith(Enclosed.class)
public class TestClass {
#RunWith(Parameterized.class)
public static class TheParameterizedPart {
#Parameters
public static Object[][] data() {
...
}
#Test
public void someTest() {
...
}
#Test
public void anotherTest() {
...
}
}
public static class NotParameterizedPart {
#Test
public void someTest() {
...
}
}
}
You can associate any number of test classes to run together using a suite. This way all the tests are run when you test your class and you can mix different test runners.
Create a test suite associated with the class you are testing
Add a reference to the parameterized test class
Add the other class(es) containing non parameterized tests.
import org.junit.runners.Suite;
import org.junit.runner.RunWith;
#RunWith(Suite.class)
#Suite.SuiteClasses({ParameterizedTestClass.class, UnitTests.class, MoreUnitTests.class})
public class SutTestSuite{
//Empty...
}
there is a number of junit plugins that give you some more features/power regarding parameterized tests. check zohhak, junit-parames and junit-dataprovider. they allow you to mix parametrized and simple junit tests
Before I knew about "#RunWith(Enclosed.class)" approach, I used the following (similar) solution, with inner classes extending outer class. I keep using this structure because I like that the tests are in same place and share some properties and methods and things seems clearer to me. Then, using Eclipse, in my run configuration, I choose that option "Run all tests in the selected project, package or source folder" and all these tests will be performed with just a click.
public class TestBooksDAO {
private static BooksDAO dao;
#Parameter(0)
public String title;
#Parameter(1)
public String author;
#Before
public void init() {
dao = BooksDAO.getInstancia();
}
/** Tests that run only once. */
public static class SingleTests extends TestBooksDAO {
#Test(timeout=10000)
public void testGetAll() {
List<Book> books = dao.getBooks();
assertNotNull(books);
assertTrue(books.size()>0);
}
#Test(timeout=10000)
public void testGetNone() {
List<Book> books = dao.getBooks(null);
assertNull(books);
}
}
/** Tests that run for each set of parameters. */
#RunWith(Parameterized.class)
public static class ParameterizedTests1 extends TestBooksDAO {
#Parameters(name = "{index}: author=\"{2}\"; title=\"{0}\";")
public static Collection<Object[]> values() {
return Arrays.asList(new Object[][] {
{"title1", ""},
{"title2", ""},
{"title3", ""},
{"title4", "author1"},
{"title5", "author2"},
});
}
#Test(timeout=10000)
public void testGetOneBook() {
Book book = dao.getBook(author, title);
assertNotNull(book);
}
}
/** Other parameters for different tests. */
#RunWith(Parameterized.class)
public static class ParameterizedTests2 extends TestBooksDAO {
#Parameters(name = "{index}: author=\"{2}\";")
public static Collection<Object[]> values() {
return Arrays.asList(new Object[][] {
{"", "author1"},
{"", "author2"},
{"", "author3"},
});
}
#Test(timeout=10000)
public void testGetBookList() {
List<Book> books = dao.getBookByAuthor(author);
assertNotNull(books);
assertTrue(books.size()>0);
}
}
}
I want to create 2 JUnit TestSuites. They both utilize the same test classes, but they should each use different parameters. For example, in test suite A, I want my data to be collected from file A and to be written to database A. In test suite B, I want my data to be collected from file B and to be written to databaseB.
The reason I use testSuites for this is because:
I can put all the specific parameters in the testsuite classes
I can reuse the testclasses
I can choose which testsuite to run. I do not want all tests to always run with all possible paramaters!
The problem is I cannot really pass the parameters. I understand the way the Parameterized class works with JUnit, but it does not allow point 3 in the list above. If I use the code below it will run my test class with both databse connections, which is not what I want to achieve.
#RunWith(value = Parameterized.class)
public class TestCheckData
{
private File file;
private DatabaseSource databaseSource;
public TestCheckData(File file, DatabaseSource databaseSource)
{
this.file = file;
this.databaseSource = databaseSource;
}
#Parameters
public static Iterable<Object[]> data1()
{
return Arrays.asList(new Object[][]
{
{ TestSuiteA.DATA_FILE_A, TestSuite1.DATABASE_A },
{ TestSuiteB.DATA_FILE_B, TestSuite1.DATABASE_B }
});
}
I already find some way of passing configurations in a spring context in this question, but I'm not using any special framework.
Well, this would be a little unconventional, but you could add a different Test class to the beginning of each suite run that would set the parameters you want to use for that test. So you'd have classes like:
public abstract class StaticParameters {
public static File dataFileToUse = null;
public static DatabaseSource databaseToUse = null;
}
public class Suite1Params extends StaticParameters {
#BeforeClass
public static void setParams() {
dataFileToUse = DATA_FILE_A;
databaseToUse = DATABASE_A;
}
}
public class Suite2Params extends StaticParameters {
#BeforeClass
public static void setParams() {
dataFileToUse = DATA_FILE_B;
databaseToUse = DATABASE_B;
}
}
Then you'd just make Suite1Params or Suite2Params the first in your suite list. You might have to add a fake #Test entry to the params classes, I'm not sure if the Suite runner requires that.
You could modify the tests so that they get the parameters from a config file. This way you would always only have 1 Suite.
The path of the config file can be looked up via a System property.
Then on the invocation of the test suite, you could pass in a different config file by changing the property using the -D option on the JVM.
So for example if you named the proprerty env.properties then your command would be:
%java -Denv.properties=prod.config runMyTests
or
%java -Denv.properties=dev.config runMyTests
etc
I have to run junit test from command line and one of the guy in the team created junit classes like below:
public Test extends TestCore
{
String some;
public Test(String some)
{
this.some = some;
}
//some test here
}
this work from the eclipse but doesn't from command line.
The result of execution this kind of file gave me error like below:
Test class should have exactly one public zero-argument constructor.
Anyone could help me?
Cheers Jaroslaw.
Eclipse uses a different testrunner. Maybe the parameterized constructors are caused by TestCore being a parameterized test, e.g. like this:
#RunWith(Parameterized.class)
public class TestCore {
String someThatWillBeHidden;
public TestCore(String some) {
this.someThatWillBeHidden = some;
}
#Parameters
public static List<Object[]> data() {
Object[][] data = new Object[][] { {"Hello"}, {" "}, {"world"}};
return Arrays.asList(data);
}
//some test here
}
So which version of junit are you using?
I basically want to run all JUnit unit tests in my IntelliJ IDEA project (excluding JUnit integration tests), using the static suite() method of JUnit. Why use the static suite() method? Because I can then use IntelliJ IDEA's JUnit test runner to run all unit tests in my application (and easily exclude all integration tests by naming convention). The code so far looks like this:
package com.acme;
import junit.framework.Test;
import junit.framework.TestCase;
import junit.framework.TestSuite;
import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;
public class AllUnitTests extends TestCase {
public static Test suite() {
List classes = getUnitTestClasses();
return createTestSuite(classes);
}
private static List getUnitTestClasses() {
List classes = new ArrayList();
classes.add(CalculatorTest.class);
return classes;
}
private static TestSuite createTestSuite(List allClasses) {
TestSuite suite = new TestSuite("All Unit Tests");
for (Iterator i = allClasses.iterator(); i.hasNext();) {
suite.addTestSuite((Class<? extends TestCase>) i.next());
}
return suite;
}
}
The method getUnitTestClasses() should be rewritten to add all project classes extending TestCase, except if the class name ends in "IntegrationTest".
I know I can do this easily in Maven for example, but I need to do it in IntelliJ IDEA so I can use the integrated test runner - I like the green bar :)
How about putting each major group of junit tests into their own root package. I use this package structure in my project:
test.
quick.
com.acme
slow.
com.acme
Without any coding, you can set up IntelliJ to run all tests, just the quick ones or just the slow ones.
I've written some code to do most of the work. It works only if your files are on the local disk instead of in a JAR. All you need is one class in the package. You could, for this purpose, create a Locator.java class, just to be able to find the package.
public class ClassEnumerator {
public static void main(String[] args) throws ClassNotFoundException {
List<Class<?>> list = listClassesInSamePackage(Locator.class, true);
System.out.println(list);
}
private static List<Class<?>> listClassesInSamePackage(Class<?> locator, boolean includeLocator)
throws ClassNotFoundException {
File packageFile = getPackageFile(locator);
String ignore = includeLocator ? null : locator.getSimpleName() + ".class";
return toClassList(locator.getPackage().getName(), listClassNames(packageFile, ignore));
}
private static File getPackageFile(Class<?> locator) {
URL url = locator.getClassLoader().getResource(locator.getName().replace(".", "/") + ".class");
if (url == null) {
throw new RuntimeException("Cannot locate " + Locator.class.getName());
}
try {
return new File(url.toURI()).getParentFile();
}
catch (URISyntaxException e) {
throw new RuntimeException(e);
}
}
private static String[] listClassNames(File packageFile, final String ignore) {
return packageFile.list(new FilenameFilter(){
#Override
public boolean accept(File dir, String name) {
if (name.equals(ignore)) {
return false;
}
return name.endsWith(".class");
}
});
}
private static List<Class<?>> toClassList(String packageName, String[] classNames)
throws ClassNotFoundException {
List<Class<?>> result = new ArrayList<Class<?>>(classNames.length);
for (String className : classNames) {
// Strip the .class
String simpleName = className.substring(0, className.length() - 6);
result.add(Class.forName(packageName + "." + simpleName));
}
return result;
}
}
What about using JUnit4 and the Suite-Runner?
Example:
#RunWith(Suite.class)
#Suite.SuiteClasses({
UserUnitTest.class,
AnotherUnitTest.class
})
public class UnitTestSuite {}
I made a small Shell-Script to find all Unit-Tests and another one to find my Integration-Tests. Have a look at my blog entry:
http://blog.timomeinen.de/2010/02/find-all-junit-tests-in-a-project/
If you use Spring TestContext you can use the #IfProfile Annotation to declare different tests.
Kind regards,
Timo Meinen
Spring has implemented an excellent classpath search function in the PathMatchingResourcePatternResolver. If you use the classpath*: prefix, you can find all the resources, including classes in a given hierarchy, and even filter them if you want. Then you can use the children of AbstractTypeHierarchyTraversingFilter, AnnotationTypeFilter and AssignableTypeFilter to filter those resources either on class level annotations or on interfaces they implement.
http://static.springsource.org/spring/docs/2.0.x/api/org/springframework/core/io/support/PathMatchingResourcePatternResolver.html
http://static.springsource.org/spring/docs/2.5.x/api/org/springframework/core/type/filter/AbstractTypeHierarchyTraversingFilter.html
Solution: https://github.com/MichaelTamm/junit-toolbox
Use the following features
#RunWith(WildcardPatternSuite.class)
#SuiteClasses({"**/*.class", "!**/*IntegrationTest.class"})
public class AllTestsExceptionIntegrationSuit {
}
assuming you following a naming pattern where you integration tests end in ...IntegrationTest and you place the file in the top-most package (so the **/*.class search will have the opportunity to pick up all your tests)