Develop Reference

Setup JUnit test timeout using reflection

I have found so far 2 ways to setup JUnit test timeout. Either using:
#Test(timeout=XXX)
Or using something like:
#ClassRule
public static Timeout timeoutRule = new Timeout(XXX, TimeUnit.MILLISECONDS);
In my case, I have a Test Runner as a main class to run all my test suites, so I can execute the tests as an executable jar.
I'd like this runner to setup the timeouts dinamically using reflection.
Is it possible to do?

You could add the timeout feature to a custom test runner like so:
public class TimeoutTestRunner extends BlockJUnit4ClassRunner {
public TimeoutTestRunner(Class<?> clazz) throws InitializationError {
super(clazz);
}
#Override
protected Statement withPotentialTimeout(FrameworkMethod method, Object test, Statement next) {
return FailOnTimeout.builder()
// you'll probably want to configure/inject this value rather than hardcode it ...
.withTimeout(1, TimeUnit.MILLISECONDS)
.build(next);
}
}
Using this test runner the tests in the following test case ...
#RunWith(TimeoutTestRunner.class)
public class YourTest {
#Test
public void willTimeout() throws InterruptedException {
Thread.sleep(50);
assertTrue(true);
}
#Test
public void willNotTimeout() throws InterruptedException {
assertTrue(true);
}
}
... will behave as follows:
willTimeout: will fail with a TestTimedOutException
willNotTimeout: will pass
Although you will need your tests to be run via this runner you will be able to control their timeout setting from one place and provide custom timeout derivation strategies such as if test name matches <some regex> then timeout is x else ....

blend parameterized and programmatic unit tests

With JUnit you can use #RunWith(Parameterized.class) to provide a set of parameters to pass to the test constructor and then run tests with each object.
I'm trying to move as much test logic as possible into data, but there are some tests that won't easily be converted into data-driven tests. Is there a way to use JUnit's Parameterized runner to run some tests with parameters, and then also add non-data-driven tests that aren't run repeatedly for each test object construction?

My workaround for this was to create a single class and place the programmatic and data-driven tests in two separate sub-classes. A sub-class must be static for JUnit to run its tests. Here's a skeleton:
#RunWith(Enclosed.class) // needed for working well with Ant
public class MyClassTests {
public static class Programmatic {
#Test
public void myTest(){
// test something here
}
}
#RunWith(Parameterized.class)
public static class DataDriven {
#Parameters
public static Collection<Object[]> getParams() {
return Collections.emptyList();
}
private String data;
public DataDriven(String testName, String data){
this.data = data;
}
#Test
public void test() throws AnalyzeExceptionEN{
// test data string here
}
}
}

one way is to use Junit's Enclosed runner. it's very verbose but also pretty powerful. it allows you to combine multiple different runners in one file.
other option is to use custom junit runner. for sure zohhak supports tests with parameters and without. small extract:
#RunWith(ZohhakRunner.class)
public class CoercingTest {
#TestWith("ONE_OF_ENUM_VALUES")
public void should_coerce_enum(SampleEnum param) {
assertThat(param).isEqualTo(SampleEnum.ONE_OF_ENUM_VALUES);
}
#Test
public void should_run_standard_junit_test() {
//this will also work
}
}
if it's not enough for you, for sure you can find other runners that support both kind of tests.

How to run specific junit tests at run time

I'm trying to figure out a good solution to having specific unit tests run with certain runtime configurations. For example:
public class TestClassAlpha() {
#setup
public void setup() {
}
#After
public void tearDown() {
}
#Test
#<only run in particular env>
public void testA() {
//whatever A
}
//always run below test no mater what env
#Test
public void testB() {
//whatever B
}
}
I am contemplating a custom annotation or custom rule perhaps, but i thought this has to be a question that comes up frequently as running tests in certain conditions (envs) is a very valid scenario. I did some limited searching within stack, and I didn't find anything that really that helped solidify either way.

This post shows you exactly what you require.
You should Write a Custom TestRule and an annotation to mark the condition.

Loading configuration only once before a set of test cases

I have a java package which contains all my test case classes. Each class contains a lot of test cases. Each class is loaded and run one by one by JUnit. However each of the classes contains common configuration code which is run again and again and initialised everytime each of the classes are run.
These initializations take a lot of time.
Is there some way to load these configuration changes first and then run the test case so that I do not need to load them everytime.

JUnit4 has #BeforeClass annotation.
Just do something like this:
public class TestClass {
private static SomeConnection connection;
#BeforeClass
public static void setUp() {
//do common setup
connection = new SomeConnection();
}
#Test
public void testSomething() { }
#Test
public void testSomethingElse() { }
#AfterClass
public static void tearDown() {
//do teardown operations
connection.close();
}
}
Method marked with #BeforeClass will run only once. Just make sure you use JUnit4.
Update:
Also note, that it should be static, and as #ChristopheRoussy mentioned, you can use #AfterClass to destroy your common setup.

You can create a static method that is run before the tests within the class are ran.
#BeforeClass
public static void ranOnlyOnce() {
/*...*/
}
Moreover, if you want to run this once before all of your tests, then you should group your tests in a suite, and put this method in this class, and use JUnit to run the suite instead of the tests.
#RunWith(Suite.class)
#SuiteClasses(value = { Test1.class, ... , Testn.class })
public class AllTests {
#BeforeClass
public static void beforeAllTests() {
/*...*/
}
}
Note that annotations can be used only in Junit 4+.

I would recommend grouping your tests into a testsuite and doing the initialization from the testsuite. There is a good discussion of the possibilities here http://www.xoriant.com/blog/software-testing-and-qa/using-customized-junit-testsuite-for-testing.html

There are a number of possibilities. The first and easiest is to use #BeforeClass and #AfterClass as has been suggested by Matyas and jFrenetic. This is the easiest way to do it if none of the test classes share setup code.
If the classes share the same setup and teardown, look at using the TestRule, specifically the ExternalResource, which allows you to run code before and after each class. The difference between this and #BeforeClass and #AfterClass is that it uses the same class, and therefore the code can be shared more easily. This can be done for example:
#RunWith(Suite.class)
#SuiteClasses({A.class, B.class, C.class})
public class UsesExternalResource {
public static Server myServer= new Server();
#ClassRule
public static ExternalResource resource= new ExternalResource() {
#Override
protected void before() throws Throwable {
myServer.connect();
};
#Override
protected void after() {
myServer.disconnect();
};
};
}
This works in a test class as well, so you could have different, but shared setups for different classes.

Conditionally ignoring tests in JUnit 4

OK, so the #Ignore annotation is good for marking that a test case shouldn't be run.
However, sometimes I want to ignore a test based on runtime information. An example might be if I have a concurrency test that needs to be run on a machine with a certain number of cores. If this test were run on a uniprocessor machine, I don't think it would be correct to just pass the test (since it hasn't been run), and it certainly wouldn't be right to fail the test and break the build.
So I want to be able to ignore tests at runtime, as this seems like the right outcome (since the test framework will allow the build to pass but record that the tests weren't run). I'm fairly sure that the annotation won't give me this flexibility, and suspect that I'll need to manually create the test suite for the class in question. However, the documentation doesn't mention anything about this and looking through the API it's also not clear how this would be done programmatically (i.e. how do I programatically create an instance of Test or similar that is equivalent to that created by the #Ignore annotation?).
If anyone has done something similar in the past, or has a bright idea of how else I could go about this, I'd be happy to hear about it.

The JUnit way is to do this at run-time is org.junit.Assume.
#Before
public void beforeMethod() {
org.junit.Assume.assumeTrue(someCondition());
// rest of setup.
}
You can do it in a #Before method or in the test itself, but not in an #After method. If you do it in the test itself, your #Before method will get run. You can also do it within #BeforeClass to prevent class initialization.
An assumption failure causes the test to be ignored.
Edit: To compare with the #RunIf annotation from junit-ext, their sample code would look like this:
#Test
public void calculateTotalSalary() {
assumeThat(Database.connect(), is(notNull()));
//test code below.
}
Not to mention that it is much easier to capture and use the connection from the Database.connect() method this way.

You should checkout Junit-ext project. They have RunIf annotation that performs conditional tests, like:
#Test
#RunIf(DatabaseIsConnected.class)
public void calculateTotalSalary() {
//your code there
}
class DatabaseIsConnected implements Checker {
public boolean satisify() {
return Database.connect() != null;
}
}
[Code sample taken from their tutorial]

In JUnit 4, another option for you may be to create an annotation to denote that the test needs to meet your custom criteria, then extend the default runner with your own and using reflection, base your decision on the custom criteria. It may look something like this:
public class CustomRunner extends BlockJUnit4ClassRunner {
public CTRunner(Class<?> klass) throws initializationError {
super(klass);
}
#Override
protected boolean isIgnored(FrameworkMethod child) {
if(shouldIgnore()) {
return true;
}
return super.isIgnored(child);
}
private boolean shouldIgnore(class) {
/* some custom criteria */
}
}

Additionally to the answer of #tkruse and #Yishai:
I do this way to conditionally skip test methods especially for Parameterized tests, if a test method should only run for some test data records.
public class MyTest {
// get current test method
#Rule public TestName testName = new TestName();
#Before
public void setUp() {
org.junit.Assume.assumeTrue(new Function<String, Boolean>() {
#Override
public Boolean apply(String testMethod) {
if (testMethod.startsWith("testMyMethod")) {
return <some condition>;
}
return true;
}
}.apply(testName.getMethodName()));
... continue setup ...
}
}

A quick note: Assume.assumeTrue(condition) ignores rest of the steps but passes the test.
To fail the test, use org.junit.Assert.fail() inside the conditional statement. Works same like Assume.assumeTrue() but fails the test.