Develop Reference

Strange behavior of Mockito, a small portion of mock lost effect when ran in mvn test

I'm working on merging 2 legacy projects, every junit test works fine in IDE(Intellij). But some error happens when I execute mvn test. From the error stacktrace, it looks like mock lost effects at some point somewhere. (In a total of about 700 tests, 6 failed).
A failuire example looks like this.
#RunWith(MockitoJUnitRunner.class)
public class SomeHandlerTest {
#Mock
private Child child;
#InjectMocks
private SomeHandler handler;
#Before
public void setUp() throws Exception {
}
#Test
public void testCall() {
handler.call();
}
}
public class SomeHandler {
private Child child;
public String call() {
child.doA();
child.doB();
return "ccc";
}
}
public class Child extends Parent {
public void doB() {
System.out.println("bbb");
}
}
abstract class Parent {
void doA() {
System.out.println("aaa");
}
}
This is just an example, and works as expected(real doA and doB are not invoked during test).
But in real codebase, doA got invoked unexpecttedly and some error happened.
What I have tried:
run failed test class alone in mvn test -Dtest=xxx, failures disappeared.
add <forkMode>always</forkMode> to surefile plugin configuration, failures disappeared.
change doA() method in the Parent class from default to public, failures disappeared.
Does any of these behaviors looks familiar to your guys? How can I troubleshoot this problem so that it can just ran normally? I don't want to change source code just for unit test or change forkMode to always(too slow).

If you add add "always to surefile plugin configuration", failures disappeared.
I think your problem is that you have tests which have side effects which influence your failed tests.

Answering my own question.
It turns out some other test changed Classloader during test.
ClassLoader loader = xxx
Thread.currentThread().setContextClassLoader(loader);
Save the original classloader before that and restore it in the #After solved the problem.
public class XXXTest {
private static ThreadLocal<ClassLoader> threadLocalClassloader = new ThreadLocal<>() ;
#Before
public void setUp() {
threadLocalClassloader.set(Thread.currentThread().getContextClassLoader());
}
#After
public void tearDown() {
Thread.currentThread().setContextClassLoader(threadLocalClassloader.get());
}
}
This well explained
add <forkMode>always</forkMode> to surefile plugin configuration, failures disappeared.
As for the reason of
every junit test works fine in IDE(Intellij)
change doA() method in the Parent class from default to public, failures disappeared.
It's too complicated and I can't explain why, maybe they're just coincidences...

Run after all cucumber tests

Is there a way to run a method after all of the cucumber tests have been run?
The #After annotation would run after every individual test, right? I wan't something that would only run once, but at the very end.

You could use the standard JUnit annotations.
In your runner class write something similar to this:
#RunWith(Cucumber.class)
#Cucumber.Options(format = {"html:target/cucumber-html-report", "json-pretty:target/cucumber-json-report.json"})
public class RunCukesTest {
#BeforeClass
public static void setup() {
System.out.println("Ran the before");
}
#AfterClass
public static void teardown() {
System.out.println("Ran the after");
}
}

What you could do is to register event handler for TestRunFinished event. For that you can create a custom plugin which will register your hook for this event :
public class TestEventHandlerPlugin implements ConcurrentEventListener {
#Override
public void setEventPublisher(EventPublisher eventPublisher) {
eventPublisher.registerHandlerFor(TestRunFinished.class, teardown);
}
private EventHandler<TestRunFinished> teardown = event -> {
//run code after all tests
};
}
and then you will have to register the plugin :
if you are running cucumber CLI you can use -p/--plugin option and pass fully qualified name of the java class : your.package.TestEventHandlerPlugin
for Junit runner :
#RunWith(Cucumber.class)
#CucumberOptions(plugin = "your.package.TestEventHandlerPlugin") //set features/glue as you need.
public class TestRunner {
}

With TestNG suite annotations would work as well.
#BeforeSuite
public static void setup() {
System.out.println("Ran once the before all the tests");
}
#AfterSuite
public static void cleanup() {
System.out.println("Ran once the after all the tests");
}

cucumber is a scenario base test, you should write your own scenario in .feature file step by step and these steps are executed respectively by their step definitions.
So if you want something to happen after all steps, you should write it in the last step and develop this step in its step definition.
Also, for what you want to execute before other steps you should consider a step before all the steps in the .feature file and develop it in its step definition.

JUNIT : run setup only once for a large number of test classes

I have a class, which I use as a basis for my unit tests. In this class I initialize the whole environment for my tests, setting up database mappings, enter a number of database records across multiple tables, etc. That class has a method with a #BeforeClass annotation which does the initialization. Next thing, I extend that class with specific classes in which I have #Test methods.
My question is, since the before class is exactly the same for all these test classes, how can I ensure that they are run only once for all the tests.
One simple solution is that I could keep all the tests in one class. However, the number of tests is huge, also they are categorised based on functional heads. So they are located in different classes. However since they need the exact same setup, they inherit the #BeforeClass. As a result the whole setup is done at least once per test class, taking much more time in total than I would prefer.
I could, though, put them all in various subpackages under one package, hence if there is a way, how I can run set up once for all the tests within that package, it would be great.

With JUnit4 test suite you can do something like this :
#RunWith(Suite.class)
#Suite.SuiteClasses({ Test1IT.class, Test2IT.class })
public class IntegrationTestSuite
{
#BeforeClass
public static void setUp()
{
System.out.println("Runs before all tests in the annotation above.");
}
#AfterClass
public static void tearDown()
{
System.out.println("Runs after all tests in the annotation above.");
}
}
Then you run this class as you would run a normal test class and it will run all of your tests.

JUnit doesn't support this, you will have to use the standard Java work-arounds for singletons: Move the common setup code into a static code block and then call an empty method in this class:
static {
...init code here...
}
public static void init() {} // Empty method to trigger the execution of the block above
Make sure that all tests call init(), for example my putting it into a #BeforeClass method. Or put the static code block into a shared base class.
Alternatively, use a global variable:
private static boolean initialize = true;
public static void init() {
if(!initialize) return;
initialize = false;
...init code here...
}

Create one base class for all tests:
public class BaseTest {
static{
/*** init code here ***/
}
}
and every test should inherit from it:
public class SomeTest extends BaseTest {
}

You can make one BaseTest class with a #BeforeClass method, then have all the other tests inherit from it. This way, when each test object is constructed, #BeforeClass gets executed.
Also avoid executing it just once for all the test suite, since all the test cases should be independent. #BeforeClass should execute only once each test case, not test suite.

If you can tolerate adding spring-test to your project, or you are using it already, then a good approach is to use the technique described here: How to load DBUnit test data once per case with Spring Test

Not sure if anyone still is using JUnit and trying to fix it without using Spring Runner (aka no spring integration). TestNG has this feature. But here is a JUnit based solution.
Create a RunOnce per thread operation like so. This maintains a list of classes for which the operation has run.
public class RunOnceOperation {
private static final ThreadLocal t = new ThreadLocal();
public void run(Function f) {
if (t.get() == null) {
t.set(Arrays.asList(getClass()));
f.apply(0);
} else {
if (!((List) t.get()).contains(getClass())) {
((List) t.get()).add(getClass());
f.apply(0);
}
}
}
}
Back in your unit test
#Before
public beforeTest() {
operation.run(new Function<Integer, Void>() {
#Override
public Void apply(Integer t) {
checkBeanProperties();
return null;
}
});
}
private void checkBeanProperties() {
//I only want to check this once per class.
//Also my bean check needs instance of the class and can't be static.
}
My function interface is like this:
interface Function<I,O> {
O apply(I i);
}
When you use this way, you can perform operations once per class using ThreadLocal.

Loading configuration only once before a set of test cases

I have a java package which contains all my test case classes. Each class contains a lot of test cases. Each class is loaded and run one by one by JUnit. However each of the classes contains common configuration code which is run again and again and initialised everytime each of the classes are run.
These initializations take a lot of time.
Is there some way to load these configuration changes first and then run the test case so that I do not need to load them everytime.

JUnit4 has #BeforeClass annotation.
Just do something like this:
public class TestClass {
private static SomeConnection connection;
#BeforeClass
public static void setUp() {
//do common setup
connection = new SomeConnection();
}
#Test
public void testSomething() { }
#Test
public void testSomethingElse() { }
#AfterClass
public static void tearDown() {
//do teardown operations
connection.close();
}
}
Method marked with #BeforeClass will run only once. Just make sure you use JUnit4.
Update:
Also note, that it should be static, and as #ChristopheRoussy mentioned, you can use #AfterClass to destroy your common setup.

You can create a static method that is run before the tests within the class are ran.
#BeforeClass
public static void ranOnlyOnce() {
/*...*/
}
Moreover, if you want to run this once before all of your tests, then you should group your tests in a suite, and put this method in this class, and use JUnit to run the suite instead of the tests.
#RunWith(Suite.class)
#SuiteClasses(value = { Test1.class, ... , Testn.class })
public class AllTests {
#BeforeClass
public static void beforeAllTests() {
/*...*/
}
}
Note that annotations can be used only in Junit 4+.

I would recommend grouping your tests into a testsuite and doing the initialization from the testsuite. There is a good discussion of the possibilities here http://www.xoriant.com/blog/software-testing-and-qa/using-customized-junit-testsuite-for-testing.html

There are a number of possibilities. The first and easiest is to use #BeforeClass and #AfterClass as has been suggested by Matyas and jFrenetic. This is the easiest way to do it if none of the test classes share setup code.
If the classes share the same setup and teardown, look at using the TestRule, specifically the ExternalResource, which allows you to run code before and after each class. The difference between this and #BeforeClass and #AfterClass is that it uses the same class, and therefore the code can be shared more easily. This can be done for example:
#RunWith(Suite.class)
#SuiteClasses({A.class, B.class, C.class})
public class UsesExternalResource {
public static Server myServer= new Server();
#ClassRule
public static ExternalResource resource= new ExternalResource() {
#Override
protected void before() throws Throwable {
myServer.connect();
};
#Override
protected void after() {
myServer.disconnect();
};
};
}
This works in a test class as well, so you could have different, but shared setups for different classes.

How do I use JUnitPerf with JWebUnit and JUnit 4?

I have a series of functional tests against a web application that correctly run, but each require the class level setup and teardown provided with the #BeforeClass and #AfterClass annotations, and hence require JUnit 4.0 or above.
Now I want to perform load testing using a small number of these functional tests, which simulate a large number of users requesting the related page of the web application. In order for each user to have their own "simulated browser" in JWebUnit, I need to use a TestFactory in JUnitPerf to instantiate the class under test, but since JUnit 4 tests are annotated with #Test instead of being derived from TestCase, I'm getting a TestFactory must be constructed with a TestCase class exception.
Is anyone successfully using JUnitPerf and its TestFactory with JUnit 4? And what is the secret sauce that lets it all work?

You need a JUnit4 aware TestFactory. I've included one below.
import junit.framework.JUnit4TestAdapter;
import junit.framework.TestCase;
import junit.framework.TestSuite;
import com.clarkware.junitperf.TestFactory;
class JUnit4TestFactory extends TestFactory {
static class DummyTestCase extends TestCase {
public void test() {
}
}
private Class<?> junit4TestClass;
public JUnit4TestFactory(Class<?> testClass) {
super(DummyTestCase.class);
this.junit4TestClass = testClass;
}
#Override
protected TestSuite makeTestSuite() {
JUnit4TestAdapter unit4TestAdapter = new JUnit4TestAdapter(this.junit4TestClass);
TestSuite testSuite = new TestSuite("JUnit4TestFactory");
testSuite.addTest(unit4TestAdapter);
return testSuite;
}
}