Develop Reference

JUnit concurrent access to synchronizedSet

I have a problem with running JUnit tests on my server. When I run the test on my machine, there is no problem at all. When I run it on the server, there is a failure on all my server "sometimes". It means tests pass sometimes in 60% of attempts and 40% fail.
I am using Mockito. My test starts with mocking some replies using MessageListener and map every request to a response and under the hood I am using Collections.synchronizedSet(new HashSet<>()) which is thread-safe.(Every modification on my synchronizedSet happens in a synchronized(mySynchronizedSet){....}) Then, I am using RestAssurd to get the response of a particular REST endpoint and assert some values.
When a test fails and I look on the Stacktrace, I see that one of my mappings (always on the same object) didn't work and there is no map between this specific request and response in my collection and naturally, I get null on requesting this endpoint.
I am using Jenkins to automate the compilation and running the test and I get the stack trace on fail or my Printlns otherwise, there are no debug facilities available.
It sounds like a concurrency problem to me. I mean it seems my collection does not have time to get ready before RestAssurd request for an endpoint. I've tested locks, sleep, and another simple java concurrency solutions but they don't help and the probabilistic character of this problem has led me to a dead end.
Every thought will be appreciated.

Judging by what you said, it seems you have a misunderstanding of how things work in 3 specific cases.
First
and most obvious, and I apologize for even mentioning this, but the reason that I do at all is because I'm gathering that you're still learning (I apologize further if you're not still learning! and at the same rate, you might not have even implied it with the way I read it, so sorry if I misread): you aren't compiling with Jenkins, you're compiling with whatever JDK flavor you have on your machine (be it Oracle, Apple, GCJ, etc). Jenkins is an automation tool that helps facilitate your tedious jobs you expect to run regularly. I only mention this because I know college students nowadays use IDE's in there opening classes and can't distinguish between the compiler, the runtime, and the IDE.
Secondly
by using a threadsafe library, it doesn't automatically make everything you do inherently threadsafe. Consider the following example:
final Map<Object, Object> foo = Collections.synchronizedMap(new HashMap <>());
final String bar = "bar";
foo.put(bar, new Object());
new Thread(new Runnable(){
#Override
public void run(){
foo.remove(bar);
}
}).start();
new Thread(new Runnable(){
#Override
public void run(){
if(foo.containsKey(bar)){
foo.get(bar).toString();
}
}
}).start();
There is no guarantee that the second thread's call to #get(Object) will happen before or after the first thread's call to #remove(Object). Consider that
the second thread could call #containsKey(Object)
then the first thread obtains CPU time and calls #remove(Object)
then the second thread now has CPU time and calls #get(Object)
at this point, the returned value from get(Object) will be null, and the call to #toString() will result in a NullPointerDereference. You say you're using Set, so this example using a Map is mainly to prove a point: just because you're using a threadsafe collection, doesn't automatically make everything you do threadsafe. I imagine there are things you are doing with your set that match this sort of behavior, but without code snippets, I can only speculate.
And Lastly
You should be careful with how you write JUnits. A proper JUnit test is what's called a "whitebox" test. In otherwords, you know everything that is happening in the test, and you are explicitly testing everything that is happening in only the unit under test. The unit under test is just the method you are calling - not the methods that are called by your method, only the method itself. What that means, is that you need a good mocking framework, and mock out any subsequent method calls that your unit under test may invoke. Some good frameworks are JMockit, Mockito+PowerMock, etc.
The importance of this is that your test is supposed to test your isolated code. If you're allowing network access, disk access, etc, then your test may fail and it may have nothing to do with code you wrote, and it invalidates the test entirely. In your case, you hint at network access, so imagine that there is some throughput issue with your switches/router/etc, or that your NIC buffer gets full and can't process fast enough for what your program is trying to do. Sure, the failure is not good, and should be fixed, but that should be tested in "blackbox" testing. Your tests should be written so that you eliminate these sort of issues from being present and only test your code in the particular method for the unit under test, and nothing else.
Edit: I actually posted an answer to a separate discussion about whitebox testing that might be relevant: Is using a test entity manager a legitamate testing practice?

Junit, testing timer.schedule without relying on thread sleeping or time

I have implemented a method that executes some logic, after a certain amount of time, using a TimerTask and Timer.schedule.
I want to verify this behaviour using Junit, however, I would like to know if there are better ways to test it, without using thread sleeping, or measuring time.
Thanks.

You can use a "own thread" excecutor service to get around the "multiple threads" complications.
You can further test that some class A pushes tasks into such a service; and you can also use unit tests to ensure that the parameters used when pushing tasks are what you expect them to be.
In other words: you really don't want to use unit tests to prove that scheduling is working (assuming that you didn't completely re-invent the wheel and you implemented your own scheduling ... which is something that you simply should not do). You want use unit tests to prove that your code is using existing (well tested) frameworks with the arguments you expect to see.

Unit testing parts of the application that use Thread Local

I am trying to implement unit testing in a web application and certain parts of it use ThreadLocal.
I cannot figure out how to go about testing it.
It looks like Junit runs all its tests using a single thread, namely the main thread.
I need to be able to assign different values to my ThreadLocal variable.
Has anyone come across such a scenario ? What do you guys recommend.

Groboutils has support for running multi-threaded tests, which will allow you to test your ThreadLocal variables.
http://groboutils.sourceforge.net/testing-junit/using_mtt.html

I would simply start threads within my unit test.
I recommend you use Futures and execute them using a ThreadPoolExecutor.

It might be enough to adorn the respective test methods with a timeout, i.e. #Test(timeout=100). Please note the "THREAD SAFETY WARNING" in Test::timeout , especially when you use #BeforeClass and #After annotations.

Poor JUnit test using springframework has fragile Thread.sleep() calls. How to fix?

I have recently joined a group with some severe JUnit testing issues. One problem is an 8 minute long test! The test has several sections; each makes calls to org.springframework.context.ApplicationEventPublisher.publishEvent()
followed by Thread.sleep() of various amounts of time, then tests conditions.
There are several obvious problems with this approach, the timing of the Thread.sleep() calls is fragile:
tests occasionally fail on busy machines; and
tests take far too long when they do not fail.
Is the pool upon which these events are handled accessible for testing and is there a call to see if the event cascade has quiesced?

Worth mentioning is that test code that actually calls external services are integration tests and not unit tests. If you're truly unit testing here you should replace those calls with mocks. That way you can better control the values returned to your business logic and test for specific conditions. Also, as you've seen, this all but eliminates false positives due to external (non-code) situations. Obviously these tests aren't failing, the facility they expect to use is.

You can overwrite the default applicationEventMulticaster by adding this bean id to your application context.
Instead of the default SimpleApplicationEventMulticaster, you could set a TaskExecutor on this bean to perform the event publishing asynchronously in multiple threads.
Or you could implement your own multicaster, which prints out which event listener took so long or was blocking, for how long and on which events. That could help you to track down the real problem of the 8-Minute-Testcase.
Interestingly, the JavaDoc of the SimpleApplicationEventMulticaster, which is used by default by Spring when you are using ApplicationContext, states the following:
By default, all listeners are invoked in the calling thread. This allows the danger of a rogue listener blocking the entire application, but adds minimal overhead. Specify an alternative TaskExecutor to have listeners executed in different threads, for example from a thread pool.

I (intentionally) avoid Spring so I'm not sure I can help with the specifics but just looking at the sleep issue, you can use something like WaitFor in tempus-fugit (shameless plug) to poll for a Condition rather than "sleep and hope". It's not ideal and usually a change in the way you test (as suggested before) is preferable but it does mean you get finer grained "waits" which are more likely to avoid race-conditions / flaky tests and generally speed up the test.
See the project's documentation for details and post back if you find it useful!

How to write multi-threaded unit tests?

I'd like to know if there are some unit testing frameworks which are capable of writing multi-threaded tests easily?
I would imagine something like:
invoke a special test method by n threads at the same time for m times. After all test threads finished, an assertion method where some constraints should be validated would be invoked.
My current approach is to create Thread objects inside a junit test method, loop manually the real test cases inside each run() method, wait for all threads and then validate the assertions. But using this, I have a large boilerplate code block for each test.
What are your experiences?

There is ConTest, and also GroboUtils.
I've used GroboUtils many years ago, and it did the job. ConTest is newer, and would be my preferred starting point now, since rather than just relying on trial and error, the instrumentation forces specific interleavings of the threads, providing a deterministic test. In contrast, GroboUtils MultiThreadedTestRunner simply runs the tests and hopes the scheduler produces an interleaving that causes the thread bug to appear.
EDIT: See also ConcuTest which also forces interleavings and is free.

There is also MultithreadedTC by Bill Pugh of FindBugs fame.

Just using the concurrency libraries would simplify your code. You can turn your boiler plate code into one method.
Something like
public static void runAll(int times, Runnable... tests) {
}