I would like to add tests for method, which contains CompletableFuture:
public void report(List<String> srcList) {
if (srcList != null) {
...
CompletableFuture.runAsync(() ->
....
srcList.forEach(src-> downloader.send(url)));
}
}
I would like to test, that method send is called. My test looks like:
#Test
public void _test() {
List<String> events = new ArrayList();
events.add("http://xxxx//");
events.add("http://xxxx//");
expect(downloader.send(events.get(0))).andReturn("xxx").times(2);
replay(downloader);
eventReporter.report(events);
verify(downloader);
}
And I get such error Downloader.send("http://xxxx//"): expected: 2, actual: 0
One way to avoid this error, is to set Thread.sleep(100); timeout. Then the thread will wait and verify that the method had called. But this will increase tests time.
Is there other way to test multithreading with EasyMock?
It is a bad practice to unit test asynchronoys code with Thread.sleep() method
because if it even works test will be unstable and flicker (run 3 times 2 pass and 1 fail)
if you set up big time of sleep and write few tests like this you meet a big time of execution
that may be exceed dozens of seconds. For complete this task you need to decouple asynchronous part
of you code from synchronous. Example how to do it:
class Service {
private Downloader downloader;
private ExecutorService service;
public Service (Downloader downloader, ExecutorService service) {
//set variables
}
public void doWork(List<String> list) {
for (String item : list) {
service.submit(() -> {
downloader.download(item);
});
}
}
}
ExecutorService is interface and we need to make our Service that will be synchronous
class SycnronousService impliments ExecutorService {
//methods empty implementations
public void submit(Runnable runnable) {
runnable.run(); //run immediately
}
//methods empty implementations
}
public class ServiceTest {
public void shouldPassAllItemsToDownloader() {
Downloader mockDownloader = AnyMockFramework.mockIt();
Service service = new Service(mockDownloader, new SycnronousService());
List<String> tasks = Arrays.asList("A", "B");
service.doWork(tasks);
verify(mockDownloader).download("A"); //verify in your way with EasyMock
verify(mockDownloader).download("B"); //verify in your way with EasyMock
// no more Timer.sleep() , test runs immeadetely
}
}
You need to replace CompletableFuture to something like in my example, because
unit testing this code not able in this way.
Later in you app you will be able to replace SycnronousService to asynchronous implementation and all will be work as expected.
I agree with #joy-dir 's answer. And you should probably do what she said to simplify your testing.
For the sake of completeness, your problem here is that the verify is called before your tasks being actually finished. There are many things you could do.
One is to loop on verify.
#Test
public void test() throws Exception {
List<String> events = new ArrayList();
events.add("http://xxxx//");
events.add("http://xxxx//");
expect(downloader.send(events.get(0))).andReturn("xxx").times(2);
replay(downloader);
report(events);
for (int i = 0; i < 10; i++) {
try {
verify(downloader);
return;
} catch(AssertionError e) {
// wait until it works
}
Thread.sleep(10);
}
verify(downloader);
}
It won't sleep a long time for nothing when successful. However, you indeed need to make sure you wait enough to prevent the test from being flaky.
Another solution is actually to use the CompletableFuture returned by runAsync. I prefer this solution.
public CompletableFuture<Void> report(List<String> srcList) {
if (srcList != null) {
return CompletableFuture.runAsync(() -> srcList.forEach(src-> downloader.send(src)));
}
return CompletableFuture.completedFuture(null);
}
#Test
public void test2() throws Exception {
List<String> events = new ArrayList();
events.add("http://xxxx//");
events.add("http://xxxx//");
expect(downloader.send(events.get(0))).andReturn("xxx").times(2);
replay(downloader);
CompletableFuture<Void> future = report(events);
future.get(100, TimeUnit.MILLISECONDS);
verify(downloader);
}
Finally, there is a hackish way. You ask the common pool if it is done. It is hackish because something else might use it. So it's cute but I wouldn't really recommend it.
#Test
public void test3() throws Exception {
List<String> events = new ArrayList();
events.add("http://xxxx//");
events.add("http://xxxx//");
expect(downloader.send(events.get(0))).andReturn("xxx").times(2);
replay(downloader);
report(events);
while(!ForkJoinPool.commonPool().isQuiescent()) {
Thread.sleep(10);
}
verify(downloader);
}
Related
Here's a sample of the code I've been working on
items contains 100 elements, thus obtaining data using synchronous calling takes up a lot of time. Can someone suggest a way to increase the speed of this operation so that it takes less time.
Currently this takes 15-20 seconds to execute. I'm new to rxjava so please provide a detailed solution to this problem if possible. dataResponses contains RouteDistance objects for each of the 100 items.
for(int i = 0 ; i<items.size();i++){
Map<String, String> map2 = new HashMap<>();
map2.put("units", "metric");
map2.put("origin", currentLocation.getLatitude()+","+currentLocation.getLongitude());
map2.put("destination", items.get(i).getPosition().get(0)+","+items.get(i).getPosition().get(1));
map2.put("transportMode", "car");
requests.add(RetrofitClient4_RouteDist.getClient().getRouteDist(map2));
}
Observable.zip(requests, new Function<Object[], List<RouteDist>>() {
#Override
public List<RouteDist> apply(Object[] objects) throws Exception {
Log.i("onSubscribe", "apply: " + objects.length);
List<RouteDist> dataaResponses = new ArrayList<>();
for (Object o : objects) {
dataaResponses.add((RouteDist) o);
}
return dataaResponses;
}
})
.observeOn(AndroidSchedulers.mainThread())
.subscribeOn(Schedulers.io())
.subscribe(
new Consumer<List<RouteDist>>() {
#Override
public void accept(List<RouteDist> dataaResponses) throws Exception {
Log.i("onSubscribe", "YOUR DATA IS HERE: "+dataaResponses.toString());
recyclerViewAdapter_profile = new RecyclerViewAdapter_Profile(items,dataaResponses);
recyclerView.setAdapter(recyclerViewAdapter_profile);
}
},
new Consumer<Throwable>() {
#Override
public void accept(Throwable e) throws Exception {
Log.e("onSubscribe", "Throwable: " + e);
}
});
API
interface Client {
Observable<RouteDist> routeDist();
}
final class RouteDist {
}
final class ClientImpl implements Client {
#Override
public Observable<RouteDist> routeDist() {
return Observable.fromCallable(() -> {
// with this log, you see, that each subscription to an Observable is executed on the ThreadPool
// Log.e("---------------------", Thread.currentThread().getName());
return new RouteDist();
});
}
}
Apply threading via subscribeOn
final class ClientProxy implements Client {
private final Client api;
private final Scheduler scheduler;
ClientProxy(Client api, Scheduler scheduler) {
this.api = api;
this.scheduler = scheduler;
}
#Override
public Observable<RouteDist> routeDist() {
// apply #subscribeOn in order to move subscribeAcutal call on given Scheduler
return api.routeDist().subscribeOn(scheduler);
}
}
AndroidTest
#Test
public void name() {
// CachedThreadPool, in order to avoid creating 100-Threads or more. It is always a good idea to use own Schedulers (e.g. Testing)
ThreadPoolExecutor threadPool = new ThreadPoolExecutor(0, 10,
60L, TimeUnit.SECONDS,
new SynchronousQueue<>());
// wrap real client with Proxy, in order to move the subscribeActual call to the ThreadPool
Client client = new ClientProxy(new ClientImpl(), Schedulers.from(threadPool));
List<Observable<RouteDist>> observables = Arrays.asList(client.routeDist(), client.routeDist(), client.routeDist());
TestObserver<List<RouteDist>> test = Observable.zip(observables, objects -> {
return Arrays.stream(objects).map(t -> (RouteDist) t).collect(Collectors.toList());
})
.observeOn(AndroidSchedulers.mainThread())
.test();
test.awaitCount(1);
// verify that onNext in subscribe is called in Android-EventLoop
assertThat(test.lastThread()).isEqualTo(Looper.getMainLooper().getThread());
// verify that 3 calls were made and merged into one List
test.assertValueAt(0, routeDists -> {
assertThat(routeDists).hasSize(3);
return true;
});
}
Further reading:
http://tomstechnicalblog.blogspot.de/2016/02/rxjava-understanding-observeon-and.html
Note:
It is not recommanded to call an API 100-times concurrently at once. Furthermore when using Zip, this is what will acutally happen, when you have a ThreadPool, which is big enough. When one API-call times-out, an onError will probably emitted for this API-calls. The onError will be propagated further to the subscriber. You will not get any result, even if only on API-call fails. It is recommanded to have some onErrorResumeNext or some other error-handling operator, in order to ensure, that one API-call does not cancel the overall result.
Awaitility is a great tool to unit-test concurrent production code.
Question: Is there a tool to ease writing concurrent test code?
Suppose I wanted to test java.util.concurrent.LinkedBlockingQueue.
public class BlockingQueueTest {
private LinkedBlockingQueue<String> out;
#Before
public void setUp() {
out = new LinkedBlockingQueue<>();
}
#Test
public void putThenGet() throws InterruptedException {
// that's easy because it can be done in one thread
out.put("Hello");
String taken = out.take();
assertThat(taken).isEqualTo("Hello");
}
#Test
public void getBeforePut() throws InterruptedException {
// that's more tricky because it can't be done with one thread
ExecutorService executorService = Executors.newSingleThreadExecutor();
executorService.submit(() -> {
Thread.sleep(100);
out.put("Hello");
return null;
});
executorService.shutdown();
String taken = out.take();
assertThat(taken).isEqualTo("Hello");
}
}
getBeforePut() is no fun to code. Is there a way to make it less hard and more readable, like so?
#Test
public void getBeforePut2() throws InterruptedException {
// Wanted: DSL for concurrent test-code
Concurrently.sleep(100, TimeUnit.MILLISECONDS).andThen(() -> out.put("Hello"));
String taken = out.take();
assertThat(taken).isEqualTo("Hello");
}
Using TestNG is the easiest way as for me:
#Test(threadPoolSize = 10, invocationCount = 15, timeOut = 1000)
public void testPut(){
out.put("Hello");
}
This test will run 15 times in 10 threads and should take no longer than 1000 ms.
Also you can create tests which depends on other test
#Test(dependsOnMethods = "testPut")
public void testGetAfterPut{
String taken = out.take();
assertThat(taken).isEqualTo("Hello");
}
(A) you can just use Thread without the need of an ExecutorService
#Test
public void getBeforePutWithThread() throws InterruptedException {
new Thread(() -> {
Thread.sleep(100);
out.put("Hello");
}).run();
String taken = out.take();
assertThat(taken).isEqualTo("Hello");
}
(B) You can take this all functionality to a simple function and avoid dependency on external lib
private void runWithDelay(long delay, Runnable action) {
new Thread(() -> {
try {
Thread.sleep(delay);
action.run();
} catch (InterruptedException e) {
e.printStackTrace();
}
}).run();
}
#Test
public void getBeforePutWithFunction() {
runWithDelay(100, () -> out.put("Hello"));
String taken = out.take();
assertThat(taken).isEqualTo("Hello");
}
By now I'm writing all my tests in kotlin. And with kotlin testing this is easy peasy and fun!
Worth mentioning when testing with threads is JUnit's #Timeout Annotation, which prevents false tests from running infinitely.
import org.assertj.core.api.Assertions.assertThat
import org.junit.jupiter.api.Test
import org.junit.jupiter.api.Timeout
import java.util.concurrent.LinkedBlockingQueue
import kotlin.concurrent.thread
class BlockingQueueKotlinTests {
// objectUnderTest
private val out = LinkedBlockingQueue<String>()
#Test
fun `put then get`() {
// that's easy because it can be done in one thread
out.put("Hello")
val taken = out.take()
assertThat(taken).isEqualTo("Hello")
}
#Test
#Timeout(1)
fun `get before put`() {
// thanks to kotlin it's really easy to do that in another thread
thread {
Thread.sleep(100)
out.put("kotlin is great!")
}
val taken = out.take()
assertThat(taken).isEqualTo("kotlin is great!")
}
}
In a loop i need to make some checks, performed actually in some another verticle. In each iteration of my loop i need to check the response code, returned from those verticle and make some decision accordingly. In some other words i need to stop the execution of my loop and somehow wait till asynch. call returns. But such execution stop violates the vert.x philosophy, which states that main thread execution should be never stopped. How can i do it in the scope of Vert.x? So far i don't know how to do this. Any suggestions/code samples/urls to smth. like a solution would b highly appreciated.
Thanks!
When working with Vert.x you need to think less in terms of loops, and more in terms of callbacks.
You should use eventBus to communicate between vertices.
Let's say that what you want is something similar to this pseudocode:
for (int i = 0; i < 4; i++) {
int result = getVerticleResult();
System.out.println(result);
}
So, just a very basic example
class LooperVerticle extends AbstractVerticle {
private int i = 4;
#Override
public void start() throws Exception {
doWork();
}
private void doWork() {
vertx.eventBus().send("channel", "", (o) -> {
if (o.succeeded()) {
System.out.println(o.result().body());
i--;
if (i > 0) {
doWork();
}
}
});
}
}
class WorkerVerticle extends AbstractVerticle {
#Override
public void start() throws Exception {
vertx.eventBus().consumer("channel", (o) -> {
// Generate some random number
int num = ThreadLocalRandom.current().nextInt(0, 9);
// Simulate slowness
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
o.reply(num);
});
}
}
To test:
public class EventBusExample {
public static void main(String[] args) {
Vertx vertx = Vertx.vertx();
vertx.deployVerticle(new LooperVerticle());
vertx.deployVerticle(new WorkerVerticle());
}
}
I think you need to use FutureTask and store them in a Collection and use FutureTask.get() to retrieve the result when needed which is a blocking call.
It sounds like a use case for reactive steam processing.
In general such problem could be solved using 2 parties:
a producer that executes tasks and returns asynchronous results
a handler that subscribes to results and performs another tasks
There is a way to configure producer to perform tasks only when there is a subscriber. And on other side subscriber can decide to unsubscribe from producer on some condition.
I'm not familiar with vertx capabilities for reactive streams. But I would start from RxJava integration
http://vertx.io/docs/vertx-rx/java/
I have a test of communication over a TCP socket where I expect the server to not respond within a set time frame when I send a certain message.
The servers behaviour is nothing I can control.
I know how to fail a test if it has not completed within a set time frame. But how can I do the opposite, make it pass for not completing within the timeframe?
I can use #Test (timeout=1000) to make a test fail if not complete within a second.
But, using Junit 4, is there a function to test for an expected timeout as a positive result? I.e. The test will fail if completed within the time frame and pass if not?
Good question. Actually you can do this using only junit tools. My idea is to inverse Timeout rule behaviour + use expected attribute of Test annotation. The only one limitation: you have to place your test in separate class, because Rule applies to all tests inside it:
public class Q37355035 {
private static final int MIN_TIMEOUT = 100;
#Rule
public Timeout timeout = new Timeout(MIN_TIMEOUT) {
public Statement apply(Statement base, Description description) {
return new FailOnTimeout(base, MIN_TIMEOUT) {
#Override
public void evaluate() throws Throwable {
try {
super.evaluate();
throw new TimeoutException();
} catch (Exception e) {}
}
};
}
};
#Test(expected = TimeoutException.class)
public void givesTimeout() throws InterruptedException {
TimeUnit.SECONDS.sleep(1);
}
}
I'm building upon the great answer by Andremoniy, don't forget to upvote his answer if you like this one!
I used the following modification to skip a test if it doesn't finish in the expected time frame. The benefit of this is that the test will be marked as Skipped by JUnit instead of Successful. This is good for optimistic execution of tests which can sometimes hang or not finish fast enough, but you don't want to mark them as failed or delete them.
public class OptimisticTestClass {
private static final int TEST_METHOD_TIMEOUT_MS = 100;
#Rule
public Timeout timeout = new Timeout(TEST_METHOD_TIMEOUT_MS, TimeUnit.MILLISECONDS) {
public Statement apply(Statement base, Description description) {
return new FailOnTimeout(base, TEST_METHOD_TIMEOUT_MS) {
#Override
public void evaluate() throws Throwable {
try {
super.evaluate();
} catch (TestTimedOutException e) {
Assume.assumeNoException("Test did not finish in the allocated time, skipping!", e);
}
}
};
}
};
// The test times out and is skipped
public void givesTimeout() throws InterruptedException {
Thread.sleep(1000);
}
}
It's simpler in Java 9
CompletableFuture.supplyAsync(() -> dormammu.bargain())
.orTimeout(1, TimeUnit.SECONDS)
.handle((result, throwable) -> {
if (!(throwable instanceof TimeoutException)) {
Assertions.fail();
}
return result;
}).get();
If the method doesn't return within 1 second, it will time out. In handle method you can make sure TimeoutException was thrown, otherwise fail the test.
The issue is that I have a method starting a new thread for a time-consuming work. I want to test the callback result, but the child thread may still running, so as a result, what I get is not the right stub.
I think the code may explain itself:
public class JustAClass {
//it is a callback for async
public interface JustACallBack {
void callFunc(JustAResult result);
}
//this is the result interface
public interface JustAResult {
}
//this is a real class for the interface
public class JustAResultReal implements JustAResult{
public JustAResultReal(String content) {this.content = content;}
public String content;
}
//here is the key function
public void threadFunc(final JustACallBack callBack) {
BCCache.executorService.execute(new Runnable() {
#Override
public void run() {
//just to simulate a time-consuming task
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
//now we callback
callBack.callFunc(new JustAResultReal("can you reach me"));
}
});
}
}
and the test function could be(I am using mockito):
#Test
public void testThreadFunc() throws Exception {
JustAClass justAClass = new JustAClass();
JustAClass.JustACallBack callBack = Mockito.mock(JustAClass.JustACallBack.class);
justAClass.threadFunc(callBack);
//add this line, we can get the expected result
Thread.sleep(1200);
Mockito.verify(callBack).callFunc(captor.capture());
System.out.println(((JustAClass.JustAResultReal)captor.getValue()).content);
}
I know we can add a sleep to wait and expect that the child thread would exit within the period, but could there be a better way? Actually how could I know how long the child thread would take? Setting a very long time can be an approach but just seems not very nice.
The general approach in #stalet's answer is close, but doesn't quite work since any assertion failures from a separate thread are not noticed by the main thread. Therefore your test will always pass, even when it shouldn't. Instead, try using ConcurrentUnit (which I authored):
#Test
public void testInvoke() throws Throwable {
Waiter waiter = new Waiter();
JustAClass justAClass = new JustAClass();
JustAClass.JustACallBack callBack = new JustAClass.JustACallBack() {
#Override
public void callFunc(final JustAClass.JustAResult result) {
waiter.assertNotNull(result);
waiter.assertTrue(result instanceof JustAClass.JustAResultReal);
waiter.resume();
}
};
justAClass.threadFunc(callBack);
waiter.await(1200, TimeUnit.SECONDS);
}
The key here is ConcurrentUnit's Waiter will properly report any assertions failures to the main test thread and the test will pass or fail as it should.
I aggree with #Gimbys comment about this is no longer a unit-test when you start testing the the threading aspect.
Nevertheless it is interesting as a way to integration-test a asynchronous invokation.
To avvoid sleep i tend to use the class CountDownLatch to wait for invokations.
In order to count down you need an actuall implementation of the callback interface - so in my example I have made a mock implementation of this.
Since there is no actual methods to fetch the data - i am just testing that it is in fact a instance of the JustAReal interface.
#Test
public void testInvoke() throws Exception {
final CountDownLatch countDownLatch = new CountDownLatch(1); //1 is how many invokes we are waiting for
JustAClass justAClass = new JustAClass();
JustAClass.JustACallBack callBack = new JustAClass.JustACallBack() {
#Override
public void callFunc(final JustAClass.JustAResult result) {
assertNotNull("Result should not be null", result);
assertTrue("Result should be instance of JustAResultReal", result instanceof JustAClass.JustAResultReal);
countDownLatch.countDown();
}
};
justAClass.threadFunc(callBack);
if(!countDownLatch.await(1200, TimeUnit.MILLISECONDS)){
fail("Timed out, see log for errors");
}
}