I have a test of communication over a TCP socket where I expect the server to not respond within a set time frame when I send a certain message.
The servers behaviour is nothing I can control.
I know how to fail a test if it has not completed within a set time frame. But how can I do the opposite, make it pass for not completing within the timeframe?
I can use #Test (timeout=1000) to make a test fail if not complete within a second.
But, using Junit 4, is there a function to test for an expected timeout as a positive result? I.e. The test will fail if completed within the time frame and pass if not?
Good question. Actually you can do this using only junit tools. My idea is to inverse Timeout rule behaviour + use expected attribute of Test annotation. The only one limitation: you have to place your test in separate class, because Rule applies to all tests inside it:
public class Q37355035 {
private static final int MIN_TIMEOUT = 100;
#Rule
public Timeout timeout = new Timeout(MIN_TIMEOUT) {
public Statement apply(Statement base, Description description) {
return new FailOnTimeout(base, MIN_TIMEOUT) {
#Override
public void evaluate() throws Throwable {
try {
super.evaluate();
throw new TimeoutException();
} catch (Exception e) {}
}
};
}
};
#Test(expected = TimeoutException.class)
public void givesTimeout() throws InterruptedException {
TimeUnit.SECONDS.sleep(1);
}
}
I'm building upon the great answer by Andremoniy, don't forget to upvote his answer if you like this one!
I used the following modification to skip a test if it doesn't finish in the expected time frame. The benefit of this is that the test will be marked as Skipped by JUnit instead of Successful. This is good for optimistic execution of tests which can sometimes hang or not finish fast enough, but you don't want to mark them as failed or delete them.
public class OptimisticTestClass {
private static final int TEST_METHOD_TIMEOUT_MS = 100;
#Rule
public Timeout timeout = new Timeout(TEST_METHOD_TIMEOUT_MS, TimeUnit.MILLISECONDS) {
public Statement apply(Statement base, Description description) {
return new FailOnTimeout(base, TEST_METHOD_TIMEOUT_MS) {
#Override
public void evaluate() throws Throwable {
try {
super.evaluate();
} catch (TestTimedOutException e) {
Assume.assumeNoException("Test did not finish in the allocated time, skipping!", e);
}
}
};
}
};
// The test times out and is skipped
public void givesTimeout() throws InterruptedException {
Thread.sleep(1000);
}
}
It's simpler in Java 9
CompletableFuture.supplyAsync(() -> dormammu.bargain())
.orTimeout(1, TimeUnit.SECONDS)
.handle((result, throwable) -> {
if (!(throwable instanceof TimeoutException)) {
Assertions.fail();
}
return result;
}).get();
If the method doesn't return within 1 second, it will time out. In handle method you can make sure TimeoutException was thrown, otherwise fail the test.
Related
I have a piece of code that retries when it fails with ExceptionOne, and if it continues to fail, it will throw a ExceptionTwo. I want to test this behavior, but am not sure how to.
public void someMethod(String x) {
boolean retry;
int try = 0;
do {
retry = false;
try {
someFunction(x);
} catch (ExceptionOne e) {
if (try < 5) {
retry = true;
attempt++;
} else {
throw new ExceptionTwo();
}
}
} while (retry);
}
private void someFunction(String x) {
doSomething(); //This can throw ExceptionOne
}
Suppose these functions are inside a SomeClass. I mocked SomeClass and tried something like
doThrow(new ExceptionOne()).doNothing().when(mockObject).someMethod(any())
for consecutive calls, but this isn't quite what I am looking for since I believe this is really testing the someMethod, not someFunction or the retry functionality.
How do you test something like this? That is, how do I verify that the behavior that when someFunction fails with ExceptionOne, it will retry until it succeeds or until it runs out of retries and throws an ExceptionTwo?
I ended up scrapping my current retry logic and using an external library to do the retry instead.
I would like to add tests for method, which contains CompletableFuture:
public void report(List<String> srcList) {
if (srcList != null) {
...
CompletableFuture.runAsync(() ->
....
srcList.forEach(src-> downloader.send(url)));
}
}
I would like to test, that method send is called. My test looks like:
#Test
public void _test() {
List<String> events = new ArrayList();
events.add("http://xxxx//");
events.add("http://xxxx//");
expect(downloader.send(events.get(0))).andReturn("xxx").times(2);
replay(downloader);
eventReporter.report(events);
verify(downloader);
}
And I get such error Downloader.send("http://xxxx//"): expected: 2, actual: 0
One way to avoid this error, is to set Thread.sleep(100); timeout. Then the thread will wait and verify that the method had called. But this will increase tests time.
Is there other way to test multithreading with EasyMock?
It is a bad practice to unit test asynchronoys code with Thread.sleep() method
because if it even works test will be unstable and flicker (run 3 times 2 pass and 1 fail)
if you set up big time of sleep and write few tests like this you meet a big time of execution
that may be exceed dozens of seconds. For complete this task you need to decouple asynchronous part
of you code from synchronous. Example how to do it:
class Service {
private Downloader downloader;
private ExecutorService service;
public Service (Downloader downloader, ExecutorService service) {
//set variables
}
public void doWork(List<String> list) {
for (String item : list) {
service.submit(() -> {
downloader.download(item);
});
}
}
}
ExecutorService is interface and we need to make our Service that will be synchronous
class SycnronousService impliments ExecutorService {
//methods empty implementations
public void submit(Runnable runnable) {
runnable.run(); //run immediately
}
//methods empty implementations
}
public class ServiceTest {
public void shouldPassAllItemsToDownloader() {
Downloader mockDownloader = AnyMockFramework.mockIt();
Service service = new Service(mockDownloader, new SycnronousService());
List<String> tasks = Arrays.asList("A", "B");
service.doWork(tasks);
verify(mockDownloader).download("A"); //verify in your way with EasyMock
verify(mockDownloader).download("B"); //verify in your way with EasyMock
// no more Timer.sleep() , test runs immeadetely
}
}
You need to replace CompletableFuture to something like in my example, because
unit testing this code not able in this way.
Later in you app you will be able to replace SycnronousService to asynchronous implementation and all will be work as expected.
I agree with #joy-dir 's answer. And you should probably do what she said to simplify your testing.
For the sake of completeness, your problem here is that the verify is called before your tasks being actually finished. There are many things you could do.
One is to loop on verify.
#Test
public void test() throws Exception {
List<String> events = new ArrayList();
events.add("http://xxxx//");
events.add("http://xxxx//");
expect(downloader.send(events.get(0))).andReturn("xxx").times(2);
replay(downloader);
report(events);
for (int i = 0; i < 10; i++) {
try {
verify(downloader);
return;
} catch(AssertionError e) {
// wait until it works
}
Thread.sleep(10);
}
verify(downloader);
}
It won't sleep a long time for nothing when successful. However, you indeed need to make sure you wait enough to prevent the test from being flaky.
Another solution is actually to use the CompletableFuture returned by runAsync. I prefer this solution.
public CompletableFuture<Void> report(List<String> srcList) {
if (srcList != null) {
return CompletableFuture.runAsync(() -> srcList.forEach(src-> downloader.send(src)));
}
return CompletableFuture.completedFuture(null);
}
#Test
public void test2() throws Exception {
List<String> events = new ArrayList();
events.add("http://xxxx//");
events.add("http://xxxx//");
expect(downloader.send(events.get(0))).andReturn("xxx").times(2);
replay(downloader);
CompletableFuture<Void> future = report(events);
future.get(100, TimeUnit.MILLISECONDS);
verify(downloader);
}
Finally, there is a hackish way. You ask the common pool if it is done. It is hackish because something else might use it. So it's cute but I wouldn't really recommend it.
#Test
public void test3() throws Exception {
List<String> events = new ArrayList();
events.add("http://xxxx//");
events.add("http://xxxx//");
expect(downloader.send(events.get(0))).andReturn("xxx").times(2);
replay(downloader);
report(events);
while(!ForkJoinPool.commonPool().isQuiescent()) {
Thread.sleep(10);
}
verify(downloader);
}
I've been looking for hints on how to best test Spring MVC Controller methods that return SseEmitters. I have come up pretty short, but have a trial-and-error solution that tests against asynchronous, threaded behavior. The below is sample code just to demonstrate concept, there may be a typo or two:
Controller Class:
#Autowired
Publisher<MyResponse> responsePublisher;
#RequestMapping("/mypath")
public SseEmitter index() throws IOException {
SseEmitter emitter = new SseEmitter();
Observable<MyResponse> responseObservable = RxReactiveStreams.toObservable(responsePublisher);
responseObservable.subscribe(
response -> {
try {
emitter.send(response);
} catch (IOException ex) {
emitter.completeWithError(ex);
}
},
error -> {
emitter.completeWithError(error);
},
emitter::complete
);
return emitter;
}
Test Class:
//A threaded dummy publisher to demonstrate async properties.
//Sends 2 responses with a 250ms pause in between.
protected static class MockPublisher implements Publisher<MyResponse> {
#Override
public void subscribe(Subscriber<? super MyResponse> subscriber) {
new Thread() {
#Override
public void run() {
try {
subscriber.onNext(response1);
Thread.sleep(250);
subscriber.onNext(response2);
} catch (InterruptedException ex) {
}
subscriber.onComplete();
}
}.start();
}
}
//Assume #Configuration that autowires the above mock publisher in the controller.
//Tests the output of the controller method.
#Test
public void testSseEmitter() throws Exception {
String path = "http://localhost/mypath/";
String expectedContent = "data:" + response1.toString() + "\n\n" +
"data:" + response2.toString() + "\n\n");
//Trial-and-Error attempts at testing this SseEmitter mechanism have yielded the following:
//- Returning an SseEmitter triggers 'asyncStarted'
//- Calling 'asyncResult' forces the test to wait for the process to complete
//- However, there is no actual 'asyncResult' to test. Instead, the content is checked for the published data.
mockMvc.perform(get(path).contentType(MediaType.ALL))
.andExpect(status().isOk())
.andExpect(request().asyncStarted())
.andExpect(request().asyncResult(nullValue()))
.andExpect(header().string("Content-Type", "text/event-stream"))
.andExpect(content().string(expectedContent))
}
As noted in the comments, asyncResult() is called to ensure that the publisher finishes its work and sends both responses before the test completes. Without it, the content check fails due to only one response being present in the content. However there is no actual result to check, hence asyncResult is null.
My specific question is whether there is a better, more precise way to force the test to wait for the async process to finish, rather than the klugie method here of waiting for a non-existent asyncResult. My broader question is whether there are other libs or Spring methods that are better suited to this vs. these async functions. Thanks!
This is a more general answer as it is meant to test an SseEmitter that will run forever, but will disconnect from SSE stream after a given timeout.
As for a different approach than MVC, as #ErinDrummond commented to the OP, you might want to investigate WebFlux.
It is a minimal example. One might want to expand with headers to the request, different matchers or maybe work on the stream output separately.
It is setting a delayed thread for disconnecting from SSE Stream which will allow to perform assertions.
#Autowired
MockMvc mockMvc;
#Test
public void testSseEmitter(){
ScheduledExecutorService execService = Executors.newScheduledThreadPool(1);
String streamUri = "/your-get-uri");
long timeout = 500L;
TimeUnit timeUnit = TimeUnit.MILLISECONDS;
MvcResult result = mockMvc.perform(get(streamURI)
.andExpect(request().asyncStarted()).andReturn();
MockAsyncContext asyncContext = (MockAsyncContext) result.getRequest().getAsyncContext();
execService.schedule(() -> {
for (AsyncListener listener : asyncContext.getListeners())
try {
listener.onTimeout(null);
} catch (IOException e) {
e.printStackTrace();
}
}, timeout, timeUnit);
result.getAsyncResult();
// assertions, e.g. response body as string contains "xyz"
mvc.perform(asyncDispatch(result)).andExpect(content().string(containsString("xyz")));
}
I have the following method and I am struggling to get 100% code coverage.
public final class SleepingHelper {
public static void sleepInMillis(Duration timeOfNextTry) {
try {
Thread.sleep(timeOfNextTry.toMillis());
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
The question is how can I force Thread.sleep to throw an exception?
Edit: since it was marked as duplicate, I am still wondering what I would assert in the test ? The other question Is more generic.
You need to interrupt it from another thread. For example:
Thread t = new Thread() {
public void run () {
SleeperMillis.sleepInMillis(new Duration(10000000l));
}
}.start();
Thread.sleep(100); // let the other thread start
t.interrupt;
You don't need to actually interrupt the thread. You can use PowerMockito to mock the static method Thread.sleep()
#RunWith(PowerMockRunner.class)
#PrepareForTest(Thread.class)
public class TestClass {
#Test
public void testSleepInMillis() throws Exception {
PowerMockito.mockStatic(Thread.class);
PowerMockito.doThrow(new InterruptedException ()).when(Thread.class);
try {
SleepHelper.sleepInMillis(11);
fail("expected exception");
} catch (InterruptedException e) {
System.out.println("all good");
}
}
You don't test it, because you can't assert its results, and you can't assert it because Thread.sleep is not accurate or guaranteed to sleep for this duration of time, and the test results will differ from run to run.
Mocking is a better option here.
Btw, it is not just that your tests aren't predictable, your code that uses Thread.sleep in production is going to be unpredictable for the same reasons. Thread.sleep(some magic number goes here) usually indicates a badly written program.
I wouldn't bother testing it. 100% coverage is excessive. However, you could do it like this:
#Test
public void testException() throws Exception {
// Capture the system error stream, so that we can test that the expected exception is printed.
ByteArrayOutputStream capturedErrors = new ByteArrayOutputStream();
System.setErr(new PrintStream(capturedErrors));
// Create a new thread on which to run the candidate method.
Thread thread = new Thread() {
#Override
public void run() {
SleepingHelper.sleepInMillis(Duration.ofMillis(10));
}
};
// Start the second thread.
thread.start();
// Interrupt the second thread. (The candidate method hasn't finished yet. It takes 10 milliseconds to run.)
thread.interrupt();
// Wait for the thread to die (and write out the stack-trace).
thread.join();
// Test that the expected exception's stack trace was printed to the system error stream.
// The output should start with the exception's name.
String output = capturedErrors.toString();
int lengthOfExceptionName = "java.lang.InterruptedException".length();
assertEquals(output.substring(0, lengthOfExceptionName), "java.lang.InterruptedException");
}
The issue is that I have a method starting a new thread for a time-consuming work. I want to test the callback result, but the child thread may still running, so as a result, what I get is not the right stub.
I think the code may explain itself:
public class JustAClass {
//it is a callback for async
public interface JustACallBack {
void callFunc(JustAResult result);
}
//this is the result interface
public interface JustAResult {
}
//this is a real class for the interface
public class JustAResultReal implements JustAResult{
public JustAResultReal(String content) {this.content = content;}
public String content;
}
//here is the key function
public void threadFunc(final JustACallBack callBack) {
BCCache.executorService.execute(new Runnable() {
#Override
public void run() {
//just to simulate a time-consuming task
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
//now we callback
callBack.callFunc(new JustAResultReal("can you reach me"));
}
});
}
}
and the test function could be(I am using mockito):
#Test
public void testThreadFunc() throws Exception {
JustAClass justAClass = new JustAClass();
JustAClass.JustACallBack callBack = Mockito.mock(JustAClass.JustACallBack.class);
justAClass.threadFunc(callBack);
//add this line, we can get the expected result
Thread.sleep(1200);
Mockito.verify(callBack).callFunc(captor.capture());
System.out.println(((JustAClass.JustAResultReal)captor.getValue()).content);
}
I know we can add a sleep to wait and expect that the child thread would exit within the period, but could there be a better way? Actually how could I know how long the child thread would take? Setting a very long time can be an approach but just seems not very nice.
The general approach in #stalet's answer is close, but doesn't quite work since any assertion failures from a separate thread are not noticed by the main thread. Therefore your test will always pass, even when it shouldn't. Instead, try using ConcurrentUnit (which I authored):
#Test
public void testInvoke() throws Throwable {
Waiter waiter = new Waiter();
JustAClass justAClass = new JustAClass();
JustAClass.JustACallBack callBack = new JustAClass.JustACallBack() {
#Override
public void callFunc(final JustAClass.JustAResult result) {
waiter.assertNotNull(result);
waiter.assertTrue(result instanceof JustAClass.JustAResultReal);
waiter.resume();
}
};
justAClass.threadFunc(callBack);
waiter.await(1200, TimeUnit.SECONDS);
}
The key here is ConcurrentUnit's Waiter will properly report any assertions failures to the main test thread and the test will pass or fail as it should.
I aggree with #Gimbys comment about this is no longer a unit-test when you start testing the the threading aspect.
Nevertheless it is interesting as a way to integration-test a asynchronous invokation.
To avvoid sleep i tend to use the class CountDownLatch to wait for invokations.
In order to count down you need an actuall implementation of the callback interface - so in my example I have made a mock implementation of this.
Since there is no actual methods to fetch the data - i am just testing that it is in fact a instance of the JustAReal interface.
#Test
public void testInvoke() throws Exception {
final CountDownLatch countDownLatch = new CountDownLatch(1); //1 is how many invokes we are waiting for
JustAClass justAClass = new JustAClass();
JustAClass.JustACallBack callBack = new JustAClass.JustACallBack() {
#Override
public void callFunc(final JustAClass.JustAResult result) {
assertNotNull("Result should not be null", result);
assertTrue("Result should be instance of JustAResultReal", result instanceof JustAClass.JustAResultReal);
countDownLatch.countDown();
}
};
justAClass.threadFunc(callBack);
if(!countDownLatch.await(1200, TimeUnit.MILLISECONDS)){
fail("Timed out, see log for errors");
}
}