I am trying to create unit test for the following code. The code utilizes AWS Java 2 SDK. The code calls selectObjectContent in S3AsyncClient class which returns a CompletableFuture (https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/S3AsyncClient.html). My test is returning null pointer exception while invoking future.get()
Here is the method I want to unit test.
public <T> Collection<T> queryWithS3Select(
List<String> s3Keys,
String s3SelectQuery,
InputSerialization inputSerialization,
Class<T> modelObject,
Comparator<T> comparator
) throws ExecutionException, InterruptedException, IOException {
TreeSet<T> collection = new TreeSet<>(comparator);
List<SelectObjectContentRequest> selectObjectContentRequest =
buildS3SelectRequests(s3Keys, s3SelectQuery, inputSerialization);
S3SelectContentHandler s3SelectContentHandler = new S3SelectContentHandler();
StringBuilder selectionResult = new StringBuilder();
for (SelectObjectContentRequest socr : selectObjectContentRequest) {
CompletableFuture<Void> future = s3AsyncClient.selectObjectContent(socr, s3SelectContentHandler);
future.get();
s3SelectContentHandler.getReceivedEvents().forEach(e -> {
if (e.sdkEventType() == SelectObjectContentEventStream.EventType.RECORDS) {
RecordsEvent response = (RecordsEvent) e;
selectionResult.append(response.payload().asUtf8String());
}
});
}
JsonParser parser = objectMapper.createParser(selectionResult.toString());
collection.addAll(Lists.newArrayList(objectMapper.readValues(parser, modelObject)));
return collection;
}
My unit test so far. Running this code I get null pointer exception at future.get() line above. How can I use the mock s3AsyncClient to return a valid future?
#Mock
private S3AsyncClient s3AsyncClient;
#Test
public void itShouldReturnQueryResults() throws IOException, ExecutionException, InterruptedException {
List<String> keysToQuery = List.of("key1", "key2");
InputSerialization inputSerialization = InputSerialization.builder()
.json(JSONInput.builder().type(JSONType.DOCUMENT).build())
.compressionType(String.valueOf(CompressionType.GZIP))
.build();
Comparator<S3SelectObject> comparator =
Comparator.comparing((S3SelectObject e) -> e.getStartTime());
underTest.queryWithS3Select(keysToQuery, S3_SELECT_QUERY, inputSerialization, S3SelectObject.class, comparator );
}
Here is the S3SelectContentHandler
public class S3SelectContentHandler implements SelectObjectContentResponseHandler {
private SelectObjectContentResponse response;
private List<SelectObjectContentEventStream> receivedEvents = new ArrayList<>();
private Throwable exception;
#Override
public void responseReceived(SelectObjectContentResponse response) {
this.response = response;
}
#Override
public void onEventStream(SdkPublisher<SelectObjectContentEventStream> publisher) {
publisher.subscribe(receivedEvents::add);
}
#Override
public void exceptionOccurred(Throwable throwable) {
exception = throwable;
}
#Override
public void complete() {}
public List<SelectObjectContentEventStream> getReceivedEvents() {
return receivedEvents;
}
}
I will share unit test for similar functionality and show you how to work with completable future when your code has .join() to continue the execution
Code under test
private final S3AsyncClient s3AsyncClient;
public long getSize(final S3AsyncClient client, final String bucket, final String key) {
return client.headObject(HeadObjectRequest.builder().bucket(bucket).key(key).build()).join().contentLength();
}
and in this code the client.headObject() returns the CompletableFuture<HeadObjectResponse> which we are going to mock and test in Unit Test as shown below
#Test
#DisplayName("Verify getSize returns the size of the given key in the bucket")
void verifyGetSizeRetunsSizeOfFileInS3() {
CompletableFuture<HeadObjectResponse> headObjectResponseCompletableFuture =
CompletableFuture.completedFuture(HeadObjectResponse.builder().contentLength(20000L).build());
when(s3AsyncClient.headObject(headObjectRequestArgumentCaptor.capture()))
.thenReturn(headObjectResponseCompletableFuture);
long size = s3Service.getSize(s3AsyncClient, "somebucket", "someFile");
assertThat(headObjectRequestArgumentCaptor.getValue()).hasFieldOrPropertyWithValue("bucket", "somebucket")
.hasFieldOrPropertyWithValue("key", "someFile");
assertThat(size).isEqualTo(20000L);
}
Related
I have a FileRepository class to fill MutableLiveData. I have two functions in there. One is calling web service and setting list (getAllFilms). Other one is assigning MutableLiveData with first function List (getFilmData). I try to write unit test for MutableLiveData. Can you help me? This function data always comes null.
public class FilmRepositoryTest
#Mock
FilmRepository frepo;
#Rule
public InstantTaskExecutorRule instantTaskExecutorRule = new InstantTaskExecutorRule();
#Before
public void setUp() throws Exception {
}
#Test
public void getFilmData_forData() throws IOException {
ArrayList<film> filmlistesi = new ArrayList<>();
MutableLiveData<List<film>> bilgi = new MutableLiveData<>();
String arama = "ankara";
filmlistesi.add(new film("Behzat Ç.: Bir Ankara Polisiyesi","2010–2019","tt1795096","series","https://m.media-amazon.com/images/M/MV5BZDZjY2I5ZjEtZGE2MS00ZjRmLTlmMGEtMDQ5ZmZhZWJjYzk3XkEyXkFqcGdeQXVyNDg4MjkzNDk#._V1_SX300.jpg"));
when(frepo.getAllFilms(arama)).thenReturn(filmlistesi);
bilgi.setValue(filmlistesi);
System.out.println(frepo.getFilmData(arama)); // print(NULL)
System.out.println(bilgi.getValue()); // print(filmlistesi)
assertEquals(frepo.getFilmData(arama),bilgi.getValue());
}
public class FilmRepository
public MutableLiveData<List<film>> getFilmData(String a) throws IOException {
MutableLiveData<List<film>> data = new MutableLiveData<>();
data.setValue(getAllFilms(a));
return data;
}
public ArrayList getAllFilms(String filmName) throws IOException {
ArrayList<film> dataset = new ArrayList<>();
return dataset;
}
LiveData or MutableLiveData must have an Observer set of it or else it returns null
Observer<List<film>> observer = new Observer<List<film>>() {
#Override
public void onChanged(#Nullable List<film> films) {
assertEquals(frepo.getFilmData(arama), films.getValue());
biligi.removeObserver(this)
}
};
bilgi.observeForever(observer);
bilgi.setValue(filmlistesi);
I have an interface:
public interface SenderService {
String send(long amount);
}
And I have an implementation of this interface:
public class SenderServiceAdapter implements SenderService {
private final ThirdPartyService thirdPartyService;
public SenderServiceAdapter(ThirdPartyService thirdPartyService) {
this.thirdPartyService = thirdPartyService;
}
#Override
public String send(long amount) {
ThirdPartyRequest thirdPartyRequest = new ThirdPartyRequest();
thirdPartyRequest.setAmount(amount);
thirdPartyRequest.setId(UUID.randomUUID().toString());
thirdPartyRequest.setDate(new Date());
ThirdPartyResponse thirdPartyResponse = thirdPartyService.send(thirdPartyRequest);
String status = thirdPartyResponse.getStatus();
if (status.equals("Error")) throw new RuntimeException("blablabla");
return thirdPartyResponse.getMessage();
}
}
Now I want to write Unit test for this service. I need to mock thirdPartyService's method send. But I don't understand how.
public class SenderServiceAdapterTest {
private ThirdPartyService thirdPartyService;
private SenderService senderService;
#Before
public void setUp() throws Exception {
thirdPartyService = Mockito.mock(ThirdPartyService.class);
senderService = new SenderServiceAdapter(thirdPartyService);
}
#Test
public void send() {
when(thirdPartyService.send(new ThirdPartyRequest())).thenReturn(new ThirdPartyResponse());
String message = senderService.send(100L);
}
}
The ThirdPartyRequest creates in SenderServiceAdapter. How can I mock it?
Try this:
doReturn(new ThirdPartyResponse()).when(thirdPartyService).send(any(ThirdPartyRequest.class));
Also by looking your code, you will need to set something in the response, so you will have to do this:
ThirdPartyResponse response = new ThirdPartyResponse(); //or mock
response.setStatus(...);
response.setMessage(...);
doReturn(response).when(thirdPartyService).send(any(ThirdPartyRequest.class));
I have the code below:
private SaveTransactionClient mockedTransactionClient;
private static Publisher publisher;
private static MyDTO mDTO1;
private static MyDTO mDTO2;
private static MyDTO mDTO3;
#BeforeClass
public void setUp() throws IOException {
TransactionResponse successResponse = new TransactionResponse();
successResponse.setDateRequest("2016-04-27 18:47:50");
successResponse.setResponse("OK");
successResponse.setTransactionNumber("1");
TransactionResponse failedResponse = new TransactionResponse();
failedResponse.setDateRequest("2016-04-27 18:47:50");
failedResponse.setResponse("Dublicate Transaction Error");
failedResponse.setTransactionNumber("1");
mDTO1 = new MyDTO(1, LocalDateTime.now(), 0);
mDTO2 = new MyDTO(2, LocalDateTime.now(), 0);
mDTO3 = new MyDTO(3, LocalDateTime.now(), 0);
mockedTransactionClient = mock(SaveTransactionClient.class);
when(mockedTransactionClient.sendTransactionRequest(mDTO1)).thenReturn(successResponse);
when(mockedTransactionClient.sendTransactionRequest(mDTO2)).thenReturn(failedResponse);
when(mockedTransactionClient.sendTransactionRequest(mDTO3)).thenThrow(new IOException());
when(mockedTransactionClient.sendTransactionRequest(any(MDTO.class))).thenThrow(new IOException());
publisher = new publisherImpl(mockedTransactionClient);
}
The actual tests are
#Test
public void TestOnlyExceptionalPublishing() {
BlockingQueue<MDTO> mDTOs = new LinkedBlockingQueue<>(Arrays.asList(mDTO3));
assertEquals(mDTOs.size(), 1);
List<MDTO> successful = publisher.publish(wDTOs);
assertEquals(successful.size(), 0);
}
#Test
public void TestOneSuccessContainsExceptionalPublishing() {
BlockingQueue<MDTO> mDTOs = new LinkedBlockingQueue<>(Arrays.asList(mDTO3,mDTO1, mDTO2));
assertEquals(mDTOs.size(), 3);
List<MDTO> successful = publisher.publish(mDTOs);
assertEquals(successful.size(), 1);
}
Now the MDTO is immutable and the way the publisher.publish(mDTO) works is that on Exception increments a "retries" counter in MDTO and retry up to 3 times. When the counter is incremented a new MDTO is generated which, with the response not being stubbed by Mockito, creates a problem for me. I added the any but this makes the other tests fail since it throws for all and not for any OTHER except for the objects I have already supplied.
Is there any anyOther type of way to do this in Mockito?
I think (if I did understand correctly) that you are looking for doAnswer/thenAnswer:
when(mockedTransactionClient.sendTransactionRequest(any(MyDTO.class)).thenAnswer(new Answer<TransactionResponse>() {
#Override
public TransactionResponse answer(final InvocationOnMock invocation) {
final MyDTO arg = invocation.getArgumentAt(0, MyDTO.class)
// do stuff here based on arg
return someTransactionResponse; // or throw some exception
}
});
I have a library which is being used by customer and they are passing DataRequest object which has userid, timeout and some other fields in it. Now I use this DataRequest object to make a URL and then I make an HTTP call using RestTemplate and my service returns back a JSON response which I use it to make a DataResponse object and return this DataResponse object back to them.
Below is my DataClient class used by customer by passing DataRequest object to it. I am using timeout value passed by customer in DataRequest to timeout the request if it is taking too much time in getSyncData method.
public class DataClient implements Client {
private RestTemplate restTemplate = new RestTemplate();
// first executor
private ExecutorService service = Executors.newFixedThreadPool(15);
#Override
public DataResponse getSyncData(DataRequest key) {
DataResponse response = null;
Future<DataResponse> responseFuture = null;
try {
responseFuture = getAsyncData(key);
response = responseFuture.get(key.getTimeout(), key.getTimeoutUnit());
} catch (TimeoutException ex) {
response = new DataResponse(DataErrorEnum.CLIENT_TIMEOUT, DataStatusEnum.ERROR);
responseFuture.cancel(true);
// logging exception here
}
return response;
}
#Override
public Future<DataResponse> getAsyncData(DataRequest key) {
DataFetcherTask task = new DataFetcherTask(key, restTemplate);
Future<DataResponse> future = service.submit(task);
return future;
}
}
DataFetcherTask class:
public class DataFetcherTask implements Callable<DataResponse> {
private DataRequest key;
private RestTemplate restTemplate;
public DataFetcherTask(DataRequest key, RestTemplate restTemplate) {
this.key = key;
this.restTemplate = restTemplate;
}
#Override
public DataResponse call() throws Exception {
// In a nutshell below is what I am doing here.
// 1. Make an url using DataRequest key.
// 2. And then execute the url RestTemplate.
// 3. Make a DataResponse object and return it.
// I am calling this whole logic in call method as LogicA
}
}
As of now my DataFetcherTask class is responsible for one DataRequest key as shown above..
Problem Statement:-
Now I have a small design change. Customer will pass DataRequest (for example keyA) object to my library and then I will make a new http call to another service (which I am not doing in my current design) by using user id present in DataRequest (keyA) object which will give me back list of user id's so I will use those user id's and make few other DataRequest (keyB, keyC, keyD) objects one for each user id returned in the response. And then I will have List<DataRequest> object which will have keyB, keyC and keyD DataRequest object. Max element in the List<DataRequest> will be three, that's all.
Now for each of those DataRequest object in List<DataRequest> I want to execute above DataFetcherTask.call method in parallel and then make List<DataResponse> by adding each DataResponse for each key. So I will have three parallel calls to DataFetcherTask.call. Idea behind this parallel call is to get the data for all those max three keys in the same global timeout value.
So my proposal is - DataFetcherTask class will return back List<DataResponse> object instead of DataResponse and then signature of getSyncData and getAsyncData method will change as well. So here is the algorithm:
Use DataRequest object passed by customer to make List<DataRequest> by calling another HTTP service.
Make a parallel call for each DataRequest in List<DataRequest> to DataFetcherTask.call method and return List<DataResponse> object to customer instead of DataResponse.
With this way, I can apply same global timeout on step 1 along with step 2 as well. If either of above step is taking time, we will just timeout in getSyncData method.
DataFetcherTask class after design change:
public class DataFetcherTask implements Callable<List<DataResponse>> {
private DataRequest key;
private RestTemplate restTemplate;
// second executor here
private ExecutorService executorService = Executors.newFixedThreadPool(10);
public DataFetcherTask(DataRequest key, RestTemplate restTemplate) {
this.key = key;
this.restTemplate = restTemplate;
}
#Override
public List<DataResponse> call() throws Exception {
List<DataRequest> keys = generateKeys();
CompletionService<DataResponse> comp = new ExecutorCompletionService<>(executorService);
int count = 0;
for (final DataRequest key : keys) {
comp.submit(new Callable<DataResponse>() {
#Override
public DataResponse call() throws Exception {
return performDataRequest(key);
}
});
}
List<DataResponse> responseList = new ArrayList<DataResponse>();
while (count-- > 0) {
Future<DataResponse> future = comp.take();
responseList.add(future.get());
}
return responseList;
}
// In this method I am making a HTTP call to another service
// and then I will make List<DataRequest> accordingly.
private List<DataRequest> generateKeys() {
List<DataRequest> keys = new ArrayList<>();
// use key object which is passed in contructor to make HTTP call to another service
// and then make List of DataRequest object and return keys.
return keys;
}
private DataResponse performDataRequest(DataRequest key) {
// This will have all LogicA code here which is shown in my original design.
// everything as it is same..
}
}
Now my question is -
Does it have to be like this? What is the right design to solve this problem? I mean having call method in another call method looks weird?
Do we need to have two executors like I have in my code? Is there any better way to solve this problem or any kind of simplification/design change we can do here?
I have simplified the code so that idea gets clear what I am trying to do..
As already mentioned in the comments of your question, you can use Java's ForkJoin framework. This will save you the extra thread pool within your DataFetcherTask.
You simply need to use a ForkJoinPool in your DataClient and convert your DataFetcherTask into a RecursiveTask (one of ForkJoinTask's subtypes). This allows you to easily execute other subtasks in parallel.
So, after these modifications your code will look something like this:
DataFetcherTask
The DataFetcherTask is now a RecursiveTask which first generates the keys and invokes subtasks for each generated key. These subtasks are executed in the same ForkJoinPool as the parent task.
public class DataFetcherTask extends RecursiveTask<List<DataResponse>> {
private final DataRequest key;
private final RestTemplate restTemplate;
public DataFetcherTask(DataRequest key, RestTemplate restTemplate) {
this.key = key;
this.restTemplate = restTemplate;
}
#Override
protected List<DataResponse> compute() {
// Create subtasks for the key and invoke them
List<DataRequestTask> requestTasks = requestTasks(generateKeys());
invokeAll(requestTasks);
// All tasks are finished if invokeAll() returns.
List<DataResponse> responseList = new ArrayList<>(requestTasks.size());
for (DataRequestTask task : requestTasks) {
try {
responseList.add(task.get());
} catch (InterruptedException | ExecutionException e) {
// TODO - Handle exception properly
Thread.currentThread().interrupt();
return Collections.emptyList();
}
}
return responseList;
}
private List<DataRequestTask> requestTasks(List<DataRequest> keys) {
List<DataRequestTask> tasks = new ArrayList<>(keys.size());
for (DataRequest key : keys) {
tasks.add(new DataRequestTask(key));
}
return tasks;
}
// In this method I am making a HTTP call to another service
// and then I will make List<DataRequest> accordingly.
private List<DataRequest> generateKeys() {
List<DataRequest> keys = new ArrayList<>();
// use key object which is passed in contructor to make HTTP call to another service
// and then make List of DataRequest object and return keys.
return keys;
}
/** Inner class for the subtasks. */
private static class DataRequestTask extends RecursiveTask<DataResponse> {
private final DataRequest request;
public DataRequestTask(DataRequest request) {
this.request = request;
}
#Override
protected DataResponse compute() {
return performDataRequest(this.request);
}
private DataResponse performDataRequest(DataRequest key) {
// This will have all LogicA code here which is shown in my original design.
// everything as it is same..
return new DataResponse(DataErrorEnum.OK, DataStatusEnum.OK);
}
}
}
DataClient
The DataClient will not change much except for the new thread pool:
public class DataClient implements Client {
private final RestTemplate restTemplate = new RestTemplate();
// Replace the ExecutorService with a ForkJoinPool
private final ForkJoinPool service = new ForkJoinPool(15);
#Override
public List<DataResponse> getSyncData(DataRequest key) {
List<DataResponse> responsList = null;
Future<List<DataResponse>> responseFuture = null;
try {
responseFuture = getAsyncData(key);
responsList = responseFuture.get(key.getTimeout(), key.getTimeoutUnit());
} catch (TimeoutException | ExecutionException | InterruptedException ex) {
responsList = Collections.singletonList(new DataResponse(DataErrorEnum.CLIENT_TIMEOUT, DataStatusEnum.ERROR));
responseFuture.cancel(true);
// logging exception here
}
return responsList;
}
#Override
public Future<List<DataResponse>> getAsyncData(DataRequest key) {
DataFetcherTask task = new DataFetcherTask(key, this.restTemplate);
return this.service.submit(task);
}
}
Once you are on Java8 you may consider changing the implementation to CompletableFutures. Then it would look something like this:
DataClientCF
public class DataClientCF {
private final RestTemplate restTemplate = new RestTemplate();
private final ExecutorService executor = Executors.newFixedThreadPool(15);
public List<DataResponse> getData(DataRequest initialKey) {
return CompletableFuture.supplyAsync(() -> generateKeys(initialKey), this.executor)
.thenApply(requests -> requests.stream().map(this::supplyRequestAsync).collect(Collectors.toList()))
.thenApply(responseFutures -> responseFutures.stream().map(future -> future.join()).collect(Collectors.toList()))
.exceptionally(t -> { throw new RuntimeException(t); })
.join();
}
private List<DataRequest> generateKeys(DataRequest key) {
return new ArrayList<>();
}
private CompletableFuture<DataResponse> supplyRequestAsync(DataRequest key) {
return CompletableFuture.supplyAsync(() -> new DataResponse(DataErrorEnum.OK, DataStatusEnum.OK), this.executor);
}
}
As mentioned in the comments, Guava's ListenableFutures would provide similar functionality for Java7 but without Lambdas they tend to get clumsy.
As I know, RestTemplate is blocking, it is said in ForkJoinPool JavaDoc in ForkJoinTask:
Computations should avoid synchronized methods or blocks, and should minimize other blocking synchronization apart from joining other tasks or using synchronizers such as Phasers that are advertised to cooperate with fork/join scheduling. ...
Tasks should also not perform blocking IO,...
Call in call is redundant.
And you don't need two executors. Also you can return partial result in getSyncData(DataRequest key). This can be done like this
DataClient.java
public class DataClient implements Client {
private RestTemplate restTemplate = new RestTemplate();
// first executor
private ExecutorService service = Executors.newFixedThreadPool(15);
#Override
public List<DataResponse> getSyncData(DataRequest key) {
List<DataResponse> responseList = null;
DataFetcherResult response = null;
try {
response = getAsyncData(key);
responseList = response.get(key.getTimeout(), key.getTimeoutUnit());
} catch (TimeoutException ex) {
response.cancel(true);
responseList = response.getPartialResult();
}
return responseList;
}
#Override
public DataFetcherResult getAsyncData(DataRequest key) {
List<DataRequest> keys = generateKeys(key);
final List<Future<DataResponse>> responseList = new ArrayList<>();
final CountDownLatch latch = new CountDownLatch(keys.size());//assume keys is not null
for (final DataRequest _key : keys) {
responseList.add(service.submit(new Callable<DataResponse>() {
#Override
public DataResponse call() throws Exception {
DataResponse response = null;
try {
response = performDataRequest(_key);
} finally {
latch.countDown();
return response;
}
}
}));
}
return new DataFetcherResult(responseList, latch);
}
// In this method I am making a HTTP call to another service
// and then I will make List<DataRequest> accordingly.
private List<DataRequest> generateKeys(DataRequest key) {
List<DataRequest> keys = new ArrayList<>();
// use key object which is passed in contructor to make HTTP call to another service
// and then make List of DataRequest object and return keys.
return keys;
}
private DataResponse performDataRequest(DataRequest key) {
// This will have all LogicA code here which is shown in my original design.
// everything as it is same..
return null;
}
}
DataFetcherResult.java
public class DataFetcherResult implements Future<List<DataResponse>> {
final List<Future<DataResponse>> futures;
final CountDownLatch latch;
public DataFetcherResult(List<Future<DataResponse>> futures, CountDownLatch latch) {
this.futures = futures;
this.latch = latch;
}
//non-blocking
public List<DataResponse> getPartialResult() {
List<DataResponse> result = new ArrayList<>(futures.size());
for (Future<DataResponse> future : futures) {
try {
result.add(future.isDone() ? future.get() : null);
//instead of null you can return new DataResponse(DataErrorEnum.NOT_READY, DataStatusEnum.ERROR);
} catch (InterruptedException | ExecutionException e) {
e.printStackTrace();
//ExecutionException or CancellationException could be thrown, especially if DataFetcherResult was cancelled
//you can handle them here and return DataResponse with corresponding DataErrorEnum and DataStatusEnum
}
}
return result;
}
#Override
public List<DataResponse> get() throws ExecutionException, InterruptedException {
List<DataResponse> result = new ArrayList<>(futures.size());
for (Future<DataResponse> future : futures) {
result.add(future.get());
}
return result;
}
#Override
public List<DataResponse> get(long timeout, TimeUnit timeUnit)
throws ExecutionException, InterruptedException, TimeoutException {
if (latch.await(timeout, timeUnit)) {
return get();
}
throw new TimeoutException();//or getPartialResult()
}
#Override
public boolean cancel(boolean mayInterruptIfRunning) {
boolean cancelled = true;
for (Future<DataResponse> future : futures) {
cancelled &= future.cancel(mayInterruptIfRunning);
}
return cancelled;
}
#Override
public boolean isCancelled() {
boolean cancelled = true;
for (Future<DataResponse> future : futures) {
cancelled &= future.isCancelled();
}
return cancelled;
}
#Override
public boolean isDone() {
boolean done = true;
for (Future<DataResponse> future : futures) {
done &= future.isDone();
}
return done;
}
//and etc.
}
I wrote it with CountDownLatch and it looks great, but note there is a nuance.
You can get stuck for a little while in DataFetcherResult.get(long timeout, TimeUnit timeUnit) because CountDownLatch is not synchronized with future's state. And it could happen that latch.getCount() == 0 but not all futures would return future.isDone() == true at the same time. Because they have already passed latch.countDown(); inside finally {} Callable's block but didn't change internal state which is still equals to NEW.
And so calling get() inside get(long timeout, TimeUnit timeUnit) can cause a small delay.
Similar case was described here.
Get with timeout DataFetcherResult.get(...) could be rewritten using futures future.get(long timeout, TimeUnit timeUnit) and you can remove CountDownLatch from a class.
public List<DataResponse> get(long timeout, TimeUnit timeUnit)
throws ExecutionException, InterruptedException{
List<DataResponse> result = new ArrayList<>(futures.size());
long timeoutMs = timeUnit.toMillis(timeout);
boolean timeout = false;
for (Future<DataResponse> future : futures) {
long beforeGet = System.currentTimeMillis();
try {
if (!timeout && timeoutMs > 0) {
result.add(future.get(timeoutMs, TimeUnit.MILLISECONDS));
timeoutMs -= System.currentTimeMillis() - beforeGet;
} else {
if (future.isDone()) {
result.add(future.get());
} else {
//result.add(new DataResponse(DataErrorEnum.NOT_READY, DataStatusEnum.ERROR)); ?
}
}
} catch (TimeoutException e) {
result.add(new DataResponse(DataErrorEnum.TIMEOUT, DataStatusEnum.ERROR));
timeout = true;
}
//you can also handle ExecutionException or CancellationException here
}
return result;
}
This code was given as an example and it should be tested before using in production, but seems legit :)
I have the following method go() I'd like to test:
private Pair<String, String> mPair;
public void go() {
Observable.zip(
mApi.webCall(),
mApi.webCall2(),
new Func2<String, String, Pair<String, String>>() {
#Override
public Pair<String, String> call(String s, String s2) {
return new Pair(s, s2);
}
}
)
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(new Action1<Pair<String, String>>() {
#Override
public void call(Pair<String, String> pair) {
mApi.webCall3(pair.first, pair.second);
}
});
}
This method uses Observable.zip() to execute to http requests asynchronously, and merge them together in one Pair. In the end, another http request is executed with the result of these previous requests.
I'd like to verify that calling the go() method makes the webCall() and webCall2() requests, followed by the webCall3(String, String) request. Therefore, I'd like the following test to pass (using Mockito to spy the Api):
#Test
public void testGo() {
/* Given */
Api api = spy(new Api() {
#Override
public Observable<String> webCall() {
return Observable.just("First");
}
#Override
public Observable<String> webCall2() {
return Observable.just("second");
}
#Override
public void webCall3() {
}
});
Test test = new Test(api);
/* When */
test.go();
/* Then */
verify(api).webCall();
verify(api).webCall2();
verify(api).webCall3("First", "second");
}
However when running this, web calls are executed asynchronously, and my test executes the assertion before the subscriber is done causing my test to fail.
I have read that you can use RxJavaSchedulersHook and RxAndroidSchedulersHook to return Schedulers.immediate() for all methods, but this results in the test running indefinitely.
I am running my unit tests on a local JVM.
How can I achieve this, preferably without having to modify the signature of go()?
(Lambdas thanks to retrolambda)
For starters, I would rephrase go as:
private Pair<String, String> mPair;
public Observable<Pair<String, String>> go() {
return Observable.zip(
mApi.webCall(),
mApi.webCall2(),
(String s, String s2) -> new Pair(s, s2)
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.doOnNext(pair -> mPair = pair);
}
public Pair<String, String> getPair() {
return mPair;
}
doOnNext allows you to intercept the value that is being processed in the chain whenever someone will subscribe to the Observable
Then, I would call the test like that:
Pair result = test.go().toBlocking().lastOrDefault(null);
Then you can test what result is.
I would use the TestScheduler and TestSubscriber in your tests. In order to use this you'll have to receive the observable that composes the zip so you can subscribe to that work with the testScheduler. Also you'll have to parameterize your schedulers. You won't have to change your go method's signature but you would have to parameterize the schedulers in underlying functionality. You could inject the schedulers by constructor, override a protected field by inheritance, or call to a package protected overload. I have written my examples assuming an overload that accepts the schedulers as arguments and returns the Observable.
The TestScheduler gives you a synchronous way to trigger async operator behavior in a predictable reproducible way. The TestSubscriber gives you a way to await termination and assert over values and signals received. Also you might want to be aware that the delay(long, TimeUnit) operator by default schedules work on the computation scheduler. You'll need to use the testScheduler there as well.
Scheduler ioScheduler = Schedulers.io();
Scheduler mainThreadScheduler = AndroidSchedulers.mainThread();
public void go() {
go(subscribeOnScheduler, mainThreadScheduler).toBlocking().single();
}
/*package*/ Observable<Pair<String, String>> go(Scheduler ioScheduler, Scheduler mainThreadScheduler) {
return Observable.zip(
mApi.webCall(),
mApi.webCall2(),
new Func2<String, String, Pair<String, String>>() {
#Override
public Pair<String, String> call(String s, String s2) {
return new Pair(s, s2);
}
})
.doOnNext(new Action1<Pair<String, String>>() {
#Override
public void call(Pair<String, String>() {
mApi.webCall3(pair.first, pair.second);
})
})
.subscribeOn(ioScheduler)
.observeOn(mainThreadScheduler);
}
Test code
#Test
public void testGo() {
/* Given */
TestScheduler testScheduler = new TestScheduler();
Api api = spy(new Api() {
#Override
public Observable<String> webCall() {
return Observable.just("First").delay(1, TimeUnit.SECONDS, testScheduler);
}
#Override
public Observable<String> webCall2() {
return Observable.just("second");
}
#Override
public void webCall3() {
}
});
Test test = new Test(api);
/* When */
test.go(testScheduler, testScheduler).subscribe(subscriber);
testScheduler.triggerActions();
subscriber.awaitTerminalEvent();
/* Then */
verify(api).webCall();
verify(api).webCall2();
verify(api).webCall3("First", "second");
}
I have found out that I can retrieve my Schedulers in a non-static way, basically injecting them into my client class. The SchedulerProvider replaces the static calls to Schedulers.x():
public interface SchedulerProvider {
Scheduler io();
Scheduler mainThread();
}
The production implementation delegates back to Schedulers:
public class SchedulerProviderImpl implements SchedulerProvider {
public static final SchedulerProvider INSTANCE = new SchedulerProviderImpl();
#Override
public Scheduler io() {
return Schedulers.io();
}
#Override
public Scheduler mainThread() {
return AndroidSchedulers.mainThread();
}
}
However, during tests I can create a TestSchedulerProvider:
public class TestSchedulerProvider implements SchedulerProvider {
private final TestScheduler mIOScheduler = new TestScheduler();
private final TestScheduler mMainThreadScheduler = new TestScheduler();
#Override
public TestScheduler io() {
return mIOScheduler;
}
#Override
public TestScheduler mainThread() {
return mMainThreadScheduler;
}
}
Now I can inject the SchedulerProvider in to the Test class containing the go() method:
class Test {
/* ... */
Test(Api api, SchedulerProvider schedulerProvider) {
mApi = api;
mSchedulerProvider = schedulerProvider;
}
void go() {
Observable.zip(
mApi.webCall(),
mApi.webCall2(),
new Func2<String, String, Pair<String, String>>() {
#Override
public Pair<String, String> call(String s, String s2) {
return new Pair(s, s2);
}
}
)
.subscribeOn(mSchedulerProvider.io())
.observeOn(mSchedulerProvider.mainThread())
.subscribe(new Action1<Pair<String, String>>() {
#Override
public void call(Pair<String, String> pair) {
mApi.webCall3(pair.first, pair.second);
}
});
}
}
Testing this works as follows:
#Test
public void testGo() {
/* Given */
TestSchedulerProvider testSchedulerProvider = new TestSchedulerProvider();
Api api = spy(new Api() {
#Override
public Observable<String> webCall() {
return Observable.just("First");
}
#Override
public Observable<String> webCall2() {
return Observable.just("second");
}
#Override
public void webCall3() {
}
});
Test test = new Test(api, testSchedulerProvider);
/* When */
test.go();
testSchedulerProvider.io().triggerActions();
testSchedulerProvider.mainThread().triggerActions();
/* Then */
verify(api).webCall();
verify(api).webCall2();
verify(api).webCall3("First", "second");
}
I had a similar issue that took one more step in order to be solved.:
existingObservable
.zipWith(Observable.interval(100, TimeUnit.MILLISECONDS), new Func1<> ...)
.subscribeOn(schedulersProvider.computation())
Was still not using the provided TestScheduler schedulersProvider returned. It was necessary to specify .subscribeOn() on the individual streams that i was zipping in order to work.:
existingObservable.subscribeOn(schedulersProvider.computation())
.zipWith(Observable.interval(100, TimeUnit.MILLISECONDS).subscribeOn(schedulersProvider.computation()), new Func1<> ...)
.subscribeOn(schedulersProvider.computation())
Note that schedulersProvider is a mock returning the TestScheduler of my Test!