Multiple WS calls in one Java Play action - java

I am developing a Play 2 (v2.2.3) Java application (with Java 8) whose task is to fetch data from two web services and return a composed result.
From my first WS call I get a JSON array object with let's say job listings. Now for every job in this collection I want to call another WS which gives me something like the stock price of the company offering the job. After all this I expect my action to return this new 'enhanced' JSON array where along with each job listing I also see the stock price.
I started putting something together and I managed to get the list of jobs back but somehow the part where I glue the two call together is not working/being called.
public static Promise<Result> index() {
final Promise<WS.Response> resultPromiseJobs = WS.url("https://jobs.github.com/positions.json?description=java").get();
final Promise<Result> result = resultPromiseJobs.flatMap(response -> {
JsonNode jobListings = response.asJson();
// convert jobListings to a collection
List<Job> jobs = ...
// call map on this collection to fetch stock prices for each element
jobs.stream().map(job -> {
Promise<WS.Response> resultPromiseStock = WS.url("secondURL?someQuery=job").get()); // this also returns a promise
// so let's map it to something
// ??? what to map to and how to convert all this to a 'Result'
resultPromiseStock.map();
...
});
});
return result; // what is the correct object to return here?
}
This is basically the same question but about Scala, I am interested in the Java solution

From our chat I can see that you want to you want to map each WS.Response to a JsonNode, with will leave you with a List<Promise<JsonNode>>. Promise.sequence can be used to convert that List<Promise<JsonNode>> to a Promise<List<JsonNode>>, which you can then map to Promise<Result>.
public static Promise<Result> index() {
final Promise<WS.Response> resultPromiseJobs = WS.url("https://jobs.github.com/positions.json?description=java").get();
final Promise<Result> result = resultPromiseJobs.flatMap(response -> {
JsonNode jobListings = response.asJson();
List<Job> jobs = ...
List<Promise<JsonNode>> jsonResultsPromise = jobs.stream().map(job -> {
Promise<WS.Response> resultPromiseStock = WS.url("secondURL?someQuery=job").get()); // this also returns a promise
// map this to the retrieved JsonNode
resultPromiseStock.map(...);
});
// Convert the list of promises into a promise of lists
Promise<List<JsonNode>> jsonResults = Promise.sequence(jsonResultsPromise);
// Now map to Promise<Result>
jsonResults.map(...);
});
return result;
}
The catch here though is that if one of the Promises fails to complete, the entire list will fail. In many cases you might want this to happen if you need all the results. It's easy enough to discard failed Promises from a list in Scala using this method, but I'm not sure how to do it in Java without fold.

Related

Quarkus Mutiny Uni/Multi wait for the request response to finish

I can't find a correct solution to this problem and I'm stuck. Let's say I have this method
#GET
#Path("/testingAsync")
public Uni<List<String>> testingMutiny() {
List<String> completeList = new ArrayList<>();
completeList.add("hello");
completeList.add("RestEasy");
List<String> finalList = new ArrayList<>();
completeList.forEach(e -> Uni.createFrom().item(e)
.onItem().delayIt().by(Duration.ofMillis(10000))
.map(value -> finalList.add(value.toUpperCase()))
.subscribe().asCompletionStage());
return Uni.createFrom().item(finalList);
}
As you see the method is simple it just takes the values from 1 list and adds them to the second one but what's the problem? When you add the waiting .onItem().delayIt().by(Duration.ofMillis(10000)) the program will return an empty list and after a while, it will just update the list. I created this method to simulate a request that the response that has some delay in it.
Let's say you hit 2 URLs with 2 different Unis after that you try to combine them and return it as one Uni. The problem is if one of those 2 URLs delay for some reason we will return the list empty but I don't want that to happen I either want the list to be completed 100% or return an error if it takes a while.
What is the best approach to that? I understand that if you add await() you are blocking the main thread and you lose all the value of using the reactive library but still, I can't find a way for this to work
EDIT
I have found out that the external URL I try to call takes about 5 seconds to do the job so I want my code to stop when creating the Uni and continue after I have received an answer from the server. I have seen in their docs (here) That I can also call await.indefinitely but I receive The current thread cannot be blocked: vert.x-eventloop-thread-14. How do I wait for a response from the server?
EDIT 2
I understand that with strings it doesn't make sense my question so much so let's say I have the following one
#GET
#Path("/testingAsync")
public Uni<List<Car>> testingMutiny() {
//ALL THIS IS IN A FOR EACH FOR EVERY CAR
//HIT ENDPOINT GET DOORS
Uni<List<JsonObjectCar>> carDoorsUni = getDoors(variable1,
variable2, variable3);
//HIT ENDPOINT GET WHEELS
Uni<List<JsonObjectCar>> carWheelsUni = getWheels(variable1,
variable2, variable3);
//HIT ENDPOINT GET WINDOWS
Uni<List<JsonObjectCar>> carWindowsUni = getWindows(variable1,
variable2, variable3);
Uni.combine()
.all()
.unis(carDoorsUni, carWheelsUni, carWindowsUni)
.combinedWith((carDoors, carWheels, carWindows) -> {
//Check if cardoors is present and set the doors into the car object
Optional.of(carDoors)
.ifPresent(val -> car.setDoors(val.getDoors()));
Optional.of(carWheels)
.ifPresent(val -> car.setWheels(val.getWheels()));
Optional.of(carWindows)
.ifPresent(val -> car.setWindows(val.getWindows()));
return car;
}).subscribe().with(e-> System.out.println("Okay it worked"));
//END OF FOR EACH
//Return car (Should have been returned with new doors / wheels/ windows but instead its empty)
return Uni.createFrom().item(car);
}
As you see in the above code It should have hit some endpoints for doors / wheels / windows and set them into the variable car but what happens, in reality, is that the car is empty because one of those endpoints has been delayed so i return a car without those values inside it. I want to first update the car object and then actually return it
You could rewrite the method like this:
#GET
#Path("/testingAsync")
public Uni<List<String>> testingMutiny() {
List<Uni<String>> unis = new ArrayList<>();
List.of("hello", "RestEasy").forEach( e -> {
unis.add( Uni.createFrom().item( e )
.onItem().delayIt().by( Duration.ofMillis( 10000 ) ) );
} );
return Uni.combine().all().unis( unis )
.combinedWith( list -> (List<String>) list);
}
Note that when you write reactive code, you want to avoid using .await().indefinetly. It shouldn't be needed anyway when using Quarkus, because it recognizes async API and interpret the results accordingly.
You also don't need to subscribe the Uni or Multi when using Quarkus, for the same reason.
Based on my previous example, you can rewrite your use case with endpoints as:
#GET
#Path("/testingAsync")
public Uni<Car> testingMutiny() {
Uni<List<JsonObjectCar>> carDoorsUni = getDoors(variable1, variable2, variable3);
Uni<List<JsonObjectCar>> carWheelsUni = getWheels(variable1,variable2, variable3);
Uni<List<JsonObjectCar>> carWindowsUni = getWindows(variable1,variable2, variable3);
return Uni.combine()
.all()
.unis(carDoorsUni, carWheelsUni, carWindowsUni)
.combinedWith(list -> {
// Result of carDoorsUni
List<JsonObjectCar> carDoors = list.get(0);
// Result of carWheelsUni
List<JsonObjectCar> carWheels = list.get(1);
// Result of carWindowsUni
List<JsonObjectCar> carWindows = list.get(2);
// Create a car instance with the previous results
Car car = createCar(...);
// You can also return a list of cars, but you need to change the return type of testingMutiny to Uni<List<Car>>
return car;
})
.invoke( () -> System.out.println("Okay it worked"));
}
You return a list, but the asynchronous processing on the Uni is delayed, so your list will be empty.
You should try returning a Uni from the pipeline that you create (and also see collect(), toUni() to put into lists) instead of doing a subscription, collect the results and re-wrap into a Uni.

Transform a CompletableFuture of a list of objects to another CompletableFuture of a list of objects

I have a method (getIds) that returns a CompletableFuture<List<String>>. For each of the IDs in the list it returns, I need to call a method that fetches the corresponding username for that ID. That fetching takes some time, so it returns a CompletableFuture of String. Ultimately the method should return a CompletableFuture of the usernames List<String>:
public CompletableFuture<List<String>> getUsernames() {
CompletableFuture<List<String>> future = getIds();
return future.thenCompose(ids -> {
return CompletableFuture.allOf(ids.stream()
.map(this::getUsername)
...);
});
private CompletableFuture<String> getUsername(String id) {
...
}
Can anyone advise how to achieve this?
The BetterFuture library I wrote some time ago makes this quite easy:
public BetterFuture<List<String>> getUsernames() {
BetterFuture<List<String>> future = getIds();
return future.andThen(ids ->
BetterFuture.reduce(ids.stream().map(this::getUsername))
.map(Stream::toList));
}
BetterFuture.reduce reduces a Stream<BetterFuture<T>> to a BetterFuture<Stream<T>> and acts in parallel on the stream of futures it receives as an argument.
Since BetterFuture is just thin wrapper around CompletableFuture, it shouldn't be too hard to peel the the relevant implementation out and make above snippet work directly on CompletableFuture.

Zipping two independent Flux in spring web Flux Returns Empty

I was combing two flux both do independent task Asynchronously then I combine them using
Flux<Tweet> tweetsByUserId = restUtils.getTweetsByUserId(userId);
Flux<DeleteStatus> stringFlux = deleteTweets(tweetsByUserId);
Flux<Tweet> tweetFlux = saveTweets(tweetsByUserId);
return Flux.
zip(stringFlux, tweetFlux)
.map(TweetService::apply
)
.doOnError( e -> System.out.println(e.getMessage()))
;
Here I combine two flux and combine them to a ResponseStatus Object
private static ResponseStatus apply(Tuple2<DeleteStatus, Tweet> res) {
return new ResponseStatus(res.getT2().getId(), res.getT1().deleted());
}
the result was supposed to return Flux but it is returning and empty array
#DeleteMapping("/user/{id}")
public Flux<ResponseStatus> getDeleteallUserTweets(#PathVariable("id") String id) {
return tweetService.getAndDeleteTweets(id);
}
why my code is returning an empty flux instead of Flux< ResponseStatus>
Don't use the zip operator just to join something. It creates 1:1 pairs of two sources, where both elements share a certain relationship. This requires both sources to have the same length. The .zip will complete when one source completes.
I assume both of your fluxes are quite independent and do not really belong together. Otherwise, you could just combine deleteTweets() and saveTweets() into one method an return a single result to apply later on.
If I wanted to delete all tweets and return a response, I would:
Generate a Flux<Tweet> for a given userId
Apply the delete operation on that flux and map each element to the individual response (responseFlux = tweetFlux.map())
Aggregate the responses into a result object and return it.

How to do multiple API calls concurrently in Spring service without changing main?

I need to make a service in an existing fat code to get results from 4 APIs and I need to merge them and reformat each responses, but it takes very slow due to 4 calls that I don't know how to do it concurrently. I am also unable to change the main to add Runnable or such executor in the main as it may have snowballing effect to another code.
So currently, I have made a controller which handle the request, a service which get the request from user and call 5 different service-middleware (SM) functions. Every SM functions used to call an external API, and in every SM, I reformat each return map of the APIs there as well. I use java.net.HttpURLConnection to do the API calls. Thus, I got my API "worked" but can't be faster than 4 seconds. Those APIs needs additional OAuth, so it would be roughly 10 API calls in total.
Since the current returns of API calls are Object type, I can treat it as Map, and reformat the output by doing looping for the data inside it. So the SM function would likely have the code similarly to below:
token = sendHttpRequest(authUrl, authRequestHeader, null, null, "GET");
Map response = sendHttpRequest(url, requestHeader, bodyParam, null, "POST");
List<Map> data = (List) ((Map) response.get("output")).get("data");
List<Map> result = new HashMap();
for(Map m : data) {
Map temp = new HashMap();
temp.put("name", m.get("Name"));
temp.put("health_status", m.get("HealthStatus"));
result.add(temp);
}
// This format is mandatory
Map finalResult = new HashMap();
finalResult.put("output", result);
finalResult.put("status", "OK");
return finalResult;
And the sendHttpRequest is the method to send request, serializing params to JSON and deserializing API output to be an Object. Here's the sendHttpRequest look like:
CloseableHttpClient httpClient = HttpClients.custom()
.setSSLSocketFactory(csf)
.build();
HttpComponentsClientHttpRequestFactory requestFactory =
new HttpComponentsClientHttpRequestFactory();
requestFactory.setConnectTimeout(this.connectTimeOut);
requestFactory.setReadTimeout(this.readTimeOut);
requestFactory.setHttpClient(httpClient);
RestTemplate rt = new RestTemplate(requestFactory);
HttpEntity<Map> request = null;
if(method.equals("POST"))
request = new HttpEntity<Map>(objBody, headers);
else if(method.equals("GET"))
request = new HttpEntity<Map>(headers);
try {
ResponseEntity<Map> response = null;
if(method.equals("POST"))
restTemplate.postForEntity(url, request , Map.class);
if(method.equals("GET"))
restTemplate.postForEntity(url, request , Map.class);
if(this.outputStream){
logger.debug("Output : " + response.getBody());
}
return response.getBody();
} catch(HttpClientErrorException e) {
logger.debug(e.getMessage());
}
The sendHttpRequest method is also an existing method that I am disallowed to change except if I just make a new method for doing my requests only.
Simply say, here's the things I need to do:
For each of the API calls:
Get the Authorization token from an external API.
Do the request (POST/GET) to another external API to get data.
Reformat the data to be expected format for response (each has its own format) <Mostly loop the array of the response object to remap the field names as it's necessary>.
After all APIs finished calling, I need to do:
Merge output from API 1 and 3 to a Map/Object
Merge output from API 2 & 4 to an Array and sort them all
Put response from API 5 in the inner object of a defined attribute/field.
Things I had tried
I had tried the use of ExecutorCompletionService to call the 5 SMs. I also created an inner class that implements Callable for this.
ExecutorService executor = Executors.newFixedThreadPool(5);
CompletionService completionService = new ExecutorCompletionService<>(executor);
List<Future<Map>> results = new ArrayList<>();
for(int i=1; i<6; i++) {
// i here is used to define which api calls to be done
results.add(completionService.submit(new CallAPIClass(paramMap, i)));
}
for (int i=0; i < results.size(); i++) {
try {
Map result = (Map) completionService.take().get();
int code = (int) result.get("code");
// Collect the results for each SM (SM function has described above)
} catch (Exception e) {
logger.debug(e.getMessage());
}
}
// Merge the outputs.
In the Merge the outputs, I need to construct the map, so it will be like this:
{
"details": {"api1": {...}, "api3": {...}},
"list_items": [{...}, {...}, ...], // Results of sorted merged lists from api2 & api4
"api5": [{...}, {...}, {...}, ...]
}
Meanwhile, from the api responses, basically I just retrieve all of their output_schema when exists.
Any tips to optimize and speed up this API call, so by the same number of calls, this can be executed faster??? Any help is greatly appreciated.
Edit
I have read #Ananthapadmanabhan's answer, but it seems that I need to change the main class file which I can't do. Or is it actually possible to apply the use of CompletableFuture without using #EnableAsync in main class?
I also wonder how to get this done in a faster time even with CompletableFuture and EnableAsync with this chain of processes.
The solution you tried looks quite decent to me:
ExecutorService executor = Executors.newFixedThreadPool(5);
CompletionService completionService = new ExecutorCompletionService<>(executor);
List<Future<Map>> results = new ArrayList<>();
for(int i=1; i<6; i++) {
// i here is used to define which api calls to be done
results.add(completionService.submit(new CallAPIClass(paramMap, i)));
}
for (int i=0; i < results.size(); i++) {
try {
Map result = (Map) completionService.take().get();
int code = (int) result.get("code");
// Collect the results for each SM (SM function has described above)
} catch (Exception e) {
logger.debug(e.getMessage());
}
}
// Merge the outputs.
I am not quite sure if, in addition to a probably more fluent API, using CompletableFuture will give you any benefit related to the performance of the program - the subject has been broadly discussed here in SO, see for example 1 2 3 - but here it is a possible solution.
In fact, the next code is based in one of my previous answers, in turn closely related to this article from the Tomasz Nurkiewicz blog.
The CompletableFuture counterpart for the code you provided will look like:
ExecutorService executor = Executors.newFixedThreadPool(5);
// List of the different parameters to perform every external API invocations
final List<Map> smParameters = Arrays.asList(
...
);
// Submit invoke external task to the thread pool
final List<CompletableFuture<Map>> futures = smParameters.stream().
map(paramMap -> CompletableFuture.supplyAsync(() -> invokeExternalAPI(paramMap), executor)).
collect(Collectors.<CompletableFuture<Map>>toList())
;
// The next code is based on the sequence method proposed in the blog I cited
// The idea is to turn the `List<CompletableFuture<Map>>` we have into a
// CompletableFuture<List<Map>> with the results of every single async task
final CompletableFuture<Void> allDoneFuture =
CompletableFuture.allOf(futures.toArray(new CompletableFuture[futures.size()]));
final CompletableFuture<List<Map>> allDone = allDoneFuture.thenApply(v ->
futures.stream().
map(future -> future.join()).
collect(Collectors.<Map>toList())
);
// Merge the outputs.
final Map result = allDone.thenAccept(results ->
// Merge the outputs. The results variable contains the different Mapz
// obtained from the every different API invocation
);
Please, verify the above code, probably it will require the definition of the type of the different parameters of your Map objects.
The mentioned invokeExternalAPI could accept a Map with the different parameters required to perform the individual APIs invocations, something like:
private Map invokeExternalAPI(Map configuration) {
// Pass and extract from the configuration the authUrl, etcetera, everything you need to
// Your code...
token = sendHttpRequest(authUrl, authRequestHeader, null, null, "GET");
Map response = sendHttpRequest(url, requestHeader, bodyParam, null, "POST");
List<Map> data = (List) ((Map) response.get("output")).get("data");
List<Map> result = new HashMap();
for(Map m : data) {
Map temp = new HashMap();
temp.put("name", m.get("Name"));
temp.put("health_status", m.get("HealthStatus"));
result.add(temp);
}
// This format is mandatory
Map finalResult = new HashMap();
finalResult.put("output", result);
finalResult.put("status", "OK");
return finalResult;
}
I think you don't need to modify your main class nor any configuration, as the solution is pure Java based.
Please, bear in mind that this generic approach can be customized to accommodate different requirements.
For example, according to your comments, it seems that you need to invoke from your service the functionality implemented in your different services middleware.
In order to define the list of tasks you wanna perform concurrently you could try something like the following instead of my initial suggestion:
List<CompletableFuture<Map>> futures = new ArrayList<>(5);
// Obtain a reference to the second middleware, and submit it
final ServiceMiddleware1 sm1 = new ServiceMiddleware1();
final CompletableFuture<Map> sm1Cf = CompletableFuture.supplyAsync(() -> sm1.doYourStuff(), executor);
futures.add(sm1Cf);
// Now obtain a reference to the second middleware, and submit it again
final ServiceMiddleware2 sm2 = new ServiceMiddleware2();
final CompletableFuture<Map> sm2Cf = CompletableFuture.supplyAsync(() -> sm2.doYourStuff(), executor);
futures.add(sm2Cf);
// the rest of service middleware. I think here a common interface
// or some kind of inheritance could be of help in the invocation
// At the end, you will get the list of futures you wanna execute in parallel
// The rest of the code is the same
final CompletableFuture<Void> allDoneFuture =
CompletableFuture.allOf(futures.toArray(new CompletableFuture[futures.size()]));
final CompletableFuture<List<Map>> allDone = allDoneFuture.thenApply(v ->
futures.stream().
map(future -> future.join()).
collect(Collectors.<Map>toList())
);
// Merge the outputs.
final Map result = allDone.thenAccept(results ->
// Merge the outputs. The results variable contains the different Mapz
// obtained from the every different API invocation
);
To deal with errors you have several options.
One obvious is to handle the error in the service middleware itself, in such a way that it will never raise any exceptions, but return some kind of information in its result Map like result code, status, etcetera.
CompletableFuture itself gives you different options as well to deal with errors. As you probably need to perform some changes in the result Map you can, when necessary, use the handle method. It basically takes as argument the result and the hypothetical exception obtained in the execution of the task associated with a CompletableFuture, and return a new one CompletableFuture with the appropriate customizations, based on that result and possible error. For example, in your 4th and 5th service middlewares, which seems to raise errors, you can use something like:
final ServiceMiddleware4 sm4 = new ServiceMiddleware4();
final CompletableFuture<Map> sm4Cf = CompletableFuture.supplyAsync(() -> sm4.doYourStuff(), executor)
.handle((result, exception) -> {
if (exception == null) {
return result;
}
Map actualResult = new HashMap();
actualResult.put("errorCode", "xxx")
actualResult.put("errorMessage", exception.getMessage());
return actualResult;
});
)
;
futures.add(sm4Cf);
This great article, for instance, explains in detail further error handling approaches.
All these approaches assume that your code doesn't throw checked exceptions. If you need to deal with them, as it seems according to your comment, you could use a modified version of the code that Holger posted in this SO answer. The idea is to create a method that will handle the check exception, completing it with the appropriate error if necessary:
public static <T> CompletableFuture<T> supplyAsync(Supplier supplier, Executor executor) {
CompletableFuture<T> f=new CompletableFuture<>();
CompletableFuture.runAsync(() -> {
try { f.complete(supplier.get()); } catch(Throwable t) { f.completeExceptionally(t); }
}, executor);
return f;
}
Then, use this method to submit every service middleware task:
List<CompletableFuture<Map>> futures = new ArrayList<>(5);
// Obtain a reference to the second middleware, and submit it
final ServiceMiddleware1 sm1 = new ServiceMiddleware1();
final CompletableFuture<Map> sm1Cf = supplyAsync(() -> sm1.doYourStuff(), executor)
// this method will only be executed if any exception is thrown
.exceptionally(exception -> {
Map errorResult = new HashMap();
errorResult.put("errorCode", "xxx")
errorResult.put("errorMessage", exception.getMessage());
return errorResult;
});
futures.add(sm1Cf);
// Apply a similar logic to the rest of services middlewares...
// The rest of the code is the same as above
final CompletableFuture<Void> allDoneFuture =
CompletableFuture.allOf(futures.toArray(new CompletableFuture[futures.size()]));
final CompletableFuture<List<Map>> allDone = allDoneFuture.thenApply(v ->
futures.stream().
map(future -> future.join()).
collect(Collectors.<Map>toList())
);
// Merge the outputs.
// Please, be aware that in the lambda expression results
// is a List of the different Maps obtained as the individual
// results of every single service middleware call
// I would create an object that agglutinates these results in
// the right format, as you indicated in your question. Let's call
// this container class ServiceMiddlewareResult. Then, the merge
// results code will looks like similar to this
final ServiceMiddlewareResult result = allDone.thenAccept(results -> {
ServiceMiddlewareResult serviceMiddlewareResult = new ServiceMiddlewareResult();
// Variable used for storing temporarily the Api 2 and 4 results
// Parameterize it as necessary
List tempResultsFromApi2AndApi4 = new ArrayList();
// Honestly I don't remember if the order of the results is the
// same as the order of the futures that generated them, my guess
// is that not, as it depends on the actual future completion,
// but in any way I always try thinking that the results can be
// in any order, so it is important that every Map contains the
// minimal information to identify the corresponding service
// middleware. With that assumption in mind, your code will look
// similar to this:
results.forEach(result -> {
// The suggested idea, identify the service middleware that
// produced the results
String serviceMiddleware = result.get("serviceMiddleware");
switch(serviceMiddleware) {
// handle every case appropriately
case 'sm1': {
// it should be similar to sm3
serviceMiddlewareResult.getDetails().setApi1(...);
break;
}
case 'sm2':
case 'sm4': {
// Extract results from the Map, and add to the temporary list
tempResultsFromApi2AndApi4.add(...)
break;
}
case 'sm5': {
// extract results and populate corresponding object
serviceMiddlewareResult.setApi5(...);
break;
}
}
});
List sortedResultsFromApi2AndApi4 = Collections.sort(
sortedResultsFromApi2AndApi4, ... the appropriate comparator...
);
result.setListItems(sortedResultsFromApi2AndApi4);
return result;
});
I modified the example to provide a posible approach to merge your results.
Please, consider include logging information within your service middleware code if you need to trace and improve the debugging capabilities offered by the overall solution.
If you have used them before, as an alternative, you could try solutions based in libraries like RxJava or Project Reactor as well.
If all the 4 api calls are independent of each other and you are using java 8 , you could extract them to separate functions in a separate service layer if needed and use spring #Async annotation on the method along with CompletableFuture as return type to make parallel calls.
#Service
public class TestClient {
RestTemplate restTemplate = new RestTemplate();
#Async
public CompletableFuture<List<TestPojo>> getTestPojoByLanguage(String language) {
String url = "https://test.eu/rest/v2/lang/" + language + "?fields=name";
Country[] response = restTemplate.getForObject(url, Country[].class);
return CompletableFuture.completedFuture(Arrays.asList(response));
}
#Async
public CompletableFuture<List<TestPojo>> getCountriesByRegion(String region) {
String url = "https://testurl.eu/rest/v2/region/" + region + "?fields=name";
Country[] response = restTemplate.getForObject(url, Country[].class);
return CompletableFuture.completedFuture(Arrays.asList(response));
}
}
Completable Future guide.

Waiting for multiple observable to complete that return different number of elements

Scenario: I have a customerID string that is used to query multiple different backend systems: calendar, helpdesk, ERP, CRM etc. I want to compile a single report.
So I have roughly (psydocode):
Result myResult = new Result();
Observable<Cal> cal = Calbackend.get(customerid);
cal.subscribe(calentry -> myResult.addCal(calentry));
Observable<Erp> erp = ERPbackend.get(customerid);
erp.subscribe(erpentry -> myResult.addErp(erpentry));
Observable<Help> help = Helpbackend.get(customerid);
help.subscribe(helpentry -> myResult.addHelp(helpentry));
Observable<Crm> crm = CRMbackend.get(customerid);
crm.subscribe(crmentry -> myResult.addCrm(crmentry));
// Magic here?
return result;
The approach I was thinking of: using defer() to prevent the start and then additionally subscribe to count() for each. Then I could ZIP the count elements since they only will emit a single item each (while the others will have different numbers of events). However that could lead to loss of data if the myResult.add is performing slower than the count().
The other option I was thinking of, is to set an array of boolean flags for each subscription and check in each completion (and error) event if all of them are done and do a callback or use blocking for that one.
I had a look here and here but that examples deal with constant numbers or data types.
Or is there a better / recommended way?
Operator toList can be used together with zip like this:
Observable<List<Cal>> cal = Calbackend.get(customerid).toList();
Observable<List<Erp>> erp = ERPbackend.get(customerid).toList();
Observable<List<Help>> help = Helpbackend.get(customerid).toList();
Observable<List<Crm>> crm = CRMbackend.get(customerid).toList();
Observable.zip(cal, erp, help, crm,
new Func4<List<Cal>, List<Erp>, List<Help>, List<Crm>, Result>() {
#Override
public Result call(List<Cal> cals, List<Erp> erps, List<Help> helps, List<Crm> crms) {
Result myResult = new Result();
// add all cals, erps, helps and crms to result
return myResult;
}
})
.subscribe(new Subscriber<Result>() {
#Override
public void onNext(Result result) {
// do something with the result
}
...
});
Explanation: As the name suggests, the toList operator creates a list of the items emitted by the source observable (the list is emitted just once, when the source observable completes) and zip is then used to combine the results of the observables.
Edit: In case of the possibility that those Observables can emit an error, you could use onErrorReturn to keep the normal flow going:
Observable<List<Cal>> cal = Calbackend.get(customerid)
.onErrorReturn(new Func1<Throwable, Cal>() {
#Override
public Cal call(Throwable throwable) {
// Return something in the error case
return null;
}
})
.toList();

Categories

Resources