I want to execute multiple DB queries parallelly and store the results in a map. I am trying to do it like this but the map is not getting populated completely when I am accessing the map.
Am I doing anything wrong?
public Map<MapKeyEnums, Set<String>> doDBCalls(String phoneNumber, long timestamp) {
Map<MapKeyEnums, Set<String>> instrumentsEdgesMap = new EnumMap<>(MapKeyEnums.class);
CompletableFuture.supplyAsync(() -> dbReadService.getCall(phoneNumber, PhoneNumber.class, "ABC", timestamp)).
thenApply(x -> instrumentsEdgesMap.put(MapKeyEnums.ABC, x));
CompletableFuture.supplyAsync(() -> dbReadService.getCall(phoneNumber, PhoneNumber.class, "XYZ", timestamp)).
thenApply(x -> instrumentsEdgesMap.put(MapKeyEnums.XYZ, x));
CompletableFuture.supplyAsync(() -> dbReadService.getCall(phoneNumber, PhoneNumber.class, "DEF", timestamp)).
thenApply(x -> instrumentsEdgesMap.put(MapKeyEnums.DEF, x));
return instrumentsEdgesMap;
}
Any help will be much appreciated, thanks in advance.
In the above approach supplyAsync will be executed by the Async thread from ForkJoinPool, but thenApply method is always executed by calling thread. So your queries will run one after the another in sequence which it is not Asynchronous
All async methods without an explicit Executor argument are performed using the ForkJoinPool.commonPool() (unless it does not support a parallelism level of at least two, in which case, a new Thread is created to run each task).
Here is the example
CompletableFuture.supplyAsync(()->{
System.out.println(Thread.currentThread().getName());
return "SupplyAsync";
}).thenAccept(i->{
System.out.println(Thread.currentThread().getName()+"--"+i);
});
Output :
ForkJoinPool.commonPool-worker-3
main--SupplyAsync
So if you want your process to be Async then first trigger all three db queries with supplyAsync and capture the output within CompletableFuture
CompletableFuture<Set<String>> first = CompletableFuture.supplyAsync(() -> dbReadService.getCall(phoneNumber, PhoneNumber.class, "ABC", timestamp));
CompletableFuture<Set<String>> second = CompletableFuture.supplyAsync(() -> dbReadService.getCall(phoneNumber, PhoneNumber.class, "XYZ", timestamp));
CompletableFuture<Set<String>> third = CompletableFuture.supplyAsync(() -> dbReadService.getCall(phoneNumber, PhoneNumber.class, "DEF", timestamp));
And then now create a stream with three of them and then collect them to Map
Stream.of(new AbstractMap.SimpleEntry<MapKeyEnums, CompletableFuture<Set<String>>>(MapKeyEnums.ABC, first),
new AbstractMap.SimpleEntry<MapKeyEnums, CompletableFuture<Set<String>>>(MapKeyEnums.XYZ, second),
new AbstractMap.SimpleEntry<MapKeyEnums, CompletableFuture<Set<String>>>(MapKeyEnums.DEF, third))
.forEach(entry->{
entry.getValue().thenAccept(val-> instrumentsEdgesMap.put(entry.getKey(), val));
});
You have to wait for the futures to complete before you return the result.
Try something like
public Map<MapKeyEnums, Set<String>> doDBCalls(String phoneNumber, long timestamp) {
Map<MapKeyEnums, Set<String>> instrumentsEdgesMap = new EnumMap<>(MapKeyEnums.class);
CompletableFuture.allOf(
CompletableFuture.supplyAsync(() -> dbReadService.getCall(phoneNumber, PhoneNumber.class, "ABC", timestamp))
.thenAccept(x -> instrumentsEdgesMap.put(MapKeyEnums.ABC, x)),
CompletableFuture.supplyAsync(() -> dbReadService.getCall(phoneNumber, PhoneNumber.class, "XYZ", timestamp))
.thenAccept(x -> instrumentsEdgesMap.put(MapKeyEnums.XYZ, x)),
CompletableFuture.supplyAsync(() -> dbReadService.getCall(phoneNumber, PhoneNumber.class, "DEF", timestamp))
.thenAccept(x -> instrumentsEdgesMap.put(MapKeyEnums.DEF, x)))
.get(); // wait for completion of all three subtasks
return instrumentsEdgesMap;
}
Related
Do I understand it well, that when I use
CompletableFuture.allOf("array of CompletableFuture")
.runAsync(()-> { "piece of code" });
first I have to wait until the array of CF are all done , and than the Runnable "piece of code" is done?
The CompletableFuture.allOf static method allows to wait for completion of all of the Futures provided as a var-arg.
For example
CompletableFuture<String> future1
= CompletableFuture.supplyAsync(() -> "Hello");
CompletableFuture<String> future2
= CompletableFuture.supplyAsync(() -> "Beautiful");
CompletableFuture<String> future3
= CompletableFuture.supplyAsync(() -> "World");
CompletableFuture<Void> combinedFuture
= CompletableFuture.allOf(future1, future2, future3);
That is explicit in the documentation:
Returns a new CompletableFuture that is completed when all of the given CompletableFutures complet
We have an async method:
public CompletableFuture<OlderCat> asyncGetOlderCat(String catName)
Given a list of Cats:
List<Cat> cats;
We like to create a bulk operation that will result in a map between the cat name and its async result:
public CompletableFuture<Map<String, OlderCat>>
We also like that if an exception was thrown from the asyncGetOlderCat, the cat will not be added to the map.
We were following this post and also this one and we came up with this code:
List<Cat> cats = ...
Map<String, CompletableFuture<OlderCat>> completableFutures = cats
.stream()
.collect(Collectors.toMap(Cat::getName,
c -> asynceGetOlderCat(c.getName())
.exceptionally( ex -> /* null?? */ ))
));
CompletableFuture<Void> allFutures = CompletableFuture
.allOf(completableFutures.values().toArray(new CompletableFuture[completableFutures.size()]));
return allFutures.thenApply(future -> completableFutures.keySet().stream()
.map(CompletableFuture::join) ???
.collect(Collectors.toMap(????)));
But it is not clear how in the allFutureswe can get access to the cat name and how to match between the OlderCat & the catName.
Can it be achieved?
You are almost there. You don't need to put an exceptionally() on the initial futures, but you should use handle() instead of thenApply() after the allOf(), because if any future fails, the allOf() will fail as well.
When processing the futures, you can then just filter out the failing ones from the result, and rebuild the expected map:
Map<String, CompletableFuture<OlderCat>> completableFutures = cats
.stream()
.collect(toMap(Cat::getName, c -> asyncGetOlderCat(c.getName())));
CompletableFuture<Void> allFutures = CompletableFuture
.allOf(completableFutures.values().toArray(new CompletableFuture[0]));
return allFutures.handle((dummy, ex) ->
completableFutures.entrySet().stream()
.filter(entry -> !entry.getValue().isCompletedExceptionally())
.collect(toMap(Map.Entry::getKey, e -> e.getValue().join())));
Note that the calls to join() are guaranteed to be non-blocking since the thenApply() will only be executed after all futures are completed.
As I get it, what you need is CompletableFuture with all results, the code below does exactly what you need
public CompletableFuture<Map<String, OlderCat>> getOlderCats(List<Cat> cats) {
return CompletableFuture.supplyAsync(
() -> {
Map<String, CompletableFuture<OlderCat>> completableFutures = cats
.stream()
.collect(Collectors.toMap(Cat::getName,
c -> asyncGetOlderCat(c.getName())
.exceptionally(ex -> {
ex.printStackTrace();
// if exception happens - return null
// if you don't want null - save failed ones to separate list and process them separately
return null;
}))
);
return completableFutures
.entrySet()
.stream()
.collect(Collectors.toMap(
Map.Entry::getKey,
e -> e.getValue().join()
));
}
);
}
What it does here - returns future, which creates more completable future inside and waits at the end.
Given this piece of code:
public List<String> findPrices(String product){
List<CompletableFuture<String>> priceFutures =
shops.stream()
.map(shop -> CompletableFuture.supplyAsync(
() -> shop.getPrice(product), executor))
.map(future -> future.thenApply(Quote::parse))
.map(future -> future.thenCompose(quote ->
CompletableFuture.supplyAsync(
() -> Discount.applyDiscount(quote), executor
)))
.collect(toList());
return priceFutures.stream()
.map(CompletableFuture::join)
.collect(toList());
}
This part of it:
.map(future -> future.thenCompose(quote ->
CompletableFuture.supplyAsync(
() -> Discount.applyDiscount(quote), executor
)))
Could it be rewrite as:
.map(future ->
future.thenComposeAsync(quote -> Discount.applyDiscount(quote), executor))
I took this code from an example of a book and says the two solutions are different, but I do not understand why.
Let's consider a function that looks like this:
public CompletableFuture<String> requestData(String parameters) {
Request request = generateRequest(parameters);
return CompletableFuture.supplyAsync(() -> sendRequest(request));
}
The difference will be with respect to which thread generateRequest() gets called on.
thenCompose will call generateRequest() on the same thread as the upstream task (or the calling thread if the upstream task has already completed).
thenComposeAsync will call generateRequest() on the provided executor if provided, or on the default ForkJoinPool otherwise.
Its possible to filter and continue emiting itens like below ?
My code that calls subscriber 2 times:
Observable<Map.Entry<String, ArrayList<MockOverview>>> requestEntries =
this.requestView.request(request)
.map(HashMap::entrySet)
.flatMapIterable(entries -> entries);
requestEntries.filter(entry -> entry.getKey().equals("featured"))
.map((Func1<Map.Entry<String, ArrayList<MockOverview>>, List<MockOverview>>) Map.Entry::getValue)
.subscribe(mockOverviews -> {
Log.i("subscrive", "featured");
});
requestEntries.filter(entry -> entry.getKey().equals("done"))
.map((Func1<Map.Entry<String, ArrayList<MockOverview>>, List<MockOverview>>) Map.Entry::getValue)
.subscribe(mockOverviews -> {
Log.i("subscrive", "featured");
});
What i want:
requestEntries.filter(entry -> entry.getKey().equals("featured"))
.map((Func1<Map.Entry<String, ArrayList<MockOverview>>, List<MockOverview>>) Map.Entry::getValue)
.subscribe(mockOverviews -> {
})
.filter(entry -> entry.getKey().equals("done"))
.map((Func1<Map.Entry<String, ArrayList<MockOverview>>, List<MockOverview>>) Map.Entry::getValue)
.subscribe(mockOverviews -> {
});
By the looks of things your second version is not equal to your first: the former looks at the requestEntries stream twice, filters on featured and done keys respectively and does its own things with it. Your second version however first filters on featured first then does some transformations and side-effects and then filter out the done. However, that Observable<entryset> is not at all in scope in that second filter lambda.
What you need to do here is use publish(<lambda>) on requestEntries and in the lambda do the stuff from your first version, use onNext instead of subscribe, merge the streams and return that combined stream. Then outside of the publish you subscribe once (and do nothing in there) or go on and use the result of your stream somewhere else.
requestEntries.publish(re -> {
Observable<...> x = re.filter(...<featured>...).map(...).doOnNext(...Log.i(...));
Observable<...> y = re.filter(...<done>...).map(...).doOnNext(...Log.i(...));
return x.mergeWith(y);
})
You can use doOnNext in the place of the first subscribe()
requestEntry.filter(v -> ...)
.map(v -> ...)
.doOnNext(v -> ...)
.filter(v -> ...)
.map(v -> ...)
.subscribe(...)
or use publish(Func1):
requestEntry.filter(v -> ...)
.map(v -> ...)
.publish(o -> {
o.subscribe(...);
return o;
})
.filter(v -> ...)
.map(v -> ...)
.subscribe(...)
i want to run the three methods posted below using CompletableFuture asynchronous supplier so that, when the Executor finishes the Futurelist should contain three values returned from the three methods respectively.
i know how to use the Futurelist, for an example:
futureList = CompletableFuture.supplyAsync()
but in my case, i want something like:
futureList.add(CompletableFuture.supplyAsync())
please let me know how can i do that.
methods:
this.compStabilityMeasure(this.frameIjList, this.frameIkList, SysConsts.STABILITY_MEASURE_TOKEN);
this.setTrackingRepValue(this.compTrackingRep(this.frameIjList, this.frameIkList, SysConsts.TRACKING_REPEATABILITY_TOKEN));
this.setViewPntRepValue(this.compViewPntRep(this.frameIjList, this.frameIkList, SysConsts.VIEWPOINT_REPEATABILITY_TOKEN));
compStabilityMeasure method implementation:
private void compStabilityMeasure(ArrayList<Mat> frameIjList, ArrayList<Mat>
frameIkList, String token) throws InterruptedException, ExecutionException {
// TODO Auto-generated method stub
synchronized (frameIjList) {
synchronized (frameIjList) {
this.setRepValue(this.compRep(frameIjList, frameIkList, token));
this.setSymRepValue(this.compSymRep(this.getRepValue(), frameIkList, frameIjList, token));
}
}
}
You want to look at using "thenCombineAsync", eg:
CompletableFuture<String> firstFuture = firstMethod();
CompletableFuture<String> secondFuture = secondMethod();
CompletableFuture<String> thirdFuture = thirdMethod();
CompletableFuture<List<String>> allCompleted = firstFuture
.thenCombineAsync(secondFuture, (first, second) -> listOf(first, second))
.thenCombineAsync(thirdFuture, (list, third) -> {
list.add(third);
return list;
});
You can use allOf, and then create a CompletableFuture that gets completed with a Stream containing the results of your individual CompletableFutures:
CompletableFuture<String> cf1 = CompletableFuture.supplyAsync(() -> "hi1");
CompletableFuture<String> cf2 = CompletableFuture.supplyAsync(() -> "hi2");
List<CompletableFuture<String>> cfsList = Arrays.asList(cf1, cf2);
CompletableFuture<Void> allCfs = CompletableFuture.allOf((CompletableFuture<String>[]) cfsList.toArray());
CompletableFuture<Stream<String>> cfWithFinishedStream = allCfs.thenApply((allCf) ->
cfsList.stream().map(cf -> cf.getNow("")));
Example to get the values from the stream when the CF completes:
cfWithFinishedStream.thenAccept(stream ->
stream.forEach(string -> System.out.println(string)));
If you don't like streams, you can convert them to a List using collect:
CompletableFuture<List<String>> cfWithFinishedList = allCfs
.thenApply((allCf) ->
cfsList.stream().map(cf ->
cf.getNow("")).collect(Collectors.toList()));