Given this piece of code:
public List<String> findPrices(String product){
List<CompletableFuture<String>> priceFutures =
shops.stream()
.map(shop -> CompletableFuture.supplyAsync(
() -> shop.getPrice(product), executor))
.map(future -> future.thenApply(Quote::parse))
.map(future -> future.thenCompose(quote ->
CompletableFuture.supplyAsync(
() -> Discount.applyDiscount(quote), executor
)))
.collect(toList());
return priceFutures.stream()
.map(CompletableFuture::join)
.collect(toList());
}
This part of it:
.map(future -> future.thenCompose(quote ->
CompletableFuture.supplyAsync(
() -> Discount.applyDiscount(quote), executor
)))
Could it be rewrite as:
.map(future ->
future.thenComposeAsync(quote -> Discount.applyDiscount(quote), executor))
I took this code from an example of a book and says the two solutions are different, but I do not understand why.
Let's consider a function that looks like this:
public CompletableFuture<String> requestData(String parameters) {
Request request = generateRequest(parameters);
return CompletableFuture.supplyAsync(() -> sendRequest(request));
}
The difference will be with respect to which thread generateRequest() gets called on.
thenCompose will call generateRequest() on the same thread as the upstream task (or the calling thread if the upstream task has already completed).
thenComposeAsync will call generateRequest() on the provided executor if provided, or on the default ForkJoinPool otherwise.
Related
Do I understand it well, that when I use
CompletableFuture.allOf("array of CompletableFuture")
.runAsync(()-> { "piece of code" });
first I have to wait until the array of CF are all done , and than the Runnable "piece of code" is done?
The CompletableFuture.allOf static method allows to wait for completion of all of the Futures provided as a var-arg.
For example
CompletableFuture<String> future1
= CompletableFuture.supplyAsync(() -> "Hello");
CompletableFuture<String> future2
= CompletableFuture.supplyAsync(() -> "Beautiful");
CompletableFuture<String> future3
= CompletableFuture.supplyAsync(() -> "World");
CompletableFuture<Void> combinedFuture
= CompletableFuture.allOf(future1, future2, future3);
That is explicit in the documentation:
Returns a new CompletableFuture that is completed when all of the given CompletableFutures complet
I want to execute multiple DB queries parallelly and store the results in a map. I am trying to do it like this but the map is not getting populated completely when I am accessing the map.
Am I doing anything wrong?
public Map<MapKeyEnums, Set<String>> doDBCalls(String phoneNumber, long timestamp) {
Map<MapKeyEnums, Set<String>> instrumentsEdgesMap = new EnumMap<>(MapKeyEnums.class);
CompletableFuture.supplyAsync(() -> dbReadService.getCall(phoneNumber, PhoneNumber.class, "ABC", timestamp)).
thenApply(x -> instrumentsEdgesMap.put(MapKeyEnums.ABC, x));
CompletableFuture.supplyAsync(() -> dbReadService.getCall(phoneNumber, PhoneNumber.class, "XYZ", timestamp)).
thenApply(x -> instrumentsEdgesMap.put(MapKeyEnums.XYZ, x));
CompletableFuture.supplyAsync(() -> dbReadService.getCall(phoneNumber, PhoneNumber.class, "DEF", timestamp)).
thenApply(x -> instrumentsEdgesMap.put(MapKeyEnums.DEF, x));
return instrumentsEdgesMap;
}
Any help will be much appreciated, thanks in advance.
In the above approach supplyAsync will be executed by the Async thread from ForkJoinPool, but thenApply method is always executed by calling thread. So your queries will run one after the another in sequence which it is not Asynchronous
All async methods without an explicit Executor argument are performed using the ForkJoinPool.commonPool() (unless it does not support a parallelism level of at least two, in which case, a new Thread is created to run each task).
Here is the example
CompletableFuture.supplyAsync(()->{
System.out.println(Thread.currentThread().getName());
return "SupplyAsync";
}).thenAccept(i->{
System.out.println(Thread.currentThread().getName()+"--"+i);
});
Output :
ForkJoinPool.commonPool-worker-3
main--SupplyAsync
So if you want your process to be Async then first trigger all three db queries with supplyAsync and capture the output within CompletableFuture
CompletableFuture<Set<String>> first = CompletableFuture.supplyAsync(() -> dbReadService.getCall(phoneNumber, PhoneNumber.class, "ABC", timestamp));
CompletableFuture<Set<String>> second = CompletableFuture.supplyAsync(() -> dbReadService.getCall(phoneNumber, PhoneNumber.class, "XYZ", timestamp));
CompletableFuture<Set<String>> third = CompletableFuture.supplyAsync(() -> dbReadService.getCall(phoneNumber, PhoneNumber.class, "DEF", timestamp));
And then now create a stream with three of them and then collect them to Map
Stream.of(new AbstractMap.SimpleEntry<MapKeyEnums, CompletableFuture<Set<String>>>(MapKeyEnums.ABC, first),
new AbstractMap.SimpleEntry<MapKeyEnums, CompletableFuture<Set<String>>>(MapKeyEnums.XYZ, second),
new AbstractMap.SimpleEntry<MapKeyEnums, CompletableFuture<Set<String>>>(MapKeyEnums.DEF, third))
.forEach(entry->{
entry.getValue().thenAccept(val-> instrumentsEdgesMap.put(entry.getKey(), val));
});
You have to wait for the futures to complete before you return the result.
Try something like
public Map<MapKeyEnums, Set<String>> doDBCalls(String phoneNumber, long timestamp) {
Map<MapKeyEnums, Set<String>> instrumentsEdgesMap = new EnumMap<>(MapKeyEnums.class);
CompletableFuture.allOf(
CompletableFuture.supplyAsync(() -> dbReadService.getCall(phoneNumber, PhoneNumber.class, "ABC", timestamp))
.thenAccept(x -> instrumentsEdgesMap.put(MapKeyEnums.ABC, x)),
CompletableFuture.supplyAsync(() -> dbReadService.getCall(phoneNumber, PhoneNumber.class, "XYZ", timestamp))
.thenAccept(x -> instrumentsEdgesMap.put(MapKeyEnums.XYZ, x)),
CompletableFuture.supplyAsync(() -> dbReadService.getCall(phoneNumber, PhoneNumber.class, "DEF", timestamp))
.thenAccept(x -> instrumentsEdgesMap.put(MapKeyEnums.DEF, x)))
.get(); // wait for completion of all three subtasks
return instrumentsEdgesMap;
}
We have an async method:
public CompletableFuture<OlderCat> asyncGetOlderCat(String catName)
Given a list of Cats:
List<Cat> cats;
We like to create a bulk operation that will result in a map between the cat name and its async result:
public CompletableFuture<Map<String, OlderCat>>
We also like that if an exception was thrown from the asyncGetOlderCat, the cat will not be added to the map.
We were following this post and also this one and we came up with this code:
List<Cat> cats = ...
Map<String, CompletableFuture<OlderCat>> completableFutures = cats
.stream()
.collect(Collectors.toMap(Cat::getName,
c -> asynceGetOlderCat(c.getName())
.exceptionally( ex -> /* null?? */ ))
));
CompletableFuture<Void> allFutures = CompletableFuture
.allOf(completableFutures.values().toArray(new CompletableFuture[completableFutures.size()]));
return allFutures.thenApply(future -> completableFutures.keySet().stream()
.map(CompletableFuture::join) ???
.collect(Collectors.toMap(????)));
But it is not clear how in the allFutureswe can get access to the cat name and how to match between the OlderCat & the catName.
Can it be achieved?
You are almost there. You don't need to put an exceptionally() on the initial futures, but you should use handle() instead of thenApply() after the allOf(), because if any future fails, the allOf() will fail as well.
When processing the futures, you can then just filter out the failing ones from the result, and rebuild the expected map:
Map<String, CompletableFuture<OlderCat>> completableFutures = cats
.stream()
.collect(toMap(Cat::getName, c -> asyncGetOlderCat(c.getName())));
CompletableFuture<Void> allFutures = CompletableFuture
.allOf(completableFutures.values().toArray(new CompletableFuture[0]));
return allFutures.handle((dummy, ex) ->
completableFutures.entrySet().stream()
.filter(entry -> !entry.getValue().isCompletedExceptionally())
.collect(toMap(Map.Entry::getKey, e -> e.getValue().join())));
Note that the calls to join() are guaranteed to be non-blocking since the thenApply() will only be executed after all futures are completed.
As I get it, what you need is CompletableFuture with all results, the code below does exactly what you need
public CompletableFuture<Map<String, OlderCat>> getOlderCats(List<Cat> cats) {
return CompletableFuture.supplyAsync(
() -> {
Map<String, CompletableFuture<OlderCat>> completableFutures = cats
.stream()
.collect(Collectors.toMap(Cat::getName,
c -> asyncGetOlderCat(c.getName())
.exceptionally(ex -> {
ex.printStackTrace();
// if exception happens - return null
// if you don't want null - save failed ones to separate list and process them separately
return null;
}))
);
return completableFutures
.entrySet()
.stream()
.collect(Collectors.toMap(
Map.Entry::getKey,
e -> e.getValue().join()
));
}
);
}
What it does here - returns future, which creates more completable future inside and waits at the end.
Its possible to filter and continue emiting itens like below ?
My code that calls subscriber 2 times:
Observable<Map.Entry<String, ArrayList<MockOverview>>> requestEntries =
this.requestView.request(request)
.map(HashMap::entrySet)
.flatMapIterable(entries -> entries);
requestEntries.filter(entry -> entry.getKey().equals("featured"))
.map((Func1<Map.Entry<String, ArrayList<MockOverview>>, List<MockOverview>>) Map.Entry::getValue)
.subscribe(mockOverviews -> {
Log.i("subscrive", "featured");
});
requestEntries.filter(entry -> entry.getKey().equals("done"))
.map((Func1<Map.Entry<String, ArrayList<MockOverview>>, List<MockOverview>>) Map.Entry::getValue)
.subscribe(mockOverviews -> {
Log.i("subscrive", "featured");
});
What i want:
requestEntries.filter(entry -> entry.getKey().equals("featured"))
.map((Func1<Map.Entry<String, ArrayList<MockOverview>>, List<MockOverview>>) Map.Entry::getValue)
.subscribe(mockOverviews -> {
})
.filter(entry -> entry.getKey().equals("done"))
.map((Func1<Map.Entry<String, ArrayList<MockOverview>>, List<MockOverview>>) Map.Entry::getValue)
.subscribe(mockOverviews -> {
});
By the looks of things your second version is not equal to your first: the former looks at the requestEntries stream twice, filters on featured and done keys respectively and does its own things with it. Your second version however first filters on featured first then does some transformations and side-effects and then filter out the done. However, that Observable<entryset> is not at all in scope in that second filter lambda.
What you need to do here is use publish(<lambda>) on requestEntries and in the lambda do the stuff from your first version, use onNext instead of subscribe, merge the streams and return that combined stream. Then outside of the publish you subscribe once (and do nothing in there) or go on and use the result of your stream somewhere else.
requestEntries.publish(re -> {
Observable<...> x = re.filter(...<featured>...).map(...).doOnNext(...Log.i(...));
Observable<...> y = re.filter(...<done>...).map(...).doOnNext(...Log.i(...));
return x.mergeWith(y);
})
You can use doOnNext in the place of the first subscribe()
requestEntry.filter(v -> ...)
.map(v -> ...)
.doOnNext(v -> ...)
.filter(v -> ...)
.map(v -> ...)
.subscribe(...)
or use publish(Func1):
requestEntry.filter(v -> ...)
.map(v -> ...)
.publish(o -> {
o.subscribe(...);
return o;
})
.filter(v -> ...)
.map(v -> ...)
.subscribe(...)
The code I'm having problems with is:
Executor executor = (Executor) callList;
List<ProgState> newProgList = executor.invokeAll(callList).stream()
.map(future -> {try {return future.get();} catch(Exception e){e.printStackTrace();}})
.filter(p -> p!=null).collect(Collectors.toList());
The method invokeAll(List>) is undefined for the type Executor
I am told I should use an executor like the one in the code snippet.
The Callables are defined within the following code:
List<Callable<ProgState>> callList = (List<Callable<ProgState>>) lst.stream()
.map(p -> ((Callable<ProgState>)(() -> {return p.oneStep();})))
.collect(Collectors.toList());
Here is the teacher's code:
//prepare the list of callables
List<Callable<PrgState>> callList = prgList.stream().map(p -> (() -> {return p.oneStep();})).collect(Collectors.toList());
//start the execution of the callables
//it returns the list of new created threads
List<PrgState> newPrgList = executor.invokeAll(callList).stream()
.map(future -> { try {
return future.get();
}
catch(Exception e) {
//here you can treat the possible
// exceptions thrown by statements
// execution
}
})
.filter(p -> p!=null).collect(Collectors.toList());
//add the new created threads to the list of existing threads
prgList.addAll(newPrgList);
If you can use stream(), why not parallelStream() as it would be much simpler.
List<PrgState> prgStates = prgList.parallelStream()
.map(p -> p.oneStep())
.collect(Collectors.toList());
This way you have no thread pool to configure, start or stop when finished.
Some might suggest that parallelStream() was the main reason for adding Stream and lambdas to Java 8 in the first place. ;)
You can't cast list of Callables with ExecutorService. You need to define ExecutorService which will inturn pick up callables and execute them in one or multiple threads in parallel.
This is what i think you are after:
ExecutorService executor = Executors.newCachedThreadPool();//change executor type as per your need.
List<ProgState> newProgList = executor.invokeAll(callList).stream().map(future -> {...