I am investigating Java 8 CompletableFutures and read (and seen) that I should employ thenCompose instead of thenApply.
I have converted my code to use thenCompose but I have a feeling in an incorrect manner.
Here is my controlling code...
final CompletableFuture<List<String>> extractor = get(htmlPageSource);
#SuppressWarnings("unchecked")
final CompletableFuture<List<Documentable>>[] completableFutures =
new CompletableFuture[ENDPOINT.EXTRACTABLES.size()];
int index = 0;
for( ENDPOINT endpoint : ENDPOINT.EXTRACTABLES ) {
final CompletableFuture<List<Documentable>> metaData =
extractor.thenComposeAsync(
s -> endpoint.contactEndpoit(s), executorService );
completableFutures[index++] = metaData.exceptionally(x -> failedList(x));
}
CompletableFuture
.allOf( completableFutures )
.thenComposeAsync( dummy -> combineDocuments( completableFutures ))
.thenAccept ( x -> finish( x ))
.exceptionally( x -> failed( x ));
private List<Documentable> failedList(final Throwable x) {
LOGGER.error("failedList", x);
final List<Documentable> metaData = new ArrayList<>();
return metaData;
}
private Void failed(final Throwable x) {
LOGGER.error("failed", x);
return null;
}
Which I believe is acceptable
However the code that makes me uneasy is this:-
WWW_SITE_ONE("https://example.site.one/") {
#Override
public <T extends Documentable> CompletionStage<List<T>> contactEndpoit( final List<String> elements) {
LOGGER.info("WWW_SITE_ONE " + Thread.currentThread().getName());
final List<T> SITE_ONEs = new ArrayList<>();
for (final String element : elements) {
try {
final String json = Jsoup.connect(ENDPOINT.WWW_SITE_ONE.getBaseUrl() + element).ignoreContentType(true).ignoreHttpErrors(true).maxBodySize(0).timeout(60000).execute().body();
if (json.contains("errors")) {
continue;
}
final T SITE_ONE = OBJECT_READER_SITE_ONE.readValue(json);
SITE_ONEs.add(SITE_ONE);
}
catch( final Throwable e ) {
LOGGER.error("WWW_SITE_ONE failed", e);
throw new RuntimeException(e);
}
}
return CompletableFuture.supplyAsync(() -> SITE_ONEs);
}
},
WWW_SITE_TWO("https://example.site.two/") {
#Override
public <T extends Documentable> CompletionStage<List<T>> contactEndpoit(final List<String> elements) {
LOGGER.info("WWW_SITE_TWO " + Thread.currentThread().getName());
final List<T> SITE_TWOs = new ArrayList<>();
for (final String element : elements) {
try {
final String json = Jsoup.connect(ENDPOINT.WWW_SITE_TWO.getBaseUrl() + element).ignoreContentType(true).ignoreHttpErrors(true).maxBodySize(0).timeout(60000).execute().body();
if (json.equals("Resource not found.")) {
continue;
}
final T SITE_TWO = OBJECT_READER_SITE_TWO.readValue(json);
SITE_TWOs.add(SITE_TWO);
}
catch (final Throwable e) {
LOGGER.error("WWW_SITE_TWO failed", e);
throw new RuntimeException(e);
}
}
return CompletableFuture.supplyAsync(() -> SITE_TWOs);
}
},
WWW_SITE_THREE("https://example.site.three/") {
#Override
public <T extends Documentable> CompletionStage<List<T>> contactEndpoit(final List<String> elements) {
LOGGER.info("WWW_SITE_THREE " + Thread.currentThread().getName());
final List<T> SITE_THREEs = new ArrayList<>();
for (final String element : elements) {
try {
final String SITE_THREEJsonString = Jsoup
.connect( ENDPOINT.WWW_SITE_THREE.getBaseUrl() + element)
.ignoreContentType(true)
.ignoreHttpErrors(true)
.maxBodySize(0)
.timeout(60000)
.execute()
.body();
final SITE_THREE SITE_THREE_Json = OBJECT_READER_SITE_THREE.readValue(SITE_THREEJsonString);
final T SITE_THREE = (T) SITE_THREE_Json;
if (SITE_THREE_Json.getHitCount() > 0) {
SITE_THREEs.add(SITE_THREE);
}
}
catch (final Throwable e) {
LOGGER.error("WWW_SITE_THREE failed", e);
throw new RuntimeException(e);
}
}
return CompletableFuture.supplyAsync(() -> SITE_THREEs);
}
};
Its where I am returning CompletableFuture.supplyAsync(() -> SITE_THREEs);
Is this the correct approach?
Or does this start another asynchronous thread to simply return my List<>?
As the name suggests, supplyAsync will perform an asynchronous operation, executing the Supplier’s get() method, hence the body of the lambda expression, in a background thread, regardless of how trivial it is. Since the implementation of supplyAsync has no way to check how trivial the code encapsulated by the Supplier is, it has to work this way.
Instead of CompletableFuture.supplyAsync(() -> SITE_THREEs), you should use CompletableFuture.completedFuture(SITE_THREEs) which returns a future that has already been completed with the result, hence, not requiring additional actions.
If the method only returns completed stages or throws an exception, you may also change it to return the result value instead of a CompletionStage and use thenApply instead of thenCompose, simplifying your code—unless you want to keep the option of introducing asynchronous operations in a future version of that method.
Related
I'm making 5 parallel network calls, mocking 4 of them to succeed and one of them to fail.
The failed call makes the entire Single.zip() fail and I can't get the results of the 4 other network calls even though they have succeeded.
How can I handle the error for the single failed network call in the Single.zip() and get the results of the ones that have succeeded?
private Single<BigInteger> createNetworkCall(){
return Single.fromCallable(() -> {
return service.getBalance("validaddress").execute();
}).subscribeOn(Schedulers.io());
}
private Single<BigInteger> createFailedNetworkCall(){
return Single.fromCallable(() -> {
return service.getBalance("invalidaddress").execute();
}).subscribeOn(Schedulers.io());
}
private void makeParallelCalls(){
List<Single<BigInteger>> iterable = new ArrayList<>();
iterable.add(createNetworkCall());
iterable.add(createNetworkCall());
iterable.add(createNetworkCall());
iterable.add(createNetworkCall());
iterable.add(createFailedNetworkCall());
Single.zip(iterable, (results) -> {
Log.d(TAG, "makeParallelCalls: " + Arrays.toString(results));
return results;
}).observeOn(AndroidSchedulers.mainThread())
.subscribe(results-> {
Log.d(TAG, "onSuccess: makeParallelCalls: " + results);
}, (exception) -> {
Log.e(TAG, "onError: makeParallelCalls", exception);
});
}
Catch the exception and don't allow the error to interrupt the Single.zip.
For example inside your request factory, return an Optional instead of the response.
private Single<Optional<Long>> performNetworkCall(int n){
return Single.fromCallable(() -> {
if (n % 2 == 0) {
throw new Exception("failed call");
}
return 0L;
}).subscribeOn(Schedulers.io())
.map(Optional::of)
.onErrorReturnItem(Optional.empty());
// or .onErrorReturn(exception -> Optional.empty());
}
private void makeParallelCalls(){
List<Single<BigInteger>> iterable = new ArrayList<>();
iterable.add(performNetworkCall(1));
iterable.add(performNetworkCall(2));
iterable.add(performNetworkCall(3));
iterable.add(performNetworkCall(4));
iterable.add(performNetworkCall(5));
Single.zip(iterable, (results) -> {
for (Object result : results) {
var optional = (Optional<Integer>)result;
if (optional.isEmpty()) {
// this one failed, no data
} else {
var response = optional.get();
}
}
return results;
});
}
You could replace the Optional with a custom class that holds additional information, such as the exception or error information that caused the failure.
If you have recovery options, you could use .onErrorResumeNextinstead.
I'm quite new to RxJava so I don't know if there is a more better solution or if this is a really bad practice, but I used the onErrorReturn callback on the Single.
If someone has a better solution, please share so I can mark yours correct!
public static final int INVALID_USER_ADDRESS = -101;
private Single<BigInteger> createNetworkCall(){
return Single.fromCallable(() -> {
return service.getBalance("validaddress").execute();
}).subscribeOn(Schedulers.io());
}
private Single<BigInteger> createFailedNetworkCall(){
return Single.fromCallable(() -> {
return service.getBalance("invalidaddress").execute();
}).subscribeOn(Schedulers.io()).subscribeOn(Schedulers.io()).onErrorReturn(throwable -> {
if(throwable instanceof IllegalArgumentException){
return BigInteger.valueOf(INVALID_USER_ADDRESS);
}else{
return BigInteger.valueOf(-100);
}
});;
}
private void makeParallelCalls(){
List<Single<BigInteger>> iterable = new ArrayList<>();
iterable.add(createNetworkCall());
iterable.add(createNetworkCall());
iterable.add(createNetworkCall());
iterable.add(createNetworkCall());
iterable.add(createFailedNetworkCall());
Single.zip(iterable, (results) -> {
Log.d(TAG, "makeParallelCalls: " + Arrays.toString(results));
return results;
}).observeOn(AndroidSchedulers.mainThread())
.subscribe(results-> {
for(int i = 0; i < results.length; i++){
if(results[i].equals(INVALID_USER_ADDRESS)){
//Handle error here
}else{
//Handle success value here
}
}
}, (exception) -> {
Log.e(TAG, "onError: makeParallelCalls", exception);
});
}
.subscribe(results-> {
returns:
[2583397195825000000000, 2583397195825000000000, 2583397195825000000000, 2583397195825000000000, -101]
TL;DR: I want to perform an asynchronous Call to a REST-API. The standard call would give me a CompleteableFuture<Response>, however because the API has a limit on how many calls it allows in a certain amount of time I want to be able to queue up calls to 1. execute them in order and 2. execute them only when I am not exceeding the APIs limits at that current moment, otherwise wait.
Long verson:
I am using Retrofit to perform Rest calls to an API and Retrofit returns a CompleteableFuture<WhateverResponseClassIDeclare> when I call it. However due to limitations of the API I am calling I want to have tight control over when and in what order my calls go out to it. In detail, too many calls in a certain timeframe would cause me to get IP banned. Similarly I want to maintain the order of my calls, even if they won't get executed immediately. The goal is to call a Wrapper of the API that returns a CompleteableFuture just like the original API but performs those in-between steps asynchronously.
I was playing around with BlockingQueues, Functions, Callables, Suppliers and everything inbetween, but I couldn't get it to work yet.
Following there is my currently NON FUNCTIONAL code I created as a Mockup to test the concept.
import java.util.concurrent.BlockingDeque;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.LinkedBlockingDeque;
import java.util.function.Function;
public class Sandbox2 {
public static void main(String[] args) throws ExecutionException, InterruptedException {
MockApi mockApi = new MockApi();
CompletableFuture<Integer> result1 = mockApi.requestAThing("Req1");
CompletableFuture<Integer> result2 = mockApi.requestAThing("Req2");
CompletableFuture<Integer> result3 = mockApi.requestAThing("Req3");
System.out.println("Result1: " + result1.get());
System.out.println("Result2: " + result2.get());
System.out.println("Result3: " + result3.get());
}
public static class MockApi {
ActualApi actualApi = new ActualApi();
BlockingDeque<Function<String, CompletableFuture<Integer>>> queueBlockingDeque = new LinkedBlockingDeque();
public CompletableFuture<Integer> requestAThing(String req1) {
Function<String, CompletableFuture<Integer>> function = new Function<String, CompletableFuture<Integer>>() {
#Override
public CompletableFuture<Integer> apply(String s) {
return actualApi.requestHandler(s);
}
};
return CompletableFuture
.runAsync(() -> queueBlockingDeque.addLast(function))
.thenRun(() -> waitForTheRightMoment(1000))
.thenCombine(function)
}
private void waitForTheRightMoment(int time) {
try {
Thread.sleep(time);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
public static class ActualApi {
public CompletableFuture<Integer> requestHandler(String request) {
return CompletableFuture.supplyAsync(() -> {
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
return Integer.parseInt(request.substring(3));
});
}
}
}
Pre JDK 9 (JDK 1.8)
You can make use of ScheduledExecutor that accepts items to execute asynchronously on a pre-configured thread pool at a pre-fixed rate / delay.
You can obtain such a service as follows:
private final ScheduledExecutorService executorService = Executors.newSingleThreadScheduledExecutor();
Once an instance of ScheduledExecutorService is created, you can start submitting items (requests) to be executed as follows:
executorService.schedule(
() -> actualApi.requestHandler(req),
delay,
unit
);
Meanwhile, using a direct call want lead a CompletableFuture<Integer> but instead would lead a ScheduledFuture<CompletableFuture<Integer>> on which you will have to block to get the wrapped result.
Instead, you would need to block on your final requests results inside the ScheduledExecutorService then wrap your final request result in a completed ComppletableFuture:
public <T> CompletableFuture<T> scheduleCompletableFuture(
final CompletableFuture<T> command,
final long delay,
final TimeUnit unit) {
final CompletableFuture<T> completableFuture = new CompletableFuture<>();
this.executorService.schedule(
(() -> {
try {
return completableFuture.complete(command.get());
} catch (Throwable t) {
return completableFuture.completeExceptionally(t);
}
}),
delay,
unit
);
return completableFuture;
}
Here down a review version of your implementation:
public class Sandbox2 {
public static void main(String[] args) throws ExecutionException, InterruptedException {
MockApi mockApi = new MockApi();
CompletableFuture<Integer> result1 = mockApi.requestAThing("Req1");
CompletableFuture<Integer> result2 = mockApi.requestAThing("Req2");
CompletableFuture<Integer> result3 = mockApi.requestAThing("Req3");
System.out.println("Result1: " + result1.get());
System.out.println("Result2: " + result2.get());
System.out.println("Result3: " + result3.get());
}
public static class MockApi {
private final AtomicLong delay = new AtomicLong(0);
private final ScheduledExecutorService executorService = Executors.newSingleThreadScheduledExecutor();
public CompletableFuture<Integer> requestAThing(String req1) {
return this.scheduleCompletableFuture(new ActualApi().requestHandler(req1), delay.incrementAndGet(), TimeUnit.SECONDS);
}
public <T> CompletableFuture<T> scheduleCompletableFuture(
final CompletableFuture<T> command,
final long delay,
final TimeUnit unit) {
final CompletableFuture<T> completableFuture = new CompletableFuture<>();
this.executorService.schedule(
(() -> {
try {
return completableFuture.complete(command.get());
} catch (Throwable t) {
return completableFuture.completeExceptionally(t);
}
}),
delay,
unit
);
return completableFuture;
}
}
public static class ActualApi {
public CompletableFuture<Integer> requestHandler(String request) {
return CompletableFuture.supplyAsync(() -> {
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
return Integer.parseInt(request.substring(3));
});
}
}
}
JDK 9 and onward
If you are using a JDK 9 version, you may make use of the supported delayed Executor:
CompletableFuture<String> future = new CompletableFuture<>();
future.completeAsync(() -> {
try {
// do something
} catch(Throwable e) {
// do something on error
}
}, CompletableFuture.delayedExecutor(1, TimeUnit.SECONDS));
Your MockApi#requestAThing would then be cleaner and shorter and you are no more in need of a custom ScheduledExecutor:
public static class MockApi {
private final AtomicLong delay = new AtomicLong(0);
public CompletableFuture<Integer> requestAThing(String req1) {
CompletableFuture<Void> future = new CompletableFuture<>();
return future.completeAsync(() -> null, CompletableFuture.delayedExecutor(delay.incrementAndGet(), TimeUnit.SECONDS))
.thenCombineAsync(new ActualApi().requestHandler(req1), (nil, result) -> result);
}
// ...
}
You might consider using bucket4j
I have found a way to produce my desired behaviour. By limiting my Executor to a single Thread I can queue up calls and they will follow the order I queued them up in.
I will supply the code of my mock classes below for anyone interested:
import java.util.Random;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
public class Sandbox2 {
public static void main(String[] args) throws ExecutionException, InterruptedException {
MockApi mockApi = new MockApi();
CompletableFuture<Integer> result1 = mockApi.requestAThing("Req1");
System.out.println("Request1 queued up");
CompletableFuture<Integer> result2 = mockApi.requestAThing("Req2");
System.out.println("Request2 queued up");
CompletableFuture<Integer> result3 = mockApi.requestAThing("Req3");
System.out.println("Request3 queued up");
//Some other logic happens here
Thread.sleep(10000);
System.out.println("Result1: " + result1.get());
System.out.println("Result2: " + result2.get());
System.out.println("Result3: " + result3.get());
System.exit(0);
}
public static class MockApi {
ActualApi actualApi = new ActualApi();
private ExecutorService executorService = Executors.newSingleThreadExecutor();
;
public CompletableFuture<Integer> requestAThing(String req1) {
CompletableFuture<Integer> completableFutureCompletableFuture = CompletableFuture.supplyAsync(() -> {
try {
System.out.println("Waiting with " + req1);
waitForTheRightMoment(new Random().nextInt(1000) + 1000);
System.out.println("Done Waiting with " + req1);
return actualApi.requestHandler(req1).get();
} catch (InterruptedException e) {
e.printStackTrace();
} catch (ExecutionException e) {
e.printStackTrace();
}
return null;
}, executorService);
return completableFutureCompletableFuture;
}
private void waitForTheRightMoment(int time) {
try {
Thread.sleep(time);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
public static class ActualApi {
public CompletableFuture<Integer> requestHandler(String request) {
return CompletableFuture.supplyAsync(() -> {
try {
Thread.sleep(new Random().nextInt(1000) + 1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("Request Handled " + request);
return Integer.parseInt(request.substring(3));
});
}
}
}
I want to retrieve data of different types from a database and return to the user within an HTTP result from a Spring Boot service. Because the database retrieval takes a significant amount of time for each, I am making these DB calls asynchronously with CompletableFuture. The pattern I have works and saves time compared to doing this synchronously, but I feel that it can and should be laid out in a cleaner fashion.
I edited the code to change the types to 'PartA', 'PartB', 'PartC', but this is otherwise how it appears. Currently, the service accepts the lists of different types (PartA, PartB, PartC), creates Completable future types of each list calling its own CompletableFuture method that calls the DB, builds a generic list of CompleteableFutures with each type, "gets" the generic list, then adds all the contents of each Future list to the list passed into the service.
This is how the Service methods are coded:
Service.java:
public void metadata(final List<PartA> partAs,final List<PartB> partBs,final List<PartC> partCs,
String prefix,String base,String suffix) throws Exception {
try {
CompletableFuture<List<PartA>> futurePartAs = partACompletableFuture(prefix,base,suffix).thenApply(list -> {
logger.info("PartA here");
return list;
});
CompletableFuture<List<PartB>> futurePartBs = partBCompletableFuture(prefix,base,suffix).thenApply(list -> {
logger.info("PartBs here");
return list;
});
CompletableFuture<List<PartC>> futurePartCs = partCCompletableFuture(prefix,base,suffix).thenApply(list -> {
logger.info("PartCs here");
return list;
});
CompletableFuture<?> combinedFuture = CompletableFuture.allOf(CompletableFuture.allOf(futurePartAs, futurePartBs, futurePartCs));
combinedFuture.get();
partAs.addAll(futurePartAs.get());
partBs.addAll(futurePartBs.get());
partCs.addAll(futurePartCs.get());
} catch (Exception e) {
logger.error("Exception: ", e);
throw e;
}
}
#Async("asyncExecutor")
public CompletableFuture<List<PartA>> partACompletableFuture(String prefix,String base,String suffix) {
return CompletableFuture.supplyAsync(() -> {
try {
logger.info("start PartA");
return getPartAs(prefix,base,suffix);
} catch (Exception e) {
logger.error("Exception: ", e);
throw e;
}
});
}
#Async("asyncExecutor")
public CompletableFuture<List<PartB>> partBCompletableFuture(String prefix,String base,String suffix) {
return CompletableFuture.supplyAsync(() -> {
try {
logger.info("start B");
return getPartBs(prefix,base,suffix);
} catch (Exception e) {
logger.error("Exception: ", e);
throw e;
}
});
}
#Async("asyncExecutor")
public CompletableFuture<List<PartC>> partCCompletableFuture(String prefix,String base,String suffix) {
return CompletableFuture.supplyAsync(() -> {
try {
logger.info("start PartC");
return getPartCs(prefix,base,suffix);
} catch (Exception e) {
logger.error("Exception: ", e);
throw e;
}
});
}
In case you wish to view the Controller and Response type:
Controller.java
#GetMapping(value="/parts/metadata",produces = { MediaType.APPLICATION_JSON_VALUE })
public ResponseEntity<MetadataResponse> metadata (#ApiParam(name="prefix",value = "Prefix value for a part",required = false)
#RequestParam(required=false) String prefix,
#ApiParam(name="base",value = "Base value for a part",required= true)
#RequestParam String base,
#ApiParam(name="suffix",value = "Suffix value for a part",required=false)
#RequestParam(required=false) #NotBlank String suffix ) throws Exception {
final List<PartA> partAs = new ArrayList<>();
final List<PartB> partBs = new ArrayList<>();
final List<PartC> partCs = new ArrayList<>();
service.metadata(partAs,partBs,partCs,prefix,base,suffix);
MetadataResponse.MetadataResponseResult res = MetadataResponse.MetadataResponseResult.builder()
.partAs(partAs)
.partBs(partBs)
.partCs(partCs)
.build();
return ResponseEntity.ok(MetadataResponse.result(res, MetadataResponse.class));
}
MetadataResponse.java
#ApiModel(value = "MetadataResponse", parent = BaseBodyResponse.class, description = "Part A, B, C")
public class MetadataResponse extends BaseBodyResponse<MetadataResponse.MetadataResponseResult> {
#Data
#Builder
#NoArgsConstructor
#AllArgsConstructor
#ApiModel(value = "MetadataResponseResult", description = "This Model holds Part As, Bs, Cs")
public static class MetadataResponseResult {
List<PartA> partAs;
List<PartB> partBs;
List<PartC> partCs;
}
}
I don't understand exactly why you need to pass all these lists as parameters in this case: public void metadata(final List<PartA> partAs,final List<PartB> partBs,final List<PartC> partCs, String prefix,String base,String suffix) throws Exception You could modify this method to return the MetadataResponseResult class you already have and use the lists from the ComparableFutures directly
I would remove the thenApply methods since you just log a statement and you don't actually change the results.
Instead of having the three methods (partACompletableFuture, partABCompletableFuture, partCCompletableFuture) you could have one method that receives a Supplier as a parameter.
#Async("asyncExecutor")
public <T> CompletableFuture<T> partCompletableFuture(Supplier<T> supplier) {
return CompletableFuture.supplyAsync(() -> {
try {
logger.info("start Part");
return supplier.get();
} catch (Exception e) {
logger.error("Exception: ", e);
throw e;
}
});
}
Aftewards you can use it as so:
CompletableFuture<List<PartA>> futurePartAs = partCompletableFuture(() ->
getPartAs(prefix,base,suffix));
It should much cleaner. Hope this helped!
Given a Stream and a method that returns a Stream for different arguments as data source, I'm looking for a way to merge the streams via flatMap(..) and catching certain Exceptions during the execution.
Let's take the following code snippet:
public class FlatMap {
public static void main(final String[] args) {
long count;
// this might throw an exception
count = Stream.of(0.2, 0.5, 0.99).flatMap(chance -> getGenerator(chance, 20)).count();
// trying to catch the exception in flatMap() will not work
count = Stream.of(0.2, 0.5, 0.99).flatMap(chance -> {
try {
return getGenerator(chance, 20);
} catch (final NullPointerException e) {
return Stream.empty();
}
}).count();
System.out.println(count);
}
// !! we cannot change this method, we simply get a Stream
static Stream<Object> getGenerator(final double chance, final long limit) {
return Stream.generate(() -> {
if (Math.random() < chance) return new Object();
throw new NullPointerException();
}).limit(limit);
}
}
Is there any way to catch the exception of each individual Stream that was created by getGenerator(..) and simply suppress the Exception, replacing the "corrupted" Stream with an empty one or skip those elements from the specific generator Stream?
It is possible to wrap the Stream into another using the Spliterator. This method will protect a given Stream by catching the Exception and saving this state:
static <T> Stream<T> protect(final Stream<T> stream) {
final Spliterator<T> spliterator = stream.spliterator();
return StreamSupport.stream(
new Spliterators.AbstractSpliterator<T>(Long.MAX_VALUE,
spliterator.characteristics() & ~Spliterator.SIZED) {
private boolean corrupted = false;
#Override
public boolean tryAdvance(final Consumer<? super T> action) {
if (!corrupted) try {
return spliterator.tryAdvance(action);
} catch (final Exception e) {
// we suppress this one, stream ends here
corrupted = true;
}
return false;
}
}, false);
}
Then we can wrap our Stream method and safely pass it in flatMap(..):
// we protect the stream by a wrapper Stream
count = Stream.of(0.2, 0.5, 0.99)
.flatMap(chance -> protect(getGenerator(chance, 20)))
.count();
One work around is to force the Stream created by getGenerator to be evaluated within the flatMap method implementation. This forces the NullPointerException to be thrown within the try-catch block, and therefore, able to be handled.
To do this, you can collect the Stream (to a List for example):
getGenerator(chance, 20).collect(Collectors.toList()).stream()
Incorporating this into your original snippet:
public class FlatMap {
public static void main(final String[] args) {
long count;
// trying to catch the exception in flatMap() will not work
count = Stream.of(0.2, 0.5, 0.99)
.flatMap(chance -> {
try {
return getGenerator(chance, 20).collect(Collectors.toList()).stream();
}
catch (final NullPointerException e) {
return Stream.empty();
}
})
.count();
System.out.println(count);
}
// !! we cannot change this method, we simply get a Stream
static Stream<Object> getGenerator(final double chance, final long limit) {
return Stream.generate(() -> {
if (Math.random() < chance) return new Object();
throw new NullPointerException();
}).limit(limit);
}
}
Warning: this approach may reduce performance if the getGenerator Stream would be better to evaluate lazily.
Try this:
static <T> Supplier<T> getOrNull(Supplier<T> supplier) {
return () -> {
try {
return supplier.get();
} catch (Throwable e) {
return null;
}
};
}
static Stream<Object> getGenerator(final double chance, final long limit) {
return Stream.generate(
getOrNull(
() -> {
if (Math.random() < chance) return new Object();
throw new NullPointerException();
// You can throw any exception here
}))
.limit(limit)
.filter(Objects::isNull);
}
Then simply call getGenerator:
count = Stream.of(0.2, 0.5, 0.99)
.flatMap(chance -> getGenerator(chance, 20))
.count();
I'm having some tasks created like follows (this is just for demonstration normally network calls):
public class RandomTask implements Function<String, String> {
private int number;
private int waitTime;
private boolean throwError;
public RandomTask(int number, int waitTime, boolean throwError) {
this.number = number;
this.waitTime = waitTime;
this.throwError = throwError;
}
#Override
public String apply(String s) {
System.out.println("Job " + number + " started");
try {
Thread.sleep(waitTime);
if (throwError) {
throw new InterruptedException("Something happened");
}
} catch (InterruptedException e) {
System.out.println("Error " + e.getLocalizedMessage());
}
return "RandomTask " + number + " finished";
}
}
Then I have a Chain class where I chain some tasks together per job.
static CompletableFuture<String> start(ExecutorService executorService) {
CompletableFuture<String> future2 = CompletableFuture.supplyAsync(() -> "Foo", executorService)
.thenApplyAsync(new RandomTask(3, 100, false), executorService)
.thenApplyAsync(new RandomTask(4, 100, false), executorService);
return future2;
}
I then start 2 chains as follows:
CompletableFuture<Void> combinedFuture = CompletableFuture.allOf(Chain1.start(fixedThreadPool), Chain2.start(fixedThreadPool));
try {
combinedFuture.get();
} catch (InterruptedException | ExecutionException e) {
e.printStackTrace();
}
That way the two chains start off at the same time.
Now I want to throw an exception in a task and catch it where I call combinedFuture.get() so that I know which task has failed in my chain.
The thing is dat I can't adapt the Function because CompletableFutures complains about this. I tried with:
#FunctionalInterface
public interface CheckedFunction<T, R> {
R apply(T t) throws InterruptedException;
}
But this doesn't work. Is this not possible or how can I achieve my goal?
“That way the two chains start off at the same time.” indicates that you have a fundamentally wrong understanding of how CompletableFuture works.
Asynchronous operations are submitted to the executor service right when you create them or as soon as their prerequisites are available. So in case of supplyAsync, which has no dependencies, the asynchronous operation starts right within the supplyAsync invocation.
All, a construct like CompletableFuture.allOf(job1, job2).get() does, is to create a new stage depending on both jobs and waiting for its completion, so the net result is just to wait for the completion of both jobs. It does not start the jobs. They are already running. Waiting for a completion has no influence of the process of completing.
Chaining a CompletableFuture with a custom function type allowing checked exceptions can be done as
public static <T,R> CompletableFuture<R> thenApplyAsync(
CompletableFuture<T> f, CheckedFunction<? super T, ? extends R> cf,
Executor e) {
CompletableFuture<R> r = new CompletableFuture<>();
f.whenCompleteAsync((v,t) -> {
try {
if(t != null) r.completeExceptionally(t);
else r.complete(cf.apply(v));
} catch(Throwable t2) {
r.completeExceptionally(t2);
}
}, e);
return r;
}
To use this method, instead of chaining calls on the CompletableFuture, you have to nest them. E.g.
static CompletableFuture<String> start(ExecutorService executorService) {
CompletableFuture<String> future2 =
thenApplyAsync(thenApplyAsync(
CompletableFuture.supplyAsync(() -> "Foo", executorService),
new RandomTask(3, 100, false), executorService),
new RandomTask(4, 100, false), executorService);
return future2;
}
given
public class RandomTask implements CheckedFunction<String, String> {
private int number, waitTime;
private boolean throwError;
public RandomTask(int number, int waitTime, boolean throwError) {
this.number = number;
this.waitTime = waitTime;
this.throwError = throwError;
}
#Override
public String apply(String s) throws InterruptedException {
System.out.println("Job " + number + " started");
Thread.sleep(waitTime);
if (throwError) {
throw new InterruptedException("Something happened in "+number);
}
return "RandomTask " + number + " finished";
}
}
You can still create two tasks and wait for both like
CompletableFuture.allOf(Chain1.start(fixedThreadPool), Chain2.start(fixedThreadPool))
.join();