I'm kinda new to using CompletableFuture API and I have a question regarding usage of allOf. From what I read, completable-future should be in complete state and allOf logic should be executed when all associated futures complete, including completed-exceptionally. But here's my smaple code for which allOf block never gets executed -
public static void test() {
CompletableFuture<String> r1 = CompletableFuture.supplyAsync(() -> {
try{
Thread.sleep(1000);
throw new RuntimeException("blahh !!!");
}catch (Exception e) {
throw new RuntimeException(e);
}
});
CompletableFuture<String> r2 = CompletableFuture.supplyAsync(() -> "55");
CompletableFuture<String> r3 = CompletableFuture.supplyAsync(() -> "56");
CompletableFuture.allOf(r1, r2, r3).thenRun(() -> { System.out.println(Thread.currentThread()+" --- End."); });
Stream.of(r1, r2, r3).forEach(System.out::println);
try{
System.out.println(Thread.currentThread()+" --- SLEEPING !!!");
Thread.sleep(3000);
System.out.println(Thread.currentThread()+" --- DONE !!!");
} catch (Exception e) {
//e.printStackTrace();
}
Stream.of(r1, r2, r3).forEach(System.out::println);
}
The problem is not that your allOf CompletableFuture never completes. It does.
What causes your code not to run is thenRun's expectation:
Returns a new CompletionStage that, when this stage completes normally, executes the given action. See the CompletionStage documentation for rules covering exceptional completion.
You probably already know that when one of allOf's futures completes exceptionally, the resulting future also completes exceptionally:
Returns a new CompletableFuture that is completed when all of the given CompletableFutures complete. If any of the given CompletableFutures complete exceptionally, then the returned CompletableFuture also does so, with a CompletionException holding this exception as its cause.
In short, don't use thenRun if you want to run an action on your allOf future irrespective of how it comples. As an alternative, you can use whenComplete:
CompletableFuture.allOf(r1, r2, r3)
.whenComplete((a, ex) -> System.out.println(Thread.currentThread() + " --- End."));
You can also use a combination of thenRun + exceptionally, one of which will run:
CompletableFuture<Void> all = CompletableFuture.allOf(r1, r2, r3);
all.thenRun(() -> {
System.out.println(Thread.currentThread() + " --- End.");
});
all.exceptionally(ex -> {
System.out.println(ex);
return null;
});
Related
I am writing a function that creates multiple (7) CompletableFutures. Each of these futures basically does two things :
using supplyAsync(), fetch data from some DB
using thenAccept(), write this data to a CSV file
When all the 7 futures have finished the job, I want to continue with further code execution. So, I am using allOf() and then calling a join() on the Void CompletableFuture returned by allOf().
The problem is, even after all futures have executed (I can see the CSVs getting generated), the join() call remains stuck and further code execution is blocked forever.
I have tried the following things :
Waiting on each future one by one calling a join() after each future. This works but, at the cost of concurrency. I don't want to do this.
Tried using get() with a TIMEOUT instead of join(). But, this always ends up throwing an exception (as get always times out) which is undesirable.
Saw this JDK bug : https://bugs.openjdk.java.net/browse/JDK-8200347 . Not sure if this is a similar issue.
Tried running without a join() or get() which will not hold the thread execution and again is not desirable.
The main function which creates all futures.
public CustomResponse process() {
CustomResponse msgResponse = new CustomResponse();
try {
// 1. DbCall 1
CompletableFuture<Void> f1 = dataHelper.fetchAndUploadCSV1();
// 2. DbCall 2
CompletableFuture<Void> f2 = dataHelper.fetchAndUploadCSV2();
// 3. DbCall 3
CompletableFuture<Void> f3 = dataHelper.fetchAndUploadCSV3();
// 4. DbCall 4
CompletableFuture<Void> f4 = dataHelper.fetchAndUploadCSV4();
// 5. DbCall 5
CompletableFuture<Void> f5 = dataHelper.fetchAndUploadCSV5();
// 6. DbCall 6
CompletableFuture<Void> f6 = dataHelper.fetchAndUploadCSV6();
// 7. DbCall 7
CompletableFuture<Void> f7 = dataHelper.fetchAndUploadCSV7();
CompletableFuture<Void>[] fAll = new CompletableFuture[] {f1, f2, f3, f4, f5, f6, f7};
CompletableFuture.allOf(fAll).join();
msgResponse.setProcessed(true);
msgResponse.setMessageStatus("message");
} catch (Exception e) {
msgResponse.setMessageStatus(ERROR);
msgResponse.setErrorMessage("error");
}
return msgResponse;
}
Each of the fetchAndUploadCSV() functions looks like this :
public CompletableFuture<Void> fetchAndUploadCSV1() {
return CompletableFuture.supplyAsync(() -> {
try {
return someService().getAllData1();
} catch (Exception e) {
throw new RuntimeException(e);
}
}).thenAccept(results -> {
try {
if (results.size() > 0) {
csvWriter.uploadAsCsv(results);
}
else {
log.info(" No data found..");
}
} catch (Exception e) {
throw new RuntimeException(e);
}
});
}
And this is what csvWriter.uploadAsCsv(results) looks like -
public <T> void uploadAsCsv(List<T> objectList) throws Exception {
long objListSize = ((objectList==null) ? 0 : objectList.size());
log.info("Action=Start, objectListSize=" + objListSize);
ByteArrayInputStream inputStream = getCsvAsInputStream(objectList);
Info fileInfo = someClient.uploadFile(inputStream);
log.info("Action=Done, FileInfo=" + ((fileInfo==null ? null : fileInfo.getID())));
}
I am using OpenCSV here to convert the data to CSV stream. And I can always see the last log line.
Expected Results :
All data fetched, CSVs generated and CustomResponse should return as processed with no error message.
Actual Results :
All data fetched, CSVs generated and main thread hung.
You can use join on each created CompletableFuture without sacrificing concurrency:
public CustomResponse process() {
CustomResponse msgResponse = new CustomResponse();
List<CompletableFuture<Void>> futures = Arrays.asList(dataHelper.fetchAndUploadCSV1(),
dataHelper.fetchAndUploadCSV2(),
dataHelper.fetchAndUploadCSV3(),
dataHelper.fetchAndUploadCSV4(),
dataHelper.fetchAndUploadCSV5(),
dataHelper.fetchAndUploadCSV6(),
dataHelper.fetchAndUploadCSV7());
return CompletableFuture.allOf(futures.toArray(new CompletableFuture<?>[0]))
.thenApply(v -> {
msgResponse.setProcessed(true);
msgResponse.setMessageStatus("message");
return msgResponse;
})
.exceptionally(throwable -> {
msgResponse.setMessageStatus("ERROR");
msgResponse.setErrorMessage("error");
return msgResponse;
}).join();
}
allOf returns a new CompletableFuture that is completed when all of the given CompletableFutures complete. So, when join is invoked in thenApply, it returns immediately. In essence, joining is happening to already completed futures. This way blocking is eliminated. Also, to handle possible exceptions, exceptionally should be invoked.
I am trying to learn the basics of RxJava2 library and right now I am stuck at the following moment:
I have generated myFlowable via Flowable.generate(...) and now I need to wait while all the tasks will finish its execution, before I can proceed further.
This is the code to showcase the problem:
myFlowable.parallel()
.runOn(Schedulers.computation())
.map(val -> myCollection.add(val))
.sequential()
.subscribe(val -> {
System.out.println("Thread from subscribe: " + Thread.currentThread().getName());
System.out.println("Value from subscribe: " + val.toString());
});
System.out.println("Before sleep - Number of objects: " + myCollection.size());
try {
Thread.sleep(1000);
System.out.println("After sleep - Number of objects: " + myCollection.size());
} catch (InterruptedException e) {
e.printStackTrace();
}
I run through all my tasks and add the results to collection. And if I check the collection size right after myFlowable block then it will be different, if I check it after small Thread.sleep(). Is there any way to check that all the tasks finished its execution and we can proceed further? Any help or guidance will be greatly appreciated.
As RxJava is asynchronous the java code below observable will run while the observable will run in a different thread thets why if you want to be notified if Flowable has finished emitting data you should do that in RxJava stream. for that you have an operator .doOnComplete
here you have an example how to detect when stream is finished
Flowable.range(0, 100).parallel()
.runOn(Schedulers.computation())
.map(integer -> {
return integer;
})
.sequential()
.doOnComplete(() -> {
System.out.println("finished");
})
.subscribe(integer -> System.out.println(integer));
You could use an AtomicBoolean, initialize it to false and set it to true using doFinally().
doFinally() is called after the Observable signals onError or onCompleted or it gets disposed by the downstream.
Then sleep the main thread until completed value is true.
Using your example:
AtomicBoolean completed = new AtomicBoolean(false);
myFlowable.parallel()
.runOn(Schedulers.computation())
.map(val -> myCollection.add(val))
.sequential()
.doFinally(() -> completed.set(true))
.subscribe(val -> {
...
});
...
try {
while(!completed.get()){
Thread.sleep(1000);
...
}
...
} catch (InterruptedException e) {
e.printStackTrace();
}
Use Flowable::blockingSubscribe() - Runs the current Flowable to a terminal event, ignoring any values and rethrowing any exception.
http://reactivex.io/RxJava/3.x/javadoc/io/reactivex/rxjava3/core/Flowable.html#blockingSubscribe--
I am running a list of runnables using ExecutorService and collating all results using CompletableFuture. I would like to correlate which instance of CompletableFuture ran a specific runnable.
Here is the actual code
public static void runTasks(final List<Runnable> tasks, final int threadCount) {
final ExecutorService es = Executors.newFixedThreadPool(threadCount);
final CompletableFuture<?>[] futures = tasks.stream()
.map(task -> CompletableFuture.runAsync(task, es))
.toArray(CompletableFuture[]::new);
try {
CompletableFuture.allOf(futures).join();
es.shutdown();
} catch (Exception e) {
System.exit(1);
}
}
I have the results stored in futures variable
CompletableFuture<?>[] futures
Is there a way to get the class name of runnable whose result is stored in an instance of future?
I am trying to print the individual task result as follows:
for (CompletableFuture future : futures) {
final boolean taskCompletedSuccessfully = future.isDone() && !(future.isCompletedExceptionally() || future.isCancelled());
LOGGER.info("Task completion status for {} : {}", <runnable class name>, (taskCompletedSuccessfully ? "SUCCESSFUL" : "FAILED"));
}
It is not possible to retrieve any information about the Runnable because the CompletableFuture does not hold any reference to it.
You will thus have to store the future and the runnable (or its class name) together in some Pair implementation, e.g.:
final List<Pair<Runnable, CompletableFuture<Void>>> futures = tasks.stream()
.map(task -> new Pair<>(task, CompletableFuture.runAsync(task, es)))
.collect(toList());
try {
CompletableFuture.allOf(futures.stream().map(Pair::getB).toArray(CompletableFuture[]::new)).join();
} catch (Exception e) {
log.warn("At least one future failed", e);
}
es.shutdown();
futures.forEach(pair -> {
CompletableFuture<Void> future = pair.getB();
final boolean taskCompletedSuccessfully = !future.isCompletedExceptionally();
log.info("Task completion status for {} : {}", pair.getA().getClass().getSimpleName(), (taskCompletedSuccessfully ? "SUCCESSFUL" : "FAILED"));
});
A few notes:
if any of the tasks fails, the allOf() will fail as well. You probably don't want to exit() in that case – otherwise you will always have only successful tasks logged;
after allOf().join(), you are guaranteed that isDone() holds true for all tasks, no need to check it;
isCancelled() (which is not possible here) implies isCompletedExceptionally()
In the following code
public CompletableFuture<String> getMyFuture(String input)
{
CompletableFuture<String> future = new CompletableFuture<String>().thenApply((result) -> result+ "::");
ExecutorService service = Executors.newFixedThreadPool(6);
service.submit(() -> {
try {
future.complete(getResult(input));
} catch (InterruptedException e) {
e.printStackTrace();
}
});
return future;
}
public String getResult(String input) throws InterruptedException
{
Thread.sleep(3000);
return "hello "+ input +" :" + LocalTime.now();
}
I am expecting the output to contain trailing "::" but program doesn't is "hello first :16:49:30.231
" Is my implementation of apply correct ?
You're invoking complete() method of the CompletionStage that you got at the first line (where you call "thenApply" method).
If your intention is to complete the CompletableFuture with some string value (future.complete(getResult(input))) and then apply some function, you'd better place thenApply() at the end (where you return the future).
public CompletableFuture<String> getMyFuture(String input)
{
CompletableFuture<String> future = new CompletableFuture<String>();
ExecutorService service = Executors.newFixedThreadPool(6);
service.submit(() -> {
try {
future.complete(getResult(input));
} catch (InterruptedException e) {
e.printStackTrace();
}
});
return future.thenApply(result -> result+ "::");
}
I don't know how to explain it in a more understandable way. But in short: you're calling complete() method on the wrong object reference inside your Runnable.
You are creating two CompletableFuture instances. The first, created via new CompletableFuture<String>() will never get completed, you don’t even keep a reference to it that would make completing it possible.
The second, created by calling .thenApply((result) -> result+ "::") on the first one, could get completed by evaluating the specified function once the first one completed, using the first’s result as an argument to the function. However, since the first never completes, the function becomes irrelevant.
But CompletableFuture instances can get completed by anyone, not just a function passed to a chaining method. The possibility to get completed is even prominently displayed in its class name. In case of multiple completion attempts, one would turn out to be the first one, winning the race and all subsequent completion attempts will be ignored. In your code, you have only one completion attempt, which will successfully complete it with the value returned by getResult, without any adaptations.
You could change your code to keep a reference to the first CompletableFuture instance to complete it manually, so that the second gets completed using the function passed to thenApply, but on the other hand, there is no need for manual completion here:
public CompletableFuture<String> getMyFuture(String input) {
ExecutorService service = Executors.newFixedThreadPool(6);
return CompletableFuture.supplyAsync(() -> getResult(input), service)
.thenApply(result -> result + "::");
}
public String getResult(String input) {
LockSupport.parkNanos(TimeUnit.SECONDS.toNanos(3));
return "hello "+ input +" :" + LocalTime.now();
}
When specifying the executor to supplyAsync, the function will be evaluated using that executor. More is not needed.
Needless to say, that’s just for example. You should never create a temporary thread pool executor, as the whole point of a thread pool executor is to allow reusing the threads (and you’re using only one of these six threads at all) and it should get shut down after use.
I've found that CompletableFuture::join seems uninterruptible when not completed:
// CompletableFuture::join implementation from JDK 8 sources
public T join() {
Object r;
return reportJoin((r = result) == null ? waitingGet(false) : r);
}
In above implementation, waitingGet(false) will ignore the interrupt flag of the working Thread and continue waiting. I'm wondering how can I interrupt a Thread in which I call CompletableFuture::join.
Do not use join() if you want to support interruption, use get() instead. Basically they are the same except:
join() is only defined in CompletableFuture whereas get() comes form interface Future
join() wraps exceptions in CompletionException whereas get() wraps them in ExecutionException
get() might be interrupted and would then throw an InterruptedException
Note that what you interrupt is the Thread, not the Future. For example, the following code interrupts the main thread while it is waiting on myFuture.get():
CompletableFuture<Void> myFuture = new CompletableFuture<>();
Thread mainThread = Thread.currentThread();
CompletableFuture.runAsync(() -> {
try {
Thread.sleep(1000);
System.out.println("Interrupting…");
mainThread.interrupt();
Thread.sleep(1000);
System.out.println("Completing");
myFuture.complete(null);
} catch (InterruptedException e) {
e.printStackTrace();
}
});
try {
myFuture.get();
System.out.println("Get succeeded");
} catch (Exception e) {
System.out.println("Get failed");
e.printStackTrace();
}
Output:
Interrupting…
Get failed
java.lang.InterruptedException
at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:347)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
at CompletableFutureInteruption.main(CompletableFutureInteruption.java:37)
…
If you replace get() by join(), the interrupt will indeed not work.
I finally give up to interrupt the Thread which blocks in waiting for CompletableFuture::join finish.
Instead I use CompletableFuture::allof to get a CompletableFuture all which ends when all my joined Futures end. And then just call the get() method of the all Future in the working thread. When get() returns, I then collect all my results by iterating all my joined Futures and call getNow on them. Such a procedure is interruptible.
I clearly known in the end you are not actually looking for the interruptible, but as a way around it can be interrupted by exceptions as follows (though it's join()):
private static void testCompleteExceptionally() {
String name = "Hearen";
CompletableFuture<String> completableFuture
= CompletableFuture.supplyAsync(() -> {
delay(500L);
if (name == null) {
throw new RuntimeException("Computation error!");
}
return "Hello, " + name;
});
if (name != null) {
completableFuture.completeExceptionally(new RuntimeException("Calculation failed!"));
}
out.println(completableFuture.handle((s, t) -> s != null ? s : "Hello, Stranger!" + t.toString()).join());
}