I want to run CompletableFuture in a TestNG class with #Test annotation.
Below is the code snippet, recursionFuture method is called recursively to do a certain task by the main() method.
Now if I use main() as public static void main(String[] args)) then everything works as desired. But when I used main() with #Test annotation then TestNG stops the execution in between and the whole task is not performed.
What should I do to make #Test to wait until all tasks are completed by recursionFuture?
I've to use CompletableFuture for async tasks and need to use #Test also.
Any help would be appreciated. Thanks.
//public static void main(String[] args) throws FileNotFoundException, InterruptedException, ExecutionException
#Test // --> this logic of recursion with threads is problematic with testNG
public static void main() throws FileNotFoundException, InterruptedException, ExecutionException
{
System.setOut(new PrintStream(new File("/Users/Pankaj/Desktop/Thread")));
Method[] method = ConcurrencyPoC_CompletableFuture.class.getMethods();
for(int i=0; i<method.length; i++)
{
if(method[i].getName().startsWith("task"))
{
TreeMap<Object, Object> m = new TreeMap<>();
m.put(method[i].getName(), method[i]);
m.put("status", new AtomicBoolean(true));
taskmap.put(method[i].getName(), m);
}
}
//submitting tasks
ExecutorService ex = Executors.newCachedThreadPool();
CompletableFuture<?> [] arrayFutures = new CompletableFuture[3];
recursionFuture(ex, arrayFutures);
}
public static String recursionFuture(ExecutorService ex, CompletableFuture<?> [] arrayFutures)
{
try
{
//check if any of submitted future is completed - to be used in case of array
for(CompletableFuture<?> future : arrayFutures)
{
future = CompletableFuture.supplyAsync(() -> new ConcurrencyPoC_CompletableFuture().executeTask(), ex);
//storing future in a final variable for further calculation
final CompletableFuture<?> task = future;
CompletableFuture.anyOf(task).thenRunAsync(() ->
{
try {
//apply recursion only when future's output is not null, otherwise there will be hell lot of futures get created which
//don't do anything just eating up memory and executing the else block of executeTask() method.
// Latest change, as soon as any task is free, create a new array of 1 size to do the next task
if(task.get() != null)
recursionFuture(ex, new CompletableFuture[1]);
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
, ex);
}
}
catch(Exception e)
{
e.printStackTrace();
}
return "ALl TASK COMPLETED";
}
There is no correspondence between TestNG #Test annotation and your completable future.
The problem is in your tast.get() method. It should be blocking running threads until all completable futures are finished.
I use Completable Future in my tests and never ever I ever got into a problem with TestNG.
The problem is one of your future's finishes, and you return to the test method instead of waiting for all the futures. You should combine all your futures with thenApplyAsync() or compose() because your Future is final you are waiting only for one future. Plus you should not use CompletbleFuture.Any() because it returns to execution when the first future is complete.
Related
From CompletableFuture javadocs:
Actions supplied for dependent completions of non-async methods may be
performed by the thread that completes the current CompletableFuture,
or by any other caller of a completion method.
and looking into CompletableFuture.complete() it calls postComplete() which seems to pop off the dependent chains and tries to complete them.
To test my understanding, I wrote a simple program
import java.util.concurrent.*;
public class TestCompletableFuture {
public static void main(String[] args) throws Exception {
CompletableFuture<Void> future = new CompletableFuture<Void>()
.whenComplete((res, exc) -> {
System.out.println("inside handle.");
if (exc != null) {
System.out.println("exception.");
}
System.out.println("completed.");
}
);
future.completeExceptionally(new Exception("exception"));
System.out.println("done.");
}
}
the output of the code:
done.
From my understanding, when the main thread calls future.completeExceptionally() it should invoke the function passed into CompletableFuture.whenComplete().
Why is this not the case?
This is because you are completing the wrong future. You need to get a reference to the first stage and complete that to see the whenComplete in action:
public class TestCompletableFuture {
public static void main(String[] args) throws Exception {
// get reference
CompletableFuture<Void> future = new CompletableFuture<>();
// configure the action that to be run when the future is complete
CompletableFuture<Void> future2 = future
.whenComplete((res, exc) -> {
System.out.println("inside handle.");
if (exc != null) {
System.out.println("exception.");
}
System.out.println("completed.");
}
);
future.completeExceptionally(new Exception("exception"));
System.out.println("done.");
}
}
So, now the code speaks for itself... When the future is complete run that action (res, exc) -> {...}. And then just trigger that completion on the future by calling completeExceptionally(...) on it.
Another thing to note is that all of the above stages (futures) are completed exceptionally now:
System.out.println(future.isDone()); // true
System.out.println(future2.isDone()); // true
System.out.println(future.isCompletedExceptionally()); // true
System.out.println(future2.isCompletedExceptionally()); // true
I have a problem with concurrent programming in Java. I am working on my bachelor thesis and I have to make several methods which will return me a String value. In the Futures TriggerMessageFututre and getMeterValuesFuture is a process running which takes between 1-5 seconds and returns a String Value when it's finished.
The problem is now that future.get() is blocking my main thread. I want to call the TriggerMessage and the getMeterValue methode in my main without blocking my main thread and get their answer as a return value when they are finished. I wasn't able to find a way to solve my problem, because either it was a solution without return value or it was a solution which blocked the thread.
private String TriggerMessage(String Messagetyp) throws InterruptedException, ExecutionException{
Future<String> future = new communicator().TriggerMessageFuture(queue,centralSystem,Messagetyp);
while(!future.isDone()) {
System.out.println("[TriggerMessage]: Calculating... ");
Thread.sleep(500);
}
String result = future.get(); //blocking
return result;
}
private String getMeterValue(String key) throws Exception{
Future<String> future = new communicator().getMeterValueFuture(queue,centralSystem,key);
while(!future.isDone()) {
System.out.println("[getMeterValue]: Calculating...");
Thread.sleep(500);
}
String result = future.get(); //blocking
return result;
}
It depends on what main thread are you referring to, plus if you can use CompletableFutures instead of plain old Java Futures.
Using the main(String[] args) thread
It's not possible to do it without any form of blocking. If you are not blocking on get, you'll have to block on a BlockingQueue implementation, otherwise the main thread just ends.
Using the Swing Event Dispatch thread
You'd need to submit a continuation task which is not possible with Future.get from the outside. So either you include this submission inside the task Future has been created for, or switch to CompletableFuture
ExecutorService exec = ...
Future<?> future = exec.submit(() -> {
var value = someCalculation();
SwingUtilities.invokeLater(() -> {
useValueOnEDT(value);
});
});
or
CompletableFuture<ValueType> cf = ...
cf.whenComplete((value, error) -> {
SwingUtilities.invokeLater(() -> {
if (error != null) {
handleErrorOnEdt(error);
} else {
useValueOnEDT(value);
}
});
});
Android Main Thread
The idea is the same as with Swing, but you'll have to use a Handler
// given value
new Handler(Looper.getMainLooper()).post(() -> {
useValueOnMainLooper(value);
});
You can wrap the Future into a CompletableFuture like so
static <T> CompletableFuture<T> from(Future<T> future) {
var delegate = new CompletableFuture<T>();
CompletableFuture.runAsync(() -> {
try {
delegate.complete(future.get());
} catch (Throwable e) {
delegate.completeExceptionally(e);
}
});
return delegate;
}
And then use that CompletableFuture to asynchronously handle the completion via its various then... and when... methods.
I am building a library that needs to some bluetooth operations on Android. I want to return a Future instance, so whoever is using my library can call .get() on the future returned and can handle ExecutionException, TimeoutException and InterruptedException themselves. However, I want to detect a timeout myself because I need to some cleanup logic like disconnecting from the device and so on. How can I achieve this?
You could implement a wrapper class around Future which delegates to a different one (the one returned by wherever you're getting your Future at the moment). Something like:
final class DelegatingFuture<T> implements Future<T> {
private final Future<T> delegate;
DelegatingFuture(final Future<T> delegate) {
this.delegate = Objects.requireNonNull(delegate);
}
// All other methods simply delegate to 'delegate'
#Override
public T get()
throws InterruptedException, ExecutionException {
try {
return this.delegate.get();
} catch (final Exception ex) {
// Handle cleanup...
throw ex;
}
}
// Something similar for get(long timeout, TimeUnit unit)
}
And then simply return new DelegatingFuture<>(currentFuture); wherever your handing these out.
The timeout is relevant to the caller of the get method with timeout and only to that caller. A timeout is nowhere meant to imply a cancellation. E.g., the following code is a legitimate usage of the Future API:
ExecutorService es = Executors.newSingleThreadExecutor();
Future<String> f = es.submit(() -> {
Thread.sleep(3000);
return "hello";
});
for(;;) try {
String s = f.get(500, TimeUnit.MILLISECONDS);
System.out.println("got "+s);
break;
}
catch(TimeoutException ex) {
// perhaps, do some other work
System.out.println("will wait something more");
}
catch (ExecutionException ex) {
System.out.println("failed with "+ex);
break;
}
es.shutdown();
Tying the cleanup to the methods actually intended to query the result, is not a useful approach. The timeout provided by the caller(s) of that method do not relate to the actual operation. There’s not even a guaranty that the result will be queried before the operations ends or that it gets queried at all.
The cleanup should happen when either, the operation finished or when the future gets cancelled explicitly. If the caller intends a cancellation after a timeout, the caller only needs to invoke cancel after catching a TimeoutException.
One approach, often pointed to, is to use a CompletionService, e.g.
static final ExecutorService MY__EXECUTOR = Executors.newCachedThreadPool();
static final CompletionService<String> COMPLETION_SERVICE
= new ExecutorCompletionService<>(MY__EXECUTOR);
static final Future<?> CLEANER = MY__EXECUTOR.submit(() -> {
for(;;) try {
Future<String> completed = COMPLETION_SERVICE.take();
System.out.println("cleanup "+completed);
} catch(InterruptedException ex) {
if(MY__EXECUTOR.isShutdown()) break;
}
});
public static Future<String> doSomeWork() {
return COMPLETION_SERVICE.submit(() -> {
Thread.sleep(3000);
return "hello";
});
}
You are in control over when to poll the completed futures, like in another background thread, as shown in the example, or right before commencing new jobs.
You can test it like
Future<String> f = doSomeWork();
try {
String s = f.get(500, TimeUnit.MILLISECONDS);
System.out.println("got "+s);
}
catch(TimeoutException ex) {
System.out.println("no result after 500ms");
}
catch (ExecutionException ex) {
System.out.println("failed with "+ex);
}
if(f.cancel(true)) System.out.println("canceled");
f = doSomeWork();
// never calling get() at all
But honestly, I never understood why such complicated things are actually necessary. If you want a cleanup at the right time, you can use
static final ExecutorService MY__EXECUTOR = Executors.newCachedThreadPool();
public static Future<String> doSomeWork() {
Callable<String> actualJob = () -> {
Thread.sleep(3000);
return "hello";
};
FutureTask<String> ft = new FutureTask<>(actualJob) {
#Override
protected void done() {
System.out.println("cleanup "+this);
}
};
MY__EXECUTOR.execute(ft);
return ft;
}
to achieve the same.
Or even simpler
static final ExecutorService MY__EXECUTOR = Executors.newCachedThreadPool();
public static Future<String> doSomeWork() {
Callable<String> actualJob = () -> {
Thread.sleep(3000);
return "hello";
};
return MY__EXECUTOR.submit(() -> {
try {
return actualJob.call();
}
finally {
// perform cleanup
System.out.println("cleanup");
}
});
}
In either case, the cleanup will be performed whether the job was completed successfully, failed, or got canceled. If cancel(true) was used and the actual job supports interruption, the cleanup also will be performed immediately after.
When I do async call using ExecutorService, it returns Future Object. Based on the boolean value it returns, I have to log the status of the async call.
But when I try to call the method get method from the future object, it blocks the main thread execution.
Is it possible to unblock the main thread execution?
public class FutureExample {
static HystrixCommand<Boolean> hystrixCommand;
public FutureExample(HystrixCommand<Boolean> hystrixCommand){
FutureExample.hystrixCommand = hystrixCommand;
}
public static void main(String[] args) throws InterruptedException, ExecutionException {
Boolean something = asyncCall();
if(something) {
System.out.println("Future task is done");
}
System.out.println("Don't wait for async call");
}
private static Boolean asyncCall() throws InterruptedException, ExecutionException {
Future<Boolean> response = hystrixCommand.queue(); // Aysnc Call to remote server
return response.get(); //this is blocking main thread
}
}
The good thing about futures is to be able to release threads until the answer arrives.
So I recommend you to use a Future implementation, like CompletableFuture:
final ExecutorService executorService = Executors.newFixedThreadPool(10);
CompletableFuture.supplyAsync(() -> {
try {
return hystrixCommand.queue();
} catch (Exception e) {
return false;
}
}, executorService);
This will work on another thread and when that future is over it will be done.
According to JavaDocs, get() method waits if necessary for the computation to complete, and then retrieves its result.
If you want to get the result once the task is completed, use isDone() function, which returns true if the task completed (normally, exceptionally, etc.). And invoke get() afterwards.
Also, you can use get(long timeout, TimeUnit unit) function to wait only for the given period of time. In this case the main thread will be "unblocked" automatically if either the time is out or the task has completed.
If you need to execute code in the main thread while the async task is running, you will need to redesign your asyncCall method to make it return a future instead.
An example:
private static Future<Boolean> asyncCall()
throws InterruptedException, ExecutionException {
return hystrixCommand.queue();
}
This way, the main method makes the call regarding when to block/wait:
public static void main(String[] args)
throws InterruptedException, ExecutionException {
Future<Boolean> something = asyncCall();
//do something while async call is running
//to check whether it's done running:
if(something.isDone()) {
System.out.println("Future task is done");
}
//when you're finally ready to wait:
System.out.println("Waiting for async call to finish");
Boolean result = something.get();
}
I have a method in which I create a some files using ThreadPoolExecuter, and later zip the files created.
private void createAndZip(){
// Some Code
ThreadPoolExecutor executer = (ThreadPoolExecutor) Executors.newFixedThreadPool(5);
for(String str : someStringList){
// This piece of code creates files and drops to certain location.
executer.execute(new MyRunnable());
}
executer.shutdown();
// Code to Zip the files created above.
}
Now my piece of code to create zip files runs even before all files are created, so not all files are zipped.
Please help. I tried Sleep, but can't gaurantee how much time the files creation will take.
You need to invoke awaitTermination on the executor object, in order to wait for the executor to finish shutting down.
I used a CountDownLatch to solve the problem. Here is sample code.
private void createAndZip() throws Exception{
CountDownLatch latch = new CountDownLatch(someStringList.size());
// Some Code
ThreadPoolExecutor executer = (ThreadPoolExecutor) Executors.newFixedThreadPool(5);
for(String str : someStringList){
// This piece of code creates files and drops to certain location.
executer.execute(new MyRunnable(latch));
}
executer.shutdown();
// Code to Zip the files created above.
try {
latch.await();
} catch (InterruptedException exception) {
throw new GIException(exception);
}
//Code here.
}
public class MyRunnable implements Runnable{
CountDownLatch latch = null;
MyRunnable(CountDownLatch latch){
this.latch = latch;
}
#Override
public void run() {
try {
// Some Logic
latch.countDown();
} catch (Exception e) {
e.printStackTrace();
}
}
}
I think you can use Future objects here. Instead of calling execute() on the executor use submit() method. This should give you a Future object for each task you are submitting to the executor. Once you submit all tasks just loop over the list of futures you got and call get() on each. This is a blocking call and it waits until the corresponding task finishes.
Here the advantage is that you can retrieve any exception thrown from your task and then decide whether to zip the files or not.
Please refer this code -
private void createAndZip() throws Exception {
// Some Code
ThreadPoolExecutor executer = (ThreadPoolExecutor) Executors.newFixedThreadPool(5);
// collect all futures
List<Future> futures = new ArrayList<>();
for(String str : someStringList){
// This piece of code creates files and drops to certain location.
futures.add(executer.submit(new MyRunnable()));
}
// wait for all tasks to finish
try {
for (Future future : futures) {
future.get();
}
} catch (Exception e) {
e.printStackTrace();
if (e instanceof ExecutionException) {
throw e;
}
} finally {
executer.shutdown();
}
// Code to Zip the files created above.
}
In your code block you are narrowing the return of Executors.newFixedThreadPool(5). One option you have is to use the ExecutorService it returns. This class already has facilities that avoid having to re-implement synchronization code such as latches. For example:
Using Futures
private void createAndZip(ExecutorService executor) throws ExecutionException, InterruptedException {
// Some Code
List<String> list = new ArrayList<>();
// For a number of reasons ExecutorService should be constructed outside
// ExecutorService executer = Executors.newFixedThreadPool(5);
List<Future<?>> futures = new ArrayList<>();
for(String str : list){
// This piece of code creates files and drops to certain location.
futures.add(executer.submit(new MyRunnable()));
}
// async work
for (Future<?> future : futures) {
future.get(); // blocks
}
// Code to Zip the files created above.
}
There are some advantages here:
Error management: when executing the the background if you use another technique you have to arrange for errors to be delivered from the background thread to your master thread. Here the future takes care of this. If your worker throws then the exception will back it back to your controlling thread.
Keeping few threadpools in your code. The reason to pool threads in the first place is to make the startup costs smaller. If you have any significant sized program you wouldn't want to create and destroy threadpools whenever you wanted to perform an operation in parallel.
With Java8 Lambda this the loops can be written in a more compact way.
Fork/Join
Perhaps better suited to your task, particularly if you are going to process a tree of files is the Fork/Join framework. Here you could roll the processing and the zipping into a collection of tasks that are submitted to the fork-join pool. That's neat because you can get a Future for the whole zip file allowing you to produce the entire zip off your main thread. Something similar to your design using fork/join might be:
static class PrepareFile extends RecursiveTask<Void> {
private String filePath;
PrepareFile(String filePath) {
this.filePath = filePath;
}
#Override
protected Void compute() {
try {
System.out.println(filePath);
Thread.sleep(1009L);
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
return null; // void
}
}
static class ZipTask extends RecursiveTask<String>
{
private List<String> files;
ZipTask(List<String> files) {
this.files = files;
}
#Override
protected String compute() {
List<PrepareFile> prepareTasks = new ArrayList<>();
for(String file : files) {
PrepareFile fileTask = new PrepareFile(file);
prepareTasks.add(fileTask);
fileTask.fork();
}
for(PrepareFile task : prepareTasks) {
task.join(); // can collect results here
}
System.out.println("Zipping");
try {
Thread.sleep(5000);
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("Done task");
return "filename.zip";
}
}
public static void main(String[] args) {
ForkJoinPool pool = new ForkJoinPool();
List<String> toProcess = Arrays.asList("a","b");
String filename = pool.invoke(new ZipTask(toProcess));
System.out.println("Zipped " + filename);
}
This is an illustration you'd want to change a few things, like the the return types of the tasks and how the tasks are invoked perhaps.
On awaitTermination
It is possible to use the awaitTermination method after calling shutdown to wait for all processes to terminate. However this may not be so desirable in longer running services or programs where thread-pools may be shared between operations.
private void createAndZip() throws Exception{
// Some Code
ThreadPoolExecutor executer = (ThreadPoolExecutor) Executors.newFixedThreadPool(5);
for(String str : someStringList){
// This piece of code creates files and drops to certain location.
executer.execute(new MyRunnable());
}
executer.shutdown();
while (true) {
boolean result_ = threadPoolExecutor.awaitTermination(TimeUnit.DAYS, 1);
if(result_)
break;
}
// Code to Zip the files created above.
//Code here.
}