Given this code,
Collection<?> callables = ...
ExecutorService executorService = ...
List<Future<?>> futures = executorService.invokeAll(callables, TIMEOUT, TimeUnit.SECONDS);
I'd like to know which of the callables were cancelled (did not finish in time). I know I can query each Future with .isCancelled() but that gives me no information as to which exact Callable was cancelled.
One solution would be for Future to implement a toString() method that delegates to the underlying toString() of Callable. Unfortunately, this is not done.
I know I can query each Future with .isCancelled() but that gives me no information as to which exact Callable was cancelled.
The list of Futures is in the same order as the list of Callables it was created from. So you can keep track that way.
Related
I have to start unknown number of threads then wait or all threads to finish their job. I am using executor service. I tried using countdownlatch - so that I may wait till countdown is zero.
But there is no way I can get number of threads I have started. Can someone give me any idea how may I achieve this ?
Thank you for your responses. I came across the answer, and it helped. Sharing a link for reference.
Flexible CountDownLatch?
In case that you want combine a List of CompletableFutures, you can do this :
// Waits for *all* futures to complete and returns a list of results.
// If *any* future completes exceptionally then the resulting future will also complete exceptionally.
public static <T> CompletableFuture<List<T>> all(List<CompletableFuture<T>> futures) {
CompletableFuture[] cfs = futures.toArray(new CompletableFuture[futures.size()]);
return CompletableFuture.allOf(cfs)
.thenApply(ignored -> futures.stream()
.map(CompletableFuture::join)
.collect(Collectors.toList())
);
}
For more details on Future & CompletableFuture, useful links:
Future: https://www.baeldung.com/java-future
CompletableFuture: https://www.baeldung.com/java-completablefuture
CompletableFuture: https://www.callicoder.com/java-8-completablefuture-tutorial/
Waiting on a list of Future
invokeAll() doesn't return until all the Callables in the submitted Collection have completed, so what is the reason for making the results Futures?
Because the task might terminate normally or exceptionally, Futures can wrap the exception for you. For example,
Callable<Integer> c1 = () -> 1;
Callable<Integer> c2 = () -> {
throw new RuntimeException();
};
List<Future<Integer>> futures = executor.invokeAll(Arrays.asList(c1,c2));
for (Future<Integer> future : futures) {
System.out.println(future.get());
}
Note that because of Future, we were able to get a result of the future that terminated normally and the that terminated exceptionally.
If invokeAll returned a List<T>, it would have to return those that completed successfully and discarded those with exceptions.
Checking the ExecutorService#invokeAll Javadoc
Executes the given tasks, returning a list of Futures holding their
status and results when all complete. Future.isDone() is true for each
element of the returned list. Note that a completed task could have
terminated either normally or by throwing an exception. The results
of this method are undefined if the given collection is modified while
this operation is in progress.
Means that we're getting a list of Future objects because there may be situations where we do not get a value even if the task is complete.
I have One Callable which I invoked using
FutureTask<Integer> task = new FutureTask<Integer>(new MyCallable(name, type));
pool = Executors.newSingleThreadExecutor();
pool.submit(task);
I want to know Is execution is continue after pool.submit(task) or It will wait for callable to complete its execution?
In short I just want to know is there any method like thread.join() for Callable?
... is there any method like thread.join() for Callable?
The pool.submit(callable) method returns a Future and will start executing immediately if the threads are available in the pool. To do a join, you can call future.get() which joins with the thread, returning the value returned by the call() method. It is important to note that get() may throw an ExecutionException if the call() method threw.
You do not need to wrap your Callable in a FutureTask. The thread-pool does that for you. So your code would be:
pool = Executors.newSingleThreadExecutor();
Future<String> future = pool.submit(new MyCallable(name, type));
// now you can do something in the foreground as your callable runs in the back
// when you are ready to get the background task's result you call get()
// get() waits for the callable to return with the value from call
// it also may throw an exception if the call() method threw
String value = future.get();
This is if your MyCallable implements Callable<String> of course. The Future<?> will match whatever type your Callable is.
task.get() (task being a FutureTask) expects the current thread to wait for the completion of the managed task by the thread pooler.
This method ends up returning either a concrete result or throwing the same checked exception (although wrapped into an ExecutionException) that the job thread would throw during its task.
I'n new to ExecutorService, but am unsure about my approach to this.
I could be dealing with up to 100 threads for a known task.
I'm using the general format below, where I create a List of FutureTasks, then submit these to the ExecutorService.
The ExecutorService returns and adds these pending results to another list.
I then iterate over this list, calling get() on each pending result.
My query is : won't this block on each get() in turn until all 100 threads have completed ? Is there a better way to do this ?
And am I right in assuming that get() returns the result of the Callable implementation's call() method ? I'm using the default FutureTask class, and haven't subclassed it.
ExecutorService exec = Executors.newFixedThreadPool( NUM_THREADS );
List<JobClass> originalList = new ArrayList<JobClass>();
List<SomeOtherClass> pendingResult = new ArrayList<SomeOtherClass>();
List<Future<SomeOtherClass>> resultList = new ArrayList<Future<SomeOtherClass>>();
for( JobClass sc : originalList )
pendingResult.add( submit( sc );
for( Future<SomeOtherClass> future : futures )
resultList.add( future.get(5, TimeUnit.SECONDS) );
Good question, if I understand you correctly, you are worried about consumption of the result of completed tasks. Yes the thread will block. Java's answer to that is using the CompletionService.
As mentioned in the documentation page "A service that decouples the production of new asynchronous tasks from the consumption of the results of completed tasks".
If you only proceed if all tasks are completed, you can do what you suggested. It will not matter in which order tasks complete. However, if you need to pass task results to other processor as soon as possible or do something else while tasks are executing, you may want to check if task is complete first using isDone method, for example, and call get() if it is complete.
This question already has answers here:
How to wait for all threads to finish, using ExecutorService?
(27 answers)
Closed 5 years ago.
I need to submit a number of task and then wait for them until all results are available. Each of them adds a String to a Vector(that is synchronized by default). Then I need to start a new task for each result in the Vector but I need to do this only when all the previous tasks have stopped doing their job.
I want to use Java Executor, in particular I tried using Executors.newFixedThreadPool(100) in order to use a fixed number of thread (I have a variable number of task that can be 10 or 500) but I'm new with executors and I don't know how to wait for task termination.
This is something like a pseudocode of what my program needs to do:
ExecutorService e = Executors.newFixedThreadPool(100);
while(true){
/*do something*/
for(...){
<start task>
}
<wait for all task termination>
for each String in result{
<start task>
}
<wait for all task termination>
}
I can't do a e.shutdown because I'm in a while(true) and I need to reuse the executorService...
Can you help me? Can you suggest me a guide/book about java executors?
The ExecutorService gives you a mechanism to execute multiple tasks simultaneously and get a collection of Future objects back (representing the asynchronous computation of the task).
Collection<Callable<?>> tasks = new LinkedList<Callable<?>>();
//populate tasks
for (Future<?> f : executorService.invokeAll(tasks)) { //invokeAll() blocks until ALL tasks submitted to executor complete
f.get();
}
If you have Runnables instead of Callables, you can easily turn a Runnable into a Callable<Object> using the method:
Callable<?> c = Executors.callable(runnable);
Can you suggest me a guide/book about
java executors??
I can answer this part:
Java Concurrency in Practice by Brian Goetz (with Tim Peierls, Joshua Bloch, Joseph Bowbeer, David Holmes and Doug Lea) is most likely your best bet.
It's not only about executors though, but instead covers java.util.concurrent package in general, as well as basic concurrency concepts and techniques, and some advanced topics such as the Java memory model.
Rather than submitting Runnables or Callables to an Executor directly and storing the corresponding Future return values I'd recommend using a CompletionService implementation to retrieve each Future when it completes. This approach decouples the production of tasks from the consumption of completed tasks, allowing for example new tasks to originate on a producer thread over a period of time.
Collection<Callable<Result>> workItems = ...
ExecutorService executor = Executors.newSingleThreadExecutor();
CompletionService<Result> compService = new ExecutorCompletionService<Result>(executor);
// Add work items to Executor.
for (Callable<Result> workItem : workItems) {
compService.submit(workItem);
}
// Consume results as they complete (this would typically occur on a different thread).
for (int i=0; i<workItems.size(); ++i) {
Future<Result> fut = compService.take(); // Will block until a result is available.
Result result = fut.get(); // Extract result; this will not block.
}
When you submit to an executor service, you'll get a Future object back.
Store those objects in a collection, and then call get() on each in turn. get() blocks until the underlying job completes, and so the result is that calling get() on each will complete once all underlying jobs have finished.
e.g.
Collection<Future> futures = ...
for (Future f : futures) {
Object result = f.get();
// maybe do something with the result. This could be a
// genericised Future<T>
}
System.out.println("Tasks completed");
Once all these have completed, then begin your second submission. Note that this might not be an optimal use of your thread pool, since it will become dormant, and then you're re-populating it. If possible try and keep it busy doing stuff.
ExecutorService executor = ...
//submit tasks
executor.shutdown(); // previously submitted tasks are executed,
// but no new tasks will be accepted
while(!executor.awaitTermination(1, TimeUnit.SECONDS))
;
There's no easy way to do what you want without creating custom ExecutorService.