Parallel execution of callables - java

I'd like to execute multiple callables parallel. But it seems that the ExecutorService always waits until all callables are finnished.
I've tried the following:
final int nThreads = 10;
ExecutorService executorService = Executors.newFixedThreadPool(nThreads);
List<PrimeCallable> tasks = new ArrayList<PrimeCallable>();
for(int i = 0; i < nThreads; i++) {
tasks.add(new PrimeCallable(0, i * 100 + 100, "thread" + i));
}
try {
for(Future<List<Integer>> result : executorService.invokeAll(tasks)) {
List<Integer> integers = result.get();
for(Integer i : integers){
System.out.println(i);
}
}
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (ExecutionException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
Now, the for loop is called when all callables in the executorService are finnished. As far as I know, there is no executorService.isParallel setter ;-).
What would be the right approach to let callables run parallel?
Thanks for your hints!

The javadocs for invokeAll says;
Executes the given tasks, returning a
list of Futures holding their status
and results when all complete. Future.isDone() is true for each element of the returned list.
So invokeAll blocks until each task in the collection is complete.

Executor service runs all your callables in parallel. All it does is , it waits for all parallel tasks to complete before it moves on. So its not like where all the tasks are run in serial.

It sounds like part of what you want is lazy execution - you don't want to have to make a copy of the structure in memory before extracting results.
I would treat this as an iteration + transformation problem. First, define an iterator over your input, such that each call to next() returns a Callable that will produce the next value in your series.
The transform stage is to apply a parallel or concurrent evaluation of those Callables, something like this (not tested):
public class ConcurrentTransform
{
private final ExecutorService executor;
private final int maxBuffer;
public ConcurrentTransform(ExecutorService executor, int maxWorkBuffer) {
this.executor = executor;
this.maxBuffer = Math.max(1, maxWorkBuffer);
}
public <T> Iterator<T> apply(final Iterator<Callable<T>> input) {
// track submitted work
final BlockingQueue<Future<T>> submitted = new LinkedBlockingQueue<Future<T>>();
// submit first N tasks
for (int i=0; i<maxBuffer && input.hasNext(); i++) {
Callable<T> task = input.next();
Future<T> future = executor.submit(task);
submitted.add(future);
}
return new Iterator<T>(){
#Override
public synchronized boolean hasNext() {
return !submitted.isEmpty();
}
#Override
public T next() {
Future<T> result;
synchronized (this) {
result = submitted.poll();
if (input.hasNext()) {
submitted.add(executor.submit(input.next()));
}
}
if (result != null) {
try {
return result.get(); // blocking
} catch (Exception e) {
if (e instanceof RuntimeException) {
throw (RuntimeException) e;
} else {
throw new RuntimeException(e);
}
}
} else {
throw new NoSuchElementException();
}
}
#Override
public void remove() {
throw new UnsupportedOperationException();
}};
}
}
After calling apply(...), you'd iterate over the resulting values, which under the covers would be executing the Callable objects in parallel and returning results in the same order as they were input. Some refinements would be to allow an optional timeout for the blocking result.get() call, or to manage the thread pool within the transform itself.

If you want to view results as they happen, use the ExecutorCompletionService.

Related

Multi threading issue --> retrieving thread result after his completion

Here is my code :
List<Object> array= new ArrayList<Object>();
int i=0;
ExecutorService pool = Executors.newFixedThreadPool(50);
for(String str : strList) {
LittleDwarfWorker littleDwarfWorker = new LittleDwarfWorker(params including a datasource);
try {
pool.execute(littleDwarfWorker);
}catch(Exception e) {
e.printStackTrace();
}
finally{
i++;
array.add(littleDwarfWorker.getResult());
if((i%100)==0) {
log.info("Progression :"+i+"/"+listeEan.size());
}
}
}
pool.shutdown();
Here my beloved dwarf :
public void run() {
JdbcTemplate localJdbcTemplate = new JdbcTemplate(this.dataSource);
//dwarf dig in database to find some diamonds
}
My issue is when I run, arrayis empty. I guess my code is bad-formatted but I'm not comfortable enought with multi threading to find my error. I suppose the array.add() instruction is executed before my thread finishes his work, so value is empty.
What I'm looking for :
each thread get his own worker, when worker has result it add the result to my array.
For me finally would be executed AFTER my thread retrieve info from db.
I looked at submit method here Returning value from Thread but i'm not sure about how retrieve "future" value. Because if my run method isn't void I get an error.
The ExecutorService in java does not work this way. I guess that you LittleDwarfWorker implmenets Runnable and that the getResult() is your creation. To make is the java way you your worker needs to implements Callable<Object> which allows you to directly get the result after the task has finished. You also need a CompletionService. So you first submit all tasks and afterwards collected their result. The .take() returns a Future<V> which hold you result, so it will block until it is ready.
ExecutorService executor = Executors.newFixedThreadPool(50);
CompletionService<Obejct> completionService = new ExecutorCompletionService<> (executor);
for(String str : strList) {
completionService.submit(new LittleDwarfWorker(...));
}
for ( int i = 0; i < strList.size(); i++ ) {
try {
Object result = completionService.take().get();
// ... do whatever something with the object
} catch ( InterruptedException | ExecutionException e ) {
e.printStackTrace();
}
}
executor.shutdown();

Java ExecutorService - Monitor Task Completion/Status bar

So I have an ExecutorService successfully blocking and running linearly right now. My trouble is, I am trying to add a status update and I can't figure out how to get Futures to settle one-item at a time. It seems that by the time the first item in my Future<> is ready so is the last. I'm hoping to find a place where I can know how many tasks my executorService has remaining/total so I can calculate a simple percentage indicator. Please note I intend on recycling my Executor and don't want to shut it down.
ExecutorService updateService = Executors.newSingleThreadExecutor();
Callable<String> callHour = () -> {
//doStuff, unaware of total number of hourCalls
return "done";
};
private void startMe(int hours){
List<Future<String>> futureHours;
List<Callable<String>> hourCalls = new ArrayList<>(hours);
for (int hour = 0; hour < hours; ++hour) {
hourCalls.add(callHour); //queue list (not running yet)
}
try {
//executes queue and blocks thread
futureHours = updateService.invokeAll(hourCalls);
futureHours.get(0).get();//performs blocking
} catch (Exception e) {
e.printStackTrace();
}
}
}
There are two things at work here.
Firstly, if we take a look at the documentation of ExecutorService#invokeAll(...), we see that it returns
[...] a list of Futures holding their status and results when all complete. [...]
(emphasis added by me)
You most probably want to use Executor#submit(...) instead.
Secondly, you have no guarantee that the task coupled to futureHours.get(0) is executed first. I would suggest using Future#isDone() with some additional logic:
private void startMe(int hours) {
[...]
try {
[...]
ArrayList<Future<String>> futureHoursDone = new ArrayList<>();
final int numTasks = futureHours.size();
int done = 0;
double percentageDone = 0.0d;
while (futureHours.isEmpty() == false) {
for (int index = 0; index < futureHours.size(); ++index) {
Future<String> futureHour = futureHours.get(index);
if (futureHour.isDone()) {
futureHours.remove(index);
futureHoursDone.add(futureHour);
--index;
++done;
percentageDone = done / (double) numTasks;
}
}
}
} catch (Exception e) {
// TODO: don't forget to HCF (https://en.wikipedia.org/wiki/Halt_and_Catch_Fire) :)
e.printStackTrace();
}
}
(This is a rough sketch. To make the progress, i.e. percentage, visible to the outside, you would have to make it an attribute and accessible through, e.g., some getter)

Java8 - idiomatic way to process a Stream<Callable<...>> in parallel delivering to a non-thread-safe consumer?

Suppose I have a Stream<Callable<SomeClass>> stream;. The stream is accessing over a million objects which will not fit in memory.
What is the idiomatic way to convert this to a Stream<SomeClass> in a manner that ensures the Callable::call are executed in parallel before being delivered to a consumer that is non-threaded-safe (perhaps by calling .sequential().forEach() or some other bottlenecking mechanism)?
i.e. Process the stream in parallel but deliver the output sequentially (random order ok, as long as it's single-threaded).
I know I could do what I want by setting up an ExecutionService and a Queue between the original stream and the consumer. But that seems like a lot of code, is there a magic one-liner?
You could still employ an ExecutorService for parallelization. Like this:
ExecutorService service = Executors.newFixedThreadPool(4);
stream.map(c -> service.submit(c)).map(future -> {
try {
return future.get(); //retrieve callable result
} catch (InterruptedException | ExecutionException ex) {
//Exception handling
throw new RuntimeException(ex);
}
});
You can process the resulting Stream<SomeClass> further sequentially.
If you use forEach/forEachOrdered directly on the Stream<Future<SomeClass>> you can process a resulting SomeClass-object directly once the current future is done (different from when you use invokeAll() which blocks until every task is done).
If you want to process the results of the callables in the exact order they are available you will have to use CompletionService which can't be used along with a single chain of stream operations due to the necessary call of Future<SomeClass> f = completionService.take() after submitting the callables.
EDIT:
Using an ExecutorService within streams doesn't work the way I showed above, because every Callable is submitted and requested via future.get() one after the other.
I found a possible even side-effect heavier solution dividing the Callables in fixed parallelized chunks.
I use a class TaskMapper as mapping-function for submitting the Callables and mapping them to chunks:
class TaskMapper implements Function<Callable<Integer>, List<Future<Integer>>>{
private final ExecutorService service;
private final int chunkSize;
private List<Future<Integer>> chunk = new ArrayList<>();
TaskMapper(ExecutorService service, int chunkSize){
this.service = service;
this.chunkSize = chunkSize;
}
#Override
public List<Future<Integer>> apply(Callable<Integer> c) {
chunk.add(service.submit(c));
if(chunk.size() == chunkSize){
List<Future<Integer>> fList = chunk;
chunk = new ArrayList<>();
return fList;
}else{
return null;
}
}
List<Future<Integer>> getChunk(){
return chunk;
}
}
This how the chain of stream-operations looks like:
ExecutorService service = Executors.newFixedThreadPool(4);
TaskMapper taskMapper = new TaskMapper(service, 4);
stream.map(taskMapper)
.filter(fl -> fl != null) //filter for the chunks
.flatMap(fl -> fl.stream()) //flat-map the chunks to futures
.map(future -> {
try {
return future.get();
} catch (InterruptedException | ExecutionException ex) {
throw new RuntimeException(ex);
}
});
//process the remaining futures
for(Future<Integer> f : taskMapper.getChunk()){
try {
Integer i = f.get();
//process i
} catch (InterruptedException | ExecutionException ex) {
//exception handling
}
}
This works as follows: The TaskMapper takes 4 callables each time submits them to the service and maps them to a chunk of futures (without Spliterator). This is solved by mapping to null for the 1st, 2nd and 3rd callable each time. null could be replaced by a dummy object for example. The mapping function that maps the futures to the results waits for the result of each future of the chunk. I use Integer in my example instead of SomeClass. When all results of the futures in the current chunk are mapped, a new chunk will be created and parallelized. Finally, if the number of elements in the stream is not dividable by the chunkSize(4 in my example), the remaining futures will have to be retrieved from the TaskMapper and processed outside of the stream.
This construct works for the tests I carried out, but I am aware that it is possible fragile due to the side-effects, statefullness and the undefined evaluation behavior of the stream.
EDIT2:
I made a version of the construct from the previous EDIT using a custom Spliterator:
public class ExecutorServiceSpliterator<T> extends AbstractSpliterator<Future<T>>{
private final Spliterator<? extends Callable<T>> srcSpliterator;
private final ExecutorService service;
private final int chunkSize;
private final Queue<Future<T>> futures = new LinkedList<>();
private ExecutorServiceSpliterator(Spliterator<? extends Callable<T>> srcSpliterator) {
this(srcSpliterator, Executors.newFixedThreadPool(8), 30); //default
}
private ExecutorServiceSpliterator(Spliterator<? extends Callable<T>> srcSpliterator, ExecutorService service, int chunkSize) {
super(Long.MAX_VALUE, srcSpliterator.characteristics() & ~SIZED & ~CONCURRENT);
this.srcSpliterator = srcSpliterator;
this.service = service;
this.chunkSize = chunkSize;
}
public static <T> Stream<T> pipeParallelized(Stream<? extends Callable<T>> srcStream){
return getStream(new ExecutorServiceSpliterator<>(srcStream.spliterator()));
}
public static <T> Stream<T> pipeParallelized(Stream<? extends Callable<T>> srcStream, ExecutorService service, int chunkSize){
return getStream(new ExecutorServiceSpliterator<>(srcStream.spliterator(), service, chunkSize));
}
private static <T> Stream<T> getStream(ExecutorServiceSpliterator<T> serviceSpliterator){
return StreamSupport.stream(serviceSpliterator, false)
.map(future -> {
try {
return future.get();
} catch (InterruptedException | ExecutionException ex) {
throw new RuntimeException(ex);
}
}
);
}
#Override
public boolean tryAdvance(Consumer<? super Future<T>> action) {
boolean didAdvance = true;
while((didAdvance = srcSpliterator.tryAdvance(c -> futures.add(service.submit(c))))
&& futures.size() < chunkSize);
if(!didAdvance){
service.shutdown();
}
if(!futures.isEmpty()){
Future<T> future = futures.remove();
action.accept(future);
return true;
}
return false;
}
}
This class provides functions (pipeParallelized()) which take a stream of Callable-elements execute them chunk-wise in parallel and then ouput a sequential stream containing the results. Spliterators are allowed to be stateful. Therefore this version should hopefully not violate any stream operation constraints. This is how the Splitterator can be used (close to a "magic oneliner"):
ExecutorServiceSpliterator.pipeParallelized(stream);
This line takes the stream of Callables stream parallelizes the execution of them and returns a sequential stream containing the results (piping happens lazily -> should work with millions of callables) which can be processed further with regular stream operations.
The implementation of ExecutorServiceSpliteratoris very basic. It should mainly demonstrate how it could be done in principle. The resupplying of the service and the retrieving of the results could be optimized. For example if the resulting stream is allowed to be unordered, a CompletionService could be used.
You are asking for an idiomatic solution. Streams with sideeffects in its behavioral parameters are discouraged (explicitly stated in the javadoc of Stream).
So the idiomatic solution is basically ExecutorService + Futures and some loops/forEach(). If you have a Stream as parameter, just transform it to a List with the standard Collector.
Something like that:
ExecutorService service = Executors.newFixedThreadPool(5);
service.invokeAll(callables).forEach( doSomething );
// or just
return service.invokeAll(callables);
First Example:
ExecutorService executor = Executors.newWorkStealingPool();
List<Callable<String>> callables = Arrays.asList(
() -> "job1",
() -> "job2",
() -> "job3");
executor.invokeAll(callables).stream().map(future -> {
return future.get();
}).forEach(System.out::println);
Second Example:
Stream.of("1", "2", "3", "4", "", "5")
.filter(s->s.length() > 0)
.parallel()
.forEachOrdered(System.out::println);
public static void main(String[] args) {
testInfititeCallableStream();
}
private static void testInfititeCallableStream() {
ExecutorService service = Executors.newFixedThreadPool(100);
Consumer<Future<String>> consumeResult = (Future<String> future)->{
try {
System.out.println(future.get());
} catch (InterruptedException | ExecutionException e) {
e.printStackTrace();
}
};
getCallableStream().parallel().map(callable -> service.submit(callable)).forEach(consumeResult);
}
private static Stream<Callable<String>> getCallableStream() {
Random randomWait = new Random();
return Stream.<Callable<String>>generate(() ->
new Callable<String>() {
public String call() throws Exception {
//wait for testing
long time = System.currentTimeMillis();
TimeUnit.MILLISECONDS.sleep(randomWait.nextInt(5000));
return time + ":" +UUID.randomUUID().toString();
};
}).limit(Integer.MAX_VALUE);
}
None of the other answers worked for me.
I finally settled on something like this (pseudo-code):
ExecutorService executor = Executors.newWorkStealingPool();
CompletionService completor = new CompletionService(executor);
int count = stream.map(completor::submit).count();
while(count-- > 0) {
SomeClass obj = completor.take();
consume(obj);
}
The consume(obj) loop is executed sequentially in a single thread while the individual callable tasks asynchronously work their way through the CompletionService's multiple threads. Memory consumption is limited as the CompletionService will have only as many items in progress at a time as there are threads available. The Callables waiting for execution are eagerly materialized from the stream, but the impact of that is negligible compared to the memory each consumes once it starts executing (your use-case may vary).

Java Producer Consumer ArrayBlockingQueue deadlock on take()

In my app there are 2 phases, one download some big data, and the other manipulates it.
so i created 2 classes which implements runnable: ImageDownloader and ImageManipulator, and they share a downloadedBlockingQueue:
public class ImageDownloader implements Runnable {
private ArrayBlockingQueue<ImageBean> downloadedImagesBlockingQueue;
private ArrayBlockingQueue<String> imgUrlsBlockingQueue;
public ImageDownloader(ArrayBlockingQueue<String> imgUrlsBlockingQueue, ArrayBlockingQueue<ImageBean> downloadedImagesBlockingQueue) {
this.downloadedImagesBlockingQueue = downloadedImagesBlockingQueue;
this.imgUrlsBlockingQueue = imgUrlsBlockingQueue;
}
#Override
public void run() {
while (!this.imgUrlsBlockingQueue.isEmpty()) {
try {
String imgUrl = this.imgUrlsBlockingQueue.take();
ImageBean imageBean = doYourThing(imgUrl);
this.downloadedImagesBlockingQueue.add(imageBean);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
public class ImageManipulator implements Runnable {
private ArrayBlockingQueue<ImageBean> downloadedImagesBlockingQueue;
private AtomicInteger capacity;
public ImageManipulator(ArrayBlockingQueue<ImageBean> downloadedImagesBlockingQueue,
AtomicInteger capacity) {
this.downloadedImagesBlockingQueue = downloadedImagesBlockingQueue;
this.capacity = capacity;
}
#Override
public void run() {
while (capacity.get() > 0) {
try {
ImageBean imageBean = downloadedImagesBlockingQueue.take(); // <- HERE I GET THE DEADLOCK
capacity.decrementAndGet();
} catch (InterruptedException e) {
e.printStackTrace();
}
// ....
}
}
}
public class Main {
public static void main(String[] args) {
String[] imageUrls = new String[]{"url1", "url2"};
int capacity = imageUrls.length;
ArrayBlockingQueue<String> imgUrlsBlockingQueue = initImgUrlsBlockingQueue(imageUrls, capacity);
ArrayBlockingQueue<ImageBean> downloadedImagesBlockingQueue = new ArrayBlockingQueue<>(capacity);
ExecutorService downloaderExecutor = Executors.newFixedThreadPool(3);
for (int i = 0; i < 3; i++) {
Runnable worker = new ImageDownloader(imgUrlsBlockingQueue, downloadedImagesBlockingQueue);
downloaderExecutor.execute(worker);
}
downloaderExecutor.shutdown();
ExecutorService manipulatorExecutor = Executors.newFixedThreadPool(3);
AtomicInteger manipulatorCapacity = new AtomicInteger(capacity);
for (int i = 0; i < 3; i++) {
Runnable worker = new ImageManipulator(downloadedImagesBlockingQueue, manipulatorCapacity);
manipulatorExecutor.execute(worker);
}
manipulatorExecutor.shutdown();
while (!downloaderExecutor.isTerminated() && !manipulatorExecutor.isTerminated()) {
}
}
}
The deadlock happens because this scenario:
t1 checks capacity its 1.
t2 checks its 1.
t3 checks its 1.
t2 takes, sets capacity to 0, continue with flow and eventually exits.
t1 and t3 now on deadlock, cause there will be no adding to the downloadedImagesBlockingQueue.
Eventually i want something like that: when the capacity is reached && the queue is empty = break the "while" loop, and terminate gracefully.
to set "is queue empty" as only condition won't work, cause in the start it is empty, until some ImageDownloader puts a imageBean into the queue.
There area a couple of things you can do to prevent deadlock:
Use a LinkedBlockingQueue which has a capacity
Use offer to add to the queue which does not block
Use drainTo or poll to take items from the queue which are not blocking
There are also some tips you might want to consider:
Use a ThreadPool:
final ExecutorService executorService = Executors.newFixedThreadPool(4);
If you use a fixed size ThreadPool you can add "poison pill"s when you finished adding data to the queue corresponding to the size of your ThreadPool and check it when you poll
Using a ThreadPool is as simple as this:
final ExecutorService executorService = Executors.newFixedThreadPool(4);
final Future<?> result = executorService.submit(new Runnable() {
#Override
public void run() {
}
});
There is also the less known ExecutorCompletionService which abstracts this whole process. More info here.
You don't need the capacity in your consumer. It's now read and updated in multiple threads, which cause the synchronization issue.
initImgUrlsBlockingQueue creates the url blocking queue with capacity number of URL items. (Right?)
ImageDownloader consumes the imgUrlsBlockingQueue and produce images, it terminates when all the URLs are downloaded, or, if capacity means number of images that should be downloaded because there may be some failure, it terminates when it added capacity number of images.
Before ImageDownloader terminates, it add a marker in to the downloadedImagesBlockingQueue, for example, a null element, a static final ImageBean static final ImageBean marker = new ImageBean().
All ImageManipulator drains the queue use the following construct, and when it sees the null element, it add it to the queue again and terminate.
// use identity comparison
while ((imageBean = downloadedImagesBlockingQueue.take()) != marker) {
// process image
}
downloadedImagesBlockingQueue.add(marker);
Note that the BlockingQueue promises its method call it atomic, however, if you check it's capacity first, and consume an element according to the capacity, the action group won't be atomic.
Well i used some of the features suggested, but this is the complete solution for me, the one which does not busy waiting and wait until the Downloader notify it.
public ImageManipulator(LinkedBlockingQueue<ImageBean> downloadedImagesBlockingQueue,
LinkedBlockingQueue<ImageBean> manipulatedImagesBlockingQueue,
AtomicInteger capacity,
ManipulatedData manipulatedData,
ReentrantLock downloaderReentrantLock,
ReentrantLock manipulatorReentrantLock,
Condition downloaderNotFull,
Condition manipulatorNotFull) {
this.downloadedImagesBlockingQueue = downloadedImagesBlockingQueue;
this.manipulatedImagesBlockingQueue = manipulatedImagesBlockingQueue;
this.capacity = capacity;
this.downloaderReentrantLock = downloaderReentrantLock;
this.manipulatorReentrantLock = manipulatorReentrantLock;
this.downloaderNotFull = downloaderNotFull;
this.manipulatorNotFull = manipulatorNotFull;
this.manipulatedData = manipulatedData;
}
#Override
public void run() {
while (capacity.get() > 0) {
downloaderReentrantLock.lock();
if (capacity.get() > 0) { //checks if the value is updated.
ImageBean imageBean = downloadedImagesBlockingQueue.poll();
if (imageBean != null) { // will be null if no downloader finished is work (successfully downloaded or not)
capacity.decrementAndGet();
if (capacity.get() == 0) { //signal all the manipulators to wake up and stop waiting for downloaded images.
downloaderNotFull.signalAll();
}
downloaderReentrantLock.unlock();
if (imageBean.getOriginalImage() != null) { // the downloader will set it null iff it failes to download it.
// business logic
}
manipulatedImagesBlockingQueue.add(imageBean);
signalAllPersisters(); // signal the persisters (which has the same lock/unlock as this manipulator.
} else {
try {
downloaderNotFull.await(); //manipulator will wait for downloaded image - downloader will signalAllManipulators (same as signalAllPersisters() here) when an imageBean will be inserted to queue.
downloaderReentrantLock.unlock();
} catch (InterruptedException e) {
logger.log(Level.ERROR, e.getMessage(), e);
}
}
}
}
logger.log(Level.INFO, "Manipulator: " + Thread.currentThread().getId() + " Ended Gracefully");
}
private void signalAllPersisters() {
manipulatorReentrantLock.lock();
manipulatorNotFull.signalAll();
manipulatorReentrantLock.unlock();
}
For full flow you can check this project on my github: https://github.com/roy-key/image-service/
Your issue is that you are trying to use a counter to track queue elements and aren't composing operations that need to be atomic. You are doing check, take, decrement. This allows the queue size and counter to desynchronize and your threads block forever. It would be better to write a synchronization primitive that is 'closeable' so that you don't have to keep an associated counter. However, a quick fix would be to change it so you are get and decrementing the counter atomically:
while (capacity.getAndDecrement() > 0) {
try {
ImageBean imageBean = downloadedImagesBlockingQueue.take();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
In this case if there are 3 threads and only one element left in the queue then only one thread will atomically decrement the counter and see that it can take without blocking. Both other threads will see 0 or <0 and break out of the loop.
You also need to make all of your class instance variables final so that they have the correct memory visibility. You should also determine how you are going to handle interrupts rather than relying on the default print trace template.

is invokeAll() a blocking call in java 7

ExecutorService executorService = Executors.newSingleThreadExecutor();
Set<Callable<String>> callables = new HashSet<Callable<String>>();
callables.add(new Callable<String>() {
public String call() throws Exception {
return "Task 1";
}
});
callables.add(new Callable<String>() {
public String call() throws Exception {
return "Task 2";
}
});
callables.add(new Callable<String>() {
public String call() throws Exception {
return "Task 3";
}
});
List<Future<String>> futures = executorService.invokeAll(callables);
for(Future<String> future : futures){
System.out.println("future.get = " + future.get());
}
For this code piece. My question is "is invokeAll() a blocking call "?
I mean, when code ran to invokeAll() line, are we bloking there to wait for all result been generated?
Executes the given tasks, returning a list of Futures holding their
status and results when all complete. Future.isDone() is true for each
element of the returned list. Note that a completed task could have
terminated either normally or by throwing an exception. The results of
this method are undefined if the given collection is modified while
this operation is in progress.
Futures can only be done when execution is finished, therefore this method can only return when the tasks have been executed.
That it can throw an InterruptedException is also indicative of a blocking action.
Looking at the implementation of invokeAll in java.util.concurrent.AbstractExecutorService (comment inline):
// from OpenJDK source; GPL-2.0-with-classpath-exception
public <T> List<Future<T>> invokeAll(Collection<? extends Callable<T>> tasks)
throws InterruptedException {
if (tasks == null)
throw new NullPointerException();
ArrayList<Future<T>> futures = new ArrayList<Future<T>>(tasks.size());
boolean done = false;
try {
for (Callable<T> t : tasks) {
RunnableFuture<T> f = newTaskFor(t);
futures.add(f);
execute(f);
}
for (int i = 0, size = futures.size(); i < size; i++) {
Future<T> f = futures.get(i);
if (!f.isDone()) {
try {
f.get(); // <== *** BLOCKS HERE ***
} catch (CancellationException ignore) {
} catch (ExecutionException ignore) {
}
}
}
done = true;
return futures;
} finally {
if (!done)
for (int i = 0, size = futures.size(); i < size; i++)
futures.get(i).cancel(true);
}
}
In fact, looking at a reference implementation is what you generally should do in these cases when the Javadoc-Specese appears to be difficult to decipher. (with the caveat in mind that some implementation details are not part of the spec.)
You mean if the parent thread will wait for all the thread created using your ExecutorService invocation? Then answer is yes, parent thread will wait and once all threads are finished you will get the list of Futures object which will hold the result of each thread execution.
See below from ExecutorService.invokeAll()
Executes the given tasks, returning a list of Futures holding their
status and results when all complete.
InvokeAll method blocks till all the tasks are completed and list of futures are returned,
Solution:
If we don't want this to happen and continue with execution of program ,we can Loop through the tasks and pass it to Submit method of ExecutorService and add it to the List of Future Objects
ExecutorService es=Executors.newFixedThreadPool(4);
List<SampleClassimplementingCallable<String>> tasks=new ArrayList<>();
List<Future<String>> futures=new ArrayList<>();
for(SampleClassimplementingCallable<String> s:tasks)
{
//This Won't Block the Calling Thread and We will get the list of futures
futures.add(es.submit(s));
}
However, When the Futures are retrieved from the list and get method is called on indivual future object ,then the thread is blocked.

Categories

Resources