Develop Reference

Parallel execution of directed acyclic graph of tasks

I have a list of tasks [Task-A,Task-B,Task-C,Task-D, ...].
One task can be optionally dependent on other tasks.
For example:
A can be dependent on 3 tasks: B, C and D
B can be dependent on 2 tasks: C and E
It's basically a directed acyclic graph and execution of a task should happen only after the dependent tasks are executed.
Now it might happen that at any point of time, there are multiple tasks that are ready for execution. In such a case, we can run them in parallel.
Any idea on how to implement such an execution while having as much parallelism as possible?
class Task{
private String name;
private List<Task> dependentTasks;
public void run(){
// business logic
}
}

The other answer works fine but is too complicated.
A simpler way is to just execute Kahn's algorithm but in parallel.
The key is to execute all the tasks in parallel for whom all dependencies have been executed.
import java.time.Instant;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.atomic.AtomicInteger;
class DependencyManager {
private final ConcurrentHashMap<String, List<String>> _dependencies = new ConcurrentHashMap<>();
private final ConcurrentHashMap<String, List<String>> _reverseDependencies = new ConcurrentHashMap<>();
private final ConcurrentHashMap<String, Runnable> _tasks = new ConcurrentHashMap<>();
private final ConcurrentHashMap<String, Integer> _numDependenciesExecuted = new ConcurrentHashMap<>();
private final AtomicInteger _numTasksExecuted = new AtomicInteger(0);
private final ExecutorService _executorService = Executors.newFixedThreadPool(16);
private static Runnable getRunnable(DependencyManager dependencyManager, String taskId){
return () -> {
try {
Thread.sleep(2000); // A task takes 2 seconds to finish.
dependencyManager.taskCompleted(taskId);
} catch (InterruptedException e) {
e.printStackTrace();
}
};
}
/**
* In case a vertex is disconnected from the rest of the graph.
* #param taskId The task id
*/
public void addVertex(String taskId) {
_dependencies.putIfAbsent(taskId, new ArrayList<>());
_reverseDependencies.putIfAbsent(taskId, new ArrayList<>());
_tasks.putIfAbsent(taskId, getRunnable(this, taskId));
_numDependenciesExecuted.putIfAbsent(taskId, 0);
}
private void addEdge(String dependentTaskId, String dependeeTaskId) {
_dependencies.get(dependentTaskId).add(dependeeTaskId);
_reverseDependencies.get(dependeeTaskId).add(dependentTaskId);
}
public void addDependency(String dependentTaskId, String dependeeTaskId) {
addVertex(dependentTaskId);
addVertex(dependeeTaskId);
addEdge(dependentTaskId, dependeeTaskId);
}
private void taskCompleted(String taskId) {
System.out.println(String.format("%s:: Task %s done!!", Instant.now(), taskId));
_numTasksExecuted.incrementAndGet();
_reverseDependencies.get(taskId).forEach(nextTaskId -> {
_numDependenciesExecuted.computeIfPresent(nextTaskId, (__, currValue) -> currValue + 1);
int numDependencies = _dependencies.get(nextTaskId).size();
int numDependenciesExecuted = _numDependenciesExecuted.get(nextTaskId);
if (numDependenciesExecuted == numDependencies) {
// All dependencies have been executed, so we can submit this task to the threadpool.
_executorService.submit(_tasks.get(nextTaskId));
}
});
if (_numTasksExecuted.get() == _tasks.size()) {
topoSortCompleted();
}
}
private void topoSortCompleted() {
System.out.println("Topo sort complete!!");
_executorService.shutdownNow();
}
public void executeTopoSort() {
System.out.println(String.format("%s:: Topo sort started!!", Instant.now()));
_dependencies.forEach((taskId, dependencies) -> {
if (dependencies.isEmpty()) {
_executorService.submit(_tasks.get(taskId));
}
});
}
}
public class TestParallelTopoSort {
public static void main(String[] args) {
DependencyManager dependencyManager = new DependencyManager();
dependencyManager.addDependency("8", "5");
dependencyManager.addDependency("7", "5");
dependencyManager.addDependency("7", "6");
dependencyManager.addDependency("6", "3");
dependencyManager.addDependency("6", "4");
dependencyManager.addDependency("5", "1");
dependencyManager.addDependency("5", "2");
dependencyManager.addDependency("5", "3");
dependencyManager.addDependency("4", "1");
dependencyManager.executeTopoSort();
// Parallel version takes 8 seconds to execute.
// Serial version would have taken 16 seconds.
}
}
The Directed Acyclic Graph constructed in this example is this:

We can create a DAG where each vertex of the graph is one of the tasks.
After that, we can compute its topological sorted order.
We can then decorate the Task class with a priority field and run the ThreadPoolExecutor with a PriorityBlockingQueue which compares Tasks using the priority field.
The final trick is to override run() to first wait for all the dependent tasks to finish.
Since each task waits indefinitely for its dependent tasks to finish, we cannot afford to let the thread-pool be completely occupied with tasks that are higher up in the topological sort order; the thread pool will get stuck forever.
To avoid this, we just have to assign priorities to tasks according to the topological order.
import java.util.ArrayList;
import java.util.Collections;
import java.util.Comparator;
import java.util.List;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.FutureTask;
import java.util.concurrent.PriorityBlockingQueue;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;
public class Testing {
private static Callable<Void> getCallable(String taskId){
return () -> {
System.out.println(String.format("Task %s result", taskId));
Thread.sleep(100);
return null;
};
}
public static void main(String[] args) throws ExecutionException, InterruptedException {
Callable<Void> taskA = getCallable("A");
Callable<Void> taskB = getCallable("B");
Callable<Void> taskC = getCallable("C");
Callable<Void> taskD = getCallable("D");
Callable<Void> taskE = getCallable("E");
PrioritizedFutureTask<Void> pfTaskA = new PrioritizedFutureTask<>(taskA);
PrioritizedFutureTask<Void> pfTaskB = new PrioritizedFutureTask<>(taskB);
PrioritizedFutureTask<Void> pfTaskC = new PrioritizedFutureTask<>(taskC);
PrioritizedFutureTask<Void> pfTaskD = new PrioritizedFutureTask<>(taskD);
PrioritizedFutureTask<Void> pfTaskE = new PrioritizedFutureTask<>(taskE);
// Create a DAG graph.
pfTaskB.addDependency(pfTaskC).addDependency(pfTaskE);
pfTaskA.addDependency(pfTaskB).addDependency(pfTaskC).addDependency(pfTaskD);
// Now that we have a graph, we can just get its topological sorted order.
List<PrioritizedFutureTask<Void>> topological_sort = new ArrayList<>();
topological_sort.add(pfTaskE);
topological_sort.add(pfTaskC);
topological_sort.add(pfTaskB);
topological_sort.add(pfTaskD);
topological_sort.add(pfTaskA);
ThreadPoolExecutor executor = new ThreadPoolExecutor(5, 5, 0L, TimeUnit.MILLISECONDS,
new PriorityBlockingQueue<Runnable>(1, new CustomRunnableComparator()));
// Its important to insert the tasks in the topological sorted order, otherwise its possible that the thread pool will be stuck forever.
for (int i = 0; i < topological_sort.size(); i++) {
PrioritizedFutureTask<Void> pfTask = topological_sort.get(i);
pfTask.setPriority(i);
// The lower the priority, the sooner it will run.
executor.execute(pfTask);
}
}
}
class PrioritizedFutureTask<T> extends FutureTask<T> implements Comparable<PrioritizedFutureTask<T>> {
private Integer _priority = 0;
private final Callable<T> callable;
private final List<PrioritizedFutureTask> _dependencies = new ArrayList<>();
;
public PrioritizedFutureTask(Callable<T> callable) {
super(callable);
this.callable = callable;
}
public PrioritizedFutureTask(Callable<T> callable, Integer priority) {
this(callable);
_priority = priority;
}
public Integer getPriority() {
return _priority;
}
public PrioritizedFutureTask<T> setPriority(Integer priority) {
_priority = priority;
return this;
}
public PrioritizedFutureTask<T> addDependency(PrioritizedFutureTask dep) {
this._dependencies.add(dep);
return this;
}
#Override
public void run() {
for (PrioritizedFutureTask dep : _dependencies) {
try {
dep.get();
} catch (InterruptedException e) {
e.printStackTrace();
} catch (ExecutionException e) {
e.printStackTrace();
}
}
super.run();
}
#Override
public int compareTo(PrioritizedFutureTask<T> other) {
if (other == null) {
throw new NullPointerException();
}
return getPriority().compareTo(other.getPriority());
}
}
class CustomRunnableComparator implements Comparator<Runnable> {
#Override
public int compare(Runnable task1, Runnable task2) {
return ((PrioritizedFutureTask) task1).compareTo((PrioritizedFutureTask) task2);
}
}
Output:
Task E result
Task C result
Task B result
Task D result
Task A result
PS: Here is a well-tested and simple implementation of topological sort in Python which you can easily port in Java.

Getting a result in the future?

I'm looking to get a result from a method which can take a while to complete and doesn't actually return the object, so I'd like to deal with it as effectively as possible. Here's an example of what I'm trying to achieve:
public static void main (String[] args) {
Object obj = someMethod();
System.out.println("The object is" + obj + ", wooh!");
}
public void callObject() {
// Sends request for the object
}
public void receiveObject(Object object) {
// Received the object
}
public Object someMethod() {
callObject();
// delay whilst the object is being received
// return received object once received, but how?
}
The method callObject will call to get the object, however a different method is called with the object in. I want someMethod() to be able to call for the object, and then return what it eventually receives, even though the actual call and receive are separate methods.
I've looked into using FutureTasks and Callables which I think is the way forward, I'm just not too sure how to implement it.
Sorry if I didn't explain myself too well, I'll give more information if necessary.
Thanks!

You could write a method, that kicks of some long running task asynchronously. You would then return a future object, that is empty but gets filled when the long running task is completed. In other programming languages, this is called a promise.
Here is an simple example. I created a method called someLongAsyncOperation which executes something that takes a while. To simulate this, I just sleep for 3 seconds before generating an answer.
import java.util.UUID;
import java.util.concurrent.*;
public class Test {
private static final ExecutorService executorService = Executors.newSingleThreadExecutor();
public Future<MyAnswer> someLongAsyncOperation(){
Future<MyAnswer> future = executorService.submit(() -> {
Thread.sleep(3000);
return new MyAnswer(UUID.randomUUID().toString());
});
return future;
}
public static void main(String[] args) throws Exception {
System.out.println("calling someLongAsyncOperation ...");
Future<MyAnswer> future = new Test().someLongAsyncOperation();
System.out.println("calling someLongAsyncOperation done.");
// do something else
System.out.println("wait for answer ...");
MyAnswer myAnswer = future.get();
System.out.printf("wait for answer done. Answer is: %s", myAnswer.value);
executorService.shutdown();
}
static class MyAnswer {
final String value;
MyAnswer(String value) {
this.value = value;
}
}
}
If you execute this little test class, you'll see, that someLongAsyncOperation returns fast, but when calling future.get(); we wait for the operation to complete.
You could now do something like starting of more than one longAsyncOperation, so they would run in parallel. And then wait until all of them are done.
Does this work as a starting point for you?
EDIT
You could implement someMethod like this:
public MyAnswer someMethod() throws ExecutionException, InterruptedException {
Future<MyAnswer> future = someLongAsyncOperation(); // kick of async operation
return future.get(); // wait for result
}
Which will make the async operation synchron again, by calling it and waiting for the result.
EDIT2
Here's another example that uses wait/notify:
import java.util.UUID;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import java.util.concurrent.locks.Condition;
import java.util.concurrent.locks.Lock;
import java.util.concurrent.locks.ReentrantLock;
public class Test2 {
private static final ExecutorService executorService = Executors.newSingleThreadExecutor();
private Object receivedObject;
private final Object mutex = new Object();
public static void main (String[] args) throws InterruptedException {
Object obj = new Test2().someMethod();
System.out.println("The object is" + obj + ", wooh!");
executorService.shutdown();
}
public void callObject() {
System.out.println("callObject ...");
// Sends request for the object asynchronously!
executorService.submit(() -> {
// some wait time to simulate slow request
try {
Thread.sleep(3000);
} catch (InterruptedException e) {
e.printStackTrace();
}
// provide object to callback
receiveObject(UUID.randomUUID().toString());
});
System.out.println("callObject done.");
}
public void receiveObject(Object object) {
System.out.println("receiveObject ...");
synchronized (mutex) {
this.receivedObject = object;
mutex.notify();
}
System.out.println("receiveObject done.");
}
public Object someMethod() throws InterruptedException {
System.out.println("someMethod ...");
synchronized (mutex) {
callObject();
while(this.receivedObject == null){
mutex.wait();
}
}
System.out.println("someMethod done.");
return this.receivedObject;
}
}
someMethod waits until receivedObject exists. receiveObject notifies upon arrival.

You need a callback:
private abstract class Callback<T>{
run(T object);
}
public Object someMethod() {
callObject(new Callback<Object>()
{
#Override
public void run(Object object)
{
System.out.println("The object is" + object + ", wooh!");
}
})
}
public void callObject(Callback<Object> callback) {
// Sends request for the object
callback.run(object);
}

import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
class ThreadExample implements Callable<String>{
#Override
public String call() throws Exception {
// TODO Auto-generated method stub
return "Ashish";
}
}
public class FutureThreadExample {
public static void main(String a[]) throws InterruptedException, ExecutionException {
ExecutorService executorService=Executors.newFixedThreadPool(1);
List <Future<String>>objList=new ArrayList<Future<String>>();
for(int i=0;i<10;i++) {
Future<String> obj=executorService.submit(new ThreadExample());
objList.add(obj);
}
for( Future<String> fut:objList) {
System.out.println(fut.get());
}
executorService.shutdown();
}
}

how to use a synchronized linked hash map correctly

trying to make an lru map by subclassing linked hash map.
the map is run through collections.synchronized.
all usages of the map are surrounded by a synchronized block. the unit test also fails if they are all removed. one would think they are not necessary since the map was run through collections.synchronized.
one thread puts sequential numbers (0,1,2,3 ...) into the map. removals are handled by removed eldest entry. no one else removes entries from the map.
the other thread gets the data from the map.
the following unit test fails usually at "oops". this is when a non zero number shows up in the first position (it should be zero until the map gets full). other strange things can happen like null values in the entry set.
any pointers will be appreciated.
thanks
import static org.junit.Assert.*;
import java.util.Collections;
import java.util.LinkedHashMap;
import java.util.Map;
import java.util.Set;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
class LruMap<K,V> extends LinkedHashMap<K,V> {
public LruMap() {
super(defaultMaxSize+1,.75f,true);
maxSize=defaultMaxSize;
}
public LruMap(int arg0) {
super(arg0+1,.75f,true);
maxSize=arg0;
}
public LruMap(int arg0,float arg1) {
super(arg0+1,arg1,true);
maxSize=arg0;
}
public LruMap(int arg0,float arg1,boolean arg2) {
super(arg0+1,arg1,arg2);
if(!arg2)
throw new RuntimeException("you did not construct an lru map!");
maxSize=arg0;
}
public LruMap(Map<K,V> arg0) {
super(arg0);
throw new RuntimeException("you did not construct an lru map!");
}
public boolean removeEldestEntry(Map.Entry<K,V> eldest) {
return size()>maxSize;
}
public final int maxSize;
public static final int defaultMaxSize=2048;
static final long serialVersionUID=0;
}
class Server implements Runnable {
public Server(final int pieces,final int period) {
this.pieces=pieces;
this.period=period;
lruMap=Collections.synchronizedMap(new LruMap<Long,Long>(3*pieces/2));
}
#Override public void run() {
t0=System.currentTimeMillis();
while(piece<stopAtPiece) {
final long dt=System.currentTimeMillis()-t0;
final long target=piece(dt);
System.out.println("adding "+(target-piece+1)+" items");
for(;piece<=target;piece++) {
synchronized(lruMap) {
lruMap.put(piece,piece);
}
}
checkMap(piece,true);
try {
Thread.sleep(100);
} catch(InterruptedException e) {
e.printStackTrace();
break;
}
}
}
Map.Entry<Long,Long>[] checkMap(final long n,boolean print) {
synchronized(lruMap) {
Map.Entry<Long,Long>[] entries=null;
if(lruMap.size()>0) {
final Set<Map.Entry<Long,Long>> entrySet=lruMap.entrySet();
entries=new Map.Entry[entrySet.size()];
entrySet.toArray(entries);
long first=entries[0].getKey();
long last=entries[entries.length-1].getKey();
if(print)
for(Map.Entry<Long,Long> entry:entries)
System.out.print(entry.getKey()+" ");
System.out.println();
if(n<pieces&&first!=0) {
System.out.println("lru: first!=0! "+first);
if(throwWhenfirstIsNotZero) { throw new RuntimeException("oops"); }
}
for(int i=0;i<entries.length-1;i++) {
long p0=entries[i].getKey();
long p1=entries[i+1].getKey();
if(p0>p1)
System.out.println("out of order! "+p0+" "+p1);
else if(p0==p1)
System.out.println("dupicate "+p0+" "+p1);
else if(p0+1==p1)
; // ok
else if(p0+1<p1)
System.out.println("skipped "+p0+" "+p1);
else System.out.println("some case i mssed!");
}
}
return entries;
}
}
public long piece(final long dt) {
return dt/period*pieces+dt%period*pieces/period;
}
public boolean throwWhenfirstIsNotZero=true;
protected long piece;
public long t0;
protected long stopAtPiece=Long.MAX_VALUE;
public final int period;
public final int pieces;
public final Map<Long,Long> lruMap;
}
public class ServerTestCase {
#Before public void setUp() throws Exception {}
#After public void tearDown() throws Exception {}
#Test public void testRun() {
server.stopAtPiece=server.pieces;
server.throwWhenfirstIsNotZero=true;
Thread thread=new Thread(server);
thread.setName("server");
thread.start();
while(thread.isAlive()) {
for(long i=0;i<server.piece;i++)
synchronized(server.lruMap) {
server.lruMap.get(i);
}
}
}
final int period=2*1000;
final int serverPieces=100;
Server server=new Server(serverPieces,period);
}

If you are accessing the collection inside a synchronized(lruMap) block, then you probably don't want to wrap it in Collections.synchronizedMap() - use one or the other. This is because they will probably be using different locks - in fact it's almost certain, because it's extremely unlikely that synchronizedMap() is using synchronized(this) internally.
Also I recommend enter link description here

How to know when a CompletionService is finished delivering results?

I want to use a CompletionService to process the results from a series of threads as they are completed. I have the service in a loop to take the Future objects it provides as they become available, but I don't know the best way to determine when all the threads have completed (and thus to exit the loop):
import java.util.concurrent.Callable;
import java.util.concurrent.CompletionService;
import java.util.concurrent.ExecutorCompletionService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import java.util.concurrent.ThreadPoolExecutor;
public class Bar {
final static int MAX_THREADS = 4;
final static int TOTAL_THREADS = 20;
public static void main(String[] args) throws Exception{
final ThreadPoolExecutor threadPool = (ThreadPoolExecutor) Executors.newFixedThreadPool(MAX_THREADS);
final CompletionService<Integer> service = new ExecutorCompletionService<Integer>(threadPool);
for (int i=0; i<TOTAL_THREADS; i++){
service.submit(new MyCallable(i));
}
int finished = 0;
Future<Integer> future = null;
do{
future = service.take();
int result = future.get();
System.out.println(" took: " + result);
finished++;
}while(finished < TOTAL_THREADS);
System.out.println("Shutting down");
threadPool.shutdown();
}
public static class MyCallable implements Callable<Integer>{
final int id;
public MyCallable(int id){
this.id = id;
System.out.println("Submitting: " + id);
}
#Override
public Integer call() throws Exception {
Thread.sleep(1000);
System.out.println("finished: " + id);
return id;
}
}
}
I've tried checking the state of the ThreadPoolExecutor, but I know the getCompletedTaskCount and getTaskCount methods are only approximations and shouldn't be relied upon. Is there a better way to ensure that I've retrieved all the Futures from the CompletionService than counting them myself?
Edit: Both the link that Nobeh provided, and this link suggest that counting the number of tasks submitted, then calling take() that many times, is the way to go. I'm just surprised there isn't a way to ask the CompletionService or its Executor what's left to be returned.

See http://www.javaspecialists.eu/archive/Issue214.html for a decent suggestion on how to extend the ExecutorCompletionService to do what you're looking for. I've pasted the relevant code below for your convenience. The author also suggests making the service implement Iterable, which I think would be a good idea.
FWIW, I agree with you that this really should be part of the standard implementation, but alas, it's not.
import java.util.concurrent.*;
import java.util.concurrent.atomic.*;
public class CountingCompletionService<V> extends ExecutorCompletionService<V> {
private final AtomicLong submittedTasks = new AtomicLong();
private final AtomicLong completedTasks = new AtomicLong();
public CountingCompletionService(Executor executor) {
super(executor);
}
public CountingCompletionService(
Executor executor, BlockingQueue<Future<V>> queue) {
super(executor, queue);
}
public Future<V> submit(Callable<V> task) {
Future<V> future = super.submit(task);
submittedTasks.incrementAndGet();
return future;
}
public Future<V> submit(Runnable task, V result) {
Future<V> future = super.submit(task, result);
submittedTasks.incrementAndGet();
return future;
}
public Future<V> take() throws InterruptedException {
Future<V> future = super.take();
completedTasks.incrementAndGet();
return future;
}
public Future<V> poll() {
Future<V> future = super.poll();
if (future != null) completedTasks.incrementAndGet();
return future;
}
public Future<V> poll(long timeout, TimeUnit unit)
throws InterruptedException {
Future<V> future = super.poll(timeout, unit);
if (future != null) completedTasks.incrementAndGet();
return future;
}
public long getNumberOfCompletedTasks() {
return completedTasks.get();
}
public long getNumberOfSubmittedTasks() {
return submittedTasks.get();
}
public boolean hasUncompletedTasks() {
return completedTasks.get() < submittedTasks.get();
}
}

The code below is inspired by #Mark's answer, but I find it more convenient to use:
package com.example;
import java.util.Iterator;
import java.util.concurrent.Callable;
import java.util.concurrent.CompletionService;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorCompletionService;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicInteger;
public class CompletionIterator<T> implements Iterator<T>, AutoCloseable {
private AtomicInteger count = new AtomicInteger(0);
private CompletionService<T> completer;
private ExecutorService executor = Executors.newWorkStealingPool(100);
public CompletionIterator() {
this.completer = new ExecutorCompletionService<>(executor);
}
public void submit(Callable<T> task) {
completer.submit(task);
count.incrementAndGet();
}
#Override
public boolean hasNext() {
return count.decrementAndGet() >= 0;
}
#Override
public T next() {
try {
return completer.take().get();
} catch (InterruptedException | ExecutionException e) {
throw new RuntimeException(e);
}
}
#Override
public void close() {
try {
executor.shutdown();
executor.awaitTermination(Long.MAX_VALUE, TimeUnit.MILLISECONDS);
executor = null;
completer = null;
count = null;
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
}
}
This is how it can be used :
try(CompletionIterator service = new CompletionIterator()) {
service.submit(task1);
service.submit(task2);
// all tasks must be submitted before iterating, to avoid race condition
for (Future<Integer> future : service) {
System.out.printf("Job %d is done%n", future.get());
}
}

Answering to these questions gives you the answer?
Do your asynchronous tasks create other tasks submitted to CompletionService?
Is service the only object that is supposed to handle the tasks created in your application?
Based on reference documentation, CompletionService acts upon a consumer/producer approach and takes advantage of an internal Executor. So, as long as, you produce the tasks in one place and consume them in another place, CompletionService.take() will denote if there are any more results to give out.
I believe this question also helps you.

My take based on Alex R' variant. Implying this will only be called in one thread, so no atomics just plain int counter
public class CompletionIterator<T> implements Iterable<T> {
private int _count = 0;
private final CompletionService<T> _completer;
public CompletionIterator(ExecutorService executor) {
this._completer = new ExecutorCompletionService<>(executor);
}
public void submit(Callable<T> task) {
_completer.submit(task);
_count++;
}
#Override
public Iterator<T> iterator() {
return new Iterator<T>() {
#Override
public boolean hasNext() {
return _count > 0;
}
#Override
public T next() {
try {
T ret = _completer.take().get();
_count--;
return ret;
} catch (InterruptedException | ExecutionException e) {
throw new RuntimeException(e);
}
}
};
}
}

Controlling Task execution order with ExecutorService

I have a process which delegates asynch tasks to a pool of threads. I need to ensure that certain tasks are executed in order.
So for example
Tasks arrive in order
Tasks a1, b1, c1, d1 , e1, a2, a3, b2, f1
Tasks can be executed in any order except where there is a natural dependancy, so a1,a2,a3 must be processed in that order by either allocating to the same thread or blocking these until I know the previous a# task was completed.
Currently it doesn't use the Java Concurrency package, but I'm considering changing to take avantage of the thread management.
Does anyone have a similar solution or suggestions of how to achieve this

I write own Executor that warrants task ordering for tasks with same key. It uses map of queues for order tasks with same key. Each keyed task execute next task with the same key.
This solution don't handle RejectedExecutionException or other exceptions from delegated Executor! So delegated Executor should be "unlimited".
import java.util.HashMap;
import java.util.LinkedList;
import java.util.Map;
import java.util.Queue;
import java.util.concurrent.Executor;
/**
* This Executor warrants task ordering for tasks with same key (key have to implement hashCode and equal methods correctly).
*/
public class OrderingExecutor implements Executor{
private final Executor delegate;
private final Map<Object, Queue<Runnable>> keyedTasks = new HashMap<Object, Queue<Runnable>>();
public OrderingExecutor(Executor delegate){
this.delegate = delegate;
}
#Override
public void execute(Runnable task) {
// task without key can be executed immediately
delegate.execute(task);
}
public void execute(Runnable task, Object key) {
if (key == null){ // if key is null, execute without ordering
execute(task);
return;
}
boolean first;
Runnable wrappedTask;
synchronized (keyedTasks){
Queue<Runnable> dependencyQueue = keyedTasks.get(key);
first = (dependencyQueue == null);
if (dependencyQueue == null){
dependencyQueue = new LinkedList<Runnable>();
keyedTasks.put(key, dependencyQueue);
}
wrappedTask = wrap(task, dependencyQueue, key);
if (!first)
dependencyQueue.add(wrappedTask);
}
// execute method can block, call it outside synchronize block
if (first)
delegate.execute(wrappedTask);
}
private Runnable wrap(Runnable task, Queue<Runnable> dependencyQueue, Object key) {
return new OrderedTask(task, dependencyQueue, key);
}
class OrderedTask implements Runnable{
private final Queue<Runnable> dependencyQueue;
private final Runnable task;
private final Object key;
public OrderedTask(Runnable task, Queue<Runnable> dependencyQueue, Object key) {
this.task = task;
this.dependencyQueue = dependencyQueue;
this.key = key;
}
#Override
public void run() {
try{
task.run();
} finally {
Runnable nextTask = null;
synchronized (keyedTasks){
if (dependencyQueue.isEmpty()){
keyedTasks.remove(key);
}else{
nextTask = dependencyQueue.poll();
}
}
if (nextTask!=null)
delegate.execute(nextTask);
}
}
}
}

When I've done this in the past I've usually had the ordering handled by a component which then submits callables/runnables to an Executor.
Something like.
Got a list of tasks to run, some with dependencies
Create an Executor and wrap with an ExecutorCompletionService
Search all tasks, any with no dependencies, schedule them via the completion service
Poll the completion service
As each task completes
Add it to a "completed" list
Reevaluate any waiting tasks wrt to the "completed list" to see if they are "dependency complete". If so schedule them
Rinse repeat until all tasks are submitted/completed
The completion service is a nice way of being able to get the tasks as they complete rather than trying to poll a bunch of Futures. However you will probably want to keep a Map<Future, TaskIdentifier> which is populated when a task is schedule via the completion service so that when the completion service gives you a completed Future you can figure out which TaskIdentifier it is.
If you ever find yourself in a state where tasks are still waiting to run, but nothing is running and nothing can be scheduled then your have a circular dependency problem.

When you submit a Runnable or Callable to an ExecutorService you receive a Future in return. Have the threads that depend on a1 be passed a1's Future and call Future.get(). This will block until the thread completes.
So:
ExecutorService exec = Executor.newFixedThreadPool(5);
Runnable a1 = ...
final Future f1 = exec.submit(a1);
Runnable a2 = new Runnable() {
#Override
public void run() {
f1.get();
... // do stuff
}
}
exec.submit(a2);
and so on.

You can use Executors.newSingleThreadExecutor(), but it will use only one thread to execute your tasks. Another option is to use CountDownLatch. Here is a simple example:
public class Main2 {
public static void main(String[] args) throws InterruptedException {
final CountDownLatch cdl1 = new CountDownLatch(1);
final CountDownLatch cdl2 = new CountDownLatch(1);
final CountDownLatch cdl3 = new CountDownLatch(1);
List<Runnable> list = new ArrayList<Runnable>();
list.add(new Runnable() {
public void run() {
System.out.println("Task 1");
// inform that task 1 is finished
cdl1.countDown();
}
});
list.add(new Runnable() {
public void run() {
// wait until task 1 is finished
try {
cdl1.await();
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("Task 2");
// inform that task 2 is finished
cdl2.countDown();
}
});
list.add(new Runnable() {
public void run() {
// wait until task 2 is finished
try {
cdl2.await();
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("Task 3");
// inform that task 3 is finished
cdl3.countDown();
}
});
ExecutorService es = Executors.newFixedThreadPool(200);
for (int i = 0; i < 3; i++) {
es.submit(list.get(i));
}
es.shutdown();
es.awaitTermination(1, TimeUnit.MINUTES);
}
}

Another option is to create your own executor, call it OrderedExecutor, and create an array of encapsulated ThreadPoolExecutor objects, with 1 thread per internal executor. You then supply a mechanism for choosing one of the internal objects, eg, you can do this by providing an interface that the user of your class can implement:
executor = new OrderedExecutor( 10 /* pool size */, new OrderedExecutor.Chooser() {
public int choose( Runnable runnable ) {
MyRunnable myRunnable = (MyRunnable)runnable;
return myRunnable.someId();
});
executor.execute( new MyRunnable() );
The implementation of OrderedExecutor.execute() will then use the Chooser to get an int, you mod this with the pool size, and that's your index into the internal array. The idea being that "someId()" will return the same value for all the "a's", etc.

I created an OrderingExecutor for this problem. If you pass the same key to to method execute() with different runnables, the execution of the runnables with the same key will be in the order the execute() is called and will never overlap.
import java.util.Arrays;
import java.util.Collection;
import java.util.Iterator;
import java.util.Queue;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentLinkedQueue;
import java.util.concurrent.ConcurrentMap;
import java.util.concurrent.Executor;
/**
* Special executor which can order the tasks if a common key is given.
* Runnables submitted with non-null key will guaranteed to run in order for the same key.
*
*/
public class OrderedExecutor {
private static final Queue<Runnable> EMPTY_QUEUE = new QueueWithHashCodeAndEquals<Runnable>(
new ConcurrentLinkedQueue<Runnable>());
private ConcurrentMap<Object, Queue<Runnable>> taskMap = new ConcurrentHashMap<Object, Queue<Runnable>>();
private Executor delegate;
private volatile boolean stopped;
public OrderedExecutor(Executor delegate) {
this.delegate = delegate;
}
public void execute(Runnable runnable, Object key) {
if (stopped) {
return;
}
if (key == null) {
delegate.execute(runnable);
return;
}
Queue<Runnable> queueForKey = taskMap.computeIfPresent(key, (k, v) -> {
v.add(runnable);
return v;
});
if (queueForKey == null) {
// There was no running task with this key
Queue<Runnable> newQ = new QueueWithHashCodeAndEquals<Runnable>(new ConcurrentLinkedQueue<Runnable>());
newQ.add(runnable);
// Use putIfAbsent because this execute() method can be called concurrently as well
queueForKey = taskMap.putIfAbsent(key, newQ);
if (queueForKey != null)
queueForKey.add(runnable);
delegate.execute(new InternalRunnable(key));
}
}
public void shutdown() {
stopped = true;
taskMap.clear();
}
/**
* Own Runnable used by OrderedExecutor.
* The runnable is associated with a specific key - the Queue<Runnable> for this
* key is polled.
* If the queue is empty, it tries to remove the queue from taskMap.
*
*/
private class InternalRunnable implements Runnable {
private Object key;
public InternalRunnable(Object key) {
this.key = key;
}
#Override
public void run() {
while (true) {
// There must be at least one task now
Runnable r = taskMap.get(key).poll();
while (r != null) {
r.run();
r = taskMap.get(key).poll();
}
// The queue emptied
// Remove from the map if and only if the queue is really empty
boolean removed = taskMap.remove(key, EMPTY_QUEUE);
if (removed) {
// The queue has been removed from the map,
// if a new task arrives with the same key, a new InternalRunnable
// will be created
break;
} // If the queue has not been removed from the map it means that someone put a task into it
// so we can safely continue the loop
}
}
}
/**
* Special Queue implementation, with equals() and hashCode() methods.
* By default, Java SE queues use identity equals() and default hashCode() methods.
* This implementation uses Arrays.equals(Queue::toArray()) and Arrays.hashCode(Queue::toArray()).
*
* #param <E> The type of elements in the queue.
*/
private static class QueueWithHashCodeAndEquals<E> implements Queue<E> {
private Queue<E> delegate;
public QueueWithHashCodeAndEquals(Queue<E> delegate) {
this.delegate = delegate;
}
public boolean add(E e) {
return delegate.add(e);
}
public boolean offer(E e) {
return delegate.offer(e);
}
public int size() {
return delegate.size();
}
public boolean isEmpty() {
return delegate.isEmpty();
}
public boolean contains(Object o) {
return delegate.contains(o);
}
public E remove() {
return delegate.remove();
}
public E poll() {
return delegate.poll();
}
public E element() {
return delegate.element();
}
public Iterator<E> iterator() {
return delegate.iterator();
}
public E peek() {
return delegate.peek();
}
public Object[] toArray() {
return delegate.toArray();
}
public <T> T[] toArray(T[] a) {
return delegate.toArray(a);
}
public boolean remove(Object o) {
return delegate.remove(o);
}
public boolean containsAll(Collection<?> c) {
return delegate.containsAll(c);
}
public boolean addAll(Collection<? extends E> c) {
return delegate.addAll(c);
}
public boolean removeAll(Collection<?> c) {
return delegate.removeAll(c);
}
public boolean retainAll(Collection<?> c) {
return delegate.retainAll(c);
}
public void clear() {
delegate.clear();
}
#Override
public boolean equals(Object obj) {
if (!(obj instanceof QueueWithHashCodeAndEquals)) {
return false;
}
QueueWithHashCodeAndEquals<?> other = (QueueWithHashCodeAndEquals<?>) obj;
return Arrays.equals(toArray(), other.toArray());
}
#Override
public int hashCode() {
return Arrays.hashCode(toArray());
}
}
}

In Habanero-Java library, there is a concept of data-driven tasks which can be used to express dependencies between tasks and avoid thread-blocking operations. Under the covers Habanero-Java library uses the JDKs ForkJoinPool (i.e. an ExecutorService).
For example, your use case for tasks A1, A2, A3, ... could be expressed as follows:
HjFuture a1 = future(() -> { doA1(); return true; });
HjFuture a2 = futureAwait(a1, () -> { doA2(); return true; });
HjFuture a3 = futureAwait(a2, () -> { doA3(); return true; });
Note that a1, a2, and a3 are just references to objects of type HjFuture and can be maintained in your custom data structures to specify the dependencies as and when the tasks A2 and A3 come in at runtime.
There are some tutorial slides available.
You can find further documentation as javadoc, API summary and primers.

I have written my won executor service which is sequence aware. It sequences the tasks which contain certain related reference and currently inflight.
You can go through the implementation at https://github.com/nenapu/SequenceAwareExecutorService