Develop Reference

Concurrent tests: test case scenario automatization

Task definition: I need to test custom concurrent collection or some container which manipulates with collections in concurrent environment. More precisely - I've read-API and write-API. I should test if there is any scenarios where I can get inconsistent data.
Problem: All concurrent test frameworks (like MultiThreadedTC, look at MultiThreadedTc section of my question) just provides you an ability to control the asynchronous code execution sequence. I mean you should suppose a critical scenarios by your own.
Broad question: Is there frameworks that can take annotations like #SharedResource, #readAPI, #writeAPI and check if your data will always be consistent? Is that impossible or I just leak a startup idea?
Annotation: If there is no such framework, but you find the idea attractive, you are welcome to contact me or propose your ideas.
Narrow question: I'm new in concurrency. So can you suggest which scenarios should I test in the code below? (look at PeerContainer class)
PeerContainer:
public class PeersContainer {
public class DaemonThreadFactory implements ThreadFactory {
private int counter = 1;
private final String prefix = "Daemon";
#Override
public Thread newThread(Runnable r) {
Thread thread = new Thread(r, prefix + "-" + counter);
thread.setDaemon(true);
counter++;
return thread;
}
}
private static class CacheCleaner implements Runnable {
private final Cache<Long, BlockingDeque<Peer>> cache;
public CacheCleaner(Cache<Long, BlockingDeque<Peer>> cache) {
this.cache = cache;
Thread.currentThread().setDaemon(true);
}
#Override
public void run() {
cache.cleanUp();
}
}
private final static int MAX_CACHE_SIZE = 100;
private final static int STRIPES_AMOUNT = 10;
private final static int PEER_ACCESS_TIMEOUT_MIN = 30;
private final static int CACHE_CLEAN_FREQUENCY_MIN = 1;
private final static PeersContainer INSTANCE;
private final Cache<Long, BlockingDeque<Peer>> peers = CacheBuilder.newBuilder()
.maximumSize(MAX_CACHE_SIZE)
.expireAfterWrite(PEER_ACCESS_TIMEOUT_MIN, TimeUnit.MINUTES)
.removalListener(new RemovalListener<Long, BlockingDeque<Peer>>() {
public void onRemoval(RemovalNotification<Long, BlockingDeque<Peer>> removal) {
if (removal.getCause() == RemovalCause.EXPIRED) {
for (Peer peer : removal.getValue()) {
peer.sendLogoutResponse(peer);
}
}
}
})
.build();
private final Striped<Lock> stripes = Striped.lock(STRIPES_AMOUNT);
private final ScheduledExecutorService scheduledExecutorService = Executors.newScheduledThreadPool(1, new DaemonThreadFactory());
private PeersContainer() {
scheduledExecutorService.schedule(new CacheCleaner(peers), CACHE_CLEAN_FREQUENCY_MIN, TimeUnit.MINUTES);
}
static {
INSTANCE = new PeersContainer();
}
public static PeersContainer getInstance() {
return INSTANCE;
}
private final Cache<Long, UserAuthorities> authToRestore = CacheBuilder.newBuilder()
.maximumSize(MAX_CACHE_SIZE)
.expireAfterWrite(PEER_ACCESS_TIMEOUT_MIN, TimeUnit.MINUTES)
.build();
public Collection<Peer> getPeers(long sessionId) {
return Collections.unmodifiableCollection(peers.getIfPresent(sessionId));
}
public Collection<Peer> getAllPeers() {
BlockingDeque<Peer> result = new LinkedBlockingDeque<Peer>();
for (BlockingDeque<Peer> deque : peers.asMap().values()) {
result.addAll(deque);
}
return Collections.unmodifiableCollection(result);
}
public boolean addPeer(Peer peer) {
long key = peer.getSessionId();
Lock lock = stripes.get(key);
lock.lock();
try {
BlockingDeque<Peer> userPeers = peers.getIfPresent(key);
if (userPeers == null) {
userPeers = new LinkedBlockingDeque<Peer>();
peers.put(key, userPeers);
}
UserAuthorities authorities = restoreSession(key);
if (authorities != null) {
peer.setAuthorities(authorities);
}
return userPeers.offer(peer);
} finally {
lock.unlock();
}
}
public void removePeer(Peer peer) {
long sessionId = peer.getSessionId();
Lock lock = stripes.get(sessionId);
lock.lock();
try {
BlockingDeque<Peer> userPeers = peers.getIfPresent(sessionId);
if (userPeers != null && !userPeers.isEmpty()) {
UserAuthorities authorities = userPeers.getFirst().getAuthorities();
authToRestore.put(sessionId, authorities);
userPeers.remove(peer);
}
} finally {
lock.unlock();
}
}
void removePeers(long sessionId) {
Lock lock = stripes.get(sessionId);
lock.lock();
try {
peers.invalidate(sessionId);
authToRestore.invalidate(sessionId);
} finally {
lock.unlock();
}
}
private UserAuthorities restoreSession(long sessionId) {
BlockingDeque<Peer> activePeers = peers.getIfPresent(sessionId);
return (activePeers != null && !activePeers.isEmpty()) ? activePeers.getFirst().getAuthorities() : authToRestore.getIfPresent(sessionId);
}
public void resetAccessedTimeout(long sessionId) {
Lock lock = stripes.get(sessionId);
lock.lock();
try {
BlockingDeque<Peer> deque = peers.getIfPresent(sessionId);
peers.invalidate(sessionId);
peers.put(sessionId, deque);
} finally {
lock.unlock();
}
}
}
MultiThreadedTC test case sample: [optional section of question]
public class ProducerConsumerTest extends MultithreadedTestCase {
private LinkedTransferQueue<String> queue;
#Override
public void initialize() {
super.initialize();
queue = new LinkedTransferQueue<String>();
}
public void thread1() throws InterruptedException {
String ret = queue.take();
}
public void thread2() throws InterruptedException {
waitForTick(1);
String ret = queue.take();
}
public void thread3() {
waitForTick(1);
waitForTick(2);
queue.put("Event 1");
queue.put("Event 2");
}
#Override
public void finish() {
super.finish();
assertEquals(true, queue.size() == 0);
}
}

Sounds like a job for static analysis, not testing, unless you have time to run multiple trillions of test cases. You pretty much can't test multithreaded behaviour - test behaviour in a single thread, then prove the abscence of threading bugs.
Try:
http://www.contemplateltd.com/threadsafe
http://checkthread.org/

Executor wait for ever in a singleton class

I have implemented a singleton (manager) to manage some related tasks, inside this manager I am using an executor to handle 10 task at the same time, I was using linkedBlockingQueue with no limit, and that's working good so far, but now I need to set a limitation to my executor queue because I have a lot of tasks (hundreds of thousands tasks), and I don’t want to put them all in my queue that causing me a performance issues, so what I have done:
here is my Executor :
public class MyThreadPoolExecutor extends ThreadPoolExecutor {
public MyThreadPoolExecutor(int corePoolSize, BlockingQueue<Runnable> workQueue) {
super(corePoolSize, corePoolSize + 5, 500, TimeUnit.MILLISECONDS, workQueue);
}
#Override
protected void beforeExecute(Thread t, Runnable r) {
super.beforeExecute(t, r);
//Do something to my task
}
#Override
protected void afterExecute(Runnable r, Throwable t) {
super.afterExecute(r, t);
if(t != null) {
//
} else {
//Do something to my task
}
}
}
and here is my manager :
public final class MyManager {
private static MyManager manager = new MyManager();
public static final int queueMaxSize = 100;
private BlockingQueue<Runnable> workQueue = new ArrayBlockingQueue<Runnable>(queueMaxSize);
private ExecutorService executor = new MyThreadPoolExecutor(10, workQueue);
/**
* constructor
*/
private MyManager() {}
public static MyManager getInstance(){
if (manager == null){
synchronized(MyManager.class){
if (manager == null){
manager = new MyManager();
}
}
}
return manager;
}
/**
*/
public void executeTask(Integer key){
executeTask(key, Locale.getDefault());
}
/**
*/
public void executeTask(Integer key, Locale locale) {
Tasker task = new Tasker(key, locale);
executor.execute(task);
}
}
and here the class that asking to do the tasks :
public class MyClass {
public void doTasks() {
//geting my tasks in array of list, its holding more than 900 000 tasks,
//sometimes its holding up to 10 million task like :
MyManager.getInstance().isFull() {\\wait, then ask again}
ArrayList<Integer> myTasks = getAllTasksIds();
for(Integer id : myTasks) {
//if i perform a waiting here it will be waiting for ever.
MyManaget.getInstance().executeTask(id);
}
}
}
What I want exactly to wait the executor until finish his queue tasks, then re-full it again.
But the problem is when I try to wait based on queue size, the executor won’t work, and its wait forever because the queue still full.

Why wouldn't you just use a bounded blocking queue (i.e. specify a bound of a BlockingQueue)? If you use a bounded blocking queue (of which size you can choose yourself), your producer will block when the queue is full, and will resume publishing tasks when a task is consumed from a queue. This way, you can avoid putting too much stuff too quickly onto the queue, but also avoid putting too less on the queue. That's kind of the point of blocking queues...

I tested your code but instead of using ArrayBlockingQueue I extended it with this... And it works. Try it:
public static class MyBlockingQueue extends ArrayBlockingQueue<Runnable> {
private static final long serialVersionUID= -9016421283603545618L;
public static Lock lock= new ReentrantLock();
public static Condition condition= lock.newCondition();
public static volatile Boolean isWaiting= false;
public MyBlockingQueue(int capacity) {
super(capacity, true);
}
#Override
public boolean offer(Runnable e) {
if (remainingCapacity() == 0) {
try {
isWaiting= true;
condition.await();
} catch (InterruptedException e1) {
e1.printStackTrace();
}
}
return super.offer(e);
}
#Override
public Runnable take() throws InterruptedException {
Runnable take= super.take();
if (remainingCapacity() > 0 && isWaiting) {
isWaiting= false;
condition.signal();
}
return take;
}
}

Determine when a ScheduledExecutorService will fire next

Is there a way to determine the current millisecond or other time measure of when a ScheduledExecutorService is going to fire next?
scheduleTaskExecutorUpdate = Executors.newSingleThreadScheduledExecutor();
I have a longer running ScheduledExecutorService(A) and from a shorter running ScheduledExecutorService(B) I would like to update a TextView, display a countdown of when ScheduledExecutorService(A) is going to fire next.

If you keep track of the ScheduledFutures for all tasks scheduled with the executor, then yes. This becomes a problem of determining the minimum delay until the next task must fire, which should be a fairly reliable estimate.
final Collection<ScheduledFuture<?>> futures = ...;
/* for each schedule, add it to the above collection */
...
final long delay = Collections.min(futures).getDelay(TimeUnit.MILLISECONDS);
... or, for one task, you merely do:
final ScheduledFuture<?> future = ...;
final long delay = future.getDelay(TimeUnit.MILLISECONDS);
Now, if you're going to be doing it a lot, with mutiple tasks, I'd suggest you maintain a DelayQueue. However, you can't merely throw the ScheduledFutures in the queue without maintaining the changes caused by periodic tasks. Luckily, the class ScheduledThreadPoolExecutor should handle this nicely via its decorateTask methods.
Note this means you will need to create your ownScheduledThreadPoolExecutor directly. Something like the below might work.
public class TrackingSingleThreadScheduledExecutor
extends ScheduledThreadPoolExecutor {
private final DelayQueue<ScheduledFuture<?>> tasks
= new DelayQueue<RunnableScheduledFuture<?>>();
public TrackingSingleThreadScheduledExecutor() {
super(1);
}
public DelayQueue<? extends ScheduledFuture<V>> tasks() {
return tasks;
}
public ScheduledFuture<V> next() {
return tasks.peek();
}
protected <V> RunnableScheduledFuture<V> decorateTask
(final Callable<V> callable, final RunnableScheduledFuture<V> task) {
return new QueueAwareTask(task);
}
protected <V> RunnableScheduledFuture<V> decorateTask
(final Runnable runnable, final RunnableScheduledFuture<V> task) {
return new QueueAwareTask(task);
}
private final class QueueAwareTask<V> implements RunnableScheduledFuture<V> {
private final RunnableScheduledFuture<V> inner;
public QueueAwareTask(final RunnableScheduledFuture<V> inner) {
this.inner = inner;
}
public boolean isPeriodic() {
return inner.isPeriodic();
}
public long getDelay(final TimeUnit unit) {
return inner.getDelay(unit);
}
public void run() {
inner.run();
if (queue.remove(inner) && inner.isPeriodic()
&& !inner.isCancelled()) {
queue.add(inner);
}
}
public int compareTo(final Delayed other) {
return inner.compareTo(other);
}
public boolean cancel(final boolean mayInterruptIfRunning) {
final boolean cancelled = inner.cancel(mayInterruptIfRunning);
if (cancelled) {
queue.remove(inner);
}
return cancelled;
}
public boolean isCancelled() {
return inner.isCancelled();
}
public boolean isDone() {
return inner.isDone();
}
public V get() throws InterruptedException, ExecutionException {
return inner.get();
}
public V get(final long timeout, final TimeUnit unit)
throws InterruptedException, ExecutionException {
return inner.get(timeout, unit);
}
}
}
Then, usage is as follows.
final TrackingSingleThreadScheduledExecutor executor
= new TrackingSingleThreadScheduledExecutor();
...
final long delay = executor.next().getDelay(TimeUnit.MILLISECONDS);

Controlling Task execution order with ExecutorService

I have a process which delegates asynch tasks to a pool of threads. I need to ensure that certain tasks are executed in order.
So for example
Tasks arrive in order
Tasks a1, b1, c1, d1 , e1, a2, a3, b2, f1
Tasks can be executed in any order except where there is a natural dependancy, so a1,a2,a3 must be processed in that order by either allocating to the same thread or blocking these until I know the previous a# task was completed.
Currently it doesn't use the Java Concurrency package, but I'm considering changing to take avantage of the thread management.
Does anyone have a similar solution or suggestions of how to achieve this

I write own Executor that warrants task ordering for tasks with same key. It uses map of queues for order tasks with same key. Each keyed task execute next task with the same key.
This solution don't handle RejectedExecutionException or other exceptions from delegated Executor! So delegated Executor should be "unlimited".
import java.util.HashMap;
import java.util.LinkedList;
import java.util.Map;
import java.util.Queue;
import java.util.concurrent.Executor;
/**
* This Executor warrants task ordering for tasks with same key (key have to implement hashCode and equal methods correctly).
*/
public class OrderingExecutor implements Executor{
private final Executor delegate;
private final Map<Object, Queue<Runnable>> keyedTasks = new HashMap<Object, Queue<Runnable>>();
public OrderingExecutor(Executor delegate){
this.delegate = delegate;
}
#Override
public void execute(Runnable task) {
// task without key can be executed immediately
delegate.execute(task);
}
public void execute(Runnable task, Object key) {
if (key == null){ // if key is null, execute without ordering
execute(task);
return;
}
boolean first;
Runnable wrappedTask;
synchronized (keyedTasks){
Queue<Runnable> dependencyQueue = keyedTasks.get(key);
first = (dependencyQueue == null);
if (dependencyQueue == null){
dependencyQueue = new LinkedList<Runnable>();
keyedTasks.put(key, dependencyQueue);
}
wrappedTask = wrap(task, dependencyQueue, key);
if (!first)
dependencyQueue.add(wrappedTask);
}
// execute method can block, call it outside synchronize block
if (first)
delegate.execute(wrappedTask);
}
private Runnable wrap(Runnable task, Queue<Runnable> dependencyQueue, Object key) {
return new OrderedTask(task, dependencyQueue, key);
}
class OrderedTask implements Runnable{
private final Queue<Runnable> dependencyQueue;
private final Runnable task;
private final Object key;
public OrderedTask(Runnable task, Queue<Runnable> dependencyQueue, Object key) {
this.task = task;
this.dependencyQueue = dependencyQueue;
this.key = key;
}
#Override
public void run() {
try{
task.run();
} finally {
Runnable nextTask = null;
synchronized (keyedTasks){
if (dependencyQueue.isEmpty()){
keyedTasks.remove(key);
}else{
nextTask = dependencyQueue.poll();
}
}
if (nextTask!=null)
delegate.execute(nextTask);
}
}
}
}

When I've done this in the past I've usually had the ordering handled by a component which then submits callables/runnables to an Executor.
Something like.
Got a list of tasks to run, some with dependencies
Create an Executor and wrap with an ExecutorCompletionService
Search all tasks, any with no dependencies, schedule them via the completion service
Poll the completion service
As each task completes
Add it to a "completed" list
Reevaluate any waiting tasks wrt to the "completed list" to see if they are "dependency complete". If so schedule them
Rinse repeat until all tasks are submitted/completed
The completion service is a nice way of being able to get the tasks as they complete rather than trying to poll a bunch of Futures. However you will probably want to keep a Map<Future, TaskIdentifier> which is populated when a task is schedule via the completion service so that when the completion service gives you a completed Future you can figure out which TaskIdentifier it is.
If you ever find yourself in a state where tasks are still waiting to run, but nothing is running and nothing can be scheduled then your have a circular dependency problem.

When you submit a Runnable or Callable to an ExecutorService you receive a Future in return. Have the threads that depend on a1 be passed a1's Future and call Future.get(). This will block until the thread completes.
So:
ExecutorService exec = Executor.newFixedThreadPool(5);
Runnable a1 = ...
final Future f1 = exec.submit(a1);
Runnable a2 = new Runnable() {
#Override
public void run() {
f1.get();
... // do stuff
}
}
exec.submit(a2);
and so on.

You can use Executors.newSingleThreadExecutor(), but it will use only one thread to execute your tasks. Another option is to use CountDownLatch. Here is a simple example:
public class Main2 {
public static void main(String[] args) throws InterruptedException {
final CountDownLatch cdl1 = new CountDownLatch(1);
final CountDownLatch cdl2 = new CountDownLatch(1);
final CountDownLatch cdl3 = new CountDownLatch(1);
List<Runnable> list = new ArrayList<Runnable>();
list.add(new Runnable() {
public void run() {
System.out.println("Task 1");
// inform that task 1 is finished
cdl1.countDown();
}
});
list.add(new Runnable() {
public void run() {
// wait until task 1 is finished
try {
cdl1.await();
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("Task 2");
// inform that task 2 is finished
cdl2.countDown();
}
});
list.add(new Runnable() {
public void run() {
// wait until task 2 is finished
try {
cdl2.await();
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("Task 3");
// inform that task 3 is finished
cdl3.countDown();
}
});
ExecutorService es = Executors.newFixedThreadPool(200);
for (int i = 0; i < 3; i++) {
es.submit(list.get(i));
}
es.shutdown();
es.awaitTermination(1, TimeUnit.MINUTES);
}
}

Another option is to create your own executor, call it OrderedExecutor, and create an array of encapsulated ThreadPoolExecutor objects, with 1 thread per internal executor. You then supply a mechanism for choosing one of the internal objects, eg, you can do this by providing an interface that the user of your class can implement:
executor = new OrderedExecutor( 10 /* pool size */, new OrderedExecutor.Chooser() {
public int choose( Runnable runnable ) {
MyRunnable myRunnable = (MyRunnable)runnable;
return myRunnable.someId();
});
executor.execute( new MyRunnable() );
The implementation of OrderedExecutor.execute() will then use the Chooser to get an int, you mod this with the pool size, and that's your index into the internal array. The idea being that "someId()" will return the same value for all the "a's", etc.

I created an OrderingExecutor for this problem. If you pass the same key to to method execute() with different runnables, the execution of the runnables with the same key will be in the order the execute() is called and will never overlap.
import java.util.Arrays;
import java.util.Collection;
import java.util.Iterator;
import java.util.Queue;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentLinkedQueue;
import java.util.concurrent.ConcurrentMap;
import java.util.concurrent.Executor;
/**
* Special executor which can order the tasks if a common key is given.
* Runnables submitted with non-null key will guaranteed to run in order for the same key.
*
*/
public class OrderedExecutor {
private static final Queue<Runnable> EMPTY_QUEUE = new QueueWithHashCodeAndEquals<Runnable>(
new ConcurrentLinkedQueue<Runnable>());
private ConcurrentMap<Object, Queue<Runnable>> taskMap = new ConcurrentHashMap<Object, Queue<Runnable>>();
private Executor delegate;
private volatile boolean stopped;
public OrderedExecutor(Executor delegate) {
this.delegate = delegate;
}
public void execute(Runnable runnable, Object key) {
if (stopped) {
return;
}
if (key == null) {
delegate.execute(runnable);
return;
}
Queue<Runnable> queueForKey = taskMap.computeIfPresent(key, (k, v) -> {
v.add(runnable);
return v;
});
if (queueForKey == null) {
// There was no running task with this key
Queue<Runnable> newQ = new QueueWithHashCodeAndEquals<Runnable>(new ConcurrentLinkedQueue<Runnable>());
newQ.add(runnable);
// Use putIfAbsent because this execute() method can be called concurrently as well
queueForKey = taskMap.putIfAbsent(key, newQ);
if (queueForKey != null)
queueForKey.add(runnable);
delegate.execute(new InternalRunnable(key));
}
}
public void shutdown() {
stopped = true;
taskMap.clear();
}
/**
* Own Runnable used by OrderedExecutor.
* The runnable is associated with a specific key - the Queue<Runnable> for this
* key is polled.
* If the queue is empty, it tries to remove the queue from taskMap.
*
*/
private class InternalRunnable implements Runnable {
private Object key;
public InternalRunnable(Object key) {
this.key = key;
}
#Override
public void run() {
while (true) {
// There must be at least one task now
Runnable r = taskMap.get(key).poll();
while (r != null) {
r.run();
r = taskMap.get(key).poll();
}
// The queue emptied
// Remove from the map if and only if the queue is really empty
boolean removed = taskMap.remove(key, EMPTY_QUEUE);
if (removed) {
// The queue has been removed from the map,
// if a new task arrives with the same key, a new InternalRunnable
// will be created
break;
} // If the queue has not been removed from the map it means that someone put a task into it
// so we can safely continue the loop
}
}
}
/**
* Special Queue implementation, with equals() and hashCode() methods.
* By default, Java SE queues use identity equals() and default hashCode() methods.
* This implementation uses Arrays.equals(Queue::toArray()) and Arrays.hashCode(Queue::toArray()).
*
* #param <E> The type of elements in the queue.
*/
private static class QueueWithHashCodeAndEquals<E> implements Queue<E> {
private Queue<E> delegate;
public QueueWithHashCodeAndEquals(Queue<E> delegate) {
this.delegate = delegate;
}
public boolean add(E e) {
return delegate.add(e);
}
public boolean offer(E e) {
return delegate.offer(e);
}
public int size() {
return delegate.size();
}
public boolean isEmpty() {
return delegate.isEmpty();
}
public boolean contains(Object o) {
return delegate.contains(o);
}
public E remove() {
return delegate.remove();
}
public E poll() {
return delegate.poll();
}
public E element() {
return delegate.element();
}
public Iterator<E> iterator() {
return delegate.iterator();
}
public E peek() {
return delegate.peek();
}
public Object[] toArray() {
return delegate.toArray();
}
public <T> T[] toArray(T[] a) {
return delegate.toArray(a);
}
public boolean remove(Object o) {
return delegate.remove(o);
}
public boolean containsAll(Collection<?> c) {
return delegate.containsAll(c);
}
public boolean addAll(Collection<? extends E> c) {
return delegate.addAll(c);
}
public boolean removeAll(Collection<?> c) {
return delegate.removeAll(c);
}
public boolean retainAll(Collection<?> c) {
return delegate.retainAll(c);
}
public void clear() {
delegate.clear();
}
#Override
public boolean equals(Object obj) {
if (!(obj instanceof QueueWithHashCodeAndEquals)) {
return false;
}
QueueWithHashCodeAndEquals<?> other = (QueueWithHashCodeAndEquals<?>) obj;
return Arrays.equals(toArray(), other.toArray());
}
#Override
public int hashCode() {
return Arrays.hashCode(toArray());
}
}
}

In Habanero-Java library, there is a concept of data-driven tasks which can be used to express dependencies between tasks and avoid thread-blocking operations. Under the covers Habanero-Java library uses the JDKs ForkJoinPool (i.e. an ExecutorService).
For example, your use case for tasks A1, A2, A3, ... could be expressed as follows:
HjFuture a1 = future(() -> { doA1(); return true; });
HjFuture a2 = futureAwait(a1, () -> { doA2(); return true; });
HjFuture a3 = futureAwait(a2, () -> { doA3(); return true; });
Note that a1, a2, and a3 are just references to objects of type HjFuture and can be maintained in your custom data structures to specify the dependencies as and when the tasks A2 and A3 come in at runtime.
There are some tutorial slides available.
You can find further documentation as javadoc, API summary and primers.

I have written my won executor service which is sequence aware. It sequences the tasks which contain certain related reference and currently inflight.
You can go through the implementation at https://github.com/nenapu/SequenceAwareExecutorService

How do I implement task prioritization using an ExecutorService in Java 5?

I am implementing a thread pooling mechanism in which I'd like to execute tasks of varying priorities. I'd like to have a nice mechanism whereby I can submit a high priority task to the service and have it be scheduled before other tasks. The priority of the task is an intrinsic property of the task itself (whether I express that task as a Callable or a Runnable is not important to me).
Now, superficially it looks like I could use a PriorityBlockingQueue as the task queue in my ThreadPoolExecutor, but that queue contains Runnable objects, which may or may not be the Runnable tasks I've submitted to it. Moreover, if I've submitted Callable tasks, it's not clear how this would ever map.
Is there a way to do this? I'd really rather not roll my own for this, since I'm far more likely to get it wrong that way.
(An aside; yes, I'm aware of the possibility of starvation for lower-priority jobs in something like this. Extra points (?!) for solutions that have a reasonable guarantee of fairness)

I have solved this problem in a reasonable fashion, and I'll describe it below for future reference to myself and anyone else who runs into this problem with the Java Concurrent libraries.
Using a PriorityBlockingQueue as the means for holding onto tasks for later execution is indeed a movement in the correct direction. The problem is that the PriorityBlockingQueue must be generically instantiated to contain Runnable instances, and it is impossible to call compareTo (or similiar) on a Runnable interface.
Onto solving the problem. When creating the Executor, it must be given a PriorityBlockingQueue. The queue should further be given a custom Comparator to do proper in place sorting:
new PriorityBlockingQueue<Runnable>(size, new CustomTaskComparator());
Now, a peek at CustomTaskComparator:
public class CustomTaskComparator implements Comparator<MyType> {
#Override
public int compare(MyType first, MyType second) {
return comparison;
}
}
Everything looking pretty straight forward up to this point. It gets a bit sticky here. Our next problem is to deal with the creation of FutureTasks from the Executor. In the Executor, we must override newTaskFor as so:
#Override
protected <V> RunnableFuture<V> newTaskFor(Callable<V> c) {
//Override the default FutureTask creation and retrofit it with
//a custom task. This is done so that prioritization can be accomplished.
return new CustomFutureTask(c);
}
Where c is the Callable task that we're trying to execute. Now, let's have a peek at CustomFutureTask:
public class CustomFutureTask extends FutureTask {
private CustomTask task;
public CustomFutureTask(Callable callable) {
super(callable);
this.task = (CustomTask) callable;
}
public CustomTask getTask() {
return task;
}
}
Notice the getTask method. We're gonna use that later to grab the original task out of this CustomFutureTask that we've created.
And finally, let's modify the original task that we were trying to execute:
public class CustomTask implements Callable<MyType>, Comparable<CustomTask> {
private final MyType myType;
public CustomTask(MyType myType) {
this.myType = myType;
}
#Override
public MyType call() {
//Do some things, return something for FutureTask implementation of `call`.
return myType;
}
#Override
public int compareTo(MyType task2) {
return new CustomTaskComparator().compare(this.myType, task2.myType);
}
}
You can see that we implement Comparable in the task to delegate to the actual Comparator for MyType.
And there you have it, customized prioritization for an Executor using the Java libraries! It takes some bit of bending, but it's the cleanest that I've been able to come up with. I hope this is helpful to someone!

At first blush it would seem you could define an interface for your tasks that extends Runnable or Callable<T> and Comparable. Then wrap a ThreadPoolExecutor with a PriorityBlockingQueue as the queue, and only accept tasks that implement your interface.
Taking your comment into account, it looks like one option is to extend ThreadPoolExecutor, and override the submit() methods. Refer to AbstractExecutorService to see what the default ones look like; all they do is wrap the Runnable or Callable in a FutureTask and execute() it. I'd probably do this by writing a wrapper class that implements ExecutorService and delegates to an anonymous inner ThreadPoolExecutor. Wrap them in something that has your priority, so that your Comparator can get at it.

You can use these helper classes:
public class PriorityFuture<T> implements RunnableFuture<T> {
private RunnableFuture<T> src;
private int priority;
public PriorityFuture(RunnableFuture<T> other, int priority) {
this.src = other;
this.priority = priority;
}
public int getPriority() {
return priority;
}
public boolean cancel(boolean mayInterruptIfRunning) {
return src.cancel(mayInterruptIfRunning);
}
public boolean isCancelled() {
return src.isCancelled();
}
public boolean isDone() {
return src.isDone();
}
public T get() throws InterruptedException, ExecutionException {
return src.get();
}
public T get(long timeout, TimeUnit unit) throws InterruptedException, ExecutionException, TimeoutException {
return src.get(timeout, unit);
}
public void run() {
src.run();
}
public static Comparator<Runnable> COMP = new Comparator<Runnable>() {
public int compare(Runnable o1, Runnable o2) {
if (o1 == null && o2 == null)
return 0;
else if (o1 == null)
return -1;
else if (o2 == null)
return 1;
else {
int p1 = ((PriorityFuture<?>) o1).getPriority();
int p2 = ((PriorityFuture<?>) o2).getPriority();
return p1 > p2 ? 1 : (p1 == p2 ? 0 : -1);
}
}
};
}
AND
public interface PriorityCallable<T> extends Callable<T> {
int getPriority();
}
AND this helper method:
public static ThreadPoolExecutor getPriorityExecutor(int nThreads) {
return new ThreadPoolExecutor(nThreads, nThreads, 0L, TimeUnit.MILLISECONDS,
new PriorityBlockingQueue<Runnable>(10, PriorityFuture.COMP)) {
protected <T> RunnableFuture<T> newTaskFor(Callable<T> callable) {
RunnableFuture<T> newTaskFor = super.newTaskFor(callable);
return new PriorityFuture<T>(newTaskFor, ((PriorityCallable<T>) callable).getPriority());
}
};
}
AND then use it like this:
class LenthyJob implements PriorityCallable<Long> {
private int priority;
public LenthyJob(int priority) {
this.priority = priority;
}
public Long call() throws Exception {
System.out.println("Executing: " + priority);
long num = 1000000;
for (int i = 0; i < 1000000; i++) {
num *= Math.random() * 1000;
num /= Math.random() * 1000;
if (num == 0)
num = 1000000;
}
return num;
}
public int getPriority() {
return priority;
}
}
public class TestPQ {
public static void main(String[] args) throws InterruptedException, ExecutionException {
ThreadPoolExecutor exec = getPriorityExecutor(2);
for (int i = 0; i < 20; i++) {
int priority = (int) (Math.random() * 100);
System.out.println("Scheduling: " + priority);
LenthyJob job = new LenthyJob(priority);
exec.submit(job);
}
}
}

I will try to explain this problem with a fully functional code. But before diving into the code I would like to explain about PriorityBlockingQueue
PriorityBlockingQueue : PriorityBlockingQueue is an implementation of BlockingQueue. It accepts the tasks along with their priority and submits the task with the highest priority for execution first. If any two tasks have same priority, then we need to provide some custom logic to decide which task goes first.
Now lets get into the code straightaway.
Driver class : This class creates an executor which accepts tasks and later submits them for execution. Here we create two tasks one with LOW priority and the other with HIGH priority. Here we tell the executor to run a MAX of 1 threads and use the PriorityBlockingQueue.
public static void main(String[] args) {
/*
Minimum number of threads that must be running : 0
Maximium number of threads that can be created : 1
If a thread is idle, then the minimum time to keep it alive : 1000
Which queue to use : PriorityBlockingQueue
*/
PriorityBlockingQueue queue = new PriorityBlockingQueue();
ThreadPoolExecutor executor = new ThreadPoolExecutor(0,1,
1000, TimeUnit.MILLISECONDS,queue);
MyTask task = new MyTask(Priority.LOW,"Low");
executor.execute(new MyFutureTask(task));
task = new MyTask(Priority.HIGH,"High");
executor.execute(new MyFutureTask(task));
task = new MyTask(Priority.MEDIUM,"Medium");
executor.execute(new MyFutureTask(task));
}
MyTask class : MyTask implements Runnable and accepts priority as an argument in the constructor. When this task runs, it prints a message and then puts the thread to sleep for 1 second.
public class MyTask implements Runnable {
public int getPriority() {
return priority.getValue();
}
private Priority priority;
public String getName() {
return name;
}
private String name;
public MyTask(Priority priority,String name){
this.priority = priority;
this.name = name;
}
#Override
public void run() {
System.out.println("The following Runnable is getting executed "+getName());
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
MyFutureTask class : Since we are using PriorityBlocingQueue for holding our tasks, our tasks must be wrapped inside FutureTask and our implementation of FutureTask must implement Comparable interface. The Comparable interface compares the priority of 2 different tasks and submits the task with the highest priority for execution.
public class MyFutureTask extends FutureTask<MyFutureTask>
implements Comparable<MyFutureTask> {
private MyTask task = null;
public MyFutureTask(MyTask task){
super(task,null);
this.task = task;
}
#Override
public int compareTo(MyFutureTask another) {
return task.getPriority() - another.task.getPriority();
}
}
Priority class : Self explanatory Priority class.
public enum Priority {
HIGHEST(0),
HIGH(1),
MEDIUM(2),
LOW(3),
LOWEST(4);
int value;
Priority(int val) {
this.value = val;
}
public int getValue(){
return value;
}
}
Now when we run this example, we get the following output
The following Runnable is getting executed High
The following Runnable is getting executed Medium
The following Runnable is getting executed Low
Even though we submitted the LOW priority first, but HIGH priority task later, but since we are using a PriorityBlockingQueue, any task with a higher priority will execute first.

My solution preserves submition order of tasks for same priorities. It's an improvement of this answer
Task execution order is based on:
Priority
Submit order (within same priority)
Tester class:
public class Main {
public static void main(String[] args) throws InterruptedException, ExecutionException {
ExecutorService executorService = PriorityExecutors.newFixedThreadPool(1);
//Priority=0
executorService.submit(newCallable("A1", 200)); //Defaults to priority=0
executorService.execute(newRunnable("A2", 200)); //Defaults to priority=0
executorService.submit(PriorityCallable.of(newCallable("A3", 200), 0));
executorService.submit(PriorityRunnable.of(newRunnable("A4", 200), 0));
executorService.execute(PriorityRunnable.of(newRunnable("A5", 200), 0));
executorService.submit(PriorityRunnable.of(newRunnable("A6", 200), 0));
executorService.execute(PriorityRunnable.of(newRunnable("A7", 200), 0));
executorService.execute(PriorityRunnable.of(newRunnable("A8", 200), 0));
//Priority=1
executorService.submit(PriorityRunnable.of(newRunnable("B1", 200), 1));
executorService.submit(PriorityRunnable.of(newRunnable("B2", 200), 1));
executorService.submit(PriorityCallable.of(newCallable("B3", 200), 1));
executorService.execute(PriorityRunnable.of(newRunnable("B4", 200), 1));
executorService.submit(PriorityRunnable.of(newRunnable("B5", 200), 1));
executorService.shutdown();
}
private static Runnable newRunnable(String name, int delay) {
return new Runnable() {
#Override
public void run() {
System.out.println(name);
sleep(delay);
}
};
}
private static Callable<Integer> newCallable(String name, int delay) {
return new Callable<Integer>() {
#Override
public Integer call() throws Exception {
System.out.println(name);
sleep(delay);
return 10;
}
};
}
private static void sleep(long millis) {
try {
Thread.sleep(millis);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
throw new RuntimeException(e);
}
}
}
Result:
A1 B1 B2 B3 B4 B5 A2 A3 A4 A5 A6 A7 A8
First task is A1 because there were no higher priority in the queue when it was inserted. B tasks are 1 priority so executed earlier, A tasks are 0 priority so executed later, but execution order is follows submition order: B1, B2, B3, ... A2, A3, A4 ...
The solution:
public class PriorityExecutors {
public static ExecutorService newFixedThreadPool(int nThreads) {
return new PriorityExecutor(nThreads, nThreads, 0L, TimeUnit.MILLISECONDS);
}
private static class PriorityExecutor extends ThreadPoolExecutor {
private static final int DEFAULT_PRIORITY = 0;
private static AtomicLong instanceCounter = new AtomicLong();
#SuppressWarnings({"unchecked"})
public PriorityExecutor(int corePoolSize, int maximumPoolSize,
long keepAliveTime, TimeUnit unit) {
super(corePoolSize, maximumPoolSize, keepAliveTime, unit, (BlockingQueue) new PriorityBlockingQueue<ComparableTask>(10,
ComparableTask.comparatorByPriorityAndSequentialOrder()));
}
#Override
public void execute(Runnable command) {
// If this is ugly then delegator pattern needed
if (command instanceof ComparableTask) //Already wrapped
super.execute(command);
else {
super.execute(newComparableRunnableFor(command));
}
}
private Runnable newComparableRunnableFor(Runnable runnable) {
return new ComparableRunnable(ensurePriorityRunnable(runnable));
}
#Override
protected <T> RunnableFuture<T> newTaskFor(Callable<T> callable) {
return new ComparableFutureTask<>(ensurePriorityCallable(callable));
}
#Override
protected <T> RunnableFuture<T> newTaskFor(Runnable runnable, T value) {
return new ComparableFutureTask<>(ensurePriorityRunnable(runnable), value);
}
private <T> PriorityCallable<T> ensurePriorityCallable(Callable<T> callable) {
return (callable instanceof PriorityCallable) ? (PriorityCallable<T>) callable
: PriorityCallable.of(callable, DEFAULT_PRIORITY);
}
private PriorityRunnable ensurePriorityRunnable(Runnable runnable) {
return (runnable instanceof PriorityRunnable) ? (PriorityRunnable) runnable
: PriorityRunnable.of(runnable, DEFAULT_PRIORITY);
}
private class ComparableFutureTask<T> extends FutureTask<T> implements ComparableTask {
private Long sequentialOrder = instanceCounter.getAndIncrement();
private HasPriority hasPriority;
public ComparableFutureTask(PriorityCallable<T> priorityCallable) {
super(priorityCallable);
this.hasPriority = priorityCallable;
}
public ComparableFutureTask(PriorityRunnable priorityRunnable, T result) {
super(priorityRunnable, result);
this.hasPriority = priorityRunnable;
}
#Override
public long getInstanceCount() {
return sequentialOrder;
}
#Override
public int getPriority() {
return hasPriority.getPriority();
}
}
private static class ComparableRunnable implements Runnable, ComparableTask {
private Long instanceCount = instanceCounter.getAndIncrement();
private HasPriority hasPriority;
private Runnable runnable;
public ComparableRunnable(PriorityRunnable priorityRunnable) {
this.runnable = priorityRunnable;
this.hasPriority = priorityRunnable;
}
#Override
public void run() {
runnable.run();
}
#Override
public int getPriority() {
return hasPriority.getPriority();
}
#Override
public long getInstanceCount() {
return instanceCount;
}
}
private interface ComparableTask extends Runnable {
int getPriority();
long getInstanceCount();
public static Comparator<ComparableTask> comparatorByPriorityAndSequentialOrder() {
return (o1, o2) -> {
int priorityResult = o2.getPriority() - o1.getPriority();
return priorityResult != 0 ? priorityResult
: (int) (o1.getInstanceCount() - o2.getInstanceCount());
};
}
}
}
private static interface HasPriority {
int getPriority();
}
public interface PriorityCallable<V> extends Callable<V>, HasPriority {
public static <V> PriorityCallable<V> of(Callable<V> callable, int priority) {
return new PriorityCallable<V>() {
#Override
public V call() throws Exception {
return callable.call();
}
#Override
public int getPriority() {
return priority;
}
};
}
}
public interface PriorityRunnable extends Runnable, HasPriority {
public static PriorityRunnable of(Runnable runnable, int priority) {
return new PriorityRunnable() {
#Override
public void run() {
runnable.run();
}
#Override
public int getPriority() {
return priority;
}
};
}
}
}

Would it be possible to have one ThreadPoolExecutor for each level of priority? A ThreadPoolExecutor can be instanciated with a ThreadFactory and you could have your own implementation of a ThreadFactory to set the different priority levels.
class MaxPriorityThreadFactory implements ThreadFactory {
public Thread newThread(Runnable r) {
Thread thread = new Thread(r);
thread.setPriority(Thread.MAX_PRIORITY);
}
}