Develop Reference

Iterate through threads run via ThreadPoolTaskExecutor

I have a ThreadPoolTaskExecutor and when I create a Process which implements Runnable I run it via: executor.execute(process).
Now, before calling execute I want to check one field from Process object and compare it with ALL other currently running processes, executed by my ThreadPoolTaskExecutor. How I can do that, not generating a concurrent problem?
Code:
public class MyApp {
ThreadPoolTaskExecutor executor;
//...
public void runProcesses {
Process firstone = new Process(1);
Process nextOne = new Process(1);
// iterate through all processes started via executor and currently running,
// verify if there is any process.getX() == 1, if not run it
executor.execute(firstone );
//wait till firstone will end becouse have the same value of X
executor.execute(nextOne); // this cant be perform until the first one will end
}
}
public class Process {
private int x;
//...
public Process (int x){
this.x = x;
}
public int getX(){
return this.x;
}
}
I was thinking about createing simple Set of process started and add new one to it. But I have problem how to determine is it still running and remove it from set when it is done. So now I'm thinking about iterating through running threads, but completly dunno how.

I think that your initial idea is pretty good and can be made to work with not too much code.
It will require some tinkering in order to decouple "is a Runnable for this value already running" from "execute this Runnable", but here's a rough illustration that doesn't take care about that:
Implement equals() and hashCode() in Process, so that instances can safely be used in unordered sets and maps.
Create a ConcurrentMap<Process, Boolean>
You won't be using Collections.newSetFromMap(new ConcurrentHashMap<Process, Boolean>) because you'd want to use the map's putIfAbsent() method.
Try to add in it using putIfAbsent() each Process that you will be submitting and bail if the returned value is not null.
A non-null return value means that there's already an equivalent Process in the map (and therefore being processed).
The trivial and not very clean solution will be to inject a reference to the map in each Process instance and have putIfAbsent(this, true) as the first thing you do in your run() method.
Remove from it each Process that has finished processing.
The trivial and not very clean solution will be inject a reference to the map in each Process instance and have remove(this) as the last thing you do in your run() method.
Other solutions can have Process implement Callable and return its unique value as a result, so that it can be removed from the map, or use CompletableFuture and its thenAccept() callback.
Here's a sample that illustrates the trivial and not very clean solution described above (code too long to paste directly here).

Though #Dimitar provided very good solution for solving this problem I want to make an addition with another approach.
Having your requirements, it seems like you need to keep all submitted Processes, slicing them by x into separate queues and executing processes in queues one by one.
API of ThreadPoolExecutor empowers to enhance behaviour of Executor and I came to the following implementation of ThreadPoolExecutor:
ThreadPoolExecutor executor = new ThreadPoolExecutor(2, 2,
0L, TimeUnit.MILLISECONDS,
new LinkedBlockingQueue<>()) {
private final ConcurrentMap<Integer, Queue<Runnable>> processes = new ConcurrentHashMap<>();
#Override
public void execute(Runnable command) {
if (command instanceof Process) {
int id = ((Process) command).getX();
Queue<Runnable> t = new ArrayDeque<>();
Queue<Runnable> queue = this.processes.putIfAbsent(id, t);
if (queue == null) {
queue = t;
}
synchronized (queue) {
queue.add(command);
if (!processes.containsKey(id)) {
processes.put(id, queue);
}
if (queue.size() == 1) {
super.execute(queue.peek()); // removal of current process would be done in #afterExecute
}
}
} else {
super.execute(command);
}
}
#Override
protected void afterExecute(Runnable r, Throwable t) {
super.afterExecute(r, t);
if (r instanceof Process) {
int id = ((Process) r).getX();
Queue<Runnable> queue = this.processes.get(id);
synchronized (queue) {
queue.poll(); // remove completed prev process
Runnable nextProcess = queue.peek(); // retrieve next process
if (nextProcess != null) {
super.execute(nextProcess);
} else {
this.processes.remove(id);
}
}
}
}
}

Java How to implement lock on ConcurrentHashMap read

TL;DR: in Java I have N threads, each using a shared collection. ConcurrentHashMap allows me to lock on write, but not on read. What I need is to lock a specific item of the collection, read the previous data, do some computation, and update the values. If two threads receive two messages from the same sender, the second thread has to wait for the first one to finish, before doing its stuff.
Long version:
These threads are receiving chronologically ordered messages, and they have to update the collection basing on a messageSenderID.
My code simplified is as follow:
public class Parent {
private Map<String, MyObject> myObjects;
ExecutorService executor;
List<Future<?>> runnables = new ArrayList<Future<?>>();
public Parent(){
myObjects= new ConcurrentHashMap<String, MyObject>();
executor = Executors.newFixedThreadPool(10);
for (int i = 0; i < 10; i++) {
WorkerThread worker = new WorkerThread("worker_" + i);
Future<?> future = executor.submit(worker);
runnables.add(future);
}
}
private synchronized String getMessageFromSender(){
// Get a message from the common source
}
private synchronized MyObject getMyObject(String id){
MyObject myObject = myObjects.get(id);
if (myObject == null) {
myObject = new MyObject(id);
myObjects.put(id, myObject);
}
return myObject;
}
private class WorkerThread implements Runnable {
private String name;
public WorkerThread(String name) {
this.name = name;
}
#Override
public void run() {
while(!isStopped()) {
JSONObject message = getMessageFromSender();
String id = message.getString("id");
MyObject myObject = getMyObject(id);
synchronized (myObject) {
doLotOfStuff(myObject);
}
}
}
}
}
So basically I have one producer and N consumers, to speed-up processing, but the N consumers have to deal with a common base of data and chronological order has to be respected.
I am currently using a ConcurrentHashMap, but I'm willing to change it if needed.
The code seems to work if messages with same ID arrive enough apart (> 1 second), but if I get two messages with the same ID in the distance of microseconds, I get two threads dealing with the same item in the collection.
I GUESS that my desired behavior is:
Thread 1 Thread 2
--------------------------------------------------------------
read message 1
find ID
lock that ID in collection
do computation and update
read message 2
find ID
lock that ID in collection
do computation and update
While I THINK that this is what happens:
Thread 1 Thread 2
--------------------------------------------------------------
read message 1
read message 2
find ID
lock that ID in collection
do computation and update
find ID
lock that ID in collection
do computation and update
I thought about doing something like
JSONObject message = getMessageFromSender();
synchronized(message){
String id = message.getString("id");
MyObject myObject = getMyObject(id);
synchronized (myObject) {
doLotOfStuff(myObject);
} // well maybe this inner synchronized is superfluous, at this point
}
But I think that would kill the whole purpose of having a multithreaded structure, since I would read one message at a time, and the workers are not doing anything else; and it would be like if I was using a SynchronizedHashMap instead of a ConcurrentHashMap.
For the record, I report here the solution I implemented eventually. I'm not sure it is optimal and I still have to test for performances, but at least the input is handed properly.
public class Parent implements Runnable {
private final static int NUM_WORKERS = 10;
ExecutorService executor;
List<Future<?>> futures = new ArrayList<Future<?>>();
List<WorkerThread> workers = new ArrayList<WorkerThread>();
#Override
public void run() {
executor = Executors.newFixedThreadPool(NUM_WORKERS);
for (int i = 0; i < NUM_WORKERS; i++) {
WorkerThread worker = new WorkerThread("worker_" + i);
Future<?> future = executor.submit(worker);
futures.add(future);
workers.add(worker);
}
while(!isStopped()) {
byte[] message = getMessageFromSender();
byte[] id = getId(message);
int n = Integer.valueOf(Byte.toString(id[id.length-1])) % NUM_WORKERS;
if(n >= 0 && n <= (NUM_WORKERS-1)){
workers.get(n).addToQueue(line);
}
}
}
private class WorkerThread implements Runnable {
private String name;
private Map<String, MyObject> myObjects;
private LinkedBlockingQueue<byte[]> queue;
public WorkerThread(String name) {
this.name = name;
}
public void addToQueue(byte[] line) {
queue.add(line);
}
#Override
public void run() {
while(!isStopped()) {
byte[] message= queue.poll();
if(line != null) {
String id = getId(message);
MyObject myObject = getMyObject(id);
doLotOfStuff(myObject);
}
}
}
}
}

Conceptually this is kind of routing problem. What you need to is:
Get your your main thread (single thread) reading messages of the queue and push the data to a FIFO queue per id.
Get a single thread to consume messages from each queue.
Locking examples will (probably) not work as after the second message order is not guaranteed even if fair=true.
From Javadoc:
Even when this lock has been set to use a fair ordering policy, a call to tryLock() will immediately acquire the lock if it is available, whether or not other threads are currently waiting for the lock.
One thing for you to decide is if you want to create a a thread per queue (which will exit once the queue is empty) or keep the fixed size thread pool and manage get the extra bits to assign threads to queues.
So, you get a single thread reading from the original queue and writing to the per-id-queues and the you also get one thread per id reading from individual queues. This will ensure task serialization.
In terms of performance, you should see significant speed-up as long as the incoming messages have a nice distribution (id-wise). If you get mostly same-id messages then task will be serialized and also include the overhead for control object creation and synchronization.

You could use a separate Map for your locks. There's also a WeakHashMap that will automatically discard entries when the key is no longer present.
static final Map<String, Lock> locks = Collections.synchronizedMap(new WeakHashMap<>());
public void lock(String id) throws InterruptedException {
// Grab a Lock out of the map.
Lock l = locks.computeIfAbsent(id, k -> new ReentrantLock());
// Lock it.
l.lockInterruptibly();
}
public void unlock(String id) throws InterruptedException {
// Is it locked?
Lock l = locks.get(id);
if ( l != null ) {
l.unlock();
}
}

I think you have the right idea with your synchronized blocks, except you mis-analyze a bit and go too far in any case. The outer synchronized block shouldn't force you into dealing with only one message at a time, it just keeps multiple threads from accessing the same message at once. But you don't need it. You really only need that inner synchronized block, on the MyObject instance. That will ensure that only one thread at a time can access any given MyObject instance, while enabling other threads to access messages, the Map and other MyObject instances as much as they want.
JSONObject message = getMessageFromSender();
String id = message.getString("id");
MyObject myObject = getMyObject(id);
synchronized (myObject) {
doLotOfStuff(myObject);
}
If you don't like that, and the updates to the MyObject instances all involve single-method invocations, then you could just synchronize all of those methods. You still retain concurrency in the Map, but you're protecting the MyObject itself from concurrent updates.
class MyObject {
public synchronize void updateFoo() {
// ...
}
public synchronize void updateBar() {
// ...
}
}
When any Thread accesses any updateX() method it will automatically lock out any other Thread from accessing that or any other synchronized method. That would be simplest, if your updates match that pattern.
If not, then you'll need to make all of your worker Threads cooperate by using some sort of locking protocol. The ReentrantLock that OldCurmudgeon suggests is a good choice, but I would put it on MyObject itself. To keep things ordered properly, you should use the fairness parameter (see http://docs.oracle.com/javase/8/docs/api/java/util/concurrent/locks/ReentrantLock.html#ReentrantLock-boolean-). "When set true, under contention, locks favor granting access to the longest-waiting thread."
class MyObject {
private final ReentrantLock lock = new ReentrantLock(true);
public void lock() {
lock.lock();
}
public void unlock() {
lock.unlock();
}
public void updateFoo() {
// ...
}
public void updateBar() {
// ...
}
}
Then you could update things like this:
JSONObject message = getMessageFromSender();
String id = message.getString("id");
MyObject myObject = getMyObject(id);
myObject.lock();
try {
doLotOfStuff(myObject);
}
finally {
myObject.unlock();
}
The important takeaway is that you don't need to control access to the messages, nor the Map. All you need to do is ensure that any given MyObject is being updated by at most one thread at a time.

Actually here is a design idea: when a consumer takes a request to work on your Object it should actually remove the object with that ID from your list of Objects and then re-insert it back once the processing is done. Then any other consumer getting request to work on the object with the same id should be in blocking mode waiting for the object with that ID to re-appear in your list. You will need to add a management to keep record of all existing objects so when you can distinguish between the object that exists already but is not currently in the list (i.e. being processed by some other consumer) and the object that does not exist yet.

You could get some speedup if you split up the JSON parsing from the doLotsOfStuff(). One thread listens for messages, parses them, then puts the parsed message on a Queue to maintain chronological order. A second thread reads from that Queue and doesLotsOfStuff with no need for locking.
However, since you apparently need more than a 2X speedup this is probably insufficient.
Added
Another possibility is multiple HashMaps. For example, if all the IDs are ints, make 10 HashMaps for IDs ending with 0,1,2... Incoming messages get directed to one of 10 threads, which parse the JSON and update their relevant Map. Order is maintained within each Map, and there are no locking or contention issues. Assuming the message IDs are randomly distributed this yields up to a 10x speedup, though there is one extra layer of overhead to get at your Map. e.g.
Thread JSON Threads 0-9
--------------------------------------------------------------
while (notInterrupted) {
read / parse next JSON message
mapToUse = ID % 10
pass JSON to that Thread's queue
}
while (notInterrupted) {
take JSON off queue
// I'm the only one with writing to Map#N
do computation and update ID
}

Java concurrency counter not properly clean up

This is a java concurrency question. 10 jobs need to be done, each of them will have 32 worker threads. Worker thread will increase a counter . Once the counter is 32, it means this job is done and then clean up counter map. From the console output, I expect that 10 "done" will be output, pool size is 0 and counterThread size is 0.
The issues are :
most of time, "pool size: 0 and countThreadMap size:3" will be
printed out. even those all threads are gone, but 3 jobs are not
finished yet.
some time, I can see nullpointerexception in line 27. I have used ConcurrentHashMap and AtomicLong, why still have concurrency
exception.
Thanks
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.atomic.AtomicLong;
public class Test {
final ConcurrentHashMap<Long, AtomicLong[]> countThreadMap = new ConcurrentHashMap<Long, AtomicLong[]>();
final ExecutorService cachedThreadPool = Executors.newCachedThreadPool();
final ThreadPoolExecutor tPoolExecutor = ((ThreadPoolExecutor) cachedThreadPool);
public void doJob(final Long batchIterationTime) {
for (int i = 0; i < 32; i++) {
Thread workerThread = new Thread(new Runnable() {
#Override
public void run() {
if (countThreadMap.get(batchIterationTime) == null) {
AtomicLong[] atomicThreadCountArr = new AtomicLong[2];
atomicThreadCountArr[0] = new AtomicLong(1);
atomicThreadCountArr[1] = new AtomicLong(System.currentTimeMillis()); //start up time
countThreadMap.put(batchIterationTime, atomicThreadCountArr);
} else {
AtomicLong[] atomicThreadCountArr = countThreadMap.get(batchIterationTime);
atomicThreadCountArr[0].getAndAdd(1);
countThreadMap.put(batchIterationTime, atomicThreadCountArr);
}
if (countThreadMap.get(batchIterationTime)[0].get() == 32) {
System.out.println("done");
countThreadMap.remove(batchIterationTime);
}
}
});
tPoolExecutor.execute(workerThread);
}
}
public void report(){
while(tPoolExecutor.getActiveCount() != 0){
//
}
System.out.println("pool size: "+ tPoolExecutor.getActiveCount() + " and countThreadMap size:"+countThreadMap.size());
}
public static void main(String[] args) throws Exception {
Test test = new Test();
for (int i = 0; i < 10; i++) {
Long batchIterationTime = System.currentTimeMillis();
test.doJob(batchIterationTime);
}
test.report();
System.out.println("All Jobs are done");
}
}

Let’s dig through all the mistakes of thread related programming, one man can make:
Thread workerThread = new Thread(new Runnable() {
…
tPoolExecutor.execute(workerThread);
You create a Thread but don’t start it but submit it to an executor. It’s a historical mistake of the Java API to let Thread implement Runnable for no good reason. Now, every developer should be aware, that there is no reason to treat a Thread as a Runnable. If you don’t want to start a thread manually, don’t create a Thread. Just create the Runnable and pass it to execute or submit.
I want to emphasize the latter as it returns a Future which gives you for free what you are attempting to implement: the information when a task has been finished. It’s even easier when using invokeAll which will submit a bunch of Callables and return when all are done. Since you didn’t tell us anything about your actual task, it’s not clear whether you can let your tasks simply implement Callable (may return null) instead of Runnable.
If you can’t use Callables or don’t want to wait immediately on submission, you have to remember the returned Futures and query them at a later time:
static final ExecutorService cachedThreadPool = Executors.newCachedThreadPool();
public static List<Future<?>> doJob(final Long batchIterationTime) {
final Random r=new Random();
List<Future<?>> list=new ArrayList<>(32);
for (int i = 0; i < 32; i++) {
Runnable job=new Runnable() {
public void run() {
// pretend to do something
LockSupport.parkNanos(TimeUnit.SECONDS.toNanos(r.nextInt(10)));
}
};
list.add(cachedThreadPool.submit(job));
}
return list;
}
public static void main(String[] args) throws Exception {
Test test = new Test();
Map<Long,List<Future<?>>> map=new HashMap<>();
for (int i = 0; i < 10; i++) {
Long batchIterationTime = System.currentTimeMillis();
while(map.containsKey(batchIterationTime))
batchIterationTime++;
map.put(batchIterationTime,doJob(batchIterationTime));
}
// print some statistics, if you really need
int overAllDone=0, overallPending=0;
for(Map.Entry<Long,List<Future<?>>> e: map.entrySet()) {
int done=0, pending=0;
for(Future<?> f: e.getValue()) {
if(f.isDone()) done++;
else pending++;
}
System.out.println(e.getKey()+"\t"+done+" done, "+pending+" pending");
overAllDone+=done;
overallPending+=pending;
}
System.out.println("Total\t"+overAllDone+" done, "+overallPending+" pending");
// wait for the completion of all jobs
for(List<Future<?>> l: map.values())
for(Future<?> f: l)
f.get();
System.out.println("All Jobs are done");
}
But note that if you don’t need the ExecutorService for subsequent tasks, it’s much easier to wait for all jobs to complete:
cachedThreadPool.shutdown();
cachedThreadPool.awaitTermination(Long.MAX_VALUE, TimeUnit.DAYS);
System.out.println("All Jobs are done");
But regardless of how unnecessary the manual tracking of the job status is, let’s delve into your attempt, so you may avoid the mistakes in the future:
if (countThreadMap.get(batchIterationTime) == null) {
The ConcurrentMap is thread safe, but this does not turn your concurrent code into sequential one (that would render multi-threading useless). The above line might be processed by up to all 32 threads at the same time, all finding that the key does not exist yet so possibly more than one thread will then be going to put the initial value into the map.
AtomicLong[] atomicThreadCountArr = new AtomicLong[2];
atomicThreadCountArr[0] = new AtomicLong(1);
atomicThreadCountArr[1] = new AtomicLong(System.currentTimeMillis());
countThreadMap.put(batchIterationTime, atomicThreadCountArr);
That’s why this is called the “check-then-act” anti-pattern. If more than one thread is going to process that code, they all will put their new value, being confident that this was the right thing as they have checked the initial condition before acting but for all but one thread the condition has changed when acting and they are overwriting the value of a previous put operation.
} else {
AtomicLong[] atomicThreadCountArr = countThreadMap.get(batchIterationTime);
atomicThreadCountArr[0].getAndAdd(1);
countThreadMap.put(batchIterationTime, atomicThreadCountArr);
Since you are modifying the AtomicInteger which is already stored into the map, the put operation is useless, it will put the very array that it retrieved before. If there wasn’t the mistake that there can be multiple initial values as described above, the put operation had no effect.
}
if (countThreadMap.get(batchIterationTime)[0].get() == 32) {
Again, the use of a ConcurrentMap doesn’t turn the multi-threaded code into sequential code. While it is clear that the only last thread will update the atomic integer to 32 (when the initial race condition doesn’t materialize), it is not guaranteed that all other threads have already passed this if statement. Therefore more than one, up to all threads can still be at this point of execution and see the value of 32. Or…
System.out.println("done");
countThreadMap.remove(batchIterationTime);
One of the threads which have seen the 32 value might execute this remove operation. At this point, there might be still threads not having executed the above if statement, now not seeing the value 32 but producing a NullPointerException as the array supposed to contain the AtomicInteger is not in the map anymore. This is what happens, occasionally…

After creating your 10 jobs, your main thread is still running - it doesn't wait for your jobs to complete before it calls report on the test. You try to overcome this with the while loop, but tPoolExecutor.getActiveCount() is potentially coming out as 0 before the workerThread is executed, and then the countThreadMap.size() is happening after the threads were added to your HashMap.
There are a number of ways to fix this - but I will let another answer-er do that because I have to leave at the moment.

How to correctly use synchronized?

This piece of code:
synchronized (mList) {
if (mList.size() != 0) {
int s = mList.size() - 1;
for (int i = s; i > 0; i -= OFFSET) {
mList.get(i).doDraw(canv);
}
getHead().drawHead(canv);
}
}
Randomly throws AIOOBEs. From what I've read, the synchronized should prevent that, so what am I doing wrong?
Edits:
AIOOBE = Array Index Out Of Bounds Exception
The code's incomplete, cut down to what is needed. But to make you happy, OFFSET is 4, and just imagine that there is a for-loop adding a bit of data at the beginning. And a second thread reading and / or modifying the list.
Edit 2:
I've noticed it happens when the list is being drawn and the current game ends. The draw-thread hasn't drawn all elements when the list is emptied. Is there a way of telling the game to wait with emtying the list untill it's empty?
Edit 3:
I've just noticed that I'm not sure if this is a multi-threading problem. Seems I only have 2 threads, one for calculating and drawing and one for user input.. Gonna have to look into this a bit more than I thought.

What you're doing looks right... but that's all:
It doesn't matter on what object you synchronize, it needn't be the list itself.
What does matter is if all threads always synchronize on the same object, when accessing a shared resource.
Any access to SWING (or another graphic library) must happen in the AWT-Thread.
To your edit:
I've noticed it happens when the list is being drawn and the current game ends. The draw-thread hasn't drawn all elements when the list is emptied. Is there a way of telling the game to wait with emtying the list untill it's empty?
I think you mean "...wait with emptying the list until the drawing has completed." Just synchronize the code doing it on the same lock (i.e., the list itself in your case).
Again: Any access to a shared resource must be protected somehow. It seems like you're using synchronized just here and not where you're emptying the list.

The safe solution is to only allow one thread to create objects, add and remove them from a List after the game has started.
I had problems myself with random AIOOBEs erros and no synchornize could solve it properly plus it was slowing down the response of the user.
My solution, which is now stable and fast (never had an AIOOBEs since) is to make UI thread inform the game thread to create or manipulate an object by setting a flag and coordinates of the touch into the persistent variables.
Since the game thread loops about 60 times per second this proved to be sufficent to pick up the message from the UI thread and do something.
This is a very simple solution and it works great!

My suggestion is to use a BlockingQueue and I think you are looking for this solution also. How you can do it? It is already shown with an example in the javadoc :)
class Producer implements Runnable {
private final BlockingQueue queue;
Producer(BlockingQueue q) { queue = q; }
public void run() {
try {
while (true) { queue.put(produce()); }
} catch (InterruptedException ex) { ... handle ...}
}
Object produce() { ... }
}
class Consumer implements Runnable {
private final BlockingQueue queue;
Consumer(BlockingQueue q) { queue = q; }
public void run() {
try {
while (true) { consume(queue.take()); }
} catch (InterruptedException ex) { ... handle ...}
}
void consume(Object x) { ... }
}
class Setup {
void main() {
BlockingQueue q = new SomeQueueImplementation();
Producer p = new Producer(q);
Consumer c1 = new Consumer(q);
Consumer c2 = new Consumer(q);
new Thread(p).start();
new Thread(c1).start();
new Thread(c2).start();
}
}
The beneficial things for you are, you need not to worry about synchronizing your mList. BlockingQueue offers 10 special method. You can check it in the doc. Few from javadoc:
BlockingQueue methods come in four forms, with different ways of handling operations that cannot be satisfied immediately, but may be satisfied at some point in the future: one throws an exception, the second returns a special value (either null or false, depending on the operation), the third blocks the current thread indefinitely until the operation can succeed, and the fourth blocks for only a given maximum time limit before giving up.
To be in safe side: I am not experienced with android. So not certain whether all java packages are allowed in android. But at least it should be :-S, I wish.

You are getting Index out of Bounds Exception because there are 2 threads that operate on the list and are doing it wrongly.
You should have been synchronizing at another level, in such a way that no other thread can iterate through the list while other thread is modifying it! Only on thread at a time should 'work on' the list.
I guess you have the following situation:
//piece of code that adds some item in the list
synchronized(mList){
mList.add(1, drawableElem);
...
}
and
//code that iterates you list(your code simplified)
synchronized (mList) {
if (mList.size() != 0) {
int s = mList.size() - 1;
for (int i = s; i > 0; i -= OFFSET) {
mList.get(i).doDraw(canv);
}
getHead().drawHead(canv);
}
}
Individually the pieces of code look fine. They seam thread-safe. But 2 individual thread-safe pieces of code might not be thread safe at a higher level!
It's just you would have done the following:
Vector v = new Vector();
if(v.length() == 0){ v.length() itself is thread safe!
v.add("elem"); v.add() itself is also thread safe individually!
}
BUT the compound operation is NOT!
Regards,
Tiberiu

Producer-consumer problem with a twist

The producer is finite, as should be the consumer.
The problem is when to stop, not how to run.
Communication can happen over any type of BlockingQueue.
Can't rely on poisoning the queue(PriorityBlockingQueue)
Can't rely on locking the queue(SynchronousQueue)
Can't rely on offer/poll exclusively(SynchronousQueue)
Probably even more exotic queues in existence.
Creates a queued seq on another (presumably lazy) seq s. The queued
seq will produce a concrete seq in the background, and can get up to
n items ahead of the consumer. n-or-q can be an integer n buffer
size, or an instance of java.util.concurrent BlockingQueue. Note
that reading from a seque can block if the reader gets ahead of the
producer.
http://clojure.github.com/clojure/clojure.core-api.html#clojure.core/seque
My attempts so far + some tests: https://gist.github.com/934781
Solutions in Java or Clojure appreciated.

class Reader {
private final ExecutorService ex = Executors.newSingleThreadExecutor();
private final List<Object> completed = new ArrayList<Object>();
private final BlockingQueue<Object> doneQueue = new LinkedBlockingQueue<Object>();
private int pending = 0;
public synchronized Object take() {
removeDone();
queue();
Object rVal;
if(completed.isEmpty()) {
try {
rVal = doneQueue.take();
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
pending--;
} else {
rVal = completed.remove(0);
}
queue();
return rVal;
}
private void removeDone() {
Object current = doneQueue.poll();
while(current != null) {
completed.add(current);
pending--;
current = doneQueue.poll();
}
}
private void queue() {
while(pending < 10) {
pending++;
ex.submit(new Runnable() {
#Override
public void run() {
doneQueue.add(compute());
}
private Object compute() {
//do actual computation here
return new Object();
}
});
}
}
}

Not exactly an answer I'm afraid, but a few remarks and more questions. My first answer would be: use clojure.core/seque. The producer needs to communicate end-of-seq somehow for the consumer to know when to stop, and I assume the number of produced elements is not known in advance. Why can't you use an EOS marker (if that's what you mean by queue poisoning)?
If I understand your alternative seque implementation correctly, it will break when elements are taken off the queue outside your function, since channel and q will be out of step in that case: channel will hold more #(.take q) elements than there are elements in q, causing it to block. There might be ways to ensure channel and q are always in step, but that would probably require implementing your own Queue class, and it adds so much complexity that I doubt it's worth it.
Also, your implementation doesn't distinguish between normal EOS and abnormal queue termination due to thread interruption - depending on what you're using it for you might want to know which is which. Personally I don't like using exceptions in this way — use exceptions for exceptional situations, not for normal flow control.