Develop Reference

Queue isEmpty and poll incosistency?

Please see the following code segment:
private static Queue<Message> m_Queue;
public boolean isQueueEmpty()
{
if (m_Queue.isEmpty())
return true;
else
return false;
}
public WgwConferenceMessage dequeue(){
try{
if(!isQueueEmpty())
{
Message message = m_Queue.poll();
if (message != null)
{
if (!message.getMessage().equals(""))
Log4jWrapper.writeLog("Retrieved " + message.getMessage() + " from queue");
else
Log4jWrapper.writeLog(LogLevelEnum.ERROR, "<Queue> dequeue", "Message empty");
return message;
}
else
{
Log4jWrapper.writeLog(LogLevelEnum.TRACE, "<Queue> dequeue", " Q is empty!");
return null;
}
}
else
return null;
}
catch (Exception e)
{
ExceptionHandler.printException(e, "<Queue>", "dequeue");
return null;
}
}
public void enqueue(Message a_Message) throws Exception
{
try
{
if (m_Queue.offer(a_Message))
Log4jWrapper.writeLog(LogLevelEnum.TRACE, "<Queue> enqueue", "Pushed " + a_Message.getMessage() + " to queue");
else
throw new Exception("Queue - Could not push message to queue");
}
catch (Exception e)
{
ExceptionHandler.printException(e, "Queue", "enqueue");
}
}
My problem is that eventually I get " Q is empty!" log line.
And I can't understand how can it be?
isQueueEmpty() says the Q is not empty, and poll says it is!
Can you advice please?
Thank you.

Assuming this code is accessed by multiple threads, the reason is that the check for emptiness and the subsequent polling are not done atomically: they are two separate actions. This means that it is possible for a different thread to call poll on the queue in between the first thread checking if it is empty and calling poll itself; if there happens only to have been one element in the queue, one of these threads is going to get null back from the call to poll.
Quoting the Javadoc for Queue:
Queue implementations generally do not allow insertion of null elements, although some implementations, such as LinkedList, do not prohibit insertion of null. Even in the implementations that permit it, null should not be inserted into a Queue, as null is also used as a special return value by the poll method to indicate that the queue contains no elements.
This means that you should use the fact that null is returned by poll as an indication that the queue was empty - you don't need to do the calls separately.
poll may be atomic - depending on the implementation of Queue you are actually using:
If you're using a non-synchronized implementation like LinkedList, you should be synchronizing it anyway if multiple threads are modifying the list, making poll atomic;
Concurrent implementations like BlockingQueue implements poll atomically, so you don't need to worry about doing anything explicitly.
TL;DR:
Remove the !isQueueEmpty() check
Ensure that your poll method is atomic either by choosing a concurrent implementation, or by synchronizing mutations of the queue.

your initialization of Queue is as follows:
m_Queue = new LinkedList<Message>();
LinkedinList is an implementation of Queue which allows null to be added.
so basically you are adding null values into your m_Queue
And as #Andy mentioned, such implementation should not be used when using poll() method.
There are two ways to avoid that
before adding Message to m_Queue you can check if its null or not
new LinkedList<Message>(); to new ArrayDequeue<Message>(); where it throws Exception if you are adding null to your Queue
I prefer second one as it makes it as a Queue should really be.

In Java, multiple threads want to manipulate one object, how to get one manipulate that and the others wait until the work is done?

My problem is a little complicated:
I have a concurrent map, the threads want to visit and update the map, if two thread want to fetch the same entry of the map, one should first get the map, update it, and the other should wait until the entry has been updated successfully and then fetch the entry.
My original thought is that: I can use another concurrent map, same key with the targeting map and use a latch as its value.
My code is like:
private final ConcurrentMap<Long, List<RowKeyMap>> targetmap;
private final ConcurrentMap<Long, CountDownLatch> helpermap;
long keyMillis; //key
CountDownLatch restoreLatch = helpermap.get(keyMillis);
if (restoreLatch != null) {
try {
restoreLatch.await();
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
throw new RuntimeException("Interrupted trying to get " + keyMillis);
}
}
List<RowKeyMap> restoredata = targetmap.get(keyMillis);
if (restoredata == null) {
//find the entry should be restored, put a latch into the helpermap and restore it
restoreLatch = new CountDownLatch(1);
CountDownLatch existingLatch = helpermap.putIfAbsent(keyMillis, restoreLatch);
if (existingLatch == null) {
microshards = new ArrayList<>(count);
for (int i = 0; i < count; ++i) {
microshards.add(new RowKeyMap(some parameters));
}
List<RowKeyMap> existing = targetmap.putIfAbsent(keyMillis, microshards);
if (existing == null) {
{do actual restore job here}
} else {
microshards = existing;
}
restoreLatch.countDown();
restoresByDate.remove(keyMillis);
} else {
// Lost the race, wait for the restore task is complete and get the new restoredata
try {
existingLatch.await();
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
throw new RuntimeException("Interrupted trying to get " + keyMillis);
}
{get the new restoredata}
}
}
But the current version has a bug:
Thread A executes through first line, gets null for restoreLatch
Thread B wakes up and executes through first line, also gets null for
restoreLatch
Thread B continues on to following lines, sees existingLatch is null
Thread B continues, puts a created-but-not-yet-restored-into list
into restoredata
Thread A wakes up and executes through, get the
created-but-not-yet-restored-into list from restoredata
Anyone has any ideas to solve this? Thanks!

So what you want is a lock per each map entry. I'm not sure a CountDownLatch is ideal here, because it cannot be re-used, and creating a new one each time complicates your problem.
But your basic problem is that you are not preventing the race condition for the lock itself.
In order to do that, you must first ensure that a lock object for the entry exists, and that if two threads go to the same entry, they will get the same lock.
You can do this by first creating a lock object, then use putIfAbsent to put it in the lock map:
Object entryLock = new Object();
Object returnedLock = helpermap.putIfAbsent( keyMillis, entryLock );
entryLock = returnedLock == null ? entryLock : returnedLock;
What this does is ensure that any two threads that are trying to access the same entry (keyMillis) will get the same lock instance. If thread A is the first to run the putIfAbsent line, then the new object it created in the first line is going to be the one to be placed in the helper map, and it's going to get null back, which means it will also use the object it just placed in the map - entryLock. Thread B then comes along and create its own entryLock. But when it tries the putIfAbsent line, there is already an object mapped to keyMillis, returnedLock, and that's the object it will use (in this case, the original new lock it created will be discarded to the garbage collection).
So whichever order thy get to the putIfAbsent line, they will be using the same lock instance. Now the next step is:
Lock the lock.
Run your processing of the data in the targetMap, creating it if it doesn't exist, updating it if it does, etc. All this time, other threads for this particular keyMillis are waiting, but threads with other keyMillis don't.
Unlock the lock. One of the other threads that wait for this keyMillis entry will now lock the lock.
To do this is pretty simple:
synchronized(entryLock) {
// All operations on the particular entry
}
If you need fancier lock facilities then use ReentrantLock or a CyclicBarrier. A CountDownLatch will need to be replaced with a new one to be usable, and that would defeat the arrangement above, which pretty much relies on the lock object being the same for all threads.

My problem is a little complicated...
Uh oh, I've got bad news for you. If you think that problem is complicated.... You actually have identified the simplest problem in multi-threaded program. It's called mutual exclusion.
You can do it like this in Java:
final Object lock = new Object();
synchronized (lock) {
// some protected statements
}
synchronized (lock) {
// some more protected statements
}
The JVM guarantees that no more than one thread can be in a synchronized(foo) block for the same object foo at the same time.

use the synchronized either at the method declaration or inside a block of code.
see here : https://docs.oracle.com/javase/tutorial/essential/concurrency/syncmeth.html

Use objects that implement the Lock interface. Java has several of these objects by default that you can use. Here is a tutorial that explains what locks are and how to use them.

Waiting for an object to be initialized

I have an object that is being initialized in a separate thread. Initialization can take several seconds while a local DB is being populated.
SpecialAnalysis currentAnalysis = new SpecialAnalysis(params_here);
I'm trying to implement a "cancel" button, that sets the object's isCancelled boolean to true. What is the proper Java way to implement this?
while (currentAnalysis == null) {
}
currentAnalysis.cancel();
This method freezes the program as it appears to have entered a computationally inefficient loop. Is this a case where I could use Object.wait()?
My current bad/semi-successful solution is:
while (currentAnalysis == null) {
Thread.sleep(500);
}
currentAnalysis.cancel();
Thanks!

Firstly, yes Object.wait() and Object.notify() / Object.notifyAll() are what you need. Whether or not you use them directly is a different matter. Due to the ease of making mistakes programming directly with wait/notify it is generally recommended to use the concurrency tools added in Java 1.5 (see second approach below).
The traditional wait/notify approach:
Initialisation:
synchronized (lockObject) {
SpecialAnalysis currentAnalysis = new SpecialAnalysis(params_here);
lockObject.notifyAll();
}
In the 'cancel' thread:
synchronized (lockObject) {
while (currentAnalysis == null) {
try { lockObject.wait(); }
catch Exception(e) { } // FIXME: ignores exception
}
}
currentAnalysis.cancel();
Of course these could be synchronized methods instead of blocks. Your choice of lockObject will depend on how many 'cancel' threads you need etc. In theory it could be anything, i.e. Object lockObject = new Object(); as long as you are careful the correct threads have access to it.
Note that it is important to put the call to wait() in a while loop here due to the possibility of spurious wakeups coming from the underlying OS.
A simpler approach would be to use a CountDownLatch, sparing you from the nuts and bolts of wait()&notify():
(I'm making a couple of assumptions here in order to suggest a possibly cleaner approach).
class AnalysisInitialiser extends Thread {
private CountDownLatch cancelLatch = new CountDownLatch(1);
private SpecialAnalysis analysis = null;
#Override
public void run() {
analysis = new SpecialAnalysis(params);
cancelLatch.countDown();
}
public SpecialAnalysis getAnalysis() {
cancelLatch.await();
return analysis;
}
}
Then in the thread that needs to send the cancel signal: (obviously you need to get hold of the AnalysisInitialiser object in some way)
analysisInit.getAnalysis.cancel();
No concurrency primitive boilerplate, yay!

i like this question so voted up..
you can do like below
do {
if(currentAnalysis != null){
currentAnalysis.cancel();
}
}
while (currentAnalysis == null)
here your do keeps checking the value of currentAnalysis and once its not null then it performs cancel else keeps looping and checking currentAnalysis value.
this is one better approach i am finding right now

java.util.ConcurrentModificationException: Unexpected List modification while multithreading?

I'm using multithreading to process a List of Strings in batches, however I'm getting this error when the Runnable task is iterating over the List to process each String.
For example the code roughly follows this structure:
public class RunnableTask implements Runnable {
private List<String> batch;
RunnableTask(List<String> batch){
this.batch = batch;
}
#Override
public void run() {
for(String record : batch){
entry = record.split(",");
m = regex.matcher(entry[7]);
if (m.find() && m.group(3) != null){
currentKey = m.group(3).trim();
currentValue = Integer.parseInt(entry[4]);
if ( resultMap.get(currentKey) == null ){
resultMap.put(currentKey, currentValue);
} else {
resultMap.put(currentKey, resultMap.get(currentKey) + currentValue);
}
}
}
}
}
Where the thread that is passing these batches for processing never modifies "batch" and NO CHANGES to batch are made inside the for loop. I understand that this exception ConcurrentModificationException is due to modifying the List during iteration but as far as I can tell that isn't happening. Is there something I'm missing?
Any help is appreciated,
Thankyou!
UPDATE1: It seems instance-variables aren't thread safe. I attempted to use CopyOnWriteArrayList in place of the ArrayList but I received inconsistent results - suggesting that the full iteration doesn't complete before the list is modified in some way and not every element is being processed.
UPDATE2: Locking on the loop with sychronized and/or a reentrantlock both still give the same exception.
I need a way to pass Lists to Runnable tasks and iterate over those lists without new threads causing concurrency issues with that list.

I understand that this exception ConcurrentModificationException is due to modifying the List during iteration but as far as I can tell that isn't happening
Ok, consider what happens when you create a new thread, passing a reference to RunnableTask instance, initialized with a different list as constructor parameter? You just changed the list reference to point to different list. And consider what happens when at the same time, a different thread inside the run() method, is changing the list, at any point. This will at some point of time, throw ConcurrentModificationException.
Instance Variables are not Thread-Safe.

Try this in your code:
public void run() {
for(String record : new ArrayList(batch)){
//do processing with record
}
}
There is a sort of problem with all your threads processing the list (is the list modified during the process?) but is difficult to tell with the code you're providing

Problem is due to multiple thread concurrently modifying the the source List structure. What I would suggest you should devide the source list to new sublist(according to size) and pass that list to threads.
Say your source List have 100 elements. and you are running 5 concurrent thread.
int index = 0;
List<TObject> tempList = new ArrayList<>();
for(TObject obj:srcList){
if(i==(srcList.size()/numberOfthread)){
RunnableTask task = new RunnableTask(tempList);
tempList = new ArrayList<>();
}else
tempList.add(obj);
}
In this case your original list would not be modified.

you need to lock the list before accessing its elements. because List is not thread safe. Try this
public void run() {
synchronizd(batch){
for(String record : batch){//do processing with record}
}
}

yes you are getting ConcurrentModificationException because your List is getting modified during iteration. If performance is not a critical issue I suggest use synchronization.
public class RunnableTask implements Runnable {
private List<String> batch = new ArrayList<String>();
RunnableTask(List<String> batch){
this.batch = batch;
}
public void run() {
synchronized (batch) {
for(String record : batch){//do processing with record}
}
}
}
}
or even better use ReentrantLock.

Your followups indicate that you are trying to reuse the same List multiple times. Your caller must create a new List for each Runnable.

Obviously someone else is changing the content of the list, which is out of picture of the code you mentioned. (If you are sure that the ConcurrentModificationException is complaining for the batch list, but not resultMap, and you are actually showing all code in RunnableTask)
Try to search in your code, for places that is updating the content of the list, check if it is possible concurrently with your RunnableTask.
Simply synchronizing in the RunnableTask is not going to help, you need to synchronize all access to the list, which is obviously happening somewhere else.
If performance is an issue to you so that you cannot synchronize on the batch list (which prohibit multiple RunnableTask to execute concurrently), consider making use of ReaderWriterLock: RunnableTask acquires read lock, while the list update logic acquire the write lock.

How to correctly use synchronized?

This piece of code:
synchronized (mList) {
if (mList.size() != 0) {
int s = mList.size() - 1;
for (int i = s; i > 0; i -= OFFSET) {
mList.get(i).doDraw(canv);
}
getHead().drawHead(canv);
}
}
Randomly throws AIOOBEs. From what I've read, the synchronized should prevent that, so what am I doing wrong?
Edits:
AIOOBE = Array Index Out Of Bounds Exception
The code's incomplete, cut down to what is needed. But to make you happy, OFFSET is 4, and just imagine that there is a for-loop adding a bit of data at the beginning. And a second thread reading and / or modifying the list.
Edit 2:
I've noticed it happens when the list is being drawn and the current game ends. The draw-thread hasn't drawn all elements when the list is emptied. Is there a way of telling the game to wait with emtying the list untill it's empty?
Edit 3:
I've just noticed that I'm not sure if this is a multi-threading problem. Seems I only have 2 threads, one for calculating and drawing and one for user input.. Gonna have to look into this a bit more than I thought.

What you're doing looks right... but that's all:
It doesn't matter on what object you synchronize, it needn't be the list itself.
What does matter is if all threads always synchronize on the same object, when accessing a shared resource.
Any access to SWING (or another graphic library) must happen in the AWT-Thread.
To your edit:
I've noticed it happens when the list is being drawn and the current game ends. The draw-thread hasn't drawn all elements when the list is emptied. Is there a way of telling the game to wait with emtying the list untill it's empty?
I think you mean "...wait with emptying the list until the drawing has completed." Just synchronize the code doing it on the same lock (i.e., the list itself in your case).
Again: Any access to a shared resource must be protected somehow. It seems like you're using synchronized just here and not where you're emptying the list.

The safe solution is to only allow one thread to create objects, add and remove them from a List after the game has started.
I had problems myself with random AIOOBEs erros and no synchornize could solve it properly plus it was slowing down the response of the user.
My solution, which is now stable and fast (never had an AIOOBEs since) is to make UI thread inform the game thread to create or manipulate an object by setting a flag and coordinates of the touch into the persistent variables.
Since the game thread loops about 60 times per second this proved to be sufficent to pick up the message from the UI thread and do something.
This is a very simple solution and it works great!

My suggestion is to use a BlockingQueue and I think you are looking for this solution also. How you can do it? It is already shown with an example in the javadoc :)
class Producer implements Runnable {
private final BlockingQueue queue;
Producer(BlockingQueue q) { queue = q; }
public void run() {
try {
while (true) { queue.put(produce()); }
} catch (InterruptedException ex) { ... handle ...}
}
Object produce() { ... }
}
class Consumer implements Runnable {
private final BlockingQueue queue;
Consumer(BlockingQueue q) { queue = q; }
public void run() {
try {
while (true) { consume(queue.take()); }
} catch (InterruptedException ex) { ... handle ...}
}
void consume(Object x) { ... }
}
class Setup {
void main() {
BlockingQueue q = new SomeQueueImplementation();
Producer p = new Producer(q);
Consumer c1 = new Consumer(q);
Consumer c2 = new Consumer(q);
new Thread(p).start();
new Thread(c1).start();
new Thread(c2).start();
}
}
The beneficial things for you are, you need not to worry about synchronizing your mList. BlockingQueue offers 10 special method. You can check it in the doc. Few from javadoc:
BlockingQueue methods come in four forms, with different ways of handling operations that cannot be satisfied immediately, but may be satisfied at some point in the future: one throws an exception, the second returns a special value (either null or false, depending on the operation), the third blocks the current thread indefinitely until the operation can succeed, and the fourth blocks for only a given maximum time limit before giving up.
To be in safe side: I am not experienced with android. So not certain whether all java packages are allowed in android. But at least it should be :-S, I wish.

You are getting Index out of Bounds Exception because there are 2 threads that operate on the list and are doing it wrongly.
You should have been synchronizing at another level, in such a way that no other thread can iterate through the list while other thread is modifying it! Only on thread at a time should 'work on' the list.
I guess you have the following situation:
//piece of code that adds some item in the list
synchronized(mList){
mList.add(1, drawableElem);
...
}
and
//code that iterates you list(your code simplified)
synchronized (mList) {
if (mList.size() != 0) {
int s = mList.size() - 1;
for (int i = s; i > 0; i -= OFFSET) {
mList.get(i).doDraw(canv);
}
getHead().drawHead(canv);
}
}
Individually the pieces of code look fine. They seam thread-safe. But 2 individual thread-safe pieces of code might not be thread safe at a higher level!
It's just you would have done the following:
Vector v = new Vector();
if(v.length() == 0){ v.length() itself is thread safe!
v.add("elem"); v.add() itself is also thread safe individually!
}
BUT the compound operation is NOT!
Regards,
Tiberiu