I have an ArrayList which is accessed by multiple threads. Main thread only clears the list and others add to it. I don't want the list get cleared while another thread is adding item to it. I want the list is locked while thread is adding item to it.
this is the code of adding thread:
synchronized (items)
{
int length = jsonArray.length();
if ((length > 0)) for (int i = 0; i < length; i++)
{
items.add(new Item(jsonArray.getJSONObject(i)));
}
}
But I don't use synchronized block for clearing. Is synchronized block necessary for clear too?
A quick way to handle this is to just use
List<Foo> items = Collections.synchronizedList(new ArrayList<Foo>());
All the methods will be synchronized and you'll be safe.
If anyone comments here something about performance, provide actual data of the OP's scenario to back up your claims.
I don't want the list get cleared while another thread is adding item to it.
Then yes, you need to have the clear routine synchronized as well.
In the below sample, the synchronized statement in add does not block clear.
public void add(T t) {
synchronized(items) {
items.add(t);
}
}
public void clear() {
items.clear();
}
Related
I'm using multithreading to process a List of Strings in batches, however I'm getting this error when the Runnable task is iterating over the List to process each String.
For example the code roughly follows this structure:
public class RunnableTask implements Runnable {
private List<String> batch;
RunnableTask(List<String> batch){
this.batch = batch;
}
#Override
public void run() {
for(String record : batch){
entry = record.split(",");
m = regex.matcher(entry[7]);
if (m.find() && m.group(3) != null){
currentKey = m.group(3).trim();
currentValue = Integer.parseInt(entry[4]);
if ( resultMap.get(currentKey) == null ){
resultMap.put(currentKey, currentValue);
} else {
resultMap.put(currentKey, resultMap.get(currentKey) + currentValue);
}
}
}
}
}
Where the thread that is passing these batches for processing never modifies "batch" and NO CHANGES to batch are made inside the for loop. I understand that this exception ConcurrentModificationException is due to modifying the List during iteration but as far as I can tell that isn't happening. Is there something I'm missing?
Any help is appreciated,
Thankyou!
UPDATE1: It seems instance-variables aren't thread safe. I attempted to use CopyOnWriteArrayList in place of the ArrayList but I received inconsistent results - suggesting that the full iteration doesn't complete before the list is modified in some way and not every element is being processed.
UPDATE2: Locking on the loop with sychronized and/or a reentrantlock both still give the same exception.
I need a way to pass Lists to Runnable tasks and iterate over those lists without new threads causing concurrency issues with that list.
I understand that this exception ConcurrentModificationException is due to modifying the List during iteration but as far as I can tell that isn't happening
Ok, consider what happens when you create a new thread, passing a reference to RunnableTask instance, initialized with a different list as constructor parameter? You just changed the list reference to point to different list. And consider what happens when at the same time, a different thread inside the run() method, is changing the list, at any point. This will at some point of time, throw ConcurrentModificationException.
Instance Variables are not Thread-Safe.
Try this in your code:
public void run() {
for(String record : new ArrayList(batch)){
//do processing with record
}
}
There is a sort of problem with all your threads processing the list (is the list modified during the process?) but is difficult to tell with the code you're providing
Problem is due to multiple thread concurrently modifying the the source List structure. What I would suggest you should devide the source list to new sublist(according to size) and pass that list to threads.
Say your source List have 100 elements. and you are running 5 concurrent thread.
int index = 0;
List<TObject> tempList = new ArrayList<>();
for(TObject obj:srcList){
if(i==(srcList.size()/numberOfthread)){
RunnableTask task = new RunnableTask(tempList);
tempList = new ArrayList<>();
}else
tempList.add(obj);
}
In this case your original list would not be modified.
you need to lock the list before accessing its elements. because List is not thread safe. Try this
public void run() {
synchronizd(batch){
for(String record : batch){//do processing with record}
}
}
yes you are getting ConcurrentModificationException because your List is getting modified during iteration. If performance is not a critical issue I suggest use synchronization.
public class RunnableTask implements Runnable {
private List<String> batch = new ArrayList<String>();
RunnableTask(List<String> batch){
this.batch = batch;
}
public void run() {
synchronized (batch) {
for(String record : batch){//do processing with record}
}
}
}
}
or even better use ReentrantLock.
Your followups indicate that you are trying to reuse the same List multiple times. Your caller must create a new List for each Runnable.
Obviously someone else is changing the content of the list, which is out of picture of the code you mentioned. (If you are sure that the ConcurrentModificationException is complaining for the batch list, but not resultMap, and you are actually showing all code in RunnableTask)
Try to search in your code, for places that is updating the content of the list, check if it is possible concurrently with your RunnableTask.
Simply synchronizing in the RunnableTask is not going to help, you need to synchronize all access to the list, which is obviously happening somewhere else.
If performance is an issue to you so that you cannot synchronize on the batch list (which prohibit multiple RunnableTask to execute concurrently), consider making use of ReaderWriterLock: RunnableTask acquires read lock, while the list update logic acquire the write lock.
While using Java Threading Primitives to construct a thread safe bounded queue - whats the difference between these 2 constructs
Creating an explicit lock object.
Using the list as the lock and waiting on it.
Example of 1
private final Object lock = new Object();
private ArrayList<String> list = new ArrayList<String>();
public String dequeue() {
synchronized (lock) {
while (list.size() == 0) {
lock.wait();
}
String value = list.remove(0);
lock.notifyAll();
return value;
}
}
public void enqueue(String value) {
synchronized (lock) {
while (list.size() == maxSize) {
lock.wait();
}
list.add(value);
lock.notifyAll();
}
}
Example of 2
private ArrayList<String> list = new ArrayList<String>();
public String dequeue() {
synchronized (list) { // lock on list
while (list.size() == 0) {
list.wait(); // wait on list
}
String value = list.remove(0);
list.notifyAll();
return value;
}
}
public void enqueue(String value) {
synchronized (list) { // lock on list
while (list.size() == maxSize) {
list.wait(); // wait on list
}
list.add(value);
list.notifyAll();
}
}
Note
This is a bounded list
No other operation is being performed apart from enqueue and dequeue.
I could use a blocking queue, but this question is more for improving my limited knowledge of threading.
If this question is repeated please let me know.
The short answer is, no, there is no functional difference, other than the extra memory overhead of maintaining that extra lock object. However, there are a couple of semantics-related items I would consider before making a final decision.
Will I ever need to perform synchronized operations on more than just my internal list?
Let's say you wanted to maintain a parallel data structure to your ArrayList, such that all operations on the list and that parallel data structure needed to be synchronized. In this case, it might be best to use the external lock, as locking on either the list or the structure might be confusing to future development efforts on this class.
Will I ever give access to my list outside of my queue class?
Let's say you wanted to provide an accessor method for your list, or make it visible to extensions of your Queue class. If you were using an external lock object, classes that retrieved references to the list would never be able to perform thread-safe operations on that list. In that case, it'd be better to synchronize on the list and make it clear in the API that external accesses/modifications to the list must also synchronize on that list.
I'm sure there are more reasons why you might choose one over the other, but these are the two big ones I can think of.
I asked here a question about iterating over a Vector, and I have been answered with some good solutions. But I read about another simpler way to do it. I would like to know if it is good solution.
synchronized(mapItems) {
Iterator<MapItem> iterator = mapItems.iterator();
while(iterator.hasNext())
iterator.next().draw(g);
}
mapItems is a synchronized collection: Vector. Is that make the iterating over the Vector safe from ConcurrentModificationException?
Yes, it will make it safe from ConcurrentModificationException at the expense of everything essentially being single-threaded.
Yes, I believe that this will prevent a ConcurrentModificationException. You are synchronizing on the Vector. All methods on Vector that modify it are also synchronized, which means that they would also lock on that same object. So no other thread could change the Vector while you're iterating over it.
Also, you are not modifying the Vector yourself while you're iterating over it.
Simply synchronizing the entire collection would not prevent a ConcurrentModificationException. This will still throw a CME
synchronized(mapItems) {
for(MapItem item : mapsItems){
mapItems.add(new MapItem());
}
}
You may want to consider using a ReadWriteLock.
For processes which iterate over the list without modifying its contents, get a read lock on the shared ReentrantReadWriteLock. This allows multiple threads to have read access to the lock.
For processes which will modify the list, acquire the write lock on the shared lock. This will prevent all other threads from accessing the list (even read-only) until you release the write lock.
Is that make the iterating over the Vector safe from
ConcurrentModificationException?
YES It makes the iterating over Vector safe from ConcurrentModificationException.If it is not synchronized then in that case , if you are accessing the Vector via various threads and some other Thread is structurally modifying the Vector at any time after the iterator is created , the iterator will throw ConcurrentModificationException.
Consider running this code:
import java.util.*;
class VVector
{
static Vector<Integer> mapItems = new Vector<Integer>();
static
{
for (int i = 0 ; i < 200 ; i++)
{
mapItems.add(i);
}
}
public static void readVector()
{
Iterator<Integer> iterator = mapItems.iterator();
try
{
while(iterator.hasNext())
{
System.out.print(iterator.next() + "\t");
}
}
catch (Exception ex){ex.printStackTrace();System.exit(0);}
}
public static void main(String[] args)
{
VVector v = new VVector();
Thread th = new Thread( new Runnable()
{
public void run()
{
int counter = 0;
while ( true )
{
mapItems.add(345);
counter++;
if (counter == 100)
{
break;
}
}
}
});
th.start();
v.readVector();
}
}
At my system it is showing following output while execution:
0 1 2 3 4 5 6 7 8 9
java.util.ConcurrentModificationException
at java.util.AbstractList$Itr.checkForComodification(Unknown Source)
at java.util.AbstractList$Itr.next(Unknown Source)
at VVector.readVector(VVector.java:19)
at VVector.main(VVector.java:38)
But on the other hand if you make the block of code containing Iterator to access that Vector synchronized using mapItems as lock , it will prevent the execution of other methods related to Vector until that synchronized block is completed atomically .
if we invoke add method inside while loop then throws exception.
synchronized(mapItems) {
Iterator<MapItem> iterator = mapItems.iterator();
while(iterator.hasNext())
iterator.next();
mapItems.add("Something"); // throws ConcurrentModificationException
}
This piece of code:
synchronized (mList) {
if (mList.size() != 0) {
int s = mList.size() - 1;
for (int i = s; i > 0; i -= OFFSET) {
mList.get(i).doDraw(canv);
}
getHead().drawHead(canv);
}
}
Randomly throws AIOOBEs. From what I've read, the synchronized should prevent that, so what am I doing wrong?
Edits:
AIOOBE = Array Index Out Of Bounds Exception
The code's incomplete, cut down to what is needed. But to make you happy, OFFSET is 4, and just imagine that there is a for-loop adding a bit of data at the beginning. And a second thread reading and / or modifying the list.
Edit 2:
I've noticed it happens when the list is being drawn and the current game ends. The draw-thread hasn't drawn all elements when the list is emptied. Is there a way of telling the game to wait with emtying the list untill it's empty?
Edit 3:
I've just noticed that I'm not sure if this is a multi-threading problem. Seems I only have 2 threads, one for calculating and drawing and one for user input.. Gonna have to look into this a bit more than I thought.
What you're doing looks right... but that's all:
It doesn't matter on what object you synchronize, it needn't be the list itself.
What does matter is if all threads always synchronize on the same object, when accessing a shared resource.
Any access to SWING (or another graphic library) must happen in the AWT-Thread.
To your edit:
I've noticed it happens when the list is being drawn and the current game ends. The draw-thread hasn't drawn all elements when the list is emptied. Is there a way of telling the game to wait with emtying the list untill it's empty?
I think you mean "...wait with emptying the list until the drawing has completed." Just synchronize the code doing it on the same lock (i.e., the list itself in your case).
Again: Any access to a shared resource must be protected somehow. It seems like you're using synchronized just here and not where you're emptying the list.
The safe solution is to only allow one thread to create objects, add and remove them from a List after the game has started.
I had problems myself with random AIOOBEs erros and no synchornize could solve it properly plus it was slowing down the response of the user.
My solution, which is now stable and fast (never had an AIOOBEs since) is to make UI thread inform the game thread to create or manipulate an object by setting a flag and coordinates of the touch into the persistent variables.
Since the game thread loops about 60 times per second this proved to be sufficent to pick up the message from the UI thread and do something.
This is a very simple solution and it works great!
My suggestion is to use a BlockingQueue and I think you are looking for this solution also. How you can do it? It is already shown with an example in the javadoc :)
class Producer implements Runnable {
private final BlockingQueue queue;
Producer(BlockingQueue q) { queue = q; }
public void run() {
try {
while (true) { queue.put(produce()); }
} catch (InterruptedException ex) { ... handle ...}
}
Object produce() { ... }
}
class Consumer implements Runnable {
private final BlockingQueue queue;
Consumer(BlockingQueue q) { queue = q; }
public void run() {
try {
while (true) { consume(queue.take()); }
} catch (InterruptedException ex) { ... handle ...}
}
void consume(Object x) { ... }
}
class Setup {
void main() {
BlockingQueue q = new SomeQueueImplementation();
Producer p = new Producer(q);
Consumer c1 = new Consumer(q);
Consumer c2 = new Consumer(q);
new Thread(p).start();
new Thread(c1).start();
new Thread(c2).start();
}
}
The beneficial things for you are, you need not to worry about synchronizing your mList. BlockingQueue offers 10 special method. You can check it in the doc. Few from javadoc:
BlockingQueue methods come in four forms, with different ways of handling operations that cannot be satisfied immediately, but may be satisfied at some point in the future: one throws an exception, the second returns a special value (either null or false, depending on the operation), the third blocks the current thread indefinitely until the operation can succeed, and the fourth blocks for only a given maximum time limit before giving up.
To be in safe side: I am not experienced with android. So not certain whether all java packages are allowed in android. But at least it should be :-S, I wish.
You are getting Index out of Bounds Exception because there are 2 threads that operate on the list and are doing it wrongly.
You should have been synchronizing at another level, in such a way that no other thread can iterate through the list while other thread is modifying it! Only on thread at a time should 'work on' the list.
I guess you have the following situation:
//piece of code that adds some item in the list
synchronized(mList){
mList.add(1, drawableElem);
...
}
and
//code that iterates you list(your code simplified)
synchronized (mList) {
if (mList.size() != 0) {
int s = mList.size() - 1;
for (int i = s; i > 0; i -= OFFSET) {
mList.get(i).doDraw(canv);
}
getHead().drawHead(canv);
}
}
Individually the pieces of code look fine. They seam thread-safe. But 2 individual thread-safe pieces of code might not be thread safe at a higher level!
It's just you would have done the following:
Vector v = new Vector();
if(v.length() == 0){ v.length() itself is thread safe!
v.add("elem"); v.add() itself is also thread safe individually!
}
BUT the compound operation is NOT!
Regards,
Tiberiu
I have written this piece of code
public class Test{
public static void main(String[] args) {
List<Integer> list = new ArrayList<Integer>();
for(int i = 1;i<= 4;i++){
new Thread(new TestTask(i, list)).start();
}
while(list.size() != 4){
// this while loop required so that all threads complete their work
}
System.out.println("List "+list);
}
}
class TestTask implements Runnable{
private int sequence;
private List<Integer> list;
public TestTask(int sequence, List<Integer> list) {
this.sequence = sequence;
this.list = list;
}
#Override
public void run() {
list.add(sequence);
}
}
This code works and prints all the four elements of list on my machine.
My question is that will this code always work. I think there might be a issue in this code when two/or more threads add element to this list at the same point. In that case it while loop will never end and code will fail.
Can anybody suggest a better way to do this? I am not very good at multithreading and don't know which concurrent collection i can use?
Thanks, Shekhar
Use this in order to get a real thread-safe list:
List<Integer> list = Collections.synchronizedList(new ArrayList<Integer>());
Depending on your usage, also a CopyOnWriteArrayList could be interesting for you. Precisly, when traversal operations vastly outnumber mutations on that list.
Afair, Lists are not thread-safe in Java, so you might get anything from working to crashing. Use synchronized access to the list in order to get a well-defined behaviour:
#Override
public void run() {
synchronized(list) {
list.add(sequence);
}
}
This way, access to the list is only possible for a single thread at a time.
Also you I'd use Thread.join() to wait for the threads to finish (you have to keep them in a separate list for doing that ...)
I think you need to do two things. First of all you need to join the threads. Because atm the other loop will sometimes run even if the threads are not completed.
You have to do it like this:
Threads threads[4] = new Thread[4];
for(int i = 1;i<= 4;i++){
threads[i] = new Thread(new TestTask(i, list));
threads[i].start();
}
// to wait that all threads finish..
for(int i = 1;i<= 4;i++){
threads[i].join();
}
while(list.size() != 4){
// this while loop required so that all threads complete their work
}
and you can make your ArrayList thread safe with packing it into:
List<Integer> list = Collections.synchronizedList(new ArrayList<Integer>());
No, there are no guarantees. The simplest solution would be to join with each thread.
Read up on wait() and notify() instead of having a busy-waiting while-loop.