Develop Reference

Java - synchronized ArrayList still ConcurrentModificationException

I got an final ArrayList<RoutingTableEntry> routingTable = new ArrayList<>(); which is accessed multiple times.
But I only get at one Point an ConcurrentModificationException which is in the following Thread:
Thread checkReplies = new Thread(() -> {
while (true) {
synchronized (routingTable) {
for (RoutingTableEntry entry : routingTable) { // throws it here
// do smth
}
}
// [...]
}
});
checkReplies.start();
It throws the exception at the loop even though the routingTable is already synchronized. This thread gets only executed once per class.
Any ideas?

There are two possibilities:
You have other code in the class that modifies routingTable, and doesn't use synchronized (routingTable) when doing so. So when the other code modifies the list during that iteration, you get the error.
You're modifying the list where you have the comment "do smth". Just because you have have the list synchronized, that doesn't mean you can modify it while looping through with its iterator. You can't (except through the iterator itself, which would mean you couldn't use the enhanced for loop). (Sometimes you get away with it because of the details of the ArrayList implementation, but other times you don't.)
Here's an example of #2 (live copy):
var routingTable = new ArrayList<String>();
routingTable.add("one");
routingTable.add("two");
routingTable.add("three");
synchronized (routingTable) {
for (String entry : routingTable) {
if (entry.equals("two")) {
routingTable.add("four");
}
}
}
That fails with JDK12's implementation of ArrayList (at least, probably others).
One key thing to understand is that synchronization and modifying the list during iteration are largely unrelated concepts. Synchronization (done properly) prevents multiple threads from accessing the list at the same time. But as you can see in the example above, just a single thread can cause a ConcurrentModificationException by modifying the list during the iteration. They only relate in that if you have one thread reading the list and another thread that may modify it, synchronization prevents the modification while the read is happening. Other than that, they're unrelated.
In a comment you've said:
i call a method which removes then
If you're removing the entry for the loop, you can do that via a list iterator's remove method:
for (var it = routingTable.listIterator(); it.hasNext; ) {
var entry = it.next();
if (/*...some condition...*/) {
it.remove(); // Removes the current entry
}
}
(There are also add and set operations.)

ConcurrentModificationException is not necessarily 'concurrent' in the sense of threading, it can be 'concurrent' in the sense of that you shall not directly modify a collection at the same time when you are iterating over it.
It is in the docs too, for a long time (excerpt from Java7: https://docs.oracle.com/javase/7/docs/api/java/util/ConcurrentModificationException.html)
Note that this exception does not always indicate that an object has been concurrently modified by a different thread. If a single thread issues a sequence of method invocations that violates the contract of an object, the object may throw this exception. For example, if a thread modifies a collection directly while it is iterating over the collection with a fail-fast iterator, the iterator will throw this exception.
And for(x:y) uses an iterator, which can easily end up being a 'fail-fast' one.

Starting from your comment in the original question, you are removing items in the routingTable while you are iterating.
You can't do it (also if the routingTable is synchronized)
for (RoutingTableEntry entry : routingTable) { // throws it here
// routingTable.remove(....);
}

AtomicReference<ArrayList> difficulties and alternatives

So in various programs that I have been writing for fun, I have come across concurrent modification exceptions.
In my naive attempt to solve this problem I used an Atomicinstead of some sort of concurrent collection. I am somewhat familiar with why this is casing errors. Essentially the individual elements of the ArrayList are not synchronized and can be modified at whim by different threads.
Could someone summarize for me
What errors occur when i try to make an atomic reference to a collection
What is a legitimate use case for an atomic reference to an Array or list
What are some better alternatives for storing instances for a game which can be used by multiple threads

Using AtomicReference to store an object such as a collection is not enough to make it thread safe. Indeed if the object that you put in the AtomicReference is not thread safe like an ArrayList for example, using it would still be unsafe if we have multiple threads trying to modify its state concurrently. So the good approach is still to put into your AtomicReference an immutable object such that its state cannot be modified by multiple threads anymore. In case of a collection you can for example use the methods of type Collections.unmodifiable* such as Collections.unmodifiableList(List) for the lists, in order to put into the AtomicReference the immutable version of your collection.
If you need thread safe collections, you should have a look to the classes in the package java.util.concurrent, you will find Collections natively thread safe. For example, if you mostly read and rarely modify your List, you can use the thread safe and efficient list CopyOnWriteArrayList.

I think you are confusing what a ConcurrentModification means...
The most common occurrence for this is when you iterate over a collection and modify it in the loop.
For instance if you do the following
public static void main(String[] args) {
List<String> l = new LinkedList<>();
for(int i=0; i < 100; i++) {
l.add("banana"+i);
}
for (String s : l) {
if("banana10".equals(s)) {
l.remove(s);
}
}
}
...this will give you a ConcurrentModificationException. Note, I have not spawned any threads.
The correct way to do the same is as follows:
public static void main(String[] args) {
List<String> l = new LinkedList<>();
for(int i=0; i < 100; i++) {
l.add("banana"+i);
}
for (Iterator<String> iterator = l.iterator(); iterator.hasNext();) {
String s = iterator.next();
if("banana10".equals(s)) {
iterator.remove();
}
}
}
Note the use of an iterator to modify the collection whilst you are looping over it.
So, I don't think you have a concurrency issue!
If you want to make your collection Thread safe, you need to look at the semantics of the thread safety. If you want to allow multiple threads to access the same collection, a ConcurrentList would be a good approach. If you want a list reference which is atomically set, as a whole, you can use an Atomic reference.

JAVA java.util.ConcurrentModificationException

I am trying to create simple JAVA program. But since last day, I am having this issue, I cannot fix.
public void receiveUpdate(ArrayList<Connection> connections) {
for(Connection senderConnection : connections) {
for(Connection receiverConnection : this.connections) {
if(senderConnection.getDestination() == receiverConnection.getDestination()) {
if(senderConnection.getDistance() + 1 < receiverConnection.getDistance()) {
senderConnection.setEnabled(false);
senderConnection.setDistance(senderConnection.getDistance() + 1);
this.connections.remove(receiverConnection);
this.connections.add(senderConnection);
}
}
}
senderConnection.setEnabled(false);
senderConnection.setDistance(senderConnection.getDistance() + 1);
this.connections.add(senderConnection);
}
}
In this method I always get error:
Exception in thread "main" java.util.ConcurrentModificationException
at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:859)
at java.util.ArrayList$Itr.next(ArrayList.java:831)
at router.Router.receiveUpdate(Router.java:56)
at router.Router.sendUpdate(Router.java:49)
at loader.Loader.notifyNetworkChanges(Loader.java:61)
at loader.Loader.main(Loader.java:102)
I have noticed, that exception doesn't appear if I comment block:
senderConnection.setEnabled(false);
senderConnection.setDistance(senderConnection.getDistance() + 1);
this.connections.add(senderConnection);
Can anyone tell me, where is the problem?

ConcurrentModificationException This exception may be thrown by methods that have detected concurrent modification of an object when such modification is not permissible.
The problem is here
this.connections.remove(receiverConnection);
This exception may be thrown by methods that have detected concurrent modification of an object when such modification is not permissible.
For example, it is not generally permissible for one thread to modify a Collection while another thread is iterating over it. In general, the results of the iteration are undefined under these circumstances. Some Iterator implementations (including those of all the general purpose collection implementations provided by the JRE) may choose to throw this exception if this behavior is detected. Iterators that do this are known as fail-fast iterators, as they fail quickly and cleanly, rather that risking arbitrary, non-deterministic behavior at an undetermined time in the future.
Note that this exception does not always indicate that an object has been concurrently modified by a different thread. If a single thread issues a sequence of method invocations that violates the contract of an object, the object may throw this exception. For example, if a thread modifies a collection directly while it is iterating over the collection with a fail-fast iterator, the iterator will throw this exception.
Check this out
To solve your problem you need to use Iterator and remove() method like this
Iterator<String> connectionsIterator = connections.iterator();
while (connectionsIterator.hasNext()) {
if (senderConnection.getDistance() + 1 < receiverConnection.getDistance()){
connectionsIterator.remove();
// add the connection to another list other than this.connections.add(senderConnection);
// then when you finish add them back the this.connections
// so you avoid modifying this.connections
}
}

You are actually modifying the collections, while iterating.
this.connections.remove(receiverConnection);
this.connections.add(senderConnection);
Use an Iterator and call remove():
Iterator<String> iter = connections.iterator();
while (iter.hasNext()) {
if (someCondition)
iter.remove();
}
Use Iterator remove method instead of ArrayList

If you are modifying (eg deleting from) the same list you are looping then you are bound to get the ConcurrentModificationException
You can either use an iterator as #ankur-singhal mentioned or make a deep copy of the list and remove from that copy. Then you can assign the reference of the original list to point to the new one that has the modifications.
Another "greedy" approach (but without using iterators) would be to keep all items to be removed in another new list and after ending your for loop then call removeAll(listOfConnectionsToBeRemoved), for example:
before your for loops
ArrayList<Connection> connectionsToBeRemoved = new ArrayList<Connection>();
inside your for loops
connectionsToBeRemoved.add(receiverConnection)
after ending your for loops
this.connections.removeAll(connectionsToBeRemoved)
*you may do the same for connections to be added. Keep them in a new list and then add them all to your connection list.

You are looping through this.connections and inside you are trying to remove from this.connections this will java.util.ConcurrentModificationException. Change your code to use iterator.

Java Enumeration vs Iterator

Enumeration doesn't throw ConcurrentModificationException , why?
See below code .
public static void main(String[] args) {
Vector<String> v=new Vector<String>();
v.add("Amit");
v.add("Raj");
v.add("Pathak");
v.add("Sumit");
v.add("Aron");
v.add("Trek");
Enumeration<String> en=v.elements();
while(en.hasMoreElements())
{
String value=(String) en.nextElement();
System.out.println(value);
v.remove(value);
}
}
It only prints :
Amit
Pathak
Aron
Why is this such behavior . Can we say that Enumerator is thread safe.
Edit: When working with Iterator it throws ConcurrentModificationException in single thread application.
public static void main(String[] args) {
Vector<String> v=new Vector<String>();
v.add("Amit");
v.add("Raj");
v.add("Pathak");
v.add("Sumit");
v.add("Aron");
v.add("Trek");
Iterator<String> it=v.iterator();
while(it.hasNext())
{
String value=(String) it.next();
System.out.println(value);
v.remove(value);
}
}
Please check.

Note that ConcurrentModificationException has nothing to do with concurrency in the sense of multithreading or thread safety. Some collections allow concurrent modifications, some don't. Normally you can find the answer in the docs. But concurrent doesn't mean concurrently by different threads. It means you can modify the collection while you iterate.
ConcurrentHashMap is a special case, as it is explicitly defined to be thread-safe AND editable while iterated (which I think is true for all thread-safe collections).
Anyway, as long as you're using a single thread to iterate and modify the collection, ConcurrentHashMap is the wrong solution to your problem. You are using the API wrongly. You should use Iterator.remove() to remove items. Alternatively, you can make a copy of the collection before you iterate and modify the original.
EDIT:
I don't know of any Enumeration that throws a ConcurrentModificationException. However, the behavior in case of a concurrent modification might not be what you expect it to be. As you see in you example, the enumeration skips every second element in the list. This is due to it's internal index being incremented regardless of removes. So this is what happens:
en.nextElement() - returns first element from Vector, increments index to 1
v.remove(value) - removes first element from Vector, shifts all elements left
en.nextElement() - returns second element from Vector, which now is "Pathak"
The fail-fast behavior of Iterator protects you from this kind of things, which is why it is generally preferable to Enumberation. Instead, you should do the following:
Iterator<String> it=v.iterator();
while(it.hasNext())
{
String value=(String) it.next();
System.out.println(value);
it.remove(); // not v.remove(value); !!
}
Alternatively:
for(String value : new Vector<String>(v)) // make a copy
{
String value=(String) it.next();
System.out.println(value);
v.remove(value);
}
The first one is certainly preferable, as you don't really need the copy as long as you use the API as it is intended.

Enumeration doesn't throw ConcurrentModificationException , why?
Because there is no path in the code being invoked which throws this exception. Edit: I am referring to the implementation provided by the Vector class, not about the Enumeration interface in general.
Why is this such behavior . Can we say that Enumerator is thread safe.
It is thread-safe in a sense that the code executed is properly synchronized. However, I don't think the result your loop yields is what you would except.
The reason for your output is that the Enumeration object maintains a counter, which is incremented after every invokation of nextElement(). This counter is not aware of your invokation of remove().

the fail-fast behaviour which throws the ConcurrentModificationException is only implemented for the Iterator as you can read in
http://docs.oracle.com/javase/6/docs/api/java/util/Vector.html

Concurrent modification here has nothing to do with threads.
Concurrency here simply means that you are modifying the collection while you are iterating over it. (In your example this happens in the same thread.)
Iterators and enumerations of collections can throw ConcurrentModificationException in this case, but don't have to. The ones that do exhibit fail-fast behaviour. Apparently, the enumeration of Vector isn't fail-fast.
Thread-safety obviously involves multiple threads somehow. Vector is thread-safe only in the sense that its operations (like add, get, etc.) are synchronized. This is to avoid non-deterministic behaviour when one thread is adding an element while at the same time another thread is trying to remove one for example.
Now, when one thread is structurally modifying your collection while another thread is iterating over it, you have to deal with both thread-safety and concurrent modification issues. In this case, and probably in general, it is safest not to rely on ConcurrentModificationException being thrown. It is best to choose the appropriate collection implementation (a thread-safe one for example) and avoid/disallow concurrent modification yourself.
Some iterators allow adding/setting/removing elements through the iterator itself. This can be a good alternative if you really need concurrent modification.

Short answer: It's a bonus feature that was invented after enumeration was already there, so the fact that the enumerator does not throw it does not suggest anything particular.
Long answer:
From wikipedia:
Collection implementations in pre-JDK 1.2 [...] did not contain a
collections framework. The standard methods for grouping Java
objects were via the array, the Vector, and the Hashtable classes,
which unfortunately were not easy to extend, and did not implement a
standard member interface. To address the need for reusable
collection data structures [...] The
collections framework was designed and developed primarily by Joshua
Bloch, and was introduced in JDK 1.2.
When the Bloch team did that, they thought it was a good idea to put in a new mechanism that sends out an alarm (ConcurrentModificationException) when their collections were not synchronized properly in a multi-threaded program. There are two important things to note about this mechanism: 1) it's not guaranteed to catch concurrency bugs -- the exception is only thrown if you are lucky. 2) The exception is also thrown if you misuse the collection using a single thread (as in your example).
So, a collection not throwing an ConcurrentModificationException when accessed by multiple threads does not imply it's thread-safe either.

It depends on how you get the Enumeration. See the following example, it does throw ConcurrentModificationException:
import java.util.*;
public class ConcurrencyTest {
public static void main(String[] args) {
Vector<String> v=new Vector<String>();
v.add("Amit");
v.add("Raj");
v.add("Pathak");
v.add("Sumit");
v.add("Aron");
v.add("Trek");
Enumeration<String> en=Collections.enumeration(v);//v.elements();
while(en.hasMoreElements())
{
String value=(String) en.nextElement();
System.out.println(value);
v.remove(value);
}
System.out.println("************************************");
Iterator<String> iter = v.iterator();
while(iter.hasNext()){
System.out.println(iter.next());
iter.remove();
System.out.println(v.size());
}
}
}
Enumeration is just an interface, it's actual behavior is implementation-dependent. The Enumeration from the Collections.enumeration() call wraps the iterator in someway, so it's indeed fail-fast, but the Enumeration obtained from calling the Vector.elements() is not.
The not-fail-fast Enumeration could introduce arbitrary, non-deterministic behavior at an undetermined time in the future. For example: if you write the main method as this, it will throw java.util.NoSuchElementException after the first iteration.
public static void main(String[] args) {
Vector<String> v=new Vector<String>();
v.add("Amit");
v.add("Raj");
v.add("Pathak");
Enumeration<String> en = v.elements(); //Collections.enumeration(v);
while(en.hasMoreElements())
{
v.remove(0);
String value=(String) en.nextElement();
System.out.println(value);
}
}

remove the v.remove(value) and everything will work as expected
Edit: sorry, misread the question there
This has nothing to do with threadsafety though. You're not even multithreading so there is no reason Java would throw an exception for this.
If you want exceptions when you are changing the vector make it Unmodifiable

Problems with ListIterator and Concurrent Modification Exception

I have two ArrayLists, each holding blocks of certain size: blockList, eraserList. Blocks are objects with two fields: start and end. I need to subtract one set of blocks from the other set of blocks.
I must walk through the eraserList and "erase" blocks out of the blockList where they overlap. Thus my code looks like:
void eraseBlocks (Arrylist<Blocks> blockList, ArrayList<Blocks> eraserList) {
ListIterator<Blocks> it = blockList.listIterator();
for (Blocks eraser: eraserList) {
while (it.hasNext()) {
Blocks block= it.next();
if ((eraser.start <= block.start) && (eraser.end >= block.end))
blockList.remove(block);
else if ((eraser.start <= block.start) && (eraser.end < block.end)){
block.set(start, eraser.end);
else if () {
...
//more code for where the eraser partially erases the beginning, end, or splits the block
//if statements call the .add(), .set(), and remove() methods on the blockList.
...
}
}
}
I don't understand why I am getting a Concurrent Modification Exception. I never modify the eraserList.
I am trying to modify the block object that is assigned in the "Block block = it.next();" statement. I am also modifying the blockList by removing or adding blocks to the list. I thought the whole point of the ListIterator was that it allowed you to modify, add, or subtract a list you are walking through.
The Failure Trace points to the Blocks eraser = it.next(); as the line drawing the exception, but I don't know what that is telling me.
Can anyone help me figure out what I am doing wrong?
Thanks!

Yes, ListIterator is designed to allow the modification of the list. But you are not using the remove() method of the ListIterator, but directly manipulate the underlying list itself.

Replace
blockList.remove(block);
with
it.remove();
If you remove an element another way, you can get a CME.

You need to call remove() on the Iterator, not on the List.
From the javadoc:
If a single thread issues a sequence of method invocations that
violates the contract of an object, the object may throw this
exception. For example, if a thread modifies a collection directly
while it is iterating over the collection with a fail-fast iterator,
the iterator will thow this exception.