Develop Reference

ConcurrentModificationException coming for CopyOnWriteArrayList as well [duplicate]

I'm occasionally getting a ConcurrentModificationException when I iterate over a list. A Google search informs me that it's probably because I'm altering that list in another thread while iterating over it and that to make this problem go away I should use java.util.concurrent.CopyOnWriteArrayList....
... except I already am.
Apparently, I'm doing something really stupid somewhere.
Does anybody have any insight into how one might induce CopyOnWriteArrayList to toss a ConcurrentModificationException? If it matters, I'm using Java 5.
Edit: Since the mutators I'm using may matter, I'm modifying this list in two ways:
Adding elements to the front. (list.add(0, newElement);)
Using subList to let older items fall off the back. (list = list.subList(0, MAX_LIST_SIZE);)
Do those raise red flags? If so, why? My understanding was that because these operations make a copy of the thing first, any existing iterators would be pointing at the unmodified original and would thus not care. Do I have a hole in my knowledge?
Edit 2: The precise code that's causing the problem is still a bit murky, but I can at least post the exception I'm seeing:
java.util.ConcurrentModificationException
at java.util.concurrent.CopyOnWriteArrayList$COWSubList.checkForComodification(Unknown Source)
at java.util.concurrent.CopyOnWriteArrayList$COWSubList.iterator(Unknown Source)
at....
... where it points to a for-each loop instantiation in my code.
That COWSubList does seem to imply that my call to subList is the root of my problem; I'd still like to understand why.
Edit 3: *facepalm*
CopyOnWriteArrayList.subList() returns a List, not a CopyOnWriteArrayList. The list it returns is under no implied obligation to provide any of COWAL's protections. Which makes using subList() like this to remove elements a Very Bad Idea.
Don't know for certain if this is my culprit, but it's damned suspicious and needs to be corrected regardless.

CopyOnWriteArrayList.subLists throw ConcurrentModificationExceptions if the containing list changes out from underneath it:
public class ListTest {
private static List<int[]> intList;
public static void main (String[] args) {
CopyOnWriteArrayList<Integer> cowal = new CopyOnWriteArrayList<Integer>();
cowal.add(1);
cowal.add(2);
cowal.add(3);
List<Integer> sub = cowal.subList(1, 2);
cowal.add(4);
sub.get(0); //throws ConcurrentModificationException
}
}

Sbodd has the correct answer, but it sounds like using CopyOnWriteArrayList instead of ArrayList is just an attempt to mask the error. The true problem is an attempt to modify the underlying list while iterating over it. You need to find where in your code you are accessing it as such and remove that usage or work around it.

Does Collectors.toSet() always return a HashSet ? What is the contract?

Javadoc says
Returns a Collector that accumulates the input elements into a new
Set. There are no guarantees on the type, mutability, serializability,
or thread-safety of the Set returned; if more control over the
returned Set is required, use
toCollection(java.util.function.Supplier).
So Collectors.toCollection(HashSet::new) seems like a good idea to avoid problems here (SO question).
My problem is, as hard as I've tried, I can not get anything else returned from toSet() than a HashSet
Here is the code I used :
public static void main(String[] args) {
List<Integer> l = Arrays.asList(1,2,3);
for (int i = 0 ; i++<1_000_000;){
Class clazz = l.stream().collect(Collectors.toSet()).getClass();
if (!clazz.equals(HashSet.class)) {
System.out.println("Not a HashSet");
}
}
}
Why then, does the Javadoc state that there is no guarantee when in fact, there is...

The JavaDoc states there is no guarantee, but that doesn't prevent any specific implementation from always returning a specific type of set. This is just the designers saying they don't want to limit what a future implementation can do. It says nothing about what the current implementation actually does.
In other words, you have discovered implementation defined behavior (always return a HashSet), but if you count on that you may have problems in the future.

The current OpenJDK's implementation (and AFAIK, Oracle's too) indeed always returns a HashSet - but there's no guarantee of that. A future release of the JDK may very well change this behavior and break you code if you somehow assume that Collectors.toSet() will return a HashSet (e.g., explicitly down-cast it).

The type of Set returned by Collectors::toSet is an implementation detail. You should not rely on implementation details to remain the same in future versions. Right now, they use a HashSet, but in the future they might want to use a different kind of set.

Future java versions might, for example, return specialized immutable set implementations that are more efficient for reading and consume less memory than the current HashSet implementation, which is really just a wrapper around HashMap. Project valhalla may eventually result in such optimizations.
They might even choose to return different set types based on the amount of data, e.g. an empty or singleton-set if it knows in advance that only zero or one elements will be returned.
So by giving fewer guarantees than possible based on the current implementation they keep the door open for future improvements.

I think what you're looking for is this:
Collectors.toCollection(LinkedHashSet::new)

Is it useful to check if a Java collection is empty before beginning iteration?

In the two styles below, an Iterator object is allocated. Is it useful to check if the collection is empty before iterating? I don't know if this qualifies as "premature optimization". Hopefully, someone with deep knowledge of JVM garbage collectors can provide insight.
Also, I don't know how the Java compiler handles for-each loops. I assume style B is converted to style A automatically. But... maybe an empty check is included.
Loop Style A
Collection<String> collection = ...
Iterator<String> iter = collection.iterator();
while (iter.hasNext()) {
String value = iter.next();
// do stuff
// maybe call iter.remove()
}
Loop Style B
Collection<String> collection = ...
for (String value : collection) {
// do stuff
}
Loop Style A (Modified)
Collection<String> collection = ...
if (!collection.isEmpty()) {
Iterator<String> iter = collection.iterator();
while (iter.hasNext()) {
String value = iter.next();
// do stuff
// maybe call iter.remove()
}
}
Loop Style B (Modified)
Collection<String> collection = ...
if (!collection.isEmpty()) {
for (String value : collection) {
// do stuff
}
}

Yes, this is pretty definitely going to be a premature optimization, if it can ever help. Your loop would have to be extremely performance critical, and usually called with empty collections, and for some reason unable to optimize away most of the cost of creating an actual iterator object.
In that perfect storm, this massive source ugliness could possibly be justified. But more likely you could rearrange something to help the compiler optimize better, like keeping your iterator local.
The iterator object is (usually?) function-local and therefore cheap to create (could just live in registers, no heap allocation needed). See https://www.beyondjava.net/escape-analysis-java for some details about how JVMs do "scalar replacement" on an Object if escape analysis proves that it's purely local, no references to it are visible to other code. So the possible savings don't even include a memory allocation.
If this did JIT-compile to a separate check before doing something, it's extra instructions that always run even when the collection is not empty.
Optimize for the most common case. Instead of adding extra code to slightly speed up the rare empty case, leave it out to speed up the common non-empty case.
I think most loops tend to run on non-empty collections. Small is common for some cases, but empty is usually rare. Perhaps you have a loop that often or usually runs on empty collections, e.g. a rarely-used feature in a program. Then it's worth thinking about optimizing for that case. (Whether this is a useful way to go about it is another matter)
This extra call to collection.isEmpty() might just optimize into the loop condition anyway, if it JIT-compiles to a simple pointer-increment loop over an array, with a start and end pointer both kept in registers. That's the best case, but then the extra source noise is useless and what you would have gotten anyway.
You could argue that if for (String value : collection) doesn't already compile to the most efficient way to loop over a collection, that's the compiler + JVM's fault, and you shouldn't have to make your source ugly for it. That's probably true to a point, although introducing a call to .isEmpty() isn't something a compiler or runtime can do unless they can inline that method to see that it's really checking the same thing an iterator would. But with JIT compilation, everything is available to be inlined.
TL:DR: It's likely that a good JIT compiler doesn't really spend any real work creating an iterator for most simple collections, and there's nothing to be saved.
In other cases, it's still probably better (for performance) not to do this unless your loop usually runs on empty collections, or (even more unlikely) iterator creation is somehow very expensive.

No , you don't have to check for empty. The first iteration will do the trick for you.

iter.hasNext() method will return true/false value. if collection has no element in it then iterator simply returns false on executing statement iter.hasNext() and loop will be terminated gracefully.

It is not necessary to check if the collection is empty. If you are iterating using a for loop or using while loop with iterator, it will not get into the iteration if the collection is empty.
But when you are iterating the collection, you should check whether the collection is not null. It might throw NullPointerException if the collection is null and you try to iterate using loop or iterator.
You need not to check if the collection is empty.

Concurrent Modification Exception Cause by Painting?

I have been having a lot of trouble with the Concurrent Modification Exception error. With the help of the community I managed to fix my last error, however I think this error is more puzzling.
I am now getting a concurrent modification error while painting, I am worried that this error occurs in the paint function itself. Unlike my last error I am nto removing an entity, in that case I completely understood what was going on just not how to fix it, this error on the other hand I do not understand.
TowerItr = activeTowers.iterator();
while (TowerItr.hasNext()) {
try {
theTower = (ArcherTower) TowerItr.next();
g.drawImage(tower, theTower.y * Map.blockSize, theTower.x * Map.blockSize, this);
} catch (ConcurrentModificationException e) {
}
}
The line that throws the exception is this one:
theTower = (ArcherTower) TowerItr.next();

There are always two sides to a ConcurrentModificationException (CME), and only one side reports the error.
In this case, your code looks perfectly fine. You are looping through the members of the activeTowers.
The stack trace will show you nothing that you don't already know.... you have a CME.
What you need to find out is where you are adding/removing data from the activeTowers collection.
In many cases, there are reasonably easy fixes. A probable fix is to synchronize the access to the activeTowers array in every place it is used. This may be quite hard to do.
Another option is to use a collection from the java.util.concurrent.* package, and to use the toArray(...) methods to get a snapshot of the collection, and then you can iterate that snapshot in your while loop.
// activeTower must be something from java.util.concurrent.*
for (ArcherTower theTower : activeTower.toArray(new ArcherTower[0]) {
g.drawImage(tower, theTower.y * Map.blockSize, theTower.x * Map.blockSize, this);
}
I should add that if you use a java.util.concurrent.* collection, then the iterator() will return a 'stable' iterator too (will not throw CME, and may (or may not) reflect changes in the collection)
Bottom line is that a CME only tells you half the story.... and only your code will tell you the rest....

Carrying on from what our esteemed members have suggested:
1) This error is generally seen when the List is being modified after an Iterator has been extracted.
2) I am looking at your code and it seems alright except this being used in the drawImage() method. This may potentially be altering your List because This has direct access to class members / class variables.
3) This can also happen if multiple threads are accessing the List concurrently. And one of the threads may be trying to alter your List from some other method which share the same instance of the List. I am saying some other method because concurrent read access should not create trouble if multiple threads were accessing this method.
Note: Please all posible code and Stake trace to debug exact problem.

CopyOnWriteArrayList throwing CurrentModificationException

I'm occasionally getting a ConcurrentModificationException when I iterate over a list. A Google search informs me that it's probably because I'm altering that list in another thread while iterating over it and that to make this problem go away I should use java.util.concurrent.CopyOnWriteArrayList....
... except I already am.
Apparently, I'm doing something really stupid somewhere.
Does anybody have any insight into how one might induce CopyOnWriteArrayList to toss a ConcurrentModificationException? If it matters, I'm using Java 5.
Edit: Since the mutators I'm using may matter, I'm modifying this list in two ways:
Adding elements to the front. (list.add(0, newElement);)
Using subList to let older items fall off the back. (list = list.subList(0, MAX_LIST_SIZE);)
Do those raise red flags? If so, why? My understanding was that because these operations make a copy of the thing first, any existing iterators would be pointing at the unmodified original and would thus not care. Do I have a hole in my knowledge?
Edit 2: The precise code that's causing the problem is still a bit murky, but I can at least post the exception I'm seeing:
java.util.ConcurrentModificationException
at java.util.concurrent.CopyOnWriteArrayList$COWSubList.checkForComodification(Unknown Source)
at java.util.concurrent.CopyOnWriteArrayList$COWSubList.iterator(Unknown Source)
at....
... where it points to a for-each loop instantiation in my code.
That COWSubList does seem to imply that my call to subList is the root of my problem; I'd still like to understand why.
Edit 3: *facepalm*
CopyOnWriteArrayList.subList() returns a List, not a CopyOnWriteArrayList. The list it returns is under no implied obligation to provide any of COWAL's protections. Which makes using subList() like this to remove elements a Very Bad Idea.
Don't know for certain if this is my culprit, but it's damned suspicious and needs to be corrected regardless.

CopyOnWriteArrayList.subLists throw ConcurrentModificationExceptions if the containing list changes out from underneath it:
public class ListTest {
private static List<int[]> intList;
public static void main (String[] args) {
CopyOnWriteArrayList<Integer> cowal = new CopyOnWriteArrayList<Integer>();
cowal.add(1);
cowal.add(2);
cowal.add(3);
List<Integer> sub = cowal.subList(1, 2);
cowal.add(4);
sub.get(0); //throws ConcurrentModificationException
}
}

Sbodd has the correct answer, but it sounds like using CopyOnWriteArrayList instead of ArrayList is just an attempt to mask the error. The true problem is an attempt to modify the underlying list while iterating over it. You need to find where in your code you are accessing it as such and remove that usage or work around it.