If I use Iterator.remove(), everything is fine. If I use ArryaList.remove(), I always receive the error java.util.ConcurrentModificationException.
Can anyone point out the reason?
The javadocs say it all:
This exception may be thrown by methods that have detected concurrent
modification of an object when such modification is not permissible.
For example, it is not generally permissible for one thread to modify
a Collection while another thread is iterating over it. In general,
the results of the iteration are undefined under these circumstances.
Some Iterator implementations (including those of all the general
purpose collection implementations provided by the JRE) may choose to
throw this exception if this behavior is detected. Iterators that do
this are known as fail-fast iterators, as they fail quickly and
cleanly, rather that risking arbitrary, non-deterministic behavior at
an undetermined time in the future.
You're modifying a collection and iterating over it at the same time.
Basically because that's how it's designed to work. If you delete an element from the list, the iterator doesn't know about it, and when it tries to access another element in the list, the list has changed and an error is raised. But if you remove an element through the iterator, then the itertor knows about the removal, makes the appropriate adjustment to its data structures, and continues.
So exactly from docs
This exception may be thrown by methods that have detected concurrent
modification of an object when such modification is not permissible.
For example, it is not generally permissible for one thread to modify
a Collection while another thread is iterating over it. In general,
the results of the iteration are undefined under these circumstances.
Some Iterator implementations (including those of all the general
purpose collection implementations provided by the JRE) may choose to
throw this exception if this behavior is detected. Iterators that do
this are known as fail-fast iterators, as they fail quickly and
cleanly, rather that risking arbitrary, non-deterministic behavior at
an undetermined time in the future.
Have look at Class ConcurrentModificationException
If you need to remove an element while you are iterating through a list then you should always use iterator.
e.g.
List<String> list = //...
//...
Iterator<String> iter = list.iterator();
while(iter.hasNext()) {
String str = itr.next();
//...
if(/* ... */) {
iter.remove(str);
}
}
That's the safest way of removing an element from a list while you are iterating through the list.
Note: You should not do something as follows:
List<String> list = //...
//...
for(String str : list) {
if(/* ... */) {
list.remove(); // <-- should not do this
}
}
Where mostly you will end up in a ConcurrentModificationException. That enhanced for loop uses the iterator under the hood. And you are manually removing the element from the list instead of going through the iterator. That's why you get that exception.
Related
A ConcurrentModificationException was returned removing an entry from a map but I am unable to see what's wrong.
Iterator<String> mapIterator = this.attributeMap.keySet().iterator();
while (mapIterator.hasNext())
{
String attributeName = mapIterator.next();
if (attributeName.matches(pattern))
{
mapIterator.remove();
}
}
In this case the line
mapIterator.next()
throws ConcurrentModificationException .
The reason behind this is:
There is an int variable modCount which provides the number of times list size has been changed, this value is used in every next() call to check for any modifications in a function checkForComodification(). if mapIterator.next() found change in modCount while iteration object then it will throw ConcurrentModificationException .
To avoid this please follow the below points:
You can convert the list to an array and then iterate on the array. This approach works well for small or medium size list but if the list is large then it will affect the performance a lot.
You can lock the list while iterating by putting it in a synchronized block. This approach is not recommended because it will cease the benefits of multithreading.
If you are using JDK1.5 or higher then you can use ConcurrentHashMap and CopyOnWriteArrayList classes. It is the recommended approach.
You can check this out on some details about CME: https://docs.oracle.com/javase/7/docs/api/java/util/ConcurrentModificationException.html
This exception may be thrown by methods that have detected concurrent modification of an object when such modification is not permissible.
For example, it is not generally permissible for one thread to modify a Collection while another thread is iterating over it. In general, the results of the iteration are undefined under these circumstances. Some Iterator implementations (including those of all the general purpose collection implementations provided by the JRE) may choose to throw this exception if this behavior is detected. Iterators that do this are known as fail-fast iterators, as they fail quickly and cleanly, rather that risking arbitrary, non-deterministic behavior at an undetermined time in the future.
Here are some solutions provided by Baeldung, linked below:
Using an Iterator Directly
Not removing during iteration
Using removeIf()
Filtering using Streams
You can check his site out for the details for each solution to avoid CME: https://www.baeldung.com/java-concurrentmodificationexception
Hope this helps.
Java is throwing ConcurrentModificationException when I am running the following code. Any idea why is that?
ArrayList<String> list1 = new ArrayList<String>();
list1.add("Hello");
list1.add("World");
list1.add("Good Evening");
for (String s : list1){
list1.remove(2);
System.out.println(s);
}
If you take a look at documentation of ConcurrentModificationException you will find that
This exception may be thrown by methods that have detected concurrent
modification of an object when such modification is not permissible.
For example, it is not generally permissible for one thread to modify
a Collection while another thread is iterating over it
...
Note that this exception does not always indicate that an object has
been concurrently modified by a different thread. If a single thread
issues a sequence of method invocations that violates the contract of
an object, the object may throw this exception. For example, if a
thread modifies a collection directly while it is iterating over the
collection with a fail-fast iterator, the iterator will throw this
exception.
Important thing about this exception is that we can't guarantee it will always be thrown as stated in documentation
Note that fail-fast behavior cannot be guaranteed as it is, generally
speaking, impossible to make any hard guarantees in the presence of
unsynchronized concurrent modification. Fail-fast operations throw
ConcurrentModificationException on a best-effort basis.
Also from ArrayList documentation
The iterators returned by this class's iterator and listIterator
methods are fail-fast: if the list is structurally modified at any
time after the iterator is created, in any way except through the
iterator's own remove or add methods, the iterator will throw a
ConcurrentModificationException.
(emphasis mine)
So you can't manipulate content of Collection (in your case List) while iterating over it via enhanced for loop because you are not doing it via iterator for-each is using internally.
To solve it just get your own Iterator and use it in your loop. To remove elements from collection use remove like in this example
Iterator<String> it = list1.iterator();
int i=0;
while(it.hasNext()){
String s = it.next();
i++;
if (i==2){
it.remove();
System.out.println("removed: "+ s);
}
}
form the doc you can read
This exception may be thrown by methods that have detected concurrent
modification of an object when such modification is not permissible.
For example, it is not generally permissible for one thread to modify
a Collection while another thread is iterating over it. In general,
the results of the iteration are undefined under these circumstances.
Some Iterator implementations (including those of all the general
purpose collection implementations provided by the JRE) may choose to
throw this exception if this behavior is detected. Iterators that do
this are known as fail-fast iterators, as they fail quickly and
cleanly, rather that risking arbitrary, non-deterministic behavior at
an undetermined time in the future.
further more, suppose that you can remove item 2 at each iteration. You will end up in a index out of bound exception:
at first iteration you remove item #2 ("Good evening") and list size become 1 (item 0 "hello" and item 1 "World")
at next iteration you remove item #2 which actually does not exist in your list. Your list is indeed of size two, but counter starts from 0, thus you end up removing something which is not there: this is an example of the non-deterministic behavior.
You cann't iterating over an list after the underlying list is modified.if you do that its give you ConcurrentModificationException
to resolve this issue use java.util.ListIterator for iteration of list.
ListIterator<String> it = list1.listIterator();
for (String s : list1) {
if (it.hasNext()) {
String item = it.next();
System.out.println(item);
}
}
You can't delete an item with ArrayList's remove() method while iterating over it. Use Iterator if you want to delete the item also while iterating.
But you are deleting an item based on index then simply moving below line outside the loop will solve your problem.
list1.remove(2);
I am confused by one specific point regarding synchronization feature of LinkedHashMap. Here are the related Javadoc where I am confused. My confusion point is, why remove method is special here, which is mentioned by -- "except through the iterator's own remove method"?
http://docs.oracle.com/javase/6/docs/api/java/util/LinkedHashMap.html
The iterators returned by the iterator method of the collections returned by all of this class's collection view methods are fail-fast: if the map is structurally modified at any time after the iterator is created, in any way except through the iterator's own remove method, the iterator will throw a ConcurrentModificationException. Thus, in the face of concurrent modification, the iterator fails quickly and cleanly, rather than risking arbitrary, non-deterministic behavior at an undetermined time in the future.
thanks in advance,
Lin
Basically, you're not allowed to structurally modify a map while iterating over it, since doing so would invalidate the iterator.
Iterator.remove() is specifically exempt from this, to enable you to easily write code like this:
Iterator<E> it = coll.iterator();
while (it.hasNext()) {
E val = it.next();
if (some_condition) {
it.remove();
}
}
It isn't special, it's the normal way of removing some item from a collection when using an iterator. If an element is removed "outside" the iterator, the iterator view of the given collection becomes inconsistent as compared to the actual collection since the iterator has no way of knowing why or how the element was removed.
Iterators can be of different types. There are iterators which "operate" on a given state of the collection in which case modifying the collection outside the iterator makes no difference. This is common for immutable collections. The other type of iterator is a "fail-fast" one which throws up an exception as soon as it finds out that the iterator is now looking at an old state of the collection (i.e. the collection was modified from outside). As mentioned in the docs, LinkedHashMap uses a fail-fast iterator.
It's not special, it acquires a sort of lock on the LinkedHashMap so that you are able to remove elements just through its remove method.
This because, as specified before:
A structural modification is any operation that adds or deletes one or more mappings or, in the case of access-ordered linked hash maps, affects iteration order.
This means that you have a pending iterator on the data structure, allowing any modification would generate problems with respect to deterministic behavior (since the iterator would become inconsistent, as the underlying structure changed). So the implementation just make any structural modification fail with an exception as soon as you invoke any method if you have an open iterator on it.
I have a List, and I want to loop through that List and remove some record base on some condition. Here is what I do
public void foo(List<Bar> recordList){
for(Bar bar : recordList){
if(bar.someCondition()){
recordList.remove(bar);
}
}
}
This code generate exception. If I use Iterator then it works fine
public void foo(List<Bar> recordList){
Iterator<Bar> iter = recordList.iterator();
while(iter.hasNext()){
Bar bar = iter.next();
if(bar.someCondition()){
iter.remove();
}
}
}
I guess my question:
Why the first piece of code does not work?
How do I make the first piece of code work?
The documentation is very clear on this.
http://download.oracle.com/javase/6/docs/api/java/util/ArrayList.html
The iterators returned by this class's iterator and listIterator methods are fail-fast: if the list is structurally modified at any time after the iterator is created, in any way except through the iterator's own remove or add methods, the iterator will throw a ConcurrentModificationException. Thus, in the face of concurrent modification, the iterator fails quickly and cleanly, rather than risking arbitrary, non-deterministic behavior at an undetermined time in the future.
It doesn't work because you are modifying a collection WHILE iterating on it. It means you are changing its state and reading it at the same time. This provokes unexpected behaviours in the inners of the collection, and to prevent data corruption, an exception is thrown.
You don't make it work. You have to use iterators.
final Multimap<Term, BooleanClause> terms = getTerms(bq);
for (Term t : terms.keySet()) {
Collection<BooleanClause> C = new HashSet(terms.get(t));
if (!C.isEmpty()) {
for (Iterator<BooleanClause> it = C.iterator(); it.hasNext();) {
BooleanClause c = it.next();
if(c.isSomething()) C.remove(c);
}
}
}
Not a SSCCE, but can you pick up the smell?
The Iterator for the HashSet class is a fail-fast iterator. From the documentation of the HashSet class:
The iterators returned by this class's iterator method are fail-fast:
if the set is modified at any time after the iterator is created, in
any way except through the iterator's own remove method, the Iterator
throws a ConcurrentModificationException. Thus, in the face of
concurrent modification, the iterator fails quickly and cleanly,
rather than risking arbitrary, non-deterministic behavior at an
undetermined time in the future.
Note that the fail-fast behavior of an iterator cannot be guaranteed
as it is, generally speaking, impossible to make any hard guarantees
in the presence of unsynchronized concurrent modification. Fail-fast
iterators throw ConcurrentModificationException on a best-effort
basis. Therefore, it would be wrong to write a program that depended
on this exception for its correctness: the fail-fast behavior of
iterators should be used only to detect bugs.
Note the last sentence - the fact that you are catching a ConcurrentModificationException implies that another thread is modifying the collection. The same Javadoc API page also states:
If multiple threads access a hash set concurrently, and at least one
of the threads modifies the set, it must be synchronized externally.
This is typically accomplished by synchronizing on some object that
naturally encapsulates the set. If no such object exists, the set
should be "wrapped" using the Collections.synchronizedSet method. This
is best done at creation time, to prevent accidental unsynchronized
access to the set:
Set s = Collections.synchronizedSet(new HashSet(...));
I believe the references to the Javadoc are self explanatory in what ought to be done next.
Additionally, in your case, I do not see why you are not using the ImmutableSet, instead of creating a HashSet on the terms object (which could possibly be modified in the interim; I cannot see the implementation of the getTerms method, but I have a hunch that the underlying keyset is being modified). Creating a immutable set will allow the current thread to have it's own defensive copy of the original key-set.
Note, that although a ConcurrentModificationException can be prevented by using a synchronized Set (as noted in the Java API documentation), it is a prerequisite that all threads access the synchronized collection and not the backing collection directly (which might be untrue in your case as the HashSet is probably created in one thread, while the underlying collection for the MultiMap is modified by other threads). The synchronized collection classes actually maintain an internal mutex for threads to acquire access to; since you cannot access the mutex directly from other threads (and it would be quite ridiculous to do so here), you ought to look at using a defensive copy of either the keyset or of the MultiMap itself using the unmodifiableMultimap method of the MultiMaps class (you'll need to return an unmodifiable MultiMap from the getTerms method). You could also investigate the necessity of returning a synchronized MultiMap, but then again, you'll need to ensure that the mutex must be acquired by any thread to protect the underlying collection from concurrent modifications.
Note, I have deliberately omitted mentioning the use of a thread-safe HashSet for the sole reason that I'm unsure of whether concurrent access to the actual collection will be ensured; it most likely will not be the case.
Edit: ConcurrentModificationExceptions thrown on Iterator.next in a single-threaded scenario
This is with respect to the statement: if(c.isSomething()) C.remove(c); that was introduced in the edited question.
Invoking Collection.remove changes the nature of the question, for it now becomes possible to have ConcurrentModificationExceptions thrown even in a single-threaded scenario.
The possibility arises out of the use of the method itself, in conjunction with the use of the Collection's iterator, in this case the variable it that was initialized using the statement : Iterator<BooleanClause> it = C.iterator();.
The Iterator it that iterates over Collection C stores state pertinent to the current state of the Collection. In this particular case (assuming a Sun/Oracle JRE), a KeyIterator (an internal inner class of the HashMap class that is used by the HashSet) is used to iterate through the Collection. A particular characteristic of this Iterator is that it tracks the number of structural modifications performed on the Collection (the HashMap in this case) via it's Iterator.remove method.
When you invoke remove on the Collection directly, and then follow it up with an invocation of Iterator.next, the iterator throws a ConcurrentModificationException, as Iterator.next verifies whether any structural modifications of the Collection have occurred that the Iterator is unaware of. In this case, Collection.remove causes a structural modification, that is tracked by the Collection, but not by the Iterator.
To overcome this part of the problem, you must invoke Iterator.remove and not Collection.remove, for this ensures that the Iterator is now aware of the modification to the Collection. The Iterator in this case, will track the structural modification occurring through the remove method. Your code should therefore look like the following:
final Multimap<Term, BooleanClause> terms = getTerms(bq);
for (Term t : terms.keySet()) {
Collection<BooleanClause> C = new HashSet(terms.get(t));
if (!C.isEmpty()) {
for (Iterator<BooleanClause> it = C.iterator(); it.hasNext();) {
BooleanClause c = it.next();
if(c.isSomething()) it.remove(); // <-- invoke remove on the Iterator. Removes the element returned by it.next.
}
}
}
The reason is that you are trying to modify the collection outside iterator.
How it works :
When you create an iterator the collection maintains a modificationNum-variable for both the collection and the iterator independently.
1. The variable for collection is being incremented for each change made to the collection and and iterator.
2. The variable for iterator is being incremented for each change made to the iterator.
So when you call it.remove() through iterator that increases the value of both the modification-number-variable by 1.
But again when you call collection.remove() on collection directly, that increments only the value of the modification-numbervariable for the collection, but not the variable for the iterator.
And rule is : whenever the modification-number value for the iterator does not match with the original collection modification-number value, it gives ConcurrentModificationException.
Vineet Reynolds has explained in great details the reasons why collections throw a ConcurrentModificationException (thread-safety, concurrency). Swagatika has explained in great details the implementation details of this mechanism (how collection and iterator keep count of the number of modifications).
Their answers were interesting, and I upvoted them. But, in your case, the problem does not come from concurrency (you have only one thread), and implementation details, while interesting, should not be considered here.
You should only consider this part of the HashSet javadoc:
The iterators returned by this class's iterator method are fail-fast:
if the set is modified at any time after the iterator is created, in
any way except through the iterator's own remove method, the Iterator
throws a ConcurrentModificationException. Thus, in the face of
concurrent modification, the iterator fails quickly and cleanly,
rather than risking arbitrary, non-deterministic behavior at an
undetermined time in the future.
In your code, you iterate over your HashSet using its iterator, but you use the HashSet's own remove method to remove elements ( C.remove(c) ), which causes the ConcurrentModificationException. Instead, as explained in the javadoc, you should use the Iterator's own remove() method, which removes the element being currently iterated from the underlying collection.
Replace
if(c.isSomething()) C.remove(c);
with
if(c.isSomething()) it.remove();
If you want to use a more functional approach, you could create a Predicate and use Guava's Iterables.removeIf() method on the HashSet:
Predicate<BooleanClause> ignoredBooleanClausePredicate = ...;
Multimap<Term, BooleanClause> terms = getTerms(bq);
for (Term term : terms.keySet()) {
Collection<BooleanClause> booleanClauses = Sets.newHashSet(terms.get(term));
Iterables.removeIf(booleanClauses, ignoredBooleanClausePredicate);
}
PS: note that in both cases, this will only remove elements from the temporary HashSet. The Multimap won't be modified.