Is this operation safe?
if not, in order to do the same thing, how to write the code??
Set<Object> set;
......
for (Object o: set) {
if (some_condition) {
set.remove(o);
}
}
No, it's not - because you aren't allowed to modify a collection you're iterating over, other than via the iterator. There are various options for fix this. Here's one:
for (Iterator<Object> iterator = set.iterator(); iterator.hasNext(); )
{
Object value = iterator.next();
if (someCondition)
{
iterator.remove();
}
}
Another option is to build a separate list of items to remove, and then remove them all after you've iterated over the set:
List<Object> itemsToRemove = new ArrayList<Object>();
for (Object x in set)
{
if (someCondition)
{
itemsToRemove.add(x);
}
}
set.removeAll(itemsToRemove);
No - the first time your condition passes, you'll get a ConcurrentModificationException thrown on the next iteration.
In general, it is not safe to modify a collection directly while iterating over it, and iterators are "fail-fast" in this situation (because on the whole while they can detect changes, there's no general way to work out how to cope with them).
In this situation, one idiom that does work is to use the Iterator's own remove() method. Removing the element in this controlled fashion keeps the iterator aware of what is happening, and has consistent semantics for what should happen to the iteration order, and so works as you would expect:
Iterator<Object> iter = set.iterator();
while (iter.hasNext()) {
final Object o = iter.next();
if (some_condition) {
iter.remove();
}
}
Note however that remove() is an "optional operation", and not all collections' iterators support it. If you're stuck in this situation, an alternative that will always work is to take a copy of the set, iterate over that and remove the objects from the original set, like so:
Set<Object> copy = new HashSet<Object>(set);
for (Object o: copy) {
if (some_condition) {
set.remove(o);
}
}
Because the iteration is over copy, while the modifications happen to set, no concurrent modification occurs and the iterator is happy.
Now, it will throw a ConcurrentModificationException.
The for() loop uses an Iterator internally that keeps an edit count and if that edit count doesn't match the backing Set's edit count, it throws a ConcurrentModificationException
Of course this depends on the actual implementation of the collection, but that's the documented behavior, e.g. in the HashSet docs:
The iterators returned by this class's
iterator method are fail-fast: if the
set is modified at any time after the
iterator is created, in any way except
through the iterator's own remove
method, the Iterator throws a
ConcurrentModificationException. Thus,
in the face of concurrent
modification, the iterator fails
quickly and cleanly, rather than
risking arbitrary, non-deterministic
behavior at an undetermined time in
the future.
Set<Object> set;
......
Set<Object> objectsToBeDeleted = new HashSet<Object>();
for (Object o : set) {
if (some_condition) {
objectsToBeDeleted.add(o);
}
}
for (Object o : objectsToBeDeleted) {
set.remove(o);
}
Related
I am trying to understand CopyOnWriteArrayList. Normally as for my understanding we cannot remove elements in for loop while we can remove in iterator if it exists..
public class HelloWorld {
public static void main(String arfs[]) {
CopyOnWriteArrayList<String> copyOnWrite = new CopyOnWriteArrayList<String> ();
copyOnWrite.add("One");
copyOnWrite.add("Two");
copyOnWrite.add("Three");
copyOnWrite.add("Four");
for(int i=0; i<copyOnWrite.size(); i++) {
copyOnWrite.remove(i);
}
}
}
Why this is not throwing an exception, as i am trying to remove an element from CopyOnWriteArrayList...
Iterator<String> itr = copyOnWrite.iterator();
while(itr.hasNext()) {
itr.remove();
}
But when using Iterator it is throwing an exception..normally this should be opposite right...
From docs regarding iterator of that collection
The iterator does NOT support the remove method.
see https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/CopyOnWriteArrayList.html#iterator()
Normally your expectation is correct for eg ArrayList - where removing item in enchanced FOR loop will throw ConcurrentModificationException and using iterator will not, but this is different implementation - thus different behavior.
No, the iterator is not supposed to make mutations, from the javadoc
https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/CopyOnWriteArrayList.html
"Element-changing operations on iterators themselves (remove, set, and add) are not supported. These methods throw UnsupportedOperationException."
In the first case it must be making a copy of the list upon mutation, which is the expected behavior for CopyOnWriteArrayList.
This question already has answers here:
ConcurrentModificationException using Iterator
(2 answers)
Closed 5 years ago.
I have the following code
public static void main(String[] args) {
List<String> list = new ArrayList<>();
Arrays.stream("hello how are you".split(" ")).forEach(s -> list.add(s));
Iterator<String> it = list.iterator();
ListIterator<String> lit = list.listIterator();
while (it.hasNext()) {
String s = it.next();
if (s.startsWith("a")) {
it.remove();
} else {
System.out.println(s);
}
}
System.out.println(list);
// {here}
while (lit.hasNext()) {
String s = lit.next();
if (s.startsWith("a")) {
lit.set("1111" + s);
} else {
System.out.println(s);
}
}
System.out.println(list);
}
Here, after iterating through the Iterator, I try to iterate through the ListIterator. But the code throws a ConcurrentModificationException. I do the modification using the ListIterator only after the Iterator is done, but why do I get this exception.
When I initalize the ListIterator at {here} instead at the top, the code runs perfectly.
Isn't ConcurrentModificationException thrown when the list is being modified by two threads simultaneously?
Does initializing the iterator, create a lock on the list ? If yes, then why does Java let us to initialize an Iterator after it has already been initialized by another Iterator?
Isn't ConcurrentModificationException thrown when the list is being modified by two threads simultaneously ?
Not necessarily. The ConcurrentModificationException indicates that the list has been structurally changed (except by the Iterator's own remove method) after the Iterator was created. This could be due to multiple threads using the same list, or it could be due to an attempt to for example remove items from an ArrayList inside a for each loop without using an Iterator.
Does initializing the iterator, create a lock on the list ?
No, there are no locks. When the Iterator is created it records the modCount of the ArrayList (a crude representation of the list's state that is incremented on every structural change). If an iterator detects a change to the List's modcount that wasn't caused by its own methods the exception is thrown.
You are getting the exception from the second iterator because of the structural changes made to the list between the second iterator being instantiated and used.
why does Java let us to initialize an Iterator after it has already been initialized by another Iterator?
The ArrayList does not keep track of all the iterators that it has created, or their state. To do so would massively complicate the implementation. The modCount approach is not perfect and is a bit crude, but it is simple and identifies many real bugs.
You have to load the second iterator after you have used the first iterator. Otherwise the second iterator "thinks" the list hasn't been changed but in reality it did. Because the list got changed the second iterator react like "Wait a minute, that shouldn't be here/gone" and throws a ConcurrentModificationException.
It let you initialize the iterator at any time. When you don't change the content you might be even fine with it and you don't get a ConcurrentModificationException because nothing has been changed.
A ConcurrentModificationException may be thrown whenever you try to use an invalid iterator - which can happen whenever you create an iterator and then modify the underlying collection from a different access point. Here, lit is initialized and then the list is modified through it, so its invalidated, which explains the exception.
ListIterator throws ConcurrentModificationException it there is a modification in list after its creation. In your code you have created Iterator and ListIterator at the same time an later you are deleting something from the list which causes ConcurrentModificationException.
To avoid this changes your code to below one. You just need to move the ListIterator initialization after the operations of iterator.
public static void main(String[] args) {
List<String> list = new ArrayList<>();
Arrays.stream("hello how are you".split(" ")).forEach(s -> list.add(s));
Iterator<String> it = list.iterator();
while (it.hasNext()) {
String s = it.next();
if (s.startsWith("a")) {
it.remove();
} else {
System.out.println(s);
}
}
System.out.println(list);
ListIterator<String> lit = list.listIterator();
while (lit.hasNext()) {
String s = lit.next();
if (s.startsWith("a")) {
lit.set("1111" + s);
} else {
System.out.println(s);
}
}
System.out.println(list);
}
when remove an element of ArrayList from inside foreach loop. its break the loop.
throw ConcurrentModificationException
My question is why its break the loop.
Code.
ArrayList gStudyLst=getLst();
int recordCount=0;
for(String study : gStudyLst){
if(++recordCount < 10){
if(generateFile(study)){
gStudyLst.remove(study); // there its break throw ConcurrentModificationException
}else{
System.out.println("File not generated"):
gStudyLst.clear();
return false;
}
}
}else{
return true;
}
}
Use Iterator to avoid ConcurrentModificationException.
Java doc of the Iterator interface says:
Iterators allow the caller to remove elements from the underlying
collection during the iteration with well-defined semantics.
Here is an example:
Iterator<String> iter = myArrayList.iterator();
while (iter.hasNext()) {
String str = iter.next();
if (someCondition)
iter.remove();
}
It is because "The iterators returned by this class's iterator and listIterator methods are fail-fast: if the list is structurally modified at any time after the iterator is created, in any way except through the iterator's own remove or add methods, the iterator will throw a ConcurrentModificationException. Thus, in the face of concurrent modification, the iterator fails quickly and cleanly, rather than risking arbitrary, non-deterministic behavior at an undetermined time in the future."
See http://docs.oracle.com/javase/7/docs/api/java/util/ArrayList.html for details.
for each loop is just used to iterate through the values of the arraylist. If you want to remove values inside the loop, use iterator as shown below
for (Iterator iterator = gStudyLst.iterator(); iterator.hasNext();) {
String study = (String) iterator.next();
}
The enhanced for statement has the form:
for ( Type Identifier : Expression ) Statement
The enhanced for statement is equivalent to a basic for statement of the form:
for (I iter = Expression.iterator(); iter.hasNext(); ) {
Type Identifier = iter.next();
Statement
}
From javaDoc:
The iterators returned by this class's iterator and listIterator methods are fail-fast: if the list is structurally modified at any time after the iterator is created, in any way except through the iterator's own remove or add methods, the iterator will throw a ConcurrentModificationException
So instead of using forEach use Iterater and modify your List using the iterator's own remove or add methods.
SITUATION: I have a TreeSet of custom Objects and I have also used a custom Comparator. I have created an iterator to use on this TreeSet.
TreeSet<Custom> ts=new TreeSet<Custom>();
Iterator<Custom> itr=ts.iterator();
while(itr.hasNext()){
Custom c=itr.next();
//Code to add a new element to the TreeSet ts
}
QUESTION: Well I want to know that if I add a new element to the TreeSet within the while loop, then will that new element get sorted immediately. In other words, if I add a new element within the while loop and it is less than the one which I am currently holding in c, then in the next iteration will I be getting the same element in c as in the last iteration?(since after sorting, the newly added element will occupy a place somewhere before the current element).
If you add an element during your iteration, your next iterator call will likely throw a ConcurrentModificationException. See the fail-fast behavior in TreeSet docs.
To iterate and add elements, you could copy first to another set:
TreeSet<Custom> ts = ...
TreeSet<Custom> tsWithExtra = new TreeSet(ts);
for (Custom c : ts) {
// possibly add to tsWithExtra
}
// continue, using tsWithExtra
or create a separate collection to be merged with ts after iteration, as Colin suggests.
You will get a java.util.ConcurrentModificationException if you add an element into the TreeSet inside while loop.
Set<String> ts = new TreeSet<>();
ts.addAll(Arrays.asList(new String[]{"abb", "abd", "abg"}));
Iterator<String> itr = ts.iterator();
while(itr.hasNext()){
String s = itr.next();
System.out.println("s: " + s);
if (s.equals("abd"))
ts.add("abc");
}
###Output
Exception in thread "main" java.util.ConcurrentModificationException
public static void main(String[] args) {
TreeSet<Integer> ts=new TreeSet<Integer>();
ts.add(2);
ts.add(4);
ts.add(0);
Iterator<Integer> itr=ts.iterator();
while(itr.hasNext()){
Integer c=itr.next();
System.out.println(c);
//Code
ts.add(1);
}
}
Exception in thread "main" java.util.ConcurrentModificationException
This will come to all collections like List , Map , Set
Because when iterator starts it may be putting some lock on it .
if you iterate list using iterator then this exception will come. I think otherwise this loop will be infinite as you are adding element whole iterating.
Consider without iterator:
public static void main(String[] args) {
List<Integer> list=new ArrayList<Integer>();
list.add(2);
list.add(4);
list.add(0);
for (int i = 0; i < 3; i++) {
System.out.println(list.get(i));
list.add(3);
}
System.out.println("Size" +list.size());
}
this will be fine .
In order to avoid the ConcurrentModificationException you might want to check out my UpdateableTreeSet. I have even added a new test case showing how to add elements during a loop. To be more exact, you mark new elements for a later, deferred update of the set. This works quite nicely. Basically you do something like
for (MyComparableElement element : myUpdateableTreeSet) {
if (someCondition) {
// Add new element (deferred)
myUpdateableTreeSet.markForUpdate(
new MyComparableElement("foo", "bar", 1, 2)
);
}
}
// Perform bulk update
myUpdateableTreeSet.updateMarked();
I guess this is quite exactly what you need. :-)
To prevent the ConcurrentModificationException while walking.
Below is my version to allow high frequency insertion into the TreeSet() and allow concurrently iterate on it. This class use a extra queue to store the inserting object when the TreeSet is being iterating.
public class UpdatableTransactionSet {
TreeSet <DepKey> transactions = new TreeSet <DepKey> ();
LinkedList <DepKey> queue = new LinkedList <DepKey> ();
boolean busy=false;
/**
* directly call it
* #param e
*/
void add(DepKey e) {
boolean bb = getLock();
if(bb) {
transactions.add(e);
freeLock();
} else {
synchronized(queue) {
queue.add(e);
}
}
}
/**
* must getLock() and freeLock() while call this getIterator function
* #return
*/
Iterator<DepKey> getIterator() {
return null;
}
synchronized boolean getLock() {
if(busy) return false;
busy = true;
return true;
}
synchronized void freeLock() {
synchronized(queue) {
for(DepKey e:queue) {
transactions.add(e);
}
}
busy = false;
}
}
While the question has already been answered, I think the most satisfactory answer lies in javadoc of TreeSet itself
The iterators returned by this class's iterator method are fail-fast: if the set is modified at any time after the iterator is created, in any way except through the iterator's own remove method, the iterator will throw a ConcurrentModificationException. Thus, in the face of concurrent modification, the iterator fails quickly and cleanly, rather than risking arbitrary, non-deterministic behavior at an undetermined time in the future.
Note that the fail-fast behavior of an iterator cannot be guaranteed as it is, >generally speaking, impossible to make any hard guarantees in the presence of unsynchronized concurrent modification. Fail-fast iterators throw ConcurrentModificationException on a best-effort basis. Therefore, it would be wrong to write a program that depended on this exception for its correctness: the fail-fast behavior of iterators should be used only to detect bugs.
To avoid the concurrent modification error that's bound to occur when you're doing the insertion, you could also create a temporary copy of the Set, iterate through the copy instead, and modify the original.
I have a Server class and a Timer inside it which is supposed to clear dead clients (clients who crashed). I followed the example below by locking the collection when the Timer iterates over the users but I still get this exception (after I crash a connected client).
http://www.javaperformancetuning.com/articles/fastfail2.shtml
List<User> users;
List<User> connectedUsers;
ConcurrentMap<User, IClient> clients;
...
users = Collections.synchronizedList(new ArrayList<User>());
connectedUsers = new ArrayList<User>();
clients = new ConcurrentHashMap<User, IClient>();
timer = new Timer();
timer.schedule(new ClearDeadClients(), 5000, 5000);
...
class ClearDeadClients extends TimerTask {
public void run() {
synchronized (users) {
Iterator<User> it = users.iterator();
while (it.hasNext()) {
User user = it.next(); // Throws exception
if (!connectedUsers.contains(user)) {
users.remove(user);
clients.remove(user);
}
}
}
connectedUsers.clear();
}
}
You need to remove from the iterator not the collection. It would look like this instead:
Iterator<User> it = users.iterator();
while (it.hasNext()) {
User user = it.next();
if (!connectedUsers.contains(user)) {
it.remove();
clients.remove(user);
}
}
You cannot modify a collection while iterating over it - unfortunately you do that here with users, and the ConcurrentModificationException is the result. From ArrayList's own javadocs:
The iterators returned by this class's iterator and listIterator methods are fail-fast: if the list is structurally modified at any time after the iterator is created, in any way except through the iterator's own remove or add methods, the iterator will throw a ConcurrentModificationException. Thus, in the face of concurrent modification, the iterator fails quickly and cleanly, rather than risking arbitrary, non-deterministic behavior at an undetermined time in the future.
To fix this particular situation, you could instead use the Iterator's own remove() method, by replacing this line:
users.remove(user);
with
it.remove();
This latter operation removes from the collection the last element that the iterator returned. (This usage avoids the exception because the iterator is aware of the change and is able to ensure it's safe; with external modifications the iterator has no way to know whether the state of its traversal is still consistent, and so fails fast).
In some situations, this immediate removal may not be workable, in which case there are three alternative general approaches:
Take a copy of the collection (users in this case), iterate over the copy and remove elements from the original.
During iteration, build up a set of elements to remove, and then perform a bulk removal after iteration has completed.
Use an List implementation which can deal with concurrent modifications, like the CopyOnWriteArrayList
This is quite a common question - see also (for example) the loop on list with remove question for other answers.