Develop Reference

For loop vs. Iterator to avoid ConcurrentModificationException with an ArrayList

Question: What is the optimal (performance-wise) solution for the add, removal, modification of items within an ArrayList which at the same time avoids the ConcurrentModificationException from being thrown during operations?
Context: Based on my research looking into this question, there doesn't seem to be any straight-forward answers to the question at hand - most recommend using CopyOnWriteArrayList, but my understanding is that it is not recommended for array lists of large size (which I am working with, hence the performance-aspect of the question).
Thus, my understanding can be summarized as the following, but want to make sure if is correct/incorrect:
IMPORTANT NOTE: The following statements all assume that the operation is done within a synchronized block.
Remove during iteration of an ArrayList should be done with an Iterator, because for loop results in unpredictable behavior if removal is done within the middle of a collection. Example:
Iterator<Item> itemIterator = items.iterator();
while (itemIterator.hasNext()) {
Item item = itemIterator.next();
// check if item needs to be removed
itemIterator.remove();
}
For add operations, cannot be done with an Iterator, but can be with ListIterator. Example:
ListIterator<Item> itemIterator = list.listIterator();
while(itemIterator.hasNext()){
\\ do some operation which requires iteration of the ArrayList
itemIterator.add(item);
}
For add operations, a ListIterator does NOT have to be necessarily be used (i.e. simply items.add(item) should not cause any problems).
For add operations while going through the collection can be done with EITHER a ListIterator or a for loop, but NOT an Iterator. Example:
Iterator<Item> itemIterator = item.iterator();
while (itemIterator.hasNext()) {
\\ do some operation which requires iteration of the ArrayList
items.add(item); \\ NOT acceptable - cannot modify ArrayList while in an Iterator of that ArrayList
}
Modification of an item within an ArrayList can be done with either an Iterator or a for loop with the same performance complexity (is this true?). Example:
\\ iterator example
Iterator<Item> itemIterator = item.iterator();
while (itemIterator.hasNext()) {
Item item = itemIterator.next();
item.update(); // modifies the item within the ArrayList during iteration
}
\\ for loop example
for (Item item : items){
item.update();
}
Will modification during iteration with the Iterator have the same performance as the for loop? Are there any thread-safety differences between the approaches?
Bonus question: what advantage does using a synchronizedList of the ArrayList for add/remove/modify operations vs. for loop vs. iterator if it also requires a synchronized block?

There is no difference between while loops and for loops and in fact, the idiomatic form of a loop using an iterator explicitly, is a for loop:
for(Iterator<Item> it = items.iterator(); it.hasNext(); ) {
Item item = it.next();
item.update();
}
which gets compiled to exactly the same code as
for(Item item: items) {
item.update();
}
Try it online!
There are no performance differences for identical compiled code dependent to the original source code used to produce it.
Instead of focusing on the loop form, you have to focus on the fundamental limitations when inserting or removing elements of an ArrayList. Each time you insert or remove an element, the elements behind the affected index have to be copied to a new location. This isn’t very expensive, as the array only consists of references to the objects, but the costs can easily add up when doing it repeatedly.
So, if you know that the number of insertions or removals is predictably small or will happen at the end or close to the end (so there is only a small number of elements to copy), it’s not a problem. But when inserting or removing an arbitrary number of elements at arbitrary positions in a loop, you run into a quadratic time complexity.
You can avoid this, by using
items.removeIf(item -> /* check and return whether to remove the item*/);
This will use an internal iteration and postpone the moving of elements until their final position is known, leading to a linear time complexity.
If that’s not feasible, you might be better off copying the list into a new list, skipping the unwanted elements. This will be slightly less efficient but still have a linear time complexity. That’s also the solution for inserting a significant number of items at arbitrary positions.
The item.update(); in an entirely different category. “the item within the ArrayList” is a wrong mindset. As said above, the ArrayList contains references to objects whereas the object itself is not affected by “being inside the ArrayList”. In fact, objects can be in multiple collections at the same time, as all standard collections only contain references.
So item.update(); changes the Item object, which is an operation independent of the ArrayList, which is dangerous when you assume a thread safety based on the list.
When you have code like
Item item = items.get(someIndex);
// followed by using item
where get is from a synchronizedList
or a manually synchronized retrieval operation which returns the item to the caller or any other form of code which uses a retrieved Item outside the synchronized block,
then your code is not thread safe. It doesn’t help when the update() call is done under a synchronization or lock when looping over the list, when the other uses are outside the synchronization or lock. To be thread safe, all uses of an object must be protected by the same thread safety construct.
So even when you use the synchronizedList, you must not only guard your loops manually, as the documentation already tells, you also have to expand the protection to all other uses of the contained elements, if they are mutable.
Alternatively, you could have different mechanisms for the list and the contained elements, if you know what you are doing, but it still means that the simplicity of “just wrap the list with synchronizedList” isn’t there.
So what advantage does it have? Actually none. It might have helped developers during the migration from Java 1.1 and its all-synchronized Vector and Hashtable to Java 2’s Collection API. But I never had a use for the synchronized wrappers at all. Any nontrivial use case requires manual synchronization (or locking) anyway.

How can I modify an element of a PriorityQueue in Java?

I got the following issue to solve: I'm with a PriorityQueue of a specific object, and the attribute I use to compare it with others are set with the same value for all the objects.
The problem is: I need to modify one of it's objects (I mean, find it by another attribute, and modify the comparable attribute) and take it off of the queue. And I got no ideia of how to do it, since peek() and poll() just remove and return the head of the queue, and remove() just remove the object, and it's not exactly what I want. I also don't know how could I use Iterator here as well.
That's the code I got until now:
public void inicializaDijkstra(Grafo grafo, Vertice v0){
Comparator<Grafo> comparator = new verticecomparator();
PriorityQueue<Grafo> Queue = new PriorityQueue<Grafo>(grafo.getNumeroDeVertices,grafo);
for (Vertice vertice : conjuntoDeVertices) {
queue.add(vertice);
}
I just though of gettinng the element I want with the Iterator, remove it from the queue, modify it and (if I didn't want to remove it) add it again on the queue. Would it work?

I just thought of getting the element I want with the Iterator, remove it from the queue, modify it and (if I didn't want to remove it) add it again on the queue. Would it work?
It should work1.
Indeed, I can't think of a better / more efficient way of doing this given your data structure choices.
Note that this approach is O(N) where N is the queue length. In a multi-threaded context you would probably need to do the entire sequence under an exclusive lock, and that could make it a concurrency bottleneck.
1 - Actually, with some queue implementations, adding (back) an element while you are iterating the priority queue can result in a ConcurrentModificationException. If that is a problem, then you may need to make a list of elements that need re-inserting and then re-insert them after the you have finished iterating. The javadocs seem to say that PriorityQueue would give CME's but PriorityBlockingQueue would not.

What does lockstep mean in programming?

I have been reading Effective Java on
Item 46: Prefer for-each loops to traditional for loops
In the part where are mentioned the cases when is iterator/for loop needed isntead of for-each loop, there is this point:
Parallel iteration—If you need to traverse multiple collections in
parallel, then you need explicit control over the iterator or index
variable, so that all iterators or index variables can be advanced
in lockstep.
Now, I understand what explicit control over iterator/index variable mean (not controller by for each loop). But I could not understand the meaning of lockstep in this sense. I tried to google it and found an article on Wikipedia which states:
Lockstep systems are fault-tolerant computer systems that run the same
set of operations at the same time in parallel.
This I understand as having aditional instance of for example server for fail-over That's ok. But I fail to fully understand what could be the exact meaning in the context of iterating over collection in programming.

In this context, the meaning is more like the military marching.
Or, when one operation advances, other operations advances/follows with it.
Or more specifically, if you want to iterate over two collections, you cannot easily the foreach construct:
for (Item i : list1) { //only allows you to iterate over 1 list.
}
Iterate over 2 collections )
Iterator iter1 = list1.iterator();
Iterator iter2 = list2.iterator();
while (iter1.hasNext() && iter2.hasNext()){
Item a = iter1.next();
Item b = iter2.next();
doSomething(a, b);
}
i.e. while iterating list1, iterating list2 follows with it - "in lockstep"

Lockstep execution means that the same statement will be executed on all the processors at the same time "in parallel". This is of special importance when you are dealing with GPGPU (General Purpose Graphics Processing Unit) programming. GPU's actually do the exact same operation in parallel on a different data set.
Example: In a for loop with independent operations on data (say a vector addition problem), all the processors may call the add operation simultaneously, then assignment operation simultaneously on two separate vector index, in a lockstep fashion, as one addition and assignment is independent from another.

The meaning of "lockstep" in this context is not special, but is the English-language meaning, interpreted as "at the same time".

Here, it just means that the index and iterator are advanced at the same time, so that they always correspond to the same element. Kind of like two people walking side-by-side--they have to step forward together if they're to remain side-by-side.

What does it mean to say that a list is locked internally?

This code is from the book Effective Java
Object[] snapshot = list.toArray();// Locks list internally
I am mainly interested in the comment here . Does it make the list unmodifiable ? What does it mean to say that a list is locked internally ? How long is this lock kept ? Is there a better alternative to convert a List to an array ?

I would imagine that it means the list doesn't maintain a reference to the returned array, meaning that the array can be modified without affecting the original list from where it came. Likewise, any modifications to the list won't be reflected in the array.
This is important in terms of thread safety, because it means you can iterate on the contents of the list from a thread-safe perspective, without worrying about another thread altering the sttae of the list in the meantime. In this sense the state of the list is "locked" in the returned array, no matter what changes are made to the list afterwards - you can see it as taking a snapshot.
toArray(); doesn't alter the state of the list - so it doesn't make it unmodifiable or anything like that.

Like the others said, I think that is about concurrency:
Text from javadoc of java.uitl.List
The returned array will be "safe" in that no references to it are
maintained by this list. (In other words, this method must
allocate a new array even if this list is backed by an array).
The caller is thus free to modify the returned array.

Its about thread safety - i.e. conversion of the list to Array will be thread safe
Edit:
In simplest way - you can take it as
when Thread one is converting List -> Array no other thread is allowed to alter the list till the time Thread one has not completed the conversion

For those wondering where the "internal locking" takes place:
Please note that J. Bloch writes as an introduction for the given code: "For example, suppose you have a synchronized list (of the sort returned by Collections.synchroniedList) (...)"
In that case toArray() really "locks internal" because the implementation of the synchronized list will do just that (with a mutex) preventing any modification by other threads while the decoupled array is created.

java - general synchronizedList question

I have a general question regarding synchronized List.
Lets say that in the constructor I am createing a list
List synchronizedList = Collections.synchronizedList(list);
and I have one method adds an object to the list.
public void add(String s){
synchronizedList.add(s)
}
There is another thread that checks every few seconds if there are a few rows , dump it to a file and deletes them all.
Now lets say I iterate each row and save it to the db.
after all iteration I clear the list.
How does the multithread support help me?
I could add an element to the list just before the clear() in the other thread occurs .
Unless I manage the lock myself (which I dont realy need a synched list for that ) it myself.

The synchronized list returned by Collections won't help in your case. It's only good if you need to guarantee serial access to individual method calls. If you need to synchronize around a larger set of operations, then you need to manually wrap that code in a synchronized block. The Javadoc states:
It is imperative that the user manually synchronize on the returned list when iterating over it.
If your list is used elsewhere you can at least safeguard it from individual method calls that would otherwise not be thread-safe. If you're entirely managing the list however, you can just add a synchronized block to your add method and use the same lock that you'll use when iterating over it.

synchronizedList indeed only guarantees that every method call on the list is synchronized. If you need multiple operations to be done in a synchronized way, you have to handle the synchronization yourself.
BTW, this is explicitely said in the javadoc for Collections.synchronizedList :
It is imperative that the user
manually synchronize on the returned
list when iterating over it:
List list = Collections.synchronizedList(new ArrayList());
...
synchronized(list) {
Iterator i = list.iterator(); // Must be in synchronized block
while (i.hasNext())
foo(i.next());
}

synchronized list means that all the operations on that list are guaranteed to be atomic. The scenario you describe requires to have some locking outside the list. Consider semaphores or making synchronized block to implement monitors. Take a look at java.util.concurrent.