I know that concurrent modification exception is thrown when the collection is structurally changed while iterating it, but why?
what are the potential problems if we don't throw concurrent modification exception?
how concurrent modification exception prevents undetermined behaviour in future?
if its because to prevent multithreading related issues why its also thrown when the same thread which called the iterator modifies the collection structure?
this may be a pretty basic question but I do need some proper scenario to convince myself that checking ConcurrentmodificationException is absolutely necessary.
It's not "absolutely necessary", but it helps prevent a number of potential bugs, both in single-threaded and multi-threaded code. Suppose for instance you're iterating over a set and add or remove an element - how should the iterator handle that? There isn't an obviously correct behavior (you might want it to appear later in the iteration, or you might not).
To avoid CMEs you'll generally want to use two separate collections - one you iterate and one you mutate. This also often leads to cleaner code that's easier to reason about. In the specific case of lists you can iterate over the indices (for (int i = 0; i < list.size(); i++)) but you need to be careful about how i changes when you modify the list so you don't skip or double-iterate over an element.
Related
In my Android app I use this code while drawing some waypoints on a map
Iterator<Waypoint> iterator = waypoints.iterator();
while (iterator.hasNext()) {
Waypoint w = iterator.next();
}
But I am getting this error
Fatal Exception: java.util.ConcurrentModificationException
java.util.ArrayList$ArrayListIterator.next (ArrayList.java:573)
I am not modifying the list directly in the loop I am iterating over.
But it is possible that I modify the list in another thread because a user can move some waypoints. And the drawing of a waypoint can happen the same time a user uses the touch display to move a waypoint.
Can I avoid that exception somehow?
If you want to maintain a List you use in several threads, it's best you use a concurrent list, such as CopyOnWriteArrayList.
Locally, you can avoid the exception by creating a copy of the waypoints list first and iterate that:
Iterator<Waypoint> iterator = new ArrayList<>(waypoints).iterator();
while (iterator.hasNext()) {
handle(iterator.next());
}
The iterator provided by array list is fail-fast iterator - meaning it fails as soon as the underlying list is modified.
One way to avoid the exception is to take a snapshot of the list into another list and then iterate over it.
Iterator<Waypoint> iterator = new ArrayList<>(waypoints).iterator();
while (iterator.hasNext()) {
Waypoint w = iterator.next();
}
another way is to use collection that implements fail-safe iterators such as CopyOnWriteArrayList.
I see some options:
a. Avoid multithreading. Well, you don't have to avoid multithreading completely, just for access to the array. All accesses to the array (even read) must happen from the same thread. Heavy computations can happen on some other threads, of course. This might be a reasonable approach when you can iterate fast.
b. Lock the ArrayList, even for reading. This can be tricky, as excessive locking can introduce deadlocks.
c. Use data copies. Remember, you copy just references, but you usually don't have to clone all the objects. For large data structures, it might be worth considering some persistent data structure, which does not require to copy all the data.
d. Deal with the ConcurrentModificationException somehow. Maybe restart the computation. This might be useful in some cases, but it might get tricky in complex code. Also, in some cases when accessing multiple shared data structures, you might get a livelock – two (or more) threads causing ConcurrentModificationException repeatedly to each other.
EDIT: For some approaches (at least A), you might find reactive programming useful, because this programming style reduces the time spent in the main thread.
This question already has answers here:
How do I make my ArrayList Thread-Safe? Another approach to problem in Java?
(8 answers)
Closed 7 years ago.
This question is more about asking if my way of doing something is the "correct" way or not. I have some program that involves constantly updating graphical components. To that effect, I have the method below.
public void update(){
for (BaseGameEntity movingEntity : movingEntityList) {
((MovingEntity)movingEntity).update();
}
}
Essentially, the class containing this method has a list of all graphical objects that need updating and it loops through, calling their respective update methods.
The issue comes when I have to add new entities or remove current entities from this list. The addition and removal of entities is handled by a different thread, and as you can guess, this results in a Concurrent Modification Exception if I try to add/remove entities while also looping through and updating their graphical components.
My ad hoc solution was to simply throw a try-catch block around this and just ignore any concurrent modification exceptions that crop up - in effect, not updating at that specific time. This does exactly what I want and no problems occur.
public void update(){
try{
for (BaseGameEntity movingEntity : movingEntityList) {
((MovingEntity)movingEntity).update();
}
}catch(ConcurrentModificationException e){
//Do Nothing
}
}
However, my question is, is this a "proper" way of handling this issue? Should I perhaps be doing something akin to what is outlined in this answer? What is the "correct" way to handle this issue, if mine is wrong? I'm not looking specifically for ways to make my arraylist thread safe such as through synchronized lists, I'm specifically asking if my method is a valid method or if there is some reason I should avoid it and actually use a synchronized list.
The proper way would be to synchronize the list with Collections.synchronizedList():
List list = Collections.synchronizedList(new ArrayList());
...
synchronized (list) {
Iterator i = list.iterator(); // Must be in synchronized block
while (i.hasNext())
foo(i.next());
}
If you are traversing way more than the number of times you update your list, you can also use CopyOnWriteArrayList.
If you don't mind occasional missing updates (or if they happen way too infrequently for the price of synchronization), your way is fine.
Is this a "proper" way of handling this issue?
If you do not mind getting an increase of concurrency at the expense of dropping the updates on error, then the answer is "yes". You do run the risk of not completing an update multiple times in a row, when significant additions and removals to the list happen often.
On the other hand, when the frequency of updates is significantly higher than the frequency of adding/removing an object, this solution sounds reasonable.
Should I perhaps be [using synchronized]?
This is also a viable solution. The difference is that an update would no longer be able to proceed when an update is in progress. This may not be desirable when the timing of calls to update is critical (yet it is not critical to update everything on every single call).
Some people consider it as a duplicate of all the generic synchronization questions. I think this is not the case. You are asking for a very specific constellation, and whether your solution is "OK", in this sense.
Based on what you described, the actual goal seems to be clear: You want to quickly and concurrently iterate over the entities to call the update method, and avoid any synchronization overhead that may be implied by using Collections#synchronizedList or similar approaches.
Additionally, I assume that the main idea behind the solution that you proposed was that the update calls have to be done very often and as fast as possible, whereas adding or removing entities happens "rarely".
So, adding and removing elements is an exception, compared to the regular operations ;-)
And (as dasblinkenlight already pointed out in his answer) for such a setup, the solution of catching and ignoring the ConcurrentModificationException is reasonable, but you should be aware of the consequences.
It might happen that the update method of some entities is called, and then the loop bails out due to the ConcurrentModificationException. You should be absolutely sure that this does not have undesirable side-effects. Depending on what update actually does, this might, for example, cause some entities to move faster over the screen, and others to not move at all, because their update calls had been missed due to several ConcurrentModificationExceptions. This may be particularly problematic if adding and removing entities is not an operation that happens rarely: If one thread constantly adds or removes elements, then the last elements of the list may never receive an update call at all.
If you want some "justification by example": I first encountered this pattern in the JUNG Graph Library, for example, in the SpringLayout class and others. When I first saw it, I cringed a little, because at the first glance it looks horribly hacky and dangerous. But here, the justification is the same: The process has to be as fast as possible, and modifications to the graph structure (which would cause the exception) are rare. Note that the JUNG guys actually do recursive calls to the respective method when the ConcurrentModificationException happens - simply because they can't always assume the method to be called constantly by another thread. This, in turn, can have nasty side-effects: If another thread does constant modifications, and the ConcurrentModificationException is thrown each time when the method is called, then this will end with a StackOverflowError... But this is not the case for you, fortunately.
What happens when a map has to be resized to accommodate more items? What will happen if another thread calls get() when resizing is underway?
Your question on what will happen if another thread calls get() while the map is being resized reveals a general lack of intuition about the Java Memory Model. Specifically, if the map implementation you use is not thread-safe, it will be irrelevant when get() is called—in the middle of resizing or while the map is sitting completely idle: the call from another thread will always be broken due to write visibility issues.
Simply put, there are only two conditions in Java: thread-safe and not thread-safe. No further details are necessary.
The answer is one word it will got crashed and will throw Exception might be UnsupportedOperationException.the reason behind this is ..
As HashMap is not thread safe so when another thread will try to read the data, it will not allow and stuff will occurs which completely depend upon implementation.
I will try to explain what will happen there. Just in case - one should not do this.
As far as I understand HashMap implementation, if you are writing to it with a single thread, it would not crash, but might not find your item. For two concurrent threads it would crash very early because of ConcurrentModificationException.
Why it won't crash
Resize operation in HashMap is sort of atomic - it first allocates your table and re-indexes items into it and then replaces an existing table with a new one
Table size grows, but does not shrink, so even with broken indexing function you are still within bounds.
get will never throw a null pointer exception - one can check that getEntry(Object key) would never throw a NullPointerException.
Why it might not find your item
Because table[hash(key)] operation is non-atomic. You might run into a situation when you have a hash value for an old map, but after resize it's already a new one.
I was reading about ConcurrentHashMap.
I read that it provides an Iterator that requires no synchronization and even allows the Map to be modified during iteration and thus there will be no ConcurrentModificationException.
I was wondering if this is a good thing as I might not get the element, put into ConcurrentHashMap earlier, during iteration as another thread might have changed it.
Is my thinking correct? If yes, is it good or bad?
I was wondering if this is a good thing as I might not get the element, put into ConcurrentHashMap earlier, during iteration as another thread might have changed it.
I don't think this should be a concern - the same statement is true if you use synchronization and the thread doing the iteration happens to grab the lock and execute it's loop prior to the thread that would insert the value.
If you need some sort of coordination between your threads to ensure that some action takes place after (and only after) another action, then you still need to manage this coordination, regardless of the type of Map used.
Usually, the ConcurrentHashMap weakly consistent iterator is sufficient. If instead you want a strongly consistent iterator, then you have a couple of options:
The ctrie is a hash array mapped trie that provides constant time snapshots. There is Java source code available for the data structure.
Clojure has a PersistentHashMap that you can use - this lets you iterate over a snapshot of the data.
Use a local database, e.g. HSQLDB to store the data instead of using a ConcurrentHashMap. Use a composite primary key of key|timestamp, and when you "update" a value you instead store a new entry with the current timestamp. To get an iterator, retrieve a resultset with a where timetamp < System.currentTimeMillis() clause, and iterate over the resultset.
In either case you're iterating over a snapshot, so you've got a strongly consistent iterator; in the former case you run the risk of running out of memory, while the latter case is a more complex solution.
The whole point of concurrent -anything is that you acknowledge concurrent activity, and don't trust that all access is serialized. With most collections, you cannot expect inter-element consistency without working for it.
If you don't care about seeing the latest data, but want a consistent (but possibly old) view of data, have a look at purely functional structures like Finger Trees.
I have a list of personId. There are two API calls to update it (add and remove):
public void add(String newPersonName) {
if (personNameIdMap.get(newPersonName) != null) {
myPersonId.add(personNameIdMap.get(newPersonName)
} else {
// get the id from Twitter and add to the list
}
// make an API call to Twitter
}
public void delete(String personNAme) {
if (personNameIdMap.get(newPersonName) != null) {
myPersonId.remove(personNameIdMap.get(newPersonName)
} else {
// wrong person name
}
// make an API call to Twitter
}
I know there can be concurrency problem. I read about 3 solutions:
synchronized the method
use Collections.synchronizedlist()
CopyOnWriteArrayList
I am not sure which one to prefer to prevent the inconsistency.
1) synchronized the method
2) use Collections.synchronizedlist
3) CopyOnWriteArrayList ..
All will work, it's a matter of what kind of performance / features you need.
Method #1 and #2 are blocking methods. If you synchronize the methods, you handle concurrency yourself. If you wrap a list in Collections.synchronizedList, it handles it for you. (IMHO #2 is safer -- just be sure to use it as the docs say, and don't let anything access the raw list that is wrapped inside the synchronizedList.)
CopyOnWriteArrayList is one of those weird things that has use in certain applications. It's a non-blocking quasi-immutable list, namely, if Thread A iterates through the list while Thread B is changing it, Thread A will iterate through a snapshot of the old list. If you need non-blocking performance, and you are rarely writing to the list, but frequently reading from it, then perhaps this is the best one to use.
edit: There are at least two other options:
4) use Vector instead of ArrayList; Vector implements List and is already synchronized. However, it's generally frowned, upon as it's considered an old-school class (was there since Java 1.0!), and should be equivalent to #2.
5) access the List serially from only one thread. If you do this, you're guaranteed not to have any concurrency problems with the List itself. One way to do this is to use Executors.newSingleThreadExecutor and queue up tasks one-by-one to access the list. This moves the resource contention from your list to the ExecutorService; if the tasks are short, it may be fine, but if some are lengthy they may cause others to block longer than desired.
In the end you need to think about concurrency at the application level: thread-safety should be a requirement, and find out how to get the performance you need with the simplest design possible.
On a side note, you're calling personNameIdMap.get(newPersonName) twice in add() and delete(). This suffers from concurrency problems if another thread modifies personNameIdMap between the two calls in each method. You're better off doing
PersonId id = personNameIdMap.get(newPersonName);
if (id != null){
myPersonId.add(id);
}
else
{
// something else
}
Collections.synchronizedList is the easiest to use and probably the best option. It simply wraps the underlying list with synchronized. Note that multi-step operations (eg for loop) still need to be synchronized by you.
Some quick things
Don't synchronize the method unless you really need to - It just locks the entire object until the method completes; hardly a desirable effect
CopyOnWriteArrayList is a very specialized list that most likely you wouldn't want since you have an add method. Its essentially a normal ArrayList but each time something is added the whole array is rebuilt, a very expensive task. Its thread safe, but not really the desired result
Synchronized is the old way of working with threads. Avoid it in favor of new idioms mostly expressed in the java.util.concurrent package.
See 1.
A CopyOnWriteArrayList has fast read and slow writes. If you're making a lot of changes to it, it might start to drag on your performance.
Concurrency isn't about an isolated choice of what mechanism or type to use in a single method. You'll need to think about it from a higher level to understand all of its impacts.
Are you making changes to personNameIdMap within those methods, or any other data structures access to which should also be synchronized? If so, it may be easiest to mark the methods as synchronized; otherwise, you might consider using Collections.synchronizedList to get a synchronized view of myPersonId and then doing all list operations through that synchronized view. Note that you should not manipulate myPersonId directly in this case, but do all accesses solely through the list returned from the Collections.synchronizedList call.
Either way, you have to make sure that there can never be a situation where a read and a write or two writes could occur simultaneously to the same unsynchronized data structure. Data structures documented as thread-safe or returned from Collections.synchronizedList, Collections.synchronizedMap, etc. are exceptions to this rule, so calls to those can be put anywhere. Non-synchronized data structures can still be used safely inside methods declared to be synchronized, however, because such methods are guaranteed by the JVM to never run at the same time, and therefore there could be no concurrent reading / writing.
In your case from the code that you posted, all 3 ways are acceptable. However, there are some specific characteristics:
#3: This should have the same effect as #2 but may run faster or slower depending on the system and workload.
#1: This way is the most flexible. Only with #1 can you make the the add() and delete() methods more complex. For example, if you need to read or write multiple items in the list, then you cannot use #2 or #3, because some other thread can still see the list being half updated.
Java concurrency (multi-threading) :
Concurrency is the ability to run several programs or several parts of a program in parallel. If a time consuming task can be performed asynchronously or in parallel, this improve the throughput and the interactivity of the program.
We can do concurrent programming with Java. By java concurrency we can do parallel programming, immutability, threads, the executor framework (thread pools), futures, callables and the fork-join framework programmings.