I need to iterate on FastMap.values.
My problem is that basic loop fails over NullPointerException
basic loop
Collection<Order> orders = myObject.getOpenOrders();
for (Order order : orders) {
}
problem is that another thread in my system edits the fastmap
It is adding and removing elements to it and I get the NullPointerException.
Rarely, but It should be solved.
So i added Null check
Collection<Order> orders = myObject.getOpenOrders();
for (Order order : orders) {
if (order != null )
}
and still I get the NullPointerExcetion
So I tried iterate it as follows
FastMap<String, Order> openOrders = myObject.getOpenOrdersMap();
for (FastMap.Entry<String, Order> e = openOrders.head(), end = openOrders.tail(); (e = e.getNext()) != end && e != null;) {
Order order = e.getValue();
}
But then the loop stops when it gets to null instead of throwing NullPointerExcetion.
And this is also a problem, since I need to iterate all of the elements.
I assume that the problem is that the for iteration uses values() and it is actually a pointer to the list.
I tried to copy the list but then I also get NullPointerExcetion in the copy process.
any sugggestions
BTW: I know that changing the whole design is the best solution and using locks in every insert and read. but is there some smaller change I can make in order to solve my problem?
Any idea?
This is not JDK classes? (Javolution?)
From what doc says :
If the map is marked shared then all operations are thread-safe
including iterations over the map's collections
So you may encounter some concurrency problem ?
Try that :
FastMap<String, Order> openOrders = myObject.getOpenOrdersMap().shared()
If you have a multi threaded environment you MUST absolutely use some synchronization, especially when working with iterators.
You cannot iterate a collection while another thread is editing it, this will invalidate iterators and break everything.
So swallow the pill and use some synchronization, this situation requires it.
Related
I am getting ConcurrentModificationException when executing the following code:
public void refreshAvailableCaseSettings() throws Exception {
//getAvailableCases() returns reference to the instance variable
LinkedHashMap<Integer, CaseSetting> cases = getAvailableCases();
/* java.util.ConcurrentModificationException even after trying entryset()
for(Map.Entry<Integer, CaseSetting> entry : cases.entrySet()){
entry.getValue().refresh(false);
}
*/
// java.util.ConcurrentModificationException
Iterator casesIterator = cases.values().iterator();
while (casesIterator.hasNext()) {
CaseSetting caseSetting = casesIterator.next();
//refresh() updates caseSetting state by getting setting info from DB
caseSetting.refresh(false);
}
}
Error:
java.util.ConcurrentModificationException
at java.util.LinkedHashMap$LinkedHashIterator.nextNode(LinkedHashMap.java:719)
at java.util.LinkedHashMap$LinkedEntryIterator.next(LinkedHashMap.java:752)
at java.util.LinkedHashMap$LinkedEntryIterator.next(LinkedHashMap.java:750)
I am not modifying the map by adding or removing elements. Please help me to understand the issue here.
The general contract for using iterators is this:
While an iterator over a collection is in progress, the underlying collection must not be modified.
You can do whatever you want on the elements of the collection, but you must not touch the collection itself.
You are getting a ConcurrentModificationException because a part of your code within the while loop does exactly that - it changes the underlying collection.
A standard approach is to either (a) create a new copy of the collection and iterate over this read-only copy, or (b) put the changes into a separate collection.
What refresh function does ? According to my understanding you are getting these only because you are modifided HashMap while iterating it,
I used to get these error when I used to put something in map, means size was increasing.
Also you need to put everything in question, like CaseSetting class
The below code is throwing concurrent Modification exception , the line where the exception is pointing is the first line of for loop
private synchronized void updateAllCacheValues() {
for (Map.Entry<Configurations, SalesConfiguration> entry : ConfigurationCache.entrySet()) {
Configurations conf = entry.getKey();
SalesConfiguration saleConfiguration = ConfigurationCache.get(conf);
Map<String, String> newMap = generateKeyValueMapFromConfigurations(conf);
lastLoadTimestamp = new Date();
saleConfiguration.setMap(newMap, lastLoadTimestamp);
}
logger.debug("Successfully updated all cached configurations., cache size " + ConfigurationCache.size() + "LAST_LOAD_TIME" + lastLoadTimestamp);
}
Below is the exception trace
java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextNode(HashMap.java:1429)
at java.util.HashMap$EntryIterator.next(HashMap.java:1463)
at java.util.HashMap$EntryIterator.next(HashMap.java:1461)
at com.learning.java.daily.updateAllCacheValues(ConfigurationLoader.java:237)
at com.learning.java.daily.updateAllCacheValues.impl.ConfigurationLoader.loadConfigurations(ConfigurationLoader.java:156)
I am not able to guess what could have caused this exception , because I wrote a sample test where I am modifying the attributes of the value of Map but concurrent modification exception did not appear.
To answer the question in your title: No, modifying value objects in your HashMap while iterating the entry set cannot alone produce a ConcurrentModificationException. There must be something else going on.
Could either the call to generateKeyValueMapFromConfigurations() or to saleConfiguration.setMap() modify the map? Might it be that your ConfigurationCache could be modified by some other, concurrent thread?? I know I am just guessing, it’s the best we can do with the information at hand.
I think this line is the culprit.
saleConfiguration.setMap(newMap, lastLoadTimestamp);
Literally ConcurrentModificationException comes when you update a map entry while iterating through it. Could you please comment out that line and check whether you are getting the exception anymore? If not you need to change the logic so that you do not change the map entries while iterating through them.
You are iterating entryset. So, modifying the entryset values is not allowed while you are iterating it. Use ConcurrentHashMap instead of HashMap.
Here is the analysis.
We can modify the key/value's attributes. But we cannot modify the map by adding or deleting elements to the map while we iterate. Suppose a map has five elements, while we iterate it, we can neither add an element to map nor delete an element from map. For this, we should use ConcurrentHashMap.
Hope it is more clear now.
occupants is an java.util.ArrayList.
I am iterating through it like so
public void hitOccupants(SnakeController snakeController){
ListIterator<Hitable> i = occupants.listIterator();
while( i.hasNext()){
Hitable hitable = i.next();
if(hitable.hit(snakeController)){//returns true if it should be deleted
i.remove();
}
}
}
I can figure out why this would give a ConcurrenModificationException...
I am using libgdx, so there could be a threading issue. This is my first libgdx project though, so I am not sure. Running the code on android. Stacktrace:
java.util.ConcurrentModificationException
at java.util.AbstractList$SimpleListIterator.remove(AbstractList.java:71)
at com.ninovanhooff.snake.model.BoardSpace.hitOccupants(BoardSpace.java:65)
at com.ninovanhooff.snake.controller.SnakeController.act(SnakeController.java:77)
at com.ninovanhooff.snake.controller.BoardController.act(BoardController.java:72)
at com.ninovanhooff.snake.GameActor$2.act(GameActor.java:77)
at com.badlogic.gdx.scenes.scene2d.Actor.act(Actor.java:86)
at com.badlogic.gdx.scenes.scene2d.Group.act(Group.java:48)
at com.badlogic.gdx.scenes.scene2d.Group.act(Group.java:48)
at com.badlogic.gdx.scenes.scene2d.Stage.act(Stage.java:225)
at com.ninovanhooff.snake.SnakeGame.render(SnakeGame.java:66)
at com.badlogic.gdx.backends.android.AndroidGraphics.onDrawFrame(AndroidGraphics.java:510)
at android.opengl.GLSurfaceView$GLThread.guardedRun(GLSurfaceView.java:1516)
at android.opengl.GLSurfaceView$GLThread.run(GLSurfaceView.java:1240)
ConcurrentModificationException as per java docs
This exception may be thrown by methods that have detected concurrent
modification of an object when such modification is not permissible.
For example, it is not generally permissible for one thread to modify
a Collection while another thread is iterating over it. In general,
the results of the iteration are undefined under these circumstances.
Some Iterator implementations (including those of all the general
purpose collection implementations provided by the JRE) may choose to
throw this exception if this behavior is detected. Iterators that do
this are known as fail-fast iterators, as they fail quickly and
cleanly, rather that risking arbitrary, non-deterministic behavior at
an undetermined time in the future.
In short one cannot modify list while iterating through it .
There are multiple ways :
1) Create a new list
2) Use of different collections such as Map or Set.
3) Change some state in hitable object
Hitable.hit() adds an element to the array, thus causing concurrent modification.
To place this into context:
I am making the classic snake game. When the snake hits the Hitable apple:
1) The apple has to be removed from the BoardSpace's occupants //BoardSpace == tile
this is done in the code discussed here
2) The snakeś body is elongated into the BoardSpace where the apple was. The snake is now an occupant of the BoardSpace, thus the snake is added to occupants.
Both these are done inside Apple.hit(), ergo: concurrent modification.
Solution: I used a CopyOnWriteArrayList to trave occupants in the original order and contents, and added the elements to be removed for later removal.
CopyOnWriteArrayList<Hitable> occupantsSnapshot = new CopyOnWriteArrayList<Hitable>(occupants);
ArrayList<Hitable> removals = new ArrayList<Hitable>();
Iterator<Hitable> i = occupantsSnapshot.iterator();
while (i.hasNext()) {
Hitable hitable = i.next();
boolean remove = hitable.hit(snakeController);
if (remove) {//returns true if it should be deleted
removals.add(hitable);
}
}
for(Hitable hitable: removals){
occupants.remove(hitable);
boardController.removeHitable(hitable);
}
For educational purposes: Bitbucket snapshot
See AppleController and SnakeBodyParts.
You're not allowed to remove items from an ArrayList while iterating through it. This is done for a number of reasons. Most obviously, an ArrayList is dynamic, so if you remove the object at index 2, the object at index 3 now shifts down to index 2. This matters because, while iterating, it could easily lead to an out-of-bounds exception if you were to shorten the length of the list without re-adjusting your iterator.
There are a number of ways to get around this, but there's a standing rule: if you're iterating, you can't remove. So either think of a way to do it without iterating (a while loop that only increments the index when hitable.hit is false) or store the objects to be removed in a separate list and then remove those items one by one.
As per Sun ,
"Iterator.remove is the only safe way to modify a collection during
iteration; the behavior is unspecified if the underlying collection is
modified in any other way while the iteration is in progress."
I have two questions :
What makes this operation "Iterator.remove()" stable than the others ?
Why did they provide a "Collection.remove()" method if it will not be useful in most of the use-cases?
First of all, Collection.remove() is very useful. It is applicable in a lot of use cases, probably more so than Iterator.remove().
However, the latter solves one specific problem: it allows you to modify the collection while iterating over it.
The problem solved by Iterator.remove() is illustrated below:
List<Integer> l = new ArrayList<Integer>(Arrays.asList(1, 2, 3, 4));
for (int el : l) {
if (el < 3) {
l.remove(el);
}
}
This code is invalid since l.remove() is called during iteration over l.
The following is the correct way to write it:
Iterator<Integer> it = l.iterator();
while (it.hasNext()) {
int el = it.next();
if (el < 3) {
it.remove();
}
}
If you're iterating over a collection and use:
Collection.remove()
you can get runtime errors (specifically ConcurrentModifcationException) because you're changing the state of the object used previously to construct the explicit series of calls necessary to complete the loop.
If you use:
Iterator.remove()
you tell the runtime that you would like to change the underlying collection AND re-evaluate the explicit series of calls necessary to complete the loop.
As the documentation you quoted clearly states,
Iterator.remove is the only safe way to modify a collection during iteration
(emphasis added)
While using am iterator, you cannot modify the collection, except by calling Iterator.remove().
If you aren't iterating the collection, you would use Collection.remove().
What makes this operation "Iterator.remove()" stable than the others ?
It means that iterator knows you removed the element so it won't produce a ConcurrentModifcationException.
Why did they provide a "Collection.remove()" method if it will not be useful in most of the use-cases ?
Usually you would use Map.remove() or Collection.remove() as this can be much more efficient than iterating over every objects. If you are removing while iterating often I suspect you should be using different collections.
Is just a design choice. It would have been possible to specify a different behavior (i.e. the iterator has to skip values that were removed by Collection.remove()), but that would have made the implementation of the collection framework much more complex. So the choice to leave it unspecified.
It's quite useful. If you know the object you want to remove, why iterate?
From what I understand, the Collection.remove(int index) will also return the removed object. Iterative.remove() will not.
I'm iterating over a JRE Collection which enforces the fail-fast iterator concept, and thus will throw a ConcurrentModificationException if the Collection is modified while iterating, other than by using the Iterator.remove() method . However, I need to remove an object's "logical partner" if the object meets a condition. Thus preventing the partner from also being processed. How can I do that? Perhaps by using better collection type for this purpose?
Example.
myCollection<BusinessObject>
for (BusinessObject anObject : myCollection)
{
if (someConditionIsTrue)
{
myCollection.remove(anObjectsPartner); // throws ConcurrentModificationException
}
}
Thanks.
It's not a fault of the collection, it's the way you're using it. Modifying the collection while halfway through an iteration leads to this error (which is a good thing as the iteration would in general be impossible to continue unambiguously).
Edit: Having reread the question this approach won't work, though I'm leaving it here as an example of how to avoid this problem in the general case.
What you want is something like this:
for (Iterator<BusinessObject> iter = myCollection.iterator; iter.hasNext(); )
{
BusinessObject anObject = iter.next();
if (someConditionIsTrue)
{
iter.remove();
}
}
If you remove objects through the Iterator itself, it's aware of the removal and everything works as you'd expect. Note that while I think all standard collections work nicely in this respect, Iterators are not required to implement the remove() method so if you have no control over the class of myCollection (and thus the implementation class of the returned iterator) you might need to put more safety checks in there.
An alternative approach (say, if you can't guarantee the iterator supports remove() and you require this functionality) is to create a copy of the collection to iterate over, then remove the elements from the original collection.
Edit: You can probably use this latter technique to achieve what you want, but then you still end up coming back to the reason why iterators throw the exception in the first place: What should the iteration do if you remove an element it hasn't yet reached? Removing (or not) the current element is relatively well-defined, but you talk about removing the current element's partner, which I presume could be at a random point in the iterable. Since there's no clear way that this should be handled, you'll need to provide some form of logic yourself to cope with this. In which case, I'd lean towards creating and populating a new collection during the iteration, and then assigning this to the myCollection variable at the end. If this isn't possible, then keeping track of the partner elements to remove and calling myCollection.removeAll would be the way to go.
You want to remove an item from a list and continue to iterate on the same list. Can you implement a two-step solution where in step 1 you collect the items to be removed in an interim collection and in step 2 remove them after identifying them?
Some thoughts (it depends on what exactly the relationship is between the two objects in the collection):
A Map with the object as the key and the partner as the value.
A CopyOnWriteArrayList, but you have to notice when you hit the partner
Make a copy into a different Collection object, and iterate over one, removing the other. If this original Collection can be a Set, that would certaily be helpful in removal.
You could try finding all the items to remove first and then remove them once you have finished processing the entire list. Skipping over the deleted items as you find them.
myCollection<BusinessObject>
List<BusinessObject> deletedObjects = new ArrayList(myCollection.size());
for (BusinessObject anObject : myCollection)
{
if (!deletedObjects.contains(anObject))
{
if (someConditionIsTrue)
{
deletedObjects.add(anObjectsPartner);
}
}
}
myCollection.removeAll(deletedObjects);
CopyOnWriteArrayList will do what you want.
Why not use a Collection of all the original BusinessObject and then a separate class (such as a Map) which associates them (ie creates partner)? Put these both as a composite elements in it's own class so that you can always remove the Partner when Business object is removed. Don't make it the responsibility of the caller every time they need to remove a BusinessObject from the Collection.
IE
class BusinessObjectCollection implements Collection<BusinessObject> {
Collection<BusinessObject> objects;
Map<BusinessObject, BusinessObject> associations;
public void remove(BusinessObject o) {
...
// remove from collection and dissasociate...
}
}
The best answer is the second, use an iterator.