Develop Reference

ConcurrentModificationException while calling size() on a sublist

I am trying to save a bunch of SQL transaction. I am in a context of ESB routes, transferring from a SQL source to a SQL target, and the order of the SQL transactions is not guaranteed, so you can have a SQL update before the object was inserted.
Due to the architecture, I'm saving those SQL transactions 1000 by 1000 (I'm using a messageQueue). So some of these can fail, and I re-route them in order to retry or reject them. To improve efficiency, I'm willing to improve the older system, where if the 1000 fail, you save 1 by 1, to implements dichotomia (if the save fail, you split the list and try again), via recursivity. I am also tracking an attribute of my objects, thanks to another list (objectsNo) for further operations.
However I am getting a ConcurrentModificationException when in my first recursivity, when calling objectsList.size(). How can I avoid it ? I'm also opened, and would be very thankful to any solutions which would provide another way than dichotomia to improve efficiency (and would by such bypass my issue).
Suppressed: java.util.ConcurrentModificationException: null
at java.util.ArrayList$SubList.checkForComodification(ArrayList.java:1231)
at java.util.ArrayList$SubList.size(ArrayList.java:1040)
at fr.company.project.esbname.mariadb.MariaDbDatabase.saveObjectWithDichotomie(MariaDbDatabase.java:398)
at fr.company.project.esbname.mariadb.MariaDbDatabase.saveObjectWithDichotomie(MariaDbDatabase.java:404)
at fr.company.project.esbname.mariadb.MariaDbDatabase.saveObject(MariaDbDatabase.java:350)
at sun.reflect.GeneratedMethodAccessor324.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.camel.component.bean.MethodInfo.invoke(MethodInfo.java:472)
at org.apache.camel.component.bean.MethodInfo$1.doProceed(MethodInfo.java:291)
at org.apache.camel.component.bean.MethodInfo$1.proceed(MethodInfo.java:264)
at org.apache.camel.component.bean.BeanProcessor.process(BeanProcessor.java:178)
at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:77)
at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:541)
... 22 common frames omitted
I tried to understand, but there should not be any mistake. Even if I used recursivity, it stays single-threaded. I considered that the issue could be with hibernate (some requests from the save which failed could stay in the cache, and lock modification), but the issue is with the size, which is on a sublist of the original list.
private List<String> saveObjectWithDichotomie(List<Object> objects,
List<String> objectsNo,
Exchange exchange) throws JsonProcessingException {
try {
objectRepository.save(objects);
return objectsNo;
} catch (DataIntegrityViolationException e) {
if (objects.size() == 1) {
objectsNo.clear();
errorProcessor.sendErrorToRejets(objects.get(0), exchange, e);
return objectsNo;
} else {
List<Object> objectsFirstHalf = objects.subList(0, objects.size()/2);
List<Object> objectsSecondHalf = objects.subList(objects.size()/2, objects.size());
List<String> objectsNoFirstHalf = objectsNo.subList(0, objectsNo.size()/2);
List<String> objectsNoSecondHalf = objectsNo.subList(objectsNo.size()/2, objectsNo.size());
objectsNo.clear();
objectsNo.addAll(
saveObjectWithDichotomie(objects, objectsNoFirstHalf, exchange)
);
objectsNo.addAll(
saveObjectWithDichotomie(objects, objectsNoSecondHalf, exchange)
);
return objectsNo;
}
}
}

If you would read the documentation of sublist is clearly says:
The returned list is backed by this list, so non-structural changes in the returned list are reflected in this list, and vice-versa.
That being the reason for your Exception (no need for multiple threads for this to happen). Thus when you create a new List, create it via:
List<Object> objectsFirstHalf = new ArrayList<>(objects.subList(0, objects.size()/2));

Two things:
ConcurrentModificationException does not mean that the list was modified by another thread, but that something is trying to access the list in an expected state but it was changed in the meantime.
subList does not create an actual new list, it creates a view on the original list. That means that you cannot change the original list without making the retrieved sublist invalid.
So,
objectsNo.clear();
is your problem.
See this MCVE:
public class Sublist {
public static void main(String[] args) {
List<String> list = new ArrayList<>(
IntStream.range(0, 100).mapToObj(Integer::toString).collect(Collectors.toList()));
List<String> sublist = list.subList(10, 20);
// outputs "15"
System.out.println(sublist.get(5));
list.clear();
// throws ConcurrentModificationException
System.out.println(sublist.get(5));
}
}

Multi Threading with ArrayList() got ArrayIndexOutOfBoundsException [duplicate]

This question already has answers here:
What are the possible problems caused by adding elements to unsynchronized ArrayList's object by multiple threads simultaneously?
(4 answers)
Java MultiThreading behaviour explanation
(4 answers)
Why is arraylist's size not right when multiple threads add elements into it?
(5 answers)
Data race in Java ArrayList class
(3 answers)
Closed 4 years ago.
I've made the following code to run multiple thread through a non thread safe object (here an ArrayList) :
import java.time.LocalDateTime;
import java.util.List;
import java.util.ArrayList;
public class A implements Runnable {
String name;
static List<Integer> list = new ArrayList();
private static Object lock = new Object();
A(String name) {
this.name = name;
}
#Override
public void run() {
for(int i = 1; i <= 1000; i++) {
list.add(i);
}
System.out.println(list.size());
}
}
I was expecting this code just to produce wrong answers since ArrayList is not thread-safe. But instead I get this error :
Exception in thread "Thread-1" 1003
2401
2799
3799
java.lang.ArrayIndexOutOfBoundsException: 109
at java.util.ArrayList.add(Unknown Source)
at threads.A.run(A.java:16)5123
at java.lang.Thread.run(Unknown Source)
Exception in thread "Thread-5" java.lang.ArrayIndexOutOfBoundsException: 4164
at java.util.ArrayList.add(Unknown Source)
at threads.A.run(A.java:16)
at java.lang.Thread.run(Unknown Source)
6123
Can anyone explain to me what is leading to this specific error?

Well, you are using a non thread-safe collection in a multi threaded environment without any synchronization.
Let's examine the add method, where you get the exception:
/**
* Appends the specified element to the end of this list.
*
* #param e element to be appended to this list
* #return <tt>true</tt> (as specified by {#link Collection#add})
*/
public boolean add(E e) {
ensureCapacityInternal(size + 1); // Increments modCount!!
elementData[size++] = e;
return true;
}
When multiple threads call this method at the same time, it is quite possible that ensureCapacityInternal(size + 1) verifies there is enough space for 1 new element, but then multiple threads try to add an element at the same time, so elementData[size++] throws ArrayIndexOutOfBoundsException for some of them.

ArrayList is not a thread-safe class.
Underlying storage for elements is Object[] which is an array. Any array requires the allocation space in memory which is predefined in the compile time. However, when an ArrayList "wants" to add the new element (beyond the underlying array bound), several things have to be done (without your knowledge). Underlying array gets a new (increased) length. Every element of the old array is copied to the new array, and then the new element is added. So, you can expect the ArrayIndexOutOfBoundsException exception when an ArrayList is used in multi-thread environment.
You are adding elements too fast so ArrayList#add() -> grow() -> newCapacity() can't calculate the correct capacity to allocate the memory for all of the elements coming in.
private void add(E e, Object[] elementData, int s) {
if (s == elementData.length)
elementData = grow();
elementData[s] = e;
size = s + 1;
}
At some point of time, the condition s == elementData.length inside ArrayList#add says that there is a space for a new element A. Immediately after that other threads put their elements into the list. Now there is no space for A and elementData[s] = e; throws an exception.

ConcurrentModificationException -> How can I change my code to stop it from throwing this error? [duplicate]

This question already has answers here:
Iterating through a Collection, avoiding ConcurrentModificationException when removing objects in a loop
(31 answers)
Closed 5 years ago.
for (FPlayer p : fPlayer.getFaction().getOnline()) {
p.setFaction(null);
}
Basically, the getOnline method returns an array list of the current FPlayers that are online. When the FPlayer is removed from their faction, the faction is set to null (p.setFaction(null)).
I cannot think about how to change my code to stop it from throwing the ConcurrentModificationException. I have used an iterator but still, it.next().setFaction(null) still throws the same exception.
EDIT:
USING A LIST ITERATOR:
ListIterator<FPlayer> it = fPlayer.getFaction().getOnline().listIterator();
while (it.hasNext()) {
it.next().setFaction(null);
}
Caused by: java.util.ConcurrentModificationException
At the line
it.next().setFaction(null)
EDIT #2:
Set faction method:
public void setFaction(Faction faction) {
if (hasFaction()) {
this.faction.getOnline().remove(this);
}
this.faction = faction;
if (faction != null) {
this.faction.getOnline().add(this);
}
}

This is happening because while iterating you are removing the data from the list .
Couple of solutions .
If the list size is small convert it to array and then loop over
Use for loop for iteration .
for(int i=0;i<fPlayer.getFaction().getOnline().size();i++)
{
// Condition to check if true
if(true)
{
fPlayer.getFaction().getOnline().remove(i);
i--;
}
}

Yes, change your code so it doesn't change the collection inside the loop you are running. For example, create a copy of the collection before iterating.
for (Foo foo : new ArrayList(myFoos)) {
if (foo.isBar()) {
myFoos.remove(foo);
}
}
Iterating and changing the list without the new ArrayList() would have caused a ConcurrentModificationException

Java 8 Stream - is throwing stream has already been operated upon or closed [duplicate]

This question already has answers here:
Why is this java Stream operated upon twice?
(4 answers)
Closed 6 years ago.
I would like to implement a pagination in stream api.
I used if statement to modify the stream, but I was surprise when I encountered
java.lang.IllegalStateException: stream has already been operated upon or closed
Here is my code
#Override
public List<Foo> search(String[] names, Integer offset, Integer size) {
final Stream<Foo> fooStream = findAllFoos().stream();
if(offset!=null) {
fooStream
.skip(offset)
.limit(size);
}
if(ArrayUtils.isNotEmpty(names)) {
fooStream.filter(foo -> ArrayUtils.contains(names, foo.getName()));
}
final List<Foo> foos = fooStream.collect(Collectors.toList()); //this lines throws the exception, according to stacktrace
return foos;
}
Here is my Foo.class
public class Foo {
private String name;
//..setters and getters
}
Here is sample call to that search method
fooService.search(fooNamesArray, 0, 1);
fooService.search(fooNamesArray, null, null);

fooStream.skip(offset)
does nothing by itself. It returns a new Stream; it does not modify fooStream.
Generally speaking you can only use a Stream for one operation, though that might be a transformation into another Stream.
What you could do, though it would still make your code awkward, would be to write fooStream = fooStream.skip(offset).limit(size).

You may want to add all your validation before hand and combine all stream operations in one go.
List<Foo> foos = fooStream.skip(offset)
.limit(size)
.filter(foo -> ArrayUtils.contains(names, foo.getName()))
.collect(Collectors.toList());
In other case, you can take the result of each stream operation into another stream and operate on them.
Stream<Foo> skippedFooStream = Collections.<Foo>emptyList().stream();
if(offset!=null) {
skippedFooStream = fooStream
.skip(offset)
.limit(size);
}
if(ArrayUtils.isNotEmpty(names)) {
// take this into another temp stream and proceed
skippedFooStream.filter(foo -> ArrayUtils.contains(names, foo.getName()));
}

Java - weird ConcurrentModificationException [duplicate]

This question already has answers here:
Java Set gets full
(2 answers)
Closed 8 years ago.
I loop through a copy of a set of objects, but it still gives me an error starting at this method:
private static Set<Updated> updates = new HashSet<>();
public static Set<GameObject> getGameObjects() {
Set<GameObject> objs = new HashSet<>();
for (Updated up : new HashSet<Updated>(updates)) {
if (up instanceof GameObject)
objs.add((GameObject) up);
}
return objs;
}
(Where the for loop is).
This only happens when there are quite a few GameObjects, and never occurs when there's only a few (like 7). Thank you!

The problem is that somebody in another thread changes the updates set while it is being copied in new HashSet<Updated>(updates).
You cannot do this without synchronization. Or use ConcurrentHashMap instead of HashSet