I have a class like the following:
class Test
{
private LinkedList<Person> persons = new LinkedList<Person>;
public synchronized void remove(Person person)
{
persons.remove(person);
}
public List<Person> getAllPersons()
{
// Clients may iterate over the copy returned and modify the structure.
return new ArrayList<Person>(persons);
}
}
persons may be modified concurrently: one is via remove() by one thread and two via the shallow copied instance returned by getAllPersons().
I have tested the above scenario in a multithreaded environment to see if I can avoid ConcurrentModificationException by returning a shallow copy when getAllPersons() is called. It seemed to work. I have never once encountered a ConcurrentModificationException.
Why, in this case, does making only a shallow copy of persons avoid a ConcurrentModificationException?
A ConcurrentModificationException is thrown when a collection changes in a manner which invalidates open iterators. This usually happens when a collection which is not thread safe is accessed by multiple threads (although this is not the only cause)
There is still a small error in your code - to safely access a member which is not itself thread safe, you should synchronize on the getAllPersons method.
Assuming that is fixed -- because you are returning a copy, the collection itself cannot be modified by other callers (each gets their own copy). That means that you can never get a ConcurrentModificationException.
Note that this does not protect you against thread safety issues with your Person class, only the collections themselves. If Person is immutable, you should be OK.
In this case, a better solution would be to directly use a CopyOnWriteArrayList which implements similar semantics, but only copies when you actually write to the list - not every time you read from it.
That's because you are returning a copy of the List rather than the list itself. remove() is the only method which is modifying the actual list, accessible by multiple threads. Threads calling the getAllPersons() method will anyways getting a new list, so if they modify this list, its not going to change the original list.
So as your collection is not getting modified concurrently by threads you are not getting ConcurrentModificationException.
Related
How do I lock a data structure (such as List) when someone is iterating over it?
For example, let's say I have this class with a list in it:
class A{
private List<Integer> list = new ArrayList<>();
public MyList() {
// initialize this.list
}
public List<Integer> getList() {
return list;
}
}
And I run this code:
public static void main(String[] args) {
A a = new A();
Thread t1 = new Thread(()->{
a.getList().forEach(System.out::println);
});
Thread t2 = new Thread(()->{
a.getList().removeIf(e->e==1);
});
t1.start();
t2.start();
}
I don't have a single block of code that uses the list, so I can't use synchronized().
I was thinking of locking the getList() method after it has been called but how can I know if the caller has finished using it so I could unlock it?
And I don't want to use CopyOnWriteArrayList because of I care about my performance;
after it has been called but how can I know if the caller has finished using it so I could unlock it?
That's impossible. The iterator API fundamentally doesn't require that you explicitly 'close' them, so, this is simply not something you can make happen. You have a problem here:
Iterating over the same list from multiple threads is an issue if anybody modifies that list in between. Actually, threads are immaterial; if you modify a list then interact with an iterator created before the modification, you get ConcurrentModificationException guaranteed. Involve threads, and you merely usually get a CoModEx; you may get bizarre behaviour if you haven't set up your locking properly.
Your chosen solution is "I shall lock the list.. but how do I do that? Better ask SO". But that's not the correct solution.
You have a few options:
Use a lock
It's not specifically the iteration that you need to lock, it's "whatever interacts with this list". Make an actual lock object, and define that any interaction of any kind with this list must occur in the confines of this lock.
Thread t1 = new Thread(() -> {
a.acquireLock();
try {
a.getList().forEach(System.out::println);
} finally {
a.releaseLock();
}
});
t1.start();
Where acquireLock and releaseLock are methods you write that use a ReadWriteLock to do their thing.
Use CopyOnWriteArrayList
COWList is an implementation of java.util.List with the property that it copies the backing store anytime you change anything about it. This has the benefit that any iterator you made is guaranteed to never throw ConcurrentModificationException: When you start iterating over it, you will end up iterating each value that was there as the list was when you began the iteration. Even if your code, or any other thread, starts modifying that list halfway through. The downside is, of course, that it is making lots of copies if you make lots of modifications, so this is not a good idea if the list is large and you're modifying it a lot.
Get rid of the getList() method, move the tasks into the object itself.
I don't know what a is (the object you call .getList() on, but apparently one of the functions that whatever this is should expose is some job that you really can't do with a getList() call: It's not just that you want the contents, you want to get the contents in a stable fashion (perhaps the method should instead have a method that gives you a copy of the list), or perhaps you want to do a thing to each element inside it (e.g. instead of getting the list and calling .forEach(System.out::println) on it, instead pass System.out::println to a and let it do the work. You can then focus your locks or other solutions to avoid clashes in that code, and not in callers of a.
Make a copy yourself
This doesn't actually work, even though it seems like it: Immediately clone the list after you receive it. This doesn't work, because cloning the list is itself an operation that iterates, just like .forEach(System.out::println) does, so if another thread interacts with the list while you are making your clone, it fails. Use one of the above 3 solutions instead.
I'm getting an java.util.ConcurrentModificationException.
The related code is:
for (Iterator<Audit> it = this.pendingAudits.iterator(); it.hasNext();) {
// Do something
it.remove();
}
When it.remove() is called it throws me this exception.
This code is inside an #Serviceannotated class:
#Service
public class AuditService {
public AuditService(
this.pendingAudits = new ArrayList<Audit>();
}
public void flush(Audit... audits) {
this.pendingAudits.addAll(Arrays.asList(audits));
try {
for (Iterator<Audit> it = this.pendingAudits.iterator(); it.hasNext();) {
// Do something
it.remove();
}
}
}
}
The problem appears when two requests reach the code.
How could I avoid this concurrent access exception?
First things first, this is not a Spring-related problem. It's just a problem with a concurrent modification of one not-so-concurrent-friendly ArrayList class.
The simplest solution possible would be to synchronize access to the method that modifies it.
public synchronized void flush(Audit... audits) { }
Keep in mind that it will enforce the sequential execution of the flush method which imposes a huge performance penalty.
Sidenote, it's not enough to synchronize the collection itself by using Collections.synchronizedList - iterator instances returned by synchronized wrappers require manual synchronization.
Isn't it obvious? You are sharing data without proper synchronization.
#Service annotated class generally is a singleton scope class and hence the same instance will be shared among all calling threads.
This results into all threads accessing the flush method on the same instance.
And guess what?
Your flush method is trying to modify an ArrayList which is a member variable. This makes it unsafe in a multithreaded scenario.
It's a good time to revisit the documentation of ArrayList which tells a lot more things about it's iterator.
Note that this implementation is not synchronized. If multiple threads access an ArrayList instance concurrently, and at least one of the threads modifies the list structurally, it must be synchronized externally. (A structural modification is any operation that adds or deletes one or more elements, or explicitly resizes the backing array; merely setting the value of an element is not a structural modification.) This is typically accomplished by synchronizing on some object that naturally encapsulates the list. If no such object exists, the list should be "wrapped" using the Collections.synchronizedList method. This is best done at creation time, to prevent accidental unsynchronized access to the list:
List list = Collections.synchronizedList(new ArrayList(...));
The iterators returned by this class's iterator and listIterator methods are fail-fast: if the list is structurally modified at any time after the iterator is created, in any way except through the iterator's own remove or add methods, the iterator will throw a ConcurrentModificationException. Thus, in the face of concurrent modification, the iterator fails quickly and cleanly, rather than risking arbitrary, non-deterministic behavior at an undetermined time in the future.
Note that the fail-fast behavior of an iterator cannot be guaranteed as it is, generally speaking, impossible to make any hard guarantees in the presence of unsynchronized concurrent modification. Fail-fast iterators throw ConcurrentModificationException on a best-effort basis. Therefore, it would be wrong to write a program that depended on this exception for its correctness: the fail-fast behavior of iterators should be used only to detect bugs.
I think you probably put Audit into the list pendingAudits somewhere and you want to flush them all when you call the flush(Audit...) method, you can use ConcurrentLinkedQueue instead of ArrayList.
public AuditService(
this.pendingAudits = new ConcurrentLinkedQueue<Audit>();
}
public void flush(Audit... audits) {
this.pendingAudits.addAll(Arrays.asList(audits));
try {
Audit audit;
while ((audit = this.pendingAudits.poll) != null) {
// Do something
}
}
}
I am not sure how to properly use the Collections.synchronizedList() implementation.
I have these two:
public synchronized static List<CurrencyBox> getOrderList() {
return Collections.synchronizedList(orderList);
}
and
public static List<CurrencyBox> getOrderList() {
return Collections.synchronizedList(orderList);
}
So as far as I understood, synchronizedList really returns the orderList and not a copy, correct?
So If I want to gurantee atomic operations, like add and remove, which of the implementation above is correct?
And does something maybe changes with Java9? Or is it still the way to go or have you any other suggestion?
Thank you
Without context it's a bit hard to tell, from the snippets provided neither give you guaranteed atomic operations.
The documentation states:
Returns a synchronized (thread-safe) list backed by the specified
list. In order to guarantee serial access, it is critical that all
access to the backing list is accomplished through the returned list.
So even if you synchronize the method the best you'll get is a guarantee that no two objects are creating the synchronized list at the same time.
You need to wrap the original orderList with Collections.synchronizedList to begin with and return the stored result of that each time.
private static List<CurrencyBox> orderList = Collections.synchronizedList(new ArrayList<CurrencyBox>());
public static List<CurrencyBox> getOrderList() {
return orderList
}
A synchronized list only synchronized methods of this list.
It means a thread won't be able to modify the list while another thread is currently running a method from this list. The object is locked while processing method.
As an example, Let's say two threads run addAllon your list, with 2 different lists (A=A1,A2,A3, B=B1,B2,B3) as parameter.
As the method is synchronized, you can be sure those lists won't be merged randomly like A1,B1,A2,A3,B2,B3
You don't decide when a thread handover the process to the other thread so you can either get A1,A2,A3,B1,B2,B3 or B1,B2,B3,A1,A2,A3.
Credit : jhamon
In thread A, an ArrayList is created. It is managed from thread A only.
In thread B, I want to copy that to a new instance.
The requirement is that copyList should not fail and should return a consistent version of the list (= existed at some time at least during the copying process).
My approach is this:
public static <T> ArrayList<T> copyList(ArrayList<? extends T> list) {
List<? extends T> unmodifiableList = Collections.unmodifiableList(list);
return new ArrayList<T>(unmodifiableList);
}
Q1: Does that satisfy the requirements?
Q2: How can I do the same without Collections.unmodifiableList with proably iterators and try-catch blocks?
UPD. That is an interview question I was asked a year ago. I understand this a bad idea to use non-thread-safe collections like ArrayList in multithreaded environment
No. ArrayList is not thread safe and you are not using an explicit synchronization.
While you are executing the method unmodifiableList the first thread can modify the original list and you will have a not valid unmodifiable list.
The simplest way I think is the following:
Replace the List with a synchronized version of it.
On the copy list synchronize on the arrayList and make a copy
For example, something like:
List<T> l = Collections.synchronizedList(new ArrayList<T>());
...
public static <T> List<T> copyList(List<? extends T> list) {
List<T> copyList = null;
synchronized(list) {
copyList = new ArrayList<T>(list);
}
return copyList;
}
You should synchronize access to the ArrayList, or replace ArrayList with a concurrent collection like CopyOnWriteArrayList.
Without doing that you might end up with a copy that is inconsistent.
There is absolutely no way to create a copy of a plain ArrayList if the "owning" thread does not offer some protocol to do so.
Without any protocol, thread A can modify the list potentially at any time, meaning thread B never gets a chance to ensure that is sees a consistent state of the list.
To actually allow a consistent copy to be made, thread A must ensure that any modifications it has made are written to memory and are visible to other threads.
Normally, the VM is allowed to reorder instructions, reads and writes as it sees fit, provided no difference can be observed from within the thread executing the program. This includes, for example, delaying writes by holding values in CPU registers or on the local stack.
The only way to ensure that everything is consistently written to main menory, is for thread A to execute an instruction that presents a reordering barrier to the VM (e.g. synchronized block or volatile field access).
So without some cooperation from thread A, there is no way to ensure above conditions are guaranteed to be fulfilled.
Common methods of circumventing this are to synchronize access to the List by only using it in a safely wrapped form (Collections.synchronizedCollection), or use of a List implementation that has these guarantees built in (any type of concurrent list implementation).
The javadoc for Collections.unmodifiableList(...) says, "Returns an unmodifiable view of the specified list."
The key word there is "view". That means it does not copy the data. All it does is create a wrapper for the given list with mutators that all throw exceptions rather than modify the base list.
Yes, but I acually create new ArrayList(Collections.unmodif...), wouldn't this work?
Oops! I missed that. If you're going to copy the list, then there's no point in calling unmodifiableList(). The only code that will ever access the unmodifiable view is the code that's right there in the same method where it's created. You don't have to worry about that code modifying the list contents because you wrote it.
On the other hand, if you're going to copy the list when other threads could be updating the list, then you're going to need synchronized all around. Every place where code could update the list needs to be in a synchronized block, as does the code that makes the copy. Of course, all of those synchronized blocks must synchronize on the same object.
Some programmers will use the list object itself as the lock object. Others will prefer to use a separate object.
Q1: Does that satisfy the requirements?
If the provided list is modified while copying it using new ArrayList<T>(unmodifiableList), you will get a ConcurrentModificationException even if you wrapped it using Collections.unmodifiableList because the Iterator of an UnmodifiableList simply calls the Iterator of the wrapped list and here as it is a non thread safe list you can still get a ConcurrentModificationException.
What you could do is indeed use CopyOnWriteArrayList instead as it is a thread safe list implementation that provides consistent snapshots of the List when you try to iterate over it. Another way could be to make the Thread A push for other threads regularly a safe copy of it using new ArrayList<T>(myList) as it is the only thread that modifies it we know that while creating the copy no other thread will modify it so it would be safe.
Q2: How can I do the same without Collections.unmodifiableList with
probably iterators and try-catch blocks?
As mentioned above Collections.unmodifiableList is not helping here to make it thread safe, for me the only thing that could make sense is actually the opposite: the thread A (the only thread that can modify the list) creates a safe copy of your ArrayList using new ArrayList<T>(list) then it pushes to other threads an unmodified list of it using Collections.unmodifiableList(list).
Generally speaking you should avoid specifying implementations in your method's definition especially public ones, you should only use interfaces or abstract classes because otherwise you would provide an implementation details to the users of your API which is not expected. So here it should be List or Collection not ArrayList.
i have a list of string(tagList) which need to shared among multiple threads for reading, so i create a unmodifiable version of it and pass it to threads, i m not sure if it's thread safe, since threads only read that list so i guess it should be ok?
also when i pass that unmodifialble list to the threads, does it pass a single copy and shared by threads or does it create multiple copy and pass one copy to each thread?
here is my code:
final List<String> tList = Collections.unmodifiableList(tagList);
List<Future<Void>> calls = new ArrayList<Future<Void>>();
FileStatus[] fsta = _fileSystem.listStatus(p);
for (FileStatus sta : fsta) {
final Path path = new Path(sta.getPath(), "data.txt");
if (!_fileSystem.exists(path)) {
continue;
}
else {
calls.add(_exec.submit(new Callable<Void>() {
#Override
public Void call() throws Exception {
filterData(path, tList);
return null;
}
}));
}
}
This completely depends on whether underlying list is thread safe on read operations. Unmodifiable list just passes all read calls, such as size (), get (int) etc to underlying list without additional synchronization.
Imagine, for example, implementation of List which caches hash code instead of calculating it each time it is needed. For such implementation, hashCode () method is actually not read-only, because it may modify internally cached hash code value.
Another example is a flavor of LinkedList which caches reference to last accessed entry together with its index, so further attempts to access nearby elements will be performed much faster. For such implementation, method get (int) will be not read-only because it updates cached reference and index, and thus will probably be not thread safe.
It is thread safe (since it can't be modified). It passes the same copy around.
However, the wrapped List (tagList) is still NOT thread safe. You must not modify the wrapped List while it's being shared. The only reason the list returned by unmodifiableList() is safe is because modifications are not allowed through it to the wrapped List.
As you say that the list is unmodifialble; hence it will be thread safe.
When you pass an object you actually pass the reference to the object (and not the actual object). As there is a single copy and its not getting modified so it will remain thread safe.
Just one caution; never access tagList directly. Access it always through wrapped unmodifiable collection named as tList. This can be achieved if you encapsulate it properly.