I'm trying to figure out how to synchronize read/write access to a synchronized list from a different class.
A small example: I have a synchronized list in one class ListProvider and I access it in a different class Filter. As the name suggests, this class performs some filtering based on a (in)validation check isInvalid.
The filter method first gets the list reference, then collects the entries to remove in a temporary list to not run into concurrent modification issues, and finally removes the entries from the list:
public class Filter {
ListProvider listProvider;
...
public void filter() {
List<String> listProviderList = listProvider.getList();
List<String> entriesToRemove = new ArrayList<>();
// collect
for (String entry : listProviderList)
if (isInvalid(entry)) {
entriesToRemove.add(entry);
}
}
// remove
for (String entry : entriesToRemove) {
listProviderList.remove(entry);
}
}
}
My question: How can I make sure that no other thread modifies the list while filter does its reading and writing?
If it were Filter's own list, I'd just do:
synchronized(myList) {
// collect
// remove
}
but in this case I'm not sure what to use as a monitor.
but in this case I'm not sure what to use as a monitor.
To create a monitor for a specific task, it is a good pattern to use a private final Object:
private final Object listUpdateLock = new Object();
...
synchronized(listUpdateLock) {
...
}
It's important to make sure that ListProvider is private and that all accesses to the list are done within a synchronized block -- even if only reading from it.
In this case, you are updating the list, you could create a temporary list and then replace it when you are done. I'm not sure you can do that with ListProvider however. Then you could just make the list volatile.
Here it seems like you should use a lock. A lock is like synchronized but it's a bit more flexible. It doesn't require a surrounding block and it has some extended features. There are also some different kinds of locks. ReentrantLock is much like synchronized.
public class ListProvider<E> {
private final List<E> theList = new ArrayList<E>();
private final ReentrantLock listLock = new ReentrantLock();
public final List<E> lockList() {
listLock.lock();
return theList;
}
public final void unlockList() {
listLock.unlock();
}
}
/* somewhere else */ {
List<E> theList = listProvider.lockList();
/*
* perform
* multiple
* operations
*
*/
listProvider.unlockList();
}
The main differences between this and synchronized are:
The actual locking mechanism is hidden. This is good for abstraction; however,
Clients must remember to unlock explicitly whereas a synchronized monitor exit is at a block delimiter.
There is a lock called ReentrantReadWriteLock which you might find useful because multiple threads can read simultaneously. ReadWriteLock explains how it works.
Do not iterate over original list, but create a copy of it to find invalid elements. When you are done with filtering you can remove invalid elements from original list safely:
public class Filter {
ListProvider listProvider;
...
public void filter() {
List<String> listProviderCopy = new ArrayList<>(listProvider.getList());
List<String> entriesToRemove = new ArrayList<>();
// collect
for (String entry : listProviderCopy)
if (isInvalid(entry)) {
entriesToRemove.add(entry);
}
}
listProvider.getList().removeAll(entriesToRemove);
}
}
You may want to use SynchronizedList
List<String> list = new ArrayList<>();
List<String> synch = Collections.synchronizedList(list);
more
Related
I have a HashMap of objects with nested ArrayLists that is accessed by multiple threads.
I am wondering if declaring it as a synchronized HashMap is enough to make it thread-safe.
public class ExampleRepository {
private static Map<String, Example> examples = Collections.synchronizedMap(new HashMap<>());
public static void addExample(Example example) {
examples.put(example.getKey(), example);
}
public static Example getExample(String key) {
return examples.get(key);
}
}
public class Example {
private String key;
// More attributes
private List<AnotherObject> anotherObjectList = new ArrayList<>();
// Constructor
public List<AnotherObject> getAnotherObjectList() {
return anotherObjectList;
}
// More getters & Setters
}
public class Doer {
// This function runs in an ExecutorService with many threads
public static one(String key) {
Example example = ExampleRepository.getExample(key);
if (example != null) {
// Do stuff
example = new Example(values);
AnotherObject anotherObject = new AnotherObject(values);
example.getAnotherObjectList().add(anotherObject);
ExampleRepository.addExample(example);
}
two(example);
}
private static two(Example example) {
// Do stuff
AnotherObject anotherObject = new AnotherObject(values);
trim(example.getAnotherObjectList(), time);
example.getAnotherObjectList().add(anotherObject);
}
private static void trim(List<AnotherObject> anotherObjectList, int time) {
short counter = 0;
for (AnotherObject anotherObject : anotherObjectList) {
if (anotherObject.getTime() < time - ONE_HOUR) {
counter++;
} else {
break;
}
}
if (counter > 0) {
anotherObjectList.subList(0, counter).clear();
}
}
}
I guess the question is adding Example objects to the HashMap thread safe? Also, is removing and adding AnotherObject objects to the nested list thread-safe or should I declared it as synchronized ArrayList?
I would greatly appreciate any insights. Thank you very much!
Thank you very much for the answers. I just realized that I actually loop a little over the nested AnotherObject. If i make the ArrayList a synchronized ArrayList, should I still put it in a synchronized block?
Thank you again!
The thing you have to be clear about is what you mean by "thread safe".
I guess the question is adding Example objects to the HashMap thread safe?
Making the map synchronized guarantees that the structural modifications you make to the map are visible to all threads.
Also, is removing and adding AnotherObjet objects to the nested list thread-safe or should I declared it as synchronized ArrayList?
No: you would need to externally synchronize accesses to the lists if you want structural modifications to the lists to be visible in all threads.
That could mean using synchronizedList, but you could "manually" synchronize on the list, or even on the map (or one of a number of other ways that create happens-before guarantees).
I guess the question is adding Example objects to the HashMap thread
safe?
-> Yes putting Example object to map is thread-safe.
Also, is removing and adding AnotherObjet objects to the nested list
thread-safe or should I declared it as synchronized ArrayList?
-> Removing objects from the list does not guarantee that it will be thread-safe.
Any operation on the map will be thread-safe as you have used the Synchronized map. ArrayList in Example object will be still unsafe to thread.
Thread-safe in a collection/map does not mean it will make API of any/every object it contains thread-safe.
Is the compute() function thread safe? Will multiple threads loop correctly over the list?
class Foo {
private List<Integer> list;
public Foo(List<Integer> list) {
this.list = list;
}
public void compute() {
for (Integer i: list) {
// do some thing with it
// NO LIST modifications
}
}
}
Considering that data does not mutate (as you mentioned in the comment) there will not be any dirty / phantom reads.
If the list is created specifically for the purposes of that method, then you're good to go. That is, if the list isn't modified in any other method or class, then that code is thread safe, since you're only reading.
A general recommendation is to make a read-only copy of the collection, if you're not sure the argument comes from a trustworthy origin (and even if you are sure).
this.list = Collections.unmodifiableList(new ArrayList<Integer>(list));
Note, however, that the elements of the list must also be thread-safe. If, in your real scenario, the list contains some mutable structure, instead of Integer (which are immutable), you should make sure that any modifications to the elements are also thread-safe.
If you can guarantee that the list is not modified elsewhere while you're iterating over it that code is thread safe.
I would create a read-only copy of the list though to be absolutely sure that it won't be modified elsewhere:
class Foo {
private List<Integer> list;
public Foo(List<Integer> list) {
this.list = Collections.unmodifiableList(new ArrayList<>(list));
}
public void compute() {
for (Integer i: list) {
// do some thing with it
// NO LIST modifications
}
}
}
If you don't mind adding a dependency to your project I suggest using Guava's ImmutableList:
this.list = ImmutableList.copyOf(list);
It is also a good idea to use Guavas immutable collections wherever you're using collections that aren't changing since they are inherently thread safe due to being immutable.
You can easily inspect the behavior when having for example 2 threads:
public class Test {
public static void main(String[] args) {
Runnable task1 = () -> { new Foo().compute(); };
Runnable task2 = () -> { new Foo().compute(); };
new Thread(task1).start();
new Thread(task2).start();
}
}
If the list is guaranteed not to be changed anywhere else, iterating on it is thread safe, if you implement compute to simply print the list content, debugging your code should help you understanding it is thread safe.
There is thread safe list in cocncurent library. If you want thread-safe collections always use it. Thread-safe list is CopyOnWriteArrayList
This version
class Foo {
private final List<Integer> list;
public Foo(List<Integer> list) {
this.list = new ArrayList<>(list);
}
public void compute() {
for(Integer i: list) {
// ...
}
}
}
is thread-safe, if following holds:
list arg to ctor can't be modified during ctor run time (e.g., it is local variable in caller) or thread-safe itself (e.g., CopyOnWriteArrayList);
compute won't modify list contents (just as OP stated). I guess compute should be not void but return some numeric value, to be of any utility...
I got a multithreaded application in which n threads write to an ConcurrentHashMap. Another n Threads read from that Map and copy its Value to a copy List.
After that the original List is removed from the map.
For some reason I always get a ConcurrentModificationException.
I even tried to create my own lock mechanism with a volatile boolean, but it won't work. When using Google Guava with Lists.newLinkedList() i get a ConcurrentModificationException. When using the StandardWay new LinkedList(list) I get an ArrayOutOfBoundsException.
Here is the compiling code example:
public class VolatileTest {
public static Map<String, List<String>> logMessages = new ConcurrentHashMap<String, List<String>>();
public static AtomicBoolean lock = new AtomicBoolean(false);
public static void main(String[] args) {
new Thread() {
public void run() {
while (true) {
try {
if (!VolatileTest.lock.get()) {
VolatileTest.lock.set(true);
List<String> list = VolatileTest.logMessages.get("test");
if (list != null) {
List<String> copyList = Collections.synchronizedList(list);
for (String string : copyList) {
System.out.println(string);
}
VolatileTest.logMessages.remove("test");
}
VolatileTest.lock.set(false);
}
} catch (ConcurrentModificationException ex) {
ex.printStackTrace();
System.exit(1);
}
}
};
}.start();
new Thread() {
#Override
public void run() {
while (true) {
if (!VolatileTest.lock.get()) {
VolatileTest.lock.set(true);
List<String> list = VolatileTest.logMessages.get("test");
if (list == null) {
list = Collections.synchronizedList(new LinkedList<String>());
}
list.add("TestError");
VolatileTest.logMessages.put("test", list);
VolatileTest.lock.set(false);
}
}
}
}.start();
}
You have ConcurrentModificationException because you have your locking broken and the reader thread reads the same list (by Iterator) the writer writes to at the same time.
Your code looks like a try of lock-free coding. If so, you must use CAS operation like this:
while (!VolatileTest.lock.compareAndSet(false, true) { } // or while (VolatileTest.lock.getAndSet(true)) {} - try to get lock
try {
// code to execute under lock
} finally {
VolatileTest.lock.set(false); // unlock
}
Your
if (!VolatileTest.lock.get()) {
VolatileTest.lock.set(true);
...
}
is not atomic. Or you can use synchronized section or any other standard locking mechanism (ReadWriteLock, for instance)
Also, if you deal with a list for reading and writing using one lock, you don't have to use synchronized list then. And moreover, you don't need even ConcurrentHashMap.
So:
use one global lock and plain HashMap/ArrayList OR
remove your global lock, use ConcurrentHashMap and plain ArrayList with synchronized on each particular instance of the list OR
use a Queue (some BlockingQueue or ConcurrentLinkedQueue) instead of all of your current stuff OR
use something like Disruptor (http://lmax-exchange.github.io/disruptor/) for inter-thread communication with many options. Also, here is a good example of how to build lock-free queues http://psy-lob-saw.blogspot.ru/2013/03/single-producerconsumer-lock-free-queue.html
ConcurrentHashMap is fail safe meaning you will not encounter ConcurrentModificationException. It's your List<String> within the map where one of your thread tries to read the data while other thread is trying to remove the data while iterating.
I would suggest, you don't try locking on whole map operation, but instead look out for making thread safe access to list may be using Vector or SynchronizedList.
Also note, your entry condition if (!VolatileTest.lock) { for both the threads means they can both run at the same time as initially by default boolean would hold false value and both may try to work on same list at the same time.
As already mentioned the locking pattern does not look valid. It is better to use synchronized. The below code works for me
final Object obj = new Object();
and then
synchronized (obj){....} instead of if (!VolatileTest.lock) {.....}
Consider you have a shared memory (List) which will serve as the "critic section".
Now, consider you that you always have items in the list for these scenarios and you want that your system will behave this way:
Thread1 get some item from the list, in the very same time Thread2 wants to add item to the list. Allow this scenario( in assumption I will take first item from begining and insert the new item in the end of the list - in the SAME TIME!).
Thread1 wants to get an item and in the same time Thread2 wants to get an item too.
This should fail.
THANKS
One possibility is to wrap your List in a class that proxies or overrides the get and add methods.
That way, you can use an explicit Lock on the add method, so that only one thread can add at any given time.
See for instance:
http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/locks/ReentrantLock.html
You could do this either by extending a List implementation, and overriding the add and get methods (or all relevant methods), or by using composition instead of inheritance, having a proxy class that forwards the calls to the list, but decorates the add and get with the explicit obtaining of the Lock.
A very simple example would be something like:
public class SharedMemory<K> {
private final List<K> memoryList;
private static final ReentrantLock lock = new ReentrantLock();
public SharedMemory() {
memoryList = new ArrayList<>();
}
public void storeItem(K item) {
memoryList.add(item);
}
public K getItem(int pos){
lock.lock();
try{
return memoryList.get(pos);
} finally {
lock.unlock();
}
}
}
Will the following code snippet of a synchronized ArrayList work in a multi-threaded environment?
class MyList {
private final ArrayList<String> internalList = new ArrayList<String>();
void add(String newValue) {
synchronized (internalList) {
internalList.add(newValue);
}
}
boolean find(String match) {
synchronized (internalList) {
for (String value : internalList) {
if (value.equals(match)) {
return true;
}
}
}
return false;
}
}
I'm concerned that one thread wont be able to see changes by another thread.
Your code will work and is thread-safe but not concurrent. You may want to consider using ConcurrentLinkedQueue or other concurrent thread-safe data structures like ConcurrentHashMap or CopyOnWriteArraySet suggested by notnoop and employ contains method.
class MyList {
private final ConcurrentLinkedQueue<String> internalList =
new ConcurrentLinkedQueue<String>();
void add(String newValue) {
internalList.add(newValue);
}
boolean find(String match) {
return internalList.contains(match);
}
}
This should work, because synchronizing on the same object establishes a happens-before relationship, and writes that happen-before reads are guaranteed to be visible.
See the Java Language Specification, section 17.4.5 for details on happens-before.
It will work fine, because all access to the list is synchronized. Hovewer, you can use CopyOnWriteArrayList to improve concurrency by avoiding locks (especially if you have many threads executing find).
It will work, but better solution is to create a List by calling Collections.synchronizedList().
You may want to consider using a Set(Tree or Hash) for your data as you are doing lookups by a key. They have methods that will be much faster than your current find method.
HashSet<String> set = new HashSet<String>();
Boolean result = set.contains(match); // O(1) time