I want to reproduce one scenario in which there are two threads accessing a shared HashMap. While one thread is copying the contents of the shared map into localMap using putAll() operation, second thread changes the shared map and CocurrentModificationException should be thrown.
I have tried but not able to reproduce the exception at the time when putAll operation is running. Each time either putAll gets complete before other thread does modification or putAll is called after other thread modification.
Can anyone please suggest how can I generate the scenario in java?
Thanks.
Spin up both threads, have them running continuously.
Have one thread constantly doing putAll, the other constantly doing the modification.
import java.util.HashMap;
import java.util.Map;
public class Example {
private final HashMap<String, String> map = new HashMap<String, String>();
public static void main(String[] args) {
new Example();
}
public Example() {
Thread thread1 = new Thread(new Runnable() {
#Override
public void run() {
int counter = 0;
while (true) {
Map<String, String> tmp = new HashMap<String, String>();
tmp.put("example" + counter, "example");
counter++;
map.putAll(tmp);
}
}
});
Thread thread2 = new Thread(new Runnable() {
#Override
public void run() {
while (true) {
map.values().remove("example");
}
}
});
thread1.start();
thread2.start();
}
}
Unfortunately I cannot copy/paste the running code directly from my current workstation so I retyped it here so there might be a typing error.
As you can see the first thread is continuously adding values while the second thread iterates over the values in the Map. When it starts iterating over the values it expects a number of values (this value is initialized at the construction of the iterator). However because thread1 is continuously adding items this value is not as expected when the Iterator checks the actual amount of values that are in the map when it actual executes the remove code. This causes the ConcurrentModificationException.
If you just need a CocurrentModificationException to be thrown, you could implement your own Map implementation (HackedMap) to remove items from the HashMap when the HashMap tries to copy values from your HackedMap
Related
public class MapDem {
final HashMap<Integer,Integer> map = new HashMap<Integer,Integer>();
public HashMap<Integer,Integer> getMap(){
return map;
}
public void putValue(int key,int value){
map.put(key,value);
}
public static void main(String args[]){
MapDem demo = new MapDem();
new Thread(new Runnable(){
#Override
public void run() {
demo.putValue(1, 10);
}
}).start();
new Thread(new Runnable(){
#Override
public void run() {
demo.putValue(1, 10);
}
}).start();
System.out.println(demo.getMap().size());
}
}
Are final fields inherently thread-safe? In the above code the map variable is marked as final, does that mean that it is thread-safe?
If the variable is not thread-safe I expect that the output from the main-method should be a "2" but I am getting either "1" or "0"
EDIT
If I declare the variable using the volatile keyword, i.e.
volatile HashMap<Integer,Integer> map = new HashMap<Integer,Integer>();
the map variable is seemingly not thread-safe, why is this? However the below method seems to work, why is this?
public synchronized void putValue(int key,int value){
if(map.isEmpty()){
System.out.println("hello");
map.put(key,value);
}
Will using Collections.unmodifiableMap(map) work?
Your test ist faulty. If two values are stored with the same key, HashMap.put(K key, V value) will overwrite the former value with the later. Thus, even without concurrency, your "test" will return a size of 1.
Code:
import java.util.HashMap;
public class MapDem {
final HashMap<Integer, Integer> map = new HashMap<Integer, Integer>();
public HashMap<Integer, Integer> getMap() {
return map;
}
public void putValue(int key, int value) {
map.put(key, value);
}
public static void main(String args[]) {
MapDem demo = new MapDem();
demo.putValue(1, 10);
demo.putValue(1, 10);
System.out.println(demo.getMap().size());
}
}
Output (Ideone demo):
1
The fact that sometimes one can see a size of 0 is due to the lack of blocking constructs. You should wait for completion of both Threads before querying the size of yur map by calling join() on your Thread-objects.
Thread t1 = new Thread(new Runnable() {
#Override
public void run() {
demo.putValue(1, 10);
}
});
t1.start();
Thread t2 = new Thread(new Runnable() {
#Override
public void run() {
demo.putValue(1, 10);
}
});
t2.start();
try {
t1.join();
t2.join();
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println(demo.getMap().size());
As mentioned by #SachinSarawgi, final does not make your code thread-safe and as further explained by #assylias, volatile does not cut it in this case.
If you need a thread-safe Hashmap, use ConcurrentHashMap.
If you are determined to write your own thread-safe implementation of the Map interface, I recommend Oracle's Lesson on Concurrency to start with, followed by Brian Goetz's "Java Concurrency in Practice" and maybe a little bit of Javier Fernández González' "Mastering Concurrency Programming with Java 8".
The direct immediate answer to your question is: no, the final keyword does not make fields thread safe.
That keyword only tells the compiler that it has to ensure that there is exactly one value assigned to that field (not zero or multiple assignments).
You know, there are reasons why getting multi-threaded code correct is considered hard.
The essence of correct multi-threading is: when some state can be updated by one thread, but is used (or updated) by other threads .. to make sure that you only get those state changes that you want to see.
Long story short: you have a lot of learning to do; a good starting point would be here.
What is thread safe is the access to the map variable: all threads reading that variable will see the same object reference.
However the operations on the HashMap (get/put) are not thread safe and this has nothing to do with the fact that map is final or not.
So your code is not thread safe unless you add some concurrency control around the putValue method - for example by making it synchronized.
Making reference variable final make sure that the reference variable cant change the object reference it has been assigned to.
But the value of the object could change. Same thing is happening here your object value could change. Now the code which change the value need to be synchronized.
You can get rid of all the synchronization problem by using ConcurrentHashMap. You can read about it here.
Using ConcurrentHashMap make sure that every write operation on the object should be handled by one thread at a time. Also it optimize the reading of HashMap too. It divide the HashMap into block and different threads may read from different blocks.
I'm trying to figure out the best way to have multiple threads working from the same list of strings. For example, say I have a list of words, and I want multiple threads to work on printing out each word on this list.
Here is what I came up with. The thread uses a while loop, and while the iterator has next, it prints out and removes it from the list.
import java.util.*;
public class ThreadsExample {
static Iterator it;
public static void main(String[] args) throws Exception {
ArrayList<String> list = new ArrayList<>();
list.add("comet");
list.add("planet");
list.add("moon");
list.add("star");
list.add("asteroid");
list.add("rocket");
list.add("spaceship");
list.add("solar");
list.add("quasar");
list.add("blackhole");
it = list.iterator();
//launch three threads
RunIt rit = new RunIt();
rit.runit();
rit.runit();
rit.runit();
}
}
class RunIt implements Runnable {
public void run()
{
while (ThreadsExample.it.hasNext()) {
//Print out and remove string from the list
System.out.println(ThreadsExample.it.next());
ThreadsExample.it.remove();
}
}
public void runit() {
Thread thread = new Thread(new RunIt());
thread.start();
}
}
This seems to work, although I get some Exception in thread "Thread-2" Exception in thread "Thread-0" java.lang.IllegalStateException errors during the run:
Exception in thread "Thread-1" Exception in thread "Thread-0"
java.lang.IllegalStateException at
java.util.ArrayList$Itr.remove(ArrayList.java:864) at
RunIt.run(ThreadsExample.java:44) at
java.lang.Thread.run(Thread.java:745) java.lang.IllegalStateException
at java.util.ArrayList$Itr.remove(ArrayList.java:864) at
RunIt.run(ThreadsExample.java:44) at
java.lang.Thread.run(Thread.java:745)
Am I doing this correctly or is there a better way to have multiple threads working on the same pool of strings?
A better way to do this is to use a concurrent queue. The Queue interface is designed to hold elements in a structure prior to processing them.
final Queue<String> queue = new ConcurrentLinkedQueue<String>();
queue.offer("asteroid");
ExecutorService executorService = Executors.newFixedThreadPool(4);
executorService.execute(new Runnable() {
public void run() {
System.out.println(queue.poll());
}
});
executorService.shutdown();
Try creating the list as a synchronized list using List.synchronizedList
Update your code like this:
ArrayList<String> list = Collections.synchronizedList(new ArrayList<>());
Am I doing this correctly or is there a better way to have multiple threads working on the same pool of strings?
You are not doing it correctly. Your code is not properly synchronized, and therefore its behavior is not well defined. There are a great number of ways you could approach the general problem you present, but one way the issues in your particular code could be fixed would be to change RunIt.run() to properly synchronize:
public void run()
{
while (true) {
synchronized(ThreadsExample.it) {
if (ThreadsExample.it.hasNext()) {
//Print out and remove string from the list
System.out.println(ThreadsExample.it.next());
ThreadsExample.it.remove();
} else {
break;
}
}
}
}
Note here that the hasNext() check, retrieval of the next element, and removal of that element are all handled within the same synchronized block to ensure mutual consistency of these operations. On the other hand, the scope of that block is contained within the loop, so that different threads executing the loop concurrently each get a chance to execute.
Note, too, that although in this case all the threads synchronize on the Iterator object, that's basically just a convenience (for me). As long as they all synchronize on the same object, it doesn't matter so much which object that is.
I got a multithreaded application in which n threads write to an ConcurrentHashMap. Another n Threads read from that Map and copy its Value to a copy List.
After that the original List is removed from the map.
For some reason I always get a ConcurrentModificationException.
I even tried to create my own lock mechanism with a volatile boolean, but it won't work. When using Google Guava with Lists.newLinkedList() i get a ConcurrentModificationException. When using the StandardWay new LinkedList(list) I get an ArrayOutOfBoundsException.
Here is the compiling code example:
public class VolatileTest {
public static Map<String, List<String>> logMessages = new ConcurrentHashMap<String, List<String>>();
public static AtomicBoolean lock = new AtomicBoolean(false);
public static void main(String[] args) {
new Thread() {
public void run() {
while (true) {
try {
if (!VolatileTest.lock.get()) {
VolatileTest.lock.set(true);
List<String> list = VolatileTest.logMessages.get("test");
if (list != null) {
List<String> copyList = Collections.synchronizedList(list);
for (String string : copyList) {
System.out.println(string);
}
VolatileTest.logMessages.remove("test");
}
VolatileTest.lock.set(false);
}
} catch (ConcurrentModificationException ex) {
ex.printStackTrace();
System.exit(1);
}
}
};
}.start();
new Thread() {
#Override
public void run() {
while (true) {
if (!VolatileTest.lock.get()) {
VolatileTest.lock.set(true);
List<String> list = VolatileTest.logMessages.get("test");
if (list == null) {
list = Collections.synchronizedList(new LinkedList<String>());
}
list.add("TestError");
VolatileTest.logMessages.put("test", list);
VolatileTest.lock.set(false);
}
}
}
}.start();
}
You have ConcurrentModificationException because you have your locking broken and the reader thread reads the same list (by Iterator) the writer writes to at the same time.
Your code looks like a try of lock-free coding. If so, you must use CAS operation like this:
while (!VolatileTest.lock.compareAndSet(false, true) { } // or while (VolatileTest.lock.getAndSet(true)) {} - try to get lock
try {
// code to execute under lock
} finally {
VolatileTest.lock.set(false); // unlock
}
Your
if (!VolatileTest.lock.get()) {
VolatileTest.lock.set(true);
...
}
is not atomic. Or you can use synchronized section or any other standard locking mechanism (ReadWriteLock, for instance)
Also, if you deal with a list for reading and writing using one lock, you don't have to use synchronized list then. And moreover, you don't need even ConcurrentHashMap.
So:
use one global lock and plain HashMap/ArrayList OR
remove your global lock, use ConcurrentHashMap and plain ArrayList with synchronized on each particular instance of the list OR
use a Queue (some BlockingQueue or ConcurrentLinkedQueue) instead of all of your current stuff OR
use something like Disruptor (http://lmax-exchange.github.io/disruptor/) for inter-thread communication with many options. Also, here is a good example of how to build lock-free queues http://psy-lob-saw.blogspot.ru/2013/03/single-producerconsumer-lock-free-queue.html
ConcurrentHashMap is fail safe meaning you will not encounter ConcurrentModificationException. It's your List<String> within the map where one of your thread tries to read the data while other thread is trying to remove the data while iterating.
I would suggest, you don't try locking on whole map operation, but instead look out for making thread safe access to list may be using Vector or SynchronizedList.
Also note, your entry condition if (!VolatileTest.lock) { for both the threads means they can both run at the same time as initially by default boolean would hold false value and both may try to work on same list at the same time.
As already mentioned the locking pattern does not look valid. It is better to use synchronized. The below code works for me
final Object obj = new Object();
and then
synchronized (obj){....} instead of if (!VolatileTest.lock) {.....}
This is a little piece of code
public Map<String,Object> findTruckParts(Map<String,Object> output){
Map<String,Object>findPartsMap = null;
NewFooInstance newFooInstance = new NewFooInstance();
findPartsMap = PartBuilder.buildPartsOutputMap(output, outputMap);
newFooInstance.buildItem(findPartsMap);
return findPartsMap;
}
outputMap is a new hashMap and output is a hashmap with some spare parts info.
buildItem calls a few other private methods passing around the findPartsMap.
public class NewFooInstance{
buildItem(Map<String,Object> partsMap){
checkPartsValidity(partsMap, fetchMapOfValidParts());
}
checkPartsValidity(Map<String,Object> partsMap,Map<String,Object> partsMap){
//partsMap = update partsMap with missing items fetched from list of valid parts
}
}
Is the above thread safe ? Since all the maps are local to the respective methods, My assumption is that this is thread safe.
Edit: I have modified the method a little bit. Takes in a map and returns another one. So, my question is, is this map that is being returned be thread safe? It is local to the method so I think this is going to be threadsafe (no other thread entering will be able to change its value in case this map loses its monitor), however, since this map is being modified in other classes and other methods, does this method-localness of this map carry over across different classes/methods and assure thread safety ?
The answer is "no", because HashMap itself is not threadsafe.
Consider using a threadsafe Map implementation such as ConcurrentHashMap.
The problem is here:
public Map<String,Object> findTruckParts(Map<String,Object> output)
Even though things look thread-safe in the methods and sub-methods with respect to resulting map, you still have a thread safety issue with the source map (i.e. 'output'). As you extract data from it to put into the new resulting map, if it is altered by another thread at the same time, you will get a ConcurrentModificationException.
Below is some code to illustrate the issue:
import java.util.HashMap;
import java.util.Map;
public class Test {
public static void main(String[] args) throws Exception {
final Map<String, Object> test = new HashMap<String, Object>();
new Thread(new Runnable() {
public void run() {
System.out.println("Thread 1: started");
findTruckParts(test);
System.out.println("Thread 1: done");
}
public Map<String,Object> findTruckParts(Map<String,Object> output) {
Map<String, Object> result = new HashMap<String, Object>();
for(int i=0; i<100000000; i++) {
for(String key : output.keySet()) {
result.put("x", output.get(key));
}
}
return result;
}
}).start();
new Thread(new Runnable() {
public void run() {
System.out.println("Thread 2: started");
for(int i=0; i<100000; i++) {
test.put("y", "y"+i);
test.remove("y");
}
System.out.println("Thread 2: done");
}
}).start();
}
}
And the output is invariably:
Thread 1: started Thread 2: started Exception in thread "Thread-1"
java.util.ConcurrentModificationException at
java.util.HashMap$HashIterator.nextEntry(HashMap.java:793) at
java.util.HashMap$KeyIterator.next(HashMap.java:828) at
Test$1.findTruckParts(Test.java:19) at Test$1.run(Test.java:12) at
java.lang.Thread.run(Thread.java:680) Thread 2: done
So even though the findTruckParts() method creates its own map to return, if it has to look into the source map and some other thread is modifying its keys/values, there will be a problem. The the other threads are only reading, it shouldn't blow up. But I'm not sure you want to talk about thread safety in this case because it's still precarious.
One way to help with thread safety is to change the first line of the main method with:
final ConcurrentHashMap<String, Object> test = new ConcurrentHashMap<String, Object>(new HashMap<String, Object>());
But you can see how safety requirement is pushed on to the caller, which isn't great. So to help that, you could also alter the signature of the method:
public Map<String,Object> findTruckParts(ConcurrentHashMap<String,Object> output);
And now there is thread safety.
Therefore, as I stated on the first line, the problem is here:
The problem is here:
public Map<String,Object> findTruckParts(Map<String,Object> output)
I hope this helps.
I'm having a bit of trouble concerning concurrency and maps in Java.
Basically I have multiple threads using (reading and modifying) their own maps, however each of these maps is a part of a larger map which is being read and modified by a further thread:
My main method creates all threads, the threads create their respective maps which are then put into the "main" map:
Map<String, MyObject> mainMap = new HashMap<String, Integer>();
FirstThread t1 = new FirstThread();
mainMap.putAll(t1.getMap());
t1.start();
SecondThread t2 = new SecondThread();
mainMap.putAll(t2.getMap());
t2.start();
ThirdThread t3 = new ThirdThread(mainMap);
t3.start();
The problem I'm facing now is that the third (main) thread sees arbitrary values in the map, depending on when one or both of the other threads update "their" items.
I must however guarantee that the third thread can iterate over - and use the values of - the map without having to fear that a part of what is being read is "old":
FirstThread (analogue to SecondThread):
for (MyObject o : map.values()) {
o.setNewValue(getNewValue());
}
ThirdThread:
for (MyObject o : map.values()) {
doSomethingWith(o.getNewValue());
}
Any ideas? I've considered using a globally accessible (static final Object through a static class) lock which will be synchronized in each thread when the map must be modified.
Or are there specific Map implementations that assess this particular problem which I could use?
Thanks in advance!
Edit:
As suggested by #Pyranja, it would be possible to synchronize the getNewValue() method. However I forgot to mention that I am in fact trying to do something along the lines of transactions, where t1 and t2 modify multiple values before/after t3 works with said values. t3 is implemented in such a way that doSomethingWith() will not actually do anything with the value if it hasn't changed.
To synchronize at a higher level than the individual value objects, you need locks to handle the synchronization between the various threads. One way to do this, without changing your code too much, is a ReadWriteLock. Thread 1 and Thread 2 are writers, Thread 3 is a reader.
You can either do this with two locks, or one. I've sketched out below doing it with one lock, two writer threads, and one reader thread, without worrying about what happens with an exception during data update (ie, transaction rollback...).
All that said, this sounds like a classic producer-consumer scenario. You should consider using something like a BlockingQueue for communication between threads, as is outlined in this question.
There's other things you may want to consider changing as well, like using Runnable instead of extending Thread.
private static final class Value {
public void update() {
}
}
private static final class Key {
}
private final class MyReaderThread extends Thread {
private final Map<Key, Value> allValues;
public MyReaderThread(Map<Key, Value> allValues) {
this.allValues = allValues;
}
#Override
public void run() {
while (!isInterrupted()) {
readData();
}
}
private void readData() {
readLock.lock();
try {
for (Value value : allValues.values()) {
// Do something
}
}
finally {
readLock.unlock();
}
}
}
private final class WriterThread extends Thread {
private final Map<Key, Value> data = new HashMap<Key, Value>();
#Override
public void run() {
while (!isInterrupted()) {
writeData();
}
}
private void writeData() {
writeLock.lock();
try {
for (Value value : data.values()) {
value.update();
}
}
finally {
writeLock.unlock();
}
}
}
private final ReentrantReadWriteLock lock = new ReentrantReadWriteLock();
private final ReadLock readLock;
private final WriteLock writeLock;
public Thing() {
readLock = lock.readLock();
writeLock = lock.writeLock();
}
public void doStuff() {
WriterThread thread1 = new WriterThread();
WriterThread thread2 = new WriterThread();
Map<Key, Value> allValues = new HashMap<Key, Value>();
allValues.putAll(thread1.data);
allValues.putAll(thread2.data);
MyReaderThread thread3 = new MyReaderThread(allValues);
thread1.start();
thread2.start();
thread3.start();
}
ConcurrentHashMap from java.util.concurrent - a thread-safe implementation of Map, which provides a much higher degree of concurrency than synchronizedMap. Just a lot of reads can almost always be performed in parallel, simultaneous reads and writes can usually be done in parallel, and multiple simultaneous recordings can often be done in parallel. (The class ConcurrentReaderHashMap offers a similar parallelism for multiple read operations, but allows only one active write operation.) ConcurrentHashMapis designed to optimize the retrieval operations.
Your example code may be misleading. In your first example you create a HashMap<String,Integer> but the second part iterates the map values which in this case are MyObject. The key to synchronization is to understand where and which mutable state is shared.
An Integer is immutable. It can be shared freely (but the reference to an Integer is mutable - it must be safely publicated and/or synchronized). But your code example suggests that the maps are populated with mutable MyObject instances.
Given that the map entries (key -> MyObject references) are not changed by any thread and all maps are created and safely publicated before any thread starts it would be in my opinion sufficient to synchronize the modification of MyObject. E.g.:
public class MyObject {
private Object value;
synchronized Object getNewValue() {
return value;
}
synchronized void setNewValue(final Object newValue) {
this.value = newValue;
}
}
If my assumptions are not correct, clarify your question / code example and also consider #jacobm's comment and #Alex answer.