I have written this piece of code
public class Test{
public static void main(String[] args) {
List<Integer> list = new ArrayList<Integer>();
for(int i = 1;i<= 4;i++){
new Thread(new TestTask(i, list)).start();
}
while(list.size() != 4){
// this while loop required so that all threads complete their work
}
System.out.println("List "+list);
}
}
class TestTask implements Runnable{
private int sequence;
private List<Integer> list;
public TestTask(int sequence, List<Integer> list) {
this.sequence = sequence;
this.list = list;
}
#Override
public void run() {
list.add(sequence);
}
}
This code works and prints all the four elements of list on my machine.
My question is that will this code always work. I think there might be a issue in this code when two/or more threads add element to this list at the same point. In that case it while loop will never end and code will fail.
Can anybody suggest a better way to do this? I am not very good at multithreading and don't know which concurrent collection i can use?
Thanks, Shekhar
Use this in order to get a real thread-safe list:
List<Integer> list = Collections.synchronizedList(new ArrayList<Integer>());
Depending on your usage, also a CopyOnWriteArrayList could be interesting for you. Precisly, when traversal operations vastly outnumber mutations on that list.
Afair, Lists are not thread-safe in Java, so you might get anything from working to crashing. Use synchronized access to the list in order to get a well-defined behaviour:
#Override
public void run() {
synchronized(list) {
list.add(sequence);
}
}
This way, access to the list is only possible for a single thread at a time.
Also you I'd use Thread.join() to wait for the threads to finish (you have to keep them in a separate list for doing that ...)
I think you need to do two things. First of all you need to join the threads. Because atm the other loop will sometimes run even if the threads are not completed.
You have to do it like this:
Threads threads[4] = new Thread[4];
for(int i = 1;i<= 4;i++){
threads[i] = new Thread(new TestTask(i, list));
threads[i].start();
}
// to wait that all threads finish..
for(int i = 1;i<= 4;i++){
threads[i].join();
}
while(list.size() != 4){
// this while loop required so that all threads complete their work
}
and you can make your ArrayList thread safe with packing it into:
List<Integer> list = Collections.synchronizedList(new ArrayList<Integer>());
No, there are no guarantees. The simplest solution would be to join with each thread.
Read up on wait() and notify() instead of having a busy-waiting while-loop.
Related
I have been trying to reproduce (and solve) a ConcurrentModificationException when an instance of HashMap is being read and written by multiple Threads.
Disclaimer: I know that HashMap is not thread-safe.
In the following code:
import java.util.*;
public class MyClass {
public static void main(String args[]) throws Exception {
java.util.Map<String, Integer> oops = new java.util.HashMap<>();
oops.put("1", 1);
oops.put("2", 2);
oops.put("3", 3);
Runnable read = () -> {
System.out.println("Entered read thread");
/*
* ConcurrentModificationException possibly occurs
*
for (int i = 0; i < 100; i++) {
List<Integer> numbers = new ArrayList<>();
numbers.addAll(oops.values());
System.out.println("Size " + numbers.size());
}
*/
for (int i = 0; i < 100; i++) {
List<Integer> numbers = new ArrayList<>();
numbers.addAll(oops.values()
.stream()
.collect(java.util.stream.Collectors.toList()));
System.out.println("Size " + numbers.size());
}
};
Runnable write = () -> {
System.out.println("Entered write thread");
for (int i = 0; i < 100; i++) {
System.out.println("Put " + i);
oops.put(Integer.toString(i), i);
}
};
Thread writeThread = new Thread(write, "write-thread");
Thread readThread = new Thread(read, "read-thread");
readThread.start();
writeThread.start();
readThread.join();
writeThread.join();
}
}
Basically, I make two threads: one keeps putting elements into a HashMap, the other is iterating on HashMap.values().
In the read thread, if I'm using numbers.addAll(oops.values()), the ConcurrentModificationException randomly occurs. Though the lines are printed randomly as expected.
But if I switch to numbers.addAll(oops.values().stream().., I don't get any error. However, I have observed a strange phenomenon. All the lines by the read thread are printed after the lines printed by the write thread.
My question is, does Collection.stream() have somehow internal synchronization?
UPDATE:
Using JDoodle https://www.jdoodle.com/a/IYy, it seems on JDK9 and JDK10, I will get ConcurrentModificationException as expected.
Thanks!
What you are seeing is absolutely by chance; bear in mind that internally System.out.println does a synchronzied; thus may be that somehow makes it look like the results appear in order.
I have not looked too deep into your code - because analyzing why HashMap, which is not thread safe, is miss behaving is most probably futile; as you know, it is documented to be non-thread safe.
About that ConcurrentModificationException, the documentation is specific that it will try at best odds to throw that; so it's either java-8 was weaker in this point, or this was again by accident.
I was able to get ConcurrentModificationException with streams on Java 8 but with some changes in code: increased number of iterations and number of added elements to map in a separate thread from 100 to 10000. And also added CyclicBarrier so that loops in reader and writer threads are started more or less at the same time. I've also checked source code of spliterator for Hashmap.values() and it throws ConcurrentModificationException if some modifications to map were made.
if (m.modCount != mc) //modCount is number of modifications mc is expected modifications count which is stored before trying to fetch next element
throw new ConcurrentModificationException();
I've looked at the source code of Java 8 quickly, it does throw ConcurrentModificationException.
HashMap's values()method returns a subclass of AbstractCollection, whose spliterator() method returns a ValueSpliterator, which throws ConcurrentModificationException.
For information Collection.stream() uses a spliterator to traverse or partition elements of a source.
There is one ArrayList with 1 million element and we are using two threads to read from this ArrayList. The first thread will read first half of the list and second thread will read the second half of list and I am using two threads to achieve this, but I don't see any difference in performance between using one thread and two threads.
I have written below program to achieve this, but I am not sure If this is the right way to implement and achieve this.
Can someone check if my code is correct or how I can fix the multithreading?
import java.util.ArrayList;
import java.util.List;
public class ThreadTask {
public static void main(String[] args) throws InterruptedException {
// TODO Auto-generated method stub
List<Integer> list = new ArrayList<>();
for(int i = 0; i <=1000000; i++){
list.add(i);
}
Thread t1 = new Thread(new PrintList(list));
Thread t2 = new Thread(new PrintList(list));
t1.setName("thread1");
t2.setName("thread2");
long starttime = System.currentTimeMillis();
System.out.println(starttime);
t1.start();
t2.start();
t1.join();
t2.join();
long endtime = System.currentTimeMillis();
System.out.println(endtime);
System.out.println("Total time "+(endtime - starttime));
}
}
class PrintList implements Runnable{
private List list = new ArrayList();
public PrintList(List list){
this.list = list;
}
#Override
public void run() {
if(Thread.currentThread().getName() != null && Thread.currentThread().getName().equalsIgnoreCase("thread1")){
for(int i = 0; i< list.size()/2;i++){
// System.out.println("Thread 1 "+list.get(i));
}
}else if(Thread.currentThread().getName() != null && Thread.currentThread().getName().equalsIgnoreCase("thread2")){
for(int i = list.size()/2; i<list.size(); i++){
//System.out.println("Thread 2 "+list.get(i));
}
}
}
}
Also, If someone can help me on how can we implement it to make it generic to use more then to thread.
System.out.println is synchronized internally (in order that you don't get mixing between the messages printed by multiple threads), so only one thread is actually printing at once.
Basically, it behaves like a single thread.
Even if in reality System.out is synchronized, still you dont want to have manually initialized threads reading from your ArrayList. Plus I doubt that your end goal is the System.out. You should use a higher abstraction. Such abstraction can easily be present either through Java8 Stream API either by ExecutorServices.
Here is one example of paralelism with Java 8 api.
Arraylist toprint;
toPrint.parallelstream().forEach(DoSometing);
This will work in parallel threads.
If you use ExecutorService You can slice your Arraylist and pass each slice to a Callable to perform the work for you in a separate thread.
class Task implements Callable {
List sublist;
public Task(List sublist) {
this.sublist = sublist;
}
public void call() {
// do something
}
}
ArrayList listToSlice;
List<List> slicedList;
ExecutorService executor = Executors.newFixedThreadPool(2);
for (List sublist:slicedList) {
Future<Integer> future = executor.submit(new Task(sublist));
......
.......s on
}
I'm trying to figure out the best way to have multiple threads working from the same list of strings. For example, say I have a list of words, and I want multiple threads to work on printing out each word on this list.
Here is what I came up with. The thread uses a while loop, and while the iterator has next, it prints out and removes it from the list.
import java.util.*;
public class ThreadsExample {
static Iterator it;
public static void main(String[] args) throws Exception {
ArrayList<String> list = new ArrayList<>();
list.add("comet");
list.add("planet");
list.add("moon");
list.add("star");
list.add("asteroid");
list.add("rocket");
list.add("spaceship");
list.add("solar");
list.add("quasar");
list.add("blackhole");
it = list.iterator();
//launch three threads
RunIt rit = new RunIt();
rit.runit();
rit.runit();
rit.runit();
}
}
class RunIt implements Runnable {
public void run()
{
while (ThreadsExample.it.hasNext()) {
//Print out and remove string from the list
System.out.println(ThreadsExample.it.next());
ThreadsExample.it.remove();
}
}
public void runit() {
Thread thread = new Thread(new RunIt());
thread.start();
}
}
This seems to work, although I get some Exception in thread "Thread-2" Exception in thread "Thread-0" java.lang.IllegalStateException errors during the run:
Exception in thread "Thread-1" Exception in thread "Thread-0"
java.lang.IllegalStateException at
java.util.ArrayList$Itr.remove(ArrayList.java:864) at
RunIt.run(ThreadsExample.java:44) at
java.lang.Thread.run(Thread.java:745) java.lang.IllegalStateException
at java.util.ArrayList$Itr.remove(ArrayList.java:864) at
RunIt.run(ThreadsExample.java:44) at
java.lang.Thread.run(Thread.java:745)
Am I doing this correctly or is there a better way to have multiple threads working on the same pool of strings?
A better way to do this is to use a concurrent queue. The Queue interface is designed to hold elements in a structure prior to processing them.
final Queue<String> queue = new ConcurrentLinkedQueue<String>();
queue.offer("asteroid");
ExecutorService executorService = Executors.newFixedThreadPool(4);
executorService.execute(new Runnable() {
public void run() {
System.out.println(queue.poll());
}
});
executorService.shutdown();
Try creating the list as a synchronized list using List.synchronizedList
Update your code like this:
ArrayList<String> list = Collections.synchronizedList(new ArrayList<>());
Am I doing this correctly or is there a better way to have multiple threads working on the same pool of strings?
You are not doing it correctly. Your code is not properly synchronized, and therefore its behavior is not well defined. There are a great number of ways you could approach the general problem you present, but one way the issues in your particular code could be fixed would be to change RunIt.run() to properly synchronize:
public void run()
{
while (true) {
synchronized(ThreadsExample.it) {
if (ThreadsExample.it.hasNext()) {
//Print out and remove string from the list
System.out.println(ThreadsExample.it.next());
ThreadsExample.it.remove();
} else {
break;
}
}
}
}
Note here that the hasNext() check, retrieval of the next element, and removal of that element are all handled within the same synchronized block to ensure mutual consistency of these operations. On the other hand, the scope of that block is contained within the loop, so that different threads executing the loop concurrently each get a chance to execute.
Note, too, that although in this case all the threads synchronize on the Iterator object, that's basically just a convenience (for me). As long as they all synchronize on the same object, it doesn't matter so much which object that is.
I'm using multithreading to process a List of Strings in batches, however I'm getting this error when the Runnable task is iterating over the List to process each String.
For example the code roughly follows this structure:
public class RunnableTask implements Runnable {
private List<String> batch;
RunnableTask(List<String> batch){
this.batch = batch;
}
#Override
public void run() {
for(String record : batch){
entry = record.split(",");
m = regex.matcher(entry[7]);
if (m.find() && m.group(3) != null){
currentKey = m.group(3).trim();
currentValue = Integer.parseInt(entry[4]);
if ( resultMap.get(currentKey) == null ){
resultMap.put(currentKey, currentValue);
} else {
resultMap.put(currentKey, resultMap.get(currentKey) + currentValue);
}
}
}
}
}
Where the thread that is passing these batches for processing never modifies "batch" and NO CHANGES to batch are made inside the for loop. I understand that this exception ConcurrentModificationException is due to modifying the List during iteration but as far as I can tell that isn't happening. Is there something I'm missing?
Any help is appreciated,
Thankyou!
UPDATE1: It seems instance-variables aren't thread safe. I attempted to use CopyOnWriteArrayList in place of the ArrayList but I received inconsistent results - suggesting that the full iteration doesn't complete before the list is modified in some way and not every element is being processed.
UPDATE2: Locking on the loop with sychronized and/or a reentrantlock both still give the same exception.
I need a way to pass Lists to Runnable tasks and iterate over those lists without new threads causing concurrency issues with that list.
I understand that this exception ConcurrentModificationException is due to modifying the List during iteration but as far as I can tell that isn't happening
Ok, consider what happens when you create a new thread, passing a reference to RunnableTask instance, initialized with a different list as constructor parameter? You just changed the list reference to point to different list. And consider what happens when at the same time, a different thread inside the run() method, is changing the list, at any point. This will at some point of time, throw ConcurrentModificationException.
Instance Variables are not Thread-Safe.
Try this in your code:
public void run() {
for(String record : new ArrayList(batch)){
//do processing with record
}
}
There is a sort of problem with all your threads processing the list (is the list modified during the process?) but is difficult to tell with the code you're providing
Problem is due to multiple thread concurrently modifying the the source List structure. What I would suggest you should devide the source list to new sublist(according to size) and pass that list to threads.
Say your source List have 100 elements. and you are running 5 concurrent thread.
int index = 0;
List<TObject> tempList = new ArrayList<>();
for(TObject obj:srcList){
if(i==(srcList.size()/numberOfthread)){
RunnableTask task = new RunnableTask(tempList);
tempList = new ArrayList<>();
}else
tempList.add(obj);
}
In this case your original list would not be modified.
you need to lock the list before accessing its elements. because List is not thread safe. Try this
public void run() {
synchronizd(batch){
for(String record : batch){//do processing with record}
}
}
yes you are getting ConcurrentModificationException because your List is getting modified during iteration. If performance is not a critical issue I suggest use synchronization.
public class RunnableTask implements Runnable {
private List<String> batch = new ArrayList<String>();
RunnableTask(List<String> batch){
this.batch = batch;
}
public void run() {
synchronized (batch) {
for(String record : batch){//do processing with record}
}
}
}
}
or even better use ReentrantLock.
Your followups indicate that you are trying to reuse the same List multiple times. Your caller must create a new List for each Runnable.
Obviously someone else is changing the content of the list, which is out of picture of the code you mentioned. (If you are sure that the ConcurrentModificationException is complaining for the batch list, but not resultMap, and you are actually showing all code in RunnableTask)
Try to search in your code, for places that is updating the content of the list, check if it is possible concurrently with your RunnableTask.
Simply synchronizing in the RunnableTask is not going to help, you need to synchronize all access to the list, which is obviously happening somewhere else.
If performance is an issue to you so that you cannot synchronize on the batch list (which prohibit multiple RunnableTask to execute concurrently), consider making use of ReaderWriterLock: RunnableTask acquires read lock, while the list update logic acquire the write lock.
I asked here a question about iterating over a Vector, and I have been answered with some good solutions. But I read about another simpler way to do it. I would like to know if it is good solution.
synchronized(mapItems) {
Iterator<MapItem> iterator = mapItems.iterator();
while(iterator.hasNext())
iterator.next().draw(g);
}
mapItems is a synchronized collection: Vector. Is that make the iterating over the Vector safe from ConcurrentModificationException?
Yes, it will make it safe from ConcurrentModificationException at the expense of everything essentially being single-threaded.
Yes, I believe that this will prevent a ConcurrentModificationException. You are synchronizing on the Vector. All methods on Vector that modify it are also synchronized, which means that they would also lock on that same object. So no other thread could change the Vector while you're iterating over it.
Also, you are not modifying the Vector yourself while you're iterating over it.
Simply synchronizing the entire collection would not prevent a ConcurrentModificationException. This will still throw a CME
synchronized(mapItems) {
for(MapItem item : mapsItems){
mapItems.add(new MapItem());
}
}
You may want to consider using a ReadWriteLock.
For processes which iterate over the list without modifying its contents, get a read lock on the shared ReentrantReadWriteLock. This allows multiple threads to have read access to the lock.
For processes which will modify the list, acquire the write lock on the shared lock. This will prevent all other threads from accessing the list (even read-only) until you release the write lock.
Is that make the iterating over the Vector safe from
ConcurrentModificationException?
YES It makes the iterating over Vector safe from ConcurrentModificationException.If it is not synchronized then in that case , if you are accessing the Vector via various threads and some other Thread is structurally modifying the Vector at any time after the iterator is created , the iterator will throw ConcurrentModificationException.
Consider running this code:
import java.util.*;
class VVector
{
static Vector<Integer> mapItems = new Vector<Integer>();
static
{
for (int i = 0 ; i < 200 ; i++)
{
mapItems.add(i);
}
}
public static void readVector()
{
Iterator<Integer> iterator = mapItems.iterator();
try
{
while(iterator.hasNext())
{
System.out.print(iterator.next() + "\t");
}
}
catch (Exception ex){ex.printStackTrace();System.exit(0);}
}
public static void main(String[] args)
{
VVector v = new VVector();
Thread th = new Thread( new Runnable()
{
public void run()
{
int counter = 0;
while ( true )
{
mapItems.add(345);
counter++;
if (counter == 100)
{
break;
}
}
}
});
th.start();
v.readVector();
}
}
At my system it is showing following output while execution:
0 1 2 3 4 5 6 7 8 9
java.util.ConcurrentModificationException
at java.util.AbstractList$Itr.checkForComodification(Unknown Source)
at java.util.AbstractList$Itr.next(Unknown Source)
at VVector.readVector(VVector.java:19)
at VVector.main(VVector.java:38)
But on the other hand if you make the block of code containing Iterator to access that Vector synchronized using mapItems as lock , it will prevent the execution of other methods related to Vector until that synchronized block is completed atomically .
if we invoke add method inside while loop then throws exception.
synchronized(mapItems) {
Iterator<MapItem> iterator = mapItems.iterator();
while(iterator.hasNext())
iterator.next();
mapItems.add("Something"); // throws ConcurrentModificationException
}