I have a list and a pointer to an element of the list. From time to time I need to:
append a value at the end of the queue
remove a value from the head of the queue
make the pointer advance to the next value in the list
That is:
from WRITE point of view it's a queue.
from READ point of view it's a list.
If I use a normal Iterator I get ConcurrentModificationException when modifying the queue; if I use a ListIterator I can only remove/add value at the iterator position.
Is there any standard data structure I can use to achieve this?
You can use a ConcurrentLinkedQueue. It allows modifying and iterating at the same time because it includes the necessary mechanisms for synchronization.
The followin snippet shows a working example where there are 3 Threads accessing the same queue without a problem:
1. Iterating and outputting elements
2. Adding new elements once in a while
3. Removing outputted elements once in a while
package test;
import java.util.Iterator;
import java.util.Queue;
import java.util.concurrent.ConcurrentLinkedQueue;
import java.util.concurrent.atomic.AtomicInteger;
public class QueueListTest
{
// private static final Queue<Integer> numbers = new ConcurrentLinkedQueue<>();
public static void main(String[] args)
{
final Queue<Integer> numbers = new ConcurrentLinkedQueue<>();
final AtomicInteger insert = new AtomicInteger(0);
final AtomicInteger output = new AtomicInteger();
for(int j = 0; j < 100; j++)
{
numbers.add(insert.getAndIncrement());
}
// print 1 number every 100ms
Thread t1 = new Thread() {
public void run()
{
Iterator<Integer> iter = numbers.iterator();
while(iter.hasNext())
{
int first = numbers.peek();
int size = numbers.size();
int last = first + size - 1;
int current = iter.next();
System.out.println("list from " + first + " to " + last + " # " + current);
output.set(current);
try
{
Thread.sleep(100);
}
catch(InterruptedException e)
{
e.printStackTrace();
}
}
}
};
// add 5 number every 500ms
Thread t2 = new Thread() {
public void run()
{
while(true)
{
for(int j = 0; j < 5; j++)
{
numbers.add(insert.getAndIncrement());
}
try
{
Thread.sleep(500);
}
catch(InterruptedException e)
{
e.printStackTrace();
}
}
}
};
// remove all printed numbers every 1000ms
Thread t3 = new Thread() {
public void run()
{
while(true)
{
try
{
Thread.sleep(1000);
}
catch(InterruptedException e)
{
e.printStackTrace();
}
int current = output.intValue();
while(numbers.peek() < current)
numbers.poll();
}
}
};
t1.start();
t2.start();
t3.start();
try
{
t1.join();
t2.join();
t3.join();
}
catch(InterruptedException e)
{
e.printStackTrace();
}
}
}
Because the queue is "linked" it should handle iterating, adding and removing in constant time and hence be the most efficient implementation you could use.
Not really. The problem is, there's no structure that does what you want in an efficient manner.
You could use ArrayList, iterate over indices and keep an updated current index after inserting at the start (increment by 1), but inserting at the start will not be efficient
You can't use LinkedList because it does not expose the current Node
Your best bet would probably be CursorableLinkedList from Apache Commons Collections (https://commons.apache.org/proper/commons-collections/apidocs/org/apache/commons/collections4/list/CursorableLinkedList.html)
Create a copy of your list before iterating over the items.
Or do you have any other constraints?
Related
I have a Output class which just prints everything that it gets to print.
public class Output {
private static List<String> textList = new ArrayList<>();
private static Output output = null;
private Output() {
Runnable task = () -> {
int lastIndex = 0;
while (true) {
while (lastIndex < textList.size()) {
System.out.println(lastIndex + " - " + textList.size() + ": " + textList.get(lastIndex));
outputText(textList.get(lastIndex));
lastIndex ++;
}
}
};
new Thread(task).start();
}
private static void outputText(String text) {
synchronized (System.out) {
System.out.println(text);
}
}
public static void say(String text) {
if (output == null) {
output = new Output();
}
textList.add(text);
}
}
When I add something to print, everything works fine:
for (int i = 0; i < 10; i++) {
Output.say("" + i);
}
But when I add a Thread.sleep to the loop it stops on the first output:
for (int i = 0; i < 10; i++) {
Output.say("" + i);
Thread.sleep(100);
}
How can I prevent it? I mean, I'm stopping with sleep just the main thread and not the separate thread.
When you don’t synchronize threads correctly, there is no guaranty that threads see updates made by other threads. They may either completely miss updates or see only parts of them, creating an entirely inconsistent result. Sometimes they may even appear to do the right thing. Without proper synchronization (in the sense of any valid construct specified to be thread safe), this is entirely unpredictable.
Sometimes, the chances of seeing a particular behavior are higher, like in your example. In most runs, the loop without sleep will complete before the other thread even starts its work, whereas inserting sleep raises the chance of lost updates after the second thread has seen values. Once the second thread has seen a value for textList.size(), it might reuse the value forever, evaluating lastIndex < textList.size() to false and executing the equivalent of while(true) { }.
It’s funny that the only place where you inserted a construct for thread safety, is the method outputText that is called by a single thread only (and printing to System.out is synchronized internally in most environments anyway).
Besides, it’s not clear why you are creating an object of type Output that has no relevance here, as all fields and methods are static.
Your code can be corrected and simplified to
public static void main(String[] args) throws InterruptedException {
List<String> textList = new ArrayList<>();
new Thread( () -> {
int index=0;
while(true) synchronized(textList) {
for(; index<textList.size(); index++)
System.out.println(textList.get(index));
}
}).start();
for (int i = 0; i < 10; i++) {
synchronized(textList) {
textList.add(""+i);
}
Thread.sleep(100);
}
}
though it still contains the issues of you original code of never terminating due to the infinite second thread and also burning the CPU with a polling loop. You should let the second thread wait for new items and add a termination condition:
public static void main(String[] args) throws InterruptedException {
List<String> textList = new ArrayList<>();
new Thread( () -> {
synchronized(textList) {
for(int index=0; ; index++) {
while(index>=textList.size()) try {
textList.wait();
} catch(InterruptedException ex) { return; }
final String item = textList.get(index);
if(item==null) break;
System.out.println(item);
}
}
}).start();
for (int i = 0; i < 10; i++) {
synchronized(textList) {
textList.add(""+i);
textList.notify();
}
Thread.sleep(100);
}
synchronized(textList) {
textList.add(null);
textList.notify();
}
}
This is still only an academic example that you shouldn’t use in real life code. There are classes for thread safe data exchange provided by the Java API removing the burden of implementing such things yourself.
public static void main(String[] args) throws InterruptedException {
ArrayBlockingQueue<String> queue = new ArrayBlockingQueue<>(10);
String endMarker = "END-OF-QUEUE"; // the queue does not allow null
new Thread( () -> {
for(;;) try {
String item = queue.take();
if(item == endMarker) break;// don't use == for ordinary strings
System.out.println(item);
} catch(InterruptedException ex) { return; }
}).start();
for (int i = 0; i < 10; i++) {
queue.put(""+i);
Thread.sleep(100);
}
queue.put(endMarker);
}
I have a Queue containing 10 elements, and I start 100 threads of which 6 may run concurrently, controlled by a Semaphore. When each thread runs, it takes the head element then adds it to the tail. But sometimes I get this exception:
java.util.NoSuchElementException
at java.util.LinkedList.removeFirst(LinkedList.java:270)
at java.util.LinkedList.remove(LinkedList.java:685)
at IBM.SemApp$1.run(SemApp.java:27)
at java.lang.Thread.run(Thread.java:745)
import java.util.LinkedList;
import java.util.Queue;
import java.util.Random;
import java.util.concurrent.Semaphore;
public class SemApp {
public static void main(String[] args) {
Queue queueB = new LinkedList<>();
for (int i = 0; i < 10; i++) {
queueB.add("Object " + i);
}
Runnable limitedCall = new Runnable() {
final Random rand = new Random();
final Semaphore available = new Semaphore(6);
int count = 0;
public void run() {
int time = rand.nextInt(15);
try {
available.acquire();
String A = (String) queueB.remove();
queueB.add(A);
available.release();
count++;
System.out.println(count);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
};
for (int i = 0; i < 100; i++) {
new Thread(limitedCall).start();
}
}
}
What am I doing wrong?
The problem is that LinkedList is not a thread-safe structure.
Therefore, it should not be shared and modified by multiple concurrent threads as the changes on queueB might not be properly "communicated" to other threads.
Try using a LinkedBlockingQueue instead.
Also, use an AtomicLong for count for the same reason: it is shared in between several threads and you want to avoid race conditions.
The fact that up to six threads may be operating on the queue concurrently means that modifications are not safe.
package workouts;
public class synchro {
private int count = 0;
public void counting() {
Thread T1 = new Thread(new Runnable() {
#Override
public void run() {
for (int i = 0; i < 10000; i++) {
count++;
}
}
});
Thread T2 = new Thread(new Runnable() {
#Override
public void run() {
for (int i = 0; i < 10000; i++) {
count++;
}
}
});
T1.start();
T2.start();
try {
T1.join();
T2.join();
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
System.out.println("counting =" + count);
}
public static void main(String[] args) {
synchro sync = new synchro();
sync.counting();
}
}
but when introduced a synchronised method and call it inside run method like below.. the output is 20000 for how many times if u run it..can someone explain the difference between the above and below code
public class synchro {
private int count = 0;
public synchronized void dosinglethread(){
count++;
}
public void counting() {
Thread T1 = new Thread(new Runnable() {
#Override
public void run() {
for (int i = 0; i < 10000; i++) {
dosinglethread();
}
}
});
Thread T2 = new Thread(new Runnable() {
#Override
public void run() {
for (int i = 0; i < 10000; i++) {
dosinglethread();
}
}
});
T1.start();
T2.start();
try {
T1.join();
T2.join();
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
System.out.println("counting =" + count);
}
public static void main(String[] args) {
synchro sync = new synchro();
sync.counting();
}
}
When you say count++, three things happen:
The current value of count is retrieved by the VM
The VM increments the value by 1
The new value is put back into count
It's entirely possible that T1 and T2 both get the value of count, then increment it separately, and then but back the result, like so:
Timeslot T1 T2
1 count = 3 ----
2 ---- count = 3
3 3 + 1 = 4 ----
4 ---- 3 + 1 = 4
5 store 4 in count ----
6 ---- store 4 in count
So now, count++ has been called twice, but the value has only increased by one. To prevent this, you have to make the increment atomic. Atomic means that either the entire sequence of operations is executed, or none of it is. Simply put, if two statements are synchronized on the same Object, they will not interleave.
In your second piece of code, dosinglethread() is declared synchronized. This is the equivalent of:
public void dosinglethread() {
synchronized (this) {
count++;
}
}
This means that when one of the threads starts executing it, they acquire a lock on your synchro instance. When the second thread also tries to execute the method, it will see that another thread already owns the lock on this, so it has to wait. When the first thread completes the method, it will release the lock and the other thread can then take it.
So why didn't volatile work? volatile variables will not cause threads to wait until they are available. Instead, when count is volatile and you call count++, the following happens: (Code adapted from Javamex's Tutorial)
int temp;
synchronized (count) {
temp = count;
}
temp = temp + 1;
synchronized (count) {
count = temp;
}
Note that this code is for illustration only: synchronized can not be used on primitives.
It's clear, then, that the threads might still be paused at temp = temp + 1, giving the same problems as when you don't do any synchronization.
For more information, check the tutorial I mentioned.
This question already has answers here:
Java: How to use Thread.join
(3 answers)
Closed 8 years ago.
I am writing application using multi threads to count number of char inside txt file.
File contains 10 000 000 chars. 10 000 rows and 1 000 columns.
EDITED
About first part of the question:
Prevoius questions was about threads, I used a thread.join(); in wrong way.
Second part:
Could you help me improve the performance and safety? Here is my code (Use of the Semaphore is required):
public class MultiThread implements Runnable {
HashMap<String, AtomicInteger> asciiMap = Maps.newHashMap();
LinkedList<String> asciiLines = ReadDataFromFile.lines;
Semaphore mutex = new Semaphore(1);
AtomicInteger i = new AtomicInteger(0);
int index;
#Override
public void run() {
long actual = 0;
try {
Calculate calculate = new Calculate();
long multiStart = System.currentTimeMillis();
Thread first = new Thread(calculate);
Thread second = new Thread(calculate);
Thread third = new Thread(calculate);
first.start();
second.start();
third.start();
first.join();
second.join();
third.join();
long multiEnd = System.currentTimeMillis();
actual = multiEnd - multiStart;
} catch (InterruptedException ex) {
Logger.getLogger(MultiThread.class.getName()).log(Level.SEVERE, null, ex);
}
int sum = 0;
for (Map.Entry<String, AtomicInteger> entry : asciiMap.entrySet()) {
System.out.println("Char: " + entry.getKey() + " , number: " + entry.getValue());
sum = sum + entry.getValue().get();
}
System.out.println("Time: " + actual);
}
int increment() {
try {
mutex.acquire();
index = i.incrementAndGet();
mutex.release();
} catch (InterruptedException ex) {
Logger.getLogger(MultiThread.class.getName()).log(Level.SEVERE, null, ex);
}
return index;
}
public class Calculate implements Runnable {
public Calculate() {
}
#Override
public void run() {
while (i.get() < asciiLines.size()) {
for (String oneCharacter : asciiLines.get(i.get()).split("")) {
if (asciiMap.containsKey(oneCharacter)) {
asciiMap.replace(oneCharacter, new AtomicInteger(asciiMap.get(oneCharacter).incrementAndGet()));
} else {
asciiMap.put(oneCharacter, new AtomicInteger(1));
}
}
i = new AtomicInteger(increment());
}
}
}
}
Every element inside LinkedList contains one row (1 000 chars).
Your code does absolutely no multithreading. Thread.join means wait until that thread has finished executing, then continue the current thread of execution. Right now, your code is executing each thread serially. You want to interleave your calls to start and join.
Thread first = new Thread(calculate);
Thread third = new Thread(calculate);
Thread second = new Thread(calculate);
first.start();
second.start();
third.start();
first.join();
second.join();
third.join();
I have very similar multithreaded code elsewhere in my codebase that works fine, but I can't see quite what's going wrong here.
This is a simple multi-threaded process to generate some result XML for a search query. The output of running this method is:
Returning from threads
The line System.out.println("Finished multithreading loop");" is never reached.
Modifying the number of threads doesn't help.
private void fillAllResults() {
int threads = 2;
final FutureTask[] tasks = new FutureTask[threads];
final ExecutorService executor = Executors.newCachedThreadPool();
for (int i = 0; i < allResults.size(); i++) {
tasks[i] = new FutureTask<Integer>(new Callable<Integer>() {
public Integer call() throws Exception {
int index;
while ((index = getResultsIndex()) < allResults.size()) {
System.out.println("Processing result " + index);
Result result = allResults.get(index);
fillResultXML(result);
}
System.out.println("Returning from threads");
return 1;
}
});
executor.execute(tasks[i]);
}
for (int i = 0; i < threads; i++) {
try {
tasks[i].get();
} catch (InterruptedException e) {
e.printStackTrace();
} catch (ExecutionException e) {
e.printStackTrace();
}
}
executor.shutdown();
System.out.println("Finished multithreading loop");
}
Edit, thanks all for the quick replies! Here's the answers:
It shows 'processing result' as many times as I have results. If allResults.size() is 25, it shows processing result 1, processing result 2 ... processing result 24.
Here's the extra code that's missing:
private List<Result> allResults = new ArrayList<Result>();
private int resultsIndex = 0;
private synchronized int getResultsIndex() {
return resultsIndex++;
}
And in case anyone's wondering, I can guarantee that none of the code within the loop increases the size of allResults.
I suppose it is related to the fact, that your array tasks has a length of threads (i.e. two in your case) but you assign more values to it within the lines
for (int i = 0; i < allResults.size(); i++) {
tasks[i] = ...
....
}
If your list allResults has more than two entries your thread will be stopped by an ArrayIndexOutOfBoundsException. Maybe you catch this one but do not handle it properly outside the code you presented.
It looks like getResultsIndex() isn't updating after every loop resulting in an infinite loop.
It's not clear from your code what allResults and getResultsIndex are, but you never seem to update whatever getResultsIndex returns.