I want to write a traditional producer and consumer multithread program in Java. Producer thread will send message to thread-safe List until this list get full. once buffer is full, it will notify worker thread and buffer will be cleared. In my coding, the worker thread is not notified.
Do you know the reason ? Thanks.
package com;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.concurrent.locks.Condition;
import java.util.concurrent.locks.ReentrantLock;
public class ThreadTest {
public static void main(String[] args) {
final List<String> bufferSafeList = Collections.synchronizedList(new ArrayList<String>());
final ReentrantLock bufferLock = new ReentrantLock();
final Condition bufferFull = bufferLock.newCondition();
// final Condition bufferEmpty = bufferLock.newCondition();
Thread producerThread = new Thread(new Runnable() {
#Override
public void run() {
while (true) {
try {
bufferLock.lock();
bufferSafeList.add(System.currentTimeMillis() + "");
System.out.println("add to buffer " + bufferSafeList.size());
if (bufferSafeList.size() > 100) {
System.out.println("send wake up signal");
bufferFull.signalAll();
//waiting for buff cleared
while(!bufferSafeList.isEmpty()){
Thread.sleep(1000);
}
}
Thread.sleep(1000);
} catch(Exception e){
e.printStackTrace();
}
finally {
bufferLock.unlock();
}
}
}
});
producerThread.start();
Thread workerThread = new Thread(new Runnable() {
#Override
public void run() {
while (true) {
try {
bufferLock.lock();
System.out.println("waiting for wakeup signal");
bufferFull.await();
System.out.println("clear buffer");
bufferSafeList.clear();
} catch (Exception ex) {
ex.printStackTrace();
} finally {
bufferLock.unlock();
}
}
}
});
workerThread.start();
}
}
You only unlock the bufferLock in the finally, so the workerThread will never be able to progress further than trying to obtain the lock
When the buffer is full unlock so that the workerThread can continue
If the producer thread happens to run first, as it is likely to do because it is started first, it will likely lock bufferLock first, in which case it will fill the list and invoke bufferFull.signalAll() while the consumer is still waiting to acquire the lock. You then busy-wait for the worker to clear the list, which it never will do because it cannot proceed.
Moreover, even if you unlocked the lock after signalling, that would be too late. By the time the worker is await()ing the condition, the signal has already come and gone. Indeed, there is a race condition there even if the producer waits to acquire the lock until it is ready to signal.
The correct usage of a condition variable always involves testing whether the expected condition is satisfied before waiting, and looping back to wait some more if it is not satisfied after resuming from the wait. You can overcome the race in your code by implementing this.
Related
I have a situation in my application where events come in and the thread that handles them (signalling thread) must signal to another thread (working thread), thus far in an idle state, that it can run some code. Once the working thread is done it should wait to be signalled again. It is possible that events will arrive while the working thread is working. In this case it should move on and keep working immediately. One action by the working thread does enough work for any amount of incoming events, so there is no need to work once per event, just once as soon as possible after each event. Example correct behavior:
event comes in
worker thread starts work
worker thread finishes work
event comes in
worker thread starts work
event comes in
event comes in
worker thread finishes work
worker thread starts work
worker thread finishes work
4 events, 3 periods of work. It's an unfortunate but unavoidable requirement that the signalling thread cannot block while handling the event. I have implemented this at the moment using a BlockingQueue, which has the pointless side effect of filling itself up even though the contents are not interesting or even looked at. I was expecting to be able to make this work using CountDownLatch or CyclicBarrier or similar but I haven't been able to find a way. Here is my implementation:
import java.util.Random;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.LinkedBlockingQueue;
public class Main {
private static final class MyBarrier {
private BlockingQueue<Boolean> queue = new LinkedBlockingQueue<>();
void await() throws InterruptedException {
queue.take();
queue.clear();
}
void signal() {
queue.add(true);
}
}
private static Random random = new Random(0);
private static void sleepForMax(int maxMillis) {
sleep(random.nextInt(maxMillis));
}
private static void sleep(long millis) {
try {
Thread.sleep(millis);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}
public static void main(String[] args) {
MyBarrier myBarrier = new MyBarrier();
final ExecutorService singallingThread = Executors.newSingleThreadExecutor();
singallingThread.submit(() -> {
while (!Thread.currentThread().isInterrupted()) {
sleepForMax(1_000); // simulate period between events arriving
myBarrier.signal();
System.out.println("Signalling work to be done");
}
System.out.println("Thread interrupted");
});
final ExecutorService workingThread = Executors.newSingleThreadExecutor();
workingThread.submit(() -> {
while (!Thread.currentThread().isInterrupted()) {
try {
System.out.println("Waiting for work");
myBarrier.await();
} catch (InterruptedException e) {
break;
}
System.out.println("Doing work...");
sleepForMax(3_000); // simulate work being done
System.out.println("Work done");
}
System.out.println("Thread interrupted");
});
sleep(10_000);
singallingThread.shutdownNow();
workingThread.shutdownNow();
}
}
What's the better way to do this?
When I run your code with your implementation that uses Phaser, having changed the sleep times so that signalling occurs every 800 ms and processing takes 1000 ms, I get e.g. this output:
00008: Waiting for work
00808: Signalling work to be done
00808: Doing work... <-- worker starts working
01608: Signalling work to be done <-- signal came, so there's more work
01808: Work done
01809: Waiting for work <-- waits for work...
02409: Signalling work to be done <-- ...for 600 ms, until the next signal
02409: Doing work...
(The number to the left is milliseconds since start. Also, you can reproduce it with your code with random delays, but that's harder to reproduce and see there.)
If I understood it correctly, this is wrong. E.g. imagine what happens if signals stop coming.
Your code can probably work with this adjustment for your specific case:
private static final class MyBarrierWithPhaser {
private final Phaser phaser = new Phaser(1);
private int lastObservedPhase; // Phaser has initial phase 0
void await() throws InterruptedException {
// only works for 1 producer 1 worker; lastObservedPhase is kind of thread-local
lastObservedPhase = phaser.awaitAdvanceInterruptibly(lastObservedPhase);
}
void signal() {
phaser.arrive();
}
}
With this, the worker records the last phase it advanced to, and if the signal thread "arrives" before the next awaitAdvanceInterruptibly, then the Phaser phase gets updated, and when worker tries to wait using a stale phase, it will progress immediately; if the signal thread does not arrive before awaitAdvanceInterruptibly, then the worker will wait until the signal thread finally arrives.
Using simpler synchronization primitives, I can think of how to implement it using the synchronized-wait()-notify() mechanism:
private static final class MyBarrierWithSynchronized {
private boolean hasWork = false;
synchronized void await() throws InterruptedException {
while (!hasWork) {
wait();
}
hasWork = false;
}
synchronized void signal() {
hasWork = true;
notifyAll(); // or notify() if we are sure there is 1 signal thread and 1 worker thread
}
}
It has a couple of drawbacks: await() won't be interrupted if the thread is waiting to enter it. Also, some don't like synchronizing on this, I kept it so in order to be short. This can be rewritten using the java.util.concurrent.* analogues, this implementation will not have both of these drawbacks:
private static final class MyBarrierWithLock {
private boolean hasWorkFlag = false;
private final Lock lock = new ReentrantLock();
private final Condition hasWorkCond = lock.newCondition();
void await() throws InterruptedException {
lock.lockInterruptibly();
try {
while (!hasWorkFlag) {
hasWorkCond.await();
}
hasWorkFlag = false;
} finally {
lock.unlock();
}
}
void signal() {
lock.lock();
try {
hasWorkFlag = true;
hasWorkCond.signalAll(); // or signal() if we are sure there is 1 signal thread and 1 worker thread
} finally {
lock.unlock();
}
}
}
I'm experimenting with this, using java.util.concurrent.Phaser, which may work, but I haven't used Phaser before so I'm not sure.
private static final class MyBarrier2 {
private Phaser phaser = new Phaser(1);
void await() throws InterruptedException {
phaser.awaitAdvanceInterruptibly(phaser.getPhase());
}
void signal() {
phaser.arrive();
}
}
I am running code for one producer and multiple consumers. I want to prioritize execution of consumer threads. i.e. if I have consThread1, consThread2, consThread3. my question is how to restrict consThread3 to consume before consThread1 and consThread2
Producer.java
import java.util.concurrent.BlockingQueue;
import org.json.simple.JSONObject;
public class Producer implements Runnable {
private final BlockingQueue<Message> sharedQueue;
#SuppressWarnings("unchecked")
public Producer(BlockingQueue<Message> sharedQueue){
this.sharedQueue=sharedQueue;
}
#Override
public void run() {
try{
for(int i=0;i<4;i++) {
Message msg=new Message(""+i);
System.out.println("Producer Produced: " +msg.getMessage());
sharedQueue.put(msg);
Thread.sleep(400);
}
sharedQueue.put(new Message("exit")); // end of producing
System.out.println("-------Producer STOPPED------");
}
catch (InterruptedException e) {
e.printStackTrace();
}
}
}
Consumer.java
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.TimeUnit;
import org.json.simple.JSONObject;
public class Consumer implements Runnable{
private final BlockingQueue<Message> sharedQueue;
private String threadId;
public Consumer(BlockingQueue<Message> sharedQueue) {
this.sharedQueue=sharedQueue;
}
#SuppressWarnings("unchecked")
#Override
public void run() {
threadId = "Consumer-" + Thread.currentThread().getName();
try {
Message msg;
while (true){
msg=sharedQueue.poll(5,TimeUnit.SECONDS);
if(msg.getMessage()=="exit" || msg.getMessage()==null){
sharedQueue.put(new Message("exit"));
break;
}
System.out.println(threadId + ": Consuming Message " + msg.getMessage());
Thread.sleep(1000);
}
System.out.println(threadId + " STOPPED Consuming ");
}
catch (InterruptedException ie) {
ie.printStackTrace();
}
}
}
Test program ProducerConsumer.java
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.LinkedBlockingQueue;
import org.json.simple.JSONObject;
public class ProducerConsumer {
public static void main(String[] args) throws InterruptedException {
BlockingQueue<Message> sharedQueue = new LinkedBlockingQueue<>(10);
//Creating Producer and Consumer Thread
Thread prodThread = new Thread(new Producer(sharedQueue));
Thread consThread1 = new Thread(new Consumer(sharedQueue));
Thread consThread2 = new Thread(new Consumer(sharedQueue));
Thread consThread3 = new Thread(new Consumer(sharedQueue));
//Starting producer and Consumer thread
System.out.println("Producer and consumer threads started \n\n\n---------------------------------------");
prodThread.start();
consThread1.start();
consThread2.start();
consThread1.join();
consThread2.join();
consThread3.start();
}
}
If you want to execute one-by-one, why do you use multiple threads at all? You should just refactor to a single thread.
However, if you want to skip the refactoring, you can just put the consuming threads into a fixed thread pool. In a thread pool, you can set the maximum number of active threads, so you can set the maximum to one and the thread pool will execute the threads one by one.
Another alternative is to create a cyclic barrier where the barrier action is your third thread (it will be invoked after the others). You can execute the first two thread through the cyclic barrier. The barrier can count the finishing threads and will execute the third when the threshold reached. This should meet your goal of wanting for the 3rd consumer thread to wait until the event can be consumed.
My application will keep monitoring a folder, once it is not empty, it will wake up the worker thread. IllegalMonitorStateException will be thrown in the wait .
what is the reason ?
import java.io.File;
import java.util.concurrent.locks.Condition;
import java.util.concurrent.locks.ReentrantLock;
import org.apache.commons.io.FileUtils;
public class LockTest {
public static void main(String[] args) {
String folder = "C:\\temp\\test";
final ReentrantLock messageArrivedLock = new ReentrantLock();
final Condition messageArrivedCondition = messageArrivedLock.newCondition();
Thread workerThread = new Thread(new Runnable() {
#Override
public void run() {
System.out.println("worker thread is running");
messageArrivedLock.lock();
while (true) {
System.out.println("worker thread is waiting");
try {
messageArrivedCondition.wait(); //Exception here
System.out.println("worker thread wakes up");
} catch (Exception e) {
e.printStackTrace();
} finally {
if (messageArrivedLock.isHeldByCurrentThread()) {
messageArrivedLock.unlock();
}
}
}
}
});
workerThread.start();
while (true) {
long size = FileUtils.sizeOf(new File(folder));
System.out.println("size:" + size); // 1000
messageArrivedLock.lock();
try {
if (size > 0) {
messageArrivedCondition.signalAll();
}
} finally {
if (messageArrivedLock.isHeldByCurrentThread()) {
messageArrivedLock.unlock();
}
}
}
}
}
I'm going to assume you meant to invoke Condition#await, which will typically (as is the case here) have the same behavior you experienced with Object#wait.
The current thread is assumed to hold the lock associated with this
Condition when this method is called. It is up to the implementation
to determine if this is the case and if not, how to respond.
Typically, an exception will be thrown (such as
IllegalMonitorStateException) and the implementation must document
that fact.
Presumably your while loop iterated once, released the lock inside the finally. On its second iteration, your thread doesn't have the lock, so calling wait will throw the IllegalMonitorStateException. Your thread needs to own the lock to invoke await on the associated Condition.
You can acquire the lock within the while loop.
I've been trying to make a simple Java thread application, where one thread waits, and another wakes it up after 3 seconds. However, I can't seem to make it work and I just can't figure out why.
public class Deadlock extends Thread {
Object lock = new Object();
public static void main(String[] args) {
//WAITER THREAD
Deadlock waiter = new Deadlock() {
#Override
public void run() {
System.out.println("Waiter started");
synchronized(lock) {
try{
System.out.println("Waiter will wait for notify...");
lock.wait();
System.out.println("Woke up!");
} catch(InterruptedException e) {
e.printStackTrace();
}
}
}//run()
};
//WAKER THREAD
Deadlock waker = new Deadlock() {
#Override
public void run() {
System.out.println("Waker started");
synchronized(lock) {
System.out.println("Waker sleeping for 3 seconds.");
try{
Thread.sleep(3000);
}catch(InterruptedException e) {}
System.out.println("Waker notifying...");
lock.notifyAll();
}
}//run
};
waiter.start();
waker.start();
}
}
The output I get is:
Waiter started
Waiter will wait for notify...
Waker started
Waker sleeping for 3 seconds.
Waker notifying...
...and keeps running forever. I expected the waiter thread to wake up and the program to terminate.
Thanks
Your main problem is that the 'lock' is a class instance property so the two Deadlock instances do not share the same 'lock'. Hence, calling notifyAll() in the waker has no effect on the waiter because it's waiting on a different object. The simplest solution is to make 'lock' static:
static Object lock = new Object();
... I'd also make it private and final for good measure.
The second issue is that by starting the two threads together you don't really guarantee that the waiter will run first - I'd add a short delay before starting the waker.
How can I make a thread run only if the other thread is running too, meaning, if I return from run in one thread, then I want the other to stop running too,
my code looks something like this:
ClientMessageHandler clientMessagehandler = new ClientMessageHandler();
ServerMessageHandler serverMessagehandler = new ServerMessageHandler();
Thread thread1 = new Thread(serverMessagehandler);
Thread thread2 = new Thread(clientMessagehandler);
thread2.start();
thread1.start();
I want to cause thread1 to stop running when thread2 stops running.
edit: detecting when thread2 stops running in order stop thread1 from running, and not how to stop thread1 from running
thanks
This minimal example should demonstrate the basic idea:
import java.io.*;
import java.util.concurrent.LinkedBlockingQueue;
public class Test {
static LinkedBlockingQueue<String> msgBuf = new LinkedBlockingQueue<String>();
static volatile boolean keepRunning = true;
static Thread thread1, thread2;
public static void main(String[] args) throws IOException {
ClientMessageHandler clientMessagehandler = new ClientMessageHandler();
ServerMessageHandler serverMessagehandler = new ServerMessageHandler();
thread1 = new Thread(serverMessagehandler);
thread2 = new Thread(clientMessagehandler);
thread2.start();
thread1.start();
}
}
class ClientMessageHandler implements Runnable {
public void run() {
while (Test.keepRunning) {
try {
String msg = Test.msgBuf.take();
System.out.println("Eating " + msg);
} catch (InterruptedException ie) {
}
}
}
}
class ServerMessageHandler implements Runnable {
public void run() {
BufferedReader br = new BufferedReader(new InputStreamReader(System.in));
String in;
try {
while (!(in = br.readLine()).equals("quit")) {
System.out.println("Feeding " + in);
Test.msgBuf.offer(in);
}
} catch (IOException e) {
}
Test.keepRunning = false;
Test.thread2.interrupt();
}
}
Edit for question clarification
I see two immediate options:
Option 1. Have the ClientMessageHandler implementation terminate the ServerMessageHandler as it terminates. This means the client needs a reference to the server thread.
public class ClientMessageHandler implements Runnable {
Thread serverThread;
public ClientMessageHandler(Thread srvThread) {
this.serverThread = srvThread;
}
public void run() {
try {
while (true) { ... }
} finally {
serverThread.interrupt();
}
}
}
Option 2. Use thread2.join() or a CountDownLatch to wait for thread2 to terminate. When control returns from the join (or CountDownLatch#await()).
ClientMessageHandler clientMessagehandler = new ClientMessageHandler();
ServerMessageHandler serverMessagehandler = new ServerMessageHandler();
Thread thread1 = new Thread(serverMessagehandler);
Thread thread2 = new Thread(clientMessagehandler);
thread2.start();
thread1.start();
thread2.join(); //blocks until the client terminates
thread1.interrupt();
Make sure that inside thread1's run method, you have some logical place where you can check the interrupt status (Thread#isInterrupted()) and decide to terminate. Also, you must take care to handle InterruptedException properly and either terminate or reset the interrupt flag.
A Thread will only stop when the run() method returns. The Thread#interrupt() only signals that a request for interruption has made. You still have to write the code in run() method accordingly that it periodically checks Thread#isInterrupted() and handle accordingly. E.g. check for it on every unit of task the Thread is doing, or on every certain progress when sort of progresslistener is attached.