Single Producer Multiple Consumer Java - java

I am new to Java Concurrency and trying to achieve/implement Single Producer[P1] and Multiple Consumer [C1,C2,C3].
The idea is producer [P1] puts in the value and consumers C1,C2,C3 all runs their task to read the value individually as put in by P1. Once C1,C2,C3 reads the values, P1 again puts a new data. Then C1,C2,C3 reads data and this loop goes on.
Wait Notify works fine for Single Producer Single Consumer, but in this case of Single Producer Multiple Consumer wait notify concept doesn't look to be good strategy. How should I approach this problem.

Thanks to #Ivan and #Andreas.
#Ivan - In his comment made me understand how Producer Consumer pattern behaves.
#Andreas - In his Comment suggested the usage of Phaser. (I used Cyclic Barrier instead since my number of registered threads does not vary dynamically)
With both their comments sharing the below sample code.
Please do suggest improvisation if there any or a better way to handle this.
Main Class
public static void main(String[] args)
{
SharedSpace sharedSpace = new SharedSpace(new LinkedBlockingQueue<Integer>(1));
new Thread(new Producer(sharedSpace)).start();
Consumer consumerRunnable = new Consumer(sharedSpace);
new Thread(consumerRunnable).start();
CyclicBarrier barrier = new CyclicBarrier(3,consumerRunnable);
new Thread(new EndUser(barrier,consumerRunnable)).start();
new Thread(new EndUser(barrier,consumerRunnable)).start();
new Thread(new EndUser(barrier,consumerRunnable)).start();
}
Producer
private SharedSpace sharedSpace;
public Producer(SharedSpace sharedSpace) {
super();
this.sharedSpace = sharedSpace;
}
public SharedSpace getSharedSpace() {
return sharedSpace;
}
public void setSharedSpace(SharedSpace sharedSpace) {
this.sharedSpace = sharedSpace;
}
#Override
public void run() {
for(int i=0;i<3;i++)
{
int value = (int) (Math.random()*30);
sharedSpace.addValue(value);
}
}
Queue Shared by Producer and Consumer
private BlockingQueue<Integer> queue;
public SharedSpace(BlockingQueue<Integer> queue) {
super();
this.queue = queue;
}
public BlockingQueue<Integer> getQueue() {
return queue;
}
public void setQueue(BlockingQueue<Integer> queue) {
this.queue = queue;
}
public void addValue(int value)
{
try {
queue.put(value);
System.out.println(System.nanoTime()+" Producer added value "+value);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
public int getValue() throws InterruptedException
{
return queue.take();
}
Consumer
private SharedSpace sharedSpace;
private Integer value;
public Consumer(SharedSpace sharedSpace) {
super();
this.sharedSpace = sharedSpace;
}
public SharedSpace getSharedSpace() {
return sharedSpace;
}
public void setSharedSpace(SharedSpace sharedSpace) {
this.sharedSpace = sharedSpace;
}
public Integer getValue() {
return value;
}
public void setValue(Integer value) {
this.value = value;
}
#Override
public void run()
{
try {
setValue(sharedSpace.getValue());
} catch (InterruptedException e) {
e.printStackTrace();
}
}
EndUser
CyclicBarrier barrier;
Consumer consumer;
public EndUser(CyclicBarrier barrier) {
super();
this.barrier = barrier;
}
public EndUser(CyclicBarrier barrier, Consumer consumer) {
super();
this.barrier = barrier;
this.consumer = consumer;
}
public Consumer getConsumer() {
return consumer;
}
public void setConsumer(Consumer consumer) {
this.consumer = consumer;
}
public CyclicBarrier getBarrier() {
return barrier;
}
public void setBarrier(CyclicBarrier barrier) {
this.barrier = barrier;
}
#Override
public void run() {
try
{
while(true)
{
System.out.println(consumer.getValue());
barrier.await();
}
}
catch (InterruptedException e) {
e.printStackTrace();
} catch (BrokenBarrierException e) {
e.printStackTrace();
}
}
Output [Consumer doesn't read from Producer unless all EndUser has taken their Data]
Producer added value 24
Producer added value 10
24
24
24
10
10
Producer added value 0
10
0
0
0

Related

Why does the ThreadpoolExecutor code never stop running?

I run the code in IntelliJ IDEA and it would not stop. I use exec.shutdownNow() instead and it stops successfully. So, I think is the problem of generator, but I can't figure out what problem it is.
The main method:
public static void main(String[] args) {
PrimeProducer generator = new PrimeProducer(new ArrayBlockingQueue<>(10));
ExecutorService exec = Executors.newFixedThreadPool(1);
exec.execute(generator);
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
throw new RuntimeException(e);
} finally {
generator.cancel();
}
//generator.get();
exec.shutdown();
}
And the PrimeProducer class:
public class PrimeProducer extends Thread{
private final BlockingQueue<BigInteger> queue;
PrimeProducer(BlockingQueue<BigInteger> queue){
this.queue = queue;
}
#Override
public void run() {
try {
BigInteger p = BigInteger.ONE;
while(!Thread.currentThread().isInterrupted()){
queue.put(p = p.nextProbablePrime());
}
} catch (InterruptedException e) {
}
}
public void cancel(){interrupt();}
public void get(){
for(BigInteger i : queue){
System.out.println(i.toString());
}
}
}
The latest version that still can't work:
public class PrimeProducer implements Runnable {
private final BlockingQueue<BigInteger> queue;
PrimeProducer(BlockingQueue<BigInteger> queue){
this.queue = queue;
}
#Override
public void run() {
try {
BigInteger p = BigInteger.ONE;
while(!Thread.currentThread().isInterrupted()){
queue.put(p = p.nextProbablePrime());
}
} catch (InterruptedException e) {
}
}
public void cancel(){Thread.currentThread().interrupt();}
public synchronized void get(){
for(BigInteger i : queue){
System.out.println(i.toString());
}
}
}
You have PrimeProducer extend Thread, but you don't actually use it as a thread, you only use it as a Runnable, which is executed on a thread of the thread pool. The cancel has no effect, because it calls the interrupt() of a different thread than the one that is actually running. When you then call exec.shutdown(), the thread pool will interrupt its threads to signal them they should end, when then means PrimeProducer.run will stop.
Instead, use a volatile boolean to end your run (and implement Runnable instead of extending Thread):
public class PrimeProducer implements Runnable {
private final BlockingQueue<BigInteger> queue;
private volatile boolean cancelled;
PrimeProducer(BlockingQueue<BigInteger> queue) {
this.queue = queue;
}
#Override
public void run() {
try {
BigInteger p = BigInteger.ONE;
while (!cancelled) {
queue.put(p = p.nextProbablePrime());
}
} catch (InterruptedException e) {
}
}
public void cancel() {
cancelled = true;
}
public void get() {
for (BigInteger i : queue) {
System.out.println(i.toString());
}
}
}

Stop the whole producer and consumer threads and yield the control to main thread

DefaultRunners are producers
and OrderTaker is a consumer
They both share a OrderQueue.
Currently, I use the variable isDone to indicate if a game is finished.
Once each round is done, I want to make it repeat again and again.
However, in my current implementation it will only run once.
How could I solve it?
public class OrderQueue {
public synchronized void pushOrder(Order order) throws InterruptedException {
if (isDone) {
wait();
} else {
runnersQueue.addLast(order);
notifyAll();
}
}
public void pullOrder() {
try {
if (runnersQueue.size() == 0) {
} else if (isDone) {
wait();
} else {
handleOrder(runnersQueue.pop());
}
} catch (InterruptedException e) {
}
}
In my main class
while(true){
enterYesToStart();
DefaultRunners dfltRunner = new DefaultRunners(queue);
OrderTaker taker = new OrderTaker(queue);
taker.run();
System.out.println("This round is finished"); # never reach to this line
}
Here's the full source code for the example
https://gist.github.com/poc7667/d98e3bf5b3b470fcb51e00d9a0d80931
I've taken a look at your code snippets and the problem is fairly obvious.
The main thread runs the OrderTaker runnable. The main thread is stuck in an eternal loop as the while statement cannot complete unless it throws an exception. (Note that the same is true for your ThreadRunner runnable.)
This means that the main thread i still pulling orders while the race is already done.
The OrderTaker should exit it's while loop while once the race is done. I guess that there are multiple ways achieve this, but one way is use a shared variable.
I took your code and adapted it into a working example.
import java.util.*;
import java.util.concurrent.ConcurrentLinkedDeque;
import java.util.concurrent.locks.Lock;
import java.util.concurrent.locks.ReadWriteLock;
import java.util.concurrent.locks.ReentrantReadWriteLock;
public class RaceApp {
public static void main(String[] args) throws InterruptedException {
final RaceUpdateManager queue = new RaceUpdateManager();
for (int i = 0; i < 3; i++) {
queue.reset();
List<Thread> threads = Arrays.asList(
new Thread(new Runner("Tortoise", 0, 10, queue)),
new Thread(new Runner("Hare", 90, 100, queue))
);
for (Thread thread : threads) {
thread.start();
}
RaceUpdatesProcessor processor = new RaceUpdatesProcessor(queue);
processor.run();
System.out.println("Game finished");
}
}
private static class RaceUpdateManager {
private static final int TOTAL_DISTANCE = 300;
//thread-safe implementation for queue so no external syncrhonization is required when adding/removing updates
private final Deque<RaceUpdate> runnersQueue = new ConcurrentLinkedDeque<>();
//lock used to sync changes to runnersRecords and done variables
private final ReadWriteLock raceStatusLock = new ReentrantReadWriteLock();
private final Map<String, Integer> runnersRecords = new HashMap<>();
private volatile boolean raceDone = false;//volatile keyword guarantees visibility of changes to variables across threads
public boolean isRaceDone() {
return raceDone;
}
//updates can by added simultaneously (read lock)
public void register(RaceUpdate raceUpdate) throws InterruptedException {
Lock readLock = raceStatusLock.readLock();
readLock.lock();
try {
if (!raceDone) {
runnersQueue.addLast(raceUpdate);
}//ignore updates when the race is done
} finally {
readLock.unlock();
}
}
//but they need to be processed in order (exclusive write lock)
public void processOldestUpdate() {
Lock writeLock = raceStatusLock.writeLock();
writeLock.lock();
try {
RaceUpdate raceUpdate = runnersQueue.poll();
if (raceUpdate != null) {
handleUpdate(raceUpdate);
}
} finally {
writeLock.unlock();
}
}
private void handleUpdate(RaceUpdate raceUpdate) {
Integer distanceRun = runnersRecords.merge(
raceUpdate.runner, raceUpdate.distanceRunSinceLastUpdate, (total, increment) -> total + increment
);
System.out.printf("%s: %d\n", raceUpdate.runner, distanceRun);
if (distanceRun >= TOTAL_DISTANCE) {
raceDone = true;
System.out.printf("Winner %s\n", raceUpdate.runner);
}
}
public void reset() {
Lock writeLock = raceStatusLock.writeLock();
writeLock.lock();
try {
runnersQueue.clear();
runnersRecords.clear();
raceDone = false;
} finally {
writeLock.unlock();
}
}
}
public static class Runner implements Runnable {
private final String name;
private final int rest;
private final int speed;
private final RaceUpdateManager queue;
private final Random rand = new Random();
public Runner(String name, int rest, int speed, RaceUpdateManager queue) {
this.name = name;
this.rest = rest;
this.speed = speed;
this.queue = queue;
}
#Override
public void run() {
while (!queue.isRaceDone()) {
try {
if (!takeRest()) {
queue.register(new RaceUpdate(this.name, this.speed));
}
Thread.sleep(100);
} catch (InterruptedException e) {
//signal that thread was interrupted and exit method
Thread.currentThread().interrupt();
return;
}
}
}
private boolean takeRest() {
return rand.nextInt(100) < rest;
}
}
public static class RaceUpdatesProcessor implements Runnable {
private final RaceUpdateManager queue;
public RaceUpdatesProcessor(RaceUpdateManager queue) {
this.queue = queue;
}
#Override
public void run() {
while (!queue.isRaceDone()) {
try {
queue.processOldestUpdate();
Thread.sleep(50);
} catch (InterruptedException e) {
//signal that thread was interrupted and exit method
Thread.currentThread().interrupt();
return;
}
}
}
}
public static class RaceUpdate {
public final String runner;
public final int distanceRunSinceLastUpdate;
public RaceUpdate(String runner, int distanceRunSinceLastUpdate) {
this.runner = runner;
this.distanceRunSinceLastUpdate = distanceRunSinceLastUpdate;
}
}
}

Is this Double-Checked Locking that is not Broken?

This article explains "Double-Checked Locking" where the idea is to reduce lock contention. As the article explains it does not work. See the code sample in the table "(Still) Broken multithreaded version "Double-Checked Locking" idiom".
Now I think I found a variant that should work. Question is whether that is correct. Let's say we have a consumer and a producer that exchange data through a shared queue:
class Producer {
private Queue queue = ...;
private AtomicInteger updateCount;
public void add(Data data) {
synchronized(updateCount) {
queue.add(task);
updateCount.incrementAndGet();
}
}
}
class Consumer {
private AtomicInteger updateCount = new AtomicInteger(0);
private int updateCountSnapshot = updateCount.get();
public void run() {
while(true) {
// do something
if(updateCountSnapshot != updateCount.get()) {
// synchronizing on the same updateCount
// instance the Producer has
synchronized(updateCount) {
Data data = queue.poll()
// mess with data
updateCountSnapshot = updateCount.get();
}
}
}
}
}
Question now is whether you think this approach works. I'm asking to be sure, because tons of things would break if it doesn't ... The idea is to reduce lock contention when only entering a synchronized block in the consumer when the updateCount has changed in the meanwhile.
I suspect you are looking more for a Code Review.
You should consider the following:
This is not double-checked locking.
Your consumer will spin on nothing and eat cpu while no data is arriving.
You use an AtomicInteger as a Semaphore.
A BlockingQueue will do all of this for you.
You haven't properly ensured that updateCount is shared.
You do not have to synchronize on atomics.
Here's a simple Producer/Consumer pair for demonstration.
public class TwoThreads {
public static void main(String args[]) throws InterruptedException {
System.out.println("TwoThreads:Test");
new TwoThreads().test();
}
// The end of the list.
private static final Integer End = -1;
static class Producer implements Runnable {
final Queue<Integer> queue;
public Producer(Queue<Integer> queue) {
this.queue = queue;
}
#Override
public void run() {
try {
for (int i = 0; i < 1000; i++) {
queue.add(i);
Thread.sleep(1);
}
// Finish the queue.
queue.add(End);
} catch (InterruptedException ex) {
// Just exit.
}
}
}
static class Consumer implements Runnable {
final Queue<Integer> queue;
public Consumer(Queue<Integer> queue) {
this.queue = queue;
}
#Override
public void run() {
boolean ended = false;
while (!ended) {
Integer i = queue.poll();
if (i != null) {
ended = i == End;
System.out.println(i);
}
}
}
}
public void test() throws InterruptedException {
Queue<Integer> queue = new LinkedBlockingQueue<>();
Thread pt = new Thread(new Producer(queue));
Thread ct = new Thread(new Consumer(queue));
// Start it all going.
pt.start();
ct.start();
// Wait for it to finish.
pt.join();
ct.join();
}
}

Java Multithreading synchronization

I am having a trouble to share a resource with three threads...
public class Subject{
int i;
boolean valueSet1 = false;
boolean valueSet2 = true;
boolean valueSet3 = true;
void put(int i){
while(valueSet1){
try{
wait();
}catch(InterruptedException e){
System.out.println("Producer thread interrupted");
}
}
System.out.println("Producer thread wakesup");
valueSet1=true;
valueSet2=false;
this.i=i;
System.out.println("Put: "+i);
notify();
}
void makesquare(){
int a;
while(valueSet2){
try{
System.out.println("Convertor thread goin to sleep");
wait();
}catch(InterruptedException e){
System.out.println("Convertor thread interrupted");
}
}
System.out.println("Convertor thread wakesup");
valueSet2 = true;
valueSet3=false;
a = this.i;
this.i = a*a;
System.out.println("Made: "+i);
notify();
}
void get(){
while(valueSet3){
try{
System.out.println("Consumer thread goin to sleep");
wait();
}catch(InterruptedException e){
System.out.println("Consumer thread interrupted");
}
}
System.out.println("Consumer thread wakesup");
valueSet3 = true;
valueSet1 = false;
System.out.println("Got: "+i);
notify();
}
}
class Producer implements Runnable{
Subject q;
Thread t;
String msg;
Producer(Subject q, String msg){
this.q=q;
this.msg = msg;
t = new Thread(this, this.msg);
}
#Override
public void run(){
int i=2;
while(true){
synchronized(q){
q.put(i++);
}
}
}
}
class Consumer implements Runnable{
Subject q;
Thread t;
String msg;
Consumer(Subject q,String msg){
this.q = q;
this.msg = msg;
t = new Thread(this, this.msg);
}
#Override
public void run(){
while(true){
synchronized(q){
q.get();
}
}
}
}
class Convertor implements Runnable{
Subject q;
Thread t;
String msg;
Convertor(Subject q, String msg){
this.q=q;
this.msg = msg;
t = new Thread(this, this.msg);
}
#Override
public void run(){
while(true){
synchronized(q){
q.makesquare();
}
}
}
}
There are three threads in the program. One thread produces while another makes square of the produced quantity. While the last thread consumes the squared product.They all share the same object.
public class Thread3way {
public static void main(String[] args) {
Subject q = new Subject();
Producer P = new Producer(q, "producer");
Convertor Cv = new Convertor(q, "convertor");
Consumer Cs = new Consumer(q, "consumer");
P.t.start();
Cv.t.start();
Cs.t.start();
}
}
Your wait loop is not quite correct, you need to keep looping until your condition is true before you do any modifications. I'd do it like this (using a State enum to make things clearer):
public class Subject {
static enum State { EMPTY, WAITING_TO_SQUARE, WAITING_TO_GET };
State state;
int value;
public synchronized void put(int i) {
while (state != State.EMPTY) {
try {
wait();
}catch(InterruptedException e) {
System.out.println("Put interrupted");
}
}
value = i;
state = State.WAITING_TO_SQUARE;
}
}
As a commenter pointed out, you don't need to synchronize twice, either synchronized(q) around the Subject calls, or declare your methods synchronized. You don't need both.
No need to use Runnables explicitly, just make Producer, Convertor, and Consumer Threads directly. Then you don't need the t fields, and you can start the threads like P.start().
Sounds to me like you're doing some kind of pipeline with 3 stages. Why not enjoy the benefits of ExecutorService:
class ConvertorTask implements Runnable {
private int number;
private static ExecutorService consumer = Executors.newSingleThreadExecutor();
public ConvertorTask(int number) {
this.number = number;
}
public void run() {
consumer.submit(new ConsumerTask(number * number));
}
}
class ConsumerTask implements Runnable {
private int number;
public ConsumerTask(int number) {
this.number = number;
}
public void run() {
System.out.println(number);
}
}
class Producer implements Runnable {
private ExecutorService convertor = Executors.newSingleThreadExecutor();
public void run() {
int i = 0;
while(true) {
convertor.submit(new ConvertorTask(i++));
}
}
}
public class Test {
public static void main(String[] args) throws InterruptedException {
Thread t = new Thread(new Producer());
t.start();
}
}

Producer/Consumer threads using a Queue

I'd like to create some sort of Producer/Consumer threading app. But I'm not sure what the best way to implement a queue between the two.
So I've some up with two ideas (both of which could be entirely wrong). I would like to know which would be better and if they both suck then what would be the best way to implement the queue. It's mainly my implementation of the queue in these examples that I'm concerned about. I'm extending a Queue class that is an in house class and is thread safe. Below are two examples with 4 classes each.
Main class-
public class SomeApp
{
private Consumer consumer;
private Producer producer;
public static void main (String args[])
{
consumer = new Consumer();
producer = new Producer();
}
}
Consumer class-
public class Consumer implements Runnable
{
public Consumer()
{
Thread consumer = new Thread(this);
consumer.start();
}
public void run()
{
while(true)
{
//get an object off the queue
Object object = QueueHandler.dequeue();
//do some stuff with the object
}
}
}
Producer class-
public class Producer implements Runnable
{
public Producer()
{
Thread producer = new Thread(this);
producer.start();
}
public void run()
{
while(true)
{
//add to the queue some sort of unique object
QueueHandler.enqueue(new Object());
}
}
}
Queue class-
public class QueueHandler
{
//This Queue class is a thread safe (written in house) class
public static Queue<Object> readQ = new Queue<Object>(100);
public static void enqueue(Object object)
{
//do some stuff
readQ.add(object);
}
public static Object dequeue()
{
//do some stuff
return readQ.get();
}
}
OR
Main class-
public class SomeApp
{
Queue<Object> readQ;
private Consumer consumer;
private Producer producer;
public static void main (String args[])
{
readQ = new Queue<Object>(100);
consumer = new Consumer(readQ);
producer = new Producer(readQ);
}
}
Consumer class-
public class Consumer implements Runnable
{
Queue<Object> queue;
public Consumer(Queue<Object> readQ)
{
queue = readQ;
Thread consumer = new Thread(this);
consumer.start();
}
public void run()
{
while(true)
{
//get an object off the queue
Object object = queue.dequeue();
//do some stuff with the object
}
}
}
Producer class-
public class Producer implements Runnable
{
Queue<Object> queue;
public Producer(Queue<Object> readQ)
{
queue = readQ;
Thread producer = new Thread(this);
producer.start();
}
public void run()
{
while(true)
{
//add to the queue some sort of unique object
queue.enqueue(new Object());
}
}
}
Queue class-
//the extended Queue class is a thread safe (written in house) class
public class QueueHandler extends Queue<Object>
{
public QueueHandler(int size)
{
super(size); //All I'm thinking about now is McDonalds.
}
public void enqueue(Object object)
{
//do some stuff
readQ.add();
}
public Object dequeue()
{
//do some stuff
return readQ.get();
}
}
And go!
Java 5+ has all the tools you need for this kind of thing. You will want to:
Put all your Producers in one ExecutorService;
Put all your Consumers in another ExecutorService;
If necessary, communicate between the two using a BlockingQueue.
I say "if necessary" for (3) because from my experience it's an unnecessary step. All you do is submit new tasks to the consumer executor service. So:
final ExecutorService producers = Executors.newFixedThreadPool(100);
final ExecutorService consumers = Executors.newFixedThreadPool(100);
while (/* has more work */) {
producers.submit(...);
}
producers.shutdown();
producers.awaitTermination(Long.MAX_VALUE, TimeUnit.NANOSECONDS);
consumers.shutdown();
consumers.awaitTermination(Long.MAX_VALUE, TimeUnit.NANOSECONDS);
So the producers submit directly to consumers.
OK, as others note, the best thing to do is to use java.util.concurrent package. I highly recommend "Java Concurrency in Practice". It's a great book that covers almost everything you need to know.
As for your particular implementation, as I noted in the comments, don't start Threads from Constructors -- it can be unsafe.
Leaving that aside, the second implementation seem better. You don't want to put queues in static fields. You are probably just loosing flexibility for nothing.
If you want to go ahead with your own implementation (for learning purpose I guess?), supply a start() method at least. You should construct the object (you can instantiate the Thread object), and then call start() to start the thread.
Edit: ExecutorService have their own queue so this can be confusing.. Here's something to get you started.
public class Main {
public static void main(String[] args) {
//The numbers are just silly tune parameters. Refer to the API.
//The important thing is, we are passing a bounded queue.
ExecutorService consumer = new ThreadPoolExecutor(1,4,30,TimeUnit.SECONDS,new LinkedBlockingQueue<Runnable>(100));
//No need to bound the queue for this executor.
//Use utility method instead of the complicated Constructor.
ExecutorService producer = Executors.newSingleThreadExecutor();
Runnable produce = new Produce(consumer);
producer.submit(produce);
}
}
class Produce implements Runnable {
private final ExecutorService consumer;
public Produce(ExecutorService consumer) {
this.consumer = consumer;
}
#Override
public void run() {
Pancake cake = Pan.cook();
Runnable consume = new Consume(cake);
consumer.submit(consume);
}
}
class Consume implements Runnable {
private final Pancake cake;
public Consume(Pancake cake){
this.cake = cake;
}
#Override
public void run() {
cake.eat();
}
}
Further EDIT:
For producer, instead of while(true), you can do something like:
#Override
public void run(){
while(!Thread.currentThread().isInterrupted()){
//do stuff
}
}
This way you can shutdown the executor by calling .shutdownNow(). If you'd use while(true), it won't shutdown.
Also note that the Producer is still vulnerable to RuntimeExceptions (i.e. one RuntimeException will halt the processing)
I have extended cletus proposed answer to working code example.
One ExecutorService (pes) accepts Producer tasks.
One ExecutorService (ces) accepts Consumer tasks.
Both Producer and Consumer shares BlockingQueue.
Multiple Producer tasks generates different numbers.
Any of Consumer tasks can consume number generated by Producer
Code:
import java.util.concurrent.*;
public class ProducerConsumerWithES {
public static void main(String args[]){
BlockingQueue<Integer> sharedQueue = new LinkedBlockingQueue<Integer>();
ExecutorService pes = Executors.newFixedThreadPool(2);
ExecutorService ces = Executors.newFixedThreadPool(2);
pes.submit(new Producer(sharedQueue,1));
pes.submit(new Producer(sharedQueue,2));
ces.submit(new Consumer(sharedQueue,1));
ces.submit(new Consumer(sharedQueue,2));
// shutdown should happen somewhere along with awaitTermination
/ * https://stackoverflow.com/questions/36644043/how-to-properly-shutdown-java-executorservice/36644320#36644320 */
pes.shutdown();
ces.shutdown();
}
}
class Producer implements Runnable {
private final BlockingQueue<Integer> sharedQueue;
private int threadNo;
public Producer(BlockingQueue<Integer> sharedQueue,int threadNo) {
this.threadNo = threadNo;
this.sharedQueue = sharedQueue;
}
#Override
public void run() {
for(int i=1; i<= 5; i++){
try {
int number = i+(10*threadNo);
System.out.println("Produced:" + number + ":by thread:"+ threadNo);
sharedQueue.put(number);
} catch (Exception err) {
err.printStackTrace();
}
}
}
}
class Consumer implements Runnable{
private final BlockingQueue<Integer> sharedQueue;
private int threadNo;
public Consumer (BlockingQueue<Integer> sharedQueue,int threadNo) {
this.sharedQueue = sharedQueue;
this.threadNo = threadNo;
}
#Override
public void run() {
while(true){
try {
int num = sharedQueue.take();
System.out.println("Consumed: "+ num + ":by thread:"+threadNo);
} catch (Exception err) {
err.printStackTrace();
}
}
}
}
output:
Produced:11:by thread:1
Produced:21:by thread:2
Produced:22:by thread:2
Consumed: 11:by thread:1
Produced:12:by thread:1
Consumed: 22:by thread:1
Consumed: 21:by thread:2
Produced:23:by thread:2
Consumed: 12:by thread:1
Produced:13:by thread:1
Consumed: 23:by thread:2
Produced:24:by thread:2
Consumed: 13:by thread:1
Produced:14:by thread:1
Consumed: 24:by thread:2
Produced:25:by thread:2
Consumed: 14:by thread:1
Produced:15:by thread:1
Consumed: 25:by thread:2
Consumed: 15:by thread:1
Note. If you don't need multiple Producers and Consumers, keep single Producer and Consumer. I have added multiple Producers and Consumers to showcase capabilities of BlockingQueue among multiple Producers and Consumers.
You are reinventing the wheel.
If you need persistence and other enterprise features use JMS (I'd suggest ActiveMq).
If you need fast in-memory queues use one of the impementations of java's Queue.
If you need to support java 1.4 or earlier, use Doug Lea's excellent concurrent package.
This is a very simple code.
import java.util.*;
// #author : rootTraveller, June 2017
class ProducerConsumer {
public static void main(String[] args) throws Exception {
Queue<Integer> queue = new LinkedList<>();
Integer buffer = new Integer(10); //Important buffer or queue size, change as per need.
Producer producerThread = new Producer(queue, buffer, "PRODUCER");
Consumer consumerThread = new Consumer(queue, buffer, "CONSUMER");
producerThread.start();
consumerThread.start();
}
}
class Producer extends Thread {
private Queue<Integer> queue;
private int queueSize ;
public Producer (Queue<Integer> queueIn, int queueSizeIn, String ThreadName){
super(ThreadName);
this.queue = queueIn;
this.queueSize = queueSizeIn;
}
public void run() {
while(true){
synchronized (queue) {
while(queue.size() == queueSize){
System.out.println(Thread.currentThread().getName() + " FULL : waiting...\n");
try{
queue.wait(); //Important
} catch (Exception ex) {
ex.printStackTrace();
}
}
//queue empty then produce one, add and notify
int randomInt = new Random().nextInt();
System.out.println(Thread.currentThread().getName() + " producing... : " + randomInt);
queue.add(randomInt);
queue.notifyAll(); //Important
} //synchronized ends here : NOTE
}
}
}
class Consumer extends Thread {
private Queue<Integer> queue;
private int queueSize;
public Consumer(Queue<Integer> queueIn, int queueSizeIn, String ThreadName){
super (ThreadName);
this.queue = queueIn;
this.queueSize = queueSizeIn;
}
public void run() {
while(true){
synchronized (queue) {
while(queue.isEmpty()){
System.out.println(Thread.currentThread().getName() + " Empty : waiting...\n");
try {
queue.wait(); //Important
} catch (Exception ex) {
ex.printStackTrace();
}
}
//queue not empty then consume one and notify
System.out.println(Thread.currentThread().getName() + " consuming... : " + queue.remove());
queue.notifyAll();
} //synchronized ends here : NOTE
}
}
}
Java code "BlockingQueue" which has synchronized put and get method.
Java code "Producer" , producer thread to produce data.
Java code "Consumer" , consumer thread to consume the data produced.
Java code "ProducerConsumer_Main", main function to start the producer and consumer thread.
BlockingQueue.java
public class BlockingQueue
{
int item;
boolean available = false;
public synchronized void put(int value)
{
while (available == true)
{
try
{
wait();
} catch (InterruptedException e) {
}
}
item = value;
available = true;
notifyAll();
}
public synchronized int get()
{
while(available == false)
{
try
{
wait();
}
catch(InterruptedException e){
}
}
available = false;
notifyAll();
return item;
}
}
Consumer.java
package com.sukanya.producer_Consumer;
public class Consumer extends Thread
{
blockingQueue queue;
private int number;
Consumer(BlockingQueue queue,int number)
{
this.queue = queue;
this.number = number;
}
public void run()
{
int value = 0;
for (int i = 0; i < 10; i++)
{
value = queue.get();
System.out.println("Consumer #" + this.number+ " got: " + value);
}
}
}
ProducerConsumer_Main.java
package com.sukanya.producer_Consumer;
public class ProducerConsumer_Main
{
public static void main(String args[])
{
BlockingQueue queue = new BlockingQueue();
Producer producer1 = new Producer(queue,1);
Consumer consumer1 = new Consumer(queue,1);
producer1.start();
consumer1.start();
}
}
Use this typesafe pattern with poison pills:
public sealed interface BaseMessage {
final class ValidMessage<T> implements BaseMessage {
#Nonnull
private final T value;
public ValidMessage(#Nonnull T value) {
this.value = value;
}
#Nonnull
public T getValue() {
return value;
}
#Override
public boolean equals(Object o) {
if (this == o) return true;
if (o == null || getClass() != o.getClass()) return false;
ValidMessage<?> that = (ValidMessage<?>) o;
return value.equals(that.value);
}
#Override
public int hashCode() {
return Objects.hash(value);
}
#Override
public String toString() {
return "ValidMessage{value=%s}".formatted(value);
}
}
final class PoisonedMessage implements BaseMessage {
public static final PoisonedMessage INSTANCE = new PoisonedMessage();
private PoisonedMessage() {
}
#Override
public String toString() {
return "PoisonedMessage{}";
}
}
}
public class Producer implements Callable<Void> {
#Nonnull
private final BlockingQueue<BaseMessage> messages;
Producer(#Nonnull BlockingQueue<BaseMessage> messages) {
this.messages = messages;
}
#Override
public Void call() throws Exception {
messages.put(new BaseMessage.ValidMessage<>(1));
messages.put(new BaseMessage.ValidMessage<>(2));
messages.put(new BaseMessage.ValidMessage<>(3));
messages.put(BaseMessage.PoisonedMessage.INSTANCE);
return null;
}
}
public class Consumer implements Callable<Void> {
#Nonnull
private final BlockingQueue<BaseMessage> messages;
private final int maxPoisons;
public Consumer(#Nonnull BlockingQueue<BaseMessage> messages, int maxPoisons) {
this.messages = messages;
this.maxPoisons = maxPoisons;
}
#Override
public Void call() throws Exception {
int poisonsReceived = 0;
while (poisonsReceived < maxPoisons && !Thread.currentThread().isInterrupted()) {
BaseMessage message = messages.take();
if (message instanceof BaseMessage.ValidMessage<?> vm) {
Integer value = (Integer) vm.getValue();
System.out.println(value);
} else if (message instanceof BaseMessage.PoisonedMessage) {
++poisonsReceived;
} else {
throw new IllegalArgumentException("Invalid BaseMessage type: " + message);
}
}
return null;
}
}
public class QueueHandler
{
//winstead of Queue<Object> will replace BlockingQueue <String> queue = new LinkedBlockingQueue <> ();
public static Queue<Object> readQ = new Queue<Object>(100);
public static void enqueue(Object object)
{
readQ.add(object);
}
public static Object dequeue()
{
return readQ.get();
}
}
When
public static BlockingQueue <String> queue = new LinkedBlockingQueue <> ();
it is static it works, but when it is non-static it doesn't work properly.
How to fix it?

Categories

Resources