Producer/Consumer: no producer/consumer should block other producer/consumer - java

I am designing a system where there will be n producers and m consumers, where n and m are numbers, and n != m.
I wanted to design the system such a way that,
no producer should block other producer when producing
no consumer should block other consumer when consuming neither
producer nor consumer block each other while producing/consuming
For eg: in java if i use the synchronized key word, then it will be blocking the respective caller.
I am not sure what data structure and algorithm i should use to implement this system.
Can some one provide me help/pointers on this?

You probably want something like the ConcurrentLinkedQueue. The idea is that you create a single queue. Each of your n producers adds work items to the queue, and each of the m consumers reads work items from the queue. The producer is simply:
while not done
create work item
add work item to queue
The consumer is just as simple:
while not done
get next work item from queue
process work item
The ConcurrentLinkedQueue methods handle adding and removing items, synchronizing with the other producers and consumers as required.
The only real drawback is that you have to poll the queue to see if there are items. So you'll probably want an auto reset event that gets tripped whenever an item is added to the queue. For example:
add work item to queue
set ItemAvailable event
And the consumer would poll the queue and if no item is available, wait on the event:
while not done
while ((item = queue.poll) == null)
wait on ItemAvailable event
process item
Take a look at the example I linked. It really isn't difficult to use.

Depending on how much heavy lifting you need to do, and how well your solution need to scale, RxJava has a bit of a steep learning curve, but once you got past that it's probably the most elegant, scaling and performing solution.
Run all your producers in different threads, combine them with Merg(), move the consumers to there own thread on a unbound buffer with .observeOn(Scheduler.newThread()).
If you need something that runs well parallel on multiple systems, look at mapreduce.
If you need something at the complete other end of the spectrum (something simple), simply stick to a ConcurrentQueue. That doesn't support multicast, but at least solves the producer side of the problem.

You want an approach where every action would be atomic and uninterruptible, so yes, in my opinion the best approach would be to use synchronized modifier on methods to set the lock.
The other interesting approach would be to use atomic variables -> http://baptiste-wicht.com/posts/2010/09/java-concurrency-atomic-variables.html
That depends on your data in these producer/consumer structures.

use wait() and notify() for thread communication , u can create n producer and m consumer threads
class Q{
int n;
boolean value=false;
synchronized int get() {
if(!value)
try { wait(); }
catch(InterruptedException e)
{ System.out.println("thread interrupted"); }
System.out.println("Got : "+n);
value=false;
notify();
return n;}
synchronized void put(int n) {
if(value)
try { wait();}
catch(InterruptedException e)
{ System.out.println("thread interrupted"); }
this.n=n;
value=true;
System.out.println("Put : "+n);
notify();}}
class Producer implements Runnable{
Q q;
Producer(Q q){
this.q=q;
new Thread(this,"Producer").start();}
public void run(){
int i=0;
while(true)
{
q.put(i++);}}
}
class Consumer implements Runnable{
Q q;
Consumer(Q q) {
this.q=q;
new Thread(this,"Consumer").start();}
public void run(){
while(true)
{
q.get();
}}}
class PCFixed
{
public static void main(String ar[])
{
Q q=new Q();
new Producer(q);
new Consumer(q);
System.out.println("PRESS CONTROL-C TO STOP");
}
}
it goes to infinity, change that based on ur requirements

Related

Java wait/notify implementation without synchronized

I have a scenario with dozens of producer and one single consumer. Timing is critical: for performance reason I want to avoid any locking of producers and I want the consumer to wait as little as possible when no messages are ready.
I've started using a ConcurrentLinkedQueue, but I don't like to call sleep on the consumer when queue.poll() == null because I could waste precious milliseconds, and I don't want to use yield because I end up wasting cpu.
So I came to implement a sort of ConcurrentBlockingQueue so that the consumer can run something like:
T item = queue.poll();
if(item == null) {
wait();
item = queue.poll();
}
return item;
And producer something like:
queue.offer(item);
notify();
Unfortunately wait/notify only works on synchronized block, which in turn would drastically reduce producer performance. Is there any other implementation of wait/notify mechanism that does not require synchronization?
I am aware of the risks related to not having wait and notify synchronized, and I managed to resolve them by having an external thread running the following:
while(true) {
notify();
sleep(100);
}
I've started using a ConcurrentLinkedQueue, but I don't like to call sleep on the consumer when queue.poll() == null
You should check the BlockingQueue interface, which has a take method that blocks until an item becomes available.
It has several implementations as detailed in the javadoc, but ConcurrentLinkedQueue is not one of them:
All Known Implementing Classes:
ArrayBlockingQueue, DelayQueue, LinkedBlockingDeque, LinkedBlockingQueue, LinkedTransferQueue, PriorityBlockingQueue, SynchronousQueue
I came out with the following implementation:
private final ConcurrentLinkedQueue<T> queue = new ConcurrentLinkedQueue<>();
private final Semaphore semaphore = new Semaphore(0);
private int size;
public void offer(T item) {
size += 1;
queue.offer(item);
semaphore.release();
}
public T poll(long timeout, TimeUnit unit) {
semaphore.drainPermits();
T item = queue.poll();
if (item == null) {
try {
semaphore.tryAcquire(timeout, unit);
} catch (InterruptedException ex) {
}
item = queue.poll();
}
if (item == null) {
size = 0;
} else {
size = Math.max(0, size - 1);
}
return item;
}
/** An inaccurate representation O(1)-access of queue size. */
public int size() {
return size;
}
With the following properties:
producers never go to SLEEP state (which I think can go with BlockingQueue implementations that use Lock in offer(), or with synchronized blocks using wait/notify)
consumer only goes to SLEEP state when queue is empty but it is soon woken up whenever a producer offer an item (no fixed-time sleep, no yield)
consumer can be sometime woken up even with empty queue, but it's ok here to waste some cpu cycle
Is there any equivalent implementation in jdk that I'm not aware of? Open for criticism.

java concurrency: multi-producer one-consumer

I have a situation where different threads populate a queue (producers) and one consumer retrieve element from this queue. My problem is that when one of these elements are retrieved from the queue some is missed (missing signal?). The producers code is:
class Producer implements Runnable {
private Consumer consumer;
Producer(Consumer consumer) { this.consumer = consumer; }
#Override
public void run() {
consumer.send("message");
}
}
and they are created and run with:
ExecutorService executor = Executors.newSingleThreadExecutor();
for (int i = 0; i < 20; i++) {
executor.execute(new Producer(consumer));
}
Consumer code is:
class Consumer implements Runnable {
private Queue<String> queue = new ConcurrentLinkedQueue<String>();
void send(String message) {
synchronized (queue) {
queue.add(message);
System.out.println("SIZE: " + queue.size());
queue.notify();
}
}
#Override
public void run() {
int counter = 0;
synchronized (queue) {
while(true) {
try {
System.out.println("SLEEP");
queue.wait(10);
} catch (InterruptedException e) {
Thread.interrupted();
}
System.out.println(counter);
if (!queue.isEmpty()) {
queue.poll();
counter++;
}
}
}
}
}
When the code is run I get sometimes 20 elements added and 20 retrieved, but in other cases the elements retrieved are less than 20. Any idea how to fix that?
I'd suggest you use a BlockingQueue instead of a Queue. A LinkedBlockingDeque might be a good candidate for you.
Your code would look like this:
void send(String message) {
synchronized (queue) {
queue.put(message);
System.out.println("SIZE: " + queue.size());
}
}
and then you'd need to just
queue.take()
on your consumer thread
The idea is that .take() will block until an item is available in the queue and then return exactly one (which is where I think your implementation suffers: missing notification while polling). .put() is responsible for doing all the notifications for you. No wait/notifies needed.
The issue in your code is probably because you are using notify instead of notifyAll. The former will only wake up a single thread, if there is one waiting on the lock. This allows a race condition where no thread is waiting and the signal is lost. A notifyAll will force correctness at a minor performance cost by requiring all threads to wake up to check whether they can obtain the lock.
This is best explained in Effective Java 1st ed (see p.150). The 2nd edition removed this tip since programmers are expected to use java.util.concurrent which provides stronger correctness guarantees.
It looks like bad idea to use ConcurrentLinkedQueue and synchronization both at the same time. It defies the purpose of concurrent data structures in the first place.
There is no problem with ConcurrentLinkedQueue data structure and replacing it with BlockingQueue will solve the problem but this is not the root cause.
Problem is with queue.wait(10). This is timed wait method. It will acquire lock again once 10ms elapses.
Notification (queue.notify() ) will get lost because there is no consumer thread waiting on it if 10ms has elapsed.
Producer will not be able to add to the queue since they can't acquire lock because lock is claimed again by the consumer.
Moving to BlockingQueue solved your problem because you removed your wait(10) code and wait and notify was taken care by BlockingQueue data structure.

Waiting for an update

What is the best way to wait for an event/update. For example, I am waiting for this data structure to be updated before doing something. Implementing it inside a loop is not the best way since it consumes much CPU time like:
while (true) {
// keep checking the data structure
// if data structure is updated
// break;
}
// do something here
What's a simple but efficient way to implement something like this in Java?
wait-notifyAll is more efficient way than loop.
Standard idiom for wait():
synchronized (obj) {
while(condition not hold)
obj.wait();
}
But it's primitive way to control threads, you'd better use classes in java.util.concurrent package. Moreover, I will choose Chris Dail's answer if I meet such problem.
It really depends on the structure of the rest of your program. I would probably start by looking through java.util.concurrent to see if something in there suits you.
Examples of ways you could do this:
Futures - If you have some 'work' to be done, you can have a thread pool executor service to perform the work. When you call submit() to do your work, you get a future that you can check or block until the work is completed.
Queues - If you have one component doing the work and one component doing the waiting, you could have their communication done with queues. Any time one is done with working on the data, it can add to a queue. You could use the LinkedBlockingQueue and poll() for the work to be completed.
Listeners - Without concurrent at all, you could use the Listener/Observer pattern.
There are lots of different options depending on your application structure.
This is a code sample i would do.
In this logic I use join method in threads. This makes sure all the threads are joined before the execution of the main thread continues. I have put TODO for locations u need to add your code
import java.util.ArrayList;
import java.util.List;
public class MultiThread extends Thread{
public void run() {
System.out.println("Starting Thread - " + this.getName()+this.getThreadGroup());
//TODO data structure is updated here
}
public static void main(String[] args) {
List dataStructureList = new ArrayList() ;//TODO need to replace with list of data structure
//TODO dataStructureList contain list of items
Thread[] threadArr = new Thread[dataStructureList.size()];
for (int j = 0; j < threadArr.length; j++) {
threadArr[j] = new MultiThread();
threadArr[j].setName("Thread " + j);
threadArr[j].start();
}
try {
for (int j = 0; j < threadArr.length; j++) {
threadArr[j].join();
}
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
System.out.println("All thread finished");
//TODO do something here
}
}
Assuming that you use multi-threading in an application. To use one object with several threads you should use synchronization. While one thread initializes data structure, other wait for finishing of initialization. This logic is usually implemented using wait/notify methods which can be called on any object.
Working thread(s):
while (initialized) {
synchronized (object) {
object.wait();
}
}
Initialization thread:
synchronized (object) {
// initialization
initialized = true;
object.notifyAll();
}
object is the data structure which should be initialized. The initialized flag used to indicate that the initialization has completed. It is better to use this flag because sometimes wait can be finished without corresponded notify.

How to correctly use synchronized?

This piece of code:
synchronized (mList) {
if (mList.size() != 0) {
int s = mList.size() - 1;
for (int i = s; i > 0; i -= OFFSET) {
mList.get(i).doDraw(canv);
}
getHead().drawHead(canv);
}
}
Randomly throws AIOOBEs. From what I've read, the synchronized should prevent that, so what am I doing wrong?
Edits:
AIOOBE = Array Index Out Of Bounds Exception
The code's incomplete, cut down to what is needed. But to make you happy, OFFSET is 4, and just imagine that there is a for-loop adding a bit of data at the beginning. And a second thread reading and / or modifying the list.
Edit 2:
I've noticed it happens when the list is being drawn and the current game ends. The draw-thread hasn't drawn all elements when the list is emptied. Is there a way of telling the game to wait with emtying the list untill it's empty?
Edit 3:
I've just noticed that I'm not sure if this is a multi-threading problem. Seems I only have 2 threads, one for calculating and drawing and one for user input.. Gonna have to look into this a bit more than I thought.
What you're doing looks right... but that's all:
It doesn't matter on what object you synchronize, it needn't be the list itself.
What does matter is if all threads always synchronize on the same object, when accessing a shared resource.
Any access to SWING (or another graphic library) must happen in the AWT-Thread.
To your edit:
I've noticed it happens when the list is being drawn and the current game ends. The draw-thread hasn't drawn all elements when the list is emptied. Is there a way of telling the game to wait with emtying the list untill it's empty?
I think you mean "...wait with emptying the list until the drawing has completed." Just synchronize the code doing it on the same lock (i.e., the list itself in your case).
Again: Any access to a shared resource must be protected somehow. It seems like you're using synchronized just here and not where you're emptying the list.
The safe solution is to only allow one thread to create objects, add and remove them from a List after the game has started.
I had problems myself with random AIOOBEs erros and no synchornize could solve it properly plus it was slowing down the response of the user.
My solution, which is now stable and fast (never had an AIOOBEs since) is to make UI thread inform the game thread to create or manipulate an object by setting a flag and coordinates of the touch into the persistent variables.
Since the game thread loops about 60 times per second this proved to be sufficent to pick up the message from the UI thread and do something.
This is a very simple solution and it works great!
My suggestion is to use a BlockingQueue and I think you are looking for this solution also. How you can do it? It is already shown with an example in the javadoc :)
class Producer implements Runnable {
private final BlockingQueue queue;
Producer(BlockingQueue q) { queue = q; }
public void run() {
try {
while (true) { queue.put(produce()); }
} catch (InterruptedException ex) { ... handle ...}
}
Object produce() { ... }
}
class Consumer implements Runnable {
private final BlockingQueue queue;
Consumer(BlockingQueue q) { queue = q; }
public void run() {
try {
while (true) { consume(queue.take()); }
} catch (InterruptedException ex) { ... handle ...}
}
void consume(Object x) { ... }
}
class Setup {
void main() {
BlockingQueue q = new SomeQueueImplementation();
Producer p = new Producer(q);
Consumer c1 = new Consumer(q);
Consumer c2 = new Consumer(q);
new Thread(p).start();
new Thread(c1).start();
new Thread(c2).start();
}
}
The beneficial things for you are, you need not to worry about synchronizing your mList. BlockingQueue offers 10 special method. You can check it in the doc. Few from javadoc:
BlockingQueue methods come in four forms, with different ways of handling operations that cannot be satisfied immediately, but may be satisfied at some point in the future: one throws an exception, the second returns a special value (either null or false, depending on the operation), the third blocks the current thread indefinitely until the operation can succeed, and the fourth blocks for only a given maximum time limit before giving up.
To be in safe side: I am not experienced with android. So not certain whether all java packages are allowed in android. But at least it should be :-S, I wish.
You are getting Index out of Bounds Exception because there are 2 threads that operate on the list and are doing it wrongly.
You should have been synchronizing at another level, in such a way that no other thread can iterate through the list while other thread is modifying it! Only on thread at a time should 'work on' the list.
I guess you have the following situation:
//piece of code that adds some item in the list
synchronized(mList){
mList.add(1, drawableElem);
...
}
and
//code that iterates you list(your code simplified)
synchronized (mList) {
if (mList.size() != 0) {
int s = mList.size() - 1;
for (int i = s; i > 0; i -= OFFSET) {
mList.get(i).doDraw(canv);
}
getHead().drawHead(canv);
}
}
Individually the pieces of code look fine. They seam thread-safe. But 2 individual thread-safe pieces of code might not be thread safe at a higher level!
It's just you would have done the following:
Vector v = new Vector();
if(v.length() == 0){ v.length() itself is thread safe!
v.add("elem"); v.add() itself is also thread safe individually!
}
BUT the compound operation is NOT!
Regards,
Tiberiu

producer/consumer work queues

I'm wrestling with the best way to implement my processing pipeline.
My producers feed work to a BlockingQueue. On the consumer side, I poll the queue, wrap what I get in a Runnable task, and submit it to an ExecutorService.
while (!isStopping())
{
String work = workQueue.poll(1000L, TimeUnit.MILLISECONDS);
if (work == null)
{
break;
}
executorService.execute(new Worker(work)); // needs to block if no threads!
}
This is not ideal; the ExecutorService has its own queue, of course, so what's really happening is that I'm always fully draining my work queue and filling the task queue, which slowly empties as the tasks complete.
I realize that I could queue tasks at the producer end, but I'd really rather not do that - I like the indirection/isolation of my work queue being dumb strings; it really isn't any business of the producer what's going to happen to them. Forcing the producer to queue a Runnable or Callable breaks an abstraction, IMHO.
But I do want the shared work queue to represent the current processing state. I want to be able to block the producers if the consumers aren't keeping up.
I'd love to use Executors, but I feel like I'm fighting their design. Can I partially drink the Kool-ade, or do I have to gulp it? Am I being wrong-headed in resisting queueing tasks? (I suspect I could set up ThreadPoolExecutor to use a 1-task queue and override it's execute method to block rather than reject-on-queue-full, but that feels gross.)
Suggestions?
I want the shared work queue to
represent the current processing
state.
Try using a shared BlockingQueue and have a pool of Worker threads taking work items off of the Queue.
I want to be able to block the
producers if the consumers aren't
keeping up.
Both ArrayBlockingQueue and LinkedBlockingQueue support bounded queues such that they will block on put when full. Using the blocking put() methods ensures that producers are blocked if the queue is full.
Here is a rough start. You can tune the number of workers and queue size:
public class WorkerTest<T> {
private final BlockingQueue<T> workQueue;
private final ExecutorService service;
public WorkerTest(int numWorkers, int workQueueSize) {
workQueue = new LinkedBlockingQueue<T>(workQueueSize);
service = Executors.newFixedThreadPool(numWorkers);
for (int i=0; i < numWorkers; i++) {
service.submit(new Worker<T>(workQueue));
}
}
public void produce(T item) {
try {
workQueue.put(item);
} catch (InterruptedException ex) {
Thread.currentThread().interrupt();
}
}
private static class Worker<T> implements Runnable {
private final BlockingQueue<T> workQueue;
public Worker(BlockingQueue<T> workQueue) {
this.workQueue = workQueue;
}
#Override
public void run() {
while (!Thread.currentThread().isInterrupted()) {
try {
T item = workQueue.take();
// Process item
} catch (InterruptedException ex) {
Thread.currentThread().interrupt();
break;
}
}
}
}
}
"find an available existing worker thread if one exists, create one if necessary, kill them if they go idle."
Managing all those worker states is as unnecessary as it is perilous. I would create one monitor thread that constantly runs in the background, who's only task is to fill up the queue and spawn consumers... why not make the worker threads daemons so they die as soon as they complete? If you attach them all to one ThreadGroup you can dynamically re-size the pool... for example:
**for(int i=0; i<queue.size()&&ThreadGroup.activeCount()<UPPER_LIMIT;i++ {
spawnDaemonWorkers(queue.poll());
}**
You could have your consumer execute Runnable::run directly instead of starting a new thread up. Combine this with a blocking queue with a maximum size and I think that you will get what you want. Your consumer becomes a worker that is executing tasks inline based on the work items on the queue. They will only dequeue items as fast as they process them so your producer when your consumers stop consuming.

Categories

Resources