I'm trying to implement a simple blocking queue in Java ME. In JavaME API, the concurrency utilities of Java SE are not available, so I have to use wait-notify like in the old times.
This is my provisional implementation. I'm using notify instead of notifyAll because in my project there are multiple producers but only a single consumer. I used an object for wait-notify on purpose to improve readability, despite it wastes a reference:
import java.util.Vector;
public class BlockingQueue {
private Vector queue = new Vector();
private Object queueLock = new Object();
public void put(Object o){
synchronized(queueLock){
queue.addElement(o);
queueLock.notify();
}
}
public Object take(){
Object ret = null;
synchronized (queueLock) {
while (queue.isEmpty()){
try {
queueLock.wait();
} catch (InterruptedException e) {}
}
ret = queue.elementAt(0);
queue.removeElementAt(0);
}
return ret;
}
}
My main question is about the put method. Could I put the queue.addElement line out of the synchronized block? Will performance improve if so?
Also, the same applies to take: could I take the two operations on queue out of the synchronized block?
Any other possible optimization?
EDIT:
As #Raam correctly pointed out, the consumer thread can starve when being awakened in wait. So what are the alternatives to prevent this? (Note: In JavaME I don't have all these nice classes from Java SE. Think of it as the old Java v1.2)
The Vector class makes no guarantees to be thread safe, and you should synchronize access to it, like you have done. Unless you have evidence that your current solution has performance problems, I wouldn't worry about it.
On a side note, I see no harm in using notifyAll rather than notify to support multiple consumers.
synchronized is used to protect access to shared state and ensure atomicity.
Note that methods of Vector are already synchronized, therefore Vector protects it own shared state itself. So, your synchronization blocks are only needed to ensure atomicity of your operations.
You certainly cannot move operations on queue from the synchronized block in your take() method, because atomicity is crucial for correctness of that method. But, as far as I understand, you can move queue operation from the synchronized block in the put() method (I cannot imagine a situation when it can go wrong).
However, the reasoning above is purely theoretical, because in all cases you have double synchronization: your synchronize on queueLock and methods of Vector implicitly synchronize on queue. Therefore proposed optimization doesn't make sense, its correctness depends on presence of that double synchronization.
To avoid double synchronization you need to synchronize on queue as well:
synchronized (queue) { ... }
Another option would be to use non-synchronized collection (such as ArrayList) instead of Vector, but JavaME doesn't support it. In this case you won't be able to use proposed optimization as well because synchronized blocks also protect shared state of the non-synchronized collection.
Unless you have performance issues specifically due to garbage collection, I would rather use a linked list than a Vector to implement a queue (first in,first out).
I would also write code that would be reused when your project (or another) gets multiple consumers. Although in that case, you need to be aware that the Java language specifications do not impose a way to implement monitors. In practice, that means that you don't control which consumer thread gets notified (half of the existing Java Virtual Machines implement monitors using a FIFO model and the other half implement monitors using a LIFO model)
I also think that whoever is using the blocking class is also supposed to deal with the InterruptedException. After all, the client code would have to deal with a null Object return otherwise.
So, something like this:
/*package*/ class LinkedObject {
private Object iCurrentObject = null;
private LinkedObject iNextLinkedObject = null;
LinkedObject(Object aNewObject, LinkedObject aNextLinkedObject) {
iCurrentObject = aNewObject;
iNextLinkedObject = aNextLinkedObject;
}
Object getCurrentObject() {
return iCurrentObject;
}
LinkedObject getNextLinkedObject() {
return iNextLinkedObject;
}
}
public class BlockingQueue {
private LinkedObject iLinkedListContainer = null;
private Object iQueueLock = new Object();
private int iBlockedThreadCount = 0;
public void appendObject(Object aNewObject) {
synchronized(iQueueLock) {
iLinkedListContainer = new iLinkedListContainer(aNewObject, iLinkedListContainer);
if(iBlockedThreadCount > 0) {
iQueueLock.notify();//one at a time because we only appended one object
}
} //synchonized(iQueueLock)
}
public Object getFirstObject() throws InterruptedException {
Object result = null;
synchronized(iQueueLock) {
if(null == iLinkedListContainer) {
++iBlockedThreadCount;
try {
iQueueLock.wait();
--iBlockedThreadCount; // instead of having a "finally" statement
} catch (InterruptedException iex) {
--iBlockedThreadCount;
throw iex;
}
}
result = iLinkedListcontainer.getCurrentObject();
iLinkedListContainer = iLinkedListContainer.getNextLinkedObject();
if((iBlockedThreadCount > 0) && (null != iLinkedListContainer )) {
iQueueLock.notify();
}
}//synchronized(iQueueLock)
return result;
}
}
I think that if you try to put less code in the synchronized blocks, the class will not be correct anymore.
There seem to be some issues with this approach. You can have scenarios where the consumer can miss notifications and wait on the queue even when there are elements in the queue.
Consider the following sequence in chronological order
T1 - Consumer acquires the queueLock and then calls wait. Wait will release the lock and cause the thread to wait for a notification
T2 - One producer acquires the queueLock and adds an element to the queue and calls notify
T3 - The Consumer thread is notified and attempts to acquire queueLock BUT fails as another producer comes at the same time. (from the notify java doc - The awakened thread will compete in the usual manner with any other threads that might be actively competing to synchronize on this object; for example, the awakened thread enjoys no reliable privilege or disadvantage in being the next thread to lock this object.)
T4 - The second producer now adds another element and calls notify. This notify is lost as the consumer is waiting on queueLock.
So theoretically its possible for the consumer to starve (forever stuck trying to get the queueLock) also you can run into a memory issue with multiple producers adding elements to the queue which are not being read and removed from the queue.
Some changes that I would suggest is as follows -
Keep an upper bound to the number of items that can be added to the queue.
Ensure that the consumer always read all the elements. Here is a program which shows how the producer - consumer problem can be coded.
Related
The common implementation is here, Java's built-in implementation is here. I have two questions regarding these two implementations:
1) The first implementation use synchronized key word on put() and take() methods, which means only one thread can access one method. Let's say if thread A call put() and found the queue is full, so it's waiting, then no one can ever call take() method since the lock is not released yet, how can the implementation be used?
2) Java's built-in uses two locks: takeLock and putLock, and used in put() and take() respectively. I saw that the interval queue is a linked list, which is not thread-safe, how can that be done?
As already mentioned in some of the comments the first implementation just uses traditional wait()/notify() mechanism where one thread waits (and of course releasing lock) for being notified by other threads.
The second one uses different locks each for put and take operations. So the individual operations (simultaneous put() or take()) are synchronous. But they need to communicate with each other when queue is full or empty. So they interact with each other through condition. Checkout the two private methods-
/**
* Signals a waiting take. Called only from put/offer (which do not
* otherwise ordinarily lock takeLock.)
*/
private void signalNotEmpty() {
final ReentrantLock takeLock = this.takeLock;
takeLock.lock();
try {
notEmpty.signal();
} finally {
takeLock.unlock();
}
}
/**
* Signals a waiting put. Called only from take/poll.
*/
private void signalNotFull() {
final ReentrantLock putLock = this.putLock;
putLock.lock();
try {
notFull.signal();
} finally {
putLock.unlock();
}
}
put method signals other threads trying to take/poll from empty queue and take method signals other threads trying to put elements into full queue.
I have two methods, foo() and bar(). There will be multiple threads calling these methods, possibly at the same time. It is potentially troublesome if foo() and bar() are run concurrently, as the interleaving of their internal logic can leave the system in an an inconsistent state. However, it it is perfectly ok, and in fact desirable, for multiple threads to be able to call foo() at the same time, and for multiple threads to be able to call bar() at the same time. The final condition is that foo() is expected to return asap, whereas there is no hard time constraint on bar().
I have been considering various ways in which it might be best to control this behaviour. Using synchronized in its simplest form doesn't work because this will block concurrent calls to each method. At first I thought ReadWriteLock might be a good fit, but this would only allow one of the methods to be called concurrently with itself. Another possibility I considered was queuing up requests for these methods on two separate queues and having a consumer which will concurrently execute every foo() in the queue, and then every bar() in the queue, but this seems like it would be difficult to tune so as to avoid unnecessary blocking of foo().
Any suggestions?
I think a good solution would be to make a separate class that controlled access to each of the methods. You would create a singleton of this class, and then use it to control when it is OK to proceed with entering either method.
This is the third iteration. This one prevents starvation.
Usage could be external to the foo() call:
em.enterFoo(Thread.currentThread());
foo();
em.exitFoo();
but would probably be cleaner as calls at the entry and exit of foo() instead, if possible.
Code:
public static class EntryManager
{
private int inFoo = 0;
private int inBar = 0;
private Queue<Thread> queue = new LinkedList<>();
public synchronized void enterBar(Thread t) throws InterruptedException
{
// Place the Thread on the queue
queue.add(t);
while(queue.peek() != t)
{
// Wait until the passed Thread is at the head of the queue.
this.wait();
}
while(inFoo > 0)
{
// Wait until there is no one in foo().
this.wait();
}
// There is no one in foo. So this thread can enter bar.
// Remove the thread from the queue.
queue.remove();
inBar++;
// Wakeup everyone.
this.notifyAll();
}
public synchronized void enterFoo(Thread t) throws InterruptedException
{
// Place the thread on the queue
queue.add(t);
while(queue.peek() != t)
{
// Wait until the passed Thread is at the head of the queue.
this.wait();
}
while(inBar > 0)
{
this.wait();
}
// There is no one in bar. So this thread can enter foo.
// Remove the thread from the queue.
queue.remove();
inFoo++;
// Wakeup everyone.
this.notifyAll();
}
public synchronized void exitBar()
{
inBar--;
// Wakeup everyone.
this.notifyAll();
}
public synchronized void exitFoo()
{
inFoo--;
// Wakeup everyone.
this.notifyAll();
}
}
I don't know of a name for that problem, so I would write my own synchronization helper object to deal with it. It sounds a lot like a reader/writer lock, except that where a reader/writer lock allows any number of readers at the same time, or exactly one writer, but not both; your lock would allow any number of foo() or any number of bar(), but not both.
The tricky part is going to be ensuring that the lock is fair. No problem if there's no contention, but what if the lock is in "foo" mode, and there's a steady stream of threads that want to call foo(), and just one or two that want to call bar(). How do the bar() threads ever get to run?
Actually, it reminds me a lot of a traffic light at a busy highway intersection. The traffic light can allow cars to flow on the east/west route, or on the north/south route, but not both. You don't want the light to switch too often and just let one or two cars through per cycle because that would be inefficient. But you also don't want the light to make drivers wait so long that they get angry.
I've got a feeling that the policy may have to be custom-tailored for your particular application. I.e., it may depend on how often the two functions are called, whether they are called in bursts, etc.
I would start from the source code of a reader/writer lock, and try to hack it up until it worked for me.
I am trying to find a good way to achieve the following API:
void add(Object o);
void processAndClear();
The class would store the objects and upon calling processAndClear would iterate through the currently stored ones, process them somehow, and then clear the store. This class should be thread safe.
the obvious approach is to use locking, but I wanted to be more "concurrent". This is the approach which I would use:
class Store{
private AtomicReference<CopyOnWriteArrayList<Object>> store = new AtomicReference<>(new CopyOnWriteArrayList <>());
void add(Object o){
store.get().add(o);
}
void processAndClear(){
CopyOnWriteArrayList<Object> objects = store.get();
store.compareAndSet(objects, new CopyOnWriteArrayList<>());
for (Object object : objects) {
//do sth
}
}
}
This would allow threads that try to add objects to proceed almost immediately without any locking/waiting for the xlearing to complete. Is this the more or less correct approach?
Your above code is not thread-safe. Imagine the following:
Thread A is put on hold at add() right after store.get()
Thread B is in processAndClear(), replaces the list, processes all elements of the old one, then returns.
Thread A resumes and adds a new item to the now obsolete list that will never be processed.
The probably easiest solution here would be to use a LinkedBlockingQueue, which would as well simplify the task a lot:
class Store{
final LinkedBlockingQueue<Object> queue = new LinkedBlockingQueue<>();
void add(final Object o){
queue.put(o); // blocks until there is free space in the optionally bounded queue
}
void processAndClear(){
Object element;
while ((element = queue.poll()) != null) { // does not block on empty list but returns null instead
doSomething(element);
}
}
}
Edit: How to do this with synchronized:
class Store{
final LinkedList<Object> queue = new LinkedList<>(); // has to be final for synchronized to work
void add(final Object o){
synchronized(queue) { // on the queue as this is the shared object in question
queue.add(o);
}
}
void processAndClear() {
final LinkedList<Object> elements = new LinkedList<>(); // temporary local list
synchronized(queue) { // here as well, as every access needs to be properly synchronized
elements.addAll(queue);
queue.clear();
}
for (Object e : elements) {
doSomething(e); // this is thread-safe as only this thread can access these now local elements
}
}
}
Why this is not a good idea
Although this is thread-safe, it is much slower if compared to the concurrent version. Assume that you have a system with 100 threads that frequently call add, while one thread calls processAndClear. Then the following performance bottle-necks will occur:
If one thread calls add the other 99 are put on hold in the meantime.
During the first part of processAndClear all 100 threads are put on hold.
If you assume that those 100 adding threads have nothing else to do, you can easily show, that the application runs at the same speed as a single-threaded application minus the cost for synchronization. That means: adding will effectively be slower with 100 threads than with 1. This is not the case if you use a concurrent list as in the first example.
There will however be a minor performance gain with the processing thread, as doSomething can be run on the old elements while new ones are added. But again the concurrent example could be faster, as you could have multiple threads do the processing simultaneously.
Effectively synchronized can be used as well, but you will automatically introduce performance bottle-necks, potentially causing the application to run slower as single-threaded, forcing you to do complicated performance tests. In addition extending the functionality always contains a risk of introducing threading issues, as locking needs to be done manually.A concurrent list in contrast solves all these problems without additional code and the code can easily changed or extended later on.
The class would store the objects and upon calling processAndClear would iterate through the currently stored ones, process them somehow, and then clear the store.
This seems like you should use a BlockingQueue for this task. Your add(...) method would add to the queue and your consumer would call take() which blocks waiting for the next item. The BlockingQueue (ArrayBlockingQueue is a typical implementation) takes care of all of the synchronization and signaling for you.
This means that you don't have to have a CopyOnWriteArrayList nor an AtomicReference. What you would lose is a collection and you can iterate through for other reasons than your post articulates currently.
i need to write a java program but i need some advice before starting on my own.
The program i will be writing is to do the following:
Simulate a shop takes advanced order for donuts
The shop would not take further orders, once 5000 donuts have been ordered
Ok i am kind of stuck thinking if i should be writing the java-class to act as a Monitor or should i use Java-Semaphore class instead?
Please advice me. Thanks for the help.
Any java object can work as a monitor via the wait/notify methods inherited from Object:
Object monitor = new Object();
// thread 1
synchronized(monitor) {
monitor.wait();
}
// thread 2
synchronized(monitor) {
monitor.notify();
}
Just make sure to hold the lock on the monitor object when calling these methods (don't worry about the wait, the lock is released automatically to allow other threads to acquire it). This way, you have a convenient mechanism for signalling among threads.
It seems to me like you are implementing a bounded producer-consumer queue. In this case:
The producer will keep putting items in a shared queue.
If the queue size reaches 5000, it will call wait on a shared monitor and go to sleep.
When it puts an item, it will call notify on the monitor to wake up the consumer if it's waiting.
The consumer will keep taking items from the queue.
When it takes an item, it will call notify on the monitor to wake up the producer.
If the queue size reaches 0 the consumer calls wait and goes to sleep.
For an even more simplified approach, have a loop at the various implementation of BlockingQueue, which provides the above features out of the box!
It seems to me that the core of this exercise is updating a counter (number of orders taken), in a thread-safe and atomic fashion. If implemented incorrectly, your shop could end up taking more than 5000 pre-orders due to missed updates and possibly different threads seeing stale values of the counter.
The simplest way to update a counter atomically is to use synchronized methods to get and increment it:
class DonutShop {
private int ordersTaken = 0;
public synchronized int getOrdersTaken() {
return ordersTaken;
}
public synchronized void increaseOrdersBy(int n) {
ordersTaken += n;
}
// Other methods here
}
The synchronized methods mean that only one thread can be calling either method at any time (and they also provide a memory barrier to ensure that different threads see the same value rather than locally cached ones which may be outdated). This ensures a consistent view of the counter across all threads in your application.
(Note that I didn't have a "set" method but an "increment" method. The problem with "set" is that if client has to call shop.set(shop.get() + 1);, another thread could have incremented the value between the calls to get and set, so this update would be lost. By making the whole increment operation atomic - because it's in the synchronized block - this situation cannot occur.
In practice I would probably use an AtomicInteger instead, which is basically a wrapper around an int to allow for atomic queries and updates, just like the DonutShop class above. It also has the advantage that it's more efficient in terms of minimising exclusive blocking, and it's part of the standard library so will be more immediately familiar to other developers than a class you've written yourself.
In terms of correctness, either will suffice.
Like Tudor wrote, you can use any object as monitor for general purpose locking and synchronization.
However, if you got the requirement that only x orders (x=5000 for your case) can be processed at any one time, you could use the java.util.concurrent.Semaphore class. It is made specifically for use cases where you can only have fixed number of jobs running - it is called permits in the terminology of Semaphore
If you do the processing immediately, you can go with
private Semaphore semaphore = new Semaphore(5000);
public void process(Order order)
{
if (semaphore.tryAcquire())
{
try
{
//do your processing here
}
finally
{
semaphore.release();
}
}
else
{
throw new IllegalStateException("can't take more orders");
}
}
If if takes more than that (human input required, starting another thread/process, etc.), you need to add callback for when the processing is over, like:
private Semaphore semaphore = new Semaphore(5000);
public void process(Order order)
{
if (semaphore.tryAcquire())
{
//start a new job to process order
}
else
{
throw new IllegalStateException("can't take more orders");
}
}
//call this from the job you started, once it is finished
public void processingFinished(Order order)
{
semaphore.release();
//any other post-processing for that order
}
I have a multithreaded class A which accesses the following insert() method of another class B (A has only a single instance of B).
Instead of making the entire method synchronized, are there any better ways to synchronize the following method? (to reduce the synchronization overhead)
private void insert(byte[] shardKey, byte[] queueKey,
byte[] value, PipelineMessageType msgType) {
PipelineMessage pipelineMessage = new PipelineMessage(queueKey,
value, msgType);
LinkedBlockingQueue<PipelineMessage> queue;
JedisShardInfo shardInfo = shardedJedis.getShardInfo(shardKey); // shardedJedis is an instance variable of this class
String mapKey = shardInfo.getHost() + shardInfo.getPort();
queue = shardQueue.get(mapKey); // shardQueue is an instance variable of this class
boolean insertSuccessful = queue.offer(pipelineMessage);
if(!insertSuccessful) {
// perform the pipeline sync - flush the queue
// use another thread for this
// (processing of queue entries is given to another thread here)
// queue would be empty now. Insert (k,v)
queue.offer(pipelineMessage);
}
}
I tried to synchronize only the fragment which accesses the instance variables but there might be a scenario where 2 threads try to insert into a full queue and enter the if block. Then 2 threads might process the queue entries which I don't want to happen.
Any suggestions are appreciated. Thank you in advance.
Seems to me that if JedisShardInfo would be a read-only item, then you should need to protect/synchronize it. So you could synchronize only from the line
queue= ...
Otherwise, almost everything should be synchronized, except the first statement (declaration of pipeline message), and then I really wonder if it changes much compared to declaring the whole method synchronized.
Also, if you got other points of synchronization, I mean other methods or block codes that are synchronized on this, you should consider splitting them and synchronize on different data members of this depending on which data members you wish to protect from multi-threading :
Object lockerA = new Object() {};
synchronized( lockerA )
{}//sync
Well, not much to say. :)
Regards,
Stéphane
The key to correct synchronization is to follow this pattern:
synchronize(lockObjectForState) { // All code that alters state must also synchronise on the same lock
while(!stateOkToProceed()) {
try {
lockForState.wait();
} catch (InterruptedException e) {
// handle if your thread was interrupted deliberately as a single to exit, or spuriously (in which case do nothing)
}
}
updateState();
lockForState.notifyAll();
}
java.util.concurrent package offer many thread-safe implementations of classes needed to solve common threading problems. Consider using a BlockingQueue.