Develop Reference

How efficient are BlockingQueues / what's their effect on CPU time?

I am making an online game in Java and I ran into one particular issue where I was trying to find the most efficient way to send clients spawn entity NPC packets. I of course understand how to send them but I wanted to do it off of the main game loop since it requires looping through a map of NPC's (I also made sure its thread safe). To do this I thought a BlockingQueue was my best option so I created a new thread set it to daemon then passed in a runnable object. Then whenever I needed to send one of these packets I would use the insertElement() method to add to the queue. Here is how it looks.
public class NpcAsyncRunnable implements Runnable {
private final BlockingQueue<NpcObject> blockingQueue;
public NpcAsyncRunnable() {
blockingQueue = new LinkedBlockingQueue<>();
}
#Override
public void run() {
while(true) {
try {
final NpcObject obj = blockingQueue.take();
//Run my algorithm here
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
public void insertElement(final NpcObject obj) {
blockingQueue.add(obj);
}
}
Now my question is how efficient is this? I am running the thread the whole time in an infinite loop because I always want it to be checking for another inserted element. However, my concern is if I have too many async threads listening would it start to clog up the CPU? I ask this because I know a CPU core can only run 1 thread of execution at a time but with hyperthreading (AMD has the same thing but its called something different) it can jump between executing multiple threads when one needs to search for something in memory. But does this infinite loop without making it sleep mean it will always be checking if the queue has a new entry? My worry is I will make a CPU core waste all its resources infinitely looping over this one thread waiting for another insertion.
Does the CPU instead auto assign small breaks to allow other threads to execute or do I need to include sleep statements so that this thread is not using way more resources than is required? How much CPU time will this use just idling?

...does this infinite loop without making it sleep mean...?
blockingQueue.take() does sleep until there's something in the queue to be taken. The Javadoc for the take method says, "Retrieves and removes the head of this queue, waiting if necessary until an element becomes available."
"Waiting" means it sleeps. Any time you are forced to write catch (InterruptedException...), it's because you called something that sleeps.
how does it know when something is added if its sleeping? It has to be running in order to check if something has been added to the queue right?
No. It doesn't need to run. It doesn't need to "check." A BlockingQueue effectively* uses object.wait() to make a thread "sleep," and it uses object.notify() to wake it up again. When one thread in a Java program calls o.wait() for any Object o, the wait() call will not return** until some other thread calls o.notify() for the same Object o.
wait() and notify() are thin wrappers for operating system-specific calls that do approximately the same thing. All the magic happens in the OS. In a nutshell;
The OS suspends the thread that calls o.wait(), and it adds the thread's saved execution context to a queue associated with the object o.
When some other thread calls o.notify(), the OS takes the saved execution context at the head of the queue (if there is one***), and moves it to the "ready-to-run" queue.
Some time later, the OS scheduler will find the saved thread context at the head of the "ready-to-run" queue, and it will restore the context on one of the system's CPUs.
At that point, the o.wait() call will return, and the thread that waited can then proceed to deal with whatever it was waiting for (e.g., an NpcAsyncRunnable object in your case.)
* I don't know whether any particular class that implements BlockingQueue actually uses object.wait() and object.notify(), but even if they don't use those methods, then they almost certainly use the same operating system calls that underlie wait() and notify().
** Almost true, but there's something called "spurious wakeup." Correctly using o.wait() and o.notify() is tricky. I strongly recommend that you work through the tutorial if you want to try it yourself.
*** o.notify() does absolutely nothing at all if no other thread is already waiting at the moment when it is called. Beginners who don't understand this often ask, "Why did wait() never return?" It didn't return because the thread that wait()ed was too late. Again, I urge you to work through the tutorial if you want to learn how to avoid that particular bug.

Do I need to synchronize my calls to BlockingQueue (java)?

I want to store a list of objects in a thread safe manner, while maintaining priority. Originally I started out using a BlockingQueue for this as it's thread safe and has ability to maintain custom priority.
I'm wondering if I need to synchronize my methods? My code looks like:
void addToQueue(SomeObject obj) {
... put it on my priority queue
... do some logging
}
What I have noticed is the logging is happening out of order, when accessing addToQueue from multiple threads. So I wrapped my method like so:
void addToQueue(SomeObject obj) {
syncronized(myMutex) {
... put it on my priority queue
... do some logging
}
}
This seemed to keep the logging in order. So now I've come to the conclusion that if I'm going this route, then maybe my code would be more efficient by not using a BlockingQueue but instead use a Set or List and manage the priority myself.
Maybe I have some misunderstanding of the BlockingQueue.

A Queue that additionally supports operations that wait for the queue to become non-empty when retrieving an element, and wait for space to become available in the queue when storing an element.
This is the javadoc for BlockingQueue. You use it if you need this blocking behavior, otherwise you don't.
BlockingQueue does not maintain any priority, it is strictly first-in-first-out. Perhaps you are using PriorityBlockingQueue?
Coming to your pseudocode:
void addToQueue(SomeObject obj) {
... put it on my priority queue
... do some logging
}
The queue is thread-safe, but that only means that multiple threads can concurrently call put it on my priority queue without any data corruption. It does not guarantee any of the following:
If there are multiple threads blocked which one will succeed first
If a thread X completes the put before a thread Y then thread X will also complete the logging before thread Y.
If you need all of addToQueue occur without interleaving from other threads then you need to synchronize. Note that you can use the queue object itself:
void addToQueue(SomeObject obj) {
synchronized (queue) {
... put it on my priority queue
... do some logging
}
}

http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/BlockingQueue.html
As with other concurrent collections, actions in a thread prior to placing an object into a BlockingQueue happen-before actions subsequent to the access or removal of that element from the BlockingQueue in another thread.
If you want to use that safety to get ordered logging you have to log before putting items into the queue and after taken an item from the queue.
I wouldn’t use synchronized for getting ordered logging. Multi-threading means parallel execution and that implies that certain actions don’t have an ordering. Log Records can have a time-stamp and seeing them in the wrong order, i.e. in the console, looks like a minor glitch to me that is not worth sacrificing the advantages of parallel execution for.

Java BlockingQueue with batching?

I am interested in a data structure identical to the Java BlockingQueue, with the exception that it must be able to batch objects in the queue. In other words, I would like the producer to be able to put objects into the queue, but have the consumer block on take() untill the queue reaches a certain size (the batch size).
Then, once the queue has reached the batch size, the producer must block on put() untill the consumer has consumed all of the elements in the queue (in which case the producer will start producing again and the consumer block untill the batch is reached again).
Does a similar data structure exist? Or should I write it (which I don't mind), I just don't want to waste my time if there is something out there.
UPDATE
Maybe to clarify things a bit:
The situation will always be as follows. There can be multiple producers adding items to the queue, but there will never be more than one consumer taking items from the queue.
Now, the problem is that there are multiple of these setups in parallel and serial. In other words, producers produce items for multiple queues, while consumers in their own right can also be producers. This can be more easily thought of as a directed graph of producers, consumer-producers, and finally consumers.
The reason that producers should block until the queues are empty (#Peter Lawrey) is because each of these will be running in a thread. If you leave them to simply produce as space becomes available, you will end up with a situation where you have too many threads trying to process too many things at once.
Maybe coupling this with an execution service could solve the problem?

I would suggest you use BlockingQueue.drainTo(Collection, int). You can use it with take() to ensure you get a minimum number of elements.
The advantage of using this approach is that your batch size grows dynamically with the workload and the producer doesn't have to block when the consumer is busy. i.e. it self optimises for latency and throughput.
To implement exactly as asked (which I think is a bad idea) you can use a SynchronousQueue with a busy consuming thread.
i.e. the consuming thread does a
list.clear();
while(list.size() < required) list.add(queue.take());
// process list.
The producer will block when ever the consumer is busy.

Here is a quick ( = simple but not fully tested) implementation that i think may be suitable for your requests - you should be able to extend it to support the full queue interface if you need to.
to increase performance you can switch to ReentrantLock instead of using "synchronized" keyword..
public class BatchBlockingQueue<T> {
private ArrayList<T> queue;
private Semaphore readerLock;
private Semaphore writerLock;
private int batchSize;
public BatchBlockingQueue(int batchSize) {
this.queue = new ArrayList<>(batchSize);
this.readerLock = new Semaphore(0);
this.writerLock = new Semaphore(batchSize);
this.batchSize = batchSize;
}
public synchronized void put(T e) throws InterruptedException {
writerLock.acquire();
queue.add(e);
if (queue.size() == batchSize) {
readerLock.release(batchSize);
}
}
public synchronized T poll() throws InterruptedException {
readerLock.acquire();
T ret = queue.remove(0);
if (queue.isEmpty()) {
writerLock.release(batchSize);
}
return ret;
}
}
Hope you find it useful.

I recently developed this utility that batch BlockingQueue elements using a flushing timeout if queue elements doesn't reach the batch size. It also supports fanOut pattern using multiple instances to elaborate the same set of data:
// Instantiate the registry
FQueueRegistry registry = new FQueueRegistry();
// Build FQueue consumer
registry.buildFQueue(String.class)
.batch()
.withChunkSize(5)
.withFlushTimeout(1)
.withFlushTimeUnit(TimeUnit.SECONDS)
.done()
.consume(() -> (broadcaster, elms) -> System.out.println("elms batched are: "+elms.size()));
// Push data into queue
for(int i = 0; i < 10; i++){
registry.sendBroadcast("Sample"+i);
}
More info here!
https://github.com/fulmicotone/io.fulmicotone.fqueue

Not that I am aware of. If I understand correctly you want either the producer to work (while the consumer is blocked) until it fills the queue or the consumer to work (while the producer blocks) until it clears up the queue. If that's the case may I suggest that you don't need a data structure but a mechanism to block the one party while the other one is working in a mutex fasion. You can lock on an object for that and internally have the logic of whether full or empty to release the lock and pass it to the other party. So in short, you should write it yourself :)

This sounds like how the RingBuffer works in the LMAX Disruptor pattern. See http://code.google.com/p/disruptor/ for more.
A very rough explanation is your main data structure is the RingBuffer. Producers put data in to the ring buffer in sequence and consumers can pull off as much data as the producer has put in to the buffer (so essentially batching). If the buffer is full, the producer blocks until the consumer has finished and freed up slots in the buffer.

Implement Producer-Consumer Pattern Using Executor Framework

I have two threads, a consumer and a producer. The consumer thread is the main thread while the producer thread is created by a third party library I use.
You make a request for a List of data to the producer thread using ProducerChannel's requestData(), which returns immediately. Then, the producer thread will generate data one by one asynchronously and uses a call back method to send each of them. I want the method that requests data to return the result synchronously. The most straightforward way would be to use wait() and notify() like below.
public class DataFeed {
boolean done;
private List<Data> dataList;
private ProducerChannel producerChannel;
// This method should be synchronous.
public List<Data> getDataList() {
this.producerChannel.requestData();
while (!done) {
wait();
}
List<Data> dataList = this.dataList;
this.dataList = null;
return dataList;
}
// This is the call back method invoked by the producer thread.
public void generated(Data data) {
if (data == null) {
done = true; // End of data.
notify();
}
else {
this.dataList.add(data);
}
}
}
Note that there's only one consumer thread in the entire application. That's why DataFeed has only one List to hold the result for each request. I learned that the Executor framework is now the preferred way to manage threads. How can I refactor this class so that it does not use Thread objects explicitly while not creating additional threads?

I think you should take a look at the producer-consumer example on the BlockingQueue javadoc:
http://download.oracle.com/javase/6/docs/api/java/util/concurrent/BlockingQueue.html

I don't really follow you. This class doesn't create any thread. So using the executor framework won't hep you here.
Instead os using explicit synchronization and thread communication using wait and notify, you could just use a BlockingQueue. The producer adds the list to the blocking queue when data is ready, and the consuler blocks while the blocking queue is empty.

If you don't care about the size of the resulting collection or getting items by index, then BlockingQueue might be what you need. If you DO care about those things (i.e. you actually need a java.util.List, instead of Collection, or if you're not ok using a delayed collection for some reason) then you can still use wait() and notify().
You'll need to add they synchronized keyword to your getDataList and generated methods, otherwise you'll get an IllegalStateMonitorException.
Like this: public synchronized List<Data> getDataList() throws InterruptedException{
But if that's the route that you want to go, use caution. Any number of threads would be able to call getDataList() at the same time. Even though it's synchronized, you're releasing the monitor lock by calling wait()
Personally, I'd go with the BlockingQueue

Here is the example on how to use Blocking Queue to tackle producer consumer problems

I'm the OP. While BlockingQueue would certainly work, I learned that Semaphore would be a simpler solution.

How to use ConcurrentLinkedQueue?

How do I use a ConcurrentLinkedQueue in Java?
Using this LinkedQueue, do I need to be worried about concurrency in the queue? Or do I just have to define two methods (one to retrive elements from the list and another to add elements to the list)?
Note: obviously these two methods have to be synchronized. Right?
EDIT: What I'm trying to do is this: I have a class (in Java) with one method to retrieve items from the queue and another class with one method to add items to the queue. The items added and retrieved from the list are objects of my own class.
One more question: do I need to do this in the remove method:
while (queue.size() == 0){
wait();
queue.poll();
}
I only have one consumer and one producer.

No, the methods don't need to be synchronized, and you don't need to define any methods; they are already in ConcurrentLinkedQueue, just use them. ConcurrentLinkedQueue does all the locking and other operations you need internally; your producer(s) adds data into the queue, and your consumers poll for it.
First, create your queue:
Queue<YourObject> queue = new ConcurrentLinkedQueue<YourObject>();
Now, wherever you are creating your producer/consumer objects, pass in the queue so they have somewhere to put their objects (you could use a setter for this, instead, but I prefer to do this kind of thing in a constructor):
YourProducer producer = new YourProducer(queue);
and:
YourConsumer consumer = new YourConsumer(queue);
and add stuff to it in your producer:
queue.offer(myObject);
and take stuff out in your consumer (if the queue is empty, poll() will return null, so check it):
YourObject myObject = queue.poll();
For more info see the Javadoc
EDIT:
If you need to block waiting for the queue to not be empty, you probably want to use a LinkedBlockingQueue, and use the take() method. However, LinkedBlockingQueue has a maximum capacity (defaults to Integer.MAX_VALUE, which is over two billion) and thus may or may not be appropriate depending on your circumstances.
If you only have one thread putting stuff into the queue, and another thread taking stuff out of the queue, ConcurrentLinkedQueue is probably overkill. It's more for when you may have hundreds or even thousands of threads accessing the queue at the same time. Your needs will probably be met by using:
Queue<YourObject> queue = Collections.synchronizedList(new LinkedList<YourObject>());
A plus of this is that it locks on the instance (queue), so you can synchronize on queue to ensure atomicity of composite operations (as explained by Jared). You CANNOT do this with a ConcurrentLinkedQueue, as all operations are done WITHOUT locking on the instance (using java.util.concurrent.atomic variables). You will NOT need to do this if you want to block while the queue is empty, because poll() will simply return null while the queue is empty, and poll() is atomic. Check to see if poll() returns null. If it does, wait(), then try again. No need to lock.
Finally:
Honestly, I'd just use a LinkedBlockingQueue. It is still overkill for your application, but odds are it will work fine. If it isn't performant enough (PROFILE!), you can always try something else, and it means you don't have to deal with ANY synchronized stuff:
BlockingQueue<YourObject> queue = new LinkedBlockingQueue<YourObject>();
queue.put(myObject); // Blocks until queue isn't full.
YourObject myObject = queue.take(); // Blocks until queue isn't empty.
Everything else is the same. Put probably won't block, because you aren't likely to put two billion objects into the queue.

This is largely a duplicate of another question.
Here's the section of that answer that is relevant to this question:
Do I need to do my own synchronization if I use java.util.ConcurrentLinkedQueue?
Atomic operations on the concurrent collections are synchronized for you. In other words, each individual call to the queue is guaranteed thread-safe without any action on your part. What is not guaranteed thread-safe are any operations you perform on the collection that are non-atomic.
For example, this is threadsafe without any action on your part:
queue.add(obj);
or
queue.poll(obj);
However; non-atomic calls to the queue are not automatically thread-safe. For example, the following operations are not automatically threadsafe:
if(!queue.isEmpty()) {
queue.poll(obj);
}
That last one is not threadsafe, as it is very possible that between the time isEmpty is called and the time poll is called, other threads will have added or removed items from the queue. The threadsafe way to perform this is like this:
synchronized(queue) {
if(!queue.isEmpty()) {
queue.poll(obj);
}
}
Again...atomic calls to the queue are automatically thread-safe. Non-atomic calls are not.

This is probably what you're looking for in terms of thread safety & "prettyness" when trying to consume everything in the queue:
for (YourObject obj = queue.poll(); obj != null; obj = queue.poll()) {
}
This will guarantee that you quit when the queue is empty, and that you continue to pop objects off of it as long as it's not empty.

Use poll to get the first element, and add to add a new last element. That's it, no synchronization or anything else.

The ConcurentLinkedQueue is a very efficient wait/lock free implementation (see the javadoc for reference), so not only you don't need to synchronize, but the queue will not lock anything, thus being virtually as fast as a non synchronized (not thread safe) one.

Just use it as you would a non-concurrent collection. The Concurrent[Collection] classes wrap the regular collections so that you don't have to think about synchronizing access.
Edit: ConcurrentLinkedList isn't actually just a wrapper, but rather a better concurrent implementation. Either way, you don't have to worry about synchronization.