Develop Reference

What is the point of BlockingQueue not being able to work in synchronized Producer/Consumer methods?

When I first read about interface BlockingQueue I read that: Producer blocks any more put() calls in a queue if it has no more space. And the opposite, it blocks method take(), if there are no items to take. I thought that it internally works same as wait() and notify(). For example, when there are no more elements to read internally wait() is called until Producer adds one more and calls notify()..or that's what we would do in 'old producer/consumer pattern. BUT IT DOESN'T WORK LIKE THAT IN BLOCKING QUEUE. How? What is the point? I am honestly surprised!
I will demonstrate:
public class Testing {
BlockingQueue<Integer> blockingQueue = new ArrayBlockingQueue<>(3);
synchronized void write() throws InterruptedException {
for (int i = 0; i < 6; i++) {
blockingQueue.put(i);
System.out.println("Added " + i);
Thread.sleep(1000);
}
}
synchronized void read() throws InterruptedException {
for (int i = 0; i < 6; i++) {
System.out.println("Took: " + blockingQueue.take());
Thread.sleep(3000);
}
}
}
class Test1 {
public static void main(String[] args) {
Testing testing = new Testing();
new Thread(new Runnable() {
#Override
public void run() {
try {
testing.write();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}).start();
new Thread(new Runnable() {
#Override
public void run() {
try {
testing.read();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}).start();
}
}
OUTPUT:
Added 0
Added 1
Added 2
'program hangs'.
My questions is how does take() and put() BLOCK if they don't use wait() or notify() internally? Do they have some while loops that burns CPU circles fast? I am frankly confused.

Here's the current implementation of ArrayBlockingQueue#put:
/**
* Inserts the specified element at the tail of this queue, waiting
* for space to become available if the queue is full.
*
* #throws InterruptedException {#inheritDoc}
* #throws NullPointerException {#inheritDoc}
*/
public void put(E e) throws InterruptedException {
Objects.requireNonNull(e);
final ReentrantLock lock = this.lock;
lock.lockInterruptibly();
try {
while (count == items.length)
notFull.await();
enqueue(e);
} finally {
lock.unlock();
}
}
You'll see that, instead of using wait() and notify(), it invokes notFull.await(); where notFull is a Condition.
The documentation of Condition states the following:
Condition factors out the Object monitor methods (wait, notify and notifyAll) into distinct objects to give the effect of having multiple wait-sets per object, by combining them with the use of arbitrary Lock implementations. Where a Lock replaces the use of synchronized methods and statements, a Condition replaces the use of the Object monitor methods.

If you go through below code, you will get an idea that how producer/consumer problem will get resolve using BlokingQueue interface.
Here you are able to see that same queue has been shared by Producer and Consumer.
And from main class you are starting both thread Producer and Consumer.
class Producer implements Runnable {
protected BlockingQueue blockingQueue = null;
public Producer(BlockingQueue blockingQueue) {
this.blockingQueue = blockingQueue;
}
#Override
public void run() {
for (int i = 0; i < 6; i++) {
try {
blockingQueue.put(i);
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("Added " + i);
}
}
}
class Consumer implements Runnable {
protected BlockingQueue blockingQueue = null;
public Consumer(BlockingQueue blockingQueue) {
this.blockingQueue = blockingQueue;
}
#Override
public void run() {
for (int i = 0; i < 6; i++) {
try {
System.out.println("Took: " + blockingQueue.take());
Thread.sleep(3000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
class Test1 {
public static void main(String[] args) throws InterruptedException {
BlockingQueue queue = new ArrayBlockingQueue(3);
Producer producer = new Producer(queue);
Consumer consumer = new Consumer(queue);
new Thread(producer).start();
new Thread(consumer).start();
Thread.sleep(4000);
}
}
This code will print output like
Took: 0
Added 0
Added 1
Added 2
Took: 1
Added 3
Added 4
Took: 2
Added 5
Took: 3
Took: 4
Took: 5

(I'm sure some or all parts of my answer could be something that you have already understood, in that case, please just consider it as a clarification :)).
1. Why did your code example using BlockingQueue get to ‘program hangs’?
1.1 Conceptually
First of all, if we can leave out the implementation level detail such as ‘wait()’, ‘notify()’, etc for a second, conceptually, all implementation in JAVA of BlockingQueue do work to the specification, i.e. like you said:
‘Producer blocks any more put() calls in a queue if it has no more
space. And the opposite, it blocks method take(), if there are no
items to take.’
So, conceptually, the reason that your code example hangs is because
1.1.1.
the thread calling the (synchronized) write() runs first and alone, and not until ‘testing.write()’ returns in this thread, the 2nd thread calling the (synchronized) read() will ever have a chance to run — this is the essence of ‘synchronized’ methods in the same object.
1.1.2.
Now, in your example, conceptually, ‘testing.write()’ will never return, in that for loop, it will ‘put’ the first 3 elements onto the queue and then kinda ‘spin wait’ for the 2nd thread to consume/’take’ some of these elements so it can ‘put’ more, but that will never happen due to aforementioned reason in 1.1.1
1.2 Programmatically
1.2.1.
(For producer) In ArrayBlockingQueue#put, the ‘spin wait’ I mentioned in 1.1.2 took form of
while (count == items.length) notFull.await();
1.2.2.
(For consumer) In ArrayBlockingQueue#take, it calls dequeue(), which in turn calls notFull.signal(), which will end the ‘spin wait’ in 1.2.1
2.Now, back to your original post’s title ‘What is the point of BlockingQueue not being able to work in synchronized Producer/Consumer methods?’.
2.1.
If I take the literal meaning of this question, then an answer could be ‘there are reasons for a convenient BlockingQueue facility to exist in JAVA other than using them in synchronized methods/blocks’, i.e. they can certainly live outside of any ‘synchronized’ structure and facilitate a vanilla producer/consumer implementation.
2.2.
However, if you meant to inquire one step further - Why can’t JAVA BlockQueue implementations work easily/nicely/smoothly in synchronized methods/blocks?
That will be a different question, a valid and interesting one that I am also incidentally puzzling about.
Specifically, see this post for further information (note that in this post, the consumer thread ‘hangs’ because of EMPTY queue and its possession of the exclusive lock, as opposed to your case where the producer thread ‘hangs’ because of FULL queue and its possession of the exclusive lock; but the core of the problems should be the same)

How to interrupt thread to do work and then sleep after doing work?

I want to have a thread which does some I/O work when it is interrupted by a main thread and then go back to sleep/wait until the interrupt is called back again.
So, I have come up with an implementation which seems to be not working. The code snippet is below.
Note - Here the flag is a public variable which can be accessed via the thread class which is in the main class
// in the main function this is how I am calling it
if(!flag) {
thread.interrupt()
}
//this is how my thread class is implemented
class IOworkthread extends Thread {
#Override
public void run() {
while(true) {
try {
flag = false;
Thread.sleep(1000);
} catch (InterruptedException e) {
flag = true;
try {
// doing my I/O work
} catch (Exception e1) {
// print the exception message
}
}
}
}
}
In the above snippet, the second try-catch block catches the InterruptedException. This means that both of the first and second try-catch block are catching the interrupt. But I had only called interrupt to happen during the first try-catch block.
Can you please help me with this?
EDIT
If you feel that there can be another solution for my objective, I will be happy to know about it :)

If it's important to respond fast to the flag you could try the following:
class IOworkthread extends Thread {//implements Runnable would be better here, but thats another story
#Override
public void run() {
while(true) {
try {
flag = false;
Thread.sleep(1000);
}
catch (InterruptedException e) {
flag = true;
}
//after the catch block the interrupted state of the thread should be reset and there should be no exceptions here
try {
// doing I/O work
}
catch (Exception e1) {
// print the exception message
// here of course other exceptions could appear but if there is no Thread.sleep() used here there should be no InterruptedException in this block
}
}
}
}
This should do different because in the catch block when the InterruptedException is caught, the interrupted flag of the thread is reset (at the end of the catch block).

It does sound like a producer/consumer construct. You seem to kind of have it the wrong way around, the IO should be driving the algorithm. Since you stay very abstract in what your code actually does, I'll need to stick to that.
So let's say your "distributed algorithm" works on data of type T; that means that it can be described as a Consumer<T> (the method name in this interface is accept(T value)). Since it can run concurrently, you want to create several instances of that; this is usually done using an ExecutorService. The Executors class provides a nice set of factory methods for creating one, let's use Executors.newFixedThreadPool(parallelism).
Your "IO" thread runs to create input for the algorithm, meaning it is a Supplier<T>. We can run it in an Executors.newSingleThreadExecutor().
We connect these two using a BlockingQueue<T>; this is a FIFO collection. The IO thread puts elements in, and the algorithm instances take out the next one that becomes available.
This makes the whole setup look something like this:
void run() {
int parallelism = 4; // or whatever
ExecutorService algorithmExecutor = Executors.newFixedThreadPool(parallelism);
ExecutorService ioExecutor = Executors.newSingleThreadExecutor();
// this queue will accept up to 4 elements
// this might need to be changed depending on performance of each
BlockingQueue<T> queue = new ArrayBlockingQueue<T>(parallelism);
ioExecutor.submit(new IoExecutor(queue));
// take element from queue
T nextElement = getNextElement(queue);
while (nextElement != null) {
algorithmExecutor.submit(() -> new AlgorithmInstance().accept(nextElement));
nextElement = getNextElement(queue);
if (nextElement == null) break;
}
// wait until algorithms have finished running and cleanup
algorithmExecutor.awaitTermination(Integer.MAX_VALUE, TimeUnit.YEARS);
algorithmExecutor.shutdown();
ioExecutor.shutdown(); // the io thread should have terminated by now already
}
T getNextElement(BlockingQueue<T> queue) {
int timeOut = 1; // adjust depending on your IO
T result = null;
while (true) {
try {
result = queue.poll(timeOut, TimeUnits.SECONDS);
} catch (TimeoutException e) {} // retry indefinetely, we will get a value eventually
}
return result;
}
Now this doesn't actually answer your question because you wanted to know how the IO thread can be notified when it can continue reading data.
This is achieved by the limit to the BlockingQueue<> which will not accept elements after this has been reached, meaning the IO thread can just keep reading and try to put in elements.
abstract class IoExecutor<T> {
private final BlockingQueue<T> queue;
public IoExecutor(BlockingQueue<T> q) { queue = q; }
public void run() {
while (hasMoreData()) {
T data = readData();
// this will block if the queue is full, so IO will pause
queue.put(data);
}
// put null into queue
queue.put(null);
}
protected boolean hasMoreData();
protected abstract T readData();
}
As a result during runtime you should at all time have 4 threads of the algorithm running, as well as (up to) 4 items in the queue waiting for one of the algorithm threads to finish and pick them up.

How to process contents from large text file using multiple threads?

I have to read a huge file contains text, around 3GB (and 40 Million lines). Just reading it happens really fast:
try (BufferedReader br = new BufferedReader(new FileReader("file.txt"))) {
while ((line = br.readLine()) != null) {
//Nothing here
}
}
With each read line from above code i do some parsing on the string and process it further.(a huge task). I try to do that multiple threads.
A) I have tried BlockingQueue like this
try (BufferedReader br = new BufferedReader(new FileReader("file.txt"))) {
String line;
BlockingQueue<String> queue = new ArrayBlockingQueue<>(100);
int numThreads = 5;
Consumer[] consumer = new Consumer[numThreads];
for (int i = 0; i < consumer.length; i++) {
consumer[i] = new Consumer(queue);
consumer[i].start();
}
while ((line = br.readLine()) != null) {
queue.put(line);
}
queue.put("exit");
} catch (FileNotFoundException ex) {
Logger.getLogger(ReadFileTest.class.getName()).log(Level.SEVERE, null, ex);
} catch (IOException | InterruptedException ex) {
Logger.getLogger(ReadFileTest.class.getName()).log(Level.SEVERE, null, ex);
}
class Consumer extends Thread {
private final BlockingQueue<String> queue;
Consumer(BlockingQueue q) {
queue = q;
}
public void run() {
while (true) {
try {
String result = queue.take();
if (result.equals("exit")) {
queue.put("exit");
break;
}
System.out.println(result);
} catch (InterruptedException ex) {
Logger.getLogger(ReadFileTest.class.getName()).log(Level.SEVERE, null, ex);
}
}
}
}
This approach took more time than normal single threaded processing.
I am not sure why - what am I doing wrong?
B) I have tried ExecutorService:
try (BufferedReader br = new BufferedReader(new FileReader("file.txt"))) {
String line;
ExecutorService pool = Executors.newFixedThreadPool(10);
while ((line = br.readLine()) != null) {
pool.execute(getRunnable(line));
}
pool.shutdown();
} catch (FileNotFoundException ex) {
Logger.getLogger(ReadFileTest.class.getName()).log(Level.SEVERE, null, ex);
} catch (IOException ex) {
Logger.getLogger(ReadFileTest.class.getName()).log(Level.SEVERE, null, ex);
}
private static Runnable getRunnable(String run){
Runnable task = () -> {
System.out.println(run);
};
return task;
}
This approach also takes more time than printing directly inside while loop. What am I doing wrong?
What is the correct way to do it?
How can I efficiently process the read line with multiple threads?

Answering one part here - why is the BlockingQueue option slower.
It is important to understand that threads don't come for "free". There is always certain overhead required to get them up and "manage" them.
And of course, when you are actually using more threads than your hardware can support "natively" then context switching is added to the bill.
Beyond that, also the BlockingQueue doesn't come free either. You see, in order to preserve order, that ArrayBlockingQueue probably has to synchronize somewhere. Worst case, that means locking and waiting. Yes, the JVM and JIT are usually pretty good about such things; but again, a certain "percentage" gets added to the bill.
But just for the record, that shouldn't matter. From the javadoc:
This class supports an optional fairness policy for ordering waiting producer and consumer threads. By default, this ordering is not guaranteed. However, a queue constructed with fairness set to true grants threads access in FIFO order. Fairness generally decreases throughput but reduces variability and avoids starvation.
As you are not setting "fairness"
BlockingQueue queue = new ArrayBlockingQueue<>(100);
that shouldn't affect you. On the other hand: I am pretty sure you expected those lines to be processed in order, so you would actually want to go for
BlockingQueue<String> queue = new ArrayBlockingQueue<>(100, true);
and thereby further slowing down the whole thing.
Finally: I agree with the comments given so far. Benchmarking such things is a complex undertaking; and many aspects influence the results. The most important question is definitely: where is your bottle neck?! Is it IO performance (then more threads don't help much!) - or is it really overall processing time (and then the "correct" number of threads for processing should definitely speed up things).
And regarding "how to do this in the correct way" - I suggest to check out this question on softwareengineering.SE.

How to process contents from large text file using multiple threads?
If your computer has enough RAM, I would do the following:
read the entire file into a variable (an ArrayList for example) - using only one thread to read the whole file.
then launch one ExecutorService (with a thread pool that uses no more than the maximum number of cores that your computer can run simultaneously)
int cores = Runtime.getRuntime().availableProcessors();
ExecutorService executorService = Executors.newFixedThreadPool(cores);
finally, divide the lines read, among a limited number of callables/runnables and submit those callables/runnables to your ExecutorService (so all of them can execute simultaneously in your ExecutorService).
unless your processing of lines uses I/O, I assume that you will reach near 100% CPU utilization, and none of your threads will be in waiting state.
do you want even faster processing?
scale vertically is the easiest option: buy even more RAM, better CPU (with more cores), use a Solid State Drive

May be all thread accessing same shared resource concurrently so result more contentious.
One thing you can try reader thread put all line in single key do submit in partition way so it will less contentious.
public void execute(Runnable command) {
final int key= command.getKey();
//Some code to check if it is runing
final int index = key != Integer.MIN_VALUE ? Math.abs(key) % size : 0;
workers[index].execute(command);
}
Create worker with queue so that if you want some task required sequentially then run.
private final AtomicBoolean scheduled = new AtomicBoolean(false);
private final BlockingQueue<Runnable> workQueue = new LinkedBlockingQueue<Runnable>(maximumQueueSize);
public void execute(Runnable command) {
long timeout = 0;
TimeUnit timeUnit = TimeUnit.SECONDS;
if (command instanceof TimeoutRunnable) {
TimeoutRunnable timeoutRunnable = ((TimeoutRunnable) command);
timeout = timeoutRunnable.getTimeout();
timeUnit = timeoutRunnable.getTimeUnit();
}
boolean offered;
try {
if (timeout == 0) {
offered = workQueue.offer(command);
} else {
offered = workQueue.offer(command, timeout, timeUnit);
}
} catch (InterruptedException e) {
throw new RejectedExecutionException("Thread is interrupted while offering work");
}
if (!offered) {
throw new RejectedExecutionException("Worker queue is full!");
}
schedule();
}
private void schedule() {
//if it is already scheduled, we don't need to schedule it again.
if (scheduled.get()) {
return;
}
if (!workQueue.isEmpty() && scheduled.compareAndSet(false, true)) {
try {
executor.execute(this);
} catch (RejectedExecutionException e) {
scheduled.set(false);
throw e;
}
}
}
public void run() {
try {
Runnable r;
do {
r = workQueue.poll();
if (r != null) {
r.run();
}
}
while (r != null);
} finally {
scheduled.set(false);
schedule();
}
}
As suggested above there is no fixed rule for thread pool size .But there is some suggestion or best practice available can be used depending upon your use case.
CPU Bound Tasks
For CPU bound tasks, Goetz (2002, 2006) recommends
threads = number of CPUs + 1
IO Bound Tasks
Working out the optimal number for IO bound tasks is less obvious. During an IO bound task, a CPU will be left idle (waiting or blocking). This idle time can be better used in initiating another IO bound request.

Writing to/Reading from a Vector (or ArrayList) with two threads

I have two threads both of which accesses an Vector. t1 adds a random number, while t2 removes and prints the first number. Below is the code and the output. t2 seems to execute only once (before t1 starts) and terminates forever. Am I missing something here? (PS: Tested with ArrayList as well)
import java.util.Random;
import java.util.Vector;
public class Main {
public static Vector<Integer> list1 = new Vector<Integer>();
public static void main(String[] args) throws InterruptedException {
System.out.println("Main started!");
Thread t1 = new Thread(new Runnable() {
#Override
public void run() {
System.out.println("writer started! ");
Random rand = new Random();
for(int i=0; i<10; i++) {
int x = rand.nextInt(100);
list1.add(x);
System.out.println("writer: " + x);
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
});
Thread t2 = new Thread(new Runnable() {
#Override
public void run() {
System.out.println("reader started! ");
while(!list1.isEmpty()) {
int x = list1.remove(0);
System.out.println("reader: "+x);
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
});
t2.start();
t1.start();
t1.join();
t2.join();
}
}
Output:
Main started!
reader started!
writer started!
writer: 40
writer: 9
writer: 23
writer: 5
writer: 41
writer: 29
writer: 72
writer: 73
writer: 95
writer: 46

This sounds like a toy to understand concurrency, so I didn't mention it before, but I will now (at the top because it is important).
If this is meant to be production code, don't roll your own. There are plenty of well implemented (debugged) concurrent data structures in java.util.concurrent. Use them.
When consuming, you need to not shutdown your consumer based on "all items consumed". This is due to a race condition where the consumer might "race ahead" of the producer and detect an empty list only because the producer hasn't yet written the items for consumption.
There are a number of ways to accomplish a shutdown of the consumer, but none of them can be done by looking at the data to be consumed in isolation.
My recommendation is that the producer "signals" the consumer when the producer is done producing. Then the consumer will stop when it has both the "signal" no more data is being produced AND the list is empty.
Alternative techniques include creating a "shutdown" item. The "producer" adds the shutdown item, and the consumer only shuts down when the "shutdown" item is seen. If you have a group of consumers, keep in mind that you shouldn't remove the shutdown item (or only one consumer would shutdown).
Also, the consumer could "monitor" the producer, such that if the producer is "alive / existent" and the list is empty, the consumer assumes that more data will become available. Shutdown occurs when the producer is dead / non-existent AND no data is available.
Which technique you use will depend on the approach you prefer and the problem you're trying to solve.
I know that people like the elegant solutions, but if your single producer is aware of the single consumer, the first option looks like.
public class Producer {
public void shutdown() {
addRemainingItems();
consumer.shutdown();
}
}
where the Consumer looks like {
public class Consumer {
private boolean shuttingDown = false;
public void shutdown() {
shuttingDown = true;
}
public void run() {
if (!list.isEmpty() && !shuttingDown) {
// pull item and process
}
}
}
Note that such lack of locking around items on the list is inherently dangerous, but you stated only a single consumer, so there's no contention for reading from the list.
Now if you have multiple consumers, you need to provide protections to assure that a single item isn't pulled by two threads at the same time (and need to communicate in such a manner that all threads shutdown).

I think this is a typical Producer–consumer problem. Try to have a look into Semaphore.

Update: The issue`s gone after changing the while loop in the consumer (reader). Instead of exiting the thread if the list is empty, it now enters the loop but does not do anything. Below is the updated reader thread. Of course a decent shutdown mechanism can also be added to the code such as Edwin suggested.
public void run() {
System.out.println("reader started! ");
while(true) {
if(!list1.isEmpty()) {
int x = list1.remove(0);
System.out.println("reader: "+x);
try {
Thread.sleep(100);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
}
Please note, this is not a code snippet taken from a real product or will go in one!

java program a infinite loop in thread cause hundred percent cpu

I am quite new to Threads in Java, I am using an API which is using thread internally and listening data from the counter party, I am putting this data in an queue for further processing. I have created another Thread which is continuously reading this queue for retrieving data and processing and to write the results into text file. I am using while(true) statement to run infinite loop in thread this cause a hundred per cent of CPU usage and if I use sleep(10) in it add up latency which keep on increasing with time as I am receiving about 20 data item in one second.
public void run() {
while(true) {
try { Thread.sleep(10); }
catch (InterruptedException e2) { // TODO Auto-generated catch block
e2.printStackTrace();
}
if (!(queue.isEmpt())) {
Tick quote=queue.take();
processTuple(quote);
}
} // end while(true)
} // end run()
Could anyone suggest me solution where I can reduce CPU usage without adding latency.

Check out ArrayBlockingQueue.
EDIT:
Example of how to use a queue based on your code:
LinkedBlockingQueue<Tick> queue;
public void run() {
while (true) {
// No need to check the queue. No need to sleep().
// take() will wait until there's anything available
Tick quote = queue.take();
processTuple(quote);
}
}

Ya. Use a BlockingQueue implementation instead of busy- wait. while(true) will keep scheduling the thread.

Use queue implementations instead of Threads. See this link to know more about queue implementations. You can use ArrayBlockingQueue.

You may change your code something like this:
BlockingQueue<Tick> queue = ..
public void run()
{
for (Tick quote; quote = queue.take(); )
{
if (quote == someSpecialObjectToIndicateStop)
break; // To stop this thread Or you may catch InterruptedException
processTuple(quote);
}
}
See BlockingQueue documentation here