new SynchronousQueue()
new LinkedBlockingQueue(1)
What is the difference? When I should use SynchronousQueue against LinkedBlockingQueue with capacity 1?
the SynchronousQueue is more of a handoff, whereas the LinkedBlockingQueue just allows a single element. The difference being that the put() call to a SynchronousQueue will not return until there is a corresponding take() call, but with a LinkedBlockingQueue of size 1, the put() call (to an empty queue) will return immediately.
I can't say that i have ever used the SynchronousQueue directly myself, but it is the default BlockingQueue used for the Executors.newCachedThreadPool() methods. It's essentially the BlockingQueue implementation for when you don't really want a queue (you don't want to maintain any pending data).
As far as I understand code above do the same things.
No, the code is not the same at all.
Sync.Q. requires to have waiter(s) for offer to succeed. LBQ will keep the item and offer will finish immediately even if there is no waiter.
SyncQ is useful for tasks handoff. Imagine you have a list w/ pending task and 3 threads available waiting on the queue, try offer() with 1/4 of the list if not accepted the thread can run the task on its own. [the last 1/4 should be handled by the current thread, if you wonder why 1/4 and not 1/3]
Think of trying to hand the task to a worker, if none is available you have an option to execute the task on your own (or throw an exception). On the contrary w/ LBQ, leaving the task in the queue doesn't guarantee any execution.
Note: the case w/ consumers and publishers is the same, i.e. the publisher may block and wait for consumers but after offer or poll returns, it ensures the task/element is to be handled.
One reason to use SynchronousQueue is to improve application performance. If you must have a hand-off between threads, you will need some synchronization object. If you can satisfy the conditions required for its use, SynchronousQueue is the fastest synchronization object I have found. Others agree. See: Implementation of BlockingQueue: What are the differences between SynchronousQueue and LinkedBlockingQueue
[Just trying to put it in (possibly) more clearer words.]
I believe the SynchronousQueue API docs states things very clearly:
A blocking queue in which each insert operation must wait for a corresponding remove operation by another thread, and vice versa.
A synchronous queue does not have any internal capacity, not even a capacity of one. You cannot peek at a synchronous queue because an element is only present when you try to remove it; you cannot insert an element (using any method) unless another thread is trying to remove it; you cannot iterate as there is nothing to iterate.
The head of the queue is the element that the first queued inserting thread is trying to add to the queue; if there is no such queued thread then no element is available for removal and poll() will return null.
And BlockingQueue API docs:
A Queue that additionally supports operations that wait for the queue to become non-empty when retrieving an element, and wait for space to become available in the queue when storing an element.
So the difference is obvious and somewhat critically subtle, especially the 3rd point below:
If the queue is empty when you are retrieving from BlockingQueue, the operation block till the new element is inserted. Also, if the queue is full when you are inserting in the BlockingQueue, the operation will block till the element is removed from the queue and a space is made for the new queue. However note that in SynchronousQueue, as operation is blocked for opposite operation (insert and remove are opposite of each other) to occur on another thread. So, unlike BlockingQueue, the blocking depends on the existence of the operation, instead of existence or non existence of an element.
As the blocking is dependent on existence of opposite operation, the element never really gets inserted in the queue. Thats why the second point: "A synchronous queue does not have any internal capacity, not even a capacity of one."
As a consequence, peek() always returns null (again, check the API doc) and iterator() returns an empty iterator in which hasNext() always returns false. (API doc). However, note that the poll() method neatly retrieves and removes the head of this queue, if another thread is currently making an element available and if no such thread exists, it returns null. (API doc)
Finally, a small note, both SynchronousQueue and LinkedBlockingQueue classes implement BlockingQueue interface.
SynchronousQueue works in a similar fashion with following major differences:
1) The size of SynchronousQueue is 0
2) put() method will only insert an element if take() method will be able to fetch that element from the queue at the same moment i.e. an element cannot be inserted if the consumer take() call is going to take some time to consume it.
SynchronousQueue - Insert only when someone will receive it at that moment itself.
Synchronous queues are basically used for handoff purposes. They do not have any capacity and a put operation is blocked until some other thread performs get operation.
If we want to safely share a variable between two threads, we can put that variable in synchrounous queue and let other thread take it from the queue.
Code Sample from https://www.baeldung.com/java-synchronous-queue
ExecutorService executor = Executors.newFixedThreadPool(2);
SynchronousQueue<Integer> queue = new SynchronousQueue<>();
Runnable producer = () -> {
Integer producedElement = ThreadLocalRandom
.current()
.nextInt();
try {
queue.put(producedElement);
} catch (InterruptedException ex) {
ex.printStackTrace();
}
};
Runnable consumer = () -> {
try {
Integer consumedElement = queue.take();
} catch (InterruptedException ex) {
ex.printStackTrace();
}
};
executor.execute(producer);
executor.execute(consumer);
executor.awaitTermination(500, TimeUnit.MILLISECONDS);
executor.shutdown();
assertEquals(queue.size(), 0);
They are also used in CachedThreadPool to achieve an effect of unlimited(Integer.MAX) thread creation as tasks arrive.
CachedPool has coreSize as 0 and maxPoolSize as Integer.MAX with synchronous queue
As tasks arrive onto queue, other tasks are blocked until the first one is fetched out. Since it does not have any queue capacity, thread pool will create one thread and this thread will take out task allowing more tasks to be put onto the queue. This will continue until thread creation reaches maxPoolSize. Based on timeOut, idle threads maybe terminated and new ones are created without crossing the maxPoolSize.
Related
I want a thread pool with a single thread but with a peculiar behavior for the BlockingQueue:
If I add a job to the queue and then add another job (making the queue hold two jobs), I want the thread to ignore the first job added and get the last one. So, everytime the thread gets a task from the queue, I want it to get the last job added to the queue and discard the others.
Is there any default BlockingQueue with this behavior? What would be the best strategy to achieve this? Should I implement my own BlockingQueue? If yes, from which BlockingQueue should I start from?
My initial idea was to create a bounded blocking queue with capacity for only one task, but that when it's full and receives another task, it swaps the tasks discarding the task that was added earlier. Do I make sense?
You are on the right track with creating a bounded blocking queue with capacity for only one task. Additionally, configure your ThreadPoolExecutor with a DiscardOldestPolicy. Thus, whenever a second task is submitted, it doesn’t fit into the queue and according to the policy the older one gets discarded.
Compare with the constructor ThreadPoolExecutor(int, int, long, TimeUnit, BlockingQueue, RejectedExecutionHandler)
I would use a an atomic variable.
https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/atomic/AtomicReference.html
Example:
AtomicReference<X> task_;
//Pop
public X pop() { return task_.getAndSet(NULL); }
//Push
public X push (X val) { return task_.getAndSet(val); }
It is also wait-free in design.
Im implimenting a java game with multiplayer and i have a sender thread to send a messages from a Queue to another player.
I have read here:
https://developer.android.com/reference/java/util/concurrent/BlockingQueue.html
"A Queue that additionally supports operations that wait for the queue to become non-empty"
which operetions and how to use them , and i know that an infinte loop that always checks if something Queue is what i want to avoid.
Since there appears to be no out-of-the-box solution to this problem, I had to implement my own.
So, what I do is that my void block() method invokes poll() and saves the item returned. Then, every method that obtains an item from the queue first checks if we have a saved item, and if so, returns it and clears it, otherwise it delegates to the underlying java blocking queue.
Luckily I only consume items from a single thread, so I do not have to worry about race conditions.
take() will allow you to wait until an element becomes available in the queue or use poll(long timeout, TimeUnit unit) to wait until a specified time.
I have a thread pool of m threads. Let's say m were 10 and fix. Then there are n queues with the possibility of n becoming large (like 100'000 or more). Every queue holds tasks to be executed by those m threads. Now, very important, every queue must be worked off sequentially task by task. This is a requirement to make sure that tasks are executed in the order they were added to the queue. Otherwise the data could become inconsistent (same as, say, with JMS queues).
So the question is now how to make sure that the tasks in those n queues are processed by the available m threads in a way that no task added to the same queue can be executed "at the same time" by different threads.
I tried to solve this problem myself and figured out that it is quite demanding. Java ThreadPoolExecutor is nice, but you would have to add quite a bit of functionality that is not easy to develop. So the question is whether anyone knows of some framework or system for Java that already solves this problem?
Update
Thanks to Adrian and Tanmay for their suggestions. The number of queues may be very large (like 100'000 or more). So one thread per queue is unhappily not possible although it would be simple and easy. I will look into the fork join framework. Looks like an interesting path to pursue.
My current first iteration solution is to have a global queue to which all tasks are added (using a JDK8 TransferQueue, which has very little locking overhead). Tasks are wrapped into a queue stub with the lock of the queue and its size. The queue itself does not exist physically, only its stub.
An idle thread first needs to obtain a token before it can access the global queue (the token would be a single element in a blocking queue, e.g. JDK8 TransferQueue). Then it does a blocking take on the global queue. When a task was obtained, it checks whether the queue lock of the task's queue stub is down. Actually, I think just using an AtomicBoolean would be sufficient and create less lock contention than a lock or synchronized block.
When the queue lock is obtained, the token is returned to the global queue and the task is executed. If it is not obtained, the task is added to a 2nd level queue and another blocking take from the global queue is done. Threads need to check whether the 2nd level queue is empty and take a task from it to be executed as well.
This solution seems to work. However, the token every thread needs to acquire before being allowed to access the global queue and the 2nd level queue looks like a bottleneck. I believe it will create high lock contention. So, I'm not so happy with this. Maybe I start with this solution and elaborate on it.
Update 2
All right, here now the "best" solution I have come up with so far. The following queues are defined:
Ready Queue (RQ): Contains all tasks that can be executed immediately by any thread in the thread pool
Entry Queue (EQ): Contains all tasks the user wants to be executed as well as internal admin tasks. The EQ is a priority queue. Admin tasks have highest priority.
Channels Queues (CQ): For every channel there is an internal channel queue that is used to preserve the ordering of the tasks, e.g. make sure task are executed sequentially in the order they were added to EQ
Scheduler: Dedicated thread that takes tasks from EQ. If the task is a user task it is added to the CQ of the channel the task was added to. If the head of the CQ equals the just inserted user task it is also added to the EQ (but remains in the CQ) so that it is executes as soon as the next thread of the thread pool becomes available.
If a user task has finished execution an internal task TaskFinished is added to RQ. When executed by the scheduler, the head is taken from the associated CQ. If the CQ is not empty after the take, the next task is polled (but not taken) from the CQ and added to the RQ. The TaskFinished tasks have higher priority than user tasks.
This approach contains in my opinion no logical errors. Note that EQ and RQ need to be synchronized. I prefer using TransferQueue from JDK8 which is very fast and where checking for it to be empty or not, polling the head item is also very fast. The CQs need not be synchronized as they are always accessed by the Scheduler only.
So far I'm quite happy with this solution. What makes me think is whether the Scheduler could turn into a bottleneck. If there are much more tasks in the EQ than it can handle the EQ might grow building up some backlog. Any opinions about that would be appreciated :-)
You can use Fork Join Framework if you are working in Java 7 or Java 8.
You can create a RecursiveTask using popped first element from each queue.
Remember to provide a reference to the queues to the corresponding RecursiveTasks.
Invoke all of the at once. (In a loop or stream).
Now at the end of the compute method (after processing of a task is completed), create another RecursiveTask by popping another element from the corresponding queue and call invoke on it.
Notes:
Each task will be responsible for extracting new element from the queue, so all tasks from the queue would be executed sequentially.
There should be a new RecursiveTask created and invoked separately for each element in the queues. This ensures that some queues do not hog the threads and starvation is avoided.
Using an ExecutorService is also a viable option, but IMO ForkJoin's API if friendlier for your use case
Hope this helps.
One simple solution would be to create a task whenever an element is added to an empty queue. This task would be responsible for only that queue and would end when the queue has been worked off. Ensure that the Queue implementations are thread-safe and the task stops after removing the last element.
EDIT: These tasks should be added to a ThreadPoolExecutor with an internal queue, for example one created by ExecutorService.newFixedThreadPool, which will work off the tasks in parallel with a limited number of threads.
Alternatively, just divide the queues among a fixed number of threads:
public class QueueWorker implements Runnable {
// should be unique and < NUM_THREADS:
int threadId;
QueueWorker(int threadId) {
this.threadId = threadId;
}
#Override
public void run() {
int currentQueueIndex = threadId;
while (true) {
Queue currentQueue = queues.get(currentQueue);
// execute tasks until empty
currentQueueIndex += NUM_THREADS;
if (currentQueueIndex > queues.size()) {
currentQueueIndex = threadId;
}
}
}
}
Im looking for a way to synchronize multiple asynchronous operations. I'd like to use a BlockingQueue with a size equal to my operations but who can i wait till the Queue is full?
Im looking for something like a reversed Blocking Queue.
I need to gather the Results of each Thread at the End.
The AsyncHandler is fixed, its already a ThreadExecutor underlying, i cannot start new Threads.
//3 Times
makeAsync(new AsyncHandler() {
onSuccess() {
..
queue.put(result)
}
onFailure() {
..
}
});
//Blocking till the Queue is full
List<Results> = queue.takeAll()
Bonus Question: I need a way to end the wait when one of my Requests fails
I've never had need to do this sort of thing, but you might have some luck using a CountDownLatch or CyclicBarrier from your various threads.
What you describe with
//Blocking till the Queue is full
List<Results> results = queue.takeAll();
does not differ semantically from “take as much items as the queue’s capacity”. If you know the capacity you can achieve this by:
// preferably a constant which you also use to construct the bounded queue
int capacity;
…
List<Results> results = new ArrayList<>(capacity);
queue.drainTo(results, capacity);
while(result.size()<capacity)
queue.drainTo(results, capacity-result.size());
This will block until it has received as much items as the capacity which is, as said, the same as waiting for the queue to become full (has a size equal to its capacity) and than take all items. The only difference is that the event of the queue becoming full is not guaranteed to happen, e.g. if you intend your async operations to offer items until the queue is full, it does not work this way.
If you don’t know the capacity, you are out of luck. There is not even a guaranty that an arbitrary BlockingQueue is bounded, read, it might have an unlimited capacity.
On the other hand, if the asynchronous operations are able to detect when they have finished, they could simply collect the items in a list locally and put the entire list into a BlockingQueue<List<Results>> as a single item once they are done. Then your code waiting for it needs only a single take to get the entire list.
If you're using Java 8, do the following:
With each call to makeAsync, create a CompletableFuture<Result> instance and make it available to the AsyncHandler, and have the caller keep a reference too, say in a list.
When an async task completes normally, have it call complete(result) on its CompletableFuture instance.
When an async task completes with an error, have it call completeExceptionally(exception) on its CompletableFuture instance.
After initiating all the asynchronous tasks, have the caller call CompletableFuture.allOf(cfArray).join(). Unfortunately this takes an array, not a list, so you have to convert. The join() call will throw an exception if any one of the tasks completed with an error. Otherwise, you can collect the results from the individual CompletableFuture instances by calling their get() methods.
If you don't have Java 8, you'll have to sort of roll your own mechanism. Initialize a CountDownLatch to the number of async tasks you're going to fire off. Have each async task store its result (or an exception, or some other means of indicating failure) into a thread-safe data structure and then decrement ('countDown`) the latch. Have the caller wait for the latch to reach zero and then collect the results and errors. This isn't terribly difficult, but you have to determine a means for storing valid results as well as recording whether an error occurred, and also maintain a count manually.
If you can modify methodAsync(), then it's as simple as to use a CountDownLatch after each time you put some elements in the queue and have the main thread wait for such a CountDownLatch.
If unfortunately you cannot modify methodAsync(), then simply wrap the queue and give it a count down latch, and then override the add() method to count down this latch. The main method just wait it to be done.
Having said the above, your program structure smells not well organized.
I am using a LinkedBlockingQueue together with the producer/consumer pattern to buffer tasks. To add tasks to the queue I use the method for my producers: Queue.put(Object); To take a task form my queue I use for my consumers: Queue.take(Object);
I found in the Java api that both these methods will block until they the queue becomes available. My problem is: I know for a fact that there are more producers of tasks in my system then consumers. And all my tasks need to be processed. So I need my consumers, when blocked, to have priority over the producers to get the queue.
Is their a way to do this without changing the methods of LinkedBlockingQueue to much?
LinkedBlockingQueue uses two ReenterantLocks lock.
private final ReentrantLock putLock = new ReentrantLock();
private final ReentrantLock takeLock = new ReentrantLock();
Since both the locks are seperate and put and take aquires seperate locks for carrying out their operating blocking one operation would not impact other operation.
Cheers !!
There is no need to prioritize consumers over producers, because they block under entirely different conditions: if the producer is blocked because the queue is full, then the consumers won't be blocked as a result of the queue being empty.
For example, producer1 has a blocked put call because the queue is full. Consumer1 then executes take, which proceeds as normal because the queue is not empty (unless your queue has a capacity of 0, which would be silly) - the consumer doesn't know or care that a producer's put call is blocked, all it cares about is that the queue is not empty.
The producers being blocked doesn't block consumers due to multiple independent locks.
take( states:
Retrieves and removes the head of this queue, waiting if necessary until an element becomes available.
put( states:
Inserts the specified element at the tail of this queue, waiting if necessary for space to become available
If there is no space, then put will block but take won't get blocked as it's by design waiting only if the queue is empty, obviously not the case here.
Original comment:
As far as I know, this queue, by design, won't block consumers even if producers are blocked due to the queue being full.