Related
I have the code sample:
public class ThreadPoolTest {
public static void main(String[] args) throws InterruptedException {
for (int i = 0; i < 100; i++) {
if (test() != 5 * 100) {
throw new RuntimeException("main");
}
}
test();
}
private static long test() throws InterruptedException {
ExecutorService executorService = Executors.newFixedThreadPool(100);
CountDownLatch countDownLatch = new CountDownLatch(100 * 5);
Set<Thread> threads = Collections.synchronizedSet(new HashSet<>());
AtomicLong atomicLong = new AtomicLong();
for (int i = 0; i < 5 * 100; i++) {
Thread.sleep(100);
executorService.submit(new Runnable() {
#Override
public void run() {
try {
threads.add(Thread.currentThread());
atomicLong.incrementAndGet();
countDownLatch.countDown();
Thread.sleep(1000);
} catch (Exception e) {
System.out.println(e);
}
}
});
}
executorService.shutdown();
countDownLatch.await();
if (threads.size() != 100) {
throw new RuntimeException("test");
}
return atomicLong.get();
}
}
I especially made application to work long.
And I see jvisualVM.
Each time gap threadpool was recreated.
After several minutes I see:
but if I use newCachedThreadPool instead of newFixedThreadPool I see constant picture:
Can you explain this behaviour?
P.S.
Problem was that exception occures in code and second iteration was not started
To answer your question; just look here:
private static long test() throws InterruptedException {
ExecutorService executorService = Executors.newFixedThreadPool(100);
The JVM creates a new ThreadPool during each run of test(), because you tell it to do so.
In other words: if you intend to re-use the same threadpool, then avoid creating/shutting down your instances all the time.
In that sense, the simple fix is: move the creation of that ExecutorService into your main() method; and pass the service as argument to your test() method.
Edit: regarding your last comment on cached vs. fixed threadpool; you probably want to look into this question.
Because you asked it to, in your code ? :) Try moving the Pool creation code outside the test.
From docs:
newFixedThreadPool
Creates a thread pool that reuses a fixed number of threads operating off a shared unbounded queue. At any point, at most nThreads threads will be active processing tasks. If additional tasks are submitted when all threads are active, they will wait in the queue until a thread is available. If any thread terminates due to a failure during execution prior to shutdown, a new one will take its place if needed to execute subsequent tasks. The threads in the pool will exist until it is explicitly shutdown.
newCachedThreadPool
Creates a thread pool that creates new threads as needed, but will reuse previously constructed threads when they are available. These pools will typically improve the performance of programs that execute many short-lived asynchronous tasks. Calls to execute will reuse previously constructed threads if available. If no existing thread is available, a new thread will be created and added to the pool. Threads that have not been used for sixty seconds are terminated and removed from the cache. Thus, a pool that remains idle for long enough will not consume any resources. Note that pools with similar properties but different details (for example, timeout parameters) may be created using ThreadPoolExecutor constructors.
I was reading through the java.util.concurrent API, and found that
CountDownLatch: A synchronization aid that allows one or more threads to wait until a set of operations being performed in other threads completes.
CyclicBarrier: A synchronization aid that allows a set of threads to all wait for each other to reach a common barrier point.
To me both seems equal, but I am sure there is much more to it.
For example, in CoundownLatch, the countdown value could not be reset, that can happen in the case of CyclicBarrier.
Is there any other difference between the two?
What are the use cases where someone would want to reset the value of countdown?
There's another difference.
When using a CyclicBarrier, the assumption is that you specify the number of waiting threads that trigger the barrier. If you specify 5, you must have at least 5 threads to call await().
When using a CountDownLatch, you specify the number of calls to countDown() that will result in all waiting threads being released. This means that you can use a CountDownLatch with only a single thread.
"Why would you do that?", you may say. Imagine that you are using a mysterious API coded by someone else that performs callbacks. You want one of your threads to wait until a certain callback has been called a number of times. You have no idea which threads the callback will be called on. In this case, a CountDownLatch is perfect, whereas I can't think of any way to implement this using a CyclicBarrier (actually, I can, but it involves timeouts... yuck!).
I just wish that CountDownLatch could be reset!
One major difference is that CyclicBarrier takes an (optional) Runnable task which is run once the common barrier condition is met.
It also allows you to get the number of clients waiting at the barrier and the number required to trigger the barrier. Once triggered the barrier is reset and can be used again.
For simple use cases - services starting etc... a CountdownLatch is fine. A CyclicBarrier is useful for more complex co-ordination tasks. An example of such a thing would be parallel computation - where multiple subtasks are involved in the computation - kind of like MapReduce.
One point that nobody has yet mentioned is that, in a CyclicBarrier, if a thread has a problem (timeout, interrupted...), all the others that have reached await() get an exception. See Javadoc:
The CyclicBarrier uses an all-or-none breakage model for failed synchronization attempts: If a thread leaves a barrier point prematurely because of interruption, failure, or timeout, all other threads waiting at that barrier point will also leave abnormally via BrokenBarrierException (or InterruptedException if they too were interrupted at about the same time).
I think that the JavaDoc has explained the differences explicitly.
Most people know that CountDownLatch can not be reset, however, CyclicBarrier can. But this is not the only difference, or the CyclicBarrier could be renamed to ResetbleCountDownLatch.
We should tell the differences from the perspective of their goals, which are described in JavaDoc
CountDownLatch: A synchronization aid that allows one or more threads to wait until a set of operations being performed in other threads completes.
CyclicBarrier: A synchronization aid that allows a set of threads to all wait for each other to reach a common barrier point.
In countDownLatch, there is one or more threads, that are waiting for a set of other threads to complete. In this situation, there are two types of threads, one type is waiting, another type is doing something, after finishes their tasks, they could be waiting or just terminated.
In CyclicBarrier, there are only one type of threads, they are waiting for each other, they are equal.
The main difference is documented right in the Javadocs for CountdownLatch. Namely:
A CountDownLatch is initialized with a
given count. The await methods block
until the current count reaches zero
due to invocations of the countDown()
method, after which all waiting
threads are released and any
subsequent invocations of await return
immediately. This is a one-shot
phenomenon -- the count cannot be
reset. If you need a version that
resets the count, consider using a
CyclicBarrier.
source 1.6 Javadoc
A CountDownLatch is used for one-time synchronization. While using a CountDownLatch, any thread is allowed to call countDown() as many times as they like. Threads which called await() are blocked until the count reaches zero because of calls to countDown() by other unblocked threads. The javadoc for CountDownLatch states:
The await methods block until the current count reaches zero due to
invocations of the countDown() method, after which all waiting threads
are released and any subsequent invocations of await return
immediately.
...
Another typical usage would be to divide a problem into N parts,
describe each part with a Runnable that executes that portion and
counts down on the latch, and queue all the Runnables to an Executor.
When all sub-parts are complete, the coordinating thread will be able
to pass through await. (When threads must repeatedly count down in
this way, instead use a CyclicBarrier.)
In contrast, the cyclic barrier is used for multiple sychronization points, e.g. if a set of threads are running a loop/phased computation and need to synchronize before starting the next iteration/phase. As per the javadoc for CyclicBarrier:
The barrier is called cyclic because it can be re-used after the
waiting threads are released.
Unlike the CountDownLatch, each call to await() belongs to some phase and can cause the thread to block until all parties belonging to that phase have invoked await(). There is no explicit countDown() operation supported by the CyclicBarrier.
This question has been adequately answered already, but I think I can value-add a little by posting some code.
To illustrate the behaviour of cyclic barrier, I have made some sample code. As soon as the barrier is tipped, it is automatically reset so that it can be used again (hence it is "cyclic"). When you run the program, observe that the print outs "Let's play" are triggered only after the barrier is tipped.
import java.util.concurrent.BrokenBarrierException;
import java.util.concurrent.CyclicBarrier;
public class CyclicBarrierCycles {
static CyclicBarrier barrier;
public static void main(String[] args) throws InterruptedException {
barrier = new CyclicBarrier(3);
new Worker().start();
Thread.sleep(1000);
new Worker().start();
Thread.sleep(1000);
new Worker().start();
Thread.sleep(1000);
System.out.println("Barrier automatically resets.");
new Worker().start();
Thread.sleep(1000);
new Worker().start();
Thread.sleep(1000);
new Worker().start();
}
}
class Worker extends Thread {
#Override
public void run() {
try {
CyclicBarrierCycles.barrier.await();
System.out.println("Let's play.");
} catch (InterruptedException e) {
e.printStackTrace();
} catch (BrokenBarrierException e) {
e.printStackTrace();
}
}
}
When I was studying about Latches and cyclicbarriers I came up with this metaphors.
cyclicbarriers: Imagine a company has a meeting room. In order to start the meeting, a certain number of meeting attendees have to come to meeting (to make it official). the following is the code of a normal meeting attendee (an employee)
class MeetingAtendee implements Runnable {
CyclicBarrier myMeetingQuorumBarrier;
public MeetingAtendee(CyclicBarrier myMileStoneBarrier) {
this.myMeetingQuorumBarrier = myMileStoneBarrier;
}
#Override
public void run() {
try {
System.out.println(Thread.currentThread().getName() + " i joined the meeting ...");
myMeetingQuorumBarrier.await();
System.out.println(Thread.currentThread().getName()+" finally meeting stared ...");
} catch (InterruptedException e) {
e.printStackTrace();
} catch (BrokenBarrierException e) {
System.out.println("Meeting canceled! every body dance <by chic band!>");
}
}
}
employee joins the meeting, waits for others to come to start meeting. also he gets exited if the meeting gets canceled :) then we have THE BOSS how doses not like to wait for others to show up and if he looses his patient, he cancels meeting.
class MeetingAtendeeTheBoss implements Runnable {
CyclicBarrier myMeetingQuorumBarrier;
public MeetingAtendeeTheBoss(CyclicBarrier myMileStoneBarrier) {
this.myMeetingQuorumBarrier = myMileStoneBarrier;
}
#Override
public void run() {
try {
System.out.println(Thread.currentThread().getName() + "I am THE BOSS - i joined the meeting ...");
//boss dose not like to wait too much!! he/she waits for 2 seconds and we END the meeting
myMeetingQuorumBarrier.await(1,TimeUnit.SECONDS);
System.out.println(Thread.currentThread().getName()+" finally meeting stared ...");
} catch (InterruptedException e) {
e.printStackTrace();
} catch (BrokenBarrierException e) {
System.out.println("what WHO canceled The meeting");
} catch (TimeoutException e) {
System.out.println("These employees waste my time!!");
}
}
}
On a normal day, employee come to meeting wait for other to show up and if some attendees don`t come they have to wait indefinitely! in some special meeting the boss comes and he does not like to wait.(5 persons need to start meeting but only boss comes and also an enthusiastic employee) so he cancels the meeting (angrily)
CyclicBarrier meetingAtendeeQuorum = new CyclicBarrier(5);
Thread atendeeThread = new Thread(new MeetingAtendee(meetingAtendeeQuorum));
Thread atendeeThreadBoss = new Thread(new MeetingAtendeeTheBoss(meetingAtendeeQuorum));
atendeeThread.start();
atendeeThreadBoss.start();
Output:
//Thread-1I am THE BOSS - i joined the meeting ...
// Thread-0 i joined the meeting ...
// These employees waste my time!!
// Meeting canceled! every body dance <by chic band!>
There is another scenario in which another outsider thread (an earth quake) cancels the meeting (call reset method). in this case all the waiting threads get woken up by an exception.
class NaturalDisasters implements Runnable {
CyclicBarrier someStupidMeetingAtendeeQuorum;
public NaturalDisasters(CyclicBarrier someStupidMeetingAtendeeQuorum) {
this.someStupidMeetingAtendeeQuorum = someStupidMeetingAtendeeQuorum;
}
void earthQuakeHappening(){
System.out.println("earth quaking.....");
someStupidMeetingAtendeeQuorum.reset();
}
#Override
public void run() {
earthQuakeHappening();
}
}
running code will result in funny output:
// Thread-1I am THE BOSS - i joined the meeting ...
// Thread-0 i joined the meeting ...
// earth quaking.....
// what WHO canceled The meeting
// Meeting canceled! every body dance <by chic band!>
You can also add a secretary to meeting room, if a meeting is held she will document every thing but she is not part of the meeting:
class MeetingSecretary implements Runnable {
#Override
public void run() {
System.out.println("preparing meeting documents");
System.out.println("taking notes ...");
}
}
Latches: if the angry boss wants to hold an exhibition for company customers, every thing needs to be ready (resources). we provide a to-do list every worker (Thread) dose his job and we check the to-do list (some workers do painting, others prepare sound system ...). when all the items in to-do list are complete (resources are provided) we can open the doors to customers.
public class Visitor implements Runnable{
CountDownLatch exhibitonDoorlatch = null;
public Visitor (CountDownLatch latch) {
exhibitonDoorlatch = latch;
}
public void run() {
try {
exhibitonDoorlatch .await();
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("customer visiting exebition");
}
}
And the workers how are preparing the exhibition:
class Worker implements Runnable {
CountDownLatch myTodoItem = null;
public Worker(CountDownLatch latch) {
this.myTodoItem = latch;
}
public void run() {
System.out.println("doing my part of job ...");
System.out.println("My work is done! remove it from todo list");
myTodoItem.countDown();
}
}
CountDownLatch preperationTodoList = new CountDownLatch(3);
// exhibition preparation workers
Worker electricalWorker = new Worker(preperationTodoList);
Worker paintingWorker = new Worker(preperationTodoList);
// Exhibition Visitors
ExhibitionVisitor exhibitionVisitorA = new ExhibitionVisitor(preperationTodoList);
ExhibitionVisitor exhibitionVisitorB = new ExhibitionVisitor(preperationTodoList);
ExhibitionVisitor exhibitionVisitorC = new ExhibitionVisitor(preperationTodoList);
new Thread(electricalWorker).start();
new Thread(paintingWorker).start();
new Thread(exhibitionVisitorA).start();
new Thread(exhibitionVisitorB).start();
new Thread(exhibitionVisitorC).start();
In a nutshell, just to understand key functional differences between the two :
public class CountDownLatch {
private Object mutex = new Object();
private int count;
public CountDownLatch(int count) {
this.count = count;
}
public void await() throws InterruptedException {
synchronized (mutex) {
while (count > 0) {
mutex.wait();
}
}
}
public void countDown() {
synchronized (mutex) {
if (--count == 0)
mutex.notifyAll();
}
}
}
and
public class CyclicBarrier {
private Object mutex = new Object();
private int count;
public CyclicBarrier(int count) {
this.count = count;
}
public void await() throws InterruptedException {
synchronized (mutex) {
count--;
while(count > 0)
mutex.wait();
mutex.notifyAll();
}
}
}
except, of course, features like non-blocking, timed waiting, diagnostics and everything which has been in details explained in the above answers.
The above classes are, however, fully functional and equivalent, within the provided functionality, to their correspondent namesakes.
On a different note, CountDownLatch's inner class subclasses AQS, while CyclicBarrier uses ReentrantLock (my suspicion is it could be other way around or both could use AQS or both use Lock -- without any loss of performance efficiency)
In CountDownLatch, threads waits for other threads to complete their execution. In CyclicBarrier, worker threads wait for each other to complete their execution.
You can not reuse same CountDownLatch instance once count reaches to zero and latch is open, on the other hand CyclicBarrier can be reused by resetting Barrier, Once barrier is broken.
One obvious difference is, only N threads can await on a CyclicBarrier of N to be release in one cycle. But unlimited number of threads can await on a CountDownLatch of N. The count down decrement can be done by one thread N times or N threads one time each or combinations.
In the case of CyclicBarrier, as soon as ALL child threads begins calling barrier.await(), the Runnable is executed in the Barrier. The barrier.await in each child thread will take different lengh of time to finish, and they all finish at the same time.
CountDownLatch is a count down of anything; CyclicBarrier is a count down for thread only
assume there are 5 worker threads and one shipper thread, and when workers produce 100 items, shipper will ship them out.
For CountDownLatch, the counter can be on workers or items
For CyclicBarrier, the counter can only on workers
If a worker falls infinite sleep, with CountDownLatch on items, Shipper can ship; However, with CyclicBarrier, Shipper can never be called
#Kevin Lee and #Jon I tried CyclicBarrier with Optional Runnable. Looks like it runs in the beginning and after the CyclicBarrier is tipped. Here is the code and output
static CyclicBarrier barrier;
public static void main(String[] args) throws InterruptedException {
barrier = new CyclicBarrier(3, new Runnable() {
#Override
public void run() {
System.out.println("I run in the beginning and after the CyclicBarrier is tipped");
}
});
new Worker().start();
Thread.sleep(1000);
new Worker().start();
Thread.sleep(1000);
new Worker().start();
Thread.sleep(1000);
System.out.println("Barrier automatically resets.");
new Worker().start();
Thread.sleep(1000);
new Worker().start();
Thread.sleep(1000);
new Worker().start();
}
Output
I run in the beginning and after the CyclicBarrier is tipped
Let's play.
Let's play.
Let's play.
Barrier automatically resets.
I run in the beginning and after the CyclicBarrier is tipped
Let's play.
Let's play.
Let's play.
I've been frustrated for some time with the default behavior of ThreadPoolExecutor which backs the ExecutorService thread-pools that so many of us use. To quote from the Javadocs:
If there are more than corePoolSize but less than maximumPoolSize threads running, a new thread will be created only if the queue is full.
What this means is that if you define a thread pool with the following code, it will never start the 2nd thread because the LinkedBlockingQueue is unbounded.
ExecutorService threadPool =
new ThreadPoolExecutor(1 /*core*/, 50 /*max*/, 60 /*timeout*/,
TimeUnit.SECONDS, new LinkedBlockingQueue<Runnable>(/* unlimited queue*/));
Only if you have a bounded queue and the queue is full are any threads above the core number started. I suspect a large number of junior Java multithreaded programmers are unaware of this behavior of the ThreadPoolExecutor.
Now I have specific use case where this is not-optimal. I'm looking for ways, without writing my own TPE class, to work around it.
My requirements are for a web service that is making call-backs to a possibly unreliable 3rd party.
I don't want to make the call-back synchronously with the web-request, so I want to use a thread-pool.
I typically get a couple of these a minute so I don't want to have a newFixedThreadPool(...) with a large number of threads that mostly are dormant.
Every so often I get a burst of this traffic and I want to scale up the number of threads to some max value (let's say 50).
I need to make a best attempt to do all callbacks so I want to queue up any additional ones above 50. I don't want to overwhelm the rest of my web-server by using a newCachedThreadPool().
How can I work around this limitation in ThreadPoolExecutor where the queue needs to be bounded and full before more threads will be started? How can I get it to start more threads before queuing tasks?
Edit:
#Flavio makes a good point about using the ThreadPoolExecutor.allowCoreThreadTimeOut(true) to have the core threads timeout and exit. I considered that but I still wanted the core-threads feature. I did not want the number of threads in the pool to drop below the core-size if possible.
How can I work around this limitation in ThreadPoolExecutor where the queue needs to be bounded and full before more threads will be started.
I believe I have finally found a somewhat elegant (maybe a little hacky) solution to this limitation with ThreadPoolExecutor. It involves extending LinkedBlockingQueue to have it return false for queue.offer(...) when there are already some tasks queued. If the current threads are not keeping up with the queued tasks, the TPE will add additional threads. If the pool is already at max threads, then the RejectedExecutionHandler will be called which does the put(...) into the queue.
It certainly is strange to write a queue where offer(...) can return false and put() never blocks so that's the hack part. But this works well with TPE's usage of the queue so I don't see any problem with doing this.
Here's the code:
// extend LinkedBlockingQueue to force offer() to return false conditionally
BlockingQueue<Runnable> queue = new LinkedBlockingQueue<Runnable>() {
private static final long serialVersionUID = -6903933921423432194L;
#Override
public boolean offer(Runnable e) {
// Offer it to the queue if there is 0 items already queued, else
// return false so the TPE will add another thread. If we return false
// and max threads have been reached then the RejectedExecutionHandler
// will be called which will do the put into the queue.
if (size() == 0) {
return super.offer(e);
} else {
return false;
}
}
};
ThreadPoolExecutor threadPool = new ThreadPoolExecutor(1 /*core*/, 50 /*max*/,
60 /*secs*/, TimeUnit.SECONDS, queue);
threadPool.setRejectedExecutionHandler(new RejectedExecutionHandler() {
#Override
public void rejectedExecution(Runnable r, ThreadPoolExecutor executor) {
try {
// This does the actual put into the queue. Once the max threads
// have been reached, the tasks will then queue up.
executor.getQueue().put(r);
// we do this after the put() to stop race conditions
if (executor.isShutdown()) {
throw new RejectedExecutionException(
"Task " + r + " rejected from " + e);
}
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
return;
}
}
});
With this mechanism, when I submit tasks to the queue, the ThreadPoolExecutor will:
Scale the number of threads up to the core size initially (here 1).
Offer it to the queue. If the queue is empty it will be queued to be handled by the existing threads.
If the queue has 1 or more elements already, the offer(...) will return false.
If false is returned, scale up the number of threads in the pool until they reach the max number (here 50).
If at the max then it calls the RejectedExecutionHandler
The RejectedExecutionHandler then puts the task into the queue to be processed by the first available thread in FIFO order.
Although in my example code above, the queue is unbounded, you could also define it as a bounded queue. For example, if you add a capacity of 1000 to the LinkedBlockingQueue then it will:
scale the threads up to max
then queue up until it is full with 1000 tasks
then block the caller until space becomes available to the queue.
Also, if you needed to use offer(...) in the
RejectedExecutionHandler then you could use the offer(E, long, TimeUnit) method instead with Long.MAX_VALUE as the timeout.
Warning:
If you expect tasks to be added to the executor after it has been shutdown, then you may want to be smarter about throwing RejectedExecutionException out of our custom RejectedExecutionHandler when the executor-service has been shutdown. Thanks to #RaduToader for pointing this out.
Edit:
Another tweak to this answer could be to ask the TPE if there are idle threads and only enqueue the item if there is so. You would have to make a true class for this and add ourQueue.setThreadPoolExecutor(tpe); method on it.
Then your offer(...) method might look something like:
Check to see if the tpe.getPoolSize() == tpe.getMaximumPoolSize() in which case just call super.offer(...).
Else if tpe.getPoolSize() > tpe.getActiveCount() then call super.offer(...) since there seem to be idle threads.
Otherwise return false to fork another thread.
Maybe this:
int poolSize = tpe.getPoolSize();
int maximumPoolSize = tpe.getMaximumPoolSize();
if (poolSize >= maximumPoolSize || poolSize > tpe.getActiveCount()) {
return super.offer(e);
} else {
return false;
}
Note that the get methods on TPE are expensive since they access volatile fields or (in the case of getActiveCount()) lock the TPE and walk the thread-list. Also, there are race conditions here that may cause a task to be enqueued improperly or another thread forked when there was an idle thread.
Set core size and max size to the same value, and allow core threads to be removed from the pool with allowCoreThreadTimeOut(true).
I've already got two other answers on this question, but I suspect this one is the best.
It's based on the technique of the currently accepted answer, namely:
Override the queue's offer() method to (sometimes) return false,
which causes the ThreadPoolExecutor to either spawn a new thread or reject the task, and
set the RejectedExecutionHandler to actually queue the task on rejection.
The problem is when offer() should return false. The currently accepted answer returns false when the queue has a couple of tasks on it, but as I've pointed out in my comment there, this causes undesirable effects. Alternately, if you always return false, you'll keep spawning new threads even when you have threads waiting on the queue.
The solution is to use Java 7 LinkedTransferQueue and have offer() call tryTransfer(). When there is a waiting consumer thread the task will just get passed to that thread. Otherwise, offer() will return false and the ThreadPoolExecutor will spawn a new thread.
BlockingQueue<Runnable> queue = new LinkedTransferQueue<Runnable>() {
#Override
public boolean offer(Runnable e) {
return tryTransfer(e);
}
};
ThreadPoolExecutor threadPool = new ThreadPoolExecutor(1, 50, 60, TimeUnit.SECONDS, queue);
threadPool.setRejectedExecutionHandler(new RejectedExecutionHandler() {
#Override
public void rejectedExecution(Runnable r, ThreadPoolExecutor executor) {
try {
executor.getQueue().put(r);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}
});
Note: I now prefer and recommend my other answer.
Here's a version which feels to me much more straightforward: Increase the corePoolSize (up to the limit of maximumPoolSize) whenever a new task is executed, then decrease the corePoolSize (down to the limit of the user specified "core pool size") whenever a task completes.
To put it another way, keep track of the number of running or enqueued tasks, and ensure that the corePoolSize is equal to the number of tasks as long as it is between the user specified "core pool size" and the maximumPoolSize.
public class GrowBeforeQueueThreadPoolExecutor extends ThreadPoolExecutor {
private int userSpecifiedCorePoolSize;
private int taskCount;
public GrowBeforeQueueThreadPoolExecutor(int corePoolSize, int maximumPoolSize, long keepAliveTime, TimeUnit unit, BlockingQueue<Runnable> workQueue) {
super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue);
userSpecifiedCorePoolSize = corePoolSize;
}
#Override
public void execute(Runnable runnable) {
synchronized (this) {
taskCount++;
setCorePoolSizeToTaskCountWithinBounds();
}
super.execute(runnable);
}
#Override
protected void afterExecute(Runnable runnable, Throwable throwable) {
super.afterExecute(runnable, throwable);
synchronized (this) {
taskCount--;
setCorePoolSizeToTaskCountWithinBounds();
}
}
private void setCorePoolSizeToTaskCountWithinBounds() {
int threads = taskCount;
if (threads < userSpecifiedCorePoolSize) threads = userSpecifiedCorePoolSize;
if (threads > getMaximumPoolSize()) threads = getMaximumPoolSize();
setCorePoolSize(threads);
}
}
As written the class doesn't support changing the user specified corePoolSize or maximumPoolSize after construction, and doesn't support manipulating the work queue directly or via remove() or purge().
We have a subclass of ThreadPoolExecutor that takes an additional creationThreshold and overrides execute.
public void execute(Runnable command) {
super.execute(command);
final int poolSize = getPoolSize();
if (poolSize < getMaximumPoolSize()) {
if (getQueue().size() > creationThreshold) {
synchronized (this) {
setCorePoolSize(poolSize + 1);
setCorePoolSize(poolSize);
}
}
}
}
maybe that helps too, but yours looks more artsy of course…
The recommended answer resolves only one (1) of the issue with the JDK thread pool:
JDK thread pools are biased towards queuing. So instead of spawning a new thread, they will queue the task. Only if the queue reaches its limit will the thread pool spawn a new thread.
Thread retirement does not happen when load lightens. For example if we have a burst of jobs hitting the pool that causes the pool to go to max, followed by light load of max 2 tasks at a time, the pool will use all threads to service the light load preventing thread retirement. (only 2 threads would be needed…)
Unhappy with the behavior above, I went ahead and implemented a pool to overcome the deficiencies above.
To resolve 2) Using Lifo scheduling resolves the issue. This idea was presented by Ben Maurer at ACM applicative 2015 conference:
Systems # Facebook scale
So a new implementation was born:
LifoThreadPoolExecutorSQP
So far this implementation improves async execution perfomance for ZEL.
The implementation is spin capable to reduce context switch overhead, yielding superior performance for certain use cases.
Hope it helps...
PS: JDK Fork Join Pool implement ExecutorService and works as a "normal" thread pool, Implementation is performant, It uses LIFO Thread scheduling, however there is no control over internal queue size, retirement timeout..., and most importantly tasks cannot be interrupted when canceling them
Note: I now prefer and recommend my other answer.
I have another proposal, following to the original idea of changing the queue to return false. In this one all tasks can enter the queue, but whenever a task is enqueued after execute(), we follow it with a sentinel no-op task which the queue rejects, causing a new thread to spawn, which will execute the no-op immediately followed by something from the queue.
Because worker threads may be polling the LinkedBlockingQueue for a new task, it's possible for a task to get enqueued even when there's an available thread. To avoid spawning new threads even when there are threads available, we need to keep track of how many threads are waiting for new tasks on the queue, and only spawn a new thread when there are more tasks on the queue than waiting threads.
final Runnable SENTINEL_NO_OP = new Runnable() { public void run() { } };
final AtomicInteger waitingThreads = new AtomicInteger(0);
BlockingQueue<Runnable> queue = new LinkedBlockingQueue<Runnable>() {
#Override
public boolean offer(Runnable e) {
// offer returning false will cause the executor to spawn a new thread
if (e == SENTINEL_NO_OP) return size() <= waitingThreads.get();
else return super.offer(e);
}
#Override
public Runnable poll(long timeout, TimeUnit unit) throws InterruptedException {
try {
waitingThreads.incrementAndGet();
return super.poll(timeout, unit);
} finally {
waitingThreads.decrementAndGet();
}
}
#Override
public Runnable take() throws InterruptedException {
try {
waitingThreads.incrementAndGet();
return super.take();
} finally {
waitingThreads.decrementAndGet();
}
}
};
ThreadPoolExecutor threadPool = new ThreadPoolExecutor(1, 50, 60, TimeUnit.SECONDS, queue) {
#Override
public void execute(Runnable command) {
super.execute(command);
if (getQueue().size() > waitingThreads.get()) super.execute(SENTINEL_NO_OP);
}
};
threadPool.setRejectedExecutionHandler(new RejectedExecutionHandler() {
#Override
public void rejectedExecution(Runnable r, ThreadPoolExecutor executor) {
if (r == SENTINEL_NO_OP) return;
else throw new RejectedExecutionException();
}
});
The best solution that I can think of is to extend.
ThreadPoolExecutor offers a few hook methods: beforeExecute and afterExecute. In your extension you could maintain use a bounded queue to feed in tasks and a second unbounded queue to handle overflow. When someone calls submit, you could attempt to place the request into the bounded queue. If you're met with an exception, you just stick the task in your overflow queue. You could then utilize the afterExecute hook to see if there is anything in the overflow queue after finishing a task. This way, the executor will take care of the stuff in it's bounded queue first, and automatically pull from this unbounded queue as time permits.
It seems like more work than your solution, but at least it doesn't involve giving queues unexpected behaviors. I also imagine that there's a better way to check the status of the queue and threads rather than relying on exceptions, which are fairly slow to throw.
Note: For JDK ThreadPoolExecutor when you have a bounded queue, you are only creating new threads when offer is returning false. You might obtain something usefull with CallerRunsPolicy which creates a bit of BackPressure and directly calls run in caller thread.
I need tasks to be executed from threads created by the pool and have an ubounded queue for scheduling, while the number of threads within the pool may grow or shrink between corePoolSize and maximumPoolSize so...
I ended up doing a full copy paste from ThreadPoolExecutor and change a bit the execute method because
unfortunately this could not be done by extension(it calls private methods).
I didn't wanted to spawn new threads just immediately when new request arrive and all threads are busy(because I have in general short lived tasks). I've added a threshold but feel free to change it to your needs ( maybe for mostly IO is better to remove this threshold)
private final AtomicInteger activeWorkers = new AtomicInteger(0);
private volatile double threshold = 0.7d;
protected void beforeExecute(Thread t, Runnable r) {
activeWorkers.incrementAndGet();
}
protected void afterExecute(Runnable r, Throwable t) {
activeWorkers.decrementAndGet();
}
public void execute(Runnable command) {
if (command == null)
throw new NullPointerException();
int c = ctl.get();
if (workerCountOf(c) < corePoolSize) {
if (addWorker(command, true))
return;
c = ctl.get();
}
if (isRunning(c) && this.workQueue.offer(command)) {
int recheck = this.ctl.get();
if (!isRunning(recheck) && this.remove(command)) {
this.reject(command);
} else if (workerCountOf(recheck) == 0) {
this.addWorker((Runnable) null, false);
}
//>>change start
else if (workerCountOf(recheck) < maximumPoolSize //
&& (activeWorkers.get() > workerCountOf(recheck) * threshold
|| workQueue.size() > workerCountOf(recheck) * threshold)) {
this.addWorker((Runnable) null, false);
}
//<<change end
} else if (!this.addWorker(command, false)) {
this.reject(command);
}
}
Below is a solution using two Threadpools both with core and max pool size as same. The second pool is used when the 1st pool is busy.
import java.util.concurrent.Future;
import java.util.concurrent.LinkedBlockingQueue;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;
public class MyExecutor {
ThreadPoolExecutor tex1, tex2;
public MyExecutor() {
tex1 = new ThreadPoolExecutor(15, 15, 5, TimeUnit.SECONDS, new LinkedBlockingQueue<>());
tex1.allowCoreThreadTimeOut(true);
tex2 = new ThreadPoolExecutor(45, 45, 100, TimeUnit.MILLISECONDS, new LinkedBlockingQueue<>());
tex2.allowCoreThreadTimeOut(true);
}
public Future<?> submit(Runnable task) {
ThreadPoolExecutor ex = tex1;
int excessTasks1 = tex1.getQueue().size() + tex1.getActiveCount() - tex1.getCorePoolSize();
if (excessTasks1 >= 0) {
int excessTasks2 = tex2.getQueue().size() + tex2.getActiveCount() - tex2.getCorePoolSize();;
if (excessTasks2 <= 0 || excessTasks2 / (double) tex2.getCorePoolSize() < excessTasks1 / (double) tex1.getCorePoolSize()) {
ex = tex2;
}
}
return ex.submit(task);
}
}
I want to understand logic of thread pool, and below there is a simple incorrect and not full implementation of it:
class ThreadPool {
private BlockingQueue<Runnable> taskQueue;
public ThreadPool(int numberOfThreads) {
taskQueue = new LinkedBlockingQueue<Runnable>(10);
for (int i = 0; i < numberOfThreads; i++) {
new PoolThread(taskQueue).start();
}
}
public void execute(Runnable task) throws InterruptedException {
taskQueue.put(task);
}
}
class PoolThread extends Thread {
private BlockingQueue<Runnable> taskQueue;
public PoolThread(BlockingQueue<Runnable> queue) {
taskQueue = queue;
}
public void run() {
while (true) {
try {
taskQueue.take().run();
} catch (Exception e) {
e.printStackTrace();
}
}
}
}
What if the number of threads to execute exceed the taskQueue size, will the calling thread be blocked?ThreadPoolExecutor - here we can see that in this case it's a work of rejected execution handler, but I still can not understand how does it work. Thanks in advance for any help.
EDIT:
set max size of blocking queue to 10
Imagine a group of bricklayers (your threads) building a wall, and a pile of bricks (your BlockingQueue).
Each bricklayer takes a brick from the pile, positions it, and then pick another one (taskQueue.take()) - until there are bricks in the pile, the bricklayers are kept busy.
A truck arrives from time to time, filling the pile with more bricks - but there is only a limited space on the pile, if there is no space the truck stops and wait until enough bricks have been used by the bricklayers.
As long there are enough bricks in the pile (more than the number of bricklayers) you can rest assured all bricklayers will have enough to work with - but when the pile start being empty the bricklayers will have to stop working until new bricks are delivered.
You have to pick a suitable number of bricklayers, to few and the truck will be often waiting for space in the pile, too many and most of them will be idle waiting for new bricks.
Implementation-wise, in general, Java gives you a threadpool, you rarely create your own -
ExecutorService threadExecutor = Executors.newFixedThreadPool( 3 );
and then you call:
threadExecutor.submit(Runnable...);
to add a task to the queue.
What if the number of threads to execute exceed the taskQueue size, will the calling thread be blocked?
The size of the queue is the number of tasks which are NOT running. Typically it will be empty even when the threads are busy. Having a queue length which matches the number of threads has no significance and nothing special happens at this point.
here we can see that in this case it's a work of rejected execution handler
The rejection handler is only called if the queue is full. Your queue has no limit so it wouldn't be called even if you supported this feature.
However, if it did have a limit and it supported this feature, the typical behaviour is to throw an exception. You can make it do other things such as block, have the current thread run the task (which is my preference) or ignore the task.
I still can not understand how does it work.
When you offer() a task to a queue, it return false if the queue could not accept it. When this happens call the rejected execution handler.
I would like to ask basic question about Java threads. Let's consider a producer - consumer scenario. Say there is one producer, and n consumer. Consumer arrive at random time, and once they are served they go away, meaning each consumer runs on its own thread. Should I still use run forever condition for consumer ?
public class Consumer extends Thread {
public void run() {
while (true) {
}
}
}
Won't this keep thread running forever ?
I wouldn't extend Thread, instead I would implement Runnable.
If you want the thread to run forever, I would have it loop forever.
A common alternative is to use
while(!Thread.currentThread().isInterrupted()) {
or
while(!Thread.interrupted()) {
It will, so you might want to do something like
while(beingServed)
{
//check if the customer is done being served (set beingServed to false)
}
This way you'll escaped the loop when it's meant to die.
Why not use a boolean that represents the presence of the Consumer?
public class Consumer extends Thread {
private volatile boolean present;
public Consumer() {
present = true;
}
public void run() {
while (present) {
// Do Stuff
}
}
public void consumerLeft() {
present = false;
}
}
First, you can create for each consumer and after the consumer will finish it's job than the consumer will finish the run function and will die, so no need for infinite loop. however, creating thread for each consumer is not good idea since creation of thread is quite expensive in performance point of view. threads are very expensive resources. In addition, i agree with the answers above that it is better to implement runnable and not to extends thread. extend thread only when you wish to customize your thread.
I strongly suggest you will use thread pool and the consumer will be the runnable object that ran by the thread in the thread pool.
the code should look like this:
public class ConsumerMgr{
int poolSize = 2;
int maxPoolSize = 2;
long keepAliveTime = 10;
ThreadPoolExecutor threadPool = null;
final ArrayBlockingQueue<Runnable> queue = new ArrayBlockingQueue<Runnable>(
5);
public ConsumerMgr()
{
threadPool = new ThreadPoolExecutor(poolSize, maxPoolSize,
keepAliveTime, TimeUnit.SECONDS, queue);
}
public void runTask(Runnable task)
{
// System.out.println("Task count.."+threadPool.getTaskCount() );
// System.out.println("Queue Size before assigning the
// task.."+queue.size() );
threadPool.execute(task);
// System.out.println("Queue Size after assigning the
// task.."+queue.size() );
// System.out.println("Pool Size after assigning the
// task.."+threadPool.getActiveCount() );
// System.out.println("Task count.."+threadPool.getTaskCount() );
System.out.println("Task count.." + queue.size());
}
It is not a good idea to extend Thread (unless you are coding a new kind of thread - ie never).
The best approach is to pass a Runnable to the Thread's constructor, like this:
public class Consumer implements Runnable {
public void run() {
while (true) {
// Do something
}
}
}
new Thread(new Consumer()).start();
In general, while(true) is OK, but you have to handle being interrupted, either by normal wake or by spurious wakeup. There are many examples out there on the web.
I recommend reading Java Concurrency in Practice.
for producer-consumer pattern you better use wait() and notify(). See this tutorial. This is far more efficient than using while(true) loop.
If you want your thread to processes messages until you kill them (or they are killed in some way) inside while (true) there would be some synchronized call to your producer thread (or SynchronizedQueue, or queuing system) which would block until a message becomes available. Once a message is consumed, the loop restarts and waits again.
If you want to manually instantiate a bunch of thread which pull a message from a producer just once then die, don't use while (true).