Develop Reference

ThreadPoolTaskExecutor should Finish Task Queue before Main thread ends

Problem statement:
I have 1,000 tasks and need to process them via ThreadPoolTaskExecutor. ThreadPoolTaskExecutor has corePoolSize = 5, maxPoolSize = 10 and queueCapacity = 1000.
Now from the main method, I am executing the following code
CountDownLatch latch = new CountDownLatch(5);
Collection<Future<?>> futures = new LinkedList<Future<?>>();
for (Map.Entry<String,Boolean> entry : map.entrySet()){
FutureTask task = new FutureTask(new CustomTask(entry));
executor.execute(task);
}
log.info("ACTIVE COUNT : "+executor.getActiveCount());
log.info("SIZE of the QUEUE : "+executor.getThreadPoolExecutor().getQueue().size());
log.info("LATCH WAIT : "+latch.getCount());
latch.wait();
.....
#Override
public Object call() throws Exception {
latch.countDown();
//some logic
return entry;
}
Now, the map has 1,000 entries in it and I want to process all tasks in queue(1,000) and then print these log lines. Whats happening here is, the corePoolSize(which is equal to the CountDounLatch count) create this number of thread and executes them 'Right-Away'. However, when this number is hit, it starts filling up the queue(which is totally fine and desired). However, this queue tasks are processed ONLY AFTER the main thread reaches the end, only then these tasks start executing. This is something that I don't want. I want the Executor to start picking up items from queue as soon as threads get free from processing batch-1.
But in my case, once the batch-1 is processed, the next task is picked only when the main threads ends(which I do not want).
Anyone with a solution on how can this be achieved? (The processing of queue as soon as the thread is available for processing)
P.S : I do understand that latch.await() waits for the threads to complete their execution, but I am looking for a behavior in which it should wait for all the threads to be finished(which is happening) and all the queue should be empty(my expectations).
Thank You

If you are going to do it this way, you need to initialize the latch with the number of tasks that you are going to submit; i.e. 1,000. Also you should decrement the latch at the end of each task, not at its start (as your code currently seems to be doing.)
But you don't need a latch or a counter or anything to implement this. Instead, if you are using a Java SE ExecutorService directly, just do this:
public static void main(String[] args) {
// Submit lots of tasks
executorService.shutdown();
try {
// Waits until all tasks in the queue have completed
executorService.awaitTermination(1_000_000, TimeUnit.SECONDS);
} catch (InterruptedException ex) {
// OK ... will end now
}
}
And if you are using the SpringFramework specific ThreadPoolTaskExecutor class:
public static void main(String[] args) {
// Submit lots of tasks
executor.setAwaitTerminationSeconds(1_000_000);
executor.setWaitForTasksToCompleteOnShutdown(true);
executor.shutdown();
}

How does newCachedThreadPool reuse threads?

I want to ask for a little more detail to the same question posted by Zeller over a year ago...
The javadoc says that the service returned by Executors.newCachedThreadPool reuses threads. How is this possible?
I get how the queue structure is setup internally, what I don't see is how it reuses threads in the queue.
All examples I've seen have the developer create an instance of their thread and pass it in through the "execute" method.
For example...
ExecutorService executor = Executors.newCachedThreadPool();
for (int i = 0; i < 10; i++) {
Runnable worker = new WorkerThread(i); //will create 10 instances
executor.execute(worker);
}
I understand that a thread pool can easily manage the life cycle of each thread, but again, I see no methods nor the ability to access or restart any of the threads in the pool.
In the above example, I would then expect that each thread would be started, run, terminated and disposed of by the thread pool, but never reused.
A messaging system would be an example of where you'd need this. Say you have an onMessage handler and you'd like to reuse one of the threads in the pool to handle it, so I'd expect methods like...
worker = executor.getIdleThread;
worker.setData(message);
executor.resubmit(worker);
or maybe have the ExecutorService acting as a factory class and have it return an instance of your threads, where internally it decides to create a new one or reuse an old one.
ExecutorService executor = Executors.newCachedThreadPool(WorkerThread);
Runnable worker = executor.getThread;
worker.setData(message);
So I'm missing something. It's probably something simple but I've spent the afternoon reading tutorials and examples and still haven't figured it out. Can someone shed some light on the subject?

I was curious too how this was possible since Threads can't be restarted, so I analyzed the code of ThreadPoolExecutor which is the implementation of all the ThreadPool ExecutorService you get through the static constructor.
First of all as stated in the other answer you don't use Threads, but Runnables in ThreadPools, because that would defeat the purpose. So here is a detailed explaination how an ExecutorService reuses Threads:
You usually add a Runnable through submit() which internally calls the execute() method. Basically this adds the runnable to a queue and adds a Worker if none is working ATM
public void execute(Runnable command) {
...
int c = ctl.get();
if (workerCountOf(c) < corePoolSize) {
if (addWorker(command, true))
return;
c = ctl.get();
}
if (isRunning(c) && workQueue.offer(command)) {
int recheck = ctl.get();
if (! isRunning(recheck) && remove(command))
reject(command);
else if (workerCountOf(recheck) == 0)
addWorker(null, false);
}
else if (!addWorker(command, false))
reje
ct(command);
}
The executor maintains a bunch of Worker (inner Class of ThreadPoolExecutor). It has your submitted runnable and a Thread that will be created through the ThreadFactory you maybe set or else just a default one; also the Worker itself is a Runnable, which is used to create the Thread from the factory
private final class Worker
extends AbstractQueuedSynchronizer
implements Runnable
{
...
Worker(Runnable firstTask) {
this.firstTask = firstTask;
this.thread = getThreadFactory().newThread(this);
}
public void run() {
runWorker(this);
}
...
}
When adding a Worker it gets kickstarted
private boolean addWorker(Runnable firstTask, boolean core) {
...
Worker w = new Worker(firstTask);
Thread t = w.thread;
...
t.start();
...
return true;
}
the runWorker() method runs in a loop and get with getTask() the runnables you submitted which are queued in the workingQueue and will wait at getTask unitl a timeout happens.
final void runWorker(Worker w) {
Runnable task = w.firstTask;
w.firstTask = null;
boolean completedAbruptly = true;
try {
while (task != null || (task = getTask()) != null) {
w.lock();
clearInterruptsForTaskRun();
try {
beforeExecute(w.thread, task);
Throwable thrown = null;
try {
task.run();
} catch (RuntimeException x) {
...
} finally {
afterExecute(task, thrown);
}
} finally {
task = null;
w.completedTasks++;
w.unlock();
}
}
completedAbruptly = false;
} finally {
processWorkerExit(w, completedAbruptly);
}
}
Here is the getTask() method
private Runnable getTask() {
...
try {
Runnable r = timed ?
workQueue.poll(keepAliveTime, TimeUnit.NANOSECONDS) :
workQueue.take();
..
} catch (InterruptedException retry) {
...
}
}
}
tl;dr So basically the Threadpool maintains worker threads that run in loops and execute runnables given by a blocking queue. The worker will be created and destroyed due to demand (no more tasks, a worker will end; if no free worker and < maxPoolSize then create new worker). Also I wouldn't call it "reuse" more the thread will be used as a looper to execute all runnables.

I understand that a thread pool can easily manage the life cycle of
each thread, but again, I see no methods nor the ability to access or
restart any of the threads in the pool.
The management of threads is done internally. The ExecutorService interface only provides the externally visible methods.
The javadoc of newCachedThreadPool simply states
Creates a thread pool that creates new threads as needed, but will
reuse previously constructed threads when they are available. [...] Calls to execute will reuse
previously constructed threads if available. If no existing thread is
available, a new thread will be created and added to the pool. Threads
that have not been used for sixty seconds are terminated and removed
from the cache. Thus, a pool that remains idle for long enough will
not consume any resources. [...]
So that is the guarantee you get. If you want to know how it is implemented, you can look at the source code, in particular, the code of ThreadPoolExecutor. Basically, idle threads will terminate after not executing a task after some time.

Executor Service and scheduleWithFixedDelay()

Here is my task. I have a static queue of jobs in a class and a static method that adds jobs to the queue. Have n amount of threads that poll from a queue and perform the pulled job. I need to have the n threads poll simultaneously at an interval. AKA, all 3 should poll every 5 seconds and look for jobs.
I have this:
public class Handler {
private static final Queue<Job> queue = new LinkedList<>();
public static void initialize(int maxThreads) { // maxThreads == 3
ScheduledExecutorService executorService =
Executors.newScheduledThreadPool(maxThreads);
executorService.scheduleWithFixedDelay(new Runnable() {
#Override
public void run() {
Job job = null;
synchronized(queue) {
if(queue.size() > 0) {
job = queue.poll();
}
}
if(job != null) {
Log.log("start job");
doJob(job);
Log.log("end job");
}
}
}, 15, 5, TimeUnit.SECONDS);
}
}
I get this output when I add 4 tasks:
startjob
endjob
startjob
endjob
startjob
endjob
startjob
endjob
It is obvious that these threads perform that jobs serially, whereas I need them to be done 3 at a time. What am I doing wrong? Thanks!

From the documentation:
If any execution of this task takes longer than its period, then subsequent executions may start late, but will not concurrently execute.
So you must schedule three independent tasks to have them run concurrently. Also note that the scheduled executor service is a fixed thread pool, which is not flexible enough for many use cases. A good idiom is to use the scheduled service just to submit tasks to a regular executor service, which may be configured as a resizable thread pool.

You are running ScheduledExecutorService with fixed delay, what means, that your jobs will run one after one. Use fixed thread pool, and submit 3 threads at a time. Here is an explanation with examples

If you declare Job extends Runnable then your code simplifies dramatically:
First declare the Executor somewhere globally accessible:
public static final ExecutorService executor = Executors.newFixedThreadPool(MAX_THREADS);
Then add a job like this:
executor.submit(new Job());
You are done.

How can we save a thread for next task after its execution in java (Implementation of Thread pooling )

I need to ask about how thread pooling is implemented for having constant number of thread executing each time when there is task submission happened . (In Executor to avoid each time thread creation and deletion overhead)
executor.submit(Runnable)
Lets say we create some threads in the start and when task come we assign task to them(Thread) using any Queue impl . But after completing it s task how could a thread return to its pool again when as per the lifecycle of thread says that
"After execution of its run method it goes into TERMINATED state and can't be used again"
I am not understood how thread pool works for having constant number of threads for execution of any task to its queue .
It would be great if anyone could provide me an example of thread reuse after its completion of task .
!!Thanks in advance .!!

"After execution of its run method it goes into TERMINATED state and can't be used again"
It doesn't finish its run() Instead it has a loop which runs the run() of the tasks you provide it.
Simplifying the thread pool pattern dramatically you have code which looks like this.
final BlockingQueue<Runnable> tasks = new LinkedBlockingQueue<Runnable>();
public void submit(Runnable runs) {
tasks.add(runs);
}
volatile boolean running = true;
// running in each thread in the pool
class RunsRunnable implement Runnable {
public void run() {
while(running) {
Runnable runs = tasks.take();
try {
runs.run();
} catch(Throwable t) {
// handles t
}
}
}
}
In this example, you can see that while the run() of each task completes, the run() of the thread itself does not until the pool is shutdown.

Usually what happens when we use thread pool , Its inside Run method it is forced to run iteratively. Until there are tasks available in the Queue.
in the below example pool.removeFromQueue() will run iteratively.
public class MyThread<V> extends Thread {
private MyThreadPool<V> pool;
private boolean active = true;
public boolean isActive() {
return active;
}
public void setPool(MyThreadPool<V> p) {
pool = p;
}
/**
* Checks if there are any unfinished tasks left. if there are , then runs
* the task and call back with output on resultListner Waits if there are no
* tasks available to run If shutDown is called on MyThreadPool, all waiting
* threads will exit and all running threads will exit after finishing the
* task
*/
#Override
public void run() {
ResultListener<V> result = pool.getResultListener();
Callable<V> task;
while (true) {
task = pool.removeFromQueue();
if (task != null) {
try {
V output = task.call();
result.finish(output);
} catch (Exception e) {
result.error(e);
}
} else {
if (!isActive())
break;
else {
synchronized (pool.getWaitLock()) {
try {
pool.getWaitLock().wait();
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
}
}
}
void shutdown() {
active = false;
}
Need to design your thread pool
public MyThreadPool(int size, ResultListener<V> myResultListener) {
tasks = new LinkedList<Callable<V>>();
threads = new LinkedList<MyThread<V>>();
shutDown = false;
resultListener = myResultListener;
for (int i = 0; i < size; i++) {
MyThread<V> myThread = new MyThread<V>();
myThread.setPool(this);
threads.add(myThread);
myThread.start();
}
}

You can take a look here: http://www.ibm.com/developerworks/library/j-jtp0730/index.html for more details and an implementation example. The threads in the pool will wait if the queue is empty and will each start consome messages once they are notified that the queue has some elements.

ExecutorService executor = Executors.newFixedThreadPool(2);
- The above statement created a ThreadPool with fixed size of 2.
executor.execute(new Worker());
- The above statement takes an instance of the class Worker which has implemented Runnable Interface.
- Now here the Executors is an intermediate object, executing the task. Which manages the Thread Objects.
- By executing the above statement the run() method will be executed, and once the run() method completes, the thread doesNot go into dead state but moves back into the pool, waiting to have another work assigned to it, so it can once again move into Runnable state and then to running, all this is handled by Executors .
executor.shutdown();
- The above statement will shutdown the Executors itself, gracefully handling the shutdown of all the threads managed by it..shutdown() on that central object, which in turn could terminate each of the registered executors.
////////// Edited Part//////////////////////
- First of all Runnable has a run() method which canNot return anything, and run() method canNot throw a checked exception, So Callable was introduced in Java 5, which is of Parametric type , and has a method called call(), and it is capable of returning , and throwing Checked exceptions.
Now see this Example:
Thread t = new Thread(new Worker());
t.run();
t.start();
- t.run() is just a simple call to run() method, this won't span a thread of execution.
- t.start() whereas prepares for the things important for the initialization of the thread of execution, and then calls the run() method of the Runnable, and then assign the Task to the newly formed thread of execution, and returns quickly....
Threads in Java becomes a necessity when using Swing and AWT. Mainly the GUI component.

I am totally agree with Peter but want add steps related to ExecutorService execution flow, for clear understanding.
If you create pool (fixed size pool) of threads it does not means that threads were created.
If you submit and/or execute new Task (Runnuble or Callable) new thread will be created JUTS if count of created threads < size of pool
Created threads not returning to pool, threads can wait for new value in blocking queue, this point we can call RETURNING TO POOL
All threads from pool execs like Peter described above.

How to implement an ExecutorService to execute batches of tasks

I am looking for a way to execute batches of tasks in java. The idea is to have an ExecutorService based on a thread pool that will allow me to spread a set of Callable among different threads from a main thread. This class should provide a waitForCompletion method that will put the main thread to sleep until all tasks are executed. Then the main thread should be awaken, and it will perform some operations and resubmit a set of tasks.
This process will be repeated numerous times, so I would like to use ExecutorService.shutdown as this would require to create multiple instances of ExecutorService.
Currently I have implemented it in the following way using a AtomicInteger, and a Lock/Condition:
public class BatchThreadPoolExecutor extends ThreadPoolExecutor {
private final AtomicInteger mActiveCount;
private final Lock mLock;
private final Condition mCondition;
public <C extends Callable<V>, V> Map<C, Future<V>> submitBatch(Collection<C> batch){
...
for(C task : batch){
submit(task);
mActiveCount.incrementAndGet();
}
}
#Override
protected void afterExecute(Runnable r, Throwable t) {
super.afterExecute(r, t);
mLock.lock();
if (mActiveCount.decrementAndGet() == 0) {
mCondition.signalAll();
}
mLock.unlock();
}
public void awaitBatchCompletion() throws InterruptedException {
...
// Lock and wait until there is no active task
mLock.lock();
while (mActiveCount.get() > 0) {
try {
mCondition.await();
} catch (InterruptedException e) {
mLock.unlock();
throw e;
}
}
mLock.unlock();
}
}
Please not that I will not necessarily submit all the tasks from the batch at once, therefore CountDownLatch does not seem to be an option.
Is this a valid way to do it? Is there a more efficient/elegant way to implement that?
Thanks

I think the ExecutorService itself will be able to perform your requirements.
Call invokeAll([...]) and iterate over all of your Tasks. All Tasks are finished, if you can iterate through all Futures.

As the other answers point out, there doesn't seem to be any part of your use case that requires a custom ExecutorService.
It seems to me that all you need to do is submit a batch, wait for them all to finish while ignoring interrupts on the main thread, then submit another batch perhaps based on the results of the first batch. I believe this is just a matter of:
ExecutorService service = ...;
Collection<Future> futures = new HashSet<Future>();
for (Callable callable : tasks) {
Future future = service.submit(callable);
futures.add(future);
}
for(Future future : futures) {
try {
future.get();
} catch (InterruptedException e) {
// Figure out if the interruption means we should stop.
}
}
// Use the results of futures to figure out a new batch of tasks.
// Repeat the process with the same ExecutorService.

I agree with #ckuetbach that the default Java Executors should provide you with all of the functionality you need to execute a "batch" of jobs.
If I were you I would just submit a bunch of jobs, wait for them to finish with the ExecutorService.awaitTermination() and then just start up a new ExecutorService. Doing this to save on "thread creations" is premature optimization unless you are doing this 100s of times a second or something.
If you really are stuck on using the same ExecutorService for each of the batches then you can allocate a ThreadPoolExecutor yourself, and be in a loop looking at ThreadPoolExecutor.getActiveCount(). Something like:
BlockingQueue jobQueue = new LinkedBlockingQueue<Runnable>();
ThreadPoolExecutor executor = new ThreadPoolExecutor(NUM_THREADS, NUM_THREADS,
0L, TimeUnit.MILLISECONDS, jobQueue);
// submit your batch of jobs ...
// need to wait a bit for the jobs to start
Thread.sleep(100);
while (executor.getActiveCount() > 0 && jobQueue.size() > 0) {
// to slow the spin
Thread.sleep(1000);
}
// continue on to submit the next batch