Develop Reference

How to properly shutdown java ExecutorService

I have a simple java ExecutorService that runs some task objects (implements Callable).
ExecutorService exec = Executors.newSingleThreadExecutor();
List<CallableTask> tasks = new ArrayList<>();
// ... create some tasks
for (CallableTask task : tasks) {
Future future = exec.submit(task);
result = (String) future.get(timeout, TimeUnit.SECONDS);
// TASKS load some classes and invoke their methods (they may create additional threads)
// ... catch interruptions and timeouts
}
exec.shutdownNow();
After all tasks are finished (either DONE or TIMEOUT-ed), I try to shutdown the executor, but it wont stop: exec.isTerminated() = FALSE.
I suspect that some tasks that are timeouted are not properly terminated.
And yes, I know that executor's shutdown is not guaranteing anything:
There are no guarantees beyond best-effort attempts to stop
processing actively executing tasks. For example, typical
implementations will cancel via {#link Thread#interrupt}, so any
task that fails to respond to interrupts may never terminate.
My question is, is there a way to ensure those (task) threads will terminate?
The best solution I came up with, is to call the System.exit() at the end of my program, but that is plain silly.

Recommended way from Oracle API documentation page of ExecutorService:
void shutdownAndAwaitTermination(ExecutorService pool) {
pool.shutdown(); // Disable new tasks from being submitted
try {
// Wait a while for existing tasks to terminate
if (!pool.awaitTermination(60, TimeUnit.SECONDS)) {
pool.shutdownNow(); // Cancel currently executing tasks
// Wait a while for tasks to respond to being cancelled
if (!pool.awaitTermination(60, TimeUnit.SECONDS))
System.err.println("Pool did not terminate");
}
} catch (InterruptedException ie) {
// (Re-)Cancel if current thread also interrupted
pool.shutdownNow();
// Preserve interrupt status
Thread.currentThread().interrupt();
}
If your pool is taking more time to shutdown, you can change
if (!pool.awaitTermination(60, TimeUnit.SECONDS))
to
while (!pool.awaitTermination(60, TimeUnit.SECONDS))
A brief summary of the shutdown related methods
shutdown():
Initiates an orderly shutdown in which previously submitted tasks are executed, but no new tasks will be accepted.
shutdownNow():
Attempts to stop all actively executing tasks, halts the processing of waiting tasks, and returns a list of the tasks that were awaiting execution.
awaitTermination(long timeout, TimeUnit unit) throws InterruptedException:
Blocks until all tasks have completed execution after a shutdown request, or the timeout occurs, or the current thread is interrupted, whichever happens first.

Do you have control of those tasks ? i.e. are you creating those yourself ? I suspect somewhere in those a thread interrupt is being ignored e.g.
try {
....
}
catch {InterruptedException e) {
// do nothing
}
When an InterruptedException is thrown, the interrupt flag on the thread needs to be reset, otherwise the thread will not exit. See here for more info.
Unfortunately you may be using a library that doesn't obey this, in which case you can't easily circumvent this. In that case one heavyweight option is to fork off a sub process to perform the job of the Callable, and that will clear up all resources upon process exit. Heavyweight and possibly non-trivial, but reliable.

Is there a way to inadvertently stop the threads of an Executor?

I tried crashing the Thread of a fixed thread pool of a standard ExecutorService with this horrible code, in order to test its resilience against exceptions and errors happening inside the tasks passed to the ExecutorService:
ExecutorService executorService = Executors.newFixedThreadPool(1);
executorService.execute(() -> {
throw new RuntimeException("Trying to kill the thread");
});
Thread.sleep(100);
executorService.execute(() -> {
System.out.println("still executing");
});
but the second task prints still executing so I think the exception is being caught or the Thread instance is restarted anyway.
What is happening? Can I trust that no matter what happens inside a task (less than System.exit()) other submitted tasks will continue to be executed by the pool?

Your executor service uses a thread pool. If a thread is killed, one another will be instantiated to execute next task.
Your call Executors.newFixedThreadPool(1); set a limit on number of active threads only.
In order to stop executor service, you must call:
executorService.shutdown();

Difference between shutdown and shutdownNow of Executor Service

I want to know the basic difference between shutdown() and shutdownNow() for shutting down the Executor Service?
As far as I understood:
shutdown() should be used for graceful shutdown which means all tasks that were running and queued for processing but not started should be allowed to complete
shutdownNow() does an abrupt shut down meaning that some unfinished tasks are cancelled and unstarted tasks are also cancelled. Is there anything else which is implicit/explicit that I am missing?
P.S: I found another question on How to shutdown an executor service related to this but not exactly what I want to know.

In summary, you can think of it that way:
shutdown() will just tell the executor service that it can't accept new tasks, but the already submitted tasks continue to run
shutdownNow() will do the same AND will try to cancel the already submitted tasks by interrupting the relevant threads. Note that if your tasks ignore the interruption, shutdownNow will behave exactly the same way as shutdown.
You can try the example below and replace shutdown by shutdownNow to better understand the different paths of execution:
with shutdown, the output is Still waiting after 100ms: calling System.exit(0)... because the running task is not interrupted and continues to run.
with shutdownNow, the output is interrupted and Exiting normally... because the running task is interrupted, catches the interruption and then stops what it is doing (breaks the while loop).
with shutdownNow, if you comment out the lines within the while loop, you will get Still waiting after 100ms: calling System.exit(0)... because the interruption is not handled by the running task any longer.
public static void main(String[] args) throws InterruptedException {
ExecutorService executor = Executors.newFixedThreadPool(1);
executor.submit(new Runnable() {
#Override
public void run() {
while (true) {
if (Thread.currentThread().isInterrupted()) {
System.out.println("interrupted");
break;
}
}
}
});
executor.shutdown();
if (!executor.awaitTermination(100, TimeUnit.MICROSECONDS)) {
System.out.println("Still waiting after 100ms: calling System.exit(0)...");
System.exit(0);
}
System.out.println("Exiting normally...");
}

shutdown():
To terminate the threads inside the ExecutorService you call its shutdown() method. The ExecutorService will not shut down immediately, but it will no longer accept new tasks, and once all threads have finished current tasks, the ExecutorService shuts down. All tasks submitted to the ExecutorService before shutdown() is called, are executed.
shutdownNow():
If you want to shut down the ExecutorService immediately, you can call the shutdownNow() method. This will attempt to stop all executing tasks right away, and skips all submitted but non-processed tasks. There are no guarantees given about the executing tasks. Perhaps they stop, perhaps the execute until the end. It is a best effort attempt.

From the javadocs:
void shutdown
Initiates an orderly shutdown in which previously submitted tasks are
executed, but no new tasks will be accepted.
List<Runnable> shutdownNow()
Attempts to stop all actively executing tasks, halts the processing of
waiting tasks, and returns a list of the tasks that were awaiting
execution.
There are no guarantees beyond best-effort attempts to stop
processing actively executing tasks.
For example, typical implementations will cancel via
Thread.interrupt(), so any task that fails to respond to interrupts
may never terminate.
Returns: list of tasks that never commenced execution

How to interrupt ExecutorService's threads

When using the ExecutorService returned by Executors.newSingleThreadExecutor(), how do I interrupt it?

In order to do this, you need to submit() a task to an ExecutorService, rather than calling execute(). When you do this, a Future is returned that can be used to manipulate the scheduled task. In particular, you can call cancel(true) on the associated Future to interrupt a task that is currently executing (or skip execution altogether if the task hasn't started running yet).
By the way, the object returned by Executors.newSingleThreadExecutor() is actually an ExecutorService.

Another way to interrupt the executor's internally managed thread(s) is to call the shutdownNow(..) method on your ExecutorService. Note, however, that as opposed to #erickson's solution, this will result in the whole ThreadPoolExecutor becoming unfit for further use.
I find this approach particularly useful in cases where the ExecutorService is no longer needed and keeping tabs on the Future instances is otherwise unnecessary (a prime example of this being the exit(..) method of your application).
Relevant information from the ExecutorService#shutdownNow(..) javadocs:
Attempts to stop all actively executing tasks, halts the processing of
waiting tasks, and returns a list of the tasks that were awaiting
execution.
There are no guarantees beyond best-effort attempts to stop processing
actively executing tasks. For example, typical implementations will
cancel via Thread.interrupt, so any task that fails to respond to
interrupts may never terminate.

One proper way could be customizing/injecting the ThreadFactory for the ExecutorService and from within the thread factory, you got the handle of the thread created, then you can schedule some task to interrupt the thread being interested.
Demo code part for the overwrited method newThread in ThreadFactory:
ThreadFactory customThreadfactory new ThreadFactory() {
public Thread newThread(Runnable runnable) {
final Thread thread = new Thread(runnable);
if (namePrefix != null) {
thread.setName(namePrefix + "-" + count.getAndIncrement());
}
if (daemon != null) {
thread.setDaemon(daemon);
}
if (priority != null) {
thread.setPriority(priority);
}
scheduledExecutorService.schedule(new Callable<String>() {
public String call() throws Exception {
System.out.println("Executed!");
thread.interrupt();
return "Called!";
}
}, 5, TimeUnit.SECONDS);
return thread;
}
}
Then you can use below code to construct your ExecutorService instance:
ExecutorService executorService = Executors.newFixedThreadPool(3,
customThreadfactory);
Then after 5 seconds, an interrupt signal will be sent to the threads in ExecutorService.

Time limit on individual threads with ExecutorService

I have an ExecutorService managing a number of Callables. The tasks that the Callables run are mostly black box transformations and number crunching. Under certain conditions, the data being transformed will oscillate and the thread will take over an hour to finish. For comparison, most threads are completed within a minute.
It's been deteremined that the data from the long-running threads is not relevent. I would like to interrupt any thread that runs longer than a certain amount of time. What would the best way to do this?

Use a ScheduleExecutorService to schedule a task to taskFuture.cancel(true) the long running task when the timeout is reached. If the task finishes before then it won't be cancelled.
ExecutorService service = Executors.newFixedThreadPool(N);
ScheduledExecutorService canceller = Executors.newSingleThreadScheduledExecutor();
public <T> Future<T> executeTask(Callable<T> c, long timeoutMS){
final Future<T> future = service.submit(c);
canceller.schedule(new Callable<Void>(){
public Void call(){
future.cancel(true);
return null;
}
}, timeoutMS, TimeUnit.MILLI_SECONDS);
return future;
}

You could cancel the future etc as in the other answers, but you need to make sure that your threads which are "number crunching" can handle the interrupt and terminate gracefully. You say that this a black box operation - how certain are you that the interrupted status of the thread is being checked actively within the black box? If it isn't, you can't cancel it with an interrupt. The black box needs to be written with interruption in mind.

The best way for you to do this would be to introduce one more Executor. You can use a ScheduledExecutorService to cancel all long running tasks for example:
ExecutorService service = Executors.newFixedThreadPool(N);
ScheduledExecutorService canceller = Executors.newScheduledThreadPool(1);
public void executeTask(Callable<?> c){
final Future<?> future = service.submit(c);
canceller.schedule(new Runnable(){
public void run(){
future.cancel(true);
}
}, SECONDS_UNTIL_TIMEOUT, TimeUnit.SECONDS);
}

You could get a list of your corresponding Futures (that are created when you submit a Callable) together with its startup time.
Another task could then check every minute if there are some task running for more than a defined time and if so, invoke cancel(true) in the Future. Done futures would be removed off the list.

You can use this method
<T> List<Future<T>> invokeAll(Collection<? extends Callable<T>> tasks,
long timeout,
TimeUnit unit)
throws InterruptedException
and set the maximum timeout to one minute. If your thread takes more than that it is just aborted.

The problem is you can not kill/stop/suspend the java thread.
All are deprecated.
future.cancel will only considered before worker thread
So if the thread is stuck with any code execution and doesn't have the logic to consider the thread interruption, then execution will still continue.
This is even true for executorService.shutdown();.
The documentation says.
There are no guarantees beyond best-effort attempts to stop processing
actively executing tasks. For example, typical implementations will
cancel via Thread. interrupt, so any task that fails to respond to
interrupts may never terminate.
So the only solution is to log such a situation and fix the inherent issue.
Future<?> future = executor.submit(task);
try {
future.get(15, TimeUnit.MINUTES);
} catch (InterruptedException e) {
// retry waiting. iterative approach not shown here
} catch (ExecutionException e) {
// your task exploded
} catch (TimeoutException e) {
// The task is running longer than usual
// Log it for future analysis
//If the interrupt signal is handled
//OR the task is stuck in the queue itself for a long time
//then worth calling below
future.cancel(true);
}