Develop Reference

Cleaning up thread submitted using ExecutorService

My code is as follows:
public Future<String> getFuture() {
ExecutorService executorService = Executors.newSingleThreadExecutor();
Future<String> future = executorService.submit(() -> {
//do something
return "test string";
});
executorService.shutDown(); // is this correct?
return future;
}
I am calling this service from other class to get the future:
Future<String> future = getFuture();
String result = future.get();
future.cancel(true); // will this assure that there wont be any thread leak?
Now out of executorService.shutDown() and future.cancel(true) which will assure that there wont be thread leaks?
Note that after calling future.cancel(true) when I check currently running threads in the result of Thread.getAllStackTraces() I can still find the thread where future executed.

You are asking the wrong question!
There is no point in creating a service within a method to then throw it away right there.
Creating that service instance doesn't come for free. The whole idea of this abstraction is to ensure to make efficient usage of infrastructure elements!
In other words: step back and rework your design; so that this service becomes a field of some class for example! And yes, that might turn out to be complicated. But most likely, spending time in that corner will pay out much more long term - compared to continuing the approach shown in your question.

It is a bad idea to create an executor then throw it away in each method call.
Now out of executorService.shutDown() and future.cancel(true) which will assure that there wont be thread leaks?
none of them.
executorService.shutdown() will just keep running the current tasks and reject new submitted tasks.
future.cancel(true) will interrupt the corresponding task if it is currently running (but it is your responsability to check if the task was interrupted and finish the execution of the task as soon as possible)
Note that after calling future.cancel(true) when I check currently running threads in the result of Thread.getAllStackTraces() I can still find the thread where future executed.
as I mentioned before, future.cancel(true) doesn't stop the thread. it only sends an interruption.

When the executorService.shutdown(); should be called

We have a service method GetDataParallel( ) which maybe called by many clients currently, and we use the ExecutorService to called the MyCallable inside it. However I found unless I called the executorService.shutdown(); the application never exit, so why the application cannot exit , we must shut down all thread pool threads manually before the application exit? and in service environment I think we don't need to call the executorService.shutdown(); to keep the application alive, right ?
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
public class MultiThreading {
static ExecutorService executorService = Executors.newFixedThreadPool(100);
private List<String> _BusinessUnits= new ArrayList<String>();
public static void main(String[] args) throws Exception {
MultiThreading kl =new MultiThreading();
kl.GetDataParallel();
Thread.sleep(10000);
System.out.println("111111111");
//executorService.shutdown();
}
public void GetDataParallel( ) throws Exception
{
_BusinessUnits.add("BU1");
_BusinessUnits.add("BU2");
_BusinessUnits.add("BU3");
for(final String v : _BusinessUnits)
{
ExecutorServiceTest.executorService.submit( new MyCallable());
}
}
}
class MyCallable implements Callable {
#Override
public String call() throws Exception {
Thread.sleep(1000);
//return the thread name executing this callable task
System.out.println(Thread.currentThread().getName());
return Thread.currentThread().getName();
}
}

Typically you shut down an ExecutorService when the application is exiting - most applications have some sort of lifecycle and shutdown sequence. If you expect your application to exit when the main thread completes whatever it has to do, you want to shut it down when its work is done (meaning there's a well-defined point at which it's work is done, and you can tell when that is).
Server-side frameworks frequently have lifecycle methods that you can hook into to detect when your code is being shut down and cleanly exit. Or you can use VM shutdown hooks (perhaps to delay shutdown until all currently queued jobs are complete), so that no matter what code causes the program to exit, your cleanup code will be run.
In general, it's a good idea to create a well-defined exit point if there isn't one provided by a framework - it lets you have an application that can be cleanly unloaded (and perhaps, reloaded with updated configuration) without the VM necessarily shutting down at all - I've used that trick to be able to have a server application reconfigure itself and reload with zero downtime in response to a Unix signal.
So, the simple answer to your question is "When it isn't going to be used anymore". In almost any application there is a point at which that's unambiguously true.
BTW, to contradict one of the other responders, an ExecutorService can use daemon threads - you can provide a ThreadFactory that configures threads however you want before you start them. But I'd encourage you to not use daemon threads and explicitly shut down the thread pool at a well-defined point - that's not the typical practice, but it will both mean your code has the potential to be shut down cleanly, and it will encourage you to think about lifecycle, which is likely to lead to better code.

There are two flavors of threads in Java (of course depending on how you look at them). 'User' threads and 'Daemon' threads. You application ends in one of the following cases:
You call System.exit()
You have no User threads left in your application. This is explained here.
Note that your main function is executed by the JVM on a 'User' thread, meaning that as long as you have not completed your main function. Most multithreaded applications will run the main function only to start all the threads needed.
The idea behind Daemon threads is that you can do something (regularly), but if all other tasks are done, it will not prevent the application from exiting.
By default new threads are 'Non Daemon' threads, the same goes for the threads craeted by your ExecutorService. If you want to change this, you have to create your own ThreadFactory. A ThreadFactory allows you to manually create the threads for your ExecutorService, it will be called when the ExecutorService needs a new thread. Here is an example of one that created 'Daemon' threads:
public class DaemonThreadFactory implements ThreadFactory
{
#Override
public Thread newThread(final Runnable r)
{
Thread t = new Thread(r);
t.setDaemon(true);
return t;
}
}
This can then be used by creating the executor service:
ExecutorService service = Executors.newFixedThreadPool(100, new DaemonThreadFactory());
Note that this is also the way to give your threads custom names, which is very useful as many logframeworks log the thread name (and the debugger shows it).
If you were to do this, in your application, it would exit right away, because you only create 'Daemon' threads, so you would have to either keep another thread alive (this could be done implicitly by another framework, for instance if you have GUI).
An alternative is to manually call System.exit(). Normally calling System.exit() in your code is not recommended. Mostly because it does not allow for good refactoring, reusing, testing and many exit points make your application unpredictable. In order to circumvent these problems, you could create a callback function that handles the job complete event. In this application you call System.exit(), your test code or other applications could do something else.

In application environment, you must call shutdown to ensure threads launched by ExecutorService must stop and it should not accept any more new tasks. Otherwise JVM will not exit.
In case of Service, you should call shutdown prior to stopping your Service execution. For e.g. in case of web-app, the contextDestroyed method of ServletContextListener is useful to invoke shutdown method of ExecutorService. This will ensure that any existing tasks must be completed prior to abrupt termination of application, but no new task will be accepted for processing.

However I found unless I called the executorService.shutdown(); the application never exit,
I think we don't need to call the executorService.shutdown(); to keep the application alive, right ?
Yes. Application is alive unless you call executorService.shutdown()
Every application should have an exit point. You can call shutdown during that exit point.
If you don't have any known exit points, ShutdownHook is one way to address your problem. But you should be aware that
In rare circumstances the virtual machine may abort, that is, stop running without shutting down cleanly
For the example you have quoted, you can address it in multiple ways (invokeAll, Future.get(), CountDownLatch etc). Have a look at related SE question.
ExecutorService, how to wait for all tasks to finish

Reason for calling shutdown() on ExecutorService

I was reading about it quite a bit in the past couple of hours, and I simply cannot see any reason (valid reason) to call shutdown() on the ExecutorService, unless we have a humongous application that stores, dozens and dozens of different executor services that are not used for a long time.
The only thing (from what I gather) the shutdown does, is doing what a normal Thread does once it's done. When the normal Thread will finish the run method of the Runnable(or Callable), it will be passed to Garbage Collection to be collected. With Executor Service the threads will simply be put on hold, they will not be ticked for the garbage collection. For that, the shutdown is needed.
Ok back to my question. Is there any reason to call shutdown on ExecutorService very often, or even right after submitting to it some tasks? I would like to leave behind the case someone is doing it and right after that calls to awaitTermination() as this is validated. Once we do that, we have to recreate a new ExecutorService all over again, to do the same thing. Isn't the whole idea for the ExecutorService to reuse the threads? So why destroy the ExecutorService so soon?
Isn't it a rational way to simply create ExecutorService (or couple depending on how many you need), then during the application running pass to them the tasks once they come along, and then on the application exit or some other important stages shutdown those executors?
I'd like an answer from some experienced coders who do write a lot of asynchronous code using the ExecutorServices.
Second side question, a bit smaller deals with the android platform. IF some of you will say that it's not the best idea to shutdown executors every time, and your program on android, could you tell me how do you handle those shutdowns (to be specific - when you execute them) when we deal with different events of the application life cycle.
Because of the CommonsWare comment, I made the post neutral. I really am not interested in arguing about it to death and it seems it's leading there. I'm only interested in learning about what I asked here from experienced developers if they are willing to share their experiences. Thanks.

The shutdown() method does one thing: prevents clients to send more work to the executor service. This means all the existing tasks will still run to completion unless other actions are taken. This is true even for scheduled tasks, e.g., for a ScheduledExecutorService: new instances of the scheduled task won't run. It also frees up any background thread resources. This can be useful in various scenarios.
Let's assume you have a console application which has an executor service running N tasks. If the user hits CTRL-C, you expect the application to terminate, possibly gracefully. What does it mean gracefully? Maybe you want your application to not be able to submit more tasks to the executor service and at the same time you want to wait for your existing N tasks to complete. You could achieve this using a shutdown hook as a last resort:
final ExecutorService service = ... // get it somewhere
Runtime.getRuntime().addShutdownHook(new Thread(new Runnable() {
#Override
public void run() {
System.out.println("Performing some shutdown cleanup...");
service.shutdown();
while (true) {
try {
System.out.println("Waiting for the service to terminate...");
if (service.awaitTermination(5, TimeUnit.SECONDS)) {
break;
}
} catch (InterruptedException e) {
}
}
System.out.println("Done cleaning");
}
}));
This hook will shutdown the service, which will prevent your application to submit new tasks, and wait for all the existing tasks to complete before shutting down the JVM. The await termination will block for 5 seconds and return true if the service is shutdown. This is done in a loop so that you're sure the service will shutdown eventually. The InterruptedException gets swallowed each time. This is the best way to shutdown an executor service that gets reused all over your application.
This code isn't perfect. Unless you're absolutely positive your tasks will eventually terminate, you might want to wait for a given timeout and then just exit, abandoning the running threads. In this case it would make sense to also call shutdownNow() after the timeout in a final attempt to interrupt the running threads (shutdownNow() will also give you a list of tasks waiting to run). If your tasks are designed to respond to interruption this will work fine.
Another interesting scenario is when you have a ScheduledExecutorService that performs a periodic task. The only way to stop the chain of periodic tasks is to call shutdown().
EDIT: I'd like to add that I wouldn't recommend using a shutdown hook as shown above in the general case: it can be error-prone and should be a last resort only. Moreover, if you have many shutdown hooks registered, the order in which they will run is undefined, which might be undesirable. I'd rather have the application explicitly call shutdown() on InterruptedException.

Isn't the whole idea for the ExecutorService to reuse the threads? So why destroy the ExecutorService so soon?
Yes. You should not destroy and re-create ExecutorService frequently. Initialize ExecutorService when you require (mostly on start-up) and keep it active until you are done with it.
Isn't it a rational way to simply create ExecutorService (or couple depending on how many you need), then during the application running pass to them the tasks once they come along, and then on the application exit or some other important stages shutdown those executors?
Yes. It's rational to shutdown ExecutorService on important stages like application exit etc.
Second side question, a bit smaller deals with android platform. IF some of you will say that it's not best idea to shutdown executors every time, and you program on android, could you tell me how you handle those shutdowns (to be specific, when you execute them) when we deal with different events of application life cycle.
Assume that ExecutorService is shared across different Activities in your application. Each activity will be paused/resumed at different intervals of time and still you need one ExecutorService per your application.
Instead of managing the state of ExecutorService in Activity life cycle methods, move ExecutorService management ( Creation/Shutdown) to your custom Service.
Create ExecutorService in Service => onCreate() and shutdown it properly in onDestroy()
Recommended way of shutting down ExecutorService :
How to properly shutdown java ExecutorService

An ExecutorService should be shut down once it is no longer needed to
free up system resources and to allow graceful application shutdown.
Because the threads in an ExecutorService may be nondaemon threads,
they may prevent normal application termination. In other words, your
application stays running after completing its main method.
Reference Book
Chaper:14
Page:814

Reason for calling shutdown() on ExecutorService
Today I encountered a situation where I have to wait until a machine is ready, before starting a series of tasks on that machine.
I make a REST call to this machine, if I don't receive 503 (Server Unavailable) then the machine is ready to process my requests. So, I wait until I get 200 (Success) for the first REST call.
There are multiple ways to achieve it, I used ExecutorService to create a thread and scheduled it to run after every X Seconds. So, I need to stop this thread on a condition, check this out...
final ScheduledExecutorService executor = Executors.newScheduledThreadPool(1);
Runnable task = () -> {
try {
int statusCode = restHelper.firstRESTCall();
if (statusCode == 200) {
executor.shutdown();
}
} catch (Exception e) {
e.printStackTrace();
}
};
int retryAfter = 60;
executor.scheduleAtFixedRate(task, 0, retryAfter, TimeUnit.SECONDS);
Second side question, a bit smaller deals with android platform.
Maybe I can answer if you'll provide bit more context!
Also from my experience with Android development it's rarely you need Threads. Are you developing a Game or an app which needs threads for performance? If not, in Android you have other ways to tackle problems like the scenario that I explained above. You can rather use TimerTask, AsyncTask or Handlers or Loaders based on context. This is because if UIThread waits for long you know what happens :/

This is genuine notwithstanding for planned undertakings, e.g., for a ScheduledExecutorService: new cases of the booked assignment won't run.
We should expect you have a comfort application which has an agent administration running N errands.
I'm not catching it's meaning effortlessly? Perhaps you need your application to not have the option to submit more assignments to the agent administration and in the meantime you need to sit tight for your current N undertakings to finish.
Except if you're totally positive your errands will in the end, you should need to sit tight for a given break and after that simply exit, deserting the running strings.
In the event that your activitys are intended to react to interference this will work fine.
Another intriguing situation is the point at which you have a ScheduledExecutorService that plays out an activity.
The best way to stop the chain of activity is to call shutdown()

Stop a Runnable submitted to ExecutorService

I've implemented subscription in my Java app. When new subscriber added, the application creates new task (class which implements Runnable to be run in the separate thread) and it is added to the ExecutorService like:
public void Subscribe()
{
es_.execute(new Subscriber(this, queueName, handler));
}
//...
private ExecutorService es_;
Application may register as many subscribers as you want. Now I want implement something like Unsubscribe so every subscriber has an ability to stop the message flow. Here I need a way to stop one of the tasks running in the ExecutorService. But I don't know how I can do this.
The ExecutorService.shutdown() and its variations are not for me: they terminates all the tasks, I want just terminate one of them. I'm searching for a solution. As simple as possible. Thanks.

You can use ExecutorService#submit instead of execute and use the returned Future object to try and cancel the task using Future#cancel
Example (Assuming Subscriber is a Runnable):
Future<?> future = es_.submit(new Subscriber(this, queueName, handler));
...
future.cancel(true); // true to interrupt if running
Important note from the comments:
If your task doesn't honour interrupts and it has already started, it will run to completion.

Instead of using ExecutorService.execute(Runnable) try using Future<?> submit(Runnable). This method will submit the Runnable into the pool for execution and it will return a Future object. By doing this you will have references to all subscriber threads.
In order to stop particular thread just use futureObj.cancel(true). This will interrupt the running thread, throwing an InterruptedException. The subscriber thread should be coded such way it will stop processing in the case of this exception (for example Thread.sleep(millis) with wrapper try / catch block for the whole method).
You cand find more information on the official API:
http://docs.oracle.com/javase/6/docs/api/java/util/concurrent/Future.html
http://docs.oracle.com/javase/6/docs/api/java/util/concurrent/ExecutorService.html

How to wait for all tasks in an ThreadPoolExecutor to finish without shutting down the Executor?

I can't use shutdown() and awaitTermination() because it is possible new tasks will be added to the ThreadPoolExecutor while it is waiting.
So I'm looking for a way to wait until the ThreadPoolExecutor has emptied it's queue and finished all of it's tasks without stopping new tasks from being added before that point.
If it makes any difference, this is for Android.
Thanks
Update: Many weeks later after revisiting this, I discovered that a modified CountDownLatch worked better for me in this case. I'll keep the answer marked because it applies more to what I asked.

If you are interested in knowing when a certain task completes, or a certain batch of tasks, you may use ExecutorService.submit(Runnable). Invoking this method returns a Future object which may be placed into a Collection which your main thread will then iterate over calling Future.get() for each one. This will cause your main thread to halt execution until the ExecutorService has processed all of the Runnable tasks.
Collection<Future<?>> futures = new LinkedList<Future<?>>();
futures.add(executorService.submit(myRunnable));
for (Future<?> future:futures) {
future.get();
}

My Scenario is a web crawler to fetch some information from a web site then processing them. A ThreadPoolExecutor is used to speed up the process because many pages can be loaded in the time. So new tasks will be created in the existing task because the crawler will follow hyperlinks in each page. The problem is the same: the main thread do not know when all the tasks are completed and it can start to process the result. I use a simple way to determine this. It is not very elegant but works in my case:
while (executor.getTaskCount()!=executor.getCompletedTaskCount()){
System.err.println("count="+executor.getTaskCount()+","+executor.getCompletedTaskCount());
Thread.sleep(5000);
}
executor.shutdown();
executor.awaitTermination(60, TimeUnit.SECONDS);

Maybe you are looking for a CompletionService to manage batches of task, see also this answer.

(This is an attempt to reproduce Thilo's earlier, deleted answer with my own adjustments.)
I think you may need to clarify your question since there is an implicit infinite condition... at some point you have to decide to shut down your executor, and at that point it won't accept any more tasks. Your question seems to imply that you want to wait until you know that no further tasks will be submitted, which you can only know in your own application code.
The following answer will allow you to smoothly transition to a new TPE (for whatever reason), completing all the currently-submitted tasks, and not rejecting new tasks to the new TPE. It might answer your question. #Thilo's might also.
Assuming you have defined somewhere a visible TPE in use as such:
AtomicReference<ThreadPoolExecutor> publiclyAvailableTPE = ...;
You can then write the TPE swap routine as such. It could also be written using a synchronized method, but I think this is simpler:
void rotateTPE()
{
ThreadPoolExecutor newTPE = createNewTPE();
// atomic swap with publicly-visible TPE
ThreadPoolExecutor oldTPE = publiclyAvailableTPE.getAndSet(newTPE);
oldTPE.shutdown();
// and if you want this method to block awaiting completion of old tasks in
// the previously visible TPE
oldTPE.awaitTermination();
}
Alternatively, if you really no kidding want to kill the thread pool, then your submitter side will need to cope with rejected tasks at some point, and you could use null for the new TPE:
void killTPE()
{
ThreadPoolExecutor oldTPE = publiclyAvailableTPE.getAndSet(null);
oldTPE.shutdown();
// and if you want this method to block awaiting completion of old tasks in
// the previously visible TPE
oldTPE.awaitTermination();
}
Which could cause upstream problems, the caller would need to know what to do with a null.
You could also swap out with a dummy TPE that simply rejected every new execution, but that's equivalent to what happens if you call shutdown() on the TPE.

If you don't want to use shutdown, follow below approaches:
Iterate through all Future tasks from submit on ExecutorService and check the status with blocking call get() on Future object as suggested by Tim Bender
Use one of
Using invokeAll on ExecutorService
Using CountDownLatch
Using ForkJoinPool or newWorkStealingPool of Executors(since java 8)
invokeAll() on executor service also achieves the same purpose of CountDownLatch
Related SE question:
How to wait for a number of threads to complete?

You could call the waitTillDone() on Runner class:
Runner runner = Runner.runner(10);
runner.runIn(2, SECONDS, runnable);
runner.run(runnable); // each of this runnables could submit more tasks
runner.waitTillDone(); // blocks until all tasks are finished (or failed)
// and now reuse it
runner.runIn(500, MILLISECONDS, callable);
runner.waitTillDone();
runner.shutdown();
To use it add this gradle/maven dependency to your project: 'com.github.matejtymes:javafixes:1.0'
For more details look here: https://github.com/MatejTymes/JavaFixes or here: http://matejtymes.blogspot.com/2016/04/executor-that-notifies-you-when-task.html

Try using queue size and active tasks count as shown below
while (executor.getThreadPoolExecutor().getActiveCount() != 0 || !executor.getThreadPoolExecutor().getQueue().isEmpty()){
try {
Thread.sleep(500);
} catch (InterruptedException e) {
}
}