Develop Reference

Short but frequent jobs: HandlerThread or ThreadPoolExecutor?

First of all, I could not determine what the title should be, so if it's not specific enough, the question itself will be.
We have an application that uses a foreground service and stays alive forever, and in this service, there are frequent database access jobs, network access jobs and some more, that needs to run on background threads. One job itself consumes a small amount of time, but the jobs themselves are frequent. Obviously, they need to run on worker threads, so I'm here to ask which design we should follow.
HandlerThread is a structure that creates a singular thread and uses a queue to execute tasks but always loops and waits for messages which consumes power, while ThreadPoolExecutor creates multiple threads for each job and deletes threads when the jobs are done, but because of too many threads there could be leaks, or out-of-memory even. The job count may be 5, or it may be 20, depending on how the user acts in a certain way. And, between 2 jobs, there can be a 5 second gap, or a day gap, totally depending on user. But, remember, the application stays alive forever and waits for these jobs to execute.
So, for this specific occasion, which one is better to use? A thread pool executor or a handler thread? Any advice is appreciated, thanks.

Caveat: I do not do Android work, so I am no expert there. My opinions here are based a quick reading of Android documentation.
tl;dr
➥ Use Executors rather than HandlerThread.
The Executors framework is more modern, flexible, and powerful than the legacy Thread facility used by HandlerThread. Everything you can do in HandlerThread you can do better with executors.
Differences
One big difference between HandlerThread and ThreadPoolExecutor is that the first comes from Android while the second comes from Java. So if you'll be doing other work with Java, you might not want to get in the habit of using HandlerThread.
Another big difference is age. The android.os.HandlerThread class inherits from java.lang.Thread, and dates back to the original Android API level 1. While nice for its time, the Thread facility in Java is limited in its design. That facility was supplanted by the more modern, flexible, and powerful Executors framework in later Java.
Executors
Your Question is not clear about whether these are recurring jobs or sporadically scheduled. Either can be handled with Executors.
For jobs that run once at a specific time, and for recurring scheduled jobs, use a ScheduledExecutorService. You can schedule a job to run once at a certain time by specifying a delay, a span of time to wait until execution. For repeated jobs, you can specify an amount to wait, then run, then wait, then run, and so on. I'll not address this further, as you seem to be talking about sporadic immediate jobs rather than scheduled or repeating jobs. If interested, search Stack Overflow as ScheduledExecutorService has been covered many times already on Stack Overflow.
Single thread pool
HandlerThread is a structure that creates a singular thread
If you want to recreate that single thread behavior, use a thread pool consisting of only a single thread.
ExecutorService es = Executors.newSingleThreadExecutor() ;
Make your tasks. Implement either Runnable or Callable using (a) a class implementing either interface, (b) without defining a class, via lambda syntax or conventional syntax.
Conventional syntax.
Runnable sayHelloJob = new Runnable()
{
#Override
public void run ( )
{
System.out.println( "Hello. " + Instant.now() );
}
};
Lambda syntax.
Runnable sayBonjourJob = ( ) -> System.out.println( "Bonjour. " + Instant.now() );
Submit as many of these jobs to the executor service as you wish.
es.submit( sayHelloJob ) ;
es.submit( sayBonjourJob ) ;
Notice that the submit method returns a Future. With that Future object, you can check if the computation is complete, wait for its completion, or retrieve the result of the computation. Or you may choose to ignore the Future object as seen in the code above.
Fixed thread pool
If you want multiple thread behavior, just create your executor with a different kind of thread pool.
A fixed thread pool has a maximum number of threads servicing a single queue of submitted jobs (Runnable or Callable objects). The threads continue to live, and are replaced as needed in case of failure.
ExecutorService es = Executors.newFixedThreadPool​( 3 ) ; // Specify number of threads.
The rest of the code remains the same. That is the beauty of using the ExecutorService interface: You can change the implementation of the executor service to get difference behavior while not breaking your code that calls upon that executor service.
Cached thread pool
Your needs may be better service by a cached thread pool. Rather than immediately creating and maintaining a certain number of threads as the fixed thread pool does, this pool creates threads only as needed, up to a maximum. When a thread is done, and resting for over a minute, the thread is terminated. As the Javadoc notes, this is ideal for “many short-lived asynchronous tasks” such as yours. But notice that there is no upper limit of threads that may be running simultaneously. If the nature of your app is such that you may see often spikes of many jobs arriving simultaneously, you may want to use a different implementation other than cached thread pool.
ExecutorService es = Executors.newCachedThreadPool() ;
Managing executors and threads
but because of too many threads there could be leaks, or out-of-memory even
It is the job of you the programmer and your sysadmin to not overburden the production server. You need to monitor performance in production. The managagement is easy enough to perform, as you control the number of threads available in the thread pool backing your executor service.
We have an application that uses a foreground service and stays alive forever
Of course your app does eventually come to end, being shutdown. When that happens, be sure to shutdown your executor and its backing thread pool. Otherwise the threads may survive, and continue indefinitely. Be sure to use the life cycle hooks of your app’s execution environment to detect and react to the app shutting down.
The job count may be 5, or it may be 20, depending on how the user acts in a certain way.
Jobs submitted to an executor service are buffered up until they can be scheduled on a thread for execution. So you may have a thread pool of, for example, 3 threads and 20 waiting jobs. No problem. The waiting jobs will be eventually executed when their time comes.
You may want to prioritize certain jobs, to be done ahead of lower priority jobs. One easy way to do this is to have two executor services. Each executor has its own backing thread pool. One executor is for the fewer but higher-priority jobs, while the other executor is for the many lower-priority jobs.
Remember that threads in a thread pool doing no work, on stand-by, have virtually no overhead in Java for either CPU or memory. So there is no downside to having a special higher-priority executor service sitting around and waiting for eventual jobs to arrive. The only concern is that your total number of all background threads and their workload not overwhelm your machine. Also, the implementation of the thread pool may well shut down unused threads after a period of disuse.

Don't really think its a question of the number of threads you are running, more how you want them run. If you want them run one at at time (i.e. you only want to execute on database query at a time) then use a HandlerThread. If you want multi-threading / a pool of threads, then use and Executor.
In my experience, leaks are really more down to how you have coded your threads, not really the chosen implementation.
Personally, I'd use a HandlerThread, here's a nice article on implementing them and how to avoid memory leaks ... Using HandlerThread in Android

creating and killing a thread vs using .notify () and .wait ()

Assuming a control thread has access to a bunch of threads and to the objects this thread would wait on. Which one will have a greater impact in performance if I have to start and stop, what several of these threads are doing, from this single control thread ?
Wouldn't it just be better for example to kill it via interruption and just create a new one with the same Runnable?

Creating (actually start()-ing) a new thread is relatively expensive, so from a performance perspective it would be better to use wait / notify.
Secondly, interrupt is not guaranteed to "stop" a thread. The thread may choose to ignore the interrupt ... or if it is purely CPU bound, it may not notice it at all.
There is also a third option: use an existing thread pool mechanism. For example, the ExecutorService API has a various implementations that provided bounded and unbounded thread pools. These can take care of scaling up and down, and pool shutdown. You use them by submit(...)-ing tasks as Runnable instances and you optionally get a Future that allows you to wait for the task completion.
Finally, for most concurrent programming use-cases, there are standard classes that support the use-case, and it is better to use them rather than attempting to implement from scratch; e.g. using wait / notify directly. In your case, you probably need some kind of "barrier" mechanism: java.util.concurrent.Phaser might be the one that you need.

Threads are fairly independent from one another and in most cases each thread will know better than the control thread when it's ready to terminate. Killing a thread is very abrupt thing, it's much better to wait for the threat to terminate itself cleanly.

Reason for calling shutdown() on ExecutorService

I was reading about it quite a bit in the past couple of hours, and I simply cannot see any reason (valid reason) to call shutdown() on the ExecutorService, unless we have a humongous application that stores, dozens and dozens of different executor services that are not used for a long time.
The only thing (from what I gather) the shutdown does, is doing what a normal Thread does once it's done. When the normal Thread will finish the run method of the Runnable(or Callable), it will be passed to Garbage Collection to be collected. With Executor Service the threads will simply be put on hold, they will not be ticked for the garbage collection. For that, the shutdown is needed.
Ok back to my question. Is there any reason to call shutdown on ExecutorService very often, or even right after submitting to it some tasks? I would like to leave behind the case someone is doing it and right after that calls to awaitTermination() as this is validated. Once we do that, we have to recreate a new ExecutorService all over again, to do the same thing. Isn't the whole idea for the ExecutorService to reuse the threads? So why destroy the ExecutorService so soon?
Isn't it a rational way to simply create ExecutorService (or couple depending on how many you need), then during the application running pass to them the tasks once they come along, and then on the application exit or some other important stages shutdown those executors?
I'd like an answer from some experienced coders who do write a lot of asynchronous code using the ExecutorServices.
Second side question, a bit smaller deals with the android platform. IF some of you will say that it's not the best idea to shutdown executors every time, and your program on android, could you tell me how do you handle those shutdowns (to be specific - when you execute them) when we deal with different events of the application life cycle.
Because of the CommonsWare comment, I made the post neutral. I really am not interested in arguing about it to death and it seems it's leading there. I'm only interested in learning about what I asked here from experienced developers if they are willing to share their experiences. Thanks.

The shutdown() method does one thing: prevents clients to send more work to the executor service. This means all the existing tasks will still run to completion unless other actions are taken. This is true even for scheduled tasks, e.g., for a ScheduledExecutorService: new instances of the scheduled task won't run. It also frees up any background thread resources. This can be useful in various scenarios.
Let's assume you have a console application which has an executor service running N tasks. If the user hits CTRL-C, you expect the application to terminate, possibly gracefully. What does it mean gracefully? Maybe you want your application to not be able to submit more tasks to the executor service and at the same time you want to wait for your existing N tasks to complete. You could achieve this using a shutdown hook as a last resort:
final ExecutorService service = ... // get it somewhere
Runtime.getRuntime().addShutdownHook(new Thread(new Runnable() {
#Override
public void run() {
System.out.println("Performing some shutdown cleanup...");
service.shutdown();
while (true) {
try {
System.out.println("Waiting for the service to terminate...");
if (service.awaitTermination(5, TimeUnit.SECONDS)) {
break;
}
} catch (InterruptedException e) {
}
}
System.out.println("Done cleaning");
}
}));
This hook will shutdown the service, which will prevent your application to submit new tasks, and wait for all the existing tasks to complete before shutting down the JVM. The await termination will block for 5 seconds and return true if the service is shutdown. This is done in a loop so that you're sure the service will shutdown eventually. The InterruptedException gets swallowed each time. This is the best way to shutdown an executor service that gets reused all over your application.
This code isn't perfect. Unless you're absolutely positive your tasks will eventually terminate, you might want to wait for a given timeout and then just exit, abandoning the running threads. In this case it would make sense to also call shutdownNow() after the timeout in a final attempt to interrupt the running threads (shutdownNow() will also give you a list of tasks waiting to run). If your tasks are designed to respond to interruption this will work fine.
Another interesting scenario is when you have a ScheduledExecutorService that performs a periodic task. The only way to stop the chain of periodic tasks is to call shutdown().
EDIT: I'd like to add that I wouldn't recommend using a shutdown hook as shown above in the general case: it can be error-prone and should be a last resort only. Moreover, if you have many shutdown hooks registered, the order in which they will run is undefined, which might be undesirable. I'd rather have the application explicitly call shutdown() on InterruptedException.

Isn't the whole idea for the ExecutorService to reuse the threads? So why destroy the ExecutorService so soon?
Yes. You should not destroy and re-create ExecutorService frequently. Initialize ExecutorService when you require (mostly on start-up) and keep it active until you are done with it.
Isn't it a rational way to simply create ExecutorService (or couple depending on how many you need), then during the application running pass to them the tasks once they come along, and then on the application exit or some other important stages shutdown those executors?
Yes. It's rational to shutdown ExecutorService on important stages like application exit etc.
Second side question, a bit smaller deals with android platform. IF some of you will say that it's not best idea to shutdown executors every time, and you program on android, could you tell me how you handle those shutdowns (to be specific, when you execute them) when we deal with different events of application life cycle.
Assume that ExecutorService is shared across different Activities in your application. Each activity will be paused/resumed at different intervals of time and still you need one ExecutorService per your application.
Instead of managing the state of ExecutorService in Activity life cycle methods, move ExecutorService management ( Creation/Shutdown) to your custom Service.
Create ExecutorService in Service => onCreate() and shutdown it properly in onDestroy()
Recommended way of shutting down ExecutorService :
How to properly shutdown java ExecutorService

An ExecutorService should be shut down once it is no longer needed to
free up system resources and to allow graceful application shutdown.
Because the threads in an ExecutorService may be nondaemon threads,
they may prevent normal application termination. In other words, your
application stays running after completing its main method.
Reference Book
Chaper:14
Page:814

Reason for calling shutdown() on ExecutorService
Today I encountered a situation where I have to wait until a machine is ready, before starting a series of tasks on that machine.
I make a REST call to this machine, if I don't receive 503 (Server Unavailable) then the machine is ready to process my requests. So, I wait until I get 200 (Success) for the first REST call.
There are multiple ways to achieve it, I used ExecutorService to create a thread and scheduled it to run after every X Seconds. So, I need to stop this thread on a condition, check this out...
final ScheduledExecutorService executor = Executors.newScheduledThreadPool(1);
Runnable task = () -> {
try {
int statusCode = restHelper.firstRESTCall();
if (statusCode == 200) {
executor.shutdown();
}
} catch (Exception e) {
e.printStackTrace();
}
};
int retryAfter = 60;
executor.scheduleAtFixedRate(task, 0, retryAfter, TimeUnit.SECONDS);
Second side question, a bit smaller deals with android platform.
Maybe I can answer if you'll provide bit more context!
Also from my experience with Android development it's rarely you need Threads. Are you developing a Game or an app which needs threads for performance? If not, in Android you have other ways to tackle problems like the scenario that I explained above. You can rather use TimerTask, AsyncTask or Handlers or Loaders based on context. This is because if UIThread waits for long you know what happens :/

This is genuine notwithstanding for planned undertakings, e.g., for a ScheduledExecutorService: new cases of the booked assignment won't run.
We should expect you have a comfort application which has an agent administration running N errands.
I'm not catching it's meaning effortlessly? Perhaps you need your application to not have the option to submit more assignments to the agent administration and in the meantime you need to sit tight for your current N undertakings to finish.
Except if you're totally positive your errands will in the end, you should need to sit tight for a given break and after that simply exit, deserting the running strings.
In the event that your activitys are intended to react to interference this will work fine.
Another intriguing situation is the point at which you have a ScheduledExecutorService that plays out an activity.
The best way to stop the chain of activity is to call shutdown()

How to use Thread Pool concept in Java?

I am creating a http proxy server in java. I have a class named Handler which is responsible for processing the requests and responses coming and going from web browser and to web server respectively. I have also another class named Copy which copies the inputStream object to outputStream object . Both these classes implement Runnable interface. I would like to use the concept of Thread pooling in my design, however i don't know how to go about that! Any hint or idea would be highly appreciated.

I suggest you look at Executor and ExecutorService. They add a lot of good stuff to make it easier to use Thread pools.
...
#Azad provided some good information and links. You should also buy and read the book Java Concurrency in Practice. (often abbreviated as JCiP) Note to stackoverflow big-wigs - how about some revenue link to Amazon???
Below is my brief summary of how to use and take advantage of ExecutorService with thread pools. Let's say you want 8 threads in the pool.
You can create one using the full featured constructors of ThreadPoolExecutor, e.g.
ExecutorService service = new ThreadPoolExecutor(8,8, more args here...);
or you can use the simpler but less customizable Executors factories, e.g.
ExecutorService service = Executors.newFixedThreadPool(8);
One advantage you immediately get is the ability to shutdown() or shutdownNow() the thread pool, and to check this status via isShutdown() or isTerminated().
If you don't care much about the Runnable you wish to run, or they are very well written, self-contained, never fail or log any errors appropriately, etc... you can call
execute(Runnable r);
If you do care about either the result (say, it calculates pi or downloads an image from a webpage) and/or you care if there was an Exception, you should use one of the submit methods that returns a Future. That allows you, at some time in the future, check if the task isDone() and to retrieve the result via get(). If there was an Exception, get() will throw it (wrapped in an ExecutionException). Note - even of your Future doesn't "return" anything (it is of type Void) it may still be good practice to call get() (ignoring the void result) to test for an Exception.
However, this checking the Future is a bit of chicken and egg problem. The whole point of a thread pool is to submit tasks without blocking. But Future.get() blocks, and Future.isDone() begs the questions of which thread is calling it, and what it does if it isn't done - do you sleep() and block?
If you are submitting a known chunk of related of tasks simultaneously, e.g., you are performing some big mathematical calculation like a matrix multiply that can be done in parallel, and there is no particular advantage to obtaining partial results, you can call invokeAll(). The calling thread will then block until all the tasks are complete, when you can call Future.get() on all the Futures.
What if the tasks are more disjointed, or you really want to use the partial results? Use ExecutorCompletionService, which wraps an ExecutorService. As tasks get completed, they are added to a queue. This makes it easy for a single thread to poll and remove events from the queue. JCiP has a great example of an web page app that downloads all the images in parallel, and renders them as soon as they become available for responsiveness.

I hope below will help you:,
class Executor
An object that executes submitted Runnable tasks. This interface provides a way of decoupling task submission from the mechanics of how each task will be run, including details of thread use, scheduling, etc. An Executor is normally used instead of explicitly creating threads. For example, rather than invoking new Thread(new(RunnableTask())).start() for each of a set of tasks, you might use:
Executor executor = anExecutor;
executor.execute(new RunnableTask1());
executor.execute(new RunnableTask2());
...
class ScheduledThreadPoolExecutor
A ThreadPoolExecutor that can additionally schedule commands to run after a given delay, or to execute periodically. This class is preferable to Timer when multiple worker threads are needed, or when the additional flexibility or capabilities of ThreadPoolExecutor (which this class extends) are required.
Delayed tasks execute no sooner than they are enabled, but without any real-time guarantees about when, after they are enabled, they will commence. Tasks scheduled for exactly the same execution time are enabled in first-in-first-out (FIFO) order of submission.
and
Interface ExecutorService
An Executor that provides methods to manage termination and methods that can produce a Future for tracking progress of one or more asynchronous tasks.
An ExecutorService can be shut down, which will cause it to stop accepting new tasks. After being shut down, the executor will eventually terminate, at which point no tasks are actively executing, no tasks are awaiting execution, and no new tasks can be submitted.
Edited:
you can find example to use Executor and ExecutorService herehereand here Question will be useful for you.

What's the proper background process behaviour for a non-GUI Java app?

What's the proper way for a Java command line application to do background work without hogging resources? Should it use sleep() in the loop or is there a more elegant/efficient way?

Some heuristics:
Don't attempt to make scheduling decisions in your application. The operating system's scheduler is way better than yours will be. Let it do its job.
Don't poll if you don't have to. For instance, instead of sleeping n seconds, then waking up to check a non-blocked socket, block on the socket. This second strategy plays better with the operating system's scheduler.
Don't use an enormous heap if you don't have to, and try not to allocate enormous chunks of memory at one time. A thrashing application tends to have a negative effect on system performance.
Use buffered I/O. Always. If you think you need non-buffered I/O, be absolutely sure you're right. (You're probably wrong.)
Don't spawn a lot of threads. Threads are surprisingly expensive; beyond a certain point, more threads will reduce your application's performance profile. If you have lots of work to do concurrently, learn and use java.util.concurrent.
Of course, this is just a starter list...

I'd only use sleep() if there's no work to be done. For example, if you're doing something like polling a task queue periodically and there's nothing there, sleep for a while then check again, etc.
If you're just trying to make sure you don't hog the CPU but you're still doing real work, you could call Thread.yield() periodically. That will relinquish control of the CPU and let other threads run, but it won't put you to sleep. If other processes don't need the CPU you'll get control back and continue to do your work.
You can also set your thread to a low priority:
myThread.setPriority(Thread.MIN_PRIORITY);
As Ishmael said, don't do this in your main thread. Create a "worker thread" instead. That way your UI (GUI or CLI) will still be responsive.

There are several ways. I would use ExecutorService... for example:
ExecutorService service = Executors.newCachedThreadPool();
Callable<Result> task = new Callable<Result>() {
public Result call() throws Exception {
// code which will be run on background thread
}
};
Future<Result> future = service.submit(task);
// Next line wait until background task is complete
// without killing CPU. Of course, you can do something
// different here and check your 'future' later.
//
// Note also that future.get() may throw various exceptions too,
// you'll need to handle them properly
Result resultFromBackgroundThread = future.get();
This is Java 5 code, ExecutorService, Callable, Future and similar are in java.util.concurrent package.

One place to start is to make sure that only those resources are being used and no other objects (so that they become garbage collected).
Placing sleep() in a single threading application is only going to halt the current thread. If you're trying to accomplish data being processed in the background while information still needs to be presented to the user then it is best to put the background process in a seperate thread.