Develop Reference

Does process have any overhead in context switching compared to threads?

A few words about what I'm planing to do. I need to create some task executor, that will poll tasks from queue and just execute code in this task. And for this I need to implement some interrupt mechanism to enable user to stop this task.
So I see two possible solutions: 1. start a pool of threads and stop them by using .destroy() method of a thread. (I will not use any shared objects) 2. Use pool of separated processes and System.exit() or kill signal to process. Option 2. looks much safer for me as I can ensure that thread killing will not lead to any concurrency problems. But I'm not sure that it won't produce a big overhead.
Also I'm not sure about JVM, if I will use separated processes, each process will be using the separated JVM, and it can bring a lot of overhead. Or not. So my question in this. Choosing a different language without runtime for worker process is possible option for me, but I still don't have enough experience with processes and don't know about overhead.

start a pool of threads and stop them by using .destroy() method of a thread. (I will not use any shared objects)
You can't stop threads on modern VMs unless said thread is 'in on it'. destroy and friends do not actually do what you want and this is unsafe. The right way is to call interrupt(). If the thread wants to annoy you and not actually stop in the face of an interrupt call, they can. The solution is to fix the code so that it doesn't do that anymore. Note that raising the interrupt flag will guaranteed stop any method that is sleeping which is specced to throw InterruptedException (sleep, wait, etc), and on most OSes, will also cause any I/O call that is currently frozen to exit by throwing an IOException, but there is no guarantee for this.
Use pool of separated processes and System.exit() or kill signal to process.
Hella expensive; a VM is not a light thing to spin up; it'll have its own copy of all the classes (even something as simple as java.lang.String and company). 10 VMs is a stretch. Whereas 1000 threads is no problem.
And for this I need to implement some interrupt mechanism to enable user to stop this task.
The real problem is that this is very difficult to guarantee. But if you control the code that needs interrupting, then usually no big deal. Just use the interrupt() mechanism.
EDIT: In case you're wondering how to do the interrupt thing: Raising the interrupt flag on a thread just raises the flag; nothing else happens unless you write code that interacts with it, or call a method that does.
There are 3 main interactions:
All things that block and are declared to throw InterruptedEx will lower the flag and throw InterruptedEx. If the flag is up and you call Thread.sleep, that will immediately_ clear the flag and throw that exception without ever even waiting. Thus, catch that exception, and return/abort/break off the task.
Thread.interrupted() will lower the flag and return true (thus, does so only once). Put this in your event loops. It's not public void run() {while (true) { ... }} or while (running) {} or whatnot, it's while (!Thread.interrupted() or possibly while (running && !Thread.interrupted9)).
Any other blocking method may or may not; java intentionally doesn't specify either way because it depends on OS and architecture. If they do (and many do), they can't throw interruptedex, as e.g. FileInputStream.read isn't specced to throw it. They throw IOException with a message indicating an abort happened.
Ensure that these 3 code paths one way or another lead to a task that swiftly ends, and you have what you want: user-interruptible tasks.

Executors framework
Java already provides a facility with your desired features, the Executors framework.
You said:
I need to create some task executor, that will poll tasks from queue and just execute code in this task.
The ExecutorService interface does just that.
Choose an implementation meeting your needs from the Executors class. For example, if you want to run your tasks in the sequence of their submission, use a single-threaded executor service. You have several others to choose from if you want other behavior.
ExecutorService executorService = Executors.newSingleThreadExecutor() ;
You said:
start a pool of threads
The executor service may be backed by a pool of threads.
ExecutorService executorService = Executors.newFixedThreadPool​( 3 ) ; // Create a pool of exactly three threads to be used for any number of submitted tasks.
You said:
just execute code in this task
Define your task as a class implementing either Runnable or Callable. That means your class carries a run method, or a call method.
Runnable task = ( ) -> System.out.println( "Doing this work on a background thread. " + Instant.now() );
You said:
will poll tasks from queue
Submit your tasks to be run. You can submit many tasks, either of the same class or of different classes. The executor service maintains a queue of submitted tasks.
executorService.submit( task );
Optionally, you may capture the Future object returned.
Future future = executorService.submit( task );
That Future object lets you check to see if the task has finished or has been cancelled.
if( future.isDone() ) { … }
You said:
enable user to stop this task
If you want to cancel the task, call Future::cancel.
Pass true if you want to interrupt the task if it has already begun execution.
Pass false if you only want to cancel the task before it has begun execution.
future.cancel( true );
You said:
looks much safer for me as I can ensure that thread killing will not lead to any concurrency problems.
Using the Executors framework, you would not be creating or killing any threads. The executor service implementation handles the threads. Your code never addresses the Thread class directly.
So no concurrency problems of that kind.
But you may have other concurrency problems if you share any resources across threads. I highly recommend reading Java Concurrency in Practice by Brian Goetz et al.
You said:
But I'm not sure that it won't produce a big overhead.
As the correct Answer by rzwitserloot explained, your approach would certainly create much more overhead that would the use of the Executors framework.
FYI, in the future Project Loom will bring virtual threads (fibers) to the Java platform. This will generally make background threading even faster, and will make practical having many thousands or even millions of non-CPU-bound tasks. Special builds available now on early-access Java 16.
ExecutorService executorService = newVirtualThreadExecutor() ;
executorService.submit( task ) ;

Short but frequent jobs: HandlerThread or ThreadPoolExecutor?

First of all, I could not determine what the title should be, so if it's not specific enough, the question itself will be.
We have an application that uses a foreground service and stays alive forever, and in this service, there are frequent database access jobs, network access jobs and some more, that needs to run on background threads. One job itself consumes a small amount of time, but the jobs themselves are frequent. Obviously, they need to run on worker threads, so I'm here to ask which design we should follow.
HandlerThread is a structure that creates a singular thread and uses a queue to execute tasks but always loops and waits for messages which consumes power, while ThreadPoolExecutor creates multiple threads for each job and deletes threads when the jobs are done, but because of too many threads there could be leaks, or out-of-memory even. The job count may be 5, or it may be 20, depending on how the user acts in a certain way. And, between 2 jobs, there can be a 5 second gap, or a day gap, totally depending on user. But, remember, the application stays alive forever and waits for these jobs to execute.
So, for this specific occasion, which one is better to use? A thread pool executor or a handler thread? Any advice is appreciated, thanks.

Caveat: I do not do Android work, so I am no expert there. My opinions here are based a quick reading of Android documentation.
tl;dr
➥ Use Executors rather than HandlerThread.
The Executors framework is more modern, flexible, and powerful than the legacy Thread facility used by HandlerThread. Everything you can do in HandlerThread you can do better with executors.
Differences
One big difference between HandlerThread and ThreadPoolExecutor is that the first comes from Android while the second comes from Java. So if you'll be doing other work with Java, you might not want to get in the habit of using HandlerThread.
Another big difference is age. The android.os.HandlerThread class inherits from java.lang.Thread, and dates back to the original Android API level 1. While nice for its time, the Thread facility in Java is limited in its design. That facility was supplanted by the more modern, flexible, and powerful Executors framework in later Java.
Executors
Your Question is not clear about whether these are recurring jobs or sporadically scheduled. Either can be handled with Executors.
For jobs that run once at a specific time, and for recurring scheduled jobs, use a ScheduledExecutorService. You can schedule a job to run once at a certain time by specifying a delay, a span of time to wait until execution. For repeated jobs, you can specify an amount to wait, then run, then wait, then run, and so on. I'll not address this further, as you seem to be talking about sporadic immediate jobs rather than scheduled or repeating jobs. If interested, search Stack Overflow as ScheduledExecutorService has been covered many times already on Stack Overflow.
Single thread pool
HandlerThread is a structure that creates a singular thread
If you want to recreate that single thread behavior, use a thread pool consisting of only a single thread.
ExecutorService es = Executors.newSingleThreadExecutor() ;
Make your tasks. Implement either Runnable or Callable using (a) a class implementing either interface, (b) without defining a class, via lambda syntax or conventional syntax.
Conventional syntax.
Runnable sayHelloJob = new Runnable()
{
#Override
public void run ( )
{
System.out.println( "Hello. " + Instant.now() );
}
};
Lambda syntax.
Runnable sayBonjourJob = ( ) -> System.out.println( "Bonjour. " + Instant.now() );
Submit as many of these jobs to the executor service as you wish.
es.submit( sayHelloJob ) ;
es.submit( sayBonjourJob ) ;
Notice that the submit method returns a Future. With that Future object, you can check if the computation is complete, wait for its completion, or retrieve the result of the computation. Or you may choose to ignore the Future object as seen in the code above.
Fixed thread pool
If you want multiple thread behavior, just create your executor with a different kind of thread pool.
A fixed thread pool has a maximum number of threads servicing a single queue of submitted jobs (Runnable or Callable objects). The threads continue to live, and are replaced as needed in case of failure.
ExecutorService es = Executors.newFixedThreadPool​( 3 ) ; // Specify number of threads.
The rest of the code remains the same. That is the beauty of using the ExecutorService interface: You can change the implementation of the executor service to get difference behavior while not breaking your code that calls upon that executor service.
Cached thread pool
Your needs may be better service by a cached thread pool. Rather than immediately creating and maintaining a certain number of threads as the fixed thread pool does, this pool creates threads only as needed, up to a maximum. When a thread is done, and resting for over a minute, the thread is terminated. As the Javadoc notes, this is ideal for “many short-lived asynchronous tasks” such as yours. But notice that there is no upper limit of threads that may be running simultaneously. If the nature of your app is such that you may see often spikes of many jobs arriving simultaneously, you may want to use a different implementation other than cached thread pool.
ExecutorService es = Executors.newCachedThreadPool() ;
Managing executors and threads
but because of too many threads there could be leaks, or out-of-memory even
It is the job of you the programmer and your sysadmin to not overburden the production server. You need to monitor performance in production. The managagement is easy enough to perform, as you control the number of threads available in the thread pool backing your executor service.
We have an application that uses a foreground service and stays alive forever
Of course your app does eventually come to end, being shutdown. When that happens, be sure to shutdown your executor and its backing thread pool. Otherwise the threads may survive, and continue indefinitely. Be sure to use the life cycle hooks of your app’s execution environment to detect and react to the app shutting down.
The job count may be 5, or it may be 20, depending on how the user acts in a certain way.
Jobs submitted to an executor service are buffered up until they can be scheduled on a thread for execution. So you may have a thread pool of, for example, 3 threads and 20 waiting jobs. No problem. The waiting jobs will be eventually executed when their time comes.
You may want to prioritize certain jobs, to be done ahead of lower priority jobs. One easy way to do this is to have two executor services. Each executor has its own backing thread pool. One executor is for the fewer but higher-priority jobs, while the other executor is for the many lower-priority jobs.
Remember that threads in a thread pool doing no work, on stand-by, have virtually no overhead in Java for either CPU or memory. So there is no downside to having a special higher-priority executor service sitting around and waiting for eventual jobs to arrive. The only concern is that your total number of all background threads and their workload not overwhelm your machine. Also, the implementation of the thread pool may well shut down unused threads after a period of disuse.

Don't really think its a question of the number of threads you are running, more how you want them run. If you want them run one at at time (i.e. you only want to execute on database query at a time) then use a HandlerThread. If you want multi-threading / a pool of threads, then use and Executor.
In my experience, leaks are really more down to how you have coded your threads, not really the chosen implementation.
Personally, I'd use a HandlerThread, here's a nice article on implementing them and how to avoid memory leaks ... Using HandlerThread in Android

Reason for calling shutdown() on ExecutorService

I was reading about it quite a bit in the past couple of hours, and I simply cannot see any reason (valid reason) to call shutdown() on the ExecutorService, unless we have a humongous application that stores, dozens and dozens of different executor services that are not used for a long time.
The only thing (from what I gather) the shutdown does, is doing what a normal Thread does once it's done. When the normal Thread will finish the run method of the Runnable(or Callable), it will be passed to Garbage Collection to be collected. With Executor Service the threads will simply be put on hold, they will not be ticked for the garbage collection. For that, the shutdown is needed.
Ok back to my question. Is there any reason to call shutdown on ExecutorService very often, or even right after submitting to it some tasks? I would like to leave behind the case someone is doing it and right after that calls to awaitTermination() as this is validated. Once we do that, we have to recreate a new ExecutorService all over again, to do the same thing. Isn't the whole idea for the ExecutorService to reuse the threads? So why destroy the ExecutorService so soon?
Isn't it a rational way to simply create ExecutorService (or couple depending on how many you need), then during the application running pass to them the tasks once they come along, and then on the application exit or some other important stages shutdown those executors?
I'd like an answer from some experienced coders who do write a lot of asynchronous code using the ExecutorServices.
Second side question, a bit smaller deals with the android platform. IF some of you will say that it's not the best idea to shutdown executors every time, and your program on android, could you tell me how do you handle those shutdowns (to be specific - when you execute them) when we deal with different events of the application life cycle.
Because of the CommonsWare comment, I made the post neutral. I really am not interested in arguing about it to death and it seems it's leading there. I'm only interested in learning about what I asked here from experienced developers if they are willing to share their experiences. Thanks.

The shutdown() method does one thing: prevents clients to send more work to the executor service. This means all the existing tasks will still run to completion unless other actions are taken. This is true even for scheduled tasks, e.g., for a ScheduledExecutorService: new instances of the scheduled task won't run. It also frees up any background thread resources. This can be useful in various scenarios.
Let's assume you have a console application which has an executor service running N tasks. If the user hits CTRL-C, you expect the application to terminate, possibly gracefully. What does it mean gracefully? Maybe you want your application to not be able to submit more tasks to the executor service and at the same time you want to wait for your existing N tasks to complete. You could achieve this using a shutdown hook as a last resort:
final ExecutorService service = ... // get it somewhere
Runtime.getRuntime().addShutdownHook(new Thread(new Runnable() {
#Override
public void run() {
System.out.println("Performing some shutdown cleanup...");
service.shutdown();
while (true) {
try {
System.out.println("Waiting for the service to terminate...");
if (service.awaitTermination(5, TimeUnit.SECONDS)) {
break;
}
} catch (InterruptedException e) {
}
}
System.out.println("Done cleaning");
}
}));
This hook will shutdown the service, which will prevent your application to submit new tasks, and wait for all the existing tasks to complete before shutting down the JVM. The await termination will block for 5 seconds and return true if the service is shutdown. This is done in a loop so that you're sure the service will shutdown eventually. The InterruptedException gets swallowed each time. This is the best way to shutdown an executor service that gets reused all over your application.
This code isn't perfect. Unless you're absolutely positive your tasks will eventually terminate, you might want to wait for a given timeout and then just exit, abandoning the running threads. In this case it would make sense to also call shutdownNow() after the timeout in a final attempt to interrupt the running threads (shutdownNow() will also give you a list of tasks waiting to run). If your tasks are designed to respond to interruption this will work fine.
Another interesting scenario is when you have a ScheduledExecutorService that performs a periodic task. The only way to stop the chain of periodic tasks is to call shutdown().
EDIT: I'd like to add that I wouldn't recommend using a shutdown hook as shown above in the general case: it can be error-prone and should be a last resort only. Moreover, if you have many shutdown hooks registered, the order in which they will run is undefined, which might be undesirable. I'd rather have the application explicitly call shutdown() on InterruptedException.

Isn't the whole idea for the ExecutorService to reuse the threads? So why destroy the ExecutorService so soon?
Yes. You should not destroy and re-create ExecutorService frequently. Initialize ExecutorService when you require (mostly on start-up) and keep it active until you are done with it.
Isn't it a rational way to simply create ExecutorService (or couple depending on how many you need), then during the application running pass to them the tasks once they come along, and then on the application exit or some other important stages shutdown those executors?
Yes. It's rational to shutdown ExecutorService on important stages like application exit etc.
Second side question, a bit smaller deals with android platform. IF some of you will say that it's not best idea to shutdown executors every time, and you program on android, could you tell me how you handle those shutdowns (to be specific, when you execute them) when we deal with different events of application life cycle.
Assume that ExecutorService is shared across different Activities in your application. Each activity will be paused/resumed at different intervals of time and still you need one ExecutorService per your application.
Instead of managing the state of ExecutorService in Activity life cycle methods, move ExecutorService management ( Creation/Shutdown) to your custom Service.
Create ExecutorService in Service => onCreate() and shutdown it properly in onDestroy()
Recommended way of shutting down ExecutorService :
How to properly shutdown java ExecutorService

An ExecutorService should be shut down once it is no longer needed to
free up system resources and to allow graceful application shutdown.
Because the threads in an ExecutorService may be nondaemon threads,
they may prevent normal application termination. In other words, your
application stays running after completing its main method.
Reference Book
Chaper:14
Page:814

Reason for calling shutdown() on ExecutorService
Today I encountered a situation where I have to wait until a machine is ready, before starting a series of tasks on that machine.
I make a REST call to this machine, if I don't receive 503 (Server Unavailable) then the machine is ready to process my requests. So, I wait until I get 200 (Success) for the first REST call.
There are multiple ways to achieve it, I used ExecutorService to create a thread and scheduled it to run after every X Seconds. So, I need to stop this thread on a condition, check this out...
final ScheduledExecutorService executor = Executors.newScheduledThreadPool(1);
Runnable task = () -> {
try {
int statusCode = restHelper.firstRESTCall();
if (statusCode == 200) {
executor.shutdown();
}
} catch (Exception e) {
e.printStackTrace();
}
};
int retryAfter = 60;
executor.scheduleAtFixedRate(task, 0, retryAfter, TimeUnit.SECONDS);
Second side question, a bit smaller deals with android platform.
Maybe I can answer if you'll provide bit more context!
Also from my experience with Android development it's rarely you need Threads. Are you developing a Game or an app which needs threads for performance? If not, in Android you have other ways to tackle problems like the scenario that I explained above. You can rather use TimerTask, AsyncTask or Handlers or Loaders based on context. This is because if UIThread waits for long you know what happens :/

This is genuine notwithstanding for planned undertakings, e.g., for a ScheduledExecutorService: new cases of the booked assignment won't run.
We should expect you have a comfort application which has an agent administration running N errands.
I'm not catching it's meaning effortlessly? Perhaps you need your application to not have the option to submit more assignments to the agent administration and in the meantime you need to sit tight for your current N undertakings to finish.
Except if you're totally positive your errands will in the end, you should need to sit tight for a given break and after that simply exit, deserting the running strings.
In the event that your activitys are intended to react to interference this will work fine.
Another intriguing situation is the point at which you have a ScheduledExecutorService that plays out an activity.
The best way to stop the chain of activity is to call shutdown()

How to use Thread Pool concept in Java?

I am creating a http proxy server in java. I have a class named Handler which is responsible for processing the requests and responses coming and going from web browser and to web server respectively. I have also another class named Copy which copies the inputStream object to outputStream object . Both these classes implement Runnable interface. I would like to use the concept of Thread pooling in my design, however i don't know how to go about that! Any hint or idea would be highly appreciated.

I suggest you look at Executor and ExecutorService. They add a lot of good stuff to make it easier to use Thread pools.
...
#Azad provided some good information and links. You should also buy and read the book Java Concurrency in Practice. (often abbreviated as JCiP) Note to stackoverflow big-wigs - how about some revenue link to Amazon???
Below is my brief summary of how to use and take advantage of ExecutorService with thread pools. Let's say you want 8 threads in the pool.
You can create one using the full featured constructors of ThreadPoolExecutor, e.g.
ExecutorService service = new ThreadPoolExecutor(8,8, more args here...);
or you can use the simpler but less customizable Executors factories, e.g.
ExecutorService service = Executors.newFixedThreadPool(8);
One advantage you immediately get is the ability to shutdown() or shutdownNow() the thread pool, and to check this status via isShutdown() or isTerminated().
If you don't care much about the Runnable you wish to run, or they are very well written, self-contained, never fail or log any errors appropriately, etc... you can call
execute(Runnable r);
If you do care about either the result (say, it calculates pi or downloads an image from a webpage) and/or you care if there was an Exception, you should use one of the submit methods that returns a Future. That allows you, at some time in the future, check if the task isDone() and to retrieve the result via get(). If there was an Exception, get() will throw it (wrapped in an ExecutionException). Note - even of your Future doesn't "return" anything (it is of type Void) it may still be good practice to call get() (ignoring the void result) to test for an Exception.
However, this checking the Future is a bit of chicken and egg problem. The whole point of a thread pool is to submit tasks without blocking. But Future.get() blocks, and Future.isDone() begs the questions of which thread is calling it, and what it does if it isn't done - do you sleep() and block?
If you are submitting a known chunk of related of tasks simultaneously, e.g., you are performing some big mathematical calculation like a matrix multiply that can be done in parallel, and there is no particular advantage to obtaining partial results, you can call invokeAll(). The calling thread will then block until all the tasks are complete, when you can call Future.get() on all the Futures.
What if the tasks are more disjointed, or you really want to use the partial results? Use ExecutorCompletionService, which wraps an ExecutorService. As tasks get completed, they are added to a queue. This makes it easy for a single thread to poll and remove events from the queue. JCiP has a great example of an web page app that downloads all the images in parallel, and renders them as soon as they become available for responsiveness.

I hope below will help you:,
class Executor
An object that executes submitted Runnable tasks. This interface provides a way of decoupling task submission from the mechanics of how each task will be run, including details of thread use, scheduling, etc. An Executor is normally used instead of explicitly creating threads. For example, rather than invoking new Thread(new(RunnableTask())).start() for each of a set of tasks, you might use:
Executor executor = anExecutor;
executor.execute(new RunnableTask1());
executor.execute(new RunnableTask2());
...
class ScheduledThreadPoolExecutor
A ThreadPoolExecutor that can additionally schedule commands to run after a given delay, or to execute periodically. This class is preferable to Timer when multiple worker threads are needed, or when the additional flexibility or capabilities of ThreadPoolExecutor (which this class extends) are required.
Delayed tasks execute no sooner than they are enabled, but without any real-time guarantees about when, after they are enabled, they will commence. Tasks scheduled for exactly the same execution time are enabled in first-in-first-out (FIFO) order of submission.
and
Interface ExecutorService
An Executor that provides methods to manage termination and methods that can produce a Future for tracking progress of one or more asynchronous tasks.
An ExecutorService can be shut down, which will cause it to stop accepting new tasks. After being shut down, the executor will eventually terminate, at which point no tasks are actively executing, no tasks are awaiting execution, and no new tasks can be submitted.
Edited:
you can find example to use Executor and ExecutorService herehereand here Question will be useful for you.

What's the proper background process behaviour for a non-GUI Java app?

What's the proper way for a Java command line application to do background work without hogging resources? Should it use sleep() in the loop or is there a more elegant/efficient way?

Some heuristics:
Don't attempt to make scheduling decisions in your application. The operating system's scheduler is way better than yours will be. Let it do its job.
Don't poll if you don't have to. For instance, instead of sleeping n seconds, then waking up to check a non-blocked socket, block on the socket. This second strategy plays better with the operating system's scheduler.
Don't use an enormous heap if you don't have to, and try not to allocate enormous chunks of memory at one time. A thrashing application tends to have a negative effect on system performance.
Use buffered I/O. Always. If you think you need non-buffered I/O, be absolutely sure you're right. (You're probably wrong.)
Don't spawn a lot of threads. Threads are surprisingly expensive; beyond a certain point, more threads will reduce your application's performance profile. If you have lots of work to do concurrently, learn and use java.util.concurrent.
Of course, this is just a starter list...

I'd only use sleep() if there's no work to be done. For example, if you're doing something like polling a task queue periodically and there's nothing there, sleep for a while then check again, etc.
If you're just trying to make sure you don't hog the CPU but you're still doing real work, you could call Thread.yield() periodically. That will relinquish control of the CPU and let other threads run, but it won't put you to sleep. If other processes don't need the CPU you'll get control back and continue to do your work.
You can also set your thread to a low priority:
myThread.setPriority(Thread.MIN_PRIORITY);
As Ishmael said, don't do this in your main thread. Create a "worker thread" instead. That way your UI (GUI or CLI) will still be responsive.

There are several ways. I would use ExecutorService... for example:
ExecutorService service = Executors.newCachedThreadPool();
Callable<Result> task = new Callable<Result>() {
public Result call() throws Exception {
// code which will be run on background thread
}
};
Future<Result> future = service.submit(task);
// Next line wait until background task is complete
// without killing CPU. Of course, you can do something
// different here and check your 'future' later.
//
// Note also that future.get() may throw various exceptions too,
// you'll need to handle them properly
Result resultFromBackgroundThread = future.get();
This is Java 5 code, ExecutorService, Callable, Future and similar are in java.util.concurrent package.

One place to start is to make sure that only those resources are being used and no other objects (so that they become garbage collected).
Placing sleep() in a single threading application is only going to halt the current thread. If you're trying to accomplish data being processed in the background while information still needs to be presented to the user then it is best to put the background process in a seperate thread.