Develop Reference

Does process have any overhead in context switching compared to threads?

A few words about what I'm planing to do. I need to create some task executor, that will poll tasks from queue and just execute code in this task. And for this I need to implement some interrupt mechanism to enable user to stop this task.
So I see two possible solutions: 1. start a pool of threads and stop them by using .destroy() method of a thread. (I will not use any shared objects) 2. Use pool of separated processes and System.exit() or kill signal to process. Option 2. looks much safer for me as I can ensure that thread killing will not lead to any concurrency problems. But I'm not sure that it won't produce a big overhead.
Also I'm not sure about JVM, if I will use separated processes, each process will be using the separated JVM, and it can bring a lot of overhead. Or not. So my question in this. Choosing a different language without runtime for worker process is possible option for me, but I still don't have enough experience with processes and don't know about overhead.

start a pool of threads and stop them by using .destroy() method of a thread. (I will not use any shared objects)
You can't stop threads on modern VMs unless said thread is 'in on it'. destroy and friends do not actually do what you want and this is unsafe. The right way is to call interrupt(). If the thread wants to annoy you and not actually stop in the face of an interrupt call, they can. The solution is to fix the code so that it doesn't do that anymore. Note that raising the interrupt flag will guaranteed stop any method that is sleeping which is specced to throw InterruptedException (sleep, wait, etc), and on most OSes, will also cause any I/O call that is currently frozen to exit by throwing an IOException, but there is no guarantee for this.
Use pool of separated processes and System.exit() or kill signal to process.
Hella expensive; a VM is not a light thing to spin up; it'll have its own copy of all the classes (even something as simple as java.lang.String and company). 10 VMs is a stretch. Whereas 1000 threads is no problem.
And for this I need to implement some interrupt mechanism to enable user to stop this task.
The real problem is that this is very difficult to guarantee. But if you control the code that needs interrupting, then usually no big deal. Just use the interrupt() mechanism.
EDIT: In case you're wondering how to do the interrupt thing: Raising the interrupt flag on a thread just raises the flag; nothing else happens unless you write code that interacts with it, or call a method that does.
There are 3 main interactions:
All things that block and are declared to throw InterruptedEx will lower the flag and throw InterruptedEx. If the flag is up and you call Thread.sleep, that will immediately_ clear the flag and throw that exception without ever even waiting. Thus, catch that exception, and return/abort/break off the task.
Thread.interrupted() will lower the flag and return true (thus, does so only once). Put this in your event loops. It's not public void run() {while (true) { ... }} or while (running) {} or whatnot, it's while (!Thread.interrupted() or possibly while (running && !Thread.interrupted9)).
Any other blocking method may or may not; java intentionally doesn't specify either way because it depends on OS and architecture. If they do (and many do), they can't throw interruptedex, as e.g. FileInputStream.read isn't specced to throw it. They throw IOException with a message indicating an abort happened.
Ensure that these 3 code paths one way or another lead to a task that swiftly ends, and you have what you want: user-interruptible tasks.

Executors framework
Java already provides a facility with your desired features, the Executors framework.
You said:
I need to create some task executor, that will poll tasks from queue and just execute code in this task.
The ExecutorService interface does just that.
Choose an implementation meeting your needs from the Executors class. For example, if you want to run your tasks in the sequence of their submission, use a single-threaded executor service. You have several others to choose from if you want other behavior.
ExecutorService executorService = Executors.newSingleThreadExecutor() ;
You said:
start a pool of threads
The executor service may be backed by a pool of threads.
ExecutorService executorService = Executors.newFixedThreadPool​( 3 ) ; // Create a pool of exactly three threads to be used for any number of submitted tasks.
You said:
just execute code in this task
Define your task as a class implementing either Runnable or Callable. That means your class carries a run method, or a call method.
Runnable task = ( ) -> System.out.println( "Doing this work on a background thread. " + Instant.now() );
You said:
will poll tasks from queue
Submit your tasks to be run. You can submit many tasks, either of the same class or of different classes. The executor service maintains a queue of submitted tasks.
executorService.submit( task );
Optionally, you may capture the Future object returned.
Future future = executorService.submit( task );
That Future object lets you check to see if the task has finished or has been cancelled.
if( future.isDone() ) { … }
You said:
enable user to stop this task
If you want to cancel the task, call Future::cancel.
Pass true if you want to interrupt the task if it has already begun execution.
Pass false if you only want to cancel the task before it has begun execution.
future.cancel( true );
You said:
looks much safer for me as I can ensure that thread killing will not lead to any concurrency problems.
Using the Executors framework, you would not be creating or killing any threads. The executor service implementation handles the threads. Your code never addresses the Thread class directly.
So no concurrency problems of that kind.
But you may have other concurrency problems if you share any resources across threads. I highly recommend reading Java Concurrency in Practice by Brian Goetz et al.
You said:
But I'm not sure that it won't produce a big overhead.
As the correct Answer by rzwitserloot explained, your approach would certainly create much more overhead that would the use of the Executors framework.
FYI, in the future Project Loom will bring virtual threads (fibers) to the Java platform. This will generally make background threading even faster, and will make practical having many thousands or even millions of non-CPU-bound tasks. Special builds available now on early-access Java 16.
ExecutorService executorService = newVirtualThreadExecutor() ;
executorService.submit( task ) ;

RxJava Questions About Threads, CompositeDisposable and Scheduler in an Android Application

My code is like this:
timingObservable = getCurrentModule()
.zipWith(Observable.interval(200, TimeUnit.MILLISECONDS), (currentModule, interval) -> currentModule)
.repeat()
.distinctUntilChanged()
.getModuleDataFromDb()
compositeDisposable.add(timingObservable
.subscribeOn(Schedulers.io())
.subscribe(next -> {
.
.
.
}));
public Observable<String> getCurrentModule() {
return Observable.fromCallable(() -> {
String currentModule = "";
// doing some none database work and computation
.
.
.
return currentModule;
}
}
It is supposed to check the current module periodically and get some data from db if the module is changed. I have several questions:
In the RxThreadFactory class of RxJava and in the newThread() method we have the line t.setDaemon(true), so is it true that all RxJava threads are daemon threads? So, they are alive as long as a component of app is alive and the app process is still running, right?
I am adding the disposable return of subscribe() to a compositeDisposable and call dispose in onDestory() of my Service/Activity classes. What happens to those disposables when the service gets killed without onDestroy() being called. I mean, since the compositeDisposable object is destoryed, is it possible that I lose the ability to dispose disposables? Should I hold application wide instance of CompositeDisposable?
In terms of performance, which one is recommended in this code? subscribeOn(Schedulers.io()) or subscribeOn(Schedulers.computation()) since Observable.interval uses computation scheduler by default and we have DB work too.
Any suggestions to improve the above code for periodic tasks?

To answer the questions :
In the RxThreadFactory class of RxJava and in the newThread() method we have the line t.setDaemon(true), so is it true that all RxJava threads are daemon threads? So, they are alive as long as a component of app is alive and the app process is still running, right?
In Java Thread::setDaemon(true) simply means that once all non-daemon threads have finished then these "daemon" threads are abandoned and the JVM shut down. For reference the android "main" Thread is not a daemon thread, but has a Looper. Daemon threads can naturally finish and do not stop the process from exiting. You should not rely on this mechanism for long running tasks and use foreground services and/or WorkManager, Rx thread pools could last as long as the process they are running in, unless the Executor they are tied to is explicitly shutdown.
I am adding the disposable return of subscribe() to a compositeDisposable and call dispose in onDestory() of my Service/Activity classes. Lets say in a scenario there are one service and one activity and compositeDisposable belongs to the service. What happens to those disposables when the service gets killed without onDestroy() being called and activity remains alive. I mean, since the compositeDisposable object is destoryed, is it possible that I lose the ability to dispose disposables? Should I hold application wide instance of CompositeDisposable?
A Service would only get destroyed without the lifecycle call backs if 1) Android kills the process to reclaim resources, in that case its not relevant to clear resources, or 2) The program crashes, and again resources cleanup is not required
In terms of performance, which one is recommended in this code? subscribeOn(Schedulers.io()) or subscribeOn(Schedulers.computation()) since Observable.interval uses computation scheduler by default and we have DB work too.
Schedulers.io is an unbounded thread pool, whilst Schedulers.computation() is bounded (a pool of 8 threads I believe). Performance could vary, but in most cases negligible difference. One scenario Schedulers.computation() may be slower could be if you have a lot of concurrency using this thread pool continuously, meaning you are waiting for a Thread to become free. In contrast Schedulers.io() could have a upfront cost of creating new threads from the ThreadFactory it uses. However it will try to use existing thread from its pool. Metrics would be needed to really see any performance differences for individual use cases. By general rule io() suggests that for work such as file/database/networking should be done using this Scheduler and computation work like timers/algorithms should use the latter.
Any suggestions to improve the above code for periodic tasks?
As already suggested polling, generally is not a good idea, and Reactive Streams by concept is more useful as a pub/sub observer pattern. However it seems you have some restrictions as to what you can modify. With the provided code there is far too little context to really give any concrete improvements, and only suggestions, which will ultimately end up with "I can't do that because...".

When the executorService.shutdown(); should be called

We have a service method GetDataParallel( ) which maybe called by many clients currently, and we use the ExecutorService to called the MyCallable inside it. However I found unless I called the executorService.shutdown(); the application never exit, so why the application cannot exit , we must shut down all thread pool threads manually before the application exit? and in service environment I think we don't need to call the executorService.shutdown(); to keep the application alive, right ?
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
public class MultiThreading {
static ExecutorService executorService = Executors.newFixedThreadPool(100);
private List<String> _BusinessUnits= new ArrayList<String>();
public static void main(String[] args) throws Exception {
MultiThreading kl =new MultiThreading();
kl.GetDataParallel();
Thread.sleep(10000);
System.out.println("111111111");
//executorService.shutdown();
}
public void GetDataParallel( ) throws Exception
{
_BusinessUnits.add("BU1");
_BusinessUnits.add("BU2");
_BusinessUnits.add("BU3");
for(final String v : _BusinessUnits)
{
ExecutorServiceTest.executorService.submit( new MyCallable());
}
}
}
class MyCallable implements Callable {
#Override
public String call() throws Exception {
Thread.sleep(1000);
//return the thread name executing this callable task
System.out.println(Thread.currentThread().getName());
return Thread.currentThread().getName();
}
}

Typically you shut down an ExecutorService when the application is exiting - most applications have some sort of lifecycle and shutdown sequence. If you expect your application to exit when the main thread completes whatever it has to do, you want to shut it down when its work is done (meaning there's a well-defined point at which it's work is done, and you can tell when that is).
Server-side frameworks frequently have lifecycle methods that you can hook into to detect when your code is being shut down and cleanly exit. Or you can use VM shutdown hooks (perhaps to delay shutdown until all currently queued jobs are complete), so that no matter what code causes the program to exit, your cleanup code will be run.
In general, it's a good idea to create a well-defined exit point if there isn't one provided by a framework - it lets you have an application that can be cleanly unloaded (and perhaps, reloaded with updated configuration) without the VM necessarily shutting down at all - I've used that trick to be able to have a server application reconfigure itself and reload with zero downtime in response to a Unix signal.
So, the simple answer to your question is "When it isn't going to be used anymore". In almost any application there is a point at which that's unambiguously true.
BTW, to contradict one of the other responders, an ExecutorService can use daemon threads - you can provide a ThreadFactory that configures threads however you want before you start them. But I'd encourage you to not use daemon threads and explicitly shut down the thread pool at a well-defined point - that's not the typical practice, but it will both mean your code has the potential to be shut down cleanly, and it will encourage you to think about lifecycle, which is likely to lead to better code.

There are two flavors of threads in Java (of course depending on how you look at them). 'User' threads and 'Daemon' threads. You application ends in one of the following cases:
You call System.exit()
You have no User threads left in your application. This is explained here.
Note that your main function is executed by the JVM on a 'User' thread, meaning that as long as you have not completed your main function. Most multithreaded applications will run the main function only to start all the threads needed.
The idea behind Daemon threads is that you can do something (regularly), but if all other tasks are done, it will not prevent the application from exiting.
By default new threads are 'Non Daemon' threads, the same goes for the threads craeted by your ExecutorService. If you want to change this, you have to create your own ThreadFactory. A ThreadFactory allows you to manually create the threads for your ExecutorService, it will be called when the ExecutorService needs a new thread. Here is an example of one that created 'Daemon' threads:
public class DaemonThreadFactory implements ThreadFactory
{
#Override
public Thread newThread(final Runnable r)
{
Thread t = new Thread(r);
t.setDaemon(true);
return t;
}
}
This can then be used by creating the executor service:
ExecutorService service = Executors.newFixedThreadPool(100, new DaemonThreadFactory());
Note that this is also the way to give your threads custom names, which is very useful as many logframeworks log the thread name (and the debugger shows it).
If you were to do this, in your application, it would exit right away, because you only create 'Daemon' threads, so you would have to either keep another thread alive (this could be done implicitly by another framework, for instance if you have GUI).
An alternative is to manually call System.exit(). Normally calling System.exit() in your code is not recommended. Mostly because it does not allow for good refactoring, reusing, testing and many exit points make your application unpredictable. In order to circumvent these problems, you could create a callback function that handles the job complete event. In this application you call System.exit(), your test code or other applications could do something else.

In application environment, you must call shutdown to ensure threads launched by ExecutorService must stop and it should not accept any more new tasks. Otherwise JVM will not exit.
In case of Service, you should call shutdown prior to stopping your Service execution. For e.g. in case of web-app, the contextDestroyed method of ServletContextListener is useful to invoke shutdown method of ExecutorService. This will ensure that any existing tasks must be completed prior to abrupt termination of application, but no new task will be accepted for processing.

However I found unless I called the executorService.shutdown(); the application never exit,
I think we don't need to call the executorService.shutdown(); to keep the application alive, right ?
Yes. Application is alive unless you call executorService.shutdown()
Every application should have an exit point. You can call shutdown during that exit point.
If you don't have any known exit points, ShutdownHook is one way to address your problem. But you should be aware that
In rare circumstances the virtual machine may abort, that is, stop running without shutting down cleanly
For the example you have quoted, you can address it in multiple ways (invokeAll, Future.get(), CountDownLatch etc). Have a look at related SE question.
ExecutorService, how to wait for all tasks to finish

Reason for calling shutdown() on ExecutorService

I was reading about it quite a bit in the past couple of hours, and I simply cannot see any reason (valid reason) to call shutdown() on the ExecutorService, unless we have a humongous application that stores, dozens and dozens of different executor services that are not used for a long time.
The only thing (from what I gather) the shutdown does, is doing what a normal Thread does once it's done. When the normal Thread will finish the run method of the Runnable(or Callable), it will be passed to Garbage Collection to be collected. With Executor Service the threads will simply be put on hold, they will not be ticked for the garbage collection. For that, the shutdown is needed.
Ok back to my question. Is there any reason to call shutdown on ExecutorService very often, or even right after submitting to it some tasks? I would like to leave behind the case someone is doing it and right after that calls to awaitTermination() as this is validated. Once we do that, we have to recreate a new ExecutorService all over again, to do the same thing. Isn't the whole idea for the ExecutorService to reuse the threads? So why destroy the ExecutorService so soon?
Isn't it a rational way to simply create ExecutorService (or couple depending on how many you need), then during the application running pass to them the tasks once they come along, and then on the application exit or some other important stages shutdown those executors?
I'd like an answer from some experienced coders who do write a lot of asynchronous code using the ExecutorServices.
Second side question, a bit smaller deals with the android platform. IF some of you will say that it's not the best idea to shutdown executors every time, and your program on android, could you tell me how do you handle those shutdowns (to be specific - when you execute them) when we deal with different events of the application life cycle.
Because of the CommonsWare comment, I made the post neutral. I really am not interested in arguing about it to death and it seems it's leading there. I'm only interested in learning about what I asked here from experienced developers if they are willing to share their experiences. Thanks.

The shutdown() method does one thing: prevents clients to send more work to the executor service. This means all the existing tasks will still run to completion unless other actions are taken. This is true even for scheduled tasks, e.g., for a ScheduledExecutorService: new instances of the scheduled task won't run. It also frees up any background thread resources. This can be useful in various scenarios.
Let's assume you have a console application which has an executor service running N tasks. If the user hits CTRL-C, you expect the application to terminate, possibly gracefully. What does it mean gracefully? Maybe you want your application to not be able to submit more tasks to the executor service and at the same time you want to wait for your existing N tasks to complete. You could achieve this using a shutdown hook as a last resort:
final ExecutorService service = ... // get it somewhere
Runtime.getRuntime().addShutdownHook(new Thread(new Runnable() {
#Override
public void run() {
System.out.println("Performing some shutdown cleanup...");
service.shutdown();
while (true) {
try {
System.out.println("Waiting for the service to terminate...");
if (service.awaitTermination(5, TimeUnit.SECONDS)) {
break;
}
} catch (InterruptedException e) {
}
}
System.out.println("Done cleaning");
}
}));
This hook will shutdown the service, which will prevent your application to submit new tasks, and wait for all the existing tasks to complete before shutting down the JVM. The await termination will block for 5 seconds and return true if the service is shutdown. This is done in a loop so that you're sure the service will shutdown eventually. The InterruptedException gets swallowed each time. This is the best way to shutdown an executor service that gets reused all over your application.
This code isn't perfect. Unless you're absolutely positive your tasks will eventually terminate, you might want to wait for a given timeout and then just exit, abandoning the running threads. In this case it would make sense to also call shutdownNow() after the timeout in a final attempt to interrupt the running threads (shutdownNow() will also give you a list of tasks waiting to run). If your tasks are designed to respond to interruption this will work fine.
Another interesting scenario is when you have a ScheduledExecutorService that performs a periodic task. The only way to stop the chain of periodic tasks is to call shutdown().
EDIT: I'd like to add that I wouldn't recommend using a shutdown hook as shown above in the general case: it can be error-prone and should be a last resort only. Moreover, if you have many shutdown hooks registered, the order in which they will run is undefined, which might be undesirable. I'd rather have the application explicitly call shutdown() on InterruptedException.

Isn't the whole idea for the ExecutorService to reuse the threads? So why destroy the ExecutorService so soon?
Yes. You should not destroy and re-create ExecutorService frequently. Initialize ExecutorService when you require (mostly on start-up) and keep it active until you are done with it.
Isn't it a rational way to simply create ExecutorService (or couple depending on how many you need), then during the application running pass to them the tasks once they come along, and then on the application exit or some other important stages shutdown those executors?
Yes. It's rational to shutdown ExecutorService on important stages like application exit etc.
Second side question, a bit smaller deals with android platform. IF some of you will say that it's not best idea to shutdown executors every time, and you program on android, could you tell me how you handle those shutdowns (to be specific, when you execute them) when we deal with different events of application life cycle.
Assume that ExecutorService is shared across different Activities in your application. Each activity will be paused/resumed at different intervals of time and still you need one ExecutorService per your application.
Instead of managing the state of ExecutorService in Activity life cycle methods, move ExecutorService management ( Creation/Shutdown) to your custom Service.
Create ExecutorService in Service => onCreate() and shutdown it properly in onDestroy()
Recommended way of shutting down ExecutorService :
How to properly shutdown java ExecutorService

An ExecutorService should be shut down once it is no longer needed to
free up system resources and to allow graceful application shutdown.
Because the threads in an ExecutorService may be nondaemon threads,
they may prevent normal application termination. In other words, your
application stays running after completing its main method.
Reference Book
Chaper:14
Page:814

Reason for calling shutdown() on ExecutorService
Today I encountered a situation where I have to wait until a machine is ready, before starting a series of tasks on that machine.
I make a REST call to this machine, if I don't receive 503 (Server Unavailable) then the machine is ready to process my requests. So, I wait until I get 200 (Success) for the first REST call.
There are multiple ways to achieve it, I used ExecutorService to create a thread and scheduled it to run after every X Seconds. So, I need to stop this thread on a condition, check this out...
final ScheduledExecutorService executor = Executors.newScheduledThreadPool(1);
Runnable task = () -> {
try {
int statusCode = restHelper.firstRESTCall();
if (statusCode == 200) {
executor.shutdown();
}
} catch (Exception e) {
e.printStackTrace();
}
};
int retryAfter = 60;
executor.scheduleAtFixedRate(task, 0, retryAfter, TimeUnit.SECONDS);
Second side question, a bit smaller deals with android platform.
Maybe I can answer if you'll provide bit more context!
Also from my experience with Android development it's rarely you need Threads. Are you developing a Game or an app which needs threads for performance? If not, in Android you have other ways to tackle problems like the scenario that I explained above. You can rather use TimerTask, AsyncTask or Handlers or Loaders based on context. This is because if UIThread waits for long you know what happens :/

This is genuine notwithstanding for planned undertakings, e.g., for a ScheduledExecutorService: new cases of the booked assignment won't run.
We should expect you have a comfort application which has an agent administration running N errands.
I'm not catching it's meaning effortlessly? Perhaps you need your application to not have the option to submit more assignments to the agent administration and in the meantime you need to sit tight for your current N undertakings to finish.
Except if you're totally positive your errands will in the end, you should need to sit tight for a given break and after that simply exit, deserting the running strings.
In the event that your activitys are intended to react to interference this will work fine.
Another intriguing situation is the point at which you have a ScheduledExecutorService that plays out an activity.
The best way to stop the chain of activity is to call shutdown()

How to terminate container-created threads in Java

I am running a multi-threaded Java web application on Apache Tomcat 6. Instead of using the new Thread(); anti-pattern, I leave thread instantiation to Tomcat (see code below).
I noticed in the last days that the web application gets slower and slower. After restarting the servlet container everything is back to normal.
Since I am not terminating threads after processing them (Don't know if I have to or if the Garbage Collector will destroy them), I am guessing that this is the cause for the performance loss.
The code basically looks like this:
Custom Server Listener (I added this to web.xml):
public class MyTaskRunner implements ServletContextListener {
public static final ExecutorService EXECUTOR_SERVICE = ExecutorService.newFixedThreadPool(10000);
public void contextDestroyed(ServletContextEvent sce) {
EXECUTOR_SERVICE.shutdownNow();
}
public void contextInitialized(ServletContextEvent sce) {
}
}
Thread instantiation:
for (Object foo : bar){
MyTaskRunner.EXECUTOR_SERVICE.submit(new Runnable() {
public void run() {
doSomethingWith(foo);
});
}
So, is there anything special that I have to do after run() has finished?

Some basic facts about threads and GC:
While a thread is running, it will not be garbage collected.
When a thread is terminated, its stack is deleted and its Thread object is removed from the ThreadGroup data structures. The remainder of the threads' state is subject to the normal reachability rules.
You don't need to do anything special to make a thread terminate. It happens when the run method call ends, either because it returned, or because it exited with an (uncaught) exception.
Now to your particular problem. Many things could be causing your performance degradation. For instance:
Some of your threads may be getting stuck; e.g. waiting on a lock, waiting for a notify that isn't going to arrive, or just stuck in an infinite CPU loop.
You may have too many threads running (doing useful, or semi-useful things), and the slowdown may be a result of lock contention, CPU resource starvation, or thrashing.
You may have a memory leak of some kind, and the slowdown may be a symptom of your application starting to run out of heap space.
To figure out what is going on, you are going to need to do some performance monitoring:
Look at the OS level stats to see if the application is thrashing.
Use a memory profile to see if your application is short of memory. If it is, see if there is a memory leak, and track it down.
Use a performance profiler to see if there are particular hotspots, or if there is a lot of lock contention.
I agree with the comment. A thread pool size of 10000 is dangerously large. It should be a small constant multiplied by the number of cores available to your application.

Are you registering the context listener in web.xml correctly?
Does your run methods end at some point? or do they keep running indefinitely?
The ExecutorSerfice will simply call Thread.start for you, you are still starting threads from inside your webapp, which is not inherently such a terrible thing as long as you shut them down properly. No matter what tecnique you use, the execution of your run() method will not be truncated.

new Thread(); anti-pattern
where do you people get that this is an anti-pattern?
anyway, 10000 threads is way too much. Each thread takes up memory and OS has its own limitations, so, as a safe path, try not to exceed a couple thousand threads.
connect jconsole (jstack might be enough) to see what your threads are doing - whether they're just sleeping, or waiting for something.
and it's not Tomcat who's managing the threads, it's ExecutorService inside java.