Develop Reference

What would happen to the threads managed by ExecutorService when tomcat shutting down?

I have an web app(with Spring/Spring boot) running on tomcat 7.There are some ExecutorService defined like:
public static final ExecutorService TEST_SERVICE = new ThreadPoolExecutor(10, 100, 60L,
TimeUnit.SECONDS, new LinkedBlockingQueue<Runnable>(1000), new ThreadPoolExecutor.CallerRunsPolicy());
The tasks are important and must complete properly. I catch the exceptions and save them to db for retry, like this:
try {
ThreadPoolHolder.TEST_SERVICE.submit(new Runnable() {
#Override
public void run() {
try {
boolean isSuccess = false;
int tryCount = 0;
while (++tryCount < CAS_COUNT_LIMIT) {
isSuccess = doWork(param);
if (isSuccess) {
break;
}
Thread.sleep(1000);
}
if (!isSuccess) {
saveFail(param);
}
} catch (Exception e) {
log.error("test error! param : {}", param, e);
saveFail(param);
}
}
});
} catch (Exception e) {
log.error("test error! param:{}", param, e);
saveFail(param);
}
So, when tomcat shutting down, what will happen to the threads of the pool(running or waiting in the queue)? how to make sure that all the tasks either completed properly before shutdown or saved to db for retry?

Tomcat has built in Thread Leak detection, so you should get an error when the application is undeployed. As a developer it is your responsibility to link any object you create to the web applications lifecycle, this means You should never ever have static state which are not constants
If you are using Spring Boot, your Spring context is already linked to the applications lifecycle, so the best way is to create you executor as a Spring bean, and let Spring shut it down when the application stops. Here is an example you can put in any #Configuration class.
#Bean(destroyMethod = "shutdownNow", name = "MyExecutorService")
public ThreadPoolExecutor executor() {
ThreadPoolExecutor threadPoolExecutor = new ThreadPoolExecutor(10, 100, 60L,
TimeUnit.SECONDS, new LinkedBlockingQueue<Runnable>(1000),
new ThreadPoolExecutor.CallerRunsPolicy());
return threadPoolExecutor;
}
As you can see the #Bean annotation allows you to specify a destroy method which will be executed when the Spring context is closed. In addition I have added the name property, this is because Spring typically creates a number of ExecutorServices for stuff like async web processing. When you need to use the executor, just Autowire it as any other spring bean.
#Autowired
#Qualifier(value = "MyExecutorService")
ThreadPoolExecutor executor;
Remember static is EVIL, you should only use static for constants, and potentially immutable obbjects.
EDIT
If you need to block the Tomcats shutdown procedure until the tasks have been processed, you need to wrap the Executor in a Component for more control, like this.
#Component
public class ExecutorWrapper implements DisposableBean {
private final ThreadPoolExecutor threadPoolExecutor;
public ExecutorWrapper() {
threadPoolExecutor = new ThreadPoolExecutor(10, 100, 60L,
TimeUnit.SECONDS, new LinkedBlockingQueue<Runnable>(1000), new ThreadPoolExecutor.CallerRunsPolicy());
}
public <T> Future<T> submit(Callable<T> task) {
return threadPoolExecutor.submit(task);
}
public void submit(Runnable runnable) {
threadPoolExecutor.submit(runnable);
}
#Override
public void destroy() throws Exception {
threadPoolExecutor.shutdown();
boolean terminated = threadPoolExecutor.awaitTermination(1, TimeUnit.MINUTES);
if (!terminated) {
List<Runnable> runnables = threadPoolExecutor.shutdownNow();
// log the runnables that were not executed
}
}
}
With this code you call shutdown first so no new tasks can be submitted, then wait some time for the executor finish the current task and queue. If it does not finish in time you call shutdownNow to interrupt the running task, and get the list of unprocessed tasks.
Note: DisposableBean does the trick, but the best solution is actually to implement the SmartLifecycle interface. You have to implement a few more methods, but you get greater control, because no threads are started until all bean have been instantiated and the entire bean hierarchy is wired together, it even allows you to specify in which orders components should be started.

Tomcat as any Java application will not end untill all non-daeon threads will end. ThreadPoolExecutor in above example uses default thread factory and will create non-daemon threads.

Call ThreadPoolExecutor.execute but not really executed

I'm using Java's ThreadPoolExecutor in my project.
The constructor method is as below:
threadBlockingQueue = new ArrayBlockingQueue(100);
threadPoolExecutor = new ThreadPoolExecutor(2, 4, 100, TimeUnit.SECONDS, threadBlockingQueue, new RejectedExecutionHandler() {
#Override
public void rejectedExecution(Runnable r, ThreadPoolExecutor executor) {
LOGGER.error("Thread Pool Reject Job");
}
});
I call ThreadPoolExecutor.execute(Runnable), but finally some of the tasks has not been executed at all. Most of the tasks could be executed, but still a few disappeared without any exception in the log. And I also did not find any log in RejectedExecutionHandler which I passed as a parameter of ThreadPoolExecutor's contructor method.
Failed to find any clue for this issue. Is there any one who encountered this issue before?
Thanks.

RejectedExecutionHandler is called when quueue is full, you can implement your own and pass it to TheadPoolExecutor constructor.
The following one add the Runnable into the queue once space becomes available.
The following implementation will execute all tasks:
new RejectedExecutionHandler {
public void rejectedExecution(Runnable r, ThreadPoolExecutor executor) {
try {
executor.getQueue().put(r); // waiting if necessary for space to become available
} catch(InterruptedException ex) {
throw new RejectedExecutionException(ex);
}
}
Another way to not loose tasks is to use the right Policy as RejectedExecutionHandler:
java.util.concurrent.ThreadPoolExecutor.CallerRunsPolicy
will runs the rejected task directly in the calling thread of the execute method.
java.util.concurrent.ThreadPoolExecutor#execute

Detecting a timed out Callable instance

I am performing some operations that are time sensitive and have a timeout associated with them.
This timeout mechanism is implemented using the Java Callable class.
The problem is that within the callable instance I execute an asynchronous task, with an anonymous interface implementation (listener).
My problem is that when the timeout triggers and the callable is cancelled. The async callback still executes and corrupts the state of my program.
How do I prevent these callbacks from firing? Do I just include a boolean specififying whether or not the timeout has ocurred or is there another way of achieving this please?
Thanks.
Code reference:
Callable<Object> callableTransaction = new Callable<Object>() {
#Override
public Object call() throws Exception {
Callback callback = new Callback() {
// Do stuff here and change program state.
};
performAsyncOperation(callback);
return ActionProcessor.OPERATION_COMPLETE;
}
};

Why don't you remove performAsyncOperation and perform the operation synchronously inside the call method.
And then, if you need any async operation, you can invoke your callableTransaction using performAsyncOperation or executor services.

Wrap your Callable in a java.util.concurrent.FutureTask... call task.cancel(true) when you want to cancel things. If your Callable is in a blocking operation (I/O, for example), then an exception will be thrown marking the interruption. Otherwise, your thread will need to check using Thread.isInterrupted() periodically to see if it should continue or abort.
Yes, this is mostly the same as including your own boolean flag, but you do get the benefit of blocking operations getting interrupted as well.

You have the right idea: Use a boolean flag to track whether the Callable has been interrupted. (I'm assuming that when you cancel the Callable, you are specifying that it should be interrupted.)
Use a concurrent class like CountDownLatch to make your Callable behave synchronously:
public Object call() throws Exception {
final AtomicBoolean aborted = new AtomicBoolean();
final CountDownLatch latch = new CountDownLatch(1);
Callback callback = new Callback() {
if (aborted.get()) {
return;
}
// Do stuff here and change program state.
latch.countDown(); // Tell Callable we're done.
};
performAsyncOperation(callback);
try {
latch.await(); // Wait for callback to finish.
} catch (InterruptedException e) {
aborted.set(true);
}
return ActionProcessor.OPERATION_COMPLETE;
}

What is the difference between ExecutorService.submit and ExecutorService.execute in this code in Java?

I am learning to use ExectorService to pool threads and send out tasks. I have a simple program below
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;
class Processor implements Runnable {
private int id;
public Processor(int id) {
this.id = id;
}
public void run() {
System.out.println("Starting: " + id);
try {
Thread.sleep(5000);
} catch (InterruptedException e) {
System.out.println("sorry, being interupted, good bye!");
System.out.println("Interrupted " + Thread.currentThread().getName());
e.printStackTrace();
}
System.out.println("Completed: " + id);
}
}
public class ExecutorExample {
public static void main(String[] args) {
Boolean isCompleted = false;
ExecutorService executor = Executors.newFixedThreadPool(2);
for (int i = 0; i < 5; i++) {
executor.execute(new Processor(i));
}
//executor does not accept any more tasks but the submitted tasks continue
executor.shutdown();
System.out.println("All tasks submitted.");
try {
//wait for the exectutor to terminate normally, which will return true
//if timeout happens, returns false, but this does NOT interrupt the threads
isCompleted = executor.awaitTermination(100, TimeUnit.SECONDS);
//this will interrupt thread it manages. catch the interrupted exception in the threads
//If not, threads will run forever and executor will never be able to shutdown.
executor.shutdownNow();
} catch (InterruptedException e) {
}
if (isCompleted) {
System.out.println("All tasks completed.");
} else {
System.out.println("Timeout " + Thread.currentThread().getName());
}
}
}
It does nothing fancy, but creates two threads and submits 5 tasks in total. After each thread completes its task, it takes the next one,
In the code above, I use executor.submit. I also changed to executor.execute. But I do not see any difference in the output. In what way are the submit and execute methods different?
This what the API says
Method submit extends base method Executor.execute(java.lang.Runnable) by creating and returning a Future that can be used to cancel execution and/or wait for completion. Methods invokeAny and invokeAll perform the most commonly useful forms of bulk execution, executing a collection of tasks and then waiting for at least one, or all, to complete. (Class ExecutorCompletionService can be used to write customized variants of these methods.)
But it's not clear to me as what it exactly means?

As you see from the JavaDoc execute(Runnable) does not return anything.
However, submit(Callable<T>) returns a Future object which allows a way for you to programatically cancel the running thread later as well as get the T that is returned when the Callable completes. See JavaDoc of Future for more details
Future<?> future = executor.submit(longRunningJob);
...
//long running job is taking too long
future.cancel(true);
Moreover,
if future.get() == null and doesn't throw any exception then Runnable executed successfully

The difference is that execute simply starts the task without any further ado, whereas submit returns a Future object to manage the task. You can do the following things with the Future object:
Cancel the task prematurely, with the cancel method.
Wait for the task to finish executing, with get.
The Future interface is more useful if you submit a Callable to the pool. The return value of the call method will be returned when you call Future.get. If you don't maintain a reference to the Future, there is no difference.

execute: Use it for fire and forget calls
submit: Use it to inspect the result of method call and take appropriate action on Future objected returned by the call
Major difference: Exception handling
submit() hides un-handled Exception in framework itself.
execute() throws un-handled Exception.
Solution for handling Exceptions with submit()
Wrap your Callable or Runnable code in try{} catch{} block
OR
Keep future.get() call in try{} catch{} block
OR
implement your own ThreadPoolExecutor and override afterExecute method
Regarding tour other queries on
invokeAll:
Executes the given tasks, returning a list of Futures holding their status and results when all complete or the timeout expires, whichever happens first.
invokeAny:
Executes the given tasks, returning the result of one that has completed successfully (i.e., without throwing an exception), if any do before the given timeout elapses.
Use invokeAll if you want to wait for all submitted tasks to complete.
Use invokeAny if you are looking for successful completion of one task out of N submitted tasks. In this case, tasks in progress will be cancelled if one of the tasks completes successfully.
Related post with code example:
Choose between ExecutorService's submit and ExecutorService's execute

A main difference between the submit() and execute() method is that ExecuterService.submit()can return result of computation because it has a return type of Future, but execute() method cannot return anything because it's return type is void. The core interface in Java 1.5's Executor framework is the Executor interface which defines the execute(Runnable task) method, whose primary purpose is to separate the task from its execution.
Any task submitted to Executor can be executed by the same thread, a worker thread from a thread pool or any other thread.
On the other hand, submit() method is defined in the ExecutorService interface which is a sub-interface of Executor and adds the functionality of terminating the thread pool, along with adding submit() method which can accept a Callable task and return a result of computation.
Similarities between the execute() and submit() as well:
Both submit() and execute() methods are used to submit a task to Executor framework for asynchronous execution.
Both submit() and execute() can accept a Runnable task.
You can access submit() and execute() from the ExecutorService interface because it also extends the Executor interface which declares the execute() method.
Apart from the fact that submit() method can return output and execute() cannot, following are other notable differences between these two key methods of Executor framework of Java 5.
The submit() can accept both Runnable and Callable task but execute() can only accept the Runnable task.
The submit() method is declared in ExecutorService interface while execute() method is declared in the Executor interface.
The return type of submit() method is a Future object but return type of execute() method is void.

If you check the source code, you will see that submit is sort of a wrapper on execute
public Future<?> submit(Runnable task) {
if (task == null) throw new NullPointerException();
RunnableFuture<Void> ftask = newTaskFor(task, null);
execute(ftask);
return ftask;
}

Submit - Returns Future object, which can be used to check result of submitted task. Can be used to cancel or to check isDone etc.
Execute - doesn't return anything.

The execute(Runnable command) is the implemented method from Interface Executor. It means just execute the command and gets nothing returned.
ExecutorService has its own methods for starting tasks: submit, invokeAny and invokeAll all of which have Callable instances as their main targets. Though there're methods having Runnable as input, actulaly Runnable will be adapted to Callable in the method. why Callable? Because we can get a Future<T> result after the task is submitted.
But when you transform a Runnable to a Callable, result you get is just the value you pass:
static final class RunnableAdapter<T> implements Callable<T> {
final Runnable task;
final T result;
RunnableAdapter(Runnable task, T result) {
this.task = task;
this.result = result;
}
public T call() {
task.run();
return result;
}
}
So, what's the point that we pass a Runnable to submit instead of just getting the result when the task is finished? Because there's a method which has only Runnable as parameter without a particular result.
Read the javadoc of Future:
If you would like to use a Future for the sake of cancellability but not provide a usable result, you can declare types of the form Future<?> and return null as a result of the underlying task.
So, if you just want to execute a Runnable task without any value returned, you can use execute().
if you want to run a Callable task, or
if you want to run a Runnable task with a specified result as the completion symbol, or
if you want to run a task and have the ability to cancel it,
you should use submit().

On top of previous responses, i.e.
execute(..) runs the task and forget about it
submit(...) returns a future;
The main advantage with the future is that you can establish a timeout. This might come very handy if you have an executor with a limited number of threads and your executions are taking forever, it will not hang the process.
Example 1: hangs forever and fills the executor
ExecutorService executor = Executors.newFixedThreadPool(2);
for (int i=0; i < 5; i++) {
executor.execute(() -> {
while (true) {
System.out.println("Running...")
Thread.sleep(Long.MAX_VALUE)
}
});
}
Your output will be (i.e. only 2 and it gets stuck):
Running...
Running...
On the other hand, you can use submit and add a timeout:
ExecutorService executor = Executors.newFixedThreadPool(2);
for (int i=0; i < 5; i++) {
Future future = executor.submit(() -> {
while (true) {
System.out.println("Running...");
Thread.sleep(Long.MAX_VALUE);
}
});
try {
future.get(1, TimeUnit.SECONDS);
} catch (Exception e) {
if (!future.isDone()) {
System.out.println("Oops: " + e.getClass().getSimpleName());
future.cancel(true);
}
}
}
The output will look like this (notice that the executor does not get stuck, but you need to manually cancel the future):
Running...
Oops: TimeoutException
Running...
Oops: TimeoutException
Running...
Oops: TimeoutException
Running...
Oops: TimeoutException
Running...
Oops: TimeoutException

basically both calls execute,if u want future object you shall call submit() method
here from the doc
public <T> Future<T> submit(Callable<T> task) {
if (task == null) throw new NullPointerException();
RunnableFuture<T> ftask = newTaskFor(task);
execute(ftask);
return ftask;
}
public <T> Future<T> submit(Runnable task, T result) {
if (task == null) throw new NullPointerException();
RunnableFuture<T> ftask = newTaskFor(task, result);
execute(ftask);
return ftask;
}
as you can see java really has no way to start a thread other than calling run() method, IMO. since i also found that Callable.call() method is called inside run() method. hence if the object is callable it would still call run() method, which inturn would call call() method
from doc.
public void run() {
if (state != NEW ||
!UNSAFE.compareAndSwapObject(this, runnerOffset,
null, Thread.currentThread()))
return;
try {
Callable<V> c = callable;
if (c != null && state == NEW) {
V result;
boolean ran;
try {
result = c.call();
ran = true;
} catch (Throwable ex) {
result = null;
ran = false;
setException(ex);
}
if (ran)
set(result);
}
} finally {
// runner must be non-null until state is settled to
// prevent concurrent calls to run()
runner = null;
// state must be re-read after nulling runner to prevent
// leaked interrupts
int s = state;
if (s >= INTERRUPTING)
handlePossibleCancellationInterrupt(s);
}
}

Choose between ExecutorService's submit and ExecutorService's execute

How should I choose between ExecutorService's submit or execute, if the returned value is not my concern?
If I test both, I didn't see any differences among the two except the returned value.
ExecutorService threadExecutor = Executors.newSingleThreadExecutor();
threadExecutor.execute(new Task());
ExecutorService threadExecutor = Executors.newSingleThreadExecutor();
threadExecutor.submit(new Task());

There is a difference concerning exception/error handling.
A task queued with execute() that generates some Throwable will cause the UncaughtExceptionHandler for the Thread running the task to be invoked. The default UncaughtExceptionHandler, which typically prints the Throwable stack trace to System.err, will be invoked if no custom handler has been installed.
On the other hand, a Throwable generated by a task queued with submit() will bind the Throwable to the Future that was produced from the call to submit(). Calling get() on that Future will throw an ExecutionException with the original Throwable as its cause (accessible by calling getCause() on the ExecutionException).

execute: Use it for fire and forget calls
submit: Use it to inspect the result of method call and take appropriate action on Future objected returned by the call
From javadocs
submit(Callable<T> task)
Submits a value-returning task for execution and returns a Future
representing the pending results of the task.
Future<?> submit(Runnable task)
Submits a Runnable task for execution and returns a Future representing that
task.
void execute(Runnable command)
Executes the given command at some time in the future. The command may execute in a new thread, in a pooled thread, or in the calling thread, at the discretion of the Executor implementation.
You have to take precaution while using submit(). It hides exception in the framework itself unless you embed your task code in try{} catch{} block.
Example code: This code swallows Arithmetic exception : / by zero.
import java.util.concurrent.*;
import java.util.*;
public class ExecuteSubmitDemo{
public ExecuteSubmitDemo()
{
System.out.println("creating service");
ExecutorService service = Executors.newFixedThreadPool(10);
//ExtendedExecutor service = new ExtendedExecutor();
service.submit(new Runnable(){
public void run(){
int a=4, b = 0;
System.out.println("a and b="+a+":"+b);
System.out.println("a/b:"+(a/b));
System.out.println("Thread Name in Runnable after divide by zero:"+Thread.currentThread().getName());
}
});
service.shutdown();
}
public static void main(String args[]){
ExecuteSubmitDemo demo = new ExecuteSubmitDemo();
}
}
output:
java ExecuteSubmitDemo
creating service
a and b=4:0
Same code throws by replacing submit() with execute() :
Replace
service.submit(new Runnable(){
with
service.execute(new Runnable(){
output:
java ExecuteSubmitDemo
creating service
a and b=4:0
Exception in thread "pool-1-thread-1" java.lang.ArithmeticException: / by zero
at ExecuteSubmitDemo$1.run(ExecuteSubmitDemo.java:14)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
How to handle the these type of scenarios while using submit()?
Embed your Task code ( Either Runnable or Callable implementation) with try{} catch{} block code
Implement CustomThreadPoolExecutor
New solution:
import java.util.concurrent.*;
import java.util.*;
public class ExecuteSubmitDemo{
public ExecuteSubmitDemo()
{
System.out.println("creating service");
//ExecutorService service = Executors.newFixedThreadPool(10);
ExtendedExecutor service = new ExtendedExecutor();
service.submit(new Runnable(){
public void run(){
int a=4, b = 0;
System.out.println("a and b="+a+":"+b);
System.out.println("a/b:"+(a/b));
System.out.println("Thread Name in Runnable after divide by zero:"+Thread.currentThread().getName());
}
});
service.shutdown();
}
public static void main(String args[]){
ExecuteSubmitDemo demo = new ExecuteSubmitDemo();
}
}
class ExtendedExecutor extends ThreadPoolExecutor {
public ExtendedExecutor() {
super(1,1,60,TimeUnit.SECONDS,new ArrayBlockingQueue<Runnable>(100));
}
// ...
protected void afterExecute(Runnable r, Throwable t) {
super.afterExecute(r, t);
if (t == null && r instanceof Future<?>) {
try {
Object result = ((Future<?>) r).get();
} catch (CancellationException ce) {
t = ce;
} catch (ExecutionException ee) {
t = ee.getCause();
} catch (InterruptedException ie) {
Thread.currentThread().interrupt(); // ignore/reset
}
}
if (t != null)
System.out.println(t);
}
}
output:
java ExecuteSubmitDemo
creating service
a and b=4:0
java.lang.ArithmeticException: / by zero

if you dont care about the return type, use execute. it's the same as submit, just without the return of Future.

Taken from the Javadoc:
Method submit extends base method {#link Executor#execute} by creating and
returning a {#link Future} that can be used to cancel execution and/or wait for
completion.
Personally I prefer the use of execute because it feels more declarative, although this really is a matter of personal preference.
To give more information: in the case of the ExecutorService implementation, the core implementation being returned by the call to Executors.newSingleThreadedExecutor() is a ThreadPoolExecutor.
The submit calls are provided by its parent AbstractExecutorService and all call execute internally. execute is overridden/provided by the ThreadPoolExecutor directly.

The full answer is a composition of two answers that were published here (plus a bit "extra"):
By submitting a task (vs. executing it) you get back a future which can be used to get the result or cancel the action. You don't have this kind of control when you execute (because its return type id void)
execute expects a Runnable while submit can take either a Runnable or a Callable as an argument (for more info about the difference between the two - see below).
execute bubbles up any unchecked-exceptions right away (it cannot throw checked exceptions!!!), while submit binds any kind of exception to the future that returns as a result, and only when you call future.get() a the (wrapped) exception will be thrown . The Throwable that you'll get is an instance of ExecutionException and if you'll call this object's getCause() it will return the original Throwable.
A few more (related) points:
Even if the task that you want to submit does not require returning a
result, you can still use Callable<Void> (instead of using a Runnable).
Cancellation of tasks can be done using the interrupt mechanism. Here's an example of how to implement a cancellation policy
To sum up, it's a better practice to use submit with a Callable (vs. execute with a Runnable). And I'll quote from "Java concurrency in practice" By Brian Goetz:
6.3.2 Result-bearing tasks: Callable and Future
The Executor framework uses Runnable as its basic task representation. Runnable is a fairly
limiting abstraction; run cannot return a value or throw checked
exceptions, although it can have side effects such as writing to a log
file or placing a result in a shared data structure. Many tasks are
effectively deferred computations—executing a database query, fetching
a resource over the network, or computing a complicated function. For
these types of tasks, Callable is a better abstraction: it expects
that the main entry point, call, will return a value and anticipates
that it might throw an exception.7 Executors includes several utility
methods for wrapping other types of tasks, including Runnable and
java.security.PrivilegedAction, with a Callable.

From the Javadoc:
The command may execute in a new thread, in a pooled thread, or in the calling thread, at the discretion of the Executor implementation.
So depending on the implementation of Executor you may find that the submitting thread blocks while the task is executing.

Just adding to the accepted answer-
However, exceptions thrown from tasks make it to the uncaught
exception handler only for tasks submitted with execute(); for tasks
submitted with submit() to the executor service, any thrown exception
is considered to be part of the task’s return status.
Source