I have an ExecutorService managing a number of Callables. The tasks that the Callables run are mostly black box transformations and number crunching. Under certain conditions, the data being transformed will oscillate and the thread will take over an hour to finish. For comparison, most threads are completed within a minute.
It's been deteremined that the data from the long-running threads is not relevent. I would like to interrupt any thread that runs longer than a certain amount of time. What would the best way to do this?
Use a ScheduleExecutorService to schedule a task to taskFuture.cancel(true) the long running task when the timeout is reached. If the task finishes before then it won't be cancelled.
ExecutorService service = Executors.newFixedThreadPool(N);
ScheduledExecutorService canceller = Executors.newSingleThreadScheduledExecutor();
public <T> Future<T> executeTask(Callable<T> c, long timeoutMS){
final Future<T> future = service.submit(c);
canceller.schedule(new Callable<Void>(){
public Void call(){
future.cancel(true);
return null;
}
}, timeoutMS, TimeUnit.MILLI_SECONDS);
return future;
}
You could cancel the future etc as in the other answers, but you need to make sure that your threads which are "number crunching" can handle the interrupt and terminate gracefully. You say that this a black box operation - how certain are you that the interrupted status of the thread is being checked actively within the black box? If it isn't, you can't cancel it with an interrupt. The black box needs to be written with interruption in mind.
The best way for you to do this would be to introduce one more Executor. You can use a ScheduledExecutorService to cancel all long running tasks for example:
ExecutorService service = Executors.newFixedThreadPool(N);
ScheduledExecutorService canceller = Executors.newScheduledThreadPool(1);
public void executeTask(Callable<?> c){
final Future<?> future = service.submit(c);
canceller.schedule(new Runnable(){
public void run(){
future.cancel(true);
}
}, SECONDS_UNTIL_TIMEOUT, TimeUnit.SECONDS);
}
You could get a list of your corresponding Futures (that are created when you submit a Callable) together with its startup time.
Another task could then check every minute if there are some task running for more than a defined time and if so, invoke cancel(true) in the Future. Done futures would be removed off the list.
You can use this method
<T> List<Future<T>> invokeAll(Collection<? extends Callable<T>> tasks,
long timeout,
TimeUnit unit)
throws InterruptedException
and set the maximum timeout to one minute. If your thread takes more than that it is just aborted.
The problem is you can not kill/stop/suspend the java thread.
All are deprecated.
future.cancel will only considered before worker thread
So if the thread is stuck with any code execution and doesn't have the logic to consider the thread interruption, then execution will still continue.
This is even true for executorService.shutdown();.
The documentation says.
There are no guarantees beyond best-effort attempts to stop processing
actively executing tasks. For example, typical implementations will
cancel via Thread. interrupt, so any task that fails to respond to
interrupts may never terminate.
So the only solution is to log such a situation and fix the inherent issue.
Future<?> future = executor.submit(task);
try {
future.get(15, TimeUnit.MINUTES);
} catch (InterruptedException e) {
// retry waiting. iterative approach not shown here
} catch (ExecutionException e) {
// your task exploded
} catch (TimeoutException e) {
// The task is running longer than usual
// Log it for future analysis
//If the interrupt signal is handled
//OR the task is stuck in the queue itself for a long time
//then worth calling below
future.cancel(true);
}
Related
I'm unsuccessfully trying to cancel a thread submitted with this code
ExecutorService executor = Executors.newSingleThreadExecutor();
Future<Object> future = executor.submit(()-> {while(true) { System.out.println("Thread"); Thread.yield();} } );
Thread.sleep(3000);
future.cancel(true);
executor.shutdown();
but the thread stays running.
When sleep is used instead of yield, the thread does get cancelled. For example this does cancel the thread:
ExecutorService executor = Executors.newSingleThreadExecutor();
Future<Object> future = executor.submit(()-> {while(true) { System.out.println("Thread"); Thread.sleep(1000);} } );
Thread.sleep(3000);
future.cancel(true);
executor.shutdown();
What's going on? Am I missing something in the documentation?
Your first task is NOT responsive to interruption. But the second task is responsive because Thread.sleep is a blocking method which is responsive to interruption. One way to solve the issue is by making your task responsive to interruption. Here's how it looks.
Future<?> future = executor.submit(() -> {
while (!Thread.currentThread().isInterrupted())
System.out.println("Thread");
});
Moreover Thread.yield is just a scheduler hint to yield its current use of a processor. This operation is platform dependent and should not be used in practice. Hence I have removed it from my answer.
If your intention is to stop all submitted tasks you could use
executor.shutdownNow();
Attempts to stop all actively executing tasks, halts the processing of
waiting tasks, and returns a list of the tasks that were awaiting
execution.
https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ExecutorService.html#shutdownNow()
As already mentioned in the comments section of the question, it is the difference between .sleep(1000) and .yield() behaviour for interrupt calls
My problem is we have to give it a fixed schedule time to make it start task. Lets say i give 10 seconds and my task has average finish time of 10-15 seconds. Thus after some time waiting threads in quque causes huge memory consumption. If i use syncronized for the method above problem will occur. If i don't use syncronized then i am wasting resources ( cpu) because i dont need to run task if not finished. So i thought a solution of recursive call of task but i believe recursive threads will add more memory problems... what should i do? Shortly i just want to be able to call a task when it is finished. Not fixed time.
public void myScheduledTask{
doJob(); ( use countdown latch to control waiting if necessary)
TimeUnit.SECONDS.sleep(x);
new Thread( new Runnable( { mySchedulTask(); } ));
or
executor.execute( a thread that call myScheduledTask() method);
}
The option that sounds like what you're trying to accomplish:
ScheduledExecutorService executor = Executors.newScheduledThreadPool(count);
ScheduledFuture<?> future = executor.scheduleWithFixedDelay(
task,
delay,
delay,
TimeUnit.MILLISECONDS
);
This would start your task and execute it after delay milliseconds after the previous completion. Count should be the number of threads you want to use, 1 is acceptable. This also lets you stop the task using the future.
The problems with your example. a) You are sleeping on an executor thread. Dont do this let the executor handle it. If you were using a threadpool of 1 then this executor couldn't do any work while you're waiting. b) Starting a new thread is taking control from the executor... just use the executor, then you have some control over the execution.
If you really wanted to stick with the form you have.
class RecurringTask implements Runnable{
#Override
public void run(){
doJob();
executor.schedule(this, delay, TimeUnit.MILLISECONDS);
}
}
Now you will be creating Futures that you never use, so it will be harder to control the execution of the task.
Create static member in your task class - Lock.
In doJob avoid performing job if lock is already aquired :
if (lock.tryLock()) {
try {
// do the job
} finally {
lock.unlock();
}
} else {
// log the fact you skipped the job
return;
}
I have a simple java ExecutorService that runs some task objects (implements Callable).
ExecutorService exec = Executors.newSingleThreadExecutor();
List<CallableTask> tasks = new ArrayList<>();
// ... create some tasks
for (CallableTask task : tasks) {
Future future = exec.submit(task);
result = (String) future.get(timeout, TimeUnit.SECONDS);
// TASKS load some classes and invoke their methods (they may create additional threads)
// ... catch interruptions and timeouts
}
exec.shutdownNow();
After all tasks are finished (either DONE or TIMEOUT-ed), I try to shutdown the executor, but it wont stop: exec.isTerminated() = FALSE.
I suspect that some tasks that are timeouted are not properly terminated.
And yes, I know that executor's shutdown is not guaranteing anything:
There are no guarantees beyond best-effort attempts to stop
processing actively executing tasks. For example, typical
implementations will cancel via {#link Thread#interrupt}, so any
task that fails to respond to interrupts may never terminate.
My question is, is there a way to ensure those (task) threads will terminate?
The best solution I came up with, is to call the System.exit() at the end of my program, but that is plain silly.
Recommended way from Oracle API documentation page of ExecutorService:
void shutdownAndAwaitTermination(ExecutorService pool) {
pool.shutdown(); // Disable new tasks from being submitted
try {
// Wait a while for existing tasks to terminate
if (!pool.awaitTermination(60, TimeUnit.SECONDS)) {
pool.shutdownNow(); // Cancel currently executing tasks
// Wait a while for tasks to respond to being cancelled
if (!pool.awaitTermination(60, TimeUnit.SECONDS))
System.err.println("Pool did not terminate");
}
} catch (InterruptedException ie) {
// (Re-)Cancel if current thread also interrupted
pool.shutdownNow();
// Preserve interrupt status
Thread.currentThread().interrupt();
}
If your pool is taking more time to shutdown, you can change
if (!pool.awaitTermination(60, TimeUnit.SECONDS))
to
while (!pool.awaitTermination(60, TimeUnit.SECONDS))
A brief summary of the shutdown related methods
shutdown():
Initiates an orderly shutdown in which previously submitted tasks are executed, but no new tasks will be accepted.
shutdownNow():
Attempts to stop all actively executing tasks, halts the processing of waiting tasks, and returns a list of the tasks that were awaiting execution.
awaitTermination(long timeout, TimeUnit unit) throws InterruptedException:
Blocks until all tasks have completed execution after a shutdown request, or the timeout occurs, or the current thread is interrupted, whichever happens first.
Do you have control of those tasks ? i.e. are you creating those yourself ? I suspect somewhere in those a thread interrupt is being ignored e.g.
try {
....
}
catch {InterruptedException e) {
// do nothing
}
When an InterruptedException is thrown, the interrupt flag on the thread needs to be reset, otherwise the thread will not exit. See here for more info.
Unfortunately you may be using a library that doesn't obey this, in which case you can't easily circumvent this. In that case one heavyweight option is to fork off a sub process to perform the job of the Callable, and that will clear up all resources upon process exit. Heavyweight and possibly non-trivial, but reliable.
When using the ExecutorService returned by Executors.newSingleThreadExecutor(), how do I interrupt it?
In order to do this, you need to submit() a task to an ExecutorService, rather than calling execute(). When you do this, a Future is returned that can be used to manipulate the scheduled task. In particular, you can call cancel(true) on the associated Future to interrupt a task that is currently executing (or skip execution altogether if the task hasn't started running yet).
By the way, the object returned by Executors.newSingleThreadExecutor() is actually an ExecutorService.
Another way to interrupt the executor's internally managed thread(s) is to call the shutdownNow(..) method on your ExecutorService. Note, however, that as opposed to #erickson's solution, this will result in the whole ThreadPoolExecutor becoming unfit for further use.
I find this approach particularly useful in cases where the ExecutorService is no longer needed and keeping tabs on the Future instances is otherwise unnecessary (a prime example of this being the exit(..) method of your application).
Relevant information from the ExecutorService#shutdownNow(..) javadocs:
Attempts to stop all actively executing tasks, halts the processing of
waiting tasks, and returns a list of the tasks that were awaiting
execution.
There are no guarantees beyond best-effort attempts to stop processing
actively executing tasks. For example, typical implementations will
cancel via Thread.interrupt, so any task that fails to respond to
interrupts may never terminate.
One proper way could be customizing/injecting the ThreadFactory for the ExecutorService and from within the thread factory, you got the handle of the thread created, then you can schedule some task to interrupt the thread being interested.
Demo code part for the overwrited method newThread in ThreadFactory:
ThreadFactory customThreadfactory new ThreadFactory() {
public Thread newThread(Runnable runnable) {
final Thread thread = new Thread(runnable);
if (namePrefix != null) {
thread.setName(namePrefix + "-" + count.getAndIncrement());
}
if (daemon != null) {
thread.setDaemon(daemon);
}
if (priority != null) {
thread.setPriority(priority);
}
scheduledExecutorService.schedule(new Callable<String>() {
public String call() throws Exception {
System.out.println("Executed!");
thread.interrupt();
return "Called!";
}
}, 5, TimeUnit.SECONDS);
return thread;
}
}
Then you can use below code to construct your ExecutorService instance:
ExecutorService executorService = Executors.newFixedThreadPool(3,
customThreadfactory);
Then after 5 seconds, an interrupt signal will be sent to the threads in ExecutorService.
I'm preparing an application where a single producer generates several million tasks, which will then be processed by a configurable number of consumers. Communication from producer to consumer is (probably) going to be queue-based.
From the thread that runs the producer/generates the tasks, what method can I use to wait for completion of all tasks? I'd rather not resume to any periodic polling to see if my tasks queue is empty. In any case, the task queue being empty isn't actually a guarantee that the last tasks have completed. Those tasks can be relatively long-running, so it's quite possible that the queue is empty while the consumer threads are still happily processing.
Rgds, Maarten
You might want to have a look at the java.util.concurrent package.
ExecutorService
Executors
Future
The executor framework already provides means to execute tasks via threadpool. The Future abstraction allows to wait for the completition of tasks.
Putting both together allows you coordinate the executions easily, decoupling tasks, activities (threads) and results.
Example:
ExecutorService executorService = Executors.newFixedThreadPool(16);
List<Callable<Void>> tasks = null;
//TODO: fill tasks;
//dispatch
List<Future<Void>> results = executorService.invokeAll(tasks);
//Wait until all tasks have completed
for(Future<Void> result: results){
result.get();
}
Edit: Alternative Version using CountDownLatch
ExecutorService executorService = Executors.newFixedThreadPool(16);
final CountDownLatch latch;
List<Callable<Void>> tasks = null;
//TODO: fill tasks;
latch = new CountDownLatch(tasks.size());
//dispatch
executorService.invokeAll(tasks);
//Wait until all tasks have completed
latch.await();
And inside your tasks:
Callable<Void> task = new Callable<Void>()
{
#Override
public Void call() throws Exception
{
// TODO: do your stuff
latch.countDown(); //<---- important part
return null;
}
};
You want to know where every tasks completes. I would have another queue of completed task reports. (One object/message per task) When this count reaches the number of tasks you created, they have all completed. This task report can also have any errors and timing information for the task.
You could have each consumer check to see if the queue is empty when they dequeue, and, if it is, pulse a condvar (or a Monitor, since I believe that's what Java has) on which the main thread is waiting.
Having the threads check a global boolean variable (marked as volatile) is a way to let the threads know that they should stop.
You can use join() method for each thread ..so that till all the threads are done your main thread will not end! And by this way you can actually find out whether all the threads are done or not!