I have a set of tasks in multiple levels that I need to run in parallel on threads taken from a thread pool.
I am using a countdown latch to time the overall execution of the level.
Problem: there are few tasks which get to execute more than their individual time just because of other tasks present in the same level that have more execution time. I want to avoid that.
Below is the code I'm using.
private final ExecutorService executor = Executors.newCachedThreadPool(new ThreadFactoryBuilder().setNameFormat(
"TaskExecutor-thread-%d").build());
....
for (int i = 0; i < levels.size(); i++) {
Set<AbstractTask> taskSet = levels.get(i);
CountDownLatch latch = new CountDownLatch(taskSet.size());
int maxAwaitTime = TaskExecutorHelper.getMaxAwaitTime(taskSet); //this returns max of all
// execution time set for
//individual tasks
for (AbstractTask t : taskSet) {
executor.submit(() -> { t.doExecute(input); });
}
latch.await(maxAwaitTime, TimeUnit.MILLISECONDS);
}
Any help would be appreciated!
A possible solution is to set a task that will interrupt execution after given timeout. The following example may give you an idea:
private final ExecutorService executor = ...;
private final ScheduledExecutorService scheduler = ...;
Future future = executor.submit(() -> ... );
ScheduledFuture scheduledFuture = scheduler.schedule(() -> future.cancel(true), 10, TimeUnit.SECONDS);
You will need some code to cancel timeout handler after task execution.
See ScheduledExecutorService#schelude for details.
Related
I have a few tasks which are registered by
final ScheduledExecutorService ses = Executors.newScheduledThreadPool(10);
List<ScheduledFuture<?>> futures = new ArrayList<>();
tasks.forEach(task->{
var future = ses.scheduleWithFixedDelay(() -> run(task), 0, 3, TimeUnit.SECONDS);
futures.add(future);
});
// and now cancel all tasks one for one after 10 seconds..
ses.scheduleWithFixedDelay(() ->
{
log.info("cancel task----");
futures.get(0).cancel(false);
}, 0, 10, TimeUnit.SECONDS);
As you can see, for each task the futures holds a task.getId() so I can obtain the ScheduledFuture of a task afterwards. I do not want to ses.shutdown() because this will shutdown the whole schedulings for the other tasks as well, which I want to avoid.
The only solution I actually see is to create one ScheduledExecutorService for each task to be able to shutdown it afterwards for a specified task, but then I cannot make use of the pooling.
How can I shutdown only a specified task within the pool?
Use
Future<?> future;
future.cancel(false);
Cancel will cancel the task and any further scheduling of it.¹ The Boolean parameter decides if you want to throw an interruption exception on the task if it is already running and blocking on a resource.
To ensure the task is removed from the queue immediately upon cancelling, use the setRemoveOnCancelPolicy method on your ScheduledThreadPoolExecutor and set the policy to true.²
final ScheduledThreadPoolExecutor ses = (ScheduledThreadPoolExecutor) Executors.newScheduledThreadPool(10);
ses.setRemoveOnCancelPolicy(true);
¹ https://docs.oracle.com/javase/8/docs/api/index.html?java/util/concurrent/Future.html
² https://stackoverflow.com/a/36748183/4425643 , https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/ScheduledThreadPoolExecutor.html#setRemoveOnCancelPolicy-boolean-
Is there any way to schedule CompletableFuture in Java?
What I wanted to do is to schedule a task to be executed with some delay, and chain it with other operations to be performed asynchronously when it completes. So far I didn't find any way to do this.
For good ol' Futures we have e.g. ScheduledExecutorService, where we can schedule a task to be executed with some delay like this:
ScheduledExecutorService scheduledExecutorService = Executors.newSingleThreadScheduledExecutor();
Future<String> future = scheduledExecutorService.schedule(() -> "someValue", 10, TimeUnit.SECONDS);
Is there any similar way for CompletableFutures?
If you're using Java 9+ then CompletableFuture#delayedExecutor(long,TimeUnit) may fit your needs:
Returns a new Executor that submits a task to the default executor after the given delay (or no delay if non-positive). Each delay commences upon invocation of the returned executor's execute method.
Executor delayed = CompletableFuture.delayedExecutor(10L, TimeUnit.SECONDS);
CompletableFuture.supplyAsync(() -> "someValue", delayed)
.thenAccept(System.out::println)
.join();
There's also an overload where you can specify the Executor to use in place of the "default executor".
As said, there is support in Java 9.
But it’s not hard to create a similar feature under Java 8; you already named the necessary elements:
// prefer this constructor with zero core threads for a shared pool,
// to avoid blocking JVM exit
static final ScheduledExecutorService SCHEDULER = new ScheduledThreadPoolExecutor(0);
static Executor delayedExecutor(long delay, TimeUnit unit)
{
return delayedExecutor(delay, unit, ForkJoinPool.commonPool());
}
static Executor delayedExecutor(long delay, TimeUnit unit, Executor executor)
{
return r -> SCHEDULER.schedule(() -> executor.execute(r), delay, unit);
}
which can be used similarly to the Java 9 feature:
Executor afterTenSecs = delayedExecutor(10L, TimeUnit.SECONDS);
CompletableFuture<String> future
= CompletableFuture.supplyAsync(() -> "someValue", afterTenSecs);
future.thenAccept(System.out::println).join();
Care must be taken to avoid that the shared scheduled executor’s threads prevent the JVM from terminating. The alternative to a zero core pool size is to use daemon threads:
static final ScheduledExecutorService SCHEDULER
= Executors.newSingleThreadScheduledExecutor(r -> {
Thread t = new Thread(r);
t.setDaemon(true);
return t;
});
I want to use method newWorkStealingPool() to get thread and run them continuously every 1 sec. Using the following sample code :
ScheduledExecutorService executor = Executors.newScheduledThreadPool(1);
Runnable task = () -> System.out.println("Scheduling: " + System.currentTimeMillis());
int initialDelay = 0;
int period = 1;
executor.scheduleAtFixedRate(task, initialDelay, period, TimeUnit.SECONDS);
I can run task continuously but I want to use the method newWorkStealingPool() to get threads. Using the following code:
ScheduledExecutorService executor = (ScheduledExecutorService)Executors.newWorkStealingPool();
Runnable task = () -> System.out.println("Scheduling: " + System.currentTimeMillis());
int initialDelay = 0;
int period = 1;
executor.scheduleAtFixedRate(task, initialDelay, period, TimeUnit.SECONDS);
I got the error:
java.util.concurrent.ForkJoinPool cannot be cast to java.util.concurrent.ScheduledExecutorService
Using ExecutorService object it's possible to use newWorkStealingPool() but I don't know if is there any way to run ExecutorService object continuously like what object ScheduledExecutorService provides?
I think this can be achieved with creating ScheduledExecutorService and ForkJoinPool. ScheduledExecutorService will be used to submit tasks to ForkJoinPool at specified intervals. And ForkJoinPool will execute these tasks.
ForkJoinPool executor = (ForkJoinPool) Executors.newWorkStealingPool();
// this will be only used for submitting tasks, so one thread is enough
ScheduledExecutorService scheduledExecutor = Executors.newSingleThreadScheduledExecutor();
Runnable task = () -> System.out.println("Scheduling: " + System.currentTimeMillis());
int initialDelay = 0;
int period = 1;
scheduledExecutor.scheduleAtFixedRate(()->executor.submit(task), initialDelay, period, TimeUnit.SECONDS);
The Executors.newWorkStealingPool() produces a ForkJoinPool. The ForkJoinPool class does not implement the ScheduledExecutorService interface so you cannot cast it to a ScheduledExecutorService.
Furthermore the ForkJoinPool and ScheduledExecutorService are fundamentally different thread pools. If you need to schedule a task to execute once every second stick with a ScheduledExecutorService, since it is suitable for your use case. ForkJoinPools are intended to use in cases where you have many small units of work divided among many threads, not for when you want to regularly execute something.
In my application i have used a custom ThreadPoolExecutor which enables pausing and resuming of the Executor by extending the ThreadPoolExecutor class. Same way I want to have Restart functionality implemented where after the shutdown method of the ExecutorService has been executed. I first tried with creating new instance of the ThreadPoolExecutor and it failed. I found this question and tried the ExecutorCompletionService which resulted the same failure where it didn't executed as intended.
First time when I click the start button in my UI it executes fine and after the completion of the process when I again start, it won't give me the intended result. Instead will give me the same previous result of the first run. What is the best suitable way which I can achieve this task ?
Thanks in advance :)
Following lines will be executed at each button click.
private static int jobs = 10;
ExecutorService executor = new PausableThreadPoolExecutor(num_threads, num_threads, 5000, TimeUnit.MILLISECONDS, new ArrayBlockingQueue<Runnable>(jobs));
for (int i = 0; i < jobs; i++) {
Runnable worker = new TaskToDo(jobs);
executor.submit(worker);
}
executor.shutdown();
while (!executor.isTerminated()) {}
System.out.println("Finished all threads");
This is the source I used to have pause/resume implementation.
Maybe a ScheduledThreadPoolExecutor you can restart it ScheduledThreadPoolExecutor
using this the methods can return a ScheduledFuture ScheduledFuture or a Future Future to hold references to the tasks
ScheduledFuture now = null;
ScheduledThreadPoolExecutor scheduler = new ScheduledThreadPoolExecutor(1);
Runnable runner = new Runnable(){
public void run()
{
rollthedice(); //your method or runnable of choice
}
};
to start and restart something like "theres other methods too"
now = scheduler.scheduleAtFixedRate(runner, 0, 250, TimeUnit.MILLISECONDS);
to cancel or stop
now.cancel(true);
now = null;
Here is my task. I have a static queue of jobs in a class and a static method that adds jobs to the queue. Have n amount of threads that poll from a queue and perform the pulled job. I need to have the n threads poll simultaneously at an interval. AKA, all 3 should poll every 5 seconds and look for jobs.
I have this:
public class Handler {
private static final Queue<Job> queue = new LinkedList<>();
public static void initialize(int maxThreads) { // maxThreads == 3
ScheduledExecutorService executorService =
Executors.newScheduledThreadPool(maxThreads);
executorService.scheduleWithFixedDelay(new Runnable() {
#Override
public void run() {
Job job = null;
synchronized(queue) {
if(queue.size() > 0) {
job = queue.poll();
}
}
if(job != null) {
Log.log("start job");
doJob(job);
Log.log("end job");
}
}
}, 15, 5, TimeUnit.SECONDS);
}
}
I get this output when I add 4 tasks:
startjob
endjob
startjob
endjob
startjob
endjob
startjob
endjob
It is obvious that these threads perform that jobs serially, whereas I need them to be done 3 at a time. What am I doing wrong? Thanks!
From the documentation:
If any execution of this task takes longer than its period, then subsequent executions may start late, but will not concurrently execute.
So you must schedule three independent tasks to have them run concurrently. Also note that the scheduled executor service is a fixed thread pool, which is not flexible enough for many use cases. A good idiom is to use the scheduled service just to submit tasks to a regular executor service, which may be configured as a resizable thread pool.
You are running ScheduledExecutorService with fixed delay, what means, that your jobs will run one after one. Use fixed thread pool, and submit 3 threads at a time. Here is an explanation with examples
If you declare Job extends Runnable then your code simplifies dramatically:
First declare the Executor somewhere globally accessible:
public static final ExecutorService executor = Executors.newFixedThreadPool(MAX_THREADS);
Then add a job like this:
executor.submit(new Job());
You are done.