we are using Hazelcast distributed tasks a lot and realized, that sometimes just starting a task lasts > 2 sec, even before the task itself is executed. We did this on a single machine; that is, no network overhead. The task executed itself has just a single line of code in its call() method (we placed a System.currentTimeMillis() at beginning and end) stores the passed argument "client" in its constructor call - nothing else.
Task is started as follows:
FutureTask<Member> task = new DistributedTask<Member>(new NotifyWaitingClientTask(client),
theId);
ExecutorService executorService = hazelcastInstance.getExecutorService();
executorService.execute(task);
...
task.get();
Question is: is this a usual time? We expected rather milliseconds on local machines.
This is not normal unless you have two many tasks that take too much time and the Executor threads are already occupied. So the task will not start until there is an available thread to execute and you'll see that latency.
If this is not the case, can you come up with the code that we can run and reproduce the issue.
Fuad
Hazelcast
Related
I have a a single method annotated with the #Scheduled annotation in Spring, scheduled to run every 2 seconds.
#Schedule(fixedDelay=2000)
public void myOperation() {
...
Thread.sleep(1000);
...
...
Thread.sleep(1000);
}
However, the underling operations being done have sleep() introduced in order to have delays (for a specific use case). Now if the delays add up to more than 2 seconds, I am trying to understand if Spring will spin up a new thread for the next scheduled operation? Or would it wait till the current thread finishes executing and then start the new one?
Also, to prevent the number of threads from exploding to a large amount, is it safe to provide a thread pool size so that there's no sudden increase in the number of threads running?
Using fixedDelay you are specifying the time to wait since the process has finished until start it again. If your process sleeps in between it does not matter. It needs to finish to the fixedDelay time starts counting. So, you are going to have only one thread active in this case.
On the other hand, if want to start a process every two seconds, without considering whether the previous task has finish or not, you need to use fixedRate. In this case, if you want to be sure there are not more than X process running at the same time, you should use a ThreadPoolTaskScheduler.
Have a look to the documentation:
Annotation Type Scheduled
ThreadPoolTaskScheduler
If You want to have scheduled tasks and control threads by yourself, than it's better to use ScheduledPoolExecutor
My confusion starts with this snippet from the api description for the method scheduleAtFixedRate.
If any execution of this task takes longer than its period, then
subsequent executions may start late, but will not concurrently
execute.
If there will be no concurrent execution, why is there a Thread pool?
Also, is there a way to get concurrent execution? I want them running at the exact period even if the prior task hasn't finished yet. I want concurrent execution.
The documentation should be read as:
..subsequent executions [of the supplied Runnable task, per registration] .. will not concurrently execute
This does not mean that there will not no concurrency over all scheduled tasks. Rather, for each task (created by invocation of scheduleAtFixedRate), the Runnable only executes on one thread at a time - even if the execution time overruns the interval.
This is an explicit design choice, as in most situations concurrent execution of task callbacks is undesirable and leads to out-of-control resource spirals. For instance, a "task bomb" could form if an increasing number of (the same) Runnables were executed concurrently.
The Thread Pool monikor is accurate as the implementation does what it is advertised to do - reuse threads across the execution of tasks.
While there is no standard [Thread Pool] Executor that will have the requested behavior it can be emulated in a limited fashion. This is because the concurrency restriction is per-task that is registered (not per-Runnable) and multiple 'identical' tasks can be registered:
long targetPeriod = ..;
long n = targetPeriod * 2;
task1 = executor.scheduleAtFixedRate(runnable, 0, n, ..)
task2 = executor.scheduleAtFixedRate(runnable, n/2, n, ..)
The actual execution/timing behavior depends on various other factors, but both of these registered tasks could execute concurrently if they take over ~n/2 (the target period) to execute.
is there a way to get concurrent execution? I want them running at the exact period even if the prior task hasn't finished yet. I want concurrent execution.
You could launch another thread for (or submit to another executor) the actual task from the scheduled task (assuming the rate is long enough to launch a thread or submit the task). If the task consistently takes longer than the rate you want you may eventually run out of resources (as described in the other answer).
For example:
ScheduledExecutorService ex = Executors.newScheduledThreadPool(1);
ex.scheduleAtFixedRate(() -> {
new Thread(() -> {
//do task
}).start();
}, 0, 1, TimeUnit.MINUTES);
There seems some issue running the Java ExecutorService (java.util.concurrent) in a UNIX environment. This issue doesnt replicate in Windows...
Issue: The pool takes unduly long time to terminate:
The program typically creates a new pool:
ExecutorService executorService = Executors.newFixedThreadPool(noOfThreads);
Submits tasks to it:
executorService.execute(thread);
Marks it closed for new tasks, so no more tasks can be submitted:
executorService.shutdown();
Makes a blocking call to terminate the threads:
executorService.awaitTermination(10, TimeUnit.HOURS);
Now, as I can make out from the logs, I can see that the threads are finished with their jobs in very little time. Even though the thread pool takes 2 hours to finish all the tasks, the awaitTermination() call suggests the pool having taken entire timeout period of 10 hours, implying that the pool didn’t relinquish the threads once it is done with them.
Did not get any forums which deal with this issue/ aspects of multithreading specific to the unix environment. Have anyone come across this situation before & know how to overcome that?
I am using SchedulerExecuterService to execute a task after specified delay and at given intervals.
ScheduledExecutorService scheduler = Executors.newScheduledThreadPool(1);
scheduler.scheduleAtFixedRate(taskThread,60 ,120 ,TimeUnit.SECONDS);
What happening is first time taskThread is not starting after delay of 60 sec, it is started after delay of more than 60 secs. Whereas the next executions started at correct intervals of 120 secs(not exactly 120 secs but there is a very minute delay which can be ignored).
My query is why the first execution is delayed for more than 60 secs? Since the task is executed by a thread, Does the start up time depends on the thread priority?
How can I make it run at exact delay of 60 secs? What about Quartz library? Will this library solve my purpose(run the job at specific time without any delay)?
Thanks in advance.
Startup of a job depends on the scheduled time and the amount of threads available - if there are none available, it can be delayed. However, further executions will start at a scheduled time (delay + n * period) - which is takes place in your case.
Start will happen approximately at a scheduled time, I doubt there are real-time guarantees for this, it depends on the underlying OS. You could try changing a thread priority for a thread in this thread pool. This could help, but it won't give you a guarantee that it would work out in different environment.
You could also make sure you run a single task for a fixed thread pool of a single thread you use. Or try increasing the thread amount.
T.b.h. the delay you are seeing is unlikely to be ScheduledExecutorService's fault. Starting a thread won't take a second unless your machine is ridiculously overloaded, and I think the thread will get initialised up-front anyways if you use a fixed size thread pool.
I'm guessing it's the initialisation of your own tasks. I don't know what you are doing in your task, but getting the resource ready (starting DB connection pools) etc. can amount to significant time.
If you want you can test with a trivial task (like writing "hello" on the console) to see. And then you can measure each part of your task to see what's taking long. Once you have that you can think of ways to "warm" your system to prevent the delay.
I currently have code that does the following:
private final static ExecutorService pool = Executors.newCachedThreadPool();
public void foo(){
FutureTask<MyObject> first_task = createFutureTask();
FutureTask<MyObject> second_task = createFutureTask();
...
pool.execute(first_task);
pool.execute(second_task);
....
first_task.get();
second_task.get();
...
System.out.println(time taken);
}
The problem I'm having is that I get each of the future task to print out the time they take when doing computation, so for example on the console I will see
first_task : 20000ms
second_task : 18000ms
...
but the total time (System.out.println(time taken)) is much larger then the longest time taken by any future task, so in line with this example the method do would take around 1 minute (compared to the 20s of first_task).
I was under the impression that these future tasks run in parallel but from the timings it seems as though they are being run one after the other. Am I using this API correctly?
You're using the API correctly, but keep in mind that each task runs in a separate thread, not a separate process (and thus not necessarily in parallel).
Each thread would have to run on a separate CPU core to actually execute at the same time. Whether or not this is possible depends on your machine, its current load, and how the JVM and OS are able to schedule the threads across cores.