I'm reading a java8 book and come across the difference between scheduleAtFixedRate and scheduleWithFixedDelay methods from ScheduledExecutorService.
I understand the difference between these two methods on the book, however when I tried to write a simple code. It's not so clear why scheduleAtFixedRate behaves synchronously.
As you can see, I'm allocating 100 threads in the pool. And the scheduler will submit a new task every 1ms, and per task, there's a delay of 1s.
ScheduledExecutorService s = Executors.newScheduledThreadPool(100);
s.scheduleAtFixedRate(() -> {
int num = new Random().nextInt(100);
System.out.println("Hello World" + num);
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("Finished" + num);
}, 0, 1, TimeUnit.MILLISECONDS);
But why am I getting this output ? A new task will run only after the other.
Hello World94
Finished94
Hello World14
Finished14
Hello World90
Finished90
Hello World26
Finished26
Take a look at the javadoc for ScheduledThreadPoolExecutor#scheduleAtFixedRate
If any execution of this task takes longer than its period, then
subsequent executions may start late, but will not concurrently
execute.
So do not await for it to execute concurrently, it will always execute sequentially..
Related
I am using Java's concurrency library ExecutorService to run my tasks. The threshold for writing to the database is 200 QPS, however, this program can only reach 20 QPS with 15 threads. I tried 5, 10, 20, 30 threads, and they were even slower than 15 threads. Here is the code:
ExecutorService executor = Executors.newFixedThreadPool(15);
List<Callable<Object>> todos = new ArrayList<>();
for (final int id : ids) {
todos.add(Executors.callable(() -> {
try {
TestObject test = testServiceClient.callRemoteService();
SaveToDatabase();
} catch (Exception ex) {}
}));
}
try {
executor.invokeAll(todos);
} catch (InterruptedException ex) {}
executor.shutdown();
1) I checked the CPU usage of the linux server on which this program is running, and the usage was 90% and 60% (it has 4 CPUs). The memory usage was only 20%. So the CPU & memory were still fine. The database server's CPU usage was low (around 20%). What could prevent the speed from reaching 200 QPS? Maybe this service call: testServiceClient.callRemoteService()? I checked the server configuration for that call and it allows high number of calls per seconds.
2) If the count of id in ids is more than 50000, is it a good idea to use invokeAll? Should we split it to smaller batches, such as 5000 each batch?
There is nothing in this code which prevents this query rate, except creating and destroying a thread pool repeately is very expensive. I suggest using the Streams API which is not only simpler but reuses a built in thread pool
int[] ids = ....
IntStream.of(ids).parallel()
.forEach(id -> testServiceClient.callRemoteService(id));
Here is a benchmark using a trivial service. The main overhead is the latency in creating the connection.
public static void main(String[] args) throws IOException {
ServerSocket ss = new ServerSocket(0);
Thread service = new Thread(() -> {
try {
for (; ; ) {
try (Socket s = ss.accept()) {
s.getOutputStream().write(s.getInputStream().read());
}
}
} catch (Throwable t) {
t.printStackTrace();
}
});
service.setDaemon(true);
service.start();
for (int t = 0; t < 5; t++) {
long start = System.nanoTime();
int[] ids = new int[5000];
IntStream.of(ids).parallel().forEach(id -> {
try {
Socket s = new Socket("localhost", ss.getLocalPort());
s.getOutputStream().write(id);
s.getInputStream().read();
} catch (IOException e) {
e.printStackTrace();
}
});
long time = System.nanoTime() - start;
System.out.println("Throughput " + (int) (ids.length * 1e9 / time) + " connects/sec");
}
}
prints
Throughput 12491 connects/sec
Throughput 13138 connects/sec
Throughput 15148 connects/sec
Throughput 14602 connects/sec
Throughput 15807 connects/sec
Using an ExecutorService would be better as #grzegorz-piwowarek mentions.
ExecutorService es = Executors.newFixedThreadPool(8);
for (int t = 0; t < 5; t++) {
long start = System.nanoTime();
int[] ids = new int[5000];
List<Future> futures = new ArrayList<>(ids.length);
for (int id : ids) {
futures.add(es.submit(() -> {
try {
Socket s = new Socket("localhost", ss.getLocalPort());
s.getOutputStream().write(id);
s.getInputStream().read();
} catch (IOException e) {
e.printStackTrace();
}
}));
}
for (Future future : futures) {
future.get();
}
long time = System.nanoTime() - start;
System.out.println("Throughput " + (int) (ids.length * 1e9 / time) + " connects/sec");
}
es.shutdown();
In this case produces much the same results.
Why do you restrict yourself to such a low number of threads?
You're missing performance opportunities this way. It seems that your tasks are really not CPU-bound. The network operations (remote service + database query) may take up the majority of time for each task to finish. During these times, where a single task/thread needs to wait for some event (network,...), another thread can use the CPU. The more threads you make available to the system, the more threads may be waiting for their network I/O to complete while still having some threads use the CPU at the same time.
I suggest you drastically ramp up the number of threads for the executor. As you say that both remote servers are rather under-utilized, I assume the host your program runs at is the bottleneck at the moment. Try to increase (double?) the number of threads until either your CPU utilization approaches 100% or memory or the remote side become the bottleneck.
By the way, you shutdown the executor, but do you actually wait for the tasks to terminate? How do you measure the "QPS"?
One more thing comes to my mind: How are DB connections handled? I.e. how are SaveToDatabase()s synchronized? Do all threads share (and compete for) a single connection? Or, worse, will each thread create a new connection to the DB, do its thing, and then close the connection again? This may be a serious bottleneck because establishing a TCP connection and doing the authentication handshake may take up as much time as running a simple SQL statement.
If the count of id in ids is more than 50000, is it a good idea to use
invokeAll? Should we split it to smaller batches, such as 5000 each
batch?
As #Vaclav Stengl already wrote, the Executors have internal queues in which they enqueue and from which they process the tasks. So no need to worry about that one. You can also just call submit for each single task as soon as you have created it. This allows the first tasks to already start executing while you're still creating/preparing later tasks, which makes sense especially when each task creation takes comparatively long, but won't hurt in all other cases. Think about invokeAll as a convenience method for cases where you already have a collection of tasks. If you create the tasks successively yourself and you already have access to the ExecutorService to run them on, just submit() them a.s.a.p.
About batch spliting:
ExecutorService has inner queue for storing tasks. In your case ExecutorService executor = Executors.newFixedThreadPool(15); has 15 thread so max 15 tasks will run concurrently and others will be stored in queue. Size of queue can be parametrized. By default size will scale up to max int. InvokeAll call inside of method execute and this method will place tasks in to queue when all threads are working.
Imho there are 2 possible scenarios why CPU is not at 100%:
try to enlarge thread pool
thread is waiting for testServiceClient.callRemoteService() to
complete and meanwhile CPU is starwing
The problem of QPS maybe is the bandwidth limit or transaction execution(it will lock the table or row). So you just increase pool size is not worked. Additional, You can try to use the producer-consumer pattern.
So I have a repeating TimerTask that runs every x number of seconds (I say x as it can be changed by the user)
I also need to add a UI feature as "time until next execution" in seconds. I could do something where I store an AtomicInteger which I increment every second, and then do some other checking of that integer for if I want to actually 'run' the task. However I'm ideally looking for a simpler way (for example, if I run the task every second but the actual code should be running every 566ms, it won't work properly). I've taken a look at TimerTask#scheduledExecutionTime, but that returns the last time it's run. Is there an easier way to do this? Thanks
you could use ScheduledThreadPoolExecutor.scheduleAtFixedRate
here is simple example
public static void main(String[] args) throws InterruptedException {
ScheduledThreadPoolExecutor scheduledThreadPoolExecutor = new ScheduledThreadPoolExecutor(10);
ScheduledFuture<?> task = scheduledThreadPoolExecutor.scheduleAtFixedRate(() -> {
System.out.println("task");
}, 1, 1, TimeUnit.SECONDS);
scheduledThreadPoolExecutor.scheduleAtFixedRate(() -> {
System.out.println("until " + task.getDelay(TimeUnit.MILLISECONDS));
}, 500, 700, TimeUnit.MILLISECONDS);
Thread.sleep(10000);
scheduledThreadPoolExecutor.shutdownNow();
}
What's the main difference between scheduleAtFixedRate and scheduleWithFixedDelay methods of ScheduledExecutorService?
scheduler.scheduleAtFixedRate(new Runnable() {
#Override
public void run() {
System.out.println("scheduleAtFixedRate: " + new Date());
}
}, 1, 3L , SECONDS);
scheduler.scheduleWithFixedDelay(new Runnable() {
#Override
public void run() {
System.out.println("scheduleWithFixedDelay: " + new Date());
}
}, 1, 3L , SECONDS);
they print exact the same time, seems they are executed at exact the same interval.
Try adding a Thread.sleep(1000); call within your run() method... Basically it's the difference between scheduling something based on when the previous execution ends and when it (logically) starts.
For example, suppose I schedule an alarm to go off with a fixed rate of once an hour, and every time it goes off, I have a cup of coffee, which takes 10 minutes. Suppose that starts at midnight, I'd have:
00:00: Start making coffee
00:10: Finish making coffee
01:00: Start making coffee
01:10: Finish making coffee
02:00: Start making coffee
02:10: Finish making coffee
If I schedule with a fixed delay of one hour, I'd have:
00:00: Start making coffee
00:10: Finish making coffee
01:10: Start making coffee
01:20: Finish making coffee
02:20: Start making coffee
02:30: Finish making coffee
Which one you want depends on your task.
Visualize time series of invocation scheduleAtFixedRate method. Next executions will start immediately if the last one takes longer than period. Otherwise, it will start after period time.
Time series of invocation scheduleWithFixedDelay method. Next execution will start after delay time between termination of one execution and the commencement of the next, regardless of its execution time
Hope can help you
The scheduleAtFixedRate() method creates a new task and submits it to the executor every period, regardless of whether or not the previous task finished.
On the other hand, the scheduleWithFixedDelay() method creates a new task after the previous task has finished.
If you read the Java Doc it will be clearer
ScheduledFuture scheduleAtFixedRate(Runnable command, long initialDelay, long period, TimeUnit unit)
Creates and executes a periodic action that becomes enabled first after the given initial delay, and subsequently with the given period; that is executions will commence after initialDelay then initialDelay+period, then initialDelay + 2 * period, and so on.
ScheduledFuture scheduleWithFixedDelay(Runnable command, long initialDelay, long delay, TimeUnit unit)
Creates and executes a periodic action that becomes enabled first after the given initial delay, and subsequently with the given delay between the termination of one execution and the commencement of the next.
There is one catch in scheduleAtFixedRate if first thread is taking too long and not ended in given duration then second conscutive thread will not start once the first task will get finsished and will not imediately get started while the first thread has comepleted their task and gievn duration has been elapsed. JVM Will decide when the next task will get executed .
I think that will help you to choose method Becuase due to this i got big problem
I can see your premiss but your conclusion is not quite right.
Here is a good and quite complete explanation according to this Tutorial for understanding the diferences bitween these two.
scheduleAtFixedRate (Runnable, long initialDelay, long period, TimeUnit timeunit)
This method schedules a task to be executed periodically. The task is executed the first time after the initialDelay, and then recurringly every time the period expires.
If any execution of the given task throws an exception, the task is no longer executed. If no exceptions are thrown, the task will continue to be executed until the ScheduledExecutorService is shut down.
If a task takes longer to execute than the period between its scheduled executions, the next execution will start after the current execution finishes. The scheduled task will not be executed by more than one thread at a time.
scheduleWithFixedDelay (Runnable, long initialDelay, long period, TimeUnit timeunit)
This method works very much like scheduleAtFixedRate() except that the period is interpreted differently.
In the scheduleAtFixedRate() method the period is interpreted as a delay between the start of the previous execution, until the start of the next execution.
In this method, however, the period is interpreted as the delay between the end of the previous execution, until the start of the next. The delay is thus between finished executions, not between the beginning of executions.
Let's write a simple program:
import java.util.concurrent.Executors
import java.util.concurrent.TimeUnit
var time = 0L
var start = System.currentTimeMillis()
val executor = Executors.newScheduledThreadPool(1)
executor.scheduleWithFixedDelay({
if (time >= 12_000L) {
executor.shutdown()
} else {
Thread.sleep(2000L)
val now = System.currentTimeMillis()
time += now - start
System.out.println("Total $time delay ${now - start}\n")
start = now
}
}, 0L, 1000L, TimeUnit.MILLISECONDS)
And see the results:
| scheduleWithFixedDelay | scheduleAtFixedRate |
|:----------------------:|:----------------------:|
| Total 2001 delay 2001 | Total 2003 delay 2003 |
| Total 5002 delay 3001 | Total 4004 delay 2001 |
| Total 8003 delay 3001 | Total 6004 delay 2000 |
| Total 11003 delay 3000 | Total 8004 delay 2000 |
| Total 14003 delay 3000 | Total 10005 delay 2001 |
| --- | Total 12005 delay 2000 |
NOTICE the execution time is bigger than waiting
scheduleWithFixedDelay keeps delay
scheduleAtFixedRate removes delay
scheduledExecutorService.scheduleAtFixedRate(() -> {
System.out.println("runnable start"); try { Thread.sleep(5000); System.out.println("runnable end");} catch
(InterruptedException e) { // TODO Auto-generated catch block
e.printStackTrace(); }}, 2, 7, TimeUnit.SECONDS);
scheduledExecutorService.scheduleWithFixedDelay(() -> {
System.out.println("runnable start"); try { Thread.sleep(5000); System.out.println("runnable end");} catch
(InterruptedException e) { // TODO Auto-generated catch block
e.printStackTrace(); } }, 2, 7, TimeUnit.SECONDS);
Just execute it, and you will know the difference. Thank you
I would like to run several tasks in parallel until a certain amount of time has passed. Let us suppose those threads are CPU-heavy and/or may block indefinitely. After the timeout, the threads should be interrupted immediately, and the main thread should continue execution regardless of unfinished or still running tasks.
I've seen a lot of questions asking this, and the answers were always similar, often along the lines of "create thread pool for tasks, start it, join it on timeout"
The problem is between the "start" and "join" parts. As soon as the pool is allowed to run, it may grab CPU and the timeout will not even start until I get it back.
I have tried Executor.invokeAll, and found that it did not fully meet the requirements. Example:
long dt = System.nanoTime ();
ExecutorService pool = Executors.newFixedThreadPool (4);
List <Callable <String>> list = new ArrayList <> ();
for (int i = 0; i < 10; i++) {
list.add (new Callable <String> () {
#Override
public String call () throws Exception {
while (true) {
}
}
});
}
System.out.println ("Start at " + (System.nanoTime () - dt) / 1000000 + "ms");
try {
pool.invokeAll (list, 3000, TimeUnit.MILLISECONDS);
}
catch (InterruptedException e) {
}
System.out.println ("End at " + (System.nanoTime () - dt) / 1000000 + "ms");
Start at 1ms
End at 3028ms
This (27 ms delay) may not seem too bad, but an infinite loop is rather easy to break out of - the actual program experiences ten times more easily. My expectation is that a timeout request is met with very high accuracy even under heavy load (I'm thinking along the lines of a hardware interrupt, which should always work).
This is a major pain in my particular program, as it needs to heed certain timeouts rather accurately (for instance, around 100 ms, if possible better). However, starting the pool often takes as long as 400 ms until I get control back, pushing past the deadline.
I'm a bit confused why this problem is almost never mentioned. Most of the answers I have seen definitely suffer from this. I suppose it may be acceptable usually, but in my case it's not.
Is there a clean and tested way to go ahead with this issue?
Edited to add:
My program is involved with GC, even though not on a large scale. For testing purposes, I rewrote the above example and found that the results given are very inconsistent, but on average noticeably worse than before.
long dt = System.nanoTime ();
ExecutorService pool = Executors.newFixedThreadPool (40);
List <Callable <String>> list = new ArrayList <> ();
for (int i = 0; i < 10; i++) {
list.add (new Callable <String> () {
#Override
public String call () throws Exception {
String s = "";
while (true) {
s += Long.toString (System.nanoTime ());
if (s.length () > 1000000) {
s = "";
}
}
}
});
}
System.out.println ("Start at " + (System.nanoTime () - dt) / 1000000 + "ms");
try {
pool.invokeAll (list, 1000, TimeUnit.MILLISECONDS);
}
catch (InterruptedException e) {
}
System.out.println ("End at " + (System.nanoTime () - dt) / 1000000 + "ms");
Start at 1ms
End at 1189ms
invokeAll should work just fine. However, it is vital that you write your tasks to properly respond to interrupts. When catching InterruptedException, they should exit immediately. If your code is catching IOException, each such catch-block should be preceded with something like:
} catch (InterruptedIOException e) {
logger.log(Level.FINE, "Interrupted; exiting", e);
return;
}
If you are using Channels, you will want to handle ClosedByInterruptException the same way.
If you perform time-consuming operations that don't catch the above exceptions, you need to check Thread.interrupted() periodically. Obviously, checking more often is better, though there will be a point of diminishing returns. (Meaning, checking it after every single statement in your task probably isn't useful.)
if (Thread.interrupted()) {
logger.fine("Interrupted; exiting");
return;
}
In your example code, your Callable is not checking the interrupt status at all, so my guess is that it never exits. An interrupt does not actually stop a thread; it just signals the thread that it should terminate itself on its own terms.
Using the VM option -XX:+PrintGCDetails, I found that the GC runs more rarely, but with a far larger time delay than expected. That delay just so happens to coincide with the spikes I experienced.
A mundane and sad explanation for the observed behavior.
How do I delay a while loop to 1 second intervals without slowing down the entire code / computer it's running on to the one second delay (just the one little loop).
Thread.sleep(1000); // do nothing for 1000 miliseconds (1 second)
It seems your loop runs on Main thread and if you do sleep on that thread it will pause the app (since there is only one thread which has been paused), to overcome this you can put this code in new Thread that runs parallely
try{
Thread.sleep(1000);
}catch(InterruptedException ex){
//do stuff
}
My simple ways to delay a loop.
I already put the codes here after failing to follow the stackoverflow's standards.
//1st way: Thread.sleep : Less efficient compared to 2nd
try {
while (true) {//Or any Loops
//Do Something
Thread.sleep(sleeptime);//Sample: Thread.sleep(1000); 1 second sleep
}
} catch (InterruptedException ex) {
//SomeFishCatching
}
//================================== Thread.sleep
//2nd way: Object lock waiting = Most efficient due to Object level Sync.
Object obj = new Object();
try {
synchronized (obj) {
while (true) {//Or any Loops
//Do Something
obj.wait(sleeptime);//Sample obj.wait(1000); 1 second sleep
}
}
} catch (InterruptedException ex) {
//SomeFishCatching
}
//=============================== Object lock waiting
//3rd way: Loop waiting = less efficient but most accurate than the two.
long expectedtime = System.currentTimeMillis();
while (true) {//Or any Loops
while(System.currentTimeMillis() < expectedtime){
//Empty Loop
}
expectedtime += sleeptime;//Sample expectedtime += 1000; 1 second sleep
//Do Something
}
//===================================== Loop waiting
As Jigar has indicated you can use another Thread to do work which can operate, sleep etc independently of other Threads. The java.util.Timer class might help you as well since it can perform periodic tasks for you without you having to get into multithreaded programming.