Develop Reference

Why does the Java Scheduler exhibit significant time drift on Windows?

I have Java service running on Windows 7 that runs once per day on a SingleThreadScheduledExecutor. I've never given it much though as it's non critical but recently looked at the numbers and saw that the service was drifting approximately 15 minutes per day which sounds way to much so dug it up.
Executors.newSingleThreadScheduledExecutor().scheduleAtFixedRate(() -> {
long drift = (System.currentTimeMillis() - lastTimeStamp - seconds * 1000);
lastTimeStamp = System.currentTimeMillis();
}, 0, 10, TimeUnit.SECONDS);
This method pretty consistently drifts +110ms per each 10 seconds. If I run it on a 1 second interval the drift averages +11ms.
Interestingly if I do the same on a Timer() values are pretty consistent with an average drift less than a full millisecond.
new Timer().schedule(new TimerTask() {
#Override
public void run() {
long drift = (System.currentTimeMillis() - lastTimeStamp - seconds * 1000);
lastTimeStamp = System.currentTimeMillis();
}
}, 0, seconds * 1000);
Linux: doesn't drift (nor with Executor, nor with Timer)
Windows: drifts like crazy with Executor, doesn't with Timer
Tested with Java8 and Java11.
Interestingly, if you assume a drift of 11ms per second you'll get 950400ms drift per day which amounts to 15.84 minutes per day. So it's pretty consistent.
The question is: why?
Why would this happen with a SingleThreadExecutor but not with a Timer.
Update1: following Slaw's comment I tried on multiple different hardware. What I found is that this issue doesn't manifest on any personal hardware. Only on the company one. On company hardware it also manifests on Win10, though an order of magnitude less.

As pointed out in the comments, the ScheduledThreadPoolExecutor bases its calculations on System.nanoTime(). For better or worse, the old Timer API however preceeded nanoTime(), and so uses System.currentTimeMillis() instead.
The difference here might seem subtle, but is more significant than one might expect. Contrary to popular belief, nanoTime() is not just a "more accurate version" of currentTimeMillis(). Millis is locked to system time, whereas nanos is not. Or as the docs put it:
This method can only be used to measure elapsed time and is not related to any other notion of system or wall-clock time. [...] The values returned by this method become meaningful only when the difference between two such values, obtained within the same instance of a Java virtual machine, is computed.
In your example, you're not following this guidance for the values to be "meaningful" - understandably, because the ScheduledThreadPoolExecutor only uses nanoTime() as an implementation detail. But the end result is the same, that being that you can't guarantee that it will stay synchronised to the system clock.
But why not? Seconds are seconds, right, so the two should stay in sync from a certain, known point?
Well, in theory, yes. But in practice, probably not.
Taking a look at the relevant native code on windows:
LARGE_INTEGER current_count;
QueryPerformanceCounter(&current_count);
double current = as_long(current_count);
double freq = performance_frequency;
jlong time = (jlong)((current/freq) * NANOSECS_PER_SEC);
return time;
We see nanos() uses the QueryPerformanceCounter API, which works by QueryPerformanceCounter getting the "ticks" of a frequency that's defined by QueryPerformanceFrequency. That frequency will stay identical, but the timer it's based off, and its synchronistaion algorithm that windows uses, varies by configuration, OS, and underlying hardware. Even ignoring the above, it's never going to be close to 100% accurate (it's based of a reasonably cheap crystal oscillator somewhere on the board, not a Caesium time standard!) so it's going to drift out with the system time as NTP keeps it in sync with reality.
In particular, this link gives some useful background, and reinforces the above pont:
When you need time stamps with a resolution of 1 microsecond or better and you don't need the time stamps to be synchronized to an external time reference, choose QueryPerformanceCounter.
(Bolding is mine.)
For your specific case of Windows 7 performing badly, note that in Windows 8+, the TSC synchronisation algorithm was improved, and QueryPerformanceCounter was always based on a TSC (as oppose to Windows 7, where it could be a TSC, HPET or the ACPI PM timer - the latter of which is especially rather inaccurate.) I suspect this is the most likely reason the situation improves tremendously on Windows 10.
That being said, the above factors still mean that you can't rely on the ScheduledThreadPoolExecutor to keep in time with "real" time - it will always drift. If that drift is an issue, then it's not a solution you can rely on in this context.
Side note: In Windows 8+, there is a GetSystemTimePreciseAsFileTime function which offers the high resolution of QueryPerformanceCounter combined with the accuracy of the system time. If Windows 7 was dropped as a supported platform, this could in theory be used to provide a System.getCurrentTimeNanos() method or similar, assuming other similar native functions exist for other supported platforms.

CronScheduler is a project of mine designed to be proof against time drift problem, and at the same time it avoids some of the problems with the old Timer class described in this post.
Example usage:
Duration syncPeriod = Duration.ofMinutes(1);
CronScheduler cron = CronScheduler.create(syncPeriod);
cron.scheduleAtFixedRateSkippingToLatest(0, 1, TimeUnit.MINUTES, runTimeMillis -> {
// Collect and send summary metrics to a remote monitoring system
});
Note: this project was actually inspired by this StackOverflow question.

inconsistent time elapsed in java

I am trying to compute time lapsed in java using nanoTime. But everytime it gives me different results. Why it is not consistent always ?
Sample code :
long startTime=System.nanoTime();
String.valueOf(number).length();
long endTime = System.nanoTime();
System.out.println(endTime-startTime);

nanoTime() and its sister currentTimeMillis() are not exact and depending on the architecture you run your code on they suffer from rounding (see the javadoc for details):
This method provides nanosecond precision, but not necessarily nanosecond resolution
(that is, how frequently the value changes) -
no guarantees are made except that the resolution is at least as good
as that of currentTimeMillis().
If you measure the time in order to decide if alternative a or b is faster you are basically doing a micro benchmark. There are frameworks for this and you should use them. Probably the one most known for Java is JMH. If you need to do the same for larger code parts you might consider profiling.
You might want to have a look at this stackoverflow post: How do I write a correct micro-benchmark in Java?

The lapsed time will vary depending upon how JVM will execute and allocate the processing time to the thread in which this code executes.
I tried multiple runs and always got result in between 9000 to 11000 nanoseconds range. This looks fairly consistent.

Accurate spending time solution

Considering the following code snippets
class time implement Runnable{
long t=0L;
public void run(){
try{while(true){Thread.sleep(1000);t++;/*show the time*/}}catch(Throwable t){}
}
}
////
long long t=0L;
void* time(void* a){//pthread thread start
sleep(1);t++;//show the time
}
I read in some tutorial that in Java Thread.sleep(1000) is not exactly 1 second, and it might be more if the system is busy at the time, then OS switch to the thread late.
Questions:
Is this case true at all or no?
Is this scenario same for native (C/C++) codes?
What is the accurate way to count the seconds up in an application?

Others have answered about the accuracy of timing. Unfortunately, there is no GUARANTEED way to sleep for X amount of time, and wake up at exactly X.00000 seconds (or milliseconds, nanoseconds, etc).
For displaying time in seconds, you can just lower the time you are waiting to, say, half a second. Then you won't have the time jump two seconds from time to time, because half a second isn't going to be extended to more than a second (unless the OS & system you are running on is absolutely overloaded and nothing gets to run when it should - in which case you should fix that problem [get a faster processor, more memory, or whatever it takes], not fiddle with the timing of your application). This works well for "relatively long periods of time", such as one second or 1/10th of a second. For higher precision, it won't really work, since we're now entering the "scheduling jitter" zone.
If you want very accurate timing, then you will probably need to use a Real-Time OS, or at least an OS that has "real time extensions enabled", which will allow the OS to be more strict about time (at the cost of "ease of use" from the programmer, and possibly also the OS being less efficient in it's handling of processes, because it "switches more often than it needs to", compared to a more "lazy" timing approach).
Note also that the "may take longer", in an idle system is mainly the "rounding up of the timer" (if the system tick happens every 10ms or 1ms, the timer is set to 1000ms + whatever is left of the current timer tick, so may be 1009.999ms, or 1000.75ms, for example). The other overhead, that come from scheduling and general OS overheads should be in the microseconds range if not nanoseconds on any modern system - after all, an OS can do quite a lot of work in a microsecond - a modern x86 CPU will execute 3 cycles per clock, and the clock runs around 0.3ns. That's 10 instructions per nanosecond [of course, cache-misses and such will worsen this dramatically]. If the OS has more than a few thousand instructions to go from one process to another (less still for threads), then there's something quite wrong. A few thousand instructions # 10 instructions per nanonsecond = some hundreds of nanoseconds. Definitely less than a microsecond. Compare that to the 1ms or 10ms "jitter" of starting the timer just after the timer ticked off last time.
Naturally, if the CPU is busy running other tasks, this is different - then the time "left to run" on other processes will also influence the time taken to wake up a process.
Of course, in a heavily loaded memory system, the "just woken up" process may not be "ready to run", it could be swapped out to disk, for example. In which case, tens if not hundreds of milliseconds are needed to load it back from the disk.

To answer the two first questions: Yes it's true, and yes.
First there is the time between the timeout expires and the time when the OS notices it, then there the time for the OS to reschedule your process, and lastly there's the time from the process has been "woken up" until it is its turn to run. How long will all this take? There's no way of saying.
And as it's all done on the OS level, it doesn't really matter what language you program in.
As for a more accurate way? There is none. You can use more high-precision timers, but there is no way of avoiding the lag described above.

Yes, it´s true that it is not accurate.
It´s the same for simple sleep-functions in C/C++ and pretty much everything else.
Depending on your system, there could be better functions accessible,
but:
What is the accurate way
A really accurate way does not exist.
Unles you have some really expensive special computer with atomic clock included.
(and no usual OS too. And even then, we could argue what "accurate" means)
If busy waiting (high CPU load) is acceptable, look at nanoTime or native usleep, HighPerformanceCounter or whatever is applicable for your system

The sleep call tells the system to stop the thread execution for at least a time period specified as argument. The system will then resume thread execution when it has a chance (it actually depends on many factors, such as hardware, thread priorities, etc.). To more or less acurately measure the time you can store the time at the beginning of execution and then calculate the time delta each time it's needed.

The sleep function is not accurate, but if the intent is to display the total amount of seconds then you should store the current time at the beginning and then display the time difference every now and then.

This is true. Every sleep implementation in any language (C too) will fail to wait exactly 1 second. It has to deal with your OS scheduler, the sleep duration is juste a hint : the minimum sleep duration to be precise, but the actual difference depends on gigazillions of factors.
Trying to figure out the deviation is tricky if you want a very high resolution clock. In most cases, you'll have about 1~5 ms (roughly).
The thing is that the order of magnitude will be the same whatever the sleep duration. If you want something "accurate", you can divide your time application and wait for a longer period. For example, when you benchmark, you will prefer this type of implementation because the delta-time will increase, decreasing uncertainty :
// get t0
// process n times
// get t1
// compute average time : (t1-t0)/n

Accuracy of ScheduledExecutorService on normal OS / JVM

I use ScheduledExecutorService.scheduleAtFixedRate to run a daily task, like this:
executor.scheduleAtFixedRate(task, d, 24L * 3600 * 1000, TimeUnit.MILLISECONDS);
(d is the initial delay in milliseconds).
The executor is created by Executors.newSingleThreadScheduledExecutor() and runs multiple tasks, but they are all scheduled a few hours apart, and take at most a few minutes.
I know that ScheduledExecutorService makes no guarantees about accuracy, and I'd need a real-time OS and JVM to get that. That is not a requirement for my task, though.
I noticed that on a Windows 2003 Server, using JDK 1.7.0_03, the task slips by almost 10 seconds per day. That makes about 5 minutes per month, which is acceptable for my application. I'll probably have to implement re-scheduling anyway, because I want the task to run at a specific local time, and so I'll have to take care of DST myself. The service runs for long periods of time - half a year without restart is not that unusual.
Still, I think that an inaccuracy of 10 sec/day is rather high for a mostly idle system, and I wonder if I should be prepared for even worse behavior.
So my question is about your experiences with scheduleAtFixedRate. Are the 10 sec/day normal? Will I get better or worse accuracy in other environments (our customers also use Linux and Solaris servers)? Or are the 10 seconds an indication that something is amiss in our environment?

For a very long running task, it is not too surprising. Another problem you have is that it uses nanoTime() which is not synchronized with NTP or the like. This can result in drift with the wall clock.
One way to avoid this is to schedule repeatedly as you suggest. The repeating tasks actually reschedule themselves (which is why they cannot throws an exception, see below) You can have a one shot task which reschedules itself at the end, using the wall clock time and taking into account day list savings.
BTW: I would make sure you catch an exceptions or even Throwable thrown. If you don't your task will stop, possibly silently (unless you are looking at the Future object returned)
What I do is cheat a little. I have a task which wakes every 1 - 10 seconds and checks if it needs to run and if not returns. The overhead is usually trivial if you don't have thousands of tasks and it's much simpler to implement.

Big difference in timestamps when running same application multiple times on an emulator

In an android application I am trying to do a exponential modulus operation and want to calculate the time taken for that process. So i have created 2 timestamps,one just before the operation and the other just after the operation
Calendar calendar0 = Calendar.getInstance();
java.util.Date now0 = calendar0.getTime();
java.sql.Timestamp currentTimestamp0 = new java.sql.Timestamp(now0.getTime());
BigInteger en = big.modPow(e, n);
Calendar calendar1 = Calendar.getInstance();
java.util.Date now1 = calendar1.getTime();
java.sql.Timestamp currentTimestamp1 = new java.sql.Timestamp(now1.getTime());
The difference in time reported by these 2 timestamps is varying over a large range for the same inputs when i am running application multiple times. It gives time in range of [200ns-6ns]
Can someone point out the reason for such result/something that i am doing wrong?

Well for one thing, you're going about your timing in a very convoluted way. Here's something which gives just as much accuracy, but rather simpler:
long start = System.currentTimeMillis();
BigInteger en = big.modPow(e, n);
long end = System.currentTimeMillis();
Note that java.util.Date only has accuracy to the nearest millisecond, and using that same value and putting it in a java.sql.Timestamp doesn't magically make it more accurate. So any result under a millisecond (6ns-200ns) is obviously spurious - basically anything under a millisecond is 0.
You may be able to get a more accurate reading using System.nanoTime - I don't know whether that's supported on Android. There may be alternative high-precision timers available, too.
Now as for why operations may actually take very different amounts of recorded time:
The granularity of the system clock used above may well be considerably less than 1ms. For example, if it measures nothing smaller than 15ms, you could see some operations supposedly taking 0ms and some taking 15ms, even though they actually take the same amount of time.
There can easily be other factors involved, the most obvious being garbage collection

Do more calculations in the trial so that your expected time is several milliseconds.
Consider doing your output to logcat, recording that on the PC and then processing the logcat to only select trials with no intervening platform messages about garbage collection or routine background operations.

A large part of the reason for the variation is that your application is being run by a virtual machine running under a general-purpose operating system on an emulated computer which is running under a general-purpose operating system on a real computer.
With the exception of your application, every one of the things in that pile (the JVM running your app, Android's Linux OS, the emulator and whatever OS runs it) may use the CPU (real or virtual) to do something else at any time for any duration. Those cycles will take away from your program's execution and add to its wall-clock execution time. The result is completely nondeterministic behavior.
If you count emulator CPU cycles consumed and discount whatever Java does behind the scenes, I have no doubt that the standard deviation in execution times would be a lot lower than what you're seeing for the same inputs. The emulated Android environment isn't the place to be benchmarking algorithms because there are too many variables you can't control.

You can't make any conclusions only running a single function. There's so much more going on. You have no control over the underlying linux operating system taking up cycles, let alone the java virtual machine, and yet still all the other programs running. Try running it 1000 or 10000 times