How can I achieve it without giving as input very large arrays? I am measuring the running time of different algorithms and for an array of 20 elements I get very (the same) similar values. I tried divided the total time by 1000000000 to clear of the E and then used like 16 mirrors where I copied the input array and executed it again for the mirror. But still it is the same for Heap and Quick sort. Any ideas without needing to write redundant lines?
Sample output:
Random array:
MergeSort:
Total time 14.333066343496
QuickSort:
Total time 14.3330663435256
HeapSort:
Total time 14.3330663435256
If you need code snippets just notify me.
To your direct question, use System.nanoTime() for more granular timestamps.
To your underlying question of how to get better benchmarks, you should run the benchmark repeatedly and on larger data sets. A benchmark that takes ~14ms to execute is going to be very noisy, even with a more precise clock. See also How do I write a correct micro-benchmark in Java?
You can't improve the granularity of this method. According to Java SE documentation:
Returns the current time in milliseconds. Note that while the unit of
time of the return value is a millisecond, the granularity of the
value depends on the underlying operating system and may be larger.
For example, many operating systems measure time in units of tens of
milliseconds.
(source)
As others stated, for time lapses, public static long nanoTime() would give you more precision, but not resolution:
This method provides nanosecond precision, but not necessarily
nanosecond resolution.
(source)
Related
Please only answer WHEN you fully comprehend the question.
Do not close down, as there does not exist a similar question.
I am aware of System.nanoTime() gives ns from an arbitrary "random" point after the JVM starts. And I am aware that System.currentTimeMillis() only gives ms precision.
What I am looking for is for the PROOF and keep an open mind, to the hypothesis that the ms changes are not exact once we try to define what exact means.
Exact would in my world mean that everytime we were to register a new ms say, we go from 97ms, 98ms, 99ms and so forth, on every time we get an update, through whatever mechanisms, we can not expect at least observed Java to give us nanosecond precision at the switches.
I know, i know. It sounds weird to expect that, but then the question comes, how accurate are the ms switches then?
It appears to be that when you ask System.nanoTime() repeatedly you would be able to get a linear graph with nanosecond resolution.
If we at the same time ask System.currentTimeMillis() right after System.nanoTime() and we disregard the variance in cost of commands, it appears as if there would be not a linear graph on the same resolution. The ms graph would +-250ns.
This is the to be expected, yet I can not find any information on the error margin, or the accuracy of the ms.
This issue is there for second precision as well, or hour precision, day, year, and so forth. When the year comes, how big is the error?
When the ms comes, how big is the error in terms on ns?
System.currenTimeMillis() can not be trusted to stay linear against System.nanoTime() and we can not expect System.currenTimeMillis() to keep up with ns precision.
But how big is the error? In computing? In Java, in unix systems?
From the documentation:
"Note that while the unit of time of the return value is a millisecond, the granularity of the value depends on the underlying operating system and may be larger. For example, many operating systems measure time in units of tens of milliseconds.
See the description of the class Date for a discussion of slight discrepancies that may arise between "computer time" and coordinated universal time (UTC)."
So both the precision and accuracy of the call is undefined. They pass the buck to the OS and shrug. I doubt that 250 ns is an accurate measure of its quality. The gap is likely much larger than that. "Tens of milliseconds" as per the documentation is a much more likely value, especially across multiple systems.
Also they essentially disavow any knowledge of UTC as well. "Slight discrepancies" are allowed, whatever that means. Technically this allows any value at all, because what exactly is "slight?" It could be a second or a minute depending on your point of view.
Finally the system clock could be misconfigured by the person operating the system, and at that point everything goes out the window.
I am trying to compute time lapsed in java using nanoTime. But everytime it gives me different results. Why it is not consistent always ?
Sample code :
long startTime=System.nanoTime();
String.valueOf(number).length();
long endTime = System.nanoTime();
System.out.println(endTime-startTime);
nanoTime() and its sister currentTimeMillis() are not exact and depending on the architecture you run your code on they suffer from rounding (see the javadoc for details):
This method provides nanosecond precision, but not necessarily nanosecond resolution
(that is, how frequently the value changes) -
no guarantees are made except that the resolution is at least as good
as that of currentTimeMillis().
If you measure the time in order to decide if alternative a or b is faster you are basically doing a micro benchmark. There are frameworks for this and you should use them. Probably the one most known for Java is JMH. If you need to do the same for larger code parts you might consider profiling.
You might want to have a look at this stackoverflow post: How do I write a correct micro-benchmark in Java?
The lapsed time will vary depending upon how JVM will execute and allocate the processing time to the thread in which this code executes.
I tried multiple runs and always got result in between 9000 to 11000 nanoseconds range. This looks fairly consistent.
Today I did a little quick Benchmark to test speed performance of System.nanoTime() and System.currentTimeMillis():
long startTime = System.nanoTime();
for(int i = 0; i < 1000000; i++) {
long test = System.nanoTime();
}
long endTime = System.nanoTime();
System.out.println("Total time: "+(endTime-startTime));
This are the results:
System.currentTimeMillis(): average of 12.7836022 / function call
System.nanoTime(): average of 34.6395674 / function call
Why are the differences in running speed so big?
Benchmark system:
Java 1.7.0_25
Windows 8 64-bit
CPU: AMD FX-6100
From this Oracle blog:
System.currentTimeMillis() is implemented using the
GetSystemTimeAsFileTime method, which essentially just reads the low
resolution time-of-day value that Windows maintains. Reading this
global variable is naturally very quick - around 6 cycles according to
reported information.
System.nanoTime() is implemented using the
QueryPerformanceCounter/ QueryPerformanceFrequency API (if available,
else it returns currentTimeMillis*10^6).
QueryPerformanceCounter(QPC) is implemented in different ways
depending on the hardware it's running on. Typically it will use
either the programmable-interval-timer (PIT), or the ACPI power
management timer (PMT), or the CPU-level timestamp-counter (TSC).
Accessing the PIT/PMT requires execution of slow I/O port instructions
and as a result the execution time for QPC is in the order of
microseconds. In contrast reading the TSC is on the order of 100 clock
cycles (to read the TSC from the chip and convert it to a time value
based on the operating frequency).
Perhaps this answer the question. The two methods use different number of clock cycles, thus resulting in slow speed of the later one.
Further in that blog in the conclusion section:
If you are interested in measuring/calculating elapsed time, then always use System.nanoTime(). On most systems it will give a resolution on the order of microseconds. Be aware though, this call can also take microseconds to execute on some platforms.
Most OS's (you didn't mention which one you are using) have an in memory counter/clock which provides millisecond accuracy (or close to that). For nanosecond accuracy most have to read a hardware counter. Communicating with hardware is slower then reading some value already in memory.
It may only be the case on Windows. See this answer to a similar question.
Basically, System.currentTimeMillis() just reads a global variable maintained by Windows (which is why it has low granularity), whereas System.nanoTime() actually has to do IO operations.
You are measuring that on Windows, aren't you. I went through this exercise in 2008. nanoTime IS slower on Windows than currentTimeMillis. As I recall, on Linux, nanotime is faster than currentTimeMillis and is certainly faster than it is on Windows.
The important thing to note is if you are trying to measure the aggregate of multiple sub-millisecond operations, you must use nanotime as if the operation finished in less than 1/1000th of a second your code, comparing currentTimeMillis will show the operation as instantaneous so 1,000 of these will still be instantaneous. What you might want to do is use nanotime then round to the nearest millisecond, so if an operation took 8000 nanoseconds it will be counted as 1 millisecond, not 0.
What you might want to do is use nanotime then round to the nearest millisecond, so if an operation took 8000 nanoseconds it will be counted as 1 millisecond, not 0.
Arithmetic note:
8000 nanoseconds is 8 microseconds is 0.008 milliseconds. Rounding will take that to 0 milliseconds.
I have developed an image processing algorithm in core java (without using any third party API), Now I have to calculate execution time of that algorithm, for this i have used System.currentTimeMillis() like that,
public class MyAlgo {
public MyAlgo(String imagePath){
long stTime = System.currentTimeMillis();
// ..........................
// My Algorithm
// ..........................
long endTime = System.currentTimeMillis();
System.out.println("Time ==> " + (endTime - stTime));
}
public static void main(String args[]){
new MyAlgo("d:\\myImage.bmp");
}
}
But the problem is that each time I am running this program I am getting different execution time. Can anyone please suggest me that how can I do this?
If you don't want to use external profiling libraries just wrap your algorithm in a for() loop that executes it 1000 times and divide the total time by 1000. The result will be much more accurate since all the other tasks/processes will even out.
Note: The overall measure time will reflect the expected time of the algorithm to finish and not the total time that algorithms code instruction require.
For example if your algorithm uses a lot of memory and on average java VM calls garbage collector twice per each execution of algorithm - than you should take into account also the time of the garbage collector.
That is exactly what a for() loop does, so you will get good results.
You cannot get a reliable result from one execution alone; Java (well, JVMs) does runtime optimizations, plus there are other processes competing for CPU time/resource access. Also, are you sure your algorithm runs in constant time whatever the inputs?
Your best bet to have a calculation as reliable as possible is to use a library dedicated to performance measurements; one of them is caliper.
Set up a benchmark with different inputs/outputs etc and run it.
You need to apply some statistical analysis over multiple executions of your algorithm. For instance, execute it 1000 times and analyze min, max and average time.
Multiple executions in different scenarios might provide insights too, for instance, in different hardware or with images with different resolution.
I suppose your algorithm can be divided in multiple steps. You can monitor the steps independently to understand the impact of each one.
Marvin Image Processing Framework, for instance, provides methods to monitor and analyze the time and the number of executions of each algorithm step.
Is it correct to compare two values resulting from a call to System.nanoTime() on two different machines? I would say no because System.nanoTime() returns a nanosecond-precise time relative to some arbitrary point time by using the Time Stamp Counter (TSC) which is processor dependent.
If I am right, is there a way (in Java) to capture an instant on two different machines and to compare (safely) these values with at least a microsecond precision or even nanotime precision?
System.currentTimeMillis() is not a solution because it is not returning a linearly increasing number of time stamps. The user or services such as NTP can change the system clock at any time and the time will leap back and forward.
You might want to look into the various clock synchronization algorithms available. Apparently the Precision Time Protocol can get you within sub-microsecond accuracy on a LAN.
If you don't need a specific time value but rather would like to know the ordering of various events, you could for instance use Lamport timestamps.
You cannot use nanoTime between two different machines. For the Java API docs:
This method can only be used to measure elapsed time and is not
related to any other notion of system or wall-clock time. The value
returned represents nanoseconds since some fixed but arbitrary time
(perhaps in the future, so values may be negative).
There's no guarantee that nanoTime is relative to any timebase.
This is a processor & OS dependent Q. Looking at POSIX clocks, for example, there are high precision time of day aware timestamps (e.g. CLOCK_REALTIME returns a nano epoch time value) and high precision arbitrary time timestamps (e.g. CLOCK_MONOTONIC) (NB: the difference between these 2 is nicely explained in this answer).
The latter is often something like time since the box was booted and therefore there's no way to accurately compare them across servers unless you have high precision clock sync (e.g. PTP as referenced in the other answer) in the first place (as then you'd be able to share an offset between them).
Whether NTP is good enough for you depends on what you're trying to measure. For example if you're trying to measure an interval of a few hundred micros (e.g. boxes connected to the same switch) then your results will be rough, at the other extreme NTP can be perfectly good if your servers are in different geographical locations entirely (e.g. London to NY) which means the clock sync effect (as long as it's not way way off) is swamped by the latency between the locations.
FWIW the JNI required to access such clocks from java is pretty trivial.
You can synchronize the time to current time millis. However even if you use NTP this can drift by 1 ms to 10 ms between machines. The only way to be micro-second synchronization between machines is to use specialist hardware.
nanoTime is guaranteed to be determined the same way or have the same resolution on two different OSes.