I'm currently working on some sort of database benchmark application. Basically, what I'm trying to do is to simulate using threads a certain number of clients that all repeat the same operation (example: a read operation) against the database during a certain period of time.
During this time, I want, in each thread, to measure the average delay for getting an answer from the database.
My first choice was to rely on ThreadMXBean's getThreadCpuTime() method (http://docs.oracle.com/javase/7/docs/api/java/lang/management/ThreadMXBean.html) but the point is that the operation is done too quickly to be measured (getThreadCpuTime() before the operation is equal to getThreadCpuTime() after the operation).
I made a little experiment to understand and illustrate the problem:
public class ExampleClass {
class LongRunningThread extends Thread {
private int n;
public LongRunningThread(int n) {
this.n = n;
}
public void run() {
ArrayList l = new ArrayList();
for (int i = 0; i < n; i++) {
l.add(new Object());
}
long time = ManagementFactory.getThreadMXBean().getThreadCpuTime(this.getId());
System.out.println("Long running thread " + this.getId() + " execution time: " + time);
}
}
class MyThread extends Thread {
int n;
public MyThread(int n) {
this.n = n;
}
public void run() {
ArrayList l = new ArrayList();
for (int i = 0; i < n; i++) {
l.add(new Object());
}
long time = ManagementFactory.getThreadMXBean().getThreadCpuTime(this.getId());
System.out.println("My thread " + this.getId() + " execution time: " + time);
}
}
public static void main(String [] args) {
System.out.println("Cpu time supported? " + ManagementFactory.getThreadMXBean().isThreadCpuTimeSupported());
System.out.println("Cpu time enabled? " + ManagementFactory.getThreadMXBean().isThreadCpuTimeEnabled());
for (int i = 1; i < 10; ++i) {
new LongRunningThread(i*1000000).start();
}
for (int i = 1; i < 10; ++i) {
new MyThread(i*100).start();
}
}
Output:
Cpu time supported? true
Cpu time enabled? true
My thread 18 execution time: 0
My thread 26 execution time: 0
My thread 20 execution time: 0
My thread 22 execution time: 0
My thread 24 execution time: 0
My thread 21 execution time: 0
My thread 25 execution time: 0
My thread 19 execution time: 0
My thread 23 execution time: 0
Long running thread 9 execution time: 15600100
Long running thread 10 execution time: 15600100
Long running thread 11 execution time: 46800300
Long running thread 12 execution time: 31200200
Long running thread 14 execution time: 78000500
Long running thread 13 execution time: 78000500
Long running thread 17 execution time: 124800800
Long running thread 15 execution time: 140400900
Long running thread 16 execution time: 109200700
I cannot get the execution time for all MyThread instances but no problem for LongRunningThread instances. Like I said, my hypothesis is that the operation done by the first threads happen too fast to be actually measured.
Is there any way to achieve what I'm trying to do? Is it possible to measure the execution time for such short time running threads?
Thanks in advance for you help :)
Have you considerd this framework http://metrics.codahale.com/. It's very very good and comes with built in support for exposing metrics via JMX
Is it possible to measure the execution time for such short time running threads?
Without measuring wall-clock times with the nano-second clock, the answer may be no. For small loops, the measured CPU time may be smaller than the precision of the method. The javadocs for ThreadMXBean.getThreadCpuTime(...) say:
Returns the total CPU time for a thread of the specified ID in nanoseconds.
The returned value is of nanoseconds precision but
not necessarily nanoseconds accuracy.
One thing to consider would be to take the CPU time if it is > 0 and take the wall-clock time if it is == 0.
as easier solution you can use next :
class MyThread extends Thread {
int n;
public MyThread(int n) {
this.n = n;
}
public void run() {
long startTime = System.nanoTime();
ArrayList l = new ArrayList(n);
for (int i = 0; i < n; i++) {
l.add(new Object());
}
long time = System.nanoTime() - startTime;
System.out.println("My thread " + this.getId() + " execution time: " + time + " ns");
}
}
if you don't need nanoseconds precision you can use System.currentTimeMillis() instead.
Related
I'm teaching myself Java using one of the ebooks on Amazon. I was doing one of the lessons which does a "benchmark" of the computer. It does this by looping for a minute and calculating the results.
It basically doesn't show you anything for a minute until it's done. So my was making a small modification to show a dot as a progress bar of sorts every few seconds. Normally this is a trivial thing, but something's not right, and I don't know what.
What happens is miniIndex will reach the threshold I specified, and print the value of miniIndex and a period. Its then supposed to set miniIndex to zero so that counter can restart. But it doesn't reset, and never increments again. It's very odd behavior.
Here is the code in its entirety:
class Benchmark {
public static void main(String[] arguments) {
long startTime = System.currentTimeMillis();
long endTime = startTime + 60000;
long index = 0;
// My inner index
int miniIndex = 0;
//
while (true) {
double x = Math.sqrt(index);
long now = System.currentTimeMillis();
if (now > endTime){
break;
}
index++;
// my modification
miniIndex++;
if (miniIndex >= 5000) {
System.out.print(miniIndex + ".");
miniIndex = 0;
}
// end of my modification
}
System.out.println(index + " loops in one minute.");
}
}
I think you misunderstand what your miniIndex++ operation is doing, as it is not counting milliseconds but instead is counting the number of loop iterations which are not equal to each other. I have modified your code to execute the if statement inside every 5 seconds, according to what you wanted to happen:
public static void main(String[] arguments) {
long startTime = System.currentTimeMillis();
long miniTime = startTime; //Declare miniTime equal to startTime
long endTime = startTime + 60000;
long index = 0;
while (true) {
double x = Math.sqrt(index);
long now = System.currentTimeMillis();
if (now > endTime){
break;
}
index++;
// my modification
//Current time minus last time the if executed and check if 5 seconds passed
if ((now - miniTime) >= 5000) {
miniTime = System.currentTimeMillis();
System.out.println("5 seconds have passed.");
//if you want to print the actual time elapsed every 5 seconds use this print
//System.out.println((now - startTime)/1000 + " seconds have passed.");
}
// end of my modification
}
System.out.println(index + " loops in one minute.");
}
Notice how I now compare current time of now and subtract the miniTime to check to see if it is higher than or equal 5000 milliseconds. To use time you must relate it to time somehow, in this case System.currentTimeMillis() and the results. Numbers themselves such as counting the loop will never be time consistent.
A loop may execute millions of times, but only take 3 seconds.
Example Output:
5 seconds have passed.
5 seconds have passed.
5 seconds have passed.
5 seconds have passed.
5 seconds have passed.
5 seconds have passed.
5 seconds have passed.
5 seconds have passed.
5 seconds have passed.
5 seconds have passed.
5 seconds have passed.
16319642816 loops in one minute.
Note: 5 seconds have passed. prints 11 times because at the 60 second mark the loop is broken so the final pass is not printed. (And 11 * 5 is 55 for the first 55 seconds).
Your code is working fine; it's your expectations that are flawed.
Your code prints a dot every 5,000 iterations. Which is going to basically spew values. Remember, your CPU is running at something > 2 billion operations a second. You can probably do a few million of those loops per second. Call it a million loops per second, divided by 5000 is 200 dots per second, more or less, anyway.
If you still want to use it in that type of output you can make it easier using StringBuilder. It's coded like that:
StringBuilder stringBuilder = new StringBuilder();
and incide the loop like that:
if (miniIndex >= 5000) {
stringBuilder.append(miniIndex).append(".");
miniIndex = 0;
}
if (stringBuilder.length() >= 200) {
System.out.println(stringBuilder);
stringBuilder.setLength(0);
}
You're probably overwhelming your output or something, such that it doesn't print anything at all. Your loop waits until 60000 milliseconds (1 minute). When I ran it, here was the end part of my output. Since your loop iterates very very fast (as fast as the computer can print, because that is the bottleneck here), NOT one iteration per second/millisecond, your counter miniIndex goes up many many times per second. However, your code does work - it just doesn't do what you think it does.
If you try something like the following, you may get a slightly more precise output:
class Benchmark {
public static void main(String[] arguments) {
long startTime = System.currentTimeMillis();
long endTime = startTime + 60000;
long index = 0;
// My inner index
int miniIndex = 0;
//
while (true) {
double x = Math.sqrt(index);
long now = System.currentTimeMillis();
if (now > endTime){
break;
}
index++;
// my modification
miniIndex++;
if (miniIndex >= 5000) {
System.out.print(miniIndex + ".");
miniIndex = 0;
}
// end of my modification
try {
Thread.sleep(1);
}
catch(Exception e)
{
e.printStackTrace();
}
}
System.out.println(index + " loops in one minute.");
}
}
Output:
..........50976 loops in one minute.
Notice that you may not get 60,000 loops a minute, as Thread.sleep(1) can sleep for slightly over 1 millisecond.
To see why it didn't work, you need to print the variable "now" while looping. You will notice that "now" stays the same for a certain amount while in the loop and eventually it will increment by one. So when you have one thousand of those increments it equates to a period of 1 sec. What the variable "miniIndex" is doing is counting up to 5000 loops and then resets, nowadays computers can perform thousands and thousands of loops in a second. Which is why it seemed like it wasn't resetting.
You can change the wait variable in the code to show different dot progress like you wanted.
class Main {
public static void main(String[] args) {
long startTime = System.currentTimeMillis();
long endTime = startTime + 60000;
long index = 0;
long wait = 5000; //secs 5000 == 5sec
long secondsPassed = startTime + wait;
System.out.print("counting");
while (true) {
double x = Math.sqrt(index);
long now = System.currentTimeMillis();
if (now > endTime){
break;
}
index++;
if (now >= secondsPassed) {
System.out.print(".");
secondsPassed = secondsPassed + wait;
}
}
System.out.println("");
System.out.println(index + " loops in one minute.");
}
}
Recently I wrote a really simple code as a benchmark to see performance increase in my machine. It just creates some number of threads and divides some number of spins between those threads. Below is my class which extends Thread in java:
public static class LoopThread extends Thread {
int index;
long numberOfRound;
long numberOfSpins;
public LoopThread(int index, long numberOfRound, long numberOfSpins) {
this.index = index;
this.numberOfRound = numberOfRound;
this.numberOfSpins = numberOfSpins;
}
public void run() {
System.out.println("Thread " + index + " started for " + numberOfRound + " rounds and " + numberOfSpins + " spins");
for(long i = 0; i < numberOfRound; i++) {
for(long j = 0; j < numberOfSpins; j++) {
}
}
System.out.println("Thread " + index + " ended");
}
The weird thing is when I first wrote this piece of code, it was almost scaling linearly until 8 threads. However, it was taking more time with 9 threads and the increase was small after 9 threads although my machine has 16 hardware threads. In order to investigate the problem, I just changed the code in a simple way so I put a time information in the last line of the code like:
System.out.println("Thread " + index + " ended: " + System.currentTimeMillis());
This change suprisingly made my code more scalable. Since I can't see an obvious reason behind why this change caused this improvement, I tried again and again and here some results which I consistently see:
Before the change:
time passed for 8 threads and 1000000 rounds and 800000 spins is 63 seconds
time passed for 9 threads and 1000000 rounds and 800000 spins is 69 seconds
After the simple change:
time passed for 8 threads and 1000000 rounds and 800000 spins is 62 seconds
time passed for 9 threads and 1000000 rounds and 800000 spins is 56 seconds
Again, I can't see any obvious reason behind that and it seems very weird to me. Do you have any idea about why this is happening?
Thanks
Edit(the code which starts threads and times):
public static void main(String[] args) {
int numberOfThreads = Integer.parseInt(args[0]);
long numberOfRounds = Long.parseLong(args[1]);
long numberOfSpins = Long.parseLong(args[2]);
long startTime = System.currentTimeMillis();
Thread[] threads = new Thread[numberOfThreads];
for( int i = 0; i < numberOfThreads; i++) {
threads[i] = new LoopThread(i, numberOfRounds, (long) numberOfSpins/numberOfThreads);
threads[i].start();
}
for(Thread t: threads) {
try {
t.join();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
long endTime = System.currentTimeMillis() - startTime;
System.out.println("time passed for " + numberOfThreads + " threads and " + numberOfRounds + " rounds and " + numberOfSpins + " spins is " + endTime/1000 + " seconds");
}
Public class AlgorithmC{
Public class AlgorithmC{
public static void main(String[] args) {
System.out.println("Table of n and the execution time with the Algorithm c:");
for (long n = 1; n <= 10000000;n=n*10) {
long startTime = System.currentTimeMillis();
long sum = n*(n+1)/2;
long endTime = System.currentTimeMillis();
long elapsedTime = endTime - startTime;
System.out.println(n + " " + elapsedTime);
}
}
}
This outputs
Table of n and the execution time with the Algorithm c:
1 0
10 0
100 0
1000 0
10000 0
100000 0
1000000 0
10000000 0
Why is the execution time zero for all the values of n?
I don't know what I'm doing wrong exactly.
Well, the time it takes to execute n*(n+1)/2 is very low. You may try using System.nanoTime(); instead. If your only purpose is to measure the execution time of some math function, then maybe you can experiment with Math.pow();But even then the time will be quite low.
You are measuring time taken to calculate
long sum = n*(n+1)/2;
On your machine it is taking less than a milli-second
In multithreading (Executor framework), the sum total of all time-prints within x() method is not matching to the total time printed by doPerform. And this difference keeps on growing with increasing number of threads in threadpool (goes upto 20 secs). Can someone please figure out why? And is there any way to decrease time taken to return from x method?
I have tested it with:
a) 500 submissions to executor (poolsize =100)
b) 500 submissions to executor (poolsize =300)
c) 300 submissions to executor (poolsize =100)
public void x() {
long startTime = System.currentTimeMillis();
for (long l = 0; l <= 10000000; l++) {
if (l % 1000000 == 0) {
System.out.println("Thread id: "
+ Thread.currentThread().getId() + "\t"
+ (System.currentTimeMillis() - startTime));
startTime = System.currentTimeMillis();
}
}
}
public void doPerform() {
long startTime = System.currentTimeMillis();
x();
System.out.println("Thread id: " + Thread.currentThread().getId()
+ "\t" + (System.currentTimeMillis() - startTime));
}
That's expected. You have 100 or 300 parallel threads being executed, and only 1, 2 or 4 cores to execute them all (unless you're running this on a giant super computer). This means that each thread is assigned some CPU time, then some other thread, then some other thread, etc. giving the illusion of parallel execution. But in reality, instructions of various threads are interlaced and executed sequentially.
So, you could have a thread A's startTime computation in doPerform() executed, and then the thread could be replaced by several other ones one on the CPU. A number of milliseconds could elapse before the thread scheduler reassigns A to a CPU and the startTime computation in x() is executed.
I was performing some test performance on an algorithm and noticed something weird. Maybe I am missing something here.
I first measure the time in milliseconde:
long startTime = System.currentTimeMillis();
x.sort(sortStringInput);
long endTime = System.currentTimeMillis();
and then in nanoseconde:
long startTime = System.nanoTime();
x.sort(sortStringInput);
long endTime = System.nanoTime();
The results are 437ms qnd 26366ns.
I am calling the same method so how can it be possible to get a result in ns which is way smaller than the one in ms. I know that 1 ms is 1 000 000 ns so 26366 is even smaller than 1 ms...
Thanks,
Are you sorting the same list twice? The second call will be extremely fast if the list is already sorted.
Depending on what platform you're on, System.nanoTime() itself can be very slow. You're better off running your benchmark multiple times and measuring the overall duration in miliseconds.
I suggest you run the test for at least 2 seconds before counting any result and run the test for at least 2 seconds and take the average. The following code prints.
Average sort time 116 ms.
Average sort time 117100526 ns.
Average sort time 116 ms.
Average sort time 116530255 ns.
Average sort time 117 ms.
Average sort time 116905977 ns.
Code
public static void main(String... args) throws IOException {
String[] strings = new String[100 * 1000];
for (int i = 0; i < strings.length; i++)
strings[i] = "" + Math.random();
int runTimeMS = 2000;
for (int i = 0; i <= 3; i++) {
{
long start = System.currentTimeMillis();
int count = 0;
do {
Arrays.sort(strings.clone());
count++;
} while (System.currentTimeMillis() - start < runTimeMS);
long time = System.currentTimeMillis() - start;
if (i>0) System.out.println("Average sort time " + time / count + " ms.");
}
{
long start = System.nanoTime();
int count = 0;
do {
Arrays.sort(strings.clone());
count++;
} while (System.nanoTime() - start < runTimeMS * 1000L * 1000L);
long time = System.nanoTime() - start;
if (i>0) System.out.println("Average sort time " + time / count + " ns.");
}
}
}