I am trying to loop through this method 10 times that searches an array of numbers captures the run time in nano seconds and prints the results. I then want t take the 10 run times and find the average and standard deviation.
Is there a way to capture the time after 10 run and use the result to find my average and standard deviation?
This is what I have so far:
public class Search {
public static int Array[] = new int[100];
//Building my array with 100 numbers in sequential order
public static void createArray(){
int i = 0;
for(i = 0; i<Array.length; i++)
Array[i] = i + 1;
int check[] = {5, 15, 12};
int target = check[2];
boolean found = false;
int j = 0;
long startTime = System.nanoTime();
for(j=0; j<Array.length;j++){
if(Array[j] == target){
long endTime = System.nanoTime();
System.out.print(endTime - startTime + "ms" + "\t\t");
found = true;
break;
}
}
if(found){
//System.out.println("got you! "+ target + " is at index "+ j +"\t");..... just to test if it was working
}
else{
System.out.println("not available");
}
}
// Printing header
public static void main(String[]args){
System.out.print("First run\tSecond run\tThird run\tFourth run\tFifth run\tSixth run\tSeventh run\tEight run\tNinth run\tTenth run\tAverage \tStandard deviation\n");
// looping through the method 10 times
int i=0;
while(i<10){
createArray();
i++;
}
}
}
Try:
long sum = 0;
long sumSquare = 0;
for(int c = 0 ; c < 10 ; c++) {
long start = System.nanoTime();
// do work
long end = System.nanoTime();
sum += end - start;
sumSquare += Math.pow(end - start, 2);
}
double average = (sum * 1D) / 10;
double variance = (sumSquare * 1D) / 10 - Math.pow(average, 2);
double std = Math.sqrt(variance);
Try creating an array list of size 10 like:
private static List<Long> times = new ArrayList<>(10);
And then, when you find the element just add endTime - startTime to list like:
times.add(..);
And once that's done, in your main method you could do sum, average like:
long totalTime = 0;
for (Long time : times) {
totalTime += time;
}
//print average by dividing totalTime by 10.
Related
I have the following Class with some sorting algorithms like MaxSort, BubbleSort, etc.:
class ArrayUtility {
public static int returnPosMax(int[] A, int i, int j) {
int max = i;
int position = 0;
for(int c = 0; c <= j; c++){
if(c >= i){
if(A[c] > max){
max = A[c];
position = c;
}
}
}
return position;
}
public static int returnMax(int[] A, int i, int j) {
return A[returnPosMax(A, i, j)];
}
public static void swap(int[] A, int i, int j) {
int b = A[i];
A[i] = A[j];
A[j] = b;
}
public static void MaxSort(int[] A) {
int posMax;
for(int i = A.length - 1; i >= 0; i--){
posMax = returnPosMax(A, 0, i);
swap(A, posMax, i);
}
}
public static void BubbleSort(int[] A) {
boolean flag = true;
while (flag != false){
flag = false;
for(int i = 1; i <= A.length - 1; i++){
if(A[i-1]>A[i]){
swap(A, i-1, i);
flag = true;
}
}
if(flag = false) {
break;
}
for(int i = A.length - 1; i >= 1; i--){
if(A[i-1]>A[i]){
swap(A, i - 1, i);
flag = true;
}
}
}
}
public static void BubbleSortX(int[] A) {
boolean flag = true;
while (flag != false){
flag = false;
for(int i = 1; i <= A.length - 1; i++){
if(A[i-1]>A[i]){
swap(A, i-1, i);
flag = true;
}
}
}
}
}
Now i have to create a Test Class to evaluate the different sorting algorithms for different lengths of randomly created Arrays:
import java.util.Random;
import java.util.Arrays;
public class TestSorting{
public static void main(String[] args){
int[] lengthArray = {100, 1000, 10000, 100000};
for(int i = 0; i <= lengthArray.length - 1; i++){
int[] arr = new int[i];
for(int j = 0; j < i; j++){
Random rd = new Random();
int randInt = rd.nextInt();
arr[j] = randInt;
}
/* long startTime = System.nanoTime();
ArrayUtility.MaxSort(arr);
long cpuTime = System.nanoTime() - startTime;
System.out.println("Time: " + cpuTime + " - Array with Length: " + lengthArray[i] + " Using MaxSort"); */
/* long startTime = System.nanoTime();
ArrayUtility.BubbleSortX(arr);
long cpuTime = System.nanoTime() - startTime;
System.out.println("Time: " + cpuTime + " - Array with Length: " + lengthArray[i] + " Using BubbleSortX"); */
long startTime = System.nanoTime();
ArrayUtility.BubbleSort(arr);
long cpuTime = System.nanoTime() - startTime;
System.out.println("Time: " + cpuTime + " - Array with Length: " + lengthArray[i] + " Using BubbleSort");
/*long startTime = System.nanoTime();
Arrays.sort(arr)
long cpuTime = System.nanoTime() - startTime;
System.out.println("Time: " + cpuTime + " - Array with Length: " + lengthArray[i] + " Using BubbleSort"); */
}
}
}
Now when i run a certain sorting algorithm (i set the others as comment for the meantime), i get weird results, for example
Time: 1049500 - Array with Length: 100 Using BubbleSort
Time: 2200 - Array with Length: 1000 Using BubbleSort
Time: 13300 - Array with Length: 10000 Using BubbleSort
Time: 3900 - Array with Length: 100000 Using BubbleSort
And any time i run the test i get different results, such that Arrays with 10 times the length take less time to sort, also i dont understand why the array with 100 integers takes so long.
TL;DR: your benchmark is wrong.
Explanation
To make a good benchmark, you need to do a lot of research. A good starting point is this article and this talk by Alexey Shipilev, the author of micro-benchmark toolkit JMH.
Main rules for benchmarking:
warm up! Do a bunch (like, thousands) of warmup rounds before you actually measure stuff - this will allow JIT compiler to do its job, all optimizations to apply, etc.
Monitor your GC closely - GC event can skid the results drastically
To avoid that - repeat the benchmark many (hundreds thousands) times and get the average.
All this can be done in JMH.
I took a snippet out of your code to show you where your code is buggy.
public static void main(String[] args){
int[] lengthArray = {100, 1000, 10000, 100000};
for(int i = 0; i <= lengthArray.length - 1; i++) { // this loop goes from 0 - 3
int[] arr = new int[i]; // thats why this array will be of size 0 - 3
// correct line would be:
// int[] arr = new int[lengthArray[i]];
for(int j = 0; j < i; j++) {
// correct line would be:
// for (int j = 0; j < arr.length; j++) {
...
Additionally, the hint for benchmarking from Dmitry is also important to note.
Comparing creating large ArrayList with intialCapacity I found that it;s slower than creting it without one. Here is the simple program I wrote to measure it:
long start2 = System.nanoTime();
List<Double> col = new ArrayList<>(30000000); // <--- Here
for (int i = 0; i < 30000000; i++) {
col.add(Math.sqrt(i + 1));
}
long end2 = System.nanoTime();
System.out.println(end2 - start2);
System.out.println(col.get(12411325).hashCode() == System.nanoTime());
The average result for new ArrayList<>(30000000): 6121173329
The average result for new ArrayList<>(): 4883894100
on my machine. I thought that it would be faster to create large array once rather than reacreating it once we go beyond the capacity of the current underlying array of ArrayList. Eventually we should have ended up with array size greater or equal than 30000000.
I thought it was optimization, but actualy pessimization. Why that?
I ran the same program multiple times. It was not in a loop
Consider how you are profiling the code - if you include both a 'ramp up time' (to take into account things such as JIT) and average over several calls (to gather some statistics/distribution), the timing may lead you to a different conclusion. For example:
public static void main(String[] args){
//Warm up
System.out.println("Warm up");
for ( int i = 0; i < 5; i++ ){
dynamic();
constant();
}
System.out.println("Timing...");
//time
long e = 0;
long s = 0;
int total = 5;
for ( int i = 0; i < total; i++ ){
long e1 = dynamic();
System.out.print(e1 + "\t");
e += e1;
long s1 = constant();
System.out.println(s1);
s += s1;
}
System.out.println("Static Avg: " + (s/total));
System.out.println("Dynamic Avg: " + (e/total));
}
private static long dynamic(){
long start2 = System.currentTimeMillis();
List<Double> col = new ArrayList<>();
for (int i = 0; i < 30000000; i++) {
col.add(Math.sqrt(i + 1));
}
long end2 = System.currentTimeMillis();
return end2 - start2;
}
private static long constant(){
long start2 = System.currentTimeMillis();
List<Double> col = new ArrayList<>(30000000);
for (int i = 0; i < 30000000; i++) {
col.add(Math.sqrt(i + 1));
}
long end2 = System.currentTimeMillis();
return end2 - start2;
}
On my system setting the initial capacity is always faster, though not by any orders of magnitude.
Edit: As suggested in a comment, consider reading through How do I write a correct micro-benchmark in Java?
I wrote two programs that create random int[] and then generate random ints to search for. Well, really I only wrote one program, then modified the search method (like I'm supposed to for OOP, right?). I'm using System.nanoTime() to calculate the time lapsed during search. I get values of 10,000-13,000 consistently for the linear search, and MOSTLY times less than 5,000 for binary search(averaging in the sort time across all the searches, less than 4,000 typically for the search alone). Occasionally, though, it will through back times over 13,000, which is even longer than all but the very longest linear searches.
I have java priority set to Real Time in the task manager (Windows). What would cause the search time to triple or more? FWIW, I get about 1 spike in linear for every 3 or 4 in binary, and the linear spike is roughly twice the normal time (22,000 was the worst so far). Is it something with the way I've written the code?
If you don't want to read through the whole thing, in both cases I record the time in the line before calling the search method and the line after, so you can skip to the search method if you think that might be the problem. Reporting of that number is copy/pasted from one program to the other, but binary search program has more reporting that I thought would be interesting. Thanks in advance.
import java.util.Random;
public class LinearSearchTimer{
public static void main(String[] args){
int[] numbers;
final int MAX_RUNS;
double time1, time2, timeSpent, timeTotal;
int searchedNumber;
Random rand = new Random();
boolean result;
boolean[] resultArr;
int resultCounter;
double pctTrue;
MAX_RUNS = 1000;
numbers = new int[MAX_RUNS];
resultArr = new boolean[MAX_RUNS];
resultCounter = 0;
timeTotal = 0;
//build array with random ints
for(int k = 0; k < numbers.length; k++){
numbers[k] = rand.nextInt(10001);
}
//generate random number and search array for it, then store to result array
for(int i = 0; i < numbers.length; i++){
searchedNumber = rand.nextInt(10001);
time1 = System.nanoTime();
result = search(numbers, searchedNumber);
time2 = System.nanoTime();
resultArr[resultCounter] = result;
resultCounter++;
timeSpent = time2 - time1;
timeTotal += timeSpent;
}
pctTrue = resultPercent(resultArr);
System.out.println("Avg time : " + timeTotal / MAX_RUNS);
System.out.println("Percent found: " + pctTrue * 100);
}
//search algorithm, returns true if found, false if not
public static boolean search(int[] numbers, int searchedNumber){
int i = 0;
boolean found = false;
while (found == false && i < numbers.length){
if (numbers[i] == searchedNumber)
found = true;
i++;
}
return found;
}
public static double resultPercent(boolean[] arr){
int trueCount = 0;
double result;
double runs;
for(boolean element : arr){
if (element == true)
trueCount++;
}
runs = arr.length;//for calculating percentage as a forced double
result = trueCount / runs;
return result;
}
}
and then...
import java.util.Arrays;
import java.util.Random;
public class BinarySearchTimer{
public static void main(String[] args){
//declare variables
int[] numbers;
final int MAX_RUNS;
double time1, time2, timeSpent, timeTotal;
int searchedNumber;
Random rand;
boolean result;
boolean[] resultArr;
int resultCounter;
double pctTrue;
double timeSort1, timeSort2, timeSortSpent;
//instantiate variables
MAX_RUNS = 1000;
numbers = new int[MAX_RUNS];
resultArr = new boolean[MAX_RUNS];
resultCounter = 0;
timeTotal = 0;
rand = new Random();
//build array with random ints
for(int k = 0; k < numbers.length; k++){
numbers[k] = rand.nextInt(10001);
}
//sort array
timeSort1 = System.nanoTime();
Arrays.sort(numbers);
timeSort2 = System.nanoTime();
timeSortSpent = timeSort2 - timeSort1;
//generate random number and call search method
for(int i = 0; i < numbers.length; i++){
searchedNumber = rand.nextInt(10001);
//get start time
time1 = System.nanoTime();
result = search(numbers, searchedNumber, 0, numbers.length - 1);
//get end time
time2 = System.nanoTime();
resultArr[resultCounter] = result;
resultCounter++;
timeSpent = time2 - time1;
timeTotal += timeSpent;
}
pctTrue = resultPercent(resultArr);
System.out.println("Avg time : " + timeTotal / MAX_RUNS);
System.out.println("Percent found: " + pctTrue * 100);
System.out.println("Actual sort time: " + timeSortSpent);
System.out.println("Sort time shared per search: " + timeSortSpent / MAX_RUNS);
System.out.println("Effective average search time: " + (timeTotal / MAX_RUNS + timeSortSpent / MAX_RUNS));
}
//search algorithm, returns true if found, false if not
public static boolean search(int[] numbers, int searchedNumber, int low, int high){
int mid = (high + low) / 2;
if(low > high)
return false;
if(numbers[mid] == searchedNumber){
return true;
}
else if(searchedNumber < numbers[mid]){
mid--;
return search(numbers, searchedNumber, low, mid);
}
else if(searchedNumber > numbers[mid]){
mid++;
return search(numbers, searchedNumber, mid, high);
}
return false;
}
public static double resultPercent(boolean[] arr){
int trueCount = 0;
double result;
double runs;
for(boolean element : arr){
if (element == true)
trueCount++;
}
runs = arr.length;//for calculating percentage as a forced double
result = trueCount / runs;
return result;
}
}
When I run this code I got time like 0.25s, I want to improve this code to get time better than 0.16s - this is the limit. I know it depends on the hardware. Time is compute without filling array. I must work on array. How can I improve this ?
Update: I want to get the index of first element of slice which avg is minimal.
public static void main(String[] args) {
int[] A = new int[10000] ;
double randomNum ;
for(int i= 0 ; i<A.length ; i++)
{
randomNum = 0 + (double)(Math.random()*1);
if(randomNum>0.5)
A[i]=-1;
else
A[i]=1;
}
System.out.println("Compute");
long start_time = System.nanoTime();
int result = 0;
double minavg = 100000 ;
double avg;
double suma = 0;
int count = 0;
for(int i=0 ; i<A.length-1 ; i++)
{
for(int k=i ; k<A.length ; k++)
{
suma+=A[k];
count++;
if(count>1)
{
if(A[k]<A[k-1]) {
avg=suma/count;
if(minavg>avg)
{
minavg=avg;
result = i;
}
}
}
}
suma=0;
count=0;
}
long end_time = System.nanoTime();
double difference = (end_time - start_time)/1e9;
System.out.println(difference);
}
One thing that you can do is changing the datatype of suma to int, but remember to cast it back to double in avg computation . This will avoid a lot of datatype conversions in incrementations, in my case it took the time down by about 1/7th.
Secondly, I'm not sure about this condition:
if (A[k] < A[k - 1])
Imho it should be like this (and it takes the time down by 1/2):
if (A[k] < minavg)
but I'm not sure if the final result is correct then, but it fixes issue mentioned by #Crferreira
I'm still not sure about Your original algorithm and the task. here is my rewrite of it. on average it takes the time further down to 1/3rd of last measurement
private static void calculateB(int[] A) {
if (A == null || A.length < 2) {
return;
}
long start_time = System.nanoTime();
int result = 0;
double minavg = 100000;
double avg;
int suma = A[A.length - 1];
int count = 1;
for (int i = A.length - 2; i >= 0; i--) {
count += 1;
int previous = A[i];
suma += previous;
if (previous < minavg) {
avg = (double) suma / count;
if (minavg > avg) {
minavg = avg;
result = i;
}
}
}
long end_time = System.nanoTime();
double difference = (end_time - start_time) / 1e6;
System.out.println(result + " " + minavg + " " + difference);
}
BUT the results from this one are different from original, you have to check if it validates against assignment. I did make the calculations manually and it seems to be OK. This one, iterates the table from the end. Thanks to this, we have a base for sum that we reuse in further iterations.
As for the general rule - I'm not sure what you are calculating - from the code, it looks like it is index of item with lowest avg of following items including the one on index - is this correct ?
I tried to look into that in javascript. I started with an average time of 0.101 and get down to 0.076 with those small modifications:
function main() {
var A = new Array() ;
var randomNum ;
for(var i= 0 ; i<1000 ; i++)
{
randomNum = 0 + Math.random();
if(randomNum>0.5)
A[i]=-1;
else
A[i]=1;
}
console.log("Compute");
var start_time = new Date().getTime();;
var result = 0;
var minavg = 100000 ;
for(var i=0 ; i<A.length-1 ; i++)
{
var suma= A[i];
var count=1;
for(var k=i+1 ; k<A.length ; k++)
{
suma+=A[k];
count++;
if(A[k]<A[k-1]) {
var avg=suma/count;
if(minavg>avg)
{
minavg=avg;
result = i;
}
}
}
}
var end_time = new Date().getTime();
var difference = (end_time - start_time)*1e-3;
console.log(difference);
}
main();
My point was to limit the number or tests and allocate local variable at the very last time in order to maximum CPU register and context. On a compile java version you might have even better results than me.
One optimization is removing the count > 1 test by rearranging the code like this
for(int i=0 ; i<A.length-1 ; i++) {
suma=A[i];
for(int k=i+1 ; k<A.length ; k++) {
suma+=A[k];
count++;
if(A[k]<A[k-1]) {
I want to compute the average execution time of x number of runs (i.e. 10)... I can easily execute 10 times using the loop in the main method, but how can I store the execution times & compute the average? I'm thinking this is something really simple, but I'm drawing a blank at the moment... Thanks in advanced!
import java.util.Arrays;
import java.util.Random;
public class OptQSort1 {
static boolean insertionSortCalled = false;
private static final Random random = new Random();
private static final int RANDOM_INT_RANGE = 9999;
private static int[] randomArray(int size) {
// Randomize data (array)
final int[] arr = new int[size];
for (int i = 0; i < arr.length; i++) {
arr[i] = random.nextInt(RANDOM_INT_RANGE);
}
return arr;
}
// Sort
private static void sort(int[] arr) {
if (arr.length > 0)
sortInPlace(arr, 0, arr.length - 1);
}
private static void sortInPlace(int[] arr, int left, int right) {
// OptQSort1:
int size = right - left + 1;
if (size < 10 && !insertionSortCalled) {
insertionSortCalled = true;
insertionSort(arr, 0, arr.length - 1);
}
if (left >= right)
return; // sorted
final int range = right - left + 1;
int pivot = random.nextInt(range) + left;
int newPivot = partition(arr, left, right, pivot);
sortInPlace(arr, left, newPivot - 1);
sortInPlace(arr, newPivot + 1, right);
}
private static int partition(int[] arr, int left, int right, int pivot) {
int pivotVal = arr[pivot];
swapArrayVals(arr, pivot, right);
int storeIndex = left;
for (int i = left; i <= (right - 1); i++) {
if (arr[i] < pivotVal) {
swapArrayVals(arr, i, storeIndex);
storeIndex++;
}
}
swapArrayVals(arr, storeIndex, right);
return storeIndex;
}
private static void swapArrayVals(int[] arr, int from, int to) {
int fromVal = arr[from];
int toVal = arr[to];
arr[from] = toVal;
arr[to] = fromVal;
}
public static void insertionSort(int[] arr, int left, int right) {
int in, out;
for (out = left + 1; out <= right; out++) {
int temp = arr[out];
in = out;
while (in > left && arr[in - 1] >= temp) {
arr[in] = arr[in - 1];
--in;
}
arr[in] = temp;
}
}
public static void main(String[] args) {
long StartTime = System.nanoTime();
int runCount = 0;
// Array size
int[] arr = randomArray(1000);
int[] copy = Arrays.copyOf(arr, arr.length);
// Print original data (array)
System.out.println("The starting/unsorted array: \n"
+ Arrays.toString(arr));
sort(arr);
do {
// check the result
Arrays.sort(copy);
if (Arrays.equals(arr, copy)) {
System.out.println("The ending/sorted array: \n"
+ Arrays.toString(arr));
// print time
long TotalTime = System.nanoTime() - StartTime;
System.out.println("Total elapsed time (milliseconds) " + "is: "
+ TotalTime + "\n");
runCount++;
}
} while (runCount < 10);
}
}
You can compute the average by just measuring the total time for 10 iterations of your code, then divide that by 10.
e.g:
public static void main(String[] args) {
long start = System.currentTimeMillis();
for (int i = 0; i < 10; ++i) {
doSort();
}
long elapsed = System.currentTimeMillis() - start;
long average = elapsed / 10;
}
As a helpful tip, use a named constant rather than a literal value for the number of iterations:
private final static int ITERATIONS = 10;
public static void main(String[] args) {
long start = System.currentTimeMillis();
for (int i = 0; i < ITERATIONS; ++i) {
doSort();
}
long elapsed = System.currentTimeMillis() - start;
long average = elapsed / ITERATIONS;
}
This means you only have to change the number in one place if you want to run, say, 50 or 100 iterations.
You should also be aware that it is very difficult to get accurate timing results from this kind of experiment. It's a good idea to include a "warm-up" phase to allow the JIT to evaluate and optimize the code, and to have a much larger number of iterations:
private static final int WARMUP_ITERATIONS = 10000;
private static final int RUN_ITERATIONS = 100000;
public static void main(String[] args) {
// Warmup with no timing
for (int i = 0; i < WARMUP_ITERATIONS; ++i) {
doSort();
}
// Now the real test
long start = System.currentTimeMillis();
for (int i = 0; i < RUN_ITERATIONS; ++i) {
doSort();
}
long elapsed = System.currentTimeMillis() - start;
long average = elapsed / RUN_ITERATIONS;
}
To calculate the average time, you need the sum of the times. The sum of the times is the total time, so you don't even need to know the individual times or record them. Just take the end-to-end time and divide by the count.
int count = ...
long start = System.nanoTime();
for(int i=0;i<count;i++) {
// do something
}
long time = System.nanoTime() - start;
long averageTime = time/count;
The JIT doesn't fully warmup until you have done at least 10,000 iterations, so you might ignore the first 11,000 if this is practical.
A simple way to do this is
int count = ...
long start = 0;
for(int i=-11000;i<count;i++) {
if(i == 0) start = System.nanoTime();
// do something
}
long time = System.nanoTime() - start;
long averageTime = time/count;
BTW: Only include in the test time the things you want to time. Generating random numbers , for example, could take longer than the sort itself which could give misleading results.
EDIT: The compile threshold which determines when a method or loop is compiled is controlled with -XX:CompileThresholed= which defaults to 10000 on the server JVM and 1500 on the client JVM. http://www.oracle.com/technetwork/java/javase/tech/vmoptions-jsp-140102.html
-XX:CompileThreshold=10000 Number of method invocations/branches before
compiling [-client: 1,500]
You can use a list of integers to store the result for each run, but you don't need it to calculate average, just divide totaltime by number of runs.
Btw, your measurements are not very good:
1) Generation of random arrays is included there
2) 10 runs is not enought
before the execution:
long start = System.currentTimeMillis();
after the execution:
long end = System.currentTimeMillis();
add each time in an ArrayList like this:
times.add(end-start);
get the average time:
Long total = 0;
for(Long l : times)
total += l;
System.out.println("Average Time: "+(total/times.size()));
Be careful the unit of time of the return value is a millisecond.
Just keep a second long value that is a running total of the Totaltime values. When the loop exits, just divide by runCount.
Alternatively, create an ArrayList<Long> to store the times. Each time you do one run, add Totaltime to the array. After the loop exits, you can average the values and also compute other statistics (min/max, standard deviation, etc.).