How do apps measure CPU usage (as a %)? - java

So I'm trying to write an app that measures CPU usage (ie, the time CPU is working vs the time it isn't). I've done some research, but unfortunately there are a bunch of different opinions on how it should be done.
These different solutions include, but aren't limited to:
Get Memory Usage in Android
and
http://juliano.info/en/Blog:Memory_Leak/Understanding_the_Linux_load_average
I've tried writing some code myself, that I though might do the trick, because the links above don't take into consideration when the core is off (or do they?)
long[][] cpuUseVal = {{2147483647, 0} , {2147483647, 0} , {2147483647, 0} ,
{2147483647, 0} , {2147483647, 0}};
public float[] readCPUUsage(int coreNum) {
int j=1;
String[] entries; //Array to hold entries in the /proc/stat file
int cpu_work;
float percents[] = new float[5];
Calendar c = Calendar.getInstance();
// Write the dataPackage
long currentTime = c.getTime().getTime();
for (int i = 0; i <= coreNum; i++){
try {
//Point the app to the file where CPU values are located
RandomAccessFile reader = new RandomAccessFile("/proc/stat", "r");
String load = reader.readLine();
while (j <= i){
load = reader.readLine();
j++;
}
//Reset j for use later in the loop
j=1;
entries = load.split("[ ]+");
//Pull the CPU working time from the file
cpu_work = Integer.parseInt(entries[1]) + Integer.parseInt(entries[2]) + Integer.parseInt(entries[3])
+ Integer.parseInt(entries[6]) + Integer.parseInt(entries[6]) + Integer.parseInt(entries[7]);
reader.close();
percents[i] = (float)(cpu_work - cpuUseVal[i][1]) / (currentTime - cpuUseVal[i][0]);
cpuUseVal[i][0] = currentTime;
cpuUseVal[i][1] = cpu_work;
//In case of an error, print a stack trace
} catch (IOException ex) {
ex.printStackTrace();
}
}
//Return the array holding the usage values for the CPU, and all cores
return percents;
}
So here is the idea of the code I wrote...I have a global array with some dummy values that should return negative percentages the first time the function is run. The values are being stored in a database, so I would know to disregard anything negative. Anyway, the function runs, getting values of time the cpu is doing certain things, and comparing it to the last time the function is run (with the help of the global array). These values are divided by the amount of time that has passed between the function runs (with the help of the calendar)
I've downloaded some of the existing cpu usage monitors and compared them to values I get from my app, and mine are never even close to what they get. Can someone explain what I'm doing wrong?
Thanks to some help I have changed my function to look like the following, hope this helps others who have this question
// Function to read values from /proc/stat and do computations to compute CPU %
public float[] readCPUUsage(int coreNum) {
int j = 1;
String[] entries;
int cpu_total;
int cpu_work;
float percents[] = new float[5];
for (int i = 0; i <= coreNum; i++) {
try {
// Point the app to the file where CPU values are located
RandomAccessFile reader = new RandomAccessFile("/proc/stat","r");
String load = reader.readLine();
// Loop to read down to the line that corresponds to the core
// whose values we are trying to read
while (j <= i) {
load = reader.readLine();
j++;
}
// Reset j for use later in the loop
j = 1;
// Break the line into separate array elements. The end of each
// element is determined by any number of spaces
entries = load.split("[ ]+");
// Pull the CPU total time on and "working time" from the file
cpu_total = Integer.parseInt(entries[1])
+ Integer.parseInt(entries[2])
+ Integer.parseInt(entries[3])
+ Integer.parseInt(entries[4])
+ Integer.parseInt(entries[5])
+ Integer.parseInt(entries[6])
+ Integer.parseInt(entries[7]);
cpu_work = Integer.parseInt(entries[1])
+ Integer.parseInt(entries[2])
+ Integer.parseInt(entries[3])
+ Integer.parseInt(entries[6])
+ Integer.parseInt(entries[7]);
reader.close();
//If it was off the whole time, say 0
if ((cpu_total - cpuUseVal[i][0]) == 0)
percents[i] = 0;
//If it was on for any amount of time, compute the %
else
percents[i] = (float) (cpu_work - cpuUseVal[i][1])
/ (cpu_total - cpuUseVal[i][0]);
//Save the values measured for future comparison
cpuUseVal[i][0] = cpu_total;
cpuUseVal[i][1] = cpu_work;
// In case of an error, print a stack trace
} catch (IOException ex) {
ex.printStackTrace();
}
}
// Return the array holding the usage values for the CPU, and all cores
return percents;
}

Apps don't measure CPU usage, the kernel does by interrupting the process 100 times per second (or some other frequency depending on how the kernel is tuned) and incrementing a counter which corresponds to what it was doing when interrupted.
If in the process => increment the user counter.
If in the kernel => increment the system counter
If waiting for disk or network or a device => increment the waiting for IO
Otherwise increment the idle counter.
The uptime is determined by the decaying average length of the run queue i.e. how many threads are waiting to run. The first number is the average length over the last minute. You can get the load average via JMX.

Related

Time how long a function runs (short duration)

I'm relatively new to Java programming, and I'm running into an issue calculating the amount of time it takes for a function to run.
First some background - I've got a lot of experience with Python, and I'm trying to recreate the functionality of the Jupyter Notebook/Lab %%timeit function, if you're familiar with that. Here's a pic of it in action (sorry, not enough karma to embed yet):
Snip of Jupyter %%timeit
What it does is run the contents of the cell (in this case a recursive function) either 1k, 10k, or 100k times, and give you the average run time of the function, and the standard deviation.
My first implementation (using the same recursive function) used System.nanoTime():
public static void main(String[] args) {
long t1, t2, diff;
long[] times = new long[1000];
int t;
for (int i=0; i< 1000; i++) {
t1 = System.nanoTime();
t = triangle(20);
t2 = System.nanoTime();
diff = t2-t1;
System.out.println(diff);
times[i] = diff;
}
long total = 0;
for (int j=0; j<times.length; j++) {
total += times[j];
}
System.out.println("Mean = " + total/1000.0);
}
But the mean is wildly thrown off -- for some reason, the first iteration of the function (on many runs) takes upwards of a million nanoseconds:
Pic of initial terminal output
Every iteration after the first dozen or so takes either 395 nanos or 0 -- so there could be a problem there too... not sure what's going on!
Also -- the code of the recursive function I'm timing:
static int triangle(int n) {
if (n == 1) {
return n;
} else {
return n + triangle(n -1);
}
}
Initially I had the line n = Math.abs(n) on the first line of the function, but then I removed it because... meh. I'm the only one using this.
I tried a number of different suggestions brought up in this SO post, but they each have their own problems... which I can go into if you need.
Anyway, thank you in advance for your help and expertise!

How to find missing audio segments

I am writing my own audio format as part of a game console project. Part of the project requires me to write an emulator so I know exactly how to implement it's functions in hardware. I am currently writing the DSP portion, but I am having trouble writing a decoding algorithm. Before I go further, I'll explain my format.
DST (Dingo Sound Track) Audio format
The audio format only records to pieces of data per sample: the amplitude and the number of frames since the last sample. I'll explain. When converting an audio file (WAV for example), it compares the current sample with the previous one. If it detects that the current sample switches amplitude direction in relation to the previous sample, it records the previous sample and the number of frames since the last record. It keeps going until the end of the file. Here is a diagram to explain further:
What I need to do
I need my "DSP" to figure out the data between each sample, as accurately as possible using only the given information. I don't think it's my encoding algorithm, because when I play the file in Audacity, I can sort of make out the original song. But when I try to play it with my decoding algorithm, I get scattered clicks. I am able to play WAV files directly with a few mods to the algorithm with almost no quality drop, so I know it's definitely the algorithm and not the rest of the DSP.
The Code
So now I got all of the basic info out of the way, here is my code (only the important parts).
Encoding algorithm:
FileInputStream s = null;
BufferedWriter bw;
try {
int bytes;
int previous = 0;
int unsigned;
int frames = 0;
int size;
int cursor = 0;
boolean dir = true;
int bytes2;
int previous2 = 0;
int unsigned2;
int frames2 = 0;
boolean dir2 = true;
s = new FileInputStream(selectedFile);
size = (int)s.getChannel().size();
File f = new File(Directory.getPath() + "\\" + (selectedFile.getName().replace(".wav", ".dts")));
System.out.println(f.getPath());
if(!f.exists()){
f.createNewFile();
}
bw = new BufferedWriter(new FileWriter(f));
try (BufferedInputStream b = new BufferedInputStream(s)) {
byte[] data = new byte[128];
b.skip(44);
System.out.println("Loading...");
while ((bytes = b.read(data)) > 0) {
// do something
for(int i=1; i<bytes; i += 4) {
unsigned = data[i] & 0xFF;
if (dir) {
if (unsigned < previous) {
bw.write(previous);
bw.write(frames);
dir = !dir;
frames = 0;
}else{
frames ++;
}
} else {
if (unsigned > previous) {
bw.write(previous);
bw.write(frames);
dir = !dir;
frames = 0;
}else{
frames ++;
}
}
previous = unsigned;
cursor ++;
unsigned2 = data[i + 2] & 0xFF;
if (dir2) {
if (unsigned2 < previous2) {
bw.write(previous2);
bw.write(frames2);
dir2 = !dir2;
frames2 = 0;
}else{
frames2 ++;
}
} else {
if (unsigned2 > previous2) {
bw.write(previous2);
bw.write(frames2);
dir2 = !dir2;
frames2 = 0;
}else{
frames2 ++;
}
}
previous2 = unsigned2;
cursor ++;
progress.setValue((int)(((float)(cursor / size)) * 100));
}
}
b.read(data);
}
bw.flush();
bw.close();
System.out.println("Done");
convert.setEnabled(true);
status.setText("finished");
} catch (Exception ex) {
status.setText("An error has occured");
ex.printStackTrace();
convert.setEnabled(true);
}
finally {
try {
s.close();
} catch (Exception ex) {
status.setText("An error has occured");
ex.printStackTrace();
convert.setEnabled(true);
}
}
The progress and status objects can be ignored for they are part of the GUI of my converter tool. This algorithm converts WAV files to my format (DST).
Decoding algorithm:
int start = bufferSize * (bufferNumber - 1);
short current;
short frames;
short count = 1;
short count2 = 1;
float jump;
for (int i = 0; i < bufferSize; i ++) {
current = RAM.read(start + i);
i++;
frames = RAM.read(start + i);
if (frames == 0) {
buffer[count - 1] = current;
count ++;
} else {
jump = current / frames;
for (int i2 = 1; i2 < frames; i2++) {
buffer[(2 * i2) - 1] = (short) (jump * i2);
count ++;
}
}
i++;
current = RAM.read(start + i);
i++;
frames = RAM.read(start + i);
if (frames == 0) {
buffer[count2] = current;
count2 ++;
} else {
jump = current / frames;
for (int i2 = 1; i2 < frames; i2++) {
buffer[2 * i2] = (short) (jump * i2);
count2 ++;
}
}
}
bufferNumber ++;
if(bufferNumber > maxBuffer){
bufferNumber = 1;
}
The RAM object is just a byte array. bufferNumber and maxBuffer refer to the amount of processing buffers the DSP core uses. buffer is the object that the resulting audio is written to. This algorithm set is designed to convert stereo tracks, which works the same way in my format but each sample will contain two sets of data, one for each track.
The Question
How do I figure out the missing audio between each sample, as accurately as possible, and how accurate will the approach be? I would love to simply use the WAV format, but my console is limited on memory (RAM). This format halves the RAM space required to process audio. I am also planning on implementing this algorithm in an ARM microcontroller, which will be the console's real DSP. The algorithm should also be fast, but accuracy is more important. If I need to clarify or explain anything further, let me know since this is my first BIG question and I am sure I forgot something. Code samples would be nice, but aren't needed that much.
EDIT:
I managed to get the DSP to output a song, but it's sped up and filled with static. The sped up part is due to a glitch in it not splitting the track into stereo (I think). And the static is due to the initial increment being too steep. Here is a picture of what I'm getting:
Here is the new code used in the DSP:
if (frames == 0) {
buffer[i - 1] = current;
//System.out.println(current);
} else {
for (int i2 = 1; i2 < frames + 1; i2++) {
jump = (float)(previous + ((float)(current - previous) / (frames - i2 + 1)));
//System.out.println((short)jump);
buffer[(2 * i2) - 1] = (short)(jump);
}
}
previous = current;
I need a way to smooth out those initial increments, and I'd prefer not to use complex arithmetic because I am limited on performance when I port this to hardware (preferably something that can operate on a 100MHZ ARM controller while being able to keep a 44.1KHZ sample rate). Edit: the result wave should actually be backwards. Sorry.
Second Edit:
I got the DSP to output in stereo, but unfortunately that didn't fix anything else like I hoped it would. I also fixed some bugs with the encoder so now it takes 8 bit unsigned audio. This has become more of a math issue so I think I'll post a similar question in Mathematics Stack Exchange. Well that was a waste of time. It got put on fhold near instantly.
You have basically a record of the signal's local extrema and want to reconstruct the signal. The most straight-forward way would be to use some monotonic interpolation scheme. You can try if this fits your needs. But I guess, the result would be very inaccurate because the characteristics of the signal are ignored.
I am not an audio engineer, so my assumptions could be wrong. But maybe, you get somewhere with these thoughts.
The signal is basically a mixture of sines. Calculating a sine function for any segment between two key frames is quite easy. The period is given by twice their distance. The amplitude is given by half the amplitude difference. This will give you a sine that hits the two key samples exactly. Furthermore, it will give you a C1-continuous signal because the derivatives at the connection points are zero. For a nice signal, you probably need even more smoothness. So you could start to interpolate the two sines around a key frame with an appropriate window function. I would start with a simple triangle window but others may give better results. This procedure will preserve the extrema.
It is probably easier to tackle this problem visually (with a plot of the signal), so you can see the results.
If it's all about size, then maybe you want to look into established audio compression methods. They usually give much better compression ratio than 1:2. Also, I don't understand why this method saves RAM because you'll have to calculate all samples when decoding. Of course, this assumes that not the complete data are loaded into RAM but streamed in pieces.

Java wordcount: a mediocre implementation

I implemented a wordcount program with Java. Basically, the program takes a large file (in my tests, I used a 10 gb data file that contained numbers only), and counts the number of times each 'word' appears - in this case, a number (23723 for example might appear 243 times in the file).
Below is my implementation. I seek to improve it, with mainly performance in mind, but a few other things as well, and I am looking for some guidance. Here are a few of the issues I wish to correct:
Currently, the program is threaded and works properly. However, what I do is pass a chunk of memory (500MB/NUM_THREADS) to each thread, and each thread proceeds to wordcount. The problem here is that I have the main thread wait for ALL the threads to complete before passing more data to each thread. It isn't too much of a problem, but there is a period of time where a few threads will wait and do nothing for a while. I believe some sort of worker pool or executor service could solve this problem (I have not learned the syntax for this yet).
The program will only work for a file that contains integers. That's a problem. I struggled with this a lot, as I didn't know how to iterate through the data without creating loads of unused variables (using a String or even StringBuilder had awful performance). Currently, I use the fact that I know the input is an integer, and just store the temporary variables as an int, so no memory problems there. I want to be able to use some sort of delimiter, whether that delimiter be a space, or several characters.
I am using a global ConcurrentHashMap to story key value pairs. For example, if a thread finds a number "24624", it searches for that number in the map. If it exists, it will increase the value of that key by one. The value of the keys at the end represent the number of occurrences of that key. So is this the proper design? Would I gain in performance by giving each thread it's own hashmap, and then merging them all at the end?
Is there any other way of seeking through a file with an offset without using the class RandomAccessMemory? This class will only read into a byte array, which I then have to convert. I haven't timed this conversion, but maybe it could be faster to use something else.
I am open to other possibilities as well, this is just what comes to mind.
Note: Splitting the file is not an option I want to explore, as I might be deploying this on a server in which I should not be creating my own files, but if it would really be a performance boost, I might listen.
Other Note: I am new to java threading, as well as new to StackOverflow. Be gentle.
public class BigCount2 {
public static void main(String[] args) throws IOException, InterruptedException {
int num, counter;
long i, j;
String delimiterString = " ";
ArrayList<Character> delim = new ArrayList<Character>();
for (char c : delimiterString.toCharArray()) {
delim.add(c);
}
int counter2 = 0;
num = Integer.parseInt(args[0]);
int bytesToRead = 1024 * 1024 * 1024 / 2; //500 MB, size of loop
int remainder = bytesToRead % num;
int k = 0;
bytesToRead = bytesToRead - remainder;
int byr = bytesToRead / num;
String filepath = "C:/Users/Daniel/Desktop/int-dataset-10g.dat";
RandomAccessFile file = new RandomAccessFile(filepath, "r");
Thread[] t = new Thread [num];//array of threads
ConcurrentMap<Integer, Integer> wordCountMap = new ConcurrentHashMap<Integer, Integer>(25000);
byte [] byteArray = new byte [byr]; //allocates 500mb to a 2D byte array
char[] newbyte;
for (i = 0; i < file.length(); i += bytesToRead) {
counter = 0;
for (j = 0; j < bytesToRead; j += byr) {
file.seek(i + j);
file.read(byteArray, 0, byr);
newbyte = new String(byteArray).toCharArray();
t[counter] = new Thread(
new BigCountThread2(counter,
newbyte,
delim,
wordCountMap));//giving each thread t[i] different file fileReader[i]
t[counter].start();
counter++;
newbyte = null;
}
for (k = 0; k < num; k++){
t[k].join(); //main thread continues after ALL threads have finished.
}
counter2++;
System.gc();
}
file.close();
System.exit(0);
}
}
class BigCountThread2 implements Runnable {
private final ConcurrentMap<Integer, Integer> wordCountMap;
char [] newbyte;
private ArrayList<Character> delim;
private int threadId; //use for later
BigCountThread2(int tid,
char[] newbyte,
ArrayList<Character> delim,
ConcurrentMap<Integer, Integer> wordCountMap) {
this.delim = delim;
threadId = tid;
this.wordCountMap = wordCountMap;
this.newbyte = newbyte;
}
public void run() {
int intCheck = 0;
int counter = 0; int i = 0; Integer check; int j =0; int temp = 0; int intbuilder = 0;
for (i = 0; i < newbyte.length; i++) {
intCheck = Character.getNumericValue(newbyte[i]);
if (newbyte[i] == ' ' || intCheck == -1) { //once a delimiter is found, the current tempArray needs to be added to the MAP
check = wordCountMap.putIfAbsent(intbuilder, 1);
if (check != null) { //if returns null, then it is the first instance
wordCountMap.put(intbuilder, wordCountMap.get(intbuilder) + 1);
}
intbuilder = 0;
}
else {
intbuilder = (intbuilder * 10) + intCheck;
counter++;
}
}
}
}
Some thoughts on a little of most ..
.. I believe some sort of worker pool or executor service could solve this problem (I have not learned the syntax for this yet).
If all the threads take about the same time to process the same amount of data, then there really isn't that much of a "problem" here.
However, one nice thing about a Thread Pool is it allows one to rather trivially adjust some basic parameters such as number of concurrent workers. Furthermore, using an executor service and Futures can provide an additional level of abstraction; in this case it could be especially handy if each thread returned a map as the result.
The program will only work for a file that contains integers. That's a problem. I struggled with this a lot, as I didn't know how to iterate through the data without creating loads of unused variables (using a String or even StringBuilder had awful performance) ..
This sounds like an implementation issue. While I would first try a StreamTokenizer (because it's already written), if doing it manually, I would check out the source - a good bit of that can be omitted when simplifying the notion of a "token". (It uses a temporary array to build the token.)
I am using a global ConcurrentHashMap to story key value pairs. .. So is this the proper design? Would I gain in performance by giving each thread it's own hashmap, and then merging them all at the end?
It would reduce locking and may increase performance to use a separate map per thread and merge strategy. Furthermore, the current implementation is broken as wordCountMap.put(intbuilder, wordCountMap.get(intbuilder) + 1) is not atomic and thus the operation might under count. I would use a separate map simply because reducing mutable shared state makes a threaded program much easier to reason about.
Is there any other way of seeking through a file with an offset without using the class RandomAccessMemory? This class will only read into a byte array, which I then have to convert. I haven't timed this conversion, but maybe it could be faster to use something else.
Consider using a FileReader (and BufferedReader) per thread on the same file. This will avoid having to first copy the file into the array and slice it out for individual threads which, while the same amount of total reading, avoids having to soak up so much memory. The reading done is actually not random access, but merely sequential (with a "skip") starting from different offsets - each thread still works on a mutually exclusive range.
Also, the original code with the slicing is broken if an integer value was "cut" in half as each of the threads would read half the word. One work-about is have each thread skip the first word if it was a continuation from the previous block (i.e. scan one byte sooner) and then read-past the end of it's range as required to complete the last word.

For Loop is performing slow

Please have a look at the following code
//Devide the has into set of 3 pieces
private void devideHash(String str)
{
int lastIndex = 0;
for(int i=0;i<=str.length();i=i+3)
{
lastIndex = i;
try
{
String stringPiece = str.substring(i, i+3);
// pw.println(stringPiece);
hashSet.add(stringPiece);
}
catch(Exception arr)
{
String stringPiece = str.substring(lastIndex, str.length());
// pw.println(stringPiece);
hashSet.add(stringPiece);
}
}
}
The above method receives String like abcdefgjijklmnop as the parameter. Inside the method, its job is to divide this sets of 3 letters. So when the operation is completed, the hashset will have pieces like abc def ghi jkl mno p
But the problem is that if the input String is big, then this loop takes noticeable amount of time to complete. Is there any way I can use to speed this process?
As an option, you could replace all your code with this line:
private void divideHash(String str) {
hashSet.addAll(Arrays.asList(str.split("(?<=\\G...)")));
}
Which will perform well.
Here's some test code:
String str = "abcdefghijklmnop";
hashSet.addAll(Arrays.asList(str.split("(?<=\\G...)")));
System.out.println(hashSet);
Output:
[jkl, abc, ghi, def, mno, p]
There is nothing we can really tell unless you tell us what the "noticeable large amount" is, and what is the expected time. It is recommended that you start a profiler to find what logic takes most time.
Some recommendations I can give from briefly reading your code is:
If the result Set is going to be huge, it will involve lots of resize and rehashing when your HashSet resize. It is recommended you first allocate required size. e.g.
HashSet hashSet = new HashSet<String>(input.size() / 3 + 1, 1.0);
This will save you lots of time for unnecessary rehashing
Never use exception to control your program flow.
Why not simply do:
int i = 0;
for (int i = 0; i < input.size(); i += 3) {
if (i + 3 > input.size()) {
// substring from i to end
} else {
// subtring from i to i+3
}
}

How should I parallelize a computationally expensive for loop and collate the iteration results?

I'm working on an 8-core machine and am performing a computationally heavy task. However, each execution of the task (i.e., iteration of for loop) is rather independent of the previous one. There are only some variables that are 'summed up' from one execution to the next. I'm guessing this is a good example for parallelizing/threading but I'm not sure how to go about it.
Here's how the code looks. As of now, it's just part of the main method in my main executor class:
double testerPayoffSum = 0.0, developerPayoffSum = 0.0;
Random seed = new Random();
try {
for (int i = 0; i < GameConstants.MAX_GAMES; i++) {
EraserSimulator eraser = new EraserSimulator(GameConstants.MAX_TARGETS, GameConstants.MAX_RESOURCES, GameConstants.NUM_ATTACKER_TYPES, seed.nextInt());
Map<Set<SingleObjectiveTarget>, Double> gameStrategy = eraser.run();
assert (gameStrategy != null);
TestingGameSimulator testingGame = new TestingGameSimulator(GameConstants.MAX_TARGETS, gameStrategy, GameConstants.NUM_GAMES_TO_STORE_FOR_HISTORY, GameConstants.NUM_TESTING_GAMES_TO_PLAY);
PlayerPayoffs payoffs = testingGame.run(eraser.getEraserInstance());
testerPayoffSum += payoffs.getAverageTesterPayoff(GameConstants.NUM_TESTING_GAMES_TO_PLAY);
developerPayoffSum += payoffs.getAverageDeveloperPayoff(GameConstants.NUM_TESTING_GAMES_TO_PLAY);
System.out.print("Output: ERASER Games played; Number of developers caught");
System.out.print(", " + GameConstants.NUM_TESTING_GAMES_TO_PLAY + ", " + payoffs.getNumTimesCaught() + "\n");
} catch(Exception e){sendEmailAlert("Execution Failed with Exception");}
I'd like to parallelize the for-loop computation if possible and keep summing up the testerPayoffSum and developerPayofffSum variables. How might I achieve this?
Note: Each execution of the for loop takes about 20-30 minutes depending on the input size (as set by the various GameConstants). Even for a small number of MAX_GAMES the above takes close to 2-3 hours.
Create a thread object implementing Callable which returns a Future object containing your testerPayoffSum and developerPayoffSum, start the calculation and sum the results obtained from the Futures (See also https://blogs.oracle.com/CoreJavaTechTips/entry/get_netbeans_6).
are you absolutely sure you have no dependency ?
1.used classes must not share any variables
if it does then you have to add locks
but it will affect performance
if some shared variables are used extensively
then the performance can drop significantly even bellow the non-parallel execution
2.used classes must not use any kind of machine learning.
there is no solution for this
because parallelization will corrupt your results
Now how to do it (I am not JAVA coder so I stick to C++ code).
//--- globals and headers -----------------------------------------------------
unsigned long __stdcall function(LPVOID p);
Random seed = new Random();
const int N=8; // threads count (<=CPU count)
int id[N]; // thread id
int max[N]; // number of games per thread
double testerPayoffSum[N]; // sum to separate variables to avoid locks need
double developerPayoffSum[N];
volatile int run=0,stop=0; // thread control variables run is number of running threads and stop force stop...
//--- main code ---------------------------------------------------------------
// init some variables ... may be the seed init will be better here too
int i;
for (i = 0; i < N; i++)
{
id[i]=i;
max[i]=GameConstants.MAX_GAMES / N;
testerPayoffSum[i]=0.0;
developerPayoffSum[i]=0.0;
}
max[0]=GameConstants.MAX_GAMES % N;
// create threads
for (i = 0; i < N; i++)
{
HANDLE hnd=CreateThread(0,0,function,&id[i],0,0);
if (hnd!=NULL) CloseHandle(hnd); // this line is important !!!
// because if you do not close Handle it will be allocated until the end of app
// handle leaks are nasty and cause weird OS behaviour
// I saw many times this bug in commercial drivers
// it is a nightmare for 24/7 software
}
// wait for them
while (run) Sleep(200);
// sum the results to [0]
for (i = 1; i < N; i++)
{
testerPayoffSum[0] +=testerPayoffSum[i];
developerPayoffSum[0]+=developerPayoffSum[i];
}
// here do what you need to do with the results
//--- thread function ---------------------------------------------------------
unsigned long __stdcall function(LPVOID p)
{
run++;
int ix=((int*)p)[0];
for (i = 0; i < max[ix]; i++)
{
if (stop) break;
EraserSimulator eraser = new EraserSimulator(GameConstants.MAX_TARGETS, GameConstants.MAX_RESOURCES, GameConstants.NUM_ATTACKER_TYPES, seed.nextInt());
Map<Set<SingleObjectiveTarget>, Double> gameStrategy = eraser.run();
assert (gameStrategy != null);
TestingGameSimulator testingGame = new TestingGameSimulator(GameConstants.MAX_TARGETS, gameStrategy, GameConstants.NUM_GAMES_TO_STORE_FOR_HISTORY, GameConstants.NUM_TESTING_GAMES_TO_PLAY);
PlayerPayoffs payoffs = testingGame.run(eraser.getEraserInstance());
testerPayoffSum[ix] += payoffs.getAverageTesterPayoff(GameConstants.NUM_TESTING_GAMES_TO_PLAY);
developerPayoffSum[ix] += payoffs.getAverageDeveloperPayoff(GameConstants.NUM_TESTING_GAMES_TO_PLAY);
// do not call any visual stuff from thread !!! sometimes it can cause a lot of problems ...
// instead cretae some global string variable and set it to what shoud be printed out
// and inside wait while loop in main code add if string != "" then System.out.print(string);
// but in that case you should add lock to it.
// System.out.print("Output: ERASER Games played; Number of developers caught");
// System.out.print(", " + GameConstants.NUM_TESTING_GAMES_TO_PLAY + ", " + payoffs.getNumTimesCaught() + "\n");
//Sleep(100); // well placed sleep
}
run--;
}
[Notes]
from your code I am assuming that GameConstants is shared variable !!!
if it is only for read than it is OK
but if you do also write to it inside thread (I suspect that yes)
then you have a big problem because you need to add locks inside your game class then ...
if no machine learning is done then you could avoid this
by creating separate GameConstants variables for each thread like ... GameConstants[N]
but you need to rewrite the code so it access the GameConstants[ix] and not GameConstants
[lock]
have no clue how locks are implemented in JAVA
but you can also use your own something like this
class _lock
{
public:
volatile bool locked;
_lock() { locked=false; }
void lock() { while(locked) Sleep(1); locked=true; }
void unlock() { locked=false; }
};
// now for each shared variable (or group of variables) add one global _lock variable
_lock l1; int sv1; // shared variable 1 and her lock
// any write access and sometimes also read access needs lock
l1.lock();
sv1++;
l1.unlock();
beware that locks can sometimes cause App freeze especially while heavy duty use.
does not matter if it is own lock or OS lock
this occurs mainly while mixing visual stuff or some OS calls inside threads and not in main thread
in that case sometimes a well placed sleep helps but avoid OS calls inside threads if you can
because it cause very many other problems ...
also try to be locked as small time as possible because in case of conflict the conflicting threads are stopped !!!
therefore you cannot just add lock at the start of loop and unlock at the end
because the parallelism speedup will be lost then
Declare a queue to collect results and submit tasks to a thread pool:
final ArrayBloclingQueue<PlayerPayoffs> queue=new ArrayBloclingQueue<PlayerPayoffs>();
Executor exec=new Executors.newFixedThreadPool(N); // number of threads depends on hardware
for (int i = 0; i < GameConstants.MAX_GAMES; i++) {
exec.execute(new Runnable(){
EraserSimulator eraser = new EraserSimulator(GameConstants.MAX_TARGETS, GameConstants.MAX_RESOURCES, GameConstants.NUM_ATTACKER_TYPES, seed.nextInt());
Map<Set<SingleObjectiveTarget>, Double> gameStrategy = eraser.run();
assert (gameStrategy != null);
TestingGameSimulator testingGame = new TestingGameSimulator(GameConstants.MAX_TARGETS, gameStrategy, GameConstants.NUM_GAMES_TO_STORE_FOR_HISTORY, GameConstants.NUM_TESTING_GAMES_TO_PLAY);
PlayerPayoffs payoffs = testingGame.run(eraser.getEraserInstance());
queue.put(payoffs);
});
}
Then collect and sum results:
double testerPayoffSum = 0.0, developerPayoffSum = 0.0;
for (int i = 0; i < GameConstants.MAX_GAMES; i++) {
PlayerPayoffs payoffs=queue.take();
testerPayoffSum += payoffs.getAverageTesterPayoff(GameConstants.NUM_TESTING_GAMES_TO_PLAY);
developerPayoffSum += payoffs.getAverageDeveloperPayoff(GameConstants.NUM_TESTING_GAMES_TO_PLAY);
System.out.print("Output: ERASER Games played; Number of developers caught");
System.out.print(", " + GameConstants.NUM_TESTING_GAMES_TO_PLAY + ", " + payoffs.getNumTimesCaught() + "\n");
}

Categories

Resources