I need to simulate in Java N sensors that send to me an obsvertation at random time. Where an observation contain more value like:
timestamp - temperature - umiditiy - ...
When i receive an observation from anyone (the time of observation is random for all sensor) of the N sensors i need to call a rutine R that refresh a data structure (in common between all sensors) with some counting.
I need to syncronyze the R call, if i call R first time for call R second time i need R first time to finish his work.
All my observation are actually stored in a CSV file, one file for sensor. But i need to simulate a "online streaming".
How i can make that in Java? If i make N threads (one for sensor) that read his CSV file, i can't read the observation in right temporal order over all CSV.
For example, if i have this 2 csv:
Csv1:
18:00 - low - low
19:00 - low - high
Csv2
18:03 - high - low
I need first to refresh my counting with obsvervation at time 18:00 in csv1, then with obsvervation at time 18:0 in csv2 and finally with observation at time 19:00 in csv2.
EDIT1: I have make a test with SynchronousQueue because I need to syncronyze my N thread, my problem is when one thread do something on counting structure other thread can't access to it.
I have find this example:
package threadTest;
import java.util.concurrent.SynchronousQueue;
public class SynchronousQueueTest
{
private SynchronousQueue sq = new SynchronousQueue(true);
class PutThread implements Runnable
{
public void run()
{
for(int i=0; i <1000; i++)
{
try {
System.out.println("PUT");
//sq.put("A");
sq.put("A");
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
class TakeThread implements Runnable
{
public void run()
{
for(int i=0; i <1000; i++)
{
try {
System.out.println("TAKE");
System.out.println(sq.take());
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
public static void main(String[] args)
{
new Thread((new SynchronousQueueTest()).new PutThread()).start();
new Thread((new SynchronousQueueTest()).new TakeThread()).start();
}
}
But i have not the expected result. The output is only this:
PUT
TAKE
But i'm expected 1000 PUT and 1000 TAKE alternate. What is the problem?
Ha. That sounds so much like what I do usually when preparing stock tick data for back test.
Normally you massage the CSV files to be at least internally time ordered. Then you can either read them and combine at beginning - or if you have multiple readers you need to put the data into a PriorityQueue based structure, say, DelayQueue. Have your simulation data structure wrap the actual data and implement Delayed interface so you know how much delay you need. This is all you need from reading side.
From publishing side, use just 1 thread and publish to the time you needed - then schedule the next round with delay like 1ms or so. That's usually enough.
From subscriber side, the subscriber is blind and just take the data as if it's from real.
BTW, if you are really doing back testing or so, you might want to write up your own time service class which can simulate a faster tick of the data.
Related
Currently working on a university assessment, so I won't share specifics and I'm not asking for any explanation that will help me solve the main problem. I've already solved the problem, but my solution might be considered a little messy.
Basically, we're working with concurrency and semaphores. There is some shared resource that up to X (where X > 1) number of threads can access at a time and an algorithm which makes it a little more complicated than just acquiring and releasing access. Threads come at a certain time, use the resource for a certain time and then leave. We are to assume that no time is wasted when arriving, accessing, releasing and leaving the resource. The goal is to demonstrate that the algorithm we have written works by outputting the times a thread arrives, accesses the resource and leaves for each thread.
I'm using a semaphore with X number of permits to govern access. And it's all working fine, but I think the way I arrive at the expected output might be a bit janky. Here's something like what I have currently:
#Override
public void run() {
long alive = System.currentTimeMillis();
try { Thread.sleep(arrivalTime * 1000); }
catch (InterruptedException e) {} // no interrupts implemented
long actualArriveTime = System.currentTimeMillis() - alive;
boolean accessed = false;
while (!accessed) accessed = tryAcquire();
long actualAccessTime = System.currentTimeMillis() - alive;
try { Thread.sleep(useTime * 1000); }
catch (InterruptedException e) {} // no interrupts implemented
release();
long actualDepartTime = System.currentTimeMillis() - alive;
System.out.println(actualArriveTime);
System.out.println(actualAccessTime);
System.out.println(actualDepartTime);
}
I do it this way because where the expected output might be:
Thread Arrival Access Departure
A 0 0 3
B 0 0 5
C 2 2 6
... ... ... ...
My output looks something like:
Thread Arrival Access Departure
A 0 0 3006
B 0 0 5008
C 2 2 6012
... ... ... ...
I'm essentially making the time period much larger so that if the computer takes a fews milliseconds to acquire(), for example, it doesn't affect the number much. Then I can round to the nearest second to get the expected output. My algorithm works, but there are issues with this. A: It's slow; B: With enough threads, the milliseconds of delay may build so that I round to the wrong number.
I need something more like this:
public static void main(String[] args) {
int clock = 0;
while (threadsWaiting) {
clock++;
}
}
#Override
public void run() {
Thread.waitUntil(clock == arrivalTime);
boolean accessed = false;
while (!accessed) accessed = tryAcquire();
int accessTime = clock;
int depatureTime = accessTime + useTime;
Thread.waitUntil(clock == departureTime);
release();
System.out.println(arrivalTime);
System.out.println(accessTime);
System.out.println(departureTime);
}
Hopefully that's clear. Any help is appreciated.
Thanks!
I have some exercises, and one of them refers to concurrency. This theme is new for me, however I spent 6 hours and finally solve my problem. But my knowledge of corresponding API is poor, so I need advice: is my solution correct or may be there is more appropriate way.
So, I have to implement next interface:
public interface PerformanceTester {
/**
* Runs a performance test of the given task.
* #param task which task to do performance tests on
* #param executionCount how many times the task should be executed in total
* #param threadPoolSize how many threads to use
*/
public PerformanceTestResult runPerformanceTest(
Runnable task,
int executionCount,
int threadPoolSize) throws InterruptedException;
}
where PerformanceTestResult contains total time (how long the whole performance test took in total), minimum time (how long the shortest single execution took) and maximum time (how long the longest single execution took).
So, I learned many new things today - about thread pools, types Executors, ExecutorService, Future, CompletionService etc.
If I had Callable task, I could make next:
Return current time in the end of call() procedure.
Create some data structure (some Map may be) to store start time and Future object, that retuned by fixedThreadPool.submit(task) (do this executionCount times, in loop);
After execution I could just subtract start time from end time for every Future.
(Is this right way in case of Callable task?)
But! I have only Runnable task, so I continued looking. I even create FutureListener implements Callable<Long>, that have to return time, when Future.isDone(), but is seams little crazy for my (I have to double threads count).
So, eventually I noticed CompletionService type with interesting method take(), that Retrieves and removes the Future representing the next completed task, waiting if none are yet present., and very nice example of using ExecutorCompletionService. And there is my solution.
public class PerformanceTesterImpl implements PerformanceTester {
#Override
public PerformanceTestResult runPerformanceTest(Runnable task,
int executionCount, int threadPoolSize) throws InterruptedException {
long totalTime = 0;
long[] times = new long[executionCount];
ExecutorService pool = Executors.newFixedThreadPool(threadPoolSize);
//create list of executionCount tasks
ArrayList<Runnable> solvers = new ArrayList<Runnable>();
for (int i = 0; i < executionCount; i++) {
solvers.add(task);
}
CompletionService<Long> ecs = new ExecutorCompletionService<Long>(pool);
//submit tasks and save time of execution start
for (Runnable s : solvers)
ecs.submit(s, System.currentTimeMillis());
//take Futures one by one in order of completing
for (int i = 0; i < executionCount; ++i) {
long r = 0;
try {
//this is saved time of execution start
r = ecs.take().get();
} catch (ExecutionException e) {
e.printStackTrace();
return null;
}
//put into array difference between current time and start time
times[i] = System.currentTimeMillis() - r;
//calculate sum in array
totalTime += times[i];
}
pool.shutdown();
//sort array to define min and max
Arrays.sort(times);
PerformanceTestResult performanceTestResult = new PerformanceTestResult(
totalTime, times[0], times[executionCount - 1]);
return performanceTestResult;
}
}
So, what can you say? Thanks for replies.
I would use System.nanoTime() for higher resolution timings. You might want to ignroe the first 10,000 tests to ensure the JVM has warmed up.
I wouldn't bother creating a List of Runnable and add this to the Executor. I would instead just add them to the executor.
Using Runnable is not a problem as you get a Future<?> back.
Note: Timing how long the task spends in the queue can make a big difference to the timing. Instead of taking the time from when the task was created you can have the task time itself and return a Long for the time in nano-seconds. How the timing is done should reflect the use case you have in mind.
A simple way to convert a Runnable task into one which times itself.
finla Runnable run = ...
ecs.submit(new Callable<Long>() {
public Long call() {
long start = System.nanoTime();
run.run();
return System.nanoTime() - start;
}
});
There are many intricacies when writing performance tests in the JVM. You probably aren't worried about them as this is an exercise, but if you are this question might have more information:
How do I write a correct micro-benchmark in Java?
That said, there don't seem to be any glaring bugs in your code. You might want to ask this on the lower traffic code-review site if you want a full review of your code:
http://codereview.stackexchange.com
I have been trying to generate a tone (444 hz, 1000 hz, etc) and then play it in Android. My first searches gave me this stack overflow question. While this works great given a duration, I would prefer to make the duration infinite (loop-able).
First I used only use integer values for the tone, because this would mean that I could use 1 second and it should loop properly. However there are still some frequencies that don't loop right.
Second I thought I could only calculate 1 period of the sine wave, and then loop that. However I found out that is not a viable approach
How can I, given any frequency, generate a tone that I can loop?
You can create a thread that will run your tone every period of time(for example 2sec.):
private Runnable startSoundRunnable = new Runnable() {
#Override
public void run() {
while (true) {
try {
toneGenerator.startTone(ToneGenerator.TONE_CDMA_ALERT_AUTOREDIAL_LITE, 2000);
sleep(2000);
} catch (Exception ex) {}
}
}
};
// Run thread
new Thread(startSoundRunnable).start();
I have asked this question some time ago to get an idea of speeding up image writing to file.I basically have an OpenGL app.At the end of each render loop I save the frame buffer into image file.
fbo.readFrame();
glReadPixels(0, 0, _viewportWidth, _viewportHeight, GL_RGBA, GL_UNSIGNED_BYTE, _data);
_data.rewind(); //ByteBuffer
new TImageExporter(ImageExporter.TGA, "renderings/", _data, _viewportWidth, _viewportHeight, true, _frameCount++).run();
The TImageExporter extends Thread and inside the "run()" method I execute writing to File.
To my surprise The render loop for 50 frames takes almost the same time as if I use a single thread version of the Image Exporter. (3293 milliseconds -multi-threaded and 3437 milliseconds using single threaded).What do I do wrong here?
That is the code inside TImageExporter:
public void export() {
_pixels = new int[_width * _height];
int bindex;
int plenght = _pixels.length;
// convert RGB data in ByteBuffer to integer array
for (int i = 0; i < plenght; ++i) {
bindex = i * 4; //use 3 for RGB
//// here write the pixels to RGBA/////
............
.......................
}
_image.flush();
_image.setRGB(0, 0, _width, _height, _pixels, 0, _width);
_image = ImageUtils.flipY(_image);
renderByFormatType();
}
private void renderByFormatType() {
String formattedString = String.format(_formatString, _frameCount);
if (_exportType.equals(TGA)) {
try {
writeTGA(_image, new File(_renderPath + "screenshot_test" + formattedString + ".tga"), _flipX);
} catch (IOException ex) {
Logger.getLogger(TImageExporter.class.getName()).log(Level.SEVERE, null, ex);
}
}
}
#Override
public void run() {
export();
}
UPDATE:
People have asked here if I write to the same file.No, each thread writes to a completely new file.
UPDATE1:
Set a global static variable that holds array of BufferedImage.Now each new TImageExporter
writes the image data to that array to a different index.All I got is 3038 milliseconds instead of 3437 when writing directly to Disk.
Multi-threading will not speed up file transfer because:
making a non-thread safe file write will overwrite some of the information the other threads are writing when 1 thread goes to save
it.
Disk I/O is your bottle neck. The best thing to do it write what you want to into a memorystream (someone edit and give documentation i
can't find it real quick) and allow it to write to the disk.. this
will allow constant disk i/o without worrying about threads
Even though your code is multi threaded ; since all you Threads are trying to access the same file..it wont make much difference..because in this scenario even though lot of threads are running; at one instant all except 1 will be waiting to acquire write access to file.
Why do you think it will be faster? The disk isn't multithreaded.
In my current project, I am going to write a function of calculating average Temperature.
A notatable point of this function is that it receives data from many temperature sources at different time. Could you advise me "How can I write a function with such a behavior?"
For instance, following function receives tempSensorData from many temperature Sensors, which sends temperature data at different time. I need to write a logic of calculating average temperature.
public void calculateRoomAvgTemp(TempStruct tempSensorData) {
// Write logic of calculating Average Temperature.
}
Why don't you just keep a running average? Assuming that your method gets called each time a temperature reading is taken, you can do the following:
private final List<Double> temps = Collections
.synchronizedList(new ArrayList<Double>());
private double currentAverage = 0d;
public void calculateRoomAvgTemp(TempStruct tempSensorData) {
synchronized (this.temps) {
this.temps.add(tempSensorData.temp);
this.currentAverage = 0d;
for (Double temp : this.temps) {
this.currentAverage += temp;
}
this.currentAverage /= this.temps.size();
}
}
can you store past temperatures in a separate field in the class? I noticed that your method is void, therefore doesn't return anything. I'm going to assume you want to store the average in the separate field. Example:
public class TempCalculator {
private double totalTemp = 0;
private int sensors = 0;
private double averageTemp = 0;
public void calculateRoomAvgTemp(TempStruct tempSensorData) {
totalTemp += tempSensorData.getTemp(); // Assuming getTemp() exists
sensors += 1;
averageTemp = totalTemp / sensors;
}
public double getAvgTemp() {
return averageTemp;
}
}
Of course, this can be done with a list if you want to store temperatures used and shaped to your needs. Hope this helps.
one way I could think of - insert values as you receive them into an ArrayList. If you need to calculate the average temperature after each data point that you receive, run through a for loop and average out the data points in the ArrayList to get the required value. One could look into using memoization to avoid running through the loop for all values.
Another option would be to use apache commons math whereby you can use the descriptive statistics option (using a rolling array) to get the mean (average).
I am assuming your code is single threaded. If multithreaded, you could use a thread safe vector (or add locks/synchronization on your own).
Hope this helps.
To calculate the average without keeping all historic values in memory (or a database), use a "moving average". This mathematical tool can give you the new average from the last one plus the current value.
You can create one thread which will calculate the temperature and many which will measure it values from different places. When some measurement is done the thread pushes the result into e.g. LinkedBlockingQueue. It's important to synchronize this operation because many threads might try to push results at the same time. The thread which calculate average temperature would run in a loop, pop new result from the queue, recalculate the temperature and then once again try to pop another result. As far as the queue will be empty the thread would be blocked on reading from it. In this way you would have an asynchronous communication between threads and the average temperature would be recalculated immediately after the measurements are done.