Related
This is a very long post, but only the first half is really relevant.
The second half describes only what I have tried to solve it, but what seemed to me too inefficient (it can perhaps help to get the idea of what I want). The relevant part ends with the line after the bolded question.
I am trying to simulate multiple productions in an imaginary factory to calculate what amount of goods of each type known will be available at the end. There are several different goods types, and they have all a specific maximum production capacity that can only be reached if enough ingredients are available. An example of how the production lines could look is here:
The goods at the bottom all have a known rate at which they are delivered to the factory, so for those, there is nothing to calculate, though this rate can change over time (also, the maximum production capacity can also change at any point in time, e.g., the capacity can be increased by adding workers or more machines).
As shown in the picture, for the other goods there are three things to look at:
Some lines produce a good out of a single other one.
Some lines produce a good out of two others.
Some lines have a good used for creation of more than one new good
(see, for example, "R8" in the middle of the illustration).
I have the following information:
Maximum production rate of each good (5 produced per hour, for example)
for the bottom goods we have the amount delivered to the factory (5 delivered per hour, for example)
how much of each is in stock now (so in case there is not enough delivered, if we still have some in stock, we don't need to reduce production)
At what times the delivery of a good will change (can happen to any good at the bottom)
At what times the maximum production rate of a good will change (can happen to any good not at the bottom)
With this information, I want to calculate the amount of each good at a given time in the future. I would like this to be as efficient as possible, since I need these calculations quite often.
I tried to make an implementation for this in Java, but I have some problems. Instead of fixing them I looked at my algorithm again and figured out it did not look as if it was very efficient anyway, so I wanted to know if someone has already seen or solved this kind of problem?
The way I tried to solve this is following:
I create maximum production (and delivery) intervals for each good using the known information when a production (or delivery) amount changes.
I put all resources at the bottom in a remaining Set and a checked Set (bottom goods are immediately checked ones).
I calculate the actual amount of goods produced for each good: therefore, I take each good in remaining and I check what goods can be produced, if all that can be produced only are made of checked goods I calculate the actual amount produced depending of the maximum rate and the available goods (depending on the production of the things it is made of and the amount in stock if this is less). Additionally, in this step I add production intervals if due to lesser production of a source good (but some in stock at the beginning) the production needs to be reduced. When finished the goods the new ones are made of get removed from the remaining Set and the new ones are added, as well as being added to the checked Set.
Now we have all the actual good productions for each good and we can calculate it. For this we loop over each good and take the actual production and add it up using the interval borders for time. We have now the amount of goods at the wanted time in the future.
Additional info: we cannot do the point 4. without 3. since the actual amount we calculate for a good, can be consumed again for the production of the next one, so we need need this Step in between.
If it helps to understand what I have done (or wanted to do) I add my code (not working). The class is already initialized with the maximum production rate intervals of each produced good currently in production. Since other goods can be in stock, for all goods that are not included we initialize them to with a production of zero and one interval.
public class FactoryGoods {
private long future;
private long now;
private Map<String, Integer> availableGoods;
private Map<String, ArrayList<ProductionInterval>> hourlyGoodIncrease;
/**
*
* #param future long current time
* #param now long last time the factory's resources got updates
* #param availableGoods Map<String,Integer> of the goods in the factory
* #param hourlyGoodIncrease Map<String,ArrayList<ProductionInterval>> of the intervals of production quantities for the goods
*/
public factoryGoods(long future, long now, Map<String,Integer> availableGoods, Map<String,ArrayList<ProductionInterval>> hourlyGoodIncrease) {
this.future = future;
this.now = now;
this.availableGoods = availableGoods;
this.hourlyGoodIncrease = hourlyGoodIncrease;
}
/**
* Calculates the resources present in a factory's storage
* #return a Map of quantities mapped on the String name of the good
*/
public Map<String,Integer> getResources() {
// Make sure all goods to have all goods inside the loop, to work on goods,
// that are produced, but also those which are only in store
HashMap<String, Boolean> goodChecked = new HashMap<String,Boolean>();
Set<String> remaining = new HashSet<String>();
for (Goods good: Goods.values()) {
String g = good.get();
if (hourlyGoodIncrease.get(g) == null) {
ArrayList<ProductionInterval> prods = new ArrayList<ProductionInterval>();
ProductionInterval start = new ProductionInterval(now, 0);
prods.add(start);
hourlyGoodIncrease.put(g, prods);
}
if (availableGoods.get(g) == null) {
availableGoods.put(g, 0);
}
if (good.isPrimary()) {
goodChecked.put(g, true);
} else {
goodChecked.put(g, false);
}
remaining.add(g);
}
// As long as goods are remaining to be checked loops over the goods, and
// recalculates hourly good increases for goods, that have all its sources
// already calculated
while (remaining.size() > 0) {
Set<String> removes = new HashSet<String>();
for (String good: remaining) {
if (goodChecked.get(good)) {
Good g = GoodFactory.get(good);
Set<String> to = new HashSet<String>();
Map<String,Float> from = new HashMap<String,Float>();
setUpFromAndToGoods(g, to, from, availableGoods);
if (areGoodsAlreadyCalculated(to, goodChecked)) {
//remaining.remove(good);
removes.add(good);
} else {
if (areGoodsReadyForCalculation(to, goodChecked)) {
// Get all resources we are working on now
Map<String,Float> fromDecrease = new HashMap<String,Float>();
for (String t: to) {
for (String f: GoodFactory.get(t).isMadeFrom().keySet()) {
from.put(f, (float) availableGoods.get(f));
}
}
// Get all interval borders
ArrayList<Long> intervalBorders = new ArrayList<Long>();
for (String wGood: from.keySet()) {
ArrayList<ProductionInterval> intervals = hourlyGoodIncrease.get(wGood);
for (ProductionInterval interval: intervals) {
long intervalStart = interval.getStartTime();
if (!intervalBorders.contains(intervalStart)) {
intervalBorders.add(intervalStart);
}
}
}
Collections.sort(intervalBorders);
intervalBorders.add(future);
for (String f: from.keySet()) {
hourlyGoodIncrease.put(f, createNewProductionIntervalls(intervalBorders, hourlyGoodIncrease.get(f)));
}
// For all intervals
int iLast = intervalBorders.size() - 1;
for (int i = 0; i < iLast; i++) {
long elapsedTime = intervalBorders.get(i + 1) - intervalBorders.get(i);
for (String t: to) {
Map<String, Float> source = GoodFactory.get(t).isMadeFrom();
for (String s: source.keySet()) {
Float decrease = fromDecrease.get(s);
fromDecrease.put(s, (decrease != null ? decrease : 0) + source.get(s));
}
}
// Calculate amount after normal maximum production
Set<String> negatives = new HashSet<String>();
Map<String,Float> nextFrom = new HashMap<String,Float>();
for (String f: from.keySet()) {
float delta = from.get(f) + (hourlyGoodIncrease.get(f).get(i).getHourlyIncrease() - fromDecrease.get(f)) * elapsedTime / (1000 * 60 * 60);
nextFrom.put(f, delta);
if (delta < 0) {
negatives.add(f);
}
}
// Check if got under zero
if (negatives.size() == 0) {
for (String f: from.keySet()) {
float newIncrease = hourlyGoodIncrease.get(f).get(i).getHourlyIncrease() - fromDecrease.get(f);
hourlyGoodIncrease.get(f).get(i).setHourlyIncrease(newIncrease);
from.put(f, nextFrom.get(f));
}
} else {
// TODO: handle case when more is used than exists
}
// Else calculate point where at least one from is zero and add an interval
// before its maximum, after needs to be adjusted
}
// Break to remove all calculated goods from the remaining set and rerun the loop
removes = to;
break;
}
}
}
}
for (String remove: removes) {
remaining.remove(remove);
}
}
// Final calculation of the goods amounts that are available in the factory
for (String good: goodChecked.keySet()) {
ArrayList<ProductionInterval> intervals = hourlyGoodIncrease.get(good);
intervals.add(new ProductionInterval(future, 0));
float after = availableGoods.get(good);
for (int i = 0; i < (intervals.size() - 1); i++) {
after += intervals.get(i).getHourlyIncrease() * (intervals.get(i + 1).getStartTime() - intervals.get(i).getStartTime()) / (1000 * 60 * 60);
}
availableGoods.put(good, (int) after);
}
return availableGoods;
}
private static ArrayList<ProductionInterval> createNewProductionIntervalls(ArrayList<Long> intervalBorders, ArrayList<ProductionInterval> hourlyIncreases) {
System.out.print("intervalBorders\n");
System.out.print(intervalBorders + "\n");
System.out.print("hourlyIncreases\n");
System.out.print(hourlyIncreases + "\n");
ArrayList<ProductionInterval> intervalls = new ArrayList<ProductionInterval>();
int i = 0;
long iTime = 0;
long nextTime = 0;
for (long l: intervalBorders) {
float increase = 0;
iTime = hourlyIncreases.get(i).getStartTime();
if (i + 1 < hourlyIncreases.size()) {
nextTime = hourlyIncreases.get(i + 1).getStartTime();
}
if (l == iTime) {
increase = hourlyIncreases.get(i).getHourlyIncrease();
} else if (iTime < l && l < nextTime) {
increase = hourlyIncreases.get(i).getHourlyIncrease();
} else if (l == nextTime) {
increase = hourlyIncreases.get(++i).getHourlyIncrease();
}
intervalls.add(new ProductionInterval(l, increase));
}
return intervalls;
}
private static void setUpFromAndToGoods(Good g, Set<String> to, Map<String,Float> from, Map<String,Integer> availableGoods) {
Set<String> unchecked = g.isUsedToCreate();
while (unchecked.size() > 0) {
String good = unchecked.iterator().next();
unchecked.remove(good);
to.add(good);
Set<String> madeFrom = GoodFactory.get(good).isMadeFrom().keySet();
for (String fromGood: madeFrom) {
if (!from.containsKey(fromGood)) {
from.put(fromGood, (float) availableGoods.get(fromGood));
Set<String> additions = GoodFactory.get(fromGood).isUsedToCreate();
for (String addition: additions) {
if (!to.contains(addition) && !unchecked.contains(addition)) {
unchecked.add(addition);
}
}
}
}
}
}
private static boolean areGoodsReadyForCalculation(Set<String> toGoods, Map<String,Boolean> goodChecked) {
for (String t: toGoods) {
Good toGood = GoodFactory.get(t);
for (String from: toGood.isMadeFrom().keySet()) {
if (!goodChecked.get(from)) {
return false;
}
}
}
return true;
}
private static boolean areGoodsAlreadyCalculated(Set<String> toGoods, Map<String,Boolean> goodChecked) {
for (String t: toGoods) {
if (!goodChecked.get(t)) {
return false;
}
}
return true;
}
}
public class ProductionInterval {
private long startTime;
private float hourlyIncrease;
public ProductionInterval(long startTime, float hourlyIncrease) {
this.setStartTime(startTime);
this.setHourlyIncrease(hourlyIncrease);
}
public float getHourlyIncrease() {
return hourlyIncrease;
}
public void setHourlyIncrease(float hourlyIncrease) {
this.hourlyIncrease = hourlyIncrease;
}
public long getStartTime() {
return startTime;
}
public void setStartTime(long startTime) {
this.startTime = startTime;
}
public String toString() {
return "<starttime=" + this.startTime + ", hourlyIncrease=" + this.hourlyIncrease + ">";
}
}
Does someone know an algorithm that can solve my problem, or have some ideas how I can change my algorithm so that it gets more efficient? (I know it does not work at all, but with all these loops, I don't think it will be efficient and I would like to know if someone sees something I could make better before I put the work into finishing it).
You can apply a max flow algorithm like the Edmonds-Karp with few modifications, and you need to build the graph to feed to the algo:
Create a node for each good
You need one "source" node and one "sink" node
For each delivered good, create an arc from the source to respective node, with the capacity equal to delivery rate
For each final product, create an arc from its respective node to the sink, with capacity equal to production rate
For each dependency between goods, create an arc between respective nodes with capacity of one.
For each good, create an arc from source to the respective node with capacity equal to amount of the good in stock (for first iteration it's zero)
The results will be the flows from final goods nodes to the sink after the algorithm is finished. For your case, you need two modifications:
When calculating flow at a node, you take the minimum of the flows to it (since you require all dependencies to create a good), and then cap it at this good's maximum production rate for non-delivered goods
You need to account for change of goods in stock - will edit the answer later
Although, this algorithm is offline, which means it's not suited for flows changing over time, it's relatively simple, and if you're not too constrained by performance requirements, it may work - just run the algo again after adjusting the capacities. For online max flow, you can look at this,
Working out my idea of fractional simulation in C++ (sorry). Please see heavily commented code below.
(I know the prioritization in the face of constrained resources isn't what you want. It's not trivial to get a fair implementation of Derivative that produces as much as it can, so I wanted to validate this approach before going down the rabbit hole.)
#include <cassert>
#include <iostream>
#include <limits>
#include <utility>
#include <vector>
// Identifies a type of good in some Factory.
using Good = int;
// Simulates a factory. The simulation is crude, assuming continuous,
// fractional, zero-latency production. Nevertheless it may be accurate enough
// for high production volumes over long periods of time.
class Factory {
public:
// Declares a new raw material. `delivery_rate` is the amount of this good
// delivered per hour.
Good NewRawMaterial(double stock, double delivery_rate) {
assert(stock >= 0.0);
assert(delivery_rate >= 0.0);
return NewGood(stock, delivery_rate, {});
}
// Declares a new manufactured good. `max_production_rate` is the max amount
// of this good produced per hour. Producing one of this good consumes one
// `input`.
Good NewManufacturedGood(double stock, double max_production_rate,
Good input) {
assert(stock >= 0.0);
assert(max_production_rate >= 0.0);
return NewGood(stock, max_production_rate, {input});
}
// Declares a new manufactured good. `max_production_rate` is the max amount
// of this good produced per hour. Producing one of this good consumes one
// `input_a` and one `input_b`.
Good NewManufacturedGood(double stock, double max_production_rate,
Good input_a, Good input_b) {
assert(stock >= 0.0);
assert(max_production_rate >= 0.0);
return NewGood(stock, max_production_rate, {input_a, input_b});
}
// Returns the number of hours since the start of the simulation.
double Now() const { return current_time_; }
// Advances the current time to `time` hours since the start of the
// simulation.
void AdvanceTo(double time);
// Returns the amount of `good` in stock as of the current time.
double Stock(Good good) const { return stock_[good]; }
// Sets the delivery rate of `good` to `delivery_rate` as of the current time.
void SetDeliveryRate(Good good, double delivery_rate) {
assert(delivery_rate >= 0.0);
max_production_rate_[good] = delivery_rate;
}
// Sets the max production rate of `good` to `max_production_rate` as of the
// current time.
void SetMaxProductionRate(Good good, double max_production_rate) {
assert(max_production_rate >= 0.0);
max_production_rate_[good] = max_production_rate;
}
private:
// Floating-point tolerance.
static constexpr double kEpsilon = 1e-06;
// Declares a new good. We handle raw materials as goods with no inputs.
Good NewGood(double stock, double max_production_rate,
std::vector<Good> inputs) {
assert(stock >= 0.0);
assert(max_production_rate >= 0.0);
Good good = stock_.size();
stock_.push_back(stock);
max_production_rate_.push_back(max_production_rate);
inputs_.push_back(std::move(inputs));
return good;
}
// Returns the right-hand derivative of stock.
std::vector<double> Derivative() const;
// Returns the next time at which a good is newly out of stock, or positive
// infinity if there is no such time.
double NextStockOutTime(const std::vector<double> &derivative) const;
// The current time, in hours since the start of the simulation.
double current_time_ = 0.0;
// `stock_[good]` is the amount of `good` in stock at the current time.
std::vector<double> stock_;
// `max_production_rate_[good]` is the max production rate of `good` at the
// current time.
std::vector<double> max_production_rate_;
// `inputs_[good]` is the list of goods required to produce `good` (empty for
// raw materials).
std::vector<std::vector<Good>> inputs_;
// Derivative of `stock_`.
std::vector<double> stock_rate_;
};
void Factory::AdvanceTo(double time) {
assert(time >= current_time_);
bool caught_up = false;
while (!caught_up) {
auto derivative = Derivative();
double next_time = NextStockOutTime(derivative);
if (time <= next_time) {
next_time = time;
caught_up = true;
}
for (Good good = 0; good < stock_.size(); good++) {
stock_[good] += (next_time - current_time_) * derivative[good];
}
current_time_ = next_time;
}
}
std::vector<double> Factory::Derivative() const {
// TODO: this code prioritizes limited supply by the order in which production
// is declared. You probably want to use linear programming or something.
std::vector<double> derivative = max_production_rate_;
for (Good good = 0; good < stock_.size(); good++) {
for (Good input : inputs_[good]) {
if (stock_[input] <= kEpsilon) {
derivative[good] = std::min(derivative[good], derivative[input]);
}
}
for (Good input : inputs_[good]) {
derivative[input] -= derivative[good];
}
}
return derivative;
}
double Factory::NextStockOutTime(const std::vector<double> &derivative) const {
double duration = std::numeric_limits<double>::infinity();
for (Good good = 0; good < stock_.size(); good++) {
if (stock_[good] > kEpsilon && derivative[good] < -kEpsilon) {
duration = std::min(duration, stock_[good] / -derivative[good]);
}
}
return current_time_ + duration;
}
int main() {
Factory factory;
Good r1 = factory.NewRawMaterial(60.0, 3.0);
Good r2 = factory.NewRawMaterial(20.0, 1.0);
Good r3 = factory.NewManufacturedGood(0.0, 2.0, r1);
Good r4 = factory.NewManufacturedGood(0.0, 1.0, r1, r2);
auto print_stocks = [&]() {
std::cout << "t : " << factory.Now() << "\n";
std::cout << "r1: " << factory.Stock(r1) << "\n";
std::cout << "r2: " << factory.Stock(r2) << "\n";
std::cout << "r3: " << factory.Stock(r3) << "\n";
std::cout << "r4: " << factory.Stock(r4) << "\n";
std::cout << "\n";
};
print_stocks();
// Everything running smoothly
factory.AdvanceTo(24.0);
print_stocks();
// Uh oh, r1 supply cut off. Stock out at 44 hours.
factory.SetDeliveryRate(r1, 0.0);
factory.AdvanceTo(48.0);
print_stocks();
// r1 supply at 50%. r3 production prioritized.
factory.SetDeliveryRate(r1, 1.5);
factory.AdvanceTo(72.0);
print_stocks();
// r1 oversupplied.
factory.SetDeliveryRate(r1, 4.0);
factory.AdvanceTo(96.0);
print_stocks();
}
Output:
t : 0
r1: 60
r2: 20
r3: 0
r4: 0
t : 24
r1: 60
r2: 20
r3: 48
r4: 24
t : 48
r1: 0
r2: 24
r3: 88
r4: 44
t : 72
r1: 0
r2: 48
r3: 124
r4: 44
t : 96
r1: 24
r2: 48
r3: 172
r4: 68
I have client-server application in which i need to measure the rate of request arrival per second(Request rate). For this, i have a timer object that activates after every seconds, reads a synchronized counter and then sets it to zero. The counter increments on each request arrival.I used following code to detect request rate. There are so many other threads and timers in my application running.The problem is "due to the inaccuracy of timers i am not getting the perfect request rate". Is there any alternative of measuring request rate other than using timers.
public class FrequencyDetector extends TimerTask {
RequestCounter requestCounter;
FrequencyHolder frequencyHolder;
public FrequencyDetector(RequestCounter requestCounter,FrequencyHolder frequencyHolder){
this.frequencyHolder=new FrequencyHolder();
this.frequencyHolder=frequencyHolder;
}
#Override
public void run() {
int newFrequency=requestCounter.getCounter();
frequencyHolder.setFrequency(newFrequency);
requestCounter.setCounterToZero();
//calls to other fuctions
}
}
Instead of checking counter per unit time you can check time per unit counter. That will probably give you more accurate results. Algorithm is given below.
Increment counter on every request.
When counter reaches a certain FIXED_LIMIT calculate frequency by frequency=FIXED_LIMIT/duration since last record
Reset the counter and start with step 1
However this will record frequency at unpredictable intervals and if frequency of request decreases the duration between successive records will increase.
To handle it we can implement an adaptive algorithm, algorithm is given below.
Increment the counter on every request.
When counter reaches a certain ADAPTIVE_LIMIT record frequency as frequency=ADAPTIVE_LIMIT/duration since last record
Change ADAPTIVE_LIMIT as ADAPTIVE_LIMIT=frequency * DESIRED RECORD INTERVAL
Reset counter and start with step 1.
Above algorithm will reset the limit based on frequency last recorded. It's given that it will not be recording at optimal intervals but it will be pretty close.
Also it will give you highly accurate frequencies as it does not depend on any scheduled thread.
Following is an implementation of such an adaptive counter.
import java.util.Random;
import java.util.concurrent.atomic.AtomicLong;
public class TestCounter {
//Keep initial counterInterval to a small value otherwise first record may take long time
final AtomicLong counterInterval = new AtomicLong(10);
AtomicLong requestCounter = new AtomicLong();
volatile long lastTime;
/**OPTIMAL_DURATION is the duration after which frequency is expected to be recorded
* Program adaptively tries to reach this duration
*/
static final double OPTIMAL_DURATION = 1.0; // 1 second
static final Random random = new Random();
public static void main(String[] args) {
System.out.println("Started ");
TestCounter main = new TestCounter();
for(int i = 0; i < 1000; i++) {
main.requestArrived();
}
}
/*
* Simulating requests
*/
public void requestArrived() {
printCounter();
try {
Thread.sleep(random.nextInt(100));
} catch (InterruptedException e) {
e.printStackTrace();
}
}
//This will be in some Utility class
private void printCounter() {
requestCounter.incrementAndGet();
long currentTime = System.nanoTime();
long currentInterval = counterInterval.get();
if(requestCounter.get() > currentInterval) {
if(lastTime != 0) {
long timeDelta = currentTime - lastTime;
long frequency = (long)(currentInterval / (timeDelta / 1e9));
System.out.printf("time=%.2f, frequency=%d\n", (timeDelta / 1e9), frequency);
//updating the currentInterval for the miss
long newCounterInterval = (long)(frequency * OPTIMAL_DURATION);
counterInterval.set(newCounterInterval);
}
requestCounter.set(0);
lastTime = currentTime;
}
}
}
Output
Started
time=0.54, frequency=18
time=0.98, frequency=18
time=1.01, frequency=17
time=0.96, frequency=17
time=0.99, frequency=17
time=0.85, frequency=19
time=0.96, frequency=19
time=0.82, frequency=23
time=1.08, frequency=21
time=0.98, frequency=21
time=0.94, frequency=22
time=1.06, frequency=20
time=1.07, frequency=18
time=0.99, frequency=18
time=0.98, frequency=18
time=1.02, frequency=17
time=0.92, frequency=18
time=0.92, frequency=19
time=0.89, frequency=21
time=0.82, frequency=25
time=1.31, frequency=19
time=1.02, frequency=18
I am working on a project in which I need to measure Total Time taken by program and average time taken by program. And that program is a Multithreaded program.
In that program, each thread is working in a particular range. Input parameters is Number of Threads and Number of Task.
If number of threads is 2 and number of tasks is 10 then each thread will be performing 10 tasks. So that means 2 thread will be doing 20 tasks.
So that means-
First thread should be using id between 1 and 10 and second thread should be using id between 11 and 20.
I got the above scenario working. Now I want to measure what is the total time and average time taken by all the threads. So I got the below setup in my program.
Problem Statement:-
Can anyone tell me the way I am trying to measure the Total time and Average time taken by all the threads is correct or not in my below program?
//create thread pool with given size
ExecutorService service = Executors.newFixedThreadPool(noOfThreads);
long startTime = 0L;
try {
readPropertyFiles();
startTime = System.nanoTime();
// queue some tasks
for (int i = 0, nextId = startRange; i < noOfThreads; i++, nextId += noOfTasks) {
service.submit(new XMPTask(nextId, noOfTasks, tableList));
}
service.shutdown();
service.awaitTermination(Long.MAX_VALUE, TimeUnit.DAYS);
} finally {
long estimatedTime = System.nanoTime() - startTime;
logTimingInfo(estimatedTime, noOfTasks, noOfThreads);
}
private static void logTimingInfo(long elapsedTime, int noOfTasks, int noOfThreads) {
long timeInMilliseconds = elapsedTime / 1000000L;
float avg = (float) (timeInMilliseconds) / noOfTasks * noOfThreads;
LOG.info(CNAME + "::" + "Total Time taken " + timeInMilliseconds + " ms. And Total Average Time taken " + avg + " ms");
}
service.submit is getting executed only noOfThreads times. XMPTask object is created the same number of times.
The time you measure is not the consumed time but the elapsed time.
If the program tested (the JVM) is the only one on the computer, it may be relatively accurate but in a real world a lot of process runs concurrently.
I have already done this job by using a native call to the OS, on Windows (I'll complete this post Monday at my office) and Linux (/proc).
I think you would need to measure the time within the task class itself (XMPTask). Within that task you should be able to extract the id of the thread that is executing it and log that. Using this approach will require reading the logs and doing some calculations on them.
Another approach would be to keep running totals and averages as time progresses. To do this you could write a simple class that is passed into each task that has some static (per jvm) variables for tracking what each thread is doing. Then you could have a single thread outside the Threadpool that did the calculations. So if you wanted to report the average cpu time for each thread every second, this calculation thread could sleep for a second, then calculate and log all the average times, then sleep for a second....
EDIT: After re-reading the requirements, you don't need a background thread, but not sure if we are tracking the average time per thread or average time per task. I have assumed total time and average time per thread and fleshed out the idea in code below. It has not been tested or debugged but should give you a good idea of how to start:
public class Runner
{
public void startRunning()
{
// Create your thread pool
ExecutorService service = Executors.newFixedThreadPool(noOfThreads);
readPropertyFiles();
MeasureTime measure = new MeasureTime();
// queue some tasks
for (int i = 0, nextId = startRange; i < noOfThreads; i++, nextId += noOfTasks)
{
service.submit(new XMPTask(nextId, noOfTasks, tableList, measure));
}
service.shutdown();
service.awaitTermination(Long.MAX_VALUE, TimeUnit.DAYS);
measure.printTotalsAndAverages();
}
}
public class MeasureTime
{
HashMap<Long, Long> threadIdToTotalCPUTimeNanos = new HashMap<Long, Long>();
HashMap<Long, Long> threadIdToStartTimeMillis = new HashMap<Long, Long>();
HashMap<Long, Long> threadIdToStartTimeNanos = new HashMap<Long, Long>();
private void addThread(Long threadId)
{
threadIdToTotalCPUTimeNanos.put(threadId, 0L);
threadIdToStartTimeMillis.put(threadId, 0L);
}
public void startTimeCount(Long threadId)
{
synchronized (threadIdToStartTimeNanos)
{
if (!threadIdToStartTimeNanos.containsKey(threadId))
{
addThread(threadId);
}
long nanos = System.nanoTime();
threadIdToStartTimeNanos.put(threadId, nanos);
}
}
public void endTimeCount(long threadId)
{
synchronized (threadIdToStartTimeNanos)
{
long endNanos = System.nanoTime();
long startNanos = threadIdToStartTimeNanos.get(threadId);
long nanos = threadIdToTotalCPUTimeNanos.get(threadId);
nanos = nanos + (endNanos - startNanos);
threadIdToTotalCPUTimeNanos.put(threadId, nanos);
}
}
public void printTotalsAndAverages()
{
long totalForAllThreadsNanos = 0L;
int numThreads = 0;
long totalWallTimeMillis = 0;
synchronized (threadIdToStartTimeNanos)
{
numThreads = threadIdToStartTimeMillis.size();
for (Long threadId: threadIdToStartTimeNanos.keySet())
{
totalWallTimeMillis += System.currentTimeMillis() - threadIdToStartTimeMillis.get(threadId);
long totalCPUTimeNanos = threadIdToTotalCPUTimeNanos.get(threadId);
totalForAllThreadsNanos += totalCPUTimeNanos;
}
}
long totalCPUMillis = (totalForAllThreadsNanos)/1000000;
System.out.println("Total milli-seconds for all threads: " + totalCPUMillis);
double averageMillis = totalCPUMillis/numThreads;
System.out.println("Average milli-seconds for all threads: " + averageMillis);
double averageCPUUtilisation = totalCPUMillis/totalWallTimeMillis;
System.out.println("Average CPU utilisation for all threads: " + averageCPUUtilisation);
}
}
public class XMPTask implements Callable<String>
{
private final MeasureTime measure;
public XMPTask(// your parameters first
MeasureTime measure)
{
// Save your things first
this.measure = measure;
}
#Override
public String call() throws Exception
{
measure.startTimeCount(Thread.currentThread().getId());
try
{
// do whatever work here that burns some CPU.
}
finally
{
measure.endTimeCount(Thread.currentThread().getId());
}
return "Your return thing";
}
}
After writing all this, there is one thing that seems a bit strange in that the XMPTask seems to know too much about the list of tasks, when, I think you should just create an XMPTask for every task that you have, give it enough information to do the job, and submit them to the service as you create them.
I want to know if I need to measure time elapsed then Single Threaded Program is good approach or Multithreading Program is a good approach for that.
Below is my single threaded program that is measuring the time of our service-
private static void serviceCall() {
histogram = new HashMap<Long, Long>();
keys = histogram.keySet();
long total = 5;
long runs = total;
while (runs > 0) {
long start_time = System.currentTimeMillis();
result = restTemplate.getForObject("SOME URL",String.class);
long difference = (System.currentTimeMillis() - start_time);
Long count = histogram.get(difference);
if (count != null) {
count++;
histogram.put(Long.valueOf(difference), count);
} else {
histogram.put(Long.valueOf(difference), Long.valueOf(1L));
}
runs--;
}
for (Long key : keys) {
Long value = histogram.get(key);
System.out.println("MEASUREMENT " + key + ":" + value);
}
}
Output I get from this Single Threaded Program is- Total call was 5
MEASUREMENT 163:1
MEASUREMENT 42:3
MEASUREMENT 47:1
which means 1 call came back in 163 ms. 3 calls came back in 42 ms and so on.
And also I did tried using Multithreaded program as well to measure the time elapsed. Meaning hitting the service parallely with few threads and then measuring how much each thread is taking.
Below is the code for that as well-
//create thread pool with given size
ExecutorService service = Executors.newFixedThreadPool(10);
// queue some tasks
for (int i = 0; i < 1 * 5; i++) {
service.submit(new ThreadTask(i, histogram));
}
public ThreadTask(int id, HashMap<Long, Long> histogram) {
this.id = id;
this.hg = histogram;
}
#Override
public void run() {
long start_time = System.currentTimeMillis();
result = restTemplate.getForObject("", String.class);
long difference = (System.currentTimeMillis() - start_time);
Long count = hg.get(difference);
if (count != null) {
count++;
hg.put(Long.valueOf(difference), count);
} else {
hg.put(Long.valueOf(difference), Long.valueOf(1L));
}
}
And below is the result I get from the above program-
{176=1, 213=1, 182=1, 136=1, 155=1}
One call came back in 176 ms, and so on
So my question is why Multithreading program is taking a lot more time as compared to above Single threaded program? If there is some loop hole in my Multithreading program, can anyone help me to improve it?
Your multi-threaded program likely makes all the requests at the same time which puts more strain on the server which will cause it to respond slower to all request.
As an aside, the way you are doing the update isn't threadsafe, so your count will likely be off in the multithreaded scenario given enough trials.
For instance, Thread A and B both return in 100 ms at the same time. The count in histogram for 100 is 3. A gets 3. B gets 3. A updates 3 to 4. B updates 3 to 4. A puts the value 4 in the histogram. B puts the value 4 in the histogram. You've now had 2 threads believe they incremented the count but the count in the histogram only reflects being incremented once.
How to compute accurately the time it takes a Java program to write or read a number of bytes from/to a file ?
It is really important that the time is being measured accurately. (The time should be computed by the program itself).
The standard idiom is:
long startTime = System.nanoTime();
doSomething();
long elapsedTime = System.nanoTime() - startTime;
not tested, but something like:
long delta = System.nanoTime();
try {
// do your stuff
} finally {
delta = System.nanoTime() - delta;
}
There is a code sample here:
http://www.goldb.org/stopwatchjava.html
/*
Copyright (c) 2005, Corey Goldberg
StopWatch.java is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
*/
public class StopWatch {
private long startTime = 0;
private long stopTime = 0;
private boolean running = false;
public void start() {
this.startTime = System.currentTimeMillis();
this.running = true;
}
public void stop() {
this.stopTime = System.currentTimeMillis();
this.running = false;
}
//elaspsed time in milliseconds
public long getElapsedTime() {
long elapsed;
if (running) {
elapsed = (System.currentTimeMillis() - startTime);
}
else {
elapsed = (stopTime - startTime);
}
return elapsed;
}
//elaspsed time in seconds
public long getElapsedTimeSecs() {
long elapsed;
if (running) {
elapsed = ((System.currentTimeMillis() - startTime) / 1000);
}
else {
elapsed = ((stopTime - startTime) / 1000);
}
return elapsed;
}
//sample usage
public static void main(String[] args) {
StopWatch s = new StopWatch();
s.start();
//code you want to time goes here
s.stop();
System.out.println("elapsed time in milliseconds: " + s.getElapsedTime());
}
}
The way I would do that is just run it in a loop some number of times. Like if you run it 1000 times and clock it, that gives you milliseconds. Run it 1,000,000 times, and it gives you microseconds.
If you also want to find out why it's taking as long as it is, you can just pause it some number of times (like 10) while it's running, and that will tell you what it's doing and why.
The problem with the get System.xxx method is that the method itself needs a few milliseconds to compute. The usually "accepted" way of doing it is running the test a few tens of thousands of times and calculating an average of this.
Also, depending on your OS there is something called the time granularity (example for windows). This is the smallest amount of time your OS can compute. On some OS its a millisecond, on some others its a nanosecond. It might or might not be relevant in your case.