Which Java synchronisation object should I use to ensure an arbitrarily large number of tasks are completed? The constraints are that:
Each task takes a non-trivial amount of time to complete and it is appropriate to perform tasks in parallel.
There are too many tasks to fit into memory (i.e. I cannot put a Future for every task into a Collection and then call get on all the futures).
I do not know how many tasks there will be (i.e. I cannot use a CountDownLatch).
The ExecutorService may be shared so I cannot use awaitTermination( long, TimeUnit )
For example, with Grand Central Dispatch, I might do something like this:
let workQueue = dispatch_get_global_queue( QOS_CLASS_BACKGROUND, 0 )
let latch = dispatch_group_create()
let startTime = NSDate()
var itemsProcessed = 0
let countUpdateQueue = dispatch_queue_create( "countUpdateQueue", DISPATCH_QUEUE_SERIAL )
for item in fetchItems() // generator returns too many items to store in memory
{
dispatch_group_enter( latch )
dispatch_async( workQueue )
{
self.processItem( item ) // method takes a non-trivial amount of time to run
dispatch_async( countUpdateQueue )
{
itemsProcessed++
}
dispatch_group_leave( latch )
}
}
dispatch_group_wait( latch, DISPATCH_TIME_FOREVER )
let endTime = NSDate()
let totalTime = endTime.timeIntervalSinceDate( startTime )
print( "Processed \(itemsProcessed) items in \(totalTime) seconds." )
It produces output that looks like this (for 128 items): Processed 128 items in 1.846794962883 seconds.
I tried something similar with a Phaser:
final Executor executor = new ThreadPoolExecutor( 64, 64, 1l, MINUTES, new LinkedBlockingQueue<Runnable>( 8 ), new CallerRunsPolicy() );
final Phaser latch = new Phaser( 0 );
final long startTime = currentTimeMillis();
final AtomicInteger itemsProcessed = new AtomicInteger( 0 );
for( final String item : fetchItems() ) // iterator returns too many items to store in memory
{
latch.register();
final Runnable task = new Runnable() {
public void run() {
processItem( item ); // method takes a non-trivial amount of time to run
itemsProcessed.incrementAndGet();
latch.arrive();
}
};
executor.execute( task );
}
latch.awaitAdvance( 0 );
final long endTime = currentTimeMillis();
out.println( "Processed " + itemsProcessed.get() + " items in " + ( endTime - startTime ) / 1000.0 + " seconds." );
The tasks do not always complete before the last print statement and I might get output that looks like this (for 128 items): Processed 121 items in 5.296 seconds. Is the Phaser even the right object to use? The documentation indicates it only supports 65,535 parties so I would need to either batch the items to be processed or introduce some sort of Phaser tiering.
The problem with the Phaser usage in this example is that the CallerRunsPolicy allows a task to execute on the initiating thread. Thus, while the loop is still in progress, the number of arrived parties can equal the number of registered parties, causing the phase to increment. The solution is to initialise the Phaser with 1 party then, when the loop is finished, arrive and wait for the other parties to arrive. This ensures the phase does not increment to 1 until all the tasks are complete.
final Executor executor = new ThreadPoolExecutor( 64, 64, 1l, MINUTES, new LinkedBlockingQueue<Runnable>( 8 ), new CallerRunsPolicy() );
final Phaser latch = new Phaser( 1 );
final long startTime = currentTimeMillis();
final AtomicInteger itemsProcessed = new AtomicInteger( 0 );
for( final String item : fetchItems() ) // iterator returns too many items to store in memory
{
latch.register();
final Runnable task = new Runnable() {
public void run() {
processItem( item ); // method takes a non-trivial amount of time to run
itemsProcessed.incrementAndGet();
final int arrivalPhase = latch.arrive();
}
};
executor.execute( task );
}
latch.arriveAndAwaitAdvance();
final long endTime = currentTimeMillis();
out.println( "Processed " + itemsProcessed.get() + " items in " + ( endTime - startTime ) / 1000.0 + " seconds." );
"to ensure an arbitrarily large number of tasks are completed" - the simplest way is to maintain a counter of completed tasks, with blocking operation to wait that given number of task is reached. There is no such ready class, but it is easy to make one:
class EventCounter {
long counter=0;
synchronized void up () {
counter++;
notifyAll();
}
synchronized void ensure (long count) {
while (counter<count) wait();
}
}
"There are too many tasks to fit into memory" - so the process of submitting new tasks must be suspended when the number of running tasks is too high. The simplest way is to consider the number of running tasks as a resource and count it with a semaphore:
Semaphore runningTasksSema=new Semaphore(maxNumberOfRunningTasks);
EventCounter eventCounter =new EventCounter ();
for( final String item : fetchItems() ) {
final Runnable task = new Runnable() {
public void run() {
processItem( item );
runningTasksSema.release();
eventCounter.up();
}
};
runningTasksSema.aquire();
executor.execute(task);
}
When a thread wants to ensure some given number of tasks are completed, it invokes:
eventCounter.ensure(givenNumberOfFinishedTasks);
Asynchronous (nonblocking) versions of runningTasksSema.aquire() and eventCounter.ensure() operations can be designed, but they would be more complex.
In case if you're on java8 you can use CompletableFuture
java.util.concurrent.CompletableFuture.allOf(CompletableFuture<?>... cfs)
that will wait for results of all futures in passed array.
Related
I'm trying to make a ScheduledExecutorService where only one task is active at a time and only once a task has finished, the next task will begin its delay with an arbitrary delay amount.
As a very simple example of what I mean, take a look at this method. The idea is to schedule 10 Runnables to simulate a countdown from 10-1. Each interval takes one second (imagine this was an arbitrary amount of seconds though, I can't use scheduleAtFixedRate in my use case).
private ScheduledExecutorService scheduler = Executors.newSingleThreadScheduledExecutor();
public void startCountdown() {
for (int i = 10; i > 0; i--) {
int countdownNumber = i;
scheduler.schedule(() -> {
System.out.println(countdownNumber);
}, 1, TimeUnit.SECONDS);
}
}
However, this will simply print all 10 numbers at once, instead of waiting for a second between each value. The only way I can circumvent this (to my knowledge) is calculating the ABSOLUTE delay, as opposed to the relative one.
While it's possible to calculate the absolute time for each item, it would be quite a hassle. Isn't there some construct in Java that allows me to queue many items at once, but waits in between each item for the delay to finish, rather than processing every delay at once?
tl;dr
Do not use your countdown number to directly schedule your tasks. Have one number for scheduling number of seconds to wait (1,2,3,…) and another number for the countdown (9,8,7,…).
Use scheduleAtFixedRate to schedule your tasks for an increasing number of seconds. No need for executor service to be single-threaded.
Details
Task that re-schedules itself
Isn't there some construct in Java that allows me to queue many items at once, but waits in between each item for the delay to finish, rather than processing every delay at once?
If you have an arbitrary amount of time not known up front when beginning the scheduling, then you should only run one task at a time. Let the task re-schedule itself.
To enable a task to re-schedule itself, pass a reference to the ScheduledExecutorService to the task object (your Runnable or Callable) as an argument in the constructor. After the task completes its main work, it discovers/calculates the amount of time to elapse for the next run. The task then submits itself (this) to the passed executor service, along with the amount of time to elapse before the next task execution.
I have already posted Answers on Stack Overflow with code for tasks that re-schedule themselves. I would expect others have as well. Search to learn more.
Regarding the "countdown" aspect of your Question, read on.
Countdown
You have the right approach in using a scheduled executor service. The problem is that you are calling the wrong method on that class.
Your call to schedule means you are scheduling several tasks to all run after a single second. All those tasks are starting from the moment your call is made. So each runs after one second from your call to schedule. So the ten tasks are all waiting a second from almost the same moment: ten moments a split-second apart, the split-second being the time it takes for your for loop to continue.
scheduleAtFixedRate
The method you are looking for is scheduleAtFixedRate. To quote the doc:
Submits a periodic action that becomes enabled first after the given initial delay, and subsequently with the given period; that is, executions will commence after initialDelay, then initialDelay + period, then initialDelay + 2 * period, and so on.
private ScheduledExecutorService scheduler = Executors.newSingleThreadScheduledExecutor();
public void countdown( ScheduledExecutorService scheduler )
{
for ( int i = 1 ; i <= 10 ; i++ )
{
int countdownNumber = 10 - i ; // For 9 through 0. Add 1 for 10 through 1.
scheduler.scheduleAtFixedRate
(
() -> { System.out.println( countdownNumber ) ; } ,
i , // 1 second, then 2 seconds, then 3 seconds, and so on to 10 seconds.
TimeUnit.SECONDS
) ;
}
}
… Eventually shut down your scheduled executor service.
Notice how this approach does not require the ScheduledExecutorService to be single-threaded.
Full example
Here is a complete example app.
package work.basil.example.countdown;
import java.time.Duration;
import java.time.Instant;
import java.util.Objects;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.TimeUnit;
public class Countdown
{
public static void main ( String[] args )
{
Countdown app = new Countdown();
app.demo();
}
private void demo ( )
{
System.out.println( "INFO - Demo start. " + Instant.now() );
ScheduledExecutorService scheduler = Executors.newSingleThreadScheduledExecutor(); // Our code does *not* require the executor service to be single-threaded. But for this particular example, we might as well do it that way.
this.countdown( scheduler );
this.shutdownAndAwaitTermination( scheduler , Duration.ofMinutes( 1 ) , Duration.ofMinutes( 1 ) );
System.out.println( "INFO - Demo end. " + Instant.now() );
}
public void countdown ( final ScheduledExecutorService scheduler )
{
Objects.requireNonNull( scheduler ) ;
for ( int i = 1 ; i <= 10 ; i++ )
{
int countdownNumber = 10 - i; // For 9 through 0. Add 1 for 10 through 1.
scheduler.scheduleAtFixedRate
(
( ) -> { System.out.println( "Countdown: " + countdownNumber + " at " + Instant.now() ); } ,
i , // 1 second, then 2 seconds, then 3 seconds, and so on to 10 seconds.
TimeUnit.SECONDS
);
}
}
// My slightly modified version of boilerplate code taken from Javadoc of `ExecutorService`.
// https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/concurrent/ExecutorService.html
void shutdownAndAwaitTermination ( final ExecutorService executorService , final Duration waitForWork , final Duration waitForRemainingTasks )
{
Objects.requireNonNull( executorService ) ;
Objects.requireNonNull( waitForWork ) ;
Objects.requireNonNull( waitForRemainingTasks ) ;
executorService.shutdown(); // Disable new tasks from being submitted
try
{
// Wait a while for existing tasks to terminate
if ( ! executorService.awaitTermination( waitForWork.toMillis() , TimeUnit.MILLISECONDS ) )
{
executorService.shutdownNow(); // Cancel currently executing tasks
// Wait a while for tasks to respond to being cancelled
if ( ! executorService.awaitTermination( waitForRemainingTasks.toMillis() , TimeUnit.MILLISECONDS ) )
{ System.err.println( "ExecutorService did not terminate." ); }
}
}
catch ( InterruptedException ex )
{
// (Re-)Cancel if current thread also interrupted
executorService.shutdownNow();
// Preserve interrupt status
Thread.currentThread().interrupt();
}
System.out.println( "DEBUG - shutdownAndAwaitTermination ran. " + Instant.now() );
}
}
When run:
INFO - Demo start. 2023-01-20T21:24:47.379244Z
Countdown: 9 at 2023-01-20T21:24:48.390269Z
Countdown: 8 at 2023-01-20T21:24:49.390045Z
Countdown: 7 at 2023-01-20T21:24:50.389957Z
Countdown: 6 at 2023-01-20T21:24:51.386468Z
Countdown: 5 at 2023-01-20T21:24:52.390168Z
Countdown: 4 at 2023-01-20T21:24:53.386538Z
Countdown: 3 at 2023-01-20T21:24:54.387583Z
Countdown: 2 at 2023-01-20T21:24:55.386705Z
Countdown: 1 at 2023-01-20T21:24:56.389490Z
Countdown: 0 at 2023-01-20T21:24:57.387566Z
DEBUG - shutdownAndAwaitTermination ran. 2023-01-20T21:24:57.391224Z
INFO - Demo end. 2023-01-20T21:24:57.391966Z
By the way, know that scheduled tasks do not always fire exactly on time for a variety of reasons.
Also, be aware that messages sent to System.out across threads do not always appear on the console chronologically. If you care about order, always include and study a timestamp such as Instant#now.
You can schedule the next call inside the task.
void countdown(final int i) {
scheduler.schedule(() -> {
System.out.println(i);
if (i > 0) countdown(i - 1);
}, 1, TimeUnit.SECONDS);
}
// ...
countdown(10);
How do I access class variable that is modified inside a TimerTask(). Here's an example:
public class Foo{
// Global variables
private int count = 0;
private Timer fooTimer;
public void startTimerTask(){
TimerTask fooTimerTask = new TimerTask(){
int c = 0;
public void run(){
c = count; // Using count here and updating its value
c++; // Local counter is incremented here
count = c; // Reassigning count
System.out.println(count); // Prints updated value
}
}
fooTimer.schedueleAtFixedRate(fooTimerTask, 0, 2000);
}
public static void main(String args[]){
// Start new timer and start tasks
fooTimer = new Timer();
startTimerTask();
// Trying to check value of count outside TimerTask()
count = count + 5;
System.out.println("Count: "+count) // Count returns 5
}
}
Using the above example, I would like to figure out how I can access modified value of count from TimerTask() and modify it in main() so that it also reflects on count used in TimerTask().
I understand that Java creates a copy of the original count and passes that copy to the TimerTask (which is an anonymous inner class) to be used upon.
But is there a way to use count as a shared resource?
I have also tried using single-element int array instead of int variable as count. But it still shows the same result.
This topic has been addressed many times on Stack Overflow, apparently a common schoolwork assignment. So I'll be brief. Search Stack Overflow to learn more.
AtomicInteger
But is there a way to use count as a shared resource?
Yes. Make the shared count an AtomicInteger.
Also, the Timer/TimerTask classes were supplanted many years ago by the Executors framework added to Java 5, as noted in their Javadoc.
By the way, watch your naming. Class names start with uppercase letter in Java, by convention.
By the way, your // Global variables label is a misnomer. What you are labeling are commonly called "member fields" or "instance variables". Closer to a true “global variable” in Java would be a static variable.
package work.basil.demo;
import java.util.concurrent.atomic.AtomicInteger;
public class Foo{
// Member field variables
public AtomicInteger counter = new AtomicInteger( 0 ) ;
}
Do your tasks in background threads.
You can define your Runnable/Callable using either lambda syntax or conventional syntax.
Obtain a scheduled executor service from Executors.
Notice that the main thread accesses the counter to obtain its current value while the background thread may be modifying that value. The AtomicInteger class makes thread-safe our threaded calls to get and updateAndGet.
package work.basil.demo;
import java.time.Duration;
import java.time.Instant;
import java.util.concurrent.*;
import java.util.function.IntUnaryOperator;
public class FooApp
{
public static void main ( String[] args )
{
System.out.println( "Our demo's `main` method is starting. " + Instant.now() );
Foo foo = new Foo();
IntUnaryOperator unaryOperator = ( int x ) -> ( x + 5 );
Runnable task = ( ) -> {
int newCount = foo.counter.updateAndGet( unaryOperator );
System.out.println( "New count is " + newCount + " at approximately " + Instant.now() ); // Beware: These calls may *not* appear on console in expected sequence.
};
ScheduledExecutorService ses = Executors.newSingleThreadScheduledExecutor();
ScheduledFuture < ? > sf = ses.scheduleAtFixedRate( task , 0 , 2 , TimeUnit.SECONDS );
// Let our background task run a few times.
// Check the current value once in a while.
for ( int i = 0 ; i < 3 ; i++ )
{
try { Thread.sleep( Duration.ofSeconds( 5 ).toMillis() ); } catch ( InterruptedException e ) { e.printStackTrace(); }
System.out.println( "Main thread reports counter is " + foo.counter.get() + " at " + Instant.now() );
}
try { Thread.sleep( Duration.ofSeconds( 20 ).toMillis() ); } catch ( InterruptedException e ) { e.printStackTrace(); }
ses.shutdown();
try { ses.awaitTermination( 1 , TimeUnit.MINUTES ); } catch ( InterruptedException e ) { e.printStackTrace(); }
System.out.println( "Our demo's `main` method is ending. " + Instant.now() );
}
}
When run.
Our demo's `main` method is starting. 2021-06-17T22:02:47.679682Z
New count is 5 at approximately 2021-06-17T22:02:47.692962Z
New count is 10 at approximately 2021-06-17T22:02:49.696992Z
New count is 15 at approximately 2021-06-17T22:02:51.696261Z
Main thread reports counter is 15 at 2021-06-17T22:02:52.697280Z
New count is 20 at approximately 2021-06-17T22:02:53.696181Z
New count is 25 at approximately 2021-06-17T22:02:55.696448Z
New count is 30 at approximately 2021-06-17T22:02:57.696376Z
Main thread reports counter is 30 at 2021-06-17T22:02:57.702464Z
New count is 35 at approximately 2021-06-17T22:02:59.696243Z
New count is 40 at approximately 2021-06-17T22:03:01.696155Z
Main thread reports counter is 40 at 2021-06-17T22:03:02.707548Z
New count is 45 at approximately 2021-06-17T22:03:03.696134Z
New count is 50 at approximately 2021-06-17T22:03:05.696130Z
New count is 55 at approximately 2021-06-17T22:03:07.695878Z
New count is 60 at approximately 2021-06-17T22:03:09.695790Z
New count is 65 at approximately 2021-06-17T22:03:11.696289Z
New count is 70 at approximately 2021-06-17T22:03:13.696278Z
New count is 75 at approximately 2021-06-17T22:03:15.696278Z
New count is 80 at approximately 2021-06-17T22:03:17.696187Z
New count is 85 at approximately 2021-06-17T22:03:19.696101Z
New count is 90 at approximately 2021-06-17T22:03:21.696009Z
Our demo's `main` method is ending. 2021-06-17T22:03:22.712192Z
I'm studying Java multi threading and trying to check performance with multiple threads.I am trying to check whether multi threading is better than with single thread.
So, I wrote a code which sums to limit.
It is working as I expected(multiple threads are faster than single thread) when limit gets larger but it didn't when limit is small like 100000L.
Is this due to context-switching ? and is the code below is appropriate to check performance of multi threading ?
public class MultiThreadingSum {
long count = 0;
static long limit = 1000000000L;
static void compareMultipleThreadToSingleThread(int threadCnt) {
Runnable r = () -> {
MultiThreadingSum mts = new MultiThreadingSum();
long startTime = System.nanoTime();
while(++mts.count<=limit);
long endTime = System.nanoTime();
long estimatedTime = endTime - startTime;
double seconds = estimatedTime / 1000000000.0;
System.out.println(Thread.currentThread().getName()+", elapsed time : "+seconds);
};
for(int i=0; i<threadCnt; i++) {
new Thread(r, "multiThread"+i).start();
}
Runnable r2 = () -> {
MultiThreadingSum mts = new MultiThreadingSum();
long startTime = System.nanoTime();
while(++mts.count<=limit*threadCnt);
long endTime = System.nanoTime();
long estimatedTime = endTime - startTime;
double seconds = estimatedTime / 1000000000.0;
System.out.println(Thread.currentThread().getName()+", elapsed time : "+seconds);
};
new Thread(r2, "singleThread").start();
}
public static void main(String[] args) {
compareMultipleThreadToSingleThread(3);
}
}
Your code does not wait for the 3-thread experiment to finish before running the single-thread experiment. So you may be contaminating your results.
Your code seems needlessly complicated. Can't we run two separate experiments, one with 3 threads and one with 1 thread, separately, to reuse code?
In modern Java, we rarely need to address the Thread class. Instead, use the executor service framework added to Java 5.
Putting this all together, perhaps your experiment should look more like the following.
Caveat: This is just a very rough cut, I've not thought it through, and my caffeination has been exhausted. So revise this code thoughtfully. Perhaps I can revise this code in a day or two.
package work.basil.threading;
import java.time.Duration;
import java.time.Instant;
import java.util.List;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicInteger;
public class Loopy
{
public static void main ( String[] args )
{
Loopy app = new Loopy();
List < Integer > inputThreadsLimit = List.of( 1 , 3 , ( Runtime.getRuntime().availableProcessors() - 1 ) );
for ( Integer numberOfThreads : inputThreadsLimit )
{
System.out.println("----------| Experiment for thread count: " + numberOfThreads + " |--------------------------");
Duration duration = app.demo( numberOfThreads ); // Waits here for the experiment to run to completion.
System.out.println( numberOfThreads + " = " + duration + " total, each: " + duration.dividedBy( numberOfThreads ) );
}
}
// Member fields
final private AtomicInteger count = new AtomicInteger( 0 );
private Duration demo ( final int numberOfThreads )
{
ExecutorService executorService = Executors.newFixedThreadPool( numberOfThreads );
long start = System.nanoTime();
for ( int i = 0 ; i < numberOfThreads ; i++ )
{
executorService.submit( new Task() );
}
executorService.shutdown(); // Ask the executor service to shutdown its backing pool of threads after all submitted tasks are done/canceled/failed.
try { executorService.awaitTermination( 1 , TimeUnit.HOURS ); } catch ( InterruptedException e ) { e.printStackTrace(); } // Tries to force the shutdown after timeout.
Duration elapsed = Duration.ofNanos( System.nanoTime() - start );
return elapsed;
}
class Task implements Runnable
{
#Override
public void run ( )
{
int countSoFar = count.incrementAndGet(); // Thread-safe way to access, increment, and write a counter.
// … add code here to do some kind of work …
System.out.println( "Thread ID " + Thread.currentThread().getId() + " is finishing run after incrementing the countSoFar to: " + countSoFar + " at " + Instant.now() );
}
}
}
this is not a good example. the multi and single threaded solutions run simultaneously and on the same counter. so practically you run one multi threaded process with four threads. you need to run one solution until thread is complete and shutdown, then the other. the easiest solution would be to run the single threaded process as a simple loop in the main method and run the multi threaded solution after the loop completes. also, i would have two separate counters, or, you can assign zero to counter after single thread loop completes
I am working on a project in which I need to measure Total Time taken by program and average time taken by program. And that program is a Multithreaded program.
In that program, each thread is working in a particular range. Input parameters is Number of Threads and Number of Task.
If number of threads is 2 and number of tasks is 10 then each thread will be performing 10 tasks. So that means 2 thread will be doing 20 tasks.
So that means-
First thread should be using id between 1 and 10 and second thread should be using id between 11 and 20.
I got the above scenario working. Now I want to measure what is the total time and average time taken by all the threads. So I got the below setup in my program.
Problem Statement:-
Can anyone tell me the way I am trying to measure the Total time and Average time taken by all the threads is correct or not in my below program?
//create thread pool with given size
ExecutorService service = Executors.newFixedThreadPool(noOfThreads);
long startTime = 0L;
try {
readPropertyFiles();
startTime = System.nanoTime();
// queue some tasks
for (int i = 0, nextId = startRange; i < noOfThreads; i++, nextId += noOfTasks) {
service.submit(new XMPTask(nextId, noOfTasks, tableList));
}
service.shutdown();
service.awaitTermination(Long.MAX_VALUE, TimeUnit.DAYS);
} finally {
long estimatedTime = System.nanoTime() - startTime;
logTimingInfo(estimatedTime, noOfTasks, noOfThreads);
}
private static void logTimingInfo(long elapsedTime, int noOfTasks, int noOfThreads) {
long timeInMilliseconds = elapsedTime / 1000000L;
float avg = (float) (timeInMilliseconds) / noOfTasks * noOfThreads;
LOG.info(CNAME + "::" + "Total Time taken " + timeInMilliseconds + " ms. And Total Average Time taken " + avg + " ms");
}
service.submit is getting executed only noOfThreads times. XMPTask object is created the same number of times.
The time you measure is not the consumed time but the elapsed time.
If the program tested (the JVM) is the only one on the computer, it may be relatively accurate but in a real world a lot of process runs concurrently.
I have already done this job by using a native call to the OS, on Windows (I'll complete this post Monday at my office) and Linux (/proc).
I think you would need to measure the time within the task class itself (XMPTask). Within that task you should be able to extract the id of the thread that is executing it and log that. Using this approach will require reading the logs and doing some calculations on them.
Another approach would be to keep running totals and averages as time progresses. To do this you could write a simple class that is passed into each task that has some static (per jvm) variables for tracking what each thread is doing. Then you could have a single thread outside the Threadpool that did the calculations. So if you wanted to report the average cpu time for each thread every second, this calculation thread could sleep for a second, then calculate and log all the average times, then sleep for a second....
EDIT: After re-reading the requirements, you don't need a background thread, but not sure if we are tracking the average time per thread or average time per task. I have assumed total time and average time per thread and fleshed out the idea in code below. It has not been tested or debugged but should give you a good idea of how to start:
public class Runner
{
public void startRunning()
{
// Create your thread pool
ExecutorService service = Executors.newFixedThreadPool(noOfThreads);
readPropertyFiles();
MeasureTime measure = new MeasureTime();
// queue some tasks
for (int i = 0, nextId = startRange; i < noOfThreads; i++, nextId += noOfTasks)
{
service.submit(new XMPTask(nextId, noOfTasks, tableList, measure));
}
service.shutdown();
service.awaitTermination(Long.MAX_VALUE, TimeUnit.DAYS);
measure.printTotalsAndAverages();
}
}
public class MeasureTime
{
HashMap<Long, Long> threadIdToTotalCPUTimeNanos = new HashMap<Long, Long>();
HashMap<Long, Long> threadIdToStartTimeMillis = new HashMap<Long, Long>();
HashMap<Long, Long> threadIdToStartTimeNanos = new HashMap<Long, Long>();
private void addThread(Long threadId)
{
threadIdToTotalCPUTimeNanos.put(threadId, 0L);
threadIdToStartTimeMillis.put(threadId, 0L);
}
public void startTimeCount(Long threadId)
{
synchronized (threadIdToStartTimeNanos)
{
if (!threadIdToStartTimeNanos.containsKey(threadId))
{
addThread(threadId);
}
long nanos = System.nanoTime();
threadIdToStartTimeNanos.put(threadId, nanos);
}
}
public void endTimeCount(long threadId)
{
synchronized (threadIdToStartTimeNanos)
{
long endNanos = System.nanoTime();
long startNanos = threadIdToStartTimeNanos.get(threadId);
long nanos = threadIdToTotalCPUTimeNanos.get(threadId);
nanos = nanos + (endNanos - startNanos);
threadIdToTotalCPUTimeNanos.put(threadId, nanos);
}
}
public void printTotalsAndAverages()
{
long totalForAllThreadsNanos = 0L;
int numThreads = 0;
long totalWallTimeMillis = 0;
synchronized (threadIdToStartTimeNanos)
{
numThreads = threadIdToStartTimeMillis.size();
for (Long threadId: threadIdToStartTimeNanos.keySet())
{
totalWallTimeMillis += System.currentTimeMillis() - threadIdToStartTimeMillis.get(threadId);
long totalCPUTimeNanos = threadIdToTotalCPUTimeNanos.get(threadId);
totalForAllThreadsNanos += totalCPUTimeNanos;
}
}
long totalCPUMillis = (totalForAllThreadsNanos)/1000000;
System.out.println("Total milli-seconds for all threads: " + totalCPUMillis);
double averageMillis = totalCPUMillis/numThreads;
System.out.println("Average milli-seconds for all threads: " + averageMillis);
double averageCPUUtilisation = totalCPUMillis/totalWallTimeMillis;
System.out.println("Average CPU utilisation for all threads: " + averageCPUUtilisation);
}
}
public class XMPTask implements Callable<String>
{
private final MeasureTime measure;
public XMPTask(// your parameters first
MeasureTime measure)
{
// Save your things first
this.measure = measure;
}
#Override
public String call() throws Exception
{
measure.startTimeCount(Thread.currentThread().getId());
try
{
// do whatever work here that burns some CPU.
}
finally
{
measure.endTimeCount(Thread.currentThread().getId());
}
return "Your return thing";
}
}
After writing all this, there is one thing that seems a bit strange in that the XMPTask seems to know too much about the list of tasks, when, I think you should just create an XMPTask for every task that you have, give it enough information to do the job, and submit them to the service as you create them.
I need to create a java function that will only run for 30 minutes, and at the end of the 30 minutes it executes something. But it should also be able to self terminate before the given time if the right conditions are met. I don't want the function to be sleeping as it should be collecting data, so no sleeping threads.
Thanks
Use: Timer.schedule( TimerTask, long )
public void someFunctino() {
// set the timeout
// this will stop this function in 30 minutes
long in30Minutes = 30 * 60 * 1000;
Timer timer = new Timer();
timer.schedule( new TimerTask(){
public void run() {
if( conditionsAreMet() ) {
System.exit(0);
}
}
}, in30Minutes );
// do the work...
.... work for n time, it would be stoped in 30 minutes at most
... code code code
}
Get the start time with System.currentTimeMillis(), calculate the time when to stop and check the current time every now and then while you're collecting the data you want to collect. Another way would be to decouple the timer and the data collecting, so that each of them could run in their own threads.
For a more specific answer, it would be helpful if you would tell what data you are collecting and how you are collecting it.
Something like this will work:
long startTime = System.currentTimeMillis();
long maxDurationInMilliseconds = 30 * 60 * 1000;
while (System.currentTimeMillis() < startTime + maxDurationInMilliseconds) {
// carry on running - 30 minutes hasn't elapsed yet
if (someOtherConditionIsMet) {
// stop running early
break;
}
}
The modern java.util.concurrent way would be using ExecutorService. There are several invoke methods taking a timeout.
Here's a kickoff example:
public static void main(String args[]) throws Exception {
ExecutorService executor = Executors.newSingleThreadExecutor();
executor.invokeAll(Arrays.asList(new Task()), 30, TimeUnit.MINUTES);
executor.shutdown();
}
where Task look like this:
public class Task implements Callable<String> {
#Override
public String call() {
// Just a dummy long running task.
BigInteger i = new BigInteger("0");
for (long l = 0; l < Long.MAX_VALUE; l++) {
i.multiply(new BigInteger(String.valueOf(l)));
// You need to check this regularly..
if (Thread.interrupted()) {
System.out.println("Task interrupted!");
break; // ..and stop the task whenever Thread is interrupted.
}
}
return null; // Or whatever you'd like to use as return value.
}
}
See also:
Lesson: Concurrency