In my main flow I have a thread pool ExecutorService, i fill it with a "known" number of Callables, that I invokeAll().
Also there is another ExecutorService object called "globalPool" (my assamption that ExecutorService is thread-safe and I can add tasks to it from different threads ).
Now, each of above Callables spawns new tasks and put them (submit) to this shared "globalPool".
The problem is that I can't know (can't block) when all tasks are completed and if there are no more incoming tasks.
Please assist.
Code snippet:
public class Worker implements Callable {
private ExecutorService sharedPool;
public Worker (ExecutorService pool){
this.sharedPool = pool;
}
public boolean call() {
for (int i = 0; i<100; i++){
if(i % 10 == 0){
sharedPool.submit(new Runnable() {
public void run() {
heavyTaskHere();
}
}
}
}
}
}
Main flow:
ExecutorService sharedPool = new Executors.newFixedThreadPool(X);
List<Callable<Boolean>> mainthareadPool= new ArrayList<Callable<Boolean>>();
ExecutorService executor = Executors.newFixedThreadPool(N);
for (int i = 0; i<10; i++){
Wroker w = new Worker (sharedPool);
mainthareadPool.add(w);
}
executor.invokeAll(mainthareadPool);
As I see it The Callables will end before their Runable will end.
If you invokeAll your Callables you get back a List of Futures corresponding to your Callables . With that Futures you can simply see if the Callables has completed by yourFutureList.get(futureOfInterestIndex).isDone()
Related
I have a task that will run many times with different values. I'd like to prevent it from executing 2 of the same tasks (Based on the string value) at the same time. Below is an example of the strings. These values will change, but for simplicity I have included these values below in the example. I submit these tasks via an ExecutorService The tasks run, but the 2nd hi blocks the other tasks from running. So 4/5 tasks run concurrently. Once the lock is released from the first hi the 5th tasks continues and the other tasks continue fine. Is there a way to prevent this type of blocking of the task so that the other 3 tasks can run before it so there is no queuing until there is actually 5 tasks running concurrently.
Submission of the tasks:
executor.submit(new Task("hi"));
executor.submit(new Task("h"));
executor.submit(new Task("u"));
executor.submit(new Task("y"));
executor.submit(new Task("hi"));
executor.submit(new Task("p"));
executor.submit(new Task("o"));
executor.submit(new Task("bb"));
The Task is simple. It just prints out the string:
Lock l = getLock(x);
try {
l.lock();
System.out.println(x);
try {
Thread.sleep(5000);
} catch (InterruptedException ex) {
Logger.getLogger(Task.class.getName()).log(Level.SEVERE, null, ex);
}
} finally {
l.unlock();
}
I've updated the post to allow for things to be more clearly understood...
To avoid blocking a thread, you have to ensure that the action doesn’t even run before the other. For example, you can use a CompletableFuture to chain an action, to be scheduled when the previous has been completed:
public static void main(String[] args) {
ExecutorService es = Executors.newFixedThreadPool(2);
for(int i = 0; i < 5; i++) submit("one", task("one"), es);
for(int i = 0; i < 5; i++) submit("two", task("two"), es);
LockSupport.parkNanos(TimeUnit.SECONDS.toNanos(26));
es.shutdown();
}
static Runnable task(String x) {
return () -> {
System.out.println(x);
LockSupport.parkNanos(TimeUnit.SECONDS.toNanos(5));
};
}
static final ConcurrentHashMap<String, CompletableFuture<Void>> MAP
= new ConcurrentHashMap<>();
static final void submit(String key, Runnable task, Executor e) {
CompletableFuture<Void> job = MAP.compute(key,
(k, previous) -> previous != null?
previous.thenRunAsync(task, e): CompletableFuture.runAsync(task, e));
job.whenComplete((v,t) -> MAP.remove(key, job));
}
The ConcurrentHashMap allows us to handle the cases as atomic updates
If no previous future exists for a key, just schedule the action, creating the future
If a previous future exists, chain the action, to be scheduled when the previous completed; the dependent action becomes the new future
If a job completed, the two-arg remove(key, job) will remove it if and only if it is still the current job
The example in the main method demonstrates how two independent actions can run with a thread pool of two threads, never blocking at thread.
I'd like to submit tasks to a thread pool (or executor service) but task should not be executed concurrently if there's already one task in the executor with the same key.
Specifically, this is for a build tool, to prevent tasks for the same part of a source tree executing concurrently.
This is an example of why I would want this behaviour:
public class Example {
public static void main(String[] args) throws Exception {
ExecutorService service = Executors.newFixedThreadPool(2);
Path resource = Paths.get("tmp");
service.submit(() -> {
Files.write(resource, Collections.singleton("foo"));
Thread.sleep(10);
if (!new String(Files.readAllBytes(resource)).equals("foo")) {
System.err.println("someone changed my stuff");
}
return null;
});
service.submit(() -> Files.write(resource, Collections.singleton("bar")));
service.shutdown();
service.awaitTermination(1, TimeUnit.MINUTES);
}
}
The solution is to use a separate single-threaded executor for each key. Since there can exist many keys, creating a thread for each key can be expensive, and so we replace single-threaded executor with a light-weight SerialExecutor which behaves like a single-threaded executor but have no own thread, borrowing a thread from some normal backend executor when needed. SerialExecutor is described in JavaDocs of Executor. Opimized version can be found at my CodeSamples project,
See similar question Design pattern to guarantee only one Runnable object of particular id value is being executed by the pool
Try with 2 different executers for your different type of tasks and if first pool completed then start another one.
ExecutorService executor1 = Executors.newFixedThreadPool(5);
for (int i = 0; i < 10; i++) {
Runnable worker = new FirstTask("" + i);
executor1.execute(worker);
}
executor1.shutdown();
while (!executor.isTerminated()) {
}
ExecutorService executor2 = Executors.newFixedThreadPool(5);
for (int i = 0; i < 10; i++) {
Runnable worker = new AnotherTask("" + i);
executor2.execute(worker);
}
executor2.shutdown();
Having -
for (int i = 0; i<10 ; i++) {
Runnable r = new Runnable(){...}
new Thread(r).start();
}
// I want to continue here only after all the subclass threads before exited .
...
How could I make sure all the subclass threads exited before I continue on after the for section ?
Does exist any solution besides keep all the Runnable's in a List<Runnable> and finally check its isAlive() for each element ?
How could I make sure all the subclass threads exited before I continue on after the for section ?
I'd use the ExecutorService classes. See the Java tutorial on them. Something like:
// create a thread pool with 10 workers
ExecutorService threadPool = Executors.newFixedThreadPool(10);
// or you can create an open-ended thread pool
// ExecutorService threadPool = Executors.newCachedThreadPool();
for (int i = 0; i < 10; i++) {
threadPool.submit(new Runnable(){...});
}
// once we have submitted all jobs to the thread pool, it should be shutdown
threadPool.shutdown();
Then you can wait for them to finish with:
threadPool.awaitTermination(Long.MAX_VALUE, TimeUnit.MILLISECONDS);
If you still want to do your own threads then typically you keep them around in a List and call join() on each one of them:
List<Thread> threadList = new ArrayList<Thread>();
for (int i = 0; i < 10; i++) {
Thread thread = new Thread(new Runnable(){...});
thread.start();
threadList.add(thread);
}
// this waits for all of the threads to finish before continuing
for (Thread thread : threadList) {
thread.join();
}
Have a look at CountDownLatch. It is great for when you want to wait for N threads to be done with something.
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
How to know if other threads have finished?
I have a threadpool that executes threads for me, how can I tell when ALL the threads I've passed it are done?
For example:
main.java
for (int i = 0; i < objectArray.length; i++) {
threadPool.submit(new ThreadHandler(objectArray[i], i));
Thread.sleep(500);
}
ThreadHandler.java
public class ThreadHandler implements Runnable {
protected SuperHandler HandlerSH;
protected int threadNum;
public ThreadHandler(SuperHandler superH, int threadNum) {
this.threadNum = threadNum;
this.HandlerSH = superH;
}
public void run() {
//do all methods here
}
Would I just put something into the run() section to set a boolean or something? Would I make an array of boolean to check when they're all done?
Thanks.
When you submit a job into a thread pool, it returns a Future instance. You can call Future.get() to see if the job has finished. This is effectively similar to a join on the tasks running in a thread pool.
You can also call threadPool.awaitTermination(...) if the thread-pool has been shutdown and you want to wait for all of the tasks have finished.
Often when I am submitting a number of jobs into a thread pool, I record their futures in a list:
List<Future<?>> futures = new ArrayList<Future<?>>();
for (int i = 0; i < objectArray.length; i++) {
futures.add(threadPool.submit(new ThreadHandler(objectArray[i], i)));
}
// if we are done submitting, we shutdown
threadPool.shutdown();
// now we can get from the future list or awaitTermination
for (Future<?> future : futures) {
// this throws an exception if your job threw an exception
future.get();
}
I'm trying to implement a sample application to test Callable and ExecutorService interfaces.
In my app I have declared:
ExecutorService exSvc = Executors.newSingleThreadExecutor();
Then:
Future<Integer> test = exSvc.submit(
new Callable<Integer>() {
public Integer call() {
for(int i = 0; i < 1000; i++){
System.out.println(i);
}
return 1;
}
});
Now I'm trying to stop the process before it terminate, I'm using exSvc.shutdownNow() but it doesn't work.
To stop gracefully a classical Thread I usually use some kind of condition variable. Which is a common approach to follow with ExecutorService?
Future.cancel(true) and ExecutorService.shutdownNow() use thread interruption. As long as you don't make uninterruptable blocking calls in your task, all you need is to handle interrupted condition correctly, something like this:
for(int i = 0; i < 1000; i++){
// Uses isInterrupted() to keep interrupted status set
if (Thread.currentThread().isInterrupted()) {
// Cannot use InterruptedException since it's checked
throw new RuntimeException();
}
System.out.println(i);
}
If you make uninterruptable blocking calls (such as network IO), things become more complex, you need to interrupt them manually somehow, for example, by closing the underlying sockets.
This is how I'd do it with a FixedThreadPool, hope it's of some help.
ExecutorService pool = Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors());
List<Future<Void>> results = new ArrayList<>();
for (int i = 0; i < numberOfJobs; i++) {
MyCallableJob job = new MyCallableJob (...);
results.add(pool.submit(job));
}
for (Future<Void> result : results) {
try { result.get(); }
catch (InterruptedException | ExecutionException ignorable) { }
}
pool.shutdown();