I'v got ConcurrentLinkedDeque which I'm using for synchronic push/pop elements,
and I'v got some async tasks which are taking one element from stack and if this element has neighbors It's pushing it to stack.
Example code:
private ConcurrentLinkedDeque<Item> stack = new ConcurrentLinkedDeque<>();
private ExecutorService exec = Executors.newFixedThreadPool(5);
while ((item = stack.pollFirst()) != null) {
if (item == null) {
} else {
Runnable worker = new Solider(this, item);
exec.execute(worker);
}
}
class Solider{
public void run(){
if(item.hasNeighbors){
for(Item item:item.neighbors){
stack.push(item)
}
}
}
}
I would like to have additional statement in while loop which answers the question - "any task in Executor is working?"
There isn't a clean way to check if all Runnables are done if you use ExecutorService.execute(Runnable). Unless you build a mechanism to do so in the Runnable itself (which is sloppy in my opinion).
Instead:
Use ExecutorService.submit(Runnable). This method will return a Future<?> which is a handle to the result of a Runnable. Using Futures provides a clean way to check results.
All you have to do is maintain a list of Futures that you submit, and then you can iterate over the whole list of Futures and either:
A) wait for all the futures to be done in a blocking way or
B) check if all the futures are done in a non-blocking way.
Here is a code example:
List<Future<?>> futures = new ArrayList<Future<?>>();
ExecutorService exec = Executors.newFixedThreadPool(5);
// Instead of using exec.execute() use exec.submit()
// because it returns a monitorable future
while((item = stack.pollFirst()) != null){
Runnable worker = new Solider(this, item);
Future<?> f = exec.submit(worker);
futures.add(f);
}
// A) Await all runnables to be done (blocking)
for(Future<?> future : futures)
future.get(); // get will block until the future is done
// B) Check if all runnables are done (non-blocking)
boolean allDone = true;
for(Future<?> future : futures){
allDone &= future.isDone(); // check if future is done
}
Update: with Java 8+ CompletableFutures you can manage this with its new callback functions. First you will need to create all of the CompletableFutures you need which will also start running, eg:
We need to accumulate all the futures generated in an Array in order to pass them later to CompletableFuture.allOf(CompletableFutures...)
So let's say you have a list of people you want to calculate its days until birthday asynchronously:
First we create all those needed futures and collect them together in an array:
CompletableFuture<?>[] completables = people.stream()
.map(p -> createCompletableFuture(p))
.toArray(CompletableFuture<?>[]::new);
private CompletableFuture createCompletableFuture(Person p) {
return CompletableFuture.runAsync(daysUntillBirthday(p));
}
Then you pass those completables to a new CompletableFuture:
CompletableFuture c = CompletableFuture.allOf(completables)
And you can now check if there are still futures running with:
c.isDone()
This may not be the cleanest solution, but you can use ThreadPoolExecutor.getActiveCount() to check how many threads are actively executing tasks.
Implementing this within a while loop with a simple condition to check if the active thread count is zero is a palatable solution.
Here is a code example:
ThreadPoolExecutor executor = (ThreadPoolExecutor) Executors.newFixedThreadPool(5);
for (int x = 0; x < 4; x++) {
Runnable worker = new Solider(this,item);
executor.execute(worker);
}
// Now check for active threads.
while(executor.getActiveCount()!=0)
{
try {
Thread.sleep(100);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
executor.shutdown();
The while block directly answers your question.
IE - If the while block is active, tasks are being executed.
Related
I have a task that will run many times with different values. I'd like to prevent it from executing 2 of the same tasks (Based on the string value) at the same time. Below is an example of the strings. These values will change, but for simplicity I have included these values below in the example. I submit these tasks via an ExecutorService The tasks run, but the 2nd hi blocks the other tasks from running. So 4/5 tasks run concurrently. Once the lock is released from the first hi the 5th tasks continues and the other tasks continue fine. Is there a way to prevent this type of blocking of the task so that the other 3 tasks can run before it so there is no queuing until there is actually 5 tasks running concurrently.
Submission of the tasks:
executor.submit(new Task("hi"));
executor.submit(new Task("h"));
executor.submit(new Task("u"));
executor.submit(new Task("y"));
executor.submit(new Task("hi"));
executor.submit(new Task("p"));
executor.submit(new Task("o"));
executor.submit(new Task("bb"));
The Task is simple. It just prints out the string:
Lock l = getLock(x);
try {
l.lock();
System.out.println(x);
try {
Thread.sleep(5000);
} catch (InterruptedException ex) {
Logger.getLogger(Task.class.getName()).log(Level.SEVERE, null, ex);
}
} finally {
l.unlock();
}
I've updated the post to allow for things to be more clearly understood...
To avoid blocking a thread, you have to ensure that the action doesn’t even run before the other. For example, you can use a CompletableFuture to chain an action, to be scheduled when the previous has been completed:
public static void main(String[] args) {
ExecutorService es = Executors.newFixedThreadPool(2);
for(int i = 0; i < 5; i++) submit("one", task("one"), es);
for(int i = 0; i < 5; i++) submit("two", task("two"), es);
LockSupport.parkNanos(TimeUnit.SECONDS.toNanos(26));
es.shutdown();
}
static Runnable task(String x) {
return () -> {
System.out.println(x);
LockSupport.parkNanos(TimeUnit.SECONDS.toNanos(5));
};
}
static final ConcurrentHashMap<String, CompletableFuture<Void>> MAP
= new ConcurrentHashMap<>();
static final void submit(String key, Runnable task, Executor e) {
CompletableFuture<Void> job = MAP.compute(key,
(k, previous) -> previous != null?
previous.thenRunAsync(task, e): CompletableFuture.runAsync(task, e));
job.whenComplete((v,t) -> MAP.remove(key, job));
}
The ConcurrentHashMap allows us to handle the cases as atomic updates
If no previous future exists for a key, just schedule the action, creating the future
If a previous future exists, chain the action, to be scheduled when the previous completed; the dependent action becomes the new future
If a job completed, the two-arg remove(key, job) will remove it if and only if it is still the current job
The example in the main method demonstrates how two independent actions can run with a thread pool of two threads, never blocking at thread.
I have method that is checking the CompletableFuture execution time. If such CompletableFuture is executing for more than 2 seconds i want to kill this task. But how can I doit if i don't have control overy thread where CompletableFuture methods are executed ?
final CompletableFuture<List<List<Student>>> responseFuture = new CompletableFuture<>();
responseFuture.supplyAsync(this::createAllRandomGroups)
.thenAccept(this::printGroups)
.exceptionally(throwable -> {
throwable.printStackTrace();
return null;
});
createAllRandomGroups()
private List<List<Student>> createAllRandomGroups() {
System.out.println("XD");
List<Student> allStudents = ClassGroupUtils.getActiveUsers();
Controller controller = Controller.getInstance();
List<List<Student>> groups = new ArrayList<>();
int groupSize = Integer.valueOf(controller.getGroupSizeComboBox().getSelectionModel().getSelectedItem());
int numberOfGroupsToGenerate = allStudents.size() / groupSize;
int studentWithoutGroup = allStudents.size() % groupSize;
if (studentWithoutGroup != 0) groups.add(this.getListOfStudentsWithoutGroup(allStudents, groupSize));
for(int i = 0; i < numberOfGroupsToGenerate; i++) {
boolean isGroupCreated = false;
while (!isGroupCreated){
Collections.shuffle(allStudents);
List<Student> newGroup = this.createNewRandomGroupOfStudents(allStudents, groupSize);
groups.add(newGroup);
if (!DataManager.isNewGroupDuplicated(newGroup.toString())) {
isGroupCreated = true;
allStudents.removeAll(newGroup);
}
}
}
DataManager.saveGroupsToCache(groups);
return groups;
}
printGroups()
private void printGroups(List<List<Student>> lists) {
System.out.println(lists);
}
This statement responseFuture.cancel(true); does not kill thread where responseFuture is doing the methods. So what is the most elegant way to terminate CompletableFuture thread ?
When you create a chain of CompletableFuture stages like b = a.thenApply(function), this handy method creates a setup of different components. Basically, these components refer to each other as a → function → b, so the completion of a will trigger the evaluation of function which will first pre-check whether b still is not completed, then evaluate your function and attempt to complete b with the result.
But b itself has no knowledge of function or the thread that will evaluate it. In fact, function is not special to b, anyone could call complete, completeExceptionally or cancel on it from any thread, the first one winning. Hence, the completable in the class name.
The only way to get hands on the threads evaluating the functions, is to be in control of them right from the start, e.g.
ExecutorService myWorkers = Executors.newFixedThreadPool(2);
CompletableFuture<FinalResultType> future
= CompletableFuture.supplyAsync(() -> generateInitialValue(), myWorkers)
.thenApplyAsync(v -> nextCalculation(v), myWorkers)
.thenApplyAsync(v -> lastCalculation(v), myWorkers);
future.whenComplete((x,y) -> myWorkers.shutdownNow());
Now, the completion of future, e.g. via cancellation, will ensure that no new evaluation will be triggered by this chain and further makes an attempt to interrupt ongoing evaluations, if any.
So you can implement a timeout, e.g.
try {
try {
FinalResultType result = future.get(2, TimeUnit.SECONDS);
System.out.println("got "+result);
}
catch(TimeoutException ex) {
if(future.cancel(true)) System.out.println("cancelled");
else System.out.println("got "+future.get());
}
}
catch(ExecutionException|InterruptedException ex) {
ex.printStackTrace();
}
Not that the rejection of tasks due to the shutdown of the thread pool may cause some of the intermediate future to never complete, but for this chain of stages, this is irrelevant. All that matters, is, that the final stage future is completed, which is guaranteed, as it is its completion which triggers the shutdown.
The only way to terminate a thread is via interruption, which is a cooperative mechanism. This means the the thread must implement interruption logic, by handling the InterruptedException.
But it is a really bad practice to interrupt threads that you don't own, which I think is your case.
Consider the following Java code (a simplified version of what I am working with - if there are errors it's because I haven't run it through a compiler):
CountdownLatch latch = new CountdownLatch(collection.size());
for(Whatever thing : collection){
provider.doWork(thing, result -> {
process(result);
latch.countDown();
};
}
try {
latch.await();
} catch (InterruptedException ignore) {}
doMoreWork();
So I run a number of asynchronous tasks, and wait for them all to be done before proceeding. Right now I'm accumulating the result of the asynchronous tasks in a list. This works and it's fine, but I'm looking at whether there's a cleaner implementation using Futures or something similar. The issue is the asynchronous call. A Callable is supposed to return the result of its work, but the result of this work won't be known until later. It's not worth rewriting doWork to be synchronous. Should I just leave this alone, or is there an option out there? Partly my interest is in better code but partly in just learning more about concurrency options. If it matters, this is in an Android app.
This is an approach using ExecutorService and Future, didn't tested on android but all available on level 1 api:
ExecutorService executor = Executors.newFixedThreadPool(10);
List<Future<String>> pending = new ArrayList<Future<String>>();
for(Whatever thing : collection) {
Future<String> future = executor.submit(new Callable<String>() {
public String call() throws Exception {
return doWork();
}
});
pending.add(future);
}
for (Future<String> result : pending) {
System.out.println("Your result ASAP:" + result.get());
}
executor.shutdown();
It will return as soon as the current item is done, in the order the were submitted.
here is two options , just bit confuse which one is best to go.
Option 1:
ExecutorService es = Executors.newFixedThreadPool(4);
List<Runnable> tasks = getTasks();
CompletableFuture<?>[] futures = tasks.stream()
.map(task -> CompletableFuture.runAsync(task, es))
.toArray(CompletableFuture[]::new);
CompletableFuture.allOf(futures).join();
es.shutdown();
Option 2:
ExecutorService es = Executors.newFixedThreadPool(4);
List< Future<?>> futures = new ArrayList<>();
for(Runnable task : taskList) {
futures.add(es.submit(task));
}
for(Future<?> future : futures) {
try {
future.get();
}catch(Exception e){
// do logging and nothing else
}
}
es.shutdown();
Here putting future.get(); in try catch is good idea right?
Since you effectively hold each submitted Future in a list of of futures by doing this:
List< Future<?>> futures = new ArrayList<>();
for(Runnable task : taskList) {
futures.add(es.submit(task));
}
You can very easily check if all the submitted jobs are done executing just, by invoking the Future#isDone method, which wilkl return true or false based on whether the task has finished or not. You can check more on this on the related documentation here.
Hence, with the above in mind you could very well create a simple helper method, that will iterate the list of your futures and check on their state. For example:
private boolean areJobsDone() {
return futures.stream()
.allMatch(Future::isDone);
}
Note that in contrast to the Future#get method, isDone is non blocking (as it's not waiting for the task to return it's result) as it effectively queries it's state.
With this you can go ahead and check for the state of your runnables, blocking the flow with a loop that will invoke the helper method, before proceeding.
Hope this helps.
There is another way to wait for all tasks to complete. After you submitted all of your tasks, call
es.shutdown()
es.awaitTermination(Long.MAX_VALUE, TimeUnit.NANO_SECONDS)
Oracle's Java Docs read:
shutdown [...] Initiates an orderly shutdown in which previously submitted tasks are executed.
awaitTermination [...] Blocks until all tasks have completed execution after a shutdown request, or the timeout occurs, or the current thread is interrupted, whichever happens first.
Concerning the timeout: with the above values, the thread-pool will only terminate after about 300 years.
Similar to the Aris_Kortex' proposal,
List<CompletableFuture<?>> futures = new ArrayList<>();
for(Runnable task : taskList) {
futures.add(CompletableFuture.runAsync(task, es));
}
and then create the combined CompletableFuture:
CompletableFuture<Void> cf = CompletableFuture.allOf(futures.toArray(futires.size()));
then you can wait for all tasks synchronously:
cf.get();
synchronously with timeout:
cf.get(1, TimeUnit.SECOND);
of asynchronously:
cf.thenRun(()->{finalActions();});
I have method that is checking the CompletableFuture execution time. If such CompletableFuture is executing for more than 2 seconds i want to kill this task. But how can I doit if i don't have control overy thread where CompletableFuture methods are executed ?
final CompletableFuture<List<List<Student>>> responseFuture = new CompletableFuture<>();
responseFuture.supplyAsync(this::createAllRandomGroups)
.thenAccept(this::printGroups)
.exceptionally(throwable -> {
throwable.printStackTrace();
return null;
});
createAllRandomGroups()
private List<List<Student>> createAllRandomGroups() {
System.out.println("XD");
List<Student> allStudents = ClassGroupUtils.getActiveUsers();
Controller controller = Controller.getInstance();
List<List<Student>> groups = new ArrayList<>();
int groupSize = Integer.valueOf(controller.getGroupSizeComboBox().getSelectionModel().getSelectedItem());
int numberOfGroupsToGenerate = allStudents.size() / groupSize;
int studentWithoutGroup = allStudents.size() % groupSize;
if (studentWithoutGroup != 0) groups.add(this.getListOfStudentsWithoutGroup(allStudents, groupSize));
for(int i = 0; i < numberOfGroupsToGenerate; i++) {
boolean isGroupCreated = false;
while (!isGroupCreated){
Collections.shuffle(allStudents);
List<Student> newGroup = this.createNewRandomGroupOfStudents(allStudents, groupSize);
groups.add(newGroup);
if (!DataManager.isNewGroupDuplicated(newGroup.toString())) {
isGroupCreated = true;
allStudents.removeAll(newGroup);
}
}
}
DataManager.saveGroupsToCache(groups);
return groups;
}
printGroups()
private void printGroups(List<List<Student>> lists) {
System.out.println(lists);
}
This statement responseFuture.cancel(true); does not kill thread where responseFuture is doing the methods. So what is the most elegant way to terminate CompletableFuture thread ?
When you create a chain of CompletableFuture stages like b = a.thenApply(function), this handy method creates a setup of different components. Basically, these components refer to each other as a → function → b, so the completion of a will trigger the evaluation of function which will first pre-check whether b still is not completed, then evaluate your function and attempt to complete b with the result.
But b itself has no knowledge of function or the thread that will evaluate it. In fact, function is not special to b, anyone could call complete, completeExceptionally or cancel on it from any thread, the first one winning. Hence, the completable in the class name.
The only way to get hands on the threads evaluating the functions, is to be in control of them right from the start, e.g.
ExecutorService myWorkers = Executors.newFixedThreadPool(2);
CompletableFuture<FinalResultType> future
= CompletableFuture.supplyAsync(() -> generateInitialValue(), myWorkers)
.thenApplyAsync(v -> nextCalculation(v), myWorkers)
.thenApplyAsync(v -> lastCalculation(v), myWorkers);
future.whenComplete((x,y) -> myWorkers.shutdownNow());
Now, the completion of future, e.g. via cancellation, will ensure that no new evaluation will be triggered by this chain and further makes an attempt to interrupt ongoing evaluations, if any.
So you can implement a timeout, e.g.
try {
try {
FinalResultType result = future.get(2, TimeUnit.SECONDS);
System.out.println("got "+result);
}
catch(TimeoutException ex) {
if(future.cancel(true)) System.out.println("cancelled");
else System.out.println("got "+future.get());
}
}
catch(ExecutionException|InterruptedException ex) {
ex.printStackTrace();
}
Not that the rejection of tasks due to the shutdown of the thread pool may cause some of the intermediate future to never complete, but for this chain of stages, this is irrelevant. All that matters, is, that the final stage future is completed, which is guaranteed, as it is its completion which triggers the shutdown.
The only way to terminate a thread is via interruption, which is a cooperative mechanism. This means the the thread must implement interruption logic, by handling the InterruptedException.
But it is a really bad practice to interrupt threads that you don't own, which I think is your case.