Develop Reference

Catch and throw custom Exception from Java ExecutorService

I have a function that goes through a list of HappyObjects and sets their fields asynchronously. In the Callable, a JsonProcessingException can occur. I have to wrap this and other exceptions from this function into a custom exception (ControllerException) and throw that instead.
Other Stack Overflow posts seem to suggest collect into a List of Futures and use get() to catch the exceptions. Thus, this is what I have so far:
default List<HappyObj> fillfunction(final List<HappyObj> happyObjs) throws ControllerException {
ThreadPoolExecutor executor = (ThreadPoolExecutor) Executors.newCachedThreadPool();
List<Future<HappyObj>> futures = new ArrayList<>();
for (HappyObj happyObj : happyObjs) {
Future<HappyObj> future = executor.submit(
() -> {
final List<Mood> moods = getMoods();
for (Mood mood : moods) {
final String json = getJsonEmotion();
ObjectMapper mapper = new ObjectMapper();
mapper.setVisibility(PropertyAccessor.FIELD, JsonAutoDetect.Visibility.ANY);
List<Emotion> emotions =
mapper.readValue(json, new TypeReference<List<Emotion>>() {}); //JsonProcessingException can occur here
MoodMetadata metadata = mood.getMoodMetadata();
if (metadata != null && metadata.getEmotionMetadata() != null) {
metadata.getEmotionMetadata().setEmotions(emotions);
}
}
happyObj.setMoods(moods);
return happyObj;
});
futures.add(future);
}
executor.shutdown();
final long maxSlaSec = 1;
try {
executor.awaitTermination(maxSlaSec, TimeUnit.SECONDS);
List<HappyObj> happyResult = new ArrayList<>();
for (Future<HappyObj> future : futures) {
happyResult.add(future.get());
}
return happyResult;
} catch (InterruptedException | ExecutionException e) {
executor.shutdownNow();
throw new ControllerException(e);
}
}
Is there a more elegant way than iterating through List<Future> and calling get on each to catch ExecutorException? I thought about using execute() vs. submit(), but then I can't handle the JsonProcessingException. I saw another post suggesting creating a ThreadPoolExecutor subclass and override the afterExecute(), but I wasn't able to handle the JsonProcessingException.
One of the reasons I asked this question is because since this method consists mainly of setters, the function was originally manipulating the given objects and returning void.

According to the docs of ExecutionException (and also the docs of Future#get), it already has wrapped that information. That is, you can use its getCause to inspect the Exception thrown by the Callable's body.
Notice that Callable#call itself throws an Exception... When you throw an Exception from the Callable, it will be wrapped into an ExecutionException which will be thrown from the Future#get method, for each Callable, meaning that you can change your loop to catch an ExecutionException for each Future and inspect its getCause!
So you don't actually need to wrap it to a custom ControllerException.
The Callables you create, can still return null of type Void for example, without needing to do anything about them.
Unless the scenario changes, then you don't need to extend ThreadPoolExecutor in this case. You don't even have to cast to ThreadPoolExecutor, because the ExecutorService interface already has the submits you need. Just throw whatever Exception you need from the Callable (such as JsonProcessingException that you mentioned) when something goes wrong in the Callable, and then inspect the ExecutionException from each Future#get method to tell if an JsonProcessingException was thrown (and you can also determine in which one it was thrown, if you need).
Is there a more elegant way than iterating through List and calling get on each to catch ExecutorException?
In my opinion, no, there is not, because you want to submit all Callables first, then let them run in parallel, and at the end inspect their ExecutionException for any Exception thrown by the Callables' body for each Callable (via Future#get of the returned Future by submit).

ExecuterService stopped processing one thread out of two

I have a list of 40000 records that needs to be processed in a for loop. Since I have a two processor system. I have created a fixedThreadPool like this:
int threads = Runtime.getRuntime().availableProcessors();
ExecutorService service = Executors.newFixedThreadPool(threads);
And divided my ArrayList into two sublists. For each of these sublists, I have created a Callable that performs the same function (involves iterating over the sublist and doing some processing) and returns me a Future object.
I submitted both these Callable using executorServiceObject.submit(callable) and added the returned Future object into my list of Future objects
Here is my question:
I have written a System.Out.printLn("Processed Item" +item.id) // consider item as the name of reference variable for current iteration
Everything was fine for some time and i could see two threads working simutaneously. But after some time, one of the threads have stopped processing. Only one thread is running. (I know this because i can see on the console that the id's given to thread 2 are not being printed anymore).
Does anyone know how this happened? I mean why ExecutorService stopped running 2nd thread.
Thanks for your help in advance.
Adding sample code as I should have done before:
public List<Output> processInputs(List<Input> inputs)
throws InterruptedException, ExecutionException {
int threads = Runtime.getRuntime().availableProcessors();
ExecutorService service = Executors.newFixedThreadPool(threads);
List<Future<Output>> futures = new ArrayList<Future<Output>>();
for (final Input input : inputs) {
Callable<Output> callable = new Callable<Output>() {
public Output call() throws Exception {
Output output = new Output();
// process your input here and compute the output
return output;
}
};
futures.add(service.submit(callable));
}
service.shutdown();
List<Output> outputs = new ArrayList<Output>();
for (Future<Output> future : futures) {
outputs.add(future.get());
}
return outputs;

Everything was fine for some time and i could see two threads working simultaneously. But after some time, one of the threads have stopped processing. Only one thread is running. (I know this because i can see on the console that the id's given to thread 2 are not being printed anymore).
I suspect that your processing thread has thrown an exception. The Future.get() method can throw ExecutionException "if the computation threw an exception".
// the following might throw an exception if the background job threw
outputs.add(future.get());
If there was a NPE, an IOException, etc. thrown by your "process your input" code then that exception is thrown by the Callable and stored in the Future so it can be thrown by the get() method but wrapped in an ExecutionException. This is useful so the thread that is waiting can get and and handle (log, etc.) the exception thrown by the background thread.
Instead of just having your processInputs(...) method throw the exception to the caller where it might be getting lost, I'd do something like the following in your while loop:
try {
outputs.add(future.get());
} catch (InterruptedException ie) {
// always a good pattern if the thread that is waiting was interrupted
Thread.currentThread().interrupt();
return;
} catch (ExecutionException ee) {
// somehow log the error
logger.error("Computation failed to process", ee);
// now continue and get the next future in the list
}
If you don't catch and properly handle that ExecutionException then the processing exception will also kill the thread that calls processInputs(...).

Java Calling method with timeout [duplicate]

I have submitted a task using executors and I need it to stop after some time (e.g. 5 minutes). I have tried doing like this:
for (Future<?> fut : e.invokeAll(tasks, 300, TimeUnit.SECONDS)) {
try {
fut.get();
} catch (CancellationException ex) {
fut.cancel(true);
tasks.clear();
} catch(ExecutionException ex){
ex.printStackTrace(); //FIXME: gestita con printstack
}
}
But I always get an error: I have a shared Vector that needs to be modified by the tasks and then read by a thread, and even if I stop all the task, if the timeout occurs I get:
Exception in thread "Thread-1" java.util.ConcurrentModificationException
Is there something wrong? How can I stop the tasks submitted that are still working after 5 minutes?

Just because you call cancel() on Future doesn't mean that the task will stop automatically. You have to do some work within the task to make sure that it will stop:
Use cancel(true) so that an interrupt is sent to the task.
Handle InterruptedException. If a function in your task throws an InterruptedException, make sure you exit gracefully as soon as possible upon catching the exception.
Periodically check Thread.currentThread().isInterrupted() if the task does continuous computation.
For example:
class LongTask implements Callable<Double> {
public Double call() {
// Sleep for a while; handle InterruptedException appropriately
try {
Thread.sleep(10000);
} catch (InterruptedException ex) {
System.out.println("Exiting gracefully!");
return null;
}
// Compute for a while; check Thread.isInterrupted() periodically
double sum = 0.0;
for (long i = 0; i < 10000000; i++) {
sum += 10.0
if (Thread.currentThread().isInterrupted()) {
System.out.println("Exiting gracefully");
return null;
}
}
return sum;
}
}
Also, as other posts have mentioned: ConcurrentModificationException can be thrown even if using the thread-safe Vector class, because iterators you obtain from Vector are not thread-safe, and thus need to be synchronized. The enhanced for-loop uses iterators, so watch out:
final Vector<Double> vector = new Vector<Double>();
vector.add(1.0);
vector.add(2.0);
// Not thread safe! If another thread modifies "vector" during the loop, then
// a ConcurrentModificationException will be thrown.
for (Double num : vector) {
System.out.println(num);
}
// You can try this as a quick fix, but it might not be what you want:
synchronized (vector) { // "vector" must be final
for (Double num : vector) {
System.out.println(num);
}
}

The ConcurrentModificationException is coming from your call to tasks.clear() while your Exceutors is iterating over your tasks Vector. What you can try to do is call shutdownNow() on your ExecutorService

The most common case for ConcurrentModificationException is when the vector is being modified at the same time as it is being iterated. Often this will be done in a single thread. You need to hold a lock on the Vector for the whole iteration (and careful not to deadlock).

fut.get() is a blocking call, even after the timeout, you will block until the task is done. If you want to stop as close to the 5 minute mark as possible, you do need to check the interrupt flag, I just recommend you do so using the Thread.isInterrupted() method which preserves the interrupt state. If you want to just stop immediately and don't need to clean any state, then throw an exception which will be caught by the Future and indicated to you as an ExecutionException.
fut.cancel(true) does not do anything as the invokeAll() method has already done this for you.
Unless you use the "tasks" Collection somewhere else, you probably don't need to call clear() on it. This isn't going to be the source of your problem since the invokeAll() method is done with the List by the time you call clear(). But, if you need to start forming a list of new tasks to execute, I suggest you form a new List of tasks, not use an old List of new Tasks.
Unfortunately, I do not have an answer for your problem. I do not see enough information here to diagnose it. Nothing in the code snippet you provided indicates an improper (only unnecessary) use of library classes/methods. Perhaps if you included a full stack trace, instead of the one line error.

Put the fut.cancel(true); in the finally block

Java: Waiting on Get Method of ScheduledFuture

Please consider the following code:
public static void main(String... args) throws InterruptedException, ExecutionException {
ScheduledExecutorService executor = Executors.newSingleThreadScheduledExecutor();
ScheduledFuture<?> future =
executor.scheduleAtFixedRate(Dummy::iterate,
0,
1,
TimeUnit.SECONDS);
TimeUnit.MILLISECONDS.sleep(1500);
executor.shutdown();
executor.awaitTermination(1, TimeUnit.MILLISECONDS);
System.out.println("Getting the future...");
future.get();
System.out.println("Got the future...");
System.out.println("Finished");
}
private static void iterate(){
System.out.println("Iterating... counter is: " + counter++);
try {
TimeUnit.MILLISECONDS.sleep(900);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
Note that the executor.awaitTermination(...) command future.get() command are happening after 1500 ms, meaning in the middle of the iterate() method.
This means that awaitTermination(...) will return false because the scheduled task hasn't finsihed yet.
Now, future.get() will wait forever. The task will finish and the service won't start other tasks, but yet the get() will never return.
A workaround is asking the future for isDone(), and only if it is done, asking for the result.
My question is what exactly is happening?
Looks like that if shutDown() happens in the during an iteration, the ScheduledThreadPool will somehow halt and, meaning there will be no future available. So why is this happening? I have looked at the documentation but couldn't find any reference indicating this issue. Is it possible that such scenario causes a future to not done, and later for the future to not be available?

If you replace your :
future.get();
to something like:
future.get(2000,TimeUnit.MILLISECONDS);
Then you see java.util.concurrent.TimeoutException exception ocuur.
and this Exception thrown when a blocking operation times out. Blocking operations for which a timeout is specified need a means to indicate that the timeout has occurred. For many such operations it is possible to return a value that indicates timeout; when that is not possible or desirable then TimeoutException should be declared and thrown.

Not calling get on list of futures

I'm using a global Executor service with some fixed thread pool size. We have bunch of related tasks that we submit for execution and wait on list of futures.
Recently, we faced a high CPU utilization issue and on debugging I found that an exception occurred while calling get() on one of the item in list of futures. Current, we iterate over the list and there is a try catch surrounding the whole loop.
try{
List<Result> results = new ArrayList<>()
for(Future<Result> futureResult: futureResults{
Result result = futureResult.get();
results.add(result);
}
} catch(Exception e){
throw new InternalServiceException(e);
}
//Do something with results
Wanted to know the behaviour of other threads if get is never called on some of the items in future. I tried searching but was not able to find anything.
Also, can this behaviour trigger high CPU utilization ?

http://www.journaldev.com/1650/java-futuretask-example-program
I would still check if the future isDone as in the example above.
If you need to run other operations or want to utilize the CPU better then I would put the collector in a separate thread and perhaps just poll for results every minute or so.
Could be scheduled or handled by Thread.sleep.

Executors class provides various methods to execute Callable in a thread pool. Since callable tasks run in parallel, we have to wait for the returned Object.
Callable tasks return java.util.concurrent.Future object. Using Future we can find out the status of the Callable task and get the returned Object.
It provides get() method that can wait for the Callable to finish and then return the result.
There is an overloaded version of get() method where we can specify the time to wait for the result, it’s useful to avoid current thread getting blocked for longer time.
Future provides cancel() method to cancel the associated Callable task. There are isDone() and isCancelled() methods to find out the current status of associated Callable task.
Here is a simple example of Callable task that returns the name of thread executing the task after some random time.
We are using Executor framework to execute 10 tasks in parallel and use Future to get the result of the submitted tasks.
public class FutureObjectTest implements Callable<String>{
#Override
public String call() throws Exception {
long waitTime = (long) (Math.random()*10000);
System.out.println(Thread.currentThread().getName() + " waiting time in MILISECONDS " + waitTime);
Thread.sleep(waitTime);
return Thread.currentThread().getName() + " exiting call method.";
}
public static void main(String [] args){
List<Future<String>> futureObjectList = new ArrayList<Future<String>>();
ExecutorService executorService = Executors.newFixedThreadPool(5);
Callable<String> futureObjectTest = new FutureObjectTest();
for(int i=0; i<10; i++){
Future<String> futureResult = executorService.submit(futureObjectTest);
futureObjectList.add(futureResult);
}
for(Future<String> futureObj : futureObjectList){
try {
System.out.println(futureObj.get());
} catch (InterruptedException | ExecutionException e) {
e.printStackTrace();
}
}
System.out.println("Starting get method of wait");
////////////get(Timeout) method///////
futureObjectList.clear();
for(int i=0; i<10; i++){
Future<String> futureResult = executorService.submit(futureObjectTest);
futureObjectList.add(futureResult);
}
executorService.shutdown();
for(Future<String> futureObj : futureObjectList){
try {
System.out.println(futureObj.get(2000,TimeUnit.MILLISECONDS));
} catch (InterruptedException | ExecutionException | TimeoutException e) {
e.printStackTrace();
}
}
}
}