I have a function that goes through a list of HappyObjects and sets their fields asynchronously. In the Callable, a JsonProcessingException can occur. I have to wrap this and other exceptions from this function into a custom exception (ControllerException) and throw that instead.
Other Stack Overflow posts seem to suggest collect into a List of Futures and use get() to catch the exceptions. Thus, this is what I have so far:
default List<HappyObj> fillfunction(final List<HappyObj> happyObjs) throws ControllerException {
ThreadPoolExecutor executor = (ThreadPoolExecutor) Executors.newCachedThreadPool();
List<Future<HappyObj>> futures = new ArrayList<>();
for (HappyObj happyObj : happyObjs) {
Future<HappyObj> future = executor.submit(
() -> {
final List<Mood> moods = getMoods();
for (Mood mood : moods) {
final String json = getJsonEmotion();
ObjectMapper mapper = new ObjectMapper();
mapper.setVisibility(PropertyAccessor.FIELD, JsonAutoDetect.Visibility.ANY);
List<Emotion> emotions =
mapper.readValue(json, new TypeReference<List<Emotion>>() {}); //JsonProcessingException can occur here
MoodMetadata metadata = mood.getMoodMetadata();
if (metadata != null && metadata.getEmotionMetadata() != null) {
metadata.getEmotionMetadata().setEmotions(emotions);
}
}
happyObj.setMoods(moods);
return happyObj;
});
futures.add(future);
}
executor.shutdown();
final long maxSlaSec = 1;
try {
executor.awaitTermination(maxSlaSec, TimeUnit.SECONDS);
List<HappyObj> happyResult = new ArrayList<>();
for (Future<HappyObj> future : futures) {
happyResult.add(future.get());
}
return happyResult;
} catch (InterruptedException | ExecutionException e) {
executor.shutdownNow();
throw new ControllerException(e);
}
}
Is there a more elegant way than iterating through List<Future> and calling get on each to catch ExecutorException? I thought about using execute() vs. submit(), but then I can't handle the JsonProcessingException. I saw another post suggesting creating a ThreadPoolExecutor subclass and override the afterExecute(), but I wasn't able to handle the JsonProcessingException.
One of the reasons I asked this question is because since this method consists mainly of setters, the function was originally manipulating the given objects and returning void.
According to the docs of ExecutionException (and also the docs of Future#get), it already has wrapped that information. That is, you can use its getCause to inspect the Exception thrown by the Callable's body.
Notice that Callable#call itself throws an Exception... When you throw an Exception from the Callable, it will be wrapped into an ExecutionException which will be thrown from the Future#get method, for each Callable, meaning that you can change your loop to catch an ExecutionException for each Future and inspect its getCause!
So you don't actually need to wrap it to a custom ControllerException.
The Callables you create, can still return null of type Void for example, without needing to do anything about them.
Unless the scenario changes, then you don't need to extend ThreadPoolExecutor in this case. You don't even have to cast to ThreadPoolExecutor, because the ExecutorService interface already has the submits you need. Just throw whatever Exception you need from the Callable (such as JsonProcessingException that you mentioned) when something goes wrong in the Callable, and then inspect the ExecutionException from each Future#get method to tell if an JsonProcessingException was thrown (and you can also determine in which one it was thrown, if you need).
Is there a more elegant way than iterating through List and calling get on each to catch ExecutorException?
In my opinion, no, there is not, because you want to submit all Callables first, then let them run in parallel, and at the end inspect their ExecutionException for any Exception thrown by the Callables' body for each Callable (via Future#get of the returned Future by submit).
Related
I've been learning about concurrency and the streams API and came across this. The offerLast()method can throw InterruptedException, so I get that I must handle it. What I don't get is why can't I throw it at the method level by adding throws Exception?. As it is this code does not compile.
static BlockingDeque<Integer> queue = new LinkedBlockingDeque<>();
public static void testing() throws Exception {
IntStream.iterate(1, i -> i+1).limit(5)
.parallel()
.forEach(s -> queue.offerLast(s, 10000, TimeUnit.MILLISECONDS));
}
I know it can be solved by surrounding it in a try/catch, or by creating a wrapper method that handles the error, but I'm still trying to understand why it can't be thrown at the method level.
Because lambda expressions are not always evaluated immediately.
Let's you have this:
public Supplier<String> giveMeASupplier() throws Exception {
return () -> someMethodThatThrowsCheckedException()
}
According to you, the above would work. Right?
Now in another method, I can do this:
Suppler<String> supplier = null;
try {
supplier = giveMeASupplier() // no exception is thrown here.
} catch (Exception ex) {
ex.printStackTrace();
}
if (supplier != null) {
System.out.println(supplier.get()); // this might throw an exception! Yet it's not in a try...catch!
}
Now what do you think would happen if supplier.get() throws an exception? Is there anything to catch it? No. If somehow the catch block a few lines before gets run, then it would be really weird.
The simple answer is that the "method" you're referring to is Consumer.accept, not YourClass.testing.
The lambda s -> queue.offerLast(s, 10000, TimeUnit.MILLISECONDS) is an implementation of java.util.function.Consumer.accept(T), which doesn't declare that it can throw InterruptedException.
And this behavior is not particular to streams, wherever a lambda expression is defined, it must comply with the signature of the abstract method of the functional interface it implements.
I used to have a callable class
class SampleTask implements Callable<Double> {
#Override
public Double call() throws Exception {
return 0d;
}
}
I used to use ExecutorService to submit the Callable. How to change to use CompletableFuture.supplyAsync?
The following code cannot compile
SampleTask task = new SampleTask();
CompletableFuture.supplyAsync(task);
No instance of type of variable U exists so that SampleTask conforms to Supplier
For your callable as written, you could simply use CompletableFuture.supplyAsync(() -> 0d);.
If, however, you have an existing Callable, using it with CompletableFuture is not so straight-forward due to the checked exceptions that a callable might throw.
You may use an ad-hoc Supplier which catches exceptions and re-throws it wrapped in an unchecked exception like
CompletableFuture.supplyAsync(() -> {
try { return callable.call(); }
catch(Exception e) { throw new CompletionException(e); }
})
Using the specific type CompletionException instead of an arbitrary subtype of RuntimeException avoids getting a CompletionException wrapping a runtime exception wrapping the actual exception when calling join().
Still, you’ll notice the wrapping when chaining an exception handler to the CompletableFuture. Also, the CompletionException thrown by join() will be the one created in the catch clause, hence contain the stack trace of some background thread rather than the thread calling join(). In other words, the behavior still differs from a Supplier that throws an exception.
Using the slightly more complicated
public static <R> CompletableFuture<R> callAsync(Callable<R> callable) {
CompletableFuture<R> cf = new CompletableFuture<>();
CompletableFuture.runAsync(() -> {
try { cf.complete(callable.call()); }
catch(Throwable ex) { cf.completeExceptionally(ex); }
});
return cf;
}
you get a CompletableFuture which behaves exactly like supplyAsync, without additional wrapper exception types, i.e. if you use
callAsync(task).exceptionally(t -> {
t.printStackTrace();
return 42.0;
})
t will be the exact exception thrown by the Callable, if any, even if it is a checked exception. Also callAsync(task).join() would produce a CompletionException with a stack trace of the caller of join() directly wrapping the exception thrown by the Callable in the exceptional case, exactly like with the Supplier or like with runAsync.
supplyAsync() expects a Supplier<U> and you are giving it a Callable.
The error message is telling you that the compiler has tried to find a type to use for U such that your SampleTask "is a" Supplier<U>, but it can't find one.
Java will implicitly "promote" a lambda to a functional interface such as Callable or Supplier. But it won't treat functional interfaces as interchangeable -- that is, you can't use a Callable where a Supplier is expected.
You can make a suitable lambda in-place:
SimpleTask task = new SimpleTask();
CompletableFuture.supplyAsync(() -> task.call());
Note that this works if SimpleTask's call() is:
public Double call() { // note no exception declared
return 0d;
}
The fact that SimpleTask happens to implement Callable is not relevant to the code above.
If you want this to work with an arbitrary Callable, or if you declare task as a Callable:
Callable callable = new SimpleTask();
CompletableFuture.supplyAsync(() -> callable.call());
... then you will get a compiler error about the uncaught exception. Your lambda will need to catch the exception and handle it (perhaps rethrowing as an unchecked exception, as described in other answers).
Or you could make SampleTask implement Supplier<Double>.
Part of the motivation for lambdas is that writing things like Callable was too verbose. So you might well leave out the intermediate class and go directly for:
CompleteableFuture<Double> future = CompletableFuture.supplyAsync(() -> 0d);
This applies for more complicated suppliers too:
CompleteableFuture<Double> future = CompletableFuture.supplyAsync(() -> {
Foo foo = slowQuery();
return transformToDouble(foo);
});
Since CompleteableFuture::supplyAsync expects a Supplier<Double> and not Callable<Double> you should go with:
Callable<Double> task = new SampleTask();
CompletableFuture.supplyAsync(() -> {
try {
return task.call();
} catch (Exception e) {
throw new RuntimeException(e);
}
});
I had this come up recently and used Vavr to solve it (was already using it for other things, too), and it worked out great for me:
CompletableFuture.supplyAsync( () -> Try.ofCallable( callable ).get() )
Or to get a Supplier of that CompletableFuture:
() -> CompletableFuture.supplyAsync( () -> Try.ofCallable( callable ).get() )
In all cases I tested this returned exactly and threw exactly what the callable itself did.
What's the prefered method to access the underlying Callable after a CancellationException exception of a FutureTask.get?
I have the following code -
public class Ping implements Callable
{
public Ping(String serverName)
{
// stuff
}
// call() method, etc.
}
// other code
futures = executor.invokeAll(callables, TIMEOUT_SECONDS, TimeUnit.SECONDS);
for (Future<Object> future : futures)
{
try
{
PingStatus status = (PingStatus)future.get();
// do stuff
}
catch (CancellationException e)
{
// HELP: throw new RuntimeException("Could not ping " + callable.serverName);
}
}
If the timeout is reached, and a CancellationException is throwing on the get, I'd like to throw a new exception that includes the serverName passed into the Callable. What's the best pattern here? And, why doesn't FutureTask provide a reference to the underlying Callable passed in on the constructor?
The original Callable is not retrievable from the Future object due to the separation between the task and the result. There are many methods which would result in Future objects being returned that do not involve the use or creation of Callable objects. Take for instance the method submit(Runnable task) which takes a runnable instead.
Runnable and Callable do not share a common parent superclass meaning that if the future object were to give you the ability to retrieve it, it would have to return an object of type Object. And that is just plain ugly.
Fortunately, if you have read the documentation for the returned list in invokeAll() (emphasis mine):
a list of Futures representing the tasks, in the same sequential order as produced by the iterator for the given task list.
meaning that the order of the input collection of Callable to the returned list of Future is preserved. By this, you can use the current index of Future to work out which Callable was cancelled.
E.g:
futures = executor.invokeAll(callables, TIMEOUT_SECONDS, TimeUnit.SECONDS);
int index = 0; // Index used for for-loop
for (Future<Object> future : futures){
try{
PingStatus status = (PingStatus)future.get();
// do stuff
}catch(CancellationException e){
Callable<Object> offendingCallable = callables.get(index);
// deal with object here
}
index++;
}
As a side note, it seems like you are returning an object PingStatus as a result from the execution. You should thus declare your custom callable as:
public class Ping<PingStatus> implements Callable{
and your appropriate future objects as Future<PingStatus> to avoid nasty casts from Object to PingStatus.
I wanted to prototype an example where I call a ServiceC using a value returned by ServiceA using Spring Reactor Stream API. So I wrote code like this
final ExecutorService executor = new ThreadPoolExecutor(4, 4, 10, TimeUnit.MINUTES, new LinkedBlockingQueue<Runnable>());
Streams.defer(executor.submit(new CallToRemoteServiceA()))
.flatMap(s -> Streams.defer(executor.submit(new CallToRemoteServiceC(s))))
.consume(s -> System.out.println("End Result : " + s));
To simulate the latency involved in ServiceA and ServiceC the call() methods of CallToRemoteServiceA and CallToRemoteServiceC has Thread.sleep() methods. The problem is that when I comment out the Thread.sleep() method i.e. the service method calls have no latency which is not true in the real world the consume method gets called. If the Thread.sleep() methods are kept in place then the consume method doesn't get called. I understand that the Streams.defer() returns a cold stream and hence it probably only executes the consume method for items accepted after it's registration but then I was wondering how I could create a HotStream from a Future returned by the ExecutorService?
I believe this is because of a bug in the reactor.rx.stream.FutureStream.subscribe() method. In this line:
try {
// Bug in the line below since unit is never null
T result = unit == null ? future.get() : future.get(time, unit);
buffer.complete();
onNext(result);
onComplete();
} catch (Throwable e) {
onError(e); <-- With default constructor this gets called if time == 0 and
future has as yet not returned
}
In this case when the default FutureStream(Future) constructor is called the unit is never null and hence the above code always calls future.get(0, TimeUnit.SECONDS) leading to an immediate timeout exception in the catch(Throwable) block. If you guys agree that this is a bug I can make a pull request with a fix for this issue??
I think what you want is to use Streams.just. You can optionally .dispatchOn(Dispatcher) if you want, but since you're already in the thread of the thread pool, you'll probably want to use the sync Dispatcher. Here's a quick test to illustrate:
#Test
public void streamsDotJust() throws InterruptedException {
ExecutorService executor = Executors.newSingleThreadExecutor();
Streams
.just(executor.submit(() -> "Hello World!"))
.map(f -> {
try {
return f.get();
} catch (Exception e) {
throw new IllegalStateException(e);
}
})
.consume(System.out::println);
Thread.sleep(100);
}
I am creating, say, 15 Callable tasks, and submitting them:
List<Future<MyResult>> futures = new ArrayList<Future<MyResult>>();
List<MyResult> myResults = new ArrayList<MyResult>();
for(int i = 1; i <= 15; i++){
Callable<MyResult> task = new MyProcessor(//parameters);
Future<MyResult> future = executorService.submit(task);
futures.add(future);//used to iterate over to call get() to collect results in next for loop
}
Then I am collecting the 15 MyResult objects:
for(Future<MyResult> future : futures){
try {
MyResult myResult = future.get();
processorResults.add(myResult);
} catch (InterruptedException e) {
//...
} catch (ExecutionException e) {
//...
}
}
Issue is: Instead of getting ALL 15 MyResult objects back from the get() method, I am sometimes getting less than 15 objects back. Sometime 12 sometimes 10 sometimes even lesser, and sometimes all 15.
I was under the impression that get() method is a blocking call and will wait for ALL 15 threads to get back with respective results, but looks like I miss a few of them and move ahead.
What is it that I am doing wrong? Am I not collecting the results/ waiting for the results correctly? Does this happen when an ERROR is thrown from any of the MyProcessor task?
It could mean that some of your jobs threw an exception. It is hard to tell from your code but you need to do something with ExecutionException other than catching and ignoring it.
Future.get() throws ExecutionException when the Callable you submitted throws a RuntimeException from the call() method. It will only return your MyResult if the method returns normally with return. You can get the exception that was thrown by doing:
} catch (ExecutionException e) {
// account for the throw here, the original exception is in e.getCause()
// log it, count it, or ...
logger.error("Job threw exception: " + e.getCause());
}
I was under the impression that get() method is a blocking call and will wait for ALL 15 threads to get back with respective results,
This is correct. When you call future.get() that will block until the job either finishes -- either by throwing an exception or returning. If the thread that is going the get() is interrupted then the get() throws an InterruptedException which also should be caught and not just ignored.