Is there any way to handle exceptions in a Flux parallel in case N of X rails fail? I have tried with the onErrorMap, onErrorReturn, and with this try catch, but it keeps throwing error even if all the others are ok, because it is going to the catch of the processRequest method.
protected Object processRequest(RequestHolder requestHolder) {
RequestHolderImpl requestHolderImpl = (RequestHolderImpl) requestHolder;
try {
if (requestHolderImpl.getPayload().getClass().equals(LinkedList.class)) {
payload.addAll((List<DataSourceRequest>) requestHolderImpl.getPayload());
} else {
payload.add((DataSourceRequest) requestHolderImpl.getPayload());
}
List<PurposeResponse> response = Flux.fromIterable(payload)
.parallel()
.flatMap(request -> {
try {
return dataSourceCall(request);
} catch (WebClientResponseException e) {
return Mono.just(new PurposeResponse(request.getPurpose(), buildResponseFromException(e, request.getPurpose())));
} catch (Exception e) {
LOGGER.error("No response could be obtained from DS. Exception thrown: {}", e.getMessage());
return Mono.just(new PurposeResponse(request.getPurpose(), new DataSourceException(e)));
}
})
.sequential()
.collectList()
.block();
return new ResponseHolderImpl(response, products);
} catch (Exception e) {
return new DataSourceException(e.getMessage());
}
}
private Mono<PurposeResponse> dataSourceCall(DataSourceRequest purpose) {
RequestHolder requestHolder = new RequestHolderImpl(purpose,
data,
products,
token);
String purposeName = getPurposeName(requestHolder);
RequestEntity<?> requestEntity = createRequestEntity(requestHolder);
LOGGER.info("Sending request to this url: {}", requestEntity.getUrl());
return webClient.get()
.uri(requestEntity.getUrl())
.header("Authorization", "Bearer " + token)
.retrieve()
.bodyToMono(JsonNode.class)
.elapsed()
.map(data -> {
LOGGER.info("Response took {} milliseconds", data.getT1());
if (LOGGER.isDebugEnabled()) {
LOGGER.debug("Response obtained from Data Source: {}", data.getT2());
}
return new PurposeResponse(purposeName, data.getT2());
});
}
private Object buildResponseFromException(WebClientResponseException e, String purposeName) {
//do things
}
Related
I have following proactive method.
void acceptSome(Consumer<? super String> consumer) {
consumer.accept("A");
consumer.accept("B");
}
And I wrote following method for reactor.
void emitAll(Sinks.Many<String> sink) {
try {
acceptSome(e -> {
sink.emitNext...(e);
});
sink.emitComplete...
} catch (Exception e) {
sink.emitError...
}
}
And I tested with following two methods, one with full buffer and the other with single buffer using thread.
#Test
void emitAll__onBackpressureBufferAll() {
final var sink = Sinks.many().unicast().<String>onBackpressureBuffer();
TheClass.emitAll(sink);
sink.asFlux()
.doOnNext(e -> {
log.debug("next: {}", e);
})
.blockLast();
}
#Test
void emitAll__onBackpressureBufferOne() {
var sink = Sinks.many().unicast().<String>onBackpressureBuffer(new ArrayBlockingQueue<>(1));
new Thread(() -> {
sink.asFlux()
.doOnNext(e -> {
log.debug("next: {}", e);
})
.blockLast();
}).start();
TheClass.emitAll(sink);
}
Now, How can I (Can I do that?) implement a method accepts a sink and returns a Flux or CompletableFuture<Flux> so that caller simply subscribe to the result without thread, with minimum buffer?
CompletableFuture<Flux<String>> emitAllAsync(Sinks.Many<String> sink) {
}
Thank you.
I tried some and it works yet doesn't seem righteous.
void emitAll(Sinks.Many<String> sink, Semaphore semaphore) {
try {
acceptSome(v -> {
try {
semaphore.acquire();
} catch (final InterruptedException ie) {
Thread.currentThread().interrupt();
throw new RuntimeException(ie);
}
log.debug("emitting {}", v);
sink.tryEmitNext(v).orThrow();
});
sink.tryEmitComplete().orThrow();
} catch (final IOException ioe) {
log.error("failed to emit to {}", sink, ioe);
sink.tryEmitError(ioe).orThrow();
}
}
CompletableFuture<Flux<String>> emitAllAsync() {
var sink = Sinks.many().unicast().<String>onBackpressureBuffer(new ArrayBlockingQueue<>(1));
var semaphore = new Semaphore(1);
CompletableFuture
.runAsync(() -> emitAll(sink, semaphore));
return CompletableFuture.completedFuture(sink.asFlux().doOnNext(v -> semaphore.release()));
}
I am receiving a request in a 0.RestController and give it to the service for processing. If service doesnt throw exception, i just return HttpStatus.200, but if an exception occurs in the service, i need catch it in controller and return the status depending on the exception.
Inside service, i need to use Mono.fromCallable for repo access. Well, if user not found, i try throw my CustomException, but i cant cach it in controller. What am I doing wrong?
Controller:
#RestController
#RequiredArgsConstructor
#Slf4j
public class CardStatusController {
private final CardStatusService cardStatusService;
#PostMapping(value = "api/{user_id}/confirm", produces = {MediaType.APPLICATION_JSON_VALUE})
public Mono<ResponseEntity<HttpStatus>> confirmStatus(
#PathVariable("user_id") String userId,
#RequestBody CardStatusRequest statusRequest) {
statusRequest.setUserId(userId);
cardStatusService.checkCardStatus(statusRequest);
return Mono.just(new ResponseEntity<>(HttpStatus.OK));
}
#ExceptionHandler({ CheckCardStatusException.class })
public void handleException() {
log.error("error!");
}
My service:
public Mono<Void> checkCardStatus(CardStatusRequest statusRequest) throws CheckCardStatusException {
if (statusRequest.getCardEmissionStatus().equals(CardEmissionStatus.ACCEPTED)) {
String reference = statusRequest.getReference();
return Mono.fromCallable(() -> userRepository.findById(statusRequest.getUserId()))
.subscribeOn(Schedulers.boundedElastic())
.switchIfEmpty(Mono.error(new CheckCardStatusException(HttpStatus.NOT_FOUND)))
.flatMap(user -> Mono.fromCallable(() -> cardRepository.findFireCardByUserId(user.getId()))
.flatMap(optionalCard -> {
if (optionalCard.isPresent()) {
if (optionalCard.get().getExtId().isEmpty()) {
Card card = optionalCard.get();
card.setExtId(reference);
try {
cardRepository.save(card);
} catch (Exception e) {
return Mono.error(new CheckCardStatusException(HttpStatus.INTERNAL_SERVER_ERROR));
}
} else {
if (!optionalCard.get().getExtId().equals(reference)) {
return Mono.error(new CheckCardStatusException(HttpStatus.CONFLICT));
}
}
} else {
Card card = new Card();
//set card params
try {
cardRepository.save(card);
} catch (Exception e) {
return Mono.error(new CheckCardStatusException(HttpStatus.INTERNAL_SERVER_ERROR));
}
}
return Mono.error(new CheckCardStatusException(HttpStatus.OK));
})).then();
}
else {
return Mono.error(new CheckCardStatusException(HttpStatus.OK));
}
}
}
My CustomException:
#AllArgsConstructor
#Getter
public class CheckCardStatusException extends RuntimeException {
private HttpStatus httpStatus;
}
The Mono returned by checkCardStatus is never subscribed, so the error signal is ignored. You have to return the whole chain to the Webflux framework as follows:
public Mono<ResponseEntity<HttpStatus>> confirmStatus(
#PathVariable("user_id") String userId,
#RequestBody CardStatusRequest statusRequest) {
statusRequest.setUserId(userId);
return cardStatusService.checkCardStatus(statusRequest)
.then(Mono.just(new ResponseEntity<>(HttpStatus.OK)));
}
In case of an error, the corresponding ExceptionHandler will be executed.
I want to retrieve data of different types from a database and return to the user within an HTTP result from a Spring Boot service. Because the database retrieval takes a significant amount of time for each, I am making these DB calls asynchronously with CompletableFuture. The pattern I have works and saves time compared to doing this synchronously, but I feel that it can and should be laid out in a cleaner fashion.
I edited the code to change the types to 'PartA', 'PartB', 'PartC', but this is otherwise how it appears. Currently, the service accepts the lists of different types (PartA, PartB, PartC), creates Completable future types of each list calling its own CompletableFuture method that calls the DB, builds a generic list of CompleteableFutures with each type, "gets" the generic list, then adds all the contents of each Future list to the list passed into the service.
This is how the Service methods are coded:
Service.java:
public void metadata(final List<PartA> partAs,final List<PartB> partBs,final List<PartC> partCs,
String prefix,String base,String suffix) throws Exception {
try {
CompletableFuture<List<PartA>> futurePartAs = partACompletableFuture(prefix,base,suffix).thenApply(list -> {
logger.info("PartA here");
return list;
});
CompletableFuture<List<PartB>> futurePartBs = partBCompletableFuture(prefix,base,suffix).thenApply(list -> {
logger.info("PartBs here");
return list;
});
CompletableFuture<List<PartC>> futurePartCs = partCCompletableFuture(prefix,base,suffix).thenApply(list -> {
logger.info("PartCs here");
return list;
});
CompletableFuture<?> combinedFuture = CompletableFuture.allOf(CompletableFuture.allOf(futurePartAs, futurePartBs, futurePartCs));
combinedFuture.get();
partAs.addAll(futurePartAs.get());
partBs.addAll(futurePartBs.get());
partCs.addAll(futurePartCs.get());
} catch (Exception e) {
logger.error("Exception: ", e);
throw e;
}
}
#Async("asyncExecutor")
public CompletableFuture<List<PartA>> partACompletableFuture(String prefix,String base,String suffix) {
return CompletableFuture.supplyAsync(() -> {
try {
logger.info("start PartA");
return getPartAs(prefix,base,suffix);
} catch (Exception e) {
logger.error("Exception: ", e);
throw e;
}
});
}
#Async("asyncExecutor")
public CompletableFuture<List<PartB>> partBCompletableFuture(String prefix,String base,String suffix) {
return CompletableFuture.supplyAsync(() -> {
try {
logger.info("start B");
return getPartBs(prefix,base,suffix);
} catch (Exception e) {
logger.error("Exception: ", e);
throw e;
}
});
}
#Async("asyncExecutor")
public CompletableFuture<List<PartC>> partCCompletableFuture(String prefix,String base,String suffix) {
return CompletableFuture.supplyAsync(() -> {
try {
logger.info("start PartC");
return getPartCs(prefix,base,suffix);
} catch (Exception e) {
logger.error("Exception: ", e);
throw e;
}
});
}
In case you wish to view the Controller and Response type:
Controller.java
#GetMapping(value="/parts/metadata",produces = { MediaType.APPLICATION_JSON_VALUE })
public ResponseEntity<MetadataResponse> metadata (#ApiParam(name="prefix",value = "Prefix value for a part",required = false)
#RequestParam(required=false) String prefix,
#ApiParam(name="base",value = "Base value for a part",required= true)
#RequestParam String base,
#ApiParam(name="suffix",value = "Suffix value for a part",required=false)
#RequestParam(required=false) #NotBlank String suffix ) throws Exception {
final List<PartA> partAs = new ArrayList<>();
final List<PartB> partBs = new ArrayList<>();
final List<PartC> partCs = new ArrayList<>();
service.metadata(partAs,partBs,partCs,prefix,base,suffix);
MetadataResponse.MetadataResponseResult res = MetadataResponse.MetadataResponseResult.builder()
.partAs(partAs)
.partBs(partBs)
.partCs(partCs)
.build();
return ResponseEntity.ok(MetadataResponse.result(res, MetadataResponse.class));
}
MetadataResponse.java
#ApiModel(value = "MetadataResponse", parent = BaseBodyResponse.class, description = "Part A, B, C")
public class MetadataResponse extends BaseBodyResponse<MetadataResponse.MetadataResponseResult> {
#Data
#Builder
#NoArgsConstructor
#AllArgsConstructor
#ApiModel(value = "MetadataResponseResult", description = "This Model holds Part As, Bs, Cs")
public static class MetadataResponseResult {
List<PartA> partAs;
List<PartB> partBs;
List<PartC> partCs;
}
}
I don't understand exactly why you need to pass all these lists as parameters in this case: public void metadata(final List<PartA> partAs,final List<PartB> partBs,final List<PartC> partCs, String prefix,String base,String suffix) throws Exception You could modify this method to return the MetadataResponseResult class you already have and use the lists from the ComparableFutures directly
I would remove the thenApply methods since you just log a statement and you don't actually change the results.
Instead of having the three methods (partACompletableFuture, partABCompletableFuture, partCCompletableFuture) you could have one method that receives a Supplier as a parameter.
#Async("asyncExecutor")
public <T> CompletableFuture<T> partCompletableFuture(Supplier<T> supplier) {
return CompletableFuture.supplyAsync(() -> {
try {
logger.info("start Part");
return supplier.get();
} catch (Exception e) {
logger.error("Exception: ", e);
throw e;
}
});
}
Aftewards you can use it as so:
CompletableFuture<List<PartA>> futurePartAs = partCompletableFuture(() ->
getPartAs(prefix,base,suffix));
It should much cleaner. Hope this helped!
I have a method:
public void getVmsAdminToken(HttpClient httpClient, handler<AsyncResult<String>> handler) {
httpClient.postAbs(url, h -> h.bodyHandler(bh -> {
try {
switch (h.statusCode()) {
case 200:
JsonObject vmsResponse = bh.toJsonObject();
handler.handle(Future.succeededFuture(Json.encode(vmsResponse)));
break;
default:
LOG.error("VMS call failed {}", h.statusCode());
handler.handle(Future.failedFuture(500 + ""));
break;
}
} catch (Throwable t) {
LOG.error("Exception in getVmsAdminToken", t);
handler.handle(Future.failedFuture(500 + ""));
}
}))
.setTimeout(timeOutMs)
.putHeader("content-type", "application/json")
.putHeader("stub", apiKey)
.end(vehicleReqBody.encode());
}
I use this inside the following method call :
private void getAdminToken(RoutingContext ctx, RedisFleetStorage storage, HttpClient httpClient) {
getVmsAdminToken(fleetId, user, vehicle, httpClient, replyVms -> {
if (reply.succeeded()) {
// why succeeded?!!
}
});
}
And even if the getVmsToken fails, the execution falls into the if (reply.succeeded())
Why might that be?
You should check the same AsyncResult object being the result of your HTTP call:
private void getAdminToken(RoutingContext ctx, RedisFleetStorage storage, HttpClient httpClient) {
getVmsAdminToken(fleetId, user, vehicle, httpClient, replyVms -> {
if (replyVms.succeeded()) {
// do you thing
}
});
}
On the change "SortBy", my program will do a NetworkIO to retrieve the top movies and display them.
However, it seems that though I have done subscribeOn(Schedulers.io()), the NetworkIO MovieDB.getPopular() and MovieDB.getTopRated() in the function call in map are excuted on the main thread and I get a android.os.NetworkOnMainThreadException.
I was wondering how to make the public Movie[] call(SortBy sortBy) asynchronous.
sortObservable.map(new Func1<SortBy, Movie[]>() {
#Override
public Movie[] call(SortBy sortBy) {
try {
switch (sortBy) {
case POPULAR:
return MovieDB.getPopular(); // NETWORK IO
case TOP_RATED:
return MovieDB.getTopRated(); // NETWORK IO
}
} catch (IOException e) {
e.printStackTrace();
} catch (JSONException e) {
e.printStackTrace();
}
return new Movie[0];
}
})
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(new Action1<Movie[]>() {
#Override
public void call(Movie[] movies) {
imageAdapter.loadData(movies);
}
});
Please check if the below works for you. It uses flatMap instead of map.
sortObservable.flatMap(new Func1<SortBy, Observable<Movie[]>>() {
#Override
public Observable<Movie[]> call(SortBy sortBy) {
try {
switch (sortBy) {
case POPULAR:
return Observable.just(MovieDB.getPopular()); // NETWORK IO
case TOP_RATED:
return Observable.just(MovieDB.getTopRated()); // NETWORK IO
}
} catch (IOException e) {
e.printStackTrace();
} catch (JSONException e) {
e.printStackTrace();
}
return Observable.just(new Movie[0]);
}
}).subscribe(new Action1<Movie[]>() {
#Override
public void call(Movie[] movies) {
imageAdapter.loadData(movies);
}
});
From your source code on Github, it seems like you are using synchronous mode of executing requests using OkHttp. OkHttp also supports asynchronous requests and that can be preferred. Below would be the changes required in few of the methods.
run method should consume enqueue instead of execute.
Observable<String> runAsync(String url){
return Observable.create(subscriber -> {
Request request = new Request.Builder().url(url).build();
client.newCall(request).enqueue(new Callback() {
#Override
public void onResponse(Call call, Response response) throws IOException {
subscriber.onNext(response.body().string());
}
#Override
public void onFailure(Call call, IOException e) {
subscriber.onError(e);
}
});
});
}
getApi can return an Observable<Movie[]> instead of Movie[]
public Observable<Movie[]> getApiAsync(String type){
return runAsync("http://api.themoviedb.org/3/movie/" + type
+ "?api_key=412e9780d02673b7599233b1636a0f0e").flatMap(response -> {
Gson gson = new Gson();
Map<String, Object> map = gson.fromJson(response,
new TypeToken<Map<String, Object>>() {
}.getType());
Movie[] movies = gson.fromJson(gson.toJson(map.get("results")),
Movie[].class);
return Observable.just(movies);
});
}
Finally I sort it out by myself:
sortObservable.flatMap(new Func1<SortBy, Observable<Movie[]>>() {
#Override
public Observable<Movie[]> call(SortBy sortBy) {
switch (sortBy) {
case POPULAR:
return Observable.fromCallable(() -> MovieDB.getPopular()).subscribeOn(Schedulers.io());
case TOP_RATED:
return Observable.fromCallable(() -> MovieDB.getTopRated()).subscribeOn(Schedulers.io());
default:
return Observable.fromCallable(() -> new Movie[0]).subscribeOn(Schedulers.io());
}
}
})
.observeOn(AndroidSchedulers.mainThread())
.subscribe(new Action1<Movie[]>() {
#Override
public void call(Movie[] movies) {
imageAdapter.loadData(movies);
}
});