In the following code in Spring Webflux application, I am calling an endpoint "myfunction" which internally calls another endpoint. If the list contains 3 values, I will hit the "cancel" endpoint 3 times. Here is the question. I want to hit the endpoint one by one which means once I get response for 1st value in list then only I want to hit for second value and so on. I know it is reactive framework, still do we have any way to do without using delayElements.
#RestController
#RequestMapping("test")
#Slf4j
public class MyRestController {
private final WebClient webClient;
public MyRestController(WebClient webClient) {
this.webClient = webClient.mutate().baseUrl("http://localhost:7076/test/").build();
}
#GetMapping("/myfunction")
public void callTest() {
Flux.fromIterable(List.of("e1", "e2", "e3"))
//.delayElements(Duration.ofMillis(1000))
.flatMap(event -> {
log.info(event);
return sendCancelRequest(event);
}).subscribe(log::info);
}
public Mono<String> sendCancelRequest(String event) {
return webClient.get()
.uri(uriBuilder -> uriBuilder.path("cancel").queryParam("event", event).build())
.accept(MediaType.APPLICATION_JSON)
.retrieve()
.bodyToMono(String.class);
}
#GetMapping("/cancel")
public Mono<String> callMe(#RequestParam String event) {
//try{Thread.sleep(5000);}catch (Exception e){}
return Mono.just(event + " cancelled");
}
}
For example:
Once I get response for "e1" then only I wanna to call "e2" as sequence and response matters for subsequent values in the list. Please assist here guys!
Related
In the below code snippet, I am trying to match my request against a custom predicate. Upon the predicate being evaluated as false, I would like to send back a custom status code (403 Forbidden in the below snippet) instead of the default 404 that is being sent on predicate failure. Here's what I've tried.
RouteLocator
#Bean
public RouteLocator customRoutesLocator(RouteLocatorBuilder builder
AuthenticationRoutePredicateFactory arpf) {
return builder.routes()
.route("id1", r ->r.path("/app1/**")
.uri("lb://id1")
.predicate(arpf.apply(new Config()))).build();
}
AuthenticationRoutePredicateFactory
public class AuthenticationRoutePredicateFactory
extends AbstractRoutePredicateFactory<AuthenticationRoutePredicateFactory.Config> {
public AuthenticationRoutePredicateFactory() {
super(Config.class);
}
#Override
public Predicate<ServerWebExchange> apply(Config config) {
return (ServerWebExchange t) -> {
try {
Boolean isRequestAuthenticated = checkAuthenticated();
return isRequestAuthenticated;
}
} catch (HttpClientErrorException e) {
//This status code does not carried forward and 404 is displayed instead.
t.getResponse().setStatusCode(HttpStatus.FORBIDDEN);
return false;
}
};
}
#Validated
public static class Config {
public Config() {
}
}
private Boolean checkAuthenticated() {
// Some sample logic that makes a REST call and returns TRUE/FALSE/HttpClientErrorException
//Not shown here for simplicity.
return true;
}
}
When the predicate returns as true, the request is forwarded to the URI. However, on false evaluation 404 is displayed, I require 403 to be displayed (On HttpClientErrorException ). Is this the right way to expect a response with custom status code?. Additionally, I also read on implementing custom webfilters for a given route that can possibly modify the response object before forwarding the request. Is there a way to invoke a filter on predicate failure in that case?
As a newbie to spring cloud gateway, I chose the wrong direction towards approaching this problem.
Clients make requests to Spring Cloud Gateway. If the Gateway Handler Mapping determines that a request matches a route, it is sent to the Gateway Web Handler. Predicates aid the Gateway Handler Mapping in determining if that request matches the route and can only return either a true or false value.
Once the request matches the route, The handler runs the request through a filter chain that is specific to the request. This is where "pre" or "post" requests can be applied before forwarding a request.
Therefore In order to send a custom response status code before conditionally forwarding the request, One must write a custom "pre" filter and this can be achieved as below. (Setting 403 status code)
AuthenticationGatewayFilterFactory
#Component
public class AuthenticationGatewayFilterFactory
extends AbstractGatewayFilterFactory<AuthenticationGatewayFilterFactory.Config> {
public AuthenticationGatewayFilterFactory() {
super(Config.class);
}
#Override
public GatewayFilter apply(Config config) {
return (exchange, chain) -> {
try {
if(isRequestAuthenticated) {
return chain.filter(exchange);
}
else {
exchange.getResponse().setStatusCode(403); // Any status code can be set here.
return exchange.getResponse().complete();
}
} catch (HttpClientErrorException e) {
exchange.getResponse().setStatusCode(403); // Any status code can be set here.
return exchange.getResponse().complete();
}
};
}
public static class Config {
}
private Boolean isRequestAuthenticated(String authToken) {
// Some sample logic that makes a REST call and returns TRUE/FALSE/HttpClientErrorException
//Not shown here for simplicity.
return true;
}
RouteLocator
#Bean
public RouteLocator customRoutesLocator(RouteLocatorBuilder builder
AuthenticationGatewayFilterFactory agff) {
return builder.routes()
.route("id1", r ->r.path("/app1/**")
.uri("lb://id1")
.filter(agff.apply(new Config()))).build();
}
I have a Quarkus application with the following filters definition:
#ApplicationScoped
#Slf4j
public class Filters {
// some #Inject parameters i'm using
#ServerRequestFilter(preMatching = true)
public void requestLoggingFilter(ContainerRequestContext requestContext) {
log.info("Recv: [{}] {}, {}", requestContext.getHeaderString("myHeader"), requestContext.getMethod(), requestContext.getUriInfo().getRequestUri());
}
#ServerResponseFilter
public void responseBasicHeaderFilter(ContainerResponseContext responseContext) {
responseContext.getHeaders().putSingle("myHeader, "myValue");
}
#ServerResponseFilter
public void responseLoggingFilter(ContainerResponseContext responseContext) {
log.info("Sent: [{}] {} {}", responseContext.getHeaderString("myHeader"), , responseContext.getStatusInfo(), responseContext.getEntity());
}
}
And I have two tests:
Test Class config:
#QuarkusTest
public class MyTest {
...
}
Test A:
final Response response = given()
.post(BASE_URL)
.then()
.extract().response();
assertEquals(200, response.getStatusCode(), () -> "Got: " + response.prettyPrint());
assertEquals("myValue", response.getHeader("myHeader"));
final Response response2 = given()
.get(BASE_URL)
.then()
.extract().response();
assertEquals(200, response2.getStatusCode(), () -> "Got: " + response2.prettyPrint());
assertEquals("myValue", response2.getHeader("myHeader"));
Test B:
final Response response = given()
.post(BASE_URL)
.then()
.extract().response();
assertEquals(200, response.getStatusCode(), () -> "Got: " + response.prettyPrint());
assertEquals("myValue", response.getHeader("myHeader"));
If i run Test B on it's own, it passes.
If i run Test A however the last assertion fails (the header value is not there).
The #ServerResponseFilter seem to not be called beyond the first time, however #ServerRequestFilter seem to be fine.
I have tested the api manually and can confirm the same behaviour. Calling the GET request first will also have the same behaviour.
I have verified that the response generated by my Controller (pojo) is generated successfully.
What could be preventing it from being rerun?
Turns out it wasn't related to GET vs POST
my GET method was returning a Multi . I converted this to Uni> and it worked.
From the documentation i found this snippet
Reactive developers may wonder why we can't return a stream of fruits directly. It tends to eb a bad idea with a database....
The keyword being we can't so I imagine this is not supported functionality
I have the following fragment where I want to return a Flux from a ResponseEntity<Response>:
#GetMapping("/{id}")
public Mono<ResponseEntity<Response>> findByDocumentClient(#PathVariable("id") String document){
return Mono.just(new ResponseEntity<>(new Response(technomechanicalService.findByDocumentClient(document), HttpStatus.OK.value(), null),
HttpStatus.OK))
.onErrorResume(error -> {
return Mono.just(new ResponseEntity<>(new Response(null, HttpStatus.BAD_REQUEST.value(), error.getMessage()),
HttpStatus.BAD_REQUEST));
});
}
The Response object is as follows:
public class Response{
private Object body;
private Integer status;
private String descStatus;
public Response(Object body, Integer status, String descStatus) {
this.body = body;
this.status = status;
this.descStatus = descStatus;
}
}
When consuming the Get method from postman, the service responds to the following:
{
"body": {
"scanAvailable": true,
"prefetch": -1
},
"status": 200,
"descStatus": null
}
Why does it generate this response? Why is the list of objects not responding?
It's because you are trying to code imperatively (traditional java) and you are serializing a Mono and not the actually value returned from the database. You should be coding functionally as reactor/webflux uses this type of development.
A Mono<T> is a producer that produces elements when someone subscribes to it. The subscriber is the one that started the call, in this case the client/browser.
Thats why you need to return a Mono<ResponseEntity> becuase when the client subscribes it will emit a ResponseEntity
So lets Look at your code:
#GetMapping("/{id}")
public Mono<ResponseEntity<Response>> findByDocumentClient(#PathVariable("id") String document){
return Mono.just(new ResponseEntity<>(new Response(technomechanicalService.findByDocumentClient(document), HttpStatus.OK.value(), null),
HttpStatus.OK))
.onErrorResume(error -> {
return Mono.just(new ResponseEntity<>(new Response(null, HttpStatus.BAD_REQUEST.value(), error.getMessage()),
HttpStatus.BAD_REQUEST));
});
}
The first thing you do, is to put your response straight into a Mono using Mono#just. In webflux a Mono is something that can emit something and as soon as you put something in one you are also telling the server that it can freely change which thread that performs the execution. So we basically want to go into a Mono as quick as possible so we can leverage webflux thread agnostic abilities.
then this line:
technomechanicalService.findByDocumentClient(document)
returns a Mono<T> and you place that in your your Response body. So it tries to serialize that into json, while you think that it takes its internal Value and serializes that its actually serializing the Mono.
So lets rewrite your code *im leaving out the error handling for now since im writing this on mobile:
#GetMapping("/{id}")
public Mono<ServerResponse> findByDocumentClient(#PathVariable("id") String document){
// We place our path variable in a mono so we can leverage
// webflux thread agnostic abilities
return Mono.just(document)
// We access the value by flatMapping and do our call to
// the database which will return a Mono<T>
.flatMap(doc -> technomechanicalService.findByDocumentClient(doc)
// We flatmap again over the db response to a ServerResponse
// with the db value as the body
.flatMap(value -> ServerResponse.ok().body(value)));
}
All this is super basic reactor/webflux stuff. I assume this is your first time using webflux. And if so i highly recommend going through the Reactor getting started of how the basics work because otherwise you will have a very hard time with reactor, and later on understanding webflux.
Agree with #Toerktumlare's answer. Quite comprehensive.
#Juan David Báez Ramos based on your answer(better if it were a comment), seems like what you want is putting technomechanicalService.findByDocumentClient(document) result as body in a Response object.
If so you can use Flux API's collectList() operator.
Example code:
#GetMapping("/{id}")
public Mono<ResponseEntity<Response>> findByDocumentClient(#PathVariable("id") String document) {
return technomechanicalService.findByDocumentClient(document)
.collectList()
.map(
listOfDocuments -> {
return new ResponseEntity<>(
new Response(listOfDocuments, HttpStatus.OK.value(), null), HttpStatus.OK);
}
)
.onErrorResume(
error -> {
return Mono.just(new ResponseEntity<>(
new Response(null, HttpStatus.BAD_REQUEST.value(), error.getMessage()),
HttpStatus.BAD_REQUEST));
}
);
}
I am developing prototype for a new project. The idea is to provide a Reactive Spring Boot microservice to bulk index documents in Elasticsearch. Elasticsearch provides a High Level Rest Client which provides an Async method to bulk process indexing requests. Async delivers callbacks using listeners are mentioned here. The callbacks receive index responses (per requests) in batches. I am trying to send this response back to the client as Flux. I have come up with something based on this blog post.
Controller
#RestController
public class AppController {
#SuppressWarnings("unchecked")
#RequestMapping(value = "/test3", method = RequestMethod.GET)
public Flux<String> index3() {
ElasticAdapter es = new ElasticAdapter();
JSONObject json = new JSONObject();
json.put("TestDoc", "Stack123");
Flux<String> fluxResponse = es.bulkIndex(json);
return fluxResponse;
}
ElasticAdapter
#Component
class ElasticAdapter {
String indexName = "test2";
private final RestHighLevelClient client;
private final ObjectMapper mapper;
private int processed = 1;
Flux<String> bulkIndex(JSONObject doc) {
return bulkIndexDoc(doc)
.doOnError(e -> System.out.print("Unable to index {}" + doc+ e));
}
private Flux<String> bulkIndexDoc(JSONObject doc) {
return Flux.create(sink -> {
try {
doBulkIndex(doc, bulkListenerToSink(sink));
} catch (JsonProcessingException e) {
sink.error(e);
}
});
}
private void doBulkIndex(JSONObject doc, BulkProcessor.Listener listener) throws JsonProcessingException {
System.out.println("Going to submit index request");
BiConsumer<BulkRequest, ActionListener<BulkResponse>> bulkConsumer =
(request, bulkListener) ->
client.bulkAsync(request, RequestOptions.DEFAULT, bulkListener);
BulkProcessor.Builder builder =
BulkProcessor.builder(bulkConsumer, listener);
builder.setBulkActions(10);
BulkProcessor bulkProcessor = builder.build();
// Submitting 5,000 index requests ( repeating same JSON)
for (int i = 0; i < 5000; i++) {
IndexRequest indexRequest = new IndexRequest(indexName, "person", i+1+"");
String json = doc.toJSONString();
indexRequest.source(json, XContentType.JSON);
bulkProcessor.add(indexRequest);
}
System.out.println("Submitted all docs
}
private BulkProcessor.Listener bulkListenerToSink(FluxSink<String> sink) {
return new BulkProcessor.Listener() {
#Override
public void beforeBulk(long executionId, BulkRequest request) {
}
#SuppressWarnings("unchecked")
#Override
public void afterBulk(long executionId, BulkRequest request, BulkResponse response) {
for (BulkItemResponse bulkItemResponse : response) {
JSONObject json = new JSONObject();
json.put("id", bulkItemResponse.getResponse().getId());
json.put("status", bulkItemResponse.getResponse().getResult
sink.next(json.toJSONString());
processed++;
}
if(processed >= 5000) {
sink.complete();
}
}
#Override
public void afterBulk(long executionId, BulkRequest request, Throwable failure) {
failure.printStackTrace();
sink.error(failure);
}
};
}
public ElasticAdapter() {
// Logic to initialize Elasticsearch Rest Client
}
}
I used FluxSink to create the Flux of Responses to send back to the Client. At this point, I have no idea whether this correct or not.
My expectation is that the calling client should receive the responses in batches of 10 ( because bulk processor processess it in batches of 10 - builder.setBulkActions(10); ). I tried to consume the endpoint using Spring Webflix Client. But unable to work it out. This is what I tried
WebClient
public class FluxClient {
public static void main(String[] args) {
WebClient client = WebClient.create("http://localhost:8080");
Flux<String> responseFlux = client.get()
.uri("/test3")
.retrieve()
.bodyToFlux(String.class);
responseFlux.subscribe(System.out::println);
}
}
Nothing is printing on console as I expected. I tried to use System.out.println(responseFlux.blockFirst());. It prints all the responses as a single batch at the end and not in batches at .
If my approach is correct, what is the correct way to consume it? For the solution in my mind, this client will reside is another Webapp.
Notes: My understanding of Reactor API is limited. The version of elasticsearch used is 6.8.
So made the following changes to your code.
In ElasticAdapter,
public Flux<Object> bulkIndex(JSONObject doc) {
return bulkIndexDoc(doc)
.subscribeOn(Schedulers.elastic(), true)
.doOnError(e -> System.out.print("Unable to index {}" + doc+ e));
}
Invoked subscribeOn(Scheduler, requestOnSeparateThread) on the Flux, Got to know about it from, https://github.com/spring-projects/spring-framework/issues/21507
In FluxClient,
Flux<String> responseFlux = client.get()
.uri("/test3")
.headers(httpHeaders -> {
httpHeaders.set("Accept", "text/event-stream");
})
.retrieve()
.bodyToFlux(String.class);
responseFlux.delayElements(Duration.ofSeconds(1)).subscribe(System.out::println);
Added "Accept" header as "text/event-stream" and delayed Flux elements.
With the above changes, was able to get the response in real time from the server.
I have a very simple Webflux controller that just do a GET request to another service endpoint and returns a simple JSON list. The problem is the remote endpoint is always called twice.
This issue doesn't happen if I used Mono as the return type of the controller instead of Flux!
// This calls "/remote/endpoint" twice!
#GetMapping("/blabla")
fun controller() : Flux<JsonNode> {
return webClient.get()
.uri("/remote/endpoint")
.retrieve()
.bodyToMono(JsonNode::class.java)
.flatMapIterable { body ->
body.get("data")
}
}
// This calls "/remote/endpoint" once.
#GetMapping("/blabla")
fun controller() : Mono<JsonNode> {
return webClient.get()
.uri("/remote/endpoint")
.retrieve()
.bodyToMono(JsonNode::class.java)
.map { body ->
body.get("data")
}
}