Develop Reference

Apply retry to each and every endpoint in the spring webflux application

I have created below webclient and using it inside of my service to make HTTP third party calls.
#Configuration
public class WebclientConfig {
#Bean
public WebClient webClient() {
// custom client connector with connection pool disabled is being used as by default the connection pooling is done and connection reset happens after some idle time.
return WebClient.builder()
.clientConnector(new ReactorClientHttpConnector(HttpClient.newConnection()))
.defaultHeader(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE)
.defaultHeader(HttpHeaders.ACCEPT, MediaType.APPLICATION_JSON_VALUE)
.build();
}
}
and in my service, I am calling the third party service like below.
private Flux<BusinessObject> getBusinessObjects(String serviceURL) {
return this.webClient.get()
.uri(serviceURL)
.retrieve()
.bodyToFlux(BusinessObject.class) //code below this, do I have to copy for each webclient request to configure the retry, even if the values are same
.retryWhen(Retry.backoff(3, Duration.of(2, ChronoUnit.SECONDS))
.doBeforeRetry((value) -> log.info("Retrying request " + value))
.filter(error -> error instanceof WebClientRequestException)
.onRetryExhaustedThrow((retryBackoffSpec, retrySignal) ->
new RuntimeException(retrySignal.failure().getMessage())));
}
My question is as in comment in above code.
I have multiple webclient calls, but I want to configure the retry backoff configuration at a single place. How can I do that? so that my code should look like below
private Flux<BusinessObject> getBusinessObjects(String serviceURL) {
return this.webClient.get()
.uri(serviceURL)
.retrieve()
.bodyToFlux(BusinessObject.class)
.somCommonRetryCodeWrappingTheRetryLogic();
}

You can use transform operator for this purpose:
private Flux<BusinessObject> getBusinessObjects(String serviceURL) {
return this.webClient.get()
.uri(serviceURL)
.retrieve()
.bodyToFlux(BusinessObject.class)
.transform(originalFlux -> wrapWithRetry(originalFlux));
}
private <T> Flux<T> wrapWithRetry(Flux<T> originalFlux) {
return originalFlux
.retryWhen(Retry.backoff(3, Duration.of(2, ChronoUnit.SECONDS))
.doBeforeRetry((value) -> log.info("Retrying request " + value))
.filter(error -> error instanceof WebClientRequestException)
.onRetryExhaustedThrow((retryBackoffSpec, retrySignal) ->
new RuntimeException(retrySignal.failure().getMessage())));
}
Only drawback is that if you have Mono use cases as well then you need to implement it twice.
If that is still too much copy-paste, you can consider defining an ExchangeFilterFunction to apply retry for every WebClient call automatically. Something like this:
WebClient.builder()
// ...
.filter((request, next) -> next.exchange(request).retry()) // add full retry config here
.build();

How to convert Spring WebClient Response to ResponseEntity?

I'm able to return ResponseEntity using toEntity() method like below:
#GetMapping("/uri")
public Mono<ResponseEntity<Data[]>> methodName() {
return webClient
.get()
.uri("http://localhost:8088/externalService")
.accept(MediaType.APPLICATION_JSON)
.retrieve()
.toEntity(Data[].class);
}
But I want to process response headers before returning.
The above code converts WebClient response to ResponseEntity and returns immediately but I want to store it in a ResponseEntity variable, process it, and then return the ResponseEntity back.
I referred this -> Spring WebClient Documentation
WHen I tried to store it in a varibale, I get this error -> "block()/blockFirst()/blockLast() are blocking, which is not supported in thread reactor-http-nio-3"

You can simply use the Reactor's map operator to modify the headers:
return webClient
.get()
.uri("http://localhost:8088/externalService")
.accept(MediaType.APPLICATION_JSON)
.retrieve()
.toEntity(Data[].class)
.map(responseEntity -> responseEntity.getHeaders().add("header", "header-value");
Alternatively, you can use .handle operator in order to provide response processing:
.handle((responseEntity, sink) -> {
if(!isValid(responseEntity)){
sink.error(new InvalidResponseException());
} else if (isOk(responseEntity))
sink.next(responseEntity);
}
else {
//just ignore element
}
})

Spring Starter Web dependency was missing in my pom.xml. Found it and added it back.
Now able to get WebClient response in ResponseEntity format.

Why does this simple Webflux Controller calls the Webclient retrieve method twice?

I have a very simple Webflux controller that just do a GET request to another service endpoint and returns a simple JSON list. The problem is the remote endpoint is always called twice.
This issue doesn't happen if I used Mono as the return type of the controller instead of Flux!
// This calls "/remote/endpoint" twice!
#GetMapping("/blabla")
fun controller() : Flux<JsonNode> {
return webClient.get()
.uri("/remote/endpoint")
.retrieve()
.bodyToMono(JsonNode::class.java)
.flatMapIterable { body ->
body.get("data")
}
}
// This calls "/remote/endpoint" once.
#GetMapping("/blabla")
fun controller() : Mono<JsonNode> {
return webClient.get()
.uri("/remote/endpoint")
.retrieve()
.bodyToMono(JsonNode::class.java)
.map { body ->
body.get("data")
}
}

Spring 5 webflux inter-dependent webclient calls are not getting timeout

I have two interdependent webclient calls to different api's , when the first webclient call response is delayed then readtimeout excpetion is coming . but whenever the first call is success and the second call response is delayed then it is waiting for response indefinitely.
i tried creating seperate instances of webclient for each call . still issue persists.
HttpClient httpClient = HttpClient.create()
.tcpConfiguration(client ->
client.doOnConnected(conn -> conn
.addHandlerLast(new ReadTimeoutHandler(10))
.addHandlerLast(new WriteTimeoutHandler(10))));
return WebClient.builder().baseUrl(url).clientConnector(new ReactorClientHttpConnector(httpClient))
.exchangeStrategies(ExchangeStrategies.withDefaults())
.defaultHeader(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_UTF8_VALUE)
.filter(ExchangeFilterFunctions
.basicAuthentication(", "))
.build();
Two subsequent calls are below where , when the second call is delayed response then the readtimeout exception is not thrown
request = request.flatMap(req ->
tempService.getId(loggedInUser, token)
.map(response -> {
req.setRetrieveClientIdentifier(response.getId());
return seRequest;
}))
.zipWhen(request -> tempService.getIdFor(request.getIdentifier(), accountToken)).map(tuple -> {
tuple.getT1().setID(tuple.getT2().getId());
return tuple.getT1();
});

How to create request with parameters with webflux Webclient?

At backend side I have REST controller with POST method:
#RequestMapping(value = "/save", method = RequestMethod.POST)
public Integer save(#RequestParam String name) {
//do save
return 0;
}
How can I create request using WebClient with request parameter?
WebClient.create(url).post()
.uri("/save")
//?
.exchange()
.block()
.bodyToMono(Integer.class)
.block();

There are many encoding challenges when it comes to creating URIs. For more flexibility while still being right on the encoding part, WebClient provides a builder-based variant for the URI:
WebClient.create().get()
.uri(builder -> builder.scheme("http")
.host("example.org").path("save")
.queryParam("name", "spring-framework")
.build())
.retrieve()
.bodyToMono(String.class);

From: https://www.callicoder.com/spring-5-reactive-webclient-webtestclient-examples/
In configuration class you can define the host:
#Bean(name = "providerWebClient")
WebClient providerWebClient(#Value("${external-rest.provider.base-url}") String providerBaseUrl) {
return WebClient.builder().baseUrl(providerBaseUrl)
.clientConnector(clientConnector()).build();
}
Then you can use the WebClient instace:
#Qualifier("providerWebClient")
private final WebClient webClient;
webClient.get()
.uri(uriBuilder -> uriBuilder.path("/provider/repos")
.queryParam("sort", "updated")
.queryParam("direction", "desc")
.build())
.header("Authorization", "Basic " + Base64Utils
.encodeToString((username + ":" + token).getBytes(UTF_8)))
.retrieve()
.bodyToFlux(GithubRepo.class);

Assuming that you already created your WebClient instance and configured it with baseUrl.
URI Path Component
this.webClient.get()
.uri("/products")
.retrieve();
result: /products
this.webClient.get()
.uri(uriBuilder - > uriBuilder
.path("/products/{id}")
.build(2))
.retrieve();
result: /products/2
this.webClient.get()
.uri(uriBuilder - > uriBuilder
.path("/products/{id}/attributes/{attributeId}")
.build(2, 13))
.retrieve();
result: /products/2/attributes/13
URI Query Parameters
this.webClient.get()
.uri(uriBuilder - > uriBuilder
.path("/peoples/")
.queryParam("name", "Charlei")
.queryParam("job", "Plumber")
.build())
.retrieve();
result:
/peoples/?name=Charlei/job=Plumber