Develop Reference

Apply retry to each and every endpoint in the spring webflux application

I have created below webclient and using it inside of my service to make HTTP third party calls.
#Configuration
public class WebclientConfig {
#Bean
public WebClient webClient() {
// custom client connector with connection pool disabled is being used as by default the connection pooling is done and connection reset happens after some idle time.
return WebClient.builder()
.clientConnector(new ReactorClientHttpConnector(HttpClient.newConnection()))
.defaultHeader(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE)
.defaultHeader(HttpHeaders.ACCEPT, MediaType.APPLICATION_JSON_VALUE)
.build();
}
}
and in my service, I am calling the third party service like below.
private Flux<BusinessObject> getBusinessObjects(String serviceURL) {
return this.webClient.get()
.uri(serviceURL)
.retrieve()
.bodyToFlux(BusinessObject.class) //code below this, do I have to copy for each webclient request to configure the retry, even if the values are same
.retryWhen(Retry.backoff(3, Duration.of(2, ChronoUnit.SECONDS))
.doBeforeRetry((value) -> log.info("Retrying request " + value))
.filter(error -> error instanceof WebClientRequestException)
.onRetryExhaustedThrow((retryBackoffSpec, retrySignal) ->
new RuntimeException(retrySignal.failure().getMessage())));
}
My question is as in comment in above code.
I have multiple webclient calls, but I want to configure the retry backoff configuration at a single place. How can I do that? so that my code should look like below
private Flux<BusinessObject> getBusinessObjects(String serviceURL) {
return this.webClient.get()
.uri(serviceURL)
.retrieve()
.bodyToFlux(BusinessObject.class)
.somCommonRetryCodeWrappingTheRetryLogic();
}

You can use transform operator for this purpose:
private Flux<BusinessObject> getBusinessObjects(String serviceURL) {
return this.webClient.get()
.uri(serviceURL)
.retrieve()
.bodyToFlux(BusinessObject.class)
.transform(originalFlux -> wrapWithRetry(originalFlux));
}
private <T> Flux<T> wrapWithRetry(Flux<T> originalFlux) {
return originalFlux
.retryWhen(Retry.backoff(3, Duration.of(2, ChronoUnit.SECONDS))
.doBeforeRetry((value) -> log.info("Retrying request " + value))
.filter(error -> error instanceof WebClientRequestException)
.onRetryExhaustedThrow((retryBackoffSpec, retrySignal) ->
new RuntimeException(retrySignal.failure().getMessage())));
}
Only drawback is that if you have Mono use cases as well then you need to implement it twice.
If that is still too much copy-paste, you can consider defining an ExchangeFilterFunction to apply retry for every WebClient call automatically. Something like this:
WebClient.builder()
// ...
.filter((request, next) -> next.exchange(request).retry()) // add full retry config here
.build();

au.com.dius.pact.consumer.PactMismatchesException: The following requests were not received:

This test case is for mocking the health check contract.
TestClass
#Pact(consumer = "Consumer")
public RequestResponsePact getHealthCheck(PactDslWithProvider builder) {
Map<String, String> headers = new HashMap<String, String>();
headers.put("Content-Type", "text/plain");
headers.put("callchainid", "a4275861-f60a-44ab-85a6-c0c2c9df5e27");
return builder
.given("get health check")
.uponReceiving("get health data")
.path("/health")
.method("GET")
.headers(headers )
.willRespondWith()
.status(200)
.body("{\"status\":\"UP\",\"components\":{\"db\":{\"status\":\"UP\",\"details\":{\"database\":\"PostgreSQL\",\"validationQuery\":\"isValid()\"}}}}")
.toPact();
}
#Test
#PactTestFor(pactMethod = "getHealthCheck")
void getHealthData(MockServer mockServer) {
WebClient webClient=WebClient.builder().baseUrl(mockServer.getUrl()).build();
final String callChainId="a4275861-f60a-44ab-85a6-c0c2c9df5e27";
ThreadContext.put(CallChainIdService.HEADER_NAME, callChainId);
AsyncClient asyncClient=new AsyncClient(webClient);
Mono<ClientResponse> responseMono=asyncClient.getHealthCheck();
System.out.println(responseMono);
}
Here the webclient end point code which i am trying to hit,
AsyncClient Class
private final WebClient CLIENT;
#Override
public Mono<ClientResponse> getHealthCheck() {
return get(MediaType.TEXT_PLAIN, "/health");
}
private Mono<ClientResponse> get(MediaType contentType, String uri, Object... params) {
return CLIENT
.mutate()
.defaultHeader(CallChainIdService.HEADER_NAME, ThreadContext.get(CallChainIdService.HEADER_NAME))
.build()
.get()
.uri(uri, params)
.accept(contentType)
.exchange();
}
When i run the test , i got PactMismatchesException: The following requests were not received.
au.com.dius.pact.consumer.PactMismatchesException: The following requests were not received:
method: GET
path: /health
query: {}
headers: {callchainid=[a4275861-f60a-44ab-85a6-c0c2c9df5e27], Content-Type=[text/plain]}
matchers: MatchingRules(rules={})
generators: Generators(categories={})
body: MISSING
I am not sure what i am doing wrong here. Appreciate your inputs and help

It looks like you have content-type as the expected header you send in the GET call, but in fact you send only an accept header (both with the media type text/plain).
I think your test should be updated to use accept.

Spring Webclient decode custom application/multipart-related,application/dicom (Wado-RS)

I'm trying to decode a multipart-related request that is just a simple multi files download but with a specific content type by part (application/dicom and not application/octet-stream).
Since the structure of the response body might be identical, I could just tell the "multipart codec" to treat that content type as an octet-stream.
public Flux<FilePart> getDicoms(String seriesUri) {
return webClient.get()
.uri(seriesUri)
.accept(MediaType.ALL)
.retrieve()
.bodyToFlux(FilePart.class);
}
How can I do that?

An easier way of reading a multipart response:
private Mono<ResponseEntity<Flux<Part>>> queryForFiles(String uri)
final var partReader = new DefaultPartHttpMessageReader();
partReader.setStreaming(true);
return WebClient.builder()
.build()
.get()
.uri(wadoUri)
.accept(MediaType.ALL)
.retrieve()
.toEntityFlux((inputMessage, context) -> partReader.read(ResolvableType.forType(DataBuffer.class), inputMessage, Map.of())))

This is what I've done to make it work. I used directly the DefaultPartHttpMessageReader class to do it cleanly (spring 5.3).
public Flux<Part> getDicoms(String wadoUri) {
final var partReader = new DefaultPartHttpMessageReader();
partReader.setStreaming(true);
return WebClient.builder()
.build()
.get()
.uri(wadoUri)
.accept(MediaType.ALL)
//.attributes(clientRegistrationId("keycloak"))
.exchange()
.flatMapMany(clientResponse -> {
var message = new ReactiveHttpInputMessage() {
#Override
public Flux<DataBuffer> getBody() {
return clientResponse.bodyToFlux(DataBuffer.class);
}
#Override
public HttpHeaders getHeaders() {
return clientResponse.headers().asHttpHeaders();
}
};
return partReader.read(ResolvableType.forType(DataBuffer.class), message, Map.of());
});
}

Follow redirection with cookies using WebFlux

I am using WebFlux with netty to make third party calls for my spring boot app. A post request with form parameters is made on client's provided url and client responds with 302 status and a location. The code I have written below is able to follow redirections but doesn't send cookies with it which is causing subsequent redirections to fail.
WebFlux config
#Bean
public WebClient webClient() {
ExchangeStrategies exchangeStrategies = ExchangeStrategies.builder()
.codecs((configurer) -> {
configurer.defaultCodecs().jaxb2Encoder(new Jaxb2XmlEncoder());
configurer.defaultCodecs().jaxb2Decoder(new Jaxb2XmlDecoder());
}).build();
exchangeStrategies.messageWriters().stream()
.filter(LoggingCodecSupport.class::isInstance)
.forEach(writer -> ((LoggingCodecSupport) writer)
.setEnableLoggingRequestDetails(Boolean.TRUE));
return WebClient.builder()
.clientConnector(new ReactorClientHttpConnector(HttpClient.create()
.followRedirect(true)
.secure()))
.exchangeStrategies(exchangeStrategies)
.build();
}
Post Request
private <T> Mono <String> buildPostRequest(MultiValueMap <String, String> formData, String postUrl) {
return client.post()
.uri(postUrl)
.body(BodyInserters.fromFormData(formData))
.header(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_FORM_URLENCODED_VALUE)
.retrieve()
.bodyToMono(String.class);
}

Spring WebClient filter Null from Json Request

I am using Spring WebClient Api to make a rest api call.
I have an entity object--JobInfo which acts as my POST Request pay-load.
The below Rest API fails because certain attributes of JobInfo are null.
private BatchInfo createBulkUploadJob(JobInfo jobInfo) {
return webClient.post()
.uri(URL.concat("/services/data/v47.0/jobs/ingest/"))
.contentType(MediaType.APPLICATION_JSON)
.header("Authorization", "OAuth " + TOKEN)
.bodyValue(jobInfo)
.retrieve()
.bodyToMono(BatchInfo.class)
.block();
}
I need to filter out the Null attributes from sending it across the rest call.
I understand this can be easily achieved by including below annotation on JobInfo class.
#JsonInclude(JsonInclude.Include.NON_NULL)
But JobInfo is coming from a third party Jar, so I cannot touch this class.
Is there a way I can configure this in webClient to filter this out or any other option ?

Try with this:
private BatchInfo createBulkUploadJob(JobInfo jobInfo) {
ObjectMapper objectMapper = new ObjectMapper();
objectMapper.setSerializationInclusion(JsonInclude.Include.NON_NULL);
ExchangeStrategies strategies = ExchangeStrategies
.builder()
.codecs(clientDefaultCodecsConfigurer -> {
clientDefaultCodecsConfigurer.defaultCodecs().jackson2JsonEncoder(new Jackson2JsonEncoder(objectMapper, MediaType.APPLICATION_JSON));
clientDefaultCodecsConfigurer.defaultCodecs().jackson2JsonDecoder(new Jackson2JsonDecoder(objectMapper, MediaType.APPLICATION_JSON));
}).build();
WebClient webClient = WebClient.builder().exchangeStrategies(strategies).build();
return webClient.post()
.uri(URL.concat("/services/data/v47.0/jobs/ingest/"))
.contentType(MediaType.APPLICATION_JSON)
.header("Authorization", "OAuth " + TOKEN)
.bodyValue(jobInfo)
.retrieve()
.bodyToMono(BatchInfo.class)
.block();
}