Springboot v2.0.0.M6 WebClient making multiple duplicate HTTP POST calls - java

I am using spring-boot version 2.0.0.M6.
I need to make async HTTP calls from spring-boot app say APP1 to another app (play framework) say APP2.
So if I need to make 20 distinct async calls from APP1 to APP2, APP2 receives 20 requests out of which few are duplicates, which means these duplicates replaced few distinct requests.
Expected:
api/v1/call/1
api/v1/call/2
api/v1/call/3
api/v1/call/4
Actual:
api/v1/call/1
api/v1/call/2
api/v1/call/4
api/v1/call/4
I am using spring reactive WebClient.
Below is the spring boot version in build.gradle
buildscript {
ext {
springBootVersion = '2.0.0.M6'
//springBootVersion = '2.0.0.BUILD-SNAPSHOT'
}
repositories {
mavenCentral()
maven { url "https://repo.spring.io/snapshot" }
maven { url "https://repo.spring.io/milestone" }
maven {url "https://plugins.gradle.org/m2/"}
}
dependencies {
classpath("org.springframework.boot:spring-boot-gradle-plugin:${springBootVersion}")
classpath("se.transmode.gradle:gradle-docker:1.2")
}
}
My WebClient init snippet
private WebClient webClient = WebClient.builder()
.clientConnector(new ReactorClientHttpConnector((HttpClientOptions.Builder builder) -> builder.disablePool()))
.build();
My POST method
public <T> Mono<JsonNode> postClient(String url, T postData) {
return Mono.subscriberContext().flatMap(ctx -> {
String cookieString = ctx.getOrDefault(Constants.SubscriberContextConstnats.COOKIES, StringUtils.EMPTY);
URI uri = URI.create(url);
return webClient.post().uri(uri).body(BodyInserters.fromObject(postData)).header(HttpHeaders.COOKIE, cookieString)
.exchange().flatMap(clientResponse ->
{
return clientResponse.bodyToMono(JsonNode.class);
})
.onErrorMap(err -> new TurtleException(err.getMessage(), err))
.doOnSuccess(jsonData -> {
});
});
}
The code from where this postClient method is invoked
private void getResultByKey(PremiumRequestHandler request, String key, BrokerConfig brokerConfig) {
/* Live calls for the insurers */
LOG.info("[PREMIUM SERVICE] LIVE CALLLLL MADE FOR: " + key + " AND REQUEST ID: " + request.getRequestId());
String uri = brokerConfig.getHostUrl() + verticalResolver.determineResultUrl(request.getVertical()) + key;
LOG.info("[PREMIUM SERVICE] LIVE CALL WITH URI : " + uri + " FOR REQUEST ID: " + request.getRequestId());
Mono<PremiumResponse> premiumResponse = reactiveWebClient.postClient(uri, request.getPremiumRequest())
.map(json -> PlatformUtils.mapToClass(json, PremiumResponse.class));
premiumResponse.subscribe(resp -> {
resp.getPremiumResults().forEach(result -> {
LOG.info("Key " + result.getKey());
repository.getResultRepoRawType(request.getVertical())
.save(result).subscribe();
saveResult.subscriberContext(ctx -> {
MultiBrokerMongoDBFactory.setDatabaseNameForCurrentThread(brokerConfig.getBroker());
return ctx;
}).subscribe();
});
}, error -> {
LOG.info("[PREMIUM SERVICE] ERROR RECIEVED FOR " + key + " AND REQUEST ID" + request.getRequestId() + " > " + error.getMessage());
});
}
Had put logs at the end-point in the client code, can not see multiple requests at that point.
Probably it's a bug in WebClient where URI is getting swapped in multithreaded environment.
Tried mutating WebClient, still the URI is getting swapped
Please help.
Git Repo added github.com/praveenk007/ps-demo

I happend to have experienced similar problem:
When calling the same service (here marked as ExternalService) in parallel (webflux) sometimes the same request was being sent and the problem indeed seems to reside in Webclient.
The solution was in changing the way Webclient is being created.
Before:
Here ExternalCall is configured client which calls ExternalService. So the parallel execution here was of internalCall method. Note we pass WebClient.RequestBodySpec class to ExternalCall class
#Bean
ExternalCall externalCall(WebClient.Builder webClientBuilder) {
var exchangeStrategies = getExchangeStrategies();
var endpoint = "v1/data";
var timeout = 10000;
var uri = "https://externalService.com/";
var requestBodySpec = webClientBuilder.clone()
.clientConnector(new ReactorClientHttpConnector(HttpClient.create()))
.exchangeStrategies(exchangeStrategies)
.build()
.post()
.uri(endpoint)
.accept(TEXT_XML)
.contentType(TEXT_XML);
return new ExternalCall(requestBodySpec, uri, timeout);
}
Then within ExternalCall class I had had
private Mono<String> internalCall(String rq) {
return requestBodySpec.bodyValue(rq)
.retrieve()
.bodyToMono(String.class)
.timeout(timeout, Mono.error(() -> new TimeoutException(String.format("%s - timeout after %s seconds", "ExternalService", timeout.getSeconds()))));
}
After:
We pass WebClient class to ExternalCall.
#Bean
ExternalCall externalCall(WebClient.Builder webClientBuilder) {
var exchangeStrategies = getExchangeStrategies();
var timeout = 10000;
var uri = "https://externalService.com/";
var webClient = webClientBuilder.clone()
.clientConnector(new ReactorClientHttpConnector(HttpClient.create()))
.exchangeStrategies(exchangeStrategies)
.build();
return new ExternalCall(webClient, uri, timeout);
}
Now we specify RequestBodySpec within ExternalCall class:
private Mono<String> internalCall(String rq) {
return webClient
.post()
.uri(endpoint)
.accept(TEXT_XML)
.contentType(TEXT_XML)
.bodyValue(rq)
.retrieve()
.bodyToMono(String.class)
.timeout(timeout, Mono.error(() -> new TimeoutException(String.format("%s - timeout after %s seconds", "ExternalService", timeout.getSeconds()))));
}
Conclusions: So apparently the moment you create WebClient.RequestBodySpec instance matters. Hope that helps somebody

Adding my few observations:
webClient.get() or webClient.post() always returns new DefaultRequestBodyUriSpec on every call on which URI being called and I think it does not look like URI getting swapped.
class DefaultWebClient implements WebClient {
..
#Override
public RequestHeadersUriSpec<?> get() {
return methodInternal(HttpMethod.GET);
}
#Override
public RequestBodyUriSpec post() {
return methodInternal(HttpMethod.POST);
}
..
#Override
public Mono<ClientResponse> exchange() {
ClientRequest request = (this.inserter != null ?
initRequestBuilder().body(this.inserter).build() :
initRequestBuilder().build());
return exchangeFunction.exchange(request).switchIfEmpty(NO_HTTP_CLIENT_RESPONSE_ERROR);
}
private ClientRequest.Builder initRequestBuilder() {
URI uri = (this.uri != null ? this.uri : uriBuilderFactory.expand(""));
return ClientRequest.create(this.httpMethod, uri)
.headers(headers -> headers.addAll(initHeaders()))
.cookies(cookies -> cookies.addAll(initCookies()))
.attributes(attributes -> attributes.putAll(this.attributes));
}
..
}
and methodInternal method looks like below
#SuppressWarnings("unchecked")
private RequestBodyUriSpec methodInternal(HttpMethod httpMethod) {
return new DefaultRequestBodyUriSpec(httpMethod);
}
In addition while making actual request also, new ClientRequest is created.
Class source
https://github.com/spring-projects/spring-framework/blob/master/spring-webflux/src/main/java/org/springframework/web/reactive/function/client/DefaultWebClient.java

Related

How to prevent multiple authentication calls via WebClient

The following code will fetch 20 devices and will consume some data from an online service.
In order to use the online service the WebClient need to get access token (authorization-grant-type=client_credentials)
spring.security.oauth2.client.registration.web.client-id=xxxx
spring.security.oauth2.client.registration.web.client-secret=xxxx
spring.security.oauth2.client.registration.web.scope=xxxx
spring.security.oauth2.client.registration.web.authorization-grant-type=client_credentials
spring.security.oauth2.client.provider.web.token-uri=https://xxxx
Config:
#Configuration
public class Config {
#Autowired
public ClientRegistrationRepository clientRegistrationRepository;
#Bean
public WebClient webClient(ExchangeFilterFunction getOAuth2FilterFunction) {
logger.info("** start webClient **");
return WebClient.builder().filter(getOAuth2FilterFunction).build();
}
#Bean
public ExchangeFilterFunction getOAuth2FilterFunction(ReactiveClientRegistrationRepository clientRegistrationRepository) {
logger.info("** start getOAuth2FilterFunction **");
InMemoryReactiveOAuth2AuthorizedClientService authorizedClientService = new InMemoryReactiveOAuth2AuthorizedClientService(clientRegistrationRepository);
AuthorizedClientServiceReactiveOAuth2AuthorizedClientManager authorizedClientManager = new AuthorizedClientServiceReactiveOAuth2AuthorizedClientManager(
clientRegistrationRepository, authorizedClientService);
authorizedClientManager.setAuthorizedClientProvider(new ClientCredentialsReactiveOAuth2AuthorizedClientProvider());
ServerOAuth2AuthorizedClientExchangeFilterFunction oauth2FilterFunction = new ServerOAuth2AuthorizedClientExchangeFilterFunction(authorizedClientManager);
oauth2FilterFunction.setDefaultClientRegistrationId("web");
return oauth2FilterFunction;
}
#Bean
public ReactiveClientRegistrationRepository clientRegistrations() {
logger.info("** start clientRegistrations **");
ClientRegistration clientRegistration = clientRegistrationRepository.findByRegistrationId("web");
return new InMemoryReactiveClientRegistrationRepository(clientRegistration);
}
WebClient:
private void getData(List<Device> initlist) {
logger.info("** start getDeviceData **");
List<Mono<DeviceDto>> jsonDeviceList = initlist.stream()
.map(device -> webClient.post().uri(infoUri)
.header(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE)
.header(HttpHeaders.ACCEPT, MediaType.APPLICATION_JSON_VALUE)
.body(Mono.just(device.getMacAddress()),String.class)
.attributes(ServerOAuth2AuthorizedClientExchangeFilterFunction.clientRegistrationId("web"))
.retrieve()
.bodyToMono(DeviceDto.class))
.collect(Collectors.toList());
ObjectMapper mapper = new ObjectMapper();
Flux<DeviceDto> mergedMonos = Flux.fromIterable(jsonDeviceList).flatMapSequential(Function.identity());
mergedMonos.map(device -> mapper.valueToTree(device)).collectList().subscribe(list -> {
generateCsv(list);
});
}
The problem is that for each service request there will be an equivalent access token request from the authentication server. So for all 20 devices in the list the WebClient will request access token 20 times.
How can I change this to have only one call rather then 20 ?
Thank you
What #Toerktumlare said is right. If you want to call oauth server once, you just need to cache the access_token in memory or redis at first request and then use the same access_token before it is expired.
WebClient webClient = WebClient.builder()
.defaultHeader("Authorization", "Bearer " + getAccessToken())
.build();
private String getAccessToken() {
// get access_token from redis
// if expiring, it will return null
String access_token = (String) redisTemplate.opsForValue().get("access_token");
if (null != access_token) {
return access_token;
}
// get access_token from oauth server again and cache it into redis
return getAccessTokenFromOauthServer();
}

How to set baseUrl in Retrofit?

I want to set the BaseUrl in Retrofit to change dynamically between stage and live because i have an app that has stage and live version. So i made a spinner and the user can select either he wants. But the problem is that after the user select the flavor he wants and then wants to change again it doens't work because the baseUrl is not changing like it should be.
I have this class where is defined the API_URL but it's not working :
#Singleton
class SingleUrlApi {
companion object{
public var API_URL_STAGE = BuildConfig.STAGE
}
}
and then i have another function that uses this API_URL_STAGE
override fun getUrl(shopUrl: ShopUrl, vararg args: String): String {
return when (shopUrl) {
ShopUrl.API_BASE -> if (SingleUrlApi.API_URL_STAGE) {
context.localizedContext(localeManager.getCurrentLocale()).getString(R.string.base_url_stage)
} else {
context.localizedContext(localeManager.getCurrentLocale()).getString(R.string.base_url_live)
}
ShopUrl.WEB_BASE -> if (SingleUrlApi.API_URL_STAGE) {
context.localizedContext(localeManager.getCurrentLocale()).getString(R.string.base_web_url_stage)
} else {
context.localizedContext(localeManager.getCurrentLocale()).getString(R.string.base_web_url_live)
}
You can use OkHttp along with Retrofit.
Then, you can use an OkHttpInterceptor to change the URL of the request
public final class HostSelectionInterceptor implements Interceptor {
#Override public okhttp3.Response intercept(Chain chain) throws IOException {
Request request = chain.request();
String host = //logic to fetch the new URL
if (host != null) {
HttpUrl newUrl = request.url().newBuilder()
.host(host)
.build();
request = request.newBuilder()
.url(newUrl)
.build();
}
return chain.proceed(request);
}
}
An easy and effective way is to use this library: RetrofitUrlManager

Spring Webclient decode custom application/multipart-related,application/dicom (Wado-RS)

I'm trying to decode a multipart-related request that is just a simple multi files download but with a specific content type by part (application/dicom and not application/octet-stream).
Since the structure of the response body might be identical, I could just tell the "multipart codec" to treat that content type as an octet-stream.
public Flux<FilePart> getDicoms(String seriesUri) {
return webClient.get()
.uri(seriesUri)
.accept(MediaType.ALL)
.retrieve()
.bodyToFlux(FilePart.class);
}
How can I do that?
An easier way of reading a multipart response:
private Mono<ResponseEntity<Flux<Part>>> queryForFiles(String uri)
final var partReader = new DefaultPartHttpMessageReader();
partReader.setStreaming(true);
return WebClient.builder()
.build()
.get()
.uri(wadoUri)
.accept(MediaType.ALL)
.retrieve()
.toEntityFlux((inputMessage, context) -> partReader.read(ResolvableType.forType(DataBuffer.class), inputMessage, Map.of())))
This is what I've done to make it work. I used directly the DefaultPartHttpMessageReader class to do it cleanly (spring 5.3).
public Flux<Part> getDicoms(String wadoUri) {
final var partReader = new DefaultPartHttpMessageReader();
partReader.setStreaming(true);
return WebClient.builder()
.build()
.get()
.uri(wadoUri)
.accept(MediaType.ALL)
//.attributes(clientRegistrationId("keycloak"))
.exchange()
.flatMapMany(clientResponse -> {
var message = new ReactiveHttpInputMessage() {
#Override
public Flux<DataBuffer> getBody() {
return clientResponse.bodyToFlux(DataBuffer.class);
}
#Override
public HttpHeaders getHeaders() {
return clientResponse.headers().asHttpHeaders();
}
};
return partReader.read(ResolvableType.forType(DataBuffer.class), message, Map.of());
});
}

Reactive Spring Boot API wrapping Elasticsearch's async bulk indexing

I am developing prototype for a new project. The idea is to provide a Reactive Spring Boot microservice to bulk index documents in Elasticsearch. Elasticsearch provides a High Level Rest Client which provides an Async method to bulk process indexing requests. Async delivers callbacks using listeners are mentioned here. The callbacks receive index responses (per requests) in batches. I am trying to send this response back to the client as Flux. I have come up with something based on this blog post.
Controller
#RestController
public class AppController {
#SuppressWarnings("unchecked")
#RequestMapping(value = "/test3", method = RequestMethod.GET)
public Flux<String> index3() {
ElasticAdapter es = new ElasticAdapter();
JSONObject json = new JSONObject();
json.put("TestDoc", "Stack123");
Flux<String> fluxResponse = es.bulkIndex(json);
return fluxResponse;
}
ElasticAdapter
#Component
class ElasticAdapter {
String indexName = "test2";
private final RestHighLevelClient client;
private final ObjectMapper mapper;
private int processed = 1;
Flux<String> bulkIndex(JSONObject doc) {
return bulkIndexDoc(doc)
.doOnError(e -> System.out.print("Unable to index {}" + doc+ e));
}
private Flux<String> bulkIndexDoc(JSONObject doc) {
return Flux.create(sink -> {
try {
doBulkIndex(doc, bulkListenerToSink(sink));
} catch (JsonProcessingException e) {
sink.error(e);
}
});
}
private void doBulkIndex(JSONObject doc, BulkProcessor.Listener listener) throws JsonProcessingException {
System.out.println("Going to submit index request");
BiConsumer<BulkRequest, ActionListener<BulkResponse>> bulkConsumer =
(request, bulkListener) ->
client.bulkAsync(request, RequestOptions.DEFAULT, bulkListener);
BulkProcessor.Builder builder =
BulkProcessor.builder(bulkConsumer, listener);
builder.setBulkActions(10);
BulkProcessor bulkProcessor = builder.build();
// Submitting 5,000 index requests ( repeating same JSON)
for (int i = 0; i < 5000; i++) {
IndexRequest indexRequest = new IndexRequest(indexName, "person", i+1+"");
String json = doc.toJSONString();
indexRequest.source(json, XContentType.JSON);
bulkProcessor.add(indexRequest);
}
System.out.println("Submitted all docs
}
private BulkProcessor.Listener bulkListenerToSink(FluxSink<String> sink) {
return new BulkProcessor.Listener() {
#Override
public void beforeBulk(long executionId, BulkRequest request) {
}
#SuppressWarnings("unchecked")
#Override
public void afterBulk(long executionId, BulkRequest request, BulkResponse response) {
for (BulkItemResponse bulkItemResponse : response) {
JSONObject json = new JSONObject();
json.put("id", bulkItemResponse.getResponse().getId());
json.put("status", bulkItemResponse.getResponse().getResult
sink.next(json.toJSONString());
processed++;
}
if(processed >= 5000) {
sink.complete();
}
}
#Override
public void afterBulk(long executionId, BulkRequest request, Throwable failure) {
failure.printStackTrace();
sink.error(failure);
}
};
}
public ElasticAdapter() {
// Logic to initialize Elasticsearch Rest Client
}
}
I used FluxSink to create the Flux of Responses to send back to the Client. At this point, I have no idea whether this correct or not.
My expectation is that the calling client should receive the responses in batches of 10 ( because bulk processor processess it in batches of 10 - builder.setBulkActions(10); ). I tried to consume the endpoint using Spring Webflix Client. But unable to work it out. This is what I tried
WebClient
public class FluxClient {
public static void main(String[] args) {
WebClient client = WebClient.create("http://localhost:8080");
Flux<String> responseFlux = client.get()
.uri("/test3")
.retrieve()
.bodyToFlux(String.class);
responseFlux.subscribe(System.out::println);
}
}
Nothing is printing on console as I expected. I tried to use System.out.println(responseFlux.blockFirst());. It prints all the responses as a single batch at the end and not in batches at .
If my approach is correct, what is the correct way to consume it? For the solution in my mind, this client will reside is another Webapp.
Notes: My understanding of Reactor API is limited. The version of elasticsearch used is 6.8.
So made the following changes to your code.
In ElasticAdapter,
public Flux<Object> bulkIndex(JSONObject doc) {
return bulkIndexDoc(doc)
.subscribeOn(Schedulers.elastic(), true)
.doOnError(e -> System.out.print("Unable to index {}" + doc+ e));
}
Invoked subscribeOn(Scheduler, requestOnSeparateThread) on the Flux, Got to know about it from, https://github.com/spring-projects/spring-framework/issues/21507
In FluxClient,
Flux<String> responseFlux = client.get()
.uri("/test3")
.headers(httpHeaders -> {
httpHeaders.set("Accept", "text/event-stream");
})
.retrieve()
.bodyToFlux(String.class);
responseFlux.delayElements(Duration.ofSeconds(1)).subscribe(System.out::println);
Added "Accept" header as "text/event-stream" and delayed Flux elements.
With the above changes, was able to get the response in real time from the server.

Spring Webflux : Webclient : Get body on error

I am using the webclient from spring webflux, like this :
WebClient.create()
.post()
.uri(url)
.syncBody(body)
.accept(MediaType.APPLICATION_JSON)
.headers(headers)
.exchange()
.flatMap(clientResponse -> clientResponse.bodyToMono(tClass));
It is working well.
I now want to handle the error from the webservice I am calling (Ex 500 internal error). Normally i would add an doOnError on the "stream" and isu the Throwable to test the status code,
But my issue is that I want to get the body provided by the webservice because it is providing me a message that i would like to use.
I am looking to do the flatMap whatever happen and test myself the status code to deserialize or not the body.
I prefer to use the methods provided by the ClientResponse to handle http errors and throw exceptions:
WebClient.create()
.post()
.uri( url )
.body( bodyObject == null ? null : BodyInserters.fromValue( bodyObject ) )
.accept( MediaType.APPLICATION_JSON )
.headers( headers )
.exchange()
.flatMap( clientResponse -> {
//Error handling
if ( clientResponse.statusCode().isError() ) { // or clientResponse.statusCode().value() >= 400
return clientResponse.createException().flatMap( Mono::error );
}
return clientResponse.bodyToMono( clazz )
} )
//You can do your checks: doOnError (..), onErrorReturn (..) ...
...
In fact, it's the same logic used in the DefaultResponseSpec of DefaultWebClient to handle errors. The DefaultResponseSpec is an implementation of ResponseSpec that we would have if we made a retrieve() instead of exchange().
Don't we have onStatus()?
public Mono<Void> cancel(SomeDTO requestDto) {
return webClient.post().uri(SOME_URL)
.body(fromObject(requestDto))
.header("API_KEY", properties.getApiKey())
.retrieve()
.onStatus(HttpStatus::isError, response -> {
logTraceResponse(log, response);
return Mono.error(new IllegalStateException(
String.format("Failed! %s", requestDto.getCartId())
));
})
.bodyToMono(Void.class)
.timeout(timeout);
}
And:
public static void logTraceResponse(Logger log, ClientResponse response) {
if (log.isTraceEnabled()) {
log.trace("Response status: {}", response.statusCode());
log.trace("Response headers: {}", response.headers().asHttpHeaders());
response.bodyToMono(String.class)
.publishOn(Schedulers.elastic())
.subscribe(body -> log.trace("Response body: {}", body));
}
}
I got the error body by doing like this:
webClient
...
.retrieve()
.onStatus(HttpStatus::isError, response -> response.bodyToMono(String.class) // error body as String or other class
.flatMap(error -> Mono.error(new RuntimeException(error)))) // throw a functional exception
.bodyToMono(MyResponseType.class)
.block();
You could also do this
return webClient.getWebClient()
.post()
.uri("/api/Card")
.body(BodyInserters.fromObject(cardObject))
.exchange()
.flatMap(clientResponse -> {
if (clientResponse.statusCode().is5xxServerError()) {
clientResponse.body((clientHttpResponse, context) -> {
return clientHttpResponse.getBody();
});
return clientResponse.bodyToMono(String.class);
}
else
return clientResponse.bodyToMono(String.class);
});
Read this article for more examples link, I found it to be helpful when I experienced a similar problem with error handling
I do something like this:
Mono<ClientResponse> responseMono = requestSpec.exchange()
.doOnNext(response -> {
HttpStatus httpStatus = response.statusCode();
if (httpStatus.is4xxClientError() || httpStatus.is5xxServerError()) {
throw new WebClientException(
"ClientResponse has erroneous status code: " + httpStatus.value() +
" " + httpStatus.getReasonPhrase());
}
});
and then:
responseMono.subscribe(v -> { }, ex -> processError(ex));
Note that as of writing this, 5xx errors no longer result in an exception from the underlying Netty layer. See https://github.com/spring-projects/spring-framework/commit/b0ab84657b712aac59951420f4e9d696c3d84ba2
I had just faced the similar situation and I found out webClient does not throw any exception even it is getting 4xx/5xx responses. In my case, I use webclient to first make a call to get the response and if it is returning 2xx response then I extract the data from the response and use it for making the second call. If the first call is getting non-2xx response then throw an exception. Because it is not throwing exception so when the first call failed and the second is still be carried on. So what I did is
return webClient.post().uri("URI")
.header(HttpHeaders.CONTENT_TYPE, "XXXX")
.header(HttpHeaders.ACCEPT, "XXXX")
.header(HttpHeaders.AUTHORIZATION, "XXXX")
.body(BodyInserters.fromObject(BODY))
.exchange()
.doOnSuccess(response -> {
HttpStatus statusCode = response.statusCode();
if (statusCode.is4xxClientError()) {
throw new Exception(statusCode.toString());
}
if (statusCode.is5xxServerError()) {
throw new Exception(statusCode.toString());
}
)
.flatMap(response -> response.bodyToMono(ANY.class))
.map(response -> response.getSomething())
.flatMap(something -> callsSecondEndpoint(something));
}
Using what I learned this fantastic SO answer regarding the "Correct way of throwing exceptions with Reactor", I was able to put this answer together. It uses .onStatus, .bodyToMono, and .handle to map the error response body to an exception.
// create a chicken
webClient
.post()
.uri(urlService.getUrl(customer) + "/chickens")
.contentType(MediaType.APPLICATION_JSON)
.body(Mono.just(chickenCreateDto), ChickenCreateDto.class) // outbound request body
.retrieve()
.onStatus(HttpStatus::isError, clientResponse ->
clientResponse.bodyToMono(ChickenCreateErrorDto.class)
.handle((error, sink) ->
sink.error(new ChickenException(error))
)
)
.bodyToMono(ChickenResponse.class)
.subscribe(
this::recordSuccessfulCreationOfChicken, // accepts ChickenResponse
this::recordUnsuccessfulCreationOfChicken // accepts throwable (ChickenException)
);
We have finally understood what is happening :
By default the Netty's httpclient (HttpClientRequest) is configured to fail on server error (response 5XX) and not on client error (4XX), this is why it was always emitting an exception.
What we have done is extend AbstractClientHttpRequest and ClientHttpConnector to configure the httpclient behave the way the want and when we are invoking the WebClient we use our custom ClientHttpConnector :
WebClient.builder().clientConnector(new CommonsReactorClientHttpConnector()).build();
The retrieve() method in WebClient throws a WebClientResponseException
whenever a response with status code 4xx or 5xx is received.
You can handle the exception by checking the response status code.
Mono<Object> result = webClient.get().uri(URL).exchange().log().flatMap(entity -> {
HttpStatus statusCode = entity.statusCode();
if (statusCode.is4xxClientError() || statusCode.is5xxServerError())
{
return Mono.error(new Exception(statusCode.toString()));
}
return Mono.just(entity);
}).flatMap(clientResponse -> clientResponse.bodyToMono(JSONObject.class))
Reference: https://www.callicoder.com/spring-5-reactive-webclient-webtestclient-examples/
I stumbled across this so figured I might as well post my code.
What I did was create a global handler that takes career of request and response errors coming out of the web client. This is in Kotlin but can be easily converted to Java, of course. This extends the default behavior so you can be sure to get all of the automatic configuration on top of your customer handling.
As you can see this doesn't really do anything custom, it just translates the web client errors into relevant responses. For response errors the code and response body are simply passed through to the client. For request errors currently it just handles connection troubles because that's all I care about (at the moment), but as you can see it can be easily extended.
#Configuration
class WebExceptionConfig(private val serverProperties: ServerProperties) {
#Bean
#Order(-2)
fun errorWebExceptionHandler(
errorAttributes: ErrorAttributes,
resourceProperties: ResourceProperties,
webProperties: WebProperties,
viewResolvers: ObjectProvider<ViewResolver>,
serverCodecConfigurer: ServerCodecConfigurer,
applicationContext: ApplicationContext
): ErrorWebExceptionHandler? {
val exceptionHandler = CustomErrorWebExceptionHandler(
errorAttributes,
(if (resourceProperties.hasBeenCustomized()) resourceProperties else webProperties.resources) as WebProperties.Resources,
serverProperties.error,
applicationContext
)
exceptionHandler.setViewResolvers(viewResolvers.orderedStream().collect(Collectors.toList()))
exceptionHandler.setMessageWriters(serverCodecConfigurer.writers)
exceptionHandler.setMessageReaders(serverCodecConfigurer.readers)
return exceptionHandler
}
}
class CustomErrorWebExceptionHandler(
errorAttributes: ErrorAttributes,
resources: WebProperties.Resources,
errorProperties: ErrorProperties,
applicationContext: ApplicationContext
) : DefaultErrorWebExceptionHandler(errorAttributes, resources, errorProperties, applicationContext) {
override fun handle(exchange: ServerWebExchange, throwable: Throwable): Mono<Void> =
when (throwable) {
is WebClientRequestException -> handleWebClientRequestException(exchange, throwable)
is WebClientResponseException -> handleWebClientResponseException(exchange, throwable)
else -> super.handle(exchange, throwable)
}
private fun handleWebClientResponseException(exchange: ServerWebExchange, throwable: WebClientResponseException): Mono<Void> {
exchange.response.headers.add("Content-Type", "application/json")
exchange.response.statusCode = throwable.statusCode
val responseBodyBuffer = exchange
.response
.bufferFactory()
.wrap(throwable.responseBodyAsByteArray)
return exchange.response.writeWith(Mono.just(responseBodyBuffer))
}
private fun handleWebClientRequestException(exchange: ServerWebExchange, throwable: WebClientRequestException): Mono<Void> {
if (throwable.rootCause is ConnectException) {
exchange.response.headers.add("Content-Type", "application/json")
exchange.response.statusCode = HttpStatus.BAD_GATEWAY
val responseBodyBuffer = exchange
.response
.bufferFactory()
.wrap(ObjectMapper().writeValueAsBytes(customErrorWebException(exchange, HttpStatus.BAD_GATEWAY, throwable.message)))
return exchange.response.writeWith(Mono.just(responseBodyBuffer))
} else {
return super.handle(exchange, throwable)
}
}
private fun customErrorWebException(exchange: ServerWebExchange, status: HttpStatus, message: Any?) =
CustomErrorWebException(
Instant.now().toString(),
exchange.request.path.value(),
status.value(),
status.reasonPhrase,
message,
exchange.request.id
)
}
data class CustomErrorWebException(
val timestamp: String,
val path: String,
val status: Int,
val error: String,
val message: Any?,
val requestId: String,
)
Actually, you can log the body easily in the onError call:
.doOnError {
logger.warn { body(it) }
}
and:
private fun body(it: Throwable) =
if (it is WebClientResponseException) {
", body: ${it.responseBodyAsString}"
} else {
""
}
For those that wish to the details of a WebClient request that triggered a 500 Internal System error, override the DefaultErrorWebExceptionHandler like as follows.
The Spring default is to tell you the client had an error, but it does not provide the body of the WebClient call, which can be invaluable in debugging.
/**
* Extends the DefaultErrorWebExceptionHandler to log the response body from a failed WebClient
* response that results in a 500 Internal Server error.
*/
#Component
#Order(-2)
public class ExtendedErrorWebExceptionHandler extends DefaultErrorWebExceptionHandler {
private static final Log logger = HttpLogging.forLogName(ExtendedErrorWebExceptionHandler.class);
public FsErrorWebExceptionHandler(
ErrorAttributes errorAttributes,
Resources resources,
ServerProperties serverProperties,
ApplicationContext applicationContext,
ServerCodecConfigurer serverCodecConfigurer) {
super(errorAttributes, resources, serverProperties.getError(), applicationContext);
super.setMessageWriters(serverCodecConfigurer.getWriters());
super.setMessageReaders(serverCodecConfigurer.getReaders());
}
/**
* Override the default error log behavior to provide details for WebClientResponseException. This
* is so that administrators can better debug WebClient errors.
*
* #param request The request to the foundation service
* #param response The response to the foundation service
* #param throwable The error that occurred during processing the request
*/
#Override
protected void logError(ServerRequest request, ServerResponse response, Throwable throwable) {
// When the throwable is a WebClientResponseException, also log the body
if (HttpStatus.resolve(response.rawStatusCode()) != null
&& response.statusCode().equals(HttpStatus.INTERNAL_SERVER_ERROR)
&& throwable instanceof WebClientResponseException) {
logger.error(
LogMessage.of(
() ->
String.format(
"%s 500 Server Error for %s\n%s",
request.exchange().getLogPrefix(),
formatRequest(request),
formatResponseError((WebClientResponseException) throwable))),
throwable);
} else {
super.logError(request, response, throwable);
}
}
private String formatRequest(ServerRequest request) {
String rawQuery = request.uri().getRawQuery();
String query = StringUtils.hasText(rawQuery) ? "?" + rawQuery : "";
return "HTTP " + request.methodName() + " \"" + request.path() + query + "\"";
}
private String formatResponseError(WebClientResponseException exception) {
return String.format(
"%-15s %s\n%-15s %s\n%-15s %d\n%-15s %s\n%-15s '%s'",
" Message:",
exception.getMessage(),
" Status:",
exception.getStatusText(),
" Status Code:",
exception.getRawStatusCode(),
" Headers:",
exception.getHeaders(),
" Body:",
exception.getResponseBodyAsString());
}
}
You have to cast the "Throwable e" parameter to WebClientResponseException, then you can call getResponseBodyAsString() :
WebClient webClient = WebClient.create("https://httpstat.us/404");
Mono<Object> monoObject = webClient.get().retrieve().bodyToMono(Object.class);
monoObject.doOnError(e -> {
if( e instanceof WebClientResponseException ){
System.out.println(
"ResponseBody = " +
((WebClientResponseException) e).getResponseBodyAsString()
);
}
}).subscribe();
// Display : ResponseBody = 404 Not Found

Categories

Resources