I'm trying to decode a multipart-related request that is just a simple multi files download but with a specific content type by part (application/dicom and not application/octet-stream).
Since the structure of the response body might be identical, I could just tell the "multipart codec" to treat that content type as an octet-stream.
public Flux<FilePart> getDicoms(String seriesUri) {
return webClient.get()
.uri(seriesUri)
.accept(MediaType.ALL)
.retrieve()
.bodyToFlux(FilePart.class);
}
How can I do that?
An easier way of reading a multipart response:
private Mono<ResponseEntity<Flux<Part>>> queryForFiles(String uri)
final var partReader = new DefaultPartHttpMessageReader();
partReader.setStreaming(true);
return WebClient.builder()
.build()
.get()
.uri(wadoUri)
.accept(MediaType.ALL)
.retrieve()
.toEntityFlux((inputMessage, context) -> partReader.read(ResolvableType.forType(DataBuffer.class), inputMessage, Map.of())))
This is what I've done to make it work. I used directly the DefaultPartHttpMessageReader class to do it cleanly (spring 5.3).
public Flux<Part> getDicoms(String wadoUri) {
final var partReader = new DefaultPartHttpMessageReader();
partReader.setStreaming(true);
return WebClient.builder()
.build()
.get()
.uri(wadoUri)
.accept(MediaType.ALL)
//.attributes(clientRegistrationId("keycloak"))
.exchange()
.flatMapMany(clientResponse -> {
var message = new ReactiveHttpInputMessage() {
#Override
public Flux<DataBuffer> getBody() {
return clientResponse.bodyToFlux(DataBuffer.class);
}
#Override
public HttpHeaders getHeaders() {
return clientResponse.headers().asHttpHeaders();
}
};
return partReader.read(ResolvableType.forType(DataBuffer.class), message, Map.of());
});
}
Related
This test case is for mocking the health check contract.
TestClass
#Pact(consumer = "Consumer")
public RequestResponsePact getHealthCheck(PactDslWithProvider builder) {
Map<String, String> headers = new HashMap<String, String>();
headers.put("Content-Type", "text/plain");
headers.put("callchainid", "a4275861-f60a-44ab-85a6-c0c2c9df5e27");
return builder
.given("get health check")
.uponReceiving("get health data")
.path("/health")
.method("GET")
.headers(headers )
.willRespondWith()
.status(200)
.body("{\"status\":\"UP\",\"components\":{\"db\":{\"status\":\"UP\",\"details\":{\"database\":\"PostgreSQL\",\"validationQuery\":\"isValid()\"}}}}")
.toPact();
}
#Test
#PactTestFor(pactMethod = "getHealthCheck")
void getHealthData(MockServer mockServer) {
WebClient webClient=WebClient.builder().baseUrl(mockServer.getUrl()).build();
final String callChainId="a4275861-f60a-44ab-85a6-c0c2c9df5e27";
ThreadContext.put(CallChainIdService.HEADER_NAME, callChainId);
AsyncClient asyncClient=new AsyncClient(webClient);
Mono<ClientResponse> responseMono=asyncClient.getHealthCheck();
System.out.println(responseMono);
}
Here the webclient end point code which i am trying to hit,
AsyncClient Class
private final WebClient CLIENT;
#Override
public Mono<ClientResponse> getHealthCheck() {
return get(MediaType.TEXT_PLAIN, "/health");
}
private Mono<ClientResponse> get(MediaType contentType, String uri, Object... params) {
return CLIENT
.mutate()
.defaultHeader(CallChainIdService.HEADER_NAME, ThreadContext.get(CallChainIdService.HEADER_NAME))
.build()
.get()
.uri(uri, params)
.accept(contentType)
.exchange();
}
When i run the test , i got PactMismatchesException: The following requests were not received.
au.com.dius.pact.consumer.PactMismatchesException: The following requests were not received:
method: GET
path: /health
query: {}
headers: {callchainid=[a4275861-f60a-44ab-85a6-c0c2c9df5e27], Content-Type=[text/plain]}
matchers: MatchingRules(rules={})
generators: Generators(categories={})
body: MISSING
I am not sure what i am doing wrong here. Appreciate your inputs and help
It looks like you have content-type as the expected header you send in the GET call, but in fact you send only an accept header (both with the media type text/plain).
I think your test should be updated to use accept.
I have a service that is returning a value which contains delay information.
public Mono<R> authenticate(
A authenticationRequest,
#RequestHeader Map<String, String> headers,
ServerHttpResponse serverHttpResponse) {
final AuthServiceResponse<R> authenticationResponse = authService.authenticate(authenticationRequest, headers);
serverHttpResponse.setStatusCode(authenticationResponse.getStatusCode());
return Mono.just(authenticationResponse.getOperationResponse())
.delayElement(authenticationResponse.getDelay());
}
I'd like to try to convert it so it is reactive I got this far...
public Mono<R> authenticate(
A authenticationRequest,
#RequestHeader Map<String, String> headers,
ServerHttpResponse serverHttpResponse) {
return authService.authenticate(authenticationRequest, headers)
.map(authenticationResponse->{
serverHttpResponse.setStatusCode(authenticationResponse.getStatusCode());
return authenticationResponse.getOperationResponse()
});
...
but I wasn't sure how to add the "delayElement" capability.
You can use Mono.fromCallable + delayElement within a flatMap like this:
return authService.authenticate(authenticationRequest, headers)
.flatMap(authenticationResponse -> {
return Mono.fromCallable(() -> authenticationResponse.getOperationResponse())
.delayElement(authenticationResponse.getDelay())
});
One thing to note... you cannot pass ServerHttpResponse in this situation as a parameter, but you have ServerWebExchange which has the request and response along with the headers. The complete solution is
public Mono<R> authenticate(
#RequestBody SimpleAuthenticationRequest authenticationRequest,
ServerWebExchange serverWebExchange) {
return authService
.authenticate(authenticationRequest, serverWebExchange.getRequest().getHeaders())
.doOnNext(
serviceResponse ->
serverWebExchange.getResponse().setStatusCode(serviceResponse.getStatusCode()))
.flatMap(
serviceResponse ->
Mono.fromCallable(serviceResponse::getOperationResponse)
.delayElement(serviceResponse.getDelay()));
}
Try this to add delay based on your authenticationResponse.getDelay() value
public Mono<Object> authenticate(Object authenticationRequest,#RequestHeader Object headers,
Object serverHttpResponse) {
return authenticate(authenticationRequest,headers)
.flatMap(authenticationResponse -> {
Mono<String> delayElement = Mono.just("add delay")
.delayElement(Duration.ofSeconds(authenticationResponse.getDelay()));
Mono<Object> actualResponse =Mono.just(authenticationResponse.getOperationResponse());
return Mono.zip(delayElement,actualResponse).map(tupleT2 -> tupleT2.getT2());
});
}
let me know if it doesn't work. i will try to find other way.
Is there a way to add an optional header to WebClient get() request?
if (config.getRefresh()) {
webClient.header("Refresh-Cache", "true");
}
It seems the whole request is chained on webClient
return webClient
.get()
.uri(uri)
.header("Authorization", BEARER_TOKEN)
.retrieve()
.bodyToMono(String.class)
.block();
I try to switch to RequestHeadersSpec but got this generic type warning
WebClient.RequestHeadersSpec is a raw type. References to generic type
WebClient.RequestHeadersSpec<S> should be parameterized Java(16777788)
I know with post(), we can do this
requestBodySpec = webClientBuilder.build().post().uri(uri);
if (config.getRefresh()) {
requestBodySpec.header("Refresh-Cache", "true");
}
return requestBodySpec
.header("Authorization", BEARER_TOKEN)
.body(Mono.just(request), MyRequest.class)
.retrieve()
.bodyToMono(String.class)
.block();
To resolve the generic type warning, you can set the generic type as a wildcard (?) ie.
WebClient.RequestHeadersSpec<?> requestBodySpec = webClient.get().uri("https://google.com");
An alternate solution for adding a header based off of a flag in config is to use an ExchangeFilterFunction.
public class RefreshExchangeFilterFunction implements ExchangeFilterFunction {
private Config config;
#Override
public Mono<ClientResponse> filter(ClientRequest request, ExchangeFunction next) {
if(config.isRefresh()) {
return next.exchange(ClientRequest.from(request)
.header("Refresh-Cache", "true")
.build());
}
return next.exchange(request);
}
}
This can then be applied to any/all web clients that need this behaviour
WebClient.builder()
.filter(refreshExchangeFilterFunction)
.build();
This approach always worked when updating a token. That is, with each request if I received an error 401, the operator retryWhen() triggered it updated the token.
Here is the code:
private Observable<TokenModel> refreshAccessToken() {
Map<String, String> requestBody = new HashMap<>();
requestBody.put(Constants.EMAIL_KEY, Constants.API_EMAIL);
requestBody.put(Constants.PASSWORD_KEY, Constants.API_PASSWORD);
return RetrofitHelper.getApiService().getAccessToken(requestBody)
.subscribeOn(Schedulers.io())
.doOnNext((AccessToken refreshedToken) -> {
PreferencesHelper.putAccessToken(mContext, refreshedToken);
});
}
public Function<Observable<Throwable>, ObservableSource<?>> isUnauthorized (){
return throwableObservable -> throwableObservable.flatMap((Function<Throwable, ObservableSource<?>>) (Throwable throwable) -> {
if (throwable instanceof HttpException) {
HttpException httpException = (HttpException) throwable;
if (httpException.code() == 401) {
return refreshAccessToken();
}
}
return Observable.error(throwable);
});
}
I call isUnauthorized() at the retryWhen() operator where I make a request to the server
class RetrofitHelper {
static ApiService getApiService() {
return initApi();
}
private static OkHttpClient createOkHttpClient() {
final OkHttpClient.Builder httpClient = new OkHttpClient.Builder();
httpClient.addInterceptor(chain -> {
Request originalRequest = chain.request();
AccessToken accessToken= PreferencesHelper.getAccessToken(BaseApplication.getInstance());
String accessTokenStr = accessToken.getAccessToken();
Request.Builder builder =
originalRequest.newBuilder().header("Authorization", "Bearer " + accessTokenStr);
Request newRequest = builder.build();
return chain.proceed(newRequest);
});
return httpClient.build();
}
private static ApiService initApi(){
Retrofit retrofit = new Retrofit.Builder()
.baseUrl(Constants._api_url)
.addConverterFactory(GsonConverterFactory.create())
.addConverterFactory(ScalarsConverterFactory.create())
.addCallAdapterFactory(RxJava2CallAdapterFactory.create())
.client(createOkHttpClient())
.build();
return retrofit.create(ApiService.class);
}
}
But we recently added Basic Auth, and now at the first request I get 401 and retryWhen() tries to update the Token, but still gets 401. That is, the doOnNext() does not work, but immediately the onError() works
private static Observable<AccessToken> refreshAccessToken() {
return RetrofitHelper.getApiService()
.getAccessToken(
Credentials.basic(
Constants._API_USERNAME, Constants._API_PASSWORD
),
Constants._API_BODY_USERNAME,
Constants._API_BODY_PASSWORD,
Constants._API_BODY_GRANT_TYPE
)
.doOnNext((AccessToken refreshedToken) -> {
PreferencesHelper.putObject(BaseApplication.getInstance(), PreferenceKey.ACCESS_TOKEN_KEY, refreshedToken);
}
});
}
// Api Service
public interface ApiService {
// Get Bearer Token
#FormUrlEncoded
#POST("oauth/token")
Observable<AccessToken> getAccessToken(#Header("Authorization") String basicAuth,
#Field("username") String username,
#Field("password") String password,
#Field("grant_type") String grantType);
}
Here, tell me why this is a mistake? Why at the first request I get 401, and from the second request everything works?
I want to suggest a better solution.
public class RefreshTokenTransformer<T extends Response<?>> implements ObservableTransformer<T, T> {
private class HttpCode {
private static final int UNAUTHORIZED_HTTP_CODE = 401;
}
private ApiService mApiService;
private UserRepository mUserRepository;
public RefreshTokenTransformer(ApiService service, UserRepository userRepository) {
mApiService = service;
mUserRepository = userRepository;
}
#Override
public ObservableSource<T> apply(final Observable<T> stream) {
return stream.flatMap(new Function<T, ObservableSource<T>>() {
#Override
public ObservableSource<T> apply(T response) throws Exception {
if (response.code() == HttpCode.UNAUTHORIZED_HTTP_CODE) {
return mApiService.refreshToken(mUserRepository.getRefreshTokenHeaders())
.filter(new UnauthorizedPredicate<>(mUserRepository))
.flatMap(new Function<Response<TokenInfo>, ObservableSource<T>>() {
#Override
public ObservableSource<T> apply(Response<TokenInfo> tokenResponse) throws Exception {
return stream.filter(new UnauthorizedPredicate<T>(mUserRepository));
}
});
}
return stream;
}
});
}
private class UnauthorizedPredicate<R extends Response<?>> implements Predicate<R> {
private UserRepository mUserRepository;
private UnauthorizedPredicate(UserRepository userRepository) {
mUserRepository = userRepository;
}
#Override
public boolean test(R response) throws Exception {
if (response.code() == HttpCode.UNAUTHORIZED_HTTP_CODE) {
throw new SessionExpiredException();
}
if (response.body() == null) {
throw new HttpException(response);
}
Class<?> responseBodyClass = response.body().getClass();
if (responseBodyClass.isAssignableFrom(TokenInfo.class)) {
try {
mUserRepository.validateUserAccess((TokenInfo) response.body());
} catch (UnverifiedAccessException error) {
throw new SessionExpiredException(error);
}
}
return true;
}
}
}
I`ve written the custom operator, which makes next actions:
first request started, and we get 401 response code;
then we execute /refresh_token request to update the token;
after that if the token is refreshed successfully, we repeat the
first request. if /refresh_token token is failed, we throw exception
Then, you can easy implement it in the any request like that:
Observable
.compose(new RefreshTokenResponseTransformer<Response<{$your_expected_result}>>
(mApiService, mUserRepository()));
One more important thing:
Most likely, that your initial observable for retrofit has params, like that:
mApiService.someRequest(token)
if the param is expected to change during the performing RefreshTokenTransformer(e.g. /refresh_token request will get new access token and you save it somewhere, then you want to use a fresh access token to repeat the request) you will need to wrap your observable with defer operator to force the creating of new observable like that:
Observable.defer(new Callable<ObservableSource<Response<? extends $your_expected_result>>>() {
#Override
public Response<? extends $your_expected_result> call() throws Exception {
return mApiService.someRequest(token);
}
})
I think it does not need to use interceptor instead you implement Authenticator by which you can access refreshed token and okhttp automatically will handle that. if you get 401 it updates header with refreshed token and make new request.
public class TokenAuthenticator implements Authenticator {
#Override
public Request authenticate(Proxy proxy, Response response) throws IOException {
// Refresh your access_token using a synchronous api request
newAccessToken = service.refreshToken();
// Add new header to rejected request and retry it
return response.request().newBuilder()
.header(AUTHORIZATION, newAccessToken)
.build();
}
I am using spring-boot version 2.0.0.M6.
I need to make async HTTP calls from spring-boot app say APP1 to another app (play framework) say APP2.
So if I need to make 20 distinct async calls from APP1 to APP2, APP2 receives 20 requests out of which few are duplicates, which means these duplicates replaced few distinct requests.
Expected:
api/v1/call/1
api/v1/call/2
api/v1/call/3
api/v1/call/4
Actual:
api/v1/call/1
api/v1/call/2
api/v1/call/4
api/v1/call/4
I am using spring reactive WebClient.
Below is the spring boot version in build.gradle
buildscript {
ext {
springBootVersion = '2.0.0.M6'
//springBootVersion = '2.0.0.BUILD-SNAPSHOT'
}
repositories {
mavenCentral()
maven { url "https://repo.spring.io/snapshot" }
maven { url "https://repo.spring.io/milestone" }
maven {url "https://plugins.gradle.org/m2/"}
}
dependencies {
classpath("org.springframework.boot:spring-boot-gradle-plugin:${springBootVersion}")
classpath("se.transmode.gradle:gradle-docker:1.2")
}
}
My WebClient init snippet
private WebClient webClient = WebClient.builder()
.clientConnector(new ReactorClientHttpConnector((HttpClientOptions.Builder builder) -> builder.disablePool()))
.build();
My POST method
public <T> Mono<JsonNode> postClient(String url, T postData) {
return Mono.subscriberContext().flatMap(ctx -> {
String cookieString = ctx.getOrDefault(Constants.SubscriberContextConstnats.COOKIES, StringUtils.EMPTY);
URI uri = URI.create(url);
return webClient.post().uri(uri).body(BodyInserters.fromObject(postData)).header(HttpHeaders.COOKIE, cookieString)
.exchange().flatMap(clientResponse ->
{
return clientResponse.bodyToMono(JsonNode.class);
})
.onErrorMap(err -> new TurtleException(err.getMessage(), err))
.doOnSuccess(jsonData -> {
});
});
}
The code from where this postClient method is invoked
private void getResultByKey(PremiumRequestHandler request, String key, BrokerConfig brokerConfig) {
/* Live calls for the insurers */
LOG.info("[PREMIUM SERVICE] LIVE CALLLLL MADE FOR: " + key + " AND REQUEST ID: " + request.getRequestId());
String uri = brokerConfig.getHostUrl() + verticalResolver.determineResultUrl(request.getVertical()) + key;
LOG.info("[PREMIUM SERVICE] LIVE CALL WITH URI : " + uri + " FOR REQUEST ID: " + request.getRequestId());
Mono<PremiumResponse> premiumResponse = reactiveWebClient.postClient(uri, request.getPremiumRequest())
.map(json -> PlatformUtils.mapToClass(json, PremiumResponse.class));
premiumResponse.subscribe(resp -> {
resp.getPremiumResults().forEach(result -> {
LOG.info("Key " + result.getKey());
repository.getResultRepoRawType(request.getVertical())
.save(result).subscribe();
saveResult.subscriberContext(ctx -> {
MultiBrokerMongoDBFactory.setDatabaseNameForCurrentThread(brokerConfig.getBroker());
return ctx;
}).subscribe();
});
}, error -> {
LOG.info("[PREMIUM SERVICE] ERROR RECIEVED FOR " + key + " AND REQUEST ID" + request.getRequestId() + " > " + error.getMessage());
});
}
Had put logs at the end-point in the client code, can not see multiple requests at that point.
Probably it's a bug in WebClient where URI is getting swapped in multithreaded environment.
Tried mutating WebClient, still the URI is getting swapped
Please help.
Git Repo added github.com/praveenk007/ps-demo
I happend to have experienced similar problem:
When calling the same service (here marked as ExternalService) in parallel (webflux) sometimes the same request was being sent and the problem indeed seems to reside in Webclient.
The solution was in changing the way Webclient is being created.
Before:
Here ExternalCall is configured client which calls ExternalService. So the parallel execution here was of internalCall method. Note we pass WebClient.RequestBodySpec class to ExternalCall class
#Bean
ExternalCall externalCall(WebClient.Builder webClientBuilder) {
var exchangeStrategies = getExchangeStrategies();
var endpoint = "v1/data";
var timeout = 10000;
var uri = "https://externalService.com/";
var requestBodySpec = webClientBuilder.clone()
.clientConnector(new ReactorClientHttpConnector(HttpClient.create()))
.exchangeStrategies(exchangeStrategies)
.build()
.post()
.uri(endpoint)
.accept(TEXT_XML)
.contentType(TEXT_XML);
return new ExternalCall(requestBodySpec, uri, timeout);
}
Then within ExternalCall class I had had
private Mono<String> internalCall(String rq) {
return requestBodySpec.bodyValue(rq)
.retrieve()
.bodyToMono(String.class)
.timeout(timeout, Mono.error(() -> new TimeoutException(String.format("%s - timeout after %s seconds", "ExternalService", timeout.getSeconds()))));
}
After:
We pass WebClient class to ExternalCall.
#Bean
ExternalCall externalCall(WebClient.Builder webClientBuilder) {
var exchangeStrategies = getExchangeStrategies();
var timeout = 10000;
var uri = "https://externalService.com/";
var webClient = webClientBuilder.clone()
.clientConnector(new ReactorClientHttpConnector(HttpClient.create()))
.exchangeStrategies(exchangeStrategies)
.build();
return new ExternalCall(webClient, uri, timeout);
}
Now we specify RequestBodySpec within ExternalCall class:
private Mono<String> internalCall(String rq) {
return webClient
.post()
.uri(endpoint)
.accept(TEXT_XML)
.contentType(TEXT_XML)
.bodyValue(rq)
.retrieve()
.bodyToMono(String.class)
.timeout(timeout, Mono.error(() -> new TimeoutException(String.format("%s - timeout after %s seconds", "ExternalService", timeout.getSeconds()))));
}
Conclusions: So apparently the moment you create WebClient.RequestBodySpec instance matters. Hope that helps somebody
Adding my few observations:
webClient.get() or webClient.post() always returns new DefaultRequestBodyUriSpec on every call on which URI being called and I think it does not look like URI getting swapped.
class DefaultWebClient implements WebClient {
..
#Override
public RequestHeadersUriSpec<?> get() {
return methodInternal(HttpMethod.GET);
}
#Override
public RequestBodyUriSpec post() {
return methodInternal(HttpMethod.POST);
}
..
#Override
public Mono<ClientResponse> exchange() {
ClientRequest request = (this.inserter != null ?
initRequestBuilder().body(this.inserter).build() :
initRequestBuilder().build());
return exchangeFunction.exchange(request).switchIfEmpty(NO_HTTP_CLIENT_RESPONSE_ERROR);
}
private ClientRequest.Builder initRequestBuilder() {
URI uri = (this.uri != null ? this.uri : uriBuilderFactory.expand(""));
return ClientRequest.create(this.httpMethod, uri)
.headers(headers -> headers.addAll(initHeaders()))
.cookies(cookies -> cookies.addAll(initCookies()))
.attributes(attributes -> attributes.putAll(this.attributes));
}
..
}
and methodInternal method looks like below
#SuppressWarnings("unchecked")
private RequestBodyUriSpec methodInternal(HttpMethod httpMethod) {
return new DefaultRequestBodyUriSpec(httpMethod);
}
In addition while making actual request also, new ClientRequest is created.
Class source
https://github.com/spring-projects/spring-framework/blob/master/spring-webflux/src/main/java/org/springframework/web/reactive/function/client/DefaultWebClient.java