I am working on a project which until now has been using the org.springframework.web.client.RestTemplate
I have recently understood that this class is to be deprecated in favour of the Asynch org.springframework.web.reactive.function.client.WebClient framework.
Now I am massively in favour of this, as my application is suffering from long delays on waiting from
responses from RestTemplate (GET) calls (in which time I could be doing database stuff etc.).
The problem that I have now is that if I make a call like:
final Mono<String> call = webClient
.get()
.uri("/base/recordPath/1?format=json")
.header("Content-Type", MediaType.APPLICATION_JSON_VALUE)
.accept(MediaType.APPLICATION_JSON)
.retrieve()
.bodyToMono(String.class)
when I make a subsequent call like:
System.out.println(call.block());
I get the expected output (a String version of a populated Json Object).
however if I change the earlier call to (which I want to do!):
final Mono<JsonObject> call = webClient
.get()
.uri("/base/recordPath/1?format=json")
.header("Content-Type", MediaType.APPLICATION_JSON_VALUE)
.accept(MediaType.APPLICATION_JSON)
.retrieve()
.bodyToMono(JsonObject.class)
when I do:
System.out.println(call.block());
I just get {} instead of a populated JsonObject
So it looks like the bodyToMono() hasn't done what 'I' expected
When I used RestTemplate, there was a way to register serialisers with the Template (though this wasn't necessary for JsonObject),
is this also necessary with WebClient? If so how do you do it?
I would be grateful of help.
Thanks
Bill
N.B. I'm not sure if this has any relevance, but the rest endpoint I am accessing does have an IP restriction, so if in some way the WebClient were to alter the originating IP, this may have some effect. Though I would have thought it would be more like a 4** of some sort, I'm not seeing any of those!
Or possibly a Media type issue, as it is going through a DMZ, where the guardian may be changing an 'unauthorised' request from application/json to text/* for example.
Another point which may have relevance is that to get a successful start of the application it is necessary to run it with the following property:
spring.main.web-application-type=none
Update
I now have my application running, though not as I want!
The issue appeared to be transitive dependencies imported by the team pom I am required to use (as a parent pom).
I now get a successful start of the project. But still find that the json Object (which is now Jackson) is still Empty (as reported by object.isEmpty()).
my dependencies/versions now are:
org.springframework:5.2.8.RELEASE
org.springframework.boot:2.3.3.RELEASE
com.fasterxml.jackson:2.11.2
I know I am fighting a parent pom which is against what am trying to do but would like to know what the dependencies I really need are
Related
I am writing a Pact Consumer and Provider Test .
Basically I have 3 problems at the moment.
#Pact(provider = PROVIDER, consumer = CONSUMER)
public RequestResponsePact getAddress(PactDslWithProvider builder) {
PactDslJsonBody body = new PactDslJsonBody()
.stringType("key1", "73501")
.stringType("key2", "value");
final Map<String,String> headers = new HashMap<>();
headers.put("Authorization","Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsIng1");
return builder
.given("Something.")
.uponReceiving("Dto")
.path("/amz/ags")
.query("code=9999")
.method("GET")
.headers(headers)
.willRespondWith()
.status(200)
.body(body)
.toPact();
}
1) If I add a header in my Consumer Test I get a 500 status Error back
2) As you just saw in the Consumer I tried to add Authorisation Header , but in the Provider it will be expired, so I have to find a way to hack something and provide a valid token.
3) In the provider REST Api other services are getting called but it seems I have to mock them at least whenever they are called an Exception gets thrown. Honestly I also don't know how to do this. In which method I need to do it. Why at all do I have to mock them because the external Apis are running.
for 2) I found a solution
#TestTemplate
#ExtendWith(PactVerificationInvocationContextProvider.class)
void pactVerificationTestTemplate(PactVerificationContext context, HttpRequest httpRequest) {
httpRequest.addHeader("Authorization", "Bearer " + "eyJ0eXAiOiJKV1Qi");
context.verifyInteraction();
}
But now I get in context.verifyInteraction() an Exception. Very strange.
I can't answer the JVM specific question, however.
If I add a header in my Consumer Test I get a 500 status Error back
Usually, this means that you're told Pact you are going to do something but did not do it. In this case, my guess is the bearer token didn't match or it didn't receive the correct header. There should be logs or a junit report with the details.
As you just saw in the Consumer I tried to add Authorisation Header , but in the Provider it will be expired, so I have to find a way to hack something and provide a valid token.
For dealing with authentication/authorization, you may want to read the strategies here:
https://docs.pact.io/provider/handling_auth
In the provider REST Api other services are getting called but it seems I have to mock them at least whenever they are called an Exception gets thrown. Honestly I also don't know how to do this. In which method I need to do it. Why at all do I have to mock them because the external Apis are running.
Pact is intended to be closer to a unit test, running external services during Pact tests is not recommended, because it makes tests less deterministic. See also this section which discusses the provider test coverage: https://docs.pact.io/5-minute-getting-started-guide#scope-of-a-provider-pact-test.
On the question of how to handle token expiry, I did like this.
The given:
JWT token contains an expire_at claim
I wrote a small library to share within all the projects where I have a class like JwtTestHelper which generates any JWT with token started_at date as a constant date and expire_at is started_at plus like 100 years.
It's important here to make it generate the same token for the same input parameters from run-to-run or the header will be different which will cause a new pact to be considered by the broker. This dramatically changes the approach - you basically need to verify every new pact each time.
On the question of handling exceptions. Do not cheat like this
httpRequest.addHeader("Authorization", "Bearer " + "eyJ0eXAiOiJKV1Qi");
Give your client class code a chance to pass the values naturally, pass it somehow or mock using the same JwtTestHelper
I know sending a body with a GET request isn't the best idea but I'm trying to consume an existing API which requires it.
Sending a body with POST is straight-forward:
webClient.post()
.uri("/employees")
.body(Mono.just(empl), Employee.class)
.retrieve()
.bodyToMono(Employee.class);
It won't work with webClient.get() though, because while the post() method returns a WebClient.RequestBodyUriSpec, the get() method returns WebClient.RequestHeadersUriSpec<?>, which doesn't seem to allow any body definitions.
I've found a workaround for Spring RestTemplate here: RestTemplate get with body,
but had no luck finding any for the new WebClient.
While the other responses are correct that you shouldn't use a body with a GET request, that is not helpful when you do not own, or cannot change the already existing method you are calling.
The problems is WebClient#get returns a WebClient.RequestHeadersUriSpec which does not provide a way for us to set the body.
WebClient#post returns a WebClient.RequestBodyUriSpec which does provide us a way to set the body but will cause us to use the wrong HTTP method, POST instead of GET.
Thankfully for us stuck in this situation there is WebClient#method which returns a WebClient.RequestBodyUriSpec and allows us to set the HTTP method.
webClient.method(HttpMethod.GET)
.uri("/employees")
.body(Mono.just(empl), Employee.class)
.retrieve()
.bodyToMono(Employee.class);
You may still run into issues in your testing libraries though...
A GET reques has no body. It is forbidden (well, not forbidden, but not used at all) by the HTTP specification. You have two approaches here:
Do a POST. It is there just for that.
Use a query string and pass the data in that part of the URL.
Of course, you can attach the needed fields and pass a payload to the GET request, but it will probably be ignored, or worse, identified as an error and rejected by the server, before your served code has access to it. But if you are passing data to the server to do some processing with it, then POST is what you need to use.
Extracted from RFC-7231. HTTP 1.1. Semantics and code:
A payload within a GET request message has no defined semantics;
sending a payload body on a GET request might cause some existing
implementations to reject the request.
(markup is mine)
Reasons for this are, mainly, that a GET method must be idempotent, producing the same output for the same URL, if repeated. POST doesn't have these requirements, so POST is your friend.
Small question regarding Spring Webflux, and how to "chain" http calls please.
With a concrete example, here is a very straightforward sample with Spring MVC, with a rest template.
RestTemplate restTemplate = new RestTemplate();
HttpEntity<StepOneRequest> stepOneRequestHttpEntity = new HttpEntity<>(new StepOneRequest("foo", "bar"));
StepOneResponse stepOneResponse = restTemplate.postForObject("https://step-one-web-service:443/getStepTwoFromFooBar", stepOneRequestHttpEntity, StepOneResponse.class);
HttpEntity<StepTwoRequest> stepTwoRequestHttpEntity = new HttpEntity<>(new StepTwoRequest(stepOneResponse.getTheDependencyFromStepOne()));
StepTwoResponse stepTwoResponse = restTemplate.postForObject("https://step-two-web-service:443/getResponseFromStepTwo", stepTwoRequestHttpEntity, StepTwoResponse);
return ResponseEntity.ok(new MyResponse(stepTwoResponse.getImportantValueFromStepTwo()));
}
In this snippet, we see very straightforward.
Initialisation of only one rest template.
Construction of a http request payload object.
Use the constructed object to query a first external web application API to get a response.
Important, the response of the first HTTP call is needed to make the second HTTP call. They can only be sequential, it needs the result of the first call, to be able to query the second API.
Then, the http call to the second API.
Finally, response based on the second API to the original requester.
Now, if this is translated to Webflux:
WebClient webClientStepOne = WebClient.create("https://step-one-web-service:443/getStepTwoFromFooBar");
WebClient webClientStepTwo = WebClient.create("https://step-two-web-service:443/getResponseFromStepTwo");
Mono<StepOneResponse> stepOneResponseMono = webClientStepOne.post().body(BodyInserters.fromValue(new StepOneRequest("foo", "bar"))).retrieve().bodyToMono(StepOneResponse.class);
Mono<StepTwoResponse> stepTwoResponseMono = stepOneResponseMono.flatMap(aStepOneResponse -> webClientStepTwo.post().body(BodyInserters.fromValue(aStepOneResponse.getTheDependencyFromStepOne())).retrieve().bodyToMono(StepTwoResponse.class));
return stepTwoResponseMono.map(aStepTwoResponse -> ResponseEntity.ok(new MyResponse(aStepTwoResponse.getImportantValueFromStepTwo())));
}
I have one big question here. While this is correct (it yields the same response) is this the correct thing to do?
Especially, the line with the flatMap
Mono<StepTwoResponse> stepTwoResponseMono = stepOneResponseMono.flatMap(aStepOneResponse -> webClientStepTwo.post().body(BodyInserters.fromValue(aStepOneResponse.getTheDependencyFromStepOne())).retrieve().bodyToMono(StepTwoResponse.class));
It seems very out of the picture. Is there a better way here?
Also, a side question, do we really need N WebClient if I need to chain N api calls please? The URLs are different, not just the path, the complete URL. When rest template can do with only one instance, it seems here, I need N WebClient if I need to call N external services.
What is the correct way to use WebClient here please?
What is the correct way to chain http calls please?
Thank you
I have one big question here. While this is correct (it yields the same response) is this the correct thing to do?
Essentially, yes, you've done it correctly. The only things that aren't really correct there are your Webclient usage, and your style.
You certainly don't have to specify a WebClient per URI (and you shouldn't, they're meant to be reusable.) So if you have different domains and the same webclient, just create a standard instance:
WebClient wc = WebClient.create();
...and then use:
webClientStepOne.post()
.uri("https://blah")
...at the top of your chains to specify a different URI with the same instance.
In terms of style - you'll have noticed the code gets unwieldy very quickly when split up into separate variables and written on long lines. Counterintuitive as it might first seem, the generally accepted best practice in reactive land is to do everything in a single statement, formatting each new method call on a separate line (enabling you to read down and then see the execution of your reactive chain.) So in your case, it would become:
return wc.post().uri("https://step-one-web-service:443/getStepTwoFromFooBar")
.body(BodyInserters.fromValue(new StepOneRequest("foo", "bar")))
.retrieve()
.bodyToMono(StepOneResponse.class)
.flatMap(aStepOneResponse ->
wc.post().uri("https://step-two-web-service:443/getResponseFromStepTwo")
.body(BodyInserters.fromValue(aStepOneResponse.getTheDependencyFromStepOne()))
.retrieve()
.bodyToMono(StepTwoResponse.class)
)
.map(aStepTwoResponse ->
ResponseEntity.ok(new MyResponse(aStepTwoResponse.getImportantValueFromStepTwo()))
);
You could tidy it up a bit maybe, for instance by delegating the inner flatmap call to a separate method - but that really just comes down to preference.
I am receiving a MonoError caused by an IllegalStateException with the cause 'The underlying HTTP client completed without emitting a response.'.
NetworkList is a class contains lists of a network topology and has a setList which is called from within the method convertJsonToPojo.
client is a WebClient (spring-webflux) in Spring 5 (5.0.4.RELEASE) using reactor-netty (0.7.5.RELEASE).
The following snippet shows how I am using WebClient to make asynchronous calls to type = BOARDS, CARDS, PORTS, ROUTERS, etc where the network name is passed in as projectName and the topology type is passed in as type.
NetworkList netList = new NetworkList();
client.get().uri("/{project}/{type}",projectName,type).retrieve().bodyToMono(String.class).subscribe(json -> this.convertJsonToPojo(Type.BOARD, json, netList));
The rest service I am hitting against is a singular service which returns json representing the components asked for.
This seems similar to Reactive WebClient Not Emitting A Response but a response within points to the following reactor-netty issue Issue 138. That issue indicates fixed as of 0.6.6.RELEASE.
So my question is am I using WebClient wrong unknowingly or should I submit this as an unfixed bug? I am fairly new to Spring 5 and the Reactive library so do not want to naively submit a bug.
Thank you for your time in advance!
PS - I have this working with RestTemplate and CompletableFuture so no need to offer that suggestion as an alternative. My tasking is to begin the migration to a reactive architecture. Thank you!
UPDATE - I believe I understand now what is going on but would like a confirmation from someone. The rest service I am hitting against is a traditional one and not a Reactive Stream. Thus, there are no 'stage' updates being shared during the call. .subscribe() is expecting a reactive stream and thus never received the complete stage. replacing .subscribe() with .block() forced a completion and the code works now. So my learning from this is that when dealing with traditional rest services a block is required. Can someone confirm that please? Thank you!
I have a somewhat simplistic controller configured as thus:
#RequestMapping(value = "user/savearticle", method = RequestMethod.POST)
public #ResponseBody
Object saveArticle(#ModelAttribute("article")RawArticle rawArticle);
Using snippets of code taken from here, I made a test case for the controller that looks like this:
MvcResult resultActions =
mockMvc.perform(MockMvcRequestBuilders.post("/user/savearticle")
.contentType(MediaType.APPLICATION_FORM_URLENCODED)
.content(convertObjectToForumUrlEncodedBytes(rawArticle)))
.andReturn();
and I simply print out the result. In any case, the ModelAttribute "rawArticle" keeps ending up as null when it enters the controller's implementation, however when I use this:
MvcResult resultActions = mockMvc.perform(
MockMvcRequestBuilders.post("/user/savearticle")
.param("title", rawArticle.getTitle())
.param("tags", rawArticle.getTags())
.param("body", rawArticle.getBody())
.param("author", rawArticle.getAuthor())).andReturn();
the mapping actually works like a charm. What I want though is that the first test be processed correctly as it seems so wrong that it's not being mapped as I thought it should be, similarly the controller is primarily being used by another program over the network using apache http (which somehow automatically passes a urlencoded form).
Do you guys have any idea where I could've made an error? I wouldn't mind posting snippets of my context configuration if you think you need it to evaluate the problem (or my pom for that matter, but just telling me what libraries I may have missed should be enough)
Update:
I made a mistake of inserting the POJO into a session in test number 1, I simply removed it here. The question stands the same.
The content is the content body of the request, you are using a content type which expects everything encoded in the URL not in the content body. Spring does data binding based on the request parameters if you want to use the content body you have to use #RequestBody instead of #ModelAttribute.