Recently I've started to use Spring's MockRestServiceServer to verify my RestTemplate based requests in tests.
When its used for simple get/post request - all good, however, I couldn't figure out how to use it with POST multipart request:
For example, my working code that I would like to test looks like this:
public ResponseEntity<String> doSomething(String someParam, MultipartFile
file, HttpHeaders headers) { //I add headers from request
MultiValueMap<String, Object> map = new LinkedMultiValueMap<>();
map.add("file", new ByteArrayResource(file.getBytes()) {
#Override
public String getFilename() {
return file.getOriginalFilename();
}
});
map.add("someParam", someParam);
HttpEntity<MultiValueMap<String, Object>> requestEntity = new
HttpEntity<>(map, headers);
return this.restTemplate.exchange(
getDestinationURI(),
HttpMethod.POST,
requestEntity,
String.class);
}
So my question is How I can specify my expectations with org.springframework.test.web.client.MockRestServiceServer? Please notice, that I don't want to just mock the "exchange" method with mockito or something, but prefer to use MockRestServiceServer
I'm using spring-test-4.3.8.RELEASE version
A code snippet would be really appreciated :)
Thanks a lot in advance
Update:
As per James's request I'm adding non-working test snippet (Spock test):
MockRestServiceServer server = MockRestServiceServer.bindTo(restTemplate).build()
server.expect(once(), requestTo(getURI()))
.andExpect(method(HttpMethod.POST))
.andExpect(header(HttpHeaders.CONTENT_TYPE, startsWith("multipart/form-data;boundary=")))
.andExpect(content().formData(["someParam" : "SampleSomeParamValue", "file" : ???????] as MultiValueMap))
.andRespond(withSuccess("sample response", MediaType.APPLICATION_JSON))
multipartFile.getBytes() >> "samplefile".getBytes()
multipartFile.getOriginalFilename() >> "sample.txt"
I get exception while asserting the request content. The form data is different, because an actual form data is created internally with Content-Disposition, Content-Type, Content-Length per parameter and I don't know how to specify these expected values
Multipart request expectations have been added to MockRestServiceServer in Spring 5.3 - see:
pull request
final version
You can use
content().multipartData(MultiValueMap<String, ?> expectedMap)
Parse the body as multipart data and assert it contains exactly the values from the given MultiValueMap. Values may be of type:
String - form field
Resource - content from a file
byte[] - other raw content
content().multipartDataContains(Map<String,?> expectedMap)
Variant of multipartData(MultiValueMap) that does the same but only for a subset of the actual values.
I think this depends on how deeply you want to test the form data. One way, which is not 100% complete, but is a "good enough" for unit testing (usually) is to do something like:
server.expect(once(), requestTo(getURI()))
.andExpect(method(HttpMethod.POST))
.andExpect(content().string(StringContains.containsString('paramname=Value') ))....
This is ugly and incomplete, but is sometimes useful. Of course, you can also work to make the form setup it's own method and then use mocks to try to verify that the expected parameters are all in place.
Related
I have a microservice architecture, where one service acts as a proxy, and must only forward the uploaded form data payload to the downstream service using restTemplate, preferably without loading anything from the request on disk or into memory.
I managed to resolve the issue taking the following steps.
Here I will describe the approaches, and the limitations used:
I have the following rest template configuration:
#Bean
public RestTemplate myRestTemplate() {
HttpComponentsClientHttpRequestFactory requestFactory = new HttpComponentsClientHttpRequestFactory();
requestFactory.setBufferRequestBody(false);
RestTemplate restTemplate = new RestTemplate(requestFactory);
restTemplate.setInterceptors(new ArrayList<>()); // to avoid interceptors loading data into memory
return restTemplate;
}
in my controller I am processing the HttpServletRequest directly using Apache Commons FileUpload Streaming Api with one asterix:
Special care on the multipart form data, so first the form fields are processed in the while loop, and then only one file was I able to process, since:
FileItemStream fileItemStream = uploadItemIterator.next();
return fileItemStream.openStream();
must be returned without invoking itemIterator.hasNext(), because that will result in FileItemStream.ItemSkippedException
which works wonderfully, no data is saved on disk
c:\Users\myuser\AppData\Local\Temp\tomcat.11416588345568217859.8077\
note: I have set the following property as stated in the documentation.
spring.application.servlet.multipart.enabled: false
From here, Using the streaming api I have an inputStream, which I will pass further down to create my HttpEntity as follows (simplified in example, full inspiration to include filename in request: here):
MultiValueMap<String, Object> multiPartBody = new LinkedMultiValueMap<>();
multiPartBody.add(FILE, inputStream);
HttpEntity<MultiValueMap<String, Object>> requestEntity = new HttpEntity<>(multiPartBody, myHeaders);
after this, I do make the call to my rest Template:
myRestTemplate.postForEntity(url, requestEntity, MyResponse.class);
this goes all the way via the following sequence:
RestTemplate.doExecute()
HttpAccessor.createRequest()
HttpComponentsClientHttpRequestFactory.createRequest() -> which will return a **HttpComponentsStreamingClientHttpRequest** <- this one is important
RestTemplate.doWithRequest(ClientHttpRequest httpRequest) -> calls: ((HttpMessageConverter<Object>) messageConverter).write(
requestBody, requestContentType, httpRequest);
FormHttpMessageConverter.write()
FormHttpMessageConverter.writeMultipart() -> where outputMessage instanceof StreamingHttpOutputMessage is true
HttpComponentsStreamingClientHttpRequest.executeInternal -> creates a new StreamingHttpEntity(...)
after which this goes down on InternalCLientExecution, and in execChain
sooner or later it will enter in the chain:
HttpComponentsStreamingClientHttpRequest.StreamingHttpEntity.writeTo(OutputStream outputStream) throws IOException {
this.body.writeTo(outputStream);
}
where body is a FormHttpMessageConverter.lambda from above:
if (outputMessage instanceof StreamingHttpOutputMessage streamingOutputMessage) {
streamingOutputMessage.setBody(outputStream -> {
writeParts(outputStream, parts, boundary);
writeEnd(outputStream, boundary);
});
}
so we get further down, and end up in:
FormHttpMessageConverter.writeParts()
FormHttpMessageConverter.writePart()
here a multipartMessage is composed and passed further down (or invoked the superclass AbstractHttpMessageConverter method)
multipartMessage = new MultipartHttpOutputMessage(os, charset);
...
((HttpMessageConverter<Object>) messageConverter).write(partBody, partContentType, multipartMessage);
from here we get into AbstractHttpMessageConverter.write where condition
if (outputMessage instanceof StreamingHttpOutputMessage)
evaluates to false because MultipartHttpOutputMessage is not an instance of StreamingHttpOutputMessage
But this seems not to affect anything, since the whole thing is invoked in the above mentioned lambda, sooner or later, we need to write the bytes from the inputStream into the outputStream.
one impediment:
if I configure the restTemplate as follows:
#Bean
#org.springframework.cloud.client.loadbalancer.LoadBalanced
public RestTemplate myRestTemplate() {
...
}
there is an interceptor/aspect overriding the RestTemplate HttpComponentsClientHttpRequestFactory with RibbonClientHttpRequestFactory (using spring netflix stack), which does not support setBufferRequestBody(false).
That is how I managed to solve the file streaming issue, hope it helps others too:
Limitations/Constraints:
You cannot use MultipartFile in your controllers since spring by default saves data into temp files on fileSystem (can't use resolve-lazily either: because), I was able to overcome this issue only with Apache Commons FileUpload
Using Apache Commons FileUpload I managed to process only one file, and the form data need to be processed before the file data
spring.application.servlet.multipart.enabled: false -> affects other endpoints too
composing downstream form data with correct Content-Disposition: form-data; name="file"; filename="my.txt" needs some strange embedded HttpEntity constructions
#LoadBalanced overrides the whole restTemplate requestFactory
Good luck everyone, and any feedback is welcome.
I'm using the Spring Rest Template to make an Http PUT request and up until now I have been passing the request body using the following:
MultiValueMap<String, Object> body = new LinkedMultiValueMap<String, Object>();
body.add("account", testAccount);
HttpEntity<?> requestEntity = new HttpEntity<Object>(body, headers);
ResponseEntity<String> responseEntity;
try {
responseEntity = rest.exchange(uri, HttpMethod.PUT, requestEntity, String.class);
}
catch (HttpStatusCodeException e) {
log.error("Http PUT failed with response: " + e.getStatusText());
return ResponseEntity.status(e.getStatusCode()).body(e.getResponseBodyAsString());
}
return responseEntity;
The request body that gets sent to my target API appears as:
{"account":[{"account_id":"495"}]}
This works, but my target API is not expecting the account object to have an array as a value and is currently giving me a 500 Internal Server Error, so, my question is, how can I get the value of the 'account' property to be an object rather than an array? For example I would like the request body to appear as:
{"account":{"account_id":"495"}}
Is there another type of Map which can be used which does not accept multiple values?
I would still like to use the exchange method if possible.
Any on help on this would be grand! Many thanks
The answer was actually simply using a regular HashMap instead of the MultiValueMap. As seen here the MultiValueMapcase is used when you want to add an array to a single key in the request body.
Thanks for all the help.
I have looked at the various answers and they do not resolve my issue. I have a very specific client need where I cannot use the body of the request.
I have checked these posts:
Trying to use Spring Boot REST to Read JSON String from POST
Parsing JSON in Spring MVC using Jackson JSON
Pass JSON Object in Rest web method
Note: I do encode the URI.
I get various errors but illegal HTML character is one. The requirement is quite simple:
Write a REST service which accepts the following request
GET /blah/bar?object=object11&object=object2&...
object is a POJO that will come in the following JSON format
{
"foo": bar,
"alpha": {
"century": a,
}
}
Obviously I will be reading in a list of object...
My code which is extremely simplified... as below.
#RequestMapping(method=RequestMethod.GET, path = "/test")
public Greeting test(#RequestParam(value = "object", defaultValue = "World") FakePOJO aFilter) {
return new Greeting(counter.incrementAndGet(), aFilter.toString());
}
I have also tried to encapsulate it as a String and convert later which doesnt work either.
Any suggestions? This should really be extremely simple and the hello world spring rest tut should be a good dummy test framework.
---- EDIT ----
I have figured out that there is an underlying with how jackson is parsing the json. I have resolved it but will be a write up.. I will provide the exact details after Monday. Short version. To make it work for both single filter and multiple filters capture it as a string and use a json slurper
If you use #RequestParam annotation to a Map<String, String> or MultiValueMap<String, String> argument, the map will be populated with all request parameters you specified in the URL.
#GetMapping("/blah/bar")
public Greeting test(#RequestParam Map<String, String> searchParameters) {
...
}
check the documentation for a more in depth explanation.
What is the most convenient way to influence the priority of the message converters Spring applies when POSTing with RestTemplate?
Use case: I want to ensure a given entity is POSTed as JSON rather than e.g. XML when I do restTemplate.postForEntity(url, entity, Void.class).
Default
By default the entity is converted to XML because the MappingJackson2XmlHttpMessageConverter takes precedence over the MappingJackson2HttpMessageConverter. The default list of converters for my app appears to be (Spring scans the classpath to see what's available):
Option 1
You can configure the message converters explicitly for a given RestTemplate instance like so restTemplate.setMessageConverters(Lists.newArrayList(new MappingJackson2HttpMessageConverter())). This is inconvenient if the instance is shared (as a Spring bean for example) as you might need converter X in one case and converter Y in a different one.
Option 2
You can set Accept and Content-Type HTTP headers explicitly in which case Spring will use a matching message converter. The downside is that you have to resort to RestTemplate.exchange instead of RestTemplate.postForEntity which means: extra code, less convenience.
HttpHeaders requestHeaders = new HttpHeaders();
requestHeaders.setContentType(MediaType.APPLICATION_JSON);
requestHeaders.setAccept(Collections.singletonList(MediaType.APPLICATION_JSON));
HttpEntity requestEntity = new HttpEntity(entity, requestHeaders);
restTemplate.exchange(url, HttpMethod.POST, requestEntity, Void.class);
Option 3
This might be what I'm looking for :)
This issue is answered in detail here.
Basically, when you add the below-mentioned library, it adds MappingJackson2XmlHttpMessageConverter before MappingJackson2HttpMessageConverter. As a result, Spring boot assumes every request accepts application/XML.
<dependency>
<groupId>com.fasterxml.jackson.dataformat</groupId>
<artifactId>jackson-dataformat-xml</artifactId>
</dependency>
To avoid this behaviour, you might want to swap the two message converters.
Example:
#Bean
RestTemplate restTemplate() {
RestTemplate restTemplate = new RestTemplate();
// move XML converter to the end of list
List<HttpMessageConverter<?>> messageConverters = restTemplate.getMessageConverters();
for (int i = 0; i < messageConverters.size() -1 ; i++) {
if (messageConverters.get(i) instanceof MappingJackson2XmlHttpMessageConverter) {
Collections.swap(messageConverters, i,messageConverters.size() - 1);
}
}
restTemplate.setMessageConverters(messageConverters);
// add interceptors if necessary
restTemplate.setInterceptors(Collections.singletonList(catalogInterceptior()));
return restTemplate;
}
According to the Spring javadoc (https://docs.spring.io/spring-framework/docs/current/javadoc-api/index.html?org/springframework/web/client/RestTemplate.html) you can still use postForEntity,
public <T> ResponseEntity<T> postForEntity(java.lang.String url,
#Nullable
java.lang.Object request,
java.lang.Class<T> responseType,
java.util.Map<java.lang.String,?> uriVariables)
throws RestClientException
....
The request parameter can be a HttpEntity in order to add additional HTTP headers to the request.
I know that v3.0 has method getHeader() but what about 2.3? Maybe it possible to get from steam?
UPDATE:
Actually, I need the HTTP response header RESTful application. In some reason I have decided to do this in servlet filter... but without success...
Solution #javax.ws.rs.core.Context HttpHeaders requestHeaders.
For example,
#javax.ws.rs.GET
public String invoceRestMethod(#Context HttpHeaders requestHeaders){
MultivaluedMap<String, String> map = headers.getRequestHeaders();
for (Map.Entry<String, List<String>> entry : map.entrySet()) {
// processing header....
}
}
Maybe in will help for someone. But any case, for Servlet issue still is opened
You cannot get the header from the stream*.
What you have to do is insert a proxy response object into the filter chain before your Servlet is called, and have that capture the header.
* Actually, you could potentially capture stuff from the stream using a proxy response and decode the headers. But if you are inserting a proxy response, it is simpler to capture the headers directly.