I'm experiencing a very low serialization performance for a custom #RepositoryRestController method returning PagedResources<PersistentEntityResource>, e.g. I'm getting 15s serialization times for 2.5Mb JSON data instead of 0.5s after the workaround I made (more on it later).
Consider this:
#Entity
public class Content {
#OneToMany
private Set<ContentMapping> contentMappings = new HashSet<>();
// ...
}
#RepositoryRestController
public class MyController {
private final ContentService contentService;
private final PagedResourcesAssembler pagedResourcesAssembler;
public ContentRestController(
ContentService contentService,
PagedResourcesAssembler pagedResourcesAssembler) {
this.contentService = contentService;
this.pagedResourcesAssembler = pagedResourcesAssembler;
}
#RequestMapping(value = "/findContent", method = RequestMethod.GET)
#ResponseBody
public PagedResources<PersistentEntityResource> findContent(PersistentEntityResourceAssembler resourceAssembler) {
Page<Content> page = contentService.getContent();
#SuppressWarnings("unchecked")
PagedResources<PersistentEntityResource> pagedResources = pagedResourcesAssembler.toResource(page, resourceAssembler);
return pagedResources;
}
}
A call to /findContent takes 15s to fully respond (while data start streaming immideately after it is made, so this is like 15s serialization time).
After profiling I found out that the cause of the problem are persistent collection properties on Content. During serialization of a Content a new transaction is opened for every access attempt to the contentMappings collection, even when contentMappings was properly fetched before serialization inside contentService.getContent() call.
Opening an explicit transaction on a controller method did not help (cause it is closed after the method exits and before serialization occurs), but I was able to work around this behaviour using HttpServletResponse and manually serializing the response:
#RepositoryRestController
public class MyController {
private final ContentService contentService;
private final PagedResourcesAssembler pagedResourcesAssembler;
private final List<HttpMessageConverter> messageConverters;
public ContentRestController(
ContentService contentService,
PagedResourcesAssembler pagedResourcesAssembler,
List<HttpMessageConverter> messageConverters) {
this.contentService = contentService;
this.pagedResourcesAssembler = pagedResourcesAssembler;
this.messageConverters = messageConverters;
}
#RequestMapping(value = "/findContent", method = RequestMethod.GET)
#ResponseBody
#Transactional(readOnly = true)
public void findContent(PersistentEntityResourceAssembler resourceAssembler, HttpServletResponse response) throws IOException {
Page<Content> page = contentService.getContent();
#SuppressWarnings("unchecked")
PagedResources<PersistentEntityResource> pagedResources = pagedResourcesAssembler.toResource(page, resourceAssembler);
// manual response serialization
MediaType mediaType = MediaType.valueOf("application/hal+json");
ResponseEntity<String> responseEntity = messageConverters.stream()
.filter(messageConverter -> messageConverter.canWrite(pagedResources.getClass(), mediaType))
.findFirst()
.map(messageConverter -> {
HttpOutputMessage outputMessage = new HttpOutputMessage() {
private final OutputStream outputStream = new ByteArrayOutputStream();
private final HttpHeaders httpHeaders = new HttpHeaders();
#Override
public OutputStream getBody() throws IOException {
return outputStream;
}
#Override
public HttpHeaders getHeaders() {
return httpHeaders;
}
};
try {
messageConverter.write(pagedResources, mediaType, outputMessage);
return ResponseEntity.ok()
.headers(outputMessage.getHeaders())
.body(new String(((ByteArrayOutputStream) outputMessage.getBody()).toByteArray(), StandardCharsets.UTF_8));
} catch (IOException e) {
throw new IllegalStateException("Failed to convert output to " + mediaType.toString());
}
})
.orElseThrow(() -> new IllegalStateException("Failed to convert output to " + mediaType.toString()));
response.setContentType(mediaType.toString());
response.setCharacterEncoding(StandardCharsets.UTF_8.name());
response.getWriter().write(responseEntity.getBody());
responseEntity.getHeaders().entrySet().stream()
.flatMap(entry -> entry.getValue().stream()
.map(value -> Tuples.of(entry.getKey(), value)))
.forEach(t -> response.addHeader(t.getT1(), t.getT2()));
response.flushBuffer();
}
}
This way response is received in 0.5s instead of 15s.
Problems I see with this workaround are e.g. completely ignoring RequestBodyAdvice / ResponseBodyAdvice processing, and the need to manually work with HttpServletResponse and HttpMessageConverter, effectively duplicating Spring code.
So my question is what I'm doing wrong, because if everything is right, I will open a bugreport on Spring Jira.
Related
We have a RestController with the below endpoint
#PostMapping(path = "/downloadFile", produces = MediaType.MULTIPART_FORM_DATA_VALUE,
consumes = MediaType.APPLICATION_JSON_VALUE)
public FileDownloadResponse downloadFile(#RequestBody FileDownloadRequest request) {
FileDownloadResponse downloadResponse = new FileDownloadResponse();
File file = new File("c:/fileLocation/"+request.getFileName());
try (InputStream stream = new FileInputStream(file)) {
byte[] bytes = IOUtil.toByteArray(stream);
downloadResponse.setFileName(file.getName());
downloadResponse.setCheckSum(calculateCheckSum(bytes));
downloadResponse.setFileContents(new FileSystemResource(bytes, file.getName()));
} catch (Exception e) {
e.printStackTrace();
}
return downloadResponse;
}
public class FileDownloadResponse {
private String fileName;
private Long checkSum;
private Resource fileContents;
}
public static class FileSystemResource extends ByteArrayResource {
private String fileName;
public FileSystemResource(byte[] byteArray , String filename) {
super(byteArray);
this.fileName = filename;
}
public String getFilename() { return fileName; }
public void setFilename(String fileName) { this.fileName= fileName; }
}
And on the Client Side we have the below code,
public class FileDownloadResponseClient {
private String fileName;
private Long checkSum;
private MultipartFile fileContents;
}
public FileDownloadResponseClient download(FileDownloadRequest request) {
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.APPLICATION_JSON);
headers.setAccept(Collections.singletonList(Mediatype.ALL));
HttpEntity<FileDownloadRequest> requestEntity = new HttpEntity<>(request, headers);
return restTemplate.postForEntity(downloadUrl, requestEntity, FileDownloadResponseClient.class);
}
When we run the Rest Client above, we are getting the below error,
org.springframework.web.client.HttpServerErrorException$InternalServerError: 500 : [no body]
Is it possible to download a multipartfile along with other additional fields? If yes, what is that we are missing here, please let us know.
Thanks in Advance!
org.springframework.web.multipart has a method boolean isEmpty() to find if the file has no content. Best put that check there and redirect to a message about such a file multipart form.
Of [no body] i have found that message on test requests to http server but in entirety generally means there is nothing in the form or no extra information needed for the server to complete the request.
For now i presume the spring framework handles all the url decoding and boundary marker stripping (on both ends) of uploaded files.
I have this endpoint for Spring Rest API:
#PostMapping(value = "/v1/", consumes = { MediaType.APPLICATION_XML_VALUE,
MediaType.APPLICATION_JSON_VALUE }, produces = { MediaType.APPLICATION_XML_VALUE,
MediaType.APPLICATION_JSON_VALUE })
public PaymentResponse handleMessage(#RequestBody PaymentTransaction transaction, HttpServletRequest request) throws Exception {
// get here plain XML
}
XML model.
#XmlRootElement(name = "payment_transaction")
#XmlAccessorType(XmlAccessType.FIELD)
public class PaymentTransaction {
public enum Response {
failed_response, successful_response
}
#XmlElement(name = "transaction_type")
public String transactionType;
.........
}
How I can get the XML request in plain XML text?
I also tried with Spring interceptor:
I tried this code:
#SpringBootApplication
#EntityScan("org.plugin.entity")
public class Application extends SpringBootServletInitializer implements WebMvcConfigurer {
#Override
protected SpringApplicationBuilder configure(SpringApplicationBuilder application) {
return application.sources(Application.class);
}
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
........
#Bean
public RestTemplate rsestTemplate() {
List<ClientHttpRequestInterceptor> interceptors = new ArrayList<>();
RestTemplate restTemplate = new RestTemplate(
new BufferingClientHttpRequestFactory(new SimpleClientHttpRequestFactory()));
restTemplate.setInterceptors(interceptors);
return restTemplate;
}
}
Component for logging:
#Component
public class RestTemplateHeaderModifierInterceptor implements ClientHttpRequestInterceptor {
#Override
public ClientHttpResponse intercept(HttpRequest request, byte[] body, ClientHttpRequestExecution execution)
throws IOException {
StringBuilder sb = new StringBuilder();
sb.append("[ ");
for (byte b : body) {
sb.append(String.format("0x%02X ", b));
}
sb.append("]");
System.out.println("!!!!!!!!!!!!!!!");
System.out.println(sb.toString());
ClientHttpResponse response = execution.execute(request, body);
InputStream inputStream = response.getBody();
String result = IOUtils.toString(inputStream, StandardCharsets.UTF_8);
System.out.println("!!!!!!!!!!!!!!!");
System.out.println(result);
return response;
}
}
But nothing is printed into the console. Any idea where I'm wrong? Probably this component is not registered?
Shouldn't it be easy like below to get it from HttpServletRequest, unless I'm missing something. I don't think there is need to use interceptor etc.
#PostMapping(value = "/v1/", consumes = { MediaType.APPLICATION_XML_VALUE,
MediaType.APPLICATION_JSON_VALUE }, produces = { MediaType.APPLICATION_XML_VALUE,
MediaType.APPLICATION_JSON_VALUE })
public PaymentResponse handleMessage(HttpServletRequest request) throws Exception {
String str, wholeXML = "";
try {
BufferedReader br = request.getReader();
while ((str = br.readLine()) != null) {
wholeXML += str;
}
System.out.println(wholeXML);
//Here goes comment question, to convert it into PaymentTransaction
JAXBContext jaxbContext = JAXBContext.newInstance(PaymentTransaction.class);
Unmarshaller unmarshaller = jaxbContext.createUnmarshaller();
StringReader reader = new StringReader(wholeXML);
PaymentTransaction paymentTransaction = (PaymentTransaction) unmarshaller.unmarshal(reader);
}
We had the same issue and use this solution in production. Which is not framework dependent (always an upside in my book) and simple.
Just consume it without specifying it as an XML. Then read the request lines and join them by \n if you want to have new lines in your xml. If not, join them by "" or whatever you please. This presumes you are using the javax.servlet.http.HttpServletRequest
Example:
#PostMapping(value = "/v1")
public PaymentResponse handleMessage(HttpServletRequest request) throws Exception {
final InputStream xml = request.getInputStream();
final String xmlString = new BufferedReader(new InputStreamReader(xml))
.lines()
.collect(Collectors.joining("\n"));
// do whatever you please with it
}
And you have an plain xml string.
For your controller to receive the request body as a plain xml string, you need only change the #RequestBody parameter type to String:
#PostMapping(value = "/v1/", consumes = { MediaType.APPLICATION_XML_VALUE, MediaType.APPLICATION_JSON_VALUE }, produces = { MediaType.APPLICATION_XML_VALUE, MediaType.APPLICATION_JSON_VALUE })
public PaymentResponse handleMessage(#RequestBody String xmlOrJson, HttpServletRequest request) throws Exception {
...
With the above mapping, if the client has submitted xml, you'll see the raw XML. Otherwise, if the client has submitted json, you'll see the raw JSON. Make sure you check the request's "Content-Type" header to know which type you're dealing with.
See https://docs.spring.io/spring/docs/current/spring-framework-reference/web.html#mvc-ann-requestbody
We've been using the spring-mvc-logger in production for quite a while. It's written as a servlet filter, so can be added as an independent wrapper to the MVC endpoint.
Our set up is almost exactly like described on the readme.md there, though we restrict the <url-pattern> under the <filter-mapping> to just the useful endpoints.
Even if it's not exactly what you're after, the codebase there makes quite a nice small example. In particular note the request/response wrapping that is needed in the filter. (This is to avoid the IllegalStateException: getReader(), getInputStream() already called that would otherwise happen if getReader() were called twice).
You have created List<ClientHttpRequestInterceptor> interceptors = new ArrayList<>(); but did not add the RestTemplateHeaderModifierInterceptor object to it.
You can autowire in the same in Application like below:
#Autowired
ClientHttpRequestInterceptor clientHttpRequestInterceptor;
and
interceptors.add(clientHttpRequestInterceptor);
The code looks like below:
class Application {
...
#Autowired
ClientHttpRequestInterceptor clientHttpRequestInterceptor;
#Bean
public RestTemplate rsestTemplate() {
List<ClientHttpRequestInterceptor> interceptors = new ArrayList<>();
RestTemplate restTemplate = new RestTemplate(
new BufferingClientHttpRequestFactory(new SimpleClientHttpRequestFactory()));
interceptors.add(clientHttpRequestInterceptor);
restTemplate.setInterceptors(interceptors);
return restTemplate;
}
...
}
Hope it helps
I have an endpoint
#GetMapping(value = "/accounts/{accountId}/placementInfo", headers = "version=1")
#ResponseStatus(HttpStatus.OK)
public List<PlacementDetail> findPlacementDetailByPlacementInfoAtTime(#PathVariable("accountId") Long accountId,
#RequestParam(value = "searchDate", required = false)
#DateTimeFormat(iso = DateTimeFormat.ISO.DATE) LocalDate searchDate) {}
And I am sending the request by using rest template
placementResponseEntity = restTemplate.exchange(placementUriBuilder(accountId, searchDate), HttpMethod.GET,
apiRequestEntity,new ParameterizedTypeReference<List<PlacementDetail>>() {});
with a helper method
private String placementUriBuilder(long accountId, LocalDate searchDate) throws IOException {
String resourceUri = ACCOUNT_RESOURCE_URI_START + accountId + PLACEMENT_DETAIL_RESOURCE_URI_END;
String url;
if(searchDate != null) {
url = UriComponentsBuilder.fromUri(serverUri.getUri()).path(resourceUri).queryParam("searchDate", searchDate.format(DateTimeFormatter.ISO_DATE)).build().toUriString();
} else {
url = UriComponentsBuilder.fromUri(serverUri.getUri()).path(resourceUri).build().toUriString();
}
return url;
}
When I look at the SO people talk about sending the object and failing as the created JSON is in wrong format but here this is a get api and I do not understand the source of the problem.
This is commonly caused by a missing error handler on your RestTemplate. Your server responds with an error and your client tries to deserialize it to a List<PlacementDetail>. In order to address this, you should properly handle HTTP error codes.
See the below snippet.
#Configuration
public class ClientConfig {
#Bean
public RestTemplate restTemplate(RestTemplateBuilder restTemplateBuilder) {
return restTemplateBuilder
.errorHandler(new ClientErrorHandler())
.build();
}
public class ClientErrorHandler implements ResponseErrorHandler {
#Override
public boolean hasError(ClientHttpResponse httpResponse) throws IOException {
// check if HTTP status signals an error response
return !HttpStatus.OK.equals(httpResponse.getStatusCode());
}
#Override
public void handleError(ClientHttpResponse httpResponse) throws IOException {
// handle exception case as you see fit
throw new RuntimeException("Error while making request");
}
}
}
I have implemented filter and I have called getEntityStream of ContainerRequestContext and set the exact value back by using setEntitystream. If i use this filter then #FormParameter data becomes null and if i don't use filter then everything will be fine (as I am not calling getEntityStream) and i have to use filter to capture request data.
Note: I am getting form params from MultivaluedMap formParams but not from #FormParameter.
Environment :- Rest Easy API with Jboss Wildfly 8 server.
#Provider
#Priority(Priorities.LOGGING)
public class CustomLoggingFilter implements ContainerRequestFilter, ContainerResponseFilter{
final static Logger log = Logger.getLogger(CustomLoggingFilter.class);
#Context
private ResourceInfo resourceInfo;
#Override
public void filter(ContainerRequestContext requestContext)
throws IOException {
MDC.put("start-time", String.valueOf(System.currentTimeMillis()));
String entityParameter = readEntityStream(requestContext);
log.info("Entity Parameter :"+entityParameter);
}
private String readEntityStream(ContainerRequestContext requestContext){
ByteArrayOutputStream outStream = new ByteArrayOutputStream();
final InputStream inputStream = requestContext.getEntityStream();
final StringBuilder builder = new StringBuilder();
int read=0;
final byte[] data = new byte[4096];
try {
while ((read = inputStream.read(data)) != -1) {
outStream.write(data, 0, read);
}
} catch (IOException e) {
e.printStackTrace();
}
byte[] requestEntity = outStream.toByteArray();
if (requestEntity.length == 0) {
builder.append("");
} else {
builder.append(new String(requestEntity));
}
requestContext.setEntityStream(new ByteArrayInputStream(requestEntity) );
return builder.toString();
}
return null;
}
}
class customResource
{
//// This code is not working
#POST
#Path("voiceCallBack")
#ApiOperation(value = "Voice call back from Twilio")
public void voiceCallback(#FormParam("param") String param)
{
log.info("param:" + param);
}
// This code is working
#POST
#Path("voiceCallBackMap")
#ApiOperation(value = "Voice call back from Twilio")
public void voiceCallbackMap(final MultivaluedMap<String, String> formParams)
{
String param = formParams.getFirst("param");
}
}
please suggest me solution & Thanks in Advance.
I found during run time that instance of the entity stream (from http request) is of type org.apache.catalina.connector.CoyoteInputStream (I am using jboss-as-7.1.1.Final). But we are setting entity stream with the instance of java.io.ByteArrayInputStream. So Resteasy is unable to bind individual formparmeters.
There are two solutions for this you can use any one of them :
Use this approach How to read JBoss Resteasy's servlet request twice while maintaing #FormParam binding?
Get form parameters like this:
#POST
#Path("voiceCallBackMap")
#ApiOperation(value = "Voice call back from Twilio")
public void voiceCallbackMap(final MultivaluedMap<String, String> formParams)
{
String param = formParams.getFirst("param");
}
Is there anyway to force spring to always produce json, even an empty json object if there's no data to return.
Our services go through another service that rejects any response that isn't valid json (regardless of status code). It's not nice but we have no control of this.
With spring controllers you can tell them to produce json, but this only works when there's content to return. Is there a quick and elegant way to make all responses be json?
#RequestMapping(value = "/test", method = RequestMethod.GET, produces = MediaType.APPLICATION_JSON_VALUE)
public
#ResponseBody
ResponseEntity<String> test(){
// if this returns null or an empty string the response body will be emtpy
// and the content-type header will not be set.
return service.getData();
}
The simply fix here is to simply add an if statement to check for null. But that's ugly as I'll have to manually set the header and the response body.
I'm hoping someone knows of a nicer way?
Thanks
If you want all responses to return application/json, then you can set this at a single place by overriding postHandle() from HandlerInterceptorAdapter:
#Component
public class ResponseInterceptor extends HandlerInterceptorAdapter {
#Override
public void postHandle(final HttpServletRequest request, final HttpServletResponse response, final Object handler,
final ModelAndView modelAndView) throws IOException {
if (response.getContentType() == null || response.getContentType().equals("")) {
response.setContentType("application/json");
}
}
}
You can look here
You may wrap the response in a "Container" object
For example I use this BaseAjaxResponse:
public class BaseAjaxResponse implements Serializable
{
private static final long serialVersionUID = 9087132709920851138L;
private int codiceOperazione;
private String descrizioneEsitoOperazione;
private long numeroTotaleOggetti;
private long numeroOggettiRestituiti;
private List<? extends Object> payload;
//Constructors and getter/setter
}
Then in my controllers I use this strategy:
#RequestMapping(method = { RequestMethod.POST }, value = { "/find" })
public ResponseEntity<BaseAjaxResponse> createCandidato(#RequestBody CandidatoDto candidato){
BaseAjaxResponse bar = new BaseAjaxResponse();
HttpStatus statusCode = null;
List<Object> payload = null;
StopWatch sw = new StopWatch("Find");
try
{
sw.start();
payload = myService.find();
sw.stop();
if( payload == null || payload.isEmpty() )
{
statusCode = HttpStatus.NO_CONTENT;
bar.setCodiceOperazione(statusCode.value());
bar.setDescrizioneEsitoOperazione("No result");
}
else
{
statusCode = HttpStatus.OK;
bar.setCodiceOperazione(statusCode.value());
bar.setDescrizioneEsitoOperazione("Got result");
//Set the object count and the number of found objects
}
}
catch (Exception e)
{
String message = "Errore nell'inserimento di un candidato; "+e.getMessage();
statusCode = HttpStatus.INTERNAL_SERVER_ERROR;
bar.setCodiceOperazione(statusCode.value());
bar.setDescrizioneEsitoOperazione(message);
logger.error(message, e);
}
finally
{
if( sw.isRunning() )
{
sw.stop();
if( logger.isDebugEnabled() )
{
logger.debug("CHIUSURA STOPWATCH FORZATA. "+sw.toString());
}
}
}
return new ResponseEntity<BaseAjaxResponse>(bar, statusCode);
}
I hope this can be useful
Angelo