Develop Reference

Spring RestTemplate streaming response into another request

I am trying to stream the result of a file download directly into another post using spring's RestTemplate
My current approach is the following:
ResponseEntity<InputStreamResource> downloadResponse = restTemplate.getForEntity(fileToDownloadUri, InputStreamResource.class);
InputStreamResource imageInputStreamResource = downloadResponse.getBody();
ResponseEntity<String> response = restTemplate.exchange(storageUri, POST, new HttpEntity<>(imageInputStreamResource), String.class);
However, I get the following exception running the code above:
org.springframework.web.client.ResourceAccessException: I/O error on POST request for "http://host:port/path/some.jpg": stream is closed; nested exception is java.io.IOException: stream is closed
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:6
...
Caused by: java.io.IOException: stream is closed
at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.ensureOpen(HttpURLConnection.java:3348)
at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:3373)
It seems that the response is always closed as the final step of processing. With the response, the HttpURLConnection is closed, and the stream is no longer processable.
I would like to be able to implement this scenario without having to hold the file completely in memory or writing it to a file (as described here).
Any hints are highly appreciated.

If you want to forward the response directly without ever holding it in memory, you have to directly write to the response:
#RequestMapping(value = "/yourEndPoint")
public void processRequest(HttpServletResponse response) {
RestTemplate restTemplate = new RestTemplate();
response.setStatus(HttpStatus.OK.value());
restTemplate.execute(
fileToDownloadUri,
HttpMethod.GET,
(ClientHttpRequest requestCallback) -> {},
responseExtractor -> {
IOUtils.copy(responseExtractor.getBody(), response.getOutputStream());
return null;
});
}

Since you tell RestTemplate to expect InputStreamResource it will try and use an appropriate converter to convert your message to a InputStreamResource. ( I'm guessing there is none that handles this as you want )
You should be able to let it expect a Resource from where you can get an input stream and read that.
import org.springframework.core.io.Resource;
ResponseEntity<Resource> exchange = RestTemplate.exchange(url, HttpMethod.GET, new HttpEntity(httpHeaders), Resource.class);
InputStream inputStream = exchange.getBody().getInputStream();
using this you can write the response to somewhere else. Files.write(inputStream, new File("./test.json")); wrote the file for me, so I assume the inputstream can also be used somewhere else. ( I used Spring 4.3.5 )
edit:
As the OP states, this will still load the file in memory. Behind the scene the InputStream is a ByteArrayInputStream.
The default RestTemplate and MessageConverters are not made for streaming content at all.
You could write your own implementation of a org.springframework.web.client.ResponseExtractor and maybe a MessageConverter. In ResponseExtractor you have access to the org.springframework.http.client.ClientHttpResponse
imho for your use case, you might be better of using Apache Httpcomponents HttpClient where you find HttpEntity#writeTo(OutputStream).

How to close a Stream of a REST Service?

I need to make a java REST service that will return an inputstream as a response. My problem is that I don't know how to close the stream after the client receives the entire stream. I'm using Java and CXF. Thanks
#GET
#Path("/{uuid}")
#Produces(MediaType.APPLICATION_OCTET_STREAM)
public Response getAttachmentByUuid(#PathParam("uuid")String uuid)
{
//getting inputstream from AWS S3
InpputSream is=getStreamFromS3(uuid);
return Response.status(Response.Status.OK).entity(is).build();
// will this "is" stream causes memory leak,, do I have to close it. Client side is not controlled by me
}

JAX-RS is implemented using Java servlets. In case of CXF is used CXFServlet. Your stream will be sent to client using the HttpServletResponse of the servlet interface
public void doGet(HttpServletRequest request, HttpServletResponse response)
You should not close an stream source (HttpServletResponse) if you have not created it. It is responsability of the container, and you can interfere with the life cycle of the request
See also Is is necessary to close the input stream returned from HttpServletRequest?

If you have a stream to close, consider try with resources:
#GET
#Path("/{uuid}")
#Produces(MediaType.APPLICATION_OCTET_STREAM)
public Response getAttachmentByUuid(#PathParam("uuid")String uuid)
{
//getting inputstream from AWS S3
// the try block opens the stream and guarantees to close it
try (InputStream is=getStreamFromS3(uuid)) {
return Response.status(Response.Status.OK).entity(from(is)).build();
}
}
This requires Java 7 and onwards. It's also awesome!
If you're in Java 6, then you would have to make your own finally block to remember to close the stream for you.

You might be looking to use a 'Conduit'
See CXF Apache Custom Transport for more info.
Be Careful though, the documentation states :
It is strongly recommended to don’t break streaming in Conduit and Destination implementations, if physical protocol supports it. CXF is completely streaming oriented – it causes high performance and scalability.

play framework- how to send post request using java code in play framework

Currently I'm switching to play framework to develop but I'm new to this wonderful framework.
I just want to send a post request to remote server and get response.
If I use Jersey, it would be quite easy, just like this:
WebResource resource = client.resource("http://myfirstUrl");
resource.addFilter(new LoggingFilter());
Form form = new Form();
form.add("grant_type", "authorization_code");
form.add("client_id", "myclientId");
form.add("client_secret", "mysecret");
form.add("code", "mycode");
form.add("redirect_uri", "http://mysecondUrl");
String msg = resource.accept(MediaType.APPLICATION_JSON).post(String.class, form);
and then I can get the msg which is what I want.
But in Play framework, I cannot find any libs to send such post request. I believe this should be a very simple feature and Play should have integrated it. I've tried to search and found most use case are about the Form in view leve. Could anyone give me some help or examples? Thanks in advance!

You can use Play WS API for making asynchronous HTTP Calls within your Play application. First you should add javaWs as a dependency.
libraryDependencies ++= Seq(
javaWs
)
Then making HTTP POST Requests are as simple as;
WS.url("http://myposttarget.com")
.setContentType("application/x-www-form-urlencoded")
.post("key1=value1&key2=value2");
post() and other http methods returns a F.Promise<WSResponse> object which is something inherited from Play Scala to Play Java. Basically it is the underlying mechanism of asynchronous calls. You can process and get the result of your request as follows:
Promise<String> promise = WS.url("http://myposttarget.com")
.setContentType("application/x-www-form-urlencoded")
.post("key1=value1&key2=value2")
.map(
new Function<WSResponse, String>() {
public String apply(WSResponse response) {
String result = response.getBody();
return result;
}
}
);
Finally obtained promise object is a wrapper around a String object in our case. And you can get the wrapped String as:
long timeout = 1000l;// 1 sec might be too many for most cases!
String result = promise.get(timeout);
timeout is the waiting time until this asynchronous request will be considered as failed.
For much more detailed explanation and more advanced use cases checkout the documentation and javadocs.
https://www.playframework.com/documentation/2.3.x/JavaWS
https://www.playframework.com/documentation/2.3.x/api/java/index.html

How to send a large input stream to a Spring REST service?

Have a Spring Rest application that run inside an embedded Jetty container.
On Client I use RestTemplate(try to).
Use case :
Having an InputStream (I don't have the File), I want to send it to the REST service.
The InputStream can be quite large (no byte[] !).
What I've tried so far :
Added StandardServletMultipartResolver to the Dispatcher context;
On servlet registration executed :
ServletRegistration.Dynamic dispatcher = ...
MultipartConfigElement multipartConfigElement = new MultipartConfigElement("D:/temp");
dispatcher.setMultipartConfig(multipartConfigElement);
On client :
restTemplate.getMessageConverters().add(new FormHttpMessageConverter());
MultiValueMap<String, Object> parts = new LinkedMultiValueMap<String, Object>();
parts.add("attachmentData", new InputStreamResource(data) {
// hacks ...
#Override
public String getFilename() {
//avoid null file name
return "attachment.zip";
}
#Override
public long contentLength() throws IOException {
// avoid calling getInputStream() twice
return -1L;
}
}
ResponseEntity<Att> saved = restTemplate.postForEntity(url, parts, Att.class)
On server :
#RequestMapping("/attachment")
public ResponseEntity<Att> saveAttachment(#RequestParam("attachmentData") javax.servlet.http.Part part) {
try {
InputStream is = part.getInputStream();
// consume is
is.close();
part.delete();
return new ResponseEntity<Att>(att, HttpStatus.CREATED);
}
}
What is happening :
The uploaded InputStream is stored successfully in the configured temp folder (MultiPart1970755229517315824), the Part part parameter is correctly Injected in the handler method.
The delete() method does not delete the file (smth still has opened handles on it).
Anyway it looks very ugly.
Is there a smoother solution ?

You want to use HTTP's Chunked Transfer Coding. You can enable that by setting SimpleClientHttpRequestFactory.setBufferRequestBody(false). See SPR-7909.

You should rather use byte[], and write a wrapper around the webservice to actually send the "large string" in chunks. Add a parameter in the webservice which will indicate the "contentID" of the content, so that the other side knows this part belongs to which half-filled "bucket". Another parameter "chunkID" would help in sequencing of the chunks on the other side. Finally, third parameter, "isFinalChunk" would be set if whatever you are sending is the final thing. This is pretty non-fancy functionality achievable in less than 100 lines of code.
The only issue with this is that you end up making "n" calls to the webservice rather than just one call, which would aggregate the connect delays etc. For realtime stuff, some more network QoS is required, but otherwise you should be fine.
I think this is much simpler, and once you have your own class wrapper to do this simple chopping and gluing, it is scalable to a great extent if your server can handle multiple webservice calls.

how to write a file object on server response and without saving file on server?

I am using Spring with DWR . I want to return a file object as response , however I save the file (to be sent) at server temporary location and then send its location as href for anchor tag on client side , however I wonder if there could be a way to throw the file directly to browser on response object without saving it temporarily on server.
I expected if there could be a way to send file as a response via DWR.

public ModelAndView writeFileContentInResponse(HttpServletRequest request, HttpServletResponse response) throws IOException {
FileInputStream inputStream = new FileInputStream("FileInputStreamDemo.java"); //read the file
response.setHeader("Content-Disposition","attachment; filename=test.txt");
try {
int c;
while ((c = inputStream.read()) != -1) {
response.getWriter().write(c);
}
} finally {
if (inputStream != null)
inputStream.close();
response.getWriter().close();
}
}

It has been years since I've used Spring, and I'm unfamiliar with DWR, but the essence of your question is basic to the web.
The answer is yes, you can. In effect, you need to set the HTTP header Content-Disposition: attachment, then stream down the contents. All of this will be in the response to the original request (as opposed to sending back a link).
The actual code to achieve this will depend on your circumstances, but this should get you started.

you call the method from Java Script, right? I didn't really understand how Spring is related in this flow, but as far as I know DWR allows you to produce Java Script Stubs and call the Java methods of the exposed bean directly on server right from your java script client code.
You can read the file byte-by-byte and return it from your java method as long as it really returns a byte array.
However what would you do with this byte array on client?
I just think in this specific flow you shouldn't use the DWR but rather issue an ordinar AJAX request (if DWR can wrap it somehow for convenience - great). This request shouldn't come to DWRServlet, but rather be proceeded by a regular servlet/some web-based framework, like Spring MVC :)
Once the request comes to the servlet, use
response.setHeader("Content-Disposition","attachment; filename=test.txt");
as was already stated.
Hope this helps,
Good luck!
Mark

An example which return a excel to download from client:
//Java side:
public FileTransfer getExcel(Parametros param){
byte[] result = <here get data>;
InputStream myInputStream = new ByteArrayInputStream(result);
String excelFormat = "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet";
FileTransfer dwrExcelFile = new FileTransfer("excel.xlsx", excelFormat, myInputStream);
return dwrExcelFile;
}
//Javascript side:
function downloadExcelFile() {
dwr.engine.setTimeout(59000);
var params = <params_to_send>;
<Java_class>.getExcel(params, {callback:function(dataFromServer) {
downloadExcelCallback(dataFromServer);
}});
}
function downloadExcelCallback(data) {
dwr.engine.openInDownload(data);
}