Develop Reference

Tomcat HttpServletResponse writing to getOutputStream() always causes OutOfMemory error for large file downloads [duplicate]

I am using this code to download an existing file from the server on Liferay (6.2) into a local pc:
`
File file = getFile(diskImage.getImageType(), diskImage.getId());
HttpServletRequest httpReq = PortalUtil.getHttpServletRequest(request);
HttpServletResponse httpResp = PortalUtil.getHttpServletResponse(response);
httpResp.setContentType("application/octet-stream");
httpResp.setHeader("Content-Transfer-Encoding", "binary");
httpResp.setHeader("Content-Length", String.valueOf(file.length()));
httpResp.setHeader("Content-Disposition", "attachment; filename=" + file.getName());
try (InputStream input = new FileInputStream(file)) {
ServletResponseUtil.sendFile(httpReq, httpResp, file.getName(), input, "application/octet-stream");
} catch (Exception e) {
throw new FilesManagerException(e);
}
}
`
This code works fine only for small files. But downloading large files (cca 2GB) throws javax.portlet.PortletException: Error occurred during request processing: Java heap space.
How to fix this code so it works properly for larger files as well?
I guess that the suitable approach would be to use some kind of a buffer for large files and I try it but it wouldn't work even for the smaller files afterwards.

First of all: I'm assuming you're doing this in a render method - and this is just plain wrong. Sooner or later this will break, because you don't have control over the output stream: It might already be committed and transmit data to the browser when your portlet starts to render. In render you always must generate a portlet's HTML code.
Instead, you should go to the resource serving phase of a portlet. With the ResourceRequest and ResourceResponse, you have a very similar support for setting mimetypes as with HttpServletResponse.
And for exactly that reason, ServletResponseUtil is indeed the wrong place to look for. If you use anything from Liferay, you should look for PortletResponseUtil. There are various sendFile methods that accept a byte[], others accept a stream or a file. I'd recommend to try these, if they still fail, look at the implementation you are ending up with. In the worst case, just don't use any of the Util methods. Copying content from one stream to another is not too bad. (Actually, you give no clue about the static type of your variable input in the question: If that's a byte[], there's your solution)
You might want to file an issue with Liferay, if indeed the pure stream-transfer does read the whole file into memory, but your quickfix (in case this is indeed a bug) would be to copy the data yourself.

Thanks for thoughts, finally I used PortletResponseUtil.sendFile(...); method and changed actionURL to responseURL in .jsp file. So that I implemented serveResource()with above mentioned method. It seems that everything is working fine.

ServletResponseUtil.sendFile(httpReq, httpResp, file.getName(), input, "application/octet-stream"); what's this?
Don't read a file once time.Use a buffer.
response.reset();
response.setContentType("application/x-download");
response.addHeader("Content-Disposition","attachment;filename="+new String(filename.getBytes(),"utf-8"));
response.addHeader("Content-Length",""+file.length());
OutputStream toClient=new BufferedOutputStream(response.getOutputStream());
response.setContentType("application/octet-stream");
byte[] buffer=new byte[1024*1024*4];
int i=-1;
while((i=fis.read(buffer))!=-1){
toClient.write(buffer,0,i);
}
fis.close();
toClient.flush();
toClient.close();

Broken PDF document when downloading from web application

I have a "simple" problem downloading a PDF, Zip or XLSX auto-generated file in Spring MVC.
I have an Exporter component that is capable of exporting a dataset into CSV,PDF,XLSX (plain or compressed) formats directly to a generic OutputStream.
Running it in JUnit with temporary files as target OutputStreams succeeds. However, if I run that component using Spring MVC's auto-wired response OutputStream then the response gets corrupted
#RequestMapping("/export")
public void export(#RequestBody ..., HttpServletResponse response)
throws IOException, ExportException
{
webExportService.export(MyEntity.class, exportRequest, sessionFactory, response.getOutputStream(), response);
}
public <T> void export(Class<T> clazz, RequestBean exportInfo, SessionFactory sessionFactory, OutputStream outputStream,
HttpServletResponse httpResponse) {
httpResponse.setContentType(getMimeType());
httpResponse.setHeader("Content-Disposition", "attachment; filename=" + getFileName() + "");
httpResponse.setHeader("Pragma", "public");
httpResponse.setHeader("Cache-control", "must-revalidate");
export(clazz, exportInfo, sessionFactory, outputStream); //this will write to the outputStream
}
[Edit] MAJOR REPHRASE REQUIRED. Sorry now that I re-read the question I found I phrased it bad
When I try to open files of ZIP, PDF and XLSX types, those files cannot be opened by their default editors. Opening them with Notepad++, compared to a temporary folder copy, shows something interesting. The binary characters are different in the two files, like in the example below:
Good
Bad
Output stream is type org.apache.catalina.connector.CoyoteOutputStream.
It looks like it is a content encoding problem. How do I find out the correct encoding for those files, if leaving to default is not working?
Side note: I initially posted that the files were "truncated", because opening them (with Notepad++, not mentioned in early question) showed up just a few dozen lines. That looked being correct, I was misleaded by the small size of the file and thought that a double newline was cutting the response entity

java: file read, modify outside, then read again, modification not seen

I have set of files which will be served by a servlet based on a parameter. The servlet opens stream to required file, reads, writes to http response and closes the stream. Now this file can be modified manually and saved. Then next time a request comes for this file, servlet opens a stream to this file, writes it to the response, but I see the content is not the modified one, its the old one.
If I restart tomcat, servlet responds with modified content. Can it be made to read the modified content without restart?
Its not tomcat caching, the string read from file reader itself is unchanged.
thanks!
Here's the content of the servlet-
InputStream fhandle=this.getClass().getClassLoader().getResourceAsStream("responses/"+file);
if(fhandle!=null){
Reader fr=new InputStreamReader(fhandle);
PrintWriter out=resp.getWriter();
int a=-1;
FileWriter fos=new FileWriter("copyFile.xml"); //to compare what's read with actual file
while((a=fr.read())>=0){
fos.write(a);
out.write(a);
}
out.flush();
fos.close();
fr.close();
fr=null;
fhandle.close();
fhandle=null;
System.gc();
}
else
System.out.println(file+" not found");
After hitting the servlet for first time, I will edit the file and save it. Now again I will hit the same servlet for the same file, and I will not get the edited content.

Resources are not reloaded dynamically when the associated files change. Resources are part of the source tree, distributed with the application. They aren't expected to change. There is something seriously wrong with your design if you require this behaviour.

Java file download code design problem

I have a Java project which is used as a component in a webapp. This java code writes an xls file in a specific folder. I want to provide a download functionality for this file which should be triggered as soon as file writing is done.
The problem is - without a server environment, how can write a download functionality?

Don't write to file in a specific folder. Just write to the HTTP response body immediately. The downloading job should just be done in the webapp's code. I assume that you're using Servlets. If you set the HTTP response Content-Disposition header to attachment, then the browser will pop a Save as dialogue. If you also set the Content-Type header, then the browser will understand what to do with it (e.g. it will then be able to ask Do you want to open it in Excel or to save? and so on).
response.setHeader("Content-Type", "application/vnd.ms-excel");
response.setHeader("Content-Disposition", "attachment;filename=\"" + filename + "\"");
// Now write xls to response.getOutputStream() instead of FileOutputStream.
If the API of that Java project is well designed, then you should have a method something like this:
public void writeXls(OutputStream output) throws IOException {
// Do your job to write xls to output. E.g. if you were using POI HSSF:
// WritableWorkbook workBook = Workbook.createWorkbook(output);
// ...
}
This way you can call it in the servlet as follows after setting the aforementioned headers:
yourClass.writeXls(response.getOutputStream());
Even more, it could easily be reused/tested in a plain vanilla Java application like follows:
yourClass.writeXls(new FileOutputStream("/path/to/foo.xls"));

This is how i do it. I show a download sql in my page.
response.setHeader("Content-Disposition", "attachment; " +
"filename=ContactPurge.sql");
response.setContentType("application/x-sql-data");
response.getWriter().write(procsql);
response.getWriter().write(sql);
response.flushBuffer();

Issue opening Excel documents with Internet Explorer

I have run into an issue in which IE does not open up the Save As/Open dialog box for an Excel document like Firefox does.
So I have created a servlet filter that is using '*.xls' as the url pattern. The issue that I now face (since this is the first filter I have created) is how to get the name of the file that the user wants so that the dialog box gets populated correctly. Currently the filter is invoked when the user selects a link on a given page.
Here is what I came up with:
The above is what I have doFilter().
String fileName = "fileName.xls";
HttpServletRequest httpRequest = (HttpServletRequest) pRequest;
String requestURI = httpRequest.getRequestURI();
if(StringUtils.isNotBlank(requestURI))
{
String uri[] = StringUtils.split(requestURI, '/');
fileName = uri[uri.length - 1];
}
HttpServletResponse httpResponse = (HttpServletResponse) pResponse;
httpResponse.setContentType("application/vnd.ms-excel");
httpResponse.setHeader("Content-disposition", "attachment; filename=\"" + fileName +"\"");
web.xml:
<filter>
<filter-name>ExcelFilter</filter-name>
<filter-class>vsg.rp.common.ExcelFilter</filter-class>
</filter>
<filter-mapping>
<filter-name>ExcelFilter</filter-name>
<url-pattern>*.xls</url-pattern>
</filter-mapping>
This all is working on my development box: Windows XP, JBoss, Eclipse, Oracle. But when it runs on the test server—Linux, Apache/JBoss, Oracle—it does not work. It appears that the filter is not even being called, no errors thrown, etc. Any idea as to why this would happen?

You want the content type set appropriately as well as the content disposition header, thus:
response.setContentType("application/vnd.ms-excel");
response.setHeader("Content-Disposition",
"attachment; filename=\"" + filename +
"\"");

Use the Content-Disposition HTTP header and set it to something like:
attachment; filename=myworkbook.xls
The is a Microsoft Knowledge Base Article all about this problem. And here is an example of setting Content-Disposition in Java code.

In addition to setting the headers for the content type, you'll also want to ensure that the server DOES NOT tell the browser to NOT CACHE the file.
In IE land, if you tell IE not to cache the file, it happily downloads the file... then tries to open the file from the directory it saved the file to... However since the headers said "don't cache" it takes this literally, and doesn't save the file, thus Excel throws an error saying, "There is no file!".
Long story short, tell IE to CACHE the file.