Needs to download big file in chunks - java

I have 20 MB of the file. I just want to download the file in the browser in chunks/streams so that If there is another request it is not blocked for a long period of time.
I also see the pause behavior in a long file. It is not needed right now. If someone can explain this because of my curiosity to know how it works.
I have the file saved in DB. When I get the request to download the file. I will get the file from DB. I push the byte code in the response body and send it to the browser. In the browser, I have created one link and download the file in one go.
Back-end code:
#GetMapping("/job-detail/{jobExecutionId}/file-detail")
public ResponseEntity<Resource> getFileDetail(#PathVariable final Long jobExecutionId) {
final FileDetailDto fileDetailDto =
fileTaskletService.getFileDetailByExecutionId(jobExecutionId);
return ResponseEntity.ok().contentType(new MediaType("text", "csv"))
.header(HttpHeaders.CONTENT_DISPOSITION,
"attachment; filename=\"" + fileDetailDto.getFileName() + "\"")
.body(new ByteArrayResource(fileDetailDto.getData()));
}
I wants to download the file in chunks so that other requests can also start the downloading on the same time.

From the wording of your question it sounds like you want the server framework to set the following header for you so that the data flows back in chunked blocks:
Transfer-Encoding: chunked
To do that you need to supply a Spring resource where the full size is not known in advance. You have used ByteArrayResource where the full size is known and results in a Content-Length header being set in the response and chunked is not used.
Change your code to use InputStreamResource and the service will stream the response back to the client with a chunked transfer encoding and no content length. Here's a sample (syntax unchecked):
try(ByteArrayInputStream bis = new ByteArrayInputStream(fileDetailDto.getData())) {
return ResponseEntity.ok().contentType(new MediaType("text", "csv"))
.header(HttpHeaders.CONTENT_DISPOSITION,
"attachment; filename=\"" + fileDetailDto.getFileName() + "\"")
.body(new InputStreamResource(bis));
}
While this will get you a chunked response I'm not convinced it's the root of your problems with the browser because they are all very capable of asynchronously streaming back data regardless of how the server provides it.

Related

Servlet error: Cannot call sendRedirect() after the response has been committed

I want to redirect to a page after writing the excel file. The servlet code is given below:
ByteArrayOutputStream outByteStream = new ByteArrayOutputStream();
workbook.write(outByteStream);
byte [] outArray = outByteStream.toByteArray();
response.setContentType("application/ms-excel");
response.setContentLength(outArray.length);
response.setHeader("Content-Disposition", "attachment; filename=name_"+date+".xlsx");
response.setIntHeader("Refresh", 1);
OutputStream outStream = response.getOutputStream();
outStream.write(outArray);
response.sendRedirect("url/reports.jsp");
This code downloads an Excel file which i have created.
when i call the above servlet, the excel file is being downloaded but it is throwing following exception in the last line :
Servlet Error: ::java.lang.IllegalStateException: Cannot call sendRedirect() after the response has been committed
Hence i am unable to redirect to a new page. what can i do to access the response object after i write the output in "outStream"
The basic problem is that this ...
I want to redirect to a page after writing the excel file.
... describes two separate responses. The server cannot chain them together by itself because the client will expect only one response to each request. Because two requests are required to elicit two responses, automation of this sequence will require client-side scripting.
Personally, I would probably put the script on the front end: a handler on the appropriate button or link that first downloads the file and then (on success) issues a request for the new page. It would also be possible to do as suggested in comments, however: put script in the new page that downloads the file.
You cannot have a body with a redirect because the browser, when receiving a redirect, will issue a second request to the URL it has found (in header Location), and it's the response of that second request that is displayed, unless it is also a redirect, in which case, it will issue a third request, and so on...

Allowed file types for Spring REST file upload

I have a Spring REST file upload controller that I need to restrict the allowed file types. I have a predefined list to check against, and I can do it with the following code snippet.
public ResponseEntity<Object> uploadFile(MultipartFile file) {
if(!file.isEmpty()) {
log.debug("File name: " + file.getOriginalFilename());
if(ipConfigService.isFileTypeSupported(file.getOriginalFilename())) {
However, as the files being upload could be quite large, I want to reject unsupported files without having to wait for the whole file to be transferred first (which is what's happening in the above example).
Is there a way to annotate the uploadFile method to only accept certain types (dynamically pulled from cache/database), or is there a way I can intercept the request to get the name or mime type without waiting for the full binary to upload?
I can see in the request from the UX that mime type is also sent:
------WebKitFormBoundarygcPzBq2V6QNzl25V Content-Disposition: form-data; name="file"; filename="TestFile.mp4" Content-Type: video/mp4
------WebKitFormBoundarygcPzBq2V6QNzl25V--

Accept Header encodes binary result in in HTTP GET Request

My API returns an Excel (xslx) File for a GET Request. If a "Accept" Header is present, the binary result gets encoded/corrupt. I can send the request using fiddler with the minimum Headers required and everything works just fine. If i add an Accept Header like a browser does:
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
The result gets larger (8kb instead of 4kb) and seems to be encoded in some way. Fiddler detects this encoding and asks to decode it. After decoding so, the result is valid again. When i use chrome browser instead, it downloads the larger (8kb) file, not decoded and therefor corrupt.
#GET
#Path("/export-report")
#Produces(MediaType.APPLICATION_OCTET_STREAM)
//#Produces({ "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet" })
public Response exportReport() throws IOException {
byte[] fileBytes = getFileBytes();
String filename = "report.xslx";
String mimeType = "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet";
return Response.ok()
.header("Content-Disposition", "attachment; filename=\"" + filename + "\"")
.header("Content-Length", fileBytes.length)
.entity(fileBytes)
.header("Content-Type", mimeType)
.build();
}
I tried returning a File, a InputStream and a byte Array, it didn't change a thing.
I also had a look at
Input and Output binary streams using JERSEY?
I have no soltuion, any idea?

Java Spring - dynamically generated csv file download response is hanging

On my company's site we have some tables that we need to export to a csv file.
There are some varying parameters, so the csv file needs to be dynamically created on request.
My problem is that after clicking to download, the response hangs, and waits for the whole file to be created (which can take some time) and only then downloads the entire file in one instant.
I'm using AngularJS, so I'm using window.location = <url_for_file_download> In order to make the browser download the file.
On the server side I'm using Java Spring and I've followed all the instructions I could find on the web in order to create a file download controller.
My controller code is something like this:
#RequestMapping(value = "http://yada.yada.yada/csv/myFile.csv", method = RequestMethod.GET)
public #ResponseBody
void getCustomers(HttpServletResponse response,
#RequestParam(required = false) String someParameters)
throws NotAuthorizedException, IOException {
// set headers
setHeaders(response);
// generate writer
CSVWriter write = generateWriter(response);
// get data
List<String[]> data = getData();
// write and flush and all that
.
.
.
}
My code for setting the response headers are:
response.setContentType("text/csv;charset=utf-8");
response.setHeader("Content-Disposition","attachment; filename=\"" + fileName + ".csv\"");
I've also tried adding the following headers:
response.setHeader("Transfer-Encoding", "Chunked");
response.setHeader("Content-Description", "File Transfer");
and I've also tried setting the Content-type to "application/octet-stream".
Notice that I don't add a Content-length header, since the file doesn't exist yet, and is being written on the fly.
For writing the csv file I'm using OpenCSV and my code is as follows:
OutputStream resOs = response.getOutputStream();
OutputStream buffOs = new BufferedOutputStream(resOs);
OutputStreamWriter outputWriter = new OutputStreamWriter(buffOs,"UTF-8");
CSVWriter writer = new CSVWriter(outputWriter);
I iterate over the data and write it like so:
for (String[] row: data) {
writer.writeNext(line);
}
(It's not exactly the code - but this is more or else what happens in the code)
And at the end I flush and close:
writer.flush();
writer.close();
I also tried flushing after each line I write.
So why isn't the file being transferred before it has all been written?
Why is my browser (Google chrome) downloading the file in one instant after waiting a long time? And how can I fix this.
I hope I've added enough code, if there's something missing just please tell me and I'll try to add it here.
Thank you so much in advance.
Can you try returning a null value in your java
return null ;
Or you can try below code also
1. Jquery code upon clicking the submit button
$(document).ready( function() {
$('#buttonName').click(function(e){
$("#formName").submit();
//alert("The file ready to be downloaded");
});
});
Your controller code
#RequestMapping(value="/name",method=RequestMethod.POST)
public ModelAndView downloadCSV(ModelMap model,HttpSession session,#ModelAttribute(value="Pojo") Pojo pojo
,HttpServletRequest request,HttpServletResponse response){
----------------some code----------------
response.setContentType("application/csv");
("application/unknown");
response.setHeader("content-disposition","attachment;filename =filename.csv");
ServletOutputStream writer = response.getOutputStream();
logger.info("downloading contents to csv");
writer.print("A");
writer.print(',');
writer.println("B");
for(int i=0;i<limit;i++){
writer.print(""+pojo.get(i).getA());
writer.print(',');
writer.print(pojo.get(i).getB());
writer.println();
}
writer.flush();
writer.close();
---------------some code-----------
return null;
}
Hope this helps
The Controller will wait for the response to be written before the response is send back to the client.
Here is a nice post with multiple approaches / options outlined
Downloading a file from spring controllers
This post talks about flushing the output periodically to help fasten the download.
how to download large files without memory issues in java
If all you are trying to do is let the user know that the file download is in progress and due soon, I think an Ajax progress status indicaor might be your solution.
Trigger the ajax call to the back-end to generate the file
Show progress indicator to the user while file is being generated server side
once response is available, file is presented to the user.
I think something similar is being explored here download file with ajax() POST Request via Spring MVC
Hope this helps!
Thanks,
Paul
I faced the same issue. The code that didn't work for me was
#RequestMapping(value = "/test")
public void test(HttpServletResponse response)
throws IOException, InterruptedException {
response.getOutputStream().println("Hello");
response.getOutputStream().flush();
Thread.sleep(2000);
response.getOutputStream().println("How");
response.getOutputStream().flush();
Thread.sleep(2000);
response.getOutputStream().println("are");
response.getOutputStream().flush();
Thread.sleep(2000);
response.getOutputStream().println("you");
response.getOutputStream().flush();
}
The culprit was ShallowEtagHeaderFilter. When this filter is enabled the response is sent in one chunk. When this filter is diabled the response is send in multiple chunks.
From this thread Tomcat does not flush the response buffer it looks like another possible culprit can be GzipFilter

How to send a PDF to the browser from a Java webapplication?

I have a java class that generates a PDF file to a folder in my computer. I have managed to connect this class to a link on a web application and when i click this link it generates the pdf and writes it to the folder on my computer. I would want to change this and have the link send the pdf to the browser instead. How can i do this? The class does not use any HttpRequests or similar and the link isnt a hypertext link atm. Im looking for the most straight forward way to send a pdf to the browser.
/* Java Code */
response.setContentType("application/pdf");
response.setHeader("Content-Disposition", "attachment; filename=path/to/file.pdf");
i print it (any kind of file) to the response stream from a byte array, inside a servlet
if(content != null)
{
response.setContentType( "application/octet-stream" );
response.setHeader("Content-Disposition", "attachment; filename=\"" + fname + "\"");
response.setContentLength(content.length);
out.write(content);
}//where content is byte[]
You should be able to write the pdf to a stream, you can pass it the output stream from your response.
There are several ways to do that:
Put PDF file on some place available from Web, and then redirect user to URL, which will lead him to PDF file (if your web server supports this). Redirection may be easily done with "Location" HTTP header.
Send PDF file in HTTP response stream. Note, that you will have to set corresponding Mime-type in HTTP header. Implementation depends on web server / web framework you are using in your application.

Categories

Resources