How to provide large files for download through spring controller ? I followed few discussions on similar topic :
Downloading a file from spring controllers
but those solutions fails for large files ~ 300mb - 600mb.
I am getting OutOfMemoryException on the last line :
#RequestMapping(value = "/file/{dummyparam}.pdf", method = RequestMethod.GET, produces=MediaType.APPLICATION_OCTET_STREAM_VALUE)
public #ResponseBody byte[] getFile(#PathVariable("dummyparam") String dummyparam, HttpServletResponse response) {
.
.
InputStream is = new FileInputStream(resultFile);
response.setHeader("Content-Disposition", "attachment; filename=\"dummyname " + dummyparam + ".pdf\"");
.
.
return IOUtils.toByteArray(is);
My (naive) assumption was that IOUtils will handle even large files but this is not obviously happening. Is there any way how to split file into chunks as download is in progress ? Files are usually around 300 - 600mb large. Max number of concurrent downloads is estimated to 10.
Easy way would be to link files as static content in the webserver directory but we would like to try do it in within our Spring app.
It is because you are reading the entire file into memory, use a buffered read and write instead.
#RequestMapping(value = "/file/{dummyparam}.pdf", method = RequestMethod.GET, produces=MediaType.APPLICATION_OCTET_STREAM_VALUE)
public void getFile(#PathVariable("dummyparam") String dummyparam, HttpServletResponse response) {
InputStream is = new FileInputStream(resultFile);
response.setHeader("Content-Disposition", "attachment; filename=\"dummyname " + dummyparam + ".pdf\"");
int read=0;
byte[] bytes = new byte[BYTES_DOWNLOAD];
OutputStream os = response.getOutputStream();
while((read = is.read(bytes))!= -1){
os.write(bytes, 0, read);
}
os.flush();
os.close();
}
For Spring , Need use InputStreamResource class in ResponseEntity .
Demo Code :
MediaType mediaType = MediaTypeUtils.getMediaTypeForFileName(this.servletContext, fileName);
System.out.println("fileName: " + fileName);
System.out.println("mediaType: " + mediaType);
File file = new File(DIRECTORY + "/" + fileName);
InputStreamResource resource = new InputStreamResource(new FileInputStream(file));
return ResponseEntity.ok()
// Content-Disposition
.header(HttpHeaders.CONTENT_DISPOSITION, "attachment;filename=" + file.getName())
// Content-Type
.contentType(mediaType)
// Contet-Length
.contentLength(file.length()) //
.body(resource);
}
Ref Link : https://o7planning.org/en/11765/spring-boot-file-download-example
Related
I'm currently developping on a webserver for my company , and i'am facing a corrupted zip file problem. At the end of a servlet. Here's the function header called at the very end of the servlet.
sendFile(directory, "s" + this.optimizerId + "-" + this.optimizerName + "-" + start.toDate(Defs.DB.DATE_PATTERN) + "-" + end.toDate(Defs.DB.DATE_PATTERN), FileExtension.NONE, ContentType.ZIP, response);
Here is the first send File function ...
public default void sendFile(File file, String fileName, FileExtension extension, ContentType type, HttpServletResponse response) {
if(type == ContentType.ZIP) {
String[] files = file.list();
if(files != null && files.length > 0) {
byte[] bytes = FileUtils.getZipBytes(file, files);
sendFile(new String(bytes), fileName, extension, type, response);
}
}
}
The "FileUtils.getZipBytes(file, files)" function is doing fine , it returns a byteArray.
I even tried to create a zip file with ZipOutputStream a File , and on the server , the zip has litterally no problem.
( I will send the source code of this function if you think it's important , but i don't want to add useless complexity ).
Then , here's the code of the second sendFile function that is called from the first one.
public default void sendFile(String content, String fileName, FileExtension extension, ContentType type, HttpServletResponse response) {
response.setContentType(type.getExpression());
response.setHeader("Content-Disposition", "attachment; filename=\"" + FileUtils.encodeForFileName(fileName) + "." + extension.getExtension() + "\"");
OutputStream out = response.getOutputStream();
out.write(content.getBytes());
out.flush();
out.close();
}
Do you see any problem in these functions ?
The zip file is downloaded from the server. He is not empty (about 15.7KB).
But when i try to open it , the file explorer of Fedora tell me that the file is corrupted.
I also tried to use the unzip command on the file , and i have the following error message.
Archive: s192-Système U carquefou-2023-01-26 000000-2023-01-26 235959(1).zip
caution: zipfile comment truncated
error [s192-Système U carquefou-2023-01-26 000000-2023-01-26 235959(1).zip]: missing 3249308257 bytes in zipfile
(attempting to process anyway)
error [s192-Système U carquefou-2023-01-26 000000-2023-01-26 235959(1).zip]: attempt to seek before beginning of zipfile
(please check that you have transferred or created the zipfile in the
appropriate BINARY mode and that you have compiled UnZip properly)
I thought it may be the FileUtils.getZipBytes(file, files) that returns a corrupted ByteArray. But i really don't think it is ... Here's the source code of this function
public static byte[] getZipBytes(File directory, String[] files) throws IOException {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ZipOutputStream zos = new ZipOutputStream(baos);
byte bytes[] = new byte[2048];
for (String fileName : files) {
FileInputStream fis = new FileInputStream(directory.getPath() +
System.getProperty("file.separator") + fileName);
BufferedInputStream bis = new BufferedInputStream(fis);
zos.putNextEntry(new ZipEntry(fileName));
int bytesRead;
while ((bytesRead = bis.read(bytes)) != -1) {
zos.write(bytes, 0, bytesRead);
}
zos.closeEntry();
bis.close();
fis.close();
}
zos.flush();
baos.flush();
zos.close();
baos.close();
return baos.toByteArray();
}
A zip file is not a string. You should drop the round trip from byte array to string and back to byte array.
Declare your sendFile() method as
public default void sendFile(byte[] content, String fileName, FileExtension extension, ContentType type, HttpServletResponse response) {
response.setContentType(type.getExpression());
response.setHeader("Content-Disposition", "attachment; filename=\"" + FileUtils.encodeForFileName(fileName) + "." + extension.getExtension() + "\"");
OutputStream out = response.getOutputStream();
out.write(content);
out.flush();
out.close();
}
And call it as
sendFile(bytes, fileName, extension, type, response);
The problem with your approach is that new String(content).getBytes() does not return a byte array with the same content as you started with.
i would like not to download the BufferedOutputStream when return java method.
my code:
FacesContext ctx = FacesContext.getCurrentInstance();
HttpServletResponse response = (HttpServletResponse) ctx.getExternalContext().getResponse();
response.setHeader("Content-Disposition", "attachment; filename=\"" + "Invoice.zip\";");
BufferedOutputStream bos = new BufferedOutputStream(response.getOutputStream());
ZipOutputStream zos = new ZipOutputStream(bos);
for(SalesEInvObject InvoiceObj : this.InvoiceTable){ // MAIN FOR-LOOP STARTS
if (InvoiceObj.getInvoiceNo() != null) {
javax.servlet.http.HttpSession httpSession =(javax.servlet.http.HttpSession) ctx.getExternalContext().getSession(false);
httpSession.setAttribute(BaseHttpServlet.DEFAULT_JASPER_PRINT_SESSION_ATTRIBUTE,
reportOutput.getInternalReportObject());
byte[] bytes = reportOutput.getReportOutputBytes();
int length = ((bytes == null) ? 0 : bytes.length);
response.setContentLength(length*tableSize);
final ZipEntry ze = new ZipEntry(reportOutputFileName+".pdf");
zos.putNextEntry(ze);
zos.write(bytes, 0, bytes.length);
zos.closeEntry();
}else {
return null;
}
}//LOOP ENDS
zos.close();
ctx.responseComplete();
my problem is when the invoices has Number it generates invoice and download in compressed zip file. but when it has no Number i dont want to download zip. but still zip file downloads but with empty no file in it.
if no pdf generated i dont want to download zip file.
any help...
Once you have started generating and writing the ZIP to the response output stream, there is no turning back. Just opening the stream causes the response to "commit" ... meaning that you can no longer change the response code or headers.
Basically, you need to check if there are any invoices before you start generating the response. Then it should just be a matter of reorganizing the existing code.
Something like .....
boolean hasInvoices = false;
for (SalesEInvObject invoiceObj : this.InvoiceTable) {
if (invoiceObj.getInvoiceNo() != null) {
hasInvoices = true;
break;
}
}
FacesContext ctx = FacesContext.getCurrentInstance();
HttpServletResponse response =
(HttpServletResponse) ctx.getExternalContext().getResponse();
if (hasInvoices) {
response.setHeader("Content-Disposition",
"attachment; filename=\"" + "Invoice.zip\";");
BufferedOutputStream bos =
new BufferedOutputStream(response.getOutputStream());
ZipOutputStream zos = new ZipOutputStream(bos);
for (SalesEInvObject invoiceObj : this.InvoiceTable) {
if (invoiceObj.getInvoiceNo() != null) {
javax.servlet.http.HttpSession httpSession =
(javax.servlet.http.HttpSession) ctx.getExternalContext()
.getSession(false);
httpSession.setAttribute(
BaseHttpServlet.DEFAULT_JASPER_PRINT_SESSION_ATTRIBUTE,
reportOutput.getInternalReportObject());
byte[] bytes = reportOutput.getReportOutputBytes();
int length = ((bytes == null) ? 0 : bytes.length);
response.setContentLength(length * tableSize);
final ZipEntry ze = new ZipEntry(reportOutputFileName + ".pdf");
zos.putNextEntry(ze);
zos.write(bytes, 0, bytes.length);
zos.closeEntry();
}
}
zos.close();
} else {
// do you want to set a response code or something?
}
ctx.responseComplete();
I have fixed some bad style. See if you can spot the changes ...
There is another problem that I haven't addressed: namely that the various resources that are opened in this code ought to be managed using try with resources. However, it may not be necessary since it looks like the resources are all based on with the request output stream. That will be closed automatically by the servlet infrastructure.
I own a mock project maven spring-rest with the end point
#RequestMapping(value = "/rest/{nid}/{fileName:.+}", method = RequestMethod.GET, produces = MediaType.APPLICATION_PDF_VALUE)
public String getPjSae(#PathVariable String nid, #PathVariable String fileName, HttpServletResponse response) throws IOException {
LOGGER.info("NID : " + nid);
LOGGER.info("NOM FICHIER : " + fileName);
File file = new File(saePath+File.separatorChar + fileName);
LOGGER.info("CHEMIN PJ : " + file);
if (file.exists()) {
InputStream inputStream = new FileInputStream(file); //load the file
// here I use Commons IO API to copy this file to the response output stream, I don't know which API you use.
IOUtils.copy(inputStream, response.getOutputStream());
// here we define the content of this file to tell the browser how to handle it
response.setContentType("application/pdf");
response.setHeader("Content-disposition", "attachment;filename=" + fileName + ".pdf");
response.flushBuffer();
}
return response.getOutputStream().toString();
}
I get the downloaded file and convert it to base64 to write it to an XML file. I have to make the comparison later and the problem is the writing of the signature of the object in the byte stream of the base 64: CoyoteOutputStream
At the end of each base64 of pdf I have a piece:
CnN0YXJ0eHJlZgo0NjkyOAolJUVPRgpvcmcuYXBhY2hlLmNhdGFsaW5hLmNvbm5lY3Rvci5Db3lvdGVPdXRwdXRTdHJlYW1ANDA4YTViYzI=
that is different each time because:
startxref
46928
%% EOF
org.apache.catalina.connector.CoyoteOutputStream#408a5bc2
So, this pit but comparison because : #408a5bc2 is unique
You eventually return this:
return response.getOutputStream().toString();
This returns the default toString output for the output stream in question, i.e. your CoyoteOutputStream object signature. If you want to avoid this, don't return it.
I am using the following code to get the content from an object in s3 bucket. I am able to copy the data into a file locally, but the file needs to be 'downloaded' and it has to be shown in the browser's downloads list.
I searched a bit about this and cam to know this has something to do with response.setHeader( "Content-Disposition", "attachment;filename=\"" + filename + "\"" );
I tried to add that too but somehow couldn't get it to work. My understanding is that the source has to be a file on a server which is converted into a stream. but I get the s3 contents in the form of a input stream. How to i download this as a file in the browser?
Below is the code i have tried so far
AmazonS3 s3 = new AmazonS3Client(new ProfileCredentialsProvider());
S3Object fetchFile = s3.getObject(new GetObjectRequest(bucketname, fileloc));
final BufferedInputStream i = new BufferedInputStream(fetchFile.getObjectContent());
response.setContentType("application/json");
response.setHeader( "Content-Disposition", "attachment;filename=\"" + filename + "\"" );
ServletOutputStream sos = response.getOutputStream();
int bytesread = i.read();
while(bytesread!=-1)
{
sos.write(bytesread);
bytesread = i.read();
}
if(i!= null)i.close();
if(sos!= null)sos.close();
Everything is correct except the file reading part.
//Set the size of buffer to stream the data.
byte []buffer=new byte[1024*8];
instead of
int byte=i.read();
Now read the file.
while( ( length = yourInputStream.read(buffer))!=-1)
{ yourOutputStream.write(buffer);
}
System.out.println("File is downloaded.");
Additionally,puting your whole code within try/catch block will help you to know the exact reason of your problem.
I am generating csv in my code, It takes some time to generate. So, I am sending an email with link once the csv file is generated. When I click that, getting 404 not found error. When I have the same link in the html, I am able to download it. Any insight or sample to refer
Sample Link -http://localhost:9090/api/report/file?fileName=filename.csv
Java code to download the report
#RequestMapping(value = "api/report/file")
public void downloadCSV(HttpServletResponse response, #RequestParam("fileName") String fileName) throws IOException {
File file = new File(fileName);
InputStream is = new FileInputStream(file);
response.setContentType("application/octet-stream");
// Response header
response.setHeader("Content-Disposition", "attachment; filename=\"" + file.getName() + "\"");
// Read from the file and write into the response
OutputStream os = response.getOutputStream();
byte[] buffer = new byte[1024];
int len;
while ((len = is.read(buffer)) != -1) {
os.write(buffer, 0, len);
}
os.flush();
os.close();
is.close();
}
Add GET method to this mapping: #RequestMapping(value = "api/report/file")