I am trying to write a servlet to download files as a zip, which will read multiple nodes in a repository (CRX) to get Inputstream of multiple images.
I am using ZipOutputStream to download the zip file.
But as the length after zipping is not known, I can't set the content-length header in response, hence the browser is not able to show the remaining time to download.
Current code:
String[] paths = request.getRequestParameters("path");
ZipOutputStream out = new ZipOutputStream(response.getOutputStream());
for (int i=0; i<paths.length;i++){
InputStream is = getStream(paths[i]);
IOUtils.copy(is, out);
IOUtils.closeQuietly(is);
out.closeEntry();
}
Is there a way to generate the ZipFile and then write to the output stream?
Related
To begin with, I am well aware that a similar topic is available on Stack OverFlow, but this one does not answer my problem. That's why I'm making this post.
At the moment, my program is able to search for ".csv" files locally, according to certain criteria, to add them to a list, and then to create a file in ".zip" format, which contains all these files. This works without any problem.
Now, I want to download this ".zip" file, from a Web interface, with a button. The connection between the code and the button works, but when I open my downloaded ".zip" file, it contains only one file, and not all the ".csv" files it is supposed to contain. Where can the problem come from?
Here is my code:
FileOutputStream baos = new FileOutputStream("myZip.zip");
ZipOutputStream zos = new ZipOutputStream(baos);
for(String sCurrent : selectedFiles){
zos.putNextEntry(new ZipEntry(new File(sCurrent).getName()));
Files.copy(Paths.get(sCurrent), zos);
zos.closeEntry();
}
zos.close();
response.getOutputStream().flush();
response.getOutputStream().close();
You are closing the ZIP after sending the response. Swap the order:
zos.close();
response.getOutputStream().write(baos.toByteArray());
It would be more efficient if you use try-with-resources to handle close, Files.copy(Paths.get(currentFile), zos); to transfer the files, and if you ZIP straight to the output stream because you might risk out of memory exceptions for large files:
ZipOutputStream zos = new ZipOutputStream(response.getOutputStream());
Not an answer so much as consolidation based on #DuncG
ZipOutputStream zos = new ZipOutputStream(response.getOutputStream());
for (String path : selectedFiles) {
zos.putNextEntry(new ZipEntry(new File(path).getName()));
Files.copy(Paths.get(path), zos);
zos.closeEntry();
}
zos.close();
In Java for a JUnit test, I am trying to mock a function that downloads a Zip File from another external API's endpoint. To simulate the download, I need to zip a test file and transform it into bytes to use as the mock's return value. I do not need to write the zipped file back to the file system but use the bytes raw as they are.
mock(zipReturner.getZipBytes()).thenReturn(testFileAsZippedBytes("testFile.txt"))
private Optional<byte[]> testFileAsZippedBytes(String testFile) {
???
}
Sharing my answer, because all the other examples I found are much heavier, require many more lines of code looping over bytes, or require using external libraries to do the same thing.
To do this without the above, use a combination of ByteArrayOutputStream, as it has the toByteArray function, ZipOutputStream to write zipped bytes to the ByteArrayOutputStream and FileInputStream to read the test file from the file system.
private Optional<byte[]> testFileAsZippedBytes(String filePath, String fileName) throws IOException {
try (
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
ZipOutputStream zipOutputStream = new ZipOutputStream(byteArrayOutputStream);
FileInputStream fileInputStream = new FileInputStream(filePath + fileName);
) {
ZipEntry zipEntry = new ZipEntry(fileName);
zipOutputStream.putNextEntry(zipEntry);
zipOutputStream.write(fileInputStream.readAllBytes());
zipOutputStream.finish();
return Optional.of(byteArrayOutputStream.toByteArray());
}
}
Use ZipEntry to add the file as an entry to the ZipOutputStream and write the bytes to the zip. Use zipOutputStream.finish() to ensure all contents are written to the stream and are ready to be consumed in the ByteArrayOutputStream, otherwise it was my experience that you would only get partial data when you call byteArrayOutputStream.toByteArray().
I have a very confusing problem and hope that I can get some ideas here.
My problem is very simple, but I didn't find a solution yet.
I want to create a simple ZIP File with ZipEntry's in it. The ZipEntry's are created by a given byte array (saved in a Postgres-DB with Hibernate).
When I put this byte array into my ZipOutputStream.write(..) the ZIP File created is always corrupt. What am I doing wrong?
The ZIP File is transferred to a FTP-Server afterwards.
ByteArrayOutputStream bos = new ByteArrayOutputStream();
final ZipOutputStream zipOut = new ZipOutputStream(bos);
String filename = "test.zip";
for(final Attachment attachment : transportDoc.getAttachments()) {
log.debug("Adding "+attachment.getFileName()+" to ZIP file /tmp/"+filename);
ZipEntry ze = new ZipEntry(attachment.getFileName());
zipOut.putNextEntry(ze);
zipOut.write(attachment.getFileContent());
zipOut.flush();
zipOut.closeEntry();
}
zipOut.close();
org.apache.commons.io.FileUtils.writeByteArrayToFile(new File("/tmp/"+filename), bos.toByteArray());
I am confused, because when I replaced
zipOut.write(attachment.getFileContent()); //This is the byte array from db
with
zipOut.write("Bla bla".getBytes());
it worked!
But the byte array from the DB can't be corrupt, because it can be written to a file with
org.apache.commons.io.FileUtils.writeByteArrayToFile(new File("/tmp/test.png"), attachment.getFileContent());
with no problem. It is a correct file.
I hope you have some ideas left.
Thanks in advance.
EDIT:
I tried to repair the ZIP file offline and then this messages appears:
zip warning: no end of stream entry found: cglhnngplpmhipfg.png
(This png file is the byte-Array-File)
Simple unzip-command output the following:
unzip created.zip
Archive: created.zip
error [created.zip]: missing 2 bytes in zipfile
(attempting to process anyway)
error [created.zip]: attempt to seek before beginning of zipfile
(please check that you have transferred or created the zipfile in the
appropriate BINARY mode and that you have compiled UnZip properly)
(attempting to re-compensate)
replace cglhnngplpmhipfg.png? [y]es, [n]o, [A]ll, [N]one, [r]ename: y
inflating: cglhnngplpmhipfg.png
error: invalid compressed data to inflate
file #2: bad zipfile offset (local header sig): 24709
(attempting to re-compensate)
inflating: created.xml
EDIT 2:
When I write this file to the Filesystem and add this file to the ZIP by an InputStream it doesn't work either! But the File on the Filesystem is ok. I can open the Image with no problem. Its very confusing
File tmpAttachment = new File("/tmp/"+filename+attachment.getFileName());
FileUtils.writeByteArrayToFile(tmpAttachment, attachment.getFileContent());
FileInputStream inTmp = new FileInputStream(tmpAttachment);
int len;
byte[] buffer = new byte[1024];
while ((len = inTmp.read(buffer)) > 0) {
zipOut.write(buffer, 0, len);
}
inTmp.close();
EDIT 3:
This problem only appears when I try to add "complex" files like png or pdf. If I put a txt-file in it, it works.
The problem was NOT in the Zip-Library itself.
It was the transmission to an external FTP Server with wrong mode. (Not binary).
Thanks all for your help.
Try closeEntry() before flush(). Also you can try to explicitly specify the size of the entry using ze.setSize(attachment.getFileContent().length).
I want to download multiple zip files as one zip file for a request.
I have zip file paths like C, https://test12.zip etc. So how can I download these files as a one zip file. I have been searching this for a while. All i got is examples for downloading multiple files(local) and zip them. This is what i tried for downloading one file. For multiple files it won't work.
URL url = new URL("https://test12.zip");
URLConnection connection = url.openConnection();
InputStream stream = connection.getInputStream();
BufferedOutputStream outs = new BufferedOutputStream(response.getOutputStream());
int len;
byte[] buf = new byte[1024];
while ((len = stream.read(buf)) > 0) {
outs.write(buf, 0, len);
}
outs.close();
Any help would be much appreciated.
A ZIP file consists of two parts: First the compressed file entries (filename, attributes and data) and at the end of the file there is a central directory containing a list of all entries, again with filename and attributes.
Hence, you can not directly combine or concatenate zip files. In Java you can only decompress the downloaded zip files on-the-fly (without storing them in the file-system) and at the same time using the decompressed content to create a new combined ZIP file:
First create a ZipOutputStream for the zip file you want to create.
Then use the InputStream of each download and use it with a ZipInputStream.
Iterates through all the entries in every ZipInputStream and for each entry create a new identical entry in the ZipOutputStream and copy the content from the ZipInputStream to the ZipOutputStream.
How to use ZipInputStream see for example: https://stackoverflow.com/a/36648504/150978
Note that this process requires to decompress and afterwards re-compress the file content. Depending on the archive size this can result in a high utilization of one CPU core.
I occasionally receive .zip files in my app that throw start of central directory not found;
zipfile corrupt. exceptions. These zip files open just fine in my Mac's Finder.
I can fix these files every time from the command line, using zip -FF bad.zip --out good.zip
Can any Java ZIP libraries out there accomplish the same thing?
You probably want to just let Java execute this command, because in strict terms zip is more like a container and it can contain different compression algorithms.
In general investigating and solving problems related to compressed archives with a programmatic approach it's likely to be a tricky and long task.
Try this with your command.
I tried using ZipInputStream and ZipOutputStream. But ZipInputStream always failed at some point when doing: "getNextEntry()". Basically the following lines of code in "getNextEntry()":
...
if ((entry = readLOC()) == null) {
return null;
}
...
returned null after some entries and I could not get further.
But finally I could solve the issue using ZipFile together with ZipOutputStream because ZipFile was reading all zip entries without problem and the solution looks like this:
protected void repairZipFile(String file) throws IOException {
File repairZipFile = new File(file+".repair");
ZipFile zipFile = new ZipFile(file);
Enumeration<? extends ZipEntry> zipFileEntries = zipFile.entries();
InputStream zis;
ZipOutputStream zos = new ZipOutputStream(new FileOutputStream(repairZipFile));
byte[] b = new byte[1024];
while(zipFileEntries.hasMoreElements()){
ZipEntry zipEntry = zipFileEntries.nextElement();
zos.putNextEntry(zipEntry);
zis = zipFile.getInputStream(zipEntry);
int n = zis.read(b);
while(n>=0) {
zos.write(b, 0, n);
n = zis.read(b);
}
zis.close();
zos.closeEntry();
}
zipFile.close();
zos.flush();
zos.close();
Files.move(repairZipFile.toPath(), (new File(file)).toPath(), StandardCopyOption.REPLACE_EXISTING);
}
There are two ways to open ZIP files in Java, using the ZipFile class, or using ZipInputStream.
As far as I remember, ZipFile reads the central directory of a zip file first - it can do this because it uses a RandomAccessFile underneath. However, ZipInputStream uses the in-line entry information, which might be better if the central directory, which I think exists at the end of the file, is missing or corrupt.
So, it might be possible to 'repair' a ZIP file in Java by reading a ZIP file using ZipInputStream, and writing it back out to another file using a ZipOutputStream, copying entry information between them. You might end up getting IO exceptions reading from the last entry of the ZipInputStream if it got truncated, but it might still save the other previous entries from the file.