java: Zip Exception invalid codes length? - java

When I'm writing entries into a zip file like this:
ZipEntry ze = zin.getNextEntry();
while (ze != null) {
InputStream is = zf.getInputStream(ze);
zos.putNextEntry(ze);
int len;
while ((len = is.read(buffer)) >= 0) {
zos.write(buffer, 0, len);
}
zos.closeEntry();
ze = zin.getNextEntry();
}
I get the following exception on the second while loop:
java.util.zip.ZipException: invalid code lengths set
at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:164)
at java.io.FilterInputStream.read(FilterInputStream.java:107)
Anybody knows why this exception is thrown and what does it mean?
P.S. I should mention that I'm running this on a listeners on JBoss 7.1.1 in order to zip vaious log files from different folders. There's a thread for each folder. Could the fact of using multiple threads lead to this problem?

You are setting the ZipEntry of the new zip file to the same instance you got from the original file. This implies that all values must match, but this fails if the compressed size of the written entry does not match the compressed size of the source file. And the tiniest bit of difference between the original compression code and the one you are using now will yield to different compressed sizes. To make it run you have to create a copy of the ZipEntry for the output and reset the compressed size on it.
By the way, you are using zin.getNextEntry() and zf.getInputStream(ze) in your code which implies that you are using a ZipFile and a ZipInputStream at the same time. If they refer to the same file, it’s a waste of resources, if they refer to different files, you can expect even more problems.
Decide for either ZipFile
ZipFile zf = …
ZipOutputStream zos = …
byte[] buffer = …
for(Enumeration<? extends ZipEntry> e=zf.entries(); e.hasMoreElements();) {
ZipEntry ze = e.nextElement();
InputStream is = zf.getInputStream(ze);
ZipEntry zeOut=new ZipEntry(ze);
zeOut.setCompressedSize(-1);
zos.putNextEntry(zeOut);
int len;
while ((len = is.read(buffer)) >= 0) {
zos.write(buffer, 0, len);
}
zos.closeEntry();
}
zos.close();
zf.close();
or ZipInputStream
ZipInputStream zin…
ZipOutputStream zos=…
byte[] buffer = …
ZipEntry ze = zin.getNextEntry();
while (ze != null) {
ZipEntry zeOut=new ZipEntry(ze);
zeOut.setCompressedSize(-1);
zos.putNextEntry(zeOut);
int len;
while ((len = zin.read(buffer)) >= 0) {
zos.write(buffer, 0, len);
}
zos.closeEntry();
ze = zin.getNextEntry();
}
zos.close();
zin.close();

Related

Java : Unexpected end of ZLIB input stream while attempting to merge split zip files

I have split zip files that I'm trying to merge using Java. But I get Unexpected end of ZLIB input stream error. Any thoughts on what I'm doing wrong?
File bigZip = new File("bigZip.zip");
List<String> zipList = Arrays.asList("src/14thmayreceipts.zip.001","src/14thmayreceipts.zip.002", "src/14thmayreceipts.zip.003");
Collections.sort(zipList);
ZipOutputStream outputStream = new ZipOutputStream(new FileOutputStream(bigZip));
for (String entry : zipList) {
readWriteZip(outputStream, entry);
}
outputStream.close();
}
private static void readWriteZip(ZipOutputStream out, String fileName) throws IOException, EOFException {
File file = new File(fileName);
ZipInputStream inStream = new ZipInputStream(new FileInputStream(file));
byte[] buffer = new byte[1024];
int len = 0;
for (ZipEntry e; (e = inStream.getNextEntry()) != null; ) {
out.putNextEntry(e);
while ((len = inStream.read(buffer)) > 0) {
out.write(buffer, 0, len);
}
}
inStream.close();
}
JDK's zip does not support split zip files. And moreover you cannot work with Input/Outpustreams when working with split zip files. And also when merging a split zip files a lot of headers in the zip file have to be updated. zip4j supports such a feature. Sample code:
ZipFile zipFile = new ZipFile("splitZipFileThatHasToBeMerged.zip");
zipFile.mergeSplitZipFiles("mergedOutputZipFile.zip");

Split big zip file and combine it back as original zip file

I have a requirement to split a 100mb zip file(which will be having sub folders and images) into 10 zip files(each of 10mb).Then I need to send each sliced zip files to an API (as multipart reauest), in receiver API i need to combine each of the above 10 zip files back to origin 100mb zip file.
Below is the code for slicing
public static void splitZip(String zipName, String location, String NewZip) throws IOException{
FileInputStream fis = new FileInputStream(location);
ZipInputStream zipInputStream = new ZipInputStream(fis);
ZipEntry entry = null;
int currentChunkIndex = 0;
long entrySize = 0;
ZipFile zipFile = new ZipFile(location);
Enumeration enumeration = zipFile.entries();
String copDest = zipCopyDest + "\\" + NewZip + "_" + currentChunkIndex +".zip";
FileOutputStream fos = new FileOutputStream(new File(copDest));
BufferedOutputStream bos = new BufferedOutputStream(fos);
ZipOutputStream zos = new ZipOutputStream(bos);
long currentSize = 0;
try {
while ((entry = zipInputStream.getNextEntry()) != null && enumeration.hasMoreElements()) {
ZipEntry zipEntry = (ZipEntry) enumeration.nextElement();
System.out.println(zipEntry.getName());
System.out.println(zipEntry.getSize());
entrySize = zipEntry.getSize();
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
if((currentSize + entrySize) > MAX_FILE_SIZE) {
zos.close();
currentChunkIndex++;
zos = getOutputStream(currentChunkIndex, NewZip);
currentSize = 0;
}else{
currentSize += entrySize;
zos.putNextEntry(new ZipEntry(entry.getName()));
byte[] buffer = new byte[8192];
int length = 0;
while ((length = zipInputStream.read(buffer)) > 0) {
outputStream.write(buffer, 0, length);
}
byte[] unzippedFile = outputStream.toByteArray();
zos.write(unzippedFile);
unzippedFile = null;
outputStream.close();
zos.closeEntry();
}
}
} finally {
zos.close();
}
}
When i extract slices zips manually i found some images are corrupted am not able to open it. Also am not getting a proper solution for combining the zip files. Thanks in advance.
Zip specification has a feature to split zip files to any desired length (minimum split length should be 64kb). Zip4j, supports this feature to create split zip files (documentation). You can then pass each split file to the api. The api can then use the merge functionality in zip4j to merge those split files into a single zip file. On a side note: even without merging, it is a perfectly valid zip file. as long as all the split zip files are in the same directory.
The approach will not work if you have any file in the zip greater than 10 mb and also the else condition should be executed in both the cases and in this approach you have to depend on the size of file in zip may be better you go with approach of not creating smaller zips

File Compression to fixed size In Java

I want to perform file compression using zip package of java.util library. The aim is to limit the compressed file to a fixed size. If the compressed file size is greater than this limit, it should be split into multiple files.
try {
fos = new FileOutputStream(p_request.getOutputFilePath() + zipFileName);
ZipOutputStream zos = new ZipOutputStream(fos);
zipEntry1 = new ZipEntry(f.getName());
fis = new FileInputStream(f.getAbsolutePath());
int count;
while ((count = fis.read(fileRAW, 0, BUFFER)) != -1) {
zipEntry1 = new ZipEntry(f.getName());
if (currentSize >= (p_request.getMaxSizePerFileInMB() * 1024 * 1024)) {
zipSplitCount++;
zos.close();
zos = new ZipOutputStream(new FileOutputStream(
p_request.getOutputFilePath() + zipFileName
+ "_" + zipSplitCount + ".zip"));
currentSize = 0;
}
zos.putNextEntry(zipEntry1);
// zos.closeEntry();
currentSize += zipEntry1.getCompressedSize();
zos.write(fileRAW, 0, count);
}
I always get compressed size as -1. Can someone advise a clean approach for this?
EDIT:
So I compressed the file into chunks of fixed size to get multi parts compressed zip of the same file as f.1.zip, f.2.zip. Now when I decompress it, is there some way to restore the original file? Currently, it says the file must be broken.
byte[] buffer = new byte[BUFFER];
ZipInputStream zis = null;
try {
zis = new ZipInputStream(new FileInputStream(f.getAbsolutePath()));
ZipEntry zipEntry = zis.getNextEntry();
while(zipEntry!=null){
String fileName = zipEntry.getName();
File newFile = new File(p_request.getOutputFilePath() + fileName);
System.out.println("file unzip : "+ newFile.getAbsoluteFile());
new File(newFile.getParent()).mkdirs();
FileOutputStream fos = new FileOutputStream(newFile);
int len;
while ((len = zis.read(buffer)) > 0) {
fos.write(buffer, 0, len);
}
fos.close();
zipEntry = zis.getNextEntry();
}
zis.closeEntry();
zis.close();
You are getting -1 because the size is not known until the Zip file has been written to disk. Compression takes place when you save the entire zip file, not when you add a new entry.
This means that you have to either:
write the zip to disk after adding each file and then measuring the zip to determine whether to keep adding to it or creating a new file
or guesstimate the size based on an average compression rate and the size of the file before compression on disk.

Unable to unzip zip file created with java

I have a list of files from different locations. I create a zip file using the following the code which works without error. But when I try to unzip the file in Windows using Extract All it fails seeing unable to find any bytes, yet if I double click into the zip file itself with Windows Explorer I can see the files and individual ones can be opened and contains the correct data
ZipOutputStream zos = new ZipOutputStream(new FileOutputStream(zipFile));
for (File next : files)
{
ZipEntry zipEntry = new ZipEntry(next.getName());
zos.putNextEntry(zipEntry);
FileInputStream in = new FileInputStream(next);
byte[] buf = new byte[1024];
int len;
while ((len = in.read(buf)) > 0)
{
zos.write(buf, 0, len);
}
zos.closeEntry();
in.close();
}
zos.close();
This may or may not be related but I've found using fixed byte length can lead to a loss of new line characters.
This may help:
final byte[] newLine = System.getProperty(
"line.separator").getBytes("UTF-8");
while ((line = in.readLine()) != null)
final byte[] buffer = line.getBytes("UTF-8");
out.write(buffer, 0, buffer.length);
out.write(newLine, 0, newLine.length);
}

help me translate Java code making use of bytes into jython code

how do I translate this code into jython?
ZipOutputStream zos = new ZipOutputStream(new FileOutputStream(file + ".zip"));
byte[] buf = new byte[1024];
int len;
//Create a new Zip entry with the file's name.
ZipEntry zipEntry = new ZipEntry(file.toString());
//Create a buffered input stream out of the file
//we're trying to add into the Zip archive.
FileInputStream fin = new FileInputStream(file);
BufferedInputStream in = new BufferedInputStream(fin);
zos.putNextEntry(zipEntry);
//Read bytes from the file and write into the Zip archive.
while ((len = in.read(buf)) >= 0) {
zos.write(buf, 0, len);
}
//Close the input stream.
in.close();
//Close this entry in the Zip stream.
zos.closeEntry();
this is what I have but it Fails badly
buf=None <<<< ?
len=None <<<< ?
zipEntry=ZipEntry(file.toString())
fin=FileInputStream(file)
bin=BufferedInputStream(fin)
self._zos.putNextEntry(zipEntry)
while (len=bin.helpme_im_dying(buf)) >= 0): <<<< ?
self._zos.write(buf,0,len) <<<< ?
len = bin.read(buf) <<<< ?
bin.close()
self._zos.closeEntry()
refer to this page for information https://www.acm.org/crossroads/xrds6-3/ovp63.html
Here's an exact translation of that function (except, like your case, using bin instead of reserved keyword in).
from jarray import zeros
from java.io import BufferedInputStream, FileInputStream, FileOutputStream
from java.util.zip import ZipEntry, ZipOutputStream
def test(file):
zos = ZipOutputStream(FileOutputStream(file + ".zip"))
buf = zeros(1024, 'b')
zipEntry = ZipEntry(file)
fin = FileInputStream(file)
bin = BufferedInputStream(fin)
zos.putNextEntry(zipEntry)
len = bin.read(buf)
while len >= 0:
zos.write(buf, 0, len)
len = bin.read(buf)
bin.close()
zos.closeEntry()
It is not an answer to your question, but related. Here is a CPython version:
from zipfile import ZipFile, ZIP_DEFLATED
def test(file):
ZipFile(file+".zip", "w", ZIP_DEFLATED).write(file)
Don't use ZipFile without ensuring it is closed:
with ZipFile('spam.zip', 'w') as myzip:
myzip.write('eggs.txt')

Categories

Resources