A web Service expects a byte[] coming from a zip file.
I have some files in a folder that I zip with Java and then I get the byte[] from this zip file.
Is this necessary or can I create the byte[] straight from the folder?
I think something like this would allow you to do what you want without writing as long as the files are not going to be very big.
String[] sourceFiles = { "C:/file1", "C:/file2" };
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ZipOutputStream zout = new ZipOutputStream(baos);
byte[] buffer = new byte[4096];
for (int i = 0; i < sourceFiles.length; i++)
{
FileInputStream fin = new FileInputStream(sourceFiles[i]);
zout.putNextEntry(new ZipEntry(sourceFiles[i]));
int length;
while ((length = fin.read(buffer)) > 0)
{
zout.write(buffer, 0, length);
}
zout.closeEntry();
fin.close();
}
zout.close();
byte[] bytes = baos.toByteArray();
A folder is a collection of files. It is a container. It does not have a byte stream to get in the first place.
On the other hand, a ZIP (or any archive) is a file. The information about the different files is stored within the ZIP file itself
However, you can iterate through the folder contents, cook up a byte array and then use it (you are doing that anyways while creating the ZIP).
Related
my java zip problem apeears only in websphere in linux environment,
but works on my local windows developer computer in tomee all right.
I try to compress a content and send the result as mail and backup the same content in a folder.
My input data is a CSV file content in a byte-array.
I compress the input byte array to a zip result byte array.
I try to save my zipped result data in a folder and send the same content as a mail attachment.
Result: zip-attachment in mail is delivered correctly but the same zip byte array saved as a file is corrupted.
public static byte[] zipContent(String filename, byte[] content) throws IOException {
final CRC32 crc = new CRC32();
final ByteArrayOutputStream baos = new ByteArrayOutputStream();
try (final ZipOutputStream zipOut = new ZipOutputStream(baos);
final InputStream fis = new ByteArrayInputStream(content);) {
crc.reset();
final ZipEntry zipEntry = new ZipEntry(filename);
zipOut.putNextEntry(zipEntry);
int length;
byte[] bytes = new byte[1024];
while ((length = fis.read(bytes)) >= 0) {
crc.update(bytes,0,length);
zipOut.write(bytes, 0, length);
}
final long crcValue = crc.getValue();
zipEntry.setCrc(crcValue);
zipEntry.setSize(content.length);
zipEntry.setComment("CRC_" + Long.toString(crcValue));
zipOut.closeEntry();
zipOut.flush();
} finally {
baos.flush();
baos.close();
}
return baos.toByteArray();
}
I call
final byte[] csvContentArray = ... // Get CSV input anyway ...
File csvFile = new File(aCsvPathString);
Files.write(csvfile.toPath(), csvContentArray); // OKAY, saved correctly pre compress in server
//COMPRESS: byte[] --> byte[]
final byte[] zippedContent = zipContent(csvFileName, csvContentArray);
File zipFile = new File(aZipPathString);
Files.write(zipfile.toPath(), zippedContent); // CORRUPTED, saved after compress in server !!!
...
sendMail(emailAdress, zippedContent); // the same zipped content in mail is delivered correctly and can be opened without warnings !
What differs for zip file content as byte[] between saving in file system and sending as mail attachment (MimeBodyPart).
It is very strange for me, because it works fine in Windows/Tomee but not in Linux/Websphere.
My zipped csv file is huge and after compressing, first about 1000 lines are fine. Rest of file is confused.
I tried to code with and without CRC by ZipEntry, but it doesn't matter.
Opening corrupted zip in 7zip gives warnings like:
Unerwartetes Datenende. Translated: unexpected data end.
Es gibt noch Daten hinter den Hauptdaten. Translated: existing data after main data.
I'm always grateful for opinions and suggestions.
I have a requirement to split a 100mb zip file(which will be having sub folders and images) into 10 zip files(each of 10mb).Then I need to send each sliced zip files to an API (as multipart reauest), in receiver API i need to combine each of the above 10 zip files back to origin 100mb zip file.
Below is the code for slicing
public static void splitZip(String zipName, String location, String NewZip) throws IOException{
FileInputStream fis = new FileInputStream(location);
ZipInputStream zipInputStream = new ZipInputStream(fis);
ZipEntry entry = null;
int currentChunkIndex = 0;
long entrySize = 0;
ZipFile zipFile = new ZipFile(location);
Enumeration enumeration = zipFile.entries();
String copDest = zipCopyDest + "\\" + NewZip + "_" + currentChunkIndex +".zip";
FileOutputStream fos = new FileOutputStream(new File(copDest));
BufferedOutputStream bos = new BufferedOutputStream(fos);
ZipOutputStream zos = new ZipOutputStream(bos);
long currentSize = 0;
try {
while ((entry = zipInputStream.getNextEntry()) != null && enumeration.hasMoreElements()) {
ZipEntry zipEntry = (ZipEntry) enumeration.nextElement();
System.out.println(zipEntry.getName());
System.out.println(zipEntry.getSize());
entrySize = zipEntry.getSize();
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
if((currentSize + entrySize) > MAX_FILE_SIZE) {
zos.close();
currentChunkIndex++;
zos = getOutputStream(currentChunkIndex, NewZip);
currentSize = 0;
}else{
currentSize += entrySize;
zos.putNextEntry(new ZipEntry(entry.getName()));
byte[] buffer = new byte[8192];
int length = 0;
while ((length = zipInputStream.read(buffer)) > 0) {
outputStream.write(buffer, 0, length);
}
byte[] unzippedFile = outputStream.toByteArray();
zos.write(unzippedFile);
unzippedFile = null;
outputStream.close();
zos.closeEntry();
}
}
} finally {
zos.close();
}
}
When i extract slices zips manually i found some images are corrupted am not able to open it. Also am not getting a proper solution for combining the zip files. Thanks in advance.
Zip specification has a feature to split zip files to any desired length (minimum split length should be 64kb). Zip4j, supports this feature to create split zip files (documentation). You can then pass each split file to the api. The api can then use the merge functionality in zip4j to merge those split files into a single zip file. On a side note: even without merging, it is a perfectly valid zip file. as long as all the split zip files are in the same directory.
The approach will not work if you have any file in the zip greater than 10 mb and also the else condition should be executed in both the cases and in this approach you have to depend on the size of file in zip may be better you go with approach of not creating smaller zips
Currently I am using below code to copy zip inside another zip.But when zip file size increases eg:2GB, program is throwing out of memory error.I have increased xmx to 1024, still prfoblem is same.Is thr s any alternate method to handle large files ?
public static void zipFile(File srcFile, File zipFile)
throws FileNotFoundException, IOException {
BufferedInputStream origin = null;
FileOutputStream dest = new FileOutputStream(zipFile);
ZipOutputStream out = new ZipOutputStream(
new BufferedOutputStream(dest));
// out.setMethod(ZipOutputStream.DEFLATED);
byte data[] = new byte[BUFFER];
FileInputStream fi = new FileInputStream(srcFile);
origin = new BufferedInputStream(fi, BUFFER);
ZipEntry entry = new ZipEntry(srcFile.getName());
out.putNextEntry(entry);
int count;
while ((count = origin.read(data, 0, BUFFER)) != -1) {
out.write(data, 0, count);
}
origin.close();
out.close();
}
You are only copying bytes when you come right down to it. You don't need to process either file as a Zip file. Just copy the bytes.
Can you try FileChannel.transferTo method? It's more efficient. Since this is done at OS level, I'm assuming it shouldn't depend on Java Heap size.
If that fails too take a look at this question.
Getting data onto inputStream object from web url
inputStream = AWSFileUtil.getInputStream(
AWSConnectionUtil.getS3Object(null),
"cdn.generalsentiment.com", filePath);
If they are mutliple files then i want to zip them and sent the filetype as "zip" to struts.xml which does the download.
actually am converting the inputstream into byteArrayInputStream
ByteArrayInputStream byteArrayInputStream = new
ByteArrayInputStream(inputStream.toString().getBytes());
while (byteArrayInputStream.read(inputStream.toString().getBytes()) > 0) {
zipOutputStream.write(inputStream.toString().getBytes());
}
and then
zipOutputStream.close();
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
fileInputStream = new FileInputStream(file);
while (fileInputStream.read(buffer) > 0) {
byteArrayOutputStream.write(buffer);
}
byteArrayOutputStream.close();
inputStream = new ByteArrayInputStream(byteArrayOutputStream.toByteArray());
reportName = "GS_MediaValue_Reports.zip";
fileType = "zip";
}
return fileType;
But the downloaded zip when extracted gives corrupt files.
Please suggest me a solution for this issue.
The short answer is that it's not how ZipOutputStream works. Since it was designed to store multiple files, along with their file names, directory structures and so on, you need to tell the stream about that explicitly.
Furthermore, converting a stream to a string is a bad idea in general, plus it's slow, especially when you're doing it in a loop.
So your solution will be something like:
ZipEntry entry = new ZipEntry( fileName ); // You have to give each entry a different filename
zipOutputStream.putNextEntry( entry );
byte buffer[] = new byte[ 1024 ]; // 1024 is the buffer size here, but it could be anything really
int count;
while( (count = inputStream.read( buffer, 0, 1024 ) ) != -1 ) {
zipOutputStream.write( buffer, 0, count );
}
My goal is to put multiple java.io.File objects into a zip file and print to HttpServletResponse for the user to download.
The files were created by the JAXB marshaller. It's a java.io.File object, but it's not actually on the file system (it's only in memory), so I can't create a FileInputStream.
All resources I've seen use the OutputStream to print zip file contents. But, all those resources use FileInputStream (which I can't use).
Anyone know how I can accomplish this?
Have a look at the Apache Commons Compress library, it provides the functionality you need.
Of course "erickson" is right with his comment to your question. You will need the file content and not the java.io.File object. In my example I assume that you have a method
byte[] getTheContentFormSomewhere(int fileNummer) which returns the file content (in memory) for the fileNummer-th file. -- Of course this function is poor design, but it is only for illustration.
It should work a bit like this:
void compress(final OutputStream out) {
ZipOutputStream zipOutputStream = new ZipOutputStream(out);
zipOutputStream.setLevel(ZipOutputStream.STORED);
for(int i = 0; i < 10; i++) {
//of course you need the file content of the i-th file
byte[] oneFileContent = getTheContentFormSomewhere(i);
addOneFileToZipArchive(zipOutputStream, "file"+i+"."txt", oneFileContent);
}
zipOutputStream.close();
}
void addOneFileToZipArchive(final ZipOutputStream zipStream,
String fileName,
byte[] content) {
ZipArchiveEntry zipEntry = new ZipArchiveEntry(fileName);
zipStream.putNextEntry(zipEntry);
zipStream.write(pdfBytes);
zipStream.closeEntry();
}
Snipets of your http controller:
HttpServletResponse response
...
response.setContentType("application/zip");
response.addHeader("Content-Disposition", "attachment; filename=\"compress.zip\"");
response.addHeader("Content-Transfer-Encoding", "binary");
ByteArrayOutputStream outputBuffer = new ByteArrayOutputStream();
compress(outputBuffer);
response.getOutputStream().write(outputBuffer.toByteArray());
response.getOutputStream().flush();
outputBuffer.close();
Turns out I'm an idiot :) The file that was being "created" was saving to invalid path and swallowing the exception, so I thought it was being "created" ok. When I tried to to instantiate a new FileInputStream, however, it complained that file didn't exist (rightly so). I had a brainfart and assumed that the java.io.File object actually contained file information in it somewhere. But as erickson pointed out, that was false.
Thanks Ralph for the code, I used it after I solved the invalid pathing issue.
My code:
ZipOutputStream out = new ZipOutputStream(response.getOutputStream());
byte[] buf = new byte[1024];
File file;
InputStream in;
// Loop through entities
for (TitleProductAccountApproval tpAccountApproval : tpAccountApprovals) {
// Generate the file
file = xmlManager.getXML(
tpAccountApproval.getTitleProduct().getTitleProductId(),
tpAccountApproval.getAccount().getAccountId(),
username);
// Write to zip file
in = new FileInputStream(file);
out.putNextEntry(new ZipEntry(file.getName()));
int len;
while ((len = in.read(buf)) > 0) {
out.write(buf, 0, len);
}
out.closeEntry();
in.close();
}
out.close();