I have the following code that writes a text file in a zip:
FileOutputStream fOut = new FileOutputStream(fullFilename, false);
BufferedOutputStream bOut = new BufferedOutputStream(fOut);
ZipOutputStream zOut = new ZipOutputStream(bOut);
zOut.putNextEntry(new ZipEntry("aFile1.txt"));
//Do some processing and write to zOut...
zOut.write(...);
(...)
zOut.closeEntry();
zOut.close();
//Etc (close all resources)
I would need to change the filename of the zipEntry after it has been written (as its name will depend on its content written).
Also, it is not an option to write in a buffer and write to file only when final filename is known (because file size is potentially very large: not enough memory).
Any advice on how to do this would be greatly appreciated!
Thanks,
Thomas
It is a missing functionality, which could have been simple, as the entries themselves are not compressed.
The easiest way, requiring a rewrite though, is the zip FileSystem: since java 7 you may use a zip file as a virtual file system: writing, renaming and moving files in them. An example. You copy a file from the normal file system into the zip file system, and later rename the file in the zip.
// Create the zip file:
URI zipURI = URI.create("jar:file:" + fullFilename); // "jar:file:/.../... .zip"
Map<String, Object> env = new HashMap<>();
env.put("create", "true");
FileSystem zipFS = FileSystems.newFileSystem(zipURI, env, null);
// Write to aFile1.txt:
Path pathInZipfile = zipFS.getPath("/aFile1.txt");
BufferedWriter out = Files.newBufferedWriter(pathInZipfile,
StandardCharsets.UTF_8, StandardOpenOption.CREATE_NEW);
out.write("Press any key, except both shift keys\n");
out.close();
// Rename file:
Path pathInZipfile2 = zipFS.getPath("/aFile2.txt");
Files.move(pathInZipfile, pathInZipfile2);
zipFS.close();
In principle you could also keep your old code - without renaming. And use a zip file system just for renaming.
How about saving the contents of aFile1.txt to a temporary file on disk, renaming it, and then creating the zip file afterwards? The last step can then be deleting the file you created on disk.
Related
I have a jar which has a resources folder that contains a folder, let's call it toplevel. toplevel contains another folder, called level1. level1 then contains a list of directories. I'd like to retrieve these directories as java.io.File objects, so that another function can do things with these File objects. With the below example that'd be a List<File> like List{dira, dirb, dirc} How can this be done?
toplevel
---level1
------dir a
------dir b
------dir c
I would suggest extracting matching entries from the jar file and save to a temporary location to get the java.io.File reference.
Option #1:
If you are reading from a file system, use ZipFile to read the file then use ZipFile.getEntry("zip-path") to get the entry and save using Files.copy
See: ZipEntry to File
Option #2:
If you are reading from an input stream source, use ZipInputStream to read the jar file, then iterate, filter and apply action to matching entries. Each matching entry is coupled with a matching ZipInputStream and you can use those input streams to save them to a temporary location, then create the List<File> reference to hand off to another function.
I wrote a quick example in this repo:
https://github.com/nfet/java-zip-demo/tree/main/src/main/resources
The demo essentially just reads the jar file in the resource folder and finds a single matching zip entry (META-INF/license.txt) and saves it to a file.
See Example Implementation in:
https://github.com/nfet/java-zip-demo/blob/7dbdba9c47e0773f959d740d62fbb63949eaca94/src/main/java/com/example/jar/demo/ReadJarFile.java
<script src="https://gist.github.com/nfet/27fce2870b8cd42e3337f6a21b8e9711.js"></script>
Thanks for the help folks but after much ado, found a solution working atop this previous solution https://stackoverflow.com/a/1529707/9486041
to narrow down to the folders and its contents.
JarURLConnection connection = (JarURLConnection) folderURL.openConnection()
JarFile jar = new JarFile(new File(connection.getJarFileURL().toURI()))
Enumeration enumEntries = jar.entries();
while (enumEntries.hasMoreElements()) {
JarEntry file = (JarEntry) enumEntries.nextElement();
if (!file.name.startsWith(path + "/")) {
continue
}
File f = new File(System.getProperty("user.home") + "/tmp" + File.separator + file.getName());
f.getParentFile().mkdirs()
InputStream is = jar.getInputStream(file); // get the input stream
FileOutputStream fos = new FileOutputStream(f);
while (is.available() > 0) { // write contents of 'is' to 'fos'
fos.write(is.read());
}
fos.close();
is.close();
}
jar.close()
I have a very confusing problem and hope that I can get some ideas here.
My problem is very simple, but I didn't find a solution yet.
I want to create a simple ZIP File with ZipEntry's in it. The ZipEntry's are created by a given byte array (saved in a Postgres-DB with Hibernate).
When I put this byte array into my ZipOutputStream.write(..) the ZIP File created is always corrupt. What am I doing wrong?
The ZIP File is transferred to a FTP-Server afterwards.
ByteArrayOutputStream bos = new ByteArrayOutputStream();
final ZipOutputStream zipOut = new ZipOutputStream(bos);
String filename = "test.zip";
for(final Attachment attachment : transportDoc.getAttachments()) {
log.debug("Adding "+attachment.getFileName()+" to ZIP file /tmp/"+filename);
ZipEntry ze = new ZipEntry(attachment.getFileName());
zipOut.putNextEntry(ze);
zipOut.write(attachment.getFileContent());
zipOut.flush();
zipOut.closeEntry();
}
zipOut.close();
org.apache.commons.io.FileUtils.writeByteArrayToFile(new File("/tmp/"+filename), bos.toByteArray());
I am confused, because when I replaced
zipOut.write(attachment.getFileContent()); //This is the byte array from db
with
zipOut.write("Bla bla".getBytes());
it worked!
But the byte array from the DB can't be corrupt, because it can be written to a file with
org.apache.commons.io.FileUtils.writeByteArrayToFile(new File("/tmp/test.png"), attachment.getFileContent());
with no problem. It is a correct file.
I hope you have some ideas left.
Thanks in advance.
EDIT:
I tried to repair the ZIP file offline and then this messages appears:
zip warning: no end of stream entry found: cglhnngplpmhipfg.png
(This png file is the byte-Array-File)
Simple unzip-command output the following:
unzip created.zip
Archive: created.zip
error [created.zip]: missing 2 bytes in zipfile
(attempting to process anyway)
error [created.zip]: attempt to seek before beginning of zipfile
(please check that you have transferred or created the zipfile in the
appropriate BINARY mode and that you have compiled UnZip properly)
(attempting to re-compensate)
replace cglhnngplpmhipfg.png? [y]es, [n]o, [A]ll, [N]one, [r]ename: y
inflating: cglhnngplpmhipfg.png
error: invalid compressed data to inflate
file #2: bad zipfile offset (local header sig): 24709
(attempting to re-compensate)
inflating: created.xml
EDIT 2:
When I write this file to the Filesystem and add this file to the ZIP by an InputStream it doesn't work either! But the File on the Filesystem is ok. I can open the Image with no problem. Its very confusing
File tmpAttachment = new File("/tmp/"+filename+attachment.getFileName());
FileUtils.writeByteArrayToFile(tmpAttachment, attachment.getFileContent());
FileInputStream inTmp = new FileInputStream(tmpAttachment);
int len;
byte[] buffer = new byte[1024];
while ((len = inTmp.read(buffer)) > 0) {
zipOut.write(buffer, 0, len);
}
inTmp.close();
EDIT 3:
This problem only appears when I try to add "complex" files like png or pdf. If I put a txt-file in it, it works.
The problem was NOT in the Zip-Library itself.
It was the transmission to an external FTP Server with wrong mode. (Not binary).
Thanks all for your help.
Try closeEntry() before flush(). Also you can try to explicitly specify the size of the entry using ze.setSize(attachment.getFileContent().length).
This question already has answers here:
How to read file from ZIP using InputStream?
(7 answers)
Closed 1 year ago.
How can I create new File (from java.io) in memory, not on the hard disk?
I am using the Java language. I don't want to save the file on the hard drive.
I'm faced with a bad API (java.util.jar.JarFile). It's expecting File file of String filename. I have no file (only byte[] content) and can create temporary file, but it's not beautiful solution. I need to validate the digest of a signed jar.
byte[] content = getContent();
File tempFile = File.createTempFile("tmp", ".tmp");
FileOutputStream fos = new FileOutputStream(tempFile);
fos.write(archiveContent);
JarFile jarFile = new JarFile(tempFile);
Manifest manifest = jarFile.getManifest();
Any examples of how to achieve getting manifest without creating a temporary file would be appreciated.
How can I create new File (from java.io) in memory , not in the hard disk?
Maybe you are confusing File and Stream:
A File is an abstract representation of file and directory pathnames. Using a File object, you can access the file metadata in a file system, and perform some operations on files on this filesystem, like delete or create the file. But the File class does not provide methods to read and write the file contents.
To read and write from a file, you are using a Stream object, like FileInputStream or FileOutputStream. These streams can be created from a File object and then be used to read from and write to the file.
You can create a stream based on a byte buffer which resides in memory, by using a ByteArrayInputStream and a ByteArrayOutputStream to read from and write to a byte buffer in a similar way you read and write from a file. The byte array contains the "File's" content. You do not need a File object then.
Both the File... and the ByteArray... streams inherit from java.io.OutputStream and java.io.InputStream, respectively, so that you can use the common superclass to hide whether you are reading from a file or from a byte array.
It is not possible to create a java.io.File that holds its content in (Java heap) memory *.
Instead, normally you would use a stream. To write to a stream, in memory, use:
OutputStream out = new ByteArrayOutputStream();
out.write(...);
But unfortunately, a stream can't be used as input for java.util.jar.JarFile, which as you mention can only use a File or a String containing the path to a valid JAR file. I believe using a temporary file like you currently do is the only option, unless you want to use a different API.
If you are okay using a different API, there is conveniently a class in the same package, named JarInputStream you can use. Simply wrap your archiveContent array in a ByteArrayInputStream, to read the contents of the JAR and extract the manifest:
try (JarInputStream stream = new JarInputStream(new ByteArrayInputStream(archiveContent))) {
Manifest manifest = stream.getManifest();
}
*) It's obviously possible to create a full file-system that resides in memory, like a RAM-disk, but that would still be "on disk" (and not in Java heap memory) as far as the Java process is concerned.
You could use an in-memory filesystem, such as Jimfs
Here's a usage example from their readme:
FileSystem fs = Jimfs.newFileSystem(Configuration.unix());
Path foo = fs.getPath("/foo");
Files.createDirectory(foo);
Path hello = foo.resolve("hello.txt"); // /foo/hello.txt
Files.write(hello, ImmutableList.of("hello world"), StandardCharsets.UTF_8);
I think temporary file can be another solution for that.
File tempFile = File.createTempFile(prefix, suffix, null);
FileOutputStream fos = new FileOutputStream(tempFile);
fos.write(byteArray);
There is a an answer about that here.
I occasionally receive .zip files in my app that throw start of central directory not found;
zipfile corrupt. exceptions. These zip files open just fine in my Mac's Finder.
I can fix these files every time from the command line, using zip -FF bad.zip --out good.zip
Can any Java ZIP libraries out there accomplish the same thing?
You probably want to just let Java execute this command, because in strict terms zip is more like a container and it can contain different compression algorithms.
In general investigating and solving problems related to compressed archives with a programmatic approach it's likely to be a tricky and long task.
Try this with your command.
I tried using ZipInputStream and ZipOutputStream. But ZipInputStream always failed at some point when doing: "getNextEntry()". Basically the following lines of code in "getNextEntry()":
...
if ((entry = readLOC()) == null) {
return null;
}
...
returned null after some entries and I could not get further.
But finally I could solve the issue using ZipFile together with ZipOutputStream because ZipFile was reading all zip entries without problem and the solution looks like this:
protected void repairZipFile(String file) throws IOException {
File repairZipFile = new File(file+".repair");
ZipFile zipFile = new ZipFile(file);
Enumeration<? extends ZipEntry> zipFileEntries = zipFile.entries();
InputStream zis;
ZipOutputStream zos = new ZipOutputStream(new FileOutputStream(repairZipFile));
byte[] b = new byte[1024];
while(zipFileEntries.hasMoreElements()){
ZipEntry zipEntry = zipFileEntries.nextElement();
zos.putNextEntry(zipEntry);
zis = zipFile.getInputStream(zipEntry);
int n = zis.read(b);
while(n>=0) {
zos.write(b, 0, n);
n = zis.read(b);
}
zis.close();
zos.closeEntry();
}
zipFile.close();
zos.flush();
zos.close();
Files.move(repairZipFile.toPath(), (new File(file)).toPath(), StandardCopyOption.REPLACE_EXISTING);
}
There are two ways to open ZIP files in Java, using the ZipFile class, or using ZipInputStream.
As far as I remember, ZipFile reads the central directory of a zip file first - it can do this because it uses a RandomAccessFile underneath. However, ZipInputStream uses the in-line entry information, which might be better if the central directory, which I think exists at the end of the file, is missing or corrupt.
So, it might be possible to 'repair' a ZIP file in Java by reading a ZIP file using ZipInputStream, and writing it back out to another file using a ZipOutputStream, copying entry information between them. You might end up getting IO exceptions reading from the last entry of the ZipInputStream if it got truncated, but it might still save the other previous entries from the file.
I'm reading a bunch of files from an FTP. Then I need to unzip those files and write them to a fileshare.
I don't want to write the files first and then read them back and unzip them. I want to do it all in one go. Is that possible?
This is my code
FTPClient fileclient = new FTPClient();
..
ByteArrayOutputStream out = new ByteArrayOutputStream();
fileclient.retrieveFile(filename, out);
??????? //How do I get my out-stream into a File-object?
File file = new File(?);
ZipFile zipFile = new ZipFile(file,ZipFile.OPEN_READ);
Any ideas?
You should use a ZipInputStream wrapped around the InputStream returned from FTPClient's retrieveFileStream(String remote).
You don't need to create the File object.
If you want to save the file you should pipe the stream directly into a ZipOutputStream
ByteArrayOutputStream out = new ByteArrayOutputStream();
ZipOutputStream zos = new ZipOutputStream(out);
// do whatever with your zip file
If, instead, you want to open the just retrieved file work with the ZipInputStream:
new ZipInputStream(fileClient.retrieveFileStream(String remote));
Just read the doc here and here
I think you want:
ZipInputStream zis = new ZipInputStream( new ByteArrayInputStream( out.toByteArray() ) );
Then read your data from the ZipInputStream.
As others have pointed out, for what you are trying to do, you don't need to write the downloaded ZIP "file" to the file system at all.
Having said that, I'd like to point out a misconception in your question, that is also reflected in some of the answers.
In Java, a File object does no really represent a file at all. Rather, it represents a file name or *path". While this name or path often corresponds to an actual file, this doesn't need to be the case.
This may sound a bit like hair-splitting, but consider this scenario:
File dir = new File("/tmp/foo");
boolean isDirectory = dir.isDirectory();
if (isDirectory) {
// spend a long time computing some result
...
// create an output file in 'dir' containing the result
}
Now if instances of the File class represented objects in the file system, then you'd expect the code that creates the output file to succeed (modulo permissions). But in fact, the create could fail because, something deleted the "/tmp/foo", or replaced it with a regular file.
It must be said that some of the methods on the File class do seem to assume that the File object does correspond to a real filesystem entity. Examples are the methods for getting a file's size or timestamps, or for listing the names in a directory. However, in each case, the method is specified to throw an exception if the actual file does not exist or has the wrong type for the operation requested.
Well, you could just create a FileOutputStream and then write the data from that:
FileOutputStream fos = new FileOutputStream(filename);
try {
out.writeTo(fos);
} finally {
fos.close();
}
Then just create the File object:
File file = new File(filename);
You need to understand that a File object doesn't represent any real data on disk - it's just a filename, effectively. The file doesn't even have to exist. If you want to actually write data, that's what FileOutputStream is for.
EDIT: I've just spotted that you didn't want to write the data out first - but that's what you've got to do, if you're going to pass the file to something that expects a genuine file with data in.
If you don't want to do that, you'll have to use a different API which doesn't expect a file to exist... as per Qwerky's answer.
Just change the ByteArrayOutputStream to a FileOutputStream.