I want to create a zip archive in Java where each contained file is produced by serializing some objects. I have a problem with correctly closing the streams.
The code looks like this:
try (OutputStream os = new FileOutputStream(file);
ZipOutputStream zos = new ZipOutputStream(os);) {
ZipEntry ze;
ObjectOutputStream oos;
ze = new ZipEntry("file1");
zos.putNextEntry(ze); // start first file in zip archive
oos = new ObjectOutputStream(zos);
oos.writeObject(obj1a);
oos.writeObject(obj1b);
// I want to close oos here without closing zos
zos.closeEntry(); // end first file in zip archive
ze = new ZipEntry("file2");
zos.putNextEntry(ze); // start second file in zip archive
oos = new ObjectOutputStream(zos);
oos.writeObject(obj2a);
oos.writeObject(obj2b);
// And here again
zos.closeEntry(); // end second file in zip archive
}
I know of course that I should close each stream after finishing using it, so I should close the ObjectOutputStreams in the indicated positions. However, closing the ObjectOutputStreams would also close the ZipOutputStream that I still need.
I do not want to omit the call to ObjectOutputStream.close() because I do not want to rely on the fact that it currently does not more than flush() and reset().
I also cannot use a single ObjectOutputStream instance because then I miss the stream header that is written by the constructor (each single file in the zip archive would not be a full object serialization file, and I could not de-serialize them independently).
The same problem occurs when reading the file again.
The only way I see would be to wrap the ZipOutputStream in some kind of "CloseProtectionOutputStream" that would forward all methods except close() before giving it to the ObjectOutputStream. However, this seems rather hacky and I wonder if I missed a nicer solution in the API.
If your OutputStream wrapper throws an exception when closed more than once, it is not a hack. You can create a wrapper for each zip entry.
From an architectural point of view, I think the ObjectOutputStream author should have provided an option to disable close() cascading. You are just workarounding his lacking API.
In this case, and for all the reasons you mentioned, I would simply not pipe my ObjectOutputStream to the ZipOutputStream. Instead, serialize to a byte[] and then write that straight into the ZipOutputStream. This way, you are free to close the ObjectOutputStream and each byte[] you produce will have the proper header from the serializer. One down side is you wind up with a byte[] in memory that you didn't have before but if you get rid of it right away, assuming we're not talking about millions of objects, the garbage collector shouldn't have a hard time cleaning up.
Just my two cents...
It at least sounds less hacky than a stream subclass that changes the close() behavior.
If you're intending to throw the ObjectOutputStream away anyway, then it should be sufficient to call flush() rather than close(), but as you say in the question the safest approach is probably to use a wrapper around the underlying ZipOutputStream that blocks the close() call. Apache commons-io has CloseShieldOutputStream for this purpose.
Related
I'm uploading zip of excel files as multipart file, but when I create Workbook object of first file, the stream gets closed and I'm not able to read next files.
its working with single file in zip but not with multiple files.
can anybody help? TIA.
try {
ZipInputStream zis = new ZipInputStream(multipartFile.getInputStream());
ZipEntry zipEntry;
while((zipEntry = zis.getNextEntry()) != null) {
XSSFWorkbook workbook = new XSSFWorkbook(zis);
readWorkbook(workbook);
}
zis.close();
} catch (Exception e) {
LOG.error(e);
}
only option is to wrap it so you can intercept close() and prevent it from closing the zip. Something like:
var wrapper = new FilterInputStream(zis) {
#Override public void close() {}
}
Then pass wrapper and not zis to new XSSFWorkbook.
NB: WARNING - your code is severely hampered, essentially buggy. You need to ensure you close that ZipInputStream at the very end, and you're not doing that now. Your exception handling is really bad. You must end all exception blocks with a hard exit, just logging it isn't good enough. throw new RuntimeException("uncaught", e); is the fallback option (many IDEs ship with e.printStackTrace() as default. This is a known extremely stupid default, update your IDE to fix this. Use try-with-resources as well to ensure closing always happens (just calling close() isn't good enough; that close() call you do have in your snippet won't be invoked if exceptions occur.
When reading from a Zipfile, you have an InputStream for the archive. Then you traverse the different entries in there and for each of them you again have an InputStream to read from. Make sure you do not close the entries' InputStreams as that event will close the archive stream.
In your case it may be that the constructor for XSSFWorkbook or the readWorkbook method is closing the stream.
I am trying to open a file for reading or create the file if it was not there.
I use this code:
String location = "/test1/test2/test3/";
new File(location).mkdirs();
location += "fileName.properties";
Path confDir = Paths.get(location);
InputStream in = Files.newInputStream(confDir, StandardOpenOption.CREATE);
in.close();
And I get java.nio.file.NoSuchFileException
Considering that I am using StandardOpenOption.CREATE option, the file should be created if it is not there.
Any idea why I am getting this exception?
It seems that you want one of two quite separate things to happen:
If the file exists, read it; or
If the file does not exist, create it.
The two things are mutually exclusive but you seem to have confusingly merged them. If the file did not exist and you've just created it, there's no point in reading it. So keep the two things separate:
Path confDir = Paths.get("/test1/test2/test3");
Files.createDirectories(confDir);
Path confFile = confDir.resolve("filename.properties");
if (Files.exists(confFile))
try (InputStream in = Files.newInputStream(confFile)) {
// Use the InputStream...
}
else
Files.createFile(confFile);
Notice also that it's better to use "try-with-resources" instead of manually closing the InputStream.
Accordingly to the JavaDocs you should have used newOutputStream() method instead, and then you will create the file:
OutputStream out = Files.newOutputStream(confDir, StandardOpenOption.CREATE);
out.close();
JavaDocs:
// Opens a file, returning an input stream to read from the file.
static InputStream newInputStream(Path path, OpenOption... options)
// Opens or creates a file, returning an output stream that
// may be used to write bytes to the file.
static OutputStream newOutputStream(Path path, OpenOption... options)
The explanation is that OpenOption constants usage relies on wether you are going to use it within a write(output) stream or a read(input) stream. This explains why OpenOption.CREATE only works deliberatery with the OutputStream but not with InputStream.
NOTE: I agree with #EJP, you should take a look to Oracle's tutorials to create files properly.
I think you intended to create an OutputStream (for writing to) instead of an InputStream (which is for reading)
Another handy way of creating an empty file is using apache-commons FileUtils like this
FileUtils.touch(new File("/test1/test2/test3/fileName.properties"));
My app needs to do the following:
Open a FileInputStream, and obtain the underlying FileDescriptor (via getFd())
Create new FileInputStream objects based on the above FileDescriptor
So far, I only needed one FileDescriptor, so I used to close it by calling close() on the original stream (i.e. on the stream which getFd() I called). I use it because some Android API methods have such a parameter.
Now that I will have more FileInputStream objects at the same time, when will the FileDescriptor be closed? (My guess: when all FileInputStream objects are closed?)
I belive you are right. A small test shows that the FileDescriptor becomes invalid after its FileInputStream is closed. Note that, in case of more than one FileInputStream for the same FileDescriptor, the FileDescriptor becomes invalid as soon as its first FileInputStream is closed, i.e. it does not matter if you close first fis1 and then fis2 or the other way around:
FileInputStream fis1 = new FileInputStream("/tmp/adb.log");
FileDescriptor fd = fis1.getFD();
FileInputStream fis2 = new FileInputStream(fd);
System.out.println(fd.valid());
fis1.close();
System.out.println(fd.valid());
fis2.close();
System.out.println(fd.valid());
Output is:
true
false
false
Do not forget to close the stream in a finally block, to make sure you close it also in case of an I/O (read/write) error.
Android FileInputStream has a concept of fd ownership.
isFdOwner is true when you create the stream from File.
Is it false when you create it from existing FileDescriptor.
(actually there is a hidden constructor which allows you to specify
whether the new stream is also an owner or not,
but 'false' works great in almost all situations)
If you close FileInputStream that was opened with File - it will be closed.
Closing streams that were opened with fd - will do nothing.
I now have this problem. I want to write a excel file hold in this XSSFWorkbook (workbook) obj into a zip file eg(example.zip while contain this example.xlsx file) to a remote server.
I have tried following but not working, it created a folder with some odd files in the zip file
XSSFWorkbook workbook = new XSSFWorkbook();
//add some data
Zipoutputstream zipstream=new Zipoutputstream(//destination outputstream);
workbook.write(zipstream);
So do anyone knows what's the right way to do this? Thanks in advance
ps workbook.write(fileoutputstream) works but it only write to local disk as a flat file eg test.xlsx instead of inside a zip as I need.
Passing a a ZipOutputStream to XSSFWorkbook.write will result in the stream being hijacked and closed by the workbook. This is because an XSSFWorkbook writes a .xlsx which is itself a zip archive of xml and other files (you can unzip any .xslx to see what's in there).
If you're able to fit the excel file in memory, I've found this to work well:
ZipOutputStream zos = new ZipOutputStream(//destination outputstream);
zos.putNextEntry(new ZipEntry("AnExcelFile.xlsx"));
ByteArrayOutputStream bos = new ByteArrayOutputStream();
workbook.write(bos);
bos.writeTo(zos);
zos.closeEntry();
// Add other entries as needed
zos.close();
Calling close on ByteArrayOutputStream has no effect and can still be written to zos.
You are missing some necessary calls on your ZipOutputStream. You will need to create a ZipEntry for your spreadsheet file, then write it out. You'll need something like
zipstream.putNextEntry(new ZipEntry("example.xlsx"));
Then you should be able to call
workbook.write(zipstream);
But after that you'll need to close the entry before closing the stream.
zipstream.closeEntry();
Please see "Write And Read .Zip File From Java" for details on how to use Java's ZipOutputStream.
Also, be aware that .xlsx files are already compressed zip files, so placing it in a .zip file may not compress it very much.
A colleague of mine, M. Bunshaft, suggested a solution similar to that of Klugscheißer but that does not require the use of a ByteArrayOutputStream, and hence can accommodate larger output.
The idea is to subclass ZipOutputStream, overriding the close() method so it will not do a close.
public class UncloseableZipOutputStream extends ZipOutputStream
{
OutputStream os;
public UncloseableZipOutputStream( OutputStream os )
{
super(os);
}
#Override
/** just flush but do not close */
public void close() throws IOException
{
flush();
}
public void reallyClose() throws IOException
{
super.close();
}
}
Then, simply use it the way you would use the ZipOutputStream.
UncloseableZipOutputStream zos = new UncloseableZipOutputStream(//destination outputstream);
zos.putNextEntry(new ZipEntry("AnExcelFile.xlsx"));
workbook.write(zos);
zos.closeEntry(); // now this will not cause a close of the stream
// Add other entries as needed
zos.reallyClose();
I'm reading a bunch of files from an FTP. Then I need to unzip those files and write them to a fileshare.
I don't want to write the files first and then read them back and unzip them. I want to do it all in one go. Is that possible?
This is my code
FTPClient fileclient = new FTPClient();
..
ByteArrayOutputStream out = new ByteArrayOutputStream();
fileclient.retrieveFile(filename, out);
??????? //How do I get my out-stream into a File-object?
File file = new File(?);
ZipFile zipFile = new ZipFile(file,ZipFile.OPEN_READ);
Any ideas?
You should use a ZipInputStream wrapped around the InputStream returned from FTPClient's retrieveFileStream(String remote).
You don't need to create the File object.
If you want to save the file you should pipe the stream directly into a ZipOutputStream
ByteArrayOutputStream out = new ByteArrayOutputStream();
ZipOutputStream zos = new ZipOutputStream(out);
// do whatever with your zip file
If, instead, you want to open the just retrieved file work with the ZipInputStream:
new ZipInputStream(fileClient.retrieveFileStream(String remote));
Just read the doc here and here
I think you want:
ZipInputStream zis = new ZipInputStream( new ByteArrayInputStream( out.toByteArray() ) );
Then read your data from the ZipInputStream.
As others have pointed out, for what you are trying to do, you don't need to write the downloaded ZIP "file" to the file system at all.
Having said that, I'd like to point out a misconception in your question, that is also reflected in some of the answers.
In Java, a File object does no really represent a file at all. Rather, it represents a file name or *path". While this name or path often corresponds to an actual file, this doesn't need to be the case.
This may sound a bit like hair-splitting, but consider this scenario:
File dir = new File("/tmp/foo");
boolean isDirectory = dir.isDirectory();
if (isDirectory) {
// spend a long time computing some result
...
// create an output file in 'dir' containing the result
}
Now if instances of the File class represented objects in the file system, then you'd expect the code that creates the output file to succeed (modulo permissions). But in fact, the create could fail because, something deleted the "/tmp/foo", or replaced it with a regular file.
It must be said that some of the methods on the File class do seem to assume that the File object does correspond to a real filesystem entity. Examples are the methods for getting a file's size or timestamps, or for listing the names in a directory. However, in each case, the method is specified to throw an exception if the actual file does not exist or has the wrong type for the operation requested.
Well, you could just create a FileOutputStream and then write the data from that:
FileOutputStream fos = new FileOutputStream(filename);
try {
out.writeTo(fos);
} finally {
fos.close();
}
Then just create the File object:
File file = new File(filename);
You need to understand that a File object doesn't represent any real data on disk - it's just a filename, effectively. The file doesn't even have to exist. If you want to actually write data, that's what FileOutputStream is for.
EDIT: I've just spotted that you didn't want to write the data out first - but that's what you've got to do, if you're going to pass the file to something that expects a genuine file with data in.
If you don't want to do that, you'll have to use a different API which doesn't expect a file to exist... as per Qwerky's answer.
Just change the ByteArrayOutputStream to a FileOutputStream.