Using BufferedWriter.write() when is a file created?
I know from the docs that when the buffer is filled it will flush to file, does this mean that:
every-time the buffer is filled an incomplete file will appear on my file system?
or that the file is only created when the BufferedWriter is closed?
My concern is that I am writing files to a directory using a BufferedWriter and another process is polling the directory for new files and reading them. I do not want an incomplete file to be created and be read by the other process.
Using BufferedWriter.write() when is a file created?
Never. BufferedWriter itself just writes to another Writer. Now if you're using a FileOutputStream or a FileWriter (where the first would probably be wrapped in an OutputStreamWriter) the file is created (or opened for write if it already exists) when you construct the object, i.e. before you've actually written any data.
My concern is that I am writing files to a directory using a BufferedWriter and another process is polling the directory for new files and reading them. I do not want an incomplete file to be created and be read by the other process.
One typical way of handling this is to write to a staging area and then rename the file into the correct place, which is usually an atomic operation. Or even write the file into the correct directory, but with a file extension which the polling process won't spot - and then rename the file to the final filename afterwards.
BufferedWriter doesn't create a file as Jon Skeet said. And you cannot guarantee that another process won't read an incomplete file when it is being written to disk. But there are two things you can do:
Lock the file so that the other process cannot read it before writing is complete. There are several questions concerning file locking in Java on this site (search for "[java] lock file").
Create the file with another filename (ie. use an extension that is not being looked for by the other process) and rename it when writing is finished.
Related
I have some data stored in file. I have to read some data from file, do something with it and then write it to new file, and then again read - calculate - write, and so on.
My problem is, that I have method for reading bytes from file and every time this method is called I open file, read and then close. Same thing at writing.
I think because of this my app is very slow, because opening and closing files is taking some time for sure.
For reading I'm using RandomAccessFile class, and FileWritter for writting. Is there any way, that both of the files will be opened all the time from first reading and writing and just closed at the end?
It won't be opening and closing the file that causes slowness. However reading and writing files can be slow, especially if writing to the sd card. Just opening it should be pretty quick.
Also, for writing make sure you use a buffered writer in there somewhere. It will greatly increase your speed if you aren't just writing the whole file as a big block. If you're reading in the entire file you should use a buffered reader as well.
Is it possible to efficiently append a line into a zip or gzip file?
I'm storing equity market data directly into the file system and I have around 40 different files which are being updated every 5ms.
Whats the best way of doing this?
Use a database, not a zip file.
Database is recommended.
If you really want to use plain text file, put them directly on the file system (and if you are using Linux, choose a proper file system for it).
If you do want to use plain text file and put the text files in a zip file, check zip file system below:
java.nio.file.FileSystems:
http://docs.oracle.com/javase/7/docs/technotes/guides/io/fsp/zipfilesystemprovider.html
The zip file system provider introduced in the Java SE 7 release is an implementation of a custom file system provider. The zip file system provider treats a zip or JAR file as a file system and provides the ability to manipulate the contents of the file. The zip file system provider creates multiple file systems — one file system for each zip or JAR file.
TrueZip
http://truezip.java.net/
TrueZIP is a Java based plug-in framework for virtual file systems (VFS) which provides transparent access to archive files as if they were just plain directories
And remember: use memory to cache, reduce disk operations, and make the writing non-blocking.
Complete cycle of editing zip file (i mean read-modify-close) will produce too much overhead. I think it is better to accumulate changes in memory and modify target file at some reasonable rate (i.e. every 5 seconds or even more).
One approach could be like compressing data sent over a socket and flushing compressed blocks to disk from time to time.
You can use a ZipOutputStream's write method to write at a given offset.
String filepath = new String("/tmp/updated.txt")
FileOutputStream fout = new FileOutputStream("/tmp/example.zip");
ZipOutputStream zout = new ZipOutputStream(fout);
byte[] file = IOUtils.toByteArray(mWriter.toString());
short yourOffset = 0;
ZipEntry ze = new ZipEntry(filepath);
try {
zout.putNextEntry(ze);
zout.write(file, yourOffset, file.length);
zout.closeEntry();
} catch(Exception e) {
e.printStackTrace();
}
If you convert your file to a byte array using Apache commons IOUtils (import org.apache.commons.io.IOUtils) you can rewrite and replace the zip entry by calling write at your the offset where the line you want to edit begins. In this case it writes the entire file, from 0 to file.length. You can replace the file in the zip by creating a ZipEntry with a path to the updated file on your drive.
I was wondering if there was any way to create a FileInputStream object from just a file object without creating an actual file on the file system? What I am attempting to do is create a file object with some information, and then upload that file somewhere else. I have no need for it to be on the local file system. I know that I could just create a temp folder and then delete it afterwards, but was wondering if it was possible to not do it that way?
What I am attempting to do is create a file object with some
information, and then upload that file somewhere else
In that case you should not work with any file-related classes at all. Instead, crate a byte array, which you can tread as an InputStream via ByteArrayInputStream.
You are probably looking for a ByteArrayInputStream or something similar.
A file input stream reads from a file on disk, that is its purpose. By the way, a File object in Java does not really represent a file, but rather the path pointing to a (potential) file on disk.
Try creating a memory stream, your file is stored in the memory instead of the file system
Using iText, I am wanting to open a PDF file, add some more pages with text to it, and then close it. I have found some questions like this on here, but all require creating a new PDF file. Is there any way to read in the pdf file and modify it and then overwrite the original?
Of course you can create a new pdf file, and afterwards overwriting the old file with the new one.
Commons Apache File Util
forceDelete(oldPdf)
moveFile(newPdf, oldPdf)
Of course, you can always overwrite a file (if it is not locked by the OS) by writing to the whole content to the FileOutputStream. You cannot partially write to part of a file unless it is to append data at the end of file. This is limited by the operating system itself so there is nothing you can do.
I have a question concerning java and file input/output.
for an specific task, i have to transfer a file (excel to be precise) while it's opened.
imagine following scenario:
An excel file is opened and used by one user. From time to time the file is saved manually by the user. Now i want to write a java programm which reads the file and transfer it over an socket every 30 sec. No problem so far. My question: what happens if the user saves the document in the exact moment my program wants to read the file. Could this cause any troubles?
Don't know if it matters, but im using an BufferedInputStream to read the file.
My question: what happens if the user saves the document in the exact moment my program wants to read the file. Could this cause any troubles?
Yes.
One or more of the following things could happen depending on your platform, and the way that the Excel file is saved.
If Excel uses locking, then either Excel or the program trying to read the file could get an error saying that the file is in use.
If Excel does a rewrite in place and doesn't lock the file, then the program trying to read the file could see a truncated Excel file.
If Excel writes a new file and renames it, the program trying to read the file could see a state
where the file apparently does not exist.
It could work.
In short, the program doing the reading needs to very defensive ...
Don't know if it matters, but im using an BufferedInputStream to read the file.
That's irrelevant I think.
AFaik, the behaviour will depend on your underlying filesystem / operating system. A unix system typically keep an "un-named" copy of the file being read and starts creating a new file for the "being written" new copy, using inode trickery. An old Windows system would likely reply that the file cannot be written to because it is locked. I don't know about modern Windows systems.
what you can do i think is to alway check the state of the file before you do anything about it. like what have been said in some earlier posts, it all depends on the underlying platform, and you should employ a lot of defensive programming...