Pretty sure this should be really easy, but I can't write to a file. No I/O exception is thrown nothing. I was having a similar problem reading earlier and I tried a hundred different ways until
DataInputStream dis = new DataInputStream(reading.class.getResourceAsStream("hello.txt");
BufferedReader input = new BufferedReader(new InputStreamreader(dis));
this worked! and I could use scanners and such to read from this point.
FileReader, making File file = new File("hello.txt") whatever, none of that worked. I couldn't get any of it to even throw an error when it was an incorrect file name.
Now I have the same problem except for writing to a file but there's no equivilant to
reading.class.getResourceAsStream("hello.txt"); that makes an /output/ stream.
Does anyone know how to get the "ResourceAsStream" but as an output stream, /or/ does anyone know what my problem might be?
I know a lot of people on this website have reading/writing issues but none of the posts helped me.
note - yes I was closing, yes I was flushing, yes I had code that wrote to the file.
GetResourceAsStream is meant to read resources (e.g. property files) that were distributed and packages along with the code. There's no guarantee they're in writable mode, e.g. both code and resources could be distributed as a jar, or jar-inside-a-WAR-inside-an-EAR...
See here Write to a file stream returned from getResourceAsStream() for additional discussion and a workaround suggestion, though it's not very recommended IMHO. I think the reasonable practice is to distinguish between (a) your immutable code distribution (b) data editable at runtime ... the latter could reside on a different machine, have different policies for secuirty/replicatoin/backup, etc.
Related
This is my understanding regarding reading a file using BufferedReader in java. Please correct me if I am wrong somewhere...
Recently I had a requirement where we are required to read a file multiple times.
The usual way which I use is setting a mark() and doing a reset. But the input parameters to
a mark is an integer and it cannot accept a long number. Is there a way in which we can read the file, a large number of times.
In c++ we can do a seekg on the fstream and read the contents once again irrespective of the number of times we want to do so. Is there anything in java which is of this nature.
Just close the file and read it again.
But review your requirement. Why can't you process it in one pass?
Not much of a good answer but if you want to do random reading and writing then you can use Channels in java.nio package.
BufferedReader is for reading a file when you logically see it as a series of records and records are generally accessed sequentially.
Channels allow you to view your file as a series of blocks. Blocks are meant to be read randomly. :)
Using subclass of channel, FileChannel, you can read what you want from wherever you want. You need to specify two things:
Where to read from.
How much to read.
It has a read(dst,pstn) where dst is a ByteBuffer and pstn is a long position.
Don't worry that it is abstract because you use it via Files.newByteChannel() which does all the voodoo needed to make it work :)
Is it ever favorable to create a FileInputStream, like this:
InputStream fileInputStream = new FileInputStream(resource.getFile());
instead of using the InputStream created by the resource, like this:
InputStream resourceInputStream = resource.getInputStream();
The resource is an org.springframework.core.io.Resource.
A resource does not always come from a File. It may come from a network resource or be dynamically generated by the class loader. If you do that you may be fine 99% of the time, but you could have trouble in specific unexpected situations.
As a general rule it's better not to make assumptions about the concrete implementation of anything.
By using resource.getFile() your making the assumption that the resource comes from a file.
If all you need is the input stream, then there is no reason not to use it. The point of getFile is getting access to the file in ways other than opening a FileInputStream on it, such as reading attributes, moving, deleting, opening it through NIO instead of classic IO, etc.
I searched and looked at multiple questions like this, but my question is really different than anything I found. I've looked at Java Docs.
How do I get the equivalent of this c file open:
stream1 = fopen (out_file, "r+b");
Once I've done a partial read from the file, the first write makes the next read return EOF no matter how many bytes were in the file.
Essentially I want a file I/O stream that doesn't do that. The whole purpose of what I'm trying to do is to replace the bytes in an existing file in the current file. I don't want to do it in a copy or make a copy before I do the Read->Write.
You can use a RandomAccessFile.
As Perception mentions, you can use a RandomAccessFile. Also, in some situations, a FileChannel may work better. I've used these to handle binary file data with great success.
EDIT: you can get a FileChannel from the RandomAccessFile object using getChannel.
We are validating XML files and depending on the result of the validation we have to move the file into a different folder.
When the XML is valid the validator returns a value and we can move the file without a problem. Same thing happens when the XML is not valid according to the schema.
If however the XML is not well formed the validator throws an exception and when we try to move the file, it fails. We believe there is still a handle in the memory somewhere that keeps hold of the file. We tried putting System.gc() before moving the file and that sorted the problem but we can't have System.gc() as a solution.
The code looks like this. We have a File object from which we create a StreamSource. The StreamSource is then passed to the validator. When the XML is not well formed it throws a SAXException. In the exception handling we use the .renameTo() method to move the file.
sc = new StreamSource(xmlFile);
validator.validate(sc);
In the catch we tried
validator.reset();
validator=null;
sc=null;
but still .renameTo() is not able to move the file. If we put System.gc() in the catch, the move will succeed.
Can someone enlight me how to sort this without System.gc()?
We use JAXP and saxon-9.1.0.8 as the parser.
Many thanks
Try creating a FileInputStream and passing that into StreamSource then close the FileInputStream when you're done. By passing in a File you have lost control of how/when to close the file handle.
When you set sc = null, you are indicating to the garbage collector that the StreamSource file is no longer being used, and that it can be collected. Streams close themselves in their destroy() method, so if they are garbage collected, they will be closed, and therefore can be moved on a Windows system (you will not have this problem on a Unix system).
To solve the problem without manually invoking the GC, simply call sc.getInputStream().close() before sc = null. This is good practice anyway.
A common pattern is to do a try .. finally block around any file handle usage, eg.
try {
sc = new StreamSource(xmlFile);
// check stuff
} finally {
sc.getInputStream().close();
}
// move to the appropriate place
In Java 7, you can instead use the new try with resources block.
Try sc.getInputStream().close() in the catch
All the three answers already given are right : you must close the underlying stream, either with a direct call to StramSource, or getting getting the stream and closing it, or creating the stream yourself and closing it.
However, I've already seen this happening, under windows, since at least three years : even if you close the stream, really every stream, if you try to move or delete the file, it will throw exception .. unless ... you explicitly call System.gc().
However, since System.gc() is not mandatory for a JVM to actually execute a round of garbage collection, and since even if it was the JVM is not mandated to remove all possible garbage object, you have no real way of being sure that the file can be deleted "now".
I don't have a clear explanation, I can only imagine that probably the windows implementation of java.io somehow caches the file handle and does not close it, until the handle gets garbage collected.
It has been reported, but I haven't confirmed it, that java.nio is not subject to this behavior, cause it has more low level control on file descriptors.
A solution I've used in the past, but which is quite a hack, was to :
Put files to delete on a "list"
Have a background thread check that list periodically, calla System.gc and try to delete those files.
Remove from the list the files you managed to delete, and keep there those that are not yet ready to.
Usually the "lag" is in the order of a few milliseconds, with some exceptions of files surviving a bit more.
It could be a good idea to also call deleteOnExit on those files, so that if the JVM terminates before your thread finished cleaning some files, the JVM will try to delete them. However, deleteOnExit had it's own bug at the time, preventing exactly the removal of the file, so I didn't. Maybe today it's resolved and you can trust deleteOnExit.
This is the JRE bug that i find most annoying and stupid, and cannot believe it is still in existence, but unfortunately I hit it just a month ago on windows Vista with latest JRE installed.
Pretty old, but some people may still find this question.
I was using Oracle Java 1.8.0_77.
The problem occurs on Windows, not on Linux.
The StreamSource instanciated with a File seems to automatically allocate and release the file resource when processed by a validator or transformer. (getInputStream() returns null)
On Windows moving a file into the place of the source file (deleting the source file) after the processing is not possible.
Solution/Workaround: Move the file using
Files.move(from.toPath(), to.toPath(), REPLACE_EXISTING, ATOMIC_MOVE);
The use of ATOMIC_MOVE here is the critical point. Whatever the reason ist, it has something to do with the annoying behavior of Windows locking files.
So, here is the situation:
I have to read big .gz archives (GBs) and kind of "index" them to later on be able to retrieve specific pieces using random access.
In other words, I wish to read the archive line by line, and be able to get the specific location in the file for any such line. (so that I can jump directly to these specific locations upon request). (PS: ...and it's UTF-8 so we cannot assume 1 byte == 1 char.)
So, basically, what I just need is a BufferedReader which keeps track of its location in the file. However, this doesn't seem to exist.
Is there anything available or do I have to roll my own?
A few additional comments:
I cannot use BufferedReader directly since the file location corresponds to what has been buffered so far. In other words, a multiple of the internal buffer size instead of the line location.
I cannot use InputStreamReader directly for performance reasons. Unbuffered would be way to slow, and, btw, lacks convenience methods to read lines.
I cannot use RandomAccessFile since 1. it's zipped, and 2. RandomAccessFile uses "modified" UTF-8
I guess the best would be use a kind of of buffered reader keeping track of file location and buffer offset ...but this sounds quite cumbersome. But maybe I missed something. Perhaps there is already something existing to do that, to read files line by lines and keep track of location (even if zipped).
Thanks for tips,
Arnaud
I think jzran could be pretty much what you're looking for:
It's a Java library based on the
zran.c sample from zlib.
You can preprocess a large gzip
archive, producing an "index" that can
be used for random read access.
You can balance between index size and
access speed.
What you are looking for is called mark(), markSupported() and skip().
This methods are declared both in InputStream and Reader, so you are welcome to use them.
GZIP compression does not support seeking. Previous data blocks are needed to build compression tables...