The InputStream is passed as a parameter from somewhere, where it will be further processed and then closed. So I don't want to close the InputStream here. Consider the following code:
void readInputStream(final InputStream inputStream) {
final BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(inputStream));
String line;
while ((line = bufferedReader.readLine() != null) {
// do my thing
}
}
If I close the BufferedReader and/or the InputStreamReader, then the underlying InputStream will be closed as well, according to another Stackoverflow post.
My question: Do Readers need to be closed, even if the underlying InputStream is closed somewhere else? Can I get a memory leak by not closing Readers?
Do readers need to be closed, even if the underlying InputStream is closed somewhere else?
No they absolutely don't need to be in that scenario. But it is generally a good idea to close them anyway.
Can I get a memory leak by not closing readers?
No, there is no memory leak, assuming that the Reader itself becomes unreachable once you have finished with it. And besides, a Reader doesn't typically use a lot of memory.
The more important question is whether you can get a resource leak by not closing a Reader. The answer is ... it depends.
If you can guarantee that the underlying InputStream will always be closed somewhere else in the application, then that takes care of possible memory leaks.
If you can't guarantee that, then there is a risk of a resource leak. The underlying OS level file descriptors are a limited resource in (for example) Linux. If a JVM doesn't close them, they can run out and a certain system calls will start to fail unexpectedly.
But if you do close the Reader then the underlying InputStream will be closed.
Calling close() more than once on a InputStream is harmless, and costs almost nothing.
The only case where you shouldn't close the Reader is when it would be wrong to close the underlying InputStream. For example, if you close a SocketInputStream the rest of the application may not be able to reestablish the network connection. Likewise, the InputStream associated with System.in usually cannot be reopened.
In that case, it is actually safe to allow the Reader you create in your method to be garbage collected. Unlike InputStream, a typical Reader class doesn't override Object::finalize() to close its source of data.
#Pshemo brought up an important point about system design.
If you are accepting an InputStream as an argument, then it may be wrong to wrap it with a local Reader ... especially a BufferedReader. A BufferedReader is liable to read-ahead on the stream. If the stream is going to be used by the caller after your method returns, then any data that has been read into the buffer but not consumed by this method is liable to be lost.
A better idea would be for the caller to pass a Reader. Alternatively, this method should be documented as taking ownership of the InputStream. And in that case, it should always close() it.
Yes, Readers need to be closed. Use a proxy, e.g. CloseShieldInputStream, to prevent the passed parameter from being closed.
void readInputStream(InputStream inputStream) throws IOException{
try (var bufferedReader = new BufferedReader(new InputStreamReader(
new CloseShieldInputStream(inputStream)))) {
String line;
while ((line = bufferedReader.readLine()) != null) {
// do my thing
}
}
}
JIC: Similar to the input shield, Apache Commons I/O also provides an output shield to solve the similar problem with closing a wrapping output stream, - CloseShieldOutputStream
For more detailed considerations, please refer to the original answer. Credits to #stephen-c
Related
I am using ProcessBuilder to input and receive information from a C++ program, using Java. After starting the process once, I would like to be able to input new strings, and receive their output, without having to restart the entire process. This is the approach I have taken thus far:
public void getData(String sentence) throws InterruptedException, IOException{
InputStream stdout = process.getInputStream();
InputStreamReader isr = new InputStreamReader(stdout);
OutputStream stdin = process.getOutputStream();
OutputStreamWriter osr = new OutputStreamWriter(stdin);
BufferedWriter writer = new BufferedWriter(osr);
BufferedReader reader = new BufferedReader(isr);
writer.write(sentence);
writer.close();
String ch = reader.readLine();
preprocessed="";
while (ch!=null){
preprocessed = preprocessed+"~"+ch;
ch = reader.readLine();
}
reader.close();
}
Each time I want to send an input to the running process, I call this method. However, there is an issue: the first time I send an input, it is fine, and the output is received perfectly. However, the second time I call it, I receive the error
java.io.IOException: Stream closed
which is unexpected, as everything is theoretically recreated when the method is called again. Moreover, removing the line the closes the BufferedWriter results in the code halting at the following line, as if the BufferedReader is waiting for the BufferedWriter to be closed.
One final thing - even when I create a NEW BufferedWriter and instruct the method to use that when called for the second time, I get the same exception, which I do not understand at all.
Is there any way this can be resolved?
Thanks a lot!
Your unexpected IOException happens because when Readers and Writers are closed, they close their underlying streams in turn.
When you call your method the first time, everything appears to work. But you close the writer, which closes the process output stream, which closes stdin from the perspective of the process. Not sure what your C++ binary looks like, but probably it just exits happily when it's done with all its input.
So subsequent calls to your method don't work.
There's a separate but similar issue on the Reader side. You call readLine() until it returns null, meaning the Reader has felt the end of the stream. But this only happens when the process is completely done with its stdout.
You need some way of identifying when you're done processing a unit of work (whatever you mean by "sentence") without waiting for the whole entire stream to end. The stream has no concept of the logical pause between outputs. It's just a continuous stream. Reader and Writer are just a thin veneer to buffer between bytes and characters but basically work the same as streams.
Maybe the outputs could have delimiters. Or you could send the length of each chunk of output before actually sending the output and distinguish outputs that way. Or maybe you know in advance how long each response will be?
You only get one shot through streams. So they will have to outlive this method. You can't be opening and closing streams if you want to avoid restarting your process every time. (There are other ways for processes to communicate, e.g. sockets, but that's probably out of scope.)
On an orthogonal note, appending to a StringBuilder is generally more efficient than a big loop of string concatenations when you're accumulating your output.
You might also have some thread check process.exitValue() or otherwise make sure the process is working as intended.
Don't keep trying to create and close your Streams, because once you close it, it's closed for good. Create them once, then in your getData(...) method use the existing Streams. Only close your Streams or their wrapping classes when you're fully done with them.
Note that you should open and close the Streams in the same method, and thus may need additional methods or classes to help you process the Streams. Consider creating a Runnable class for this and then reading from the Streams in another Thread. Also don't ignore the error stream, as that may be sending key information that you will need to fully understand what's going on here.
Some tool I have uses Writer as output. I want to write to System.out, so I created an OutputStreamWriter for System.out. My problem is that I want to do other things with the standard output of my program after I completed this task. I did not find any means to detach the writer.
Is there any common Writer implementation that can do that? Should I write my own Writer? Should I call flush() on my OutputStreamWriter and then leak it?
You can override close and flush instead:
PrintWriter w = new PrintWriter(System.out) {
#Override
public void close() {
flush();
}
};
Yes, just .flush() it and let it be. It doesn't mean you'd "leak" it: the Writer object will get garbage collected, and the underlying stream is still in use. Nothing wrong with that.
Hi I need a answer for necessity of flush in I/O streams in java.since in my program with flush and without flush the output is same.ie,every thing is written in to the destination file.then why i need flush?will file input stream consumes buffer memory?
the below is my simple sample program
file = new File("c:/newfile.txt");
fop = new FileOutputStream("c:/newfile.txt");
// if file doesnt exists, then create it
if (!file.exists()) {
file.createNewFile();
}
// get the content in bytes
byte[] contentInBytes = content.getBytes();
fop.write(contentInBytes);
fop.flush();
fop.close();
even when i command flush and close it can write the contents in to the file properly..?then y we need flush?and will file outputstream consumes memory?
Close calls flush on the stream, so flush is not needed if you want to close the stream.
Flush is useful if you want to make sure that the data is saved, without closing a stream, e.g. when sending messages over the Internet, or writing to the console. You may notice, that if you write to console with system.out.print(), then the output is not displayed, until you call flush, or until there is a new line in the text (in which case Java will call flush for you).
See more on this question
In fact, FileOutputStream is not buffered, so the data is directly written to the file.
The abstract OutputStream defines flush (an empty method) to accomodate also the needs of buffered streams, so FileOutputStream inherits it.
If you are not certain of the underlying implementation, it is generally good practice to flush the streams before closing them.
Also, in your code there is a little error:
file = new File("c:/newfile.txt");
fop = new FileOutputStream("c:/newfile.txt");
// Will never happen, new FileOutputStream creates the file
if (!file.exists()) {
file.createNewFile();
}
EDIT:
As for the close part of the question:
When you comment out close(), then exiting main() the close method is called by the finalizer (i.e before the stream is garbage collected, a JVM thread calls its finalize() method, which in turn calls the close() method), but you can't sensibly rely on the finalizer: you don't own it and you can't be sure of when it is activated.
Again , best practice is to call close() explicitly.
Short question,
I saw in some old code where a ByteArrayInputStream was created like:
new BufferedReader(new InputStreamReader(new ByteArrayInputStream(somebytes)));
And then the BufferedReader is used to read out somebytes line by line.
All working fine, but I noticed that the BufferedReader is never closed.
This is all working in a long running websphere application, the somebytes are not terrible big (200k most), it is only invoked a few times a week and we're not experiencing any apparent memory leaks. So I expect that all the objects are successfully garbage collected.
I always (once) learned that input/output streams need to be closed, in a finally statement. Are ByteStreams the exception to this rule?
kind regards
Jeroen.
You don't have to close ByteArrayInputStream, the moment it is not referenced by any variable, garbage collector will release the stream and somebytes (of course assuming they aren't referenced somewhere else).
However it is always a good practice to close every stream, in fact, maybe the implementation creating the stream will change in the future and instead of raw bytes you'll be reading file? Also static code analyzing tools like PMD or FindBugs (see comments) will most likely complain.
If you are bored with closing the stream and being forced to handle impossible IOException, you might use IOUtils:
IOUtils.closeQuietly(stream);
It is always good practice to close your readers. However not closing a ByteArrayInputStream does not have as heavy of a potential negative effect because you are not accessing a file, just a byte array in memory.
As #TomaszNurkiewicz mentioned it's always good to close the opened stream. Another good way to let it do the try block itself. Use try with resource like.......
try ( InputStream inputStream = new ByteArrayInputStream(bytes); Workbook workBook = new XSSFWorkbook(inputStream)) {
here Workbook and InputStream both implements Closeable Interface so once try block completes ( normally or abruptly), stream will be closed for sure.
Resources need to be closed in a finally (or equivalent). But where you just have some bytes, no it doesn't matter. Although when writing, be careful to flush in the happy case.
I have two threads in Java.
First thread is closing a bufferedreader (br.close())
When the second thread does a read on the same reader I get an IOException (Stream Closed)
I get this exception even if I use br.ready()
Is there a way to know if the stream is already closed?
The method ready() will throw an exception if closed. But even if you added a closed check method, so long as the lock is released between calls (which it is for BufferedReader), the reader might be closed by the time you read it.
I see three options:
Wrap your read call with a try/catch block to handle the closed case.
Create a subclass of BufferedReader that extends close() to set your own variable that
can be used to check to see if the reader is closed. This also requires overriding a lot of
methods to do whatever the behavior you want with a closed reader if you want it do something beside throw the IOException.
Add a lock of your own and use it to both close the reader (one thread) and
check that the buffer is ready and read from it. You can either set a variable directly for the check or just group the ready() and read() calls into the same synchronized block.
I don't think there's any method you can call directly to tell if a stream is closed.
If you really need to have two threads sharing this reader, your best option might be to have the one thread call back to the other thread or set a flag to notify it that the stream has been closed so the other thread knows not to try to read from it.
If you have already started a read operation, then you will notified with an IOException that a stream is closed. Even if you have called br.ready() before, the exception is happening when your code is blocked at the read method.
There is no way to avoid this. On the contrary, you should expect a read operation to throw an exception and be prepared to handle it appropriately.
Another way would be to extend a BufferedReader in your own class, overriding close() method to indicate if it was closed.
Indeed, looking at ensureOpen() it checks if the in object is not null.
Since it is private we can't access it from outside.
But if you really need it, you can use reflection.
BufferedReader myBR = new BufferedReader(new FileReader("c:/somefile.txt"));
myBR.close();
Class c = myBR.getClass();
Field in = c.getDeclaredField("in");
in.setAccessible(true);
Object inReader = in.get(myBR);
if(inReader == null){
System.out.println("is closed");
}
else{
System.out.println("is open");
}
If only things that read from BufferedReader and close it are your threads, I would just make synchronized section and set some flat when closing the Reader. So in each thread you should:
Open synchronized section.
Check if stream isn't closed already (check the flag).
Read from stream.
Close stream.
Set the flag.
End synchronized section.
Just be careful and avoid anything that could hang in synchronized section. Basically put there only what is necessary, only read data, process it later.