I have the following piece of code in a try/catch block
InputStream inputstream = conn.getInputStream();
InputStreamReader inputstreamreader = new InputStreamReader(inputstream);
BufferedReader bufferedreader = new BufferedReader(inputstreamreader);
My question is that when I have to close these streams in the finally block, do I have to close all the 3 streams or just closing the befferedreader will close all the other streams ?
By convention, wrapper streams (which wrap existing streams) close the underlying stream when they are closed, so only have to close bufferedreader in your example. Also, it is usually harmless to close an already closed stream, so closing all 3 streams won't hurt.
Normally it is ok to just close the most outer stream, because by convention it must trigger close on the underlying streams.
So normally code looks like this:
BufferedReader in = null;
try {
in = new BufferedReader(new InputStreamReader(conn.getInputStream()));
...
in.close(); // when you care about Exception-Handling in case when closing fails
}
finally {
IOUtils.closeQuietly(in); // ensure closing; Apache Commons IO
}
Nevertheless there may be rare cases where an underlying stream constructor raises an exception where the stream is already opened. In that case the above code won't close the underlying stream because the outer constructor was never called and in is null. So the finally block does not close anything leaving the underlying stream opened.
Since Java 7 you can do this:
try (OutputStream out1 = new ...; OutputStream out2 = new ...) {
...
out1.close(); //if you want Exceptions-Handling; otherwise skip this
out2.close(); //if you want Exceptions-Handling; otherwise skip this
} // out1 and out2 are auto-closed when leaving this block
In most cases you do not want Exception-Handling when raised while closing so skip these explicit close() calls.
Edit
Here's some code for the non-believers where it is substantial to use this pattern. You may also like to read Apache Commons IOUtils javadoc about closeQuietly() method.
OutputStream out1 = null;
OutputStream out2 = null;
try {
out1 = new ...;
out2 = new ...;
...
out1.close(); // can be skipped if we do not care about exception-handling while closing
out2.close(); // can be skipped if we ...
}
finally {
/*
* I've some custom methods in my projects overloading these
* closeQuietly() methods with a 2nd param taking a logger instance,
* because usually I do not want to react on Exceptions during close
* but want to see it in the logs when it happened.
*/
IOUtils.closeQuietly(out1);
IOUtils.closeQuietly(out2);
}
Using #Tom's "advice" will leave out1 opened when creation of out2 raises an exception. This advice is from someone talking about It's a continual source of errors for obvious reasons. Well, I may be blind, but it's not obvious to me. My pattern is idiot-safe in every use-case I can think of while Tom's pattern is error-prone.
Closing the outermost one is sufficient (i.e. the BufferedReader). Reading the source code of BufferedReader we can see that it closes the inner Reader when its own close method is called:
513 public void close() throws IOException {
514 synchronized (lock) {
515 if (in == null)
516 return;
517 in.close();
518 in = null;
519 cb = null;
520 }
521 }
522 }
As a rule of thumb, you should close everything in the reverse order that you opened them.
I would close all of them in the inverse order from which you have opened them, as if when opening them would push the reader to a stack and closing would pop the reader from the stack.
In the end, after closing all, the "reader stack" must be empty.
You only need to close the actual resource. You should close the resource even if constructing decorators fails. For output, you should flush the most decorator object in the happy case.
Some complications:
Sometimes the decorators are different resources (some compression implementations use the C heap).
Closing decorators in sad cases actually causes flushes, with ensuing confusion such as not actually closing the underlying resource.
It looks like you underlying resource is a URLConnection, which doesn't have a disconnect/close method as such.
You may wish to consider using the Execute Around idiom so you don't have to duplicate this sort of thing.
Related
I have a Java program that reads some text from a file, line by line, and writes new text to an output file. But not all the text I write to my BufferedWriter appears in the output file after the program has finished. Why is that?
The details: the program takes a CSV text document and converts it into SQL commands to insert the data into a table. The text file has more than 10000 lines which look similar to following:
2007,10,9,1,1,1006134,19423882
The program seems to work fine except it just stops in the file randomly half way through creating a new SQL statement having printed it into the SQL file. It looks something like:
insert into nyccrash values (2007, 1, 2, 1, 4, 1033092, 259916);
insert into nyccrash values (2007, 1, 1, 1, 1, 1020246, 197687);
insert into nyccrash values (2007, 10, 9, 1
This happens after about 10000 lines but several hundred lines before the end of the file. Where the break happens is between a 1 and a ,. However, the characters doesn't seem important because if I change the 1 to a 42 the last thing written to the new file is 4, which is cutting off the 2 from that integer. So it seems like the reader or writer must just be dying after writing/reading a certain amount.
My Java code is as follows:
import java.io.*;
public class InsertCrashData
{
public static void main (String args[])
{
try
{
//Open the input file.
FileReader istream = new FileReader("nyccrash.txt");
BufferedReader in = new BufferedReader(istream);
//Open the output file.
FileWriter ostream = new FileWriter("nyccrash.sql");
BufferedWriter out = new BufferedWriter(ostream);
String line, sqlstr;
sqlstr = "CREATE TABLE nyccrash (crash_year integer, accident_type integer, collision_type integer, weather_condition integer, light_condition integer, x_coordinate integer, y_coordinate integer);\n\n";
out.write(sqlstr);
while((line = in.readLine())!= null)
{
String[] esa = line.split(",");
sqlstr = "insert into nyccrash values ("+esa[0]+", "+esa[1]+", "+esa[2]+", "+esa[3]+", "+esa[4]+", "+esa[5]+", "+esa[6]+");\n";
out.write(sqlstr);
}
}
catch(Exception e)
{
System.out.println(e);
}
}
}
You need to close your OutputStream which will flush the remainder of your data:
out.close();
The default buffer size for BufferedWriter is 8192 characters, large enough to easily hold hundreds of lines of unwritten data.
You must close() your BufferedWriter. You must close() your BufferedWriter because it IS-A Writer and thus implements AutoCloseable, which means (emphasis added) it is
A resource that must be closed when it is no longer needed.
Some people say you must first call flush() for your BufferedWriter before calling close(). They are wrong. The documentation for BufferedWriter.close() notes that it "Closes the stream, flushing it first" (emphasis added).
The documented semantics of flushing (flush()) are
Flushes this stream by writing any buffered output to the underlying stream
So, you must close, and close will flush any buffered output.
Your output file does not include all the text you wrote to your BufferedWriter because it stored some of that text in a buffer. The BufferedWriter never emptied that buffer, passing it through to the file, because you never told it to do so.
Since Java 7, the best way to ensure an AutoCloseable resource, such as a BufferedWriter, is closed when it is not longer need is to use automatic resource management (ARM), also known as try-with-resources:
try (BufferedWriter out = new BufferedWriter(new FileWriter(file))) {
// writes to out here
} catch (IOException ex) {
// handle ex
}
You must also close your BufferedReader when it is no longer need, so you should have nested try-with-resources blocks:
try (BufferedReader in = new BufferedReader(new FileReader("nyccrash.txt")) {
try (BufferedWriter out = new BufferedWriter(new FileWriter("nyccrash.sql"))) {
// your reading and writing code here
}
} catch (IOException ex) {
// handle ex
}
Do not be tempted (as other answers here suggest) just to call close() at the end of your method, when your code has "finished" using the writer. That will not work if your writing code throws an exception, and in particular if it throws an IOException.
A resource that must be closed when it is no longer needed.
finally {
out.close();//this would resolve the issue
}
Some things to consider:
BufferedWriter.close() flushes the buffer to the underlying stream, so if you forget to flush() and don't close, your file may not have all the text you wrote to it.
BufferedWriter.close() also closes the wrapped Writer. When that's a FileWriter, this will ultimately close a FileOutputStream and tell the OS that you're done writing to the file.
The garbage collector will automatically call close(), not on the BufferedWriter or the wrapped FileWriter, but on the FileOuputStream. So the OS will be happy, but you have to wait for the GC.
However, you always want to release OS resources as soon as you no longer need them. This goes for open files, database connections, print queues ... anything. Trust me on this one.
BufferedWriter.close() does clear up the internal character buffer, so that memory will be available for garbage collection, even while the BufferedWriter itself remains in scope.
So, Always close your resources (not just files) when you're done with them.
If you really want a peek under the covers, most of the Java API's source is available. BufferedWriter is here.
Your code does not appear to be closing the writer after you've finished writing to it. Add an out.close() (preferably in a finally block) and it should work properly.
you dint close your BufferedWriter.close it inside a finally block
finally {
out.close();//this would resolve the issue
}
Always close your resources (not just files) when you're done with them.
finally {
out.close();//this would resolve the issue
}
There might be situations when you want to flush the buffer without closing the file. In these situations you can use the flush-method.
Since you're using BufferedWriter you can also flush the buffer when appropriate:
out.flush()
This will write the rest of the buffer to the actual file. Close-method also flushes the buffer and closes the file.
out.close()
There might be situations when you want to flush the buffer without closing the file. In these situations you can use the flush-method.
You can also use BuffredWriter's newline-method instead of adding \n to the end of the line. Newline-method uses system specific line separator so your code works on different platforms.
out.newLine()
According to documentation it is no use calling flush() method. If you intent to use FileWriter then flush() would help you out.
Basically in this case, you just need to close, BufferedWriter.close() only. This will flush the remainder of your data.
create finally block and put the close method inside so that it will put all data without missing.
finally {
out.close();
}
I would like to know when does a stream close if its not closed manually. By this I mean, will the stream be closed if the scope of its reference is no more?
Consider the following sample scenario.
Class A{
InputStream in;
OutputStream out;
A(){
// Initialize and create the streams.
}
...
}
Class B{
public void myMethod(){
A a = new A();
System.out.println("End of my method.")
}
...
}
Here, once I am done with the stream, I am exiting myMethod() but the program which in turn the process, is not terminated and goes on with other operations.
I did not close the streams. Will it be closed automatically once the scope of the reference to class A ends? (ie. When myMethod() ends)? Does GC takes care of that? Also, I read that the streams will be closed once the process ends and the system releases all the resources held for it for other processes. How can we check whether the stream is open or not? Are there any linux commands or options from Java to check that?
Any help is appreciated!
I don't think that the JVM spec makes any guarantee about that. You really are supposed to finally close these resources.
When the process ends, the operating system will release all resources associated to it (including memory, file handles, and network sockets).
There are OS facilities to check about open files and streams, such as lsof.
In the case of FileInputStream there's a finalize() method that will free resources when the stream is garbage collected.
Not that you can or should rely on that. It's just what FileInputStream does. SocketInputStream on the other hand overrides finalize() to explicitly do nothing, relying on Socket to close the resources (and Socket doesn't have a finalize() method).
If you are not clossing it manually then all the unmanaged resources will be released when the process termiantes, however it is not a recommended or a best practice. You should always close your stream once you are done with it.
A better and suggested way to manage the stream is to use try with resource option like this:
try (InputStream input = new FileInputStream(...);
Reader reader = new InputStreamReader(input, ...)) {
...
}
I did not close the streams. Will it be closed automatically once the
scope of the reference to class A ends?
No it will not close automatically.
A good reference to follow is:
Better Resource Management with Java SE 7: Beyond Syntactic Sugar
There is no garantee that the resources will be closd as long as the JVM is running, Your suggested implementation is quite dangerous.
What I would suggest.
Make the class A a Closeable and use the tryResourceClose-Statement of Java.
The example is here.
https://docs.oracle.com/javase/tutorial/essential/exceptions/tryResourceClose.html
After leaving the try-block you get the chance to close your resources.
The stream itself normally doesn't know if it is open or not.
However, your own class A can keep track it the streams were closed or not.
That's a bad programming practice, an error from the programmer. Depending on the underlying data source, it might never close and you can have leaks. You MUST close any resource when you've finished with it, in a finally block, or using a try with resources if Java 7 or higher, to ensure it is closed even if an exception is thrown.
InputStream in;
OutputStream out;
try {
// Do your stuff with the streams
} finally {
if (in != null) {
in.close();
}
if (out != null) {
out.close();
}
}
With Java 7, you can create one or more “resources” in the try statement. A “resources” is something that implements the java.lang.AutoCloseable interface. This resource would be automatically closed and the end of the try block.
you can look this and java doc for more info
private static void printFileJava7() throws IOException {
try(FileInputStream input = new FileInputStream("file.txt")) {
int data = input.read();
while(data != -1){
System.out.print((char) data);
data = input.read();
}
}
}
When the try block finishes the FileInputStream will be closed automatically. This is possible because FileInputStream implements the Java interface java.lang.AutoCloseable. All classes implementing this interface can be used inside the try-with-resources construct.
I am a bit confused with an error my program started throwing recently.
java.io.IOException: No space left on device
at java.io.FileInputStream.close0(Native Method)
at java.io.FileInputStream.close(FileInputStream.java:259)
at java.io.FilterInputStream.close(FilterInputStream.java:155)
I am assuming that since this is a FileInputStream, that this file is being held in memory, and not on the physical disk. Memory levels look great, and as does disk space. This is especially confusing since it happens on the close of the FileInputStream. Thanks for any explanations you might have as to how this can occur.
EDIT: Code for review
if (this.file.exists()) {
DataInputStream is = new DataInputStream(new FileInputStream(this.file));
this.startDate = new DateTime(is.readLong(), this.timeZone);
this.endDate = new DateTime(is.readLong(), this.timeZone);
is.close();
}
As you can see above I am only opening the file, reading some content, and then closing the file.
In this case, the IOException is thrown from the native method that closes the stream.
The reason it is defined to throw an exception is because the close operation performs a final flush - thus, if an IOException occurs during the flush it will be thrown.
There are several reasons for the exception you have received:
You may lack write permissions on the specific folder.
You may have exceeded your quota.
I also personally suggest that you use the following method of closing the stream:
if (this.file.exists()) {
try {
DataInputStream is = new DataInputStream(new FileInputStream(this.file));
this.startDate = new DateTime(is.readLong(), this.timeZone);
this.endDate = new DateTime(is.readLong(), this.timeZone);
} catch (Exception ex) {
// Handle the exception here
} finally {
is.close();
}
}
You can also use IOUtils method closeQuietly that does not throw an exception 'cause in your case you are not changing the file and you are probably not interested in the result of the close method.
EDIT:
Henry is right. I read InputStream and automatically changed it in my mind to OutputStream.
A close operation on the InputStream does not change the file itself but can change the metadata of the file - such as last access time, etc.
I am new to the guava library, and I am quite confused with the InputSupplier and OutputSupplier. According to the javadoc, they are just factories for InputStream and OutputStream respectively. However, I don't see anything quite useful of these two interface, could anyone show me a example why I should use these two APIs besides for higher abstraction?
The main benefit of both of these interfaces is that they allow library code to control the whole lifecycle of the actual input/output objects. Guava's utilities never close an InputStream or OutputStream that you pass in directly, because that might not be what you want to happen. Even if they did, you'd still need try/finally to deal with errors creating objects. The suppliers are a sort of lazy callback that allow the actual objects to be created, used and closed all in one go without you having to do a lot of ugly try/finally and error handling.
For an example, just look at the code it takes to copy one file to another (with the actual copying and stream closing code minimized using Guava utilities):
File in = ...
File out = ...
FileInputStream inStream = new FileInputStream(in);
boolean threw = true;
try {
/*
* Note how two try/finally blocks are needed here, in case creating
* outStream fails.
*/
FileOutputStream outStream = new FileOutputStream(out);
try {
ByteStreams.copy(inStream, outStream);
threw = false;
} finally {
Closeables.close(outStream, threw);
}
} finally {
Closeables.close(inStream, threw);
}
Then look at the code if you use suppliers instead:
File in = ...
File out = ...
ByteStreams.copy(Files.newInputStreamSupplier(in),
Files.newOutputStreamSupplier(out));
With Guava's InputSupplier / OutputSupplier, you do not have to handle yourself the various IOExceptions thrown when instantiating FileInputStreams / FileOutputStreams. Guava will automatically handle these exceptions for you when it calls the InputSupplier.getInput() / OutputSupplier.getOutput() factory methods.
By encapsulating the input / output construction in these factory interfaces, you defer their instantiation, and thus defer the moment you will need to handle the IOException / FileNotFoundException they may throw. In fact, you defer it so much, that it's Guava that handles it for you.
You can then replace
FileInputStream inputStream = null;
try {
inputStream = new FileInputStream(file);
} catch (FileNotFoundException ex) {
throw new RuntimeException(ex);
}
doSomething(inputStream);
with
InputSupplier<FileInputStream> inputStreamSupplier = Files.newInputStreamSupplier(file);
doSomething(inputStreamSupplier);
Edit: also see ColinD's answer, on how Guava may close these inputs / outputs for you when it controls their whole lifecycle (that is, when it used an InputSupplier / OutputSupplier to obtain them).
In the following code:
DataInputStream in = new DataInputStream(
new BufferedInputStream(new FileInputStream(file)));
in.close();
Do I need to close the 2 other stream in addition to closing the "top level" stream?
if you look the source of DataInputStream you can see that it closes the underlying streams as well. so you don't need. and this is (or should be) true for all type of streams.
I will use this opportunity to answer with an answer I have already made before.
By using Project Lombok you can let Lombok correctly close the streams for you.
Details can be found here.
Karazi, is right in suggesting that. Further, just to get an idea and a little more insight, Java IO API is actually implemented using decorator pattern. You can check out decorator pattern on wiki.
I'd stick the close in a finally block just to make sure it is flushed properly in case of an exception.
public void tryToDoWhatever() throws Exception
{
DataInputStream in = null;
try
{
in = new DataInputStream(
new BufferedInputStream(new FileInputStream(file)));
}
finally
{
if (in != null)
in.close();
}
}