Develop Reference

File descriptor leak example?

Is there any good example do demonstrate file descriptor leak in Android? I read somewhere that it occurs if we don't close the streams for example FileInputStream or FileOutputStream but I could not find any good reference example which demonstrates it.
Please share some blog/code snippet. thank you!

Because Dalvik's FileInputStream will close itself when it is garbage collected (this is also true for OpenJDK/Oracle) it is less common than you'd think to actually leak file descriptors. Of course, the file descriptors will be "leaked" until the GC runs so depending on your program it could take a while before they are reclaimed.
To accomplish a more permanent leak you will have to prevent the stream from being garbage collected by keeping a reference to it somewhere in memory.
Here's a short example that loads a properties file every 1 second and keeps track of every time it has changed:
public class StreamLeak {
/**
* A revision of the properties.
*/
public static class Revision {
final ZonedDateTime time = ZonedDateTime.now();
final PropertiesFile file;
Revision(PropertiesFile file) {
this.file = file;
}
}
/*
* Container for {#link Properties} that implements lazy loading.
*/
public static class PropertiesFile {
private final InputStream stream;
private Properties properties;
PropertiesFile(InputStream stream) {
this.stream = stream;
}
Properties getProperties() {
if(this.properties == null) {
properties = new Properties();
try {
properties.load(stream);
} catch(IOException e) {
e.printStackTrace();
}
}
return properties;
}
#Override
public boolean equals(Object o) {
if(o instanceof PropertiesFile) {
return ((PropertiesFile)o).getProperties().equals(getProperties());
}
return false;
}
}
public static void main(String[] args) throws IOException, InterruptedException {
URL url = new URL(args[0]);
LinkedList<Revision> revisions = new LinkedList<>();
// Loop indefinitely
while(true) {
// Load the file
PropertiesFile pf = new PropertiesFile(url.openStream());
// See if the file has changed
if(revisions.isEmpty() || !revisions.getLast().file.equals(pf)) {
// Store the new revision
revisions.add(new Revision(pf));
System.out.println(url.toString() + " has changed, total revisions: " + revisions.size());
}
Thread.sleep(1000);
}
}
}
Because of the lazy loading we keep the InputStream in the PropertiesFile which will be kept whenever we create a new Revision and since we never close the stream we will be leaking file descriptors here.
Now, these open file descriptors will be closed by the OS when the program terminates, but as long as the program is running it will continue to leak file descriptors as can be seen by using lsof:
$ lsof | grep pf.properties | head -n 3
java 6938 raniz 48r REG 252,0 0 262694 /tmp/pf.properties
java 6938 raniz 49r REG 252,0 0 262694 /tmp/pf.properties
java 6938 raniz 50r REG 252,0 0 262694 /tmp/pf.properties
$ lsof | grep pf.properties | wc -l
431
And if we force the GC to run we can see that most of these are returned:
$ jcmd 6938 GC.run
6938:
Command executed successfully
$ lsof | grep pf.properties | wc -l
2
The remaining two descriptors are the ones stored in the Revisions.
I ran this on my Ubuntu machine but the output would look similar if run on Android.

InputStream in;
try {
in = new BufferedInputStream(socket.getInputStream());
// Do your stuff with the input stream
} catch (Exception e) {
// Handle your exception
} finally {
// Close the stream here
if (in != null) {
try {
in.close();
} catch (IOException e) {
Log.e(TAG, "Unable to close stream: " + e);
}
}
}
The idea is to close your file descriptor in the finally block. Whether you finish successfully or an exception occurs, the file descriptor will be properly closed.
Now, if you're looking for something to demonstrate how to NOT do this properly, just wrap this code in a while(1) loop, comment out the in.close() line, and put a break; in your catch block so that when it blows up you'll break out of your infinite loop.

InputStream in;
try {
in = new FileInputStream(new File("abc");
in.read(); // Do some stuff with open fileinputstream
// If an exception is generated, inputstream object will not be closed
// as the next statement will not be executed, instead jumping to
// the catch block. this will cause a leak of the fd assigned to file
// "abc" while opening it
in.close()'
} catch (Exception e) {
// Handle your exception
}

Writing to an External File on Android --- File Doesn't Register, But Java Can Read

I'm trying to write to an external txt (or csv) file for Android. I can run an app, close it, and run it again, and readData() will read back to my log what I've stored. However, the dirFile (file directory) appears nowhere within my Android files (even if I connect it to a computer and search).
Something interesting, though: if I clear my log (similar to a list of print statements shown within Eclipse) and disconnect my phone from my computer, then reconnect it, the log reappears with everything I've ever written to my file (even if I later overwrote it)...yet the app isn't even running!
Here is my code. Please help me understand why I cannot find my file!
(Note: I've tried appending a "myFile.txt" extension to the directory, but it just causes an EISDIR exception.)
public void writeData(String dirName){
try
{
File root = new File(getExternalFilesDir(null), dirName);
// Writes to file
//
// The "true" argument allows the file to be appended. Without this argument (just root),
// the file will be overwritten (even though we later call append) rather than appended to.
FileWriter writer = new FileWriter(root, true);
writer.append("Append This Text\n");
writer.flush();
writer.close();
// Checks if we actually wrote to file by reading it back in (appears in Log)
//readData(dirName);
}
catch(Exception e)
{
Log.v("2222", "2222 ERROR: " + e.getMessage());
}
}
If you're interested, here's the function I wrote to read in the data:
public void readData(String dirName){
try
{
File root = new File(getExternalFilesDir(null), dirName);
// Checks to see if we are actually writing to file by reading in the file
BufferedReader reader = new BufferedReader(new FileReader(root));
try {
String s = reader.readLine();
while (s != null) {
Log.v("2222", "2222 READ: " + s);
s = reader.readLine();
}
}
catch(Exception e) {
Log.v("2222", "2222 ERROR: " + e.getMessage());
}
finally {
reader.close();
}
}
catch(Exception e) {
Log.v("2222", "2222 ERROR: " + e.getMessage());
}
}
Thanks!

even if I connect it to a computer and search
if I clear my log (similar to a list of print statements shown within Eclipse) and disconnect my phone from my computer, then reconnect it, the log reappears with everything I've ever written to my file (even if I later overwrote it).
What you are seeing on your computer is what is indexed by MediaStore, and possibly a subset of those, depending upon whether your computer caches information it gets from the device in terms of "directory" contents.
To help ensure that MediaStore indexes your file promptly:
Use a FileOutputStream (optionally wrapped in an OutputStreamWriter), not a FileWriter
Call flush(), getFD().sync(), and close() on the FileOutputStream, instead of calling flush() and close() on the FileWriter (sync() will ensure the bytes are written to disk before continuing)
Use MediaScannerConnection and scanFile() to tell MediaStore to index your file
You can then use whatever sort of "reload" or "refresh" or whatever option is in your desktop OS's file manager, and your file should show up.
This blog post has more on all of this.

public void create(){
folder = new File(Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_MOVIES),"video");
boolean success = true;
if (!folder.exists()) {
success=folder.mkdirs();
}
if (success) {
readfile();
} else {
System.out.println("failed");
}
}
The above code will be used to crete the directory in th emobile at desired path
private void readfile() {
// TODO Auto-generated method stub
AssetManager assetManager = getResources().getAssets();
String[] files = null;
try {
files = assetManager.list("clipart");
} catch (Exception e) {
Log.e("read clipart ERROR", e.toString());
e.printStackTrace();
}
for(String filename : files) {
System.out.println("File name => "+filename);
InputStream in = null;
OutputStream out = null;
try {
in = assetManager.open("clipart/" + filename);
out = new FileOutputStream(folder + "/" + filename);
copyFile(in, out);
in.close();
in = null;
out.flush();
out.close();
out = null;
} catch(Exception e) {
Log.e("copy clipart ERROR", e.toString());
e.printStackTrace();
}
}}private void copyFile(InputStream in, OutputStream out) throws IOException {
byte[] buffer = new byte[1024];
int read;
while((read = in.read(buffer)) != -1){
out.write(buffer, 0, read);
}}
this is my code used to write file in internal memory from the assets folder in project. This code can read all type(extension) of file from asset folder to mobile.
Don't forget to add permission in manifest file
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
and call the above function by
readfile();//this call the function to read and write the file
I hope this may help you.
Thank you.

Java File.delete() does not delete all files

I have the following Java code which iterates through all the files in a directory and deletes them.
for(File file : tmpDir.listFiles())
{
file.delete();
}
It does however not delete all files. Some, usually 20-30, out of a couple of thousand, are left behind when I do this. Is it possible to fix this, or have I stumbled upon some Java voodoo that is best left alone?

It returns a boolean value, you should check that. From the JavaDoc:
Returns:
true if and only if the file or directory is successfully deleted; false otherwise
You should check the value of the return and take action.
If it returns false it may well be that you do not have permission to delete the file.
In that case you can check whether the file is writeable by the application and if not attempt to make it writeable - again this returns a boolean. If successful you can try deleting again.
You could use a utility method:
private void deleteFile(final File f) throws IOException {
if (f.delete()) {
return;
}
if (!f.canWrite() && !f.setWritable(true)) {
throw new IOException("No write permissions on file '" + f + "' and cannot set writeable.");
}
if (!f.delete()) {
throw new IOException("Failed to delete file '" + f + "' even after setting writeable; file may be locked.");
}
}
I would also take their advice in the JavaDoc:
Note that the Files class defines the delete method to throw an
IOException when a file cannot be deleted. This is useful for error
reporting and to diagnose why a file cannot be deleted.
Provided that you are using Java 7 that is. That method throws a number of exceptions that you can handle:
try {
Files.delete(path);
} catch (NoSuchFileException x) {
System.err.format("%s: no such" + " file or directory%n", path);
} catch (DirectoryNotEmptyException x) {
System.err.format("%s not empty%n", path);
} catch (IOException x) {
// File permission problems are caught here.
System.err.println(x);
}
Example taken from the Oracle tutorial page.

Forcing the garbage collector to run using System.gc(); made all the files deletable.

Make sure that you don't have any open stream like BufferedReader/Writer, FileReader/Writer etc. First close them, then you should be able to delete the file.
One more point, E.g. if you open a BufferedReader via another reader like FileReader, you must close both of the readers seperately.
So instead of this:
BufferedReader reader = new BufferedReader(new FileReader(new File(filePath)););
do this:
BufferedReader bufferedReader = null;
FileReader fileReader = null;
try{
fileReader = new FileReader(readFile);
bufferedReader = new BufferedReader(fileReader);
}catch{...}
...
try {
fileReader.close();
bufferedReader .close();
readFile.delete();
} catch (IOException e) {
e.printStackTrace();
}

Resource leak: 'in' is never closed, though it IS closed

I know that there are a couple of similarly entitled questions out there, but most of them have simply forgotten to put a close() directive on their stream. This here is different.
Lets say I have the following minimal example:
public void test() throws IOException
{
InputStream in;
if( file.exists() )
{
in = new FileInputStream( file );
}
else
{
in = new URL( "some url" ).openStream();
}
in.close();
}
This give me a Resource leak: 'in' is never closed warning in Eclipse (Juno SR1).
But when I move the in.close() into the conditional block, the warnings vanishes:
public void test() throws IOException
{
InputStream in;
if( file.exists() )
{
in = new GZIPInputStream( new FileInputStream( file ) );
in.close();
}
else
{
in = new URL( "some URL" ).openStream();
}
}
What is going on here?

Because of the IO exception, you can run into a resource leak (poentially)
Try doing the following:
public void test() throws IOException
{
InputStream in= null;
try {
if( file.exists() )
{
// In this case, if the FileInputStream call does not
// throw a FileNotFoundException (descendant of IOException)
// it will create the input stream which you are wrapping
// in a GZIPInputStream (no IO exception on construction)
in = new GZIPInputStream( new FileInputStream( file ) );
}
else
{
// Here however, if you are able to create the URL
// object, "some url" is a valid URL, when you call
// openStream() you have the potential of creating
// the input stream. new URL(String spec) will throw
// a MalformedURLException which is also a descendant of
// IOException.
in = new URL( "some url" ).openStream();
}
// Do work on the 'in' here
} finally {
if( null != in ) {
try
{
in.close();
} catch(IOException ex) {
// log or fail if you like
}
}
}
}
Doing the above will make sure you've closed the stream or at least made a best effort to do so.
In your original code, you had the InputStream declared but never initialized. That is bad form to begin with. Initialize that to null as I illustrated above. My feeling, and I'm not running Juno at the moment, is that it sees that the InputStream 'in', may potentially make it through all the hoops and hurdles to get to the point at which you are going to use it. Unfortunate, as someone pointed out, your code is a bit dodgy for an example. Doing this as I've detailed as well as #duffymo you'll get rid of the warning.

Here's how I'd write it:
public void test() throws IOException
{
InputStream in = null;
try {
if(file.exists()) {
in = new FileInputStream( file );
} else {
in = new URL( "some url" ).openStream();
}
// Do something useful with the stream.
} finally {
close(in);
}
}
public static void close(InputStream is) {
try {
if (is != null) {
is.close();
}
} catch (IOException e) {
e.printStackTrace();
}
}

I suspect the warning is incorrect. It could be checking you are closing the stream in the same scope. In the second case, you are not closing the second stream.

Your in stream may not be initialized if the file doesn't exist and you try to close a non-existent file.
Your second example would also need a close statement to avoid leaks.

This same Eclipse reporting can happen when you explicitly throw an exception after you have opened your resource like:
public void method() throws IOException {
BufferedReader br = new BufferedReader(new FileReader("myfile.txt"));
while (br.ready()) {
String line = br.readLine():
if (line.length() > 255) {
throw new IOException("I am some random IOException");
}
}
br.close();
}
This is some contrived code for demonstration purposes so don't look too hard.
If one were to comment out the line, the warning goes away. Of course, you instead want to make sure that that resource is being closed properly. You could do:
if (line.length() > 255) {
br.close();
throw new IOException("I am some random IOException");
}
Do not rely on the Eclipse warnings in this case though. Get in the habit of using the try/finally approach to make sure that resources are correctly and consistently being closed.

I have something like:
InputStream content = httpResponse.getEntity()==null?null:httpResponse.getEntity().getContent();
that gives the same warrning. But if I leave it just like this:
InputStream content =httpResponse.getEntity().getContent();
I receive no warrnings. Isn't strange or what?
-- I hope my info is adding knowledge to the original question. Thanks!

FFMPEG in Java issue

I have the following code in a java Web Service:
public boolean makeFile(String fileName, String audio)
{
if (makeUserFolder())
{
File file = new File(getUserFolderPath() + fileName + amr);
FileOutputStream fileOutputStream = null;
try
{
file.createNewFile();
fileOutputStream = new FileOutputStream(file);
fileOutputStream.write(Base64.decode(audio));
return true;
}
catch(FileNotFoundException ex)
{
return false;
}
catch(IOException ex)
{
return false;
}
finally{
try {
fileOutputStream.close();
convertFile(fileName);
} catch (IOException ex) {
Logger.getLogger(FileUtils.class.getName()).log(Level.SEVERE, null, ex);
}
}
}
else
return false;
}
public boolean convertFile(String fileName)
{
Process ffmpeg;
String filePath = this.userFolderPath + fileName;
try {
ProcessBuilder pb = new ProcessBuilder("ffmpeg","-i",filePath + amr,filePath + mp3);
pb.redirectErrorStream();
ffmpeg = pb.start();
} catch (IOException ex) {
return false;
}
return true;
}
It used to work and now it simply won't execute the ffmpeg conversion for some reason. I thought it was a problem with my file but after running the command from terminal no errors are thrown or anything, thought it was maybe permissions issue but all the permissions have been granted in the folder I'm saving the files. I noticed that the input BufferedReader ins being set to null after running the process, any idea what's happening?

First of all, a small nitpick with your code...when you create the FileOutputStream you create it using a string rather than a File, when you have already created the File before, so you might as well recycle that rather than force the FileOutputStream to instantiate the File itself.
Another small nitpick is the fact that when you are writing out the audio file, you should enclose that in a try block and close the output stream in a finally block. If you are allowed to add a new library to your project, you might use Guava which has a method Files.write(byte[],File), which will take care of all the dirty resource management for you.
The only thing that I can see that looks like a definite bug is the fact that you are ignoring the error stream of ffmpeg. If you are blocking waiting for input on the stdout of ffmpeg, then it will not work.
The easiest way to take care of this bug is to use ProcessBuilder instead of Runtime.
ProcessBuilder pb = new ProcessBuilder("ffmpeg","-i",filePath+amr,filePath+mp3);
pb.redirectErrorStream(); // This will make both stdout and stderr be redirected to process.getInputStream();
ffmpeg = pb.start();
If you start it this way, then your current code will be able to read both input streams fully. It is possible that the stderr was hiding some error that you were not able to see due to not reading it.
If that was not your problem, I would recommend using absolute paths with ffmpeg...in other words:
String lastdot = file.getName().lastIndexOf('.');
File mp3file = new File(file.getParentFile(),file.getName().substring(0,lastdot)+".mp3");
ProcessBuilder pb = new ProcessBuilder("ffmpeg","-i",file.getAbsolutePath(),mp3file.getAbsolutePath());
// ...
If that doesn't work, I would change ffmpeg to be an absolute path as well (in order to rule out path issues).
Edit: Further suggestions.
I would personally refactor the writing code into its own method, so that you can use it elsewhere necessary. In other other words:
public static boolean write(byte[] content, File to) {
FileOutputStream fos = new FileOutputStream(to);
try {
fos.write(content);
} catch (IOException io) {
// logging code here
return false;
} finally {
closeQuietly(fos);
}
return true;
}
public static void closeQuietly(Closeable toClose) {
if ( toClose == null ) { return; }
try {
toClose.close();
} catch (IOException e) {
// logging code here
}
}
The reason that I made the closeQuietly(Closeable) method is due to the fact that if you do not close it in that way, there is a possibility that an exception will be thrown by the close() method, and that exception will obscure the exception that was thrown originally. If you put these in a utility class (although looking at your code, I assume that the class that it is currently in is named FileUtils), then you will be able to use them throughout your application whenever you need to deal with file output.
This will allow you to rewrite the block as:
File file = new File(getUserFolderPath() + fileName + amr);
file.createNewFile()
write(Base64.decode(audio),file);
convertFile(fileName);
I don't know whether or not you should do this, however if you want to be sure that the ffmpeg process has completed, then you should say ffmpeg.waitFor(); to be sure that it has completed. If you do that, then you should examine ffmpeg.exitValue(); to make sure that it completed successfully.
Another thing that you might want to do is once it has completed, write what it output to a log file so you have a record of what happened, just in case something happens.