Develop Reference

Java too many open files exception

I have a problem on my code; basically I have an array containing some key:
String[] ComputerScience = { "A", "B", "C", "D" };
And so on, containing 40 entries.
My code reads 900 pdf from 40 folder corresponding to each element of ComputerScience, manipulates the extracted text and stores the output in a file named A.txt , B.txt, ecc ...
Each folder "A", "B", ecc contains 900 pdf.
After a lot of documents, an exception "Too many open files" is thrown.
I'm supposing that I am correctly closing files handler.
static boolean writeOccurencesFile(String WORDLIST,String categoria, TreeMap<String,Integer> map) {
File dizionario = new File(WORDLIST);
FileReader fileReader = null;
FileWriter fileWriter = null;
try {
File cat_out = new File("files/" + categoria + ".txt");
fileWriter = new FileWriter(cat_out, true);
} catch (IOException e) {
e.printStackTrace();
}
try {
fileReader = new FileReader(dizionario);
} catch (FileNotFoundException e) { }
try {
BufferedReader bufferedReader = new BufferedReader(fileReader);
if (dizionario.exists()) {
StringBuffer stringBuffer = new StringBuffer();
String parola;
StringBuffer line = new StringBuffer();
int contatore_index_parola = 1;
while ((parola = bufferedReader.readLine()) != null) {
if (map.containsKey(parola) && !parola.isEmpty()) {
line.append(contatore_index_parola + ":" + map.get(parola).intValue() + " ");
map.remove(parola);
}
contatore_index_parola++;
}
if (! line.toString().isEmpty()) {
fileWriter.append(getCategoryID(categoria) + " " + line + "\n"); // print riga completa documento N x1:y x2:a ...
}
} else { System.err.println("Dictionary file not found."); }
bufferedReader.close();
fileReader.close();
fileWriter.close();
} catch (IOException e) { return false;}
catch (NullPointerException ex ) { return false;}
finally {
try {
fileReader.close();
fileWriter.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return true;
}
But the error still comes. ( it is thrown at:)
try {
File cat_out = new File("files/" + categoria + ".txt");
fileWriter = new FileWriter(cat_out, true);
} catch (IOException e) {
e.printStackTrace();
}
Thank you.
EDIT: SOLVED
I found the solution, there was, in the main function in which writeOccurencesFile is called, another function that create a RandomAccessFile and doesn't close it.
The debugger sais that Exception has thrown in writeOccurencesFile but using Java Leak Detector i found out that the pdf were already opened and not close after parsing to pure text.
Thank you!

Try using this utility specifically designed for the purpose.
This Java agent is a utility that keeps track of where/when/who opened files in your JVM. You can have the agent trace these operations to find out about the access pattern or handle leaks, and dump the list of currently open files and where/when/who opened them.
When the exception occurs, this agent will dump the list, allowing you to find out where a large number of file descriptors are in use.

i have tried using try-with resources; but the problem remains.
Also running in system macos built-in console print out a FileNotFound exception at the line of FileWriter fileWriter = ...
static boolean writeOccurencesFile(String WORDLIST,String categoria, TreeMap<String,Integer> map) {
File dizionario = new File(WORDLIST);
try (FileWriter fileWriter = new FileWriter( "files/" + categoria + ".txt" , true)) {
try (FileReader fileReader = new FileReader(dizionario)) {
try (BufferedReader bufferedReader = new BufferedReader(fileReader)) {
if (dizionario.exists()) {
StringBuffer stringBuffer = new StringBuffer();
String parola;
StringBuffer line = new StringBuffer();
int contatore_index_parola = 1;
while ((parola = bufferedReader.readLine()) != null) {
if (map.containsKey(parola) && !parola.isEmpty()) {
line.append(contatore_index_parola + ":" + map.get(parola).intValue() + " ");
map.remove(parola);
}
contatore_index_parola++;
}
if (!line.toString().isEmpty()) {
fileWriter.append(getCategoryID(categoria) + " " + line + "\n"); // print riga completa documento N x1:y x2:a ...
}
} else {
System.err.println("Dictionary file not found.");
}
}
}
} catch (IOException e) {
e.printStackTrace();
}
return true;
}
This is the code that i am using now, although the bad managing of Exception, why the files seem to be not closed?
Now i am making a test with File Leak Detector

Maybe your code raises another exception that you are not handling. Try add catch (Exception e) before finally block
You also can move BufferedReader declaration out the try and close it in finally

FileWriter not appending to existing file

I am writing a method that takes in a List of Twitter Status objects as a parameter, opens a log file containing String represenatations of Tweets, checks if any of the String representations of the Status objects are already written to the file - if so, it removes them from the list, if not it appends the Status to the file.
Everything is working up until I attempt to write to the file. Nothing is being written at all. I am led to believe that it is due to the method having the file open in two different places: new File("tweets.txt") and new FileWriter("tweets.txt, true).
Here is my method:
private List<Status> removeDuplicates(List<Status> mentions) {
File mentionsFile = new File("tweets.txt");
try {
mentionsFile.createNewFile();
} catch (IOException e1) {
// Print error + stacktrace
}
List<String> fileLines = new ArrayList<>();
try {
Scanner scanner = new Scanner(mentionsFile);
while (scanner.hasNextLine()) {
fileLines.add(scanner.nextLine());
}
scanner.close();
} catch (FileNotFoundException e) {
// Print error + stacktrace
}
List<Status> duplicates = new ArrayList<>();
for (Status mention : mentions) {
String mentionString = "#" + mention.getUser().getScreenName() + " \"" + mention.getText() + "\" (" + mention.getCreatedAt() + "\")";
if (fileLines.contains(mentionString)) {
duplicates.add(mention);
} else {
try {
Writer writer = new BufferedWriter(new FileWriter("tweets.txt", true));
writer.write(mentionString);
} catch (IOException e) {
// Print error + stacktrace
}
}
}
mentions.removeAll(duplicates);
return mentions;
}

I wrote here few thoughts looking your code.
Remember to always close the object Reader and Writer.
Have a look at try-with-resources statement :
try (Writer writer = new BufferedWriter(new FileWriter("tweets.txt", true))) {
writer.write(mentionString);
} catch (IOException e) {
// Print error + stacktrace
}
To read an entire file in a List<String>:
List<String> lines = Files.readAllLines(Paths.get("tweets.txt"), StandardCharsets.UTF_8);
And again, I think it's a bad practice write in the same file you're reading of.
I would suggest to write in a different file if you don't have a particular constraint.
But if you really want have this behavior there are few alternative.
Create a temporary file as output and, when you process is successfully completed, than move it to the old one using Files.move(from, to).

Java Swing FileChooser Error

I'm following this tutorial:
http://docs.oracle.com/javase/tutorial/uiswing/components/filechooser.html
so that i can learn how to use the file chooser in a GUI i am building, but the source file i have downloaded doesn't match up with the tutorial, i am getting an error message whenever i press a button in the GUI.
if (e.getSource() == saveButton) {
FileSaveService fss = null;
FileContents fileContents = null;
ByteArrayInputStream is = new ByteArrayInputStream(
(new String("Saved by JWSFileChooserDemo").getBytes()));
//XXX YIKES! If they select an
//XXX existing file, this will
//XXX overwrite that file.
try {
fss = (FileSaveService)ServiceManager.
lookup("javax.jnlp.FileSaveService");
} catch (UnavailableServiceException exc) { }
if (fss != null) {
try {
fileContents = fss.saveFileDialog(null,
null,
is,
"JWSFileChooserDemo.txt");
} catch (Exception exc) {
log.append("Save command failed: "
+ exc.getLocalizedMessage()
+ newline);
log.setCaretPosition(log.getDocument().getLength());
}
}
if (fileContents != null) {
try {
log.append("Saved file: " + fileContents.getName()
+ "." + newline);
} catch (IOException exc) {
log.append("Problem saving file: "
+ exc.getLocalizedMessage()
+ newline);
}
} else {
log.append("User canceled save request." + newline);
}
log.setCaretPosition(log.getDocument().getLength());
}
}
I am getting the user cancelled save request.

Your main issue is that your fileContents may be null without you to know it. This can be caused by 2 reasons:
The user cancelled the request, so saveFileDialog returns null. This is the message you actually get, but the source of your error can be different ;
The error could come from the extension parameter of saveFileDialog that is null... . If an exception is raised, it would be nice to know about it, so I advise you add a message to the log (see code below).
Solution:
You should log the UnavailableServiceException to keep track of the exception, and you should respect the saveFileDialog method prototype (to avoid any ambiguity) as described here: FileSaveService Interface.
This is the part of your code where are applied the previous advices:
try {
fss = (FileSaveService)ServiceManager.lookup("javax.jnlp.FileSaveService");
} catch (UnavailableServiceException exc) {
log.append("A problem occurred while accessing the service manager." + newline);
}
if (fss != null) {
try {
fileContents = fss.saveFileDialog(null, { "txt" }, is, "JWSFileChooserDemo");
}
/* Your previous code */
}
/* Your previous code */
If this doesn't solve your problem, this will at least give your more information about where it comes from.

Using FileWriter to convert Google Analytics data into CSV file using JAVA

Hi all after getting some advice, I am attempting to use the filewriter method in order to export my google analytics queries that i got to a CSV file format here is what i have so far
private static void printGaData(GaData results) {
try {
PrintWriter pw = new PrintWriter(BufferedWriter(new FileWriter("data.csv")));
}
catch(Exception e) {
e.printStackTrace();
}
System.out.println(
"printing results for profile: " + results.getProfileInfo().getProfileName());
if (results.getRows() == null || results.getRows().isEmpty()) {
System.out.println("No results Found.");
} else {
// Print column headers.
for (ColumnHeaders header : results.getColumnHeaders()) {
System.out.printf(header.getName() + ", ");
}
System.out.println();
// Print actual data.
for (List<String> row : results.getRows()) {
for (String column : row) {
pw.printf(row + ", ");
}
pw.println();
}
pw.println();
}
}
}
doesnt output any data and keeps saying that the pw is non extent and stuff like that

Your PrintWriter is inside the try catch block. If you define it outside like
PrintWriter pw = null;
try {
pw = new PrintWriter(BufferedWriter(new FileWriter("data.csv")));
} catch (Exception e) {
// handle exception
}
then it will be available to the rest of your code.

Java writing to a text file not working properly

The Java application that I support is logging some details in a flat file. the problem I face some times is that, the entry is very low compared to the previous day. This entry is most essential because our reports are generated based on the file. I went thro code for writing I couldn't figure out any issues. the method which is writing is sync method.
Any suggestions? I can also provide the code for you is you may need?
public synchronized void log (String connID, String hotline, String callerType,
String cli, String lastMenu, String lastInput,
String status, String reason)
{
//String absoluteFP = LOG_LOC + ls + this.getFilename();
//PrintWriter pw = this.getPrintWriter(absoluteFP, true, true);
try
{
pw.print (this.getDateTime ()+ ","+connID +","+hotline+","+callerType+","+ cli+"," + lastMenu + "," + lastInput + "," + status + "," + reason);
//end 1006
pw.print (ls);
pw.flush ();
//pw.close();
}
catch (Exception e)
{
e.printStackTrace ();
return;
}
}
private synchronized PrintWriter getPrintWriter (String absoluteFileName,
boolean append, boolean autoFlush)
{
try
{
//set absolute filepath
File folder = new File (absoluteFileName).getParentFile ();//2009-01-23
File f = new File (absoluteFileName);
if (!folder.exists ())//2009-01-23
{
//System.out.println ("Call Detailed Record folder NOT FOUND! Creating a new);
folder.mkdirs ();
//System.out.println ("Configure log folder");
this.setHiddenFile (LOG_LOC);//set tmp directory to hidden folder
if (!f.exists ())
{
//System.out.println ("Creating a new Call Detailed Record...");//2009-01-23
f.createNewFile ();//2009-01-23
}
}
else
{
if (!f.exists ())
{
//System.out.println ("Creating a new Call Detailed Record...");//2009-01-23
f.createNewFile ();//2009-01-23
}
}
FileOutputStream tempFOS = new FileOutputStream (absoluteFileName, append);
if (tempFOS != null)
{
return new PrintWriter (tempFOS, autoFlush);
}
else
{
return null;
}
}
catch (Exception ex)
{
ex.printStackTrace ();
return null;
}
}
/**
* Set the given absolute file path as a hidden file.
* #param absoluteFile String
*/
private void setHiddenFile (String absoluteFile)
{
//set hidden file
//2009-01-22, KC
Runtime rt = Runtime.getRuntime ();
absoluteFile = absoluteFile.substring (0, absoluteFile.length () - 1);//2009-01-23
try
{
System.out.println (rt.exec ("attrib +H " + "\"" + absoluteFile + "\"").getInputStream ().toString ());
}
catch (IOException e)
{
e.printStackTrace ();
}
}
private String getDateTime ()
{
//2011-076-09, KC-format up to milliseconds to prevent duplicate PK in CDR table.
//return DateUtils.now ("yyyy/MM/dd HH:mm:ss");
return DateUtils.now ("yyyy/MM/dd HH:mm:ss:SSS");
//end 0609
}
private String getFilename ()
{
///return "CDR_" + port + ".dat";//2010-10-01
return port + ".dat";//2010-10-01
}
public void closePW ()
{
if (pw != null)
{
pw.close ();
}
}

You've created a FileOutputStream, but aren't closing that stream. Close that stream and try again. That might be causing the problem.
Messages are getting logged sometime because the garbage collector kicks in at some intervals and closes the FileOutStream. This then allows messages to be logged again. You're getting the unreachable error since you have a return statement in both the if & else blocks. You'll have to take the PrintWriter and FileOutStreamWriter out of the getPrintWriter put it where you usually call the getPrintWriter(). Then you'll be able to close the streams correctly. getPrintWriter should only ensure file exists, so rename it to ensureFileExistance

If you can use Apache Common IO, try this:
public synchronized void log(String connID, String hotline, String callerType,
String cli, String lastMenu, String lastInput,
String status, String reason) {
String absoluteFP = LOG_LOC + ls + this.getFilename();
File file = new File(absoluteFP);
String message = this.getDateTime() + "," + connID + "," + hotline + "," + callerType + "," + cli + "," + lastMenu + "," + lastInput + "," + status + "," + reason;
try {
// note that you must explicitly add new line character if you want the line to end with newline
FileUtils.write(file, message + "\n", "UTF-8", true);
} catch (IOException ex) {
ex.printStackTrace ();
}
}
In Common IO 2.1, you can append a file that you are writting to. You can now get rid of the closePW and getPrintwriter and since the log method is synchronized, the file can be written one at a time from the same object. However, if you try to write the same file from different object at the same time, you will end up having overwritting problem.
Also, Common IO create the missing parent folder for you automatically. There is no need to explicitly check and create the folder.