Develop Reference

Jar showing increased RAM activity after completing the task

I have a class KeywordCount which tokenizes a given sentence and tags it using a maxent tagger by Apache OpenNLP-POS tagger. I first tokenize the output and then feed it to the tagger. I have a problem of RAM usage of upto 165 MB after the jar has completed its tasks. The rest of the program just makes a DB call and checks for new tasks. I have isolated the leak to this class. You can safely ignore the Apache POI Excel code. I need to know if any of you can find the leak in the code.
public class KeywordCount {
Task task;
String taskFolder = "";
List<String> listOfWords;
public KeywordCount(String taskFolder) {
this.taskFolder = taskFolder;
listOfWords = new ArrayList<String>();
}
public void tagText() throws Exception {
String xlsxOutput = taskFolder + File.separator + "results_pe.xlsx";
FileInputStream fis = new FileInputStream(new File(xlsxOutput));
XSSFWorkbook wb = new XSSFWorkbook(fis);
XSSFSheet sheet = wb.createSheet("Keyword Count");
XSSFRow row = sheet.createRow(0);
Cell cell = row.createCell(0);
XSSFCellStyle csf = (XSSFCellStyle)wb.createCellStyle();
csf.setVerticalAlignment(CellStyle.VERTICAL_TOP);
csf.setBorderBottom(CellStyle.BORDER_THICK);
csf.setBorderRight(CellStyle.BORDER_THICK);
csf.setBorderTop(CellStyle.BORDER_THICK);
csf.setBorderLeft(CellStyle.BORDER_THICK);
Font fontf = wb.createFont();
fontf.setColor(IndexedColors.GREEN.getIndex());
fontf.setBoldweight(Font.BOLDWEIGHT_BOLD);
csf.setFont(fontf);
int rowNum = 0;
BufferedReader br = null;
InputStream modelIn = null;
POSModel model = null;
try {
modelIn = new FileInputStream("taggers" + File.separator + "en-pos-maxent.bin");
model = new POSModel(modelIn);
}
catch (IOException e) {
// Model loading failed, handle the error
e.printStackTrace();
}
finally {
if (modelIn != null) {
try {
modelIn.close();
}
catch (IOException e) {
}
}
}
File ftmp = new File(taskFolder + File.separator + "phrase_tmp.txt");
if(ftmp.exists()) {
br = new BufferedReader(new FileReader(ftmp));
POSTaggerME tagger = new POSTaggerME(model);
String line = "";
while((line = br.readLine()) != null) {
if (line.equals("")) {
break;
}
row = sheet.createRow(rowNum++);
if(line.startsWith("Match")) {
int index = line.indexOf(":");
line = line.substring(index + 1);
String[] sent = getTokens(line);
String[] tags = tagger.tag(sent);
for(int i = 0; i < tags.length; i++) {
if (tags[i].equals("NN") || tags[i].equals("NNP") || tags[i].equals("NNS") || tags[i].equals("NNPS")) {
listOfWords.add(sent[i].toLowerCase());
} else if (tags[i].equals("JJ") || tags[i].equals("JJR") || tags[i].equals("JJS")) {
listOfWords.add(sent[i].toLowerCase());
}
}
Map<String, Integer> treeMap = new TreeMap<String, Integer>();
for(String temp : listOfWords) {
Integer counter = treeMap.get(temp);
treeMap.put(temp, (counter == null) ? 1 : counter + 1);
}
listOfWords.clear();
sent = null;
tags = null;
if (treeMap != null || !treeMap.isEmpty()) {
for(Map.Entry<String, Integer> entry : treeMap.entrySet()) {
row = sheet.createRow(rowNum++);
cell = row.createCell(0);
cell.setCellValue(entry.getKey().substring(0, 1).toUpperCase() + entry.getKey().substring(1));
XSSFCell cell1 = row.createCell(1);
cell1.setCellValue(entry.getValue());
}
treeMap.clear();
}
treeMap = null;
}
rowNum++;
}
br.close();
tagger = null;
model = null;
}
sheet.autoSizeColumn(0);
fis.close();
FileOutputStream fos = new FileOutputStream(new File(xlsxOutput));
wb.write(fos);
fos.close();
System.out.println("Finished writing XLSX file for Keyword Count!!");
}
public String[] getTokens(String match) throws Exception {
InputStream modelIn = new FileInputStream("taggers" + File.separator + "en-token.bin");
TokenizerModel model = null;
try {
model = new TokenizerModel(modelIn);
}
catch (IOException e) {
e.printStackTrace();
}
finally {
if (modelIn != null) {
try {
modelIn.close();
}
catch (IOException e) {
}
}
}
Tokenizer tokenizer = new TokenizerME(model);
String tokens[] = tokenizer.tokenize(match);
model = null;
return tokens;
}
}
My system GCed the RAM after 165MB...but when I upload to the server the GC is not performed and it rises upto 480 MB(49% of RAM usage).

First of all, increased heap usage is not evidence of a memory leak. It may simply be that the GC has not run yet.
Having said that, it is doubtful that anyone can spot a memory leak just by "eyeballing" your code. The correct way to solve this is for >>you<< to read up on the techniques for finding Java memory leaks, and >>you<< then use the relevant tools (e.g. visualvm, jhat, etc) to search for the problem yourself.
Here are some references on finding storage leaks:
Troubleshooting Guide for Java SE 6 with HotSpot VM : Troubleshooting Memory Leaks. http://www.oracle.com/technetwork/java/javase/memleaks-137499.html - Note 1.
How to find a Java Memory Leak
Note 1: This link is liable to break. If it does, use Google to find the article.
I have isolated the leak to this class. You can safely ignore the Apache POI Excel code.
If we ignore the Apache POI code, the only source of a potential memory "leakage" is that the word list ( listOfWords ) is retained. (Calling clear() will null out its contents, but the backing array is retained, and that array's size is determined by the maximum list size. From a memory footprint perspective, it would be better to replace the list with a new empty list.)
However, that is only a "leak" if you keep a reference to the KeywordCount instance. And if you are doing that because you are using the instance, I wouldn't call that a leak at all.

Error in reading file java.io

So i'm trying to read the following string from the text file addToLibrary.txt
file:/Users/JEAcomputer/Music/iTunes/iTunes%20Media/Music/Flight%20Of%20The%20Conchords/Flight%20Of%20The%20Conchords/06%20Mutha'uckas.mp3
But when I do i get the following error:
java.io.FileNotFoundException: file:/Users/JEAcomputer/Music/iTunes/iTunes%20Media/Music/Flight%20Of%20The%20Conchords/Flight%20Of%20The%20Conchords/06%20Mutha'uckas.mp3 (No such file or directory)
Whats odd is that I got that string from a fileChooser using this method:
public static void addToLibrary(File f) {
String fileName = "addToLibrary.txt";
try {
FileWriter filewriter = new FileWriter(fileName, true);
BufferedWriter bufferedWriter = new BufferedWriter(filewriter);
bufferedWriter.newLine();
bufferedWriter.write(f.toURI().toString());
System.out.println("Your file has been written");
bufferedWriter.close();
} catch (IOException ex) {
System.out.println(
"Error writing to file '"
+ fileName + "'");
} finally {
}
}
An even stranger error is that my file reader can read things in another folder but not anything in iTunes Media.
I attempt to read all the files in the different folders with the following method:
public void getMusicDirectory() {
int index = 0;
try {
File[] contents = musicDir.listFiles();
//System.out.println(contents[3].toString());
for (int i = 0; i < contents.length; i++) {
//System.out.println("----------------------------------------"+contents.length);
String name = contents[i].getName();
//System.out.println(name);
if (name.indexOf(".mp3") == -1) {
continue;
}
FileInputStream file = new FileInputStream(contents[i]);
file.read();
System.out.println(contents[i].toURI().toString());
songsDir.add(new Song((new MediaPlayer(new Media(contents[i].toURI().toString()))), contents[i]));
file.close();
}
} catch (Exception e) {
System.out.println("Error -- " + e.toString());
}
try(BufferedReader br = new BufferedReader(new FileReader("addToLibrary.txt"))) {
//System.out.println("In check login try");
for (String line; (line = br.readLine()) != null; ) {
FileInputStream file = new FileInputStream(new File(line));
file.read();
songsDir.add(new Song(new MediaPlayer(new Media(line)), new File(line)));
file.close();
}
// line is not visible here.
} catch (Exception e) {
System.out.println("Error reading add to library-- " + e.toString());
}
}
So how can i make this work? why does the first part of the method work but not the second?

You are not having a problem reading the string
file:/Users/JEAcomputer/Music/iTunes/iTunes%20Media/Music/Flight%20Of%20The%20Conchords/Flight%20Of%20The%20Conchords/06%20Mutha'uckas.mp3
from a file. That part works fine. Your problem is after that, when you try to open the file with the path:
file:/Users/JEAcomputer/Music/iTunes/iTunes%20Media/Music/Flight%20Of%20The%20Conchords/Flight%20Of%20The%20Conchords/06%20Mutha'uckas.mp3
because that's not actually a path; it's a URI (although it can be converted to a path).
You could convert this to a path, in order to open it, but you have no reason to - your code doesn't actually read from the file (apart from the first byte, which it does nothing with) so there's no point in opening it. Delete the following lines:
FileInputStream file = new FileInputStream(contents[i]); // THIS ONE
file.read(); // THIS ONE
System.out.println(contents[i].toURI().toString());
songsDir.add(new Song((new MediaPlayer(new Media(contents[i].toURI().toString()))), contents[i]));
file.close(); // THIS ONE
and
FileInputStream file = new FileInputStream(new File(line)); // THIS ONE
file.read(); // THIS ONE
songsDir.add(new Song(new MediaPlayer(new Media(line)), new File(line)));
file.close(); // THIS ONE

file:/Users/JEAcomputer/Music/iTunes/iTunes%20Media/Music/Flight%20Of%20The%20Conchords/Flight%20Of%20The%20Conchords/06%20Mutha'uckas.mp3 is not a valid File reference, especially under Windows.
Since you've idendtified the String as a URI, you should treat it as such...
URI uri = URI.create("file:/Users/JEAcomputer/Music/iTunes/iTunes%20Media/Music/Flight%20Of%20The%20Conchords/Flight%20Of%20The%20Conchords/06%20Mutha'uckas.mp3");
Okay, but, there's no real way to read URI, but you can read a URL, so we need to convert the URI to URL, luckily, this is quite simple...
URL url = uri.toURL();
From there you can use URL#openStream to open an InputStream (which you can wrap in a InputStreamReader) and read the contents of the file, for example...
String imageFile = "file:/...";
URI uri = URI.create(imageFile);
try {
URL url = uri.toURL();
try (InputStream is = url.openStream()) {
byte[] bytes = new byte[1024 * 4];
int bytesRead = -1;
int totalBytesRead = 0;
while ((bytesRead = is.read(bytes)) != -1) {
// Somthing, something, something, bytes
totalBytesRead += bytesRead;
}
System.out.println("Read a total of " + totalBytesRead);
} catch (IOException ex) {
ex.printStackTrace();
}
} catch (MalformedURLException ex) {
ex.printStackTrace();
}
You could, however, save your self a lot of issues and stop using things like f.toURI().toString()); (File#toURI#toString) and simply use File#getPath instead...This would allow you to simply create a new File reference from the String...
Also, your resource management needs some work, basically, if you open it, you should close it. See The try-with-resources Statement for some more ideas

Java rename not working

Here is my code
private void edit(String search_bookname) {
String current_bookname ="", current_ISBN = "", current_author = "", current_rating = "", record = "", comma = ",", current_status = "";
int flag1 = 0, flag2 = 0;
File file = new File("Book_data.txt");
try {
BufferedReader reader = new BufferedReader (new FileReader (file));
File f = new File("Book_data_copy.txt");
FileWriter create = new FileWriter(f);
PrintWriter y = new PrintWriter(create);
while(reader.ready())
{
record = reader.readLine();
StringTokenizer st = new StringTokenizer(record, ",");
while(st.hasMoreTokens()) {
current_bookname = st.nextToken();
current_author = st.nextToken();
current_ISBN = st.nextToken();
current_rating = st.nextToken();
current_status = st.nextToken();
flag2 = 0;
if (search_bookname.equals(current_bookname)) {
flag1 = 1;
flag2 = 1;
try {
y.print(current_bookname); y.print(comma);
y.print(current_author); y.print(comma);
y.print(current_ISBN); y.print(comma);
y.print(current_rating); y.print(comma);
y.println("Borrowed");
} catch(Exception e) {}
Thread.sleep(1000);
}
}
if(flag2==0) // All non-matching records shall only be written to file. Record to be deleted will not be written to new file
{
y.print(current_bookname); y.print(comma); //One record per line..... Each field in a record is seperated by COMMA (" , " )
y.print(current_author); y.print(comma);
y.print(current_ISBN); y.print(comma);
y.print(current_rating);y.print(comma);
y.println(current_status);
}
}
reader.close();
y.close();
create.close();
} catch (Exception e) {}
if(flag1==1) //Rename File ONLY when record has been found for Edit
{
File oldFileName = new File("Book_data_copy.txt");
File newFileName = new File("Book_data.txt");
System.out.println("File renamed .....................");
try
{
newFileName.delete(); oldFileName.renameTo(newFileName);
if (oldFileName.renameTo(newFileName))
System.out.println("File renamed successfull !");
else
System.out.println("File rename operation failed !");
Thread.sleep(1000);
} catch (Exception e) {}
}
}
The project is for a library system. I am relatively new to java and I use netbeans on windows 8.1. The code outputs rename operation failed. Almost exactly the same code for edit has been used before in the program and it worked.
Any suggestions or code corrections would be helpfu.
Thanks!

Your issue is with the below code section
oldFileName.renameTo(newFileName);
if (oldFileName.renameTo(newFileName))
you are trying to rename twice. The first might pass but the 2nd will definitely fail. Check if the original file was renamed. If any error is thrown pring the stack trace and add the trace to your post.

You are trying to rename oldFileName twice:
newFileName.delete(); oldFileName.renameTo(newFileName); // rename
if (oldFileName.renameTo(newFileName)) // rename again ?!
System.out.println("File renamed successfull !");
else
System.out.println("File rename operation failed !");
The second rename in the if() fails, because the file was renamed already in the line before.

There are any number of problems here, starting with ignoring exceptions; not closing the file in a finally block; and using Reader.ready() incorrectly.

This is 2015. Use java.nio.file!
final Path srcFile = Paths.get("Book_data.txt").toAbsolutePath();
final Path dstFile = srcFile.resolveSibling("Book_data_copy.txt");
try (
final BufferedReader reader = Files.newBufferedReader(srcFile, StandardCharsets.UTF_8);
final BufferedWriter writer = Files.newBufferedWriter(dstFile, StandardCharsets.UTF_8,
StandardOpenOption.CREATE, StandardOpenOption.TRUNCATE_EXISTING);
final PrintWriter pw = new PrintWriter(writer);
) {
// work with reader and pw
}
if (flag1 == 1)
Files.move(dstFile, srcFile, StandardCopyOption.REPLACE_EXISTING,
StandardCopyOption.ATOMIC_MOVE);
If you did that from the start, not only would your resources have been closed safely (your code doesn't do that) but you would have gotten a meaningful exception as well (you are trying to rename twice; you would have had a NoSuchFileException on the second rename).
Relying on File is an error, and it has always been.

Android/Java - When a file exists, rename the file

I can't seem to get my logic right, I'm trying to rename a file to "photo2.jpg" if, say "photo.jpg" and "photo1.jpg" exists, and so on.
At the moment when I run my code, and I take a picture, only "photo.jpg" and "photo1.jpg" ever exist, and then they get written over if a third and fourth, etc. photo is taken.
String photoName = "photo.jpg";
String i = "0";
int num = 0;
File photo = new File(Environment.getExternalStorageDirectory(), photoName);
//for (File file : photo.listFiles()){
while(photo.exists()) {
//if(file.getName().equals("photo.jpg")){
//photo.delete();
num = Integer.parseInt(i);
++num;
String concatenatedNum = Integer.toString(num);
StringBuffer insertNum = new StringBuffer(photoName);
insertNum.insert(5, concatenatedNum);
photoName = insertNum.toString();
photo.renameTo(new File(Environment.getExternalStorageDirectory(), photoName));
//}
}
try {
FileOutputStream fos=new FileOutputStream(photo.getPath());
//MediaStore.Images.Media.insertImage(getContentResolver(), yourBitmap, yourTitle, yourDescription);
//write jpeg to local drive
fos.write(jpeg[0]);
fos.close();
}
catch (java.io.IOException e) {}
Thanks for your time and help!
EDIT: Half solved: I realized I was overwriting the file instead of creating a NEW file. Now I can take multiple pictures and they are saved as their own file. However, the naming of the files is now:
photo.jpg
photo1.jpg
photo11.jpg
photo111.jpg, etc.

You always base your filename on i, but you never change the value of i when you find that number is used.

I know this is older, but I ended up here when I was looking for a solution.
I ended up doing the following:
String baseFilename = "photo";
File outputFile = new File(Environment.getExternalStorageDirectory(), baseFilename + ".jpg");
int i = 2; // whatever increment you want to start with, I'm copying Windows' naming convention
while (outputFile.exists()){
outputFile = new File(Environment.getExternalStorageDirectory(), baseFilename + "(" + i + ")" + ".jpg");
i++;
}
You will end up with photo.jpg, photo(2).jpg, photo(3).jpg, etc.
Obviously you can easily change how the int is appended, but like I said I just decided to follow how Windows does it.

private void savePhoto(String fileName, final String extension)
{
// First, get all the file names from the directory
String[] allFiles = new File(Environment.getExternalStorageDirectory().toString()).list();
// Create a new empty list to put all the matching file names in
// In this case all the files names that are "photo.jpg" or "photo#.jpg"
ArrayList<String> files = new ArrayList<String>();
// Skim through all the files
for(String file : allFiles)
{
// Use a regular expression to find matching files
// fileName[0-9]+\.extension|fileName\.extension
if(file.matches(fileName + "[0-9]+\\." + extension + "|" + fileName + "\\." + extension))
{
files.add(file);
}
}
files.trimToSize();
// Now sift through the list and find out what the highest number is
// Example, if you've taken 8 photos, then highestNumber will equal 8
int highestNumber = 0;
int digit;
for(String file : files)
{
try
{
digit = Integer.parseInt(file.replace(fileName, "").replace("." + extension, ""));
}
catch(NumberFormatException e)
{
digit = 1;
}
if(digit > highestNumber)
{
highestNumber = digit;
}
}
// Create the file object
fileName = fileName + highestNumber++ + "." + extension;
File file = new File(Environment.getExternalStorageDirectory().toString(), fileName);
// In not sure what you do around here, I can't find any array titled "jpeg"
// So do what you will
FileOutputStream fostream = null;
try
{
fostream = new FileOutputStream(file);
//MediaStore.Images.Media.insertImage(getContentResolver(), yourBitmap, yourTitle, yourDescription);
//write jpeg to local drive
fostream.write(jpeg[0]);
}
catch(IOException e)
{
e.printStackTrace();
}
finally
{
if(fostream != null)
{
try
{
fostream.flush();
fostream.close();
}
catch(IOException e)
{
e.printStackTrace();
}
}
}
}

how to read a csv file from sd card or specified path in android [duplicate]

This question already has answers here:
Closed 11 years ago.
Possible Duplicate:
reading a specific file from sdcard in android
I'm trying to make a simple android app that basically imports a csv and inserts it to my database table. So far, I was able to read a csv file inside the res folder.
my sample csv file is named "test.csv" and is basically accessed through "InputStream is = this.getResources().openRawResource(R.drawable.test);".
Here's my sample code:
InputStream is = this.getResources().openRawResource
(R.drawable.test);
BufferedReader reader = new BufferedReader(new InputStreamReader
(is));
try {
String line;
String brand = "";
String model = "";
String type = "";
this.dh = new DataHelper(this);
//this.dh.deleteAllCar();
while ((line = reader.readLine()) != null) {
// do something with "line"
String[] RowData = line.split(",");
brand = RowData[1];
model = RowData[2];
type = RowData[3];
this.dh = new DataHelper(this);
//this.dh.deleteAllCar();
this.dh.insertcsv(brand, model, type);
}
}catch (IOException ex) {
// handle exception
}finally {
try {
is.close();
}
catch (IOException e) {
// handle exception
}
}
This works fine however, I want to be able to make a feature wherein the user can specify where to get the file(like from phone's sdcard, etc). But for now, I wanted to know how to access the csv from sdcard(mnt/sdcard/test.csv).
Help will be highly appreciated! thanks and happy coding!

Reading a file from an SDCard has been covered on Stack Overflow previously.
Here's the link:
reading a specific file from sdcard in android

Here is code on how to write to the SD Card, you should be able to figure out the read part using your code above:
private void writeToSDCard() {
try {
File root = Environment.getExternalStorageDirectory();
if (root.canWrite()){
InputStream from = myContext.getResources().openRawResource(rID);
File dir = new java.io.File (root, "pdf");
dir.mkdir();
File writeTo = new File(root, "pdf/" + attachmentName);
FileOutputStream to = new FileOutputStream(writeTo);
byte[] buffer = new byte[4096];
int bytesRead;
while ((bytesRead = from.read(buffer)) != -1)
to.write(buffer, 0, bytesRead); // write
to.close();
from.close();
} else {
Log.d(TAG, "Unable to access SD card.");
}
} catch (Exception e) {
Log.d(TAG, "writeToSDCard: " + e.getMessage());
}
}
}