Develop Reference

Android Reading a large text efficiently in Java

My code is too slow
How can I make my code efficiently? Currently the code needs several minutes until the file was read, which is way too long. Can this be done faster? There is no stacktrace, because it works, but too slow.
Thanks!
The Problem Code:
private void list(){
String strLine2="";
wwwdf2 = new StringBuffer();
InputStream fis2 = this.getResources().openRawResource(R.raw.list);
BufferedReader br2 = new BufferedReader(new InputStreamReader(fis2));
if(fis2 != null) {
try {
LineNumberReader lnr = new LineNumberReader(br2);
String linenumber = String.valueOf(lnr);
int i=0;
while (i!=1) {
strLine2 = br2.readLine();
wwwdf2.append(strLine2 + "\n");
String contains = String.valueOf(wwwdf2);
if(contains.contains("itisdonecomplet")){
i++;
}
}
// Toast.makeText(getApplicationContext(), strLine2, Toast.LENGTH_LONG).show();
Toast.makeText(getApplicationContext(), wwwdf2, Toast.LENGTH_LONG).show();
} catch (IOException e) {
e.printStackTrace();
}
}
}

Use StringBuilder instead of StringBuffer.
StringBuffer is synchronized, and you don't need that.
Don't use String.valueOf, which builds a string, negating the value using a StringBuffer/Builder. You are building a string from the whole buffer, checking it, discarding the string, then constructing nearly the same string again.
Use if (wwwdf2.indexOf("itisdonecomplet") >= 0) instead, which avoids creating the string.
But this will still be reasonably slow, as although you would not be constructing a string and searching through it all, you are still doing the searching.
You can make this a lot faster by only searching the very end of the string. For example, you could use wwwdf2.indexOf("itisdonecomplet", Math.max(0, wwwdf2.length() - strLine2.length() - "itisdonecomplet".length())).
Although, as blackapps points out in a comment, you could simply check if strLine2 contains that string.
Don't use string concatenation inside a call to append: make two separate calls.
wwwdf2.append(strLine2);
wwwdf2.append("\n");
You don't check if you reach the end of the file. Check if strLine2 is null, and break the loop if it is.

My new Created code:(My test device is a Samsung S8)
private void list(){
String strLine2="";
wwwdf2 = new StringBuilder();
InputStream fis2 = this.getResources().openRawResource(R.raw.list);
BufferedReader br2 = new BufferedReader(new InputStreamReader(fis2));
if(fis2 != null) {
try {
LineNumberReader lnr = new LineNumberReader(br2);
String linenumber = String.valueOf(lnr);
int i=0;
while (i!=1) {
strLine2 = br2.readLine();
wwwdf2.append(strLine2);
wwwdf2.append("\n");
if (wwwdf2.indexOf("itisdonecomplet") >= 0){
i++;
}
}
// Toast.makeText(getApplicationContext(), strLine2, Toast.LENGTH_LONG).show();
Toast.makeText(getApplicationContext(), wwwdf2, Toast.LENGTH_LONG).show();
} catch (IOException e) {
e.printStackTrace();
}
}
}

How can I read from the next line of a text file, and pause, allowing me to read from the line after that later?

I wrote a program that generates random numbers into two text files and random letters into a third according the two constant files. Now I need to read from each text file, line by line, and put them together. The program is that the suggestion found here doesn't really help my situation. When I try that approach it just reads all lines until it's done without allowing me the option to pause it, go to a different file, etc.
Ideally I would like to find some way to read just the next line, and then later go to the line after that. Like maybe some kind of variable to hold my place in reading or something.
public static void mergeProductCodesToFile(String prefixFile,
String inlineFile,
String suffixFile,
String productFile) throws IOException
{
try (BufferedReader br = new BufferedReader(new FileReader(prefixFile)))
{
String line;
while ((line = br.readLine()) != null)
{
try (PrintWriter out = new PrintWriter(new BufferedWriter(new FileWriter(productFile, true))))
{
out.print(line); //This will print the next digit to the right
}
catch (FileNotFoundException e)
{
System.err.println("File error: " + e.getMessage());
}
}
}
}
EDIT: The digits being created according to the following. Basically, constants tell it how many digits to create in each line and how many lines to create. Now I need to combine these together without deleting anything from either text file.
public static void writeRandomCodesToFile(String codeFile,
char fromChar, char toChar,
int numberOfCharactersPerCode,
int numberOfCodesToGenerate) throws IOException
{
for (int i = 1; i <= PRODUCT_COUNT; i++)
{
int I = 0;
if (codeFile == "inline.txt")
{
for (I = 1; I <= CHARACTERS_PER_CODE; I++)
{
int digit = (int)(Math.random() * 10);
try (PrintWriter out = new PrintWriter(new BufferedWriter(new FileWriter(codeFile, true))))
{
out.print(digit); //This will print the next digit to the right
}
catch (FileNotFoundException e)
{
System.err.println("File error: " + e.getMessage());
System.exit(1);
}
}
}
if ((codeFile == "prefix.txt") || (codeFile == "suffix.txt"))
{
for (I = 1; I <= CHARACTERS_PER_CODE; I++)
{
Random r = new Random();
char digit = (char)(r.nextInt(26) + 'a');
digit = Character.toUpperCase(digit);
try (PrintWriter out = new PrintWriter(new BufferedWriter(new FileWriter(codeFile, true))))
{
out.print(digit);
}
catch (FileNotFoundException e)
{
System.err.println("File error: " + e.getMessage());
System.exit(1);
}
}
}
//This will take the text file to the next line
if (I >= CHARACTERS_PER_CODE)
{
{
Random r = new Random();
char digit = (char)(r.nextInt(26) + 'a');
try (PrintWriter out = new PrintWriter(new BufferedWriter(new FileWriter(codeFile, true))))
{
out.println(""); //This will return a new line for the next loop
}
catch (FileNotFoundException e)
{
System.err.println("File error: " + e.getMessage());
System.exit(1);
}
}
}
}
System.out.println(codeFile + " was successfully created.");
}// end writeRandomCodesToFile()

Being respectfull with your code, it will be something like this:
public static void mergeProductCodesToFile(String prefixFile, String inlineFile, String suffixFile, String productFile) throws IOException {
try (BufferedReader prefixReader = new BufferedReader(new FileReader(prefixFile));
BufferedReader inlineReader = new BufferedReader(new FileReader(inlineFile));
BufferedReader suffixReader = new BufferedReader(new FileReader(suffixFile))) {
StringBuilder line = new StringBuilder();
String prefix, inline, suffix;
while ((prefix = prefixReader.readLine()) != null) {
//assuming that nothing fails and the files are equals in # of lines.
inline = inlineReader.readLine();
suffix = suffixReader.readLine();
line.append(prefix).append(inline).append(suffix).append("\r\n");
// write it
...
}
} finally {/*close writers*/}
}
Some exceptions may be thrown.
I hope you don't implement it in one single method.
You can make use of iterators too, or a very simple reader class (method).
I wouldn't use List to load the data at least I guarantee that the files will be low sized and that I can spare the memory usage.

My approach as we discussed by storing the data and interleaving it. Like Sergio said in his answer, make sure memory isn't a problem in terms of the size of the file and how much memory the data structures will use.
//the main method we're working on
public static void mergeProductCodesToFile(String prefixFile,
String inlineFile,
String suffixFile,
String productFile) throws IOException
{
try {
List<String> prefix = read(prefixFile);
List<String> inline = read(inlineFile);
List<String> suffix = read(productFile);
String fileText = interleave(prefix, inline, suffix);
//write the single string to file however you want
} catch (...) {...}//do your error handling...
}
//helper methods and some static variables
private static Scanner reader;//I just prefer scanner. Use whatever you want.
private static StringBuilder sb;
private static List<String> read(String filename) throws IOException
{
List<String> list = new ArrayList<String>;
try (reader = new Scanner(new File(filename)))
{
while(reader.hasNext())
{ list.add(reader.nextLine()); }
} catch (...) {...}//catch errors...
}
//I'm going to build the whole file in one string, but you could also have this method return one line at a time (something like an iterator) and output it to the file to avoid creating the massive string
private static String interleave(List<String> one, List<String> two, List<String> three)
{
sb = new StringBuilder();
for (int i = 0; i < one.size(); i++)//notice no checking on size equality of words or the lists. you might want this
{
sb.append(one.get(i)).append(two.get(i)).append(three.get(i)).append("\n");
}
return sb.toString()
}
Obviously there is still some to be desired in terms of memory and performance; additionally there are ways to make this slightly more extensible to other situations, but it's a good starting point. With c#, I could more easily make use of the iterator to make interleave give you one line at a time, potentially saving memory. Just a different idea!

Unusual output from using split()

I'm reading from a file that has the following format:
name : symptoms : causes : treatments : rate : prognosis
There are a total of 21 entries but when I try to read from the file and use .split(":");, the output changes each time but is always along the lines of: [Ljava.lang.String;#614951ff. I'm guessing it's the pointer or memory address but I want the String value. I'm not getting any exceptions though so I'm not sure where I've gone wrong. The purpose of the method is to read the file and split into an array using the delimiter for the given file row selected.
public String[] readCancer(int row) {
cancers = new String[22];
FileInputStream fis;
InputStreamReader isr;
BufferedReader br = null;
String eachCancer;
String[] splitCancer = null;
int j = 0;
try {
fis = new FileInputStream(myData);
isr = new InputStreamReader(fis);
br = new BufferedReader(isr);
input = new Scanner(br);
while(input.hasNext() && j < 23) {
cancers[j++] = input.nextLine();
}
eachCancer = cancers[row].toString();
splitCancer = eachCancer.split(":");
} catch (IOException iox) {
JOptionPane.showMessageDialog(null, "Problem with file input");
} finally {
try {
if(br != null) {
br.close();
}
} catch (IOException iox) {
JOptionPane.showMessageDialog(null, "Problem closing the file");
}
}
return splitCancer;
}

To print the contents of array :
1) System.out.println(Arrays.deepToString(splitCancer));
2) System.out.println(Arrays.toString(splitCancer));
3) System.out.println(Arrays.asList(splitCancer));

If you want to display the string array, you should use:
System.out.println(Arrays.toString(splitCancer));
Because when you print splitCancer you'll get the address of the array and not the content of it.
Of course you can print the content in other ways:
for(String str : splitCancer) {
System.out.println(str);
}

Currently I have the following:
public String[] readCancer() {
cancers = new String[22];
split = new String[22];
FileInputStream fis;
InputStreamReader isr;
BufferedReader br = null;
String eachCancer;
int j = 0;
try {
fis = new FileInputStream(myData);
isr = new InputStreamReader(fis);
br = new BufferedReader(isr);
input = new Scanner(br);
while(input.hasNext() && j < 23) {
cancers[j] = input.nextLine().toString();
//cancers[j] = input.nextLine();
split[j] = cancers[j].split(":");
//split[j] = "blah"; this line works
j++;
}
System.out.println(Arrays.toString(split));
} catch (IOException iox) {
JOptionPane.showMessageDialog(null, "Problem with file input");
} finally {
try {
if(br != null) {
br.close();
}
} catch (IOException iox) {
JOptionPane.showMessageDialog(null, "Problem closing the file");
}
}
return split;
//return split[j]; does not work
}
In my while loop, I keep getting compile errors saying it requires a String but found Stirng[] for split. When I try something simpler, such as split[j] = "blah";, there are no compile errors. I can return cancers perfectly but I need to split by the : delimiter and that seems to be something I cant get my head around. When I try return split[j], then I get another compile error saying it requires a String[] but found String. I've been at this for more than an hour, read through examples in my textbook and tutorials online for using split but it still isn't working. This is the only part of my program that I'm not sure how to do.

I tried pasting the entire file but it came a horrid block of text, so here are 2 lines from it. Each line has the same format but differing lengths:
The general format of the file is name : symptoms : causes : treatment : rate : prognosis
The rate is a String since it is unknown for some diseases and when it is known, the rate is not always out of 250,000. Sometimes it is out of 1,000,000 or 100,000, etc... .
acute promyelocytic leukemia : easy bruising, rapid internal bleeding, fatigue, anemia, frequent fever, infection, blood clots : PML and RARA genes : Medications, chemotherapy : 1 in 250,000 : Good
familial cylindromatosis : numerous skin tumours, ulcers, infection, impaired eyesight, hearing, smell, balance : CYLD gene : Surgery, chemotherapy : Unknown : Unknown
My most recent code attempt is at Unusual output from using split()
The 2 arrays of cancers and split are private String[] as field variables declared outside any of the methods. The variable myData is a private File also declared as a field variable outside any of the methods. I have checked and already verified the file path is correct.
The main method that calls the method:
public static void main(String[] args) {
CancerGUI _gui = new CancerGUI();
String[] resultCancer;
resultCancer = _gui.readCancer();
//System.out.println(resultCancer);
System.out.println(Arrays.toString(resultCancer));
}
I am only calling it in the main method to test whether it correctly returns the String[]. Once it does, then I will call it in a different method that adds the data to a GUI (this part I am reasonably confident I know how to do and have examples from my instructor and textbook to follow).

Print data from file to array

I need to have this file print to an array, not to screen.And yes, I MUST use an array - School Project - I'm very new to java so any help is appreciated. Any ideas? thanks
import java.io.BufferedReader;
import java.io.File;
import java.io.FileReader;
import java.io.FileNotFoundException;
import java.util.ArrayList;
import java.util.Scanner;
public class HangmanProject
{
public static void main(String[] args) throws FileNotFoundException
{
String scoreKeeper; // to keep track of score
int guessesLeft; // to keep track of guesses remaining
String wordList[]; // array to store words
Scanner keyboard = new Scanner(System.in); // to read user's input
System.out.println("Welcome to Hangman Project!");
// Create a scanner to read the secret words file
Scanner wordScan = null;
try {
wordScan = new Scanner(new BufferedReader(new FileReader("words.txt")));
while (wordScan.hasNext()) {
System.out.println(wordScan.next());
}
} finally {
if (wordScan != null) {
wordScan.close();
}
}
}
}

Nick, you just gave us the final piece of the puzzle. If you know the number of lines you will be reading, you can simply define an array of that length before you read the file
Something like...
String[] wordArray = new String[10];
int index = 0;
String word = null; // word to be read from file...
// Use buffered reader to read each line...
wordArray[index] = word;
index++;
Now that example's not going to mean much to be honest, so I did these two examples
The first one uses the concept suggested by Alex, which allows you to read an unknown number of lines from the file.
The only trip up is if the lines are separated by more the one line feed (ie there is a extra line between words)
public static void readUnknownWords() {
// Reference to the words file
File words = new File("Words.txt");
// Use a StringBuilder to buffer the content as it's read from the file
StringBuilder sb = new StringBuilder(128);
BufferedReader reader = null;
try {
// Create the reader. A File reader would be just as fine in this
// example, but hay ;)
reader = new BufferedReader(new FileReader(words));
// The read buffer to use to read data into
char[] buffer = new char[1024];
int bytesRead = -1;
// Read the file to we get to the end
while ((bytesRead = reader.read(buffer)) != -1) {
// Append the results to the string builder
sb.append(buffer, 0, bytesRead);
}
// Split the string builder into individal words by the line break
String[] wordArray = sb.toString().split("\n");
System.out.println("Read " + wordArray.length + " words");
} catch (Exception e) {
e.printStackTrace();
} finally {
try {
reader.close();
} catch (Exception e) {
}
}
}
The second demonstrates how to read the words into an array of known length. This is probably closer to the what you actually want
public static void readKnownWords()
// This is just the same as the previous example, except we
// know in advance the number of lines we will be reading
File words = new File("Words.txt");
BufferedReader reader = null;
try {
// Create the word array of a known quantity
// The quantity value could be defined as a constant
// ie public static final int WORD_COUNT = 10;
String[] wordArray = new String[10];
reader = new BufferedReader(new FileReader(words));
// Instead of reading to a char buffer, we are
// going to take the easy route and read each line
// straight into a String
String text = null;
// The current array index
int index = 0;
// Read the file till we reach the end
// ps- my file had lots more words, so I put a limit
// in the loop to prevent index out of bounds exceptions
while ((text = reader.readLine()) != null && index < 10) {
wordArray[index] = text;
index++;
}
System.out.println("Read " + wordArray.length + " words");
} catch (Exception e) {
e.printStackTrace();
} finally {
try {
reader.close();
} catch (Exception e) {
}
}
}
If you find either of these useful, I would appropriate it you would give me a small up-vote and check Alex's answer as correct, as it's his idea that I've adapted.
Now, if you're really paranoid about which line break to use, you can find the values used by the system via the System.getProperties().getProperty("line.separator") value.

Do you need more help with the reading the file, or getting the String to a parsed array? If you can read the file into a String, simply do:
String[] words = readString.split("\n");
That will split the string at each line break, so assuming this is your text file:
Word1
Word2
Word3
words will be: {word1, word2, word3}

If the words you are reading are stored in each line of the file, you can use the hasNextLine() and nextLine() to read the text one line at a time. Using the next() will also work, since you just need to throw one word in the array, but nextLine() is usually always preferred.
As for only using an array, you have two options:
You either declare a large array, the size of whom you are sure will never be less than the total amount of words;
You go through the file twice, the first time you read the amount of elements, then you initialize the array depending on that value and then, go through it a second time while adding the string as you go by.
It is usually recommended to use a dynamic collection such as an ArrayList(). You can then use the toArray() method to turnt he list into an array.

Modify a .txt file in Java

I have a text file that I want to edit using Java. It has many thousands of lines. I basically want to iterate through the lines and change/edit/delete some text. This will need to happen quite often.
From the solutions I saw on other sites, the general approach seems to be:
Open the existing file using a BufferedReader
Read each line, make modifications to each line, and add it to a StringBuilder
Once all the text has been read and modified, write the contents of the StringBuilder to a new file
Replace the old file with the new file
This solution seems slightly "hacky" to me, especially if I have thousands of lines in my text file.
Anybody know of a better solution?

I haven't done this in Java recently, but writing an entire file into memory seems like a bad idea.
The best idea that I can come up with is open a temporary file in writing mode at the same time, and for each line, read it, modify if necessary, then write into the temporary file. At the end, delete the original and rename the temporary file.
If you have modify permissions on the file system, you probably also have deleting and renaming permissions.

if the file is just a few thousand lines you should be able to read the entire file in one read and convert that to a String.
You can use apache IOUtils which has method like the following.
public static String readFile(String filename) throws IOException {
File file = new File(filename);
int len = (int) file.length();
byte[] bytes = new byte[len];
FileInputStream fis = null;
try {
fis = new FileInputStream(file);
assert len == fis.read(bytes);
} catch (IOException e) {
close(fis);
throw e;
}
return new String(bytes, "UTF-8");
}
public static void writeFile(String filename, String text) throws IOException {
FileOutputStream fos = null;
try {
fos = new FileOutputStream(filename);
fos.write(text.getBytes("UTF-8"));
} catch (IOException e) {
close(fos);
throw e;
}
}
public static void close(Closeable closeable) {
try {
closeable.close();
} catch(IOException ignored) {
}
}

You can use RandomAccessFile in Java to modify the file on one condition:
The size of each line has to be fixed otherwise, when new string is written back, it might override the string in the next line.
Therefore, in my example, I set the line length as 100 and padding with space string when creating the file and writing back to the file.
So in order to allow update, you need to set the length of line a little larger than the longest length of the line in this file.
public class RandomAccessFileUtil {
public static final long RECORD_LENGTH = 100;
public static final String EMPTY_STRING = " ";
public static final String CRLF = "\n";
public static final String PATHNAME = "/home/mjiang/JM/mahtew.txt";
/**
* one two three
Text to be appended with
five six seven
eight nine ten
*
*
* #param args
* #throws IOException
*/
public static void main(String[] args) throws IOException
{
String starPrefix = "Text to be appended with";
String replacedString = "new text has been appended";
RandomAccessFile file = new RandomAccessFile(new File(PATHNAME), "rw");
String line = "";
while((line = file.readLine()) != null)
{
if(line.startsWith(starPrefix))
{
file.seek(file.getFilePointer() - RECORD_LENGTH - 1);
file.writeBytes(replacedString);
}
}
}
public static void createFile() throws IOException
{
RandomAccessFile file = new RandomAccessFile(new File(PATHNAME), "rw");
String line1 = "one two three";
String line2 = "Text to be appended with";
String line3 = "five six seven";
String line4 = "eight nine ten";
file.writeBytes(paddingRight(line1));
file.writeBytes(CRLF);
file.writeBytes(paddingRight(line2));
file.writeBytes(CRLF);
file.writeBytes(paddingRight(line3));
file.writeBytes(CRLF);
file.writeBytes(paddingRight(line4));
file.writeBytes(CRLF);
file.close();
System.out.println(String.format("File is created in [%s]", PATHNAME));
}
public static String paddingRight(String source)
{
StringBuilder result = new StringBuilder(100);
if(source != null)
{
result.append(source);
for (int i = 0; i < RECORD_LENGTH - source.length(); i++)
{
result.append(EMPTY_STRING);
}
}
return result.toString();
}
}

If the file is large, you might want to use a FileStream for output, but that seems pretty much like it is the simplest process to do what you're asking (and without more specificity i.e. on what types of changes / edits / deletions you're trying to do, it's impossible to determine what more complicated way might work).

No reason to buffer the entire file.
Simply write each line as your read it, insert lines when necessary, delete lines when necessary, replace lines when necessary.
Fundamentally, you will not get around having to recreate the file wholesale, especially if it's just a text file.

What kind of data is it? Do you control the format of the file?
If the file contains name/value pairs (or similar), you could have some luck with Properties, or perhaps cobbling together something using a flat file JDBC driver.
Alternatively, have you considered not writing the data so often? Operating on an in-memory copy of your file should be relatively trivial. If there are no external resources which need real time updates of the file, then there is no need to go to disk every time you want to make a modification. You can run a scheduled task to write periodic updates to disk if you are worried about data backup.

In general you cannot edit the file in place; it's simply a very long sequence of characters, which happens to include newline characters. You could edit in place if your changes don't change the number of characters in each line.

Can't you use regular expressions, if you know what you want to change ? Jakarta Regexp should probably do the trick.

Although this question was a time ago posted, I think it is good to put my answer here.
I think that the best approach is to use FileChannel from java.nio.channels package in this scenario. But this, only if you need to have a good performance! You would need to get a FileChannel via a RandomAccessFile, like this:
java.nio.channels.FileChannel channel = new java.io.RandomAccessFile("/my/fyle/path", "rw").getChannel();
After this, you need a to create a ByteBuffer where you will read from the FileChannel.
this looks something like this:
java.nio.ByteBuffer inBuffer = java.nio.ByteBuffer.allocate(100);
int pos = 0;
int aux = 0;
StringBuilder sb = new StringBuilder();
while (pos != -1) {
aux = channel.read(inBuffer, pos);
pos = (aux != -1) ? pos + aux : -1;
b = inBuffer.array();
sb.delete(0, sb.length());
for (int i = 0; i < b.length; ++i) {
sb.append((char)b[i]);
}
//here you can do your stuff on sb
inBuffer = ByteBuffer.allocate(100);
}
Hope that my answer will help you!

I think, FileOutputStream.getFileChannel() will help a lot, see FileChannel api
http://java.sun.com/javase/6/docs/api/java/nio/channels/FileChannel.html

private static void modifyFile(String filePath, String oldString, String newString) {
File fileToBeModified = new File(filePath);
StringBuilder oldContent = new StringBuilder();
try (BufferedReader reader = new BufferedReader(new FileReader(fileToBeModified))) {
String line = reader.readLine();
while (line != null) {
oldContent.append(line).append(System.lineSeparator());
line = reader.readLine();
}
String content = oldContent.toString();
String newContent = content.replaceAll(oldString, newString);
try (FileWriter writer = new FileWriter(fileToBeModified)) {
writer.write(newContent);
}
} catch (IOException e) {
e.printStackTrace();
}
}

You can change the txt file to java by saving on clicking "Save As" and saving *.java extension.