I have a text file that I want to edit using Java. It has many thousands of lines. I basically want to iterate through the lines and change/edit/delete some text. This will need to happen quite often.
From the solutions I saw on other sites, the general approach seems to be:
Open the existing file using a BufferedReader
Read each line, make modifications to each line, and add it to a StringBuilder
Once all the text has been read and modified, write the contents of the StringBuilder to a new file
Replace the old file with the new file
This solution seems slightly "hacky" to me, especially if I have thousands of lines in my text file.
Anybody know of a better solution?
I haven't done this in Java recently, but writing an entire file into memory seems like a bad idea.
The best idea that I can come up with is open a temporary file in writing mode at the same time, and for each line, read it, modify if necessary, then write into the temporary file. At the end, delete the original and rename the temporary file.
If you have modify permissions on the file system, you probably also have deleting and renaming permissions.
if the file is just a few thousand lines you should be able to read the entire file in one read and convert that to a String.
You can use apache IOUtils which has method like the following.
public static String readFile(String filename) throws IOException {
File file = new File(filename);
int len = (int) file.length();
byte[] bytes = new byte[len];
FileInputStream fis = null;
try {
fis = new FileInputStream(file);
assert len == fis.read(bytes);
} catch (IOException e) {
close(fis);
throw e;
}
return new String(bytes, "UTF-8");
}
public static void writeFile(String filename, String text) throws IOException {
FileOutputStream fos = null;
try {
fos = new FileOutputStream(filename);
fos.write(text.getBytes("UTF-8"));
} catch (IOException e) {
close(fos);
throw e;
}
}
public static void close(Closeable closeable) {
try {
closeable.close();
} catch(IOException ignored) {
}
}
You can use RandomAccessFile in Java to modify the file on one condition:
The size of each line has to be fixed otherwise, when new string is written back, it might override the string in the next line.
Therefore, in my example, I set the line length as 100 and padding with space string when creating the file and writing back to the file.
So in order to allow update, you need to set the length of line a little larger than the longest length of the line in this file.
public class RandomAccessFileUtil {
public static final long RECORD_LENGTH = 100;
public static final String EMPTY_STRING = " ";
public static final String CRLF = "\n";
public static final String PATHNAME = "/home/mjiang/JM/mahtew.txt";
/**
* one two three
Text to be appended with
five six seven
eight nine ten
*
*
* #param args
* #throws IOException
*/
public static void main(String[] args) throws IOException
{
String starPrefix = "Text to be appended with";
String replacedString = "new text has been appended";
RandomAccessFile file = new RandomAccessFile(new File(PATHNAME), "rw");
String line = "";
while((line = file.readLine()) != null)
{
if(line.startsWith(starPrefix))
{
file.seek(file.getFilePointer() - RECORD_LENGTH - 1);
file.writeBytes(replacedString);
}
}
}
public static void createFile() throws IOException
{
RandomAccessFile file = new RandomAccessFile(new File(PATHNAME), "rw");
String line1 = "one two three";
String line2 = "Text to be appended with";
String line3 = "five six seven";
String line4 = "eight nine ten";
file.writeBytes(paddingRight(line1));
file.writeBytes(CRLF);
file.writeBytes(paddingRight(line2));
file.writeBytes(CRLF);
file.writeBytes(paddingRight(line3));
file.writeBytes(CRLF);
file.writeBytes(paddingRight(line4));
file.writeBytes(CRLF);
file.close();
System.out.println(String.format("File is created in [%s]", PATHNAME));
}
public static String paddingRight(String source)
{
StringBuilder result = new StringBuilder(100);
if(source != null)
{
result.append(source);
for (int i = 0; i < RECORD_LENGTH - source.length(); i++)
{
result.append(EMPTY_STRING);
}
}
return result.toString();
}
}
If the file is large, you might want to use a FileStream for output, but that seems pretty much like it is the simplest process to do what you're asking (and without more specificity i.e. on what types of changes / edits / deletions you're trying to do, it's impossible to determine what more complicated way might work).
No reason to buffer the entire file.
Simply write each line as your read it, insert lines when necessary, delete lines when necessary, replace lines when necessary.
Fundamentally, you will not get around having to recreate the file wholesale, especially if it's just a text file.
What kind of data is it? Do you control the format of the file?
If the file contains name/value pairs (or similar), you could have some luck with Properties, or perhaps cobbling together something using a flat file JDBC driver.
Alternatively, have you considered not writing the data so often? Operating on an in-memory copy of your file should be relatively trivial. If there are no external resources which need real time updates of the file, then there is no need to go to disk every time you want to make a modification. You can run a scheduled task to write periodic updates to disk if you are worried about data backup.
In general you cannot edit the file in place; it's simply a very long sequence of characters, which happens to include newline characters. You could edit in place if your changes don't change the number of characters in each line.
Can't you use regular expressions, if you know what you want to change ? Jakarta Regexp should probably do the trick.
Although this question was a time ago posted, I think it is good to put my answer here.
I think that the best approach is to use FileChannel from java.nio.channels package in this scenario. But this, only if you need to have a good performance! You would need to get a FileChannel via a RandomAccessFile, like this:
java.nio.channels.FileChannel channel = new java.io.RandomAccessFile("/my/fyle/path", "rw").getChannel();
After this, you need a to create a ByteBuffer where you will read from the FileChannel.
this looks something like this:
java.nio.ByteBuffer inBuffer = java.nio.ByteBuffer.allocate(100);
int pos = 0;
int aux = 0;
StringBuilder sb = new StringBuilder();
while (pos != -1) {
aux = channel.read(inBuffer, pos);
pos = (aux != -1) ? pos + aux : -1;
b = inBuffer.array();
sb.delete(0, sb.length());
for (int i = 0; i < b.length; ++i) {
sb.append((char)b[i]);
}
//here you can do your stuff on sb
inBuffer = ByteBuffer.allocate(100);
}
Hope that my answer will help you!
I think, FileOutputStream.getFileChannel() will help a lot, see FileChannel api
http://java.sun.com/javase/6/docs/api/java/nio/channels/FileChannel.html
private static void modifyFile(String filePath, String oldString, String newString) {
File fileToBeModified = new File(filePath);
StringBuilder oldContent = new StringBuilder();
try (BufferedReader reader = new BufferedReader(new FileReader(fileToBeModified))) {
String line = reader.readLine();
while (line != null) {
oldContent.append(line).append(System.lineSeparator());
line = reader.readLine();
}
String content = oldContent.toString();
String newContent = content.replaceAll(oldString, newString);
try (FileWriter writer = new FileWriter(fileToBeModified)) {
writer.write(newContent);
}
} catch (IOException e) {
e.printStackTrace();
}
}
You can change the txt file to java by saving on clicking "Save As" and saving *.java extension.
Related
I really do not want to do a duplicate question, but none of the answers on SO were implementable in my problem.
The answer in this question:
How to read a file from a certain offset in Java?
uses RandomAccessFile, but the implementations I found need all the file lines to have the same length.
How can I get List lines = readLinesFromLine(file);?
I tried
private static List<String> readRandomAccessFile(String filepath, int lineStart, int lineEnd, int charsPerLine, String delimiter) {
File file = new File(filepath);
String data = "";
int bytesPerLine = charsPerLine+2;
try{
RandomAccessFile randomAccessFile = new RandomAccessFile(file, "r");
for (int i = lineStart; i <lineEnd ; i++) {
randomAccessFile.seek(bytesPerLine *i);
data = randomAccessFile.readLine();
dialogLineRead.add(data);
}
randomAccessFile.close();
}catch (Exception e){
e.printStackTrace();
}
String returnData = "";
for (int i = 0; i < dialogLineRead.size(); i++) {
returnData += dialogLineRead.get(i);
returnData+=delimiter;
}
return returnData;
But like I said charsPerLine has to be the same for each line.
I tried to count the chars of each line in a file, and store it in a list, but with a log file of 2gb, that takes to much ram.
Any ideas?
For a standard text file where you don't know the line lengths in advance, there's really no way around reading the whole thing line by line, like in this answer, for example.
I'm trying to delete a line of text from a text file without copying to a temporary file. I am trying to do this by using a Printwriter and a Scanner and having them traverse the file at the same time, the writer writing what the Scanner reads and overwriting each line with the same thing, until it gets to the line that I wish to delete. Then, I advance the Scanner but not the writer, and continue as before. Here is the code:
But first, the parameters: My file names are numbers, so this would read 1.txt or 2.txt, etc, and so f specifies the file name. I convert it to a String in the constructor for a file. Int n is the index of the line that I want to delete.
public void deleteLine(int f, int n){
try{
Scanner reader = new Scanner(new File(f+".txt"));
PrintWriter writer = new PrintWriter(new FileWriter(new File(f+".txt")),false);
for(int w=0; w<n; w++)
writer.write(reader.nextLine());
reader.nextLine();
while(reader.hasNextLine())
writer.write(reader.nextLine());
} catch(Exception e){
System.err.println("Enjoy the stack trace!");
e.printStackTrace();
}
}
It gives me strange errors. It says "NoSuchElementException" and "no line found" in the stack trace. It points to different lines; it seems that any of the nextLine() calls can do this. Is it possible to delete a line this way? If so, what am I doing wrong? If not, why? (BTW, just in case you'd want this, the text file is about 500 lines. I don't know if that counts as large or even matters, though.)
As others have pointed out, you might be better off using a temporary file, if there's a slightest risk that your program crashes mid way:
public static void removeNthLine(String f, int toRemove) throws IOException {
File tmp = File.createTempFile("tmp", "");
BufferedReader br = new BufferedReader(new FileReader(f));
BufferedWriter bw = new BufferedWriter(new FileWriter(tmp));
for (int i = 0; i < toRemove; i++)
bw.write(String.format("%s%n", br.readLine()));
br.readLine();
String l;
while (null != (l = br.readLine()))
bw.write(String.format("%s%n", l));
br.close();
bw.close();
File oldFile = new File(f);
if (oldFile.delete())
tmp.renameTo(oldFile);
}
(Beware of the sloppy treatment of encodings, new-line characters and exception handling.)
However, I don't like answering questions with "I won't tell you how, because you shouldn't do it anyway.". (In some other situation for instance, you may be working with a file that's larger than half your hard drive!) So here goes:
You need to use a RandomAccessFile instead. Using this class you can both read and write to the file using the same object:
public static void removeNthLine(String f, int toRemove) throws IOException {
RandomAccessFile raf = new RandomAccessFile(f, "rw");
// Leave the n first lines unchanged.
for (int i = 0; i < toRemove; i++)
raf.readLine();
// Shift remaining lines upwards.
long writePos = raf.getFilePointer();
raf.readLine();
long readPos = raf.getFilePointer();
byte[] buf = new byte[1024];
int n;
while (-1 != (n = raf.read(buf))) {
raf.seek(writePos);
raf.write(buf, 0, n);
readPos += n;
writePos += n;
raf.seek(readPos);
}
raf.setLength(writePos);
raf.close();
}
You cannot do it this way. FileWriter can only append to a file, rather than write in the middle of it - You need RandomAccessFile if you want to write in the middle. What you do now - you override the file the first time you write to it (and it gets empty - that's why you get the exception). You can create FileWriter with append flag set to true - but this way you would append to a file rather than write in the middle of it.
I'd really recommend to write to a new file and then rename it at the end.
#shelley: you can't do what you are trying to do and what's more, you shouldn't. You should read the file and write to a temporary file for several reasons, for one, it's possible to do it this way (as opposed to what you're trying to do) and for another, if the process gets corrupted, you could bale out without loss of the original file. Now you could update a specific location of a file using a RandomAccessFile, but this is usually done (in my experience) when you are dealing with fixed sized records rather than typical text files.
I'm currently writing my project for school in which requires me to read and write to txt files. I can read them correctly but I can only write to them at the end from an appended FileWriter. I would like to be able to overwrite things in my txt files on line numbers by first deleting the data on the line and then writing in the new data. I attempted to use this method...
public void overWriteFile(String dataType, String newData) throws IOException
{
ReadFile file = new ReadFile(path);
RandomAccessFile ra = new RandomAccessFile(path, "rw");
int line = file.lineNumber(path, dataType);
ra.seek(line);
ra.writeUTF(dataType.toUpperCase() + ":" + newData);
}
but I believe that the seek method moves along in bytes rather than line numbers. Can anyone help. Thanks in advance :)
P.S. the file.lineNumber method returns the exact line that the old data was on so I already have the line number that needs to be written to.
EDIT: Soloution found! Thanks guys :) I'll post the soloution below if anyone is interested
public void overWriteFile(String dataType, String newData, Team team, int dataOrder) throws IOException
{
try
{
ReadFile fileRead = new ReadFile(path);
String data = "";
if(path == "res/metadata.txt")
{
data = fileRead.getMetaData(dataType);
}
else if(path == "res/squads.txt")
{
data = fileRead.getSquadData(dataType, dataOrder);
}
else if(path == "res/users.txt")
{
data = fileRead.getUsernameData(dataType, dataOrder);
}
else if(path == ("res/playerdata/" + team.teamname + ".txt"))
{
//data = fileRead.getPlayerData(dataType, team.teamname, dataOrder);
}
BufferedReader file = new BufferedReader(new FileReader(path));
String line;
String input = "";
while((line = file.readLine()) != null)
{
input += line + '\n';
}
input = input.replace(dataType.toUpperCase() + ":" + data, dataType.toUpperCase() + ":" + newData);
FileOutputStream out = new FileOutputStream(path);
out.write(input.getBytes());
}
catch(Exception e)
{
System.out.println("Error overwriting file: " + path);
e.printStackTrace();
}
}
A quick and dirty solution would be to use the Files.readAllLines and Files.write methods to read all lines, change the one you want to change, and overwrite the whole file:
List<String> lines = Files.readAllLines(file.toPath());
lines.set(line, dataType.toUpperCase() + ":" + newData);
Files.write(file.toPath(), lines); // You can add a charset and other options too
Of course, that's not a good idea if it's a very big file. See this answer for some ideas on how to copy the file line by line in that case.
Regardless of how you do it, though, if you are changing the byte length of the line, you will need to rewrite the whole file (AFAIK). RandomAcessFile allows you to move around the file and overwrite data, but not to insert new bytes or removes existing ones, so the length of the file (in bytes) will stay the same.
Here is a link to a question just like this with a great answer:
I want to open a text file and edit a specific line in java
Basically, you can't just edit that line, unless it'll be the exact same length.
Instead, you'll want to copy over every line, and then when you reach the line number of the line you want to change, instead of copying over the old line, just put in your new line.
The link I gave you has a great example on how to do this.
I hope this helps...if not, let me know, and I'll elaborate further on the post. Good luck :)
I have a text file with data. The file has information from all months. Imagine that the information for January occupy 50 lines. Than February starts and it occupies 40 more lines. Than I have March and so on... Is it possible to read only part of the file? Can I say "read from line X to line Y"? or is there a better way to accomplish this? I only want to print the data correspondent to one month not the all file. Here is my code
public static void readFile()
{
try
{
DataInputStream inputStream =
new DataInputStream(new FileInputStream("SpreadsheetDatabase2013.txt"));
while(inputStream.available() != 0)
{
System.out.println("AVAILABLE: " + inputStream.available());
System.out.println(inputStream.readUTF());
System.out.println(inputStream.readInt());
for (int i = 0; i < 40; i++)
{
System.out.println(inputStream.readUTF());
System.out.println(inputStream.readUTF());
System.out.println(inputStream.readUTF());
System.out.println(inputStream.readUTF());
System.out.println(inputStream.readUTF());
System.out.println(inputStream.readDouble());
System.out.println(inputStream.readUTF());
System.out.println(inputStream.readBoolean());
System.out.println();
}
}// end while
inputStream.close();
}// end try
catch (Exception e)
{
System.out.println("An error has occurred.");
}//end catch
}//end method
Thank you for your time.
My approach to this would be to read the entire contents of the text file and store it in a ArrayList and read only the lines for the requested month.
Example:
Use this function to read the all the lines from the file.
/**
* Read from a file specified by the filePath.
*
* #param filePath
* The path of the file.
* #return List of lines in the file.
* #throws IOException
*/
public static ArrayList<String> readFromFile(String filePath)
throws IOException {
ArrayList<String> temp = new ArrayList<String>();
File file = new File(filePath);
if (file.exists()) {
BufferedReader brin;
brin = new BufferedReader(new FileReader(filePath));
String line = brin.readLine();
while (line != null) {
if (!line.equals(""))
temp.add(line);
line = brin.readLine();
}
brin.close();
}
return temp;
}
Then read only the ones you need from ArrayList temp.
Example:
if you want to read February month's data assuming its 50 lines of data and starts from 40th line.
for(int i=40;i<90;i++)
{
System.out.println(temp.get(i));
}
Note: This is only just one way of doing this. I am not certain if there is any other way!
I would use the scanner class.
Scanner scanner = new Scanner(filename);
Use scanner.nextLine() to get each of the lines of the file. If you only want from line x to line y you can use a for loop to scan each of the lines that you don't need before going through the scanner for the lines you do need. Be careful not to hit an exception without throwing it though.
Or you can go through the scanner and for each line, add the String contents of the line to an ArrayList. Good luck.
Based on how you said your data was organized, I would suggest doing something like this
ArrayList<String> temp = new ArrayList<String>();
int read = 0;
File file = new File(filePath);
if (file.exists()) {
BufferedReader brin;
brin = new BufferedReader(new FileReader(filePath));
String line = brin.readLine();
while (line != null) {
if (!line.equals("")){
if(line.equals("March"))
read = 1;
else if(line.equals("April"))
break;
else if(read == 1)
temp.add(line);
}
line = brin.readLine();
}
brin.close();
Just tried it myself, that'll take in all the data between March and April. You can adjust them as necessary or make them variables. Thanks to ngoa for the foundation code. Credit where credit is due
If you have Java 7, you can use Files.readAllLines(Path path, Charset cs), e.g.
Path path = // Path to "SpreadsheetDatabase2013.txt"
Charset charset = // "UTF-8" or whatever charset is used
List<String> allLines = Files.readAllLines(path, charset);
List<String> relevantLines = allLines.subList(x, y);
Where x (inclusive) and y (exclusive) indicates the line numbers that are of interest, see List.subList(int fromIndex, int toIndex).
One benefit of this solution, as stated in the JavaDoc of readAllLines():
This method ensures that the file is closed when all bytes have been read or an I/O error, or other runtime exception, is thrown.
What ist most concise way to read the contents of a file or input stream in Java? Do I always have to create a buffer, read (at most) line by line and so on or is there a more concise way? I wish I could do just
String content = new File("test.txt").readFully();
Use the Apache Commons IOUtils package. In particular the IOUtils class provides a set of methods to read from streams, readers etc. and handle all the exceptions etc.
e.g.
InputStream is = ...
String contents = IOUtils.toString(is);
// or
List lines = IOUtils.readLines(is)
I think using a Scanner is quite OK with regards to conciseness of Java on-board tools:
Scanner s = new Scanner(new File("file"));
StringBuilder builder = new StringBuilder();
while(s.hasNextLine()) builder.append(s.nextLine());
Also, it's quite flexible, too (e.g. regular expressions support, number parsing).
Helper functions. I basically use a few of them, depending on the situation
cat method that pipes an InputStream to an OutputStream
method that calls cat to a ByteArrayOutputStream and extracts the byte array, enabling quick read of an entire file to a byte array
Implementation of Iterator<String> that is constructed using a Reader; it wraps it in a BufferedReader and readLine's on next()
...
Either roll your own or use something out of commons-io or your preferred utility library.
To give an example of such an helper function:
String[] lines = NioUtils.readInFile(componentxml);
The key is to try to close the BufferedReader even if an IOException is thrown.
/**
* Read lines in a file. <br />
* File must exist
* #param f file to be read
* #return array of lines, empty if file empty
* #throws IOException if prb during access or closing of the file
*/
public static String[] readInFile(final File f) throws IOException
{
final ArrayList lines = new ArrayList();
IOException anioe = null;
BufferedReader br = null;
try
{
br = new BufferedReader(new FileReader(f));
String line;
line = br.readLine();
while(line != null)
{
lines.add(line);
line = br.readLine();
}
br.close();
br = null;
}
catch (final IOException e)
{
anioe = e;
}
finally
{
if(br != null)
{
try {
br.close();
} catch (final IOException e) {
anioe = e;
}
}
if(anioe != null)
{
throw anioe;
}
}
final String[] myStrings = new String[lines.size()];
//myStrings = lines.toArray(myStrings);
System.arraycopy(lines.toArray(), 0, myStrings, 0, lines.size());
return myStrings;
}
(if you just want a String, change the function to append each lines to a StringBuffer (or StringBuilder in java5 or 6)
String content = (new RandomAccessFile(new File("test.txt"))).readUTF();
Unfortunately Java is very picky about the source file being valid UTF8 though, or you will get an EOFException or UTFDataFormatException.
You have to create your own function, I suppose. The problem is that Java's read routines (those I know, at least) usually take a buffer argument with a given length.
A solution I saw is to get the size of the file, create a buffer of this size and read the file at once. Hoping the file isn't a gigabyte log or XML file...
The usual way is to have a fixed size buffer or to use readLine and concatenate the results in a StringBuffer/StringBuilder.
I don't think reading using BufferedReader is a good idea because BufferedReader will return just the content of line without the delimeter. When the line contains nothing but newline character, BR will return a null although it still doesn't reach the end of the stream.
String org.apache.commons.io.FileUtils.readFileToString(File file)
Pick one from here.
How do I create a Java string from the contents of a file?
The favorite was:
private static String readFile(String path) throws IOException {
FileInputStream stream = new FileInputStream(new File(path));
try {
FileChannel fc = stream.getChannel();
MappedByteBuffer bb = fc.map(FileChannel.MapMode.READ_ONLY, 0, fc.size());
/* Instead of using default, pass in a decoder. */
return CharSet.defaultCharset().decode(bb).toString();
}
finally {
stream.close();
}
}
Posted by erickson
Or the Java 8 way:
try {
String str = new String(Files.readAllBytes(Paths.get("myfile.txt")));
...
} catch (IOException ex) {
Logger.getLogger(getClass().getName()).log(Level.SEVERE, null, ex);
}
One may pass an appropriate Charset to the String constructor.