I am creating one text file which will connected to some server.
this text file will receive its contents from the server.
It will receive the some text data continuously.
To limit the file size , I am checking no.of lines in the file and if exceeds the mark I am clearing file content. Server will write from the beginning.
Below is the code I have used to do this :
LineNumberReader myReader = new LineNumberReader( new FileReader(new File("mnt/sdcard/abc.txt")));
while(true) {
while(myReader.readLine() != null) {
counter ++;
}
if(counter > 100 ) {
PrintWriter writer = new PrintWriter("/mnt/sdcard/abc.txt");
writer.print("");
writer.close();
writer = null;
counter = 0;
}
}
But after I clear the contents in a file my "counter" not increasing.
But my file is having some data.
I think after reading done I have set my "myReader" to some intial..?
If its how to set that to initial so that .readLine() should start from begining.
Shouldn't you close myReader before writing to the file??
LineNumberReader myReader = new LineNumberReader( new FileReader(new File("mnt/sdcard/abc.txt")));
while(true)
{
while(myReader.readLine() != null) {
counter++;
}
if(counter > 100 )
{
//CLOSE myReader
myReader.close();
PrintWriter writer = new PrintWriter("/mnt/sdcard/abc.txt");
writer.print("");
writer.close();
writer = null;
counter = 0;
//REOPEN myReader
myReader = new LineNumberReader( new FileReader(new File("mnt/sdcard/abc.txt")));
}
}
Shouldn't you make sure that changes to the file done by the server and changes to the file done by this loop are synchronized??
can you show how and where counter is declared and what other code might be modifying it? it is a matter of guessing without seeing that. meanwhile, maybe you can consider not reading the file content all the time and use the file size to determine if you should clean it.
long limit= .... //add your limit in bytes
long fileSize = new File("mnt/sdcard/abc.txt").length();
if (fileSize > limit){
//clean the file
}
please also note to check what was mentioned in the other answers regarding closing the file or trying to clean it while it is open and the server is writing to it.
Issue a myReader.reset() after clearing the contents.
Related
My aim is to read from a large file, process 2 lines at a time, and write the result to a new file(s). These files can get very large, from 1GB to 150GB in size, so I'd like to attempt to do this processing using the least RAM possible
The processing is very simple: The lines split by a tab delimited, certain elements are selected, and the new String is written to the new files.
So far I have attempted using BufferedReader to read the File and PrintWriter to output the lines to a file:
while((line1 = br.readLine()) != null){
if(!line1.startsWith("#")){
line2 = br.readLine();
recordCount++;
one.println(String.format("%s\n%s\n+\n%s",line1.split("\t")[0] + ".1", line1.split("\t")[9], line1.split("\t")[10]));
two.println(String.format("%s\n%s\n+\n%s",line2.split("\t")[0] + ".2", line2.split("\t")[9], line2.split("\t")[10]));
}
}
I have also attempted to uses Java8 Streams to read and write from the file:
stream.forEach(line -> {
if(!line.startsWith("#")) {
try {
if (counter.getAndIncrement() % 2 == 0)
Files.write(path1, String.format("%s\n%s\n+\n%s", line.split("\t")[0] + ".1", line.split("\t")[9], line.split("\t")[10]).getBytes(), StandardOpenOption.APPEND);
else
Files.write(path2, String.format("%s\n%s\n+\n%s", line.split("\t")[0] + ".2", line.split("\t")[9], line.split("\t")[10]).getBytes(), StandardOpenOption.APPEND);
}catch(IOException ioe){
}
}
});
Finally, I have tried to use an InputStream and scanner to read the file and PrintWriter to output the lines:
inputStream = new FileInputStream(inputFile);
sc = new Scanner(inputStream, "UTF-8");
String line1, line2;
PrintWriter one = new PrintWriter(new FileOutputStream(dotOne));
PrintWriter two = new PrintWriter(new FileOutputStream(dotTwo));
while(sc.hasNextLine()){
line1 = sc.nextLine();
if(!line1.startsWith("#")) {
line2 = sc.nextLine();
one.println(String.format("%s\n%s\n+\n%s",line1.split("\t")[0] + ".1", line1.split("\t")[9], line1.split("\t")[10]));
two.println(String.format("%s\n%s\n+\n%s",line2.split("\t")[0] + ".2", line2.split("\t")[9], line2.split("\t")[10]));
}
}
The issue that I'm facing is that the program seems to be storing either the data to write, or the input file data into RAM.
All of the above methods do work, but use more RAM than I'd like them to.
Thanks in advance,
Sam
What you did not try is a MemoryMappedByteBuffer. The FileChannel.map might be usable for your purpose, not allocating in java memory.
Functioning code with a self made byte buffer would be:
try (FileInputStream fis = new FileInputStream(source);
FileChannel fic = fis.getChannel();
FileOutputStream fos = new FileOutputStream(target);
FileChannel foc = fos.getChannel()) {
ByteBuffer buffer = ByteBuffer.allocate(1024);
while (true) {
int nread = fic.read(buffer);
if (nread == -1) {}
break;
}
buffer.flip();
foc.write(buffer);
buffer.clear();
}
}
Using fic.map to consecutively map regions into OS memory seems easy, but
such more complex code I would need to test first.
When creating PrintWriter set autoFlush to true:
new PrintWriter(new FileOutputStream(dotOne), true)
This way the buffered data will be flushed with every println.
I have data file “ReadFile1.txt”. I want to read each data from ReadFile1.txt and manipulate those data then write the results in another file “WriteFile2.txt”. Here is my function. The problem is it only reads 2nd,4th, and so on and does write only 2nd result. What’s wrong in this code? I appreciate your help.
public void doManipulate() throws NumberFormatException, IOException {
int multiple = 10;
try {
FileInputStream file = new FileInputStream("ReadFile1.txt");
InputStreamReader input = new InputStreamReader(file);
BufferedReader reader = new BufferedReader(input);
String data1;
while ((data1 = reader.readLine()) != null) {
int data2 = 0;
data1 = reader.readLine();
data2 = Integer.parseInt(data1);
int compressedFrames = data2*multiple;
File file2 = new File("WriteFile2.txt");
FileWriter writer = new FileWriter(file2);
writer.write(String.valueOf(compressedFrames) + "\n");
writer.flush();
writer.close();
}
reader.close();
} catch (IOException e) {
e.printStackTrace();
}
}
You're calling reader.readLine() twice for every iteration of the while loop - the first time is in the loop declaration, which reads every odd line, and the second is just a couple of lines down (data1 = reader.readLine();). The second call is blowing away anything read by the first before you have a chance to parse it. Removing the second call should fix the "every other line" issue.
Another issue is that you're closing the writer at every iteration of the while loop - don't close the writer until the while loop is done or your output file will only have the first parsed data element in it after your program closes.
I know this is very basic stuff but for some reason I'm having problems with a bufferedReader/ Writer. I am trying to get the first line of text and return it to another method. However, for some reason the writer doesn't seem to be writing to the temp file and it isn't changing the name of the temp file either.
By throwing a few print statements I have been able to figure out:
The while loop is operating correctly
The if else statement is operating correctly
The tempFile is not writing to a text file correctly
The tempFile is not renaming correctly
There are no errors being thrown
private static String wavFinder() throws IOException{
String currentWav=null;
int x = 1;
File inputFile = new File("C:\\convoLists/unTranscribed.txt");
File tempFile = new File("C:\\convoLists/unTranscribedtemp.txt");
BufferedReader reader = new BufferedReader(new FileReader(inputFile));
BufferedWriter writer = new BufferedWriter(new FileWriter(tempFile));
String currentLine = null;
while((currentLine = reader.readLine()) != null) {
if(x == 1){
currentWav = currentLine;
}
else{
writer.write(currentLine);
}
x = 2;
}
boolean successful = tempFile.renameTo(inputFile);
System.out.println("Success: " + successful);
System.out.println("currentWav = " + currentWav);
return currentWav;
}
Here is the method I am using. If you notice anything please let me know and if you have any questions I will be sure to answer them quickly. Thank you :)
First flush the steam(writer) and close them.
You can not have two files with same name. You are trying to rename the temp file with input file. You need to delete input file and then rename it to that.
reader.close();
writer.flush();
writer.close();
inputFile.delete();
Add these lines before rename and it will work
Close your buffers before trying to call renameTo.
reader.close()
writer.close()
File inputFile = new File("C:\convoLists/unTranscribed.txt");
File tempFile = new File("C:\convoLists/unTranscribedtemp.txt");
Why you have different signs for path?
Always should be //.
I am trying to write contents to a file, but once the while loop prints all the lines , it get's stuck and doesn't go to the next line. The line after while loop are not being executed. My first guess was that maybe the fstream is still waiting for another line, but the while loop does check if no more data then it should come out.
Please help me out here !
Thanks.
//Code
File file;
DataInputStream inputData = new DataInputStream(newPeerConnection.getInputStream());
file = new File(
"F:\\Workspace\\PeerToPeer\\src\\ncsu\\csc\\socketClasses\\out\\out.txt");
#SuppressWarnings("resource")
PrintWriter fstream = new PrintWriter(new FileWriter(
file, true));
if (!file.exists()) {
file.createNewFile();
}
System.out.println("Writing to file: ");
while ((strval = inputData.readUTF()) != null) {
System.out.println(strval);
fstream.println(strval);
//fstream.flush();
//fstream.close();
}
//These lines are not executed
System.out.println("Do you want to download another file (Y/N)?");
wantToContinue = scannerObj.next();
out.writeUTF(wantToContinue);
out.flush();}
I guess your while loop is blocked by inputData.readUTF().
You should ensure the input stream of newPeerConnection is closed by another port, otherwise inputData will wait for next input and inputData.readUTF() blocks.
I have servlet to read log file my jsp call servlet at some interval.I want to add functionality in my servlet so that it read file from line next to last line from previous read.
I have following code but its not working
File file = new File("D:\\graph\\temp.log");
FileReader fr = new FileReader(file);
LineNumberReader lnr = new LineNumberReader(fr);\
lnr.setLineNumber(count) // count is the variable keeping track of number of line
previously read.
Is it need some extra code ? or is there any other alernative to do the same ?
From the documentation:
Note however, that setLineNumber(int) does not actually change the
current position in the stream; it only changes the value that will be
returned by getLineNumber().
Sounds to me like you'll have to start from the beginning and call readLine in a loop until getLineNumber() is greater than count.
Something like this:
File file = new File("D:\\graph\\temp.log");
FileReader fr = new FileReader(file);
LineNumberReader lineReader = new LineNumberReader(fr);
// skip the lines you don't need
while (lineReader.getLineNumber() < count) {
lineReader.readLine();
}
// begin processing input here
Have you looked at RandomAccessFile. If you keep the count of bytes read. Than followin code will help you.
RandomAccessFile raf = new RandomAccessFile(fileName,"r");
byte[] cBuf = new byte[1024];
long count = offset;
raf.seek(offset);
long lineCOunt=0l;
while(true){
int lineCOunt =raf.read(cBuf);
count +=lineCOunt;
if(lineCOunt<1024){
System.out.println((new String(cBuf)).substring(0,i));
break;
}
System.out.println(new String(cBuf));