Develop Reference

Java FileReader method is not returning/printing all characters in file record

I am trying to write to a file thru FileWriter and then read it thru FileReader. FileWriter writes the record to file okay, but the FileReader method appears to be skipping characters while printing. I have pasted the code and the file contents and output below. Can you please help identify what I might be missing or doing wrong?
***Update -
Just realized a possibility (after posting) that the "read()" in the while statement could be reading a Character ahead the "read()" in the print statement, and hence causing every other character (beginning with the first one) to skip the print statement. Not sure though. Could that be what is happening here?
Code -
public class FileIOCharClass {
public static void main(String[] args) {
File TestChar = new File("C:\\TestChar.txt");
FileIOCharClass FileIOChar = new FileIOCharClass();
try {
FileIOChar.charWriteMethod(TestChar);
}
catch (IOException a){
System.out.println("IO Exception occured in Write method");
}
try{
FileIOChar.charReadMethod(TestChar);
}
catch (IOException b){
System.out.println("IO Exception occured in Read method");
}
}
private void charWriteMethod(File CharFileToWrite) throws IOException {
FileWriter WriteRoutine = null;
try{
WriteRoutine = new FileWriter(CharFileToWrite);
WriteRoutine.write("Line-1");
WriteRoutine.write("Line-2");
WriteRoutine.write("Line-3");
}finally {
WriteRoutine.flush();
WriteRoutine.close();
}
}
private void charReadMethod(File CharFileToRead) throws IOException {
FileReader ReadRoutine = new FileReader(CharFileToRead);
while (ReadRoutine.read() != -1){
System.out.println(((char)ReadRoutine.read()));
}
}
}
Contents of TestChar file created -
Line-1Line-2Line-3
Program print output
i
e
1
i
e
2
i
e
3
Process finished with exit code 0

The issue is that you are reading twice. Once int the while condition, and another in the block statement
while (ReadRoutine.read() != -1){
System.out.println(((char)ReadRoutine.read()));
}
Instead you can try:
int data = ReadRoutine.read();
while (data != -1) {
System.out.println((char)data);
data = ReadRoutine.read();
}

Object Writing in Network

I have simple java server and client .In the server, a file is broken into several chunks of byte array Now this byte arrays has to be send through object output stream. But if every time i use a new array to load file data that is perfect but if i use the same array(It is necessary i have to memory efficient) to load the file data client receives the same (first) byte array every time.
networkUtil read and write
public Object read() {
Object o = null;
try {
o=ois.readObject();
} catch (Exception e) {
//System.out.println("Reading Error in network : " + e.toString());
}
return o;
}
public void write(Object o) {
try {
oos.writeObject(o);
} catch (IOException e) {
System.out.println("Writing Error in network : " + e.toString());
}
}
Server writing portion
public void run() {
try {
//Scanner input=new Scanner(System.in);
byte []b =new byte[1000];
int num=5;
long i=0;
//ObjectOutputStream oosp = null;
for(int j=0;j<num;j++) {
File f=new File("G:\\photography\\DSC01020.JPG");
RandomAccessFile file1=new RandomAccessFile(f,"r");
long l=file1.length();
num=(int)Math.ceil((double)l/(double)1000);
//System.out.println("it is num "+num);
//file1.close();
// RandomAccessFile file=new RandomAccessFile(f,"r");
// byte [] b =new byte[1000];
System.out.println("seeking from "+i+"left "+(l-(j*1000)));
file1.seek(i);
file1.read(b);
file1.close();
System.out.println("it is first "+b[0]+" it is second "+b[1]);
nc.write(b);//network util
//oosp.write(b);
file1.close();
i+=1000;
}
client reading portion
try {
FileOutputStream fos=new FileOutputStream("C:\\Temp\\test.jpg");
byte []a;
for(int j=0;j<225;j++) {
Object o=nc.read();//netwotk util
if(o!= null) {
if(o instanceof Data) {
Data obj=(Data)o;
//System.out.println(obj.getElement());
}
if(o instanceof byte[])
{
//System.out.println("it is byte array");
a=(byte[])o;
System.out.println("it is first "+a[0]+" it is second "+a[1]);
if(j==224)// it is hard coded for this file i have to change this for all file
{
fos.write(a,0,203);
}
else {
fos.write(a);
}
}
}
}

Do you have memory problems ? In Java, the objects and arrays not in use are garbage collected. See Deleting An Entire Array. I don't think that you will enconter any problems by reallocating each time.
Edit:
Since the reallocation is the problem, maybe a ByteBuffer http://docs.oracle.com/javase/7/docs/api/java/nio/ByteBuffer.html can solve this.
You can try to use java.nio instead with a FileChannel and a ByteBuffer. See http://www.java2s.com/Tutorial/Java/0180__File/UseFileChannelandByteBuffertoCopyFile.htm and FileChannel ByteBuffer and Hashing Files for examples.

reading variables buried in java exception handling

I am writing a function to take a text file and count how many lines it has while outputting the lines to an array of strings. Doing this I have several exceptions I need to look out for. The class function has several variables that should have a scope throughout the function but when I write a value to the function inside of an exception, the return statement cannot find it. I've moved the declaration around and nothing helps
The value returned "h5Files" "Might not have been initialized" Since I don't know how long the array will be I cannot initialize it to a certain length. I do this within the code and I need a way to tell the return statement that I now have a values
Here is the code
public String[] ReadScanlist(String fileIn){
int i;
String directory ="c:\\data\\"; // "\" is an illegal character
System.out.println(directory);
int linereader = 0;
String h5Files[];
File fileToRead = new File(directory + fileIn);
System.out.println(fileToRead);
try {
FileInputStream fin = new FileInputStream(fileToRead); // open this file
}
catch(FileNotFoundException exc) {
System.out.println("File Not Found");
}
try{
//read bytes until EOF is detected
do {
FileReader fr = new FileReader(fileToRead);// Need to convert to reader
LineNumberReader lineToRead = new LineNumberReader(fr); // Use line number reader class
//
while (lineToRead.readLine() != null){
linereader++;
}
linereader = 0;
lineToRead.setLineNumber(0); //reset line number
h5Files = new String[linereader];
while (lineToRead.readLine() != null){
h5Files[linereader] = lineToRead.readLine(); // deposit string into array
linereader++;
}
return h5Files;
}
while(i !=-1); // When i = -1 the end of the file has been reached
}
catch(IOException exc) {
System.out.println("Error reading file.");
}
try{
FileInputStream fin = new FileInputStream(fileToRead);
fin.close(); // close the file
}
catch(IOException exc) {
System.out.println("Error Closing File");
}
return h5Files;
}

Your code is very very odd. For example these two blocks make no sense:
try {
FileInputStream fin = new FileInputStream(fileToRead); // open this file
}
catch(FileNotFoundException exc) {
System.out.println("File Not Found");
}
try{
FileInputStream fin = new FileInputStream(fileToRead);
fin.close(); // close the file
}
catch(IOException exc) {
System.out.println("Error Closing File");
}
I don't know what you think they do, but besides the first one leaking memory, they do nothing at all. The comments are more worrying, they suggest that you need to do more reading on IO in Java.
Deleting those blocks and tidying the code a (moving declarations, formatting) gives this:
public String[] ReadScanlist(String fileIn) {
String directory = "c:\\data\\";
String h5Files[];
File fileToRead = new File(directory + fileIn);
try {
int i = 0;
do {
FileReader fr = new FileReader(fileToRead);
LineNumberReader lineToRead = new LineNumberReader(fr);
int linereader = 0;
while (lineToRead.readLine() != null) {
linereader++;
}
linereader = 0;
lineToRead.setLineNumber(0);
h5Files = new String[linereader];
while (lineToRead.readLine() != null) {
h5Files[linereader] = lineToRead.readLine();
linereader++;
}
return h5Files;
} while (i != -1);
} catch (IOException exc) {
System.out.println("Error reading file.");
}
return h5Files;
}
My first bone of contention is the File related code. First, File abstracts from the underlying OS, so using / is absolutely fine. Second, there is a reason File has a File, String constructor, this code should read:
File directory = new File("c:/data");
File fileToRead = new File(directory, fileIn);
But it should really be using the new Path API anyway (see below).
So, you declare h5Files[]. You then proceed to read the whole file to count the lines. You then assign h5Files[] to an array of the correct size. Finally you fill the array.
If you have an error anywhere before you assign h5Files[] you have not initialised it and therefore cannot return it. This is what the compiler is telling you.
I don't know what i does in this code, it is assigned to 0 at the top and then never reassigned. This is an infinite loop.
So, you need to rethink your logic. I would recommend throwing an IOException if you cannot read the file. Never return null - this is an anti-pattern and leads to all those thousands of null checks in your code. If you never return null you will never have to check for it.
May I suggest the following alternative code:
If you are on Java 7:
public String[] ReadScanlist(String fileIn) throws IOException {
final Path root = Paths.get("C:/data");
final List<String> lines = Files.readAllLines(root.resolve(fileIn), StandardCharsets.UTF_8);
return lines.toArray(new String[lines.size()]);
}
Or, if you have Java 8:
public String[] ReadScanlist(String fileIn) throws IOException {
final Path root = Paths.get("C:/data");
try (final Stream<String> lines = Files.lines(root.resolve(fileIn), StandardCharsets.UTF_8)) {
return lines.toArray(String[]::new);
}
}

Since I don't know how long the array will be I cannot initialize it
to a certain length.
I don't think an array is the correct solution for you then - not to say it can't be done, but you would be re-inventing the wheel.
I would suggest you use a LinkedList instead, something like:
LinkedList<String> h5Files = new LinkedList<>();
h5Files.add(lineToRead.readLine());
Alternatively you could re-invent the wheel by setting the array to an arbritary value, say 10, and then re-size it whenever it gets full, something like this:
h5Files = new String[10];
if (linereader = h5Files.size())
{
String[] temp = h5Files;
h5Files = new String[2 * linereader];
for (int i = 0; i < linereader; i++)
{
h5Files[i] = temp[i];
}
}
Either one of these solutions would allow you to initialize the array (or array alternative) in a safe constructor, prior to your try block, such that you can access it if any exceptions are thrown

Here is your problem. Please take a look on digested version of your code with my comments.
String h5Files[]; // here you define the variable. It still is not initialized.
try{
..................
do {
h5Files = new String[linereader]; // here you initialize the variable
} while(i !=-1); // When i = -1 the end of the file has been reached
..................
catch(IOException exc) {
// if you are here the variable is still not initialized
System.out.println("Error reading file.");
}
// you continue reading file even if exception was thrown while opening the file
I think that now the problem is clearer. You try to open the file and count lines. If you succeed you create array. If not (i.e. when exception is thrown) you catch the exception but still continue reading the file. But in this case you array is not initialized.
Now how to fix this?
Actually if you failed to read the file first time you cannot continue. This may happen for example if file does not exist. So, you should either return when first exception is thrown or just do not catch it at all. Indeed there is nothing to do with the file if exception was thrown at any phase. Exception is not return code. This is the reason that exceptions exist.
So, just do not catch exceptions at all. Declare your method as throws IOException and remove all try/catch blocks.

How to use a chunk delimiter in a raw data file?

I want to save raw data chunks to a file, And later on read those chunks one by one. This is no big deal except the following doubt:
What exact bytes to use as a delimiter, i.e to identify end of one chunk and beginning of next ? Given that chunk data might also contain such a sequence of bytes by random chance.
Notes: chunks are of variable size and contain random data. They are jpeg images actually.

You could first write the length of the chunk to the file as a fixed-size value, e.g. a 4 bytes integer, followed by the data itself:
public void appendChunk(byte[] data, File file) throws IOException {
DataOutputStream stream = null;
try {
stream = new DataOutputStream(new BufferedOutputStream(new FileOutputStream(file, true)));
stream.writeInt(data.length);
stream.write(data);
} finally {
if (stream != null) {
try {
stream.close();
} catch (IOException e) {
// ignore
}
}
}
}
If you later have to read the chunks back from that file, you start by reading the length of the first chunk. You now can decide whether to read the chunk data, or whether to skip it and continue with the next chunk.
public void processChunks(File file) throws IOException {
DataInputStream stream = null;
try {
stream = new DataInputStream(new BufferedInputStream(new FileInputStream(file)));
while (true) {
try {
int length = stream.readInt();
byte[] data = new byte[length];
stream.readFully(data);
// todo: do something with the data
} catch (EOFException e) {
// end of file reached
break;
}
}
} finally {
if (stream != null) {
try {
stream.close();
} catch (IOException e) {
// ignore
}
}
}
}
You can also add other meta-data about the chunks, like writing the original name of the file with stream.writeUTF(...). You only have to make sure that you write and read the same data in the same order.

Create a 2nd file in which you save the byteranges of your chunks in the chunkfile, or add that information to the header of your chunkfile. Did something similar once, don't forget that the byteranges than have the additional offset of the length of the header.
int startbyte = 0;
int lastByte = 0;
int chunkcount = 0;
File chunkfile;
File structurefile;
for (every chunk) {
append chunk to chunkfile
lastByte = startByte + chunk.sizeInBytes()
append to structurefile: chunkcount startByte lastByte
chunkcount++;
startByte = lastByte + 1
}

How do I use Java to read from a file that is actively being written to?

I have an application that writes information to file. This information is used post-execution to determine pass/failure/correctness of the application. I'd like to be able to read the file as it is being written so that I can do these pass/failure/correctness checks in real time.
I assume it is possible to do this, but what are the gotcha's involved when using Java? If the reading catches up to the writing, will it just wait for more writes up until the file is closed, or will the read throw an exception at this point? If the latter, what do I do then?
My intuition is currently pushing me towards BufferedStreams. Is this the way to go?

Could not get the example to work using FileChannel.read(ByteBuffer) because it isn't a blocking read. Did however get the code below to work:
boolean running = true;
BufferedInputStream reader = new BufferedInputStream(new FileInputStream( "out.txt" ) );
public void run() {
while( running ) {
if( reader.available() > 0 ) {
System.out.print( (char)reader.read() );
}
else {
try {
sleep( 500 );
}
catch( InterruptedException ex ) {
running = false;
}
}
}
}
Of course the same thing would work as a timer instead of a thread, but I leave that up to the programmer. I'm still looking for a better way, but this works for me for now.
Oh, and I'll caveat this with: I'm using 1.4.2. Yes I know I'm in the stone ages still.

If you want to read a file while it is being written and only read the new content then following will help you achieve the same.
To run this program you will launch it from command prompt/terminal window and pass the file name to read. It will read the file unless you kill the program.
java FileReader c:\myfile.txt
As you type a line of text save it from notepad and you will see the text printed in the console.
public class FileReader {
public static void main(String args[]) throws Exception {
if(args.length>0){
File file = new File(args[0]);
System.out.println(file.getAbsolutePath());
if(file.exists() && file.canRead()){
long fileLength = file.length();
readFile(file,0L);
while(true){
if(fileLength<file.length()){
readFile(file,fileLength);
fileLength=file.length();
}
}
}
}else{
System.out.println("no file to read");
}
}
public static void readFile(File file,Long fileLength) throws IOException {
String line = null;
BufferedReader in = new BufferedReader(new java.io.FileReader(file));
in.skip(fileLength);
while((line = in.readLine()) != null)
{
System.out.println(line);
}
in.close();
}
}

You might also take a look at java channel for locking a part of a file.
http://java.sun.com/javase/6/docs/api/java/nio/channels/FileChannel.html
This function of the FileChannel might be a start
lock(long position, long size, boolean shared)
An invocation of this method will block until the region can be locked

I totally agree with Joshua's response, Tailer is fit for the job in this situation. Here is an example :
It writes a line every 150 ms in a file, while reading this very same file every 2500 ms
public class TailerTest
{
public static void main(String[] args)
{
File f = new File("/tmp/test.txt");
MyListener listener = new MyListener();
Tailer.create(f, listener, 2500);
try
{
FileOutputStream fos = new FileOutputStream(f);
int i = 0;
while (i < 200)
{
fos.write(("test" + ++i + "\n").getBytes());
Thread.sleep(150);
}
fos.close();
}
catch (Exception e)
{
e.printStackTrace();
}
}
private static class MyListener extends TailerListenerAdapter
{
#Override
public void handle(String line)
{
System.out.println(line);
}
}
}

The answer seems to be "no" ... and "yes". There seems to be no real way to know if a file is open for writing by another application. So, reading from such a file will just progress until content is exhausted. I took Mike's advice and wrote some test code:
Writer.java writes a string to file and then waits for the user to hit enter before writing another line to file. The idea being that it could be started up, then a reader can be started to see how it copes with the "partial" file. The reader I wrote is in Reader.java.
Writer.java
public class Writer extends Object
{
Writer () {
}
public static String[] strings =
{
"Hello World",
"Goodbye World"
};
public static void main(String[] args)
throws java.io.IOException {
java.io.PrintWriter pw =
new java.io.PrintWriter(new java.io.FileOutputStream("out.txt"), true);
for(String s : strings) {
pw.println(s);
System.in.read();
}
pw.close();
}
}
Reader.java
public class Reader extends Object
{
Reader () {
}
public static void main(String[] args)
throws Exception {
java.io.FileInputStream in = new java.io.FileInputStream("out.txt");
java.nio.channels.FileChannel fc = in.getChannel();
java.nio.ByteBuffer bb = java.nio.ByteBuffer.allocate(10);
while(fc.read(bb) >= 0) {
bb.flip();
while(bb.hasRemaining()) {
System.out.println((char)bb.get());
}
bb.clear();
}
System.exit(0);
}
}
No guarantees that this code is best practice.
This leaves the option suggested by Mike of periodically checking if there is new data to be read from the file. This then requires user intervention to close the file reader when it is determined that the reading is completed. Or, the reader needs to be made aware the content of the file and be able to determine and end of write condition. If the content were XML, the end of document could be used to signal this.

There are a Open Source Java Graphic Tail that does this.
https://stackoverflow.com/a/559146/1255493
public void run() {
try {
while (_running) {
Thread.sleep(_updateInterval);
long len = _file.length();
if (len < _filePointer) {
// Log must have been jibbled or deleted.
this.appendMessage("Log file was reset. Restarting logging from start of file.");
_filePointer = len;
}
else if (len > _filePointer) {
// File must have had something added to it!
RandomAccessFile raf = new RandomAccessFile(_file, "r");
raf.seek(_filePointer);
String line = null;
while ((line = raf.readLine()) != null) {
this.appendLine(line);
}
_filePointer = raf.getFilePointer();
raf.close();
}
}
}
catch (Exception e) {
this.appendMessage("Fatal error reading log file, log tailing has stopped.");
}
// dispose();
}

You can't read a file which is opened from another process using FileInputStream, FileReader or RandomAccessFile.
But using FileChannel directly will work:
private static byte[] readSharedFile(File file) throws IOException {
byte buffer[] = new byte[(int) file.length()];
final FileChannel fc = FileChannel.open(file.toPath(), EnumSet.of(StandardOpenOption.READ));
final ByteBuffer dst = ByteBuffer.wrap(buffer);
fc.read(dst);
fc.close();
return buffer;
}

Not Java per-se, but you may run into issues where you have written something to a file, but it hasn't been actually written yet - it might be in a cache somewhere, and reading from the same file may not actually give you the new information.
Short version - use flush() or whatever the relevant system call is to ensure that your data is actually written to the file.
Note I am not talking about the OS level disk cache - if your data gets into here, it should appear in a read() after this point. It may be that the language itself caches writes, waiting until a buffer fills up or file is flushed/closed.

I've never tried it, but you should write a test case to see if reading from a stream after you have hit the end will work, regardless of if there is more data written to the file.
Is there a reason you can't use a piped input/output stream? Is the data being written and read from the same application (if so, you have the data, why do you need to read from the file)?
Otherwise, maybe read till end of file, then monitor for changes and seek to where you left off and continue... though watch out for race conditions.