I want to know when a file has rotated because because I'm watching a file and I have to get a new file with the content of this file and the new file. I have to use Java6, so I don't have the new features of JDK7.
The problem is that when I rotate the file (keeping a reference to the file), the old reference gets updated, I would like that it'd point to the old file.
I was doing something like this:
if (reader == null) {
reader = new RandomAccessFile(toWatch, "r");
}
System.out.println("LasModified:" + new Date(toWatch.lastModified()));
System.out.println("Cuando se creo to Watch long:" + reader.length()); --> lenght is okay.
//Here, I use logrotate to simulate what it could happen. (I execute the code in debug mode to be able to do it
long len = 0L;
File f = new File("/home/gortiz/logRotate/test.flume");
System.out.println("LasModified NewReader:" + new Date(f.lastModified()));
RandomAccessFile newReader = new RandomAccessFile(toWatch, "r");
len = reader.length(); --> len = 0
long newLength = newReader.length(); --> newLenght = 0
I could check if there's a file with name.1. Usually logs system renames logs like that.. but, it's not a good solution because nobody guarantees that that's going to be as log system renames files or even they could be moved to another directory.
Related
this is my directory structure
Inside the server I have the following code for saving a file that gets sent from the client
fileName = reader.readLine();
DataInputStream dis = null;
try {
dis = new DataInputStream(csocket.getInputStream());
FileOutputStream fos = new FileOutputStream(fileName);
buffer = new byte[4096];
int fileSize = 15123;
int read = 0;
int totalRead = 0;
int remaining = fileSize;
while((read = dis.read(buffer, 0, Math.min(buffer.length, remaining))) > 0) {
totalRead += read;
remaining -= read;
fos.write(buffer, 0, read);
}
fos.close();
dis.close();
} catch (IOException e) {
}
break;
I'm wondering how I would go about saving the file within the xml folder? I've tried using getClass().getResource and such but nothing seems to work.
fileName is just a simple string containing the name of the file, not a path or anything.
I get the correct path using this code:
File targetDir = new File(getClass().getResource("xml").getPath());
File targetFile = new File(targetDir, fileName);
targetFile.createNewFile();
System.out.println(targetFile.getAbsolutePath());
dis = new DataInputStream(csocket.getInputStream());
FileOutputStream fos = new FileOutputStream(targetFile.getAbsolutePath(), false);
But it still won't save it there...
The best way is to receive explicitly the target path for storing files, either through a .properties file or a command-line argument. In this way, you make your program flexible to be installed and adapted in different environments.
But if you wish your program to assume the target directory automatically, the best option is to set a relative path before creating the FileOutputStream, as long as you start your program always from the same path:
File targetDir=new File("xml");
File targetFile=new File(targetDir, fileName);
FileOutputStream fos = new FileOutputStream(targetFile);
This will work assuming the program is started from server as current directory.
Update
Other minor suggestions about your program:
Never base the exit condition of the loop on a hard-coded file size, because it is not possible to know it a priori. Instead, check explicitly if the value returned by read is less than 0 => that means End Of File reached.
Consequently, do not bother to calculate the exact amount of data to get through a call to read. Just enter the buffer size, because you are setting a maximum data size.
Never let exceptions catched without a proper treatment: If you know how to make your program recover, enter a proper code into the catch block. Otherwise, you'd better not catch them: Declare them in the throws clause and let them be propagated to the caller.
Always create stream resources through the try-with-resources instruction, to ensure they got closed at the end:
try (FileOutputStream fos = new FileOutputStream(...))
{
// ... use fos...
}
Save unnecessary instructions: If you don't care about if the file already exists on the filesystem or not, don't call createNewFile. But if you care, check the returned value and bifurcate consequently.
I try to create file and it does created but not at ProjectName\src\com\company\xml but in ProjectName\out\production\ProjectName\com\company\xml
my code:
File targetDir = new File(this.getClass().getResource("xml").getPath());
// get the parent of the file
String parentPath = targetDir.getParent( );
String fileName="xml/name.txt";
//do something
File targetFile = new File(parentPath, fileName);
targetFile.createNewFile();
Just pay attention that after compilation you will try to save it into a jar file and it a complicated thing to do.
usually you need to save file into file outside from your jar(separate in the root) like this:
I have to read a URLConnection response containing 2MB of pretty printed JSON in java.
2mb is not "small" but by no means large. It contains JSON. However, it is pretty printed JSON with around 60k lines. A
while ((line = bufferedReader.readLine()) != null) {
lineAllOfIt += line;
}
takes around 10 minutes to read this response. There must be something wrong with my approach, but I cannot picture a better approach.
For this particular case, I would cache the file locally using java you can have a low memory transfer of the file to your computer, then you can go through it line by line without loading the file into memory as well and pull out the data you need or loading it all at once.
EDIT: Made changes on variable names i pulled this from my code and forgot to neutralize the variables. Also FileChannel transferTo/transferFrom can be much more efficient as there is potentially less copies and depending on operation could go from the SocketBuffer -> Disk. FileChannel API
String urlString = "http://update.domain.com/file.json" // File URL Path
Path diskSaveLocation = Paths.get("file.json"); // This will be just help place it in your working directory
final URL url = new URL(fileUrlString);
final URLConnection conn = url.openConnection();
final long fileLength = conn.getContentLength();
System.out.println(String.format("Downloading file... %s, Size: %d bytes.", fileUrlString, fileLength));
try(
FileOutputStream stream = new FileOutputStream(diskSaveLocation.toFile(), false);
FileChannel fileChannel = stream.getChannel();
ReadableByteChannel inChannel = Channels.newChannel(conn.getInputStream());
) {
long read = 0;
long readerPosition = 0;
while ((read = fileChannel.transferFrom(inChannel, readerPosition, fileLength)) >= 0 && readerPosition < fileLength) {
readerPosition += read;
}
if (fileLength != Files.size(diskSaveLocation)) {
Files.delete(diskSaveLocation);
System.out.println(String.format("File... %s did not download correctly, deleting file artifact!", fileUrlString));
}
}
System.out.println(String.format("File Download... %s completed!", fileUrlString));
((HttpURLConnection) conn).disconnect();
You can now read this same file using a NIO2 method that allows you to read line by line without loading into memory. Using Scanner or RandomAccessFile methods you can prevent reading lines into the heap. If you want to read the whole file in you can also do so locally from the cached file using many of the methods from Javas Files utility methods.
Java Read Large Text File With 70million line of text
I am using the following code but I am not able to rename the file.There is no such file newname.txt in the system as well. I am using JRE6. Please help me out! Also I have tried to rename using Files class, That too isn't working.
File f1 = new File("oldname.txt");
File f2 = new File("newname.txt");
boolean b = f1.renameTo(f2);
The same code gets execute on SunOs UNIX but not on my windows 7. Why is it so ? Can I do something to execute it on my local machine?
File path to oldname.txt is not correct, Try giving the absolute path and check it works like "c:\oldname.txt"
File f1 = new File("c:\\oldname.txt");
File f2 = new File("c:\\newname.txt");
boolean b = f1.renameTo(f2);
at the end print the value of b
Ref: Rename a file using Java (edited little bit to fit your requirment)
File f1 = new File("oldname.txt");
File f2 = new File("newname.txt");
if (!f1.exists())
throw new java.io.FileNotFoundException("file not found");
if (f2.exists())
throw new java.io.IOException("file exists");
// Rename file (or directory)
boolean success = f1.renameTo(f2);
if (!success) {
// File was not successfully renamed
}
I have a file:
RandomAccessFile file = new RandomAccessFile("files/bank.txt", "r");
And I am reading and writing data to/from this file using:
for (int pos = 0; pos < 1000; pos++) {
file.seek(40 * pos + 30);
double money = file.readDouble();
if (pos == num) {
money += i;
System.out.println(money+" "+i);
file.seek(40 * pos + 30);
file.writeDouble(money);
}
}
In this way it reads the double - works correctly, and then it should overwrite that double with the value it held previously plus i. However this is not working as the value doesn't change. What have I done wrong?
You have opened file only for reading:
RandomAccessFile("files/bank.txt", "r");
you should open it with:
new RandomAccessFile("files/bank.txt", "rws");
which opens for reading and writing, as with "rw", and also require that every update to the file's content or metadata be written synchronously to the underlying storage device.
or with:
new RandomAccessFile("files/bank.txt", "rwd");
which opens for reading and writing, as with "rw", and also require that every update to the file's content be written synchronously to the underlying storage device.
You have the file open for reading only:
RandomAccessFile file = new RandomAccessFile("files/bank.txt", "r");
Change that line to:
RandomAccessFile file = new RandomAccessFile("files/bank.txt", "rw");
And you will be able to update with the code you have above. :)
This is a classic debug error, you should've breakpointed at money += i to see the value change, the document not changing is another issue and should have been addressed as the possibility of the file not being writable. i.e. you should've opened for writing.
Java
The code below was written to read all files in, and send the data to another method (setOutput()), and then call a method to rename the last read file to another directory, and then delete the original. Everything seems to work up until the smdrCleanup() method is called. The renameTo() is failing.
From what I understand, if a FileReader is wrapped in a BufferedReader, I only need to call BufferedReader.close() to release the last read file... which I am doing here.
I have also seen where if the file were still "open", being scanned by anti-virus programs, or otherwise locked by a process, the renameTo() function would fail. I have used Process Explorer to review what may have it locked, and I don't see anything locking it.
I have my method setup to throw any kind of IOExceptions, but I am not getting any exceptions. Everything runs, but the console merely says that the file was not copied.
I am running Eclipse Helios Release 2, Windows 7 Ultimate, local administrator, UAC disabled.
Any help would be greatly appreciated.
public void smdrReader(String path, String oldPath) throws IOException
{
output = null; //nullify the value of output to avoid duplicate data
File folder = new File(path); //setting the directory for raw data files
File[] listOfFiles = folder.listFiles(); //array of files within the folder/directory
//For loop to iterate through the available files, open, & read contents to String Buffer.
for (int i = 0; i < listOfFiles.length; i++)
{
if (listOfFiles[i].isFile()) //verifying next entry in array is a file
{
File fileName = new File(listOfFiles[i].getName());//getting file name from array iteration
StringBuffer fileData = new StringBuffer(2048);//establishing StringBuffer for reading file contents into
BufferedReader reader = new BufferedReader(new FileReader(path + fileName));//reader to actually access/read file
String readData = String.valueOf(reader);//String variable being set to value of the reader
fileData.append(readData);//appending data from String variable into StringBuffer variable
setOutput(fileData);//setting the value of "output" to the value of StringBuffer
fileData.delete(0, i);
reader.close();//closing the reader (closing the file)
smdrCleanup(oldPath,fileName.toString());//calling method to move processed file and delete original
}
}
}
//method to rename input file into another directory and delete the original file to avoid duplicate processing
public void smdrCleanup(String oldPathPassed, String fileNamePassed) throws IOException
{
File oldFile = new File(oldPathPassed);//establishing File object for path to processed folder
File fileName = new File(fileNamePassed);//establishing File object for the fileName to rename/delete
String oldFilePath = oldFile.toString();
boolean success = fileName.renameTo(new File(oldFilePath + "\\" + fileName + ".old"));//attempting to rename file
if (!success) //checking the success of the file rename operation
{
System.out.println("The File Was NOT Successfully Moved!");//reporting error if the file cannot be renamed
}
else
{
fileName.delete();//deleting the file if it was successfully renamed
}
}
oldFile.toString(); returns the full path of the file including its file name, so if your old file path is c:\path\to\file.txt, oldFilePath + "\\" + fileName + ".old" will be c:\path\to\file.txt\file.txt.old.
Since there is no folder c:\path\to\file.txt, it fails. change it to
boolean success = fileName.renameTo(new File(oldFilePath + ".old"));
And you should be good to go.
File.renameTo can fail for any number of reasons:
Many aspects of the behavior of this
method are inherently
platform-dependent: The rename
operation might not be able to move a
file from one filesystem to another,
it might not be atomic, and it might
not succeed if a file with the
destination abstract pathname already
exists. The return value should always
be checked to make sure that the
rename operation was successful.
But there's not much feedback on why it fails. Before calling renameTo, verify that the file you're moving exists, and the parent directory you are moving it to also exists, and canWrite(). Are these on the same disk volume? If not, it might fail.
*EDIT: Code sample added *
Try something like the following. Modifications:
Accepts File objects instead of Strings
Uses 2-arg File constructor to create a child File object in a parent directory
Better error checking
This might give you some clues into what is failing.
public void smdrCleanup(File oldPathPassed, File fileNamePassed) throws IOException {
if (!oldPathPassed.exists() || !oldPathPassed.isDirectory() || !oldPathPassed.canWrite() ) throw new IOException("Dest is not a writable directory: " + oldPathPassed);
if (!fileNamePassed.exists()) throw new IOException("File does not exist: " + fileNamePassed);
final File dest = new File(oldPathPassed, fileNamePassed + ".old");
if (dest.exists()) throw new IOException("File already exists: " + dest);
boolean success = (fileNamePassed).renameTo(dest);//attempting to rename file
if (!success) //checking the success of the file rename operation
{
throw new IOException("The File Was NOT Successfully Moved!");
} else {
// file was successfully renamed, no need to delete
}
}