I have two java process (JAR) one is writing to a text file on every 1 min and another is reading that file and call a web service to store data in database.
Is there any way to lock the file when it is on write mode? I have observed that when wvdial is dialing a modem its create a lock file in /var/lock/ttyUSB0..LOCK I think. I want a this kind of procedure if the file is on write mode the another process could wait till write done. After writing the process can read the file content.
Please guide me to solve my issue.
Thank you
Maybe this class can help you http://docs.oracle.com/javase/7/docs/api/java/nio/channels/FileLock.html
Edit: This post might already be covering the subject How can I lock a file using java (if possible)
Exemple:
FileInputStream in = new FileInputStream(file);
try
{
java.nio.channels.FileLock lock = in.getChannel().lock();
try
{
//write
}
finally
{
lock.release();
}
}
finally
{
in.close();
}
Now in the reading process:
FileInputStream in = new FileInputStream(file);
try
{
FileLock lock = in.getChannel().tryLock();
if (lock == null)
{
//file is locked, wait or do something else
}
else
{
try
{
//read
}
finally
{
lock.release();
}
}
}
finally
{
in.close();
}
The problem you will have here is that Java cannot open() with O_EXCL, as a result you cannot create a file atomically.
Use a directory instead: creating a directory is an atomic operation. File's .mkdir() will return false if the directory cannot be created. rmdir() it when you're done.
Of course, make sure that both of your processes have write access to the base directory!
Related
A logic that handles the rollback of a write to a file is this possible?
From my understanding a BufferWriter only writes when a .close() or .flush() is invoked.
I would like to know is it possible to, rollback a write or undo any changes to a file when an error has occurred?
This means that the BufferWriter acts as a temporary storage to store the changes done to a file.
How big is what you're writing? If it isn't too big, then you could write to a ByteArrayOutputStream so you're writing in memory and not affecting the final file you want to write to. Only once you've written everything to memory and have done whatever you want to do to verify that everything is OK can you write to the output file. You can pretty much be guaranteed that if the file gets written to at all, it will get written to in its entirety (unless you run out of disk space.). Here's an example:
import java.io.*;
class Solution {
public static void main(String[] args) {
ByteArrayOutputStream os = new ByteArrayOutputStream();
try {
// Do whatever writing you want to do here. If this fails, you were only writing to memory and so
// haven't affected the disk in any way.
os.write("abcdefg\n".getBytes());
// Possibly check here to make sure everything went OK
// All is well, so write the output file. This should never fail unless you're out of disk space
// or you don't have permission to write to the specified location.
try (OutputStream os2 = new FileOutputStream("/tmp/blah")) {
os2.write(os.toByteArray());
}
} catch (IOException e) {
e.printStackTrace();
}
}
}
If you have to (or just want to) use Writers instead of OutputStreams, here's the equivalent example:
Writer writer = new StringWriter();
try {
// again, this represents the writing process that you worry might fail...
writer.write("abcdefg\n");
try (Writer os2 = new FileWriter("/tmp/blah2")) {
os2.write(writer.toString());
}
} catch (IOException e) {
e.printStackTrace();
}
It is impossible to rollback or undo changes already applied to files/streams,
but there are tons of alternatives to do so:
One simple trick is to clean the destination and redo the process again, to clean the file:
PrintWriter writer = new PrintWriter(FILE_PATH);
writer.print("");
// other operations
writer.close();
You can remove the content entirely and re-run again.
Or if you are sure the last line(s) are the problems, you may do remove last line actions for your purpose, such as rollback the line instead:
Delete last line in text file
I want to check if the file exist on server on multithread environment and if exists return that file content diractly or download from my s3 service server.
My code like this:
final Object lock = new Object();
File file = new File("/file/path");
if (file.exists()) {
return FileUtils.readFileToByteArray(file);
} else {
byte[] bytes = this.downloadFileFromRemoteServer();
if (!file.exists()) {
synchronized (lock) {
if(!file.exists()) {
FileUtils.writeByteArrayToFile(tempFile, bytes);
}
}
}
tempFile.renameTo(file);
return bytes;
}
The above code similar java double checked locking, is method file.exists() behavior like volatile keyword? And pseudo code correctly?
File.exists() checks the file existence with the file-system, and so it should behave like a volatile, so you are covered there
Some issues though -
1) As soon as a thread sees that the file doesn't exist, it starts downloading the file, which is time consuming, so its likely that other threads will also come and start downloading the same file. So the download part should be moved inside the lock
2) You're renaming the temp file outside the lock. A thread may get to that point without creating/writing-to a temp file. Should move the rename inside the lock as-well
Since IO has much more overhead than locking, I think the above 2 steps would be beneficial
You are overly cautious: since you are writing to a temp file, there is no risk of overwriting an existing file, which carries a possibility of reading a half-written file: your reads are going to be consistent.
The only issue that your code is protecting against is writing the same downloaded content into multiple temporary files, which is not much of a performance problem in comparison to multiple downloads, which would happen anyway.
I would simplify your code as follows:
File file = new File("/file/path");
if (!file.exists()) {
byte[] bytes = this.downloadFileFromRemoteServer();
File tempFile = File.createTempFile(...);
FileUtils.writeByteArrayToFile(tempFile, bytes);
tempFile.renameTo(file);
}
return FileUtils.readFileToByteArray(file);
I need to write a custom batch File renamer. I've got the bulk of it done except I can't figure out how to check if a file is already open. I'm just using the java.io.File package and there is a canWrite() method but that doesn't seem to test if the file is in use by another program. Any ideas on how I can make this work?
Using the Apache Commons IO library...
boolean isFileUnlocked = false;
try {
org.apache.commons.io.FileUtils.touch(yourFile);
isFileUnlocked = true;
} catch (IOException e) {
isFileUnlocked = false;
}
if(isFileUnlocked){
// Do stuff you need to do with a file that is NOT locked.
} else {
// Do stuff you need to do with a file that IS locked
}
(The Q&A is about how to deal with Windows "open file" locks ... not how implement this kind of locking portably.)
This whole issue is fraught with portability issues and race conditions:
You could try to use FileLock, but it is not necessarily supported for your OS and/or filesystem.
It appears that on Windows you may be unable to use FileLock if another application has opened the file in a particular way.
Even if you did manage to use FileLock or something else, you've still got the problem that something may come in and open the file between you testing the file and doing the rename.
A simpler though non-portable solution is to just try the rename (or whatever it is you are trying to do) and diagnose the return value and / or any Java exceptions that arise due to opened files.
Notes:
If you use the Files API instead of the File API you will get more information in the event of a failure.
On systems (e.g. Linux) where you are allowed to rename a locked or open file, you won't get any failure result or exceptions. The operation will just succeed. However, on such systems you generally don't need to worry if a file is already open, since the OS doesn't lock files on open.
// TO CHECK WHETHER A FILE IS OPENED
// OR NOT (not for .txt files)
// the file we want to check
String fileName = "C:\\Text.xlsx";
File file = new File(fileName);
// try to rename the file with the same name
File sameFileName = new File(fileName);
if(file.renameTo(sameFileName)){
// if the file is renamed
System.out.println("file is closed");
}else{
// if the file didnt accept the renaming operation
System.out.println("file is opened");
}
On Windows I found the answer https://stackoverflow.com/a/13706972/3014879 using
fileIsLocked = !file.renameTo(file)
most useful, as it avoids false positives when processing write protected (or readonly) files.
org.apache.commons.io.FileUtils.touch(yourFile) doesn't check if your file is open or not. Instead, it changes the timestamp of the file to the current time.
I used IOException and it works just fine:
try
{
String filePath = "C:\sheet.xlsx";
FileWriter fw = new FileWriter(filePath );
}
catch (IOException e)
{
System.out.println("File is open");
}
I don't think you'll ever get a definitive solution for this, the operating system isn't necessarily going to tell you if the file is open or not.
You might get some mileage out of java.nio.channels.FileLock, although the javadoc is loaded with caveats.
Hi I really hope this helps.
I tried all the options before and none really work on Windows. The only think that helped me accomplish this was trying to move the file. Event to the same place under an ATOMIC_MOVE. If the file is being written by another program or Java thread, this definitely will produce an Exception.
try{
Files.move(Paths.get(currentFile.getPath()),
Paths.get(currentFile.getPath()), StandardCopyOption.ATOMIC_MOVE);
// DO YOUR STUFF HERE SINCE IT IS NOT BEING WRITTEN BY ANOTHER PROGRAM
} catch (Exception e){
// DO NOT WRITE THEN SINCE THE FILE IS BEING WRITTEN BY ANOTHER PROGRAM
}
If file is in use FileOutputStream fileOutputStream = new FileOutputStream(file); returns java.io.FileNotFoundException with 'The process cannot access the file because it is being used by another process' in the exception message.
to make sure that a java.io.File is not modified/deleted during processing, I would like to create a temporary file in a different directory than the original file (e.g. System temp directory) that
cannot be accessed by the user
but holds the information (directory, name,...) of the original file
I need the original file's information, because there are lots of accesses on file-information, such as folder-structure, filename and file-extension. Working with the temporary file would destroy this information.
Unfortunately, it is not possible to just set a file's name/directory since this would rename/move the file.
Alternative approach: One could also work on both files, grabbing the information from the source-file and reading content from the temporary file but this does not seem like the optimal way to do this.
Is there a better approach to this?
Best regards
Martin
It sounds like what you want to do is just prevent any modifications to the file while you are reading from it. This is typically accomplished by locking the file, so that only your process can access it. As an example (using FileLock from java.nio)
try{
File file = new File("randomfile.txt");
FileChannel channel = new RandomAccessFile(file, "rw").getChannel();
FileLock lock = channel.lock();//Obtain a lock on the file
try{
//Do your things
}
finally{
lock.release();//Release the lock on the file
channel.close();
}
}
catch (IOException e) {
e.printStackTrace();
}
I suggest you use java.io.File.createTempFile(String, String, File) and also use java.io.File.deleteOnExit(); the file must be accessible to the user - else the use cannot write to it (QED). That is, try something like this -
try {
File directory = new File("/tmp"); // or C:/temp ?
File f = File.createTempFile("base-temp", ".tmp", directory); // create a new
// temp file... with a prefix, suffix and in a tmp folder...
f.deleteOnExit(); // Remove it when we exit (you can still explicitly delete when you're done).
} catch (IOException e) {
e.printStackTrace();
}
I have a cluster of machines, each running a Java app.
These Java apps need to access a unique resource.txt file concurrently.
I need to atomically rename a temp.txt file to resource.txt in Java, even if resource.txt already exist.
Deleting resource.txt and renaming temp.txt doesn't work, as it's not atomic (it creates a small timeframe where resource.txt doesn't exist).
And it should be cross-platform...
For Java 1.7+, use java.nio.file.Files.move(Path source, Path target, CopyOption... options) with CopyOptions "REPLACE_EXISTING" and "ATOMIC_MOVE".
See API documentation for more information.
For example:
Files.move(src, dst, StandardCopyOption.ATOMIC_MOVE);
On Linux (and I believe Solaris and other UNIX operating systems), Java's File.renameTo() method will overwrite the destination file if it exists, but this is not the case under Windows.
To be cross platform, I think you'd have to use file locking on resource.txt and then overwrite the data.
The behavior of the file lock is
platform-dependent. On some platforms,
the file lock is advisory, which means
that unless an application checks for
a file lock, it will not be prevented
from accessing the file. On other
platforms, the file lock is mandatory,
which means that a file lock prevents
any application from accessing the
file.
try {
// Get a file channel for the file
File file = new File("filename");
FileChannel channel = new RandomAccessFile(file, "rw").getChannel();
// Use the file channel to create a lock on the file.
// This method blocks until it can retrieve the lock.
FileLock lock = channel.lock();
// Try acquiring the lock without blocking. This method returns
// null or throws an exception if the file is already locked.
try {
lock = channel.tryLock();
} catch (OverlappingFileLockException e) {
// File is already locked in this thread or virtual machine
}
// Release the lock
lock.release();
// Close the file
channel.close();
} catch (Exception e) {
}
Linux, by default, uses voluntary locking, while Windows enforces it. Maybe you could detect the OS, and use renameTo() under UNIX with some locking code for Windows?
There's also a way to turn on mandatory locking under Linux for specific files, but it's kind of obscure. You have to set the mode bits just right.
Linux, following System V (see System
V Interface Definition (SVID) Version
3), lets the sgid bit for files
without group execute permission mark
the file for mandatory locking
Here is a discussion that relates: http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4017593
As stated here, it looks like the Windows OS doesn't even support atomic file rename for older versions. It's very likely you have to use some manual locking mechanisms or some kind of transactions. For that, you might want to take a look into the apache commons transaction package.
If this should be cross-platform I suggest 2 options:
Implement an intermediate service that is responsible for all the file accesses. Here you can use several mechanisms for synchronizing the requests. Each client java app accesses the file only through this service.
Create a control file each time you need to perform synchronized operations. Each java app that accesses the file is responsible checking for the control file and waiting while this control file exists. (almost like a semaphore). The process doing the delete/rename operation is responsible for creating/deleting the control file.
If the purpose of the rename is to replace resource.txt on the fly and you have control over all the programs involved, and the frequency of replacement is not high, you could do the following.
To open/read the file:
Open "resource.txt", if that fails
Open "resource.old.txt", if that fails
Open "resource.txt" again, if that fails
You have an error condition.
To replace the file:
Rename "resource.txt" to "resource.old.txt", then
Rename "resource.new.txt" to "resource.txt", then
Delete "resource.old.txt".
Which will ensure all your readers always find a valid file.
But, easier, would be to simply try your opening in a loop, like:
InputStream inp=null;
StopWatch tmr=new StopWatch(); // made up class, not std Java
IOException err=null;
while(inp==null && tmr.elapsed()<5000) { // or some approp. length of time
try { inp=new FileInputStream("resource.txt"); }
catch(IOException thr) { err=thr; sleep(100); } // or some approp. length of time
}
if(inp==null) {
// handle error here - file did not turn up after required elapsed time
throw new IOException("Could not obtain data from resource.txt file");
}
... carry on
You might get some traction by establishing a filechannel lock on the file before renaming it (and deleting the file you're going to overwrite once you have the lock).
-r
I solve with a simple rename function.
Calling :
File newPath = new File("...");
newPath = checkName(newPath);
Files.copy(file.toPath(), newPath.toPath(), StandardCopyOption.REPLACE_EXISTING);
The checkName function checks if exits.
If exits then concat a number between two bracket (1) to the end of the filename.
Functions:
private static File checkName(File newPath) {
if (Files.exists(newPath.toPath())) {
String extractRegExSubStr = extractRegExSubStr(newPath.getName(), "\\([0-9]+\\)");
if (extractRegExSubStr != null) {
extractRegExSubStr = extractRegExSubStr.replaceAll("\\(|\\)", "");
int parseInt = Integer.parseInt(extractRegExSubStr);
int parseIntPLus = parseInt + 1;
newPath = new File(newPath.getAbsolutePath().replace("(" + parseInt + ")", "(" + parseIntPLus + ")"));
return checkName(newPath);
} else {
newPath = new File(newPath.getAbsolutePath().replace(".pdf", " (" + 1 + ").pdf"));
return checkName(newPath);
}
}
return newPath;
}
private static String extractRegExSubStr(String row, String patternStr) {
Pattern pattern = Pattern.compile(patternStr);
Matcher matcher = pattern.matcher(row);
if (matcher.find()) {
return matcher.group(0);
}
return null;
}
EDIT: Its only works for pdf. If you want other please replace the .pdf or create an extension paramter for it.
NOTE: If the file contains additional numbers between brackets '(' then it may mess up your file names.