Develop Reference

Java: How to hold off read/writing a file while it's locked

I have a number of separate applications (all written in Java) that need to access a file for short periods of time. Some of these processes will need read access while others will need to update the file. So that all of these applications play nicely I want to write some code that will use OS locks to gain exclusive access to the file for the time that each application requires it.
The obvious way to do this RandomAccessFile myFile = new RandomAccessFile(file, "rw"), however this will fail if another process already has the lock. What I need is the ability to back off and try again.
I was hoping to write some code that uses channel.tryLock() to test if a lock has been taken out. The trouble is I need a channel, and I don't appear able to get that channel object with out taking out a lock!
Update
I need to find a way of checking to see if there is a lock on a file. I want to do this without throwing an exception.
A simplified version of my code is:
void myMethod(File myFile) {
try (
RandomAccessFile myFile = new RandomAccessFile(myFile, "rw"); // Exception here
FileChannel myChannel = myFile.getChannel();
FileLock myLock = lockFile(myChannel )
) {
// Stuff to read and/or write the file
}
}
private FileLock lockFile(FileChannel channel) throws Exception {
FileLock lock;
while (lock = channel.tryLock() == null) {
Thread.sleep(100);
}
return lock;
}
the problem is that if the file is locked it fails (by throwing an exception) on the highlighted line - before the point that can get a lock for the file.
Other variations for obtaining the channel such as FileChannel channel = FileChannel.open(myFile.toPath(), StandardOpenOption.WRITE, StandardOpenOption.APPEND) also throw an exception:
Exception in thread "main" java.nio.file.FileSystemException: path\to\my\file.txt: The process cannot access the file because it is being used by another process.
So how can I get a channel to test the lock with with out throwing an exception?

The standard way to use FileLock, is to open the file, e.g. via FileChannel.open, followed by tryLock. The presence of a lock does not prevent other processes from opening the file.
This can be demonstrated by the following program:
import java.io.IOException;
import java.nio.channels.*;
import java.nio.file.*;
class Locking {
public static void main(String[] args) throws IOException, InterruptedException {
if(args.length > 0) {
String me = String.format("%6s ", ProcessHandle.current());
Path p = Paths.get(args[0]);
try(FileChannel fc = FileChannel.open(p,
StandardOpenOption.WRITE, StandardOpenOption.APPEND)) {
FileLock l = fc.tryLock();
if(l == null) System.out.println(me + "could not acquire lock");
else {
System.out.println(me + "got lock");
Thread.sleep(3000);
System.out.println(me + "releasing lock");
l.release();
}
}
}
else {
Path p = Files.createTempFile("lock", "test");
String[] command = {
Paths.get(System.getProperty("java.home"), "bin", "java").toString(),
"-cp", System.getProperty("java.class.path"),
"Locking", p.toString()
};
ProcessBuilder b = new ProcessBuilder(command).inheritIO();
Process p1 = b.start(), p2 = b.start(), p3 = b.start();
p1.waitFor();
p2.waitFor();
p3.waitFor();
Files.delete(p);
}
}
}
which prints something alike
12116 got lock
13948 could not acquire lock
13384 could not acquire lock
12116 releasing lock
which can be demonstrated online on tio.run
While this program works the same under Windows, this operating system supports opening files unshared, preventing other processes from opening. If a different process has opened the file in that way, we can’t even open it to probe the locking state.
This is not the way, Java opens the file, however, there’s a non-standard open option to replicate the behavior, com.sun.nio.file.ExtendedOpenOption.NOSHARE_WRITE. In recent JDKs, it’s in the jdk.unsupported module.
When we run the following extended test program under Windows
import java.io.IOException;
import java.nio.channels.*;
import java.nio.file.*;
import java.util.HashSet;
import java.util.Set;
class LockingWindows {
public static void main(String[] args) throws IOException, InterruptedException {
if(args.length > 0) {
String me = String.format("%6s ", ProcessHandle.current());
Path p = Paths.get(args[0]);
Set<OpenOption> options
= Set.of(StandardOpenOption.WRITE, StandardOpenOption.APPEND);
if(Boolean.parseBoolean(args[1])) options = addExclusive(options);
try(FileChannel fc = FileChannel.open(p, options)) {
FileLock l = fc.tryLock();
if(l == null) System.out.println(me + "could not acquire lock");
else {
System.out.println(me + "got lock");
Thread.sleep(3000);
System.out.println(me + "releasing lock");
l.release();
}
}
}
else {
Path p = Files.createTempFile("lock", "test");
String[] command = {
Paths.get(System.getProperty("java.home"), "bin", "java").toString(),
"-cp", System.getProperty("java.class.path"),
"LockingWindows", p.toString(), "false"
};
ProcessBuilder b = new ProcessBuilder(command).inheritIO();
for(int run = 0; run < 2; run++) {
Process p1 = b.start(), p2 = b.start(), p3 = b.start();
p1.waitFor();
p2.waitFor();
p3.waitFor();
if(run == 0) {
command[command.length - 1] = "true";
b.command(command);
System.out.println("\nNow with exclusive mode");
}
}
Files.delete(p);
}
}
private static Set<OpenOption> addExclusive(Set<OpenOption> options) {
OpenOption o;
try {
o = (OpenOption) Class.forName("com.sun.nio.file.ExtendedOpenOption")
.getField("NOSHARE_WRITE").get(null);
options = new HashSet<>(options);
options.add(o);
} catch(ReflectiveOperationException | ClassCastException ex) {
System.err.println("opening exclusive not supported");
}
return options;
}
}
we will get something like
2356 got lock
6412 could not acquire lock
9824 could not acquire lock
2356 releasing lock
Now with exclusive mode
9160 got lock
Exception in thread "main" java.nio.file.FileSystemException: C:\Users\...\Temp\lock13936982436235244405test: The process cannot access the file because it is being used by another process
at java.base/sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:92)
at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:103)
at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:108)
at java.base/sun.nio.fs.WindowsFileSystemProvider.newFileChannel(WindowsFileSystemProvider.java:121)
at java.base/java.nio.channels.FileChannel.open(FileChannel.java:298)
at LockingWindows.main(LockingWindows.java:148)
Exception in thread "main" java.nio.file.FileSystemException: C:\Users\...\Temp\lock13936982436235244405test: The process cannot access the file because it is being used by another process
at java.base/sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:92)
at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:103)
at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:108)
at java.base/sun.nio.fs.WindowsFileSystemProvider.newFileChannel(WindowsFileSystemProvider.java:121)
at java.base/java.nio.channels.FileChannel.open(FileChannel.java:298)
at LockingWindows.main(LockingWindows.java:148)
9160 releasing lock
The similarity to the outcome of your test suggests that the Windows program you ran concurrently to your Java program did use such a mode.
For your Java programs, no such issue should arise, as long as you don’t use that mode. Only when you have to interact with another Windows program not using the collaborative locking, you have to deal with this.

FileLock in Java doesn't work in Docker mount volume

In my situation, some web configuration files are shared through mount folder in Docker. In same cases we want to modify these files concurrently. That's why I want to use lock to make sure file is being modified once at the same time. But I found flock is not working in Docker. Does it not supported?
public void modifyFile() {
try {
File file = new File("/tmp/fileToLock.dat");
// Creates a random access file stream to read from, and optionally to write to
FileChannel channel = new RandomAccessFile(file, "rw").getChannel();
// Acquire an exclusive lock on this channel's file (blocks until lock can be retrieved)
FileLock lock = null;
// Attempts to acquire an exclusive lock on this channel's file (returns null or throws
// an exception if the file is already locked.
try {
lock = channel.tryLock();
if (null != lock) {
List<String> fileToString = FileUtils.readLines(file, StandardCharsets.UTF_8);
long l = 0l;
if (null != fileToString && fileToString.size() > 0) {
l = Long.valueOf(fileToString.get(fileToString.size() - 1));
}
l++;
FileUtils.writeStringToFile(file, String.valueOf(l) + "\r\n", StandardCharsets.UTF_8, true);
}
} catch (OverlappingFileLockException e) {
// thrown when an attempt is made to acquire a lock on a a file that overlaps
// a region already locked by the same JVM or when another thread is already
// waiting to lock an overlapping region of the same file
System.out.println("Overlapping File Lock Error: " + e.getMessage());
channel.close();
}
// release the lock
if (null != lock) {
lock.release();
}
// close the channel
channel.close();
} catch (IOException e) {
System.out.println("I/O Error: " + e.getMessage());
}
}

In the end, I just leave any lock implementation for file without any Redis lock thing.

Locking a process in Java

I have a Java servlet which calls another software (say S) over a TCP connection. This software S uses a network resource, and the output has to be retrived from a hyperlink(using wget).
Since it's the same hyperlink I need to download my result from (irrespective of the request), it results into incorrect results few requests. I basically need to lock the use of this network resource across different processes (I believe each call from the servlet is going to create a new process).
I tried to use ReentrantLock (but I guess it only works with threads and not accross processes).
Please let me know how can this be achieved.
Thanks

Here is how to do cross-process locking in Java. Adjust to your needs and add error/exception checking/handling as necessary.
// Tester
try {
if (crossProcessLockAcquire(SomeClassInYourApp.class, 3000)) {
// Success - This process now has the lock. (Don't keep it too long.)
}
else {
// Fail (Timeout) - Another process still had the lock after 3 seconds.
}
} finally {
crossProcessLockRelease(); // try/finally is very important.
}
// Acquire - Returns success ( true/false )
private static boolean crossProcessLockAcquire(final Class<?> c, final long waitMS) {
if (fileLock == null && c != null && waitMS > 0) {
try {
long dropDeadTime = System.currentTimeMillis() + waitMS;
File file = new File(lockTempDir, c.getName() + ".lock");
RandomAccessFile randomAccessFile = new RandomAccessFile(file, "rw");
FileChannel fileChannel = randomAccessFile.getChannel();
while (System.currentTimeMillis() < dropDeadTime) {
fileLock = fileChannel.tryLock();
if (fileLock != null) {
break;
}
Thread.sleep(250); // 4 attempts/sec
}
} catch (Exception e) {
e.printStackTrace();
}
}
return fileLock == null ? false : true;
}
// Release
private static void crossProcessLockRelease() {
if (fileLock != null) {
try {
fileLock.release();
fileLock = null;
} catch (IOException e) {
e.printStackTrace();
}
}
}
// Some class vars and a failsafe lock release.
private static File lockTempDir = new File(System.getProperty("java.io.tmpdir") + File.separator + "locks");
private static FileLock fileLock = null;
static {
Runtime.getRuntime().addShutdownHook(new Thread() {
public void run(){
crossProcessLockRelease();
}
});
}

Why are you reusing this TCP connection? If it's easy to set up, just set one up every time you need it. For example, with an HTTP request, you should just make a new request every time.
My guess is that you have something static that shouldn't be, so multiple threads are using it when they should all have their own version.
If they're expensive, consider creating one-per-thread with ThreadLocal.
If even that doesn't work, and you don't mind threads blocking, just add "synchronized" to the method that's causing the problem.

The resource you are trying to lock has to support looking. It would be better if the service didn't need to be locked externally.
As a work around you can use a ServerSocket to lock a resource between processes.

Java - locking a file for exclusive access

My problem is this: I'm using a WatchService to get notified about new files in a specific folder, now if a file gets moved/copied or created in said folder an event gets triggered and the name of the new file gets returned. The problem now is, if I try to access the file and it is not fully there yet (e.g. the copy is still in progress) an exception gets raised. What i tried was to do something like this:
RandomAccessFile raf = new RandomAccessFile(fileName, "rw");
FileChannel fc = raf.getChannel();
FileLock lck = fc.lock();
But even if a lock gets acquired, sometimes still an Exception gets raised if I try to write to the file because another process has still an open handle to it.
Now, how can a file in Java be locked for truly exclusive access?

For me, the statement
RandomAccessFile raf = new RandomAccessFile(fileName, "rw");
returns a FileNotFoundException if I cannot acquire a lock on the file. I catch the filenotfound exception and treat it...
public static boolean isFileLocked(String filename) {
boolean isLocked=false;
RandomAccessFile fos=null;
try {
File file = new File(filename);
if(file.exists()) {
fos=new RandomAccessFile(file,"rw");
}
} catch (FileNotFoundException e) {
isLocked=true;
}catch (Exception e) {
// handle exception
}finally {
try {
if(fos!=null) {
fos.close();
}
}catch(Exception e) {
//handle exception
}
}
return isLocked;
}
you could run this in a loop and wait until you get a lock on the file. Wouldn't the line
FileChannel fc = raf.getChannel();
never reach if the file is locked? You will get a FileNotFoundException thrown..

its better not to use classes in thejava.io package, instead use the java.nio package .
The latter has a FileLock class that you can use for a lock to a FileChannel.
try {
// Get a file channel for the file
File file = new File("filename");
FileChannel channel = new RandomAccessFile(file, "rw").getChannel();
// Use the file channel to create a lock on the file.
// This method blocks until it can retrieve the lock.
FileLock lock = channel.lock();
/*
use channel.lock OR channel.tryLock();
*/
// Try acquiring the lock without blocking. This method returns
// null or throws an exception if the file is already locked.
try {
lock = channel.tryLock();
} catch (OverlappingFileLockException e) {
// File is already locked in this thread or virtual machine
}
// Release the lock - if it is not null!
if( lock != null ) {
lock.release();
}
// Close the file
channel.close();
} catch (Exception e) {
}

Using java file locks within single JVM and across multiple JVMs

I guess I miss something, but I cannot understand how file locks work in Java. To be more exact - how it is implemented.
It seems I cannot acquire (even cannot attempt acquiring) two or more locks for the same file inside single JVM. First lock will be successfully acquired, all further attempts to acquire more locks will result in OverlapingFileLockException. Nevertheless it works for separate processes.
I want to implement data-storage backed by file-system which is intended to work with multiple concurrent requests (both read and write). I want to use file locks to lock on particular files in the storage.
It seems that I have to introduce one more synchronization (exclusive) on JVM-level and only then sync on files to avoid this exception.
Did anyone do anything like that?
I prepared simple test case to show what my problem is. I use Mac OS X, Java 6.
import junit.framework.*;
import javax.swing.*;
import java.io.*;
import java.nio.channels.*;
/**
* Java file locks test.
*/
public class FileLocksTest extends TestCase {
/** File path (on Windows file will be created under the root directory of the current drive). */
private static final String LOCK_FILE_PATH = "/test-java-file-lock-tmp.bin";
/**
* #throws Exception If failed.
*/
public void testWriteLocks() throws Exception {
final File file = new File(LOCK_FILE_PATH);
file.createNewFile();
RandomAccessFile raf = new RandomAccessFile(file, "rw");
System.out.println("Getting lock...");
FileLock lock = raf.getChannel().lock();
System.out.println("Obtained lock: " + lock);
Thread thread = new Thread(new Runnable() {
#Override public void run() {
try {
RandomAccessFile raf = new RandomAccessFile(file, "rw");
System.out.println("Getting lock (parallel thread)...");
FileLock lock = raf.getChannel().lock();
System.out.println("Obtained lock (parallel tread): " + lock);
lock.release();
}
catch (Throwable e) {
e.printStackTrace();
}
}
});
thread.start();
JOptionPane.showMessageDialog(null, "Press OK to release lock.");
lock.release();
thread.join();
}
/**
* #throws Exception If failed.
*/
public void testReadLocks() throws Exception {
final File file = new File(LOCK_FILE_PATH);
file.createNewFile();
RandomAccessFile raf = new RandomAccessFile(file, "r");
System.out.println("Getting lock...");
FileLock lock = raf.getChannel().lock(0, Long.MAX_VALUE, true);
System.out.println("Obtained lock: " + lock);
Thread thread = new Thread(new Runnable() {
#Override public void run() {
try {
RandomAccessFile raf = new RandomAccessFile(file, "r");
System.out.println("Getting lock (parallel thread)...");
FileLock lock = raf.getChannel().lock(0, Long.MAX_VALUE, true);
System.out.println("Obtained lock (parallel thread): " + lock);
lock.release();
}
catch (Throwable e) {
e.printStackTrace();
}
}
});
thread.start();
JOptionPane.showMessageDialog(null, "Press OK to release lock.");
lock.release();
thread.join();
}
}

From the Javadoc:
File locks are held on behalf of the
entire Java virtual machine. They are
not suitable for controlling access to
a file by multiple threads within the
same virtual machine.

You can only acquire a lock once per file. Locks are not re-entrant AFAIK.
IMHO: Using files to communicate between process is a very bad idea. Perhaps you will be able to get this to work reliably, let me know if you can ;)
I would have one and only one thread read/write in only one process.

Have you checked the documentation? The FileChannel.lock() method returns an exclusive lock across the entire file. If you want to have multiple locks active concurrently across different threads, then you cannot use this method.
Instead you need to use FileChannel.locklock(long position, long size, boolean shared) in order to lock a specific region of the file. This will allow you to have multiple locks active at the same time, provided that each one is applied to a different region of the file. If you attempt to lock the same region of the file twice, you will encounter the same exception.