I want to exclusively lock text file from Java code, so I found following example:
public class Main
{
public static void main(String args[]) throws IOException
{
String strFilePath = "M:/Projects/SafeFile/ClientSide/dump/data6.txt";
writeFileWithLock(new File(strFilePath), "some_content");
}
public static void writeFileWithLock(File file, String content) {
// auto close and release the lock
try (RandomAccessFile reader = new RandomAccessFile(file, "rw");
FileLock lock = reader.getChannel().lock()) {
// Simulate a 10s locked
TimeUnit.SECONDS.sleep(10);
reader.write(content.getBytes());
} catch (IOException | InterruptedException e) {
e.printStackTrace();
}
}
}
When I double click on data6.txt before 10 seconds elapse, my expectation is that I will receive some message like "File already opened by other process" or something similar. But I manage to open it without any problems. Does anybody see what is wrong with this code? Thanks!
FileLock does not generally use mandatory locks. This means that FileLock will generally provide locking only against other applications that use FileLock or equivalent.
Depending on the OS, there may be different system calls to lock files. For example fcntl or lockf.
To summarize, you may not safely assume that FileLock is a mandatory lock in every platform.
Related
I have 4 containers(java app) in 4 docker hosts. they need to read/write to the same file.
file lock cannot be used because of different OS.
So, I tried to create a .lock file, if one of the 4 containers created the .lock file, the other containers will have to wait.
But this still isn't working well. The other containers cannot see the .lock file created by other containers sometimes(not real-time).
Are there other solutions?
I suggest you rethink your assumptions:
What if you have not 4 but 400 containers?
What if they are on servers not sharing a file system?
The clean way to do this is to write a very basic server (if the load allows it, this can be nginx+PHP and be done in 10 minutes) that does the file writing, run this in another container and connect to it from the other containers. This will give you:
file locking easy and reliable, as the file is seen only by one server
scalability
clusterability
abstraction
Try File lock api to implement this. Demo in Java.
public void modifyFile() {
try {
File file = new File("/tmp/fileToLock.dat");
// Creates a random access file stream to read from, and optionally to write to
FileChannel channel = new RandomAccessFile(file, "rw").getChannel();
// Acquire an exclusive lock on this channel's file (blocks until lock can be retrieved)
FileLock lock = null;
// Attempts to acquire an exclusive lock on this channel's file (returns null or throws
// an exception if the file is already locked.
try {
lock = channel.tryLock();
if (null != lock) {
List<String> fileToString = FileUtils.readLines(file, StandardCharsets.UTF_8);
long l = 0l;
if (null != fileToString && fileToString.size() > 0) {
l = Long.valueOf(fileToString.get(fileToString.size() - 1));
}
l++;
FileUtils.writeStringToFile(file, String.valueOf(l) + "\r\n", StandardCharsets.UTF_8, true);
}
} catch (OverlappingFileLockException e) {
// thrown when an attempt is made to acquire a lock on a a file that overlaps
// a region already locked by the same JVM or when another thread is already
// waiting to lock an overlapping region of the same file
System.out.println("Overlapping File Lock Error: " + e.getMessage());
channel.close();
}
// release the lock
if (null != lock) {
lock.release();
}
// close the channel
channel.close();
} catch (IOException e) {
System.out.println("I/O Error: " + e.getMessage());
}
}
I want to read from a file that's actively being written by another application/process.
This code which I found in another question, does the job: It reads a file while it is being written and only read the new content.
The problem that it consumes a lot of CPU even if no data was added to the file. How can I optimize this code ?
Use a timer ? or a thread.sleep to pause ?
Another thing to add, the whole program I am trying to write reads a file in real-time and process its content. So this means that thread.sleep or the timer will pause my whole program. The ideal improvement I am looking for is not to wait few seconds, but for a certain event to happen => New data is added. Is this possible ?
public class FileReader {
public static void main(String args[]) throws Exception {
if(args.length>0){
File file = new File(args[0]);
System.out.println(file.getAbsolutePath());
if(file.exists() && file.canRead()){
long fileLength = file.length();
readFile(file,0L);
while(true){
if(fileLength<file.length()){
readFile(file,fileLength);
fileLength=file.length();
}
}
}
}else{
System.out.println("no file to read");
}
}
public static void readFile(File file,Long fileLength) throws IOException {
String line = null;
BufferedReader in = new BufferedReader(new java.io.FileReader(file));
in.skip(fileLength);
while((line = in.readLine()) != null)
{
System.out.println(line);
}
in.close();
}
}
The ideal improvement I am looking for is not to wait few seconds, but
for a certain event to happen => New data is added. Is this possible ?
Best solution : data pushing :
The application that produces the content should inform the other application as it may read the content.
You could use any channel that may convey the information between two distinct applications.
For example by using a specific file that the writer updates to notify new things to read.
The writer could write/overwrite in this file a update date and the reader would read the data file only if it doesn't read any content since this date.
A more robust way but with more overhead could be exposing a notification service from the reader side.
Why not a REST service.
In this way, the writer could notify the reader via the service as new content is ready.
Another thing to add, the whole program I am trying to write reads a
file in real-time and process its content. So this means that
thread.sleep or the timer will pause my whole program.
A workaround solution : data pulling performed by a specific thread :
You have probably a multi-core CPU.
So create a separate thread to read the produced file to allow other threads of your application to be runnable.
Besides, you also could performs some regular pause : Thread.sleep() to optimize the core use done by the reading thread.
It could look like :
Thread readingThread = new Thread(new MyReadingProcessing());
readingThread.start();
Where MyReadingProcessing is a Runnable :
public class MyReadingProcessing implements Runnable{
public void run(){
while (true){
readFile(...);
try{
Thread.sleep(1000); // 1 second here but choose the deemed reasonable time according the metrics of your producer application
}
catch(InterruptedException e){
Thread.currentThread().interrupt();
}
if (isAllReadFile()){ // condition to stop the reading process
return;
}
}
}
}
Instead of a busy wait loop use a WatchService on directory entries changing.
Path path = Paths.get("...");
try (WatchService watchService =
path.getFileSystem().newWatchService()) {
WatchKey watchKey = path.register(watchService,
StandardWatchEventKinds.ENTRY_MODIFY);
for (;;) { // watchKey.poll with timeout, or take, blocking
WatchKey taken = watchService.take();
for (WatchEvent<?> event : taken.pollEvents()) {
Path changedPath = (Path) event.context();
if (changedPath.equals(path)) {
...
}
}
boolean valid = taken.reset();
if (!valid) {
... unregistered
}
}
}
Note that the above has to be adapted to use poll or take.
I wrote a method which replace some lines in a file (it's not the purpose of this question). Everything works fine, but I'm wondering if file is closed for reading when I start writing. I'd like to ensure that my solution is safe. That's what I've done:
private void replaceCodeInTranslationFile(File file, String code) {
if (file.exists()) {
try (Stream<String> lines = Files.lines(Paths.get(file.getAbsolutePath()), Charset.defaultCharset())) {
String output = this.getLinesWithUpdatedCode(lines, code);
this.replaceFileWithContent(file, output); // is it safe?
} catch (IOException e) {
throw new UncheckedIOException(e);
}
}
}
Method replaceFileWithContent() looks like this:
private void replaceFileWithContent(File file, String content) throws IOException {
try (FileOutputStream fileOut = new FileOutputStream(file.getAbsolutePath())) {
fileOut.write(content.getBytes(Charset.defaultCharset()));
}
}
I think that try-with-resources closes resource at the end of a statement, so this code can be potentially the source of problems. Am I correct?
Read / Write lock implementations may be helpful for this kind of scenario to ensure thread safe operations.
Refer this http://tutorials.jenkov.com/java-concurrency/read-write-locks.html for more..
I have a Java servlet which calls another software (say S) over a TCP connection. This software S uses a network resource, and the output has to be retrived from a hyperlink(using wget).
Since it's the same hyperlink I need to download my result from (irrespective of the request), it results into incorrect results few requests. I basically need to lock the use of this network resource across different processes (I believe each call from the servlet is going to create a new process).
I tried to use ReentrantLock (but I guess it only works with threads and not accross processes).
Please let me know how can this be achieved.
Thanks
Here is how to do cross-process locking in Java. Adjust to your needs and add error/exception checking/handling as necessary.
// Tester
try {
if (crossProcessLockAcquire(SomeClassInYourApp.class, 3000)) {
// Success - This process now has the lock. (Don't keep it too long.)
}
else {
// Fail (Timeout) - Another process still had the lock after 3 seconds.
}
} finally {
crossProcessLockRelease(); // try/finally is very important.
}
// Acquire - Returns success ( true/false )
private static boolean crossProcessLockAcquire(final Class<?> c, final long waitMS) {
if (fileLock == null && c != null && waitMS > 0) {
try {
long dropDeadTime = System.currentTimeMillis() + waitMS;
File file = new File(lockTempDir, c.getName() + ".lock");
RandomAccessFile randomAccessFile = new RandomAccessFile(file, "rw");
FileChannel fileChannel = randomAccessFile.getChannel();
while (System.currentTimeMillis() < dropDeadTime) {
fileLock = fileChannel.tryLock();
if (fileLock != null) {
break;
}
Thread.sleep(250); // 4 attempts/sec
}
} catch (Exception e) {
e.printStackTrace();
}
}
return fileLock == null ? false : true;
}
// Release
private static void crossProcessLockRelease() {
if (fileLock != null) {
try {
fileLock.release();
fileLock = null;
} catch (IOException e) {
e.printStackTrace();
}
}
}
// Some class vars and a failsafe lock release.
private static File lockTempDir = new File(System.getProperty("java.io.tmpdir") + File.separator + "locks");
private static FileLock fileLock = null;
static {
Runtime.getRuntime().addShutdownHook(new Thread() {
public void run(){
crossProcessLockRelease();
}
});
}
Why are you reusing this TCP connection? If it's easy to set up, just set one up every time you need it. For example, with an HTTP request, you should just make a new request every time.
My guess is that you have something static that shouldn't be, so multiple threads are using it when they should all have their own version.
If they're expensive, consider creating one-per-thread with ThreadLocal.
If even that doesn't work, and you don't mind threads blocking, just add "synchronized" to the method that's causing the problem.
The resource you are trying to lock has to support looking. It would be better if the service didn't need to be locked externally.
As a work around you can use a ServerSocket to lock a resource between processes.
I guess I miss something, but I cannot understand how file locks work in Java. To be more exact - how it is implemented.
It seems I cannot acquire (even cannot attempt acquiring) two or more locks for the same file inside single JVM. First lock will be successfully acquired, all further attempts to acquire more locks will result in OverlapingFileLockException. Nevertheless it works for separate processes.
I want to implement data-storage backed by file-system which is intended to work with multiple concurrent requests (both read and write). I want to use file locks to lock on particular files in the storage.
It seems that I have to introduce one more synchronization (exclusive) on JVM-level and only then sync on files to avoid this exception.
Did anyone do anything like that?
I prepared simple test case to show what my problem is. I use Mac OS X, Java 6.
import junit.framework.*;
import javax.swing.*;
import java.io.*;
import java.nio.channels.*;
/**
* Java file locks test.
*/
public class FileLocksTest extends TestCase {
/** File path (on Windows file will be created under the root directory of the current drive). */
private static final String LOCK_FILE_PATH = "/test-java-file-lock-tmp.bin";
/**
* #throws Exception If failed.
*/
public void testWriteLocks() throws Exception {
final File file = new File(LOCK_FILE_PATH);
file.createNewFile();
RandomAccessFile raf = new RandomAccessFile(file, "rw");
System.out.println("Getting lock...");
FileLock lock = raf.getChannel().lock();
System.out.println("Obtained lock: " + lock);
Thread thread = new Thread(new Runnable() {
#Override public void run() {
try {
RandomAccessFile raf = new RandomAccessFile(file, "rw");
System.out.println("Getting lock (parallel thread)...");
FileLock lock = raf.getChannel().lock();
System.out.println("Obtained lock (parallel tread): " + lock);
lock.release();
}
catch (Throwable e) {
e.printStackTrace();
}
}
});
thread.start();
JOptionPane.showMessageDialog(null, "Press OK to release lock.");
lock.release();
thread.join();
}
/**
* #throws Exception If failed.
*/
public void testReadLocks() throws Exception {
final File file = new File(LOCK_FILE_PATH);
file.createNewFile();
RandomAccessFile raf = new RandomAccessFile(file, "r");
System.out.println("Getting lock...");
FileLock lock = raf.getChannel().lock(0, Long.MAX_VALUE, true);
System.out.println("Obtained lock: " + lock);
Thread thread = new Thread(new Runnable() {
#Override public void run() {
try {
RandomAccessFile raf = new RandomAccessFile(file, "r");
System.out.println("Getting lock (parallel thread)...");
FileLock lock = raf.getChannel().lock(0, Long.MAX_VALUE, true);
System.out.println("Obtained lock (parallel thread): " + lock);
lock.release();
}
catch (Throwable e) {
e.printStackTrace();
}
}
});
thread.start();
JOptionPane.showMessageDialog(null, "Press OK to release lock.");
lock.release();
thread.join();
}
}
From the Javadoc:
File locks are held on behalf of the
entire Java virtual machine. They are
not suitable for controlling access to
a file by multiple threads within the
same virtual machine.
You can only acquire a lock once per file. Locks are not re-entrant AFAIK.
IMHO: Using files to communicate between process is a very bad idea. Perhaps you will be able to get this to work reliably, let me know if you can ;)
I would have one and only one thread read/write in only one process.
Have you checked the documentation? The FileChannel.lock() method returns an exclusive lock across the entire file. If you want to have multiple locks active concurrently across different threads, then you cannot use this method.
Instead you need to use FileChannel.locklock(long position, long size, boolean shared) in order to lock a specific region of the file. This will allow you to have multiple locks active at the same time, provided that each one is applied to a different region of the file. If you attempt to lock the same region of the file twice, you will encounter the same exception.