to make sure that a java.io.File is not modified/deleted during processing, I would like to create a temporary file in a different directory than the original file (e.g. System temp directory) that
cannot be accessed by the user
but holds the information (directory, name,...) of the original file
I need the original file's information, because there are lots of accesses on file-information, such as folder-structure, filename and file-extension. Working with the temporary file would destroy this information.
Unfortunately, it is not possible to just set a file's name/directory since this would rename/move the file.
Alternative approach: One could also work on both files, grabbing the information from the source-file and reading content from the temporary file but this does not seem like the optimal way to do this.
Is there a better approach to this?
Best regards
Martin
It sounds like what you want to do is just prevent any modifications to the file while you are reading from it. This is typically accomplished by locking the file, so that only your process can access it. As an example (using FileLock from java.nio)
try{
File file = new File("randomfile.txt");
FileChannel channel = new RandomAccessFile(file, "rw").getChannel();
FileLock lock = channel.lock();//Obtain a lock on the file
try{
//Do your things
}
finally{
lock.release();//Release the lock on the file
channel.close();
}
}
catch (IOException e) {
e.printStackTrace();
}
I suggest you use java.io.File.createTempFile(String, String, File) and also use java.io.File.deleteOnExit(); the file must be accessible to the user - else the use cannot write to it (QED). That is, try something like this -
try {
File directory = new File("/tmp"); // or C:/temp ?
File f = File.createTempFile("base-temp", ".tmp", directory); // create a new
// temp file... with a prefix, suffix and in a tmp folder...
f.deleteOnExit(); // Remove it when we exit (you can still explicitly delete when you're done).
} catch (IOException e) {
e.printStackTrace();
}
Related
Multiple Java instances are running on my machine and I want to check whether the Hadoop file is already opened in write (fs.create(file) or fs.append(file)) mode in any of the instances.
I Tried in FileStatus of the Hadoop file, not found anything. Is there any way to check whether the Hadoop file is already opened for write?
One way is to try to create/append a file again and catch the exception, but I have thousands of files and don't want to try every file. Also, if create/append is a success, then I have to close the file and lastModifiedTime will also get changed. I don't want to modify the FileStatus of a Hadoop file.
DistributedFileSystem provides the method isFileClosed(path) to check whether the file is opened for write.
try {
DistributedFileSystem dfs = new DistributedFileSystem();
Path path = new Path("/path/to/hadoop/file");
FileSystem fs = path.getFileSystem(conf);
dfs.initialize(fs.getUri(),conf);
if (dfs.exists(path) && !dfs.isFileClosed(path)) {
System.out.println("File " + path +" already opened in write mode");
}
} catch (IOException e) {
e.printStackTrace();
}
I need to write a custom batch File renamer. I've got the bulk of it done except I can't figure out how to check if a file is already open. I'm just using the java.io.File package and there is a canWrite() method but that doesn't seem to test if the file is in use by another program. Any ideas on how I can make this work?
Using the Apache Commons IO library...
boolean isFileUnlocked = false;
try {
org.apache.commons.io.FileUtils.touch(yourFile);
isFileUnlocked = true;
} catch (IOException e) {
isFileUnlocked = false;
}
if(isFileUnlocked){
// Do stuff you need to do with a file that is NOT locked.
} else {
// Do stuff you need to do with a file that IS locked
}
(The Q&A is about how to deal with Windows "open file" locks ... not how implement this kind of locking portably.)
This whole issue is fraught with portability issues and race conditions:
You could try to use FileLock, but it is not necessarily supported for your OS and/or filesystem.
It appears that on Windows you may be unable to use FileLock if another application has opened the file in a particular way.
Even if you did manage to use FileLock or something else, you've still got the problem that something may come in and open the file between you testing the file and doing the rename.
A simpler though non-portable solution is to just try the rename (or whatever it is you are trying to do) and diagnose the return value and / or any Java exceptions that arise due to opened files.
Notes:
If you use the Files API instead of the File API you will get more information in the event of a failure.
On systems (e.g. Linux) where you are allowed to rename a locked or open file, you won't get any failure result or exceptions. The operation will just succeed. However, on such systems you generally don't need to worry if a file is already open, since the OS doesn't lock files on open.
// TO CHECK WHETHER A FILE IS OPENED
// OR NOT (not for .txt files)
// the file we want to check
String fileName = "C:\\Text.xlsx";
File file = new File(fileName);
// try to rename the file with the same name
File sameFileName = new File(fileName);
if(file.renameTo(sameFileName)){
// if the file is renamed
System.out.println("file is closed");
}else{
// if the file didnt accept the renaming operation
System.out.println("file is opened");
}
On Windows I found the answer https://stackoverflow.com/a/13706972/3014879 using
fileIsLocked = !file.renameTo(file)
most useful, as it avoids false positives when processing write protected (or readonly) files.
org.apache.commons.io.FileUtils.touch(yourFile) doesn't check if your file is open or not. Instead, it changes the timestamp of the file to the current time.
I used IOException and it works just fine:
try
{
String filePath = "C:\sheet.xlsx";
FileWriter fw = new FileWriter(filePath );
}
catch (IOException e)
{
System.out.println("File is open");
}
I don't think you'll ever get a definitive solution for this, the operating system isn't necessarily going to tell you if the file is open or not.
You might get some mileage out of java.nio.channels.FileLock, although the javadoc is loaded with caveats.
Hi I really hope this helps.
I tried all the options before and none really work on Windows. The only think that helped me accomplish this was trying to move the file. Event to the same place under an ATOMIC_MOVE. If the file is being written by another program or Java thread, this definitely will produce an Exception.
try{
Files.move(Paths.get(currentFile.getPath()),
Paths.get(currentFile.getPath()), StandardCopyOption.ATOMIC_MOVE);
// DO YOUR STUFF HERE SINCE IT IS NOT BEING WRITTEN BY ANOTHER PROGRAM
} catch (Exception e){
// DO NOT WRITE THEN SINCE THE FILE IS BEING WRITTEN BY ANOTHER PROGRAM
}
If file is in use FileOutputStream fileOutputStream = new FileOutputStream(file); returns java.io.FileNotFoundException with 'The process cannot access the file because it is being used by another process' in the exception message.
How do I create a tar or gzipped tar archive in Java that isn't backed by File and actual files?
I've found commons-compress, but the examples and most of the documentation rely on using already existing files that can be referenced by a Java File object. What if I don't want to use a File object and want to build my tar archive from byte[].
The only constructors for TarArchiveEntry that provide a way to set the content accept File and there is not setter for content.
From the documentation for TarArchiveEntry:
TarArchiveEntry(File file)
Construct an entry for a file.
TarArchiveEntry(File file, String fileName)
Construct an entry for a file.
Using commons-compress, it isn't immediately clear or exemplified in the documentation, but here is the gist of it
//Get the content you want to lay down into a byte[]
byte content[] = "I am some simple content that should be written".getBytes();
//Name your entry with the complete path relative to the base directory
//of your archive. Any directories that don't exist (e.g. "testDir") will
//be created for you
TarArchiveEntry textFile = new TarArchiveEntry("testDir/hello.txt");
//Make sure to set the size of the entry. If you don't you will not be able
//to write to it
textFile.setSize(content.length);
TarArchiveOutputStream gzOut = null;
try {
/*In this case I chose to show how to lay down a gzipped archive.
You could just as easily remove the GZIPOutputStream to lay down a plain tar.
You also should be able to replace the FileOutputStream with
a ByteArrayOutputStream to lay nothing down on disk if you wanted
to do something else with your newly created archive
*/
gzOut = new TarArchiveOutputStream(
new GZIPOutputStream(
new BufferedOutputStream(
new FileOutputStream("/tmp/mytest.tar.gz")
)));
//When you put an ArchiveEntry into the archive output stream,
//it sets it as the current entry
gzOut.putArchiveEntry(textFile);
//The write command allows you to write bytes to the current entry
//on the output stream, which was set by the above command.
//It will not allow you to write any more than the size
//that you specified when you created the archive entry above
gzOut.write(content);
//You must close the current entry when you are done with it.
//If you are appending multiple archive entries, you only need
//to close the last one. The putArchiveEntry automatically closes
//the previous "current entry" if there was one
gzOut.closeArchiveEntry();
} catch (Exception ex) {
System.err.println("ERROR: " + ex.getMessage());
} finally {
if (gzOut != null) {
try {
gzOut.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
I have a file I'm storing within my jar that I use a default setting file. I wish to write this file out to a user defined path. How do I write it out? This file that I'm trying to write out is in the same location as my class files that will be writing this file
Use getResourceAsStream to access the resource. Create a FileOutputStream for the file you wish to write. Read from one stream and write to the other. Preferably, use buffering, and don't forget to close your streams when you're done.
See Location-Independent Access to Resources.
use "getResourceAsStream"
-> http://mindprod.com/jgloss/getresourceasstream.html
given a resource that you want to write to a given Path path, then you can use:
try(InputStream is = this.getClass().getResourceAsStream(resource)){
Files.copy(is, path);
} catch (Exception e){
throw new RuntimeException(e);
}
I have a cluster of machines, each running a Java app.
These Java apps need to access a unique resource.txt file concurrently.
I need to atomically rename a temp.txt file to resource.txt in Java, even if resource.txt already exist.
Deleting resource.txt and renaming temp.txt doesn't work, as it's not atomic (it creates a small timeframe where resource.txt doesn't exist).
And it should be cross-platform...
For Java 1.7+, use java.nio.file.Files.move(Path source, Path target, CopyOption... options) with CopyOptions "REPLACE_EXISTING" and "ATOMIC_MOVE".
See API documentation for more information.
For example:
Files.move(src, dst, StandardCopyOption.ATOMIC_MOVE);
On Linux (and I believe Solaris and other UNIX operating systems), Java's File.renameTo() method will overwrite the destination file if it exists, but this is not the case under Windows.
To be cross platform, I think you'd have to use file locking on resource.txt and then overwrite the data.
The behavior of the file lock is
platform-dependent. On some platforms,
the file lock is advisory, which means
that unless an application checks for
a file lock, it will not be prevented
from accessing the file. On other
platforms, the file lock is mandatory,
which means that a file lock prevents
any application from accessing the
file.
try {
// Get a file channel for the file
File file = new File("filename");
FileChannel channel = new RandomAccessFile(file, "rw").getChannel();
// Use the file channel to create a lock on the file.
// This method blocks until it can retrieve the lock.
FileLock lock = channel.lock();
// Try acquiring the lock without blocking. This method returns
// null or throws an exception if the file is already locked.
try {
lock = channel.tryLock();
} catch (OverlappingFileLockException e) {
// File is already locked in this thread or virtual machine
}
// Release the lock
lock.release();
// Close the file
channel.close();
} catch (Exception e) {
}
Linux, by default, uses voluntary locking, while Windows enforces it. Maybe you could detect the OS, and use renameTo() under UNIX with some locking code for Windows?
There's also a way to turn on mandatory locking under Linux for specific files, but it's kind of obscure. You have to set the mode bits just right.
Linux, following System V (see System
V Interface Definition (SVID) Version
3), lets the sgid bit for files
without group execute permission mark
the file for mandatory locking
Here is a discussion that relates: http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4017593
As stated here, it looks like the Windows OS doesn't even support atomic file rename for older versions. It's very likely you have to use some manual locking mechanisms or some kind of transactions. For that, you might want to take a look into the apache commons transaction package.
If this should be cross-platform I suggest 2 options:
Implement an intermediate service that is responsible for all the file accesses. Here you can use several mechanisms for synchronizing the requests. Each client java app accesses the file only through this service.
Create a control file each time you need to perform synchronized operations. Each java app that accesses the file is responsible checking for the control file and waiting while this control file exists. (almost like a semaphore). The process doing the delete/rename operation is responsible for creating/deleting the control file.
If the purpose of the rename is to replace resource.txt on the fly and you have control over all the programs involved, and the frequency of replacement is not high, you could do the following.
To open/read the file:
Open "resource.txt", if that fails
Open "resource.old.txt", if that fails
Open "resource.txt" again, if that fails
You have an error condition.
To replace the file:
Rename "resource.txt" to "resource.old.txt", then
Rename "resource.new.txt" to "resource.txt", then
Delete "resource.old.txt".
Which will ensure all your readers always find a valid file.
But, easier, would be to simply try your opening in a loop, like:
InputStream inp=null;
StopWatch tmr=new StopWatch(); // made up class, not std Java
IOException err=null;
while(inp==null && tmr.elapsed()<5000) { // or some approp. length of time
try { inp=new FileInputStream("resource.txt"); }
catch(IOException thr) { err=thr; sleep(100); } // or some approp. length of time
}
if(inp==null) {
// handle error here - file did not turn up after required elapsed time
throw new IOException("Could not obtain data from resource.txt file");
}
... carry on
You might get some traction by establishing a filechannel lock on the file before renaming it (and deleting the file you're going to overwrite once you have the lock).
-r
I solve with a simple rename function.
Calling :
File newPath = new File("...");
newPath = checkName(newPath);
Files.copy(file.toPath(), newPath.toPath(), StandardCopyOption.REPLACE_EXISTING);
The checkName function checks if exits.
If exits then concat a number between two bracket (1) to the end of the filename.
Functions:
private static File checkName(File newPath) {
if (Files.exists(newPath.toPath())) {
String extractRegExSubStr = extractRegExSubStr(newPath.getName(), "\\([0-9]+\\)");
if (extractRegExSubStr != null) {
extractRegExSubStr = extractRegExSubStr.replaceAll("\\(|\\)", "");
int parseInt = Integer.parseInt(extractRegExSubStr);
int parseIntPLus = parseInt + 1;
newPath = new File(newPath.getAbsolutePath().replace("(" + parseInt + ")", "(" + parseIntPLus + ")"));
return checkName(newPath);
} else {
newPath = new File(newPath.getAbsolutePath().replace(".pdf", " (" + 1 + ").pdf"));
return checkName(newPath);
}
}
return newPath;
}
private static String extractRegExSubStr(String row, String patternStr) {
Pattern pattern = Pattern.compile(patternStr);
Matcher matcher = pattern.matcher(row);
if (matcher.find()) {
return matcher.group(0);
}
return null;
}
EDIT: Its only works for pdf. If you want other please replace the .pdf or create an extension paramter for it.
NOTE: If the file contains additional numbers between brackets '(' then it may mess up your file names.