In some place I create temporary file:
Files.createTempDirectory("chunk");
In some other place after processsing, I try to delete file:
Files.deleteIfExists(somePath)
and experience following trace:
java.nio.file.FileSystemException: C:\....\Temp\chunk11607697185854596263\chunk-3.csv: The process cannot access the file because it is being used by another process.
at java.base/sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:92)
at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:103)
at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:108)
at java.base/sun.nio.fs.WindowsFileSystemProvider.implDelete(WindowsFileSystemProvider.java:270)
at java.base/sun.nio.fs.AbstractFileSystemProvider.deleteIfExists(AbstractFileSystemProvider.java:110)
at java.base/java.nio.file.Files.deleteIfExists(Files.java:1180)
at my.some.project.batch.MyClass.afterStep(MyClass.java:31)
It happens when I start application on my local Windows mashine and doesn't happen in docker. Also I don't experience such error when I run application locally on MacOS. What the problem and how can I fix it ?
Check that the file isn't opened by some other process. This check can be done with Process Explorer. When you have started the program, select Find in the menu and then Find Handle or DLL..., or press Ctrl+F. Enter the name of the file that is being locked by some process and click Search.
There's a difference between Windows and Unix-like operating systems regarding the handling of removal of opened files.
On Unix-like systems a file can be removed even if it's opened by other processes. The actual file is not removed until all other processes has closed the file, but the filename entry on the filesystem is removed. The low-level file remove operation returns success as soon as the first part of the operation succeeds. Removal of the actual file data is delayed until the open count has reached 0.
On Windows the file may not be opened by any process for the low-level file remove operation to be able to succeed.
This could be a reason why you see a different behaviour when running your program on Windows versus Docker or MacOS.
There are quite a few problems around when file is deleted with Java in Windows. The most important thing is to really make sure there are no unclosed streams before trying to delete a file as VGR already mentioned.
For more info about the usual problems you could take a look at this:
delete & GC
and this: file delete
I am using this kind of awful looking "solution" for deleting files, feel free to try:
public static boolean deleteFile(File file)
throws Exception {
if (!file.exists()) {
throw new Exception("File not found: " + file.getName());
}
try {
file.setWritable(true);
} catch (Exception ignored) {
}
int delay = 50;
for (int i = 0; i < 20; i++) {
if (file.delete()) {
return true;
}
System.gc();
try {
Thread.sleep(delay);
} catch (InterruptedException ignored) {
}
delay = delay + 50;
}
throw new Exception("Could not delete the file '"
+ file.getName() + "'");
}
Related
I got a problem when i want to update my (lets say orderNumber) and i try to renaming the file of image. lets say the original file ini MainImage15 i want to rename to MainImage16 with Files.Copy or Files.Move and after that i try to delete after succesful copying. and i got the error like this
java.nio.file.FileSystemException: C:\Users\User\apache\webapps\Promotion\030000\MainImage15.jpg: The process cannot access the file because it is being used by another process.
public void renameFileToFileSystem(final String fileName, final String oldFileLocation, final String newFileLocation) {
Path source = Paths.get(oldFileLocation);
Path destination = Paths.get(newFileLocation);
try {
Files.copy(source, destination, StandardCopyOption.REPLACE_EXISTING);
Files.delete(source);
} catch (final IOException ioException) {
throw new ContentManagementException(fileName, ioException.getMessage());
}
}
i dont knwo what to do. should i use buffer close? but i just renaming the file . thankyou.
I don't think Files.copy keeps the file handle open.
To make sure its true - remove the line Files.copy and rerun - the chances are that you still won't be able to delete the file.
So you must find who keeps the handle busy. There are basically two possibilities:
Its somewhere else in your code
Its some kind of external process (antivirus, another application that you've used to render the image and so forth). You can use Process Explorer that will help to find the process that keeps the handle.
I need to write a custom batch File renamer. I've got the bulk of it done except I can't figure out how to check if a file is already open. I'm just using the java.io.File package and there is a canWrite() method but that doesn't seem to test if the file is in use by another program. Any ideas on how I can make this work?
Using the Apache Commons IO library...
boolean isFileUnlocked = false;
try {
org.apache.commons.io.FileUtils.touch(yourFile);
isFileUnlocked = true;
} catch (IOException e) {
isFileUnlocked = false;
}
if(isFileUnlocked){
// Do stuff you need to do with a file that is NOT locked.
} else {
// Do stuff you need to do with a file that IS locked
}
(The Q&A is about how to deal with Windows "open file" locks ... not how implement this kind of locking portably.)
This whole issue is fraught with portability issues and race conditions:
You could try to use FileLock, but it is not necessarily supported for your OS and/or filesystem.
It appears that on Windows you may be unable to use FileLock if another application has opened the file in a particular way.
Even if you did manage to use FileLock or something else, you've still got the problem that something may come in and open the file between you testing the file and doing the rename.
A simpler though non-portable solution is to just try the rename (or whatever it is you are trying to do) and diagnose the return value and / or any Java exceptions that arise due to opened files.
Notes:
If you use the Files API instead of the File API you will get more information in the event of a failure.
On systems (e.g. Linux) where you are allowed to rename a locked or open file, you won't get any failure result or exceptions. The operation will just succeed. However, on such systems you generally don't need to worry if a file is already open, since the OS doesn't lock files on open.
// TO CHECK WHETHER A FILE IS OPENED
// OR NOT (not for .txt files)
// the file we want to check
String fileName = "C:\\Text.xlsx";
File file = new File(fileName);
// try to rename the file with the same name
File sameFileName = new File(fileName);
if(file.renameTo(sameFileName)){
// if the file is renamed
System.out.println("file is closed");
}else{
// if the file didnt accept the renaming operation
System.out.println("file is opened");
}
On Windows I found the answer https://stackoverflow.com/a/13706972/3014879 using
fileIsLocked = !file.renameTo(file)
most useful, as it avoids false positives when processing write protected (or readonly) files.
org.apache.commons.io.FileUtils.touch(yourFile) doesn't check if your file is open or not. Instead, it changes the timestamp of the file to the current time.
I used IOException and it works just fine:
try
{
String filePath = "C:\sheet.xlsx";
FileWriter fw = new FileWriter(filePath );
}
catch (IOException e)
{
System.out.println("File is open");
}
I don't think you'll ever get a definitive solution for this, the operating system isn't necessarily going to tell you if the file is open or not.
You might get some mileage out of java.nio.channels.FileLock, although the javadoc is loaded with caveats.
Hi I really hope this helps.
I tried all the options before and none really work on Windows. The only think that helped me accomplish this was trying to move the file. Event to the same place under an ATOMIC_MOVE. If the file is being written by another program or Java thread, this definitely will produce an Exception.
try{
Files.move(Paths.get(currentFile.getPath()),
Paths.get(currentFile.getPath()), StandardCopyOption.ATOMIC_MOVE);
// DO YOUR STUFF HERE SINCE IT IS NOT BEING WRITTEN BY ANOTHER PROGRAM
} catch (Exception e){
// DO NOT WRITE THEN SINCE THE FILE IS BEING WRITTEN BY ANOTHER PROGRAM
}
If file is in use FileOutputStream fileOutputStream = new FileOutputStream(file); returns java.io.FileNotFoundException with 'The process cannot access the file because it is being used by another process' in the exception message.
I have a server that reads a list of text files from a windows shared directory and save it contents to the db before its starts to accepts user messages. This server will be running in multiple machines at one time.
I see that when I run the server in multiple machines, the server that starts processing the files, first processes all the files and the others keep waiting to get access to the files in that directory.
My code does this - (cannot post code due to security policy)
Get a list all files in the shared directory.
Sort them by modified date (its saving time series data)
While(true) till more files exist in directory
Get the first file in the list, and move it to InProgess folder and read
Save contents to database.
Move file to Archive directory.
Process the next file.
I see that when I run the same program in 2 different machines, one of them get holds of the files first and loads them all. The other one keeps waiting to get a handle on the files and if it does find a handle, they have already been processed. So it moves on.
My aim is to have the process when run in both or multiple machines to process all the file in parallel and finish faster. For now I am testing with 500 files on disk, but I can have more files on disk at any given time.
PseudoCode -
if(files exist on disk){
LOGGER.info("Files exist on disk. Lets process them up first....");
while (true) {
File dir = new File(directory);
List<File> fileList = new LinkedList<File>(Arrays.asList(dir.listFiles((FileFilter)FileFileFilter.FILE)));
LOGGER.info("No of files in this process: "+ sortedFileList.size());
if (fileList.size() > 0) {
Collections.sort(fileList, new Server().new FileComparator());
File file = fileList.get(0);
//If I cannot rename the file in the same directory, the file maybe open and I move to the next file
if(!file.renameTo(file.getAbsoluteFile())) {
LOGGER.info("Read next file...");
continue;
}
LOGGER.info("Get file handle...");
if (file.exists()) {
File inprogressFile = new File(dataDirName + FileBackupOnDisk.INPROGRESS + fileName);
saveToDB(inprogressFile);
if (savedToDB)
if(inprogressFile.renameTo(new File(dataDirName+ARCHIVE+fileName)))
LOGGER.info("Moved file to archive - " + fileName);
else
LOGGER.error("Move file " + fileName + " to failed directory!");
}
}
}
}
That's my file comparator code. This cannot be opening files -
final Map<File, Long> staticLastModifiedTimes = new HashMap<File,Long>();
for(final File f : sortedFileList) {
staticLastModifiedTimes.put(f, f.lastModified());
}
Collections.sort(sortedFileList, new Comparator<File>() {
#Override
public int compare(final File f1, final File f2) {
return
staticLastModifiedTimes.get(f1).compareTo(staticLastModifiedTimes.get(f2));
}
});
How do I make sure that both my servers/multiple servers running on different machines are able to access the shared directly in parallel. Right now it looks like the 2nd process find that files exist in the dir but hang at one point waiting to get a file handle.
Let me know if anyone has done this before and how?
I found out that my solution above works perfectly fine!!!!
Its just that running one instance from my eclipse with another from a m/c in the network was causing this latency issues.
If I run the program with 2 machines in the same network it works fine. Just that my computer was slower. Both the instances read the files when they get are able to get handle on it.
Thank you all for your help.
I've created a simple program that would write a file with a directory using the following codes:
String nameProve = nameField.getText();
String employee = ("C:\\Users\\ALLEN\\workspace32bit\\RETRIEVE_CHECKER1\\RETRIEVE_CHECKED1" + nameProve + ".txt");
PrintWriter outputStream1 = null;
try
{
outputStream1 = new PrintWriter(employeeName);
}
catch(Exception e)
{
JOptionPane.showMessageDialog(null, "Can not write to " + employeeName);
System.exit(0);
}
outputStream1.println("Employee Retrieve Executed");
outputStream1.close();
I've already exported my code to a .jar file and my code works just fine if i execute it in the computer were I've develop my program but when I copied the jar file to other computer and I also created the forlder (manually) that corresponds with the directory in my codes my program doesn't work and my catch block will show the message Can not write to "C:\Users\ALLEN\workspace32bit\RETRIEVE_CHECKER1\RETRIEVE_CHECKED1" + nameProve + ".txt"
Can anybody give me some advice on how to solve this error? Thanks!
Obviously you are the user ALLEN on your machine and run the program as that user, so everithing works fine, because you are the owner of the directory C:\Users\ALLEN.
Then you copy the jar to another machine, where the user ALLEN does not exist and you are logged in as antoher user let's say his name is Bob and create that directory unter C:\Users. You may have noticed that you when you wanted to create that directory as the user Bob windows has warned you that you need admin priviledges in order to compleate this action.
Now you try to run your program with the user Bob who has access only to his own directory C:\Users\Bob and try to write to ALLEN's own directory. So what happens is that you get an IOException telling you access denied which is good so!
You should not attepmt to write to other users private direcotries, this is a security issue.
In your code when dealing with filesystem, never hard code absolute path, always use relative paths, or if you need to defined a direcotry where all the data needed yb you program should be located, then pass ist as argument.
The simplest to do, ist use the following and work with the current working directory
String employee = "RETRIEVE_CHECKER1\\RETRIEVE_CHECKED1" + nameProve + ".txt";
You need to create the directory RETRIEVE_CHECKER1 either by hand in location where you run the program, or better yet in your program using File#mkdir and use it like this:
File employee = new File(dir, "RETRIEVE_CHECKED1" + nameProve + ".txt");
PrintWriter outputStream1 = new PrintWriter(employeeName);
Exception you catch will have all details- like access denied, directory not exist, etc. Currently you loose exception data (message, stacktrace) when catching it.
Please do something like
Do not catch just java.lang.Exception. This is too wide- will catch possible NullPointerException and present misleading message '"Can not write to'. Catch only specific exception instead- ones that are actually thrown from try block. In your case, that will be java.io.FileNotFoundException
Do not loose exception details. Print it to the standard error log, or even better use loging framework
catch block that adresses those issues:
catch(FileNotFoundException e) {
JOptionPane.showMessageDialog(null, e.getMessage());
e.printStackTrace();
}
I have a cluster of machines, each running a Java app.
These Java apps need to access a unique resource.txt file concurrently.
I need to atomically rename a temp.txt file to resource.txt in Java, even if resource.txt already exist.
Deleting resource.txt and renaming temp.txt doesn't work, as it's not atomic (it creates a small timeframe where resource.txt doesn't exist).
And it should be cross-platform...
For Java 1.7+, use java.nio.file.Files.move(Path source, Path target, CopyOption... options) with CopyOptions "REPLACE_EXISTING" and "ATOMIC_MOVE".
See API documentation for more information.
For example:
Files.move(src, dst, StandardCopyOption.ATOMIC_MOVE);
On Linux (and I believe Solaris and other UNIX operating systems), Java's File.renameTo() method will overwrite the destination file if it exists, but this is not the case under Windows.
To be cross platform, I think you'd have to use file locking on resource.txt and then overwrite the data.
The behavior of the file lock is
platform-dependent. On some platforms,
the file lock is advisory, which means
that unless an application checks for
a file lock, it will not be prevented
from accessing the file. On other
platforms, the file lock is mandatory,
which means that a file lock prevents
any application from accessing the
file.
try {
// Get a file channel for the file
File file = new File("filename");
FileChannel channel = new RandomAccessFile(file, "rw").getChannel();
// Use the file channel to create a lock on the file.
// This method blocks until it can retrieve the lock.
FileLock lock = channel.lock();
// Try acquiring the lock without blocking. This method returns
// null or throws an exception if the file is already locked.
try {
lock = channel.tryLock();
} catch (OverlappingFileLockException e) {
// File is already locked in this thread or virtual machine
}
// Release the lock
lock.release();
// Close the file
channel.close();
} catch (Exception e) {
}
Linux, by default, uses voluntary locking, while Windows enforces it. Maybe you could detect the OS, and use renameTo() under UNIX with some locking code for Windows?
There's also a way to turn on mandatory locking under Linux for specific files, but it's kind of obscure. You have to set the mode bits just right.
Linux, following System V (see System
V Interface Definition (SVID) Version
3), lets the sgid bit for files
without group execute permission mark
the file for mandatory locking
Here is a discussion that relates: http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4017593
As stated here, it looks like the Windows OS doesn't even support atomic file rename for older versions. It's very likely you have to use some manual locking mechanisms or some kind of transactions. For that, you might want to take a look into the apache commons transaction package.
If this should be cross-platform I suggest 2 options:
Implement an intermediate service that is responsible for all the file accesses. Here you can use several mechanisms for synchronizing the requests. Each client java app accesses the file only through this service.
Create a control file each time you need to perform synchronized operations. Each java app that accesses the file is responsible checking for the control file and waiting while this control file exists. (almost like a semaphore). The process doing the delete/rename operation is responsible for creating/deleting the control file.
If the purpose of the rename is to replace resource.txt on the fly and you have control over all the programs involved, and the frequency of replacement is not high, you could do the following.
To open/read the file:
Open "resource.txt", if that fails
Open "resource.old.txt", if that fails
Open "resource.txt" again, if that fails
You have an error condition.
To replace the file:
Rename "resource.txt" to "resource.old.txt", then
Rename "resource.new.txt" to "resource.txt", then
Delete "resource.old.txt".
Which will ensure all your readers always find a valid file.
But, easier, would be to simply try your opening in a loop, like:
InputStream inp=null;
StopWatch tmr=new StopWatch(); // made up class, not std Java
IOException err=null;
while(inp==null && tmr.elapsed()<5000) { // or some approp. length of time
try { inp=new FileInputStream("resource.txt"); }
catch(IOException thr) { err=thr; sleep(100); } // or some approp. length of time
}
if(inp==null) {
// handle error here - file did not turn up after required elapsed time
throw new IOException("Could not obtain data from resource.txt file");
}
... carry on
You might get some traction by establishing a filechannel lock on the file before renaming it (and deleting the file you're going to overwrite once you have the lock).
-r
I solve with a simple rename function.
Calling :
File newPath = new File("...");
newPath = checkName(newPath);
Files.copy(file.toPath(), newPath.toPath(), StandardCopyOption.REPLACE_EXISTING);
The checkName function checks if exits.
If exits then concat a number between two bracket (1) to the end of the filename.
Functions:
private static File checkName(File newPath) {
if (Files.exists(newPath.toPath())) {
String extractRegExSubStr = extractRegExSubStr(newPath.getName(), "\\([0-9]+\\)");
if (extractRegExSubStr != null) {
extractRegExSubStr = extractRegExSubStr.replaceAll("\\(|\\)", "");
int parseInt = Integer.parseInt(extractRegExSubStr);
int parseIntPLus = parseInt + 1;
newPath = new File(newPath.getAbsolutePath().replace("(" + parseInt + ")", "(" + parseIntPLus + ")"));
return checkName(newPath);
} else {
newPath = new File(newPath.getAbsolutePath().replace(".pdf", " (" + 1 + ").pdf"));
return checkName(newPath);
}
}
return newPath;
}
private static String extractRegExSubStr(String row, String patternStr) {
Pattern pattern = Pattern.compile(patternStr);
Matcher matcher = pattern.matcher(row);
if (matcher.find()) {
return matcher.group(0);
}
return null;
}
EDIT: Its only works for pdf. If you want other please replace the .pdf or create an extension paramter for it.
NOTE: If the file contains additional numbers between brackets '(' then it may mess up your file names.