Develop Reference

Java.nio.file.NoSuchFileException but files exits

I'm trying to read files from a folder. There are >1000 files in that folder and i trying read 100 files every 10s by using ScheduledThreadPoolExecutor.scheduleWithFixedDelay()
Files.list(Paths.get(path)).sorted().limit(limit).forEach(file ->{ //limit = 100
try{
String content = new String(Files.readAllBytes(file), "UTF-8");
//do business logic
doneFile.add(fileName);
}catch (Exception e){
log.error(e);
}
});
if(doneFile.size() > 0){
for(String fileName: doneFile){
try {
Files.delete(Paths.get(path + fileName));
} catch (IOException e) {
log.error(e);
}
}
}
About 60-80 files were read successfully and the others throw java.nio.file.NoSuchFileException.
Delete function also throws some java.nio.file.NoSuchFileException.
But finally, all files were read and deleted after some threads in spite of exceptions
What cause exception in this case and how can i fix it?
Many thanks!
Ps. Sr about my bad English

I commented:
It sounds like something else is adding and removing files. Is it possible that you have two scheduled tasks running? Is it possible that an external process is doing this?
You replied:
I checked log file and got that ScheduledThreadPoolExecutor.scheduleWithFixedDelay() created 10 threads instead of 8 as expected for 4 loops
So ... it sounds like you have two (or more) threads iterating through the same directory, processing and deleting files without any coordination. Combine that with the fact that Files.list is going to buffer part or all of the directory that it is listing, and you will get problems like:
One thread deletes a file that is in another file's stream before the second thread tries to open it. This leads to a NoSuchFileException when the latter tries to open the file.
Two threads process the same file at the same time, and one deletes it while the second thread is still processing it. This leads to a NoSuchFileException when the latter tries to delete the file.
Possible solutions:
Write the code to ignore these exceptions. (But you could still have two threads processing the same file at the same time, which seems like a bad thing.)
Restructure the code so that there is only one thread scanning the directory at any one time.
If you need parallelism when processing within the directory, have the directory processor thread submit a task for each file to an executor with a bounded thread pool.

How i can make a Watch Service that deletes a file when it closes?

I basically want to make a watch service (or something like it) that checks if a file has been closed and instantly remove that file if it did close(finished executing).
How I can achieve this? please give me a cmd commands or some code(i prefer Java).

Ok, this should not be hard to do, if you google a bit you find a Java-File Method called file.canWrite() which basically returns if a file is locked by an other program or so.
So codewise what you could do is something like this.
boolean isDeleted = false;
File f = new File (// Put your file here);
while (!isDeleted) {
if (f.canWrite()) {
f.delete();
isDeleted = true;
} else {
try {
Thread.sleep(10); // Throws Exception you need to catch somewhere...
} catch (Exception e) {}
}
}
This code you need to include into some Java-Program. I added a simple Thread.sleep(10) that your PC does not have to check aaaaaalllllllll the time.
See Check if a file is locked in Java
Other possibility would be trying to rename the file with file.renameTo("some_path.txt"); as this method also returns a boolean whether it was successfull! Just note that you then need to update the file again before removing it.
Last possibility I see is pretty similar to the second one. You try to delete the file by calling file.delete(); If the file still exists you know it was not successful and loop because of that.

I assume you mean when the file is not open in another program, and you cannot make changes to that other program? (If you are talking about your own program opening the file, this is much easier.)
On Windows, it is not very easy to tell which program has a file open. Take a look at https://superuser.com/questions/117902/find-out-which-process-is-locking-a-file-or-folder-in-windows for some options. I like the handle tool for this, but it has to run as Administrator, which may be a problem. You can try renaming or writing to the file, as suggested at Check if a file is locked in Java
Once you have a script that determines whether the file is open to your satisfaction, it should be fairly straightforward to write a script which loops while testing if the file is open and then deletes file.

Implementing Microsoft.CognitiveServices.Speech recognition for several files

I have gotten the coding example from here to work.
I can run a .wav file through and get the transcript, however in the example the program never ends until I hit a key:
System.out.println("Press any key to stop");
new Scanner(System.in).nextLine();
That seems to artificially pause everything while the service is being queried.
If I remove those that line, the program jumps through to fast and concludes without waiting for the service to respond.
Question: How do I resume/continue the program with the full transcription without needing to hit a key?
I would like to run this for multiple .wav files transcribing each one after the other. But so far it runs the first one then waits.
I have been scouring the documentation and I have tried multiple things including using recognizer.close(); which I would expect to end the SpeechRecognizer but which seems to do nothing.
Or using result = recognizer.recognizeOnceAsync().get(); which does not transcribe the full file.
Does anyone know of an example of this running multiple files or how to implement that?
Thanks.

You can create a function that will read and return the list of files in your directory:
private static String[] GetFiles(String directory)
{
String[] files = (new File(directory)).list(File::isFile);
return files;
}
Then loop through them to process them, and then transcribe them.
String[] files = GetFiles(args[0]);
for (String file : files)
{
//Your code goes here.
System.out.printf("File %1$s processed" + "\r\n",file);//print out which file has been successfully processed.
}
You could also try using the Batch Transcription feature!
Batch transcription is ideal if you have large amounts of audio in
storage.

Upload retry mechanism using JSch library

I have a file to upload (say abc.pdf). Very first time I want to upload this file as a temp file (say abc.pdf.temp). Then , if the file is successfully transferred (fully transferred) then I need to rename it to its original name (abc.pdf). But if the file is not fully transferred then I need to delete the temp file that I uploaded initially since I don't want to keep a corrupted file in the server. Is this achievable to do using this JSch library. Below is the sample code. Does this code make sense to achieve this?
Sample Code:
originalFile = 'abc.pdf';
tempFile = 'abc.pdf.temp';
fileInputStream = createobject("java", "java.io.FileInputStream").init('C:\abc.pdf');
SftpChannel.put(fileInputStream,tempFile);
// Comparing remote file size with local file
if(SftpChannel.lstat(tempFile).getSize() NEQ localFileSize){
// Allow to Resume the file transfer since the file size is different
SftpChannel.put(fileInputStream,tempFile,SftpChannel.RESUME);
if(SftpChannel.lstat(tempFile).getSize() NEQ localFileSize){
// Check again if the file is not fully transferred (During RESUME) then
// deleting the file since dont want to keep a corrupted file in the server.
SftpChannel.rm(tempFile);
}
}else{//assuming file is fully transferred
SftpChannel.rename(tempFile ,originalFile);
}

It's very unlikely that after the put finishes without throwing, the file size won't match. It can hardly happen. Even if it happens, it makes little sense to call RESUME. If something catastrophic goes wrong that is not detected by put, RESUME is not likely to help.
And even if you want to try with RESUME, it does not make sense to try once. If you believe it makes sense to retry, you have to keep retrying until you succeed, not only once.
You should catch exception and resume/delete/whatever. That's the primary recovery mechanism. This is 100x more likely to happen than 1.

Checking for write access in a directory before creating files inside it

My small utility application asks the user for an output directory via a GUI file selector.
Then it creates a lot of files in this output directory after some processing.
I need to check if the application has write access so that it informs the user and does
not continue with the processing (which might take a long time)
My first attempt was the canWrite() method of java.io.File. But this does not work
since it deals with the directory entry itself and not its contents. I have seen at least
one instance of a Windows XP folder that can be renamed or deleted but no files may be created
in it (because of permissions). This is actually my testcase.
I finally settled with the following solution
//User places the input file in a directory and selects it from the GUI
//All output files will be created in the directory that contains the input file
File fileBrowse = chooser.getSelectedFile(); //chooser is a JFileChooser
File sample = new File(fileBrowse.getParent(),"empty.txt");
try
{
/*
* Create and delete a dummy file in order to check file permissions. Maybe
* there is a safer way for this check.
*/
sample.createNewFile();
sample.delete();
}
catch(IOException e)
{
//Error message shown to user. Operation is aborted
}
However this does not feel elegant to me since it just tries to actually create a file and checks if the operation succeeds.
I suspect that there must be a better way for this but all solutions I have found so far
with Security Managers and stuff deal with Java Applets and not standalone applications.
Am I missing something?
What is the recommended way of checking for file access inside a directory before
actually writing the files?
I am using Java 5.

You could check the file permissions, make sure the directory exists, and do a lot of checking or find a library that does all that checking for you BUT (!) isn't the best way of checking to try ? If you check for permissions and the filesystem changes... you will have to change your code. But trying to write a file will ALWAYS tell you if you can write a file.
Your solution doesn't have to be the most elegant one. It's not a cheap hard coded patch or something ugly. It's just normal code. And it will always work. But if you don't like to see that check in the code just separate it by putting it in class which only goal is to check for the possibly of writing. In fact, you should put it in a utility class wheter you like the elegance or not.
The other solution would be to place your whole writing-to-the-hard-drive code, in the try. And if you can't write, the whole part will be skipped and you give feedback to the user with a message in the catch part.

it doesn't works even if you invoke canWrite on the final path?
File sample = new File(fileBrowse.getParent(),"empty.txt");
if (sample.canWrite()) {
doSomethingUseful(sample);
} else {
notifyUser();
}

you can use FilePermission to get the details .
I find one way where you need to implement SecurityManager the code is here and here

Using Java 1.8 I was able to use the following.
Set<PosixFilePermission> permissions = Files.getPosixFilePermissions(Paths.get(destDir), LinkOption.NOFOLLOW_LINKS);
Assert.assertTrue("User did not have read permission.", permissions.contains(PosixFilePermission.OWNER_READ));
Assert.assertTrue("User did not have execute permission.", permissions.contains(PosixFilePermission.OWNER_EXECUTE));
Assert.assertTrue("User did not have write permission.", permissions.contains(PosixFilePermission.OWNER_WRITE));
Assert.assertFalse("Group did have read permission.", permissions.contains(PosixFilePermission.GROUP_READ));
Assert.assertFalse("Group did have execute permission.", permissions.contains(PosixFilePermission.GROUP_EXECUTE));
Assert.assertFalse("Group did have write permission.", permissions.contains(PosixFilePermission.GROUP_WRITE));
Assert.assertFalse("Others did have read permission.", permissions.contains(PosixFilePermission.OTHERS_READ));
Assert.assertFalse("Others did have execute permission.", permissions.contains(PosixFilePermission.OTHERS_EXECUTE));
Assert.assertFalse("Others did have write permission.", permissions.contains(PosixFilePermission.OTHERS_WRITE));