I recently implemented Java 7's WatchService and it works perfectly. Now I wondered if there is a way to get all the Events which occured since the last run of my program. For example:
I run my program, create some files, edit some files and I get all the corresponding Events
I close my program
I create a file named foo.txt
I start my program, and the first event i get is an ENTRY_CREATE for foo.txt
I thought about saving the lastModifiedDate and searching for files and directorys newer than the last execution of my program. Is there another (and better) way to do this?
There is no better way to do this if your program is meant to scan for all file changes (apart from storing files in a content / source control repository, but that would be external to your program).
Java 7's WatchService is only a more performant way than continuously looping and comparing file dates / folder contents, hence you need to implement your own logic to solve your own problem.
There is no way to do this in Java, or in any other programming language.
The operating system doesn't (and can't) buffer file system events on the off-chance that someone might start a program to process them. The event monitor / delivery system captures the events for a running application that is listening for them. When nothing is listening, the events are not captured.
You could write a small daemon (system service on Windows) which runs continuously and listens for file system changes. It could write these to a file. When your application runs, rather than listening for changes itself, it could just read the file. As events happen while it runs, the daemon will continue to receive them and send them through the file to the application.
You would need to ensure that the file was organised in such a way that it could be written to and read from safely at the same time, and that it did not grow indefinitely.
Related
I have created and am working on a server-application that monitors for specific folders and takes appropriate actions whenever files are being added.
Now I come to the point where I want to be able to shutdown the program, for example for applying a patch.
The server runs simply in a command prompt, how can I signal that I want to perform maintenance on it? I do not think reading System.in is feasible as I am also outputting text in the prompt.
Regards.
You could try reading System.in as System.in and System.out are different file descriptors. What this means is that by writing things in console you are not writing in the same place than when you are typing, so console output should not matter for reading commands in the prompt.
A second application can be used to communicate with the server application. You can use Java Management Extensions or implement your own client/server communication using sockets.
Another way to achieve this is that server periodically checks for existence of an specific file somewhere on hard disk. If server finds that specific file, it will shut down.
Im reading a log file in Java on a Linux box on a continual schedule of 2 minutes looking for certain messages. I store the last offset (RandomAccessFile getFilePointer) and read from it onwards when I detect LastModified has changed; is this best practice or even right?
If you are using Java 7, then you can use WatchService to notify you when a File inside of a directory changes. This is similar to interrupts and doesn't need continuous polling. See this for more details on WatchService. Otherwise you are better off with the polling method you already do.
Try this Tailer:
http://commons.apache.org/io/apidocs/org/apache/commons/io/input/Tailer.html
This is Java implementation of "tail -f" Unix functionality.
For detecting changes on files, or changes in any content inside directory, you can watch that directory for new content, modification of existing content and deletion of any existing content inside any directory.
See post: Directory watching for changes in java
I want to monitor a directory, and when a file appears there open it, process it and then move it to another directory. The problem is how to check that the other program is done writing it. With Java 7, I can use a WatchService from FileSystem, but I can only check when the files are created. What I want is to know when all file handles are released.
My first thought was that I could obtain an exclusive lock, but it turned out that it was possible to kick out another application while it was actually updating the file.
What is the preferred way to do this in Java? Thanks!
The Watcher APIs currently allow you to see events when a file system object is created, modified or deleted in a watched directory. They don't tell you about other inotify events (on Linux). In fact, I don't think there is a way to do this in pure Java.
I was looking for a way to do this myself a few weeks ago and I came across a mail thread that suggested that you could write a custom implementation of the FileSystem api that provided a file watcher that supported other file system events. I decided not to pursue it because I had an alternative solution ... based on knowledge of how the files I am watching are being produced.
In my case, the files are produced by instruments that save image files to a shared drive. The solution is to watch the stream of "modified" events for a newly created file. When it stops and no more have been forthcoming for a couple of seconds (the "settling time"), then the file can be processed.
If this solution proves to be unreliable, the fallback is to implement the watching and initial processing (taking a snapshot of the file) in C / C++ using the inotify calls directly. This will allow me to directly observe the file close event.
The simplest way for a filebased interface is:
The sender writes the files with a changed filename (e.g. "example.xml_")
When the sender has finished writing the file, he renames it (e.g. "example.xml_" to "example.xml")
The receiver scans only for "*.xml"
I have 2 machines both running one process each. shell process on Machine A will scp something to machine B, and java process on B will use these files. Both processes run as crontab tasks.
How to achieve synchronization/atomicity etc? How to signal that whole of file has been written.
so that process on B always has access to latest and complete files, the handler doesn't go stale..
Assuming you're using a filesystem with atomic moves you can do that. Or use symbolic links.
A copies the file to a temp location on B. When the upload is complete, it relocates, with a move or symlink, the file into the expected location. B can then only ever see completely uploaded files.
If your process on A cannot SSH into B to make the final move, it could rather add another, zero byte marker file which indicates the upload is complete.
A uploads FOO.txt, when that upload is complete, it creates the FOO.txt.done file. B then scans the directory for *.done files, and uses the associated data file. Plus cleanup of course.
A quick'n'dirty work around / solution for the problem could be resolved by using user control, setting the ownership (and chmod'ing appropriately) when copying the file (using SCP from A), when finishing then changing the ownership / permissions so that the process on B will be able to access the file.
A few hacks that I can suggest I guess. You can either :-
create a temp file (like a lock file) at the start of file writing, the presence of which means that the file being streamed is being written to and delete the temp file once the writing is complete.
OR
Remove crontab on the consumer process and have the scp process send a signal (JMS/ unicast). The java process could just be a listener on a queue/ socket or be invoked upon receiving a unicast.
I have a button that I want to disable as long as there isn't a specific number of files in a directory.
Is there some kind of listener that notifies me at the moment a file is created or deleted in a directory?
There's no current native support in Java for file system events and monitoring. JNotify is a useful library for doing this. You should set it up to monitor the directory for modifications, and then determine yourself what's been added/removed.
Java 7 will have file system event support built into it.
One thing you might want to consider - If you're listening for creation events then you'll want to make sure that the file is completely written before you start reading it. I'm not sure what type of support Java 7 will offer for this problem.
I've implemented mechanisms like this in the past and this particular problem required special handling. If you are controlling both the file reader and writer then you can get around this with naming conventions, etc. (the writer names the file xxx.prt and renames the file when it's done being written). Since I didn't have control of the writer I had to add another polling mechanism to check the file size on an interval to make sure each new file was actually ready to be read. Not a perfect solution, but was sufficient for my case.
My two cents...