I have a java program which creates a lock file to ensure that no other executions run at the same time as it. If the program runs it creates the file, and upon exit, either successful or via an exception, the file is removed. But if the user hits Ctrl+C, closes the terminal, or in some other way interrupts execution, the file is not deleted. Is there any way to detect this interrupt command and ensure that the file is deleted in this case as well?
You might want to look into shutdown hooks.
Also, this is probably a duplicate or near-duplicate of this previous SO question from a day or two ago:
How can I "intercept" Ctrl+C in a CLI application?
The only good way to deal with this is to timestamp your log file, and update the logfile with a new timestamp every few seconds. File.deleteOnExit will help, but no method is foolproof (what happens when someone just pulls the power cord?).
If you write a timestamp, then you can check for a valid timestamp (recently written) and overwrite the log file if it is too badly out of date. That way a stale lockfile won't get in the way of the user.
Related
I have multiple executable JARs, let's call them task1.jar, task2.jar, task3.jar, etc...
They mainly access the database and do housekeeping tasks on the server.
They are run by cron jobs, one after another, early in the morning everyday. The crob jobs are set about 15 minutes apart.
The crob jobs simply just call:
java -jar taskX.jar
Depending on the data on the database, their durations to finish will vary.
The task executions must not overlap. Otherwise when they access the database at the same time, the server will crash. It has happened before when one task takes too long to run and hasn't finished before another task has started.
So my question is... is there anyway to check a JAR has started or finished so that the next JAR can be delayed?
I would preferably resolve this with Java methods over changing any configs on server which I have limited access on.
Edit: #mksmanjit Sorry, I didn't mention before. There is also one complication - not all of them are run everyday. For example, task1.jar and task3.jar may be run everyday, task2.jar may be run every two days and task4.jar may be run every Mon, Wed, Fri, etc... So the situation can get quite complicated.
Couple of ways as I see it !
Let all JAR (or tasks) write Logs in a common or a separate log file. (Usually feasible if you only want to capture minor details such as start time , stop time, result etc) Assuming format you keep is manually readable.
As per above suggestion, create a Table in DB to keep a track of task activity along with task name, it's details such as trigger by, start time, end time, result etc
Alternatively, create database Triggers to monitor event (such as write or read) on DB table and whenever your task executes, trigger will also execute depending on what DB actions you want your trigger to perform such as sending e-mail, writing to table, printing simple log
Let JARs (or tasks) also print logs on console just in case assuming they are designed to view from any console or UI at runtime !
I have a Java application that creates multiple threads. There is 1 producer thread which reads from a 10gb file, parses that information, creates objects from it and puts them into multiple blocking queues (5 queues).
The rest of the 5 consumer threads read from a blockingqueue (each consumer thread has its own blockingqueue). The consumer threads then each write to an individual file, so 5 files in total get created. It takes around 30min to create all files.
The problem:
The threads are writing to an external mount directory in a linux box. We've experience problems where other linux mounts have gone down and applications crash so I want to prevent that in this application.
What I would like to do is keep checking if the mount (directory) exists before writing to it. Im assuming if the directory goes down it will throw a FileNotFoundException. If that is the case I want it to keep checking if the directory is there for about 10-20min before completely crashing. Because I dont want to have to read the 10gb file again I want the consumer threads to be able to pick up from where they last left off.
What Im not sure would be best practice is:
Is it best to check if the directory exists in the main class before creating the threads? Or check in each consumer thread?
If I keep checking if the directory exists in each consumer thread it seems like repeatable code. I can check in the main class but it takes 30min to create these files. What if in those 30min the mount goes down then if Im only checking in the main class whether the directory exists the application will crash. Or if Im already writing to a directory is it impossible for an external directory to go down? Does it get locked?
thank you
We have something similar in our application, but in our case we are running a web app and if our mounted file system goes down we just throw an exception, but we want to do something more elegant, like you do...
I would recommend using a combination of the following patterns: State, CircuitBreaker, which I believe CircuitBreaker is a more specific version of the State pattern, and Observer/Observable.
These would work in the following way...
Create something that represents your file system. Maybe a class called MountedFileSystem. Make all your write calls to this particular class.
This class will catch all FileNotFoundException and one occurs, the CircutBreaker gets triggered. This change will be like the State pattern. One state is when things are working 'fine', the other state is when things aren't working 'fine', meaning that the mount has gone away.
Then, in the background, I would have a task that starts on a thread and checks the actual underlying file system to see if it is back. When the file system is back, change the state in the MountedFileSystem, and fire an Event (Observer/Observable) to try writing the files again to disk.
And as yuan quigfei stated, I am fairly certain you're going to have to rewrite those files. I just don't see being able to restart writing to them, but perhaps someone else has an idea.
write a method to detect folder exist or not.
call this method before actual writing.
create 5 thread based on 2. Once detect file is not existed, you seems have no choice but rewrite. Of course, you don't need re-read if all your content are in memory already(Big memory).
Using Java's java.nio.channels.FileLock I am trying to synchronize file reading and writing on a Windows filesystem. I have a test program that runs in a loop:
Lock the file X.LOCK
Test that X.JSON exists (just a consistency check)
Write the file X.TMP
Rename X.TMP to X.JSON using java.nio.files.Files.move() deletes the old X.JSON and renames X.TMP to X.JSON in a atomic action.
Test that X.JSON exists (this always returns true)
Release the lock on X.LOCK
I run this in a tight loop in multiple instances of the test program. It locks the symbolic file "X.LOCK" and not the actual file that is being written and renamed. I believe that is necessary to preserve the lock through the rename operation.
Here is what I find: In about 2% of the cases, process 1 will write/rename/release the lock, and process 2 which was waiting on that lock will get that lock, start executing, but find that X.JSON does not exist. The "exists" check returns false!
If I introduce a delay (200ms) after the rename, and before the unlock, then the whole thing runs 100% reliably. I can try smaller delays, but I am loath to add any delay since that is never the right answer to making a reliable program.
It appears that when one process atomically renames a file, it takes some time for the other process to see that. But the unlock signal goes faster! So the lock signal tells the other program to move forward, and that other program can't see the file it is supposed to be working on!
Question: is there any way I can force the unlock signal to be sent AFTER the file system has settled and guaranteed to be consistent with operations that were put in there before the unlock was called?
Any hints on where I can look for information on this kind of timing/sequencing on a Windows file system using Java? I have not tried this test program on any other platform yet, but I certainly will check Linux soon.
UPDATE
I am suspicious of interference from virus scanning. It got a test to a reproducible state, and it was failing about 1% of the time, this time reporting "AccessDeniedException". I think the virus scan might be kicking in, scanning the file between being created and being renamed, and when it does this, it runs at a higher privilege, and causes this error when trying to rename it. Anyone else run into this problem?
The solution appears to be that on a system where virus scan is running, depending upon the specific brand of virus scanner, it is possible that the call to move can be interfered with. I was calling:
java.nio.files.Files.move(src, dest, StandardCopyOption.REPLACE_EXISTING,
StandardCopyOption.ATOMIC_MOVE );
This command will effectively delete the dest if it exists, and rename the src file to the dest, and it will do it atomically. It is documented that if it can not do it atomically, it will throw an exception. I was getting AccessDeniedException which is not mentioned in the documentation specifically but apparently happens.
What appears to be happening is that -- and this all depended on a specific timing that was happening about 1% of the time -- was that the operation of the virus scan either on the src file or the dest file caused the atomic move to fail.
I tried on each of three different systems configured differently. The windows computer with the Microsoft Windows Defender never caused the AccessDeniedException while another with Trend Micro virus scan was failing regularly. That is not a thorough survey of virus scan options; they were the only options I had available for test. The machine with the Trend Micro also has an encrypted hard disk, and that might be a factor to make the timing such as to trip this problem.
I even went so far as to implement a "retry" where if the move threw an exception, the code would wait 10ms and try again. Even with this, the retry failed about 0.1% of the time. Maybe I could have waited longer, but that would in any case be a problem making the code slower.
What worked was to add a step to delete the file being replaced before doing the move. My guess is that the virus scan is either stopped by the delete, or else it continues to scan on the src or dest file without bothering the move command. The steps are these:
Lock the file X.LOCK
Test that X.JSON exists (just a consistency check)
Write the file X.TMP
(NEW) Delete the old X.JSON
Rename X.TMP to X.JSON using java.nio.files.Files.move() simply renames X.TMP to X.JSON in an atomic action.
Test that X.JSON exists (this always returns true)
Release the lock on X.LOCK
Is this now 100% reliable? I can't say for sure, since all this is timing dependent. It is possible that this just changed the timing in a way that allows it to run.
Let's assume I have a Java program with a main class in persondetails.java.
Person1 runs persondetails.java to get their details. When a second person tries to access the same program in same system, it should say someone is already using this program, wait until the specific block completes in person1.
How can I communicate between these two Java processes in the same system?
If I understand your question correctly you would like to have a program that only allows one instance to be running at a time? I can think of two possibilities off the top of my head.
Using some kind of "lock" file on the filesystem. When the program is executed it could check for the existence of the lock file and display an error if it already exists. If the lock file doesn't exist it would be created and the program allowed to continue. You would just have to make sure you delete the lock file when the program finishes.
Binding to a specific port (that is unlikely to already being used). When the program is executed you would attempt to bind to the port, if you can the program continues otherwise you give an error.
Either way you are just using a locking mechanism to detect that the other process is running.
Locking using the file system can be problematic. For example if the program fails and stops executing unexpectedly the lock file could remain and prevent future execution of the program. At the same time binding to a port to accomplish this is really a misuse of that functionality and could be problematic in it's own way (port already in use, or the application not being allowed to bind to the port).
I personally would be carefully considering whether or not it is really necessary to do this kind of locking between different processes. I would consider it favorable to avoid using either of the options I mentioned previously.
At a very very basic level, the program could create a file named lock. The process would be something like
Process A checks for existence of lock.
If lock does not exist, Process A creates lock and begins to work.
At the end of work, Process A deletes lock.
If lock exists, Process A waits until lock does not exist.
Process A can poll or wait for an interrupt or any of a variety of callback mechanisms.
Hi all i Have two process in my java application as below.
Process p1=some process;
Process p2=some process;
At the first time it starts its working perfectly.But when i reload the application the older process is also running so i just want to kill the older process if its executing.I know that we can kill a process by using its process id.But how can i get the process id of this p1 and p2.Any idea?.
Catch the reload event in your application and terminate the child process before you exit.
If that doesn't work, you probably have a bug in your design. Try to fix that bug. Really. I mean it.
A workaround is to write the PID into a text file. When you start, read the text file, check if this is the correct process and then kill it.
Without the check, you might be killing a perfectly valid process which was created by someone else. My reasoning is like so: You can't kill the process when you reload. That means you can't delete the text file either. So the PID file will always exist. Eventually, you will kill the wrong process. That will happen once per year. It will happen the first time six months after you left/changed job/etc. Nobody will understand what is going on because it's almost impossible to reproduce.
As others have said, it would be better to terminate the process in a more natural way.
However, if you do need to actually forcibly terminate it, then you could call the destroy() method on the Process object, whenever you don't care about it running anymore.
Also, do you really need to run the code as a Process? There are generally better ways.