Java application is running from a Jar on a network drive. If Jar file becomes unavailable for some reason then there is NoClassDefFoundError as expected. Like in example network connection is lost. But what I find odd is that the application will still crash completely and will not recover if Failover occurs on the network drive where Jar file is located.
Failover means Network Drive doesn't change and it only flicks briefly and becomes available again right away but it changes some sort of internal low level drive (the infrastructure guys call it Node). After the Failover happens (node is changed) all users that were connected to application which is running in a Citrix server get the same exception.
I would have thought, that once network drive is back online the JVM should be able to recover, but it seems it is trying to obtain classes from the old node where Jar was located and not from the new Node. Does anyone know why JVM will present this behavior ?
The JVM only opens a file once and keeps the handle open to read classes from it when needed. When the network drive gets disconnected the handle becomes invalid.
One presumably could write a classloader that tries to reopen the file and verifies that it is indeed the same file as before, e.g. via hashing, but the standard implementation does not since filesystems are assumed to be reliable.
Most other executable programs started from network drives are likely to experience similar problems.
You should use a clustering network filesystem that does not invalidate application-visible handles and instead does failover transparently.
Related
I am facing a very strange issue in java(J2EE App.). I have an Application that reads data from customer configuration Files placed in a location on local machine/ server , reads it via Java API and Displays it on the UI of the Tool. later, Through UI, the data can be Changed and is written back to the file by tool via Java API.
The problem is that the tool fails to read information (reads half of the file) and causes data loss on the UI. But the Issue is not Consistent. It happens About 1 in 20 times only. Rest it always reads well.
I am not able to reproduce the issue on my WINDOwS machine. But is was seen in the Production Server (ON UNIX Environment).
Please Suggest what I need to check. Are there any Permission related Issues in UNIX.
Can my tool have a Bug in it? or is it environment problem that the tool suffers from.
Should I try
try {
// my code
} catch(Throwable t) {
t.printStackTrace();
}
To debug if it's an issue in Environment?
Windows tends to lock files so you are less likely to read it while it is being written to. Linux takes the view you know what you are doing and doesn't lock by default. This means you can see files before you have finished. This is a common problem with files as they are not designed as a messaging protocol and so you have to come up with something heuristic to handle this deficiency. A better approach is to not use files for communication between processes or you have to be very aware of it's limitations.
I am using Cassandra DB in my java application. Am using Thrift client to connect Cassandra from my java application. If the Cassandra disk get full means it automatically terminates. So from my java program i could not find the correct error why the Cassandra is down.
So how to avoid the auto termination of Cassandra or is their any way to identify the disk full error ?
Also i dont have physical access to cassandra drive. Its running in some other remote machine.
Disk errors and, in general, generic hardware/system errors are not usually properly handled in any application. The database should only provide as much durability as possible in such scenarios and it is the correct behavior - shut down and break as little as possible.
As for your application - if you can not connect to the database, there is no difference as to what caused an error. You app will not work anyway.
There are special tools that can monitor your machine, i.e. Nagios. If you are the administrator of that server, use such applications. When the disk is getting filled up you will receive an email or text. Use such tools and don't break an open door by implementing several hundred of lines of code to handle random and very rare situations.
Setup ssh access to Casandra machine and use some ssh client like JSch to run df /casandra/drive (if Linux) or fsutil volume diskfree c:\casandra\drive (if Windows) from your Java client. Capture output that is simple and parse to obtain the free disk space. That way your application will monitor that is happening there and probably should alert the user and refuse to add data if there is an out of disk space threat.
You can also use standard monitoring tools or setup server side script to send the message if the disk space low. However this will not stop your application from crashing, you need to take actions after you see that the disk space is low.
I have mounted a NFS mount in my local machine (So, I suppose my machine is the NFS client) and a file is being written in the NFS mount by the remote machine hosting the NFS mount (NFS server). Now, how will I be able to detect using JAVA that the file being written to in the remote machine has been closed before me reading from my local machine?
Or, are there any alternative ways to achieve what I want?
Edit: I would also like to mention that I have no control over the remote NFS server. I can only mount in read only mode.
As you stated that you have no control over the NFS server nor the program updating the file on the NFS server, there is not a lot you can do. The only thing you can really make use of is the fact that when a file is updated its modification time is updated. This should be available to your java client using File.lastModified(). If your application remembers the last modification date, you can then at least know if there is data to process since you last read it. It's definitely not perfect. There are race conditions etc, but depending on what exactly your application needs/does, it may be enough. You can't tell if the file is closed or open on the server.
If the file modifications are slow to appear, you could also mount using the noac mount option. While this hurts NFS performance on a regular NFS client, it may work for you if your machine is not using a lot of other NFS files on that drive. See "man nfs".
As a side note, I can't see that file locking will help as you gave no indication that the application on the NFS server uses locks anyway.
we've bean struggling here at work by somebody suggestion that we should decrease the size of our war file, specifically the WEB-INF/lib directory size, in order to improve our production JBoss instance performance. Something I'm still suspicious about.
We have around 15 web apps deploy in our application server, each about 15 to 20 MB in size.
I know there are a lot of variables involved on this, but has anyone of you actually deal with this situation? Does the .war files size actually has a significant impact on web containers in general?
What advice can you offer?
Thank U.
There are many things to be suspicious of here:
What about the application is not performing to the level you would like?
Have you measured the application to find out which components are causing the lack of performance?
What are the bottlenecks in the application/system?
The size of the application alone has nothing to do with any sort of runtime performance. The number of classes loaded during the lifetime of the application has an impact on memory usage of the application, but an incredibly negligible one.
When dealing with "performance issues", the solution always follows the same general steps:
What does it mean when we say "bad performance"?
What specifically is not performing? Measure, measure, measure.
Can we improve the specific component not performing to the level we want?
If so, implement the ideas, measure again to find out if performance has truly improved.
Need you to tell us the operating system.
Do you have antivirus live protection?
A war/jar file is actually a zip file - i.e., if you renamed a .war to a .zip, you can use a zip utility to view/unzip it.
During deployment, the war file is unzipped once into a designated folder. If you have live-protection, the antivirus utility might take some time to scan the new branch of directories created and slow down any access to them.
Many web app frameworks, like JSPs, create temporary files and your live-protection would get into action to scan them.
If this is your situation, you have to decide whether you wish to exclude your web-app from antivirus live-scanning.
Are you running Linux but your web directory is accessed using ntfs-3g? If so, check if the ntfs directory is compressed. ntfs-3g has problems accessing compressed ntfs files especially when multiple files are manipulated/created/uncompressed simultaneously. In the first place, unless there are some extremely valid reasons (which I can't see any), a web app directory should be a local partition in a format native to Linux.
Use wireshark to monitor the network activity. Find out if web apps are causing accesses to remote file systems. See if there are too many retransmits whenever the web apps are active. Excessive retransmits or requests for retransmits means the network pipeline has integrity problems. I am still trying to understand this issue myself - some network cards have buffering problems (as though buffer overflow) operating in Linux but not in Windows.
Wireshark is not difficult to use as long as you have an understanding of ip addresses, and you might wish to write awk, perl or python scripts to analyze the traffic. Personally, I would use SAS.
Is there a reliable, cross-platform way to do IPC (between two JVMs running on the same host) in Java (J2SE) that doesn't rely on the network stack?
To be more specific, I have a server application that I'd like to provide a small "monitoring" GUI app for. The monitor app would simply talk to the server process and display simple status information. The server app has a web interface for most of its interaction, but sometimes things go wrong (port conflict, user forgot password) that require a local control app.
In the past I've done this by having the server listen on 127.0.01 on a specific port and the client communicates that way. However, this isn't as reliable as I'd like. Certain things can make this not work (Windows's network stack can be bizarre with VPN adapters, MediaSense, laptops lid closing/power saving modes). You can imagine the user's confusion when the tool they use to diagnose the server doesn't even think the server is running.
Named Pipes seem plausible, but Java doesn't seem to have an API for them unless I'm mistaken. Ideas? Third party libraries that support this? My performance requirements are obviously extremely lax in case that helps.
One of my specialties is really low-tech solutions. Especially if your performance requirements aren't critical:
The low-low tech alternative to named pipes is named FILES. Think yourself up a protocol where one app writes a file and another reads it. If need be, you can do semaphoring between them.
Remember that a rename is pretty much an atomic operation, so you could calmly write a file in some process and then make it magically appear in its entirety by renaming/moving it from somewhere that wasn't previously visible.
You can poll for data by checking for appearance of a file (in a loop with a SLEEP in it), and you can signal completion by deleting the file.
An added benefit is that you can debug your app using the DIR command :)
Depending on how much data you need to pass between the server and the diagnostic tool you could:
go low-tech and have a background thread check a file in the file system; fetch commands from it; write ouput into a second to be picked up by the diagnostic tool.
build a component that manages an input/output queue in shared memory connecting to it via JNI.
Consider JMX. I do not know if any of the Windows JVM's allow JMX over shared memory.
Does Windows even have named pipes? I was going to suggest it. You'd just have to use an exec() to create it.
Map a read_write byte buffer into memory from a FileChannel. Write status information into the byte buffer, then call force() to get it written out. On the monitor side, open up the same file and map it into memory too. Poll it periodically to find out the status.