I am facing a very strange issue in java(J2EE App.). I have an Application that reads data from customer configuration Files placed in a location on local machine/ server , reads it via Java API and Displays it on the UI of the Tool. later, Through UI, the data can be Changed and is written back to the file by tool via Java API.
The problem is that the tool fails to read information (reads half of the file) and causes data loss on the UI. But the Issue is not Consistent. It happens About 1 in 20 times only. Rest it always reads well.
I am not able to reproduce the issue on my WINDOwS machine. But is was seen in the Production Server (ON UNIX Environment).
Please Suggest what I need to check. Are there any Permission related Issues in UNIX.
Can my tool have a Bug in it? or is it environment problem that the tool suffers from.
Should I try
try {
// my code
} catch(Throwable t) {
t.printStackTrace();
}
To debug if it's an issue in Environment?
Windows tends to lock files so you are less likely to read it while it is being written to. Linux takes the view you know what you are doing and doesn't lock by default. This means you can see files before you have finished. This is a common problem with files as they are not designed as a messaging protocol and so you have to come up with something heuristic to handle this deficiency. A better approach is to not use files for communication between processes or you have to be very aware of it's limitations.
Related
Java application is running from a Jar on a network drive. If Jar file becomes unavailable for some reason then there is NoClassDefFoundError as expected. Like in example network connection is lost. But what I find odd is that the application will still crash completely and will not recover if Failover occurs on the network drive where Jar file is located.
Failover means Network Drive doesn't change and it only flicks briefly and becomes available again right away but it changes some sort of internal low level drive (the infrastructure guys call it Node). After the Failover happens (node is changed) all users that were connected to application which is running in a Citrix server get the same exception.
I would have thought, that once network drive is back online the JVM should be able to recover, but it seems it is trying to obtain classes from the old node where Jar was located and not from the new Node. Does anyone know why JVM will present this behavior ?
The JVM only opens a file once and keeps the handle open to read classes from it when needed. When the network drive gets disconnected the handle becomes invalid.
One presumably could write a classloader that tries to reopen the file and verifies that it is indeed the same file as before, e.g. via hashing, but the standard implementation does not since filesystems are assumed to be reliable.
Most other executable programs started from network drives are likely to experience similar problems.
You should use a clustering network filesystem that does not invalidate application-visible handles and instead does failover transparently.
I have an file system based application written in Java both on Windows and Mac OS.
My requirement is to track the changes made to files/folders under a directory. Operations to track are normal file level operations like CRUD ones.
when my application runs i can run a watch service from java nio and can track the changes (though detecting rename is still a problem in watch service ).
My problem comes when i have to detect changes when the application is not running. i have read that the file backup software do it through change journal feature of Windows NTFS.
My questions are as follows
(a) Are change journal apis available in . NET managed code of c# (or even in Java) or only availalbe through c++ as shown in the examples?
(b) Is change jounrnal or equivalent available in HFS plus (mac os) ? if yes, are there apis available (any language)?
(c ) Is there any better way to track the changes done in file system when the application is not running ?
cheers,
Saurav
Read this:
Keeping an eye in the NTFS Change Journal
Part 1 - https://technet.microsoft.com/en-us/library/bb742450.aspx
Part 2 - http://www.microsoft.com/msj/1099/journal2/journal2.aspx
I am working on a Java Web Application based on Java 6/Tomcat 6.0. It's a web based document management system. The customers may upload any kind of file to that web application. After uploading a file a new Thread is spawned, in which the uploaded file is analyzed. The analysis is done using a third party library.
This third-party-libraries works fine in about 90% of the analyze-jobs, but sometimes (depending on the uploaded file) the logic starts to use all remaining memory, leading to an OutOfMemoryError.
As the whole application is running in a single JVM, the OoM-Error is not only affecting the analyze-jobs, but has also impact on other features. In the worst case scenario, the application crashes completely or remains in an inconsistent state.
I am now looking for a rather quick (but safe) way to handle those OoM-Errors. Replacing the library currently is no option (that's why I have neither mentioned the name of the library, nor what kind of analysis is done). Does anybody have an idea of what could be done to work around this error?
I've been thinking about launching a new process (java.lang.ProcessBuilder) to have a new JVM. If the third-party-lib causes an OoM-Error there, it would not have effects on the web application. On the other hand, this would cause additional effort to synchronize the new Process with the Analysis-Part of the web application. Does anybody have any experience with such a system (especially with regards to the stability of the system)?
Some more information:
1) The analysis part can be summarized as a kind of text extraction. The module receives a file reference as input and writes the analysis result into a text file. The resulting text-file is further processed within the web applications business logic. Currently the workflow is synchronous. The business logics waits for the third-party-lib to complete its job. There is no queuing or other asynchronous approach.
2) I am quite sure that the third-party-library causes the OoM-Error. I've tested the analysis part in isolation with different files of different sizes. The file that causes the OoM-Error is quite small (about 4MB). I have done further tests with that particular file. While having a JVM with 256MB of heap, the analysis crashes due to the OoM-Error. The same test in a JVM with 512MB heap passes. However, increasing the heap size will only help for a short period of time, as a larger test file again causes the test to fail due to OoM-Error.
3) A Limit for the size of files being uploaded is in place; but of course you cannot have a limit of 4MB per file. Same is for the OS and architecture. The system has to work on both 32- and 64-bit systems (Windows and Linux)
It depends on both the client and the server as well as the design of the web app. You need to answer a few questions:
what is supposed to happen as a result of the analysis and when is it supposed to happen?
Does the client wait for the result of the analysis?
What is returned to the client?
You also need to determine the nature of the OOM.
It is possible that you might want to handle the file upload and the file analysis separately. For instance, your webapp can upload the file to somewhere in the file system and you can defer the analysis part to a web service, which would be passed a reference to the file location. The webservice may or may not be called asynchronously, depending on how and when the client that uploaded the file needs notification in the case of a problem in the analysis.
All of these factors go into your determination.
Other considerations, what JVM are you using, what is the OS and how is it configured in terms of system memory? Is it the JVM 32 or 64 bit, what is the max file size allowed on upload, what kind of garbage collectors have you tried.
It is possible that you can solve this problem from an infrastructure perspective as opposed to changing the code. Limiting the max size of the file upload, moving from 32 to 64 bit, changing the garbage collector, upgrading libraries after determining whether or not there is a bug or memory leak in one of them, etc.
One other red flag that is glaring, you say "a thread is spawned". While this sort of thing is possible it is often frowned upon in the JEE world. Spawning threads yourself can cause problems in how the container manages resources. Make sure you are not causing the issue yourself, try a file load independently in a test environment on a file that is known to cause problems (if that can be ascertained). This will help you determine ff the problem is the third party library or a design one.
Why not have a (possibly clustered) application per 3rd-party lib that handles file analysation. Those applications are called remotely (possibly asynchronously) from your main application. They are passed a URL which points to the file they should analyze and return their analysation results.
When a file upload completed the analyzation job is put into the queue. When an analyzation application is up again after it crashed it will resume consuming messages from the queue.
I am using Cassandra DB in my java application. Am using Thrift client to connect Cassandra from my java application. If the Cassandra disk get full means it automatically terminates. So from my java program i could not find the correct error why the Cassandra is down.
So how to avoid the auto termination of Cassandra or is their any way to identify the disk full error ?
Also i dont have physical access to cassandra drive. Its running in some other remote machine.
Disk errors and, in general, generic hardware/system errors are not usually properly handled in any application. The database should only provide as much durability as possible in such scenarios and it is the correct behavior - shut down and break as little as possible.
As for your application - if you can not connect to the database, there is no difference as to what caused an error. You app will not work anyway.
There are special tools that can monitor your machine, i.e. Nagios. If you are the administrator of that server, use such applications. When the disk is getting filled up you will receive an email or text. Use such tools and don't break an open door by implementing several hundred of lines of code to handle random and very rare situations.
Setup ssh access to Casandra machine and use some ssh client like JSch to run df /casandra/drive (if Linux) or fsutil volume diskfree c:\casandra\drive (if Windows) from your Java client. Capture output that is simple and parse to obtain the free disk space. That way your application will monitor that is happening there and probably should alert the user and refuse to add data if there is an out of disk space threat.
You can also use standard monitoring tools or setup server side script to send the message if the disk space low. However this will not stop your application from crashing, you need to take actions after you see that the disk space is low.
Is there a reliable, cross-platform way to do IPC (between two JVMs running on the same host) in Java (J2SE) that doesn't rely on the network stack?
To be more specific, I have a server application that I'd like to provide a small "monitoring" GUI app for. The monitor app would simply talk to the server process and display simple status information. The server app has a web interface for most of its interaction, but sometimes things go wrong (port conflict, user forgot password) that require a local control app.
In the past I've done this by having the server listen on 127.0.01 on a specific port and the client communicates that way. However, this isn't as reliable as I'd like. Certain things can make this not work (Windows's network stack can be bizarre with VPN adapters, MediaSense, laptops lid closing/power saving modes). You can imagine the user's confusion when the tool they use to diagnose the server doesn't even think the server is running.
Named Pipes seem plausible, but Java doesn't seem to have an API for them unless I'm mistaken. Ideas? Third party libraries that support this? My performance requirements are obviously extremely lax in case that helps.
One of my specialties is really low-tech solutions. Especially if your performance requirements aren't critical:
The low-low tech alternative to named pipes is named FILES. Think yourself up a protocol where one app writes a file and another reads it. If need be, you can do semaphoring between them.
Remember that a rename is pretty much an atomic operation, so you could calmly write a file in some process and then make it magically appear in its entirety by renaming/moving it from somewhere that wasn't previously visible.
You can poll for data by checking for appearance of a file (in a loop with a SLEEP in it), and you can signal completion by deleting the file.
An added benefit is that you can debug your app using the DIR command :)
Depending on how much data you need to pass between the server and the diagnostic tool you could:
go low-tech and have a background thread check a file in the file system; fetch commands from it; write ouput into a second to be picked up by the diagnostic tool.
build a component that manages an input/output queue in shared memory connecting to it via JNI.
Consider JMX. I do not know if any of the Windows JVM's allow JMX over shared memory.
Does Windows even have named pipes? I was going to suggest it. You'd just have to use an exec() to create it.
Map a read_write byte buffer into memory from a FileChannel. Write status information into the byte buffer, then call force() to get it written out. On the monitor side, open up the same file and map it into memory too. Poll it periodically to find out the status.