Let me try to explain..We have a server running Jboss 5.1 with really many concurrent connections, and well, sometimes we need to restart and do some maintenance/release.
The problem is, sometimes we start up the server and it looks like some library just didn't load at startup time, or for some other reason, we keep getting NullPointerException without any stacktrace.. we do know the class it is thrown, but by checking the code, where nullpointers could for some reason be thrown, and even forcing a few exceptions, all of them have at least the stacktrace.
I found here in stackoverflow many answers telling about the JVM doing some optimizations but we have just started up the server.I don't believe it does any optimization even when the first exception is thrown.I have also checked for any relevant setStackTrace or codes like ex.toString()..which we also do not have.
So my question actually is, for what reason other than this optimization thing, could such exception be thrown without stacktrace?
EDIT:
My question is NOT related to the -XX:-OmitStackTraceInFastThrow, since it happens even at the first exception!
Use the -XX:-OmitStackTraceInFastThrow JVM argument.
Related
I'm working on a physics program and as time goes on I continue to tie up more and more loose ends with exception handling to prevent freezing and locking up. For Now I am getting certain exceptions in my console such as StringFormatException(s), but this error does not freeze the program nor does it affect run-time in any way, it simply shows up in the IDE's terminal (Eclipse IDE, JRE 7). When dealing with errors such as this one, that (seemingly) do not affect run-time, is it still important to handle the exception even if the program works fine? If I were to export the program as a .jar with only these types of errors not handled, would users even notice? Why do some errors cause large problems while others don't?
Side Note:
I'm asking this primarily because in the future, when taking on projects much larger than this one I believe there are going to be times when a vast number of spots in my code may exist where exceptions are thrown but not yet handled. On very large projects involving tens of thousands of lines of code do many programmers dismiss these types of errors that do not affect logic or run-time and conclude that to go back and fix all of them would not be necessary or worth the time? Or is it essential to develop a habit of committing possible error throwing situations to memory and staying on top of all possible errors.
Examples of how software with similar errors to this, ones that allow the program to function as if nothing has happened would be greatly appreciated if you have the source code available and have taken note of this sort of thing.
is it still important to handle the exception even if the program works fine?
Even if it appears to work fine it may not be. If you get into the habit of ignoring Exceptions you can easily ignore an Exception which is causing a problem.
If I were to export the program as a .jar with only these types of errors not handled, would users even notice?
It depends if they see the output.
Why do some errors cause large problems while others don't?
Some errors are more serious than others. Ideally you only want serious error or none. Errors which are not too serious are more likely to have subtle problems which are harder/less likely to get fixed.
On very large projects involving tens of thousands of lines of code do many programmers dismiss these types of errors that do not affect logic or run-time and conclude that to go back and fix all of them would not be necessary or worth the time?
Its best not to handle or catch an exception unless you are going to do something useful with it. Otherwise it is better to re-throw it provided it will always get logged at the very least.
ones that allow the program to function as if nothing has happened would be greatly appreciated
FileNotFoundException, EOFException, NumberFormatException are common exceptions which are handled in code (but rarely just ignored) Google for examples.
Its almost always a bad idea to pretend an exception didn't occur. If you don't care if something worked or not, you usually don't need it in the first place.
There can be runtime errors that don't affect the execution of a program but I would strongly urge you to handle or fix the cause of every error you possibly can.
If you don't fix or handle runtime errors your logs (if you have any!) are going to be full of them and you won't be able to see the error you actually should care about from all the noise.
I've run into the issue where I have a program (not written by me, by someone else) I want to run 24/7, but sometimes it crashes. Normally, this wouldn't be an issue because I can simply create a process watcher that checks if it crashed, and then restarts it if necessary.
But, this particular program sometimes throws an exception and outputs it into the graphical interface that's integrated into it. In this instance, the program doesn't crash at all. The interface stays up, but the actual server functionality is unavailable.
Is there any way I can intercept this information from this process?
You want to use the Java Virtual Machine Tools Interface. I can't give you the code to catch your exception, but this is where to look. You'll have to do some detective work to find the class that throws the exception, or at least to find some indicator that it has been thrown.
Edit: You can also try calling the vendor to see if they know of a way. You can also look to see if it is writing the exception to a log file, which you could then watch.
This may or may not work, but if when the application displays it's error and the server stops working does the memory usage drop? If so you could probably just add some logic to your process monitor to call the windows command tasklist to see if the memory usage drops below some threshold. You'll have to check how much memory the program normally uses and how much it uses after the error though.
Since you said the server functionality stops working, another option could be to write a simple program that basically just pings the server how ever often you want to make sure it is still up. If not, kill the process and restart it.
I assume you have no access to the source code, so if it is outputting to the GUI the answer is no. Even if you could attach to the running process you would need to intercept the exception, but it is caught and sent to the GUI, not thrown from the application.
In theory, you could screen scrape the application. I don't know of any specific tools for doing this, but they may be out there.
Edit: I may have been wrong above, check out a post here where they get the stack from a running thread. You probably won't be able to capture the exception this way, but if you're lucky the stack trace will look very different when the program is operating normally compared to when an exception has been thrown.
Edit 2: I submitted a second, more accurate answer. See below.
Is the other program Java? Look at AspectJ, you may be able to hack something using it if you have control on the program startup.
Without ability to rebuild the app you are generally out of luck unless you do some extensive hacking. Here is one option I can think of.
Most likely the application replaces System.out and/or System.err with its own stream implementation. If that's the case you can try to locate the class for this stream and replace it with your own wrapper with the same name. You may rename original class using jarjar. In the wapper you can provide console output to detect the exception.
Following on from this question https://softwareengineering.stackexchange.com/questions/37294/logging-why-and-what
I was wondering what actually happens to an error that occurs during the runtime of a Java Enterprise Edition applicaiton.
Does the JVM store a log of all the errors?
Or are the errors forgotten?
It is contingent where the output is directed. If output is getting pushed to a console window then yes...it is all but lost. An enterprise application however would be making use of a logging framework to deal with all output thus rendering any exception available within the logs provided by the framework.
Application servers usually have a big catch-all net to mop up any unhandled exceptions. However, if an exception is allowed to bubble up without it ever hitting a catch close, the thread it came from will die and the exception will be passed to the thread's UncaughtExceptionHandler, if one exists.
an error happens when the program is running. and is being handled by exceptions.
Exceptions : ignore the error, handel the exception, and go back to method that was called.
What is the best practice when dealing with Errors within a server application?
In particular, how do you think an application should handle errors like OutOfMemoryError?
I'm particularly interested in Java applications running within Tomcat, but I think that is a more general problem.
The reason I'm asking is because I am reviewing a web application that frequently throws OOME, but usually it just logs them and then proceeds with execution. That results, obviously, in more OOMEs.
While that is certainly bad practice, in my opinion, I'm not entirely sure that stopping the Server would be the best solution.
There is not much you can do to fix OutOfMemoryError except to clean up the code and adjust JVM memory (but if you have a leak somewhere it's just a bandaid)
If you don't have access to the source code and/or are not willing to fix it, an external solution is to use some sort of watch dog program that will monitor java application and restart it automatically when it detects OOMEs. Here is a link to one such program.
Of course it assumes that the application will survive restarts.
The application shouldn't handle OOM at all - that should be the server's responsibility.
Next step: Check if memory settings are appropriate. If they aren't, fix them; if they are, fix the application. :)
Well, if you have OOME then the best way would be to release as many resources (especially cached ones) as possible. Restarting the web-app (in case it's web-apps fault) or the web server itself (in case something else in the server does this) would do for recovering from this state. On the development front though it'd be good to profile the app and see what is taking up the space, sometimes there are resources that are attached to a class variable and hence not collected, sometimes something else. In the past we had problems where Tomcat wouldn't release the classes of previous versions of the same app when you replace the app with a newer version. Somewhat solved the problem by nullifying class variables or re-factoring not to use them at all but some leaks still remained.
An OutOfMemoryError is by no means always unrecoverable - it may well be the result of a single bad request, and depending on the app's structure it may just abandon processing the request and continue processing others without any problems.
So if your architecture supports it, catch the Error at a point where you have a chance to stop doing what caused it and continue doing something else - for an app server, this would be at the point that dispatches requests to individual app instances.
Of course, you should also make sure that this does not go unnoticed and a real fix can be implemented as soon as possible, so the app should log the error AND send out some sort of warning (e.g. email, but preferably something harder to ignore or get lost). If something goes wrong during that, then shutting down is the only sensible thing left to do.
#Michael Borgwardt, You can't recover from an OutOfMemoryError in Java. For other errors, it might not stop the application, but OutOfMemoryError literally hangs applications.
In our application which deals with Documents heavily, we do catch OOM errors where one bad request can result in OOM but we dont want to bring down the application because of this. We catch OOM and log it.
Not sure if this is best practice but seems like its working
I'm not an expert in such things, but I'll take a chance to give my vague opinion on this problem.
Generally, I think that there's two main ways:
Server is stopped.
Resources are thus gracefully degrading throughput, reducing memory consumption, but staying alive. For this case application must have appropriate architecture, I think.
According to the javadoc about a java.lang.Error:
An Error is a subclass of Throwable that indicates serious problems that a reasonable application should not try to catch. Most such errors are abnormal conditions. The ThreadDeath error, though a "normal" condition, is also a subclass of Error because most applications should not try to catch it.
A method is not required to declare in its throws clause any subclasses of Error that might be thrown during the execution of the method but not caught, since these errors are abnormal conditions that should never occur.
So, the best practice when dealing with subclasses of Error is to fix the problem that is causing them, not to "handle" them. As it's clearly stated, they should never occur.
In the case of an OutOfMemoryError, maybe you have a process that consumes lots of memory (e.g. generating reports) and your JVM is not well sized, maybe you have a memory leak somewhere in your application, etc. Whatever it is, find the problem and fix it, don't handle it.
I strongly disagree with the notion that you should never handle an OutOfMemoryError.
Yes, it tends to be unrecoverable most of the time. However, one of my servers got one a few days ago and the server was still mostly working for more than an hour and a half. Nobody complained so I didn't notice until my monitoring software got a failure and hour and a half after the first OutOfMemoryError. I need to know as soon as possible when there is an OutOfMemoryError on my server. I need to handle it so that I can set up a notification so that I can know to restart my server ASAP.
I'm still trying to figure out how to get Tomcat to do something when it gets an Error. error-page doesn't seem to be working for it.
I'm trying to debug a problem in my Java application that throws no errors, no exceptions and doesn't even crash the app (it seems the failure happens in a separate thread).
The problem seems to be inside a call to a library function (it's JAXBContext.newInstance(String) if that matters). The program will reach the line just before the call, but not the one just after it. My catch blocks are not entered and the program just continues to run.
The problem happens while trying to render an XML response to a web request that came in via Struts. The request has been handled and the code should marshal the response object. The client gets a response right away (so the code doesn't seem to hang in a loop), but it's just empty.
I have set a breakpoint just before the problematic line but the debugger just runs over it, I haven't a clue why.
I'm using eclipse and the application runs inside an OSGi container (Apache Felix) that was started with -Xdebug -Xrunjdwp:transport=dt_socket,address=8000,server=y,suspend=y. From within Eclipse I then use the Debug settings for "Remote Java application" to connect the debugger.
What are techniques to get at such a problem?
Probably an obvious question, but are you sure you are catching Throwable? An unchecked exception could easily cause the thread in question to die (assuming no one above you in the call stack is catching it either.)
Since you are suspending the VM on startup with your debug arguments, I assume you have confirmed that the debugger is attaching correctly. The fact that you say the debugger skips right past the call is very suspect. Are you able to hit any breakpoints in this application? What about in this Class? What about in this thread?
How did you narrow down the line in question without the debugger? println/debugging to a file?
Can you paste a code snippet of the method in question?
You could confirm the theory that the thread is dying by creating a second thread before the problem occurs and joining it to the thread you think is dying. Then the second thread's run() method would be invoked when the thread in question exits, and you'd know it died (but would still not know why.)
In answer to your general question, when I have a bug in a Java app that I can't reproduce in the debugger (which happens from time to time for various reasons), I incrementally modify my code with sysout printlns or output to files. If necessary, I may also modify the code my code is invoking. If you don't have the source code to the code you are invoking, you can try one of the many BCI frameworks to inject your byte code into the methods in question. It's a tedious process, but only happens occasionally.
You could try getting a Thread Dump - that will tell you if any methods are blocking (e.g. waiting for input). [Edit: re-reading your original question, getting a thread dump probably won't help as it looks like nothing is actually blocking. But I'm leaving it here as I find it useful in many other situations!]
If you think the error is happening in another thread you could also set an UncaughtExceptionHandler to try and catch it.
If you're sure the problem is somewhere within that method, you could try looking at the JAXB source code.
EDIT:
Well, if it gets really bad you can build your own private copy with debugging instrumentation. I hope you won't have to resort to that.
perhaps inside the call there is an infitite loop happening and this is why you get no further - but this might not cause a crash (unless memory is being used in each loop).