I want to activate graceful shutdown on an existing Spring Boot (v2.3.9) app.
Initially, to test the feature out, I created a sample app and added the property
server.shutdown: graceful in the application.properties. I could see the server shutting down gracefully in the logs when I kill the app.
Satisfied with the output, I added the same property to the existing app. However, when I kill that particular app, the shutdown doesn't happen gracefully. No logs like above are produced. This made me wonder whether the property was actually getting set so just to double down I also set it as a run-time argument -Dserver.shutdown=graceful. It didn't work even after that.
I have confirmed the Spring Boot/Tomcat versions running for the existing app and they are above the minimum required to enable this property.
The existing application has a convoluted logging structure with a variety of libraries (log4j, logback etc) in the mix. Could it be the case that the graceful shutdown is happening but due to a higher logging level, the logs don't show up? If that's the case, which property should I set/override to enable them.
It was a logging issue indeed. A logging property deep inside the legacy code was overriding the outer log level. The shutdown works just fine.
Related
I'm trying to debug a Java application in Kubernetes using a Cloud Code plugin.
There is no trouble with the default debug.
I just click debug and it works, but... I don't know how to connect to application on the start.
I've tried to add option -agentlib:jdwp=transport=dt_socket,server=n,suspend=**y**,address=,quiet=y
but JVM crushed because Cloud Code adds its own option agentlib and JVM can't handle two options with the same name.
How can I edit the agentlib option for Cloud Code? (to add suspend=y) or maybe disable that option.
Or maybe there is another way to debug the application while it starts?
I've tried to add agentlib option to JDK_JAVA_OPTIONS, but scaffold(library inside cloud plugin) try to find agentlib in JAVA_TOOL_OPTIONS
I've put the option in the right place and it works well
Adding this as an answer to provide additional context.
Skaffold doesn't currently support this feature. There is an open feature request on Skaffold to add this ability.
Adding support for this has not been a high-priority item for Skaffold as suspending on startup often causes puzzling problem cascades as startup, readiness, and liveness probes time out, leading to pod restarts, and then debug sessions being terminated, and then new sessions established. And container startup problems are often more easily debugged in isolation (e.g., running as a normal Java app and emulating the Kubernetes startup through env vars, sending messages, etc).
All that said, Skaffold should respect existing -agentlib settings passed on the command-line or in JAVA_TOOL_OPTIONS. So as you found, you can pass along your own JDWP setting in your JAVA_TOOL_OPTIONS.
I have a spring boot application with a pretty standard logback configuration. I know I can turn on/off the verbose logging by changing the logging level property to only log errors (<root level = "ERROR">). And I have configured my app to only log errors in the application-insights azure resource that I have, so as to avoid verbose logging till it's necessary. All of this is working fine.
However, as soon as I get an error, I want to start verbose logging for a certain period of time let's say 2hrs, so as to help with the troubleshooting. And this is what I am unable to achieve. I am pretty new to Java as well as Azure resource configurations, so any idea will be really helpful.
Appreciate the answer and comment on the question, they will prove to be a good resource for whoever comes across this question in future. However, a more pointed answer to this situation is needed. I was able to achieve the conditional, time based, verbose logging for my application by the following approach:
Create an alert in azure portal pointing to the azure resource in use.
Configure that alert's action group to call a controller action method, selecting webhook as the action type.
Inside your controller action method, change the logging level setting to verbose logging for desired period of time, and reset it back to non-verbose after that. (I used non blocking Thread.sleep() for this)
Enjoy the automation :)
I read a lot but I couldn't figure out how I could specify for example the log level for specific classes.
Only way I could figure out was in the standalone.xml but why should I configure some application specific setting very general in the server? This complicates the deployment process unnecessary.
Isn't it somehow possible to define the specific log level and the output files somewhere inside the war without touching the server?
Btw. it doesn't matter if log4j or commons-logging or slf4j or whatever is used.
Using a logging.properties file or a log4j configuration file in your deployment will work in JBoss EAP 6.x and WildFly (formerly JBoss AS). Note though that a log4j configuration would only work if you use log4j for your logging facade.
That said I agree with Marko that this should probably be done in the server configuration. I would also encourage you to use the CLI or web interface rather than editing the raw XML as well. You can get some more information on the documentation.
I am sorry for not providing a direct answer, but consider this: the application being in charge of logging levels is a bad idea most of the time as this is something an AS admin should be able to change at any time. For example, the point of the DEBUG or TRACE log levels is to be able to place a lot of such statements in the code without hurting the production server's performance. However, once a bug is detected, you want to be able to lower the logging level without rebuilding the application. This should be a purely administrative task.
On the other hand, I do recognize the need to at least have a decent starting point for the logging configuration and I don't know of any architecture which would allow the application to provide defaults which are overridable by the server configuration.
I am running a JAX-RS application that needs to do some clean up once the application is shut down. I installed a shutdown hook for this purpose and it works fine. However, in unit tests, the shut down hook is (of course) not triggered since the JVM is not shut down in between two calls. This I want to fix and only use the shut down hook as a last resort. I imagine, the same would happen if I only undeploy the application from a servlet container without explicitly triggering a shut down. Also, I imagine that I might have created a memory leak in the case the application container continues running with my shut down hook still registered?
The question: Is there a possibility to listen for the server / servlet responsible for my JAX-RS application to be destroyed? I was thinking about something like the Servlet#destroy method.
I am looking for something like: new ResourceConfig().register(new MyShutDownListener()) or something I can do in a ShutDownBinder? Any ideas?
Well, I found the org.glassfish.jersey.server.spi.ContainerLifecycleListener interface. It seems as if this interface found its way into Jersey in v2.3 while my built was using v2.1. I now upgraded Jersey and use it happily. (This is however a Jersey interface, it does not have a sibling in JAX-RS.)
If this is not an option for a future reader of this answer, I found another solution with using GrizzlyWebContainerFactory which allows me to use define my own servlet for starting up the application. This solution was however way more verbose.
Currently we have J2EE applications that run on WebSphere and log4j 1.x is our company's standard for Java logging. We'd like to investigate the ability to change the log4j configuration xml file at runtime so we can change to the DEBUG level, for example, when problems are identified, and return to the ERROR level once those problems are cleared up and DEBUG is no longer needed. We'd like to do all this without having to recycle the application to reread the config file.
I see that DOMConfigurator.configureandWatch() is an available method that would allow the config file to be reread at runtime. However, because it spawns a separate thread that won't be shut down when the application is shut down, it is unsafe to run in a J2EE environment.
http://logging.apache.org/log4j/1.2/faq.html#a3.6
Does anyone know if this was fixed in log4 2? I haven't found anything definitive yet from the documentation and was curious if it is safe to change the config file at runtime in J2EE.
Re-reading the configuration file is not something you want to do in production. It means that you're changing resources which many app servers don't expect to change. Development environments can get away with it on the grounds that blowing up the app server isn't a big deal. It also means that your app might behave differently if it has to be restarted.
You're going to want to change the log levels programmatically. You can do this by using LogManager.getLogger() for the packages you care about. Once you have the logger, change the level to debug, and change it back when you are done.