I am trying to come up with a microservice using dropwizard.
The documentation tells how to start the application, but says nothing about terminating it gracefully. Fir example, apache tomcat has both startup and shutdown scripts.
So does anyone know how to terminate a dropwizard application other than pressing Ctrl+C of kill ?
Dropwizard Jetty has shutdown hooks. So kill -SIGINT <pid> works out really well.
Inspired by praveenag's answer, I dug into Jetty's code.
If you start DropWizard providing:
-DSTOP.PORT=xxxx -DSTOP.KEY=my_secret_key
as Java options,
It tells Jetty to listen on that port for a stop request.
You can then write to that socket to tell Jetty to shutdown. I did this from R like this:
socket = make.socket("localhost", 8082)
write.socket(socket, "my_secret_key\r\n")
write.socket(socket, "stop\r\n")
close.socket(socket)
I guess you can do the same from any other language.
The other answers here are great, but if you want to go a bit further up the stack and easily add custom logging / security / arbitrarily complex shutdown logic, then adding a shutdown hook via a dropwizard admin Task is a nice pattern.
Create a shutdown Task with any logic you like
import io.dropwizard.servlets.tasks.Task;
public class ShutdownTask extends Task {
public ShutdownTask() {
super("shutdown"); // the task name, used in the endpoint to execute it
}
public void execute(
ImmutableMultimap<String, String> immutableMultimap,
PrintWriter printWriter
) throws Exception {
// kill the process asynchronously with some nominal delay
// to allow the task http response to be sent
new Timer().schedule(new TimerTask() {
public void run() {
// any custom logging / logic here prior to shutdown
System.exit(0);
}
}, 5000);
}
}
Register the task in Application.run()
environment.admin().addTask(new ShutdownTask());
And then execute it via a POST to the following endpoint on the admin port
http://localhost:<ADMIN PORT>/tasks/shutdown
I am assuming this is not a question for your development environment but for your deployments. The answer depends on your deployment strategy. In the past we have handled deployments where the drop wizard application is bundled as a java process that can be started and the pid being recorded and forcefully kill the process. Or bundle the java process in an upstart/init script to gracefully start and shutdown the system.
On the other hand when you start a dropwizard application what it eventually does is start a jetty server. http://eureka.ykyuen.info/2010/07/26/jetty-stop-a-jetty-server-by-command. This can maybe shed some light on how you can pass the stop port as arguments when you start the dropwizard application.
Programmatically, you can also do:
environment.getApplicationContext().getServer().stop();
That's the environment you get in your Application:
#Override
public void run(ApplicationConfiguration configuration, Environment environment) throws Exception { ... }
In Java:
// In case vm shutdown
Runtime.getRuntime().addShutdownHook(new Thread() {
#Override
public void run()
{
// what should be closed if forced shudown
// ....
LOG.info(String.format("--- End of ShutDownHook (%s) ---", "APPLICATION_NAME"));
}
});
Build your own strategy, how to shutdown your app.
Related
I have a product service in Java. In our code I am creating shut down hook, but when I stop service it is not calling shut down hook consistently. Out of 5 stop calls it has called shutdown hook only once.
Runnable shutdownHandler = new Runnable() {
#Override
public void run() {
s_log.info("Shutting down thread..");
}
};
Runtime.getRuntime().addShutdownHook(
new Thread(shutdownHandler, "shutdownthread"));
Can anybody please tell me what could be the reason behind this not getting called consistently?
Check the following code:
Runnable shutdownHandler = new Runnable() {
#Override
public void run() {
System.out.println("Shutting down thread..");
}
};
Runtime.getRuntime().addShutdownHook(
new Thread(shutdownHandler, "shutdownthread"));
and if it gives you expected output, you need to check the documentation of your logging framework.
I am also finding that my framework (Jooby) and Java shutdown hooks work fine on my Mac on IntelliJ which sends a kill SIGINT (-2) however on Ubuntu Server 20.04 LTS they don't run.
As my Java app is a webapp I came up with a simple workaround:
Setup a controller to listen to some url that isn't easily guessable e.g.
/exit/fuuzfhuaBFDUWYEGLI823y82941u9y47t3u45
Have the controller simply do the following:
System.exit(0)
Do a curl or wget from a script to the URL and the shutdown hooks all fire as JVM comes down.
I suspect for some reason on Linux there is a bug and no matter what interrupt that I use besides SIGKILL they all effectively behave like SIGKILL and the JVM comes down hard/abruptly.
I use JBoss 7.1.1.Final. Here I have a startup singleton. At startup I initialize something, at shutdown I terminate/cleanup my stuff.
But now I found out, that the termination I can do whatever I want for 1 second! After one second the application is just away, and sometimes 1 second is not enough time for a real cleanup.
Code:
#Singleton
#Startup
public class ShutdownTest {
#PostConstruct
public void initialize() {
LOG.info("Initialization");
}
#PreDestroy
public void terminate() {
for(;;) {
LOG.info("loop in terminate...");
Tools.sleepQuietly(100); // just sleeps for 100 milliseconds
}
}
}
This class does report the initalization at startup, but at termination I get 10 times the output, after this, the JBoss is dead.
How can I configure this time until JBoss kills itself even if some PreDestroy methods are still running?
Currently I start and stop JBoss from Eclipse (Poller are set to Web Port).
There is no way to do this on JBoss 7 at the shutdown event (based on this answer of Tomaz Cerar from the JBoss / WildFly team).
On windows I can confirm that there is no way to do this (and I assume so for Linux).
I found it works if you halt the application before shutting the server down. That is: log on to the webconsole, navigate to Runtime (Tab on the upper right) -> Manage Deployments (menu) -> Button "Disable" for your application.
In WildFly 8 you get the "timeout" option for the shutdown. See this post on how to use it.
I am trying to get my Java program to exit gracefully on my unix server. I have a jar file, which I start through a cron job in the morning. Then in the evening, when I want to shut it down, I have a cron job which calls a script that finds the PID and calls kill -9 <PID>. However, it doesn't seem that my shutdown hook is activated when I terminate this way. I also tried kill <PID> (no -9) and I get the same problem. How can I make sure the shutdown hook gets called? Alternatively, perhaps there is a better way to kill my process daily.
class ShutdownHook {
ShutdownHook() {}
public void attachShutDownHook() {
Runtime.getRuntime().addShutdownHook(new Thread() {
#Override
public void run() {
System.out.println("Shut down hook activating");
}
});
System.out.println("Shut Down Hook Attached.");
}
}
You can use code like this on Unix to trap SIGINT (#2) signal:
Signal.handle(new Signal("INT"), new SignalHandler() {
public void handle(Signal sig) {
// Forced exit
System.exit(1);
}
});
kill -9 <pid> sends a KILL signal. This signal cannot be intercepted by the program.
If you call kill <pid>, the TERM signal (15) wil be sent. In that case, the JVM will catch the signal and the shutdown hooks will be executed.
This has nothing to do with the signals the JVM is trapping/receiving but everything to do with the terrible shutdown process of Gnome, which apparently needs to be cooperative not to absolutely shit the bed (and the jdk doesn't have the api for this). If you want to see a even worse consequence of this, try to run:
dbus-monitor --profile --session type='method_call',interface='org.gnome.SessionManager'
on a shell, and logout or restart: it will crash gnome-shell and hang the computer until you login on a TTY and order a restart.
Maybe kdbus will fix this on this case, maybe not. The only thing i know is that shutdownhooks on a java application that is using AWT (not command line) will NEVER run its shutdownhooks on GNOME3. Actually, the VM will always exit with non-zero code (failure) presumably from native code. At least it doesn't hang, although this makes shutdown hooks quite useless
(i've been trying to make a workaround by using dbus-monitor, but as you can see from the example i gave, it's a bit too dangerous too).
How to programmatically shutdown embedded jetty server?
I start jetty server like this:
Server server = new Server(8090);
...
server.start();
server.join();
Now, I want to shut it down from a request, such as http://127.0.0.1:8090/shutdown
How do I do it cleanly?
The commonly proposed solution is to create a thread and call server.stop() from this thread.
But I possibly need a call to Thread.sleep() to ensure that the servlet has finished processing the shutdown request.
I found a very clean neat method here
The magic code snippet is:-
server.setStopTimeout(10000L);;
try {
new Thread() {
#Override
public void run() {
try {
context.stop();
server.stop();
} catch (Exception ex) {
System.out.println("Failed to stop Jetty");
}
}
}.start();
Because the shutdown is running from a separate thread, it does not trip up over itself.
Try server.setGracefulShutdown(stands_for_milliseconds);.
I think it's similar to thread.join(stands_for_milliseconds);.
Having the ability for a Jetty server to be shutdown remotely through a HTTP request is not recommended as it provides as potential security threat. In most cases it should be sufficient to SSH to the hosting server and run an appropriate command there to shutdown a respective instance of a Jetty server.
The basic idea is to start a separate thread as part of Jetty startup code (so there is no need to sleep as required in one of mentioned in the comment answers) that would serve as a service thread to handle shutdown requests. In this thread, a ServerSocket could be bound to localhost and a designated port, and when an expected message is received it would call server.stop().
This blog post provides a detailed discussion using the above approach.
How do I stop a Java process gracefully in Linux and Windows?
When does Runtime.getRuntime().addShutdownHook get called, and when does it not?
What about finalizers, do they help here?
Can I send some sort of signal to a Java process from a shell?
I am looking for preferably portable solutions.
Shutdown hooks execute in all cases where the VM is not forcibly killed. So, if you were to issue a "standard" kill (SIGTERM from a kill command) then they will execute. Similarly, they will execute after calling System.exit(int).
However a hard kill (kill -9 or kill -SIGKILL) then they won't execute. Similarly (and obviously) they won't execute if you pull the power from the computer, drop it into a vat of boiling lava, or beat the CPU into pieces with a sledgehammer. You probably already knew that, though.
Finalizers really should run as well, but it's best not to rely on that for shutdown cleanup, but rather rely on your shutdown hooks to stop things cleanly. And, as always, be careful with deadlocks (I've seen far too many shutdown hooks hang the entire process)!
Ok, after all the possibilities I have chosen to work with "Java Monitoring and Management"
Overview is here
That allows you to control one application from another one in relatively easy way. You can call the controlling application from a script to stop controlled application gracefully before killing it.
Here is the simplified code:
Controlled application:
run it with the folowing VM parameters:
-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.port=9999
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false
//ThreadMonitorMBean.java
public interface ThreadMonitorMBean
{
String getName();
void start();
void stop();
boolean isRunning();
}
// ThreadMonitor.java
public class ThreadMonitor implements ThreadMonitorMBean
{
private Thread m_thrd = null;
public ThreadMonitor(Thread thrd)
{
m_thrd = thrd;
}
#Override
public String getName()
{
return "JMX Controlled App";
}
#Override
public void start()
{
// TODO: start application here
System.out.println("remote start called");
}
#Override
public void stop()
{
// TODO: stop application here
System.out.println("remote stop called");
m_thrd.interrupt();
}
public boolean isRunning()
{
return Thread.currentThread().isAlive();
}
public static void main(String[] args)
{
try
{
System.out.println("JMX started");
ThreadMonitorMBean monitor = new ThreadMonitor(Thread.currentThread());
MBeanServer server = ManagementFactory.getPlatformMBeanServer();
ObjectName name = new ObjectName("com.example:type=ThreadMonitor");
server.registerMBean(monitor, name);
while(!Thread.interrupted())
{
// loop until interrupted
System.out.println(".");
try
{
Thread.sleep(1000);
}
catch(InterruptedException ex)
{
Thread.currentThread().interrupt();
}
}
}
catch(Exception e)
{
e.printStackTrace();
}
finally
{
// TODO: some final clean up could be here also
System.out.println("JMX stopped");
}
}
}
Controlling application:
run it with the stop or start as the command line argument
public class ThreadMonitorConsole
{
public static void main(String[] args)
{
try
{
// connecting to JMX
System.out.println("Connect to JMX service.");
JMXServiceURL url = new JMXServiceURL("service:jmx:rmi:///jndi/rmi://:9999/jmxrmi");
JMXConnector jmxc = JMXConnectorFactory.connect(url, null);
MBeanServerConnection mbsc = jmxc.getMBeanServerConnection();
// Construct proxy for the the MBean object
ObjectName mbeanName = new ObjectName("com.example:type=ThreadMonitor");
ThreadMonitorMBean mbeanProxy = JMX.newMBeanProxy(mbsc, mbeanName, ThreadMonitorMBean.class, true);
System.out.println("Connected to: "+mbeanProxy.getName()+", the app is "+(mbeanProxy.isRunning() ? "" : "not ")+"running");
// parse command line arguments
if(args[0].equalsIgnoreCase("start"))
{
System.out.println("Invoke \"start\" method");
mbeanProxy.start();
}
else if(args[0].equalsIgnoreCase("stop"))
{
System.out.println("Invoke \"stop\" method");
mbeanProxy.stop();
}
// clean up and exit
jmxc.close();
System.out.println("Done.");
}
catch(Exception e)
{
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
That's it. :-)
An another way: your application can open a server socet and wait for an information arrived to it. For example a string with a "magic" word :) and then react to make shutdown: System.exit(). You can send such information to the socke using an external application like telnet.
Here is a bit tricky, but portable solution:
In your application implement a shutdown hook
When you want to shut down your JVM gracefully, install a Java Agent that calls System.exit() using the Attach API.
I implemented the Java Agent. It is available on Github: https://github.com/everit-org/javaagent-shutdown
Detailed description about the solution is available here: https://everitorg.wordpress.com/2016/06/15/shutting-down-a-jvm-process/
Similar Question Here
Finalizers in Java are bad. They add a lot of overhead to garbage collection. Avoid them whenever possible.
The shutdownHook will only get called when the VM is shutting down. I think it very well may do what you want.
Thanks for you answers. Shutdown hooks seams like something that would work in my case.
But I also bumped into the thing called Monitoring and Management beans:
http://java.sun.com/j2se/1.5.0/docs/guide/management/overview.html
That gives some nice possibilities, for remote monitoring, and manipulation of the java process. (Was introduced in Java 5)
Signalling in Linux can be done with "kill" (man kill for the available signals), you'd need the process ID to do that. (ps ax | grep java) or something like that, or store the process id when the process gets created (this is used in most linux startup files, see /etc/init.d)
Portable signalling can be done by integrating a SocketServer in your java application. It's not that difficult and gives you the freedom to send any command you want.
If you meant finally clauses in stead of finalizers; they do not get extecuted when System.exit() is called.
Finalizers should work, but shouldn't really do anything more significant but print a debug statement. They're dangerous.