Develop Reference

Kafka Streams Shutdown Hook and Unexpected Exception Handling in the same Stream application

I was tasked with tearing down a Dev environment and setting it up again from scrap to verify our CI-CD processes; only problem was that I messed up creating one topic and so the Kafka Streams application exited with an error.
I dug into it and found the issue and corrected it but as I was digging in I ran into another odd little problem.
I implemented a Unexpected Exception handler as so:
streams.setUncaughtExceptionHandler((t, e) -> {
logger.fatal("Caught unhandled Kafka Streams Exception:", e);
// Do some exception handling.
streams.close();
// Maybe do some more exception handling.
// Open a lock that is waiting after streams.start() call
// to let application exit normally
shutdownLatch.countDown();
});
Problem is that if the application threw an exception because of a topic error when the KafkaStreams::close is call the application seems to dead lock in WindowsSelectorImpl::poll after attempting a call to KafkaStreams::waitOnState.
I thought it might be an issue with calling KafkaStreams::close inside the Exception Handler but I found this SO and a comment from Matthias J. Sax that said it should be ok to call KafkaStreams::Close in the exception handler with the caveat of not calling KafkaStreams::close from multiple threads.
The issue is that I want to implement a shutdown hook to kill the steams application gracefully on request, as well as implement the UnexpectedException handler to clean up and terminate gracefully in the event of exceptions.
I came up with the following solution that checks the KafkaStreams state before calling close and it does actually work, but it seems a bit iffy since I could see other cases besides running (perhaps Pending) where we would want to ensure the KafkaStreams::close it called.
Runtime.getRuntime().addShutdownHook(new Thread(() -> {
logger.fatal("Caught Shutdown request");
// Do some shutdown cleanup.
if (streams.state().isRunning())
{
If this hook is called due to the Main exiting after handling
an exception we don't want to call close again. It doesn't
cause any errors but logs that the application was closed
a second time.
streams.close(100L, TimeUnit.MILLISECONDS);
}
// Maybe do a little bit more clean up before system exits.
System.exit(0);
}));
streams.setUncaughtExceptionHandler((t, e) -> {
logger.fatal("Caught unhandled Kafka Streams Exception:", e);
// Do some exception handling.
if (streams.state().isRunning())
{
streams.close(100L, TimeUnit.MILLISECONDS);
}
// Maybe do some more exception handling.
// Open the Gate to let application exit normally
shutdownLatch.countDown();
// Or Optionally call halt to immediately terminate and prevent call to Shutdown hook.
Runtime.getRuntime().halt(0);
});
Any suggestions about why calling the KafkaSteams:close in the exception handler would be causing such troubles or if there is a better way to implement shutdown hook and the exception handler at the same time it would be greatly appreciated?

Calling close() from an exception handler and from shutdown hook is slightly different. close() might deadlock if called from the shutdown hook (cf. https://issues.apache.org/jira/browse/KAFKA-4366) and thus, you should call it with a timeout.
Also, the issue is related with calling System.exit() from within an uncaught exception handler as described in the Jira. In general, calling System.exit() is quite harsh and should be avoided IMHO.
Your solution seems not to be 100% robust, either, because streams.state().isRunning() could result in a race condition.
An alternative to using a timeout might be to only set an AtomicBoolean within both the shutdown hook and the exception handler and use the "main()" thread to call close if the boolean flag is set to true:
private final static AtomicBoolean stopStreams = new AtomicBoolean(false);
public static void main(String[] args) {
// do stuff
KafkaStreams streams = ...
stream.setUncaughtExceptionHandler((t, e) -> {
stopStreams.set(true);
});
Runtime.getRuntime().addShutdownHook(new Thread(() -> {
stopStreams.set(true);
});
while (!stopStreams.get()) {
Thread.sleep(1000);
}
streams.close();
}

Java shutdown hook not running

I have a product service in Java. In our code I am creating shut down hook, but when I stop service it is not calling shut down hook consistently. Out of 5 stop calls it has called shutdown hook only once.
Runnable shutdownHandler = new Runnable() {
#Override
public void run() {
s_log.info("Shutting down thread..");
}
};
Runtime.getRuntime().addShutdownHook(
new Thread(shutdownHandler, "shutdownthread"));
Can anybody please tell me what could be the reason behind this not getting called consistently?

Check the following code:
Runnable shutdownHandler = new Runnable() {
#Override
public void run() {
System.out.println("Shutting down thread..");
}
};
Runtime.getRuntime().addShutdownHook(
new Thread(shutdownHandler, "shutdownthread"));
and if it gives you expected output, you need to check the documentation of your logging framework.

I am also finding that my framework (Jooby) and Java shutdown hooks work fine on my Mac on IntelliJ which sends a kill SIGINT (-2) however on Ubuntu Server 20.04 LTS they don't run.
As my Java app is a webapp I came up with a simple workaround:
Setup a controller to listen to some url that isn't easily guessable e.g.
/exit/fuuzfhuaBFDUWYEGLI823y82941u9y47t3u45
Have the controller simply do the following:
System.exit(0)
Do a curl or wget from a script to the URL and the shutdown hooks all fire as JVM comes down.
I suspect for some reason on Linux there is a bug and no matter what interrupt that I use besides SIGKILL they all effectively behave like SIGKILL and the JVM comes down hard/abruptly.

Schedule a executable file without Windows task scheduler

Is it possible to schedule a executable file to run just before log-off using java?
I am working on a application that needed to send a message to the sever just before the system log-off/shutdown. But I didn't find any method to solve it. If anyone know plz help me.
Thanks in advnce.

As stated in the comments a ShutdownHook is probably what you are searching for.
Short example:
Runtime.getRuntime().addShutdownHook(new Thread() {
#Override
public void run() {
// Code to be executed when JVM exits
System.out.println("JVM exit");
}
});
Note that the ShutdownHook will execute whenever the JVM exits. So it will also execute when your program terminates normally.
If you have any further questions comment on this answer.

Shutdown hook from UNIX

I am trying to get my Java program to exit gracefully on my unix server. I have a jar file, which I start through a cron job in the morning. Then in the evening, when I want to shut it down, I have a cron job which calls a script that finds the PID and calls kill -9 <PID>. However, it doesn't seem that my shutdown hook is activated when I terminate this way. I also tried kill <PID> (no -9) and I get the same problem. How can I make sure the shutdown hook gets called? Alternatively, perhaps there is a better way to kill my process daily.
class ShutdownHook {
ShutdownHook() {}
public void attachShutDownHook() {
Runtime.getRuntime().addShutdownHook(new Thread() {
#Override
public void run() {
System.out.println("Shut down hook activating");
}
});
System.out.println("Shut Down Hook Attached.");
}
}

You can use code like this on Unix to trap SIGINT (#2) signal:
Signal.handle(new Signal("INT"), new SignalHandler() {
public void handle(Signal sig) {
// Forced exit
System.exit(1);
}
});

kill -9 <pid> sends a KILL signal. This signal cannot be intercepted by the program.
If you call kill <pid>, the TERM signal (15) wil be sent. In that case, the JVM will catch the signal and the shutdown hooks will be executed.

This has nothing to do with the signals the JVM is trapping/receiving but everything to do with the terrible shutdown process of Gnome, which apparently needs to be cooperative not to absolutely shit the bed (and the jdk doesn't have the api for this). If you want to see a even worse consequence of this, try to run:
dbus-monitor --profile --session type='method_call',interface='org.gnome.SessionManager'
on a shell, and logout or restart: it will crash gnome-shell and hang the computer until you login on a TTY and order a restart.
Maybe kdbus will fix this on this case, maybe not. The only thing i know is that shutdownhooks on a java application that is using AWT (not command line) will NEVER run its shutdownhooks on GNOME3. Actually, the VM will always exit with non-zero code (failure) presumably from native code. At least it doesn't hang, although this makes shutdown hooks quite useless
(i've been trying to make a workaround by using dbus-monitor, but as you can see from the example i gave, it's a bit too dangerous too).

Eclipse/Java: uncaught exceptions on the EDT suspend execution in EventDispatchThread.run()

This is a minor annoyance I have when debugging with Eclipse. With "Suspend execution on uncaught exceptions" checked, the debugger will normally suspend a thread right where the exception was thrown. Exceptions on the Event Dispatch Thread, however, cause it to pause on the last line of EventDispatchThread.run(). There is no useful information about what the exception is or what caused it until I resume the thread to allow the stack trace to print to the console.
Debug the following code in Eclipse to demonstrate:
public class SuspendOnUncaughtTest {
public static void main(String[] args) {
javax.swing.SwingUtilities.invokeLater(new Runnable() {
public void run() {
((Object) null).toString();
}
});
}
}
And here is what it looks like after the exception is thrown:
EDIT 10/21/2011: I guess there's nothing weird going on with Eclipse or the Java debugger, it's just that exceptions are caught and rethrown in EventDispatchThread.pumpOneEventForFilters(int). I suppose there's no way to tell Eclipse to "suspend execution on exceptions that are going to be caught and possibly rethrown". Too bad.

Is see a similar result in NeteBeans, but Thread.setDefaultUncaughtExceptionHandler() may be useful in this context. There's an example here.

As far as I can tell, there is no solution to this. Eclipse is behaving correctly, the correct behavior is just annoying.

I had this issue and found something that worked for me.
I am using Kepler, and was able to stop this by going to the breakpoints tab of the debugger perspective and deselecting the checkbox on the EventDispatchThread.