Develop Reference

How to prevent Spring context closing until clustered Quartz jobs are not finished ? [ Spring + Quartz ]

I developed an application with Spring boot 2.2.4.RELEASE and Quartz (v 2.2.3) in the cluster.
I have a Master Job that finds the records in a table and schedule these records via scheduler
`org.springframework.scheduling.quartz.SchedulerFactoryBean
Every single job scheduled has a logic that interacts with DB via HikariCP (Connection pool).
The rule must be that in case of application shut-down the application has to wait until the end of every running job. I will be able to set this rule to
org.springframework.scheduling.quartz.SchedulerFactoryBean
via property
setWaitForJobsToCompleteOnShutdown(true); `
The solution it's working fine but I saw that the Connection Pool (HikariCP) is closed without to wait for the end jobs to run. It leads to the loss of the interaction logicon DB.
I'd like to avoid this thing.
During shut-down of Spring boot, is it possible to prioritize the objects close into the context to do to finish every single job process regularly ?

Spring Boot Shutdown with Scheduler

I'm using Spring Boot + Spring Task Scheduler to creating a polling application. I plan to use the actuator to shutdown the application during maintenance windows (see shutdown endpoint for actuator: http://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#production-ready-endpoints).
My question is, will the Spring be smart enough finish an in-flight task before gracefully shutting down the application? Will Spring be smart enough not to start another task if the in-flight task completes before shutdown?
Thank you.

Since spring boot 2.1 you can set the following property
# Whether to wait for running jobs to complete on shutdown.
spring.quartz.wait-for-jobs-to-complete-on-shutdown=false

You can control this by setting
ThreadPoolTaskExecutor#waitForTasksToCompleteOnShutdown
to true.
There is a really good explanation in the javadoc of the inherited setter
org.springframework.scheduling.concurrent.ExecutorConfigurationSupport#setWaitForTasksToCompleteOnShutdown
Set whether to wait for scheduled tasks to complete on shutdown,
not interrupting running tasks and executing all tasks in the queue.
Default is "false", shutting down immediately through interrupting
ongoing tasks and clearing the queue. Switch this flag to "true" if you
prefer fully completed tasks at the expense of a longer shutdown phase...
The spring scheduler uses per default a
java.util.concurrent.ThreadPoolExecutor
which will not start new tasks during shutdown as described in javadoc of
java.util.concurrent.ThreadPoolExecutor#shutdown
Initiates an orderly shutdown in which previously submitted
tasks are executed, but no new tasks will be accepted...

Should I Shutdown Scheduler at Servlet's destroy() Method?

There is a startup servlet that initialises Quartz Scheduler in the init() method and shut down the scheduler at destroy() method. I understand that Servlet itself is an object and will be GC collected at a unknown interval. If a shut down of Quartz Scheduler is placed in the destroy() method, I am expecting the scheduler to be restarted each time the servlet is GC collected. If the process of the destroy and start skipped the exact timing where I am expecting quartz to run a thread, I probably will have trouble. Am I right in this assumption?
Note: I noted that there is a context listener in the quartz library to shut down quartz when the application is out of service.

The Servlet specification does allow containers to unload Servlets at any point although they don't have to do this (Tomcat doesn't for example). You are unlikely to hit a problem but if Quartz is used by components other than the Servlet that starts it them it really belongs to the application not the Servlet and should be configured at that level.
Generally, it is better to start and stop Quartz with a ServletContextListener than with a Servlet.

How to reliably kill #Scheduled threads across servers?

I'm building a plugin that is implemented as a Spring MVC application. This plugin is deployed on 3 - 6 tomcat servers via a gui on one of the servers. Each of the instances of the plugin has an #Scheduled method to collect information on the server and store it in a central database.
My issue is that the gui interface for uninstalling the plugin leaves some of the #Scheduled threads running.
For example, I have an environment that has servers 1 - 3. I install and enable the plugin via the gui on server 1. There are now 3 instances of the application running #Scheduled threads on servers 1 - 3. If I go back to server 1 and uninstall the plugin, the thread is reliably killed on server 1 but not servers 2 or 3.
I've implemented the following but the behavior persists:
#Component
public class ContextClosedListener implements ApplicationListener<ContextClosedEvent> {
#Autowired
ThreadPoolTaskExecutor executor;
#Autowired
ThreadPoolTaskScheduler scheduler;
public void onApplicationEvent(ContextClosedEvent event) {
scheduler.shutdown();
executor.shutdown();
}
}
Additionally, I've thought of implementing this as a context listener rather than an #Scheduled method but I'd rather stick to Spring for maintenance and extensibility reasons.
How can I reliably kill threads in an environment like this?

A couple thoughts I have. ThreadPoolTaskExecutor has a method setThreadNamePrefix, which allows you to set the prefix of the thread. You could set the prefix to something unique, then find and kill those threads at runtime. You can also set the thread group using the setThreadGroup method on the same object, then just stop the threads in the threadgroup.
The better, and safer, solution would be to create a break-out method in your scheduled jobs. This is the prefered method to stopping a Thread instead of the old "shot it in the head" method of calling Thread.stop(). You could get reference to those Runnables either by setting a common prefix or by using the thread group as described above.
The next question is: how do you stop the threads easily? For that, it would depend on how your appliation is implemented. Since I deal mainly with Spring MVC apps, my first solution would be to write a Controller to handle admin tasks. If this was JBoss, or some other large app server that had JMX (Tomcat can be configured to provide JMX I believe, but I don't think its configured out of the box that way), I might write a JMX-enabled bean to allow me to stop the threads via the app servers console. Basically, give your self a method to trigger the stopping of the threads.

User thread spawning in Jetty

I have a web application running over Jetty, and I need to spawn a thread for idle connection handling. This thread is being started in the spring context.
I know it's not a good practice to spawn threads in a container, but couldn't find a better way to do this. Any ideas?

You can have the container set up a timer or thread pool for you. See the docs.

The traditional way to handle such resources is in a servlet context listener. Check the Servlet API.