Develop Reference

How to execute db/jpa operations inside quartz context?

I have this service Scheduling a task:
#ApplicationScoped
public class PaymentService {
#Transactional
public Payment scheduleNewPayment(Payment payment) throws ParseException, SchedulerException {
Payment.persist(payment);
JobDetail job = JobBuilder.newJob(PaymentJob.class)
.withIdentity(String.format("job%d", payment.id), "payment-job-group")
.build();
Date parsed = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss").parse(payment.dueDate);
SimpleTrigger trigger = (SimpleTrigger) TriggerBuilder.newTrigger()
.withIdentity(String.format("trigger%d", payment.id), "trigger-group")
.startAt(parsed)
.forJob(job)
.build();
SchedulerFactory schedulerFactory = new StdSchedulerFactory();
Scheduler scheduler = schedulerFactory.getScheduler();
scheduler.scheduleJob(job, trigger);
scheduler.start();
return payment;
}
}
And this job:
#ApplicationScoped
public class PaymentJob implements Job {
#Override
public void execute(JobExecutionContext jobExecutionContext) throws JobExecutionException {
System.out.println(Payment.count());
}
}
But I can not perform a DB operation inside the Job context (jobExecutionContext.getScheduler().getContext() is null by the way).
I'm running my app with quarkus, the hibernate operation comes from Hibernate Panache and the Scheduler is quartz.

First of all, you should use the underlying managed Quartz Scheduler instance : #Inject org.quartz.Scheduler (I suppose you're using the quarkus-quartz extension).
The other "problem" is that the default Quartz job factory simply calls new PaymentJob() and so no injection/initialization is performed. Quarkus is only using a custom factory for the jobs generated for methods annotated with #Scheduled. If you don't need injection then simply remove the superfluous#ApplicationScoped from the PaymentJob class.
Finally, you need to activate all the necessary CDI contexts manually. It's very likely that the request context is needed. You can copy the following snippet: https://github.com/quarkusio/quarkus/blob/master/extensions/arc/runtime/src/main/java/io/quarkus/arc/runtime/BeanInvoker.java#L14-L24 into your execute() method.
jobExecutionContext.getScheduler().getContext() is null by the way
This is really odd. What exception/error do you actually get?

shutdown all the quartz jobs as soon as app server is shut down?

I have a web application in which I am running some jobs periodically so for that I am using quartz framework here. Below is how I am starting all my jobs:
As soon as the server gets started up, it calls postInit method autmatically. And then I start all my jobs and it works fine:
#PostConstruct
public void postInit() {
logger.logInfo("Starting all jobs");
StdSchedulerFactory factory = new StdSchedulerFactory();
try {
factory.initialize(App.class.getClassLoader().getResourceAsStream("quartz.properties"));
Scheduler scheduler = factory.getScheduler();
// starts all our jobs using quartz_config.xml file
scheduler.start();
} catch (SchedulerException ex) {
logger.logError("error while starting scheduler= ", ExceptionUtils.getStackTrace(ex));
}
}
#PreDestroy
public void shutdown() {
logger.logInfo("Shutting down all jobs");
}
Now I want to stop all the jobs that are running as soon as we try to shutdown the app server. So whenever we try to shutdown the app server, it will call shutdown method automatically. Now I need some way where we can shutdown all the jobs as soon as shutdown method is called. What is the best way by which I can shutdown all the jobs as soon as shutdown method is called?
Below is my "quartz.properties" file. Do I really need "quartz.properties" file since I guess I am using default values anyways I think?
#------------------------- Threads ---------------------------------#
# how many jobs we should run at the same time?
org.quartz.threadPool.threadCount=15
# ----------------------------- Plugins --------------------------- #
# class from where we should load the configuration data for each job and trigger.
org.quartz.plugin.jobInitializer.class=org.quartz.plugins.xml.XMLSchedulingDataProcessorPlugin
org.quartz.plugin.jobInitializer.fileNames = quartz_config.xml
org.quartz.plugin.jobInitializer.failOnFileNotFound = true
org.quartz.jobStore.class = org.quartz.simpl.RAMJobStore

You can use Scheduler.shutdown() method as below and it is a good idea to externalize the quartz configuration even if you use the default parameters. This will make your code flexible.
private Scheduler scheduler;
#PostConstruct
public void postInit() {
logger.logInfo("Starting all jobs");
StdSchedulerFactory factory = new StdSchedulerFactory();
try {
factory.initialize(App.class.getClassLoader().getResourceAsStream("quartz.properties"));
scheduler = factory.getScheduler();
// starts all our jobs using quartz_config.xml file
scheduler.start();
} catch (SchedulerException ex) {
logger.logError("error while starting scheduler= ", ExceptionUtils.getStackTrace(ex));
}
}
#PreDestroy
public void shutdown() throws SchedulerException {
logger.logInfo("Shutting down all jobs");
scheduler.shutdown();
}

Spring: Still waiting for shutdown of 1 message listener invokers

I have a Spring Boot app which programatically starts a few JMS listeners. From config:
#Override
public void configureJmsListeners(final JmsListenerEndpointRegistrar registrar) {
final List<String> allQueueNames = getAllQueueNames();
for (final String queueName : allQueueNames) {
LOG.info("Creating JMSListener for queueName: '{}'", queueName);
final SimpleJmsListenerEndpoint endpoint = new SimpleJmsListenerEndpoint();
endpoint.setId(queueName + "_endpoint");
endpoint.setDestination(queueName);
endpoint.setMessageListener(message -> onMessage((TextMessage) message));
registrar.registerEndpoint(endpoint);
}
}
#Bean
#Override
public DefaultJmsListenerContainerFactory jmsListenerContainerFactory(
final ConnectionFactory connectionFactory,
final DefaultJmsListenerContainerFactoryConfigurer defaultJmsListenerContainerFactoryConfigurer) {
final DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
factory.setSessionTransacted(true);
factory.setErrorHandler(t -> errorService.handleException(t));
defaultJmsListenerContainerFactoryConfigurer.configure(factory, connectionFactory);
return factory;
}
Given certain circumstances, e.g. db failing, I have to stop the listeners from processing any more messages and STOP THE APP.
I use the following, from errorService, to stop the application:
((ConfigurableApplicationContext)(this).applicationContext).close();
However the application does not stop due to:
Still waiting for shutdown of 1 message listener invokers
...which repeats forever.
Is there any way that I can stop the application during processing of the JMS Message? I thought that stopping the application in the Listeners errorHandler would have worked but obviously the application thinks that the Listener is still processing the message and therefore will not stop.
Thanks in advance

Recover an Asynch ThreadPoolTaskexecutor after server crashed/shut down

I Have a Spring rest controller which is calling an asynchronous method using Spring's #Async methodology and return immediately an http 202 code (Accepted) to the client.(The asynchronous job is heavy and could lead to a timeout).
So actually, at the end of the asynchronous task, i'm sending an email to the client telling him the status of his request.
Everything works just fine but I'm asking myself what can I do if my server/jvm crashes or if it is shut down? My client would receive a 202 code and will never receive a the status email.
Is there a way to synchronize (in real time) a ThreadPoolTaskExecutor in a database or even in a file to let the server recover at startup without managing this on my own with complex rules and evolution status?
Here is my Executor configuration
#Configuration
#EnableAsync
public class AsyncConfig implements AsyncConfigurer {
#Override
public Executor getAsyncExecutor() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(4);
executor.setMaxPoolSize(8);
executor.setQueueCapacity(100);
executor.setThreadNamePrefix("asyncTaskExecutor-");
executor.setAwaitTerminationSeconds(120);
executor.setKeepAliveSeconds(30);
executor.setRejectedExecutionHandler(new ThreadPoolExecutor.CallerRunsPolicy());
executor.initialize();
return executor;
}
#Override
public AsyncUncaughtExceptionHandler getAsyncUncaughtExceptionHandler() {
return new SimpleAsyncUncaughtExceptionHandler();
}
}
The controller that launch the async task
#RequestMapping(value = "/testAsync", method = RequestMethod.GET)
public void testAsync() throws InterruptedException{
businessService.doHeavyThings();
}
The async method called:
#Async
public void doHeavyThings() throws InterruptedException {
LOGGER.error("Start doHeavyThings with configured executor - " + Thread.currentThread().getName() + " at " + new Date());
Thread.sleep(5000L);
LOGGER.error("Stop doHeavyThings with configured executor - " + Thread.currentThread().getName() + " at " + new Date());
}
}
Thx

For a web server shutdown the application lifecycle in a java web application will notifiy a ServletContextListener. If you provide an implementation of a ServletContextListener you can put your 'what is already processed' logic in the contextDestroyed method.
When the web server or the application is started again the listener can be used to recover an re-process the unprocessed items of your job using the contextInitialized method.
Another option would be using Spring destruction callbacks and place the logic here.
HTH

How can I control the rate at which Spring receives from a queue?

I am using Spring's message-driven POJO framework (and DefaultMessageListenerContainer in particular) to listen to several queues and topics.
In the case of one particularly queue, there is a need to slow the rate at which I drain the queue, on the order of one message every five minutes. The actual processing of the messages is a sub-second operation, but I would like the listener to sit idle for some time in between messages.
I have created a bit of a hack, but it is decidedly sub-optimal: What I've done is to set the max concurrency to 1 and add a Thread.sleep(..) after processing each message. I would like to find a way instead to use the DefaultMessageListenerContainer to wait between attempts to receive, rather than causing the handler to do the waiting during the would-be processing of a message.
I had considered if there was a ScheduledExecutor that would help, but I realize that the throttling would need to be done where the tasks are produced. Is there perhaps some method from DefaultMessageListenerContainer that I could override to accomplish what I'm after?

Depending on the provider of the queue, you may be able to set a max rate for consumers that consume it's queues.
For example in hornetQ you set this in the connection factory using consumer-max-rate.

An alternative to modifying the behavior of your consumer would be to make use of Apache Camel to delay the messages on that one specific queue.
http://camel.apache.org/delayer.html describes the functionality of the Camel Delayer pattern. So for example:
<route>
<from uri="jms:YOURQUEUE"/>
<delay>
<constant>1000</constant>
</delay>
<to uri="jms:DELAYEDQUEUE"/>
</route>
Where you would then consume the DELAYEDQUEUE and all messages would be delayed by 1 second.

I'm not sure for 100%, but believe that receiveTimeout is what you want.
<bean id="blahContainer" class="org.springframework.jms.listener.DefaultMessageListenerContainer">
....
<!-- 300000 = 5 * 60 * 1000 == 5 min -->
<property name="receiveTimeout" value="300000"/>
</bean>
receiveTimeout accepts timeout in milliseconds, you can read more about it in javadocs

Here's a solution that extends DefaultMessageListenerContainer to provide the throttling functionality. The advantage of this approach is that Thread.sleep() is not being called within onMessage(). This would hold a Transaction open for longer than necessary if a Transaction is in play (as configured in this example below). The call to Thread.sleep() occurs after the transaction has been committed. A limitation to implementing this throttling feature is that we can only support one consumer thread, hence the name ThrottlingSingleConsumerMessageListenerContainer.
#Configuration
#EnableJms
#EnableTransactionManagement
public class Config
{
private static final long THROTTLE_FIVE_SECONDS = 5_000;
#Bean
public DefaultMessageListenerContainer defaultMessageListenerContainer(
ConnectionFactory connectionFactory,
PlatformTransactionManager transactionManager,
MyJmsListener myJmsListner)
{
DefaultMessageListenerContainer dmlc = new ThrottlingSingleConsumerMessageListenerContainer(THROTTLE_FIVE_SECONDS);
dmlc.setConnectionFactory(connectionFactory);
dmlc.setSessionAcknowledgeMode(Session.SESSION_TRANSACTED);
dmlc.setSessionTransacted(true);
dmlc.setTransactionManager(transactionManager);
dmlc.setDestinationName("QUEUE.IN");
dmlc.setMessageListener(myJmsListner);
return dmlc;
}
}
#Component
public class MyJmsListener implements MessageListener
{
#Override
public void onMessage(Message message)
{
// process the message
}
}
public class ThrottlingSingleConsumerMessageListenerContainer extends DefaultMessageListenerContainer
{
private static final Logger log = LoggerFactory.getLogger(ThrottlingSingleConsumerMessageListenerContainer.class);
private final long delayMillis;
public ThrottlingSingleConsumerMessageListenerContainer(long delayMillis)
{
this.delayMillis = delayMillis;
super.setMaxConcurrentConsumers(1);
}
#Override
protected boolean receiveAndExecute(Object invoker, #Nullable Session session, #Nullable MessageConsumer consumer) throws JMSException
{
boolean messageReceived = super.receiveAndExecute(invoker, session, consumer);
if (messageReceived) {
log.info("Sleeping for {} millis", delayMillis);
try {
Thread.sleep(delayMillis);
} catch (InterruptedException e) {
log.warn("Sleeping thread has been interrupted");
Thread.currentThread().interrupt();
}
}
return messageReceived;
}
#Override
public void setMaxConcurrentConsumers(int maxConcurrentConsumers)
{
super.setMaxConcurrentConsumers(maxConcurrentConsumers);
Assert.isTrue(getMaxConcurrentConsumers() <= 1, "Throttling does not support maxConcurrentConsumers > 1");
}
#Override
public void setConcurrency(String concurrency)
{
super.setConcurrency(concurrency);
Assert.isTrue(getMaxConcurrentConsumers() <= 1, "Throttling does not support maxConcurrentConsumers > 1");
}
}
This has been tested on org.springframework 5.x but should run on earlier versions also.