I am new to Spring Batch framework and quartz scheduler. My task is to schedule a new Spring Batch job dynamically using quartz scheduler. All new spring batch job's entries are in my database with trigger expression. Problem is that for every new spring batch job coming from database , we need to wrap it in quartz's scheduler job. so means as many spring batch job will be there that many batch job class should be there to wrap them and run by quartz scheduler.
quartz is storing all the job's and trigger entry into its own data base tables. which i have configure in config file. This job will always be quartz's job not spring batch job.
here is my main method, here i will write my data base connection code to find out new springbatch job name and trigger expression and will bind them with quartz schedular
public static void main(String[] args) {
try {
ApplicationContext context = new ClassPathXmlApplicationContext("quartz-context.xml");
JobLauncher launcher=(JobLauncher) context.getBean("jobLauncher");
JobLocator locator= (JobLocator) context.getBean("jobRegistry");
Scheduler schedulerFactoryBean=(Scheduler) context.getBean("quartzSchedulerFactoryBean");
JobDetail job = newJob(SpringBatchJob.class).withIdentity("myJob001", "group1").build();
Trigger trigger1 =newTrigger().withIdentity("myTrigger001", "group1").startNow().withSchedule(simpleSchedule().withIntervalInSeconds(10).repeatForever()).build();
schedulerFactoryBean.scheduleJob(job, trigger1);
schedulerFactoryBean.start();
} catch (SchedulerException e) {
e.printStackTrace();
}
}
Here we can see that we have jobDetail which is quartz job and whose execute method is use to run spring batch job.
springBatchjob.java
public class SpringBatchJob implements Job {
private String jobName;
private String batchJob;
private JobLocator jobLocator;
private JobLauncher jobLauncher;
private File contentDirectory;
private String directoryPath = "inputFiles";
public void init(){
contentDirectory = new File(directoryPath);
}
boolean fileFound = false;
public void performJob(String str) {}
public String getJobName() {
return jobName;
}
public void setBatchJob(String batchJob) {
this.batchJob = batchJob;
}
public void setJobName(String jobName) {
this.jobName = jobName;
}
public void setJobLocator(JobLocator jobLocator) {
this.jobLocator = jobLocator;
}
public void setJobLauncher(JobLauncher jobLauncher) {
this.jobLauncher = jobLauncher;
}
#Override
public void execute(JobExecutionContext arg0) throws org.quartz.JobExecutionException {
JobParameter jb= new JobParameter(5L);
Map<String, JobParameter> map= new HashMap<>();
map.put(jobName,jb);
ApplicationContext context = new ClassPathXmlApplicationContext("quartz-context.xml");
JobLauncher launcher=(JobLauncher) context.getBean("jobLauncher");
JobLocator locator= (JobLocator) context.getBean("jobRegistry");
setJobLauncher(launcher);
setJobLocator(locator);
setJobName("helloWorldJob");
// TODO Auto-generated method stub
JobExecution result = null;
try {
result = jobLauncher.run(jobLocator.getJob(jobName), new JobParameters(map));
} catch (JobExecutionAlreadyRunningException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (JobRestartException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (JobInstanceAlreadyCompleteException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (JobParametersInvalidException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (NoSuchJobException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
System.out.println("ExamResult Job completetion details : "+result.toString());
}
here in setJobName method i am hard coding my spring batch job name ,
But in my project we are having nearly 800 jobs, so acc to approarch we need to make 800 wrapper classes.
Please help me , that how to resolve this solution by making generic class.
I sincerely hope you aren't really using that class, it will eventually eat all your memory as you are recreating your application over and over.
Spring already has support for quartz and especially the construction of jobs in the form of the SpringBeanJobFactory which you can use to your advantage especially if you extend it with some auto wiring capabilities.
public class AutowiringSpringBeanJobFactory extends SpringBeanJobFactory implements ApplicationContextAware {
private ApplicationContext context;
#Override
protected Object createJobInstance(TriggerFiredBundle bundle) throws Exception {
Object object = super.createJobInstance(bundle);
context.getAutowireCapableBeanFactory().autowireBean(object);
return object;
}
public void setApplicationContext(ApplicationContext applicationContext) {
this.context=applicationContext;
}
}
Then rewrite your job class as something like this
public class SpringBatchJob implements Job {
private final Logger logger = LoggerFactory.getLogger(SpringBatchJob.class);
private String jobName;
#Autowired
private JobLocator jobLocator;
#Autowired
private JobLauncher jobLauncher;
#Override
public void execute(JobExecutionContext context) throws org.quartz.JobExecutionException {
JobDataMap JobDataMap = context.getMergedJobDataMap();
JobParametersBuilder builder = new JobParametersBuilder();
for (Map.Entry<String, Object) param : jobDataMap.entrySet()) {
String key = param.getKey();
Object val = param.getValue();
builder.addString(key, String.valueOf(val)); // Or make it smarter by doing type detection.
}
Job jobToLaunch = jobLocator.getJob(jobName);
JobExecution result;
try {
result = jobLauncher.run(jobToLaunch, builder.to);
} catch (JobExecutionException e) {
throw new org.quartz.JobExecutionException("Exception execution job '"+this.jobName+"'", e);
} finally {
logger.info("{} Job completetion details ", this.jobName, result);
}
}
}
Now you need to configure the SchedulerFactoryBean to use the AutowiringSpringBeanJobFactory by setting the jobFactory property. When you done then you just need to configure the quartz jobs appropriately.
For the sample you posted the following will launch the helloWorldJob when it is triggered to launch.
public static void main(String[] args) {
try {
ApplicationContext context = new ClassPathXmlApplicationContext("quartz-context.xml");
JobLauncher launcher=(JobLauncher) context.getBean("jobLauncher");
JobLocator locator= (JobLocator) context.getBean("jobRegistry");
Scheduler schedulerFactoryBean=(Scheduler) context.getBean("quartzSchedulerFactoryBean");
JobDetail job = newJob(SpringBatchJob.class)
.withIdentity("myJob001", "group1")
.usingJobData("jobName", "helloWorldJob")
.build();
Trigger trigger1 =newTrigger().withIdentity("myTrigger001", "group1")
.startNow()
.withSchedule(simpleSchedule().withIntervalInSeconds(10).repeatForever())
.build();
schedulerFactoryBean.scheduleJob(job, trigger1);
schedulerFactoryBean.start();
} catch (SchedulerException e) {
e.printStackTrace();
}
}
Notice the .usingJobData("jobName", "helloWorldJob"), the SpringBeanJobFactory will try to satisfy all setter methods on the SpringBatchJob class. This has a setJobName which will get injected with helloWorldJob when launched. The functionality is extended by the AutowiringSpringBeanJobFactory to also auto wire the needed infrastructure beans from Spring Batch.
If you need to pass additional properties to the Spring Batch job (like keys to use or other information, just add another usingJobData("your-property", your-value). The modified SpringBatchJob will map all quarts job parameters to Spring Batch parameters before launching the job.
Note: This was typed from the top of my head, I haven't actually tested this, but it should at least give you an idea on how to do this.
Related
I have a spring boot java service I have to schedule to run on a particular time. I have enabled the #Enablescheduling and #Scheduled annotation and given the cron expression.
It's working fine. The scheduler is running at the expected time. But my concern is I should control the cron expression somewhere from outside my jar file. I have tried using it in property file but when packaging my property file also getting included in that.
Sample code:
#PostMapping(path = "getoktatodynamodb")
#Scheduled(cron = "0 0/5 0 * * ?")
#ApiOperation("Sync data to DynamoDB")
public FinalResponse getdatatodynamodb() {
FinalResponse finalResponse = new FinalResponse();
try {
LOGGER.info("Sync data to DynamoDB starts - " + new Date());
finalResponse = dynamodbuserService.dynamoDbSync();
} catch (MyRestTemplateException ex) {
LOGGER.error(ex.getMessage());
finalResponse.setResponseMessage(ex.getMessage());
finalResponse.setStatusCode(ex.getStatusCode().value());
} catch (Exception execption) {
LOGGER.error(execption.getMessage());
finalResponse.setResponseMessage(execption.getMessage());
finalResponse.setStatusCode(HttpStatus.INTERNAL_SERVER_ERROR.value());
} finally {
LOGGER.info("Sync data DynamoDB Ends - " + new Date());
}
return finalResponse;
}
The main intention is scheduler should be in our control whenever we need to change the time it should be configurable. No code change and restarting the scheduler for minor changes.
How should we achieve this also we would like to schedule this in linux ec2 instance? in case if we have better suggestion to achieve this kindly share it.
You can implement SchedulingConfigurer:
https://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/scheduling/annotation/SchedulingConfigurer.html
This DZone article shows a really good example: https://dzone.com/articles/schedulers-in-java-and-spring which I'm showing here in case the article doesn't stay permanent.
#Configuration
#EnableScheduling
public class ScheduledConfiguration implements SchedulingConfigurer {
TaskScheduler taskScheduler;
private ScheduledFuture<?> job1;
private ScheduledFuture<?> job2;
#Override
public void configureTasks(ScheduledTaskRegistrar taskRegistrar) {
ThreadPoolTaskScheduler threadPoolTaskScheduler =new ThreadPoolTaskScheduler();
threadPoolTaskScheduler.setPoolSize(10);// Set the pool of threads
threadPoolTaskScheduler.setThreadNamePrefix("scheduler-thread");
threadPoolTaskScheduler.initialize();
job1(threadPoolTaskScheduler);// Assign the job1 to the scheduler
// Assign the job1 to the scheduler
this.taskScheduler=threadPoolTaskScheduler;// this will be used in later part of the article during refreshing the cron expression dynamically
taskRegistrar.setTaskScheduler(threadPoolTaskScheduler);
}
private void job1(TaskScheduler scheduler) {
job1 = scheduler.schedule(new Runnable() {
#Override
public void run() {
System.out.println(Thread.currentThread().getName() + " The Task1 executed at " + new Date());
try {
Thread.sleep(10000);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}, new Trigger() {
#Override
public Date nextExecutionTime(TriggerContext triggerContext) {
String cronExp = "0/5 * * * * ?";// Can be pulled from a db .
return new CronTrigger(cronExp).nextExecutionTime(triggerContext);
}
});
}
private void job2(TaskScheduler scheduler){
job2=scheduler.schedule(new Runnable(){
#Override
public void run() {
System.out.println(Thread.currentThread().getName()+" The Task2 executed at "+ new Date());
}
}, new Trigger(){
#Override
public Date nextExecutionTime(TriggerContext triggerContext) {
String cronExp="0/1 * * * * ?";//Can be pulled from a db . This will run every minute
return new CronTrigger(cronExp).nextExecutionTime(triggerContext);
}
});
}
}
I am using Quartz CronScheduler to execute a job every 15 minutes. Every time the job executes it checks the DB for any change in the cron expression and updates the job scheduler for the next run. I have implemented the above in the following way:
#Service
public class QuartzSchedulerService {
private static final Logger LOG = LoggerFactory.getLogger(QuartzSchedulerService.class);
private Scheduler scheduler;
#PostConstruct
private void init() {
try {
scheduler = StdSchedulerFactory.getDefaultScheduler();
scheduler.start();
} catch (Exception e) {
LOG.error("Unable to start quartz scheduler", e);
}
}
#PreDestroy
private void destroy() {
try {
scheduler.shutdown();
} catch (Exception e) {
LOG.error("Unable to shutdown quartz scheduler", e);
}
}
public void registerCronJob(Class<? extends Job> jobClass, String cronExpression) {
try {
String jobName = jobClass.getSimpleName();
CronScheduleBuilder cronBuilder = CronScheduleBuilder.cronSchedule(cronExpression);
JobDetail jobDetail = JobBuilder.newJob(jobClass).withIdentity(jobName).build();
CronTrigger cronTrigger = TriggerBuilder.newTrigger().withIdentity(jobName).withSchedule(cronBuilder).build();
scheduler.scheduleJob(jobDetail, cronTrigger);
LOG.info("Registered Cron Job:" + jobName + " " + jobDetail.getKey());
} catch (Exception e) {
LOG.error("Unable to register cron job", e);
}
}
public void updateCronSchedule(Class<? extends Job> jobClass, String cronExpression) {
try {
String jobName = jobClass.getSimpleName();
CronScheduleBuilder cronBuilder = CronScheduleBuilder.cronSchedule(cronExpression);
CronTrigger newCronTrigger = TriggerBuilder.newTrigger().withIdentity(jobName).withSchedule(cronBuilder).build();
scheduler.rescheduleJob(TriggerKey.triggerKey(jobName), newCronTrigger);
LOG.info("Updated Cron Job:" + jobName + " " + newCronTrigger.getJobKey());
LOG.info("Jobs executed: " + scheduler.getMetaData().getNumberOfJobsExecuted());
} catch (Exception e) {
LOG.error("Unable to reschedule cron job", e);
}
}
}
The class which implements the Job interface is as belows:
#Component
#DisallowConcurrentExecution
public class PropertiesReloadJob implements Job {
private static final Logger LOG = LoggerFactory.getLogger(PropertiesReloadJob.class);
#Autowired
private QuartzSchedulerService schedulerService;
#Autowired
private PropertiesService propertiesService;
public void loadAtStartup() {
load();
LOG.info("--- Registerting PropertiesReload Cron Job ---");
schedulerService.registerCronJob(PropertiesReloadJob.class, propertiesService.getCacheReloadCronExpression());
}
#Override
public void execute(JobExecutionContext context) throws JobExecutionException {
SpringBeanAutowiringSupport.processInjectionBasedOnCurrentContext(this);
load();
LOG.info("--- Updating Pro Cron Job ---");
schedulerService.updateCronSchedule(PropertiesReloadJob.class, propertiesService.getCacheReloadCronExpression());
}
public void load(){
// Load properties from DB
}
The loadAtStartup() method is called during context initializtion and then after every 15 minutes the execute() method is called.
The cron expression used is: 0 0/15 * 1/1 * ? *
Now, the problem is as follows:
Lets say that the job starts at 3:00:00, it will execute as many times it can till 3:00:01, rather than executing only once.
Next the job will start at 3:15:00 and again will run as many times it can till 3:15:01.
The number of times the job executes is different every time.
I am not sure what is causing this behaviour. I have tested the cron expression with cronmaker.
Can somebody point out the error here ?
I have a Springboot application and I'm trying to execute an asynchronous method on a bean class inside a controller method. The problem is that my #Async method is not being executed asynchronously. Execution is halted until the method completes.
Can anyone tell me what I'm missing?
Here is my application class:
#SpringBootApplication
#EnableAsync
public class Application {
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
#Bean
public EmbeddedServletContainerFactory servletContainer() {
TomcatEmbeddedServletContainerFactory factory = new TomcatEmbeddedServletContainerFactory();
factory.addConnectorCustomizers(new TomcatConnectorCustomizer() {
#Override
public void customize(Connector connector) {
connector.setPort(9000);
connector.setAsyncTimeout(60000);
}
});
return factory;
}
}
Here is my bean class:
public class LongProcess {
#Async
public Future<String> call() {
try {
System.out.println("Sleeping now...");
Thread.sleep(10000);
return new AsyncResult<String>("Hey");
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
return null;
}
}
}
My configuration class:
#Configuration
#EnableAsync
public class LongProcessConfiguration implements AsyncConfigurer {
#Bean
public LongProcess longProcessBean() {
return new LongProcess();
}
#Override
public Executor getAsyncExecutor() {
ThreadPoolTaskExecutor taskExecutor = new ThreadPoolTaskExecutor();
taskExecutor.setMaxPoolSize(10);
taskExecutor.setThreadNamePrefix("LULExecutor-");
taskExecutor.initialize();
return taskExecutor;
}
#Override
public AsyncUncaughtExceptionHandler getAsyncUncaughtExceptionHandler() {
return new SimpleAsyncUncaughtExceptionHandler();
}
}
My controller method:
#RequestMapping("/utilities/longProcess")
public String longProcess() {
System.out.println("Starting long process...");
CsvFileDifferConfiguration context = new CsvFileDifferConfiguration();
LongProcess process = context.longProcessBean();
Future<String> result = process.call();
System.out.println("Done!");
return "{success: 1}";
}
This request unfortunately does not return immediately (I don't care about the result). The method is called successfully, but not in the background. Any idea what I might be missing?
As a test, if I change the controller method to wait for the result, the wait block is never entered:
#RequestMapping("/utilities/longProcess")
public String longProcess() throws InterruptedException {
System.out.println("Starting long process...");
CsvFileDifferConfiguration context = new CsvFileDifferConfiguration();
LongProcess process = context.longProcessBean();
Future<String> result = process.call();
while (!(result.isDone())) {
Thread.sleep(1); //10-millisecond pause between each check
System.out.println("Waiting for Long Process...");
}
System.out.println("Done!");
return "{success: 1}";
}
You have a mistake for the CDI usage.
If you manage your object using Spring Container you have to get deal just with ApplicationContext or its abilities like #Autowired.
The code
CsvFileDifferConfiguration context = new CsvFileDifferConfiguration();
is wrong.
Since you define your LongProcess as a #Bean you can just inject it to your #Controller:
#Autowired
privete LongProcess process;
and use it as before.
Using objects directly (e.g. new) loses the dependency injection features.
Read more Spring Docs, please.
The requirements were, to be able to dynamically schedule jobs using Quartz. After looking at examples on the web I found most of the examples were of static scheduling. I want to create quartz job within a loop. Currently only one job is running. Please help me. My code is given below
while (iterator.hasNext()) {
JSONObject obj =iterator.next();
ISocialMediaPoller socialMediaObj=socialMeadiaObj.getPoller(obj);
String jobName = (String)obj.get("NAME");
// long rpo =(Long)obj.get("RPO");
JobDetail job = new JobDetail();
job.setName(jobName);
job.setJobClass(Pollersheduller.class);
//configure the scheduler time
SimpleTrigger trigger = new SimpleTrigger();
trigger.setName(jobName);
trigger.setStartTime(new Date(System.currentTimeMillis() + 1000));
trigger.setRepeatCount(SimpleTrigger.REPEAT_INDEFINITELY);
trigger.setRepeatInterval(12345);
// socialMediaObj.execute();
//schedule it
Scheduler scheduler = new StdSchedulerFactory().getScheduler();
scheduler.getContext().put("socialMediaObj", socialMediaObj);
scheduler.start();
scheduler.scheduleJob(job, trigger);
}
public void execute(JobExecutionContext context) throws JobExecutionException {
// TODO Auto-generated method stub
SchedulerContext schedulerContext = null;
try {
schedulerContext = context.getScheduler().getContext();
ISocialMediaPoller socialMediaObj=
(ISocialMediaPoller)schedulerContext.get("socialMediaObj");
socialMediaObj.execute();
} catch (SchedulerException e1) {
e1.printStackTrace();
}
I would suggest managing your jobs through the db , in case your server restarts you better persist the dynamically created quartz jobs.
Here is an example
#Service public class PersistentJobSchedulerJob {
private static Logger logger = Logger.getLogger("PersistentJobSchedulerJob");
#Autowired
private JobRepository jobRepository;
#Autowired
private MailService mailService;
#SuppressWarnings({ "rawtypes", "unchecked" })
#Scheduled(fixedRate=30000)
public void schedulePersistentJobs(){
List<JobData> jobsData= jobRepository.findAll();
logger.info("Retriving Jobs from Database and Scheduling One by One | Total Number of Jobs: "+jobsData.size());
try{
Scheduler scheduler = new StdSchedulerFactory().getScheduler();
scheduler.start();
for(JobData jobData: jobsData){
JobDetail job = newJob(MailSenderJob.class)
.withIdentity(jobData.getJobName())
.usingJobData(getJobDataMap(jobData))
.build();
if(!jobData.getActive()){
logger.info("Deleting a Job");
scheduler.deleteJob(new JobKey(jobData.getJobName()));
continue;
}
if(scheduler.checkExists(new JobKey(jobData.getJobName()))){
logger.info("Rescheduling the Job");
Trigger oldTrigger = scheduler.getTrigger(new TriggerKey(jobData.getJobName()+"Trigger"));
TriggerBuilder tb = oldTrigger.getTriggerBuilder();
Trigger newTrigger = tb.withSchedule(simpleSchedule()
.withIntervalInMilliseconds(jobData.getRepeatInterval()).
repeatForever())
.build();
scheduler.rescheduleJob(oldTrigger.getKey(), newTrigger);
}else{
logger.info("Scheduling the Job");
scheduler.scheduleJob(job,getTrigger(jobData));
}
}
}catch (SchedulerException e) {
logger.error("Scheduler Exception : "+e.getMessage());
}
}
private JobDataMap getJobDataMap(JobData jobData) {
JobDataMap jobDataMap = new JobDataMap();
jobDataMap.put("recipients", jobData.getRecipients());
jobDataMap.put("mailService", mailService);
return jobDataMap;
}
private Trigger getTrigger(JobData jobData){
SimpleTrigger simpleTrigger = newTrigger().withIdentity(jobData.getJobName()+"Trigger")
.startAt(jobData.getStartDateTime())
.withSchedule(simpleSchedule()
.withIntervalInMilliseconds(jobData.getRepeatInterval()).
repeatForever())
.build();
return simpleTrigger;
} }
The full source code can be found here: Job scheduling with Quartz example
I am using JBoss5.1.x AS, EJB3.0. I am trying to add a job (using Quartz) to my deployment. I am registering a new Service, so it will init the scheduler on application deploy.
My problem is that the service never gets registered when I deploy my app.
My code:
Interface:
public interface ComponentMonitoringService
{
void create() throws Exception;
void start() throws Exception;
void stop();
void destroy();
}
Service:
#Service(objectName = "com.mirs.ecms.timer:service=ServerStartupManager")
#Management(ComponentMonitoringService.class)
public class ServerStartupManager implements ComponentMonitoringService
{
private SchedulerFactory schedulerFactory = null;
private Scheduler scheduler = null;
Logger logger = Logger.getLogger("ecms.log");
public void create() throws Exception
{
}
public void start() throws Exception
{
// Write your startup code
initScheduler();
}
private void initScheduler() throws ParseException, SchedulerException
{
schedulerFactory = new StdSchedulerFactory();
scheduler = schedulerFactory.getScheduler();
JobDetail startECMSJob = new JobDetail("startECMSJob", "group1", StartECMSJob.class);
CronTrigger trigger1 = new CronTrigger("cronTrigger", "TriggersGroup1", "0 0/5 * * * ?");
scheduler.scheduleJob(startECMSJob, trigger1);
scheduler.start();
}
public void stop()
{
try
{
scheduler.shutdown();
}
catch (Exception e)
{
logger.error("ServerStartupManager Failure occured during Manager stop", e);
}
}
public void destroy()
{
}
}
I found a solution.
I was not using the right annotation. I have to use EJB3 annotations.