I am considering using the Quartz framework to schedule the run of several hundred jobs.
According to their API, jobs can be scheduled to run at certain moments in time but not to run one after the other (and stop a chain of jobs if one fails).
The only recommended methods I was able to find are:
Using a listener which notices the completion of a job and schedule the next trigger to fire (how to coordinate this?)
Each job will receive a parameter containing the next job to run and, after completing the actual work, schedule its run. (Cooperative)
Do you know a better method to create a workflow of jobs in Quartz?
Can you recommend other methods/framework for implementing a workflow in Java ?
EDITED: In the meantime I found out about OSWorkflow which appears to be a good match for what I need. It appears that what I need to implement is a "Sequence Pattern".
When Quartz documentation talks about "Job", it is referring to a class implementing the "Job" Interface, which is really just any class with an "execute" method that takes in the Quartz Context object. When creating this implementation you can really do whatever you want.
You could create an implementation of the Quartz Job Interface which simply calls all the jobs in your workflow in series, and throws a JobExecutionException exception on failure.
It sounds to me like you want Quartz to schedule the first job, and chain everything off that.
Have you looked at encapsulating each task using the Command Pattern, and linking them together ?
I've worked on a project called Dynamic Task Scheduler that use Quartz to execute job chains implementing a simple workflow in a fault-tolerant way (definied in XML format).
Take a look at http://sourceforge.net/projects/dynatasksched/
The project is beta, but I think it can gives you some ideas to start..
Hope it's useful!
For job chaining support for Quartz, you may want to check the QuartzDesk project that I have been involved in. In version 2.0. we have added a powerful job chaining engine that enables you to orchestrate your Quartz jobs without the need to modify your application code.
The engine takes care of propagating the job execution result and other parameters from the source job to the chained target job.
QuartzDesk comes with a GUI that allows you to dynamically update your job chains without disrupting your application.
Related
I have question about approach I should use for my case. I have to schedule a job which will use user input (start date/time and file - user can pick start date/time and file on frontend). Job will do the same thing everytime, but with different file, cron expression and schedule name. So no functional changes in job - only different parameters.
For now there are java config classes with beans to configure jobs and I'm now thinking if it is possible to create new configuration class and change somehow parameters which I'm interested in? Or I shouldn't use beans and create a service where I will build everytime Job, JobDetails, Trigger etc and just schedule it?
I'm completely new in Spring Batch and I had to takeover this part from guy who left the team.
You need to dynamically create the schedule according to user input. Quartz provides the Trigger interface which you can implement as needed.
A similar question here: Dynamic Job Scheduling with Quartz or any other java api
Hope this helps.
We have an application which does a lot of imports and exports - basically between CSV files and database tables.
Some of the imports and exports are conflicting (you can't execute them simultaneously) for various reasons (like "legacy code").
We were looking into javax.batch. Conceptually it suits very well. But what we really failed to find is the possibility to somehow manage the "exclusiveness" of certain jobs we want to run.
Could someone please provide a pointer on that? How would we implement exclusive batch jobs with javax.batch? Or should we implement our own JobOperator for this?
Update
What I mean by "exclusiveness" is tha ability to define that certain jobs may not be executed in parallel. In the most trivial case this would mean "only execute one an only one job at time". In more complex cases more complex logic like "job of type A can't run with other jobs of type A or B, but C is OK". The "type of job" is, for instance, jobXmlName here (regardless of job parameters).
JSR-352 (and Spring Batch) both avoid the topic of orchestration on purpose. To do so would require a particular approach which prevents the inherent flexibility these batch frameworks offer. Because of that, the JobOperator in JSR-352 does not have any notion of preventing one job from running while another one is running.
While you could accomplish this via your own custom JobOperator, I wouldn't recommend that approach. Instead, you'd be better off moving that one layer higher, into whoever is calling the JobOperator so that the logic for that type of orchestration concern is separated from the implementation details of launching a job. For example, if you're using a scheduler to launch jobs, I'd put the logic there as to what jobs can run in parallel and which ones cannot...not in a custom JobOperator.
There is a requirement in the project that will have a scheduled task that will do some job.
The project is Spring based and the scheduled job will be part of the application war.I have
never implemented this kind of functionality before.
I have heard of Quartz. Also, I read somewhere that Spring provides some functionality to schedule tasks. So, I was thinking if I am already using Spring then why to go for some other API(Quartz).
I am not sure which one to use? what will be the pros/cons of one over another?
Please suggest what will be the best way to approach my requirement.
I have used Spring's Task execution and scheduling - http://static.springsource.org/spring/docs/3.0.x/reference/scheduling.html
I'm reading the Spring 3.0 docs regarding scheduling. I'm leaning towards Spring's JobDetailBean for Quartz. However, the #Scheduled annotation has captured my eye. It appears this is another way of scheduling task using the Spring Framework. Based on the docs, Spring provides three way of scheduling:
#Scheduled
Via Quartz
Via JDK Timer
I have no interest in the JDK Timer. Why should I choose #Scheduled over Quartz? (When I mention Quartz I mean using Spring's bean wrapper for Quartz).
Let's say my use case is complex enough that I will be communicating to a third-party web service to import and export data at specified intervals.
Quartz is an order of magnitude more complex than Spring's built in scheduler, including support for persistent, transactional and distributed jobs. It's a bit of a pig, though, even with Spring's API support.
If all you need to is to execute methods on a bean every X seconds, or on a cron schedule, then #Scheduled (or the various options in Spring's <task> config schema) is probably enough
I have to state my own experience regarding use of #Scheduled versus Quartz as scheduling implementation in a Spring application.
Scheduling jobs had the following requirements:
End users should have the ability to save and schedule (define execution time) their own tasks
Scheduled jobs during server downtime should not get omitted from jobs queue
Hence, we have to try and use Quartz implementation (version 2.2.3) in order to support persistence of jobs in a database. Some basic conclusions are the following:
Integration with a Spring 4 MVC application is not difficult at all using quartz.properties file.
We had the ability to choose a second database for storing the jobs from the main database.
Jobs scheduled during server downtime begin running as long as server comes up.
As a bonus we managed to maintain in main database some useful (and more user-oriented) information about user defined scheduled jobs using custom JobListener and TriggerListener.
Quartz is a very helpful library in applications with more complex scheduling requirements.
According to Quartz Documentation
We can use some more and complex feature that it doesn't exist in #Scheduler.
for example:
in Quartz we can placing a scheduler in stand-by mode with
scheduler.standby(); and re schedule it with scheduler.start();.
shutting down a scheduler before execution of job or after that with
scheduler.shutdown(true); and scheduler.shutdown(false);
storing a job for later use and when you need the job you can
triggered it.
JobDetail job1 =newJob(MyJobClass.class).
withIdentity("job1","group1").
storeDurably().
build();
Add the new job to the scheduler, instructing it to "replace" the
existing job with the given name and group (if any).
JobDetail job1 = newJob(MyJobClass.class).
withIdentity("job1", "group1").
build();
In Spring you could schedule task by using FixedRate,FixedDelay and cron. But most of the scheduled job requires dynamic handling of execution time. So in this scenario it is better to use Quartz as it provide the option to store scheduled jobs in DBJobstore as well as RAMJobstore.
I developed a web application and one of the function is to send emails.
I like to have the behaviour whereby the actual sending of emails is done in a batch job. Meaning when user click on the trigger button in online screen. The job for sending emails will be scheduled to run instead of immediately run.
How to do that exaclty? Any sample code references etc?
I would recommend Quartz website where I learnt Quartz.
For Spring integration, follow http://static.springsource.org/spring/docs/1.2.9/reference/scheduling.html
This tutorial will give you better idea how to schedule a job in quartz.
In example DumbJob.class will implement Job interface from Quartz and in turn provide execute() method. This method will contain batch job code.
I think what you really want to do is implement a queue / worker model here. The job gets added to a queue and periodically, workers poll the queue to determine if anything needs to be done.