My application is split into 2 web applications running in the same container sharing one db.
The first war does only background processing and the other is for the client GUI + some background stuffs.
The application with the client GUI allows the user to configure the scheduling of some tasks that will be executed by the "background application". Basically it configures the Quartz jobs and triggers.
I'd like that the scheduler of the background application handles only the jobs of a certain group (bg-jobs), and that the other scheduler handles the other group (fg-jobs).
Is it possible to configure this kind of isolation with quartz?
Note: I'd like to keep it simple and if I can avoid to use Quartz Where which seems to be liek a hammer to sledge this probably overkill for my need.
Thanks in advance
The simplest and quickest way is to create a separate load of tables for each application. So have one set of quartz tables prefixed with "bg-" and another prefixed with "fg-". Then just change your schedulers configs to point at the appropriate tables. I know it might be a little awkward but you did say you wanted to keep it simple :).
Related
I'm developing a web app in JEE technology with WildFly as production server. Il must make some tasks autorun every day for performing somes operations in the databases.
But I've never do it before with JEE technology. If someone could help me.
Thanks.
If you want to create a task/job which needs to run on every day as per schedule. I would suggest you consider the following options.
Database Events / Triggers:
Seems you have highlighted that you need some database operations right so my option is Event / Triggers where you can define the operations and schedule that when do you want run the task.
Quartz:
My second option is quartz schedular where you can configure your existing java Class or Servlets to run as per the time which you set in the Quartz configuration.
Java Thread Executor:
We can achieve your spec by using java thread itself where you can use Java Thread Executor Framework which provides many options and flexibility than traditional java. Anyhow, I'm recommending to use either option 1 or 2 consider it as the last case since we should maintain at least a thread to be live forever.
Note: I have just given the redirection to proceed so please go ahead and explore the concepts and pick the right one based on the cons and props of the approaches also suite to your spec.
Refer: the points discussed here.There are few things related to your spec covered
We have a Spring Integration application which is polling a mongodb:inbound-channel-adapter like so:
<int-mongodb:inbound-channel-adapter channel="n2s.mongoResults"
collection-name="entities"
query="{_id: {$regex: 'mpl/objectives'}})">
<!-- Run every 15 minutes -->
<int:poller fixed-rate="900000"/>
</int-mongodb:inbound-channel-adapter>
Everything works fine. However, this application is deployed to a cluster and so multiple servers are running the same poller. We'd like to coordinate these servers such that only one runs the pipeline.
Of course, the servers don't know about each other, so we probably need to coordinate them through a locking mechanism in a database. Any suggestions on how to achieve this?
Notes:
We have access to both a MongoDB database and an Oracle database in this workflow. From the perspective of the workflow, it makes more sense to lock on the Oracle database.
It's fine if all server execute the polling step and then one server locks to actually process the records, if that's easier to achieve.
Any suggestions on how to achieve this?
You could use distributed locking tool like like Zookeeper. Another alternative would be to change from a simple fixed trigger to a scheduling framework like Quartz which will ensure that the job only executes on a single node.
It's fine if all server execute the polling step and then one server
locks to actually process the records, if that's easier to achieve.
Yea that's what I would do. I think it's by far the easiest approach. See this post for details on how to do locking with Oracle.
There are several options, including:
Set auto-startup="false" and use some management tool to monitor the servers and ensure that exactly one adapter is running (you can use a control-bus or JMX to start/stop the adapter.
Run the application in SpringXD containers; set the module count for the source (containing the mongo adapter) and the XD admin will make sure an instance is running. It uses Zookeeper to manage state.
Use a distributed lock to ensure only one instance processes messages. Spring Integration itself comes with a RedisLockRegistry for such things or you can use any distributed lock mechanism.
I am looking for a pattern and/or framework which can model the following problem in an easily configurable way.
Every say 3 minutes, I needs to have a set of jobs kick off in a web application context that will concurrently hit web services to obtain the latest version of data, and push it off to a database. The problem is the database will be being heavily used to read the data from to do tons of complex calculations on the data. We are currently using spring so I have been looking at Spring Batch to run this process does anyone have any suggestions/patterns/examples of using Spring or other technologies of a similar system?
We have used ServletContextlisteners to kick off TimerTasks in our web applications when we needed processes to run repeatedly. The ServletContextListener kicks off when the app server starts the application or when the application is restarted. Then the timer tasks act like a separate thread that repeats your code for the specified period of time.
ServletContextListener
http://www.javabeat.net/examples/2009/02/26/servletcontextlistener-example/
TimerTask
http://enos.itcollege.ee/~jpoial/docs/tutorial/essential/threads/timer.html
Is refactoring the job out of the web application and into a standalone app a possibility?
That way you could stick the batch job onto a separate batch server (so that the extra load of the batch job wouldn't impact your web application), which then calls the web services and updates the database. The job can then be kicked off using something like cron or Autosys.
We're using Spring-Batch for exactly this purpose.
The database design would also depend on what the batched data is used for. If it is for reporting purposes, I would recommend separating the operational database from the reporting database, using a database link to obtain the required data from the operational database into the reporting database and then running the complex queries on the reporting database. That way the load is shifted off the operational database.
I think it's worth also looking into frameworks like camel-integration. Also take a look at the so called Enterprise Integration Patterns. Check the catalog - it might provide you with some useful vocabulary to think about the scaling/scheduling problem at hand.
The framework itself integrates really well with Spring.
I'm building a personal Web application using Java EE 6 Technologies (the container is an application server, Jboss AS 7). I'm starting from scratch to create
repetitive background tasks, I identified two possible scenarios :
Scheduled tasks (e.g, sending bulk mails every sunday night)
Trigger tasks based on web event (e.g,running some long background updates from a web action)
What I want to avoid (I don't know if is posible) is to have some background task scattered around my platformm (some of them using cron, others using TimerTask, db jobs, etc..) becoming difficult to maintain.
What are the different approaches to handling repetitive background tasks in a Java web application, taking into account the two previous requirements ?
Related:
Scheduled Tasks for Web Applications
Scheduled task in a web application?
With EE6 you can get rid of Quartz for almost all situations using the TimerService with #Timeout annotations.
And you dont need to write a line of XML to get it working.
There is a nice example in the EE Night Hacks book, also available as source here.
You can add a Timeout method to a bean processing your trigger web events. This way, they can be maintained in one place. You can also modify the timer settings by trigger events.
I'd still look at Quartz also. I can't comment on TimerService with EE6 as a substitute as I haven't used it, but I found Quartz to be quite useful.
When I used it (quite a few years ago now), it had a config file that closely resembled what you'd find for cron. You could use that to call whatever methods you need to perform your scheduled jobs, and then simply provide some other mechanism to call the method on demand.
i had a look on the struts plugins list here and wasn't able to find a plugin to do background/scheduled tasks.
What i want to do, is run a daily task that pulls files from a few servers. I'd like this task to be run from within the web app, so that my importer gets access to all the data classes, also it would be less complicated IMO.
Any common way to go about this?
Thanks
Your best bet is probably Quartz which provides a way to define jobs, and a number of triggers, including cron like expressions. It can be embedded in your app.
Together with Struts2 I use the Spring Plugin for this Issue.
With Spring you can easy define Timer Tasks or Quartz Jobs to execute
Background Jobs.