We need to monitor the status of quartz jobs by only having access to the database. Does quartz offer a way to achieve this?
Reading the documentation, in the Trigger class we have TriggerState which defines various states, ERROR and COMPLETE among others.
But during regular execution, the CompleteExecutionInstruction is always NOOP. It doesn't matter if I throw an exception or execute completely. I was expecting something like COMPLETE if I was successful or ERROR if I threw an exception.
Does anyone have any experience with this?
Trigger state in no way reflects the last execution status of your job so you cannot expect the trigger state to be COMPLETE or ERROR.
If you want to monitor your jobs, I recommend that you look into the Quartz JobListener interface. This interface allows you to intercept and process all job execution events that you may be interested in. For example, you may want to implement a job listener that stores the intercepted job execution events in a database and then you can query this database to obtain the last job execution status of your jobs etc.
You may also consider using an existing Quartz scheduler, job and trigger management and monitoring tool. I am the original author of QuartzDesk which is one of these tools. QuartzDesk is a completely non-intrusive tool that does not require you to modify your application code in any way, and it can work with all types of Java applications and nearly all Quartz versions. There is a public online demo that you can register to, try and see for yourself if the tool satisfies your requirements. When it comes to monitoring, the QuartzDesk GUI displays a so-called health indicator next to each job and trigger. This indicator depicts the status (success, error, veto) of the last 10 job/trigger executions so you can quickly see if there are any jobs that failed recently etc. You can also access complete execution history of individual jobs and trigger. If you want to be notified of any job execution related event, you can create an execution notification rule that will send you a notification message (IM, SMS, email, Slack, HipChat, SNMP Trap, ...) when a configured condition is met. I am attaching a screenshot of the GUI showing job health indicators and the selected job's execution history.
.
Related
I am working on a new functionality for a multi-tenancy web-app, which allows the admin to start a potentially very long running process (ca. 1 - 5 min) by the click of a button in the admin panel.
However it is crucial that such a task can only be executed ONCE at a time for each tenant. Of course we can disable the button after a click, but we cannot prevent the admin (or another admin) from opening another browser tab and clicking the button again.
Is there any existing library which allows us to:
Uniquely identify a job (e.g. by an id like "tenant_001_activation_task")
Start the task in the background
Query if such a task is already running in the background and if so reject any further calls to this function.
I already had a look into quartz and the Spring TaskExecutor. However these two seem to mainly focus on scheduling tasks at a given time (like a cronjob). What I'm looking for is a solution for running and monitoring a background job at any time programmatically.
If you decide to use Quartz, you can simply annotate the relevant job implementation classes with the #DisallowConcurrentExecution annotation.
Please note that this annotation is effective on the Quartz job detail level, not on the job implementation class level. Let us say you have a job implementation class com.foo.MyTenantTask and you annotation this class with the #DisallowConcurrentExecution annotation. Then you register 2 jobs that use this job implementation task - tenant_001_task and tenant_002_task.
If you run tenant_001_task and tenant_002_task, they will be allowed to run concurrently because they are different jobs (job details). However, if you attempt to run multiple instances of tenant_001_task concurrently, Quartz will only execute the first instance and the other instances will be queued up and wait for the first instance to finish executing. Then Quartz will pick one queued instance of tenant_001_task and execute it and so on until all queued up instances of tenant_001_task have been executed.
On the other hand, Quartz will not prevent concurrent execution of tenant_001_task and tenant_002_task instances since these represent different jobs (job details).
Quartz provides various (local, JMX, RMI) APIs that allow you to obtain the list of currently executing jobs, list of all registered jobs and their triggers etc. It will certainly allow you to implement the scheduling logic you described.
If you are building an app to manage and monitor your Quartz jobs, triggers etc., I recommend that you take a quick look into our product called QuartzDesk. It is a management and monitoring GUI for all types of Java Quartz-based applications. There is a public online demo and if you want to experiment locally, you can request a 30-day trial license key. If you need to interact with your Quartz schedulers programmatically (and possibly remotely), you can use various JAX-WS service APIs provided by QuartzDesk.
I'm working on a relatively simple workflow using Amazon's Flow framework for Java. I think I have a decent grasp of everything that's going on right now, but I have one area I'm still uncertain about: how should I go about handling timeouts?
The main timeout with my workflow is the executionStartToCloseTimeoutSeconds on the workflow itself, but I'd imagine the process is the same regardless of which timeout fires. It seems that most of the time, when the task times out, it just kind of disappears. I'd like to be able to know when this happens and do something (e.g. send an e-mail or log it somehow). I searched around and couldn't find any example of anything being notified that a timeout happened.
Activity timeout is delivered to the workflow code in the form of an Exception and can be easily handled.
IMHO workflow execution timeout is similar to kill -9 in Unix. It kills workflow without giving it chance to perform cleanup. So the main use for it is to ensure that broken workflow instances do not stay open forever.
For all business level timeouts do not rely on workflow timeouts, use timers instead. When timer fires your workflow code can execute notification activity and terminate the workflow with appropriate failure status.
http://docs.aws.amazon.com/amazonswf/latest/developerguide/swf-timeout-types.html
For activity related timeouts, the short answer is that your decider (i.e. workflow) logic should handle it. You should not have to worry about things timing out once you validate the logic and have retries in place.
For workflow timeouts you will need to inspect the workflow history / state to figure out that it timed out. You can definitely list workflow executions but you probably have to go through the SWF API directly (i.e. not through Flow). You want to do this anyway to catch failed workflows.
A pattern I've used and seen being used with SWF is to have an external way of keeping track of the work you've dispatched through SWF (think a DB) and use that to check in on work that was started and never completed. The workflow itself updates this when it completes (or as it's completing major pieces of work) so it's trivial to figure out which workflows are problematic.
Asynchronous jobs such as download scores from the website, or send emails after completion of some critical tasks.
Rightnow we when we download some scores, we have to wait on the current page to get the response page or to get file downloaded.
Is there a possibility that i can click on download scores and it happens in the background so that i can navigate to other parts
of the website, and in the mean-time check the status of the job. Or Schedule some job later in the future and get its execution results
via email.
Ours is a struts 2 webapplication with Hibernate 3.5 ORM. After browsing into some java scheduling libraries, got some info on Quartz.
But is Quartz the right library for the above requirements or any other library that i can try for?
Please guide me in the right direction.
You will need some sort of asynchronous processing support. You can use:
quartz-scheduler - this library is very comprehensive and allows you to schedule all sorts of jobs. If you want to use it only for the purpose of scheduling jobs in the background and run them immediately, might be an overkill
use thread pool, see Executors class
jms queue can listen on requests and process them asynchronously in mdbs
Finally you can take advantage of #Async/#Asynchronous support in spring or ejb
Then you mut somehow restore the results. Depening on whether you want to deliver them directly in the browser or via e-mail:
every time you are rendering a page, check whether there aren't any completed/in progress jobs. If there are some completed jobs, display an extra link on the page somewhere (sort of notification). If the job is in progress, start an ajax request and ask every other second or use long-polling/comet to receive the result immediately
if you want to send results by e-mail, just send it after the job finishes. Much simpler but less user-friendly IMHO.
Quartz is certainly one way to do that - and works well if you want to schedule a job to run at a particular time or with a particular frequency.
If you just want to kick something off in the background in response to a user action, and check its status, there are a few other ways to do it which may be better suited to this pattern:
the java.util.concurrent package: you can set up a ThreadPoolExecutor and submit tasks to it that implement Callable. You get back a Future<T> object that you can check for completion (isDone) and get its result when complete (get).
with EJB or Spring, there is also a concept of a (session) bean method being #Async or #Asynchronous, which return a Future<T> as well and behave as above. Basically this just abstracts away the thread-pool creation and management from your code, and moves it into the container or framework.
I have a number of backend processes (java applications) which run 24/7. To monitor these backends (i.e. to check if a process is not responding and notify via SMS/EMAIL) I have written another application.
The old backends now log heartbeat at regular time interval and this new applications checks if they are doing it regularly and notifies if necessary.
Now, We have two options
either run it as a scheduled task, which will run after every (let say) 15 min and stop after doing its job or
Run it as another backend process with 15 min sleep time.
The issue we can foresee right now is that what if this monitor application goes into non-responding state? So, my question is Is there any difference between both the cases or both are same? What option would suit my case more?
Please note this is a specific case and is not same as this or this
Environment: Java, hosted on LINUX server
By scheduled task, do you mean triggered by the system scheduler, or as a scheduled thread in the existing backend processes?
To capture unexpected termination or unresponsive states you would be best running a separate process rather than a thread. However, a scheduled thread would give you closer interaction with the owning process with less IPC overhead.
I would implement both. Maintain a record of the local state in each backend process, with a scheduled task in each process triggering a thread to update the current state of that node. This update could be fairly frequent, since it will be less expensive than communicating with a separate process.
Use your separate "monitoring app" process to routinely gather the information about all the backend processes. This should occur less frequently - whether the process is running all the time, or scheduled by a cron job is immaterial since the state is held in each backend process. If one of the backends become unresponsive, this monitoring app will be able to determine the lack of response and perform some meaningful probes to determine what the problem is. It will be this component that will then notify your SMS/Email utility to send a report.
I would go for a backend process as it can maintain state
have a look at the quartz scheduler from terracotta
http://terracotta.org/products/quartz-scheduler
It will be resilient to transient conditions and you only need provide a simple wrap so the monitor app should be robust providing you get the threading stuff right in the quartz.properties file.
You can use nagios core as core and Naptor to monitoring your application. Its easy to setup and embed with your application development.
You can check at this link:
https://github.com/agunghakase/Naptor/tree/ver1.0.0
I need to schedule a task that will perform the task at the given time by user. But this scheduling I need to run whether the application is running or not. So how could I specify the scheduling using quartz?
I am writing the code for the situation in servlet and then from where I need to run that servlet I am bit confused about it, because if I would use the load-on-startup it will call servlet every time the application is loaded so it will cause the job detail to be duplicated in data tables. And the scheduling will stop when user will logout the session. But I want the scheduling to be running till the tomcat is running.
Any help is appreciated. Thanks in advance.
To avoid duplication of the job in the database you can write your code to first check for the job before scheduling, or to first delete the job (whether or not it exists) before scheduling.
In 2.x API there is a checkExists() set of methods.