Asynchronous jobs such as download scores from the website, or send emails after completion of some critical tasks.
Rightnow we when we download some scores, we have to wait on the current page to get the response page or to get file downloaded.
Is there a possibility that i can click on download scores and it happens in the background so that i can navigate to other parts
of the website, and in the mean-time check the status of the job. Or Schedule some job later in the future and get its execution results
via email.
Ours is a struts 2 webapplication with Hibernate 3.5 ORM. After browsing into some java scheduling libraries, got some info on Quartz.
But is Quartz the right library for the above requirements or any other library that i can try for?
Please guide me in the right direction.
You will need some sort of asynchronous processing support. You can use:
quartz-scheduler - this library is very comprehensive and allows you to schedule all sorts of jobs. If you want to use it only for the purpose of scheduling jobs in the background and run them immediately, might be an overkill
use thread pool, see Executors class
jms queue can listen on requests and process them asynchronously in mdbs
Finally you can take advantage of #Async/#Asynchronous support in spring or ejb
Then you mut somehow restore the results. Depening on whether you want to deliver them directly in the browser or via e-mail:
every time you are rendering a page, check whether there aren't any completed/in progress jobs. If there are some completed jobs, display an extra link on the page somewhere (sort of notification). If the job is in progress, start an ajax request and ask every other second or use long-polling/comet to receive the result immediately
if you want to send results by e-mail, just send it after the job finishes. Much simpler but less user-friendly IMHO.
Quartz is certainly one way to do that - and works well if you want to schedule a job to run at a particular time or with a particular frequency.
If you just want to kick something off in the background in response to a user action, and check its status, there are a few other ways to do it which may be better suited to this pattern:
the java.util.concurrent package: you can set up a ThreadPoolExecutor and submit tasks to it that implement Callable. You get back a Future<T> object that you can check for completion (isDone) and get its result when complete (get).
with EJB or Spring, there is also a concept of a (session) bean method being #Async or #Asynchronous, which return a Future<T> as well and behave as above. Basically this just abstracts away the thread-pool creation and management from your code, and moves it into the container or framework.
Related
We need to monitor the status of quartz jobs by only having access to the database. Does quartz offer a way to achieve this?
Reading the documentation, in the Trigger class we have TriggerState which defines various states, ERROR and COMPLETE among others.
But during regular execution, the CompleteExecutionInstruction is always NOOP. It doesn't matter if I throw an exception or execute completely. I was expecting something like COMPLETE if I was successful or ERROR if I threw an exception.
Does anyone have any experience with this?
Trigger state in no way reflects the last execution status of your job so you cannot expect the trigger state to be COMPLETE or ERROR.
If you want to monitor your jobs, I recommend that you look into the Quartz JobListener interface. This interface allows you to intercept and process all job execution events that you may be interested in. For example, you may want to implement a job listener that stores the intercepted job execution events in a database and then you can query this database to obtain the last job execution status of your jobs etc.
You may also consider using an existing Quartz scheduler, job and trigger management and monitoring tool. I am the original author of QuartzDesk which is one of these tools. QuartzDesk is a completely non-intrusive tool that does not require you to modify your application code in any way, and it can work with all types of Java applications and nearly all Quartz versions. There is a public online demo that you can register to, try and see for yourself if the tool satisfies your requirements. When it comes to monitoring, the QuartzDesk GUI displays a so-called health indicator next to each job and trigger. This indicator depicts the status (success, error, veto) of the last 10 job/trigger executions so you can quickly see if there are any jobs that failed recently etc. You can also access complete execution history of individual jobs and trigger. If you want to be notified of any job execution related event, you can create an execution notification rule that will send you a notification message (IM, SMS, email, Slack, HipChat, SNMP Trap, ...) when a configured condition is met. I am attaching a screenshot of the GUI showing job health indicators and the selected job's execution history.
.
I have a user workflow where at a specific time a webservice is called, and the results are presented to the user.
According to the search request and the queried results, I want to perform some database updates and statistic logging.
As the workflow pauses while the webservice is requested, I thought about creating some kind of background thread that performs these database actions, while the user can already continue the workflow without having to wait for database actions to complete.
Do you think this is a good practice? How could I create such onetime running background threads?
If you only want to run in the background, then an Executor service is a good solution.
If you need to ensure that queued requests survive events like a server restart, then you need a persistent queue like a JMS Queue. There are some nice, free open source JMS implementations that serve this purpose.
If service call teakes little time (say 1 or 2 seconds) then it is a waste of time to develop such feature.
If it takes significant amount of time you should do this in background.
I am doing a web application which has Java as a front end and shell script as a back end. The concept is I need to process multiple files in the back end. I will get the date range from the user (for example from July 1st-8th) and for each day process around 100 files. So in total I have 800 files to process.
I will get these details from JSP and delegate a background call to shell script and get back the results and display the same to the user.
Now I did all these in a sequential approach - by which I mean without threads. So there is only one main thread that executes and the user has to wait till 800 files are processed sequentially. However this is really slow. And because of this I am thinking of going for threads. Since I am a beginner of threads, I read a some stuffs regarding this and I have come up with the following idea:
As I read threads work have to be split .. I thought of splitting the
8 day work to 4 threads where each thread would perform 2 day work
I would like to know whether I am following a correct approach and my major concerns are:
Is it recommended to spawn multiple threads from a web application
Whether or not this is a good approach
Some guidance of how to proceed with this. An example instance would be great. Thank you.
Yes, you can run the long processing job in multi-threaded or in any high performance environment. You should also you Servlet 3.0 Asynchronous Request Processing to suspend the request thread and wait till the Long processing task is done.
Yes, there's nothing wrong with spawning multiple threads from a web application. In fact, if you're running a Servlet container (which you most likely are since you're using Java), it's already spawning multiple threads for you. In general a Servlet container will automatically spawn a new thread (or reuse one out of a pool) to handle each request it receives.
Your approach is fine, thought you'll want to fine-tune the number of threads to something that is suitable given the hardware configuration of your system and the amount of concurrent load on your web service. Also note that while spinning up a bunch of threads will reduce the total amount of time needed to process all the data, it will still leave a potentially large chunk of time before any data is ready to go back to the user. So you might get a better result by doing smaller work units sequentially, and posting each batch of results to the user-interface as soon as it is ready. Then it will still take a long while for the user to have all the data, but that can start viewing at least a portion of it almost immediately.
The way to improve user experience is not by parallelizing at Servlet level on 100000 threads but rather to provide incremental rendering of the view. First of all it would be useful to separate your application in multiple layers, according to the MVC pattern for example.
Saying that, you will have to look on how
Create a service that is able to return partial answers and a last answer, meaning that all available data has been returned. Each of this answers can be computed in parallel to improve performance.
Fill a web page incrementally, tipically by calling back this service which returns a JSON string you use to add data to the DOM. Every time you get an answer, if this is a partial answer, you call again the service providing the previous sequence number.
If you look to Liligo to understand, you will see how this is works. The technique I described is known as polling, but there are others technique to obtain similar asynchronous results at UI Level. In general, you don't want to work directly with the Servlet API, which is a very low level API,but rather use a reasonable framework or abstraction for that.
If you want a warm advice, you should have a look to the Play! framework http://www.playframework.org/documentation/2.0.2/JavaStream HTTP streaming.
Create threads in a web application is not a good solution. It is a bad design because normally it would be the container (web server) who is charged with that activity. So I think you have to find another solution.
I suggest you putting the shell scripts in cron, scheduled to run each minute, and to "activate" them you can touch files that act as semaphores. At each run the scripts verify if the web application touched the semaphore file, if so they read the date interval from those files and then start to process.
I'm trying to write a Spring web application on a Weblogic server that makes several independent database SELECTs(i.e. they can safely be called concurrently), one of which takes 15 minutes to execute.
Once all the results are fetched, an email containing the results will be sent to a user list.
What's a good way to get around this problem? Is there a Spring library that can help or do I go ahead and create daemon threads to do the job?
EDIT: This will have to be done at the application layer (business requirement) and the email will be sent out by the web application.
Are you sure you are doing everything optimally? 15 minutes is a really long time unless you have a gabillion rows across dozens of tables and need a heckofalot of joins....this is your highest priority -- why is it taking so long?
Do you do the email job at set intervals, or is it invoked from your web app? If set intervals, you should do it in an outside job, possibly on another machine. You can use daemons or the quartz scheduler.
If you need to fire this process off from the web app, you need to do it asynchronously. You could use JMS, or you could just have a table into which you enter a new job request, with daemon process that looks for new jobs every X time period. Firing off background threads is possible, but its error prone and not worth the complication, especially since you have other valid options that are simpler.
If you are asking about Spring support for long-running, possibly asynchronous tasks, you have a choice between Spring JMS support and Spring Batch.
You can use spring quartz to schedule the job. That way the jobs will run in the same container but will not require an http request to trigger them.
I need to design a Swing application, which will need to send out multiple jobs as customer requested. each job is running the same shell scripts which will take 10-30 mins to return a value. (the jobs are not running on application server or as web services. )then the Swing application will need to decide what to do next according to the return value.
my question is if I can use JMS to send out jobs. if not, what do you suggest I look into?
multithreading....
Thank you very much!
Multi-threading is the obvious first approximation here. Take a look at SwingWorker, launch the process in a background thread, monitor the progress (as in show the user if it is still running, perhaps even a view into what is being emitted to the console), etc. These are the obvious choices.
What JMS would solve for you (and you would have to find a light weight JMS implementation that would run on the desktop) is to allow for retries and guarantees that the process runs to completion. Something that takes 20 minutes to run in a shell script doesn't sound like it is a candidate for a retry, but if it is, and it is important that the message really get through instead of just having the thread die and the process forgotten if the user closes the java application, then JMS is the type of thing to look at.
JMS can most certainly be used in a Swing based application. If the shell scripts are to be executed on a server by a service that is listening to the JMS queue and respond on another queue or topic.
There is nothing limiting you from using JMS queues or topics in a desktop application.
JMS is generally used to communicate between processes and between client/server, not really what you're looking for here, unless you're sending them out to a server to be processed, but it doesn't sound like that here. It sounds like you're looking for a work queue, such that a swing app has a button that adds a new task to the queue (where the task is running the shell script). You can then have multiple threads taking tasks of the queue and running the scripts.
You may - or may not - profit from using a job scheduler, like Quartz. Maybe it's overkill, maybe it's just what you need.