I have recently used Quartz scheduler for running some background processes (eg. report generation) that involves database access. Now, this background tasks runs in separate threads. However, the database access in done in a central point in my application, and something like
System.err.println("CURRENT THREAD: "+ Thread.currentThread().getName());
prints "main", before executing the actual query. Therefore, i am thinking that the database read is done by the UI thread ("main" thread). This is also confirmed by the fact that the UI loses (in part) his responsiveness. What i want to do, if possible, is to run every database access in a separate thread, so that a Cancel button can be implemented to a query request, if that will become necessary. So, i imagine having this workers:
UI thread: responsible for UI;
Database access thread: responsible for data retrieving from the db;
Scheduled background tasks: responsible for everything else (and using the DB access thread).
Can this be realized? Or, is there a better alternative to my approach?
P.S. I do not want to use at this point some existing frameworks, that address this issue (Hibernate, Spring, etc). I just need a home-brew, working solution.
Stack trace:
at com.mycompany.myproduct.core.db.SQL.executeQuery(SQL.java:260)
at com.mycompany.myproduct.core.db.Database.getAbstractDBObjects(Database.java:285)
at com.mycompany.myproduct.core.db.Database.getAbstractDBObjects(Database.java:305)
at com.mycompany.myproduct.core.util.job.DummyJobProcessor$1.run(DummyJobProcessor.java:61)
at org.eclipse.swt.widgets.RunnableLock.run(Unknown Source)
at org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Unknown Source)
at org.eclipse.swt.widgets.Display.runAsyncMessages(Unknown Source)
at org.eclipse.swt.widgets.Display.readAndDispatch(Unknown Source)
at com.mycompany.myproduct.core.ui.layers.AbstractView.open(AbstractView.java:915)
at com.mycompany.myproduct.mycompany.open(mycompany.java:196)
at com.mycompany.myproduct.mycompany.main(mycompany.java:365)
It seems like your db queries are run from the UI thread even though you say that a scheduled task runs the queries (?). To be sure, you can print out a stacktrace where you access the database:
new Exception().printStacktrace();
Over to your proposed solution: it seems like a decent design if your db queries are slow and otherwise would freeze your UI. You could implement an event system between your UI layer and the db layer, perhaps a simple queue based approach.
EDIT:
There are most likely examples to find how to implement a event based solution.
Disclaimer: I haven't done any real UI programming in many years.
The user clicks a button in the UI. The UI thread puts an event object (DataWantedEvent) on a queue (java.util.Queue), changes a label ("Waiting for data...") and then goes on and waits for other user interactions.
The db layer thread takes the event form the queue and queries the database. The result is posted back on another queue in a result object.
A UI thread (not the main thread probably) takes the result object from the result queue and updates the UI.
A queue for posting result objects back to the UI might not be needed. An update method could be invoked directly.
If the user clicks a cancel button, the update event/callback could be ignored or, if possible, the db query could be cancelled.
You can have a listener implementation that directly updates the UI. Something like a function callback, that gives an event to the UI thread.
That way , your UI thread can continue serving the input request, assuming you business logic does not require it to be a synchronous db request.
You could always do it asynchronously.
HTH.
Related
I need to wait for a condition in a Spring MVC request handler while I call a third party service to update some entities for a user.
The wait averages about 2 seconds.
I'm calling Thread.sleep to allow the remote call to complete and for the entities to be updated in the database:
Thread.currentThread().sleep(2000);
After this, I retrieve the updated models from the database and display the view.
However, what will be the effect on parallel requests that arrive for processing at this controller/request handler?
Will parallel requests also experience a wait?
Or will they be spawned off into separate threads and so not be affected by the delay experienced by the current request?
What are doing may work sometimes, but it is not a reliable solution.
The Java Future interface, along with a configured ExecutorService allows you to begin some operation and have one or more threads wait until the result is ready (or optionally until a certain amount of time has passed).
You can find documentation for it here:
http://download.oracle.com/javase/6/docs/api/java/util/concurrent/Future.html
I'm new to jersey, jsp's and web application development in general so hopefully this isn't a silly question. I've got a jsp and currently when the user hits a button on it, it starts a HTTP request which takes about 5-10 minutes to return. Once it finishes they're redirected to another page.
I'm wondering, is it possible or even advisable that I multithread the application so that the heavy processing will start but the user get's redirected to the next .jsp right away. If multithreading is not possible is there another method that you would recommend for dealing with heavy processing in a web application?
A JSP is basically a Servlet (it's translated in a Java Servlet Class and compiled). Teoretically you can start a new thread in a servlet (and hence in a JSP, via scriptlet), but that's really not advised for multiple reasons.
It'd be better recommended to make an asynchronous HTTP call via ajax, then, once the call is done immediately show something else to the user, and when the call back returns display the results.
Rather than create a new thread each time it might be more efficient to have a worker thread which continually polls a shared queue. Using, for example, ArrayBlockingQueue you web request can simple add an object to the queue and return to the user, and your worker thread (or repeating scheduled job) can take care of the heavy weight processing.
Instead of waiting for process to complete in a JSP, you can create a TimerTask (or Quartz Job) and set it for immediate execution and redirect user to some other page. Have that Job store the result in some central place that can be accessed by another JSP (in case you want to pull result of Job later, may be through ajax) Doing so, you save yourself from managing threads manually (which is error prone), you get async functionality, user does not need to see the blank browser screen for around 5-10 minutes.
It is possible.
Create a thread, store its reference somewhere that is available everywhere (a static Map) and store its key (in the session, in the code of the JSP's answer).
Following calls can retrieve the thread and check its state/results.
Anyway, use with care:
a) You will need to control that old results are deleted. It is inevitable that sometimes the browser will close, so you need a Watchdog to clear data obviously no longer needed.
b) The user are not used to this kind of behavior. There is a serious risk that they will just "go back" and try to launch the thread again, and again, and again. Try to control it (ideally the id of the thread will be linked to the user, so as long as an older thread is active an user cannot launch another one).
I'm new to the idea of Threading, but not asynchronous behavior. My Android app is taking ~180 millisecond to start up and ~550 milli when I use GoogleAnalytics trackViewPage method and MobFoxView constructor. Coming from Actionscript 3, anything that "took time" was automatically async and I was forced to handle it with listeners which is a bit different in Android it appears. It seems I'M responsible for deciding when something should be asynchronous. So I guess my question is, HOW do I decide what should be async? Is it by milliseconds of executing? But perhaps that changes greatly between devices. Perhaps it should be by ... or is it by ....?
You need to know one important thing - by default everything you do without starting separate thread is executed on "main" thread (also knows as UI-thread).
If you do something, which can block - your UI will lag and users will suffer.
If you doing something, which is not about UI but about database query, network call or potentially long blocking operation - you need to start thread directly or use AsyncTask.
Also you must note, if you try to do something with UI (e.g. set value to a TextView) from not-main thread you will fail. UI can be acessed only from UI-Thread.
I'm not sure if I'm on the right track, but I have previously created a Thread and stored it in the current session to be able to use it in other Servlets.
When a user logs in, I need a Thread that will periodically poll a webservices server and retrieve updates. When the user logs out I want to stop/pause the Thread.
If I create the Thread when the user logs in, I need a reference to this Thread if I want to pause it when the user is logged out, and then start it again when they login.
What is the best and correct way to go about this? Should I monitor the session from inside the Thread and if the user has logged out, have the thread return in order to end the Thread?
Then next time the user logs in, I can create a new background Thread?
***I need the thread to run in the background and poll a web services server for updates which I use to update my database. It may poll once every 5 minutes.
I don't want it to be polling all the time, hence why I was trying to control it based on the user being logged in or not... I was setting the boolean running to false when they log out in order to stop the thread. Because the thread was stored in the application context, I could then set running to true when they logged back in.
What about a Thread to run in the background and check something, for example that your internet is up?
I know how to create the Thread, but I'm confused about how to terminate it, since it needs to be continually running in the background.
Am I using threads for the wrong thing? should I be using something else?
Thanks again, hopefully this is a better explanation...
Since servlets themselves will be instantiated multiple times at the same time, creating even more threads is not a wise idea. You will run into problems if you decide to cluster your application one day across multiple machines because of serialization issues, so you're locking yourself at the beginning.
Besides, it seems your problem can be done using AJAX pooling from client side. Why pool a webservices server and retrieve updates if you can't show them to your client without them refreshing the entire page?
Think of this scenario: if a user sessions lasts for 30 minutes, and a user doesn't log out, but just closes the browser window: you won't know that this happened, and you would pool for next 30 minutes that webservice, which is a waste of resources.
A Thread is not serializable, so storing it in a session is likely to be problematic.
Also, stopping or pausing a thread can be problematic unless you code the thread's Runnable to periodically check to see if it has been paused / stopped and act accordingly. (The Thread.stop() and Thread.pause() methods are deprecated and you should avoid using them. They can potentially destabilize your JVM.)
Even if you address the above issues, there are a host of resource management issues to be addressed.
I'm not sure if I'm on the right track
In short, you are not, IMO.
You have not explained why you need to associate a thread with each session, or what these threads are intended for. If you did, you would get some good suggestions for alternative approaches that (largely) avoid the problems above.
FOLLOWUP
I wasn't using .stop and .pause.
Good.
i was using a boolean "runnable" to control the thread.
That's one alternative. I prefer to use the Thread.interrupt() mechanism, because it allows you to interrupt a thread that is blocked on an I/O operation or in a wait(...) call to stop.
What if you are polling a device, and need to access returned data? What is the best approach?
Probably, the best approach is to create a thread or thread pool in servlet initialization that is responsible for polling the device(s). If you need to save the returned data so that it can be accessed in a subsequent HTTP request, put it into (say) a queue that the request controller can access. You may need some infrastructure to remove uncollected data objects from the queue after a certain period, etc.
You really shouldn't be creating Threads inside containers at all. What's it for?
And if it's for use by other servlets it's more likely to belong in the application attributes, not the session attributes, isn't it? or are you really creating a thread per user? which you really don't want to do.
I can see, that there may be valid reasons to do the polling server side in a background thread: For example it may take longer than a user wants to wait for the answer of a request. And there is the issue that some anti-discrimination laws (to be more precise the guidelines on how to interpret those laws) deny the use of JavaScript, so no Ajax.
So if there is really no other way, I suggest to create one thread (or a very small thread pool) in Servlet.init(). This single (or small number of threads) should do the polling for all active users. In web.xml you can define a session listener so you can keep track of opened and closed/expired sessions.
Make sure to signal in Servlet.destroy() that the thread(s) should exist.
To define a session live cycle listener add this in web.xml:
<listener>
<listener-class>
com.example.SessionListener
</listener-class>
</listener>
And then remember the open sessions, so that you can iterate over them in the background thread.
public class SessionListener implements HttpSessionListener {
private Set<HttpSession> sessions = Collections.synchronizedCollections(
new HashSet<HttpSession>());
public void sessionCreated(HttpSessionEvent sessionEvent) {
HttpSession session = sessionEvent.getSession();
sessions.add(session;
}
public synchronized void sessionDestroyed(HttpSessionEvent sessionEvent) {
HttpSession session = sessionEvent.getSession();
sessions.remove(session);
}
}
It's about an application which is supposed to process (VAD, Loudness, Clipping) a lot of soundfiles (e.g. 100k). At this time, I create as many worker threads (callables) as I can put into memory, and then run all with a threadPool.invokeAll(), write results to file system, unload processed files and continue at step 1. Due to the fact it's an app with a GUI, i don't want to user to feel like the app "is not responding" while processing all soundfiles. (which it does at this time cause invokeAll is blocking). I'm not sure what is a "good" way to fix this. It shall not be possible for the user to do other things while processing, but I'd like to show a progress bar like "10 of 100000 soundfiles are done". So how do I get there? Do I have to create a "watcher thread", so that every worker hold a callback on it? I'm quite new to multi threading, and don't get the idea of such a mechanism.
If you need to know: I'm using SWT/JFace.
You could use an ExecutorCompletionService for this purpose; if you submit each of the Callable tasks in a loop, you can then call the take method of the completion service - receiving tasks one at a time as they finish. Every time you take a task, you can update your GUI.
As another option, you could implement your own ExecutorService that is also an Observable, allowing the publication of updates to subscribing Observers whenever a task is completed.
You should have a look at SwingWorker. It's a good class for doing lengthy operations whilst reporting back progress to the gui and maintaining a responsive gui.
Using a Swing Worker Thread provides some good information.