I have an web application which accept some data from user to create a task, then the task should be executed.
Since the execution of the Task is to download something from the internet which will cost some time, so I tried to create a new Thread to do the job.
This is my idea:
create a LoaderThread used to download data. And the LoaderThread hold a field of ArrayList used for put the Task.
A Servlet to handle the request and response.
When the Servlet startup, start the LoaderThread
During the servlet run, add task to the LoaderThread.
This is the code(some of them is omitted):
public class RwdServlet extends HttpServlet {
private StaticMapLoader loader;
#Override
public void init() throws ServletException {
super.init();
loader = new StaticMapLoader();
loader.startRunning();
}
#Override
protected void doGet(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
Task t=createTask(req);
loader.addTask(t);
}
#Override
public void destroy() {
loader.stopRunning();
}
}
public class StaticMapLoader extends Thread {
private List<Task> tasks = new ArrayList<Task>();
private boolean running = false;
#Override
public void run() {
while (running) {
if (tasks.size() > 0) {
Task t = tasks.get(0);
log.info(t);
if (t != null && t.status == Status.waiting) {
tasks.remove(0);
t.status = Status.running;
downLoad(t);
}
}
}
}
private void downLoad(Task t) {
//download file
}
public void addTask(Task t) {
tasks.add(t);
}
public void startRunning() {
running = true;
this.start();
}
public void stopRunning() {
running = false;
this.interrupt();
}
}
The above code worked, but I found that even the tasks were empty and there are no new task added, the loop will keep running.
So I though if I can make the LoaderThread suspend when there are no tasks, and notify it when new task come out.
So I tried this:
#Override
public void run() {
while (running) {
if (tasks.size() > 0) {
Task t = tasks.get(0);
log.info(t);
if (t != null && t.status == Status.waiting) {
tasks.remove(0);
t.status = Status.running;
downLoad(t);
}
} else {
try {
wait();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
I tried to call the wait() if the tasks is empty.
But I do not know how to wake up it?
Also, is there anything I should know to improve the application?
BWT, is it possible that more than one LoaderThread instance will be created? If so , how to avoid it?
It seems that I can use other implementation, but I wonder if my case is refactor-able?
Since I want to learn some thing I have missed. :) Thanks.
Your requirements are the standard usage of ExecutorService, so I would recommend you to use ExecutorService and not reinvent the wheel.
Base on code you provided, your servlet should look like this:
public class RwdServlet extends HttpServlet {
private ExecutorService loader;
#Override
public void init() throws ServletException {
super.init();
loader = Executors.newCachedThreadPool();//or use some other executor, google about difference between them
}
#Override
protected void doGet(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
Task t=createTask(req); //assume that task implements Runnable or Callable
loader.submit(t); // submit a task to executor after this line your task will start execution in another thread
}
#Override
public void destroy() {
loader.shutdown();//this will destroy executor service but before that it will wait until all already submitted tasks will be executed
}
}
See link with example
Your use case calls for an ExecutorService and you have started reimplementing it from scratch. Better stop now and use the finished, bug-free, flexible, and powerful product from the standard library.
Related
I am trying to write a test for my android app that communicates with a cloud service.
Theoretically the flow for the test is supposed to be this:
Send request to the server in a worker thread
Wait for the response from the server
Check the response returned by the server
I am trying to use Espresso's IdlingResource class to accomplish that but it is not working as expected. Here's what I have so far
My Test:
#RunWith(AndroidJUnit4.class)
public class CloudManagerTest {
FirebaseOperationIdlingResource mIdlingResource;
#Before
public void setup() {
mIdlingResource = new FirebaseOperationIdlingResource();
Espresso.registerIdlingResources(mIdlingResource);
}
#Test
public void testAsyncOperation() {
Cloud.CLOUD_MANAGER.getDatabase().getCategories(new OperationResult<List<Category>>() {
#Override
public void onResult(boolean success, List<Category> result) {
mIdlingResource.onOperationEnded();
assertTrue(success);
assertNotNull(result);
}
});
mIdlingResource.onOperationStarted();
}
}
The FirebaseOperationIdlingResource
public class FirebaseOperationIdlingResource implements IdlingResource {
private boolean idleNow = true;
private ResourceCallback callback;
#Override
public String getName() {
return String.valueOf(System.currentTimeMillis());
}
public void onOperationStarted() {
idleNow = false;
}
public void onOperationEnded() {
idleNow = true;
if (callback != null) {
callback.onTransitionToIdle();
}
}
#Override
public boolean isIdleNow() {
synchronized (this) {
return idleNow;
}
}
#Override
public void registerIdleTransitionCallback(ResourceCallback callback) {
this.callback = callback;
}}
When used with Espresso's view matchers the test is executed properly, the activity waits and then check the result.
However plain JUNIT4 assert methods are ignored and JUnit is not waiting for my cloud operation to complete.
Is is possible that IdlingResource only work with Espresso methods ? Or am I doing something wrong ?
I use Awaitility for something like that.
It has a very good guide, here is the basic idea:
Wherever you need to wait:
await().until(newUserIsAdded());
elsewhere:
private Callable<Boolean> newUserIsAdded() {
return new Callable<Boolean>() {
public Boolean call() throws Exception {
return userRepository.size() == 1; // The condition that must be fulfilled
}
};
}
I think this example is pretty similar to what you're doing, so save the result of your asynchronous operation to a field, and check it in the call() method.
Junit will not wait for async tasks to complete. You can use CountDownLatch to block the thread, until you receive response from server or timeout.
Countdown latch is a simple yet elegant solution and does NOT need an external library. It also helps you focus on the actual logic to be tested rather than over-engineering the async wait or waiting for a response
void testBackgroundJob() {
Latch latch = new CountDownLatch(1);
//Do your async job
Service.doSomething(new Callback() {
#Override
public void onResponse(){
ACTUAL_RESULT = SUCCESS;
latch.countDown(); // notify the count down latch
// assertEquals(..
}
});
//Wait for api response async
try {
latch.await();
} catch (InterruptedException e) {
e.printStackTrace();
}
assertEquals(expectedResult, ACTUAL_RESULT);
}
In Java, all the standard blocking methods can be interrupted by calling Thread.interrupt(), but what if we have Java bindings wrapping a native library which does its own I/O? How then should the native code hook into the Thread and respond to calls to Thread.interrupt()?
Example code
For a full writeup, including runnable sample code, see https://github.com/NWilson/javaInterruptHook.
How does Thread.interrupt() work?
In Sun's JRE (and OpenJDK), interrupt() is able to wake up a few low-level operations itself, such as waiting on a monitor (Object.wait()), and provides an internal hook for higher-level code to be notified of the interruption. This is provided through JavaLangAccess.blockedOn(), which can actually be called directly by your code through sun.misc.SharedSecrets, if you're willing to use Sun-specific implementation details.
As far as I can tell, there's only one public, documented way to register for that notification, which is using java.nio.channels.spi.AbstractSelector. This is a partial implementation of Selector which does the dirty work for you of hooking up the JavaLangAccess.blockedOn() notification to the selector's wakeup() method.
How to implement this all
Implement AbstractSelector to make your own selector; most of the methods are not relevant so just ignore them and put in stubs to shut up the compiler. When you are about to enter your JNI blocking method, call the AbstractSelector begin() method, then call end() when the JNI call has returned.
The selector should cancel the JNI method in its implementation of wakeup().
Rough code
(For a full running example see the github repo linked at the top.)
class NativeTask {
public NativeTask() {}
public void doTask()
throws InterruptedException {
NativeTaskSelector selector = new NativeTaskSelector(this);
try {
selector.registerStart();
doTask0(); // native method
if (Thread.interrupted())
throw new InterruptedException();
} finally {
selector.registerEnd();
try { selector.close(); } catch (IOException impossible) {}
}
}
/* The long-running native operation. */
native private void doTask0();
public void wakeupTask() { wakeupTask0(); }
/* A way to cause the native operation to wake up. */
native private void wakeupTask0();
}
class NativeTaskSelector extends AbstractSelector {
protected NativeTaskSelector(NativeTask task_) {
super(null);
task = task_;
}
public void registerStart() { begin(); }
public void registerEnd() { end(); }
final private NativeTask task;
#Override
public Selector wakeup() {
task.wakeupTask();
return this;
}
#Override
protected void implCloseSelector() throws IOException {
}
#Override
protected SelectionKey register(AbstractSelectableChannel arg0, int arg1,
Object arg2) {
throw new UnsupportedOperationException();
}
#Override
public Set<SelectionKey> keys() {
throw new UnsupportedOperationException();
}
#Override
public int select() throws IOException {
throw new UnsupportedOperationException();
}
#Override
public int select(long arg0) throws IOException {
throw new UnsupportedOperationException();
}
#Override
public int selectNow() throws IOException {
throw new UnsupportedOperationException();
}
#Override
public Set<SelectionKey> selectedKeys() {
throw new UnsupportedOperationException();
}
}
I was going through a expert(?) tutorial on "Asynchronous processing support in Servlet 3.0" (http://www.javaworld.com/javaworld/jw-02-2009/jw-02-servlet3.html?page=2). In it there is this following code snippet:
#WebServlet(name="myServlet", urlPatterns={"/slowprocess"}, asyncSupported=true)
public class MyServlet extends HttpServlet {
public void doGet(HttpServletRequest request, HttpServletResponse response) {
AsyncContext aCtx = request.startAsync(request, response);
ServletContext appScope = request.getServletContext();
((Queue<AsyncContext>)appScope.getAttribute("slowWebServiceJobQueue")).add(aCtx);
}
}
#WebServletContextListener
public class SlowWebService implements ServletContextListener {
public void contextInitialized(ServletContextEvent sce) {
Queue<AsyncContext> jobQueue = new ConcurrentLinkedQueue<AsyncContext>();
sce.getServletContext().setAttribute("slowWebServiceJobQueue", jobQueue);
// pool size matching Web services capacity
Executor executor = Executors.newFixedThreadPool(10);
while(true)
{
if(!jobQueue.isEmpty())
{
final AsyncContext aCtx = jobQueue.poll();
executor.execute(new Runnable(){
public void run() {
ServletRequest request = aCtx.getRequest();
// get parameteres
// invoke a Web service endpoint
// set results
aCtx.forward("/result.jsp");
}
});
}
}
}
public void contextDestroyed(ServletContextEvent sce) {
}
}
Since, this was an expert 'this-is-how-it-is-done' article I am assuming that they wouldnt be writing some silly code.
But I am unable to accept having a 'while(true)' loop inside 'contextInitialized()'.
As per my understanding, this context listener thread would never exit the 'contextInitialized()' method. Right ?
What happens to any other Servlets I have in my application ? Are they free to service requests ? Or does this bring down the entire servlet container ?
1) I would change the code as following:
#WebServletContextListener
public class SlowWebService implements ServletContextListener, Runnable {
private volatile boolean running;
ExecutorService executor;
Thread runner;
Queue<AsyncContext> jobQueue;
public void contextInitialized(ServletContextEvent sce) {
jobQueue = new ConcurrentLinkedQueue<AsyncContext>();
sce.getServletContext().setAttribute("slowWebServiceJobQueue", jobQueue);
// pool size matching Web services capacity
executor = Executors.newFixedThreadPool(10);
runner = new Thread(this);
runner.start();
}
public void run() {
running = true;
while(running)
{
try {
if(!jobQueue.isEmpty())
{
final AsyncContext aCtx = jobQueue.poll();
executor.execute(new Runnable(){
public void run() {
ServletRequest request = aCtx.getRequest();
// get parameteres
// invoke a Web service endpoint
// set results
aCtx.forward("/result.jsp");
}
});
}
}
catch (InterruptException e) {
}
}
}
public void contextDestroyed(ServletContextEvent sce) {
running = false;
runner.interrupt();
executor.shutdown();
}
}
2) Other Servlets shouldn't be affected as long as they have a different urlPattern.
I am using an AbstractScheduledService with a scheduler. A simple pattern like:
class MyService extends AbstractScheduledService {
// KEEP THIS VAR IN MIND W.R.T the SHUTDOWN_HOOK BELOW
public static volatile boolean keepRunning = true;
protected void startUp() throws Exception {
// startup stuff
}
protected void runOneIteration() throws Exception {
// main logic stuff
}
protected void shutDown() throws Exception {
// shutdown stuff
}
protected Scheduler scheduler() {
return Scheduler.newFixedRateSchedule(0, 1, TimeUnit.SECONDS);
}
}
Now, I want to implement a typical shutdown hook like this: (the below snippet will be in main method)
final Thread mainThread = Thread.currentThread();
LOGGER.debug("Adding the shutdown hook");
Runtime.getRuntime().addShutdownHook(new Thread() {
#Override
public void run() {
keepRunning = false;
LOGGER.debug("The shutdown hook was engaged. Setting keepRunnning to false.");
try {
// Is this appropriate?
mainThread.join();
} catch (InterruptedException e) {
// handle exception here
}
}
});
The shutdown hook is from typical docs example. It doesn't seem to work well with the Guava services pattern since the service itself is running on a different thread.
My service has a polling loop in the runOneIteration() logic. I want it to complete it's current task at hand and then shutdown gracefully when is sees that the keepRunning is now false since the Shutdown hook was engaged some time in the recent past when it was busy with its current iteration.
Any ideas how this can be done gracefully (complete current iteration and then shutdown)?
Wouldn't you just call stopAndWait()?
Runtime.getRuntime().addShutdownHook(new Thread() {
#Override
public void run() {
service.stopAsync().awaitTerminated(60, TimeUnit.SECONDS);
}
});
I have a simple Question:
I have a Thread named rlMF. I created it this way:
public Thread rlMF = new Thread(new Runnable() {
public void run() {
reloadMissingFiles();
stopTh();
}
public void stopTh() {
activityStopped = true;
}
});
Now i want to call the stopTh Function from outer Thread. Why can't i simply call rlMF.stopTh(); and what can i do else?
Example:
protected void onPause() {
Log.d("Info", "destroying...");
activityStopped = true;
rlMF.stopTh();
super.onPause();
}
Is not working...
Because the interface accessible is from Thread. In order to have you method accessible from out, you need to specify a type that exposes this method.
And if you take a look carefully the method is implemented in the instance of Runnable. Not even in Thread.
You could have something like this if you really need to access the Runnable object:
class MyRunnable implements Runnable {
public void run() {
...
}
public void fooBar() {
...
}
}
public void someMethod() {
MyRunnable myRunnable = new MyRunnable();
Thread thread = new Thread(myRunnable);
...
myRunnable.fooBar();
...
}
An example for Francisco approach, besides what you are trying to achieve. Maybe this can point you in the right direction
public class CustomRun implements Runnable {
public void run() {
reloadMissingFiles();
stopTh();
}
public void stopTh() {
activityStopped = true;
}
}
In your Code
// start thread with custom runner
CustomRun runner = new CustomRun();
new Thread(runner).start();
// call your stopTh method on CustomRun class
protected void onPause() {
Log.d("Info", "destroying...");
activityStopped = true;
runner.stopTh();
super.onPause();
}
Your goal is to interrupt the thread from onPause. There are several ways to do it, but essentially, you will need to include some interruptibility in reloadMissingFiles.
Option 1
You can use a boolean flag like you did - you need to declare it as volatile to make sure the changes are visible across threads:
private volatile boolean activityStopped = false;
public void reloadMissingFiles() {
while (!activityStopped) {
//load small chunks so that the activityStopped flag is checked regularly
}
}
public Thread rlMF = new Thread(new Runnable() {
public void run() {
reloadMissingFiles(); //will exit soon after activityStopped has been set to false
}
});
protected void onPause() {
//This will stop the thread fairly soon if the while loop in
//reloadMissingFiles is fast enough
activityStopped = true;
super.onPause();
}
Option 2 (better approach)
I don't know what you do in reloadMissingFiles, but I suppose it is some sort of I/O operations, which are generally interruptible. You can then have an interruption policy where you stop as soon as an InterruptedException is caught:
public void reloadMissingFiles() {
try {
//use I/O methods that can be interrupted
} catch (InterruptedException e) {
//cleanup specific stuff (for example undo the operation you started
//if you don't have time to complete it
//then let the finally block clean the mess
} finally {
//cleanup (close the files, database connection or whatever needs to be cleaned
}
}
public Thread rlMF = new Thread(new Runnable() {
public void run() {
reloadMissingFiles(); //will exit when interrupted
}
});
protected void onPause() {
runner.interrupt(); //sends an interruption signal to the I/O operations
super.onPause();
}
Note: you can also read this article for a more in depth version of it.