I am trying to use executor framework in Google App engine. Bellow is the code that I am trying to run.
Thread thread = ThreadManager.createBackgroundThread(new Runnable(){
public void run(){
try{
LOGGER.info( "Checking background thread");
Thread.sleep(10);
}
catch (InterruptedException ex){
throw new RuntimeException("Exception:", ex);
}
}
});
ScheduledThreadPoolExecutor executor = (ScheduledThreadPoolExecutor) Executors.newScheduledThreadPool(10, ThreadManager.backgroundThreadFactory());
executor.scheduleAtFixedRate(thread, 0, 30, TimeUnit.MINUTES);
But this doesn't start the thread. But if I use thread.start() it works properly. I have checked Whitelisted Classes and it does provide Executor classes. So where I am doing it wrong ?
Saikat,
You should always try to avoid creating threads on App Engine, because of it's distributed and dynamic nature it tends to have really bad/unexpected results.
In your case multiple instances will spawn multiple (local) threads sending many times the same notification. Also, bear in mind GAE front end instances have a 1 minute request limit, so after that time the server will kill that request.
Fortunately App Engine provides the Cron service for exactly this situations.
The Cron Service will allow you to schedule a job to run at a given time or every given period. When the cron is triggered GAE will call a configured URL so you can do your process, in your case send notifications.
Eg:(from the link provided)
<cron>
<url>/weeklyreport</url>
<description>Mail out a weekly report</description>
<schedule>every monday 08:30</schedule>
<timezone>America/New_York</timezone>
</cron>
will make an HTTP request to /weeklyreport every monday #8:30.
Related
Several jobs get executed on a machine. For each job, I have to call an external service (plus do some other work). The number of times the external service has to be called for a job is not known beforehand, or even when the job starts as the job processing involves streaming input data.
Before starting the job, I have to create certain number of threads for processing - as part of the processing it calls the external service.
Say the service supports only 100 TPS (transactions per second) and maximum 10 jobs can run on a machine
and I do not want to bombard the service.
The problem is in deciding the number of worker threads to be allocated for each job. Since in the worst case 10 jobs could run at a time, I could allocate 10 threads for each job. But if only one job is running with 10 threads I would be under-utilizing the service.
I think I could use PoolingHttpClientConnectionManager for pooling the threads and use them to access the service. I could use a DefaultMaxPerRoute and MaxTotal as 100.
Advantage: When only one job is running, I can make full utilization of the external service.
The problem is when 10 jobs are running, 1000 threads will be competing to call the service. Is creating the default 100 threads per job be fine or would the resource contention lead to problems?
UPDATE:
Learnt that the client I will be using must not have a connection request timeout to checkout/lease a connection from the pool else it would throw a ConnectionPoolTimeoutException.
I think PoolingHttpClientConnectionManager only as pool contain connections because It's only one thread. And thread for connect to url in your code control that ex:
public String doPost(HttpPost postMethod, ProfilerLog profilerLog, LogBuilder logBuilder)
throws Exception {
try {
// execute thread
ConnectorThread thread = new ConnectorThread(postMethod, httpClient, "Connector Thread",
profilerLog, logBuilder);
thread.setDaemon(true);
thread.start();
// tunnel response
thread.join();
return thread.getRespData();
} catch (Exception ex) {
LOGGER.error(ex.getMessage(), ex);
return null;
} finally {
shutdown();
}
}
ConnectorThread execute target url.
Regards.
I am trying to use fixed thread pool using executor framework. Each runnable instance submitted to the executor is worker thread which process a java result set. For every iteration of the result set I have to call rest webservice which used the oauth token. The oauth token is to be refreshed after every 50 min and need to be shared among all the runnable submitted to the executor.I am also using scheduled executor which execute after every 50 minutes but sometimes it is called correctly and some time not due to which the rest web service fails as it used the expired token in its header. I need to ensure that the scheduled service must be called after every 50 min without fail. But I am stuck on this. Also I need some mechanism that after the group of rest web service call is completed then only the new web service calls should be made while iterating the result set.
ThreadPoolExecutor executorPool = (ThreadPoolExecutor) Executors.newFixedThreadPool(2);
WorkerThread wt1=new WorkerThread(conn,Queries.getAddressInfo("AL"),oauth_token,restTemplate);
WorkerThread wt2=new WorkerThread(conn,Queries.getAddressInfo("AK"),oauth_token,restTemplate);
executorPool.execute(wt1);
executorPool.execute(wt2);
ScheduledFuture scheduledFuture =
scheduledExecutorService.schedule(new Runnable() {
public void run() {
System.out.println("token service");
String url="";
try {
url = WebServicePropertyFileReader.getOauthUrl()+String.format(urlToGetOauthToken, WebServicePropertyFileReader.getClientId(),
WebServicePropertyFileReader.getClientSecret());
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
Layer7Token token=restTemplate.postForObject(url, null, Layer7Token.class);
GlobalTokenAccessor.oauth_token=token.getAccessToken();
}
},
50,
TimeUnit.MINUTES);
There are two issues :
Schedule schedules it only one time.
You should use scheduleAtFixedRate to schedule periodic tasks. There is no guarantee that the thread will get scheduled within 50 minutes. So, you may want to schedule the refresh every 49 minutes or so.
Secondly, you should control the access to the shared variable though a synchronized write and read methods. Otherwise, the read thread may read incorrect values.
I have a web application that contains a java bean for executing a potentially long-running job. I'd like to find a way that I can identify when a thread has been executing for a very long time and then potentially kill it if necessary.
My application runs in Glassfish 3 so I am on Java 1.6. I am just looking for a solution to a potential problem in the future.
EDIT:
To be clear I am looking for something like a tool or utility to monitor a running web application.
Use an Executor Service.
ExecutorService executor = Executors.newSingleThreadExecutor();
Future<String> future = executor.submit(new Runnable(){ ....});//pass your runnable
And then you can wait for a specified time:
try {
int timeOut = 5;
//Waits if necessary for at most the given time for the computation to
// complete, and then retrieves its result, if available.
future.get(timeout, TimeUnit.SECONDS)
} catch (TimeoutException e) {
System.out.println("TimedOut!");
}
executor.shutdownNow();
I am trying to implement comet approach for Facebook like browser push notifications in following manner:
HTTP request is made to a simple Servlet using AJAX, it gets submitted to java.util.concurrent.Executor (using AsyncContext) and HTTP thread gets freed immediately. This request is picked by a background thread.
This background worker thread in the thread pool keeps checking for new notification every 30 seconds (Queries database). If there is any notification, a response it sent. Else after certain timeout i.e. 5 minutes, a "No Notification" response is sent. Client immediately makes a new request and the process is repeated.
Problem here is that I am using Thread.sleep(30000) for periodically checking the data.
There is still one thread occupied per request. This time (30 sec) is wasted and the thread remains un-available to cater any other request.
Is there any technique which I can use, to return the thread to pool immediately after it has checked for new notification? And then some other available thread from the pool does the same after 30 secs to check notifications and so on?
Code goes like this:
// Creation of a global async Executor on ServletContextListener. contextInitialized
Executor executor =
new ThreadPoolExecutor(1, 1, 50000L,
TimeUnit.MILLISECONDS,
new LinkedBlockingQueue<Runnable>(100));
// Delegate the request to async thread
executor.execute(new RunnableClass(asyncContext));
// In RunnableClass.run() method
while(timeout5Min)
{
boolean newNotificationPresent = checkIfNotificationPresent(reqId);
if(!newNotificationPresent)
Thread.sleep(30000);
}
// send response
Can ScheduledThreadPoolExecutor somehow be used in such case? Any other technique?
If you are looking for Thread efficiency, you should look at actors.
Have a look at Akka
Otherwise, don't sleep, schedule a recheck using a java.util.Timer
How can I create a timeout for each command that is running in parallel using java.util.concurrent.ExecutorService?
My code is something like this:
For example in the code below I need obj1 run for maximum 1 min, and obj2 for 2 mins and others 5 mins.
ExecutorService exService;
exService = Executors.newCachedThreadPool();
exService.execute(obj1);
exService.execute(obj2);
exService.execute(obj3);
exService.execute(obj4);
exService.shutdown();
boolean finshed = exService.awaitTermination(5, TimeUnit.MINUTES);
if (finshed) {
//Doing something
}
EDIT:
Unfortunately the class of obj1 - obj4 is scraping some web pages using WebHarvest that uses jakarta HttpClient for reading web pages and HttpClient (And neither WebHarvest itself) doesn't have any feature for timeout on entire page reading and/or scraping job.
This is my time consuming task and I thought about killing ExecutorService thread after a timeout to handle this problem.
In general, there is no reliable way to make a separate thread quit. In particular, there is no reliable way to interrupt and stop your task after a timeout from outside that task. What you need to do is make the tasks themselves responsible for stopping after their time runs out. Depending on what they do, you might be able to abstract this behaviour into a superclass something like:
public abstract class TimeoutRunnable implements Runnable {
private final long timeLimitMillis;
private long startTimeMillis;
public TimeoutRunnable(long timeLimitMillis) {
this.timeLimitMillis = timeLimitMillis;
}
public final void run() {
startTimeMillis = System.currentTimeMillis();
while (System.currentTimeMillis() - startTimeMillis < timeLimitMillis) {
runIteration();
}
}
protected abstract void runIteration();
}
Then in your subclass override, runIteration() and perform a single "step" of the task.
The only reasonably reliable way to kill a task is to run it in a separate process and kill that process if it times out. Using any other approach with a library which does not support timeouts is likely to be error prone at best.
from my point of view I think that such stuff requires some more robust foundations than plain Java standard classes , that 's why I would suggest to use any scheduler infrastructure (Quartz or any other project) which may gives you handles (job identifiers) to kill your time consuming tasks .
You may have something like this :
Main Thread launches the Quartz Scheduler , receive subscriptions from different jobs
saying : Job1,Job 2 and TimeCheckerJob
TimeCheckerJob would be a forever job ,notified to any new job and would check for living time for each any new job... In this job you would have to deal with start time of each job, beware of the OS clocks and don't try to setup too hard constraints (nanoseconds is pure fiction).
HTH
My 2 cents
Jerome