I have cron setup in appengine project:
<?xml version="1.0" encoding="UTF-8"?>
<cronentries>
<cron>
<url>/cron/someurl</url>
<description>cron</description>
<schedule>every monday 8:00</schedule>
<timezone>Asia/Singapore</timezone>
</cron>
</cronentries>
I am getting the error:
com.google.apphosting.api.DeadlineExceededException: This request (40811df3b6350a70) started at 2012/11/26 00:00:00.404 UTC and was still executing at 2012/11/26 00:09:59.917 UTC.
It's a 1-minute limit to run the task? I though cron doesn't have that limit. How to avoid the error in cron entry?
Thanks.
An HTTP request invoked by cron can run for up to 10 minutes, as per the documentation. If you notice the exception log closely in the HH:MM:SS value, you will find that a total of 10 minutes have passed since the job was started.
You might want to look at your code to see why it is taking that long. In case you have requirements that make your tasks run longer than 10 minutes, I suggest that you look at trapping the exception and then inserting another request to run a job with some request parameter that tells the job to start from where it left off last time.
Alternately, you could also look at Backends.
Also you can try using task queues (10 minute maximun) and re-enqueue when you are near to time limit, using for that any state variable stored in datastore, or if you are depending of a iteration in datastore you can pass the datastore cursor to the other re-enqueued task. For me that works very well.
Related
I want to poll a directory at a specific time each day. Right now I am using timer component with period=86400000 to run periodically every 24 hrs. But I wanted to start the route at a specific time and run for only weekdays mon-fri and not on weekends. Any help would be appreciated. Thanks
You can use the camel-quartz2 component for this.
From Camel Docs:
USING CRON TRIGGERS Quartz supports Cron-like expressions for
specifying timers in a handy format. You can use these expressions in
the cron URI parameter; though to preserve valid URI encoding we allow
+ to be used instead of spaces.
For example, the following will fire a message every five minutes
starting at 12pm (noon) to 6pm on weekdays:
from("quartz2://myGroup/myTimerName?cron=0+0/5+12-18+?+*+MON-FRI")
.to("activemq:Totally.Rocks");
which is equivalent to using the cron expression
0 0/5 12-18 ? * MON-FRI
You can get more usages of the cron scheduler at http://www.quartz-scheduler.org/documentation/quartz-2.3.0/tutorials/crontrigger.html
I have a Java GAE instance that runs TaskQueue jobs. In the last 48 hours I have been getting lots of CancellationException errors (about 36 in a 24 hour period).
The error always occurs on a DataStore query to a particular table. The query is on an indexed column, projectId,and the query is a composite OR (i.e. projectId = -1 or projectId = 23). The entire table in question has 96 records and any one query will pull back 20 records at most.
Does anyone have any idea why I'm getting these problems
Thanks
Paul
Update: Sorry my mistake, this isn't part of a Task Queue call it is the code that sets up the task queue call. The time the call took is on average about 59.5 secs (thanks for pointing that out Alex). Looking at the trace (thanks Alex) it seems that the delay is in the datastore call. It can be as quick as 21ms or over 59000ms, which is way to long. I just checked the logs again and there hasn't been any of those exceptions in the last 24 hours so this might be a 48 hour glitch.
Understanding "CancellationException: Task was cancelled" error while doing a Google Datastore query
Based on this question, it sounds like CancellationException means that you are hitting the 10-minute timeout that TaskQueue Jobs have to complete execution.
Do your logs show how long these requests are running for before they throw an exception (for example stackdriver shows that this deferred task ran for 902ms)?
Querying a table of 96 records should be fine, but it's hard to be certain without being able to see what the code of your task looks like.
You could split your task up into smaller 'sub-tasks' and have sub-task launch the next sub-task before returning.
EDIT: you could also try going here https://console.cloud.google.com/traces/traces to see a break down of where your request is spending its time
What else do I have to do, other than making a cron.xml file for scheduling ? I am getting the same exception:
java.security.AccessControlException: access denied
("java.lang.RuntimePermission" "modifyThreadGroup")
as I was getting before.
This is my cron.xml :
<?xml version="1.0" encoding="UTF-8"?>
<cronentries>
<cron>
<url>/tw</url>
<description>Tweet every half an hour</description>
<schedule>every 30 minutes from 8:00 to 17:00</schedule>
</cron>
</cronentries>
/tw is the servlet that has a doGet method which uses java.util.Timer to schedule the task.
You have to not use Timer (or anything else that uses threads via the normal Java API). AppEngine doesn't allow creation of additional threads as part of a request except via its own special interface (and they're not allowed to outlive the request).
The point of crons is that they're already called once for each time they're supposed to happen. You don't need to do any further "scheduling" in the servlet - just do what you want to happen when the cron fires.
I think you're doing this a bit wrong. Cron is used to specify at what intervals/time a servlet will be called. So, your actual servlet needs to do the work (i.e. send a tweet) and cron service will make sure it's called at correct times.
You can use threads, subject to restrictions described in The Sandbox. But you may not need to use threads at all. Schedule the work using a Push Queue. To reduce platform overheads, AppEngine overprovisioning may start either your cron task or your queued task multiple times, so your logic may need to take extra precautions to avoid sending duplicate tweets.
I know that I need to set up all my crons in cron.xml file. I now want to set the schedules dynamically i.e. in a database table so that I can tweak them with a simple update.
For example instead of having the below:
<cron>
<url>/cron/task1</url>
<description>Task 1 </description>
<schedule>every 5 minutes</schedule>
</cron>
I want to have something like:
<cron>
<url>/cron/task1</url>
<description>Task 1 </description>
<schedule>${TASK1_SCHEDULE}</schedule>
</cron>
where TASK1_SCHEDULE will be an entry from the database.
Is that possible ? Or is there any other way to achieve that programmatically?
You cannot dynamically modify the cron.xml file to accept new jobs or modifications to the list of jobs.
I agree with Martin in that you write a Cron Job that executes at the threshold interval that you can tolerate for your application i.e. 1 min or 2 minutes. In that, I would suggest manage a custom Job definition entity that contains various parameters depending on your needs, including the time for next execution.
In your Cron Job, once the time for next execution has passed, you should use a TaskQueue to execute the functionality, rather than running each one on our own and waiting for it to complete.
Take a look at the following article that provides a similar framework for doing so: http://pisarenko.net/blog/2013/09/03/creating-dynamic-task-scheduler-on-appengine/
Write a custom scheduler because you cannot change cron.xml from your code. Call your scheduler at the shortest interval you require. Within your scheduler, query the database for tasks that are due and execute them.
I am building an app for which I need to set up cron jobs. What I want to do is to set the specific minutes in a hour where specific crons should run. For instance:
Task1 at 1st minute of the hour
Task2 on every second minute of the hour
Task3 every 2 minute only in the second half of the hour
Building this in the standard Unix cron format is reasonably straightforward, but could not figure out how to do it in the Google-App-Engine.
The documentation does not list any non-trivial examples. Any suggestions on how to do it? Examples would be excellent.
The documentation you linked to seems to indicate that it isn't possible to do what you want using only Cron for Java (unless they have an undocumented feature for it). In particular this doesn't appear to allow for multiple times.
time specifies the time of day, as HH:MM in 24 hour time.
The Python version says the exact same thing.
However, one solution (albeit somewhat more expensive in terms of CPU usage) would be to call a URL every minute, and from the handler for that URL, dispatch out to whatever other calls you need.
In other words, something like:
<?xml version="1.0" encoding="UTF-8"?>
<cronentries>
<cron>
<url>/run-scheduled-tasks</url>
<description>Run all scheduled tasks</description>
<schedule>every 1 minutes</schedule>
</cron>
</cronentries>
Then in run-scheduled-tasks, check a database for when each task last run, and if your complex condition for triggering them has occurred since then.
If the documentation is correct you can't get as granular as you are wanting. Doesn't look like they support picking a particular minute of the hour. Or a subset of an hour.
You might have to get creative. Why do you need such specific timing?
This may seem silly. Write three servlet. And schedule them from another UNIX machine on the other part of the world :D. Or even you can write a java app to do it. enjoy
Have a look at Quartz and see if that'll solve your problem.