We have implemented quartz for scheduling.Every job produced have different key.It was working fine till now. Yesterday we come through a problem as same job is being executed twice or thrice(no particular behaviour) by two different Quartz-Worker threads.
We cant make thread pool size one as we need concurrent jobs.
One noticeable thing about our scheduled job is that it reschedules(daily, weekly or monthly) itself at every run i.e. if a job is scheduled to run daily then it would reschedule itself in next 24 hours but with a random predefined(say 3 hours) time window. For example if a job ran today at 4:10 (i.e between 4:00 and 7:00) then our job will reschedule it self to tomorrow at some random time between 4:00 and 7:00. It may be 4:01 or 6:59 or 5:23 or any other value in the given time window. This process was also working fine and it is still working fine for most of the cases except in some cases where our rescheduling algo is failing to schedule itself in next 24 hours. Instead it is scheduling itself in next 10 sec, 1 hr or any other random value. But it finally stabilizes itself after 2-3 such wrong rescheduling i.e. it finally schedule itself in next 24 hrs. We are suspecting that this may be happening due to the multiple threads accessing the Calendar object (we are using Calendar.getInstance() and cal.add(Calendar.DAY_OF_YEAR, 1) to reschedule the job in next 24 hours). Somehow calendar instance is picking wrong time or is not able to add one day in current time.
So, there are two issues:
1. Multiple Quartz threads acquiring the same job
2. Calendar is not able to add given interval or is picking wrong current time in some particular cases(multiple thread access)
Any help will be appreciated.Reply asap.
Thanks.
Thanks for reply.
I would like to know what is the difference between Statefuljob and #DisallowConcurrentExecution annotation and setting threadPool.threadCount to 1.
Code for rescheduling is as...
Calendar cal = Calendar.getInstance();
Calendar nextCal = Calendar.getInstance();
cal.setTimeZone(TimeZone.getTimeZone(obj.getTimeZone()));
nextCal.setTimeZone(TimeZone.getTimeZone(obj.getTimeZone()));
Date startTime = null;
SimpleTrigger trigger = null;
JobDataMap dataMap = new JobDataMap();
if (repeatTimeInMillis == null) {
cal.set(Calendar.HOUR_OF_DAY, obj.getStartTime());
nextCal.set(Calendar.HOUR_OF_DAY, obj.getStartTime());
cal.set(Calendar.MINUTE, 0);
nextCal.set(Calendar.MINUTE, 0);
cal.set(Calendar.SECOND, 0);
nextCal.set(Calendar.SECOND, 0);
if (obj.getScheduleType() == ScheduleType.MONTHLY) { // Monthly
log.info("in monthly schedule");
nextCal.add(Calendar.MONTH, 2);
nextCal.set(Calendar.DAY_OF_MONTH, obj.getDate());
cal.add(Calendar.MONTH, 1);
cal.set(Calendar.DAY_OF_MONTH, obj.getDate());
} else if (obj.getScheduleType() == ScheduleType.WEEKLY) { // Weekly
log.info("in weekly schedule");
nextCal.add(Calendar.WEEK_OF_YEAR, 2);
nextCal.set(Calendar.DAY_OF_WEEK, obj.getDay());
cal.add(Calendar.WEEK_OF_YEAR, 1);
cal.set(Calendar.DAY_OF_WEEK, obj.getDay());
} else if (obj.getScheduleType() == ScheduleType.DAILY) { // Daily
log.info("in daily schedule");
nextCal.add(Calendar.DAY_OF_YEAR, 2);
cal.add(Calendar.DAY_OF_YEAR, 1);
}
long time = obj.getTimeWindow() * 60 * 60 * 1000;
time = Math.round(time * Math.random());
cal.setTimeInMillis(cal.getTimeInMillis() + time);
startTime = cal.getTime();
nextCal.setTimeInMillis(nextCal.getTimeInMillis() + time);
repeatTimeInMillis = nextCal.getTimeInMillis() - cal.getTimeInMillis();
log.info("Rescheduling job at " + startTime);
trigger = newTrigger().usingJobData(dataMap)
.withIdentity(obj.getScheduleJobName(), obj.getScheduleJobGroup()).startAt(startTime)
.withSchedule(simpleSchedule().withIntervalInMilliseconds(repeatTimeInMillis).repeatForever())
.build();
} else {
log.info("Rescheduling job next " + repeatTimeInMillis + " milliseconds.");
cal.setTimeInMillis(cal.getTimeInMillis() + repeatTimeInMillis);
startTime = cal.getTime();
trigger = newTrigger().usingJobData(dataMap)
.withIdentity(obj.getScheduleJobName(), obj.getScheduleJobGroup()).startAt(startTime)
.withSchedule(simpleSchedule().withIntervalInMilliseconds(repeatTimeInMillis).withRepeatCount(1)).build();
}
The StatefulJob interface and #DisallowConcurrentExecution annotation do the same thing.
From the DisallowConcurrentExecution javadoc:
marks a Job class as one that must not have multiple instances
executed concurrently....
This can be used in lieu of implementing the StatefulJob marker
interface that was used prior to Quartz 2.0
Setting the threadPool.threadCount property to 1 would mean at most 1 job of any type can be executing
Using any of these solutions will stop a job executing concurrently and cause any trigger to be placed into a queue to be executed when the previous trigger instance has completed
Related
I want to execute two Tasks with Java ScheduledExecuter. The tasks may not run asynchron, so I only allow 1 thread.
ScheduledExecutorService scheduler= Executors.newSingleThreadScheduledExecutor();
The schedulers work is mainly to always execute the same task every few seconds. This works perfectly fine.
But there is a problem: The user is able to generate tasks at runtime and schedule them at a specific time he can choose. this works in 99,9% of cases, but if the user decides to execute 2 or more tasks at the exact same time, the scheduler only executes one of them and simply doesnt execute the others. The other tasks never get executed.
Example:
This works:
Calendar toRun = Calendar.getInstance(); //for example 17:00
toRun.set(Calendar.HOUR_OF_DAY, 17);
toRun.set(Calendar.MINUTE, 0);
toRun.set(Calendar.SECOND, 0);
long delay = toRun.getTimeInMillis() - System.currentTimeMillis();
Task_TaskOne t1 = new Task_TaskOne();
scheduler.schedule(t1, delay, TimeUnit.MILLISECONDS));
Calendar toRun2 = Calendar.getInstance(); //for example 17:01
toRun2.set(Calendar.HOUR_OF_DAY, 17);
toRun2.set(Calendar.MINUTE, 1);
toRun2.set(Calendar.SECOND, 0);
long delay2 = toRun2.getTimeInMillis() - System.currentTimeMillis();
Task_TaskTwo t2 = new Task_TaskTwo();
scheduler.schedule(t2, delay2, TimeUnit.MILLISECONDS));
In this case only the first task gets executed, the 2nd one never does:
Calendar toRun = Calendar.getInstance(); //for example 17:00
toRun.set(Calendar.HOUR_OF_DAY, 17);
toRun.set(Calendar.MINUTE, 0);
toRun.set(Calendar.SECOND, 0);
long delay = toRun.getTimeInMillis() - System.currentTimeMillis();
Task_TaskOne t1 = new Task_TaskOne();
scheduler.schedule(t1, delay, TimeUnit.MILLISECONDS));
Calendar toRun2 = Calendar.getInstance(); //for example 17:00
toRun2.set(Calendar.HOUR_OF_DAY, 17);
toRun2.set(Calendar.MINUTE, 0);
toRun2.set(Calendar.SECOND, 0);
long delay2 = toRun2.getTimeInMillis() - System.currentTimeMillis();
Task_TaskTwo t2 = new Task_TaskTwo();
scheduler.schedule(t2, delay2, TimeUnit.MILLISECONDS));
In my opinion this is weird. Shouldnt the scheduler have a que (like TimerTask has) and execute the 2nd task just after the first one?
Also does anyone has an easy fix for it? Multiple threads are not an option.
I’m currently using selenium in a web bot to purchase items on a website. When I search for the item I want to buy and it cannot be found I use driver.navigate().refresh() to refresh the page to see if it is there now, it will keep doing this until it finds the product when it is released on the page. However, I wish to start my bot a few hours before the release of the product which currently doesn’t work as after roughly 30 seconds of refreshing the page I get banned from the page due to the anti-ddos software they use. One option is to increase the delay between refreshing, however I need to catch the release of this product as soon as possible so I’m trying to find a way that my program can wait/sleep until 30 seconds before the release however I’m struggling to find a way to do this.
Just call Thread.sleep with the appropriate amount of milliseconds:
public static void main(String[] args) throws InterruptedException {
long currentTime = System.currentTimeMillis();
long releaseTime = currentTime + 1000 * 60 * 60 * 24 * 3; // 3 days
Thread.sleep(releaseTime - currentTime);
}
Another way would be to use java.time classes:
public static void main(String[] args) throws InterruptedException {
LocalDateTime now = LocalDateTime.now();
LocalDateTime release = LocalDateTime.of(2019, 10, 30, 13, 30);
long sleepDuration = Duration.between(now, release).toMillis();
TimeUnit.MILLISECONDS.sleep(sleepDuration);
}
Java 9 introduces new methods to the Duration class like toSeconds(), toMinutes() and so on.
You could also consider using a ScheduledExecutorService to schedule your tasks. This is especially useful if you have multiple tasks to schedule and don't want having multiple threads being blocked for that:
private static final ScheduledExecutorService service = new ScheduledThreadPoolExecutor(2);
private static ScheduledFuture<?> scheduleTask(Runnable task, LocalDateTime releaseTime) {
Duration duration = Duration.between(LocalDateTime.now(), releaseTime);
return service.schedule(task, duration.toSeconds(), TimeUnit.SECONDS);
}
In general, to sleep until the next Thursday at 10:59 you could use the following code:
LocalDateTime release = LocalDateTime.now()
.with(TemporalAdjusters.nextOrSame(DayOfWeek.THURSDAY))
.withHour(10)
.withMinute(59);
Duration duration = Duration.between(LocalDateTime.now(), release);
TimeUnit.MILLISECONDS.sleep(duration.toMillis());
I think rather than sleeping you should take a look at scheduled tasks with cron expressions in Spring... that way you don't have a blocked thread just sitting there.
Scheduled Tasks with Spring
Cron Expressions
I have a scheduled job which runs every 100 seconds. Sometimes the execution of this method takes a lot of time (which is ok and there is no problem with that). In this situation, the result of the running method is not important to me and I want to re-schedule the job for next 100 second.
What is the best way to force the running job to terminate (return) after a specific time?
My scheduled code is like below:
#Scheduled(fixedDelay = 100*1000)
fun calculateLastDaysStatistics() {
logger.info("affiliate statistics thread Started Successfully")
val processStartDate = Date()
for (i in 1..prevDaysToConsider) {
logger.info("AdZone-Stats prev days $i")
val yesterday = DateUtility.addDay(Date(), -i)
val startDate = DateUtility.getZeroDayTime(yesterday.time)
val endDate = DateUtility.addDay(startDate, 1)
/* This method is probable to take a lot of time */
calculateStatistics(startDate, endDate)
}
val processLength = (Date().time - processStartDate.time) / 1000
logger.info("affiliate statistics thread finished in " + processLength + "s")
}
Thanks.
Try using Fixed Rate instead of Fixed Delay
Here is the article from
Paraschiv.E. The #Scheduled Annotation in Spring. Referred from https://www.baeldung.com/spring-scheduled-tasks
Schedule a Task at a Fixed Rate
#Scheduled(fixedRate = 1000)
public void scheduleFixedRateTask() {
System.out.println(
"Fixed rate task - " + System.currentTimeMillis() / 1000);
}
Note that the beginning of the task execution doesn’t wait for the completion of the previous execution.
This option should be used when each execution of the task is independent.
You can implement a custom Task scheduler using, org.springframework.scheduling.TaskScheduler instead of Annotation based method.
private final TaskScheduler scheduler;
#Autowired
public SchedulingManager(TaskScheduler scheduler) {
this.scheduler = scheduler;
}
In this case,
ScheduledFuture scheduledeFuture = scheduler.schedule(()->{
....You job goes here..
}, new CronTrigger("*/100 * * * * *"));
You can keep track of the scheduled future to make sure it runs max time intended.
scheduledeFuture.get(100,TimeUnit.SECONDS)
I need to execute a task every 5 minutes on my server to update some datas on a db, i've found that on openshift i have the cron that executes some script every tot time. Is it possibile to make a script that makes a simple call to a servlet or to a java code to run this job?
I am quite new to server side programming so please speak easy!
Ps. I am using a Tomcat 6 (Jboss EWS 1.0), mySQL 5.5 server
AS I understand you, you need your application to run sth every XX minutes.
To calculate the start time I made a helper function "getStartTime" With that I can use the human readable time like "23:30" (attention, I am from german, so it is not for AM/PM, just change for your needs).
Helper Method:
private static long getStartTime(String startTime) {
int hour = Integer.parseInt(startTime.split(":")[0]);
int minutes = Integer.parseInt(startTime.split(":")[1]);
Calendar cal = Calendar.getInstance();
Date dateNow = cal.getTime();
cal.set(Calendar.HOUR_OF_DAY, hour);
cal.set(Calendar.MINUTE, minutes);
cal.set(Calendar.SECOND, 0);
if(cal.getTime().before(dateNow)) {
cal.set(Calendar.DAY_OF_MONTH, cal.get(Calendar.DAY_OF_MONTH) + 1);
return cal.getTime().getTime();
} else {
return cal.getTime().getTime();
}
}
Now you can use the ScheduledExecutorService from Java. Example:
ScheduledExecutorService scheduledExecutorService = Executors.newScheduledThreadPool(1);
long startClearRequests = getStartTime(DEFAULT_JOB_START_TIME) - System.currentTimeMillis();
And set your needs into the scheduleAtFiexed Rate:
scheduledExecutorService.scheduleAtFixedRate(clearRequests, startClearRequests, Math.round(DEFAULT_JOB_PERIOD_HOURS * 60 * 60 * 1000), TimeUnit.MILLISECONDS);
For example I use:
private static final int NUM_OF_THREADS = 2;
private static final String DEFAULT_JOB_START_TIME = "23:30";
private static final double DEFAULT_JOB_PERIOD_HOURS = 24;
As you see, you can change the number of threads (depends of what your application is doing), the start time (this is just needed for application start (when to start the job the first time).
And also the period (every XX hour the job shall run ... I took hours, but you need ti insert milliseconds at the end, so for 5 minutes (you have to tak 5 * 60 *1000 miliseconds.
Greetings
EDIT in respect to the athors comments:
To start things on application start, you have several methods. One method is to start a servlet on startup like this. Insert into the web.xml
<servlet>
<servlet-name>ServletStartups</servlet-name>
<servlet-class>model.initialization.ServletStartups</servlet-class>
<load-on-startup>1</load-on-startup>
</servlet>
This will call the Class ServletStartups on Application start (the number in load-on-startup is the priority, because you can have multiple entries and can decide which to start first (1, 2, 3 ...)
Now within your servlet you defines an init() method, which is automatically called, like that:
public class ServletStartups extends HttpServlet{
public void init() throws ServletException{
// HEre you can put your methods as described above //(scheduledExecutorService( ...
}
}
IMPORTANT NOTE:
above I had a method "clearRequests", sorry this was my method, I have not renamed it to add it here. THis method will be called in my application every 24 hours.
the methods you call from the ScheduledExecutorService have to be a callable, like this:
private Runnable clearRequests = new Runnable() {
public void run() {
try {
// Here do your task
} catch (Exception e) {
throw new RuntimeException(e);
}
}
};
I have some processes in jBPM which I test with unit tests (sending events, checking if nodes are triggered, etc).
KnowledgeBuilder knowledgeBuilder = KnowledgeBuilderFactory.newKnowledgeBuilder();
knowledgeBuilder.add(ResourceFactory.newClassPathResource("process.bpmn"), ResourceType.BPMN2);
knowledgeBase = knowledgeBuilder.newKnowledgeBase();
session = knowledgeBase.newStatefulKnowledgeSession();
....
In some of the processes there are fixed timers (for example 3 weeks). Is there a possibility to manipulate the time jbpm is using so that I can simulate that this period of time is already over?
Btw. I don't want to trigger these notes manually or modify the times in it.
I'm using jbpm 5.4.
One possibility is to use the clock of the session and iterate over all actual running timers.
This will not actually shift time, but can be used to cancel timers, which should be fired in a time span.
Example:
public void shiftTime(long timeToShiftInMs) {
long targetTime = System.currentTimeMillis() + timeToShiftInMs + 10; // 10ms ahead ...
JDKTimerService clock = getSession().getSessionClock();
List<TimerJobInstance> jobs = new ArrayList<>();
for (TimerJobInstance job : clock.getTimerJobInstances()) { // go through all jobs
// He keeps already executed timer without nextFirTime
Date nextFireTime = job.getTrigger().hasNextFireTime();
if (nextFireTime != null) {
long jobTime = nextFireTime.getTime();
if (targetTime > jobTime) { // look if it should be fired after this time
jobs.add(job);
}
}
}
for (TimerJobInstance job : jobs) {
job.getJob().execute(job.getJobContext());
clock.removeJob(job.getJobHandle());
}
}