I have an app that could set the time for processing. The problems is when I update the time, the processing will increase. For example:
Initially the timer start at 07:00AM
Let say, I update the timer to 08:00AM then the next day onwards, the program will run again at 07:00AM and also at 08:00AM. (The 07:00AM is still in scheduler, how to remove the 07:00AM?)
How to make the scheduler to only run the 08:00AM the next day?
public void setKonfigurasi(String name, String value) {
log.info(SERVLET_NAME + "Entering setKonfigurasi");
amBean.setParam(name,value); //update the time into database
//name = 'processFileConf|kodPT|userA|20140312 08:30 AM'
// reschedule timer after configured by user
try {
String kodPT = name.substring(name.indexOf("|") + 1, name.indexOf("|",name.indexOf("|") + 1));
String configStr = value.substring(2); //get the new time
String currentStr = CommonUtil.getCurrentDate();
DateFormat dateformat = new SimpleDateFormat("dd/MM/yyyy KK:mm:ss a");
Date currentDate=new Date() ;
Date configDate = dateformat.parse(currentStr+" "+configStr);
long config = configDate.getTime();
long current = currentDate.getTime();
// today
long delay = config-current;
if (delay < 0)
// tomorrow
delay += (1000*60*60*24);
// create the timer and timer task objects
Timer timer = new Timer();
timer.scheduleAtFixedRate(new TimerTask() {
public void run() {
System.out.println("showtime for "+kodPT);
processFile("auto"+kodPT);
}
}, delay, 1000*60*60*24);
ServletContextEvent servletContextEvent = EtServletContextListener.getContext();
ServletContext servletContext = servletContextEvent.getServletContext();
servletContext.removeAttribute ("timer");
servletContext.setAttribute ("timer", timer);
} catch (Exception e) {
log.error("Exception on date format : "+e.getMessage());
}
log.info(SERVLET_NAME + "Exiting setKonfigurasi");
}
You need to call cancel() on the previous timer and create a new one. The javadoc says:
Terminates this timer, discarding any currently scheduled tasks. Does
not interfere with a currently executing task (if it exists). Once a
timer has been terminated, its execution thread terminates gracefully,
and no more tasks may be scheduled on it.
I have found out about how to do what I want. Instead of using java.util.Timer to create the timer, we should use javax.ejb.Timer since the Timer in ejb have an info to identified each timer.
Timer Service EJB
Related
I'm using KafkaSource to read kafka messages of Events type, as per documentation providing event time extractor is optional for source kafka
KafkaSource<Events> source =
KafkaSource.<Events>builder()
.setProperties(kafkaProperties)
.setBootstrapServers(parameters.get("bootstrap-servers-source"))
.setTopics(parameters.get("source-topic"))
.setGroupId("visit-events-flink-mvp")
.setStartingOffsets(OffsetsInitializer.committedOffsets(OffsetResetStrategy.EARLIEST))
//.setStartingOffsets(OffsetsInitializer.earliest())
.setValueOnlyDeserializer(new EventsDeserializationSchema())
.build();
// event stream from kafka source
DataStream<Events> eventStream =
env.fromSource(source, WatermarkStrategy.forMonotonousTimestamps(), "Kafka Source")
//should be a unique id
.uid("kafka-source");
//stream is keyed based on the anonymousId
DataStream<Events> keyedStream =
eventStream.keyBy(Events::getAnonymousId)
// .process(new KeyedProcessing(Long.parseLong(parameters.get("ttl"))))
.process(new KeyedProcessingWithCallBack(Long.parseLong(parameters.get("ttl"))))
.uid("engager-events-keyed-processing");
In my KeyedProcessingWithCallBack, I'm setting event time timer for 60 secs and the call back is not triggering at all.
My kafka source has 8 partitions and I'm running job with parallelism 1
public void processElement(EngagerEvents value, KeyedProcessFunction<String, EngagerEvents, String>.Context ctx, Collector<String> out) throws Exception {
ObjectMapper objectMapper = new ObjectMapper();
JsonNode jsonNode = objectMapper.readTree(value.getEventString());
System.out.println("time : " +jsonNode.get("EVENT_TIMESTAMP").textValue());
if (anonymousIdHasBeenSeen.value() == null) {
System.out.println("time stamp emitting: " +jsonNode.get("EVENT_TIMESTAMP").textValue());
// key is not available in the state
anonymousIdHasBeenSeen.update(true);
System.out.println("TIMER START TIME: " +ctx.timestamp());
out.collect(value.getEventString());
ctx.timerService().registerEventTimeTimer(ctx.timestamp() + (stateTtl * 1000));
}
}
// not getting triggered
#Override
public void onTimer(long timestamp, OnTimerContext ctx, Collector<String> out)
throws Exception {
// triggers after ttl has passed
System.out.println("Call back triggered : time : " +timestamp + " value : " +anonymousIdHasBeenSeen.value());
anonymousIdHasBeenSeen.clear();
}
TEST Simulator code which will send event with anonymousId=111 with different event time stamp
try {
for (int i = 0; i < 500; i++) {
String[] anonymousId = {"111"};
String key = String.valueOf(new Random().nextInt(10));
ProducerRecord<String, String> record = new ProducerRecord<>(
"flink-visits-mvp-test-source",
key,
// getEvent(UUID.randomUUID().toString() + "-" +Thread.currentThread().getName() , event[new Random().nextInt(1)]));
// getEvent(anonymousId[new Random().nextInt(1)], event[new Random().nextInt(1)]));
getEvent(anonymousId[new Random().nextInt(1)],
System.currentTimeMillis(),
event));
//System.out.println(record.value().toString());
producer.send(record);
try {
Thread.sleep(200);
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
}
}
Am I doing something wrong here ? Why is my event time timer call back not trgiggering ?
I also experience the same error with Flink V1.16. The processElement method is invoked as expected however, observed that ctx.timerService().currentWatermark() always print as -9223372036854775808. onTimer method never invoked with 60 seconds timer.
With lots of trial and error I found that invoking env.setParallelism() method resolve the issue.
final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
env.setParallelism(1);
After fix, the currentWatermark() method return correct watermark time and onTimer method is invoked every 60 seconds.
Here is the log entries before fix:
Event Time = 1669610765000, End Of Window Time = 1669610819999,
current water mark time = -9223372036854775808
Event Time = 1669610807000, End Of Window Time = 1669610819999,
current water mark time = -9223372036854775808
Here is the log entries after fix:
Event Time = 1669610393000, End Of Window Time = 1669610399999,
current water mark time = 1669610387999
Event Time = 1669610450000, End Of Window Time = 1669610459999,
current water mark time = 1669610435999
I need to schedule to a new Date for everytime a task it's executed.
I've seen many examples where the period or interval is set by millis and stays for every iteration but i can't find any that takes a date parameter for next execution
I tried the #Scheduled annotation since I am working with Spring, but I do not know if there is a possibility to pass a parameter.
Examples i've seen
Example 1:
#Scheduled(fixedRate = 20000)
public void scheduler() {
log.info("scheduler");
log.info("Current Thread " + Thread.currentThread().getName());
log.info("Current Thread " + Thread.currentThread().getId());
}
Example 2:
ScheduledExecutorService scheduler = Executors.newScheduledThreadPool(1);
scheduler.scheduleAtFixedRate(myRunnable, 10, 10, TimeUnit.MINUTES);
I expect to read a Date from a db table to schedule my task for new iteration
Ty for the help!
Edit
Note: There will also be a time when I need to decide where to stop the next iteration, so I'm trying to call the schedule task by a method
I avoid using Spring, so I cannot help you there. But I can guide you through the use of ScheduledExecutorService to accomplish your goal.
ScheduledExecutorService::schedule( Runnable command, long delay, TimeUnit unit )
You are partially correct about the ScheduledExecutorService: Two of its three scheduling strategies are designed to keep regular intervals between runs:
scheduleAtFixedRate
scheduleWithFixedDelay
But the third strategy lets you set the next run with any amount of delay you wish.
schedule( Runnable command, long delay, TimeUnit unit )
schedule( Callable<V> callable, long delay, TimeUnit unit )
If you want a single task to be executed repeatedly but not concurrently, use a single-thread executor.
ScheduledExecutorService scheduledExecutorService = Executors.newSingleThreadScheduledExecutor() ;
On that ScheduledExecutorService, schedule your task. And make the last step of that task be the chore of scheduling the next occurrence. We have a perpetual motion machine, each time the task runs, it schedules the next run, indefinitely.
Define your Runnable task.
Runnable runnable = new Runnable() {
#Override
public void run ( ) {
// Do the work of this task.
ZonedDateTime zdt = ZonedDateTime.now( ZoneId.systemDefault() ); // Capture the current moment.
System.out.println( "Current moment: " + zdt ); // Report the current moment.
// Schedule the next run of this task.
scheduledExecutorService.schedule( this , 10L , TimeUnit.SECONDS ); // Delay will not be *exactly* this amount of time due to interruptions of scheduling cores on CPU and threads by the JVM and host OS.
}
};
Then run it.
// Jump-start this perpetual motion machine.
scheduledExecutorService.schedule( runnable , 0L , TimeUnit.SECONDS ); // Start immediately, no delay.
Let the executor do its work repeatedly for a certain length of time. Sleep the main thread while the executor service runs on a background thread(s).
try {
Thread.sleep( TimeUnit.MINUTES.toMillis( 2 ) ); // Let our app, and the executor, run for 2 minutes, then shut them both down.
} catch ( InterruptedException e ) {
e.printStackTrace();
}
Remember to always shutdown the executor. Otherwise its background thread(s) may continue running long after your main app has exited.
scheduledExecutorService.shutdown();
System.out.println( "INFO - Executor shutting down. App exiting. " + ZonedDateTime.now( ZoneId.systemDefault() ) );
Tip: Always wrap your Runnable code in a try-catch for all exceptions. Any uncaught exception reaching the executor service will cause the executor to immediately halt, and halt silently.
Runnable runnable = new Runnable() {
#Override
public void run ( ) {
try {
// Do the work of this task.
ZonedDateTime zdt = ZonedDateTime.now( ZoneId.systemDefault() ); // Capture the current moment.
System.out.println( "Current moment: " + zdt ); // Report the current moment.
// Schedule the next run of this task.
scheduledExecutorService.schedule( this , 10L , TimeUnit.SECONDS ); // Delay will not be *exactly* this amount of time due to interruptions of scheduling cores on CPU and threads by the JVM and host OS.
} catch ( Exception e ) {
// TODO: Handle unexpected exeption.
System.out.println( "ERROR - unexpected exception caught on its way to reaching a scheduled executor service. Message # 55cbae82-8492-4638-9630-60c5b28ad876." );
}
}
};
I expect to read a Date from a db table to schedule my task for new iteration
Never use Date or Calendar. Those terrible classes were supplanted years ago by the java.time with the adoption of JSR 310.
As of JDBC 4.2 and later, we can directly exchange java.time objects with the database.
OffsetDateTime now = OffsetDateTime.now( ZoneOffset.UTC ) ;
OffsetDateTime later = myResultSet.getObject( … , OffsetDateTime.class ) ;
if( ! now.isBefore( later ) ) { … } // Verify the future moment is indeed in the future.
Calculate elapsed time, the amount of time we want to delay until our next scheduled run.
Duration d = Duration.between( now , odt ) ;
long seconds = d.toSeconds() ; // Truncates any fractional second.
Use that number of seconds to schedule the next run.
scheduledExecutorService.schedule( this , seconds , TimeUnit.SECONDS );
So the Runnable now looks like this.
Runnable runnable = new Runnable() {
#Override
public void run ( ) {
try {
// Do the work of this task.
ZonedDateTime zdt = ZonedDateTime.now( ZoneId.systemDefault() ); // Capture the current moment.
System.out.println( "Current moment: " + zdt ); // Report the current moment.
// Schedule the next run of this task.
OffsetDateTime now = OffsetDateTime.now( ZoneOffset.UTC ) ;
… do your database query …
OffsetDateTime later = myResultSet.getObject( … , OffsetDateTime.class ) ;
if( ! now.isBefore( later ) ) { … } // Verify the future moment is indeed in the future.
Duration d = Duration.between( now , odt ) ;
long seconds = d.toSeconds() ; // Truncates any fractional second.
scheduledExecutorService.schedule( this , seconds , TimeUnit.SECONDS ); // Delay will not be *exactly* this amount of time due to interruptions of scheduling cores on CPU and threads by the JVM and host OS.
} catch ( Exception e ) {
// TODO: Handle unexpected exeption.
System.out.println( "ERROR - unexpected exception caught on its way to reaching a scheduled executor service. Message # 55cbae82-8492-4638-9630-60c5b28ad876." );
}
}
};
Here is the complete example in a single .java file but without the database query.
package work.basil.example;
import java.util.concurrent.*;
import java.time.*;
public class ScheduleNextTaskExample {
public static void main ( String[] args ) {
ScheduleNextTaskExample app = new ScheduleNextTaskExample();
app.doIt();
}
private void doIt ( ) {
ScheduledExecutorService scheduledExecutorService = Executors.newSingleThreadScheduledExecutor();
Runnable runnable = new Runnable() {
#Override
public void run ( ) {
try {
ZonedDateTime zdt = ZonedDateTime.now( ZoneId.systemDefault() ); // Capture the current moment.
System.out.println( "Current moment: " + zdt ); // Report the current moment.
scheduledExecutorService.schedule( this , 10L , TimeUnit.SECONDS ); // Delay will not be *exactly* this amount of time due to interruptions of scheduling cores on CPU and threads by the JVM and host OS.
} catch ( Exception e ) {
// TODO: Handle unexpected exeption.
System.out.println( "ERROR - unexpected exception caught on its way to reaching a scheduled executor service. Message # 55cbae82-8492-4638-9630-60c5b28ad876." );
}
}
};
// Jump-start this perpetual motion machine.
scheduledExecutorService.schedule( runnable , 0L , TimeUnit.SECONDS ); // Start immediately, no delay.
try {
Thread.sleep( TimeUnit.MINUTES.toMillis( 2 ) ); // Let our app, and the executor, run for 2 minutes, then shut them both down.
} catch ( InterruptedException e ) {
e.printStackTrace();
}
scheduledExecutorService.shutdown();
System.out.println( "INFO - Executor shutting down. App exiting. " + ZonedDateTime.now( ZoneId.systemDefault() ) );
}
}
You can register a new TimerTask, execute the desired logic, and register a new TimerTask at the completion of the desired logic:
public class Starter {
public void execute() {
Timer timer = new Timer();
Date firstExecutionDate = // ... compute ...
timer.schedule(
new RepeatedTimerTask(timer, this::toDoUponEachExecution, this::findNextExecutionDate),
firstExecutionDate
);
}
private Date findNextExecutionDate() {
// ... compute ...
}
private void toDoUponEachExecution() {
// ... do something ...
}
}
public class RepeatedTimerTask extends TimerTask {
private final Timer timer;
private final Runnable logic;
private final Supplier<Date> nextExecution;
public RepeatedTimerTask(Timer timer, Runnable logic, Supplier<Date> nextExecution) {
this.timer = timer;
this.logic = logic;
this.nextExecution = nextExecution;
}
#Override
public void run() {
logic.run();
timer.schedule(this, nextExecution.get());
}
}
Please can any one forward me the sample code related. Cause i tried a lot and on internet no useful info or links i can found related to it.
Thanks in Advance
This might be a workaround. But it works!
In your scheduler, have a default thread running every 1 minute (Or interval of your choice) that pings a file or DB for any changes.
The scheduler should be refreshed if the scheduler finds an entry in the DB.
From your JSP, on click of a button, create a relevant entry in the DB.
While pinging the DB, if the scheduler finds an entry, then it will do the necessary action.
Code snippet
// Default constructor.
public Scheduler()throws SchedulerException, Exception
{
try
{
SchedulerFactory sf = new StdSchedulerFactory();
sche = sf.getScheduler();
sche.start();
if(sche.isShutdown())
{
SendAlerts.sendMsgToGroup("Scheduler Failed To Start at "+sdtf3.format(new Date())+" hrs.",defaultMsgGroup);
logger.fatal("Scheduler Failed To Start At = " + sdtf1.format(new Date()) );
}
else
{
SendAlerts.sendMsgToGroup("Scheduler started at "+sdtf3.format(new Date())+" hrs.",SchStartAlertGroup);
logger.fatal("Scheduler Started At = " + sdtf1.format(new Date()) );
}
sysdate = new Date();
readFromDBAndConfigureSchedules();
while (true)
{
if(sche.isShutdown())
{
SendAlerts.sendMsgToGroup("Scheduler Failed To Start at "+sdtf3.format(new Date())+" hrs.",defaultMsgGroup);
logger.fatal("Scheduler Failed To Start At = " + sdtf1.format(new Date()) );
}
else
{
logger.info("Scheduler is Running. Table Last Pinged at : "+sdtf1.format(sysdate));
}
/*
-----------------
IN THE CHECK DB TABLE METHOD, HANDLE REQUESTS FOR STOP, PAUSE, RE-SCHEDULE ETC
------------------
*/
SchRunJob.checkDBTable();
// Loop will repeat every 1 hour = 60 minutes * 60 seconds = 3600 seconds
Thread.sleep (3600 * 1000);
} // End of while Start Flag is Y
} // End of try block
catch (Exception e)
{
SendAlerts.sendMsgToGroup( "Fatal Exception Caught.Scheduler Shut Down at " + sdtf1.format(new Date()),defaultMsgGroup);
logger.fatal("Fatal Exception Caught.Scheduler Shut Down at " + sdtf1.format(new Date()));
e.printStackTrace();
System.exit(0);
}
} // End of default constructor**
I am using following cron expression to execute a job on every Friday at specified time of day (in sample below it's 1:13 PM).
0 13 13 ? * FRI
So expected behaviour should be if I initialize this trigger any day other then Friday then it should not start executing until next Friday. But whats happening in my case is even if I initialized this trigger today (as today is Wednesday), it starts executing jobs at the very moment.
Relevant java source:
CronTrigger cronTrigger = new CronTrigger("trigger_" + groupName, groupName, cronExpression);
cronTrigger.setStartTime(startDate); //startDate = 1-Mar-2012
cronTrigger.setEndTime(endDate); //endDate = 30-Apr-2012
Your issue is configuring the startTime. startTime is meant to be the time at at which the trigger should occur. Since the date is old this causes a misfire in the scheduler and the default behavior is for the scheduler to immediately refire.
Remove setStartTime, the default behavior is for startTime to be set to the current time and the first trigger time will be the match to the cron trigger after the start time so this Thursday.
Quick little test I through together to verify:
public class Test {
public static void main(String[] args) throws ParseException, SchedulerException {
String groupName = "group";
String cronExpression = "0 13 13 ? * THUR";
CronTrigger cronTrigger = new CronTrigger("trigger_" + groupName, groupName, cronExpression);
cronTrigger.setStartTime(new Date(0));
Scheduler scheduler = StdSchedulerFactory.getDefaultScheduler();
JobDetail detail = new JobDetail("testJob", groupName, TestJob.class);
scheduler.scheduleJob(detail, cronTrigger);
scheduler.start();
try {
Thread.sleep(50001);
} catch (Exception ignore) {
}
}
public static class TestJob implements Job {
public void execute(JobExecutionContext context) throws JobExecutionException {
System.out.println("TEST");
}
}
}
When removing the setStartTime my print message does not trigger. With it there the print message triggers.
Let's say I have a trigger configured this way:
<bean id="updateInsBBTrigger"
class="org.springframework.scheduling.quartz.CronTriggerBean">
<property name="jobDetail" ref="updateInsBBJobDetail"/>
<!-- run every morning at 5 AM -->
<property name="cronExpression" value="0 0 5 * * ?"/>
</bean>
The trigger have to connect with another application and if there is any problem (like a connection failure) it should to retry the task up to five times every 10 minutes or until success. There is any way to configure the trigger to work like this?
I would recommend an implementation like this one to recover the job after a fail:
final JobDataMap jobDataMap = jobCtx.getJobDetail().getJobDataMap();
// the keys doesn't exist on first retry
final int retries = jobDataMap.containsKey(COUNT_MAP_KEY) ? jobDataMap.getIntValue(COUNT_MAP_KEY) : 0;
// to stop after awhile
if (retries < MAX_RETRIES) {
log.warn("Retry job " + jobCtx.getJobDetail());
// increment the number of retries
jobDataMap.put(COUNT_MAP_KEY, retries + 1);
final JobDetail job = jobCtx
.getJobDetail()
.getJobBuilder()
// to track the number of retries
.withIdentity(jobCtx.getJobDetail().getKey().getName() + " - " + retries, "FailingJobsGroup")
.usingJobData(jobDataMap)
.build();
final OperableTrigger trigger = (OperableTrigger) TriggerBuilder
.newTrigger()
.forJob(job)
// trying to reduce back pressure, you can use another algorithm
.startAt(new Date(jobCtx.getFireTime().getTime() + (retries*100)))
.build();
try {
// schedule another job to avoid blocking threads
jobCtx.getScheduler().scheduleJob(job, trigger);
} catch (SchedulerException e) {
log.error("Error creating job");
throw new JobExecutionException(e);
}
}
Why?
It will not block Quartz Workers
It will avoid back pressure. With setRefireImmediately the job will be fired immediately and it could lead to back pressure issues
Source: Automatically Retry Failed Jobs in Quartz
If you want to have a job which keeps trying over and over again until it succeeds, all you have to do is throw a JobExecutionException with a flag to tell the scheduler to fire it again when it fails. The following code shows how:
class MyJob implements Job {
public MyJob() {
}
public void execute(JobExecutionContext context) throws JobExecutionException {
try{
//connect to other application etc
}
catch(Exception e){
Thread.sleep(600000); //sleep for 10 mins
JobExecutionException e2 = new JobExecutionException(e);
//fire it again
e2.setRefireImmediately(true);
throw e2;
}
}
}
It gets a bit more complicated if you want to retry a certain number of times. You have to use a StatefulJob and hold a retryCounter in its JobDataMap, which you increment if the job fails. If the counter exceeds the maximum number of retries, then you can disable the job if you wish.
class MyJob implements StatefulJob {
public MyJob() {
}
public void execute(JobExecutionContext context) throws JobExecutionException {
JobDataMap dataMap = context.getJobDetail().getJobDataMap();
int count = dataMap.getIntValue("count");
// allow 5 retries
if(count >= 5){
JobExecutionException e = new JobExecutionException("Retries exceeded");
//make sure it doesn't run again
e.setUnscheduleAllTriggers(true);
throw e;
}
try{
//connect to other application etc
//reset counter back to 0
dataMap.putAsString("count", 0);
}
catch(Exception e){
count++;
dataMap.putAsString("count", count);
JobExecutionException e2 = new JobExecutionException(e);
Thread.sleep(600000); //sleep for 10 mins
//fire it again
e2.setRefireImmediately(true);
throw e2;
}
}
}
I would suggest for more flexibility and configurability to better store in your DB two offsets: the repeatOffset which will tell you
after how long the job should be retried and the trialPeriodOffset which will keep the information of the time window that the job is
allowed to be rescheduled. Then you can retrieve these two parameters like (I assume you are using Spring):
String repeatOffset = yourDBUtilsDao.getConfigParameter(..);
String trialPeriodOffset = yourDBUtilsDao.getConfigParameter(..);
Then instead of the job to remember the counter it will need to remember the initalAttempt:
Long initialAttempt = null;
initialAttempt = (Long) existingJobDetail.getJobDataMap().get("firstAttempt");
and perform the something like the following check:
long allowedThreshold = initialAttempt + Long.parseLong(trialPeriodOffset);
if (System.currentTimeMillis() > allowedThreshold) {
//We've tried enough, time to give up
log.warn("The job is not going to be rescheduled since it has reached its trial period threshold");
sched.deleteJob(jobName, jobGroup);
return YourResultEnumHere.HAS_REACHED_THE_RESCHEDULING_LIMIT;
}
It would be a good idea to create an enum for the result of the attempt that is being returned back to the core workflow of your
application like above.
Then construct the rescheduling time:
Date startTime = null;
startTime = new Date(System.currentTimeMillis() + Long.parseLong(repeatOffset));
String triggerName = "Trigger_" + jobName;
String triggerGroup = "Trigger_" + jobGroup;
Trigger retrievedTrigger = sched.getTrigger(triggerName, triggerGroup);
if (!(retrievedTrigger instanceof SimpleTrigger)) {
log.error("While rescheduling the Quartz Job retrieved was not of SimpleTrigger type as expected");
return YourResultEnumHere.ERROR;
}
((SimpleTrigger) retrievedTrigger).setStartTime(startTime);
sched.rescheduleJob(triggerName, triggerGroup, retrievedTrigger);
return YourResultEnumHere.RESCHEDULED;