I need to schedule to a new Date for everytime a task it's executed.
I've seen many examples where the period or interval is set by millis and stays for every iteration but i can't find any that takes a date parameter for next execution
I tried the #Scheduled annotation since I am working with Spring, but I do not know if there is a possibility to pass a parameter.
Examples i've seen
Example 1:
#Scheduled(fixedRate = 20000)
public void scheduler() {
log.info("scheduler");
log.info("Current Thread " + Thread.currentThread().getName());
log.info("Current Thread " + Thread.currentThread().getId());
}
Example 2:
ScheduledExecutorService scheduler = Executors.newScheduledThreadPool(1);
scheduler.scheduleAtFixedRate(myRunnable, 10, 10, TimeUnit.MINUTES);
I expect to read a Date from a db table to schedule my task for new iteration
Ty for the help!
Edit
Note: There will also be a time when I need to decide where to stop the next iteration, so I'm trying to call the schedule task by a method
I avoid using Spring, so I cannot help you there. But I can guide you through the use of ScheduledExecutorService to accomplish your goal.
ScheduledExecutorService::schedule( Runnable command, long delay, TimeUnit unit )
You are partially correct about the ScheduledExecutorService: Two of its three scheduling strategies are designed to keep regular intervals between runs:
scheduleAtFixedRate
scheduleWithFixedDelay
But the third strategy lets you set the next run with any amount of delay you wish.
schedule( Runnable command, long delay, TimeUnit unit )
schedule( Callable<V> callable, long delay, TimeUnit unit )
If you want a single task to be executed repeatedly but not concurrently, use a single-thread executor.
ScheduledExecutorService scheduledExecutorService = Executors.newSingleThreadScheduledExecutor() ;
On that ScheduledExecutorService, schedule your task. And make the last step of that task be the chore of scheduling the next occurrence. We have a perpetual motion machine, each time the task runs, it schedules the next run, indefinitely.
Define your Runnable task.
Runnable runnable = new Runnable() {
#Override
public void run ( ) {
// Do the work of this task.
ZonedDateTime zdt = ZonedDateTime.now( ZoneId.systemDefault() ); // Capture the current moment.
System.out.println( "Current moment: " + zdt ); // Report the current moment.
// Schedule the next run of this task.
scheduledExecutorService.schedule( this , 10L , TimeUnit.SECONDS ); // Delay will not be *exactly* this amount of time due to interruptions of scheduling cores on CPU and threads by the JVM and host OS.
}
};
Then run it.
// Jump-start this perpetual motion machine.
scheduledExecutorService.schedule( runnable , 0L , TimeUnit.SECONDS ); // Start immediately, no delay.
Let the executor do its work repeatedly for a certain length of time. Sleep the main thread while the executor service runs on a background thread(s).
try {
Thread.sleep( TimeUnit.MINUTES.toMillis( 2 ) ); // Let our app, and the executor, run for 2 minutes, then shut them both down.
} catch ( InterruptedException e ) {
e.printStackTrace();
}
Remember to always shutdown the executor. Otherwise its background thread(s) may continue running long after your main app has exited.
scheduledExecutorService.shutdown();
System.out.println( "INFO - Executor shutting down. App exiting. " + ZonedDateTime.now( ZoneId.systemDefault() ) );
Tip: Always wrap your Runnable code in a try-catch for all exceptions. Any uncaught exception reaching the executor service will cause the executor to immediately halt, and halt silently.
Runnable runnable = new Runnable() {
#Override
public void run ( ) {
try {
// Do the work of this task.
ZonedDateTime zdt = ZonedDateTime.now( ZoneId.systemDefault() ); // Capture the current moment.
System.out.println( "Current moment: " + zdt ); // Report the current moment.
// Schedule the next run of this task.
scheduledExecutorService.schedule( this , 10L , TimeUnit.SECONDS ); // Delay will not be *exactly* this amount of time due to interruptions of scheduling cores on CPU and threads by the JVM and host OS.
} catch ( Exception e ) {
// TODO: Handle unexpected exeption.
System.out.println( "ERROR - unexpected exception caught on its way to reaching a scheduled executor service. Message # 55cbae82-8492-4638-9630-60c5b28ad876." );
}
}
};
I expect to read a Date from a db table to schedule my task for new iteration
Never use Date or Calendar. Those terrible classes were supplanted years ago by the java.time with the adoption of JSR 310.
As of JDBC 4.2 and later, we can directly exchange java.time objects with the database.
OffsetDateTime now = OffsetDateTime.now( ZoneOffset.UTC ) ;
OffsetDateTime later = myResultSet.getObject( … , OffsetDateTime.class ) ;
if( ! now.isBefore( later ) ) { … } // Verify the future moment is indeed in the future.
Calculate elapsed time, the amount of time we want to delay until our next scheduled run.
Duration d = Duration.between( now , odt ) ;
long seconds = d.toSeconds() ; // Truncates any fractional second.
Use that number of seconds to schedule the next run.
scheduledExecutorService.schedule( this , seconds , TimeUnit.SECONDS );
So the Runnable now looks like this.
Runnable runnable = new Runnable() {
#Override
public void run ( ) {
try {
// Do the work of this task.
ZonedDateTime zdt = ZonedDateTime.now( ZoneId.systemDefault() ); // Capture the current moment.
System.out.println( "Current moment: " + zdt ); // Report the current moment.
// Schedule the next run of this task.
OffsetDateTime now = OffsetDateTime.now( ZoneOffset.UTC ) ;
… do your database query …
OffsetDateTime later = myResultSet.getObject( … , OffsetDateTime.class ) ;
if( ! now.isBefore( later ) ) { … } // Verify the future moment is indeed in the future.
Duration d = Duration.between( now , odt ) ;
long seconds = d.toSeconds() ; // Truncates any fractional second.
scheduledExecutorService.schedule( this , seconds , TimeUnit.SECONDS ); // Delay will not be *exactly* this amount of time due to interruptions of scheduling cores on CPU and threads by the JVM and host OS.
} catch ( Exception e ) {
// TODO: Handle unexpected exeption.
System.out.println( "ERROR - unexpected exception caught on its way to reaching a scheduled executor service. Message # 55cbae82-8492-4638-9630-60c5b28ad876." );
}
}
};
Here is the complete example in a single .java file but without the database query.
package work.basil.example;
import java.util.concurrent.*;
import java.time.*;
public class ScheduleNextTaskExample {
public static void main ( String[] args ) {
ScheduleNextTaskExample app = new ScheduleNextTaskExample();
app.doIt();
}
private void doIt ( ) {
ScheduledExecutorService scheduledExecutorService = Executors.newSingleThreadScheduledExecutor();
Runnable runnable = new Runnable() {
#Override
public void run ( ) {
try {
ZonedDateTime zdt = ZonedDateTime.now( ZoneId.systemDefault() ); // Capture the current moment.
System.out.println( "Current moment: " + zdt ); // Report the current moment.
scheduledExecutorService.schedule( this , 10L , TimeUnit.SECONDS ); // Delay will not be *exactly* this amount of time due to interruptions of scheduling cores on CPU and threads by the JVM and host OS.
} catch ( Exception e ) {
// TODO: Handle unexpected exeption.
System.out.println( "ERROR - unexpected exception caught on its way to reaching a scheduled executor service. Message # 55cbae82-8492-4638-9630-60c5b28ad876." );
}
}
};
// Jump-start this perpetual motion machine.
scheduledExecutorService.schedule( runnable , 0L , TimeUnit.SECONDS ); // Start immediately, no delay.
try {
Thread.sleep( TimeUnit.MINUTES.toMillis( 2 ) ); // Let our app, and the executor, run for 2 minutes, then shut them both down.
} catch ( InterruptedException e ) {
e.printStackTrace();
}
scheduledExecutorService.shutdown();
System.out.println( "INFO - Executor shutting down. App exiting. " + ZonedDateTime.now( ZoneId.systemDefault() ) );
}
}
You can register a new TimerTask, execute the desired logic, and register a new TimerTask at the completion of the desired logic:
public class Starter {
public void execute() {
Timer timer = new Timer();
Date firstExecutionDate = // ... compute ...
timer.schedule(
new RepeatedTimerTask(timer, this::toDoUponEachExecution, this::findNextExecutionDate),
firstExecutionDate
);
}
private Date findNextExecutionDate() {
// ... compute ...
}
private void toDoUponEachExecution() {
// ... do something ...
}
}
public class RepeatedTimerTask extends TimerTask {
private final Timer timer;
private final Runnable logic;
private final Supplier<Date> nextExecution;
public RepeatedTimerTask(Timer timer, Runnable logic, Supplier<Date> nextExecution) {
this.timer = timer;
this.logic = logic;
this.nextExecution = nextExecution;
}
#Override
public void run() {
logic.run();
timer.schedule(this, nextExecution.get());
}
}
Related
I'm using KafkaSource to read kafka messages of Events type, as per documentation providing event time extractor is optional for source kafka
KafkaSource<Events> source =
KafkaSource.<Events>builder()
.setProperties(kafkaProperties)
.setBootstrapServers(parameters.get("bootstrap-servers-source"))
.setTopics(parameters.get("source-topic"))
.setGroupId("visit-events-flink-mvp")
.setStartingOffsets(OffsetsInitializer.committedOffsets(OffsetResetStrategy.EARLIEST))
//.setStartingOffsets(OffsetsInitializer.earliest())
.setValueOnlyDeserializer(new EventsDeserializationSchema())
.build();
// event stream from kafka source
DataStream<Events> eventStream =
env.fromSource(source, WatermarkStrategy.forMonotonousTimestamps(), "Kafka Source")
//should be a unique id
.uid("kafka-source");
//stream is keyed based on the anonymousId
DataStream<Events> keyedStream =
eventStream.keyBy(Events::getAnonymousId)
// .process(new KeyedProcessing(Long.parseLong(parameters.get("ttl"))))
.process(new KeyedProcessingWithCallBack(Long.parseLong(parameters.get("ttl"))))
.uid("engager-events-keyed-processing");
In my KeyedProcessingWithCallBack, I'm setting event time timer for 60 secs and the call back is not triggering at all.
My kafka source has 8 partitions and I'm running job with parallelism 1
public void processElement(EngagerEvents value, KeyedProcessFunction<String, EngagerEvents, String>.Context ctx, Collector<String> out) throws Exception {
ObjectMapper objectMapper = new ObjectMapper();
JsonNode jsonNode = objectMapper.readTree(value.getEventString());
System.out.println("time : " +jsonNode.get("EVENT_TIMESTAMP").textValue());
if (anonymousIdHasBeenSeen.value() == null) {
System.out.println("time stamp emitting: " +jsonNode.get("EVENT_TIMESTAMP").textValue());
// key is not available in the state
anonymousIdHasBeenSeen.update(true);
System.out.println("TIMER START TIME: " +ctx.timestamp());
out.collect(value.getEventString());
ctx.timerService().registerEventTimeTimer(ctx.timestamp() + (stateTtl * 1000));
}
}
// not getting triggered
#Override
public void onTimer(long timestamp, OnTimerContext ctx, Collector<String> out)
throws Exception {
// triggers after ttl has passed
System.out.println("Call back triggered : time : " +timestamp + " value : " +anonymousIdHasBeenSeen.value());
anonymousIdHasBeenSeen.clear();
}
TEST Simulator code which will send event with anonymousId=111 with different event time stamp
try {
for (int i = 0; i < 500; i++) {
String[] anonymousId = {"111"};
String key = String.valueOf(new Random().nextInt(10));
ProducerRecord<String, String> record = new ProducerRecord<>(
"flink-visits-mvp-test-source",
key,
// getEvent(UUID.randomUUID().toString() + "-" +Thread.currentThread().getName() , event[new Random().nextInt(1)]));
// getEvent(anonymousId[new Random().nextInt(1)], event[new Random().nextInt(1)]));
getEvent(anonymousId[new Random().nextInt(1)],
System.currentTimeMillis(),
event));
//System.out.println(record.value().toString());
producer.send(record);
try {
Thread.sleep(200);
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
}
}
Am I doing something wrong here ? Why is my event time timer call back not trgiggering ?
I also experience the same error with Flink V1.16. The processElement method is invoked as expected however, observed that ctx.timerService().currentWatermark() always print as -9223372036854775808. onTimer method never invoked with 60 seconds timer.
With lots of trial and error I found that invoking env.setParallelism() method resolve the issue.
final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
env.setParallelism(1);
After fix, the currentWatermark() method return correct watermark time and onTimer method is invoked every 60 seconds.
Here is the log entries before fix:
Event Time = 1669610765000, End Of Window Time = 1669610819999,
current water mark time = -9223372036854775808
Event Time = 1669610807000, End Of Window Time = 1669610819999,
current water mark time = -9223372036854775808
Here is the log entries after fix:
Event Time = 1669610393000, End Of Window Time = 1669610399999,
current water mark time = 1669610387999
Event Time = 1669610450000, End Of Window Time = 1669610459999,
current water mark time = 1669610435999
I have a problem with Resilience4j RateLimiter
public static void main(final String[] args) throws InterruptedException {
final ExternalService service = new ExternalService();
final ExecutorService executorService = Executors.newFixedThreadPool(30);
final RateLimiterConfig config = RateLimiterConfig.custom()
.limitRefreshPeriod(Duration.ofSeconds(10))
.limitForPeriod(3)
.timeoutDuration(Duration.ofSeconds(12))
.build();
final RateLimiter rateLimiter = RateLimiter.of("RateLimiter", config);
final Callable<Response<String>> callable = RateLimiter.decorateCallable(
rateLimiter, () -> service.get(200, "OK")
);
executorService.submit(callable); //fine in first period
executorService.submit(callable); //fine in first period
executorService.submit(callable); //fine in first period
executorService.submit(callable); //should wait 10 sec and fine in second period
executorService.submit(callable); //should wait 10 sec and fine in second period
executorService.submit(callable); //should wait 10 sec and fine in second period
executorService.submit(callable); //should exit with timeout after 12 seconds
executorService.submit(callable); //should exit with timeout after 12 seconds
executorService.submit(callable); //should exit with timeout after 12 seconds
Thread.sleep(Duration.ofSeconds(40).toMillis());
executorService.shutdown();
}
In ExternalService I have some basic logging with localTime of responses. I think that it should work as I explained in comments, but my response is:
> Task :Main.main()
[12:24:53.5] Return standard response
[12:24:53.5] Return standard response
[12:24:53.5] Return standard response
[12:25:03.5] Return standard response
[12:25:03.5] Return standard response
[12:25:03.5] Return standard response
[12:25:03.5] Return standard response
[12:25:03.5] Return standard response
BUILD SUCCESSFUL in 40s
So it seems that the first cycle is good, but after that, FIVE next threads are allowed by RateLimiter, and the last thread is never called.
Unfortunately it was a bug which was introduced in a PR #672 which is part of release v1.2.0. The PR added the possibility to requests multiple permits per call. The bug was fixed now.
#RequestMapping(value="/api/discovery.conversations.recent")
public #ResponseBody ChannelsRecentList getChannelsList() throws InterruptedException {
Instant start = Instant.now();
ChannelsRecentList channels = new ChannelsRecentList();
for(int i=0;i<30;i++){
ChannelsRecent temp = new ChannelsRecent("ID"+randomAlphaNumeric.randomValueGenerator(),teamName,offset);
channels.getChannels().add(temp);
}
Thread.sleep(1);
Instant finish = Instant.now();
long timeElapsed = Duration.between(start, finish).toMillis();
logger.info("TimeTaken For ConversationList is "+timeElapsed+" ms");
logger.info("Total ConversationRecent APICalls_Received_Count (30 per API Request): "+APICallReceived++);
return channels;
}
Finally i made it to get throughput and system resources using
actuator/micrometer
prometheus
grafana.
as described in this article
Please can any one forward me the sample code related. Cause i tried a lot and on internet no useful info or links i can found related to it.
Thanks in Advance
This might be a workaround. But it works!
In your scheduler, have a default thread running every 1 minute (Or interval of your choice) that pings a file or DB for any changes.
The scheduler should be refreshed if the scheduler finds an entry in the DB.
From your JSP, on click of a button, create a relevant entry in the DB.
While pinging the DB, if the scheduler finds an entry, then it will do the necessary action.
Code snippet
// Default constructor.
public Scheduler()throws SchedulerException, Exception
{
try
{
SchedulerFactory sf = new StdSchedulerFactory();
sche = sf.getScheduler();
sche.start();
if(sche.isShutdown())
{
SendAlerts.sendMsgToGroup("Scheduler Failed To Start at "+sdtf3.format(new Date())+" hrs.",defaultMsgGroup);
logger.fatal("Scheduler Failed To Start At = " + sdtf1.format(new Date()) );
}
else
{
SendAlerts.sendMsgToGroup("Scheduler started at "+sdtf3.format(new Date())+" hrs.",SchStartAlertGroup);
logger.fatal("Scheduler Started At = " + sdtf1.format(new Date()) );
}
sysdate = new Date();
readFromDBAndConfigureSchedules();
while (true)
{
if(sche.isShutdown())
{
SendAlerts.sendMsgToGroup("Scheduler Failed To Start at "+sdtf3.format(new Date())+" hrs.",defaultMsgGroup);
logger.fatal("Scheduler Failed To Start At = " + sdtf1.format(new Date()) );
}
else
{
logger.info("Scheduler is Running. Table Last Pinged at : "+sdtf1.format(sysdate));
}
/*
-----------------
IN THE CHECK DB TABLE METHOD, HANDLE REQUESTS FOR STOP, PAUSE, RE-SCHEDULE ETC
------------------
*/
SchRunJob.checkDBTable();
// Loop will repeat every 1 hour = 60 minutes * 60 seconds = 3600 seconds
Thread.sleep (3600 * 1000);
} // End of while Start Flag is Y
} // End of try block
catch (Exception e)
{
SendAlerts.sendMsgToGroup( "Fatal Exception Caught.Scheduler Shut Down at " + sdtf1.format(new Date()),defaultMsgGroup);
logger.fatal("Fatal Exception Caught.Scheduler Shut Down at " + sdtf1.format(new Date()));
e.printStackTrace();
System.exit(0);
}
} // End of default constructor**
I have an app that could set the time for processing. The problems is when I update the time, the processing will increase. For example:
Initially the timer start at 07:00AM
Let say, I update the timer to 08:00AM then the next day onwards, the program will run again at 07:00AM and also at 08:00AM. (The 07:00AM is still in scheduler, how to remove the 07:00AM?)
How to make the scheduler to only run the 08:00AM the next day?
public void setKonfigurasi(String name, String value) {
log.info(SERVLET_NAME + "Entering setKonfigurasi");
amBean.setParam(name,value); //update the time into database
//name = 'processFileConf|kodPT|userA|20140312 08:30 AM'
// reschedule timer after configured by user
try {
String kodPT = name.substring(name.indexOf("|") + 1, name.indexOf("|",name.indexOf("|") + 1));
String configStr = value.substring(2); //get the new time
String currentStr = CommonUtil.getCurrentDate();
DateFormat dateformat = new SimpleDateFormat("dd/MM/yyyy KK:mm:ss a");
Date currentDate=new Date() ;
Date configDate = dateformat.parse(currentStr+" "+configStr);
long config = configDate.getTime();
long current = currentDate.getTime();
// today
long delay = config-current;
if (delay < 0)
// tomorrow
delay += (1000*60*60*24);
// create the timer and timer task objects
Timer timer = new Timer();
timer.scheduleAtFixedRate(new TimerTask() {
public void run() {
System.out.println("showtime for "+kodPT);
processFile("auto"+kodPT);
}
}, delay, 1000*60*60*24);
ServletContextEvent servletContextEvent = EtServletContextListener.getContext();
ServletContext servletContext = servletContextEvent.getServletContext();
servletContext.removeAttribute ("timer");
servletContext.setAttribute ("timer", timer);
} catch (Exception e) {
log.error("Exception on date format : "+e.getMessage());
}
log.info(SERVLET_NAME + "Exiting setKonfigurasi");
}
You need to call cancel() on the previous timer and create a new one. The javadoc says:
Terminates this timer, discarding any currently scheduled tasks. Does
not interfere with a currently executing task (if it exists). Once a
timer has been terminated, its execution thread terminates gracefully,
and no more tasks may be scheduled on it.
I have found out about how to do what I want. Instead of using java.util.Timer to create the timer, we should use javax.ejb.Timer since the Timer in ejb have an info to identified each timer.
Timer Service EJB