i am trying to execute a task at a fixed rate using the #Scheduled annotation in java spring. however, it seems that by default spring will not execute a fixedRate task at a fixed rate if the task is slower than the rate. is there some setting i can add to my spring configuration to change this behavior?
example:
#Service
public class MyTask{
#Scheduled(fixedRate = 1000)
public void doIt(){
// this sometimes takes >1000ms, in which case the next execution is late
...
}
}
i have a work-around, but it seems less than ideal. basically, i just replace the default single-thread executor with a thread pool, then i have a scheduled method call an async method since the #Async annotation allows concurrent executions:
#Service
public class MyTask{
#Async
public void doIt(){
// this sometimes takes >1000ms, but the next execution is on time
...
}
}
#Service
public class MyTaskScheduler{
...
#Scheduled(fixedRate = 1000)
public void doIt(){
myTask.doIt();
}
}
#Configuration
#EnableScheduling
#EnableAsync
public class MySpringJavaConfig{
#Bean(destroyMethod = "shutdown")
public Executor taskScheduler() {
return Executors.newScheduledThreadPool(5);
}
}
boring details of my real-world scenario: in my production code i have a task that takes between 10ms and 10 minutes depending on the current workload. ideally, i would like to capture a new thread from the pool every 1000ms so that the number of concurrent threads increases with the workload. obviously i have an upper limit on threads in place (among other controls) to keep things from getting out of hand.
The TaskScheduler API (which backs some of the Spring Scheduling behavior) seems to be defined to prevent the behavior you are requesting
Schedule the given Runnable, invoking it at the specified execution
time and subsequently with the given period.
Parameters
period the interval between successive executions of the task (in milliseconds)
subsequently and successive seem to indicate that the next execution will only occur after the current execution is complete.
What's more, ScheduledExecutorService#scheduleAtFixedRate(..) (which the built-in TaskScheduler implementations use) also says
If any execution of this task takes longer than its period, then
subsequent executions may start late, but will not concurrently
execute.
So there's another layer of the implementation that prevents the behavior you want.
A possible solution, and one I don't recommend since the API doesn't seem to be built around it, is to define and provide your own TaskScheduler that does run the task concurrently. Look into #EnableScheduling and SchedulingConfigurer for how to register a TaskScheduler.
the best solution i have found so far is to simply use a delegate to make the method calls async. this is only preferable because it allows me to declare the schedule in the same class as the method which does the work:
#Service
public class AsyncRunner {
#Async
public void run(Runnable runnable) {
runnable.run();
}
}
#Service
public class MyTask{
...
#Scheduled(fixedRate = 1000)
public void scheduleIt(){
asyncRunner.run(this::doIt);
}
public void doIt(){
// this sometimes takes >1000ms, but the next execution is on time
...
}
}
Related
I have a project which executes multiple scheduled method at start up.
I remarked that after scheduled methods are executed, the opened threads do not close, but remain in a 'parking' state.
Is this a normal behavior ?
Aren't the threads suppose to close after method is executed ? (Because keeping multiple threads open just slows down the application and consumes more RAM.)
Here are my code configurations:
#EnableScheduling
#Configuration
#ConditionalOnProperty(name = "scheduling.enabled", matchIfMissing = true)
public class SchedulingConfiguration implements SchedulingConfigurer {
}
Here is an example of method called in service:
#Scheduled(cron = "0 0 4 * * *")
protected void updateExchangeRates() {
if (enablePostConstruct) {
countryService.updateCountryExchangeRates();
}
}
I would like to run the scheduled methods asynchronously, with a max thread pool consumed between 10-15 threads. And after execution, the thread to close and reopen in case it got to the point when it needs to be executed again.
Can you guide me please how this can be achieved ?
I tried to implement SchedulingConfigurer and perform executorService.shutdown(), but it did not work.
You could use a method annotated with #PreDestroy to invoke executorService.shutdown(). I wouldn't bother about the Parking State, you probably want those threads to be ready for the next invocation, so not really harmful that they are parked.
Nothing wrong with the code.
I currently have a scheduled task within my Spring application.
However a two parts of this logic is severely time-consuming and I am wondering if there would be a way to make these two parts asynchronous so that it does not interfere with the time of the logic being executed.
The logic that I need to execute as follows.
#Scheduled(fixedDelay = 10000)
public void startAuction() throws Exception {
List<SchGoodsAuctionStartListRes> list = schedulerService.schGoodsAuctionStartList();
for (SchGoodsAuctionStartListRes item : list) {
schedulerService.schGoodsAuctionStart(item);
// 1st time consuming block that needs async
PushInfo pushInfo = pushMapper.pushGoodsSeller(item.getGoodsIdx());
pushInfo.setTitle("Start");
pushInfo.setBody("[" + pushInfo.getBrand() + "] started.");
pushInfo.setPushGrp("001");
pushInfo.setPushCode("003");
fcmPushUtil.sendPush(pushInfo);
// 2nd time consuming block that needs async
List<PushInfo> pushInfos = pushMapper.pushGoodsAuctionAll(item.getIdx());
for (PushInfo pushInfoItem : pushInfos) {
pushInfoItem.setTitle("\uD83D\uDD14 open");
pushInfoItem.setBody("[" + pushInfo.getBrand() + "] started. \uD83D\uDC5C");
pushInfoItem.setPushGrp("002");
pushInfoItem.setPushCode("008");
fcmPushUtil.sendPush(pushInfoItem);
}
}
}
From my understanding, a scheduler already is executing logic asynchronously, and I wonder if there would be any way of making those two blocks asynchronous so that it does not cause delays when executing this logic.
Any sort of advice or feedback would be deeply appreciated!
Thank you in advance!
There are several approaches that you could take here.
Configuring Thread Pool executor for Spring's scheduled tasks
By default Spring uses single thread executor to execute scheduled tasks, which means that even if you have multiple #Scheduled tasks or another execution for a task triggers before the previous one is completed, they will all have to wait in the queue.
You can configure your own executor to be used by Spring Scheduling. Take a look at the documentation of #EnableScheduling, it is pretty exhaustive on the subject.
To configure ExecutorService to be used for scheduled tasks it is enough to define a bean:
#Bean
public TaskScheduler taskScheduler() {
ThreadPoolTaskScheduler threadPoolTaskScheduler = new ThreadPoolTaskScheduler();
threadPoolTaskScheduler.setPoolSize(8);
threadPoolTaskScheduler.setThreadNamePrefix("task-scheduler");
return threadPoolTaskScheduler;
}
Additionally, if you use Spring Boot, you can use properties file:
spring.task.scheduling.pool.size=8
Executing scheduled tasks asynchronously
To execute scheduled tasks asynchronously you can use Spring's #Async annotation (and make sure to #EnableAsync somewhere in your configuration. That will make your tasks to be executed on a background thread, freeing the scheduling thread.
#EnableAsync
public class ScheduledAsyncTask {
#Async
#Scheduled(fixedRate = 10000)
public void scheduleFixedRateTaskAsync() throws InterruptedException {
// your task logic ...
}
}
Offload expensive parts of your tasks to a different executor
Finally, you could use a separate ExecutorService and run expensive parts of your tasks using that executor instead of the one used for task scheduling. This will keep the time needed to complete the execution on the thread used by Spring to schedule tasks to a minimum, allowing it to start next executions.
public class ScheduledAsyncTask implements DisposableBean {
private final ExecutorService executorService = Executors.newFixedThreadPool(4);
#Scheduled(fixedRate = 10000)
public void scheduleFixedRateTaskAsync() throws InterruptedException {
executorService.submit(() -> {
// Expensive calculations ...
});
}
#Override
public void destroy() {
executorService.shutdown();
}
}
I am trying to schedule a task inside a Spring #Bean which will update the property of the instance returning from Bean.
I am able to run this code, and the executor works fine a couple of times, but after that, it suddenly stops loading.
What exactly is the problem here? Is there a better way to work this out??
#Bean(name="service")
public Service getService(){
Service service = new Service();
ScheduledExecutorService serviceLoader = Executors.newScheduledThreadPool(1);
serviceLoader.scheduleAtFixedRate(new Runnable() {
#Override
public void run() {
service.loadAllLiveEvents();
}
}, 0, 1, TimeUnit.HOURS);
return service;
}
The lifecycle of serviceLoader looks weird - it gets initialized right during the method or service, then schedules some work and then service gets returned. What happens to the reference to this pool? when the shutdown can be called?
In addition, the first iteration runs immediately, and this happens when the application context is not ready yet, this can lead to unpredictable results depending on the actual code that runs during the iteration.
I can't say for sure what happens based on this code snippet, but here are some possible solutions:
Use #Scheduled annotation, running scheduled tasks is a bulit-in spring feature. There are a lot of tutorials, here is one of them
If you absolutely have to use the thread pool, I suggest the following configuration:
#Configuration
public class MyConfiguration {
#Bean
public Service service() {
return new Service();
}
#Bean(destroyMethod="shutdownNow") // or shutdown - now spring will close the pool when the app context gets closed
#Qualifier("serviceLoaderPool")
public ScheduledExecutorService serviceLoader() {
return Executors.newScheduledThreadPool(1);
}
#EventListener
public void onAppContextStarted(ApplicationReadyEvent evt) {
ScheduledExecutorService loader =
(ScheduledExecutorService)evt.getApplicationContext().getBean("serviceLoaderPool");
Service service = evt.getApplicationContext.getBean(Service.class);
loader.scheduleAtFixedRate(new Runnable() {
#Override
public void run() {
service.loadAllLiveEvents();
}
}, 0, 1, TimeUnit.HOURS);
}
}
With this approach you can be sure that the service will start refreshing when the application context is ready.
The lifecycle of the executor service is well-defined as well, spring manages it as a regular singleton bean, so it won't be GC'ed as long as the application context is up-and-running.
The presence of destroy method guarantees graceful shutdown (again, spring will call it for you).
Is there a way to limit the lifetime of a running spring-batch job to e.g. 23 hours?
We start a batch job daily by a cron job and he job takes about 9 hours. It happened under some circumstances that the DB connection was so slow that the job took over 60 hours to complete. The problem is that the next job instance gets started by the cronjob the next day - and then anotherone the day after - and anotherone...
If this job is not finished within e.g. 23 hours, I want to terminate it and return an error. Is there a way to do that out-of-the-box with spring-batch?
Using a StepListener you can stop a job by calling StepExecution#setTerminateOnly.
stepBuilderFactory
...
.writer(writer)
.listener(timeoutListener)
...
.build()
And the TimeoutListener could look like this
#Component
public class TimeoutListener implements StepListener {
private StepExecution stepExecution;
#BeforeStep
public void beforeStep(StepExecution stepExecution) {
this.stepExecution = stepExecution;
}
#BeforeRead
public void beforeRead() {
if (jobShouldStop()) {
stepExecution.setTerminateOnly();
}
}
private boolean jobShouldStop() {
// ...
}
}
This will gracefully stop the job, without forcefully terminate any running steps.
Spring Batch specifically avoids the issue of job orchestration which this falls into. That being said, you could add a listener to your job that checks for other instances running and calls stop on them before beginning that one. Not knowing what each job does, I'm not sure how effective that would be, but it should be a start.
If you write your own job class to launch the process you can make your class implement StatefulJob interface, which prevents concurrent launches of the same job. Apart from that you can write your own monitoring and stop the job programatically after some period, but it will require some custom coding, I dont know if there is anything build-in for such use case.
If i want a method to repeat async, can i use #Scheduled and #Async together ?
#Async
#Scheduled(fixedDelay = x)
public void doSomethingEveryXMinuteAsync() {
// action
}
or is there another standard way to achive this ?
There is no need to use #Async. Just use fixedRate attribute of #Scheduled instead of fixedDelay. Spring will make another invocation on the method after the given time regardless of any call is already being processed.
UPDATE:
Apparently fixedRate attribute does not enforce a scheduled method to be called asynchronously and increasing pool size of scheduler task executor only enables asynchronous execution of independent #Scheduled methods. Even putting #Async on the method does not make it work as OP has asked.
ScheduledAnnotationBeanPostProcessor just creates a Runnable from the #Scheduled method and does not create any pointcut as #Async method processor would. ScheduledThreadPoolExecutor waits until Runnable#run() is finished and sets the next execution time using the start time and the fixed rate. So if the method call takes more time than the scheduled time, the next task is triggered right after the previous call is finished.
An easy solution would be extracting the actual method into another class as a #Async method and calling this method from the #Scheduled method.
Implement SchedulingConfigurer and override configureTasks method.
Define poolsize more than one, It will work as you are expecting.
You should configure implementing SchedulingConfigurer:
#Configuration
#EnableScheduling
public class ScheduledConfiguration implements SchedulingConfigurer {
#Override
public void configureTasks(ScheduledTaskRegistrar scheduledTaskRegistrar)
{
ThreadPoolTaskScheduler threadPoolTaskScheduler = new ThreadPoolTaskScheduler();
threadPoolTaskScheduler.setPoolSize(10);
threadPoolTaskScheduler.setThreadNamePrefix("your-scheduler-");
threadPoolTaskScheduler.initialize();
scheduledTaskRegistrar.setTaskScheduler(threadPoolTaskScheduler);
}
}
With this code you achieve parallel execution of your method which is annotated with #Scheduled.
You may also set the property:
spring.task.scheduling.pool.size
#Async and #Scheduled method shouldn't be in a same class.
follow the manual and add AsyncConfigurerhttps://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/scheduling/annotation/EnableAsync.html