Is there a way to limit the lifetime of a running spring-batch job to e.g. 23 hours?
We start a batch job daily by a cron job and he job takes about 9 hours. It happened under some circumstances that the DB connection was so slow that the job took over 60 hours to complete. The problem is that the next job instance gets started by the cronjob the next day - and then anotherone the day after - and anotherone...
If this job is not finished within e.g. 23 hours, I want to terminate it and return an error. Is there a way to do that out-of-the-box with spring-batch?
Using a StepListener you can stop a job by calling StepExecution#setTerminateOnly.
stepBuilderFactory
...
.writer(writer)
.listener(timeoutListener)
...
.build()
And the TimeoutListener could look like this
#Component
public class TimeoutListener implements StepListener {
private StepExecution stepExecution;
#BeforeStep
public void beforeStep(StepExecution stepExecution) {
this.stepExecution = stepExecution;
}
#BeforeRead
public void beforeRead() {
if (jobShouldStop()) {
stepExecution.setTerminateOnly();
}
}
private boolean jobShouldStop() {
// ...
}
}
This will gracefully stop the job, without forcefully terminate any running steps.
Spring Batch specifically avoids the issue of job orchestration which this falls into. That being said, you could add a listener to your job that checks for other instances running and calls stop on them before beginning that one. Not knowing what each job does, I'm not sure how effective that would be, but it should be a start.
If you write your own job class to launch the process you can make your class implement StatefulJob interface, which prevents concurrent launches of the same job. Apart from that you can write your own monitoring and stop the job programatically after some period, but it will require some custom coding, I dont know if there is anything build-in for such use case.
Related
I have a project which executes multiple scheduled method at start up.
I remarked that after scheduled methods are executed, the opened threads do not close, but remain in a 'parking' state.
Is this a normal behavior ?
Aren't the threads suppose to close after method is executed ? (Because keeping multiple threads open just slows down the application and consumes more RAM.)
Here are my code configurations:
#EnableScheduling
#Configuration
#ConditionalOnProperty(name = "scheduling.enabled", matchIfMissing = true)
public class SchedulingConfiguration implements SchedulingConfigurer {
}
Here is an example of method called in service:
#Scheduled(cron = "0 0 4 * * *")
protected void updateExchangeRates() {
if (enablePostConstruct) {
countryService.updateCountryExchangeRates();
}
}
I would like to run the scheduled methods asynchronously, with a max thread pool consumed between 10-15 threads. And after execution, the thread to close and reopen in case it got to the point when it needs to be executed again.
Can you guide me please how this can be achieved ?
I tried to implement SchedulingConfigurer and perform executorService.shutdown(), but it did not work.
You could use a method annotated with #PreDestroy to invoke executorService.shutdown(). I wouldn't bother about the Parking State, you probably want those threads to be ready for the next invocation, so not really harmful that they are parked.
Nothing wrong with the code.
I'm using Spring Batch with Spring cloud tasks. I have the following configuration in my job:
#Bean
public Job jobDemo(
#Value("${jobname}")String jobName,
JobBuilderFactory jobBuilderFactory,
JobCompletionNotificationListener listener
) {
return jobBuilderFactory.get(jobName)
.incrementer(new RunIdIncrementer())
.preventRestart()
.listener(listener)
.flow(stepA())
.end()
.build();
}
I don't want the restart functionality in the job, that's why I have put .preventRestart(). I want to launch a new job every time the task runs, that is, a new instance of the job to run even when the last time the job has failed or stopped or anything. But I'm getting the following error:
org.springframework.batch.core.repository.JobRestartException: JobInstance already exists and is not restartable
This happens only in the scenarios when the job does not finish sucessfully. Any ideas about the solution?
A JobInstance can only be completed once successfully. When you are starting a Spring Batch job via Spring Boot, Spring Batch handles the logic to increment a JobParameter if there is a JobParametersIncrementer provides (as you have). However...when Spring Batch does that incrementing, it only increments if the previous job was successful. In your case, you want it to always increment. Because of that, you're going to need to write your own CommandLineRunner that always increments the JobParameters.
Spring Boot's JobLauncherCommandLineRunner is where the code to launch a job exists. You'll probably want to extend that and override it's execute method to be sure job parameters are always incremented.
I am using Quartz to schedule a job in Java.
The scheduler repeats indefinitely.
If a certain exception occurs in the Job.class, I need to shutdown the scheduler.
How can I get the Job to notify the scheduling class to shutdown()?
Thanks,
RayK
Quartz's jobs are run by calling their execute(JobExecutionContext) method. This JobExecutionContext object has a getScheduler() method to access the Scheduler that is running your job.
So you could just do:
class YourJob implements Job {
//...
#Override
public void execute(final JobExecutionContext context) throws JobExecutionException {
//...
if (someCondition) {
try {
context.getScheduler().shutdown();
} catch (SchedulerException e) {
// log or rethrow!
}
}
//...
However, I'm not really sure if shutting the Scheduler down in the middle of running a job is a good idea. Why do you want to do that? If all you're trying to do is to stop some job from running again, it may be better to just unschedule it:
Unscheduling a Particular Trigger of Job:
// Unschedule a particular trigger from the job (a job may have more than one trigger)
scheduler.unscheduleJob(triggerKey("trigger1", "group1"));
Deleting a Job and Unscheduling All of Its Triggers
scheduler.deleteJob(jobKey("job1", "group1"));
I want to configure scheduler in my application where I have to set cron expression with database values dynamically. When application starts, a method should fetch the database values in set them in cron expression for a particular job. Please help me with this. I am all new to quartz scheduler, spring scheduler concepts
You can very well use TaskScheduler class of Spring Scheduling in this case.
Please have a look at the class definition:
https://docs.spring.io/spring/docs/current/javadoc-api/org/springframework/scheduling/TaskScheduler.html
scheduler.schedule(runnableTask, new CronTrigger(cron, TimeZone.getTimeZone(timezone)));
You can create a runnable task as follows:
class RunnableTask implements Runnable {
#Override
public void run() {
//
}
}
While creating a cron trigger, you can load cron expression from database.
You may want to look at this answer. https://stackoverflow.com/a/4499229/82632
Basically you need to autowire TaskScheduler class then programmatically add jobs with it.
i am trying to execute a task at a fixed rate using the #Scheduled annotation in java spring. however, it seems that by default spring will not execute a fixedRate task at a fixed rate if the task is slower than the rate. is there some setting i can add to my spring configuration to change this behavior?
example:
#Service
public class MyTask{
#Scheduled(fixedRate = 1000)
public void doIt(){
// this sometimes takes >1000ms, in which case the next execution is late
...
}
}
i have a work-around, but it seems less than ideal. basically, i just replace the default single-thread executor with a thread pool, then i have a scheduled method call an async method since the #Async annotation allows concurrent executions:
#Service
public class MyTask{
#Async
public void doIt(){
// this sometimes takes >1000ms, but the next execution is on time
...
}
}
#Service
public class MyTaskScheduler{
...
#Scheduled(fixedRate = 1000)
public void doIt(){
myTask.doIt();
}
}
#Configuration
#EnableScheduling
#EnableAsync
public class MySpringJavaConfig{
#Bean(destroyMethod = "shutdown")
public Executor taskScheduler() {
return Executors.newScheduledThreadPool(5);
}
}
boring details of my real-world scenario: in my production code i have a task that takes between 10ms and 10 minutes depending on the current workload. ideally, i would like to capture a new thread from the pool every 1000ms so that the number of concurrent threads increases with the workload. obviously i have an upper limit on threads in place (among other controls) to keep things from getting out of hand.
The TaskScheduler API (which backs some of the Spring Scheduling behavior) seems to be defined to prevent the behavior you are requesting
Schedule the given Runnable, invoking it at the specified execution
time and subsequently with the given period.
Parameters
period the interval between successive executions of the task (in milliseconds)
subsequently and successive seem to indicate that the next execution will only occur after the current execution is complete.
What's more, ScheduledExecutorService#scheduleAtFixedRate(..) (which the built-in TaskScheduler implementations use) also says
If any execution of this task takes longer than its period, then
subsequent executions may start late, but will not concurrently
execute.
So there's another layer of the implementation that prevents the behavior you want.
A possible solution, and one I don't recommend since the API doesn't seem to be built around it, is to define and provide your own TaskScheduler that does run the task concurrently. Look into #EnableScheduling and SchedulingConfigurer for how to register a TaskScheduler.
the best solution i have found so far is to simply use a delegate to make the method calls async. this is only preferable because it allows me to declare the schedule in the same class as the method which does the work:
#Service
public class AsyncRunner {
#Async
public void run(Runnable runnable) {
runnable.run();
}
}
#Service
public class MyTask{
...
#Scheduled(fixedRate = 1000)
public void scheduleIt(){
asyncRunner.run(this::doIt);
}
public void doIt(){
// this sometimes takes >1000ms, but the next execution is on time
...
}
}