I have a spring batch process which does data reading from the database and writing to file. Basically, the scenario is, that the user can send a request and the job will start and execute the process. But the issue is if the user sends the request 5 times there will be 5 different spring jobs started and running. But those are duplicates. So is there a way that we can avoid or block creating duplicate spring jobs?
You can create a JobExecutionListener that stops the current job execution if another one is already running and configure your job with that listener...
public class SingleExecutionJobListener implements JobExecutionListener {
private static String MATCH_ALL_PATTERN = ".*";
#Autowired
private JobExplorer jobExplorer;
#Autowired
private JobRegistry jobRegistry;
private String jobNamePattern = MATCH_ALL_PATTERN;
#Override
public void beforeJob(JobExecution jobExecution) {
Collection<String> jobNames = jobRegistry.getJobNames();
for (String jobName : jobNames) {
if (jobName.matches(StringUtils.defaultIfBlank(jobNamePattern, MATCH_ALL_PATTERN))) {
Set<JobExecution> jobExecutions = jobExplorer.findRunningJobExecutions(jobName);
if (CollectionUtils.isNotEmpty(jobExecutions)) {
for (JobExecution execution : jobExecutions) {
if (execution.getJobInstance().getId().compareTo(jobExecution.getJobInstance().getId()) != 0) {
jobExecution.stop();
throw new IllegalStateException(jobName + " instance " + execution.getJobInstance().getId()
+ " is currently running. Please restart this job when " + jobName + " has finished.");
}
}
}
}
}
}
#Override
public void afterJob(JobExecution jobExecution) {}
public String getJobNamePattern() {
return jobNamePattern;
}
public void setJobNamePattern(String jobNamePattern) {
this.jobNamePattern = jobNamePattern;
}
}
Related
I am using Spring Boot 2.6.6 and I want to configure scheduled tasks at runtime. The task configs are saved in a database. Here is my code:
#Configuration
#EnableScheduling
public class SchedulerConfig implements SchedulingConfigurer, DisposableBean {
private static final Logger logger = LoggerFactory.getLogger(SchedulerConfig.class);
private BackupJobsService backupJobsService;
private BackupService backupService;
ScheduledExecutorService executor = Executors.newSingleThreadScheduledExecutor();
public SchedulerConfig(#Autowired BackupJobsService backupJobsService, #Autowired BackupService backupService) {
this.backupJobsService = backupJobsService;
this.backupService = backupService;
}
#Override
public void configureTasks(ScheduledTaskRegistrar taskRegistrar) {
logger.info("Register all custom cron jobs...");
logger.debug("====> SchedulerConfig: configureTasks <====");
List<BackupJob> listOfJobs = backupJobsService.findAll();
for (BackupJob backupJob : listOfJobs) {
BackupTask backupTask = new BackupTask(backupJob, backupService);
Trigger trigger = new Trigger() {
#Override
public Date nextExecutionTime(TriggerContext triggerContext) {
logger.debug("====> SchedulerConfig: nextExecutionTime (" + backupJob.getJobName() + ") <====");
CronTrigger crontrigger = new CronTrigger(backupJob.getJobConfig());
return crontrigger.nextExecutionTime(triggerContext);
}
};
taskRegistrar.addTriggerTask(backupTask, trigger);
}
}
#Override
public void destroy() throws Exception {
logger.debug("====> SchedulerConfig: destroy <====");
if (executor != null) {
executor.shutdownNow();
}
}
}
The object backupJob has all Infos about the job. In the field JobConfig save the cron string for the job.
This class work perfect if I start the spring boot application. How can I reconfigure the tasks without a restart of the spring boot application ?
I try to check Kafka topics on startup spring-boot application. I want to throw an exception and interrupt startup. It is my config:
#Slf4j
#Configuration
public class KafkaTopicConfig implements ApplicationRunner {
private final KafkaAdmin kafkaAdmin;
private final TopicProperties topicProperties;
public KafkaTopicConfig(KafkaAdmin kafkaAdmin, TopicProperties topicProperties) {
this.kafkaAdmin = kafkaAdmin;
this.topicProperties = topicProperties;
}
#Override
public void run(ApplicationArguments args) throws Exception {
AdminClient admin = AdminClient.create(kafkaAdmin.getConfig());
ListTopicsResult listTopicsResult = admin.listTopics();
listTopicsResult.names().whenComplete((existTopics, throwable) -> {
log.info("TOPICS LOAD: {}", existTopics.size());
topicProperties.getTopics().forEach((s, topic) -> {
if (!existTopics.contains(topic))
throw new IllegalStateException("Topic with name: " + topic + " not found in kafka.");
});
});
}
}
But after throws throw new IllegalStateException("Topic with name: " + topic + " not found in kafka."); this exception ignored and application continue works.
Instead of ApplicationRunner, implement SmartLifecycle with autoStartup=true and put your logic in start().
I have the following scheduled piece of code in my Spring Boot Application:
#Scheduled(fixedDelay = DELAY_SECONDS)
private void processJobQueue() {
BlockingQueue<ReportDeliverable> jobQueue = JobQueueFactory.getJobQueueInstance();
while (!jobQueue.isEmpty()) {
//do stuff
if (rCount == 0) {
status = send(reportDeliverable);
if (status == TransferStatus.FAILURE) {
populateQueue(reportDeliverable);
}
if (status == TransferStatus.SUCCESS) { //write the metadata to database
int i = dbMetadataWriter.writeMetadata(reportDeliverable);
}
} else if (rCount == -1) {
populateQueue(reportDeliverable);
} else
logger.info("Record exists in MetaData for {}. Ignoring the File transfer....", reportDeliverable.getFilePath());
}
}
In my DBMetadataWriter component, the writeMetadataWriter() looks something like this:
#Component
public class DBMetadataWriter {
public int writeMetadata(final ReportDeliverable reportDeliverable) {
int nbInserted = 0;
try {
nbInserted = jdbcTemplate.update(PORTAL_METADATA_INSERT, insertDataValues);
} catch (Exception e) {
logger.error("Could not insert metadata for {}, Exception: {} ", reportDeliverable.toString(), e.getMessage());
}
return nbInserted;
}
In some cases, when writing the insert to the database, I get table space issues with the database at which point I think it would be wise for me to shut down the spring boot application until table space problems are resolved.
What would be the correct way to handle these rare cases? What technique can I use to gracefully shutdown the spring boot application and how can I do it in the above code?
My entry point class where I initially validate all my database connections before processing etc has the following...
#Component
public class RegisterReportSchedules implements ApplicationListener<ContextRefreshedEvent> {
#Autowired
private ApplicationContext applicationContext;
#Override
public void onApplicationEvent(ContextRefreshedEvent contextRefreshedEvent) {
}
private void shutdownApplication() {
int exitCode = SpringApplication.exit(applicationContext, (ExitCodeGenerator) () -> 0);
System.exit(exitCode);
}
}
You have exit() method on SpringApplication class, which can be used for exiting Spring boot application gracefully.
It requires 2 paramerter:
ApplicationContext
ExitCodeGenerator
For further reading:
https://docs.spring.io/spring-boot/docs/current/api/org/springframework/boot/SpringApplication.html#exit-org.springframework.context.ApplicationContext-org.springframework.boot.ExitCodeGenerator...-
Code Example:
#Autowired
public void shutDown(ExecutorServiceExitCodeGenerator exitCodeGenerator) {
SpringApplication.exit(applicationContext, exitCodeGenerator);
}
Call this method when you get exception for No Table space
Facing terrible issue... .
I have two timers of Class java.util.Timer, scheduledReportTimer which sends daily emails and scheduleImmediateReportTimer which send emails on every 10 mins.
scheduledReportTimer is working perfectly fine.
scheduleImmediateReportTimer is runs every 10 mins but it seems its running twice in 10 mins or there are two thread gets created for scheduleImmediateReportTimer (i am not sure what exactly it is)
but it is calling the email sending method twice
i tested email sending logic for lots of conditions and which is perfect .
please help me.
public class ScheduleReportManager implements ServletContextListener{
private ServletContext application = null;
private Environment environment =null;
private Timer scheduledReportTimer=null;
private Timer scheduleImmediateReportTimer=null; //daily timer
private ConcurrentHashMap scheduledReportTimerTasks;
private ConcurrentHashMap scheduledImmediateReportTimerTasks; //daily timer hash map
ApplicationContext webContext=null;
public ScheduleReportManager() {
}
public void contextInitialized(ServletContextEvent servletContextEvent) {
try{
this.application= servletContextEvent.getServletContext();
webContext = (ApplicationContext) application.getAttribute(WebApplicationContext.ROOT_WEB_APPLICATION_CONTEXT_ATTRIBUTE);
//daily timer
if(application.getAttribute("scheduledReportTimer")==null)
{
application.setAttribute("scheduleReportManager",this);
this.scheduledReportTimer= new Timer(true);
setupSchedule(scheduledReportTimer, application);
application.setAttribute("scheduledReportTimer", scheduledReportTimer);
}
//timer for 10 mins
if(application.getAttribute("scheduleImmediateReportTimer")==null)
{
this.scheduleImmediateReportTimer= new Timer(true);
setupImmediateSchedule(scheduleImmediateReportTimer, application);
application.setAttribute("scheduleImmediateReportTimer", scheduleImmediateReportTimer);
}
Logger.global.log(Level.INFO, "ScheduledReportTimer: " + application.getServletContextName() + ": Setup completed");
} catch (Exception e)
{
Logger.global.log(Level.SEVERE, "Setup Report Timer Exception - " + e.toString());
}
}
//timer for 10 mins
public void setupImmediateSchedule(Timer scheduledImmediateReportTimer,ServletContext application ) {
scheduledImmediateReportTimerTasks= new ConcurrentHashMap();
ScheduleImmediateReportTimerTask immediateReportTimerTask = new ScheduleImmediateReportTimerTask(application,environment,this);
scheduledImmediateReportTimerTasks.put(environment.getCode(),immediateReportTimerTask);
scheduledImmediateReportTimer.schedule(immediateReportTimerTask,1000);
}
//timer for 10 mins
public void setTimerForImmediateReportExecution(ScheduleImmediateReportTimerTask immediateReportTimerTask){
Environment environment = immediateReportTimerTask.getEnvironment();
ServletContext application = immediateReportTimerTask.getApplication();
ScheduleImmediateReportTimerTask reportTimerTask= new ScheduleImmediateReportTimerTask(application,environment,this);
String environmentCode = environment.getCode();
synchronized (scheduledImmediateReportTimerTasks){
scheduledImmediateReportTimerTasks.put(environmentCode,reportTimerTask);
scheduleImmediateReportTimer.schedule(reportTimerTask,600000); // set timer for running every 10 mins
}
}
//daily timer
public void setTimerForNextExecution(ScheduledReportTimerTask timerTask, DateTime nextExecutionDateTime)
{
if(nextExecutionDateTime == null)
return;
Environment environment = timerTask.getEnvironment();
ServletContext application = timerTask.getApplication();
ScheduledReportTimerTask scheduledReportTimerTask = new ScheduledReportTimerTask(application,environment,this);
java.util.Date nextScheduleTime = nextExecutionDateTime.getDate();
String environmentCode = environment.getCode();
synchronized (scheduledReportTimerTasks){
scheduledReportTimerTasks.put(environmentCode,scheduledReportTimerTask);
Logger.global.log(Level.INFO, "ScheduledReportManager: next execution time is " + nextScheduleTime.toString());
scheduledReportTimer.schedule(scheduledReportTimerTask,nextScheduleTime);
}
}
//daily timer
public void setupSchedule(Timer scheduledReportTimer, ServletContext application) throws Exception{
this.environment = SpringBridgeUtil.retrieveEnvironment(application);
this.scheduledReportTimerTasks= new ConcurrentHashMap();
ScheduledReportTimerTask scheduledReportTimerTask = new ScheduledReportTimerTask(application,environment,this);
scheduledReportTimerTasks.put(environment.getCode(),scheduledReportTimerTask);
scheduledReportTimer.schedule(scheduledReportTimerTask,1000);
}
public void contextDestroyed(ServletContextEvent servletContextEvent) {
String contextName = application.getServletContextName();
if (scheduledReportTimer != null) {
scheduledReportTimer.cancel();
}
if(scheduleImmediateReportTimer !=null)
{
scheduleImmediateReportTimer.cancel();
}
Logger.global.log(Level.INFO, "scheduledReportTimer: " + contextName
+ ": Tasks cancelled due to context removal");
}
}
// sends email on every 10 mins
public class ScheduleImmediateReportTimerTask extends TimerTask {
#Override
public void run() {
try{
// send mail
sendNotifiationEmail();
}catch (Exception e) {
e.printStackTrace();
Logger.global.log(Level.INFO,"immediateScheduledReportTimerTask Exception " + e.toString());
}
}
}
i am using jdk 1.7 and spring mvc 3.5.0 .
Classes using java.util.Timer and import java.util.TimerTask;
Let me know if this is the correct way or is there any other way to get execute schedule task periodically .
I have one spring batch job which can be kicked of by rest URL. I want to make sure only one job instance is allowed to run. and if another instance already running then don't start another. even if the parameters are different.
I searched and found nothing out of box solution. thinking of extending SimpleJobLauncher. to check if any instance of the job running or not.
You could try to intercept the job execution, implementing the JobExecutionListener interface:
public class MyJobExecutionListener extends JobExecutionListener {
//active JobExecution, used as a lock.
private JobExecution _active;
public void beforeJob(JobExecution jobExecution) {
//create a lock
synchronized(jobExecution) {
if(_active!=null && _active.isRunning()) {
jobExecution.stop();
} else {
_active=jobExecution;
}
}
}
public void afterJob(JobExecution jobExecution) {
//release the lock
synchronized(jobExecution) {
if(jobExecution==_active) {
_active=null;
}
}
}
}
And then, inject to the Job definition:
<job id="myJobConfig">
<listeners>
<listener ref="myListener"/>
</listeners>
</job>
I solved this by creating an JobExecutionListner and with the help of JobExplorer I checked if any other instance is running if running then stop current job.I created listener so that it can be plugged in to any job that requires this kind of scenario.
Set<JobExecution> jobExecutions = ((SimpleJobExplorer) jobExplorer.getObject()).findRunningJobExecutions(jobExecution.getJobInstance().getJobName());
if(jobExecutions.size()>1){
Long currentTime = (new Date()).getTime();
for(JobExecution execution : jobExecutions ){
if(execution.getJobInstance().getId().compareTo(jobExecution.getJobInstance().getId())!=0 && (currentTime - execution.getStartTime().getTime()) <lockOverideTime){
jobExecution.stop();
throw new IllegalStateException("Another instance of the job running job name : " +jobExecution.getJobInstance().getJobName() );
}
}
}
Or, in response to REST URL, check using JobExplorer if your job is running using job's specifics business rules
I think a simple method like the following might do the trick:
#Autowire
private JobExplorer jobExplorer;
private boolean isJobRunning(Job job) {
Set<JobExecution> jobExecutions = jobExplorer.findRunningJobExecutions(job.getName());
return !jobExecutions.isEmpty();
}
Then, prior to executing your job make the check:
private void executeJob(Job job, #Nonnull JobParameters params) {
if (isJobRunning(job)) {
return;
}
try {
jobLauncher.run(job, params);
} catch (JobExecutionAlreadyRunningException | JobRestartException | JobInstanceAlreadyCompleteException | JobParametersInvalidException e) {
log.error("could not run job " + jobIdentifier, e);
}
}