I am using quartz to schedule a spring batch job.
The job reads a file from a folder( which has multiple files ) does some processing and copies it to another folder.
is it possible to create multiple instances of the job, that will run concurrenty,reading multiple files ?
My question is :
In spring batch, is it possible to spawn multiple instances of the same job? I am using quartz schedular ?
In Spring Batch it is possible to start several jobs, provided you have supplied different JobParameters for each jobLauncher.run() call. jobLauncher in your Spring configuration will spawn each job in a separate thread, if it is configured with appropriate task executor:
<bean id="jobLauncher" class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
<property name="jobRepository" ref="jobRepository" />
<property name="taskExecutor" ref="taskExecutor" />
</bean>
<bean id="taskExecutor" class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor"
p:corePoolSize="5"
p:maxPoolSize="30" />
It is possible with Quartz, using a MethodInvokingJobDetailFactoryBean, for instance:
<bean id="myjob"
class="org.springframework.scheduling.quartz.MethodInvokingJobDetailFactoryBean">
<property name="targetObject">
<ref bean="someBean" />
</property>
<property name="targetMethod" value="someMethod" />
<!-- The concurrent property is already true by default
<property name="concurrent" value="true" />
-->
</bean>
Citing the Spring documentation
By default, Quartz Jobs are stateless, resulting in the possibility of jobs interfering with each other. If you specify two triggers for the same JobDetail, it might be possible that before the first job has finished, the second one will start. If JobDetail classes implement the Stateful interface, this won't happen. The second job will not start before the first one has finished. To make jobs resulting from the MethodInvokingJobDetailFactoryBean non-concurrent, set the concurrent flag to false.
Related
I currently have multiple Quartz pollers that check a multiple flags in the database and fire a service if flags are true and I also have one spring batch project reading files, then calling a service to update the information accordingly.
What i am wondering is if it is possible to get the spring batch application to only call a service very 5 minutes (dont check for any information just call a service which will handle all the business logic) so that i can remove the Quartz jobs from my application all my jobs are in one place (spring batch project)?
If so can any point me in the right direction on how to do this as i cannot find anything online to just set up a spring batch job just to fire a service?
You can simply schedule a Quartz job and call your service from executeInternal method of your schedular class. You will not be able to autowire your service directly in schedular class, so you need to set it using jobDataas Map.
<bean class="org.springframework.scheduling.quartz.SchedulerFactoryBean">
<property name="triggers">
<list>
<bean id="cronTriggerCache"
class="org.springframework.scheduling.quartz.CronTriggerBean">
<property name="jobDetail" ref="exampleJob" />
<property name="cronExpression" value="your cron expression" />
</bean>
</list>
</bean>
<bean name="exampleJob" class="org.springframework.scheduling.quartz.JobDetailFactoryBean">
<property name="jobClass" value="example.ExampleJob"/>
<property name="jobDataAsMap">
<map>
<entry key="myService" ref="myService"/>
</map>
</property>
</bean>
package example;
public class ExampleJob extends QuartzJobBean {
private myService myService;
/**
* have getter setter for you service
*
*/
protected void executeInternal(JobExecutionContext ctx) throws JobExecutionException {
// do the actual work
//call your service method here
}
}
I am using spring batch in my application. I am using multithreading using spring batch at step level .
Below is my configuration:
<step id="testStep">
<tasklet
task-executor="taskExecutor" throttle-limit="5">
<chunk reader="reader" writer="writer"
commit-interval="10">
</chunk>
</tasklet>
</step>
<bean id="writer" class="org.springframework.batch.item.file.FlatFileItemWriter"
scope="step">
<property name="resource" value="file:${output.file.location.directory}/<<<NEED TO PASS CURRENT EXECUTING THREAD NAME>>>" />
<property name="appendAllowed" value="true"></property>
<property name="lineAggregator" ref="aggregator" />
</bean>
So in resource property of flatfileitemWriter , i need to pass current executing thread name, so that each thread wil write in a separate unique file.
I can able to get the current executing thread name in writer class but dont know how to get the same in the spring confirguration file .
Any idea to do so ?
If you want you can use spEL; with this scripting language you can use standard java method and to retrive current thread name the syntax is #{T(java.lang.Thread).getCurrentThread().getName()}.
In a more clear way use a StepExecutionListener.beforeStep() injecting the current thread name into step execution context (eg. currentThreadName key) and retrieve from step xml config in the standard way
<property name="resource" value="file:${output.file.location.directory}/#{stepExecutionContext['currentThreadName']}" />
I have seen various thread on timertask issues. However I would like some clarification on the inner working on spring Scheduler APIs (3.1).
I have a requirement to kick off a timer every 10 secs. The application runs in clustered websphere zos. (atleast 4 jvm nodes).
Here is the wiring.
<bean id="dataProcessSchedulerTask" class="org.springframework.scheduling.timer.MethodInvokingTimerTaskFactoryBean">
<property name="targetObject" ref="ondataTransferTimerWakeupService" />
<property name="targetMethod" value="processDataFeedMetadata" />
</bean>
<bean id="DATA_PROCESS_TIMER"
class="org.springframework.scheduling.timer.ScheduledTimerTask">
<property name="timerTask" ref="DATAProcessSchedulerTask" />
<property name="delay" value="#{systemProperties.DATA_PROCESS_TIMER}" />
<property name="period" value="#{systemProperties.DATA_PROCESS_TIMER}" />
</bean>
<bean class="org.springframework.scheduling.timer.TimerFactoryBean">
<property name="scheduledTimerTasks">
list>
<ref bean="DATA_PROCESS_TIMER" />
</list>
</property>
</bean>
<bean id="onDATATransferTimerWakeupService" class="com.serviceimpl.OnDATATransferTimerWakeupService" />
I have defined (#async) for processDataFeedMetadata method.
The problem I am seeing the timer which fires every 10 secs correctly to begin with (4 jvms - 24 timer occurrences every min), started misbehaving after few hours (2 or 3 jvms stop firing any timer at all - 6 to 12 timer occurrences every min). I understand timertask implementation has it's own limitation. However, if I invoke an ASYNC method from MethodInvokingTimerTaskFactoryBean, Why would the timertask misbehave as
(1) timer should complete well within the interval as soon as it invokes ASYNC method.
(2) I don't see any exception from app logs and MethodInvokingTimerTaskFactoryBean should have handled and consume any exception if there are any.
Really appreciate if anyone has input on what is going on here ?
I suggest you utilizing Quartz Scheduler for Spring Scheduler implementation
http://quartz-scheduler.org/
Do you have this problem with 1 jvm ?
I have a simple Spring Scheduled Taks defined by the following:
<context:component-scan base-package="com/test"/>
<task:scheduled-tasks>
<task:scheduled ref="myScheduler" method="doMyTask" fixed-rate="300000"/>
</task:scheduled-tasks>
<task:scheduler id="taskScheduler" pool-size="1"/>
<task:executor id="executorWithPoolSizeRange"
pool-size="1"
queue-capacity="100"/>
<bean id="cleanupClass" class="com.test.CleanupClass">
<property name="myProperty" value="3600"/>
</bean>
I would like to run a single thread synchronously every 5 minutes. However, what I get is FIVE instances of the task running consecutively every 5 minutes. Does anyone know if there is something missing from the XML description above?
I got the behavior I wanted using the #Scheduled annotations but I would rather not use annotation for the fixed-rate as I want it to be configurable outside of the code.
Thanks.
the following worked for me:
<bean id="task" class="com.foo.MyTask">
<task:scheduled-tasks scheduler="scheduler">
<task:scheduled ref="task" method="run" fixed-delay="300000" />
</task:scheduled-tasks>
<task:scheduler id="scheduler" pool-size="10" />
Greetings,
Mark
Is this the behavior you are seeing in the STS when you deploy it to tomcat? If so, you would want to undeploy the application, redeploy it and restart the application.
Another idea is to use SPEL expression from a properties file to use it with #Sched annotation. In that way it is still configurable while using that annotation.
I'm using org.springframework.scheduling.quartz.CronTriggerBean for trigger job.
Could you propose cron expression to trigger job execution only once on application startup pls?
I believe the actual answer is: no, you can't.
What you could do however when using Spring 3.1 (which is Milestone 2 at the time of writing) is create profiles which can be enabled for different environments. So you can use different beans and bean configurations depending on profiles you enable.
Instead of using scheduling, I would prefer to invoke your trigger via defining an init-method in your spring configuration
To run a job only once at startup with the Spring Quartz scheduler you can use the org.springframework.scheduling.quartz.SimpleTriggerBean which doesn't take a cronExpression but a startDelay and a repeatCount. Set the repeatCount to 0 for a single execution (see Quartz documentation on SimpleTrigger for further options).
<bean id="doJobOnceOnStartupTrigger" class="org.springframework.scheduling.quartz.SimpleTriggerBean">
<property name="jobDetail" ref="myJobDetail" />
<property name="startDelay" value="5000" />
<property name="repeatCount" value="0" />
</bean>
<bean id="myJobDetail" class="org.springframework.scheduling.quartz.MethodInvokingJobDetailFactoryBean">
<property name="targetObject" ref="myBean" />
<property name="targetMethod" value="myMethod" />
</bean>
Additional information can be found int the
spring documentation: Chapter 22. Scheduling jobs using Quartz or Timer
You can use a SimpleTriggerBean - with a cron expression, you won't achieve this