I have the following setup for my quartz job -
<bean id="jobDetail" class="org.springframework.scheduling.quartz.MethodInvokingJobDetailFactoryBean">
<property name="targetObject" ref="actualObject" /><br>
<property name="targetMethod" value="processData"/>
<property name="concurrent" value="false"/>
</bean>
<bean id="simpleTrigger" class="org.springframework.scheduling.quartz.SimpleTriggerFactoryBean">
<property name="jobDetail" ref="jobDetail" />
<property name="startDelay" value="10000" />
<property name="repeatInterval" value="1000" />
</bean>
And this works good for me.
What I would like to do is to call the processData agai,n as soon as it finishes.
I know the good old while(true) method is best suited, but I want to do this with quartz.
First of all you must explain why you "want to do this with quartz" because the "good old while(true)" is the right way to implement your use case (of course you need an extra thread, but Quartz needs that as well). It sounds like over-engineering so you'll better have a good reason.
That being said you have two choices:
reschedule the job to run immediately when you leave it. In principle:
public class HelloJob implements Job {
public HelloJob() {
}
public void execute(JobExecutionContext context) throws JobExecutionException
{
//do your job...
Trigger trigger = newTrigger().build();
JobDetail job = newJob(HelloJob.class).build();
context.getScheduler().scheduleJob(trigger, job);
}
}
You don't need your XML configuration but must schedule this job to run for the very first time somehow (e.g. in #PostConstruct using Scheduler directly). The momemnt your job finishes, it will run the same job again as it was just rescheduled.
JobChainingJobListener might work for you, see: Can Quartz Scheduler Run jobs serially?
Both solutions are very heavyweight compared to "good old while(true)".
Related
I am trying to create an Spring batch asynchronous job with in another job. Say Job-1 should be completed and Job-2 should be executed. But problem is Job-1 is waiting till Job-2 is getting completed which i don't want. I have used JobStep as well but it is happening in an synchronous way and not helpful. Can some one help me how to use Asynchronously where Job-1 should not wait till Job-2 is completed ?
Sample xml snippet below
<bean id="taskExecutorAsync" class="org.springframework.core.task.SimpleAsyncTaskExecutor" />
<bean id="jobLauncherAsync" class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
<property name="jobRepository" ref="jobRepository" />
<property name="taskExecutor" ref="taskExecutorAsync" />
</bean>
<bean id="CreationProcess" class="test.CreationProcess">
<property name="jobLauncher" ref="jobLauncherAsync" />
<property name="jobRepository" ref="jobRepository" />
<property name="jobExplorer" ref="jobExplorer" />
</bean>
Thanks
You could use a SimpleAsyncTaskExecutor executor to avoid blocking.
I tried to create a separate thread, returned back and new thread updated the details. Unable to create a new asynchronous spring batch job with in another job.
In short, you can't using the JobStep. The reason is because a Job is a state machine with each Step serving as a state. In order for the Job to transition to the next state (aka complete in your use case), the current state (your child job) needs to complete.
You can launch jobs from other jobs, but to do so, you'll need to write a Tasklet to launch the job on a new thread (using a TaskExecutor) and return immediately.
Currently we are using java.util.concurrent.ExecutorService, java.util.concurrent.Future, java.util.concurrent.Callable to run cron jobs daily. Here our task is grabbing contacts from Gmail java api. We want to implement cron jobs with spring framework. please let me know how?
Thanks in advance..
For running the process at scheduled time, you can use any of the expression (unix cron expression or fixed delay/rate) with spring framework's Scheduled annotation.
public class DemoScheduleCron
{
#Scheduled(cron="*/10 * * * * ?")
//#Scheduled(fixedDelay = 10000)
//#Scheduled(fixedRate = 10000)
public void method1()
{
System.out.println("This method executs for every 10 seconds");
}
}
To implement grabbing contacts from Gmail api, you can use spring Quartz / scheduler which is explained here.
Small snippet of configuring concurrency from above docs:
<bean id="taskExecutor" class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
<property name="corePoolSize" value="5" />
<property name="maxPoolSize" value="10" />
<property name="queueCapacity" value="25" />
</bean>
<bean id="taskExecutorExample" class="TaskExecutorExample">
<constructor-arg ref="taskExecutor" />
</bean>
Spring batch + cron
please see the below link:
[1]http://www.mkyong.com/spring-batch/spring-batch-and-spring-taskscheduler-example/
There is the following Spring Integration configuration with Retry:
<int:chain input-channel="sdCreationChannel" output-channel="debugLogger">
<int:poller fixed-delay="500" />
<int:filter ref="sdIntegrationExistingRequestSentFilter" method="filter"/>
<int:service-activator ref="sdCreationServiceImpl" method="processMessage">
<int:request-handler-advice-chain>
<ref bean="retryAdvice"/>
</int:request-handler-advice-chain>
</int:service-activator>
</int:chain>
<bean id="retryAdvice" class="org.springframework.integration.handler.advice.RequestHandlerRetryAdvice" >
<property name="retryTemplate">
<bean class="org.springframework.retry.support.RetryTemplate">
<property name="backOffPolicy">
<bean class="org.springframework.retry.backoff.ExponentialBackOffPolicy">
<property name="initialInterval" value="${integration.retry.initial.delay}"/>
<property name="multiplier" value="${integration.retry.backoff.multiplier}"/>
</bean>
</property>
<property name="retryPolicy">
<bean class="org.springframework.retry.policy.SimpleRetryPolicy">
<property name="maxAttempts" value="${integration.retry.max.attempts}" />
</bean>
</property>
</bean>
</property>
</bean>
The simplified Java code is as follows:
#Component("sdCreationServiceImpl")
public class SDCreationServiceImpl implements SDCreationService {
#Autowired
private NotifySD notifySD;
#Override
public void processMessage(IntegrationPayload integrationPayload) {
List<ConfirmationCode> sdConfCodes = findCodesFromPayLoad(integrationPayload);
notifySD.activateConfirmationCodes(sdConfCodes);
}
The problem with retrying this code is that the List sdConfCodes can be partially processed at every Retry, so every time we need to send for processing the less amount of elements. What is best way to organize this code?
Following the Artem Bilan suggestion (Thanks!) I created the 2nd method with variable list in SDCreationServiceImpl, i.e. activateConfirmationCodes and then in the XML specification pointed to this method as a method for sdCreationServiceImpl.
#Component("sdCreationServiceImpl")
public class SDCreationServiceImpl implements SDCreationService {
#Autowired
private NotifySD notifySD;
List<ConfirmationCode> sdConfCodes = new ArrayList<ConfirmationCode()>;
#Override
public void processMessage(IntegrationPayload integrationPayload) {
sdConfCodes = findCodesFromPayLoad(integrationPayload);
}
public void activateConfirmationCodes()
{
notifySD.activateConfirmationCodes(sdConfCodes);
}
And then the XML spec for service-activator is as follows:
<int:service-activator ref="sdCreationServiceImpl" method="activateConfirmationCodes">
<int:request-handler-advice-chain>
<ref bean="retryAdvice"/>
</int:request-handler-advice-chain>
</int:service-activator>
Yes, this method activateConfirmationCodes is invoked in Retry but the 1st method processMessage is not invoked at all. Is it possible to specify one method for invoking at the first try and other method for retrying?
Second with this design the list becomes the singleton and this can give problems in multithreading, correct?. Can this list be associated with a bean only for a particular message?
From big it isn't clear where is your problem. From other side let me share some my thoughts, perhaps I will guess your goal.
Having the List<ConfirmationCode> as a payload allows us to modify it at any time. So, let's assume we have list as 10 elements. On the first attempt we have processed 3 of them. The fourth has failed. We have to go to retry throwing some appropriate exception.
But we come back to the beginning of the retry-aware method, so with the same arguments. If we remove those successful items from the collection, the next retry iteration won't process them at all.
From one side you can achieve that distinguishing findCodesFromPayLoad() service and activateConfirmationCodes(), applying retry for the last one.
From other side you can mark items as processed in the activateConfirmationCodes(), so the next findCodesFromPayLoad(integrationPayload) won't return them.
In other words there are enough ways to modify collection without changing messages.
Is there any possibility to know inside the onMessage method, which queue the MessageListener is listening to?
My Spring-config (a part of it):
<bean id="abstractMessageListenerContainer" class="org.springframework.jms.listener.DefaultMessageListenerContainer" abstract="true">
<property name="connectionFactory" ref="connectionFactory" />
<property name="maxConcurrentConsumers" value="5"/>
<property name="receiveTimeout" value="10000" />
</bean>
<bean class="org.springframework.jms.listener.DefaultMessageListenerContainer" parent="abstractMessageListenerContainer">
<property name="destinationName" value="MY.QUEUE" />
<property name="messageListener" ref="myMessageListener" />
</bean>
<bean id="myMessageListener" class="my.package.structure.ListenerClass"></bean>
My Listener Class:
public class ListenerClass implements MessageListener {
public void onMessage(Message msg) {
//where do I know from, on which queue the message has been written to?
}
}
Is there any out-of-the box solution? Or any custom solution to get the queue/destination-name?
Do need the queue in a subsequent batch-processing...
Easy. in trivial cases at least:
msg.getJMSDestination() will give you the destination as a javax.jms.Destination object. Typically .toString() returns something like: queue://MYQUEUENAME
However, in some JMS implementations, there might be multihop queues, such as a static pub/sub setup in WebSphere MQ where you might write your message to one queue and it will bounce around a route to end up in a completly different queue. Also, you might have the case of an ESB with logic in the middle that routes the message. In such cases, you will need to think twice before relying to much on the JMSDestination attribute. Otherwise, go ahead.
I have a Spring-Batch job that I launch from a Spring MVC controller. The controller gets an uploaded file from the user and the job is supposed to process the file:
#RequestMapping(value = "/upload")
public ModelAndView uploadInventory(UploadFile uploadFile, BindingResult bindingResult) {
// code for saving the uploaded file to disk goes here...
// now I want to launch the job of reading the file line by line and saving it to the database,
// but I want to launch this job in a new thread, not in the HTTP request thread,
// since I so not want the user to wait until the job ends.
jobLauncher.run(
jobRegistry.getJob(JOB_NAME),
new JobParametersBuilder().addString("targetDirectory", folderPath).addString("targetFile", fileName).toJobParameters()
);
return mav;
}
I've tried the following XML config:
<job id="writeProductsJob" xmlns="http://www.springframework.org/schema/batch">
<step id="readWrite">
<tasklet task-executor="taskExecutor">
<chunk reader="productItemReader" writer="productItemWriter" commit-interval="10" />
</tasklet>
</step>
</job>
<bean id="taskExecutor"
class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
<property name="corePoolSize" value="5" />
<property name="maxPoolSize" value="5" />
</bean>
...but it seems like the multithreading happens only within the job boundaries itself. I.e., the controller thread waits until the job ends, and the job execution is handled by multiple threads (which is good but not the main thing I wanted). The main thing I wanted is that the job will be launched on a separate thread (or threads) while the controller thread will continue its execution without waiting for the job threads to end.
Is there a way to achieve this with Spring-batch?
The official documentation describes your exact problem and a solution in 4.5.2. Running Jobs from within a Web Container:
[...] The controller launches a Job using a JobLauncher that has been configured to launch asynchronously, which immediately returns a JobExecution. The Job will likely still be running, however, this nonblocking behaviour allows the controller to return immediately, which is required when handling an HttpRequest.
Spring Batch http://static.springsource.org/spring-batch/reference/html-single/images/launch-from-request.png
So you were pretty close in trying to use TaskExecutor, however it needs to be passed to the JobLauncher instead:
<bean id="jobLauncher"
class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
<property name="jobRepository" ref="jobRepository" />
<property name="taskExecutor" ref="taskExecutor"/>
</bean>
Disclaimer: I have never used Spring Batch...
The jobLauncher.run() method can be called in a new Thread like so:
#RequestMapping(value = "/upload")
public ModelAndView uploadInventory(UploadFile uploadFile, BindingResult bindingResult) {
[...]
final SomeObject jobLauncher = [...]
Thread thread = new Thread(){
#Override
public void run(){
jobLauncher.run([...]);
}
};
thread.start();
return mav;
}
The thread.start() line will spawn a new thread, and then continue to execute the code below it.
Note that, if jobLauncher is a local variable, it must be declared final in order for it to be used inside of the anonymous Thread class.
If you don't need to show the processing errors to your client, you can start the spring batch job in a seperate thread.