Develop Reference

How to run spring batch job multiple times with different job parameters?

We have created a simple spring batch job with single step. There are custom implemented ItemReader and ItemWriter. The ItemReader gets the initial data from job parameter. The batch runs perfectly when run as a standalone java process. But what we want is to host the batch on some server. Therefore, we have created REST service to initialize the batch. The service calls the job URL and passes some parameter. This parameter is passed as job parameter to the batch. The service and job run fine when it is called for one parameter.
But when we call the service more than once (twice for testing purpose), the batch behaves strangely. We are passing different job parameters. But when the execution starts for second job initialization, the job parameter value which is received by the ItemReader is the same as the one for the first execution. And both execution interfere with each other, sharing database connection, interfering with data retrieved etc.
We have tried setting the restartable parameter to false but it didn't work. We have also tried the following solution:
Can we create multiple instances of a same java(spring) batch job?
The above solution started giving "Interrupted attempting lock" error in JBoss.
On further investigation we found that ItemReader is getting initialized only once. That is why it is getting same job parameter value and is interfering with the previous execution.
EDIT
Following is the job configuration:
<bean id="jobLauncher"
class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
<property name="jobRepository" ref="jobRepository" />
<job id="jobid" restartable="false">
<step id="step1">
<tasklet>
<chunk reader="reader" writer="writer"
commit-interval="2">
</chunk>
</tasklet>
</step>
</job>
Following is the code snippet to launch the job:
JobLauncher jobLauncher = (JobLauncher) context.getBean("jobLauncher");
Job job = (Job) context.getBean("jobid");
try {
JobParameters param = new JobParametersBuilder().addString("key","value").toJobParameters();
JobExecution execution = jobLauncher.run(job, param);
} catch (Exception e) {
e.printStackTrace();
}
Can anyone please suggest some solution? Am I missing some configuration for the step?
Thanks in advance.

I found that if we create the Context and JobLauncher objects statically, that is, if there is only one instance of these two objects, the above thing can work. In this way, we can launch the same job multiple times, but with different parameters.
Class MyClass{
private static ConfigurableApplicationContext context = null;
private static JobLauncher jobLauncher = null;
static{
String[] springConfig = {BatchTokeniserConstants.SPRING_CONFIG_FILE_NAME};
try {
context = new ClassPathXmlApplicationContext(springConfig);
jobLauncher = (JobLauncher) context.getBean("jobLauncher");
BatchTokeniserUtils.loadSystemVaiables();
} catch (BeansException e) {
}
}
}
Now the jobLauncher can be used to launch any job any number of time.
I hope it helps others.

MongoDB throws NullPointerException in Class DBTCPConnector

I'm trying to get a grip using MongoDB with the Spring Data MongoDB framework. I tried severeal approaches to connect to my local DB and insert + retrieve some collections and documents, using the official Spring Reference Documentation and some simple examples like this Hello-World-Demo.
Actually, for the beginning I'm going to use the MongoTemplate to keep it simple. But now I'run into the following problem.
When I use the Spring Configuration with annotations configure the setting needed to connect to my local DB, everything works fine.
Otherwise When I use XML for the configuration, I run into an java.lang.NullPointerException at com.mongodb.DBTCPConnector.getClusterDescription
Here are my configuration files and the example code for connecting to the DB:
Use Case 1 - Spring Configuration with annotations:
//package, imports etc. here
#Configuration
public class MongoConfiguration {
public #Bean MongoDbFactory mongoDbFactory() throws Exception {
return new SimpleMongoDbFactory(new MongoClient(), "Test1");
}
public #Bean MongoTemplate mongoTemplate() throws Exception {
return new MongoTemplate(mongoDbFactory());
}
}
Using the configuration class like this ...
AnnotationConfigApplicationContext ctx = new AnnotationConfigApplicationContext(MongoConfiguration.class);
MongoOperations mongoOperation = (MongoOperations) ctx.getBean("mongoTemplate");
ctx.close();
for (String s : mongoOperation.getCollectionNames()) {
System.out.println(s);
}
.. creates this output:
documents
leute
system.indexes
system.users
Use Case 2 - XML configuration (SpringConfig2.xml):
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:context="http://www.springframework.org/schema/context"
xmlns:mongo="http://www.springframework.org/schema/data/mongo"
xsi:schemaLocation="http://www.springframework.org/schema/context
http://www.springframework.org/schema/context/spring-context-3.0.xsd
http://www.springframework.org/schema/data/mongo
http://www.springframework.org/schema/data/mongo/spring-mongo-1.0.xsd
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-3.0.xsd">
<mongo:mongo host="127.0.0.1" port="27017" />
<mongo:db-factory dbname="Test1" />
<bean id="mongoTemplate" class="org.springframework.data.mongodb.core.MongoTemplate">
<constructor-arg name="mongoDbFactory" ref="mongoDbFactory" />
</bean>
</beans>
Using the configuration file like this ...
GenericXmlApplicationContext ctx = new GenericXmlApplicationContext("SpringConfig2.xml");
MongoOperations mongoOperation = (MongoOperations) ctx.getBean("mongoTemplate");
ctx.close();
for (String s : mongoOperation.getCollectionNames()) {
System.out.println(s);
}
.. results in this error
java.lang.NullPointerException
at com.mongodb.DBTCPConnector.getClusterDescription(DBTCPConnector.java:404)
at com.mongodb.DBTCPConnector.getMaxBsonObjectSize(DBTCPConnector.java:653)
at com.mongodb.Mongo.getMaxBsonObjectSize(Mongo.java:641)
at com.mongodb.DBCollectionImpl.find(DBCollectionImpl.java:81)
at com.mongodb.DBCollectionImpl.find(DBCollectionImpl.java:66)
at com.mongodb.DB.getCollectionNames(DB.java:510)
at org.springframework.data.mongodb.core.MongoTemplate$13.doInDB(MongoTemplate.java:1501)
at org.springframework.data.mongodb.core.MongoTemplate$13.doInDB(MongoTemplate.java:1499)
at org.springframework.data.mongodb.core.MongoTemplate.execute(MongoTemplate.java:394)
at org.springframework.data.mongodb.core.MongoTemplate.getCollectionNames(MongoTemplate.java:1499)
at test.main(Test.java:28)
When debugging DBTCPConnector.getClusterDescription, it seems that in the second case the private class variable cluster is for some reason not instantiated, leading to the described error.
What I'd like to know is: am I doing anything wrong within my XML-configuration or when using this config / context? Why does using XML-configuration end in an error, while using annotation configuration just works fine?
Basically (in the end) I just "copy+paste"'d the code examples from the official references for Spring / Spring Data MongoDB.
I'd appreciate any help / suggestions :)

Thanks to the hint from Tushar Mishra in the comments I was able to track down the origin of the error and why it occurs in one case but not in the other.
I'll try to explain in short, maybe it'll save someone some research time (or remember me if I perhaps run into this or a similar error again).
When closing either the AnnotationConfigApplicationContext-object (UC1) or the AnnotationConfigApplicationContext-object (UC2) with ctx.close(), from somewhere
org.springframework.beans.factory.DisposableBean#destroy() is invoked. As far as I understood, within there used singleton beans get destroyed by invoking the destroy()-methods
of each of those beans.
In short: closing the XXApplicationContext-object also destroys the MongoFactoryBean and with it closes the connection to the MongoDB.
So using c**tx.close()** at the described position leads to using a MongoOperations-object on a closed connection at the next line, resulting in the described NullPointerException.
And for why the sample code with Annotations (UC1) runs fine, whereas the sample code configured with XML (UC2) just breaks with an NullPointerException:
In UC1, the org.springframework.beans.factory.DisposableBean#destroy()-call is interupted by some DisposableBeanMethodInterceptor, which tries to redirect to a dispose() method.
But there is no method implemented in MongoFactoryBean, so the Mongo-connection stays alive and can be used even after ctx.close() was invoked. I think the connection will get killed later by the garbage collector, leading to the same error.
I'm not sure what to make of this, yet. If I should implement a dispose()-method myself or something alike. But at least I figured out, why this code sample behave different, although supposed to do the same thing.
Maybe it helps someone. Thanks again, Tushar, for giving a hint to the right direction.

Scheduling Task in Spring/Java

I am spawning a thread which will keep pulling the chunk of records from database and putting them into the queue. This thread will be started on the server load. I want this thread to be active all the time. If there are no records in the database I want it to wait and check again after some time. I was thinking of using spring task scheduler to schedule this but not sure if that is right because I only want my task to be started once. What will be the good way of implementing this in Spring ?
Also, i need to have a boundary check that if my thread goes down (because of any error or exception condition) it should be re-instantiated after some time.
I can do all this in java by using thread communication methods but just trying if there is something available in Spring or Java for such scenarios.
Any suggestions or pointer will help.

You can use the #Scheduled annotation to run jobs. First create a class with a method that is annotated with #Scheduled.
Class
public class GitHubJob {
#Scheduled(fixedDelay = 604800000)
public void perform() {
//do Something
}
}
Then register this class in your configuration files.
spring-context.xml
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:tx="http://www.springframework.org/schema/tx"
xmlns:task="http://www.springframework.org/schema/task"
xmlns:context="http://www.springframework.org/schema/context"
xsi:schemaLocation="http://www.springframework.org/schema/task http://www.springframework.org/schema/task/spring-task-3.1.xsd
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.1.xsd
http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-3.0.xsd">
<tx:annotation-driven/>
<task:annotation-driven scheduler="myScheduler"/>
<task:scheduler id="myScheduler" pool-size="10"/>
<bean id="gitHubJob" class="org.tothought.spring.jobs.GitHubJob"/>
</beans>
For more about scheduling visit the Spring Docs.

#Scheduled(fixedDelay=3600000)
private void refreshValues() {
[...]
}
That runs a task once every hour. It needs to be void and accept no arguments. If you use Spring's Java configuration you'll also need to add the annotation #EnableScheduling to one of your #Configuration classes.

You can try using Quartz scheduler. http://quartz-scheduler.org/
That will allow you to specify the duration of time between task executions. You can set a flag (boolean) that says if the class has run before to avoid duplicating code that you only want to run the 'first' time.

Spring has out-ot-the-box support for scheduling tasks. Here are for the docs for the latest 3.2.x version of spring but check the docs for the version you are using. Looks like it uses Quartz under the hood for scheduling tasks.

I think your requirement is just regular senario which quartz or spring scheduling framework supports very well. but you want to create a spececial approach to impl it. my suggestion is to keep it simple and stupid. spring scheudling will take advantage your worker thread by pooling instead of runing it all the time.
As of statistics, it 's easy to check worker class's log. if you want see the statistics in web console, you need to record worker's log in database. I am sure you could make it easily in normal way.

Spring Batch: How to find out if job is restarted

Is there any possibility to find out, If a job is restarted in Spring Batch?
We do provide some Tasklets without restart-support from spring-batch and has to implement our own proceeding, if job is restarted.
Can't find any possibility in JobRepository, JobOperator, JobExplorer, etc.

Define a JobExplorer bean with required properties
<bean id="jobExplorer"
class="org.springframework.batch.core.explore.support.JobExplorerFactoryBean">
<property name="dataSource" ref="dataSource"/>
<property name="lobHandler" ref="lobHandler"/>
</bean>
Query it with your jobName
List<JobInstance> jobInstances= jobExplorer.getJobInstances(jobName);
for (JobInstance jobInstance : jobInstances) {
List<JobExecution> jobExecutions = jobExplorer.getJobExecutions(jobInstance);
for (JobExecution jobExecution : jobExecutions) {
if (jobExecution.getExitStatus().equals(ExitStatus.COMPLETED)) {
//You found a completed job, possible candidate for a restart
//You may check if the job is restarted comparing jobParameters
JobParameters jobParameters = jobInstance.getParameters();
//Check your running job if it has the same jobParameters
}
}
}
Did not compile this but I hope it gives an idea

Another way using jobExplorer is execute the following command:
jobExplorer.getJobExecutions(jobExplorer.getJobInstance(currentJobExecution.getJobInstance().getId())).size() > 1;
This statement verifies if another execution of the the same job (same id) exists. In environments with minimum control, does not exist possibility that the other execution be not a failed or stopped execution.

Potentially you can find this information in spring-batch's database tables, can't remeber the exact table's name, but you can figure out quickly because there are only few tables. I guess there is some information regarding restarting.

Spring Batch: starting a job from within a Spring MVC contorller WITH A NEW THREAD

I have a Spring-Batch job that I launch from a Spring MVC controller. The controller gets an uploaded file from the user and the job is supposed to process the file:
#RequestMapping(value = "/upload")
public ModelAndView uploadInventory(UploadFile uploadFile, BindingResult bindingResult) {
// code for saving the uploaded file to disk goes here...
// now I want to launch the job of reading the file line by line and saving it to the database,
// but I want to launch this job in a new thread, not in the HTTP request thread,
// since I so not want the user to wait until the job ends.
jobLauncher.run(
jobRegistry.getJob(JOB_NAME),
new JobParametersBuilder().addString("targetDirectory", folderPath).addString("targetFile", fileName).toJobParameters()
);
return mav;
}
I've tried the following XML config:
<job id="writeProductsJob" xmlns="http://www.springframework.org/schema/batch">
<step id="readWrite">
<tasklet task-executor="taskExecutor">
<chunk reader="productItemReader" writer="productItemWriter" commit-interval="10" />
</tasklet>
</step>
</job>
<bean id="taskExecutor"
class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
<property name="corePoolSize" value="5" />
<property name="maxPoolSize" value="5" />
</bean>
...but it seems like the multithreading happens only within the job boundaries itself. I.e., the controller thread waits until the job ends, and the job execution is handled by multiple threads (which is good but not the main thing I wanted). The main thing I wanted is that the job will be launched on a separate thread (or threads) while the controller thread will continue its execution without waiting for the job threads to end.
Is there a way to achieve this with Spring-batch?

The official documentation describes your exact problem and a solution in 4.5.2. Running Jobs from within a Web Container:
[...] The controller launches a Job using a JobLauncher that has been configured to launch asynchronously, which immediately returns a JobExecution. The Job will likely still be running, however, this nonblocking behaviour allows the controller to return immediately, which is required when handling an HttpRequest.
Spring Batch http://static.springsource.org/spring-batch/reference/html-single/images/launch-from-request.png
So you were pretty close in trying to use TaskExecutor, however it needs to be passed to the JobLauncher instead:
<bean id="jobLauncher"
class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
<property name="jobRepository" ref="jobRepository" />
<property name="taskExecutor" ref="taskExecutor"/>
</bean>
Disclaimer: I have never used Spring Batch...

The jobLauncher.run() method can be called in a new Thread like so:
#RequestMapping(value = "/upload")
public ModelAndView uploadInventory(UploadFile uploadFile, BindingResult bindingResult) {
[...]
final SomeObject jobLauncher = [...]
Thread thread = new Thread(){
#Override
public void run(){
jobLauncher.run([...]);
}
};
thread.start();
return mav;
}
The thread.start() line will spawn a new thread, and then continue to execute the code below it.
Note that, if jobLauncher is a local variable, it must be declared final in order for it to be used inside of the anonymous Thread class.

If you don't need to show the processing errors to your client, you can start the spring batch job in a seperate thread.