Develop Reference

How to use HibernateCursorItemReader in spring batch

In a spring batch job, I have a step in which I want to read data by chunk from the database using Hibernate, process it and write it back to the database. Right now I'm having some trouble setting up the reader :
#EnableBatchProcessing
#Configuration
#ComponentScan(basePackages = {"com.x.y.z.database", "com.x.y.p.database"}, excludeFilters={
#ComponentScan.Filter(type= FilterType.ASSIGNABLE_TYPE, value=ConfigHelperMail.class)})
#EntityScan(basePackages = {"com.x.y.z.database", "com.x.y.p.database"})
#EnableJpaRepositories({"com.x.y.z.database", "com.x.y.p.database"})
public class BatchConfiguration {
#Autowired
public JobBuilderFactory jobs;
#Bean
public Job job(CustomJobListener listener,
#Qualifier("step1") Step step1,
#Qualifier("step2") Step step2,
#Qualifier("step3") Step step3) {
return jobs.get("SimpleJobName")
.incrementer(new RunIdIncrementer())
.preventRestart()
.listener(listener)
.start(step1)
.next(step2)
.next(step3)
.build();
}
}
In step1 writer I save the data to the database using Hibernate :
#Entity
#Table(name = "tmp_z_stepone")
public class StepOneEntity {
#Id
#Column(name = "reference")
private String reference;
/** Other properties and getters and setters **/
}
#Component
public class StepOneWriter implements ItemWriter<StepOneEntity> {
#Autowired
private StepOneService stepOneService;
#Override
public void write (List<? extends StepOneEntity> items) throws WriterException {
stepOneService.saveMany(items);
}
}
StepOneService is annotated with #Service and it uses a DAO interface that extends CrudRepository<StepOneEntity, String> and its saveMany method just uses the save method from CrudRepository<StepOneEntity, String>.
Step 2
#Component
public class StepOne {
#Autowired
private StepTwoReader reader;
#Autowired
private StepTwoProcessor processor;
#Autowired
private StepTwoWriter writer;
#Bean
#JobScope
#Qualifier("step2")
public Step step2() throws ReaderException {
return stepBuilderFactory.get("step2")
.<StepOneEntity, StepTwoEntity>chunk(10)
.reader(reader.read())
.processor(processor)
.writer(writer)
.build();
}
}
#Component
public class StepTwoReader {
public ItemReader<OutputControleFormat> read () throws ReaderException {
HibernateCursorItemReader itemReader = new HibernateCursorItemReader();
itemReader.setQueryString("from tmp_z_stepone");
itemReader.setUseStatelessSession(true);
return itemReader;
}
}
I think I'm missionf some session/ORM configuration but I don't know where, the error is below :
org.springframework.batch.item.ItemStreamException: Failed to initialize the reader
at org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader.open(AbstractItemCountingItemStreamItemReader.java:147) ~[spring-batch-infrastructure-3.0.8.RELEASE.jar:3.0.8.RELEASE]
at org.springframework.batch.item.support.CompositeItemStream.open(CompositeItemStream.java:96) ~[spring-batch-infrastructure-3.0.8.RELEASE.jar:3.0.8.RELEASE]
at org.springframework.batch.core.step.item.ChunkMonitor.open(ChunkMonitor.java:114) ~[spring-batch-core-3.0.8.RELEASE.jar:3.0.8.RELEASE]
at org.springframework.batch.item.support.CompositeItemStream.open(CompositeItemStream.java:96) ~[spring-batch-infrastructure-3.0.8.RELEASE.jar:3.0.8.RELEASE]
at org.springframework.batch.core.step.tasklet.TaskletStep.open(TaskletStep.java:310) ~[spring-batch-core-3.0.8.RELEASE.jar:3.0.8.RELEASE]
at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:197) ~[spring-batch-core-3.0.8.RELEASE.jar:3.0.8.RELEASE]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_92]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_92]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_92]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_92]
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333) [spring-aop-4.3.13.RELEASE.jar:4.3.13.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190) [spring-aop-4.3.13.RELEASE.jar:4.3.13.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) [spring-aop-4.3.13.RELEASE.jar:4.3.13.RELEASE]
at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:133) [spring-aop-4.3.13.RELEASE.jar:4.3.13.RELEASE]
at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:121) [spring-aop-4.3.13.RELEASE.jar:4.3.13.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) [spring-aop-4.3.13.RELEASE.jar:4.3.13.RELEASE]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213) [spring-aop-4.3.13.RELEASE.jar:4.3.13.RELEASE]
at com.sun.proxy.$Proxy110.execute(Unknown Source) [na:na]
at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:148) [spring-batch-core-3.0.8.RELEASE.jar:3.0.8.RELEASE]
at org.springframework.batch.core.job.AbstractJob.handleStep(AbstractJob.java:392) [spring-batch-core-3.0.8.RELEASE.jar:3.0.8.RELEASE]
at org.springframework.batch.core.job.SimpleJob.doExecute(SimpleJob.java:135) [spring-batch-core-3.0.8.RELEASE.jar:3.0.8.RELEASE]
at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:306) [spring-batch-core-3.0.8.RELEASE.jar:3.0.8.RELEASE]
at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:135) [spring-batch-core-3.0.8.RELEASE.jar:3.0.8.RELEASE]
at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50) [spring-core-4.3.13.RELEASE.jar:4.3.13.RELEASE]
at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:128) [spring-batch-core-3.0.8.RELEASE.jar:3.0.8.RELEASE]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_92]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_92]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_92]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_92]
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333) [spring-aop-4.3.13.RELEASE.jar:4.3.13.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190) [spring-aop-4.3.13.RELEASE.jar:4.3.13.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) [spring-aop-4.3.13.RELEASE.jar:4.3.13.RELEASE]
at org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:127) [spring-batch-core-3.0.8.RELEASE.jar:3.0.8.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) [spring-aop-4.3.13.RELEASE.jar:4.3.13.RELEASE]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213) [spring-aop-4.3.13.RELEASE.jar:4.3.13.RELEASE]
at com.sun.proxy.$Proxy107.run(Unknown Source) [na:na]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.execute(JobLauncherCommandLineRunner.java:211) [spring-boot-autoconfigure-1.5.9.RELEASE.jar:1.5.9.RELEASE]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.executeLocalJobs(JobLauncherCommandLineRunner.java:227) [spring-boot-autoconfigure-1.5.9.RELEASE.jar:1.5.9.RELEASE]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.launchJobFromProperties(JobLauncherCommandLineRunner.java:123) [spring-boot-autoconfigure-1.5.9.RELEASE.jar:1.5.9.RELEASE]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.run(JobLauncherCommandLineRunner.java:117) [spring-boot-autoconfigure-1.5.9.RELEASE.jar:1.5.9.RELEASE]
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:732) [spring-boot-1.5.9.RELEASE.jar:1.5.9.RELEASE]
at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:716) [spring-boot-1.5.9.RELEASE.jar:1.5.9.RELEASE]
at org.springframework.boot.SpringApplication.afterRefresh(SpringApplication.java:703) [spring-boot-1.5.9.RELEASE.jar:1.5.9.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:304) [spring-boot-1.5.9.RELEASE.jar:1.5.9.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1118) [spring-boot-1.5.9.RELEASE.jar:1.5.9.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1107) [spring-boot-1.5.9.RELEASE.jar:1.5.9.RELEASE]
at com.cdn.oxc.signature.ApplicationConfig.main(ApplicationConfig.java:33) [classes/:na]
Caused by: java.lang.NullPointerException: null
at org.springframework.batch.item.database.HibernateItemReaderHelper.createQuery(HibernateItemReaderHelper.java:141) ~[spring-batch-infrastructure-3.0.8.RELEASE.jar:3.0.8.RELEASE]
at org.springframework.batch.item.database.HibernateItemReaderHelper.getForwardOnlyCursor(HibernateItemReaderHelper.java:125) ~[spring-batch-infrastructure-3.0.8.RELEASE.jar:3.0.8.RELEASE]
at org.springframework.batch.item.database.HibernateCursorItemReader.doOpen(HibernateCursorItemReader.java:185) ~[spring-batch-infrastructure-3.0.8.RELEASE.jar:3.0.8.RELEASE]
at org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader.open(AbstractItemCountingItemStreamItemReader.java:144) ~[spring-batch-infrastructure-3.0.8.RELEASE.jar:3.0.8.RELEASE]
... 46 common frames omitted

HibernateCursorItemReader implements the InitializingBean interface, which means that Spring will call its afterPropertiesSet on container initialization if it is declared as a bean. However, in your case, the HibernateCursorItemReader is not declared as a bean in the application context, it is wrapped in a class (StepTwoReader) then created with reader.read().
If you declare the HibernateCursorItemReader as a bean with your current code in your read method, you should get an IllegalStateException with message: "A SessionFactory must be provided". You actually need to configure a SessionFactory for your Hibernate reader.
Another option is to use the HibernateCursorItemReaderBuilder which will ensure the reader is correctly configured.

#StepScope
#Slf4j
#Component
public class CursorItemReader extends HibernateCursorItemReader<ItemEntity> {
public CursorItemReader(EntityManagerFactory entityManagerFactory,
#Value("#{stepExecution}")StepExecution stepExecution) {
this.setName("CursorItemReader");
this.setSessionFactory(entityManagerFactory.createEntityManager().unwrap(org.hibernate.Session.class).getSessionFactory());
this.setQueryString("from ItemEntity");
this.setUseStatelessSession(true);
this.setFetchSize(5);
}
#Override
public ItemEntity read() throws Exception {
ItemEntity item = this.doRead();
return item;
}
}
You can also refer-
https://docs.spring.io/spring-batch/docs/current/reference/html/readersAndWriters.html#HibernateCursorItemReader

Spring boot repository does not save to the DB if called from scheduled job

I have a spring boot application in which I need to schedule a job to read files from a specific directory and store the data into the DB.
I used Spring batch for handling the files part as the number of files is very large.
The application has a component named PraserStarer which has a method named startParsing. This method is annotated with #scheduled annotation.
#scheduled(fixedDelay = 60 * 1000)
public startParsing(){
// start spring batch job
}
I have a repository interface NewsRepositry injected into the writer of the spring batch first step.
The application has a simple controller to manually call the startParsing method. When calling the startParsing method from the controller, everything works fine. The spring batch job starts normally, read the files, write the data into the DB, and archive the files.
When the method startParsing is invoked from the scheduling framework, the spring batch job starts normally, and read the files but nothing is stored the DB.
I suspect the problem here is that there are two different contexts, one for the scheduling part and another for the rest of the application.
For some reason, there is no transaction manager in the scheduling context which causes nothing to go to the DB.
1- Is my suspicion correct?
2- If yes, how can I force the transaction manager to be loaded to the other context?
EDIT
The code for the parser starter class is below
#Component
public class ParserStarter {
#Autowired
JobLauncher jobLauncher;
#Value("${app.data_directory}")
private String dataDir;
#Autowired
private ParserJobListener jobListener;
#Autowired
private JobBuilderFactory jobBuilderFactory;
public Resource[] getResources() throws IOException {
// return array of file resource to be processed
}
// #Scheduled(fixedDelay = 60 * 1000)
public void startParsing() throws Exception {
String jobName = System.currentTimeMillis() + " New Parser Job";
JobParameters jobParameters = new JobParametersBuilder().addString("source", jobName).toJobParameters();
jobLauncher.run(getParsingJob(), jobParameters);
}
#Bean(name="getParsingJob")
private Job getParsingJob() throws IOException {
jobListener.setResources(getResources());
Step processingStep = jobListener.processingStep();
Step archivingStep = jobListener.archivingStep();
Job job = jobBuilderFactory.get("Store News").incrementer(new RunIdIncrementer())
.listener(jobListener).start(processingStep).next(archivingStep).build();
return job;
}
}
The code for the job listener is below
#Component
public class ParserJobListener extends JobExecutionListenerSupport {
#Autowired
private StepBuilderFactory stepBuilderFactory;
private Resource[] resources;
#Value("${app.archive_directory}")
private String archiveDirectory;
#Autowired
private Writer writer;
public MultiResourceItemReader<DataRecord> multiResourceItemReader() {
MultiResourceItemReader<DataRecord> resourceItemReader = new MultiResourceItemReader<DataRecord>();
resourceItemReader.setResources(resources);
resourceItemReader.setDelegate(new Reader());
return resourceItemReader;
}
public Step archivingStep() {
FileArchivingTask archivingTask = new FileArchivingTask(resources, archiveDirectory);
return stepBuilderFactory.get("Archiving step").tasklet(archivingTask).build();
}
public Step processingStep() {
return stepBuilderFactory.get("Process news file").<DataRecord, DataRecord>chunk(1000)
.reader(multiResourceItemReader()).writer(writer).build();
}
#Override
public void afterJob(JobExecution jobExecution) {
if (jobExecution.getStatus() == BatchStatus.COMPLETED) {
System.out.println("Job finished")
}
}
public void setResources(Resource[] resources) {
this.resources = resources;
}
}
What is remaining is the writer and it is below
#Component
public class Writer implements ItemWriter<DataRecord>{
#Autowired
private DataRepository dataRepo;
#Override
public void write(List<? extends DataRecord> items) throws Exception {
dataRepo.saveAll(items);
}
}
Edit 2
I have changed the writer's write method to save and flush each item indiviually as following
#Transactional
public void write(List<? extends GdeltRecord> items) throws Exception {
for (GdeltRecord gdeltRecord : items) {
dataRepo.saveAndFlush(gdeltRecord);
}
// dataRepo.saveAll(items);
}
This time the application throws a TransactionRequiredException: no transaction is in progress exception.
Here is the full stack trace of the exception
Caused by: javax.persistence.TransactionRequiredException: no transaction is in progress
at org.hibernate.internal.SessionImpl.checkTransactionNeeded(SessionImpl.java:3552) ~[hibernate-core-5.3.7.Final.jar:5.3.7.Final]
at org.hibernate.internal.SessionImpl.doFlush(SessionImpl.java:1444) ~[hibernate-core-5.3.7.Final.jar:5.3.7.Final]
at org.hibernate.internal.SessionImpl.flush(SessionImpl.java:1440) ~[hibernate-core-5.3.7.Final.jar:5.3.7.Final]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_191]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_191]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_191]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_191]
at org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerInvocationHandler.invoke(ExtendedEntityManagerCreator.java:350) ~[spring-orm-5.1.4.RELEASE.jar:5.1.4.RELEASE]
at com.sun.proxy.$Proxy87.flush(Unknown Source) ~[na:na]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_191]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_191]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_191]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_191]
at org.springframework.orm.jpa.SharedEntityManagerCreator$SharedEntityManagerInvocationHandler.invoke(SharedEntityManagerCreator.java:308) ~[spring-orm-5.1.4.RELEASE.jar:5.1.4.RELEASE]
at com.sun.proxy.$Proxy87.flush(Unknown Source) ~[na:na]
at org.springframework.data.jpa.repository.support.SimpleJpaRepository.flush(SimpleJpaRepository.java:533) ~[spring-data-jpa-2.1.4.RELEASE.jar:2.1.4.RELEASE]
at org.springframework.data.jpa.repository.support.SimpleJpaRepository.saveAndFlush(SimpleJpaRepository.java:504) ~[spring-data-jpa-2.1.4.RELEASE.jar:2.1.4.RELEASE]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_191]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_191]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_191]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_191]
at org.springframework.data.repository.core.support.RepositoryComposition$RepositoryFragments.invoke(RepositoryComposition.java:359) ~[spring-data-commons-2.1.4.RELEASE.jar:2.1.4.RELEASE]
at org.springframework.data.repository.core.support.RepositoryComposition.invoke(RepositoryComposition.java:200) ~[spring-data-commons-2.1.4.RELEASE.jar:2.1.4.RELEASE]
at org.springframework.data.repository.core.support.RepositoryFactorySupport$ImplementationMethodExecutionInterceptor.invoke(RepositoryFactorySupport.java:644) ~[spring-data-commons-2.1.4.RELEASE.jar:2.1.4.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) [spring-aop-5.1.4.RELEASE.jar:5.1.4.RELEASE]
at org.springframework.data.repository.core.support.RepositoryFactorySupport$QueryExecutorMethodInterceptor.doInvoke(RepositoryFactorySupport.java:608) ~[spring-data-commons-2.1.4.RELEASE.jar:2.1.4.RELEASE]
at org.springframework.data.repository.core.support.RepositoryFactorySupport$QueryExecutorMethodInterceptor.lambda$invoke$3(RepositoryFactorySupport.java:595) ~[spring-data-commons-2.1.4.RELEASE.jar:2.1.4.RELEASE]
at org.springframework.data.repository.core.support.RepositoryFactorySupport$QueryExecutorMethodInterceptor.invoke(RepositoryFactorySupport.java:595) ~[spring-data-commons-2.1.4.RELEASE.jar:2.1.4.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) [spring-aop-5.1.4.RELEASE.jar:5.1.4.RELEASE]
at org.springframework.data.projection.DefaultMethodInvokingMethodInterceptor.invoke(DefaultMethodInvokingMethodInterceptor.java:59) ~[spring-data-commons-2.1.4.RELEASE.jar:2.1.4.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) [spring-aop-5.1.4.RELEASE.jar:5.1.4.RELEASE]
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:294) ~[spring-tx-5.1.4.RELEASE.jar:5.1.4.RELEASE]
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98) ~[spring-tx-5.1.4.RELEASE.jar:5.1.4.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) [spring-aop-5.1.4.RELEASE.jar:5.1.4.RELEASE]
at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:139) ~[spring-tx-5.1.4.RELEASE.jar:5.1.4.RELEASE]
... 66 common frames omitted

I tried the approach described in this question (JpaItemWriter: no transaction is in progress) and it worked for me.
I defined a JpaTransactionManager bean and used it with step configuration.
#Bean
#Primary
public JpaTransactionManager jpaTransactionManager() {
final JpaTransactionManager transactionManager = new JpaTransactionManager();
transactionManager.setDataSource(dataSource);
return transactionManager;
}
and in the step configuration
#Autowired
JpaTransactionManager trxm;
public Step processingStep(Resource[] resources) throws IOException {
return stepBuilderFactory.get("Process CSV File")
.transactionManager(trxm)
.<DataRecord, DataRecord>chunk(1000)
.reader(multiResourceItemReader()).writer(writer).build();
}

It's difficult to analyze what exactly is happening without seeing the full code. However, based on the documentation, spring uses chunk oriented processing. This is what it says:
Once the number of items read equals the commit interval, the entire chunk is written out via the ItemWriter, and then the transaction is committed.
This might be why you are not seeing any db writes immediately.
Regarding the transaction manager, you can define it like this (explained here):
#Bean
public Job sampleJob(JobRepository jobRepository, Step sampleStep) {
return this.jobBuilderFactory.get("sampleJob")
.repository(jobRepository)
.start(sampleStep)
.build();
}
/**
* Note the TransactionManager is typically autowired in and not needed to be explicitly
* configured
*/
#Bean
public Step sampleStep(PlatformTransactionManager transactionManager) {
return this.stepBuilderFactory.get("sampleStep")
.transactionManager(transactionManager)
.<String, String>chunk(10)
.reader(itemReader())
.writer(itemWriter())
.build();
}

We have to explicitly mention JpaTransactionManager instead of default spring batch transactions.
#Configuration
#EnableBatchProcessing
public class MyJob extends DefaultBatchConfigurer {
#Autowired
private DataSource dataSource;
#Bean
#Primary
public JpaTransactionManager jpaTransactionManager() {
final JpaTransactionManager tm = new JpaTransactionManager();
tm.setDataSource(dataSource);
return tm;
}
}

QuartzJobBean could not obtain transaction-synchronized Session

When I run my SpringBootApplication and my QuartzJobBean triggers I get a HibernateSystemException that says Could not obtain transaction-synchronized Session for current thread. I found a lot of similar problems (on SO) when I googled the Exception in the first place but none of the provided answers helped me out.
My ApplicationConfig looks like this. Here I set up my TransactionManager and the Datasource etc.
#SpringBootApplication(scanBasePackages = {"de.xxx.yyy"}, exclude = HibernateJpaAutoConfiguration.class)
#EnableTransactionManagement
public class ApplicationConfig extends SpringBootServletInitializer {
...
#Bean
#Autowired
public HibernateTransactionManager transactionManager(SessionFactory sessionFactory) {
HibernateTransactionManager txManager = new HibernateTransactionManager();
txManager.setSessionFactory(sessionFactory);
return txManager;
}
#Bean
public LocalSessionFactoryBean sessionFactory() {
...
sessionFactory.setDataSource(dataSource());
sessionFactory.setHibernateProperties(hibernateProperties());
...
}
}
Then I have another Config for Quartz - the QuartzSchedulerConfig. It configures the triggers, details and the scheduler
#Configuration
#EnableTransactionManagement
public class QuartzSchedulerConfig {
#Autowired
DataSource dataSource;
#Autowired
ApplicationContext context;
#Autowired
QuartzTriggerProperties quartzTriggerProperties;
#Autowired
HibernateTransactionManager transactionManager;
#Bean
public SchedulerFactoryBean schedulerFactoryBean() throws ParseException, IOException {
SchedulerFactoryBean factory = new SchedulerFactoryBean();
AutowiringSpringBeanJobFactory jobFactory = new AutowiringSpringBeanJobFactory();
jobFactory.setApplicationContext(context);
...
factory.setTransactionManager(transactionManager);
factory.setJobFactory(jobFactory);
...
factory.setTriggers(collectorTrigger());
return factory;
}
...
#Bean
public Trigger collectorTrigger() throws ParseException {
CronTriggerFactoryBean factoryBean = new CronTriggerFactoryBean();
factoryBean.setJobDetail(collectorJobDetail());
factoryBean.setName("collectorTrigger");
factoryBean.setCronExpression(quartzTriggerProperties.getCollectorTrigger())
factoryBean.afterPropertiesSet();
return factoryBean.getObject();
}
#Bean
public JobDetail collectorJobDetail() {
JobDetailFactoryBean factoryBean = new JobDetailFactoryBean();
factoryBean.setJobClass(CollectorJob.class);
factoryBean.setDurability(true);
factoryBean.setApplicationContext(context);
factoryBean.setName("collectorJob");
factoryBean.afterPropertiesSet();
return factoryBean.getObject();
}
...
}
Now when I start the Application the CollectorJob gets executed, processes its tasks and finally when it tries to save the TransportVO to the database it fails with this Exception:
org.quartz.SchedulerException: Job threw an unhandled exception. [See nested exception:
org.springframework.orm.hibernate5.HibernateSystemException: Could not obtain transaction-synchronized Session for current thread; nested exception is org.hibernate.HibernateException: Could not obtain transaction-synchronized Session for current thread]
at org.quartz.core.JobRunShell.run(JobRunShell.java:213)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)
Caused by: org.springframework.orm.hibernate5.HibernateSystemException: Could not obtain transaction-synchronized Session for current thread; nested exception is org.hibernate.HibernateException: Could not obtain transaction-synchronized Session for current thread
at org.springframework.orm.hibernate5.SessionFactoryUtils.convertHibernateAccessException(SessionFactoryUtils.java:296)
at org.springframework.orm.hibernate5.HibernateExceptionTranslator.convertHibernateAccessException(HibernateExceptionTranslator.java:68)
at org.springframework.orm.hibernate5.HibernateExceptionTranslator.translateExceptionIfPossible(HibernateExceptionTranslator.java:49)
at org.springframework.dao.support.ChainedPersistenceExceptionTranslator.translateExceptionIfPossible(ChainedPersistenceExceptionTranslator.java:59)
at org.springframework.dao.support.DataAccessUtils.translateIfNecessary(DataAccessUtils.java:213)
at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:147)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:656)
at de.xxx.yyy.database.jobs.TransportDAO$$EnhancerBySpringCGLIB$$27fdef97.save(<generated>)
at de.xxx.yyy.business.services.collector.CollectorService.saveEventForTransporter(CollectorService.java:314)
at de.xxx.yyy.business.services.collector.CollectorService.downloadAndSaveArchiv(CollectorService.java:243)
at de.xxx.yyy.business.services.collector.CollectorService.saveArchivesToDownload(CollectorService.java:110)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:724)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:656)
at de.xxx.yyy.business.services.collector.CollectorService$$EnhancerBySpringCGLIB$$cdf948ab.saveArchivesToDownload(<generated>)
at de.xxx.yyy.business.services.collector.CollectorJob.downloadArchives(CollectorJob.java:123)
at de.xxx.yyy.business.services.collector.CollectorJob.doDownloadForAllServer(CollectorJob.java:87)
at de.xxx.yyy.business.services.collector.CollectorJob.executeInternal(CollectorJob.java:73)
at org.springframework.scheduling.quartz.QuartzJobBean.execute(QuartzJobBean.java:75)
at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
... 1 more
Caused by: org.hibernate.HibernateException: Could not obtain transaction-synchronized Session for current thread
at org.springframework.orm.hibernate5.SpringSessionContext.currentSession(SpringSessionContext.java:133)
at org.hibernate.internal.SessionFactoryImpl.getCurrentSession(SessionFactoryImpl.java:719)
at de.xxx.yyy.database.protocoll.QueryDSLSupport.getSession(QueryDSLSupport.java:35)
at de.xxx.yyy.database.jobs.TransportDAO.save(TransportDAO.java:65)
at de.xxx.yyy.database.jobs.TransportDAO$$FastClassBySpringCGLIB$$d4e14826.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:721)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:136)
... 22 more
The Service-Method that calls the save-Method of the TransportDAO is annotated with #Transactional and when the customer is using the application there is no problem with TransactionManagement an Session obtaining. So I'm guessing that my QuartzScheduler is somehow misconfigured. Any ideas what I have to change to make the QuartzJobs work?

how to test DAO Implementations methods in spring mvc with java configuration classes and no XML using jUnit

i'm working in a spring mvc project using netbeans i created a netbeans maven web project and i configured it to be a spring mvc project with java classes configuration and NO-XML, i'm trying to do a unit test of a insert DAO implementation method and my autowired DAO variable is always null
i was following this tutorial http://spring.io/blog/2011/06/21/spring-3-1-m2-testing-with-configuration-classes-and-profiles
here is my MvcConfig class
#Configuration
#ComponentScan("controller")
#EnableWebMvc
public class MvcConfig extends WebMvcConfigurerAdapter {
#Override
public void addResourceHandlers(ResourceHandlerRegistry registry) {
registry.addResourceHandler("/resources/**").addResourceLocations("/resources/");
}
#Override
public void configureDefaultServletHandling(DefaultServletHandlerConfigurer configurer) {
configurer.enable();
}
//JDBCTEMPLATE
#Bean
public DataSource getDataSource() {
DriverManagerDataSource dataSource = new DriverManagerDataSource();
dataSource.setDriverClassName("oracle.jdbc.driver.OracleDriver");
dataSource.setUrl("jdbc:oracle:thin:#127.0.0.1:1521:XE");
dataSource.setUsername("myPC");
dataSource.setPassword("fakePassword");
return dataSource;
}
//CREATE DAO BEAN
#Bean
public UserDAO getUserDAO() {
return new UserDAOImplementation( getDataSource() );
}
}
here is my User DAO
public interface UserDAO{
public void insertUser(User user);
}
and my UserDAOImplementation
public class UserDAOImplementation implements UserDAO{
private JdbcTemplate jdbcTemplate;
public UserDAOImplementation () {}
public UserDAOImplementation (DataSource datasource) {
this.jdbcTemplate = new JdbcTemplate(datasource);
}
#Override
public void insertUser(User user) {
String sql = "INSERT INTO User"
+ "(field1, field2, field3, field4, field5, field6)"
+ " VALUES (SEQ_user.nextval, ?, ?, ?, ?, ?)";
jdbcTemplate.update(sql, user.getField1(),
user.getField2(),
user.getField3(),
user.getField4(),
user.getField5() );
}
}
and here is my test class i have commented "#RunWith(SpringJUnit4ClassRunner.class)" beacuse it gives me this error
org/junit/runners/model/MultipleFailureException
java.lang.NoClassDefFoundError
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.withAfterClasses(SpringJUnit4ClassRunner.java:187)
at org.junit.runners.ParentRunner.classBlock(ParentRunner.java:145)
at org.junit.runners.ParentRunner.run(ParentRunner.java:235)
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:162)
at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:53)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:123)
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:104)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:164)
at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:110)
at org.apache.maven.surefire.booter.SurefireStarter.invokeProvider(SurefireStarter.java:175)
at org.apache.maven.surefire.booter.SurefireStarter.runSuitesInProcessWhenForked(SurefireStarter.java:107)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:68)
Caused by: java.lang.ClassNotFoundException: org.junit.runners.model.MultipleFailureException
at java.net.URLClassLoader$1.run(URLClassLoader.java:372)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 16 more
this is my test class
// this is commented because it gives me errors #RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration(classes=MvcConfig .class, loader=AnnotationConfigContextLoader.class)
public class UserTest{
User user;
#Autowired
private UserDAO userDAO;
#Before
public void setUp() {
System.out.println("#Before - setUp");
user = new User();
user.setField1("a");
user.setField2("a");
user.setField3("a");
user.setField4("a");
user.setField5("a");
}
#Test
public void insertUser () {
userDAO.insertUser(user);
}
the AutoWired userDAO variable is always null
i'm running the test by right clicking it and choose run file
this are the imports i'm using
import org.springframework.test.context.ContextConfiguration;
import org.springframework.test.context.junit4.SpringJUnit4ClassRunner;
import org.springframework.test.context.support.AnnotationConfigContextLoader;
i'm using netbeans 8.0.2, my thoughts are that the problem is because i'm not using this #RunWith(SpringJUnit4ClassRunner.class) beacuse it gives me the errors that i mention before, but i dont know maybe i'm missing something, this insert method works fine outside the test i just wanted to test it to test some new methods in the future and i wanted to learn how to test methods with a java configuration
EDIT
i looked at my folder Test Depencies and inside my package org.junit.runners.model theres no a MultipleFailureException i dont know if that could be the error
EDIT 2
i updated my junit version from junit 4.7 to junit 4.12 and know i have this error ERROR: Failed to load ApplicationContext
Failed to load ApplicationContext
java.lang.IllegalStateException
at org.springframework.test.context.DefaultCacheAwareContextLoaderDelegate.loadContext(DefaultCacheAwareContextLoaderDelegate.java:91)
at org.springframework.test.context.DefaultTestContext.getApplicationContext(DefaultTestContext.java:74)
at org.springframework.test.context.support.DependencyInjectionTestExecutionListener.injectDependencies(DependencyInjectionTestExecutionListener.java:116)
at org.springframework.test.context.support.DependencyInjectionTestExecutionListener.prepareTestInstance(DependencyInjectionTestExecutionListener.java:82)
at org.springframework.test.context.TestContextManager.prepareTestInstance(TestContextManager.java:212)
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.createTest(SpringJUnit4ClassRunner.java:199)
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner$1.runReflectiveCall(SpringJUnit4ClassRunner.java:251)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.methodBlock(SpringJUnit4ClassRunner.java:253)
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:216)
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:82)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:60)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:67)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:162)
at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:53)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:123)
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:104)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:164)
at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:110)
at org.apache.maven.surefire.booter.SurefireStarter.invokeProvider(SurefireStarter.java:175)
at org.apache.maven.surefire.booter.SurefireStarter.runSuitesInProcessWhenForked(SurefireStarter.java:107)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:68)
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'defaultServletHandlerMapping' defined in class org.springframework.web.servlet.config.annotation.DelegatingWebMvcConfiguration: Instantiation of bean failed; nested exception is org.springframework.beans.factory.BeanDefinitionStoreException: Factory method [public org.springframework.web.servlet.HandlerMapping org.springframework.web.servlet.config.annotation.WebMvcConfigurationSupport.defaultServletHandlerMapping()] threw exception; nested exception is java.lang.IllegalArgumentException: A ServletContext is required to configure default servlet handling
at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:597)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1095)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:990)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:504)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:475)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:302)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:229)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:298)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:706)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:762)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:482)
at org.springframework.test.context.support.AbstractGenericContextLoader.loadContext(AbstractGenericContextLoader.java:125)
at org.springframework.test.context.support.AbstractGenericContextLoader.loadContext(AbstractGenericContextLoader.java:60)
at org.springframework.test.context.DefaultCacheAwareContextLoaderDelegate.loadContextInternal(DefaultCacheAwareContextLoaderDelegate.java:63)
at org.springframework.test.context.DefaultCacheAwareContextLoaderDelegate.loadContext(DefaultCacheAwareContextLoaderDelegate.java:83)
... 33 more
Caused by: org.springframework.beans.factory.BeanDefinitionStoreException: Factory method [public org.springframework.web.servlet.HandlerMapping org.springframework.web.servlet.config.annotation.WebMvcConfigurationSupport.defaultServletHandlerMapping()] threw exception; nested exception is java.lang.IllegalArgumentException: A ServletContext is required to configure default servlet handling
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:188)
at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:586)
... 48 more
Caused by: java.lang.IllegalArgumentException: A ServletContext is required to configure default servlet handling
at org.springframework.util.Assert.notNull(Assert.java:112)
at org.springframework.web.servlet.config.annotation.DefaultServletHandlerConfigurer.<init>(DefaultServletHandlerConfigurer.java:54)
at org.springframework.web.servlet.config.annotation.WebMvcConfigurationSupport.defaultServletHandlerMapping(WebMvcConfigurationSupport.java:358)
at org.springframework.web.servlet.config.annotation.DelegatingWebMvcConfiguration$$EnhancerBySpringCGLIB$$ae1586a5.CGLIB$defaultServletHandlerMapping$16(<generated>)
at org.springframework.web.servlet.config.annotation.DelegatingWebMvcConfiguration$$EnhancerBySpringCGLIB$$ae1586a5$$FastClassBySpringCGLIB$$38361e62.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:311)
at org.springframework.web.servlet.config.annotation.DelegatingWebMvcConfiguration$$EnhancerBySpringCGLIB$$ae1586a5.defaultServletHandlerMapping(<generated>)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:166)
... 49 more

i found the solution since my MvcConfig class have a #EnableWebMvc annotation
like this:
#Configuration
#ComponentScan("controller")
#EnableWebMvc
public class MvcConfig extends WebMvcConfigurerAdapter {
i needed to add this annotation to my test class #WebAppConfiguration and change the loader value from AnnotationConfigContextLoader to this AnnotationConfigWebContextLoader like this:
loader=AnnotationConfigWebContextLoader.class
so in conclusion i modified my test class like this:
#WebAppConfiguration
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration(classes=MvcConfiguracion.class, loader=AnnotationConfigWebContextLoader.class)
public class UserTest{

Spring-Batch without persisting metadata to database?

I want to create a spring-batch job, but I want to run it without any database persistence. Unfortunately spring-batch requires to write metadata ob the job cycles to a database somehow, thus procing me to provide at least some kind of db with transactionmanager and entitymanager.
It it possible to prevent the metadata and run independently from txmanagers and databases?
Update:
ERROR org.springframework.batch.core.job.AbstractJob: Encountered fatal error executing job
java.lang.NullPointerException
at org.springframework.batch.core.repository.dao.MapJobExecutionDao.synchronizeStatus(MapJobExecutionDao.java:158) ~[spring-batch-core-3.0.1.RELEASE.jar:3.0.1.RELEASE]
at org.springframework.batch.core.repository.support.SimpleJobRepository.update(SimpleJobRepository.java:161) ~[spring-batch-core-3.0.1.RELEASE.jar:3.0.1.RELEASE]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.7.0_51]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[?:1.7.0_51]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.7.0_51]
at java.lang.reflect.Method.invoke(Method.java:606) ~[?:1.7.0_51]
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317) ~[spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190) ~[spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) ~[spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]
at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:98) ~[spring-tx-4.0.6.RELEASE.jar:4.0.6.RELEASE]
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:262) ~[spring-tx-4.0.6.RELEASE.jar:4.0.6.RELEASE]
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:95) ~[spring-tx-4.0.6.RELEASE.jar:4.0.6.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207) ~[spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]
at com.sun.proxy.$Proxy134.update(Unknown Source) ~[?:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.7.0_51]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[?:1.7.0_51]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.7.0_51]
at java.lang.reflect.Method.invoke(Method.java:606) ~[?:1.7.0_51]
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317) ~[spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190) ~[spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) ~[spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]
at org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:127) ~[spring-batch-core-3.0.1.RELEASE.jar:3.0.1.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207) ~[spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]
at com.sun.proxy.$Proxy134.update(Unknown Source) ~[?:?]
at org.springframework.batch.core.job.AbstractJob.updateStatus(AbstractJob.java:416) ~[spring-batch-core-3.0.1.RELEASE.jar:3.0.1.RELEASE]
at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:299) [spring-batch-core-3.0.1.RELEASE.jar:3.0.1.RELEASE]
at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:135) [spring-batch-core-3.0.1.RELEASE.jar:3.0.1.RELEASE]
at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50) [spring-core-4.0.6.RELEASE.jar:4.0.6.RELEASE]
at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:128) [spring-batch-core-3.0.1.RELEASE.jar:3.0.1.RELEASE]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.7.0_51]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[?:1.7.0_51]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.7.0_51]
at java.lang.reflect.Method.invoke(Method.java:606) ~[?:1.7.0_51]
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317) [spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190) [spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) [spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]
at org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:127) [spring-batch-core-3.0.1.RELEASE.jar:3.0.1.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) [spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207) [spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]
at com.sun.proxy.$Proxy50.run(Unknown Source) [?:?]

Simply create a configuration without datasource for Batch configuration :
#Configuration
#EnableAutoConfiguration
#EnableBatchProcessing
public class BatchConfiguration extends DefaultBatchConfigurer {
#Override
public void setDataSource(DataSource dataSource) {
// override to do not set datasource even if a datasource exist.
// initialize will use a Map based JobRepository (instead of database)
}
}
It will initialize JobRepository and JobExplorer with a in memory map based implementation.
https://github.com/spring-projects/spring-batch/blob/342d27bc1ed83312bdcd9c0cb30510f4c469e47d/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/DefaultBatchConfigurer.java#L84
and you can use you're production datasource as well even if auto configured with spring boot.

I want to run it without any database persistence
You can use MapJobRepositoryFactoryBean and ResourcelessTransactionManager
sample configuration:
<bean id="transactionManager"
class="org.springframework.batch.support.transaction.ResourcelessTransactionManager" />
<bean id="jobRepository"
class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean">
<property name="transactionManager" ref="transactionManager" />
</bean>
<bean id="jobLauncher"
class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
<property name="jobRepository" ref="jobRepository" />
</bean>
For Spring 4.X, the annotation based configuration would be as follows:
#Bean
public PlatformTransactionManager getTransactionManager() {
return new ResourcelessTransactionManager();
}
#Bean
public JobRepository getJobRepo() {
return new MapJobRepositoryFactoryBean(getTransactionManager()).getObject();
}

After tweaking #Braj's answer, my working configuration looks as follows:
#Bean
public ResourcelessTransactionManager transactionManager() {
return new ResourcelessTransactionManager();
}
#Bean
public JobRepository jobRepository(ResourcelessTransactionManager transactionManager) throws Exception {
MapJobRepositoryFactoryBean mapJobRepositoryFactoryBean = new MapJobRepositoryFactoryBean(transactionManager);
mapJobRepositoryFactoryBean.setTransactionManager(transactionManager);
return mapJobRepositoryFactoryBean.getObject();
}
#Bean
public SimpleJobLauncher jobLauncher(JobRepository jobRepository) {
SimpleJobLauncher simpleJobLauncher = new SimpleJobLauncher();
simpleJobLauncher.setJobRepository(jobRepository);
return simpleJobLauncher;
}

I came back to my own question, as the solution did not work anymore.
As of spring-batch-1.5.3 use as follows:
#SpringBootApplication(exclude = {DataSourceAutoConfiguration.class})
...
#Bean
public PlatformTransactionManager transactionManager() {
return new ResourcelessTransactionManager();
}
}

If you don't want to store job's metadata in the database, configuration looks as follows
#Configuration
public class InMemoryJobRepositoryConfiguration {
#Bean
public ResourcelessTransactionManager transactionManager() {
return new ResourcelessTransactionManager();
}
#Bean
public MapJobRepositoryFactoryBean mapJobRepositoryFactory(ResourcelessTransactionManager transactionManager)
throws Exception {
MapJobRepositoryFactoryBean factory = new MapJobRepositoryFactoryBean(transactionManager);
factory.afterPropertiesSet();
return factory;
}
#Bean
public JobRepository jobRepository(MapJobRepositoryFactoryBean repositoryFactory) throws Exception {
return repositoryFactory.getObject();
}
#Bean
public JobExplorer jobExplorer(MapJobRepositoryFactoryBean repositoryFactory) {
return new SimpleJobExplorer(repositoryFactory.getJobInstanceDao(), repositoryFactory.getJobExecutionDao(),
repositoryFactory.getStepExecutionDao(), repositoryFactory.getExecutionContextDao());
}
#Bean
public SimpleJobLauncher jobLauncher(JobRepository jobRepository) {
SimpleJobLauncher launcher = new SimpleJobLauncher();
launcher.setJobRepository(jobRepository);
return launcher;
}
}
If job needs to read/write business data to the database and the datasource is configured as follows.
#Autowired
private DataSource dataSource;
spring batch creates internal schema (BATCH_* tables and sequences) using that datasource. To prevent this from happening set flag spring.batch.initializer.enabled=false in application.properties.

I tried all of the solutions above, but it does not work with my particular scenario because my spring batch job has advanced features such as multi-threading. It needs a database. So here is what I did to solve the problem:
#Configuration
public class FakeBatchConfig implements BatchConfigurer {
PlatformTransactionManager transactionManager;
JobRepository jobRepository;
JobLauncher jobLauncher;
JobExplorer jobExplorer;
#Override
public JobRepository getJobRepository() {
return jobRepository;
}
#Override
public PlatformTransactionManager getTransactionManager() {
return transactionManager;
}
#Override
public JobLauncher getJobLauncher() {
return jobLauncher;
}
#Override
public JobExplorer getJobExplorer() {
return jobExplorer;
}
#PostConstruct
void initialize() throws Exception {
if (this.transactionManager == null) {
this.transactionManager = new ResourcelessTransactionManager();
}
MapJobRepositoryFactoryBean jobRepositoryFactory = new MapJobRepositoryFactoryBean(this.transactionManager);
jobRepositoryFactory.afterPropertiesSet();
this.jobRepository = jobRepositoryFactory.getObject();
MapJobExplorerFactoryBean jobExplorerFactory = new MapJobExplorerFactoryBean(jobRepositoryFactory);
jobExplorerFactory.afterPropertiesSet();
this.jobExplorer = jobExplorerFactory.getObject();
this.jobLauncher = createJobLauncher();
}
private JobLauncher createJobLauncher() throws Exception {
SimpleJobLauncher jobLauncher = new SimpleJobLauncher();
jobLauncher.setJobRepository(jobRepository);
jobLauncher.afterPropertiesSet();
return jobLauncher;
}
}

In addition to #Braj-s response I had to exclude Batch autoconfiguration:
#SpringBootApplication(exclude = {BatchAutoConfiguration.class})
It was not working otherwise.
This includes Spring Boot application

After reading all the answers provided I got it working. I used mix of membersound and Aure77 solution. I found out there was no need to override setDataSource method. DefaultBatchConfigurer automatically takes care of PlatformTransactionManager and DataSource is not required. Please note I am using Spring boot 2.0.3.RELEASE.
Following is the method I used
#SpringBootApplication(exclude = { DataSourceAutoConfiguration.class })
#EnableBatchProcessing
public class DemoApplication extends DefaultBatchConfigurer {
#Autowired
private JobBuilderFactory jobs;
#Autowired
private StepBuilderFactory steps;
#Bean
protected Tasklet tasklet() {
return new Tasklet() {
#Override
public RepeatStatus execute(StepContribution contribution, ChunkContext context) {
return RepeatStatus.FINISHED;
}
};
}
#Bean
public Job job() throws Exception {
return this.jobs.get("job").start(step1()).build();
}
#Bean
protected Step step1() throws Exception {
return this.steps.get("step1").tasklet(tasklet()).build();
}
public static void main(String[] args) throws Exception {
//System.exit(SpringApplication.exit(SpringApplication.run(DemoApplication.class, args)));
SpringApplication.run(DemoApplication.class, args);
}
}

The simplest of all is add an embedded database to classpath.
Naturally in production is not recommended but if you're using it to learn, save time.
Add to your pom.xml the embedded H2 database dependency:
<dependency>
<groupId>com.h2database</groupId>
<artifactId>h2</artifactId>
<scope>runtime</scope>
</dependency>
That's all.

Following changes can be made to use the Spring Batch without having to create spring batch related metadata database structures.
This worked for me:
Added bean of type ResourcelessTransactionManager and configured MapJobRepositoryFactoryBean.
Using the configured bean MapJobRepositoryFactoryBean created a SimpleJoblauncher bean.
Added annotation #EnableBatchProcessing below #SpringBootApplication.
To override the defaults I had to add spring.main.allow-bean-definition-overriding=true in application.properties.