I've got a problem using Spring Batch and can't seem to find a solution.
So, I've got a batch processing some items in chunks (size 10). I've also got a transaction manager used by this batch to persist the processed items after each chunk.
But... I would also like to persist some progress status for those items, in real time. So before processing an item I want to save a status saying this item is in progress.
And I can't seem to find a solution to achieve that. I tried the following solutions :
If I just annotate my status manager service with Transactional annotation the statuses are commited after the whole chunk processing.
If I add REQUIRES_NEW as propagation level to the annotation, it works... but the batch ends in some kind of deadlock (I read that it was common issue with REQUIRES_NEW).
So my last guess was to add a second transaction manager (on the same datasource) and use it on the status manager service... But I got the same result as solution #1 (which seem wierd to me as I expected the transaction from this manager to act independently of the chunk transaction).
Has anybody ever encountered this problem?
EDIT :
Here is my configuration, simplified on purpose :
Class DbConfiguration:
#Configuration
public class DbConfiguration {
#Bean
#Primary
public JpaTransactionManager transactionManager() throws Exception {
final JpaTransactionManager jpaTransactionManager = new JpaTransactionManager();
jpaTransactionManager.setEntityManagerFactory(entityManagerFactory());
jpaTransactionManager.afterPropertiesSet();
return jpaTransactionManager;
}
}
Class JobConfiguration:
#Configuration
#Import(DbConfiguration.class)
#EnableTransactionManagement
public class JobConfiguration {
#Autowired
private EntityManagerFactory entityManagerFactory;
#Bean
public Job jobDefinition() {
return jobBuilderFactory
.get(JOB_NAME)
.start(step())
.build();
}
#Bean
public Step step() {
return stepBuilderFactory
.get(STEP_NAME)
.<Object, Object>chunk(COMMIT_INTERVAL)
.reader(reader())
.processor(processor())
.writer(writer())
.build();
}
#Bean
public PlatformTransactionManager statusTransactionManager() {
final JpaTransactionManager jpaTransactionManager = new JpaTransactionManager();
jpaTransactionManager.setEntityManagerFactory(entityManagerFactory);
jpaTransactionManager.afterPropertiesSet();
return jpaTransactionManager;
}
}
Class StatusManagerServiceImpl:
#Transactional("statusTransactionManager")
public class StatusManagerServiceImpl implements StatusManagerService {
...
}
A way to achieve this is to use Spring TransactionTemplate as follows:
#Service
public class StatusManagerServiceImpl implements StatusManagerService {
#Autowired
private PlatformTransactionManager transactionManager;
public separateTransactionMethod() {
TransactionTemplate transactionTemplate = new TransactionTemplate(transactionManager);
transactionTemplate.execute(new TransactionCallbackWithoutResult() {
#Override
protected void doInTransactionWithoutResult(TransactionStatus status) {
// Do something here that will be committed right away.
}
});
}
}
You can alternatively use new TransactionCallback if you got a result to return. Then execute method would return that result.
Related
I am facing some issues while writing integration tests for Spring Batch jobs. The main problem is that an exception is thrown whenever a transaction is started inside the batch job.
Well, first things first. Imagine this is the step of a simple job. A Tasklet for the sake of simplicity. Of course, it is used in a proper batch config (MyBatchConfig) which I also omit for brevity.
#Component
public class SimpleTask implements Tasklet {
private final MyRepository myRepository;
public SimpleTask(MyRepository myRepository) {
this.myRepository = myRepository;
}
#Override
public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception {
myRepository.deleteAll(); // or maybe saveAll() or some other #Transactional method
return RepeatStatus.FINISHED;
}
}
MyRepository is a very unspecial CrudRepository.
Now, to test that job I use the following test class.
#SpringBatchTest
#EnableAutoConfiguration
#SpringJUnitConfig(classes = {
H2DataSourceConfig.class, // <-- this is a configuration bean for an in-memory testing database
MyBatchConfig.class
})
public class MyBatchJobTest {
#Autowired
private JobLauncherTestUtils jobLauncherTestUtils;
#Autowired
private JobRepositoryTestUtils jobRepositoryTestUtils;
#Autowired
private MyRepository myRepository;
#Test
public void testJob() throws Exception {
var testItems = List.of(
new MyTestItem(1),
new MyTestItem(2),
new MyTestItem(3)
);
myRepository.saveAll(testItems); // <--- works perfectly well
jobLauncherTestUtils.launchJob();
}
}
When it comes to the tasklet execution and more precisely to the deleteAll() method call this exception is fired:
org.springframework.transaction.CannotCreateTransactionException: Could not open JPA EntityManager for transaction; nested exception is java.lang.IllegalStateException: Already value [org.springframework.jdbc.datasource.ConnectionHolder#68f48807] for key [org.springframework.jdbc.datasource.DriverManagerDataSource#49a6f486] bound to thread [SimpleAsyncTaskExecutor-1]
at org.springframework.orm.jpa.JpaTransactionManager.doBegin(JpaTransactionManager.java:448)
...
Do you have any ideas why this is happening?
As a workaround I currently mock the repository with #MockBean and back it with an ArrayList but this is not what the inventor intended, I guess.
Any advice?
Kind regards
Update 1.1 (includes solution)
The mentioned data source configuration class is
#Configuration
#EnableJpaRepositories(
basePackages = {"my.project.persistence.repository"},
entityManagerFactoryRef = "myTestEntityManagerFactory",
transactionManagerRef = "myTestTransactionManager"
)
#EnableTransactionManagement
public class H2DataSourceConfig {
#Bean
public DataSource myTestDataSource() {
var dataSource = new DriverManagerDataSource();
dataSource.setDriverClassName("org.h2.Driver");
dataSource.setUrl("jdbc:h2:mem:myDb;DB_CLOSE_DELAY=-1");
return dataSource;
}
#Bean
public LocalContainerEntityManagerFactoryBean myTestEntityManagerFactory() {
var emFactory = new LocalContainerEntityManagerFactoryBean();
var adapter = new HibernateJpaVendorAdapter();
adapter.setDatabasePlatform("org.hibernate.dialect.H2Dialect");
adapter.setGenerateDdl(true);
emFactory.setDataSource(myTestDataSource());
emFactory.setPackagesToScan("my.project.persistence.model");
emFactory.setJpaVendorAdapter(adapter);
return emFactory;
}
#Bean
public PlatformTransactionManager myTestTransactionManager() {
return new JpaTransactionManager(myTestEntityManagerFactory().getObject());
}
#Bean
public BatchConfigurer testBatchConfigurer() {
return new DefaultBatchConfigurer() {
#Override
public PlatformTransactionManager getTransactionManager() {
return myTestTransactionManager();
}
};
}
}
By default, when you declare a datasource in your application context, Spring Batch will use a DataSourceTransactionManager to drive step transactions, but this transaction manager knows nothing about your JPA context.
If you want to use another transaction manager, you need to override BatchConfigurer#getTransactionManager and return the transaction manager you want to use to drive step transactions. In your case, you are only declaring a transaction manager bean in the application context which is not enough. Here a quick example:
#Bean
public BatchConfigurer batchConfigurer() {
return new DefaultBatchConfigurer() {
#Override
public PlatformTransactionManager getTransactionManager() {
return new JpaTransactionManager(myTestEntityManagerFactory().getObject());
}
};
}
For more details, please refer to the reference documentation.
Environment: JDK9.0.4, Spring Boot 2.0.2 with Spring Batch 4.0.1, using Postgres 9.5 and JPA via Hibernate.
My job uses JpaPagingItemReader and JpaItemWriter with a processor in between that updates values on the entity.
My issue is that my Batch job is throwing a org.hibernate.StaleObjectStateException after processing the first item and attempting to read the next page (chunk/page size 1)
Caused by: org.hibernate.StaleObjectStateException: Row was updated or deleted by another transaction (or unsaved-value mapping was incorrect) : [com.my.entity#1315]
From debugging, it appears that Batch attempts to flush the JPA session in JPAPagingItemReader doReadPage(), but it's already been flushed during write() via (I think) another EMF that's somehow instantiated, and so Hibernate detects the version number has changed and throws the exception.
Update
I've found a couple of 5 year old forum posts/JIRAs that suggest that this is a limitation in Batch and a variety of workarounds. I'd be grateful for thoughts on whether I'm reading this right.
https://jira.spring.io/browse/BATCH-1110
https://jira.spring.io/browse/BATCH-1166
http://forum.spring.io/forum/spring-projects/batch/92792-jpapagingitemreader-stale-object-exception
If anyone can point towards any obvious pitfalls in this area I'd be grateful. I've read a fair amount of the Javadocs but struggling to understand the issue.
#Configuration
public class DBConfig {
#Autowired
private DataSource dataSource;
#Bean
public PlatformTransactionManager transactionManager(EntityManagerFactory emf) {
JpaTransactionManager transactionManager = new JpaTransactionManager();
transactionManager.setEntityManagerFactory(emf);
return transactionManager;
}
#Bean
public LocalContainerEntityManagerFactoryBean entityManagerFactory() {
LocalContainerEntityManagerFactoryBean em = new LocalContainerEntityManagerFactoryBean();
em.setDataSource(dataSource);
em.setPackagesToScan("com.my.ents", "com.my.other.ends");
JpaVendorAdapter vendorAdapter = new HibernateJpaVendorAdapter();
em.setJpaVendorAdapter(vendorAdapter);
return em;
}
}
#Bean
#StepScope
public JpaPagingItemReader<ScrapeHistorySingle> getItemReader(
#Value("#{jobParameters['council']}") String councilShortName,
#Value("#{jobParameters['scrapeType']}") String scrapeTypeString) {
if (councilShortName == null) {
throw new NullPointerException("councilShortName cannot be null");
}
if (scrapeTypeString == null) {
throw new NullPointerException("scrapeType cannot be null");
}
JpaPagingItemReader<ScrapeHistorySingle> itemReader = new JpaPagingItemReader<>();
itemReader.setEntityManagerFactory(entityManagerFactory);
...
#Bean
public JpaItemWriter<ScrapeHistorySingle> itemWriterScrapeHistorySingle() {
JpaItemWriter<ScrapeHistorySingle> itemWriter = new JpaItemWriter<>();
itemWriter.setEntityManagerFactory(entityManagerFactory);
return itemWriter;
}
#Configuration
#EnableBatchProcessing
#Component
public class CustomBatchConfigurer extends DefaultBatchConfigurer {
private static final Log logger = LogFactory.getLog(CustomBatchConfigurer.class);
#Autowired
private ThreadPoolTaskExecutor taskExecutor;
#Autowired
private JobRegistry jobRegistry;
#Autowired
private DataSource dataSource;
#Qualifier("transactionManager")
#Autowired
private PlatformTransactionManager transactionManager;
In Spring Integration we have a Setup that looks something like this:
--->
--->
(dispatcher) Messages --> Gateway ----> QueueChannel ---> MessageHandler (worker)
--->
--->
So we have one Dispatcher Thread that takes Messages from a MQTT-Broker and forwards them into the Queue. The Poller for the Queue is provided with a TaskExecuter, so the Consumer is multithreaded.
We managed to implement all the functionalities. So the just described setup is already implemented.
Now to guarantee no data loss we want to make two things:
1.:
We want our queue to persist the data, so when the Programm shuts down ungracefully, all the data in the queue will still be there.
This also worked for us, we are using MongoDB as a database because we read somewhere in your docs that this is the recommended way to do it.
2.:
The second thing we want to assure is that the worker threads are working transactional. So only if the worker threads return correctly the messages will permanently be deleted from the queue (and therefore the persistent MessageStore). If the program shuts down during the processing of a message (by the worker thread) the message will still be in the queue at the next startup.
Also if the worker, for example, thows an exception during the processing of the message, it will be put back into the queue.
Our implementation:
As explained before, the basic setup of the program is already implemented. We then extended the basic implementation with a message store implementation for the queue.
QueueChannel:
#Bean
public PollableChannel inputChannel(BasicMessageGroupStore mongoDbChannelMessageStore) {
return new QueueChannel(new MessageGroupQueue(mongoDbChannelMessageStore, "inputChannel"));
}
backed by a Messagestore:
#Bean
public BasicMessageGroupStore mongoDbChannelMessageStore(MongoDbFactory mongoDbFactory) {
MongoDbChannelMessageStore store = new MongoDbChannelMessageStore(mongoDbFactory);
store.setPriorityEnabled(true);
return store;
}
the matching Poller:
#Bean(name = PollerMetadata.DEFAULT_POLLER)
public PollerMetadata poller() {
PollerMetadata poll = Pollers.fixedDelay(10).get();
poll.setTaskExecutor(consumer);
return poll;
}
Executor:
private Executor consumer = Executors.newFixedThreadPool(5);
What we have tried?
As explained now we want to extend this implementation with a transactional functionality. We tried using the setTransactionSynchronizationFactory like explained here but it wasn't working (didn't get errors or anything but the behavior was still as it was before we added the TransactionSynchronizer):
#Bean(name = PollerMetadata.DEFAULT_POLLER)
public PollerMetadata poller() {
PollerMetadata poll = Pollers.fixedDelay(10).get();
poll.setTaskExecutor(consumer);
BeanFactory factory = mock(BeanFactory.class);
ExpressionEvaluatingTransactionSynchronizationProcessor etsp = new ExpressionEvaluatingTransactionSynchronizationProcessor();
etsp.setBeanFactory(factory);
etsp.setAfterRollbackChannel(inputChannel());
etsp.setAfterRollbackExpression(new SpelExpressionParser().parseExpression("#bix"));
etsp.setAfterCommitChannel(inputChannel());
etsp.setAfterCommitExpression(new SpelExpressionParser().parseExpression("#bix"));
DefaultTransactionSynchronizationFactory dtsf = new DefaultTransactionSynchronizationFactory(etsp);
poll.setTransactionSynchronizationFactory(dtsf);
return poll;
}
What would be the best way to realize our requirements in spring integration?
EDIT:
As recommended in the answer I chose to do this with the JdbcChannelMessageStore. So I tried converting the XML Implementation described here (18.4.2) into Java. I wasn't quite sure on how to do it, this is what I have tried so far:
I created H2 database and run the script shown here on it.
Created JDBCChannelMessageStore Bean:
#Bean
public JdbcChannelMessageStore store() {
JdbcChannelMessageStore ms = new JdbcChannelMessageStore();
ms.setChannelMessageStoreQueryProvider(queryProvider());
ms.setUsingIdCache(true);
ms.setDataSource(dataSource);
return ms;
}
Created H2ChannelMessageStoreQueryProvider
#Bean
public ChannelMessageStoreQueryProvider queryProvider() {
return new H2ChannelMessageStoreQueryProvider();
}
Adapted the poller:
#Bean(name = PollerMetadata.DEFAULT_POLLER)
public PollerMetadata poller() throws Exception {
PollerMetadata poll = Pollers.fixedDelay(10).get();
poll.setTaskExecutor(consumer);
poll.setAdviceChain(Collections.singletonList(transactionInterceptor()));
return poll;
}
Autowired my PlaatformTransactionManager:
#Autowired
PlatformTransactionManager transactionManager;
And created TransactionInterceptor from the TransactonManager:
#Bean
public TransactionInterceptor transactionInterceptor() {
return new TransactionInterceptorBuilder(true)
.transactionManager(transactionManager)
.isolation(Isolation.READ_COMMITTED)
.propagation(Propagation.REQUIRED)
.build();
}
If you need to have queue as transactional, you definitely should take a look into the transactional MessageStore. And only JDBC one is like that. Just because only JDBC support transactions. So, when we perform DELETE, it is OK only if TX is committed.
The MongoDB, nor any other NoSQL DataBases, support such a model, therefore you only can push back the failed messages to the DB on rollback using TransactionSynchronizationFactory.
UPDATE
#RunWith(SpringRunner.class)
#DirtiesContext
public class So47264688Tests {
private static final String MESSAGE_GROUP = "transactionalQueueChannel";
private static EmbeddedDatabase dataSource;
#BeforeClass
public static void init() {
dataSource = new EmbeddedDatabaseBuilder()
.setType(EmbeddedDatabaseType.H2)
.addScript("classpath:/org/springframework/integration/jdbc/schema-drop-h2.sql")
.addScript("classpath:/org/springframework/integration/jdbc/schema-h2.sql")
.build();
}
#AfterClass
public static void destroy() {
dataSource.shutdown();
}
#Autowired
private PollableChannel transactionalQueueChannel;
#Autowired
private JdbcChannelMessageStore jdbcChannelMessageStore;
#Autowired
private PollingConsumer serviceActivatorEndpoint;
#Autowired
private CountDownLatch exceptionLatch;
#Test
public void testTransactionalQueueChannel() throws InterruptedException {
GenericMessage<String> message = new GenericMessage<>("foo");
this.transactionalQueueChannel.send(message);
assertTrue(this.exceptionLatch.await(10, TimeUnit.SECONDS));
this.serviceActivatorEndpoint.stop();
assertEquals(1, this.jdbcChannelMessageStore.messageGroupSize(MESSAGE_GROUP));
Message<?> messageFromStore = this.jdbcChannelMessageStore.pollMessageFromGroup(MESSAGE_GROUP);
assertNotNull(messageFromStore);
assertEquals(message, messageFromStore);
}
#Configuration
#EnableIntegration
public static class ContextConfiguration {
#Bean
public PlatformTransactionManager transactionManager() {
return new DataSourceTransactionManager(dataSource);
}
#Bean
public ChannelMessageStoreQueryProvider queryProvider() {
return new H2ChannelMessageStoreQueryProvider();
}
#Bean
public JdbcChannelMessageStore jdbcChannelMessageStore() {
JdbcChannelMessageStore jdbcChannelMessageStore = new JdbcChannelMessageStore(dataSource);
jdbcChannelMessageStore.setChannelMessageStoreQueryProvider(queryProvider());
return jdbcChannelMessageStore;
}
#Bean
public PollableChannel transactionalQueueChannel() {
return new QueueChannel(new MessageGroupQueue(jdbcChannelMessageStore(), MESSAGE_GROUP));
}
#Bean
public TransactionInterceptor transactionInterceptor() {
return new TransactionInterceptorBuilder()
.transactionManager(transactionManager())
.isolation(Isolation.READ_COMMITTED)
.propagation(Propagation.REQUIRED)
.build();
}
#Bean
public TaskExecutor threadPoolTaskExecutor() {
ThreadPoolTaskExecutor threadPoolTaskExecutor = new ThreadPoolTaskExecutor();
threadPoolTaskExecutor.setCorePoolSize(5);
return threadPoolTaskExecutor;
}
#Bean(name = PollerMetadata.DEFAULT_POLLER)
public PollerMetadata poller() {
return Pollers.fixedDelay(10)
.advice(transactionInterceptor())
.taskExecutor(threadPoolTaskExecutor())
.get();
}
#Bean
public CountDownLatch exceptionLatch() {
return new CountDownLatch(2);
}
#ServiceActivator(inputChannel = "transactionalQueueChannel")
public void handle(Message<?> message) {
System.out.println(message);
try {
throw new RuntimeException("Intentional for rollback");
}
finally {
exceptionLatch().countDown();
}
}
}
}
Thanks to Artem Bilan for your great support. I finally found the solution. It seemed like there was another bean with the name transactionManager and transactionInterceptor active. This resulted in the strange behavior, that my trans-manager was never initialized, instead the other transactionmanager (null) was used for the transactioninterceptor and the PollingConsumer. Thats why my Transactionmanager in PollingConsumer was null, and why my Transactions were never working.
The solution was to rename all my beans, for some beans I also used the annotation #Primary to tell spring to always use this speciffic bean when autowired.
I also downgraded two 4.3, just to make sure this wasn't an error related to Version 5. I haven't testet if it would work with V 5 yet, but I think it should work also.
My Spring Batch job is started every 5 minutes - basically it reads a string, uses the string as a parameter in a sql query, and prints out the resulting sql result list. Mostly it seems to be running ok, but I notice sporadic errors in my logs every 5-10 runs
2017-05-05 11:13:26.101 INFO 9572 --- [nio-8081-exec-8] c.u.r.s.AgentCollectorServiceImpl : Could not open JPA E
ntityManager for transaction; nested exception is java.lang.IllegalStateException: Transaction already active
My job is started like from my AgentCollectorServiceImpl class
#Override
public void addReportIds(List<Integer> reportIds) {
try {
.toJobParameters();
jobLauncher.run(job, jobParameters);
} catch (Exception e) {
log.info(e.getMessage());
}
}
My BatchConfig class looks like
#Configuration
#EnableBatchProcessing
#Import(AppConfig.class)
public class BatchConfig {
#Autowired
private JobBuilderFactory jobBuilderFactory;
#Autowired
private StepBuilderFactory stepBuilderFactory;
#Autowired
private AppConfig appConfig;
#Bean
public Reader reader() {
return new Reader();
}
#Bean
public Processor processor() {
return new Processor();
}
#Bean
public Writer writer() {
return new Writer();
}
#Bean
public Job job() {
return jobBuilderFactory.get("job")
.incrementer(new RunIdIncrementer())
.flow(step1())
.end()
.build();
}
#Bean
public Step step1() {
return stepBuilderFactory.get("step1")
.<String, String> chunk(1)
.reader(reader())
.processor(processor())
.writer(writer())
.build();
}
}
My AppConfig class looks like
#Configuration
#PropertySource("classpath:application.properties")
#ComponentScan
public class AppConfig {
#Bean
public LocalContainerEntityManagerFactoryBean entityManagerFactory() {
LocalContainerEntityManagerFactoryBean em = new LocalContainerEntityManagerFactoryBean();
em.setDataSource(organizationDataSource());
em.setPackagesToScan(new String[]{"com.organization.agentcollector.model"});
JpaVendorAdapter vendorAdapter = new HibernateJpaVendorAdapter();
em.setJpaVendorAdapter(vendorAdapter);
em.setJpaProperties(additionalProperties());
return em;
}
Properties additionalProperties() {
Properties properties = new Properties();
properties.setProperty("hibernate.dialect", "com.organization.agentcollector.config.SQLServerDialectOverrider");
return properties;
}
#Bean
JpaTransactionManager transactionManager(final EntityManagerFactory emf) {
JpaTransactionManager transactionManager = new JpaTransactionManager();
transactionManager.setEntityManagerFactory(entityManagerFactory().getObject());
return transactionManager;
}
My Processor class looks like
public class Processor implements ItemProcessor<String, String> {
private final Logger log = LoggerFactory.getLogger(Processor.class);
#Autowired
EventReportsDAOImpl eventReportsDAOImpl;
#Override
public String process(String reportIdsJson) throws Exception {
String eventReportsJson = eventReportsDAOImpl.listEventReportsInJsonRequest(reportIdsJson);
//System.out.println(returnContent+"PROCESSOR");
return eventReportsJson;
}
}
My DAOImpl class looks like
#Component
#Transactional
public class EventReportsDAOImpl implements EventReportsDAO {
#PersistenceContext
private EntityManager em;
#Override
public EventReports getEventReports(Integer reportId) {
return null;
}
#Override
public String listEventReportsInJsonRequest(String reportIds) {
System.out.println("Event Report reportIds processing");
ArrayList<EventReports> erArr = new ArrayList<EventReports>();
String reportIdsList = reportIds.substring(1, reportIds.length() - 1);
//System.out.println(reportIdsList);
try {
StoredProcedureQuery q = em.createStoredProcedureQuery("sp_get_event_reports", "eventReportsResult");
q.registerStoredProcedureParameter("reportIds", String.class, ParameterMode.IN);
q.setParameter("reportIds", reportIdsList);
boolean isResultSet = q.execute();
erArr = (ArrayList<EventReports>) q.getResultList();
} catch (Exception e) {
System.out.println("No event reports found for list " + reportIdsList);
}
return erArr.toString();
}
I thought Spring would manage transactions automatically. The error seems to suggest that a transaction is not being properly closed?
One thing I tried was removing all #Transactional annotations from my code as I read that #EnableBatchProcessing already injects a Transaction Manager into each step - but when I did this, I saw the 'transaction already active' error much more frequently.
Any advice appreciated on how to fix this, thank you!
The #Transactional notation establishes a transactional scope that dictates when a transaction starts and ends, also called its boundary. If you operate outside of this boundary you'll receive errors.
First off, I found this bit of documentation the most helpful on Spring transactions: http://docs.spring.io/spring-framework/docs/4.2.x/spring-framework-reference/html/transaction.html specifically this section
Secondly, you may wish to enable trace level logs and potentially the SQL statements to help debug this. In order to do so I added the following to my application.properties:
spring.jpa.properties.hibernate.show_sql=false
spring.jpa.properties.hibernate.use_sql_comments=true
spring.jpa.properties.hibernate.format_sql=true
spring.jpa.properties.hibernate.type=trace
spring.jpa.show-sql=true
logging.level.org.hibernate=TRACE
There will be ALOT of output here, but you'll get a good idea of whats happening behind the scenes.
Third, and the most important part for me in learning how to use #Transactional is that every call to the DAO creates a new session -or- reuses the existing session if within the same transactional scope. Refer to the documentation above for examples of this.
I'm writing application which connects with Oracle Database. I call function from DB which inserts new records to table. And after this callback I can decide what I want to do: commit or rollback.
Unfortunalety I'm new in Spring, so I have problems with configuration. And what's more I want to make this configuration in Java class, not in XML. And here I need your help.
UPDATED CODE:
ApplicationConfig code:
#Configuration
#EnableTransactionManagement
#ComponentScan("hr")
#PropertySource({"classpath:jdbc.properties", "classpath:functions.properties", "classpath:procedures.properties"})
public class ApplicationConfig {
#Autowired
private Environment env;
#Bean(name="dataSource")
public DataSource dataSource() {
BasicDataSource dataSource = new BasicDataSource();
dataSource.setDriverClassName(env.getProperty("jdbc.driver"));
dataSource.setUrl(env.getProperty("jdbc.url"));
dataSource.setUsername(env.getProperty("jdbc.username"));
dataSource.setPassword(env.getProperty("jdbc.password"));
dataSource.setDefaultAutoCommit(false);
return dataSource;
}
#Bean
public JdbcTemplate jdbcTemplate(DataSource dataSource) {
JdbcTemplate jdbcTemplate = new JdbcTemplate(dataSource);
jdbcTemplate.setResultsMapCaseInsensitive(true);
return jdbcTemplate;
}
#Bean(name="txName")
public PlatformTransactionManager txManager() {
DataSourceTransactionManager txManager = new DataSourceTransactionManager();
txManager.setDataSource(dataSource());
return txManager;
}
}
I have Dao and Service, where both implements proper interface.
Service implementation:
#Service
public class HumanResourcesServiceImpl implements HumanResourcesService {
#Autowired
private HumanResourcesDao hrDao;
#Override
public String generateData(int rowsNumber) {
return hrDao.generateData(rowsNumber);
}
#Override
#Transactional("txName")
public void shouldCommit(boolean doCommit, Connection connection) throws SQLException {
hrDao.shouldCommit(doCommit, connection);
}
}
Dao implementation:
#Repository
public class HumanResourcesDaoImpl implements HumanResourcesDao {
private JdbcTemplate jdbcTemplate;
private SimpleJdbcCall generateData;
#Autowired
public HumanResourcesDaoImpl(JdbcTemplate jdbcTemplate, Environment env) {
this.jdbcTemplate = jdbcTemplate;
generateData = new SimpleJdbcCall(jdbcTemplate)
.withProcedureName(env.getProperty("procedure.generateData"));
}
#Override
public String generateData(int rowsNumber) {
HashMap<String, Object> params = new HashMap<>();
params.put("i_rowsNumber", rowsNumber);
Map<String, Object> m = generateData.execute(params);
return (String) m.get("o_execution_time");
}
#Override
#Transactional("txName")
public void shouldCommit(boolean doCommit, Connection connection) throws SQLException {
if(doCommit) {
connection.commit();
} else {
connection.rollback();
}
}
}
Main class code:
public class Main extends Application implements Initializable {
#Override
public void initialize(URL url, ResourceBundle resourceBundle) {
ApplicationContext context = new AnnotationConfigApplicationContext(ApplicationConfig.class);
hrService = context.getBean(HumanResourcesService.class);
BasicDataSource ds = (BasicDataSource)context.getBean("dataSource");
Connection connection = ds.getConnection();
//do something and call
//hrService.generateData
//do something and call
//hrService.shouldCommit(true, connection);
//which commit or rollback generated data from previoues callback
}
}
UPDATE:
I think that the problem is with connection, because this statement:
this.jdbcTemplate.getDataSource().getConnection();
creates new connection, so then there is nothing to commit or rollback. But still I can't figure why this doesn't work properly. No errors, no new records...
What is wierd, is that when I debuged connection.commit(); I found out that in DelegatingConnection.java, parameter this has proper connection, but there is something like:
_conn.commit();
and _conn has different connection. Why?
Should I in some way synchronize connection for those 2 methods or what? Or this is only one connection? To be honest, I'm not sure how it works exactly. One connection and all callbacks to stored procedures are in this connection or maybe with each callback new connection is created?
Real question is how to commit or rollback data from previous callback which do insert into table?
One easy way to do this is to annotate the method with #Transactional:
#Transactional
public void myBeanMethod() {
...
if (!doCommit)
throw new IllegalStateException(); // any unchecked will do
}
and spring will roll all database changes back.
Remember to add #EnableTransactionManagement to your spring application/main class
You can use #Transactional and #EnableTransactionManagement to setup transactions without using the XML configuration. In short, annotate the methods/classes you want to have transactions with #Transactional. To setup the transactional management you use the #EnableTransactionManagement inside your #Configuration.
See Springs docs for example on how to use both. The #EnableTransactionManagement is detailed in the JavaDocs but should match the XML configuration.
UPDATE
The problem is that you are mixing raw JDBC calls (java.sql.Connection) with Spring JDBC. When you execute your SimpleJdbcCall, Spring creates a new Connection. This is not the same Connection as the one you later try to commit. Hence, nothing happens when you perform the commit. I tried to somehow get the connection that the SimpleJdbcCall uses, but could not find any easy way.
To test this I tried the following (I did not use params):
#Override
public String generateData(int rowsNumber) {
//HashMap<String, Object> params = new HashMap<>();
//params.put("i_rowsNumber", rowsNumber);
//Map<String, Object> m = generateData.execute(params);
Connection targetConnection = DataSourceUtils.getTargetConnection(generateData.getJdbcTemplate().getDataSource().getConnection());
System.out.println(targetConnection.prepareCall((generateData.getCallString())).execute());
targetConnection.commit();
return (String) m.get("o_execution_time");
}
If I don't save the targetConnection, and instead try to get the connection again by calling DataSourceUtils.getTargetConnection() when committing, nothing happens. Thus, you must commit on the same connection that you perform the statement on. This does not seem to be easy, nor the proper way.
The solution is to drop the java.sql.Connection.commit() call. Instead, you use Spring Transactions completly. If you use #Transaction on the method that performs database call, Spring will automatically commit when the method has finished. If the method body experiences any Exception (even outside the actual database call) it will automatically rollback. In other words, this should suffice for normal Transaction management.
However, if you are doing batch processing, and wish to have more control over your transactions with commits and rollbacks, you can still use Spring. To programatically control transactions with Spring, you can use TransactionTemplate which have commit and rollback methods. Don't have time to give you proper samples, but may do so in later days if you are still stuck ;)
#Configuration
#EnableTransactionManagement
#ComponentScan(basePackages="org.saat")
#PropertySource(value="classpath:resources/db.properties",ignoreResourceNotFound=true)
public class AppConfig {
#Autowired
private Environment env;
#Bean(name="dataSource")
public DataSource getDataSource() {
DriverManagerDataSource dataSource = new DriverManagerDataSource();
dataSource.setDriverClassName(env.getProperty("db.driver"));
dataSource.setUrl(env.getProperty("db.url"));
dataSource.setUsername(env.getProperty("db.username"));
dataSource.setPassword(env.getProperty("db.password"));
return dataSource;
}
#Bean(name="entityManagerFactoryBean")
public LocalContainerEntityManagerFactoryBean getSessionFactory() {
LocalContainerEntityManagerFactoryBean factoryBean = new LocalContainerEntityManagerFactoryBean ();
factoryBean.setDataSource(getDataSource());
factoryBean.setPackagesToScan("org.saat");
factoryBean.setJpaVendorAdapter(getJpaVendorAdapter());
Properties props=new Properties();
props.put("hibernate.dialect", env.getProperty("hibernate.dialect"));
props.put("hibernate.hbm2ddl.auto",env.getProperty("hibernate.hbm2ddl.auto"));
props.put("hibernate.show_sql",env.getProperty("hibernate.show_sql"));
factoryBean.setJpaProperties(props);
return factoryBean;
}
#Bean(name="transactionManager")
public JpaTransactionManager getTransactionManager() {
JpaTransactionManager jpatransactionManager = new JpaTransactionManager();
jpatransactionManager.setEntityManagerFactory(getSessionFactory().getObject());
return jpatransactionManager;
}
#Bean
public JpaVendorAdapter getJpaVendorAdapter() {
HibernateJpaVendorAdapter hibernateJpaVendorAdapter = new HibernateJpaVendorAdapter();
return hibernateJpaVendorAdapter;
}
}