I have a spring batch job which has multiple steps.
Step 1 : Loads 10 records from the database. (Tasklet does that job)
Step 2: Chunk Oriented processing in configured here using ItemReader,ItemProcessor,ItemWriter implementations with commit -interval =1
Now as I understand , for every record this would happen
Begin Transaction ( Read - Process - Write ) Commit Tx
My problem is imagine it processed six records and now with the 7th Record it got an exception in the ItemProcessor Implementation, It tries to rollback but not able to rollback due to transaction in unknown state
Even if it is not able to rollback tx for 7th record it does not processes 8th,9th,10th record at all and job is STOPPED.
Note: ItemProcessor implementation is calling services (#Service annotated) which are marked as transactional using #Transactional(readOnly=false) annotation.
Please suggest the solution.
ItemProcessor Code Below
public Long process(LoanApplication loanApplication)throws Exception {
Long createLoan = null;
LoanStatus loanStatus = null;
StaffUser user = staffUserService.getStaffUserByName(Constants.AUTO_USER);
String notes = null;
try{
try{
loanValidationService.validate(loanApplication);
loanStatus=LoanStatus.U;
}catch(LoanValidationException e){
loanStatus=LoanStatus.UC;
notes=e.getMessage();
}
dataLoadLoanManagementService.setText(notes);
createLoan = dataLoadLoanManagementService.createLoan(loanApplication, user,loanStatus);
}catch(Exception le){
logger.error("Error creating the loan application ; Parent Application Ref : "
+ loanApplication
+ " with status as " +(loanStatus!=null ?loanStatus.getStatus():loanStatus)
+"\n"
+" School Ref :"
+ loanApplication.getSchoolRef()
+"\n"
+" Client Details :"
+loanApplication.getClientDetails()
+ "Error Details: " + ExceptionUtils.getStackTrace(le));
}
return createLoan;
}
Even with Skippable Exception Classes configured this does not work.
To explain bit more i get Persistence Exception in Item Processor and i catch it , therefore Spring batch executes the writer but after writer execution i get the exception below
INFO 06-21 11:38:00 Commit failed while step execution data was already updated. Reverting to old version. (TaskletStep.java:342)
ERROR 06-21 11:38:00 Rolling back with transaction in unknown state (TaskletStep.java:351)
ERROR 06-21 11:38:00 Encountered an error executing the step (AbstractStep.java:212)
org.springframework.transaction.TransactionSystemException: Could not commit JPA transaction; nested exception is javax.persistence.RollbackException: Transaction marked as rollbackOnly
For both of your questions, skipping the exception occured during the prosessor phase will solve your problems.
This may be configured by skippable-exception-classes element if you know the root cause of the exception. For example, if you get a divide by zero exception during the processor phase and do want to neglect it, a sample configuration might be:
<chunk reader="reader" processor="processor" writer="writer"
commit-interval="1" >
<skippable-exception-classes>
<include class="java.lang.ArithmeticException" />
</skippable-exception-classes>
</chunk>
Since the given exception class and its subclasses will be skipped, you may even try java.lang.Exception
check transaction propagation option at loanValidationService.validate
Guess from "Transaction marked as rollbackOnly", "Commit failed while step execution data was already updated"
Current transaction was rollbacked and parent Transaction should be commit but transaction already over.
if chunk's transaction is same at LoanValidationService.validate
change propagation option to REQUIRES_NEW
following article may help to understand metadata's transaction.
http://blog.codecentric.de/en/2012/03/transactions-in-spring-batch-part-1-the-basics/
class LoanValidationServiceImpl implements LoanValidationService {
#Trasnactional(propagation=REQUIRES_NEW, rollbackFor=Exception.class)
validate(LoanApplication loanApplication) {
// business logic
}
}
Related
I've one application in which I'm trying to process a file which contains 500k transactions, the processing of file completes in 3 minutes and the timeout is set to 15 minutes, still getting timeout exception during commit.
Getting Below Exception in Websphere System Logs while committing the transaction from my code in spring, due to which all the transactions are rolledback;
XATransaction E J2CA0027E: An exception occurred while invoking end on an XA Resource Adapter from DataSource JMS$FundtechQConFactory$JMSManagedConnection#75, within transaction ID {XidImpl: formatId(57415344), gtrid_length(36), bqual_length(54),
data(0000017bdaabecde0000000b6a6fc584d5dba62aea917f1902693232b82dd8a4ff309f370000017bdaabecde0000000b6a6fc584d5dba62aea917f1902693232b82dd8a4ff309f37000000010000000000000000000000000097)} : javax.transaction.xa.XAException: The method 'xa_end' has failed with errorCode '106'.
at com.ibm.mq.jmqi.JmqiXAResource.end(JmqiXAResource.java:559)
at com.ibm.ejs.jms.JMSManagedSession$JMSXAResource.end(JMSManagedSession.java:1410)
at com.ibm.ejs.j2c.XATransactionWrapper.end(XATransactionWrapper.java:623)
at com.ibm.ws.Transaction.JTA.JTAResourceBase.end(JTAResourceBase.java:254)
at com.ibm.tx.jta.impl.RegisteredResources.sendEnd(RegisteredResources.java:1154)
at com.ibm.tx.jta.impl.RegisteredResources.distributeEnd(RegisteredResources.java:1130)
at com.ibm.tx.jta.impl.TransactionImpl.distributeEnd(TransactionImpl.java:1748)
at com.ibm.tx.jta.impl.TransactionImpl.prepareResources(TransactionImpl.java:1478)
at com.ibm.ws.tx.jta.TransactionImpl.stage1CommitProcessing(TransactionImpl.java:630)
at com.ibm.tx.jta.impl.TransactionImpl.processCommit(TransactionImpl.java:1040)
at com.ibm.tx.jta.impl.TransactionImpl.commit(TransactionImpl.java:974)
at com.ibm.ws.tx.jta.TranManagerImpl.commit(TranManagerImpl.java:439)
at com.ibm.tx.jta.impl.TranManagerSet.commit(TranManagerSet.java:191)
at com.ibm.ws.tx.jta.UserTransactionImpl.commit(UserTransactionImpl.java:307)
at org.springframework.transaction.jta.JtaTransactionManager.doCommit(JtaTransactionManager.java:1035)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:743)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:711)
XATransaction E J2CA0027E: An exception occurred while invoking rollback on an XA Resource Adapter from DataSource JMS$FundtechQConFactory$JMSManagedConnection#206, within transaction ID {XidImpl: formatId(57415344), gtrid_length(36), bqual_length(54),
data(0000017bdaabecde0000000b6a6fc584d5dba62aea917f1902693232b82dd8a4ff309f370000017bdaabecde0000000b6a6fc584d5dba62aea917f1902693232b82dd8a4ff309f37000000010000000000000000000000000003)} : javax.transaction.xa.XAException: The method 'xa_rollback' has failed with errorCode '-4'.
at com.ibm.mq.jmqi.JmqiXAResource.rollback(JmqiXAResource.java:874)
at com.ibm.ejs.jms.JMSManagedSession$JMSXAResource.rollback(JMSManagedSession.java:1201)
at com.ibm.ejs.j2c.XATransactionWrapper.rollback(XATransactionWrapper.java:1328)
at com.ibm.tx.jta.impl.JTAXAResourceImpl.rollback(JTAXAResourceImpl.java:381)
at com.ibm.tx.jta.impl.RegisteredResources.deliverOutcome(RegisteredResources.java:1718)
at com.ibm.tx.jta.impl.RegisteredResources.distributeOutcome(RegisteredResources.java:2004)
at com.ibm.tx.jta.impl.RegisteredResources.distributeRollback(RegisteredResources.java:2657)
at com.ibm.tx.jta.impl.TransactionImpl.internalRollback(TransactionImpl.java:1973)
at com.ibm.tx.jta.impl.TransactionImpl.internalRollback(TransactionImpl.java:1936)
at com.ibm.tx.jta.impl.TransactionImpl.coreStage2CommitProcessing(TransactionImpl.java:1156)
at com.ibm.tx.jta.impl.TransactionImpl.stage2CommitProcessing(TransactionImpl.java:1183)
at com.ibm.tx.jta.impl.TransactionImpl.processCommit(TransactionImpl.java:1044)
at com.ibm.tx.jta.impl.TransactionImpl.commit(TransactionImpl.java:974)
at com.ibm.ws.tx.jta.TranManagerImpl.commit(TranManagerImpl.java:439)
at com.ibm.tx.jta.impl.TranManagerSet.commit(TranManagerSet.java:191)
at com.ibm.ws.tx.jta.UserTransactionImpl.commit(UserTransactionImpl.java:307)
at org.springframework.transaction.jta.JtaTransactionManager.doCommit(JtaTransactionManager.java:1035)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:743)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:711)
Error code 106 that you get on xa.end is XA_RBTIMEOUT.
This means that the resource manager has rolled back the transaction due to exceeding a timeout.
Error code -4 that you get on xa.commit is XAER_NOTA.
This means that the transaction xid isn't valid anymore, which makes sense given the previous error - because transaction branch was already rolled back due to the timeout.
It is important to be aware that transaction timeouts can be set at various levels. WebSphere Application Server has a global transaction lifetime timeout. Various resource providers will often have their own timeout settings (I'd recommend checking if MQ does) which can cause them to time out their transaction branches prior to the overall transaction. Applications can specify a transaction timeout on UserTransaction, although it appears in this case you are going through Spring, which might be specifying that value, so also look into Springs settings for transaction timeout.
I am getting Commit failed while step execution data was already updated error when spring batch try to commit the records. Any help would be greatly appreciated.
I am using HIbernate JPA .
It is working for lesser number of records. Throwing error when the record count is high.
Stack Trace:
2016-01-20 08:49:45 INFO TaskletStep:359 - Commit failed while step execution data was already updated. Reverting to old version.
2016-01-20 08:49:45 ERROR TaskletStep:370 - Rolling back with transaction in unknown state
2016-01-20 08:49:45 ERROR AbstractStep:225 - Encountered an error executing step uploadFiles in job fileUploadJob
org.springframework.transaction.TransactionSystemException: Could not commit JPA transaction; nested exception is javax.persistence.RollbackException: Transaction marked as rollbackOnly
at org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:524)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:757)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:726)
I had same error "Commit failed while step execution data was already updated. Reverting to old version" and I solved it by using " #Transactional(propagation = Propagation.REQUIRES_NEW)" annotation above the method where the records were committed.
I'm using Spring Batch for import items from XML to database. After import I create log with invalid records. I get these items by configuring skippable exceptions:
<batch:chunk reader="reader" processor="processor" writer="writer" commit-interval="10" skip-limit="99999999">
<batch:skippable-exception-classes>
<batch:include class="java.lang.Exception"/>
</batch:skippable-exception-classes>
<batch:listeners>
<batch:listener ref="recordSkipListener"/>
<batch:listener ref="itemReadListener"/>
</batch:listeners>
</batch:chunk>
My log after step execution:
Start time: 30.12.2015 01:05:38
End time: 30.12.2015 01:20:59
Read count: 3842
Skip count: 0
Write count: 3522
Skip count calculated by this expression:
int skipCount = stepExecution.getReadSkipCount() + stepExecution.getProcessSkipCount() + stepExecution.getWriteSkipCount();
"RecordSkipListener" handles all exceptions throwed in processor and writer, but it read, process and write methods are never called during step execution. Only ChunkListener.afterChunkError method is called, but its arguments doesn't contains info about failed record.
Therefore I have two questions:
How I can log failed items?
Why Spring Batch doesn't implement behavior same as behavior on ItemWriter.write:
failed chunk are divided onto chunks with size=1 and then processed on dedicated transaction?
// Update
Even if I decrease chunk-size to 1 then about 50 records are not written. And I cannot log their.
I'll answer your questions in reverse order:
Why Spring Batch doesn't implement behavior same as behavior on ItemWriter.write: failed chunk are divided onto chunks with size=1 and then processed on dedicated transaction?
When an exception is thrown during the process or write phase of the step, we can provide the item that caused the error because we have a valid item. The issue with an exception that occurs during the read phase is that since an exception has been thrown, we don't have an item to give to anyone. The only way a reader has to communicate what the input was that caused the exception is by embedding information about that input into the exception being thrown. An example of this is when an exception is thrown during the parsing of a line in the FlatFileItemReader, we throw a FlatFileParseException. That exception has both the line number and the contents of the line that caused the exception. You can use the ItemReadListener#onReadError method to catch the exception and obtain the information provided in the exception there.
How I can log failed items?
With XML, the ability to provide insight into what went wrong is difficult since it's not really within the reader's control (it's really a function of the unmarshaller and what went wrong there). The best you can do is to impute the bad record from the record count (via the ExecutionContext). That would tell you what number record caused the error within your XML file.
Issue of this problem is that at chunk commiting was exception: length of one of entity fields was exceeded allowed size. In this case processing and writing doesn't retry
I am using two Message Drive Beans (MDB), to update same table. I am using EJB 3.0 and using transaction type as
#TransactionAttribute(TransactionAttributeType.REQUIRES_NEW)
some times one of the methods are getting error like "Transaction is ended due to timeout" and MDB is getting deactivated. Hence , I have to restart the server.
Can I use any other type of transaction attribute #TransactionAttribute(TransactionAttributeType.NOT_SUPPORTED) to avoid the timeout error ?
Here is my stacktrace.
[:] CWWMQ0007W: The message endpoint Rsme#RsmeEJB.jar#GemsInqMDB has been paused by the system. Message delivery failed to the endpoint more than 0 times. The last attempted delivery failed with the following error: javax.ejb.EJBTransactionRolledbackException: Transaction rolled back; nested exception is: javax.transaction.TransactionRolledbackException: Transaction is ended due to timeout
javax.transaction.TransactionRolledbackException: Transaction is ended due to timeout
at com.ibm.tx.jta.impl.EmbeddableTranManagerImpl.completeTxTimeout(EmbeddableTranManagerImpl.java:62)
at com.ibm.tx.jta.impl.EmbeddableTranManagerSet.completeTxTimeout(EmbeddableTranManagerSet.java:85)
at com.ibm.ejs.csi.TransactionControlImpl.completeTxTimeout(TransactionControlImpl.java:1347)
at com.ibm.ejs.csi.TranStrategy.postInvoke(TranStrategy.java:242)
at com.ibm.ejs.csi.TransactionControlImpl.postInvoke(TransactionControlImpl.java:579)
at com.ibm.ejs.container.EJSContainer.postInvoke(EJSContainer.java:4843)
at com.maybank.meaa.local.EJSLocal0SLMeaaEntityBean_877f3cd5.insertPD003Result(EJSLocal0SLMeaaEntityBean_877f3cd5.java)
at com.maybank.meaa.mdbs.GemsInqMDB.onMessage(GemsInqMDB.java:75)
at com.ibm.ejs.container.MessageEndpointHandler.invokeMdbMethod(MessageEndpointHandler.java:1163)
at com.ibm.ejs.container.MessageEndpointHandler.invoke(MessageEndpointHandler.java:842)
at $Proxy27.onMessage(Unknown Source)
at com.ibm.mq.connector.inbound.MessageEndpointWrapper.onMessage(MessageEndpointWrapper.java:131)
at com.ibm.mq.jms.MQSession$FacadeMessageListener.onMessage(MQSession.java:147)
at com.ibm.msg.client.jms.internal.JmsSessionImpl.run(JmsSessionImpl.java:2665)
at com.ibm.mq.jms.MQSession.run(MQSession.java:862)
at com.ibm.mq.connector.inbound.WorkImpl.run(WorkImpl.java:279)
at com.ibm.ejs.j2c.work.WorkProxy.run(WorkProxy.java:608)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1650)
javax.ejb.EJBTransactionRolledbackException: Transaction rolled back; nested exception is: javax.transaction.TransactionRolledbackException: Transaction is ended due to timeout
Caused by: javax.transaction.TransactionRolledbackException: Transaction is ended due to timeout
at com.ibm.tx.jta.impl.EmbeddableTranManagerImpl.completeTxTimeout(EmbeddableTranManagerImpl.java:62)
at com.ibm.tx.jta.impl.EmbeddableTranManagerSet.completeTxTimeout(EmbeddableTranManagerSet.java:85)
at com.ibm.ejs.csi.TransactionControlImpl.completeTxTimeout(TransactionControlImpl.java:1347)
at com.ibm.ejs.csi.TranStrategy.postInvoke(TranStrategy.java:242)
at com.ibm.ejs.csi.TransactionControlImpl.postInvoke(TransactionControlImpl.java:579)
at com.ibm.ejs.container.EJSContainer.postInvoke(EJSContainer.java:4843)
at com.maybank.meaa.local.EJSLocal0SLMeaaEntityBean_877f3cd5.insertPD003Result(EJSLocal0SLMeaaEntityBean_877f3cd5.java)
at com.maybank.meaa.mdbs.GemsInqMDB.onMessage(GemsInqMDB.java:75)
at com.ibm.ejs.container.MessageEndpointHandler.invokeMdbMethod(MessageEndpointHandler.java:1163)
at com.ibm.ejs.container.MessageEndpointHandler.invoke(MessageEndpointHandler.java:842)
at $Proxy27.onMessage(Unknown Source)
at com.ibm.mq.connector.inbound.MessageEndpointWrapper.onMessage(MessageEndpointWrapper.java:131)
at com.ibm.mq.jms.MQSession$FacadeMessageListener.onMessage(MQSession.java:147)
at com.ibm.msg.client.jms.internal.JmsSessionImpl.run(JmsSessionImpl.java:2665)
at com.ibm.mq.jms.MQSession.run(MQSession.java:862)
at com.ibm.mq.connector.inbound.WorkImpl.run(WorkImpl.java:279)
at com.ibm.ejs.j2c.work.WorkProxy.run(WorkProxy.java:608)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1650)
.
Create two different tables with the same structure to avoid the deadlock instead of using same table for two MDB
This error can be caused by normally 2 things
1- EJB Timeout
2- Datasource Timeout
You can configurate both to a greater value.
1 - WebSphere Application servers > <> container properties > Transaction service
Total transaction lifetime timeout - this will be the default value for EJB without this property
Maximum transaction timeout - this wil be the max value for any transaction managed by WAS
2 - Data sources > <> > Connection pools
Connection timeout
Unused timeout
Normally, the option 1 fix the problem.
My Application(java spring-core) has several threads running concurrently and accessing db, I am getting exception in some peaktime
07:43:33,400 WARN [org.hibernate.util.JDBCExceptionReporter] SQL Error: 1213, SQLState: 40001
07:43:33,808 ERROR [org.hibernate.util.JDBCExceptionReporter] Deadlock found when trying to get lock; try restarting transaction
07:43:33,808 ERROR [org.hibernate.event.def.AbstractFlushingEventListener] Could not synchronize database state with session
org.hibernate.exception.LockAcquisitionException: could not insert: [com.xminds.bestfriend.frontend.model.Question]
at org.hibernate.exception.SQLStateConverter.convert(SQLStateConverter.java:107)
at org.hibernate.exception.JDBCExceptionHelper.convert(JDBCExceptionHelper.java:66)
at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:2436)
at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:2856)
at org.hibernate.action.EntityInsertAction.execute(EntityInsertAction.java:79)
at org.hibernate.engine.ActionQueue.execute(ActionQueue.java:273)
at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:265)
at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:184)
at org.hibernate.event.def.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:321)
at org.hibernate.event.def.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:51)
at org.hibernate.impl.SessionImpl.flush(SessionImpl.java:1216)
at org.hibernate.impl.SessionImpl.managedFlush(SessionImpl.java:383)
at org.hibernate.transaction.JDBCTransaction.commit(JDBCTransaction.java:133)
at org.springframework.orm.hibernate3.HibernateTransactionManager.doCommit(HibernateTransactionManager.java:656)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:754)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:723)
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:147)
at com.xminds.bestfriend.consumers.Base.onMessage(Base.java:96)
at org.springframework.jms.listener.adapter.MessageListenerAdapter.onMessage(MessageListenerAdapter.java:339)
at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:535)
at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:495)
at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:467)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:325)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:263)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1058)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1050)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:947)
at java.lang.Thread.run(Thread.java:662)
Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLTransactionRollbackException: Deadlock found when trying to get lock; try restarting transaction
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
at com.mysql.jdbc.Util.getInstance(Util.java:386)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1065)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4074)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4006)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2468)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2629)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2719)
at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2155)
at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2450)
at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2371)
at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2355)
at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeUpdate(NewProxyPreparedStatement.java:105)
at org.hibernate.jdbc.NonBatchingBatcher.addToBatch(NonBatchingBatcher.java:46)
at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:2416)
... 25 more
My code looks
try
{
this.consumerTransactionTemplate.execute(new TransactionCallbackWithoutResult(){
#Override
protected void doInTransactionWithoutResult(
TransactionStatus status)
{
process();
}
});
}
catch(Exception e){
logger.error("Exception occured " , e);
//TODO: Exception handling
}
MySQL's InnoDB engine sports row-level locking, which can lead to deadlocks even when your code is inserting or updating a single row (specially if there are several indexes on the table being updated). Your best bet is to design the code around this in order to retry a transaction if it fails due to a deadlock. Some useful info about MySQL deadlock diagnose and possible workarounds is available here.
An interesting implementation of deadlock retry via AOP in Spring is available here. This way you just need to add the annotation to the method you want to retry in case of deadlock.
Emir's answer is great and it describes the problem that you are getting. However I suggest you to try spring-retry.
It's a brilliant framework that implements the retry pattern via annotation(s).
Example:
#Retryable(maxAttempts = 4, backoff = #Backoff(delay = 500))
public void doSomethingWithMysql() {
consumerTransactionTemplate.execute(
new TransactionCallbackWithoutResult(){
#Override
protected void doInTransactionWithoutResult(
TransactionStatus status)
{
process();
}
});
}
In case of exception, it will retry (call) up to 4 times the method doSomethingWithMysql() with a backoff policy of 500ms
If you are using JPA/Hibernate then simple just follow below steps to avoid dead lock. Once you have acquired the lock, don't made any call on db with same id anywhere in the transaction (I mean to say you should not get entity again on sameid), on locking object you modify and save no issues.
service level:-
employee=empDao.getForUpdate(id);
Dao level:-
public employee getForUpdate(String id)
return mySqlRepository.getForUpdate(id)
Repository(interface):-
#Lock(LockModeType.PESSIMITSIC_WRITE)
#Query("select e from employee e where id=?1")
public employee getForUpdate(String id)
Here is an example with plain Spring and no extra frameworks.
TransactionTemplate transactionTemplate = new TransactionTemplate(transactionManager); // autowired
transactionTemplate.setIsolationLevel(TransactionDefinition.ISOLATION_SERIALIZABLE); // this increases deadlocks, but prevents multiple simultaneous similar requests from inserting multiple rows
Object o = transactionTemplate.execute(txStatus -> {
for (int i=0; i<3; i++) {
try {
return findExistingOrCreate(...);
} catch (DeadlockLoserDataAccessException e) {
Logger.info(TAG, "create()", "Deadlock exception when trying to find or create. Retrying "+(2-i)+" more times...");
try { Thread.sleep(2^i*1000); } catch (InterruptedException e2) {}
}
}
return null;
});
if (o == null) throw new ApiException(HttpStatus.SERVICE_UNAVAILABLE, "Possible deadlock or busy database, please try again later.");
Using serializable transaction isolation level is specific to my situation because it converts SELECT to SELECT ... IN SHARE MODE / SELECT ... FOR UPDATE and locks those rows. The findExistingOrCreate() is doing a lot of complicated searching for existing rows and auto-generating names and checking for bad words, etc. When many of the same request came in at the same time, it would create multiple rows. With the serializable transaction isolation level, it is now idempotent; it now locks the rows, creates a single row, and all subsequent requests return the new existing row.
When you face this kind of error "deadlock detected". You should inspect your queries execution and verify if two or more concurrent transactions can cause a deadlock.
These transactions should acquire database locks in the same order in order to avoid deadlock.
This can happen on none-concurrent applications with one thread inserting records consecutively, too. In case a table has a unique constraint MySQL "builds" that constraint after the commit. That locks the table and might disturb the next insert leading to the above mentioned deadlock. Although I only noticed that error on Windows.
Like on all other answers, repeating the insert solved the issue.
With other databases - PostgreSQL, Oracle or H2 - it works without this workaround.