Develop Reference

Spring JPA + MySQL Deadlock found on commit but the deadlock is not registered into the error_log of the database

I have a method annotated with #Transactional which obtain messages and perform some operations on the database to persist things correctly.
Now, some times this method throws a DeadLock with the following stacktrace:
Caused by: org.hibernate.TransactionException: Unable to commit against JDBC Connection
at org.hibernate.resource.jdbc.internal.AbstractLogicalConnectionImplementor.commit(AbstractLogicalConnectionImplementor.java:87)
at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl$TransactionDriverControlImpl.commit(JdbcResourceLocalTransactionCoordinatorImpl.java:272)
at org.hibernate.engine.transaction.internal.TransactionImpl.commit(TransactionImpl.java:104)
at org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:532)
... 34 common frames omitted
Caused by: com.mysql.cj.jdbc.exceptions.MySQLTransactionRollbackException: Deadlock found when trying to get lock; try restarting transaction
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:123)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97)
at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122)
at com.mysql.cj.jdbc.ConnectionImpl.commit(ConnectionImpl.java:813)
at com.zaxxer.hikari.pool.ProxyConnection.commit(ProxyConnection.java:361)
at com.zaxxer.hikari.pool.HikariProxyConnection.commit(HikariProxyConnection.java)
at org.hibernate.resource.jdbc.internal.AbstractLogicalConnectionImplementor.commit(AbstractLogicalConnectionImplementor.java:81)
... 37 common frames omitted
The problem I am facing is that on the database side, MySQL is not registering any deadlock to happen in that moment.
I've enabled the innodb_print_all_deadlocks variable and I verified that it is working as expected by manually raising deadlocks which are immediately logged by the db.
I've also verified that when a real deadlock happens within the application (with real I mean that MySQL logs the deadlock on its side), the stacktrace is different and usually the deadlock is found by the database server way before the commit, for example when the application tries to ClientPreparedStatement.executeUpdateInternal.
It looks weird to me that the deadlock is raised on ConnectionImpl.commit. I've checked the ConnectionImpl code and I can't understand why a deadlock could be thrown there (at the line in the stacktrace, there's just return; )
Is it possible that JPA is somehow raising a deadlock on the app side?

Leaving this as it might help other people stuck with the same weird problem.
The reason for my problem was the fact that the application was using MariaDB driver instead of MySQL one. For some reasons, the MariaDB driver was throwing fake deadlock exceptions, so the MySQL server was correct in not logging any deadlock happening.
Switching the driver with the more correct one, all problems stopped.

ORA-02067 error on Spring MVC / Oracle / Tomcat

Randomly my Spring MVC/Oracle 11G/Tomcat 7 application is throwing the following error: “ORA-02067: transaction or savepoint rollback required”
Once that error appears ALL subsequent operations to the database that uses that same connection through the Tomcat will fail throwing the ORA-02067 error.
Example:
"detailErrorInfo": "CallableStatementCallback;
uncategorized SQLException for SQL
[{call SCHEMAX.[PACKAGEX].PROCEDUREX()}]; SQL state [60000];
error code [604];
ORA-00604: error occurred at recursive SQL level 1
ORA-02067: transaction or savepoint rollback required;
nested exception is java.sql.SQLException:
ORA-00604: error occurred at recursive SQL level 1
ORA-02067: transaction or savepoint rollback required",
Curiously, if I call that same Oracle package/procedure from other connection or application, it works.
The error stops only on the Spring application the moment the administrator restart the Tomcat.
My theory is there is a transaction that is failing and is not doing a rollback. Because the Tomcat is using a pool of JDBC connections, the error persists on the application.
I have not being able to reproduce that on the development environment.
Does Oracle provides tools to pinpoint the culprit transaction? Is there any view that display transaction that are pending a rollback?
Thanks

JDBC Connection Pool: connections are not recycled after DB restart

I added setMaxActive(8) on org.apache.tomcat.jdbc.pool.PoolProperties. Every time the DB restarts, the application is unusable because the established connections remain. I get the following error:
org.postgresql.util.PSQLException: This connection has been closed
I've tried using some other settings on the pool to no avail...
Thank you for help!

Use the validationQuery property which will check if the connection is valid before returning the connection.
Ref: Tomcat 6 JDBC Connection Pool
This property is available on latest tomcat versions.

Look at this link:
Postgres connection has been closed error in Spring Boot
Very valid question and this problem is usually faced by many. The
exception generally occurs, when network connection is lost between
pool and database (most of the time due to restart). Looking at the
stack trace you have specified, it is quite clear that you are using
jdbc pool to get the connection. JDBC pool has options to fine-tune
various connection pool settings and log details about whats going on
inside pool.
You can refer to to detailed Apache documentation on pool
configuration to specify abandon timeout
Check for removeAbandoned, removeAbandonedTimeout, logAbandoned parameters
Additionally you can make use of additional properties to further
tighten the validation
Use testXXX and validationQuery for connection validity.
My own $0.02: use these two parameters:
validationQuery=<TEST SQL>
testOnBorrow=true

Setting c3p0 configuration for long running query

I am using Hibernate version 3.3.1 and
jtds 1.2.2 as JDBC driver and
c3p0 version 0.9.1.2 for connection pooling to connect SQL Server.
My query takes about 12 seconds to run.
When i run the query, I get the following exception
ERROR [main] (JDBCExceptionReporter.java:101) - Invalid state, the Connection object is closed.
org.hibernate.exception.GenericJDBCException: could not inspect JDBC autocommit mode
at org.hibernate.exception.SQLStateConverter.handledNonSpecificException(SQLStateConverter.java:126)
at org.hibernate.exception.SQLStateConverter.convert(SQLStateConverter.java:114)
at org.hibernate.exception.JDBCExceptionHelper.convert(JDBCExceptionHelper.java:66)
at org.hibernate.exception.JDBCExceptionHelper.convert(JDBCExceptionHelper.java:52)
at org.hibernate.jdbc.JDBCContext.afterNontransactionalQuery(JDBCContext.java:275)
at org.hibernate.impl.SessionImpl.afterOperation(SessionImpl.java:444)
at org.hibernate.impl.SessionImpl.listCustomQuery(SessionImpl.java:1728)
at org.hibernate.impl.AbstractSessionImpl.list(AbstractSessionImpl.java:165)
at org.hibernate.impl.SQLQueryImpl.list(SQLQueryImpl.java:175)
If I modify the query to return a small set of data, I do not get exception.
It seems there is some configuration problem.
In my hibernate.properties file, I have the following configuration values
hibernate.format_sql=true
hibernate.dialect=org.hibernate.dialect.SQLServerDialect
hibernate.optimistic-lock=true
hibernate.connection.autocommit=true
hibernate.show_sql=false
hibernate.generate_statistics=false
c3p0.acquire_increment=1
c3p0.idle_test_period=1000
c3p0.max_size=10
c3p0.max_statements=0
c3p0.min_size=5
c3p0.timeout=800
Can you please suggest what parameters needs to be set to run a long running query?
Thank you

c3p0 requires no special setup for long-running queries, although there are some settings (in particular unreturnedConnectionTimeout) that could interfere with long-running queries if set.
i'd verify that unreturnedConnectionTimeout is not set. (there are lots of places c3p0 configuration might be set outside of your hibernate.properties file.) c3p0 dumps its config to log at INFO on pool initialization. look for the value of unreturnedConnectionTimeout. it should be 0.
if it is 0, there is nothing at the c3p0 level that should interfere with your long-duration query.

How to reestablish a JDBC connection after a timeout?

I have a long-running method which executes a large number of native SQL queries through the EntityManager (TopLink Essentials). Each query takes only milliseconds to run, but there are many thousands of them. This happens within a single EJB transaction. After 15 minutes, the database closes the connection which results in following error:
Exception [TOPLINK-4002] (Oracle TopLink Essentials - 2.1 (Build b02-p04 (04/12/2010))): oracle.toplink.essentials.exceptions.DatabaseException
Internal Exception: java.sql.SQLException: Closed Connection
Error Code: 17008
Call: select ...
Query: DataReadQuery()
at oracle.toplink.essentials.exceptions.DatabaseException.sqlException(DatabaseException.java:319)
.
.
.
RAR5031:System Exception.
javax.resource.ResourceException: This Managed Connection is not valid as the phyiscal connection is not usable
at com.sun.gjc.spi.ManagedConnection.checkIfValid(ManagedConnection.java:612)
In the JDBC connection pool I set is-connection-validation-required="true" and connection-validation-method="table" but this did not help .
I assumed that JDBC connection validation is there to deal with precisely this kind of errors. I also looked at TopLink extensions (http://www.oracle.com/technetwork/middleware/ias/toplink-jpa-extensions-094393.html) for some kind of timeout settings but found nothing. There is also the TopLink session configuration file (http://download.oracle.com/docs/cd/B14099_19/web.1012/b15901/sessions003.htm) but I don't think there is anything useful there either.
I don't have access to the Oracle DBA tables, but I think that Oracle closes connections after 15 minutes according to the setting in CONNECT_TIME profile variable.
Is there any other way to make TopLink or the JDBC pool to reestablish a closed connection?
The database is Oracle 10g, application server is Sun Glassfish 2.1.1.

All JPA implementations (running on a Java EE container) use a datasource with an associated connection pool to manage connectivity with the database.
The persistence context itself is associated with the datasource via an appropriate entry in persistence.xml. If you wish to change the connection timeout settings on the client-side, then the associated connection pool must be re-configured.
In Glassfish, the timeout settings associated with the connection pool can be reconfigured by editing the pool settings, as listed in the following links:
Changing timeout settings in GlassFish 3.1
Changing timeout settings in GlassFish 2.1
On the server-side (whose settings if lower than the client settings, would be more important), the Oracle database can be configured to have database profiles associated with user accounts. The session idle_time and connect_time parameters of a profile would constitute the timeout settings of importance in this aspect of the client-server interaction. If no profile has been set, then by default, the timeout is unlimited.

Unless you've got some sort of RAC failover, when the connection is terminated, it will end the session and transaction.
The admins may have set into some limits to prevent runaway transactions or a single job 'hogging' a connection in a pool. You generally don't want to lock a connection in a pool for an extended period.
If these queries aren't necessarily part of the same transaction, then you could try terminating and restarting a new connection.
Are you able to restructure your code so that it completes in under 15 minutes. A stored procedure in the background may be able to do the job a lot quicker than dragging the results of thousands of operations over the network.

I see you set your connection-validation-method="table" and is-connection-validation-required="true", but you do not mention that you specified the table you were validating on; did you set validation-table-name="any_table_you_know_exists" and provide any existing table-name? validation-table-name="existing_table_name" is required.
See this article for more details on connection validation.
Related StackOverflow article with similar problem - he wants to flush the entire invalid connection pool.