SQLite in-memory database encounters SQLITE_LOCKED_SHAREDCACHE intermittently - java

I am using mybatis 3.4.6 along with org.xerial:sqlite-jdbc 3.28.0. Below is my configuration to use an in-memory database with shared mode enabled
db.driver=org.sqlite.JDBC
db.url=jdbc:sqlite:file::memory:?cache=shared
The db.url is correct according to this test class
And I managed to setup the correct transaction isolation level with below mybatis configuration though there is a typo of property read_uncommitted according to this issue which is reported by me as well
<environment id="${db.env}">
<transactionManager type="jdbc"/>
<dataSource type="POOLED">
<property name="driver" value="${db.driver}" />
<property name="url" value="${db.url}"/>
<property name="username" value="${db.username}" />
<property name="password" value="${db.password}" />
<property name="defaultTransactionIsolationLevel" value="1" />
<property name="driver.synchronous" value="OFF" />
<property name="driver.transaction_mode" value="IMMEDIATE"/>
<property name="driver.foreign_keys" value="ON"/>
</dataSource>
</environment>
This line of configuration
<property name="defaultTransactionIsolationLevel" value="1" />
does the trick to set the correct value of PRAGMA read_uncommitted
I am pretty sure of it since I debugged the underneath code which initialize the connection and check the value has been set correctly
However with the above setting, my program still encounters SQLITE_LOCKED_SHAREDCACHE intermittently while reading, which I think it shouldn't happen according the description highlighted in the red rectangle of below screenshot. I want to know the reason and how to resolve it, though the occurring probability of this error is low.
Any ideas would be appreciated!!
The debug configurations is below
===CONFINGURATION==============================================
jdbcDriver org.sqlite.JDBC
jdbcUrl jdbc:sqlite:file::memory:?cache=shared
jdbcUsername
jdbcPassword ************
poolMaxActiveConnections 10
poolMaxIdleConnections 5
poolMaxCheckoutTime 20000
poolTimeToWait 20000
poolPingEnabled false
poolPingQuery NO PING QUERY SET
poolPingConnectionsNotUsedFor 0
---STATUS-----------------------------------------------------
activeConnections 5
idleConnections 5
requestCount 27
averageRequestTime 7941
averageCheckoutTime 4437
claimedOverdue 0
averageOverdueCheckoutTime 0
hadToWait 0
averageWaitTime 0
badConnectionCount 0
===============================================================
Attachments:
The exception is below
org.apache.ibatis.exceptions.PersistenceException:
### Error querying database. Cause: org.apache.ibatis.transaction.TransactionException: Error configuring AutoCommit. Your driver may not support getAutoCommit() or setAutoCommit(). Requested setting: false. Cause: org.sqlite.SQLiteException: [SQLITE_LOCKED_SHAREDCACHE] Contention with a different database connection that shares the cache (database table is locked)
### The error may exist in mapper/MsgRecordDO-sqlmap-mappering.xml
### The error may involve com.super.mock.platform.agent.dal.daointerface.MsgRecordDAO.getRecord
### The error occurred while executing a query
### Cause: org.apache.ibatis.transaction.TransactionException: Error configuring AutoCommit. Your driver may not support getAutoCommit() or setAutoCommit(). Requested setting: false. Cause: org.sqlite.SQLiteException: [SQLITE_LOCKED_SHAREDCACHE] Contention with a different database connection that shares the cache (database table is locked)

I finally resolved this issue by myself and share the workaround below in case someone else encounters similar issue in the future.
First of all, we're able to get the completed call stack of the exception shown below
Going through the source code indicated by the callback, we have below findings.
SQLite is built-in with auto commit enabled by default which is contradict with MyBatis which disables auto commit by default since we're using SqlSessionManager
MyBatis would override the auto commit property during connection initialization using method setDesiredAutoCommit which finally invokes SQLiteConnection#setAutoCommit
SQLiteConnection#setAutoCommit would incur a begin immediate operation against the database which is actually exclusive, check out below source code screenshots for detailed explanation since we configure our transaction mode to be IMMEDIATE
<property name="driver.transaction_mode" value="IMMEDIATE"/>
So until now, An apparent solution is to change the transaction mode to be DEFERRED. Furthermore, the solution of making the auto commit setting the same between MyBatis and SQLite has been considered as well, however, it's not adopted since there is no way to set the auto commit of SQLiteConnection during initialization stage, there would be always switching (from true to false or vice versa) and switch would cause the above error probably if transaction mode is not set properly

Related

How to fix "No more data to read from socket" exception? [duplicate]

We are using Oracle as the database for our Web application. The application runs well most of the time, but we get this "No more data to read from socket" error.
Caused by: java.sql.SQLRecoverableException: No more data to read from socket
at oracle.jdbc.driver.T4CMAREngine.unmarshalUB1(T4CMAREngine.java:1142)
at oracle.jdbc.driver.T4CMAREngine.unmarshalSB1(T4CMAREngine.java:1099)
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:288)
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:191)
at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:523)
at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:207)
at oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:863)
at oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1153)
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1275)
at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3576)
at oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3620)
at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1491)
at org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:93)
at org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:93)
at org.hibernate.jdbc.AbstractBatcher.getResultSet(AbstractBatcher.java:208)
at org.hibernate.loader.Loader.getResultSet(Loader.java:1869)
at org.hibernate.loader.Loader.doQuery(Loader.java:718)
at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:270)
at org.hibernate.loader.Loader.doList(Loader.java:2449)
... 63 more
We use spring, hibernate and i have the following for the datasource in my applciation context file.
<bean class="org.apache.commons.dbcp.BasicDataSource"
destroy-method="close" id="dataSource">
<property name="driverClassName" value="${database.driverClassName}" />
<property name="url" value="${database.url}" />
<property name="username" value="${database.username}" />
<property name="password" value="${database.password}" />
<property name="defaultAutoCommit" value="false" />
<property name="initialSize" value="10" />
<property name="maxActive" value="30" />
<property name="validationQuery" value="select 1 from dual" />
<property name="testOnBorrow" value="true" />
<property name="testOnReturn" value="true" />
<property name="poolPreparedStatements" value="true" />
<property name="removeAbandoned" value="true" />
<property name="logAbandoned" value="true" />
</bean>
I am not sure whether this is because of application errors, database errors or network errors.
We see the following on the oracle logs
Thu Oct 20 10:29:44 2011
Errors in file d:\oracle\diag\rdbms\ads\ads\trace\ads_ora_3836.trc (incident=31653):
ORA-03137: TTC protocol internal error : [12333] [4] [195] [3] [] [] [] []
Incident details in: d:\oracle\diag\rdbms\ads\ads\incident\incdir_31653\ads_ora_3836_i31653.trc
Thu Oct 20 10:29:45 2011
Trace dumping is performing id=[cdmp_20111020102945]
Thu Oct 20 10:29:49 2011
Sweep [inc][31653]: completed
Sweep [inc2][31653]: completed
Thu Oct 20 10:34:20 2011
Errors in file d:\oracle\diag\rdbms\ads\ads\trace\ads_ora_860.trc (incident=31645):
ORA-03137: TTC protocol internal error : [12333] [4] [195] [3] [] [] [] []
Incident details in: d:\oracle\diag\rdbms\ads\ads\incident\incdir_31645\ads_ora_860_i31645.trc
Thu Oct 20 10:34:21 2011
Oracle Version : 11.2.0.1.0
For errors like this you should involve oracle support. Unfortunately you do not mention what oracle release you are using. The error can be related to optimizer bind peeking. Depending on the oracle version different workarounds apply.
You have two ways to address this:
upgrade to 11.2
set oracle parameter _optim_peek_user_binds = false
Of course underscore parameters should only be set if advised by oracle support
We were facing same problem, we resolved it by increasing initialSize and maxActive size of connection pool.
You can check this link
Maybe this helps someone.
Another case: If you are sending date parameters to a parameterized sql, make sure you sent java.sql.Timestamp and not java.util.Date. Otherwise you get
java.sql.SQLRecoverableException: No more data to read from socket
Example statement:
In our java code, we are using org.apache.commons.dbutils and we have the following:
final String sqlStatement = "select x from person where date_of_birth between ? and ?";
java.util.Date dtFrom = new Date(); //<-- this will fail
java.util.Date dtTo = new Date(); //<-- this will fail
Object[] params = new Object[]{ dtFrom , dtTo };
final List mapList = (List) query.query(conn, sqlStatement, new MapListHandler(),params);
The above was failing until we changed the date parameters to be java.sql.Timestamp
java.sql.Timestamp tFrom = new java.sql.Timestamp (dtFrom.getTime()); //<-- this is OK
java.sql.Timestamp tTo = new java.sql.Timestamp(dtTo.getTime()); //<-- this is OK
Object[] params = new Object[]{ tFrom , tTo };
final List mapList = (List) query.query(conn, sqlStatement, new MapListHandler(),params);
This is a very low-level exception, which is ORA-17410.
It may happen for several reasons:
A temporary problem with networking.
Wrong JDBC driver version.
Some issues with a special data structure (on database side).
Database bug.
In my case, it was a bug we hit on the database, which needs to be patched.
Try two things:
Set in $ORACLE_HOME/network/admin/tnsnames.ora on the oracle server server=dedicated to server=shared to allow more than one connection at a time. Restart oracle.
If you are using Java this might help you: In java/jdk1.6.0_31/jre/lib/security/Java.security change securerandom.source=file:/dev/urandom to securerandom.source=file:///dev/urandom
I had the same problem. I was able to solve the problem from application side, under the following scenario:
JDK8, spring framework 4.2.4.RELEASE, apache tomcat 7.0.63, Oracle Database 11g Enterprise Edition 11.2.0.4.0
I used the database connection pooling apache tomcat-jdbc:
You can take the following configuration parameters as a reference:
<Resource name="jdbc/exampleDB"
auth="Container"
type="javax.sql.DataSource"
factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
testWhileIdle="true"
testOnBorrow="true"
testOnReturn="false"
validationQuery="SELECT 1 FROM DUAL"
validationInterval="30000"
timeBetweenEvictionRunsMillis="30000"
maxActive="100"
minIdle="10"
maxWait="10000"
initialSize="10"
removeAbandonedTimeout="60"
removeAbandoned="true"
logAbandoned="true"
minEvictableIdleTimeMillis="30000"
jmxEnabled="true"
jdbcInterceptors="org.apache.tomcat.jdbc.pool.interceptor.ConnectionState;
org.apache.tomcat.jdbc.pool.interceptor.StatementFinalizer"
username="your-username"
password="your-password"
driverClassName="oracle.jdbc.driver.OracleDriver"
url="jdbc:oracle:thin:#localhost:1521:xe"/>
This configuration was sufficient to fix the error. This works fine for me in the scenario mentioned above.
For more details about the setup apache tomcat-jdbc: https://tomcat.apache.org/tomcat-7.0-doc/jdbc-pool.html
Downgrading the JRE from 7 to 6 fixed this issue for me.
Yes, as #ggkmath said, sometimes a good old restart is exactly what you need. Like when "contact the author and have him rewrite the app, meanwhile wait" is not an option.
This happens when an application is not written (yet) in a way that it can handle restarts of the underlying database.
In our case we had a query which loads multiple items with select * from x where something in (...)
The in part was so long for benchmark test.(17mb as text query). Query is valid but text so long. Shortening the query solved the problem.
I got this error then restarted my GlassFish server that held connection pools between my client app and the database, and the error went away. So, try restarting your application server if applicable.
I seemed to fix my instance by removing the parameter placeholder for a parameterized query.
For some reason, using these placeholders were working fine, and then they stopped working and I got the error/bug.
As a workaround, I substituted literals for my placeholders and it started working.
Remove this
where
SOME_VAR = :1
Use this
where
SOME_VAR = 'Value'
Seemed to be an issue with a view. JDBC query was using a view. I took a guess, recompiled the view and error is gone.
In my case the error occurs running a simple query like this in SQLdeveloper:
select count(1) from tabel1 inner join tabel2 on table1.id = table2.id_table1 ;
I solved the error so...
select
/*+OPT_PARAM('_index_join_enabled' 'false') */
count(1) from tabel1 inner join tabel2 on table1.id = table2.id_table1 ;

how to avoid "lock timeout" when updating DB using multiple threads?

I am trying to update a table using multiple threads. But I am not updating the same records/rows at the same time. I am grouping the table into different groups and trying to update them simultaneously. However, I am getting the locked timeout error all the time.
I am using Hibernate, Spring MVC, ThreadPoolTaskExecutor and MySQL. I am getting the data from another DB schema and updating my own database. The data is huge which is why i want to use multi threads so it can be done faster. However, it's producing "lock timeout" error. Can anyone help please? thanks for your good heart.
I call sessionFactory.getCurrenSession() to update the database table.
here is my config:
<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource"
destroy-method="close" p:driverClassName="${jdbc.driverClassName}"
p:url="${jdbc.url}" p:username="${jdbc.username}" p:password="${jdbc.password}">
</bean>
<bean id="sessionFactory"
class="org.springframework.orm.hibernate4.LocalSessionFactoryBean">
<property name="dataSource" ref="dataSource" />
<property name="configLocation">
<value>classpath:hibernate.cfg.xml</value>
</property>
<property name="hibernateProperties">
<props>
<prop key="hibernate.dialect">org.hibernate.dialect.MySQLDialect</prop>
<prop key="hibernate.show_sql">true</prop>
</props>
</property>
</bean>
<bean id="transactionManager"
class="org.springframework.orm.hibernate4.HibernateTransactionManager">
<property name="sessionFactory" ref="sessionFactory" />
</bean>
<bean id="taskExecutor"
class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
<property name="corePoolSize" value="5" />
<property name="maxPoolSize" value="10" />
<property name="WaitForTasksToCompleteOnShutdown" value="true" />
</bean>
here is my stacktrace:
WARN : org.hibernate.engine.jdbc.spi.SqlExceptionHelper - SQL Error: 1205, SQLState: 41000
ERROR: org.hibernate.engine.jdbc.spi.SqlExceptionHelper - Lock wait timeout exceeded; try restarting transaction
Exception in thread "taskExecutor-5" Exception in thread "taskExecutor-4" Exception in thread "taskExecutor-2" org.hibernate.exception.LockTimeoutException: could not execute statement
at org.hibernate.dialect.MySQLDialect$1.convert(MySQLDialect.java:407)
at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:49)
at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:125)
at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:110)
at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:136)
at org.hibernate.hql.internal.ast.exec.BasicExecutor.execute(BasicExecutor.java:103)
at org.hibernate.hql.internal.ast.QueryTranslatorImpl.executeUpdate(QueryTranslatorImpl.java:413)
at org.hibernate.engine.query.spi.HQLQueryPlan.performExecuteUpdate(HQLQueryPlan.java:282)
at org.hibernate.internal.SessionImpl.executeUpdate(SessionImpl.java:1289)
at org.hibernate.internal.QueryImpl.executeUpdate(QueryImpl.java:116)
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:96)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:260)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:94)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: java.sql.SQLException: Lock wait timeout exceeded; try restarting transaction
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1084)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4232)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4164)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2615)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2776)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2838)
at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2082)
at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2334)
at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2262)
at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2246)
at org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:105)
at org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:105)
at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:133)
... 25 more
A "lock wait timeout" can always happen (even with a large amount of inserts in one transaction) and there is no silver bullet to solve it. But I managed to get around it when I was trying to update half of all the records in one (relative small) table while the other half was being modified by another server.
Review all SQL statements in the transaction.
Use explain to make sure indexes are used where possible. Remove any statements that are not needed as part of the transaction.
Optimize the order of the SQL statements in the transaction.
This was a bit of trial and error for me, but try to imagine which order of SQL statements coming from multiple threads/connections might be easier to deal with for the database. In my case, just switching the order of two SQL statements made the "lock wait timeout" occur less frequent.
Update smaller subsets.
This finally solved the "lock wait timeout" for me. In my case there was an indexed column that allowed me to divide the larger update set into smaller subsets. So now one big update transaction was turned into about ten smaller update transactions. Keep in mind though that you need to be able to continue the smaller transactions after a crash (i.e. data must remain consistent in such a way that your application can redo the operation and have the same result).
Whether or not multiple threads will improve the throughput (updated rows per second) remains to be seen: it depends on the size of the update sets (network latency) and how efficiently MySQL can handle the locks for the table(s) to update the rows. You might only see a marginal improvement when using two threads/connections instead of one.
[Edit] Also watch out for database triggers/procedures: they can impact performance in a bad manner.
Maybe you could try to lower isolation level. If it helps you can dig more. It should speed up also execution in multi threaded environment.
If you are using annotations you can achieve this by
#Transactional(isolation=Isolation.READ_UNCOMMITTED)
on top of your transactional class.
This appears to be a timeout on the database side. I'd guess that the database is the limiting factor, so adding threads in your application doesn't help.
If you want to use threads to speed things up, I'd suggest using only two threads. While one thread reads from the other database, the second thread writes to the MySQL database.
Note that if both databases are on the same database server, even that won't help. You would need a faster database or a beefier database machine.

How to connect to DB in hibernate using JNDI

I am using jndi/db alias name used to connect to DB. Its working fine.. But, i need to accomplish the same using Hibernate...
I am not able to connect when i use below properties,
<property name="hibernate.connection.datasource">jdbc/ASPTADMINDEV</property>
And in resurce.xml file, i am using this,
(
<factories xmi:type="resources.jdbc:DataSource" xmi:id="DataSource_1233129165895" name="Oracle JDBC Driver DataSource" jndiName="jdbc/ASPTADMINDEV" description="New JDBC Datasource" providerType="Oracle JDBC Driver" authMechanismPreference="BASIC_PASSWORD" authDataAlias="ccixdmgr-nprd1-01-nodeMgr/GITPTDEV-ASPTADMIN-ASPTI" manageCachedHandles="false" logMissingTransactionContext="true" diagnoseConnectionUsage="false" relationalResourceAdapter="builtin_rra" statementCacheSize="10" datasourceHelperClassname="com.ibm.websphere.rsadapter.Oracle10gDataStoreHelper">
<propertySet xmi:id="J2EEResourcePropertySet_1233129165915">
<resourceProperties xmi:id="J2EEResourceProperty_1233129165915" name="driverType" type="java.lang.String" value="oci8" description="The type of the driver. The possible values are: thin, oci8." required="false"/>
<resourceProperties xmi:id="J2EEResourceProperty_1233129165916" name="oracleLogFileSizeLimit" type="java.lang.Integer" value="0" description="Oracle10g and beyond: The oracleLogFileSizeLimit specifies the maximum number of bytes to be written to any one file. Property is relevant only if trace file is specified. Default is unlimited" required="false"/>
<resourceProperties xmi:id="J2EEResourceProperty_1233129165917" name="oracleLogFileCount" type="java.lang.Integer" value="1" description="Oracle10g and beyond: The oracleLogFileCount specifies the number of files to use. Property is relevant only if trace file is specified. Default is 1." required="false"/>
<resourceProperties xmi:id="J2EEResourceProperty_1233129165918" name="oracleLogFileName" type="java.lang.String" value="" description="Oracle10g and beyond: The oracleLogFileName indicates which file to write the traces to" required="false"/>
<resourceProperties xmi:id="J2EEResourceProperty_1233129165919" name="oracleLogTraceLevel" type="java.lang.String" value="INFO" description="Oracle10g and beyond: The oracleLogTraceLevel specifies which message ">
......... and so
}
please let me know what i am doing mistake here..
Forget to paste the error ;).. Below is the error, i am getting
INFO: JNDI InitialContext properties:{}
Apr 9, 2014 4:17:19 PM org.hibernate.connection.DatasourceConnectionProvider configure
SEVERE: Could not find datasource: java:comp/env/jdbc/ASPTADMINDEV
javax.naming.NameNotFoundException: Name jdbc is not bound in this Context
Initial SessionFactory creation failed.org.hibernate.HibernateException: Could not find datasource
at org.apache.naming.NamingContext.lookup(NamingContext.java:770)
at org.apache.naming.NamingContext.lookup(NamingContext.java:140)
.
Do i need to change any thing. I am using below property
<property name="hibernate.connection.datasource">java:comp/env/jdbc/ASPTADMINDEV</property>
I think you need to give correct JNDI name of the data source, try this one.
<property name="hibernate.connection.datasource">java:comp/env/jdbc/ASPTADMINDEV</property>

Many threads created using C3P0 with Hibernate/Spring

I do a project merging Hibernate and Spring in a Java web application, using Tomcat under Linux environment. Due to the Mysql 8 hours timeout problem, we want to use C3P0 to manage a connection pool with our Mysql database.
But when we use it, we have numerous threads that are created. I figured it out beacause I did on each request a print of all of them with a memory status that show me the increasing memory and that kind of threads:
name: C3P0PooledConnectionPoolManager[identityToken->1hged7o8r13kpj7n1h3ycia|39c446]-HelperThread-#0 daemon: true group! main groupParent: system alive: true interrupted: false
name: C3P0PooledConnectionPoolManager[identityToken->1hged7o8r13kpj7n1h3ycia|17ec0e8]-AdminTaskTimer daemon: true group! main groupParent: system alive: true interrupted: false
It can produce more than 500 threads like these ones, after enough time.
Here is my Hibernate.cfg.xml:
<property name="connection.provider_class">
org.hibernate.connection.C3P0ConnectionProvider</property>
<property name="hibernate.c3p0.acquire_increment">1</property>
<property name="hibernate.c3p0.idle_test_period">5</property>
<property name="hibernate.c3p0.max_size">100</property>
<property name="hibernate.c3p0.max_statements">100</property>
<property name="hibernate.c3p0.min_size">10</property>
<property name="hibernate.c3p0.timeout">5</property>
<property name="hibernate.dialect">org.hibernate.dialect.MySQLDialect</property>
<property name="hibernate.connection.driver_class">com.mysql.jdbc.Driver</property>
<property name="hibernate.connection.url">jdbc:mysql://localhost:3306/myBase</property>
<property name="hibernate.connection.username">root</property>
<property name="hibernate.connection.password"></property>
<property name="hibernate.hbm2ddl.auto">update</property>
<property name="hibernate.default_schema">myProject</property>
<property name="hibernate.query.factory_class">org.hibernate.hql.classic.ClassicQueryTranslatorFactory</property>
<property name="show_sql">false</property>
<property name="cache.provider_class">
org.hibernate.cache.NoCacheProvider
</property>
I also tried to add a C3P0 propeties file, but except reducing the helper thread number, it don't delete the unsused thread:
c3p0.maxStatements=5
c3p0.maxIdleTime=10
c3p0.numHelperThreads=1
c3p0.testConnectionOnCheckout=true
c3p0.preferredTestQuery=SELECT 1
c3p0.initialPoolSize=1
c3p0.minPoolSize=1
c3p0.maxPoolSize=10
c3p0.acquireIncrement=1
c3p0.idleConnectionTestPeriod=1
Does anyone have an idea of why this happen and how to solve this problem?
Thanks a lot.
if you are seeing a multiplication of c3p0 helper and timer threads, you are somehow creating a multitude of c3p0 DataSources when you want there to be just one. sometimes this happens if you are hot-reloading your app but forgetting to close() your old c3p0 DataSource when you recycle.
effectively it looks like you are "leaking" DataSources. you need to figure out why/where this is happening. for some clues, check out your logs for c3p0 DataSource initialization messages at INFO level. Search for the string "Initializing c3p0 pool", for example.
good luck!
Ok I found a combination of properties to solve my problem, keeping in mind that I don't need a lot of connection at a time:
c3p0.maxStatements=5
c3p0.maxIdleTime=10
c3p0.numHelperThreads=3
c3p0.testConnectionOnCheckout=true
c3p0.preferredTestQuery=SELECT 1
c3p0.initialPoolSize=1
c3p0.minPoolSize=1
c3p0.maxPoolSize=1
c3p0.acquireIncrement=1
c3p0.idleConnectionTestPeriod=1
c3p0.maxAdministrativeTaskTime=1
Thanks to everyone
I would expect it to create a number of threads proportional to c3p0.minPoolSize
and c3p0.maxPoolSize and your maximum is 10.
http://www.mchange.com/projects/c3p0/#other_ds_configuration
"numHelperThreads and maxAdministrativeTaskTime help to configure the behavior of DataSource thread pools. By default, each DataSource has only three associated helper threads. If performance seems to drag under heavy load, or if you observe via JMX or direct inspection of a PooledDataSource, that the number of "pending tasks" is usually greater than zero, try increasing numHelperThreads. maxAdministrativeTaskTime may be useful for users experiencing tasks that hang indefinitely and "APPARENT DEADLOCK" messages. (See Appendix A for more.) "
numHelperThreads defines how many threads per DataSource are used, therefore indeed you will have 10 threads with numHelperThreads=1.
The only way to make sure C3P0 consumes only one Thread is to set c3p0.minPoolSize
and c3p0.maxPoolSize to 1 but this defeats the purpose of connection pooling.

Spring JDBCTemplate other MySQL datasource than apache commons?

I am using Spring JDBCTemplate to perform SQL operations on an apache commons datasource (org.apache.commons.dbcp.BasicDataSource) and when the service is up and running to long, i end up getting this exception:
org.springframework.dao.RecoverableDataAccessException: StatementCallback; SQL [SELECT * FROM vendor ORDER BY name]; The last packet successfully received from the server was 64,206,061 milliseconds ago. The last packet sent successfully to the server was 64,206,062 milliseconds ago. is longer than the server configured value of 'wait_timeout'. You should consider either expiring and/or testing connection validity before use in your application, increasing the server configured values for client timeouts, or using the Connector/J connection property 'autoReconnect=true' to avoid this problem.; nested exception is com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: The last packet successfully received from the server was 64,206,061 milliseconds ago. The last packet sent successfully to the server was 64,206,062 milliseconds ago. is longer than the server configured value of 'wait_timeout'. You should consider either expiring and/or testing connection validity before use in your application, increasing the server configured values for client timeouts, or using the Connector/J connection property 'autoReconnect=true' to avoid this problem.
at org.springframework.jdbc.support.SQLExceptionSubclassTranslator.doTranslate(SQLExceptionSubclassTranslator.java:98)
at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72)
at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80)
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:406)
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:455)
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:463)
at com.cable.comcast.neto.nse.taac.dao.VendorDao.getAllVendor(VendorDao.java:25)
at com.cable.comcast.neto.nse.taac.controller.RemoteVendorAccessController.requestAccess(RemoteVendorAccessController.java:78)
I have tried adding the 'autoReconnect=true' to the connection string, but this problem still occurs. Is there another datasource that can be used that will manage the reconnecting for me?
BasicDataSource can manage keeping the connections alive for you. You need to set the following properties :
minEvictableIdleTimeMillis = 120000 // Two minutes
testOnBorrow = true
timeBetweenEvictionRunsMillis = 120000 // Two minutes
minIdle = (some acceptable number of idle connections for your server)
These will configure the data source to keep continually test your connections, and expire and remove them if they become stale. There's a number of other properties on the basic data source that you may want to consider checking into as well to tweak your connection pooling performance. I've run into some strange problems in the past where I was having issues with my database access and it all came down to how the connection pool was configured.
You can try to C3PO:
http://sourceforge.net/projects/c3p0/
<bean id="dataSource" class="com.mchange.v2.c3p0.ComboPooledDataSource"destroy-method="close">
<property name="user" value="${db.username}"/>
<property name="password" value="${db.password}"/>
<property name="driverClass" value="${db.driverClassName}"/>
<property name="jdbcUrl" value="${db.url}"/>
<property name="initialPoolSize" value="0"/>
<property name="maxPoolSize" value="1"/>
<property name="minPoolSize" value="1"/>
<property name="acquireIncrement" value="1"/>
<property name="acquireRetryAttempts" value="0"/>
<property name="idleConnectionTestPeriod" value="600"/> <!--in seconds-->
</bean>
grettings
pacovr

Categories

Resources