I am working on spring based application where I manage connection pool to manage connection between MySQL and Java application.
Connection pool configuration looks like this:
<bean id="dataSource"
class="org.apache.commons.dbcp.BasicDataSource" >
<property name="driverClassName" value="${database.driver}" />
<property name="url" value="${database.url}" />
<property name="username" value="${database.user}" />
<property name="password" value="${database.password}" />
<property name="initialSize" value="20" />
<property name="maxActive" value="100" />
<property name="maxIdle" value="50" />
<property name="minIdle" value="10" />
</bean>
Now my questions are:
1) I have two different applications having the same configuration, then how many connection pool will mysql maintain, it will one for all application having maxactive1=100 or it will based on per application. In my case 2* 100 = 200(maxactive).
2) what can be best maxactive value if i have 500 requests in 1 sec?
1) MySQL doesn't know about your connection pools. MySQL has own option for max allowed connections. And the number should be bigger than sum of connections required by all the connection pools. So in your case 2*100.
2) best maxactive value depends on multiple factors. 500 requsts per second if one request is processed 1 second requires 500 connections. But if the request processing time is 0.1 sec 50 connections would be enough.
Related
My application is using Spring 2.5.x and deployed on Tomcat server. Some times, I get below error when my db connection is idle:
[TeraJDBC 14.00.00.13] [Error 1095] [SQLState HY000] Cannot call a
method on closed connection
Here is the datasource configuration
<bean id="dataSource" class="org.apache.tomcat.jdbc.pool.DataSource"
destroy-method="close">
<property name="driverClassName" value="com.teradata.jdbc.TeraDriver"/>
<property name="url" >
<util:constant static-field="_DB_HOST"/>
</property>
<property name="username">
<util:constant static-field="_DB_USER"/>
</property>
<property name="password">
<util:constant static-field="_DB_PWD"/>
</property>
<property name="initialSize" value="1" />
<property name="maxActive" value="50" />
</bean>
Is there any configuration I'm missing here?
While all connections used by Spring's JdbcTemplate are closed at the end of each transaction, Tomcat's JDBC Connection Pool never actually returns the real Connection's obtained by the driver. DataSource#getConnection always returns a proxy, such that Connection#close returns the connection to the pool instead of physically closing it.
Therefore, as explained in this answer, the connections are probably closed by the server. Therefore you need to configure the pool to validate the connections, as in the answer cited by Kayaman.
I suspect that your problem is not caused by connection issues, but server policy, so I would set up:
<property name="validationQuery" value="SELECT 66353343" />
<property name="testWhileIdle" value="true" />
<property name="timeBetweenEvictionRunsMillis" value="60000" />
in order to check every 60 seconds if the physical connections are up.
I have legacy web application. Witch uses jdbcTemplate to execute stored procedures from DB (Oracle), procedures located at packages. Problem is, when database developers compiles that packages im getting Expceptions sometihng like existing state of packages been changed (ora 4061). and they does that 1 or 2 times per week. Then i have to restart application to restart pool. app uses OracleDataSource. i can't change anything at that db but can change pool manager. so the question is. is it possible to close that connection? not just return to pool. and get new connection(from pool) if that Exception apears? thanks !!
people sugested to re-execute that stored procedure when that error apears. It doest worked.
<bean id="ds_name" class="oracle.jdbc.pool.OracleDataSource"
destroy-method="close">
<property name="connectionCachingEnabled" value="true" />
<property name="URL" value="****" />
<property name="user" value="user" />
<property name="password" value="pass" />
<property name="connectionCacheProperties">
<value>
MinLimit:1
MaxLimit:100
InitialLimit:1
ConnectionWaitTimeout:120
InactivityTimeout:180
ValidateConnection:true
</value>
</property>
</bean>
<bean id="jdbcTemplate" class="org.springframework.jdbc.core.JdbcTemplate">
<property name="dataSource" ref="ds_name" />
</bean>
I'm experiencing a weird behaviour with our Java web application, configured to access a PostgreSQL database through C3p0 ComboPooledDataSource.
First of all, PostgreSQL server installation has default parameters, we didn't need to change any variable in it..
We run our queries using Spring JdbcTemplate, version 3.2.12.RELEASE. As it is for PostgreSQL, it is configured with default parameters.
Here it is our context configuration with C3p0 and Spring;
<bean id="resoilDataSource" class="com.mchange.v2.c3p0.ComboPooledDataSource"
destroy-method="close">
<!-- access configuration -->
<property name="driverClass" value="org.postgresql.Driver" />
<property name="jdbcUrl" value="jdbc:postgresql://localhost:5432/*****" />
<property name="user" value="******" />
<property name="password" value="******" />
<!-- pool sizing -->
<property name="initialPoolSize" value="1" />
<property name="minPoolSize" value="1" />
<property name="maxPoolSize" value="6" />
<property name="acquireIncrement" value="3" />
<property name="maxStatements" value="150" />
<!-- refreshing connections -->
<property name="maxIdleTime" value="180" /> <!-- 3min -->
<property name="maxIdleTimeExcessConnections" value="120" /> <!-- 3min -->
<!-- timeouts e testing -->
<property name="idleConnectionTestPeriod" value="120" /> <!-- 60 -->
<property name="testConnectionOnCheckout" value="true" />
<property name="testConnectionOnCheckin" value="false" />
<property name="preferredTestQuery" value="SELECT 1" />
</bean>
<bean id="template" class="org.springframework.jdbc.core.JdbcTemplate">
<property name="dataSource" ref="resoilDataSource"></property>
</bean>
Let me describe the problem we are encountering:
once we reach "maxPoolSize" of idle connections opened on Postgres side, this connections never expires, they remain in "idle" state and there's no way C3p0 will get any of them back for pooling..
I would expect that once one of these opened connections excess idle time, C3p0 is able to reuse it due to the "maxIdleTimeExcessConnections" parameter.
Unfortunately, this never happens.
I also tried to substitute C3p0 ComboPooledDataSource with Apache DBCP BasicDataSource, but nothing changed.
Before using PostgreSQL database for our application, our customers asked us to install our application integrating other popular databases instead of PostgreSQL (SQL Server and Oracle in particular) and we never experienced this behaviour.
Any ideas about what's going on it's truly appreciated, thanks in advance.
I'm using the following bean to connect to the DB through JDBC:
<bean id="databaseds" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
<property name="driverClassName" value="${jdbc.driverClassName}"/>
<property name="url" value="${jdbc.url}"/>
<property name="removeAbandoned" value="true"/>
<property name="initialSize" value="1" />
<property name="maxActive" value="2" />
<property name="maxIdle" value="0" />
<property name="minEvictableIdleTimeMillis" value="120000" />
</bean>
Using quartz a new class is constructed which takes advantage of Spring JDBC to run SQL statements. What I did notice is that most of the the JDBC connections are still hanging around hours after the quartz job ran and completed. Based on the bean I'm expecting to have NO idle sessions 2-3 minutes after the connection is not being used.
Am I missing anything? I don't want to keep more than 1-2 connections open to the database, the program is not doing any DB intensive work but does run every 30 minutes and if the connections are not reused or closed they pile up.
I'm fighting with configuring Tomcat JDBC Connection Pool to achieve reliability. Current issue is that in test environment I have such scanerio in webapp:
day 1: everything works fine
day 2: webapp cannot comunicate with MySQL for several hours, lot of "Broken pipe" in logs
day 3: suprisingly, everything works fine again (without ingerention or restart)
I have configured validationInterval, validationQuery, validationTimeout. This is my data source config:
<bean id="dataSource" class="org.apache.tomcat.jdbc.pool.DataSource"
destroy-method="close">
<property name="driverClassName" value="com.mysql.jdbc.Driver" />
<property name="username" value="${dbUser}" />
<property name="password" value="${dbPass}" />
<property name="url" value="${dbUrl}" />
<property name="defaultAutoCommit" value="false" />
<property name="defaultTransactionIsolation">
<util:constant static-field="java.sql.Connection.TRANSACTION_SERIALIZABLE" />
</property>
<property name="maxActive" value="300" />
<property name="maxIdle" value="25" />
<property name="initialSize" value="5" />
<property name="validationInterval" value="5000" />
<property name="validationQuery" value="SELECT 1"/>
<property name="validationQueryTimeout" value="3" />
<property name="minIdle" value="5" />
<property name="initSQL" value="SET time_zone = '+00:00';" />
</bean>
I don't have autoReconnect=true parameter in connection URL, only UTF8 encoding.
The exact error is:
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException:
The last packet successfully received from the server was 38,700,615
milliseconds ago. The last packet sent successfully to the server was
38,700,615 milliseconds ago. is longer than the server configured
value of 'wait_timeout'. You should consider either expiring and/or
testing connection validity before use in your application, increasing
the server configured values for client timeouts, or using the
Connector/J connection property 'autoReconnect=true' to avoid this
problem.
Caused by: java.net.SocketException: Broken pipe
We had some similar problems with one of our applications and after a lot of digging we added the following properties that solved all our connection problems:
maxAge="180000"
testOnBorrow="true"
testWhileIdle="true"
validationInterval="0" //forces the connection pool to validate each time a connection is given to the application
You need to set 'testOnBorrow' to 'true', and probably 'maxAge' to less than the server's configured 'wait_timeout', as hinted in the message.