Develop Reference

Tomcat database connection pool slow in high throughput

I have soap webservice written in Spring 2.X and which connect to Teradata and return the result to client. To connect the data based I am using Tomcat JDBC Connection Pool as the DataSource.
In peak hour (9AM to 6PM) application get about 60k transactions requests. I observed some of the transactions goes in hung state and return response in 2-3 minutes . I suspect some transaction goes in wait status and once connection is available in pool then complete the transactions.
Below is the configuration for the DataSource.
<bean id="dataSource" class="org.apache.tomcat.jdbc.pool.DataSource"
destroy-method="close">
<property name="driverClassName" value="com.teradata.jdbc.TeraDriver"/>
<property name="url" >
<util:constant static-field=" _DB_HOST"/>
</property>
<property name="username">
<util:constant static-field=" DB_USER"/>
</property>
<property name="password">
<util:constant static-field=" DB_PWD"/>
</property>
<property name="initialSize" value="1" />
<property name="maxActive" value="50" />
<property name="minIdle" value="0" />
<property name="maxWait" value="-1" />
<property name="minEvictableIdleTimeMillis" value="1000" />
<property name="timeBetweenEvictionRunsMillis" value="1000" />
</bean>
<bean id="transactionManager" class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
<property name="dataSource" ref="dataSource"/>
<qualifier value="txnMngr"/>
</bean>
// using JdbcTemplate to read the data from data base.
Here are my questions:
Is there any issue with above configuration based on the load which I mentioned for my application?
Is there any way I can monitor the DB connection pool uses?

C3p0 datasource with PostgreSQL

I'm experiencing a weird behaviour with our Java web application, configured to access a PostgreSQL database through C3p0 ComboPooledDataSource.
First of all, PostgreSQL server installation has default parameters, we didn't need to change any variable in it..
We run our queries using Spring JdbcTemplate, version 3.2.12.RELEASE. As it is for PostgreSQL, it is configured with default parameters.
Here it is our context configuration with C3p0 and Spring;
<bean id="resoilDataSource" class="com.mchange.v2.c3p0.ComboPooledDataSource"
destroy-method="close">
<!-- access configuration -->
<property name="driverClass" value="org.postgresql.Driver" />
<property name="jdbcUrl" value="jdbc:postgresql://localhost:5432/*****" />
<property name="user" value="******" />
<property name="password" value="******" />
<!-- pool sizing -->
<property name="initialPoolSize" value="1" />
<property name="minPoolSize" value="1" />
<property name="maxPoolSize" value="6" />
<property name="acquireIncrement" value="3" />
<property name="maxStatements" value="150" />
<!-- refreshing connections -->
<property name="maxIdleTime" value="180" /> <!-- 3min -->
<property name="maxIdleTimeExcessConnections" value="120" /> <!-- 3min -->
<!-- timeouts e testing -->
<property name="idleConnectionTestPeriod" value="120" /> <!-- 60 -->
<property name="testConnectionOnCheckout" value="true" />
<property name="testConnectionOnCheckin" value="false" />
<property name="preferredTestQuery" value="SELECT 1" />
</bean>
<bean id="template" class="org.springframework.jdbc.core.JdbcTemplate">
<property name="dataSource" ref="resoilDataSource"></property>
</bean>
Let me describe the problem we are encountering:
once we reach "maxPoolSize" of idle connections opened on Postgres side, this connections never expires, they remain in "idle" state and there's no way C3p0 will get any of them back for pooling..
I would expect that once one of these opened connections excess idle time, C3p0 is able to reuse it due to the "maxIdleTimeExcessConnections" parameter.
Unfortunately, this never happens.
I also tried to substitute C3p0 ComboPooledDataSource with Apache DBCP BasicDataSource, but nothing changed.
Before using PostgreSQL database for our application, our customers asked us to install our application integrating other popular databases instead of PostgreSQL (SQL Server and Oracle in particular) and we never experienced this behaviour.
Any ideas about what's going on it's truly appreciated, thanks in advance.

Configuring postgresql driver through Spring xml datasource

I've been trying to configure the connections made with a postgresql datasource declared in a xml Spring configuration file.
<bean id="myDataSource"
class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="driverClassName" value="org.postgresql.Driver" />
<property name="url" value="jdbc:postgresql://localhost:5432/dbname" />
<property name="username" value="postgres" />
<property name="password" value="" />
<property name="socketTimeout" value="10"/>
</bean>
I know, I shouldn't be using the DriverManagerDataSource class from spring, (we will soon move to C3p0 or DBCP) because it's not a real pooling.
I'm trying to set the socketTimeout value of a postgresql connection ( described here https://jdbc.postgresql.org/documentation/head/connect.html ) but of course, "socketTimeout" is not a property of the datasource, so it doesn't work.
Is it possible to do this through the datasource xml's configuration ? Or should I do it somewhere else ? Because the data source manages the connection I don't think I'll be able to do a
props.setProperty("timeout",30);
Connection conn = DriverManager.getConnection(url, props);
Can I even do this with the DriverManagerDataSource ? I tried to search, but I didn't find anything usefull, as not a lot of people are really using it.

Thank you M. Deinum, I was able to find how.
Actually, even knowing the property was named "connectionProperties", I didn't found a lot answers (maybe people rarely use it this way ?). So I'm posting it:
<bean id="myDataSource"
class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="driverClassName" value="org.postgresql.Driver" />
<property name="url" value="jdbc:postgresql://localhost:5432/dbname" />
<property name="username" value="postgres" />
<property name="password" value="" />
<!--<property name="socketTimeout" value="10"/>-->
<property name="connectionProperties">
<props>
<prop key="socketTimeout">10</prop>
</props>
</property>
</bean>
If anyone has a better/more complete answer, I'll check it out ;)

Spring JDBC connections left open

I'm using the following bean to connect to the DB through JDBC:
<bean id="databaseds" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
<property name="driverClassName" value="${jdbc.driverClassName}"/>
<property name="url" value="${jdbc.url}"/>
<property name="removeAbandoned" value="true"/>
<property name="initialSize" value="1" />
<property name="maxActive" value="2" />
<property name="maxIdle" value="0" />
<property name="minEvictableIdleTimeMillis" value="120000" />
</bean>
Using quartz a new class is constructed which takes advantage of Spring JDBC to run SQL statements. What I did notice is that most of the the JDBC connections are still hanging around hours after the quartz job ran and completed. Based on the bean I'm expecting to have NO idle sessions 2-3 minutes after the connection is not being used.
Am I missing anything? I don't want to keep more than 1-2 connections open to the database, the program is not doing any DB intensive work but does run every 30 minutes and if the connections are not reused or closed they pile up.

How to configure MySQL connection properties with Spring, Hibernate 3.3 and c3p0?

I am currently in the process of upgrading an application from Hibernate 3.2 to Hibernate 3.3. I though I'd stick with the default connection pool (Hibernate changed its default from Commons DBCP to c3p0) as I don't have any good reason to choose a non-default pool. At least non but having used DBCP before.
The upgrade went pretty much without any problems so far. The only thing I can't get to work is passing properties to the underlying MySQL JDBC4Connection. Up to now, I used DBCP's BasicDataSource.addConnectionProperty(String,String) to pass properties (useUnicode=true, characterEncodin=UTF-8, characterSetResults=UTF-8, zeroDateTimeBehavior=convertToNull).
However, I can't find any way to do the same with c3p0 other than including them in the JDBC URL. (That's something I'd like to avoid as I wanna keep the URL configurable without forcing users to include those parameters.)
So far, I've tried to use a ConnectionCustomizer without success. Any other suggestions?

Once again a question I answer myself (another self-learner? yes, please!):
com.mchange.v2.c3p0.ComboPooledDataSource has a property "properties". Interestingly, setting properties after user and password overrides them. But setting properties before user and password works as expected.

Follow up for the self answer.
An example of a spring way of configuring this:
The Data source bean:
<bean id="dataSource" class="com.mchange.v2.c3p0.ComboPooledDataSource" destroy-method="close">
<property name="properties" ref="mysqlConnectionProperties"></property>
<property name="driverClass" value="${jdbc.driver}" />
<property name="jdbcUrl" value="${jdbc.url}" />
<property name="user" value="${jdbc.username}" />
<property name="password" value="${jdbc.password}" />
<!-- c3p0 combo pooled data source settings -->
<property name="initialPoolSize" value="3" />
<property name="minPoolSize" value="3" />
<property name="maxPoolSize" value="50" />
<property name="maxIdleTime" value="7200" />
<property name="maxStatements" value="200" />
<property name="idleConnectionTestPeriod" value="270" />
<property name="preferredTestQuery">
<value>SELECT 1</value>
</property>
</bean>
The properties bean:
<bean id="mysqlConnectionProperties" class="java.util.Properties">
<constructor-arg>
<props>
<prop key="useTimezone">true</prop>
<prop key="serverTimezone">America/Chicago</prop>
<!-- add any other properties you have -->
</props>
</constructor-arg>
</bean>