Develop Reference

Tomcat database connection pool slow in high throughput

I have soap webservice written in Spring 2.X and which connect to Teradata and return the result to client. To connect the data based I am using Tomcat JDBC Connection Pool as the DataSource.
In peak hour (9AM to 6PM) application get about 60k transactions requests. I observed some of the transactions goes in hung state and return response in 2-3 minutes . I suspect some transaction goes in wait status and once connection is available in pool then complete the transactions.
Below is the configuration for the DataSource.
<bean id="dataSource" class="org.apache.tomcat.jdbc.pool.DataSource"
destroy-method="close">
<property name="driverClassName" value="com.teradata.jdbc.TeraDriver"/>
<property name="url" >
<util:constant static-field=" _DB_HOST"/>
</property>
<property name="username">
<util:constant static-field=" DB_USER"/>
</property>
<property name="password">
<util:constant static-field=" DB_PWD"/>
</property>
<property name="initialSize" value="1" />
<property name="maxActive" value="50" />
<property name="minIdle" value="0" />
<property name="maxWait" value="-1" />
<property name="minEvictableIdleTimeMillis" value="1000" />
<property name="timeBetweenEvictionRunsMillis" value="1000" />
</bean>
<bean id="transactionManager" class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
<property name="dataSource" ref="dataSource"/>
<qualifier value="txnMngr"/>
</bean>
// using JdbcTemplate to read the data from data base.
Here are my questions:
Is there any issue with above configuration based on the load which I mentioned for my application?
Is there any way I can monitor the DB connection pool uses?

What are the steps to identify connection leak when using DBCP, JDBCTemplate and ScheduledExecutor?

We are creating a Spring boot web app.
DB : JDBC Template and DBCP connection pool.
Java code: A runnable is called in Executors.newSingleThreadScheduledExecutor();
Time interval: 2 min
The code in runnable hits DB using JDBCTemplate.query().
Issue: The heap usage increases to several GBs in few min.
Any Pointers would be helpful to identify the memory leak.
Note: If we comment the JDBCTemplate.query() , memory usage is constant.
Settings of DBCP:
<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
<property name="driverClassName" value="oracle.jdbc.OracleDriver" />
<property name="url" value="${batch.jdbc.url}" />
<property name="username" value="******" />
<property name="password" value="******" />
<property name="connectionProperties" value="defaultRowPrefetch=10000;defaultBatchValue=200;" />
<property name="minIdle" value="10" />
<property name="maxIdle" value="12" />
<property name="maxActive" value="100" />
<property name="accessToUnderlyingConnectionAllowed" value="true" />
<property name="initialSize" value="${batch.jdbc.pool.size}"/>
<property name="validationQuery" value="select 1 from dual" />
<property name="validationQueryTimeout" value="5" />
<property name="timeBetweenEvictionRunsMillis" value="120000" />
<property name="minEvictableIdleTimeMillis" value="60000" />
<property name="testOnBorrow" value="true" />
</bean>
Suspect from Eclipse MAT report
One instance of "org.apache.commons.pool.impl.GenericObjectPool" loaded by "org.springframework.boot.loader.LaunchedURLClassLoader # 0x7fc1d90124c8" occupies 1,421,543,264 (94.69%) bytes. The memory is accumulated in one instance of "org.apache.commons.pool.impl.GenericObjectPool" loaded by "org.springframework.boot.loader.LaunchedURLClassLoader # 0x7fc1d90124c8".

using two versions of DB2 drivers at the same time?

I need to use 2 version of DB2Driver for my application. I am using spring to define dataSources. The path of two Libs are the same, the first dataSource needs driver version2 and second one needs version3. Is there any way to solve this problem?
<bean id="DataSource2" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
<property name="driverClassName" value="com.ibm.db2.jcc.DB2Driver"/>
<property name="url" value="yyy"/>
<property name="username" value="yyy"/>
<property name="password" value="yyy"/>
</bean>
<bean id="DataSource2" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
<property name="driverClassName" value="com.ibm.db2.jcc.DB2Driver"/>
<property name="url" value="xxx"/>
<property name="username" value="xxx"/>
<property name="password" value="xxx"/>
<property name="maxActive" value="100"/>
<property name="maxIdle" value="10"/>
<property name="minIdle" value="1"/>
<property name="maxWait" value="3000"/>
<property name="testOnBorrow" value="true"/>
<property name="testOnReturn" value="false"/>
<property name="testWhileIdle" value="true"/>
<property name="timeBetweenEvictionRunsMillis" value="10000"/>
<property name="numTestsPerEvictionRun" value="50"/>
<property name="minEvictableIdleTimeMillis" value="10000"/>
<property name="poolPreparedStatements" value="true"/>
</bean>

Searchpath with Postgres does not work with Spring/DBCP datasource

Postgres extensions are installed in public schema.
Set the search path for the app-specific schema on the DBCP datasource the following way:
<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close" primary="true">
<property name="driverClassName" value="org.postgresql.Driver"/>
<property name="url" value="jdbc:postgresql://myhost:myport/${db.gos_app.database}?searchpath=mySchema,public;?ApplicationName=${app.name}"/>
<property name="connectionProperties" value="currentSchema=mySchema;"/>
<property name="username" value="user"/>
<property name="password" value="pw"/>
<property name="defaultAutoCommit" value="false"/>
<property name="maxActive" value="6" />
</bean>
But somehow I cannot use extensions installed in this public schema without qualifying them like "public.hstore".

Found the solution - JDBC driver does not know the searchpath property in the URL. But this has been nowhere reported :-(
Instead currentSchema is given to driver which is then mapped to native Postgres searchpath (thus probably overwriting the default one there with public included). Not intuitive!
So the solution looks like this:
<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close" primary="true">
<property name="driverClassName" value="org.postgresql.Driver"/>
<property name="url" value="jdbc:postgresql://myhost:myport/${db.gos_app.database}?ApplicationName=${app.name}"/>
<property name="connectionProperties" value="currentSchema=mySchema,public;"/>
<property name="username" value="user"/>
<property name="password" value="pw"/>
<property name="defaultAutoCommit" value="false"/>
<property name="maxActive" value="6" />

JMS automatically reconnecting to TOPIC

I have two apps one app is publishing to a topic and another app is reading from that topic. We had a scenario where a queuemanager went down and then was available again. The publisher (without a restart) carries on working fine after the queuemanager restarts, but the topic consumer does not recieve the new messages from the publisher until it is restarted. Is there some configuration that can be setup on the topic consumer to refresh its connection somehow? We are using java / spring / spring integration over IBM MQ. The following is the configuration of our consumer.
<bean id="jmsAlertsServiceMessageListener" class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="alertConnectionFactory"/>
<property name="destination" ref="alertsServiceTopic"/>
<property name="autoStartup" value="false"/>
<property name="clientId" value="${ps.Alert.clientId}"/>
<property name="taskExecutor" ref="jmsTaskExecutor"/>
<property name="subscriptionDurable" value="true"/>
<property name="pubSubDomain" value="true"/>
<property name="messageSelector" value="AlertState = 'RESOLVED'"/>
<property name="messageListener" ref="alertsMessageListener"/>
<property name="durableSubscriptionName" value="replay"/>
<property name="recoveryInterval" value="30000"/>
</bean>
<bean id="alertConnectionFactory" class="com.ibm.mq.jms.MQConnectionFactory" >
<property name="transportType" value="1" />
<property name="queueManager" value="${alert.mq.qm}" />
<property name="hostName" value="${alert.mq.host}" />
<property name="port" value="${alert.mq.port}" />
<property name="channel" value="${alert.mq.channel}" />
<property name="SSLCipherSuite" value="SSL_RSA_WITH_RC4_128_SHA" />
<property name="brokerPubQueue" value="${alert.mq.topic_connection_factory_broker_pub_queue}"/>
<property name="brokerQueueManager" value="${alert.mq.topic_connection_factory_broker_queue_manager}"/>
<property name="providerVersion" value="${alert.mq.topic_connection_factory_provider_version}"/>
<property name="brokerVersion" value="1"/>
<property name="messageSelection" value="1"/>
</bean>
<bean id="alertsServiceTopic" class="com.ibm.mq.jms.MQTopic">
<constructor-arg value="${alert.mq.topic}" />
<property name="brokerDurSubQueue" value="${alert.mq.queue}"/>
<property name="brokerVersion" value="1"/>
</bean>
<bean id="alertsMessageListener" class="org.springframework.integration.jms.ChannelPublishingJmsMessageListener">
<property name="requestChannel" ref="alertsJmsInputChannel"/>
</bean>
<bean id="jmsTaskExecutor" class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
<property name="corePoolSize" value="1"/>
<property name="maxPoolSize" value="1"/>
<property name="waitForTasksToCompleteOnShutdown" value="true"/>
<property name="daemon" value="false"/>
<property name="threadNamePrefix" value="jmsInbound-"/>
<property name="queueCapacity" value="3"/>
<!-- Discard any task that gets rejected. -->
<property name="rejectedExecutionHandler">
<bean class="java.util.concurrent.ThreadPoolExecutor$DiscardPolicy"/>
</property>
</bean>

The DefaultMessageListenerContainer will automatically reconnect according to the recoveryInterval.
Turn on DEBUG (or even TRACE) logging to figure out what's happening.