Develop Reference

What are the steps to identify connection leak when using DBCP, JDBCTemplate and ScheduledExecutor?

We are creating a Spring boot web app.
DB : JDBC Template and DBCP connection pool.
Java code: A runnable is called in Executors.newSingleThreadScheduledExecutor();
Time interval: 2 min
The code in runnable hits DB using JDBCTemplate.query().
Issue: The heap usage increases to several GBs in few min.
Any Pointers would be helpful to identify the memory leak.
Note: If we comment the JDBCTemplate.query() , memory usage is constant.
Settings of DBCP:
<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
<property name="driverClassName" value="oracle.jdbc.OracleDriver" />
<property name="url" value="${batch.jdbc.url}" />
<property name="username" value="******" />
<property name="password" value="******" />
<property name="connectionProperties" value="defaultRowPrefetch=10000;defaultBatchValue=200;" />
<property name="minIdle" value="10" />
<property name="maxIdle" value="12" />
<property name="maxActive" value="100" />
<property name="accessToUnderlyingConnectionAllowed" value="true" />
<property name="initialSize" value="${batch.jdbc.pool.size}"/>
<property name="validationQuery" value="select 1 from dual" />
<property name="validationQueryTimeout" value="5" />
<property name="timeBetweenEvictionRunsMillis" value="120000" />
<property name="minEvictableIdleTimeMillis" value="60000" />
<property name="testOnBorrow" value="true" />
</bean>
Suspect from Eclipse MAT report
One instance of "org.apache.commons.pool.impl.GenericObjectPool" loaded by "org.springframework.boot.loader.LaunchedURLClassLoader # 0x7fc1d90124c8" occupies 1,421,543,264 (94.69%) bytes. The memory is accumulated in one instance of "org.apache.commons.pool.impl.GenericObjectPool" loaded by "org.springframework.boot.loader.LaunchedURLClassLoader # 0x7fc1d90124c8".

apache-camel throttle does not work as expected

Apache-Camel: 2.12.2, activemq: 5.7
We noticed that in the following route throttling works fine for the first 100 exchanges. After that instead of sending 100 exchanges per second, it sends only 1 exchange per second. Now if we set timePeriodMillis=100 it seems to be working fine. Note that we send 500 exchanges at the same time.
<bean id="myProject" class="org.apache.activemq.camel.component.ActiveMQComponent">
<property name="connectionFactory" ref="myProjectPooledConnectionFactory" />
<property name="preserveMessageQos" value="true" />
<property name="transacted" value="false" />
<property name="maxConcurrentConsumers" value="10" />
</bean>
<bean id="myProjectPooledConnectionFactory" class="org.apache.activemq.pool.PooledConnectionFactory" destroy-method="stop" init-method="start">
<property name="connectionFactory" ref="myProjectAmqConnectionFactory" />
<property name="maxConnections" value="20" />
<property name="idleTimeout" value="0" />
</bean>
<bean id="myProjectAmqConnectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory" >
<property name="brokerURL" value="${myProject.broker.url}" />
<property name="copyMessageOnSend" value="false" />
<property name="useAsyncSend" value="true" />
</bean>
<!-- Local ActiveMQ Configuration -->
<bean id="activemq" class="org.apache.activemq.camel.component.ActiveMQComponent">
<property name="connectionFactory" ref="pooledConnectionFactory"/>
<property name="transacted" value="false"/>
<property name="maxConcurrentConsumers" value="500"/>
</bean>
<bean id="pooledConnectionFactory" class="org.apache.activemq.pool.PooledConnectionFactory" destroy-method="stop" init-method="start">
<property name="connectionFactory" ref="amqConnectionFactory" />
<property name="maxConnections" value="1" />
<property name="idleTimeout" value="0" />
</bean>
<bean id="amqConnectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory" >
<property name="brokerURL" value="${broker.url}" />
<property name="copyMessageOnSend" value="false" />
<property name="useAsyncSend" value="true" />
</bean>
<route id="myProject.outbound.traffic" errorHandlerRef="error.handler.myProject">
<from uri="{{queue.myProject.mint.in}}{{queue.myProject.mint.in.args}}"/>
<throttle timePeriodMillis="1000" >
<constant>100</constant>
<process ref="myProjectProcessor" />
<inOnly uri="myProject:{{queue.myProject}}">
<log logName="myProject.myProjectProcessor" loggingLevel="INFO" message="this is a test message" />
</throttle>
</route>

A colleague of mine found the answer. It is a bug in camel 2.12.2 and it seems to have been solved in 2.12.3. It has to do with the way an exchange is assigned to a slot. Instead of checking if a slot is active it should check if the slot has past first.
protected synchronized TimeSlot nextSlot() {
if (slot == null) {
slot = new TimeSlot();
}
if (slot.isFull() || !slot.isActive()) {
slot = slot.next();
}
slot.assign();
return slot;
}
See here

JMS automatically reconnecting to TOPIC

I have two apps one app is publishing to a topic and another app is reading from that topic. We had a scenario where a queuemanager went down and then was available again. The publisher (without a restart) carries on working fine after the queuemanager restarts, but the topic consumer does not recieve the new messages from the publisher until it is restarted. Is there some configuration that can be setup on the topic consumer to refresh its connection somehow? We are using java / spring / spring integration over IBM MQ. The following is the configuration of our consumer.
<bean id="jmsAlertsServiceMessageListener" class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="alertConnectionFactory"/>
<property name="destination" ref="alertsServiceTopic"/>
<property name="autoStartup" value="false"/>
<property name="clientId" value="${ps.Alert.clientId}"/>
<property name="taskExecutor" ref="jmsTaskExecutor"/>
<property name="subscriptionDurable" value="true"/>
<property name="pubSubDomain" value="true"/>
<property name="messageSelector" value="AlertState = 'RESOLVED'"/>
<property name="messageListener" ref="alertsMessageListener"/>
<property name="durableSubscriptionName" value="replay"/>
<property name="recoveryInterval" value="30000"/>
</bean>
<bean id="alertConnectionFactory" class="com.ibm.mq.jms.MQConnectionFactory" >
<property name="transportType" value="1" />
<property name="queueManager" value="${alert.mq.qm}" />
<property name="hostName" value="${alert.mq.host}" />
<property name="port" value="${alert.mq.port}" />
<property name="channel" value="${alert.mq.channel}" />
<property name="SSLCipherSuite" value="SSL_RSA_WITH_RC4_128_SHA" />
<property name="brokerPubQueue" value="${alert.mq.topic_connection_factory_broker_pub_queue}"/>
<property name="brokerQueueManager" value="${alert.mq.topic_connection_factory_broker_queue_manager}"/>
<property name="providerVersion" value="${alert.mq.topic_connection_factory_provider_version}"/>
<property name="brokerVersion" value="1"/>
<property name="messageSelection" value="1"/>
</bean>
<bean id="alertsServiceTopic" class="com.ibm.mq.jms.MQTopic">
<constructor-arg value="${alert.mq.topic}" />
<property name="brokerDurSubQueue" value="${alert.mq.queue}"/>
<property name="brokerVersion" value="1"/>
</bean>
<bean id="alertsMessageListener" class="org.springframework.integration.jms.ChannelPublishingJmsMessageListener">
<property name="requestChannel" ref="alertsJmsInputChannel"/>
</bean>
<bean id="jmsTaskExecutor" class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
<property name="corePoolSize" value="1"/>
<property name="maxPoolSize" value="1"/>
<property name="waitForTasksToCompleteOnShutdown" value="true"/>
<property name="daemon" value="false"/>
<property name="threadNamePrefix" value="jmsInbound-"/>
<property name="queueCapacity" value="3"/>
<!-- Discard any task that gets rejected. -->
<property name="rejectedExecutionHandler">
<bean class="java.util.concurrent.ThreadPoolExecutor$DiscardPolicy"/>
</property>
</bean>

The DefaultMessageListenerContainer will automatically reconnect according to the recoveryInterval.
Turn on DEBUG (or even TRACE) logging to figure out what's happening.

Use property file's property in spring mvc configuration servlet xml

I use following line of code to read config.properties file in my spring mvc configuration servlet xml file.
<context:property-placeholder location="file:///${config.properties}" />
config.properties contains a property say: propertyName = propertyValue
I want to use this propertyName's value in spring mvc configuration servlet xml file.

Here is an example of configuring as pool with properties taken from a properties file
<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource"
destroy-method="close">
<property name="driverClassName" value="oracle.jdbc.driver.OracleDriver" />
<property name="url" value="jdbc:oracle:thin:#${database.url}" />
<property name="username" value="${database.username}" />
<property name="password" value="${database.password}" />
<property name="validationQuery" value="SELECT 1 FROM DUAL" />
<property name="testWhileIdle" value="true" />
<property name="timeBetweenEvictionRunsMillis" value="300000" />
<property name="numTestsPerEvictionRun" value="6" />
<property name="minEvictableIdleTimeMillis" value="1800000" />
<property name="initialSize" value="3" />
<property name="maxActive" value="75" />
<property name="maxIdle" value="75" />
<property name="maxWait" value="5000" />
<property name="poolPreparedStatements" value="true" />
<property name="maxOpenPreparedStatements" value="100" />
</bean>
The property file contains there rows
database.url=localhost:1521:xe
database.username=dbusername
database.password=dbpassword

I guess you need to add
<context:property-placeholder location="file:///${config.properties}" />
to your servlet.xml file.

How do I add database pooling - Spring3, Hibernate, MySQL

Spring3, Hibernate, MySQL: I am working on my first project, can some one please show me how to change the following code to add database pooling? thanks
applicationContext-security-JDBC.xml
<beans:bean class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<beans:property name="driverClassName" value="${database.driver}" />
<beans:property name="url" value="${database.url}" />
<beans:property name="username" value="${database.user}" />
<beans:property name="password" value="${database.password}" />
</beans:bean>
jdbc.properties
database.driver=com.mysql.jdbc.Driver
database.url=jdbc:mysql://127.0.0.1/db_mytest
database.user=root
database.password=
hibernate.dialect=org.hibernate.dialect.MySQL5Dialect
hibernate.show_sql=true
thanks.... again

I suggest you to use commons-dbcp. You have to download the jar and add it to the WEB-INF/lib directory (if not already included in your application server). This is your new applicationContext.xml with some default parameters that you should change as you wish:
<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource"
destroy-method="close">
<property name="driverClassName" value="${database.driver}" />
<property name="url" value="${database.url}" />
<property name="username" value="${database.username}" />
<property name="password" value="${database.password}" />
<property name="timeBetweenEvictionRunsMillis" value="300000" />
<property name="numTestsPerEvictionRun" value="6" />
<property name="minEvictableIdleTimeMillis" value="1800000" />
<property name="initialSize" value="3" />
<property name="maxActive" value="10" />
<property name="maxIdle" value="10" />
<property name="maxWait" value="5000" />
<property name="poolPreparedStatements" value="true" />
<property name="maxOpenPreparedStatements" value="100" />
</bean>
Hope this helps.

You don't specify what you are running your application on. Many application servers have their own connection pooling implementation that you can tap into. I've used Tomcat's DBCP in the past - was quite easy to set up.