I'm using hibernate to try and retrieve a cached query.
#Transactional
public interface ProductDAO extends JpaRepository<Product, Long> {
#QueryHints({ #QueryHint(name = "org.hibernate.cacheable", value = "true") })
Product findByCode(String code);
}
I'm load testing and I'm doing this in a large loop of 1000 iterations.
for (int i = 0; i < 500; i++) {
URL myURL = new URL("http://localhost:8080/test");
URLConnection myURLConnection = myURL.openConnection();
myURLConnection.connect();
myURLConnection.getContent();
}
I've checked with showsql and I can see only 1 SQL statement is generated for my first hit to the DB after which it is cached.
Yet I still get the following error even though no SQL is being shown:
com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Too many connections
My Hibernate properties:
#hibernate properties
hibernate.dialect = ${hibernate.dialect}
hibernate.show_sql = false
hibernate.hbm2ddl.auto = ${hibernate.hbm2ddl}
hibernate.c3p0.min_size = 10
hibernate.c3p0.max_size = 100
hibernate.c3p0.timeout = 300
hibernate.c3p0.max_statements = 50
hibernate.c3p0.acquire_increment = 5
hibernate.c3p0.idle_test_period = 3000
hibernate.cache.use_second_level_cache=true
hibernate.cache.region.factory_class=org.hibernate.cache.ehcache.SingletonEhCacheRegionFactory
hibernate.cache.use_query_cache=true
Database config:
<bean name="dataSource" class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="url" value="${db.url}" />
<property name="driverClassName" value="${db.driverClassName}" />
<property name="username" value="${db.username}" />
<property name="password" value="${db.password}" />
</bean>
<bean class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean" id="entityManagerFactory">
<property name="dataSource" ref="dataSource" />
<property name="jpaVendorAdapter">
<bean class="org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter">
<property name="showSql" value="false" />
<property name="generateDdl" value="true" />
<property name="databasePlatform" value="${hibernate.dialect}" />
</bean>
</property>
<property name="jpaProperties" ref="hibernateProperties" />
<property name="packagesToScan">
<array>
<value>com.exammple.model</value>
</array>
</property>
</bean>
<bean id="hibernateProperties" class="org.springframework.beans.factory.config.PropertiesFactoryBean">
<property name="location" value="classpath:/spring/hibernate.properties" />
</bean>
<bean id="sessionFactory" factory-bean="entityManagerFactory" factory-method="getSessionFactory" />
<bean id="transactionManager" class="org.springframework.orm.jpa.JpaTransactionManager">
<property name="dataSource" ref="dataSource" />
<property name="entityManagerFactory" ref="entityManagerFactory" />
</bean>
<tx:annotation-driven transaction-manager="transactionManager" />
<jpa:repositories base-package="com.example.dal" entity-manager-factory-ref="entityManagerFactory"
transaction-manager-ref="transactionManager" repository-impl-postfix="CustomImpl" />
The problem is your configuration you aren't using a connection pool.
You are configuring a DriverManagerDataSource which isn't a proper connection pool. You are injecting this bean into the LocalContainerEntityManagerFactoryBean which renders your hibernate.connection and hibernate.c3p0 properties useless, they aren't used.
Solution is quite easy drop the hibernate.c3p0 and hibernate.connection properties and replace the DriverManagerDataSource with a proper pool implementation. I would recommend HikariCP over C3P0 but that is personal preference.
<bean id="hikariConfig" class="com.zaxxer.hikari.HikariConfig">
<property name="poolName" value="springHikariCP" />
<property name="connectionTestQuery" value="SELECT 1" />
<property name="dataSourceClassName" value="${db.driverClassName}" />
<property name="dataSourceProperties">
<props>
<prop key="url">${db.url}</prop>
<prop key="user">${db.username}</prop>
<prop key="password">${jdb.password}</prop>
</props>
</property>
</bean>
<bean id="dataSource" class="com.zaxxer.hikari.HikariDataSource" destroy-method="close">
<constructor-arg ref="hikariConfig" />
</bean>
This problem may be have multiple issues:
You don't close your database connections.
You set a maximum connection pool pool size that exceeds the maximum connections allowed of your database server. When the number of clients exceeds the number of maximum allowed connections, you'd get an exception like this.
I could use FlexyPool to monitor the connection pool usage and find out if connections are leaking or if they are leased for long times.
Update
As M. Deinum already said, you are not using Connection Pooling.
You can still use the Hibernate C3P0 properties, but you have to remove the dataSource from the LocalContainerEntityManagerFactoryBean:
<property name="dataSource" ref="dataSource" />
This way Hibernate can use the hibernate.c3p0 properties.
Related
I have soap webservice written in Spring 2.X and which connect to Teradata and return the result to client. To connect the data based I am using Tomcat JDBC Connection Pool as the DataSource.
In peak hour (9AM to 6PM) application get about 60k transactions requests. I observed some of the transactions goes in hung state and return response in 2-3 minutes . I suspect some transaction goes in wait status and once connection is available in pool then complete the transactions.
Below is the configuration for the DataSource.
<bean id="dataSource" class="org.apache.tomcat.jdbc.pool.DataSource"
destroy-method="close">
<property name="driverClassName" value="com.teradata.jdbc.TeraDriver"/>
<property name="url" >
<util:constant static-field=" _DB_HOST"/>
</property>
<property name="username">
<util:constant static-field=" DB_USER"/>
</property>
<property name="password">
<util:constant static-field=" DB_PWD"/>
</property>
<property name="initialSize" value="1" />
<property name="maxActive" value="50" />
<property name="minIdle" value="0" />
<property name="maxWait" value="-1" />
<property name="minEvictableIdleTimeMillis" value="1000" />
<property name="timeBetweenEvictionRunsMillis" value="1000" />
</bean>
<bean id="transactionManager" class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
<property name="dataSource" ref="dataSource"/>
<qualifier value="txnMngr"/>
</bean>
// using JdbcTemplate to read the data from data base.
Here are my questions:
Is there any issue with above configuration based on the load which I mentioned for my application?
Is there any way I can monitor the DB connection pool uses?
I have a Java Batch Program which is triggered from a BPEL process used for inserting set of Record into DB.
Size of record might vary but at an average around 20,000 to 40,000.
My custom java batch picks up the data from an excel uploaded from a third party app into server location.
For performing the DML operation I have gone with Hibernate/Spring JDBC framework.
Refer to my spring configuration file as below::
<context:component-scan base-package="mybasepackage" />
<bean id="myProps"
class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="locations">
<list>
<value>classpath*:myDB.properties</value>
</list>
</property>
<property name="ignoreUnresolvablePlaceholders" value="true" />
</bean>
<bean id="dataSource"
class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="driverClassName" value="${DB_DRIVER}" />
<property name="url" value="${DB_URL}" />
<property name="username" value="${DB_USER}" />
<property name="password" value="${DB_PASSWORD}" />
</bean>
<bean id="jdbcTemplate" class="org.springframework.jdbc.core.JdbcTemplate">
<property name="dataSource" ref="dataSource" />
</bean>
<bean id="sessionFactory"
class="org.springframework.orm.hibernate5.LocalSessionFactoryBean">
<property name="dataSource" ref="dataSource" />
<property name="packagesToScan" value="mypackage" />
<property name="hibernateProperties">
<props>
<prop key="hibernate.dialect">org.hibernate.dialect.Oracle12cDialect</prop>
<prop key="hibernate.show_sql">true</prop>
<prop key="hibernate.jdbc.batch_size">20</prop>
<prop key="hibernate.c3p0.min_size">5</prop>
<prop key="hibernate.c3p0.max_size">10</prop>
<prop key="hibernate.c3p0.max_size">7000</prop>
</props>
</property>
</bean>
<bean id="myBatchDao"
class="mypackage.MyBatchDaoImpl">
<property name="jdbcTemplate" ref="jdbcTemplate" />
<property name="sessionFactory" ref="sessionFactory" />
</bean>
<bean id="myBatchService"
class="mypackage.MyBatchServiceImpl">
<property name="myBatchDao" ref="myBatchDao" />
</bean>
<bean id="transactionManager"
class="org.springframework.orm.hibernate5.HibernateTransactionManager">
<property name="sessionFactory" ref="sessionFactory" />
</bean>
<tx:annotation-driven transaction-manager="transactionManager" />
I am already using c3p0 for ConnectionPooling and also visited the following links:
link1
link2
But still I am getting following exception::
<org.hibernate.engine.jdbc.spi.SqlExceptionHelper> <BEA-000000> <IO Error: Got minus one from a read call, connect lapse 6370 ms., Authentication lapse 0 ms.>
org.springframework.transaction.CannotCreateTransactionException: Could not open Hibernate Session for transaction;nested exception is org.hibernate.exception.JDBCConnectionException: Unable to acquire JDBC Connection
at org.springframework.orm.hibernate5.HibernateTransactionManager.doBegin(HibernateTransactionManager.java:564)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.getTransaction(AbstractPlatformTransactionManager.java:371)at org.springframework.transaction.interceptor.TransactionAspectSupport.createTransactionIfNecessary(TransactionAspectSupport.java:335)
I'm working on spring boot application, which already has a database connection established in its applicationContext.xml file and the necessary transaction manager and vendors etc.
I now need to connect the app to a second database. But I'm having issues with this. In my unit tests the connection is fine and can make simple queries to retrieve data, which is all I need it to do. However when I compile the app into a jar and run it, I get the following error
NoUniqueBeanDefinitionException: No qualifying bean of type "org.springframework.transaction.PlatformTransactionManager" available: expected single matching bean but found 2: transactionManager, transactionManager2
I have spent ages looking up how to solve this, and the suggested fixes I have found here , here and here have not worked.
I have one persistence.xml with two persistence units defined. And in my applicaitonContext.xml I defined two datasources, two transaction managers and two entity Manager Factories. I then use the #persitsencecontext and #Transactional("") annotations to say which persistence unit and managers to use, but I still get an error. I also added in the <qualifier> tag to the app context file, as I saw this as a suggested fix with the #transactional annotation, still no luck.
My code is below, can anyone spot an errors I have made, and why it may not be working as expected
applicationContext.xml
<bean id="dataSource1" class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="driverClassName" value="org.hsqldb.jdbcDriver"/>
<property name="url" value="..."/>
<property name="username" value="..."/>
<property name="password" value="..."/>
</bean>
<bean id="entityManagerFactory" name="proxy">
<property name="persistenceUnitName" value="proxy" />
<property name="persistenceUnitXmlLocation" value="classpath:META-INF/persistence.xml" />
<property name="dataSource" ref="dataSource1" />
<property name="jpaVendorAdapter" ref="hiberanteVendorAdapter" />
<property name="jpaProperties">
<props>
<prop key="hiberante.hbm2ddl.auto">valudate</prop>
</props>
</property>
</bean>
<bean id="hibernateVendorAdapter" class="org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter">
<property name="databasePlatform" value="org.hibernate.dialect.HSQLDialect" />
<property name="database" value="HSQL" />
<property name="showSql" value="true" />
</bean>
<bean id="transactionManager" class="org.springframework.orm.jpa.JpaTransactionManager">
<property name="entityManagerFactory" ref="entityManagerFactory" />
<qualifier value="transactionManager1" />
</bean>
<!-- Second datasource -->
<bean id="dataSource2" class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="driverClassName" value="org.hsqldb.jdbcDriver"/>
<property name="url" value="..."/>
<property name="username" value="..."/>
<property name="password" value="..."/>
</bean>
<bean id="entityManagerFactory2" name="proxy">
<property name="persistenceUnitName" value="proxy2" />
<property name="persistenceUnitXmlLocation" value="classpath:META-INF/persistence.xml" />
<property name="dataSource" ref="dataSource2" />
<property name="jpaVendorAdapter" ref="hiberanteVendorAdapter2" />
<property name="jpaProperties">
<props>
<prop key="hiberante.hbm2ddl.auto">valudate</prop>
</props>
</property>
</bean>
<bean id="transactionManager2" class="org.springframework.orm.jpa.JpaTransactionManager">
<property name="entityManagerFactory" ref="entityManagerFactory2" />
<qualifier value="transactionManager2" />
</bean>
<bean id="hibernateVendorAdapter2" class="org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter" />
<tx:annotation-driven/>
Implementation
#Repository
#Transactional("transactionManager2")
public class myDaoImpl extends GenericJPADao<Integer, Integer> implements ImyDao {
#PersistenceContext(unitName="proxy2")
protected EntityManager em;
}
SOLUTION
The accepted answer was the correct solution for me, but a few things to note. The beans have to point to their respective entityManagerFactory's and you need to be careful on which bean you set the autowire-candidate="false" on, as I set it on the incorrect one at first, and had transactions rolled back as a result. I think there could be cleaner solution to this, but as a quick fix it works fine
try this :
<bean id="transactionManager" class="org.springframework.orm.jpa.JpaTransactionManager" autowire-candidate="false">
<property name="entityManagerFactory" ref="entityManagerFactory" />
</bean>
<bean id="transactionManager2" class="org.springframework.orm.jpa.JpaTransactionManager">
<property name="entityManagerFactory" ref="entityManagerFactory" />
</bean>
This question already has answers here:
Spring Boot, Spring Data JPA with multiple DataSources
(6 answers)
Closed 4 years ago.
I read all the related questions and tried all of them but still can't make my configuration straight.
I've two databases and i want to use them as datasources in my application.
Here is my context file:
<jee:jndi-lookup id="firstDataSource" jndi-name="java:/comp/env/jdbc/firstDS" expected-type="javax.sql.DataSource" />
<jee:jndi-lookup id="secondDataSource" jndi-name="java:/comp/env/jdbc/secondDS" expected-type="javax.sql.DataSource" />
<bean name="persistenceProvider" class="org.hibernate.jpa.HibernatePersistenceProvider"></bean>
<bean class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean" id="thisEntityManagerFactory">
<property name="dataSource" ref="firstDataSource"/>
<property name="packagesToScan" value="com.a.b.first.model"/>
<property name="jpaVendorAdapter">
<bean class="org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter">
<property name="generateDdl" value="false" />
<property name="showSql" value="true"/>
<property name="databasePlatform" value= "org.hibernate.dialect.Oracle10gDialect"/>
<property name="database" value="ORACLE"/>
</bean>
</property>
<property name="persistenceUnitName" value="firstPersistenceUnit" />
<property name="persistenceProvider" ref="persistenceProvider"></property>
<property name="jpaProperties">
<value>
hibernate.generate_statistics = true
hibernate.cache.use_second_level_cache = true
hibernate.cache.region.factory_class = org.hibernate.cache.ehcache.SingletonEhCacheRegionFactory
hibernate.cache.use_query_cache = true
<!--hibernate.hbm2ddl.auto=create-->
</value>
</property>
</bean>
<bean class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean" id="otherEntityManagerFactory">
<property name="dataSource" ref="secondDataSource"/>
<property name="packagesToScan" value="com.a.b.second.model"/>
<property name="jpaVendorAdapter">
<bean class="org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter">
<property name="generateDdl" value="false" />
<property name="showSql" value="true"/>
<property name="databasePlatform" value= "org.hibernate.dialect.Oracle10gDialect"/>
<property name="database" value="ORACLE"/>
</bean>
</property>
<property name="persistenceUnitName" value="secondPersistenceUnit" />
<property name="persistenceProvider" ref="persistenceProvider"></property>
<property name="jpaProperties">
<value>
hibernate.generate_statistics = true
hibernate.cache.use_second_level_cache = true
hibernate.cache.region.factory_class = org.hibernate.cache.ehcache.SingletonEhCacheRegionFactory
hibernate.cache.use_query_cache = true
<!--hibernate.hbm2ddl.auto=create-->
</value>
</property>
</bean>
<bean class="org.springframework.orm.jpa.JpaTransactionManager" id="thisTransactionManager">
<property name="entityManagerFactory" ref="thisEntityManagerFactory"/>
<property name="jpaDialect">
<bean class="org.springframework.orm.jpa.vendor.HibernateJpaDialect" />
</property>
</bean>
<bean class="org.springframework.orm.jpa.JpaTransactionManager" id="otherTransactionManager">
<property name="entityManagerFactory" ref="otherEntityManagerFactory"/>
<property name="jpaDialect">
<bean class="org.springframework.orm.jpa.vendor.HibernateJpaDialect" />
</property>
</bean>
<tx:annotation-driven transaction-manager="thisTransactionManager" />
<tx:annotation-driven transaction-manager="otherTransactionManager" />
<jpa:repositories base-package="com.a.b.first.intf" entity-manager-factory-ref="thisEntityManagerFactory" transaction-manager-ref="transactionManager"/>
<jpa:repositories base-package="com.a.b.second.intf" entity-manager-factory-ref="otherEntityManagerFactory" transaction-manager-ref="otherTransactionManager" />
Problem is when i try to use a repository interface which is located under package com.a.b.second.intf, it goes to firstDataSource and throws a SQLSyntaxErrorException with message "ORA-00942: table or view does not exist". Because there is no such table in the first database.
There is a line in tho logs for each EntityManagerFactory, saying
Building JPA container EntityManagerFactory for persistence unit
'default'
And in the next line it prints PersistenceUnitInfo for this EntityManagerFactory. The "Non JTA datasource" property of the PersistenceUnitInfo is same for both EntityManagerFactories. I guess that means both persistence units uses the same datasource.
What am i missing?
Thanks.
In your service layer you should have something like this:
#Service
#Transactional("thisTransactionManager")
public class ThisService{
#Autowired
private com.a.b.first.intf.Repo1 repo1;
}
#Service
#Transactional("otherEntityManagerFactory")
public class OtherService{
#Autowired
private com.a.b.second.intf.Repo1 repo1;
}
This way, when you call a service method, the TransactionInterceptor can load the appropriate transaction manager, associated to the EntityManagerFactory you want to operate with.
How would I setup container-managed datasources and embedded Active MQ resources to JTATransactionManager for global Transactions?
I am using Tomcat 6 and installed Atomikos in it to support JTA. I use Hibernate for ORM. Here is my configuration:
<bean id="AtomikosTransactionManager"
class="com.atomikos.icatch.jta.UserTransactionManager"
init-method="init" destroy-method="close">
<!-- when close is called, should we force
transactions to terminate or not? -->
<property name="forceShutdown" value="false" />
</bean>
<bean id="AtomikosUserTransaction"
class="com.atomikos.icatch.jta.UserTransactionImp">
<property name="transactionTimeout" value="300" />
</bean>
<jee:jndi-lookup expected-type="javax.sql.DataSource" id="dataSource" jndi-name="jdbc/EDITSOLUTIONS"/>
<bean id="sessionFactory" class="org.springframework.orm.hibernate.LocalSessionFactoryBean">
<property name="dataSource" ref="dataSource"/>
<property name="mappingResources"/>
<list>
<value>../../src/editsolutions.hibernate.cfg.xml</value>
</list>
<property name="hibernateProperties">
<value>
hibernate.dialect=${hibernate.dialect}
</value>
</property>
</bean>
<bean id="activemq" class="org.apache.activemq.camel.component.ActiveMQComponent">
<property name="connectionFactory">
<bean class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL" value="failover://tcp://localhost:61616"/>
</bean>
</property>
</bean>
<bean id="jmsConnectionFactory"class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL" value="failover://tcp://localhost:61616"/>
</bean>
<bean name="txManager" class="org.springframework.transaction.jta.JtaTransactionManager">
<property name="transactionManager" ref="AtomikosTransactionManager" />
<property name="userTransaction" ref="AtomikosUserTransaction" />
</bean>
Spring documentation says that JTA Transaction Manager need not be told about resources. That's what I have done.
I have the following outstanding questions:
I am not sure whether Atomikos is integrated properly or not?
Is it OK to get the datasource from <jee:jndi-lookup>?
Is Hiberante configuration correct with respect to JTATransactionManager?
As it is embedded in JVM not managed by container, would JTATransactionManager be able to recognize ActimeMQ ?
Try with this very useful link: http://www.atomikos.com/Documentation/SpringIntegration
Remember to configure the datasource in this way:
<bean id="dataSourceA" class="com.atomikos.jdbc.AtomikosDataSourceBean" init-method="init" destroy-method="close">
<qualifier value="jmsRecoveryDatabaseSchema"/>
<property name="uniqueResourceName"><value>XADS1</value></property>
<property name="xaDataSourceClassName">
<value>oracle.jdbc.xa.client.OracleXADataSource</value>
</property>
<property name="xaProperties">
<props>
<prop key="URL">${jdbc.url}</prop>
<prop key="user">${jdbc.username}</prop>
<prop key="password">${jdbc.password}</prop>
</props>
</property>
<property name="poolSize"><value>1</value></property>
<property name="testQuery" value="SELECT 1 FROM DUAL"/>
</bean>
I hope it helps!