below is my hibernate.xml file this file , for any query in sql i will use session factory for query but in this i am using a property name dataSource where it is refered to database connection,so for every query i am calling session factory and and for every call it is calling dataSource and making a new connection rather than that i want to make only one connection and make multiple queries one for each request is it possible
i am using hibernate for sql queries
i am using below hibernate.xml as i have learnt from http://www.mkyong.com/spring/maven-spring-hibernate-mysql-example/
hibernate.xml:
<property name="dataSource">
<ref bean="dataSource"/>
</property>
<property name="hibernateProperties">
<props>
<prop key="hibernate.dialect">org.hibernate.dialect.MySQLDialect</prop>
<prop key="hibernate.show_sql">true</prop>
</props>
</property>
<property name="mappingResources">
<list>
<value>resources/database/Token.xml</value>
</list>
</property>
</bean>
datasource.xml:
<bean id="dataSource"
class="org.apache.commons.dbcp.BasicDataSource">
<property name="driverClassName" value="com.mysql.jdbc.Driver"/>
<property name="url" value="jdbc:mysql://localhost:3306/get"/>
<property name="username" value="root"/>
<property name="password" value="password"/>
<property name="initialSize" value="3"/>
<property name="minPoolSize" value="1" />
<property name="maxPoolSize" value="3" />
</bean>
</beans>
Update: i have made my code connection pooling but how to make it open only one connection at start of application and use same connection for every request
as you can see for every call to session factory it calls dataSource and it makes a connection i want to stop it
Two considerations.
First of all is that you can configure all your beans in only one xml file, not need to keep different files for hibernate and spring (the Spring one is enough).
Second: you can use a datasource that supports pooling connection and more configuration like C3P0.
An example of how to declare it is:
<bean id="yourDataSource" class="com.mchange.v2.c3p0.ComboPooledDataSource" destroy-method="close">
<property name="driverClass" value="com.mysql.jdbc.Driver" />
<property name="jdbcUrl" value="jdbc:mysql://localhost:3306/get" />
<property name="user" value="username" />
<property name="password" value="password" />
<property name="minPoolSize" value="1" />
<property name="maxPoolSize" value="3" />
<property name="preferredTestQuery">
<value>select null from dual</value>
</property>
<property name="testConnectionOnCheckin">
<value>true</value>
</property>
<property name="idleConnectionTestPeriod">
<value>1000</value>
</property>
</bean>
If you want to reuse connections, you need to use DataSource implementation which supports connection pooling.
The example here is for dbcp library. There is also Tomcat implementation.
In order to use it you need to add the library to the dependencies and update the configuration of the DataSource with the implementation class and the configuration values for the connection pool.
I am not sure I understand your issue but, how are you configuring your sessionfactory? Is your sessionfactory a spring bean?
If it is spring bean, then it is singleton by default, which means only one instance of sessionfactory exists for the entire application and is shared. In that case your assumption that you are creating a new sessionfactory which then creates a new datasource is wrong.
If you are manually creating your sessionfactory in code, then you need to implement the singleton design pattern by yourself.
private static SessionFactory seesionFactory = null;
private static final SessionFactory makeSessionFactory()
{
try {
if (sessionFactory==null)
seesionFactory = new Configuration().configure().buildSessionFactory();
} catch (HibernateException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return seesionFactory;
}
Related
I have a legacy Spring Data JPA application that has a large number of Entities and CrudRepositories. JPA is configured using the XML below. We have a new requirement that requires us to insert 10,000 - 50,000 entities into the database at once via a FileUpload. With the existing configuration the database CPU spikes. After enabling hibernate statistics, its was apparent that these 10,000 insert operations were generating over 200,000 DB queries due to all the validation logic required with a single insert in the InvoiceService.
Original Configuration
<bean id="dataSource" destroy-method="close" class="org.apache.commons.dbcp2.BasicDataSource">
<property name="driverClassName" value="${db.driver}"/>
<property name="url" value="${db.jdbcurl}"/>
<property name="username" value="${db.username}"/>
<property name="password" value="${db.password}"/>
<property name="maxTotal" value="${db.maxTotal}"/>
<property name="maxIdle" value="${db.maxIdle}"/>
<property name="minIdle" value="${db.minIdle}"/>
<property name="initialSize" value="${db.initialSize}"/>
<property name="maxWaitMillis" value="${db.maxWaitMillis}"/>
<property name="minEvictableIdleTimeMillis" value="${db.minEvictableIdleTimeMillis}"/>
<property name="timeBetweenEvictionRunsMillis" value="${db.timeBetweenEvictionRunsMillis}"/>
<property name="testOnBorrow" value="${db.testOnBorrow}"/>
<property name="testOnReturn" value="${db.testOnReturn}"/>
<property name="testWhileIdle" value="${db.testWhileIdle}"/>
<property name="removeAbandonedOnBorrow" value="${db.removeAbandonedOnBorrow}"/>
<property name="removeAbandonedOnMaintenance" value="${db.removeAbandonedOnMaintenance}"/>
<property name="removeAbandonedTimeout" value="${db.removeAbandonedTimeout}"/>
<property name="logAbandoned" value="${db.logAbandoned}"/>
</bean>
<bean id="entityManagerFactory"
class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean" depends-on="flyway">
<property name="dataSource" ref="dataSource" />
<property name="packagesToScan" value="my.package.domain" />
<property name="loadTimeWeaver">
<bean class="org.springframework.instrument.classloading.InstrumentationLoadTimeWeaver" />
</property>
<property name="jpaVendorAdapter">
<bean class="org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter" />
</property>
<property name="jpaProperties">
<props>
<prop key="hibernate.hbm2ddl.auto">${hibernate.hbm2ddl.auto:validate}</prop>
<prop key="hibernate.dialect">org.hibernate.dialect.PostgreSQL9Dialect</prop>
<prop key="hibernate.show_sql">${hibernate.show_sql:false}</prop>
</props>
</property>
<property name="persistenceUnitName" value="entityManagerFactory" />
</bean>
<bean id="persistenceAnnotationBeanPostProcessor" class="org.springframework.orm.jpa.support.PersistenceAnnotationBeanPostProcessor">
<property name="defaultPersistenceUnitName" value="entityManagerFactory"/>
</bean>
<bean id="transactionManager" class="org.springframework.orm.jpa.JpaTransactionManager">
<property name="entityManagerFactory" ref="entityManagerFactory" />
</bean>
<tx:annotation-driven proxy-target-class="true" />
<bean id="persistenceExceptionTranslationPostProcessor"
class="org.springframework.dao.annotation.PersistenceExceptionTranslationPostProcessor" />
<jpa:repositories base-package="my.package.repository" entity-manager-factory-ref="entityManagerFactory"/>
The FileUploadService snippet looks like this...
EntityManager batchEntityManager = entityManagerFactory.createEntityManager();
EntityTransaction transaction = batchEntityManager.getTransaction();
transaction.begin();
try (BufferedReader buffer = new BufferedReader(new InputStreamReader(file.getInputStream()))) {
buffer.lines()
.filter(StringUtils::isNotBlank)
.forEach(csvLine -> {
invoiceService.createInvoice(csvLine);
if (counter.incrementAndGet() % (updateFrequency.equals(0) ? 1 : updateFrequency) == 0) {
FileUpload fileUpload1 = fileUploadRepository.findOne(fileUpload.getId());
fileUpload1.setNumberOfSentRecords(sentCount.get());
fileUploadRepository.save(fileUpload1);
transaction.commit();
transaction.begin();
batchEntityManager.clear();
}
});
transaction.commit();
} catch (IOException ex) {
systemException.incrementAndGet();
log.error("Unexpected error while performing send task.", ex);
transaction.rollback();
}
// Update FileUpload status.
FileUpload fileUpload1 = fileUploadRepository.findOne(fileUpload.getId());
fileUpload1.setNumberOfSentRecords(sentCount.get());
if (systemException.get() != 0) {
fileUpload1.setStatus(FileUploadStatus.SYSTEM_ERROR);
} else {
fileUpload1.setStatus(FileUploadStatus.SENT);
}
fileUploadRepository.save(fileUpload1);
batchEntityManager.close();
Most of the DB queries were select statements that return the same results for each record being inserted. It was obvious that enabling EhCache as a Second-Level cache would have a significant performance improvement. However, this application has been running flawlessly in production for several years without ehcache enabled. I am hesitant to turn this on globally as I do not know how this will affect the large number of other repositories/queries.
Question 1
Is there a way to configure a "alternate" EntityManagerFactory that uses the second level cache for this batch process only? How can I choose to use this factory instead of the primary for this batch process only?
I experimented adding something like below to my spring config. I can easily inject this additional EntityManager into my class and use it. However, the existing Repositories (such as FileUploadRepository) don't seem to use it - they just return null. I am not sure if this approach is possible. The documentation for JpaTransactionManager says
This transaction manager is appropriate for applications that use a single JPA EntityManagerFactory for transactional data access
which is exactly what I am not doing. So what other options are there?
<bean id="batchEntityManagerFactory"
class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean" depends-on="flyway">
<property name="dataSource" ref="dataSource" />
<property name="packagesToScan" value="my.package.domain" />
<property name="loadTimeWeaver">
<bean class="org.springframework.instrument.classloading.InstrumentationLoadTimeWeaver" />
</property>
<property name="jpaVendorAdapter">
<bean class="org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter" />
</property>
<property name="jpaProperties">
<props>
<prop key="hibernate.hbm2ddl.auto">${hibernate.hbm2ddl.auto:validate}</prop>
<prop key="hibernate.dialect">org.hibernate.dialect.PostgreSQL9Dialect</prop>
<prop key="hibernate.show_sql">${hibernate.show_sql:false}</prop>
<prop key="hibernate.generate_statistics">${hibernate.generate_statistics:false}</prop>
<prop key="hibernate.ejb.interceptor">my.package.HibernateStatistics</prop>
<prop key="hibernate.cache.use_query_cache">true</prop>
<prop key="hibernate.cache.use_second_level_cache">true</prop>
<prop key="hibernate.cache.region.factory_class">org.hibernate.cache.ehcache.EhCacheRegionFactory</prop>
<prop key="hibernate.jdbc.batch_size">100</prop>
<prop key="hibernate.order_inserts">true</prop>
<prop key="hibernate.order_updates">true</prop>
</props>
</property>
<property name="persistenceUnitName" value="entityManagerFactory" />
</bean>
Question 2
Assuming there is no other option to "selectively" use EhCache, I tried enabling it on the primary EntityManagerFactory only. We can certainly do regression testing to make sure we don't introduce new issues. I assume this is fairly safe to do? However, another issue came up. I am trying to commit inserts in batches as described in this post and shown in my code above. I am getting an RollbackException when trying to commit the batch due to Connection org.postgresql.jdbc.PgConnection#1e7eb804 is closed.. I presume this is due to the maxWaitMillis property on the dataSource.
I don't want to change this property for every other existing spring Service/Repository/query in the application. If I could use the "custom" EntityManagerFactory I could easily provide a different DataSource Bean with the properties I want. Again, is this possible?
Maybe I looking at this problem all wrong. Are there any other suggestions?
You can have another EntityManagerFactory bean with a different qualifier, so that's one option. I would still recommend you look into these select queries. I bet the problem is just a missing index which causes the database to do full table scans. If you add the proper indexes, you probably don't need to change a thing in your application.
I've been trying to configure the connections made with a postgresql datasource declared in a xml Spring configuration file.
<bean id="myDataSource"
class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="driverClassName" value="org.postgresql.Driver" />
<property name="url" value="jdbc:postgresql://localhost:5432/dbname" />
<property name="username" value="postgres" />
<property name="password" value="" />
<property name="socketTimeout" value="10"/>
</bean>
I know, I shouldn't be using the DriverManagerDataSource class from spring, (we will soon move to C3p0 or DBCP) because it's not a real pooling.
I'm trying to set the socketTimeout value of a postgresql connection ( described here https://jdbc.postgresql.org/documentation/head/connect.html ) but of course, "socketTimeout" is not a property of the datasource, so it doesn't work.
Is it possible to do this through the datasource xml's configuration ? Or should I do it somewhere else ? Because the data source manages the connection I don't think I'll be able to do a
props.setProperty("timeout",30);
Connection conn = DriverManager.getConnection(url, props);
Can I even do this with the DriverManagerDataSource ? I tried to search, but I didn't find anything usefull, as not a lot of people are really using it.
Thank you M. Deinum, I was able to find how.
Actually, even knowing the property was named "connectionProperties", I didn't found a lot answers (maybe people rarely use it this way ?). So I'm posting it:
<bean id="myDataSource"
class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="driverClassName" value="org.postgresql.Driver" />
<property name="url" value="jdbc:postgresql://localhost:5432/dbname" />
<property name="username" value="postgres" />
<property name="password" value="" />
<!--<property name="socketTimeout" value="10"/>-->
<property name="connectionProperties">
<props>
<prop key="socketTimeout">10</prop>
</props>
</property>
</bean>
If anyone has a better/more complete answer, I'll check it out ;)
I'm studying Spring MVC and Hibernate. I had no problems handling database connections and queries with Spring mvc (MySql DB).
Now, I'm trying to use Hibernate and I found it intricate:
create a hibernate configuration file, create a class for retrieving SessionFactory, create a xml file for any persistent object etc.
I'm sure there's a simplest way that allow me to do an easy configuration using just:
the Spring xml configuration file
annotation (in persistent object classes)
What iI want to reach is something like the following. I saw a similar code in an example, but now I'm no more able to find it on the internet
xxxx-servlet.xml
<bean id="SessionFactory" class="org.springframework.orm.hibernate3.annotation.AnnotationSessionFactoryBean">
<property name="dataSource" ref="dataSource" />
<property name="packagesToScan" value="com.springgestioneerrori.model" />
<property name="hibernateProperties">
<props>
<prop key="hibernate.dialect">org.hibernate.dialect.MySQLDialect</prop>
<prop key="hibernate.connection.autocommit">true</prop>
<prop key="hibernate.show_sql">false</prop>
<prop key="hibernate.format_sql">true</prop>
<prop key="hibernate.use_sql_comments">false</prop>
</props>
</property>
</bean>
<bean id="dataSource" class="org.springframework.jdbc.datasource.DriverManagerDataSource" >
<property name="driverClass" value="com.mysql.jdbc.Driver" />
<property name="jdbcUrl" value="jdbc:mysql://localhost:3306/dbName" />
<property name="user" value="root" />
<property name="password" value="root" />
</bean>
In the code above, I suppose, that what it's not correct is my dataSource bean. Does anybody know the way to reach my goal?
Thank everybody!
i think your properties names inside the 'data source' bean should be like the following :
<property name="driverClassName" value="com.mysql.jdbc.Driver" />
<property name="url" value="jdbc:mysql://localhost:3306/dbName" />
<property name="username" value="root" />
<property name="password" value="root" />
so instead of driverClass , it should be driverClassName and so on ..
and please refer to this answer here , it talks about using spring DriverManagerDataSource vs apache BasicDataSource .
Hope that Helps
Everything must be correct, but I'd want to offer you to use *.properties to keep connection configuration to your DB like that
in appContext.xml:
<!-- JDBC DataSource bean -->
<bean id="dataSource" class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="driverClassName" value="${jdbc.driverClassName}"/>
<property name="url" value="${jdbc.url}"/>
<property name="username" value="${jdbc.username}"/>
<property name="password" value="${jdbc.password}"/>
</bean>
<bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="locations">
<list>
<value>classpath:properties/database.properties</value>
</list>
</property>
</bean>
in database.properties:
jdbc.driverClassName = com.mysql.jdbc.Driver
jdbc.schema = schema
jdbc.url = jdbc:mysql://localhost:3306/schema
jdbc.username = root
jdbc.password = password
And what interferes you to use annotations in your classes?
There are Oracle 11g2 and application Java based of spring
Need of DataSource for oracle JDBC driver with enable/disable CacheConnection functional in runtime - i.e. if CacheConnection is enable new connection not established if there are free connection in DataSource, if CacheConnection is disable always new connection established and existence is closing after put in idle
Early we used apache DataSource:
<bean id="datasourceClassic" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
<property name="driverClassName" value="oracle.jdbc.OracleDriver" />
<property name="url" value="jdbc:oracle:oci:#TEST" />
<property name="username" value="TEST" />
<property name="password" value="TEST" />
<property name="maxActive" value="10" />
<property name="defaultAutoCommit" value="false" />
</bean>
-- but this data source don't have functional for enable/disable CacheConnection
We tested OracleDataSource:
<property name="URL" value="jdbc:oracle:oci:#TEST" />
<property name="user" value="TEST" />
<property name="password" value="TEST" />
<property name="connectionCachingEnabled" value="true" />
<property name="connectionCacheProperties">
<props>
<prop key="MinLimit">0</prop>
<prop key="MaxLimit">1</prop>
<prop key="InitialLimit">0</prop>
<prop key="InactivityTimeout">10</prop>
<prop key="ConnectionWaitTimeout">10</prop>
<prop key="ValidateConnection">true</prop>
</props>
</property>
</bean>
-- have this functional but it work only if explicity set up this options in context.xml, and method setConnectionCachingEnabled depreceted from runtime
Oracle since 11g2 recomended use Universal Connection Pool (UCP):
<bean id="datasource2" class="oracle.ucp.jdbc.PoolDataSourceFactory" factory-method="getPoolDataSource">
<property name="connectionFactoryClassName" value="oracle.jdbc.pool.OracleDataSource"/>
<property name="connectionPoolName" value="TEST"/>
<property name="URL" value="jdbc:oracle:oci:#TEST" />
<property name="user" value="TEST" />
<property name="password" value="TEST" />
<property name="initialPoolSize" value="0" />
<property name="minPoolSize" value="0" />
<property name="maxPoolSize" value="1" />
<property name="validateConnectionOnBorrow" value="true" />
</bean>
-- but this pool also didn't have functional to enable/disable CacheConnection...
Any idea of what DataSource can satisfy my requirements?
UPDATE:
I test next trik with org.apache.commons.dbcp.BasicDataSource:
/* "Disable" CacheConnection */
BasicDataSource bds = ... //get DataSource
bds.setMaxIdle(0);
and:
/* "Enable" CacheConnection */
BasicDataSource bds = ... //get DataSource
bds.setMaxIdle(bds.getMaxActive());
Works like <property name="connectionCachingEnabled" value="true" /> - what do you think about this?
Are you able to create a data source which has caching enabled and one which has caching disabled? If so, you could create two different DataSource beans with different names and configurations in your ApplicationContext. You could autowire both in your application and switch between them dynamically.
public class MyClass {
#Autowired
#Qualifier("datasourceCacheDisabled")
DataSource cacheDisabledDataSource;
#Autowired
#Qualifier("datasourceCacheEnabled")
DataSource cacheEnabledDataSource;
DataSource dataSource=null;
boolean enableCache;
public MyClass() {
// enable by default
enableDataSourceCaching();
}
public void enableDataSourceCaching() {
enableCache=true;
dataSource=cacheEnabledDataSource;
}
public void disableDataSourceCaching() {
enableCache=false;
dataSource=cacheDisabledDataSource;
}
// code which uses the current dataSource
...
}
Alternately, if you need beans with the DataSource wired already, you could create two ApplicationContexts and toggle between them. Note that the latter solution prevents results caching; I'm not sure about the effects on caching with the first solution.
I am currently in the process of upgrading an application from Hibernate 3.2 to Hibernate 3.3. I though I'd stick with the default connection pool (Hibernate changed its default from Commons DBCP to c3p0) as I don't have any good reason to choose a non-default pool. At least non but having used DBCP before.
The upgrade went pretty much without any problems so far. The only thing I can't get to work is passing properties to the underlying MySQL JDBC4Connection. Up to now, I used DBCP's BasicDataSource.addConnectionProperty(String,String) to pass properties (useUnicode=true, characterEncodin=UTF-8, characterSetResults=UTF-8, zeroDateTimeBehavior=convertToNull).
However, I can't find any way to do the same with c3p0 other than including them in the JDBC URL. (That's something I'd like to avoid as I wanna keep the URL configurable without forcing users to include those parameters.)
So far, I've tried to use a ConnectionCustomizer without success. Any other suggestions?
Once again a question I answer myself (another self-learner? yes, please!):
com.mchange.v2.c3p0.ComboPooledDataSource has a property "properties". Interestingly, setting properties after user and password overrides them. But setting properties before user and password works as expected.
Follow up for the self answer.
An example of a spring way of configuring this:
The Data source bean:
<bean id="dataSource" class="com.mchange.v2.c3p0.ComboPooledDataSource" destroy-method="close">
<property name="properties" ref="mysqlConnectionProperties"></property>
<property name="driverClass" value="${jdbc.driver}" />
<property name="jdbcUrl" value="${jdbc.url}" />
<property name="user" value="${jdbc.username}" />
<property name="password" value="${jdbc.password}" />
<!-- c3p0 combo pooled data source settings -->
<property name="initialPoolSize" value="3" />
<property name="minPoolSize" value="3" />
<property name="maxPoolSize" value="50" />
<property name="maxIdleTime" value="7200" />
<property name="maxStatements" value="200" />
<property name="idleConnectionTestPeriod" value="270" />
<property name="preferredTestQuery">
<value>SELECT 1</value>
</property>
</bean>
The properties bean:
<bean id="mysqlConnectionProperties" class="java.util.Properties">
<constructor-arg>
<props>
<prop key="useTimezone">true</prop>
<prop key="serverTimezone">America/Chicago</prop>
<!-- add any other properties you have -->
</props>
</constructor-arg>
</bean>