Use a fallback datasource for Hibernate in Spring

Use a fallback datasource for Hibernate in Spring - java

We are working on a Spring based Web application where the key to the business is high availability. Hibernate is the ORM and MySQL is the DB that is used. Our architecture forces us to have the following mechanism.
The Webapp first tries to connect to the primary MySQL server.
If that fails, it connects to the Secondary MySQL server, which is mostly out of sync with the data.
The webapp needs to know which MySQL Server it is connected to, since we want to notify the user when he is using the secondary server.
As soon as the connection re-establishes with the primary, the connected has to be switched from secondary to primary.
I am stuck at the very first phase. I am unable find out how to direct Spring/Hibernate to use multiple DB Servers.
Here is the current config file (removing the unwanted stuff):
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:aop="http://www.springframework.org/schema/aop"
xmlns:context="http://www.springframework.org/schema/context"
xmlns:jee="http://www.springframework.org/schema/jee" xmlns:lang="http://www.springframework.org/schema/lang"
xmlns:p="http://www.springframework.org/schema/p" xmlns:tx="http://www.springframework.org/schema/tx"
xmlns:util="http://www.springframework.org/schema/util"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop.xsd
http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd
http://www.springframework.org/schema/jee http://www.springframework.org/schema/jee/spring-jee.xsd
http://www.springframework.org/schema/lang http://www.springframework.org/schema/lang/spring-lang.xsd
http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx.xsd
http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util.xsd">
<context:annotation-config />
<context:component-scan base-package="com.smartshop" />
<bean
class="org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor" />
<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource"
destroy-method="close">
<property name="driverClassName" value="com.mysql.jdbc.Driver" />
<property name="url" value="jdbc:mysql://localhost:3306/primarydb" />
<property name="username" value="username" />
<property name="password" value="password" />
<property name="maxIdle" value="10" />
<property name="maxActive" value="100" />
<property name="maxWait" value="10000" />
<property name="validationQuery" value="select 1" />
<property name="testOnBorrow" value="false" />
<property name="testWhileIdle" value="true" />
<property name="timeBetweenEvictionRunsMillis" value="1200000" />
<property name="minEvictableIdleTimeMillis" value="1800000" />
<property name="numTestsPerEvictionRun" value="5" />
<property name="defaultAutoCommit" value="false" />
</bean>
<bean id="sessionFactory"
class="org.springframework.orm.hibernate3.LocalSessionFactoryBean">
<property name="dataSource" ref="dataSource" />
<property name="configLocation">
<value>/WEB-INF/hibernate.cfg.xml</value>
</property>
<property name="configurationClass">
<value>org.hibernate.cfg.AnnotationConfiguration</value>
</property>
<property name="hibernateProperties">
<props>
<prop key="hibernate.dialect">org.hibernate.dialect.MySQLDialect</prop>
<prop key="hibernate.show_sql">false</prop>
</props>
</property>
</bean>
<tx:annotation-driven />
<bean id="transactionManager"
class="org.springframework.orm.hibernate3.HibernateTransactionManager">
<property name="sessionFactory" ref="sessionFactory" />
</bean>
<bean
class="org.springframework.orm.hibernate3.support.OpenSessionInViewInterceptor"
name="openSessionInViewInterceptor">
<property name="sessionFactory" ref="sessionFactory"></property>
<property name="flushMode">
<bean
id="org.springframework.orm.hibernate3.HibernateAccessor.FLUSH_AUTO"
class="org.springframework.beans.factory.config.FieldRetrievingFactoryBean" />
</property>
</bean>
<bean id="handlerMapping"
class="org.springframework.web.servlet.mvc.annotation.DefaultAnnotationHandlerMapping">
<property name="interceptors">
<list>
<ref bean="localeChangeInterceptor" />
<ref bean="openSessionInViewInterceptor" />
</list>
</property>
</bean>
Is there a way to define Spring to connect to a backup datasource when the primary datasource is inaccessible?

if you configure your datasource as a jndi datasource you can use the following configuration
<bean id="dataSource"
class="org.springframework.jndi.JndiObjectFactoryBean">
<property name="jndiName" ref="datasourceJNDIName" />
<property name="defaultObject" ref="fallBackDataSource" />
</bean>
<!-- fall back datasource if JNDI look up of main datasource fails -->
<bean id="fallBackDataSource"
class="org.springframework.jndi.JndiObjectFactoryBean">
<property name="jndiName" ref="datasourceJNDIName-2" />
</bean>

This kind of tricks have to be done on MySQL side not on the webapp. MySQL cluster for instance can handle this.
More infos here:
-http://www.mysql.com/products/cluster/
-http://en.wikipedia.org/wiki/MySQL_Cluster
UPDATE:
Ok, so, have a look here -> org.springframework.jdbc.datasource.lookup.AbstractRoutingDataSource . Build a custom implementation that override the method getConnection() and getConnection(String username, String password). Surroud them by catching SQLException, and if occurs, choose the other datasource.

Related

Spring batch 3600 partitions long pause before starting

I have the following spring batch job with a partitioned step that creates 3600 partitions for a partitioned step. I use a ThreadPoolTaskExecutor with a max pool size 100 and a queue capacity of 100 (although it seems to make no difference for speed). Im using Visual VM to monitor the threads and I notice the taskExecutor threads don't start until > 5 minutes after starting the job.
Oddly enough, If I limit the number of partitions to 100 the threads start fairly quickly and finish in about a minute.
Another issue I notice is that there doesn't seem to be more than one database connection ever as seen in the VisualVM thread visualization
Can someone please review my batch job below and tell me if I am missing something that would limit the number of database connections to 1? Also, why would adding more partitions affect the performance if my ThreadPoolTaskExecutor parameters don't change? Shouldn't the jobs just sit in a queue until there is a thread able to service them?
--- Spring batch job ---
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:batch="http://www.springframework.org/schema/batch"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:context="http://www.springframework.org/schema/context"
xmlns:util="http://www.springframework.org/schema/util"
xmlns:lang="http://www.springframework.org/schema/lang"
xsi:schemaLocation="http://www.springframework.org/schema/batch http://www.springframework.org/schema/batch/spring-batch-3.0.xsd
http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.0.xsd
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-4.0.xsd
http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util-3.0.xsd
http://www.springframework.org/schema/lang http://www.springframework.org/schema/lang/spring-lang-4.0.xsd">
<context:property-placeholder location="${ext.properties.dataManipulation.Properties}"/>
<import resource="${ext.properties.dataManipulation.Connection}"/>
<import resource="flatFileLineProperties.xml"/>
<!-- JobRepository and JobLauncher are configuration/setup classes -->
<bean id="jobRepository"
class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean"/>
<bean id="transactionManager"
class="org.springframework.batch.support.transaction.ResourcelessTransactionManager"/>
<bean id="jobLauncher"
class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
<property name="jobRepository" ref="jobRepository"/>
</bean>
<util:map id="keys" map-class="java.util.HashMap">
<entry key="SAILING_ID" value="ASCENDING" value-type="org.springframework.batch.item.database.Order"/>
<entry key="RES_ID" value="ASCENDING" value-type="org.springframework.batch.item.database.Order" />
</util:map>
<!-- Here is my partioned step -->
<bean id="reservationsItemReader" class="org.springframework.batch.item.database.JdbcPagingItemReader" scope="step">
<property name="dataSource" ref="dataSource" />
<property name="queryProvider">
<bean class="org.springframework.batch.item.database.support.SqlPagingQueryProviderFactoryBean">
<property name="dataSource" ref="dataSource" />
<property name="selectClause">
<value>
<![CDATA[
GUEST_ID,
FIRST_NAME,
LAST_NAME,
TITLE,
HOUSEHOLD_NAME,
SAILING_ID,
RES_ID
]]>
</value>
</property>
<property name="fromClause" value="FROM RESERVATION "/>
<property name="whereClause" >
<value>
<![CDATA[ AND SAIL_ID = :sailId
]]>
</value>
</property>
<!--<property name="sortKey" value="SAILING_ID" />-->
<property name="sortKeys" ref="keys"/>
<!--<property name="sortKeys" ref="sortKeys"/>-->
</bean>
</property>
<property name="parameterValues">
<map>
<!--<entry key="shipCode" value="#{stepExecutionContext[shipCode]}" />-->
<entry key="sailId" value="#{stepExecutionContext[sailId]}" />
</map>
</property>
<!--property name="pageSize" value="500000" /-->
<property name="pageSize" value="40000" />
<property name="rowMapper">
<bean class="com.ncl.endeca.mapper.ColumnToHashMapper" />
</property>
</bean>
<bean id="sortKeys" class="java.util.HashMap" scope="prototype" >
<constructor-arg>
<map key-type="java.lang.String" value-type="org.springframework.batch.item.database.Order">
<entry key="SAILING_ID" value="ASCENDING" />
<entry key="RES_ID" value="ASCENDING" />
</map>
</constructor-arg>
</bean>
<util:list id="client_fields" value-type="java.lang.String">
<value>FIRST_NAME</value>
<value>LAST_NAME</value>
<value>TITLE</value>
<value>HOUSEHOLD_NAME</value>
</util:list>
<bean id="reservationsItemWriter" class="com.ncl.endeca.writer.ReservationWriter" scope="step">
<property name="guestFields" ref="client_fields" />
<property name="outPrefix" value="${file.out.prefix}" />
<property name="shipCode" value="#{stepExecutionContext[shipCode]}" />
<property name="sailId" value="#{stepExecutionContext[sailId]}" />
<property name="soldOutSailings" ref="soldOutSailingsList" />
</bean>
<bean id="yearsAgo" class="java.lang.Integer">
<constructor-arg>
<value>${yearsAgo}</value>
</constructor-arg>
</bean>
<bean id="yearsAhead" class="java.lang.Integer">
<constructor-arg>
<value>${yearsAhead}</value>
</constructor-arg>
</bean>
<bean id="resPartitioner" class="com.ncl.endeca.partition.ReservationPartitioner">
<property name="yearsAgo" ref="yearsAgo" />
<property name="yearsAhead" ref="yearsAhead" />
<property name="batchLimit" value="${batch.limit}" />
<property name="dataSource" ref="dataSource"/>
</bean>
<bean id="taskExecutor"
class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
<property name="corePoolSize" value="${batch.corePoolSize}" />
<property name="maxPoolSize" value="${batch.maxPoolSize}" />
<property name="queueCapacity" value="${batch.queueCapacity}" />
</bean>
<!--<bean id="taskExecutor" class="org.springframework.core.task.SimpleAsyncTaskExecutor"/>-->
<!-- each thread will run this job, with different stepExecutionContext values. -->
<step id="slave" xmlns="http://www.springframework.org/schema/batch" >
<flow parent="readReservations"/>
</step>
<!--<bean id="countrySpecificCompletionPolicy" class="org.springframework.batch.core.resource.StepExecutionSimpleCompletionPolicy">-->
<!--<property name="keyName" value="sailId"/>-->
<!--</bean>-->
<batch:flow id="readReservations">
<batch:step id="reservations" xmlns="http://www.springframework.org/schema/batch" >
<tasklet throttle-limit="${batch.corePoolSize}">
<chunk reader="reservationsItemReader" writer="reservationsItemWriter" commit-interval="50000" />
</tasklet>
</batch:step>
</batch:flow>
<!-- Actual Job -->
<batch:job id="dataManipulationJob">
<batch:step id="masterStep">
<batch:partition step="slave" partitioner="resPartitioner">
<batch:handler grid-size="100" task-executor="taskExecutor" />
</batch:partition>
</batch:step>
</batch:job>
I have tried BasicDataSource and Hikari connections but the pool sizes have no affect when I monitor VisualVM
---- connection.xml ----
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:jdbc="http://www.springframework.org/schema/jdbc"
xmlns:batch="http://www.springframework.org/schema/batch"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:context="http://www.springframework.org/schema/context"
xmlns:util="http://www.springframework.org/schema/util"
xmlns:lang="http://www.springframework.org/schema/lang"
xsi:schemaLocation="http://www.springframework.org/schema/batch http://www.springframework.org/schema/batch/spring-batch-3.0.xsd
http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.0.xsd
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-4.0.xsd
http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util-3.0.xsd
http://www.springframework.org/schema/lang http://www.springframework.org/schema/lang/spring-lang-4.0.xsd">
<bean id="hikariConfig" class="com.zaxxer.hikari.HikariConfig">
<property name="poolName" value="springHikariCP" />
<property name="connectionTestQuery" value="SELECT 10 from dual" />
<property name="dataSourceClassName" value="${hibernate.dataSourceClassName}" />
<property name="maximumPoolSize" value="${batch.maxPoolSize}" />
<property name="idleTimeout" value="${hibernate.hikari.idleTimeout}" />
<property name="dataSourceProperties">
<props>
<prop key="url">${dataSource.url}</prop>
<prop key="user">${dataSource.username}</prop>
<prop key="password">${dataSource.password}</prop>
</props>
</property>
</bean>
<!-- HikariCP configuration -->
<!--<bean id="dataSource" class="com.zaxxer.hikari.HikariDataSource" destroy-method="close">-->
<!--<constructor-arg ref="hikariConfig" />-->
<!--</bean>-->
<!--<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close" scope="step">-->
<!--<property name="driverClassName" value="${hibernate.dataSourceClassName}" />-->
<!--<property name="url" value="${dataSource.url}" />-->
<!--<property name="username" value="${dataSource.username}" />-->
<!--<property name="password" value="${dataSource.password}" />-->
<!--<property name="testWhileIdle" value="false"/>-->
<!--<property name="maxActive" value="${batch.corePoolSize}"/>-->
<!--</bean>-->
<!-- connect to database -->
<bean id="dataSource"
class="org.springframework.jdbc.datasource.DriverManagerDataSource" >
<property name="driverClassName" value="oracle.jdbc.OracleDriver" />
<property name="url" value="jdbc:oracle:thin:#******" />
<property name="username" value="****" />
<property name="password" value="****" />
</bean>

The time to create partitions depends on the performance of your ReservationPartitioner as well as the number of partitions. Creating 3600 partitions means creating 3600 StepExecution/ExecutionContext objects and persisting them in the corresponding tables. This can take time for such a high number of partitions.
In regards to the database connection, you are using the MapJobRepositoryFactoryBean with a ResourcelessTransactionManager so there are no interactions with a database for Spring Batch meta-data. The only component that interacts with the database according to your configuration is the JdbcPagingItemReader (I don't know what is the type of your ReservationWriter) so it is not surprising to see a single database connection.

hsqldb file mode with spring, db not created

I'm trying to use hsqldb in file mode with spring and hibernate,
but the the database is not being initialized and i'm getting the following error message:
2015-03-12 23:58:35,542 FATAL [hsqldb.db.HSQLDB4C0FFF0D08.ENGINE]: could not reopen database
org.hsqldb.HsqlException: Database lock acquisition failure: lockFile: org.hsqldb.persist.LockFile#6602b8e7[file =C:\User\db".lck, exists=false, locked=false, valid=false, ] method: openRAF reason: java.io.FileNotFoundException: C:\User\db".lck (The filename, directory name, or volume label syntax is incorrect)
My spring configuration is as following:
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:jdbc="http://www.springframework.org/schema/jdbc"
xmlns:context="http://www.springframework.org/schema/context" xmlns:tx="http://www.springframework.org/schema/tx"
xmlns:p="http://www.springframework.org/schema/p"
xsi:schemaLocation="http://www.springframework.org/schema/jdbc http://www.springframework.org/schema/jdbc/spring-jdbc-4.1.xsd
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-4.1.xsd
http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-4.1.xsd
http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-4.1.xsd">
<tx:annotation-driven />
<context:component-scan base-package="org.oss.orm" />
<bean
class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="location">
<value>db.properties</value>
</property>
</bean>
<!-- data source -->
<bean id="dataSource" class="com.jolbox.bonecp.BoneCPDataSource">
<property name="driverClass" value="${jdbc.driverClassName}" />
<property name="jdbcUrl" value="${jdbc.url}" />
<property name="username" value="${jdbc.username}" />
<property name="password" value="${jdbc.password}" />
<property name="partitionCount" value="3" />
<property name="acquireIncrement" value="10" />
<property name="maxConnectionsPerPartition" value="60" />
<property name="minConnectionsPerPartition" value="20" />
</bean>
<jdbc:initialize-database data-source="dataSource">
<jdbc:script location="classpath:scripts/create-table-if-not-exists" />
</jdbc:initialize-database>
<!-- Session Factory -->
<bean id="sessionFactory" name="sessionFactory"
class="org.springframework.orm.hibernate4.LocalSessionFactoryBean">
<property name="dataSource" ref="dataSource"></property>
<property name="packagesToScan" value="org.oss.beans"></property>
<property name="hibernateProperties">
<props>
<prop key="dialect">org.hibernate.dialect.HSQLDialect</prop>
<prop key="hibernate.show_sql">true</prop>
<!-- Create Tables if does not exist -->
<prop key="hibernate.hbm2ddl.auto">create</prop>
</props>
</property>
<property name="annotatedClasses">
<list>
<value>org.oss.beans.Property</value>
<value>org.oss.beans.PropertyImage</value>
</list>
</property>
</bean>
<bean id="transactionManager"
class="org.springframework.orm.hibernate4.HibernateTransactionManager"
p:sessionFactory-ref="sessionFactory">
</bean>
With the seperate properties file:
jdbc.driverClassName=org.hsqldb.jdbcDriver
jdbc.url=jdbc:hsqldb:file:#{systemProperties['user.home']}\\db"
jdbc.dialect=org.hibernate.dialect.HSQLDialect
jdbc.username=sa
jdbc.password=
What changes needs to be made for the db to be initialized?

NULL Database Record Returned After Reading Database Record Immediately After Insert

I'm using Spring JMS with the JPA Hibernate implementation and I'm seeing an intermittent issue with a insert and then instant read of the same record.
web application flow:
-Data gets posted to my web applications web service and the data is sent to a Glassfish OpenMQ queue (STUInputQ below).
-com.api.listener.backoffice.STUMessageListener reads the STUInputQ queue and does a insert into our Oracle Database and then sends a message (with the new database primary key) to another queue (ArchiveQ below).
-com.api.listener.backoffice.StorableMessageListener reads the ArchiveQ queue and attempts to do an read of the database using the primary key of the database record that was inserted by com.api.listener.backoffice.STUMessageListener.
Problem:
Sometimes (about 18%) the read operation in StorableMessageListener returns null, even though the record does exist. It seems to me the insert commit hasn't processed before the read occurs even though the insert returns the sequence generated primary key. I've put a unix timestamp at the end of the method that inserts the data and the one that reads it and when the issue occurs the unix timestamps are the same, so it seems as though the read gets the message before the commit is final.
Temporary Solution:
I've added some logic to sleep the thread and that ensures that I never get a null with the database read. I don't really think the thread sleep is a long term solution. Any ideas on why it seems the STUMessageListener isn't able to commit the transaction before the StorableMessageListener reads it?
Dependencies:
hibernate-core.3.3.2.GA
hibernate-entitymanager-3.4.0.GA
spring 3.0.6.RELEASE
Java 1.5
Spring Configuration:
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:util="http://www.springframework.org/schema/util"
xmlns:context="http://www.springframework.org/schema/context"
xmlns:aop="http://www.springframework.org/schema/aop"
xmlns:tx="http://www.springframework.org/schema/tx"
xmlns:hz="http://www.hazelcast.com/schema/spring"
xsi:schemaLocation="
http://www.hazelcast.com/schema/spring
http://www.hazelcast.com/schema/spring/hazelcast-spring-2.5.xsd
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
http://www.springframework.org/schema/util
http://www.springframework.org/schema/util/spring-util-3.0.xsd
http://www.springframework.org/schema/context
http://www.springframework.org/schema/context/spring-context-3.0.xsd
http://www.springframework.org/schema/tx
http://www.springframework.org/schema/tx/spring-tx-3.0.xsd
http://www.springframework.org/schema/aop
http://www.springframework.org/schema/aop/spring-aop-3.0.xsd">
<!-- Generic -->
<context:annotation-config />
<context:component-scan base-package="myapp.api" />
<aop:aspectj-autoproxy/>
<!-- JPA -->
<bean class="org.springframework.orm.jpa.support.PersistenceAnnotationBeanPostProcessor" />
<tx:annotation-driven />
<bean class="org.springframework.dao.annotation.PersistenceExceptionTranslationPostProcessor" />
<bean id="entityManagerFactory" class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">
<property name="dataSource" ref="dataSource" />
<property name="jpaVendorAdapter" ref="jpaVendorAdapter" />
<property name="persistenceUnitName" value="MyApp" />
<property name="jpaProperties">
<props>
<prop key="hibernate.use_sql_comments">true</prop>
<prop key="hibernate.generate_statistics">true</prop>
<prop key="hibernate.archive.autodetection">class</prop>
<prop key="hibernate.cache.use_second_level_cache">true</prop>
<prop key="hibernate.cache.provider_class">com.hazelcast.hibernate.provider.HazelcastCacheProvider</prop>
<prop key="hibernate.cache.use_query_cache">true</prop>
<prop key="hibernate.cache.use_minimal_puts">true</prop>
</props>
</property>
</bean>
<hz:hazelcast id="instance">
<hz:config>
//rest of Hazelast config maps here
</hz:config>
</hz:hazelcast>
<hz:hibernate-region-factory id="regionFactory" instance-ref="instance"/>
<!-- Define JPA Provider Adapter -->
<bean id="jpaVendorAdapter" class="org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter">
<property name="showSql" value="true" />
<property name="generateDdl" value="true" />
<property name="databasePlatform" value="org.hibernate.dialect.OracleDialect" />
</bean>
<bean id="dataSourceTarget" class="oracle.jdbc.pool.OracleDataSource" destroy-method="close">
<property name="URL" value="jdbc:oracle:thin:#server:1525:name" />
<property name="user" value="test" />
<property name="password" value="123" />
<property name="connectionCachingEnabled" value="true" />
<property name="connectionCacheProperties">
<props merge="default">
<prop key="MinLimit">5</prop>
<prop key="MaxLimit">50</prop>
</props>
</property>
</bean>
<bean id="dataSource" class="org.springframework.jdbc.datasource.LazyConnectionDataSourceProxy">
<property name="targetDataSource" ref="dataSourceTarget"/>
</bean>
<bean id="transactionManager" class="org.springframework.orm.jpa.JpaTransactionManager">
<property name="entityManagerFactory" ref="entityManagerFactory"/>
<property name="dataSource" ref="dataSource"/>
</bean>
<tx:annotation-driven transaction-manager="transactionManager" proxy-target-class="false"/>
<bean id="genericDAO" class="myapp.api.dao.impl.GenericDAOImpl">
<constructor-arg>
<value>java.io.Serializable</value>
</constructor-arg>
</bean>
<bean id="springContextHolder" class="myapp.api.util.SpringContextHolder" factory-method="getInstance" />
<bean id="executionInterceptor" class="myapp.api.listener.backoffice.ExecutionInterceptor" />
<!-- JNDI-->
<bean id="jndiTemplate" class="org.springframework.jndi.JndiTemplate"/>
<!-- JMS -->
<bean id="jmsQueueConnectionFactory" class="org.springframework.jndi.JndiObjectFactoryBean">
<property name="jndiTemplate">
<ref bean="jndiTemplate"/>
</property>
<property name="jndiName" value="${jms.jndi.qconnectionfactory}">
</property>
</bean>
<bean id="myJMSConnectionFactory" class="com.api.model.vo.backoffice.OpenMqConnectionFactoryBean">
<property name="imqAddressList" value="${jms.imq.url}" />
<property name="imqDefaultUsername" value="${jms.imq.user}" />
<property name="imqDefaultPassword" value="${jms.imq.password}" />
<property name="imqHost" value="${jms.imq.host}" />
<property name="imqPort" value="${jms.imq.port}" />
</bean>
<bean id="stuMessageListener" class="com.api.listener.backoffice.STUMessageListener" />
<bean id="storeListener" class="com.api.listener.backoffice.StorableMessageListener"/>
<bean id="executionInterceptor" class="com.api.listener.backoffice.ExecutionInterceptor" />
<bean id="stuJmsContainer" class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="jmsQueueConnectionFactory"/>
<property name="destinationName" value="STUInputQ"/>
<property name="sessionTransacted" value="false"/>
<property name="messageListener" ref="stuMessageListener" />
<property name="concurrentConsumers" value="5" />
<property name="maxConcurrentConsumers" value="100" />
<property name="receiveTimeout" value="30000" />
<property name="cacheLevelName" value="CACHE_NONE" />
</bean>
<bean id="storeJmsContainer" class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="jmsQueueConnectionFactory"/>
<property name="destinationName" value="ArchiveQ"/>
<property name="sessionTransacted" value="false"/>
<property name="messageListener" ref="storeListener" />
<property name="concurrentConsumers" value="5" />
<property name="maxConcurrentConsumers" value="100" />
<property name="receiveTimeout" value="30000" />
<property name="cacheLevelName" value="CACHE_NONE" />
</bean>
</beans>
Persistence Configuration:
<?xml version="1.0" encoding="UTF-8"?>
<persistence version="1.0" xmlns="http://java.sun.com/xml/ns/persistence" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_1_0.xsd">
<persistence-unit name="com" transaction-type="RESOURCE_LOCAL">
</persistence-unit>
</persistence>
Classes that insert record:
public class STUMessageListener implements javax.jms.MessageListener{
#Autowired
StoringService storingService;
#Transactional
public void onMessage(Message message) throws RuntimeException {
try {
Object omsg = ((ObjectMessage) message).getObject();
if (omsg instanceof StorableMessage) {
StorableMessage storableMessage = (StorableMessage) omsg;
//StorableMessage insert into Database
storingService.store(storableMessage);
//jms logic here to send message to next queue (ArchiveQ)
}
catch (Throwable ex) {
throw new RuntimeException(ex);
}
}
#Service("storingService")
public class StoringServiceImpl{
#Autowired
MessagesDAO messagesDAO;
#Transactional
public StorableMessage store(StorableMessage storableMessage) {
messagesDAO.save(storableMessage);
}
}
#Repository("messagesDAO")
public class MessagesDAOImpl{
private Class<T> type
#PersistenceContext
EntityManager entityManager;
public void save(T object) {
entityManager.persist(object);
}
public T findById(Serializable id) {
return entityManager.find(type, id);
}
}
Classes that Read the Database Record:
public class StorableMessageListener implements javax.jms.MessageListener {
#Autowired
MessageDAO messageDAO;
#Transactional
public void onMessage(Message message) throws RuntimeException {
if (omsg instanceof StorableMessage) {
//this is where null is returned for the Messages object 18% of the time
//sleep thread by 1 second logic here helps eliminate the null Messages object
//uses same MessageDAO as above
Messages msg = messageDAO.findById(storableMessage.getMessageKey());
}
}

Try to change the annotation of the insert method as
#Transactional(propagation = Propagation.REQUIRES_NEW)
This will commit the insert as soon as the method finishes.

Spring Hibernate Database Connections

I am in the process of learning Hibernate, Spring and JPA.
I would like to know whether my database connections are closed automatically by Spring or not. When I looked at database table v$session I can find there are four JDBC Thin Client
sessions. So would like to know whether these connections are from my application.
Below is my applicationContext.xml. Any help is highly appreciable.
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:tx="http://www.springframework.org/schema/tx"
xmlns:context="http://www.springframework.org/schema/context"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
http://www.springframework.org/schema/tx
http://www.springframework.org/schema/tx/spring-tx-3.0.xsd
http://www.springframework.org/schema/context
http://www.springframework.org/schema/context/spring-context-3.0.xsd"
>
<context:component-scan base-package="net.test" />
<!-- Data Source Declaration -->
<bean id="DataSource" class="org.springframework.jndi.JndiObjectFactoryBean">
<property name="jndiName" value="jdbc/myDS"/>
</bean>
<bean
class="org.springframework.dao.annotation.PersistenceExceptionTranslationPostProcessor" />
<bean class="org.springframework.orm.hibernate4.HibernateExceptionTranslator" />
<!-- JPA Entity Manager Factory -->
<bean id="entityManagerFactory"
class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">
<property name="dataSource" ref="DataSource" />
<property name="packagesToScan" value="net.test.entity" />
<property name="jpaVendorAdapter">
<bean class="org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter">
<property name="showSql" value="true" />
<property name="generateDdl" value="false" />
<property name="databasePlatform" value="${jdbc.dialectClass}" />
</bean>
</property>
</bean>
<bean id="defaultLobHandler" class="org.springframework.jdbc.support.lob.DefaultLobHandler" />
<!-- Session Factory Declaration -->
<bean id="SessionFactory"
class="org.springframework.orm.hibernate4.LocalSessionFactoryBean">
<property name="dataSource" ref="DataSource" />
<property name="annotatedClasses">
<list>
<value>net.test.entity.Employee</value>
<value>net.test.entity.Department</value>
</list>
</property>
<property name="hibernateProperties">
<props>
<prop key="hibernate.dialect">org.hibernate.dialect.Oracle10gDialect</prop>
<prop key="hibernate.show_sql">true</prop>
<prop key="hibernate.query.factory_class">org.hibernate.hql.internal.classic.ClassicQueryTranslatorFactory
</prop>
</props>
</property>
</bean>
<tx:annotation-driven transaction-manager="txManager" />
<tx:annotation-driven transaction-manager="transactionManager" />
<bean id="txManager"
class="org.springframework.orm.hibernate4.HibernateTransactionManager">
<property name="sessionFactory" ref="SessionFactory" />
</bean>
<bean id="transactionManager" class="org.springframework.orm.jpa.JpaTransactionManager">
<property name="entityManagerFactory" ref="entityManagerFactory" />
</bean>
<!-- <tx:annotation-driven transaction-manager="txManager"/> -->
<context:annotation-config />
<bean id="hibernateStatisticsMBean" class="org.hibernate.jmx.StatisticsService">
<property name="statisticsEnabled" value="true" />
<property name="sessionFactory" value="#{entityManagerFactory.sessionFactory}" />
</bean>
<bean name="ehCacheManagerMBean"
class="org.springframework.cache.ehcache.EhCacheManagerFactoryBean" />
<bean id="mbeanServer" class="org.springframework.jmx.support.MBeanServerFactoryBean">
<property name="locateExistingServerIfPossible" value="true" />
</bean>
<bean id="jmxExporter" class="org.springframework.jmx.export.MBeanExporter"
lazy-init="false">
<property name="server" ref="mbeanServer" />
<property name="registrationBehaviorName" value="REGISTRATION_REPLACE_EXISTING" />
<property name="beans">
<map>
<entry key="SpringBeans:name=hibernateStatisticsMBean"
value-ref="hibernateStatisticsMBean" />
<entry key="SpringBeans:name=ehCacheManagerMBean" value-ref="ehCacheManagerMBean" />
</map>
</property>
</bean>
</beans>

Spring closes connections when the transaction is committed of rollbacked. But the connections are pooled connections, so closing them simply puts them back into the pool of connections, and doen't physically closes them.
The first goal is to be able to get a new connection from the pool extremely quickly, without having to recreate a new physical connection each time, because it's a costly operation.
The other goal is to be able to put a limit on the number of opened connections, to avoid bringing the database to its knees.
Note that the pool of connections, in your case, isn't handled by Spring, but by your application server, which makes it available from JNDI.

Trying to setup pooling with C3P0 Pooling in Hibernate with Spring WebFlow and MySQL

I am trying to setup pooling with C3P0 Pooling in Hibernate with Spring WebFlow and MySQL but I would like to have Hibernate managed the pools and not mysql so I setup my jdbc.properties file and my database.xml file and ran my project and checked the connections to the database and it looks like if I change the database.minPoolSize properties on the MySQL side it works but its not working if I change them on the Hibernate size. Can someone please review my files and let me know what you think:
jdbc.properties:
#
# database
#
database.driver=com.mysql.jdbc.Driver
database.url=jdbc:mysql://xxx
database.user=xxx
database.password=xxx
database.acquireIncrement=1
database.minPoolSize=5
database.maxPoolSize=25
database.maxIdleTime=1
#
# Hibernate
#
hibernate.dialect=org.hibernate.dialect.MySQL5Dialect
hibernate.show_sql=false
format_sql=false
hibernate.use_sql_comments=false;
#
# C3P0
#
hibernate.c3p0.min_size=5
hibernate.c3p0.max_size=25
hibernate.c3p0.timeout=600
hibernate.c3p0.max_statements=0
hibernate.c3p0.idle_test_period=300
hibernate.c3p0.acquire_increment=5
Here is my database.xml file:
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:context="http://www.springframework.org/schema/context"
xmlns:tx="http://www.springframework.org/schema/tx" xmlns:jdbc="http://www.springframework.org/schema/jdbc"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/context
http://www.springframework.org/schema/context/spring-context-3.0.xsd
http://www.springframework.org/schema/tx
http://www.springframework.org/schema/tx/spring-tx-3.0.xsd
http://www.springframework.org/schema/jdbc
http://www.springframework.org/schema/jdbc/spring-jdbc-3.0.xsd">
<context:property-placeholder location="classpath:jdbc.properties" />
<context:component-scan base-package="org.uftwf" />
<tx:annotation-driven transaction-manager="hibernateTransactionManager" />
<bean id="dataSource" class="com.mchange.v2.c3p0.ComboPooledDataSource"
destroy-method="close">
<!-- these are C3P0 properties -->
<property name="acquireIncrement" value="${database.acquireIncrement}" />
<property name="minPoolSize" value="${database.minPoolSize}" />
<property name="maxPoolSize" value="${database.maxPoolSize}" />
<property name="maxIdleTime" value="${database.maxIdleTime}" />
<property name="driverClass" value="${database.driver}" />
<property name="jdbcUrl" value="${database.url}" />
<property name="user" value="${database.user}" />
<property name="password" value="${database.password}" />
</bean>
<!-- <bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource"
destroy-method="close"> <property name="driverClassName" value="${database.driver}"
/> <property name="url" value="${database.url}" /> <property name="username"
value="${database.user}" /> <property name="password" value="${database.password}"
/> </bean> -->
<bean id="sessionFactory"
class="org.springframework.orm.hibernate3.annotation.AnnotationSessionFactoryBean">
<property name="dataSource" ref="dataSource" />
<property name="annotatedClasses">
<list>
<value>org.uftwf.schoolvisit.model.VisitModel</value>
<value>org.uftwf.schoolvisit.model.NameID_lookupModel</value>
<value>org.uftwf.schoolvisit.model.School_lookupModel</value>
</list>
</property>
<property name="hibernateProperties">
<props>
<prop key="hibernate.dialect">${hibernate.dialect}</prop>
<prop key="hibernate.show_sql">${hibernate.show_sql}</prop>
<prop key="hibernate.use_sql_comments">${hibernate.use_sql_comments}</prop>
<prop key="format_sql">${format_sql}</prop>
<prop key="hibernate.c3p0.min_size">10</prop>
<prop key="hibernate.c3p0.max_size">25</prop>
<prop key="hibernate.c3p0.timeout">600</prop>
<prop key="hibernate.c3p0.max_statements">0</prop>
<prop key="hibernate.c3p0.idle_test_period">300</prop>
<prop key="hibernate.c3p0.acquire_increment">5</prop>
</props>
</property>
</bean>
<bean id="hibernateTransactionManager"
class="org.springframework.orm.hibernate3.HibernateTransactionManager">
<property name="sessionFactory" ref="sessionFactory" />
</bean>
</beans>
why is C3P0 not using the hibernate propertys but using the mysql onces?

You're defining your C3P0 bean with the normal properties. Change ${database...} to the associated key that you want.

Develop Reference

Java is a programming language and computing platform first released by Sun Microsystems in 1995.

Use a fallback datasource for Hibernate in Spring - java

Related

Spring batch 3600 partitions long pause before starting

hsqldb file mode with spring, db not created

NULL Database Record Returned After Reading Database Record Immediately After Insert

Spring Hibernate Database Connections

Trying to setup pooling with C3P0 Pooling in Hibernate with Spring WebFlow and MySQL

Categories

Resources