Develop Reference

HSQLDB with Postgresql: type not found or user lacks privilege: BIGSERIAL

I'm building Spring+Hibernate Java Application. I wanted to add some integration tests, done in in-memory database.
So, my normal database is Postgresql, and I populate it using .sql scripts run with flyway plugin. ID fields are set to BIGSERIAL. I wanted to use in-memory database, to resemble my original database, and then try to test some classes with it. I managed to configure preety much everything(I hope so), but when I run the test class itself I get error with CREATE TABLE scripts:
Caused by: org.hsqldb.HsqlException: type not found or user lacks privilege: BIGSERIAL
I found out that I should configure HSQLDB, to enable Postgresql compability.
Use SET DATABASE SQL SYNTAX PGS TRUE or the equivalent URL property sql.syntax_pgs=true to enable the PostgreSQL's non-standard features.
I use persistence.xml to define normal and test persistence unit. This is fragment responsible for defining test persistence unit:
<persistence-unit name="testJPA">
<properties>
<property name="hibernate.archive.autodetection" value="class, hbm"/>
<property name="hibernate.dialect" value="org.hibernate.dialect.HSQLDialect"/>
<property name="hibernate.show_sql" value="true"/>
<property name="hibernate.connection.driver_class" value="org.hsqldb.jdbcDriver"/>
<property name="hibernate.connection.username" value="sa"/>
<property name="hibernate.connection.password" value=""/>
<property name="hibernate.connection.url" value="jdbc:hsqldb:mem:butterfly;sql.syntax_pgs=true"/>
<property name="hibernate.hbm2ddl.auto" value="create"/>
</properties>
</persistence-unit>
Then I use configuration class for tests:
#Configuration
#EnableAutoConfiguration
#ComponentScan(basePackages = {"core.utilities"} )
#EnableTransactionManagement
public class TestsInitializer {
#Bean
public LocalEntityManagerFactoryBean entityManagerFactory() {
LocalEntityManagerFactoryBean factoryBean = new LocalEntityManagerFactoryBean();
factoryBean.setPersistenceUnitName("testJPA");
return factoryBean;
}
}
And in testclass itself:
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration(classes = { TestsInitializer.class }/*, loader = AnnotationConfigContextLoader.class*/)
#Transactional
public class GenreBATest {
#Autowired
private GenreBA genreBA;
#Test
public void testFindAllAccounts() {
//whatever
}
}
I added the required property at the end of URL, found examples of this exact property across the internet, but it does not resolve my problem.
I'm still getting: Message : type not found or user lacks privilege: BIGSERIAL
What am I doing wrong?

Ok, I found out that scripts were run by flyway ignoring all possible syntax commands from persistence.xml. And flyway run with HSQLDB because I used #EnableAutoConfiguration in test configuration class without excludes :) I learned through hard way that Spring Boot already configures Hibernate to create schema based on entities for an in-memory database. So I don't need to use database scripts at all, and they were run by accident.

Junit Hibernate creates database schema multiple times

I'm working on a Java Spring+Hibernate project and we have a Junit setup in order to unit-test our code.
The problem I face right now is that I don't understand why Hibernate (with Spring-boot) creates the database schema 2 times before the tests actually run. The sequence is as follows:
Alter tables drop all foreign keys
Drop tables if exist
Create tables
Alter tables add constraints (like FK)
Alter tables drop all foreign keys
Drop tables if exist
Create tables
Alter tables add constraints
Execute all tests
My questions is, if to be more specific: Why points 3-6 including are executed?
Why simply not to execute 1,2,7,8,9. Why do I want this? Because it takes precious time and I don't understand why do I need this.
Below is my persistence configuration:
<persistence-unit name="localContainerEntityForTest">
<description>Spring JPA LocalContainerEntityManagerFactoryBean</description>
<provider>org.hibernate.ejb.HibernatePersistence</provider>
<properties>
<property name="hibernate.dialect" value="org.hibernate.dialect.MySQLDialect"/>
<property name="hibernate.hbm2ddl.auto" value="create"/>
<property name="hibernate.implicit_naming_strategy" value="legacy-jpa"/>
<property name = "hibernate.show_sql" value = "true" />
<property name="javax.persistence.jdbc.driver" value="com.mysql.jdbc.Driver"/>
<property name="javax.persistence.jdbc.url" value="jdbc:mysql://localhost/myApp?createDatabaseIfNotExist=true"/>
<property name="javax.persistence.jdbc.user" value="hibernate"/>
<property name="javax.persistence.jdbc.password" value="password"/>
</properties>
</persistence-unit>
Also, below are the annotations I use for every unit test class:
#RunWith(SpringJUnit4ClassRunner.class)
#SpringApplicationConfiguration(classes = JPAConfigurationTestEnviorement.class)
#WebAppConfiguration
#FixMethodOrder(MethodSorters.NAME_ASCENDING)

I am fairly familiar with Hibernate, but less so with Spring Boot. My guess is that Hibernate is directly responsible for one create-alter-drop cycle and Spring Boot is responsible for the other one. If you are morally opposed to your current setup, then you can try updating your Hibernate XML to the following:
<property name="hibernate.hbm2ddl.auto" value="validate"/>
By choosing the validate option, you are instructing Hibernate to only validate your existing schema and to not create or drop any tables. The trick here is that I'm not sure which of the two cycles you want to remove (nor did you tell us).

Problem found, after starting the process to upgrade the spring boot version and I observed some weird code that was not telling me anything before. The reason was the creation of EntityManagerFactory before defining the LocalContainerEntityManagerBean:
#Bean
public LocalContainerEntityManagerFactoryBean getEntityManagerFactoryBean() {
Persistence.createEntityManagerFactory("localContainerEntityForTest");
LocalContainerEntityManagerFactoryBean lcemfb = new LocalContainerEntityManagerFactoryBean();
lcemfb.setPersistenceUnitName("localContainerEntityForTest");
lcemfb.setPackagesToScan("com.mybasepackage");
lcemfb.setPersistenceXmlLocation("classpath:/META-INF/persistence.xml");
return lcemfb;
}
After removing the line:
Persistence.createEntityManagerFactory("localContainerEntityForTest");
the cycle disappeared.

Ehcache second level cache not working with JPA and Hibernate?

EDIT: I should state that I've been researching this for a couple of days now - there is plenty of information on "how to configure ehcache with Hibernate" but they mostly do not refer to using JPA and annotations - they either refer to pure Hibernate or to configuration through XML. (Just want to make it clear that I have already been around the internet on this problem.) I'm using JPA and annotations, so most of these configuration guides refer to files I don't have in my app (like hbm.xml files).
I have an app that is using Hibernate 3.6.10.FINAL, Spring Data 1.3.2.RELEASE, and Spring version 3.2.1.RELEASE. I'm trying to get the second-level caching working in hibernate. According to the documentation, I can do that simply by including the following dependency and configuration:
(POM.XML)
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-ehcache</artifactId>
<version>${hibernate.version}</version>
</dependency>
(PERSISTENCE.XML)
<property name="hibernate.cache.provider_class" value="org.hibernate.cache.EhCacheProvider" />
<property name="hibernate.cache.use_second_level_cache" value="true" />
<property name="hibernate.cache.use_query_cache" value="true" />
<property name="hibernate.generate_statistics" value="true" />
I've annotated one of my entity classes using the javax.persistence.Cacheable annotation and tried to view the statistics in a JUnit test:
public void cacheTest() {
RandomDataGenerator randomData = new RandomDataGenerator();
for (int i = 0; i < 10; i++) {
AppMaster master = masterService.findOne(randomData.nextLong(1, 10));
logger.debug(String.format("Read one record back = %1$d, %2$s", master.getApplicationId(), master.getApContact()),AppMasterServiceTest.class);
// Get statistics from hibernate session
Session session = (Session)masterService.getEntityManager().getDelegate();
Statistics statistics = session.getSessionFactory().getStatistics();
logger.debug(String.format("Second level stats = %1$d, %2$d, %3$d",
statistics.getSecondLevelCachePutCount(),
statistics.getSecondLevelCacheHitCount(),
statistics.getSecondLevelCacheMissCount()), AppMasterServiceTest.class);
}
But the statistics appear to always be zero.
2013-11-11 11:20:33,908 DEBUG [main] test.service.AppMasterServiceTest - Second level stats = 0, 0, 0
Can anyone help me diagnose this?

javax.persistence.Cachable requires you set ENABLE_SELECTIVE in your persistence.xml. You should include a line like the following:
<shared-cache-mode>ENABLE_SELECTIVE</shared-cache-mode>
Check out the docs on http://docs.oracle.com/javaee/6/api/javax/persistence/Cacheable.html
Also, take a look at this related question: How to use JPA2's #Cacheable instead of Hibernate's #Cache

How will this connection pooling scenario pan out?

We have a web application that uses C3P0 to pool our connections. We inject C3P0 as a data source into JdbcTemplate. You can see how we do this here:
<bean id="dataSourceDev" class="com.mchange.v2.c3p0.ComboPooledDataSource">
<property name="driverClass" value="${databasedev.driver}" />
<property name="jdbcUrl" value="${databasedev.url}"/>
<property name="user" value="${databasedev.username}"/>
<property name="password" value="${databasedev.password}"/>
<property name="initialPoolSize" value="5" />
<property name="minPoolSize" value="5" />
<property name="maxPoolSize" value="1000" />
<property name="acquireIncrement" value="5" />
<property name="maxStatements" value="1000" />
<property name="maxStatementsPerConnection" value="1000"/>
<property name="maxIdleTime" value="10800"/> <!-- 3 hours -->
</bean>
<bean id="jdbcTemplate" class="org.springframework.jdbc.core.JdbcTemplate">
<constructor-arg>
<ref bean="dataSourceDev" />
</constructor-arg>
</bean>
<bean id="someDaoBean" class="com.gedi.platform.dao.SomeDaoClass">
<property name="jdbcTemplate" ref="jdbcTemplate" />
</bean>
<bean id="someResourceClass" class="com.gedi.platform.SomeResourceClass">
<property name="someDao" ref="someDaoBean" />
</bean>
You can see that it's a Java EE web application - it uses Jetty as its application server. My question is, how does Jetty instantiate our beans, and how will that affect connection pooling? If we have dozens of users using the web site at different times, will all of these users be placed in the same connection pool? Or is there only one connection pool per client, in which every HTTP client creates new instances of Resource, DAO, JdbcTemplate and C3P0?
Am I being clear? What I want to have is one connection pool for all HTTP requests, regardless of whether they come from web browsers originating in Boston or New Zealand. That way, the connection pool is exerting its maximum effects. However, if a new connection pool is instantiated for every HTTP client, then the pooling doesn't end up being much of an improvement.
Edit
An important tidbit - We use the Jersey reference implementation of JAX-RS to produce a RESTful interface. So our servlet dispatches requests through Jersey which finds a suitable Resource class/method to handle them. I wonder whether Jersey re-instantiates these classes on every request, or keeps one instance of them at all times.

Neither Jersey nor Jetty are relevant here. Spring is important here. And in Spring every bean (like your dataSourceDev, jdbcTemplate and someDaoBean) are singletons. That means when Spring application context starts, it will creatly exactly one instance of each of them.
That means no matter what uses your DataSource (web request, background job, etc.), the same instance (thus the same connection pool) is used. You are right that if connection pool was created per each request, it would not have been much of an improvement. Actually it would be much, much slower.
But in your case (and this is how 99% of web applications work) all code requiring database access will compete and reuse the same connections (or wait if none available). BTW make sure your database can actually handle 1000 concurrent connections.

Spring creates the beans and caches them, so unless you have specified the beans as prototype scoped(which creates a new bean for each request), all bean are singleton's by default. Jetty doesn't interfere.
When a request comes in, the DispatcherServlet catches the request and hands it off to the appropriate handler. The handler is the same bean if it has not been declared as a prototype bean.
You understood the connection pool correctly. This is exactly why the concept was created. It doesn't matter where the request came from, the maximum number of connections to the database at any point in time will be the one you have defined in the maxPoolSize property.

Spring #transactional behaving strangely with mode=aspectj

I've been trying to get Spring transactions working properly in my application for days, and now I have nowhere to turn but to the community. Thanks in advance for your help.
My application has specific requirements where I need to annotate a protected overridden method with #Transactional and call this method from the abstract parent class. From what I've read, I can not use proxies, and must use mode=aspectj.
First, some configuration:
<bean id="transactionManager"
class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
<property name="dataSource" ref="dataSource" />
</bean>
<tx:annotation-driven transaction-manager="transactionManager"
proxy-target-class="false" mode="aspectj" />
<context:load-time-weaver />
<bean name="ID_DataAccessor" class="dal.DataAccessor">
<constructor-arg ref="dataSource" />
</bean>
<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource"
destroy-method="close">
<property name="driverClassName" value="com.mysql.jdbc.Driver" />
<property name="url" value="jdbc:mysql://localhost" />
.....
<property name="defaultAutoCommit" value="true" />
</bean>
If I use the #Transactional annotation on a method in ID_DataAccessor, then the connection is transactional (DataSourceUtils.isConnectionTransactional(c, dataSource) == true). This is useless to me though, because the transaction commits as soon as the method is completed. When I use #Trasnactional on a method that calls a method in DataAccessor, the connection aquired in DataAccessor (from DataSourceUtils.getConnection(dataSource)) is NOT transactional. Additionally, I put a breakpoint in DataSourceTransactionManager.doGetTransaction(), and this is only called if the #Transactional annotation is directly on the method acquiring the connection. Forgive me for overloading the terminology, but it appears that the #Transactional annotation does not propagate up the stack.
I'm completely baffled. Please help! :)

I just about gave up on this one. It was quite non-obvious. Your problem is caused by using the EndOfDayProcess class in your entry point. It's a problem because it causes the class to be loaded, and more generally, it'll extend to any of your *Process and *EntryPoint classes. At that point in your code, Spring hasn't been started yet, so its load-time weaver isn't active, and the class gets loaded normally, without the transaction code woven into it. Of course, classes are only loaded once, so it's there to stay, and when you create the bean in Spring, it's a bean whose type doesn't have anything to do with transactions, except it has #Transactional annotations on it. By then, there's nobody there to see them anymore, though. I see two possible (good) solutions:
Switch to build-time weaving. Then it's impossible to load a class before it's woven.
Change the way you determine which *Process bean to use based on the entry point. Use anything you like except for something that causes the *Process class to be loaded.