I (after a while without touching JPA) started a project with Hibernate 4.1, JPA2 and Spring. It is a maven project with a common life cycle.
Upon running mvn test I get the expected result of not finding the required database objects (connection is successful as expected). All my researches and experiments, however, proved not enough to make something that seemed to be a common situation.
I expect to be able to have maven drop/create the schema the local development database while executing unit tests; I imagined that hibernate3-maven-plugin (version 3.0 ideally) should handle this but I didn't manage to have it working. I don't expect any automatic insertion of data (for this I could use DBUnit or even better have each test generate its own test data, but this plays no role here) but I do expect that the schema be refreshed on the test database, reflecting the current state of my annotated model classes. I suppose this would be bound to process-test-resources phase.
I expect to generate a (or set of) sql file with the (current) schema definition, but the best result I got reflects the issue described here: Maven + Spring + Hibernate: hibernate3-maven-plugin hbm2ddl fails for reason "Caused by: java.lang.NullPointerException" (no solution so far).
Am I missing something silly or is it really not possibile at this time?
I would be very happy if someone could provide me any of
proper documentation of how this is supposed to be achieved
an working example using hibernate 4
any guidelines of practical ways of achieve my goals with some other strategy.
If it's of any relevance, database is Postgres 9.1.
Thanks in advance.
One way of doing it is to use Arquillian. You can create a separate deployment package for each test or for a suite of tests, with it's own persistence.xml and datasources. Use the hbm2dll setting in the persistence.xml to either create-drop or update the schema:
<property name="hibernate.hbm2ddl.auto" value="create-drop" />
If you want to prepopulate the database, you can either add a import.sql to your deployment which will be executed by hibernate on application startup, or use the Arquillian Persistence extension.
Here is a complete arquillian test setup as example:
#RunWith(Arquillian.class)
public class MyTest {
#Deployment
public static Archive<?> createTestArchive() {
return ShrinkWrap.create(WebArchive.class, "myTest.war")
.addPackages(true, "com.example")
.addAsResource("testSeeds/seed.sql", "import.sql")
.addAsResource("in-mem-persistence.xml", "META-INF/persistence.xml")
.addAsWebInfResource("in-mem-datasource.xml");
}
One downside is that the in-container tests will be slower than simple unit tests.
I am not sure how well Arquillian is able to play nice with Spring, I have only used it for Java EE applications, so please tell if it is of any help.
Again an answer to my own question.
To achieve the result I wanted I had to chain in maven the execution of hibernate3-maven-plugin's hbm2ddl on process-classes phase with another execution of sql-maven-plugin on the process-test-resources phase. The first generates the sql for Hibernate's model and the second applies it to the database.
With hindsight, seems actually a decent and clean solution.
I'm still interested in knowing the best practices, if they differ from my solution, to achieve the goal of setting up the database for testing with maven from a Hibernate model.
Related
We have a microservice based on Spring Boot 2.7.6. For testing we employ dbunit/junit4 and a h2 database. We have some integrationtests which establish a database-baseline via a schema.sql and then some dbunit-xml-magic. Those tests are marked with #DirtiesContext.
My understanding of #DirtiesContext is, that after the test both the database and the Spring context should be discarded and rebuilt. Up to now this has worked well.
(Almost) out of the blue those tests fail. The error message is java.lang.IllegalStateException: Failed to load ApplicationContext with the cause, that a sequence cannot be created, because it already exists. This sequence is being created by the schema.sql. It is my understanding that the error message implies the databases are not discarded between the tests.
The current changes on the branch are purely business driven. While some classes have changed their packages (which seems to have some effect on the ordering during initialization by Spring), no changes on the testframework itself or the like were made.
I can't isolate the change which triggered the behavior but some experimentation showed, downgrading the h2 database from 2.1.212 to 1.4.199 relaxes the problem (fewer tests fail) and downgrading to 1.4.190 resolves the issue.
The question now is, is this some form of bug? If so, some bug in h2 2.1 oder in 1.4? Is my understanding incorrect, that #DirtiesContext should clean out the database?
I have a spring + hibernate application that uses postgres database. I need to write unit tests for the controllers. For tests I wanted to use h2 database but unfortunately test crashes during create-drop leaving me with information that bpchar data type is invalid. I wonder how to solve this issue so I could run tests.
I can't change my columns with bpchar to varchar, it need to stay as it is. I also tried to set postgresql mode but it didn't helped.
Am I right that the only solution I have is to use embedded postgres database in order to perform tests or is there any other approach that I could use?
Am I right that the only solution I have is to use embedded postgres database in order to perform tests or is there any other approach that I could use?
You try to use postgres-specific data type with h2 (which does not have it). Of course, it does not work.
If you cannot change type of this field - use embedded postgres in tests.
Actually you can do this in your application.properties to let H2 know:
spring.datasource.url=jdbc:h2:mem:testdb;INIT=CREATE TYPE BPCHAR AS CHARACTER NOT NULL
Also make sure auto configuration of the database is turned off for your test. You can do this by adding:
#AutoConfigureTestDatabase(replace = AutoConfigureTestDatabase.Replace.NONE)
public class MyTestForTableWithBpcharColumn {
One interesting approach to this issue is Test Containers
Since Postgres doesn't have an embedded mode, but you can use the aforementioned framework to start the docker container before the test, create a schema and apply migrations if you're using something like Flyway or Liquibase or integrate your custom solution.
The idea is that the container will be prepared and available to the test when it runs.
After the test passes (regardless of the actual result, success or failure) you can stop the container.
Firing up the container can be quite expensive (a matter of seconds), however you can take advantage of spring caching configurations during the tests, so when the first test in the module starts, the container is actually started, however, it gets reused between the tests and the test cases, since the application context doesn't get re-started.
Keeping the database clean between tests also becomes a trivial task due to Spring's #Transactional annotation that you put on a test case, so that spring artificially rolls back the transaction after each test. Since in Postgres, even DDL commands can be transactional it should be good enough.
The only limitation of this approach is that you should have a docker available on build machine or local development machine if you're planning to run these tests locally (on Linux and Mac OS its not a problem anyway, but on Windows you need to have Windows 10 Professional edition at least in order to be able to install docker environment).
I've used this approach in real projects and found it very effective for integration testing.
I use Play 2.3 with Hibernate.
On starting up the application the first time, I want to have some data inserted into the database as default values.
In my case I have an entity class "Studycourse". All tables are created through JPA on first run.
I use DB evolution (1.sql) to insert the default data, e.g.:
INSERT INTO studycourse (id, title) VALUES (1, 'Computer Science');
This works when using the normal "activator run" command. But if I do "activator test" and start a simple integration test with inMemoryDatabase(), I get following error:
[error] play - Table "STUDYCOURSE" not found; SQL statement: INSERT INTO studycourse (id, title) VALUES (1, 'Computer Science')
I guess, that the initial JPA setup is not done in the in-memory DB.
Question: Is there a best practice on how to do this?
The integration test looks like:
public class IntegrationTest {
#Test
public void test() {
running(testServer(3333, fakeApplication(inMemoryDatabase())), HTMLUNIT, new Callback<TestBrowser>() {
public void invoke(TestBrowser browser) {
browser.goTo("http://localhost:3333");
assertThat(browser.pageSource()).contains("Your new application is ready.");
}
});
}
}
Thanks in advance.
Your original question was essentially asking "How can I execute my JPA initialization steps in a test environment so that my in-memory database is populated when I run integration tests involving my database?".
My answer will not directly address that but it will summarize how we solve the same underlying issues you're trying to solve.
My interpretation of your objectives is that you want to:
Establish a good practice for database schema migration
Establish a common practice for database integration testing
Database Schema Migration
As I mentioned in my comment above, we use http://flywaydb.org for database schema migrations and it has been an outstanding tool. Flyway has an SBT plugin so you can run flywayClean and flywayMigrate right from activator to delete and re-initialize your database instantly.
Flyway supports sophisticated file name versions so that you can execute SQL scripts like v1.1.0.sql, v1.1.1.sql, and v1.2.0.sql. Flyway will also complain if you try to execute migration scripts that are not a pure improvement on the existing state of the database. This means we're using flyway to push database schema migrations to production, resting confident that this will fail if we've done something silly. Of course, we always take a DB backup right before the migration just to be safe.
Finally, Flyway will even let you execute java programs to populate the database in case you want to use service methods instead of just raw SQL.
Anyway, your choices here are basically Play evolutions, Flyway, or Liquibase.
In-Memory Database vs. Dev-Database
On this issue, I've seen two primary positions:
Never test on an in-memory database because then your tests won't
reveal the subtle differences between your in-memory database and
your production database, or
Use an in-memory database for local testing, but at least have your
build server use a dev database.
You can see, for example, the comments at the end of http://blog.jooq.org/2014/06/26/stop-unit-testing-database-code/.
Option #1 gives you higher speed overall, but delays the feedback time between writing bad code and getting a failing integration test.
Option #2 gives you slightly lower speed overall, but gives you immediate feedback on the real database.
As with most things in engineering, there is no "best" solution, just a set of tradeoffs which make the most sense for your team.
Choosing an ORM Layer
We initially began with Hibernate but eventually switched to http://jooq.org. See http://www.vertabelo.com/blog/technical-articles/jooq-a-really-nice-alternative-to-hibernate for a jOOQ-positive overview, and http://blog.jooq.org/2012/04/21/jooq-and-hibernate-a-discussion/ for a good discussion on the two.
Hibernate seemed attractive to us because it was so mature and so popular, but when we began running into classic SQL vs. Object-Oriented impedance mismatches like how to handle inheritance, Hibernate required a learning curve and some setup overhead.
We reasoned that, if we're going to incur that overhead at all, why not just use SQL directly to do the mappings? So, we switched to JOOQ and have been able to write some very clean, elegant, and testable code. If you're not too far down the hibernate path, I would encourage you to take a look at jOOQ.
If you're already deep into Hibernate, and it's working well for you, there's probably little value to switching.
Best Practices for Database Integration Testing
I wondered this exact question and posted about it at https://groups.google.com/forum/#!topic/jooq-user/GkBW5ZGdgwQ. Lukas, the author of jOOQ responded with some general remarks.
At this point, we integration test most of our DAO's and service classes. Our tests are run after flywayClean and flywayMigrate have been run. Then, each test is written to clean up after itself. The biggest issue is performance, which so far is not a problem, but may be an issue later.
I also posted on that and received a helpful answer. See https://groups.google.com/d/msg/play-framework/BgOCIgz_9q0/jBy8zxejPEkJ.
Disclaimer: we are close to launching our app but not yet running it in production, so others may have additional best practices to add.
I've written a JEE6 application using CDI and JPA. My tests are written in JUnit. I'd like to run the database tests against an in-memory HSQLDB database in order to make sure my JPQL (which I consider 'code') is tested. My motivation is that that changing a JPQL statement with a mocked out EntityManager would lead to successful test execution of the code unit.
I'm using Guice and Jukito to run other (non-jpa) tests.
Does anyone have an example for this? I've tried looking around and I've yet to find a good example or framework project to handle this.
Arquillian persistence? DBUnit?
This is a bit of an odd question, but it has been bothering me for a few months now. I have built a JPA-based web application using Wicket + Hibernate (built with Maven), and want to test the DAO layer directly. I created a specific src/test/resources/META-INF/persistence.xml file that I used for testing, but have been running into conflicts with WTP and the like. To get around these issues, I created a separate test project where the unit tests live. Is there a better way to manage unit tests for a JPA project without having duels between persistence files?
Addendum: Would other test frameworks (TestNG, for example) make this any easier?
You may want to try mockito. The test works like this:
You use mockito to "implement" EntityManager. Instead of the real code, you use the methods of mockito to say "if the application calls getReference(), then return this object". In the background, mockito will create a proxy instance which intercepts the Java method calls and returns the values which you specify. Calls to other methods will return null.
Mocking things like createQuery() works the same way but you first need to create a mockup of Query and then use the same approach as in getReference() (return the query mockup).
Since you don't use a real EM, you don't need a real persistence.xml.
A much more simple solution would be if you could set some property to change the name of the persistence.xml file but I don't think that this is possible.
Some other links that may help:
How to configure JPA for testing in Maven
Suggest a JPA Unit test framework
We use dual persistence.xml files for production and test runtimes but it is a classpath related issue only (we use Eclipse but do not rely on WTP plugins heavily). The only difference between the two is that the production version doesn't contain entity definitions.
We don't use a mocking framework to test JPA as this wouldn't add any value to our tests. The tests do run real data access with JPA that talks to PostgreSQL database.
Our approach to tests is based on Spring test framework for persistence layer: in-transaction testing. Our application is Spring-based but this approach is equally usable for arbitrary applications that want to take advantage of Spring test classes. The essence is that each test runs within a single transaction that never commits and at the end (in tearDown) it is automatically rolled back. This solves the problem of data pollution and test dependency in very nice unobtrusive and transparent way.
The Spring test framework is flexible to allow multi-transaction testing but these are special cases that constitute not more than 10% of tests.
We still use legacy support for JUnit 3.8 but new Spring TestContext Framework for JUnit 4 looks very attractive.
For setting up in-transaction test data we use in-house utility class that constructs business entities. Since it's shared between all tests the overhead to maintain and support it is greatly outweight by the benefits of having standard and reliable way to setup test data.
Spring DI helps to make tests concise and self-descriptive but it's not a critical feature.
Using Spring and Spring's unit testing is the best way to go. With spring, you don't require two persistence.xml's as your persistence.xml has nothing in it, everything is specified by spring (all we specify in our persistence.xml is the persistence-unit name) and thus you can change database configuration etc with spring.
And as topchef pointed out, spring's transaction based unit testing is great.
As mentioned here : http://www.devx.com/java/Article/36785/1954,
you can remove the following lines from your project's .settings/org.eclipse.wst.common.component to avoid deploying test resources with the web app.
<wb-resource deploy-path="/WEB-INF/classes" source-path="/src/test/java"/>
<wb-resource deploy-path="/WEB-INF/classes" source-path="/src/test/resources"/>
You can:
Have several persistence units
Have several persistence.xml and copy them on test, and restore them later
Setup your own properties on testing, and use mockito to return your custom entity manager factory
Use spring: https://www.baeldung.com/spring-testing-separate-data-source
The first two options are the most discussed in all suggested questions, and are by far the ones I like the least.
Solution 3. would look like this:
private EntityManager entityManager;
private static EntityManagerFactory entityManagerFactory;
#BeforeClass
public static void mainTestInitClass() {
Properties pros = new Properties();
// Override production properties
pros.setProperty("hibernate.dialect", "org.hibernate.dialect.H2Dialect");
pros.setProperty("hibernate.connection.driver_class", "org.h2.Driver");
pros.setProperty("hibernate.connection.username", "sa");
pros.setProperty("hibernate.connection.url", "jdbc:h2:mem:some_test_db;DB_CLOSE_DELAY=-1;MVCC=TRUE;DATABASE_TO_UPPER=false");
pros.setProperty("hibernate.hbm2ddl.auto", "create");
entityManagerFactory = Persistence.createEntityManagerFactory("your_unit", pros);
}
#Before
public void mainTestORMSetUp() throws Exception {
this.entityManager = entityManagerFactory.createEntityManager();
}
Now you have an entity manager available for every test. Use mockito to inject it where needed.
Solution 4: Use Spring Data+Spring Boot to setup the JPA, so you don't need the Entity Factory anymore, you simply use two different application.properties (one for main, and one for test) and then you use your defined Spring Entity Repository. Alternatively you can use different spring profiles (one for tests, other for production) which would end up allowing you to do the same. This solution is the one I use. Check the URL above for more details.