Develop Reference

Simulate / Mock a MySql Database for Unit Testing

For our applications we have created a series of healthchecks. One of these health checks checks whether or not a MySql database is up and running. In order to unit test this functionality I need to mock or simulate a database and run the health check against it.
To establish a connection to the MySql database being checked, we are using DriverManager.getConnection(). So we first tried to mock the return value of getConnection(). However, that is a static method and Mockito is unable to mock it. Then we turned to Powermock because it is able to mock static methods. Unfortunately, we found it to be poorly supported with JUnit5 and we had to clutter up our pom with a lot of dependencies. Not to mention the code was becoming overly complicated. In the end, we decide to look for other options besides mocking.
Basically, our health check just needs to make sure that the MySQL url, username, and password are correct and the database will accept a connection. Ideally, our unit tests would reflect these requirements.
So, our next idea is to use an In-Memory database, like this one from Baeldung. This seems like a great option but I'm worried it won't behave in the same way a real MySQL database will. Is there a way to create an In-memory MySQl database and establish a connection with DriverManager.getConnection()? This would effectively simulate our real life health check

One option is to use the mysql jcabi maven plugin: mysql.jcabi.com
Please note version > 0.7 doesn't work on Windows.
This will actually setup an instance of MySQL, so it allows for one to test very mysql specific stuff, though the first run is slow.
So, a couple of questions will help drive your answer: are you doing anything highly MySQL specific? Do you want your app to be portable to other DBs?
If the answer is yes to the first one and no to the second one, probably use the jcabi plugin.
If the answer is no to the first, and yes to the second, you're probably pretty safe using H2.
If the answer is yes to both, you've got bigger problems :).
I generally try to keep my code away from DB vendor specificity and use libraries like JPA to abstract this. I can then assume that JPA (and impl) has tested across the supported DBs, and I'm good enough to run unit and integration tests with an in-memory DB like H2.
I generally also have some end-to-end testing in a prod-like env with the same DB setup as prod, in order to cover any vendor-specific quirks.

TDD without local database?

When we develop a Rails application then we use a local database in our development environment, and make sure that our specs pass as part of TDD.
Is it a norm to not use a local database similar to Sqlite while doing TDD in Java? I have been told in-memory database(HSQL) is all that is needed for running unit and integration tests. Is this a standard practice being followed?
We use Sqlite in our Rails application for local development and for running our Rspecs. But my question is for Java development. We are working on rewritting a part of our application in Java. I have been told that you do not need any database for development if you write integration tests covering all functionality. And have been told that HSQL is sufficient for that. As I am used to having database for local development in Rails, I am wondering how you debug any issues later on? It is quite helpful to analyze any issues if we can replicate the data and scenario in local environment. How do you do same in Java/Spring if you do not use any database for development environment and rely completely on HSQL for testing?

For me, I never use any databases including HSQLDB to write an unit-test.
I prefer to create some interfaces like as: *Repository. and let's the SUT communicate with it. and then I write some implementation class let them implement the interface which I have created. and the classes hierarchy looks like below:
<<uses>>
SUT ---------------> Repository
^
| <<implement>>
|
|--------|--------|-------|
| | | |
JPA Hibernate JDBC .etc
this approach is known as Separation of Concerns. the application domain is a concern, data accessing is another concern. following this approach result in many plug-compatible components and independent modules, such as: domain, jpa, jdbc, and .etc, but the important thing is that will make your test is more testable.
Then I use Test Doubles to mock/stub out its collaboration in unit-test to testing them are work together as expected. the pseudo-code like as below:
repo = mock(Repository.class);
SUT it = new SUT(repository);
when(repo.find(id)).thenReturn(entity);
assert it.exercise() == expectedResult;
assert it.currentState == expectedState;
But you must write some integration test using database to testing each Repository implementation that operate on the third-party api. it is called by Martin: Test Isolation.

The answer to your question: is very common to have your test environment database as close as the development environment as possible.
I suppose that you are preoccupied with performance, there are more crucial things that you could improve before considering having an in-memory database.
Usually while TDD-ing you would only run the tests involved and later run your whole suite to check that you didn't break anything. If you are using Rspec you could use tags.
Another important thing is to clean the database at the beginning of every test since tests should be isolated and never depend on the result of previous tests. This will improve complex search queries that you could have in your system. there is a gem that could help you here.
Finally, if you are using some sort of continuous integration tool remember to set it up using rake db:schema:load instead of rake db:migrate. This will run your schema file as a single migration instead of running each single migration every time you commit. (Remember to keep this version-controlled and always up to date)

You are getting terminology wrong. TDD is about writing test cases in general. But most of the time, and also in your question, one thinks about using TDD for unit testing.
And unfortunately, terms are not very clear. When you turn to wikipedia, you find there (my words): "anything you do to test a piece of software" can be called a unit test.
But that isn't helpful. You should rather look for definitions such as here. And the main aspect there: unit tests work in isolation. Quoting from that link:
Runs in memory (no DB or File access, for example)
Thus:
when doing unit testing, you should not use any database
when you integration tests, you want to ensure that your solution works "end to end". In that sense you might be using a special instance of your database, but not a different kind of database.

Play framework with JPA initial insertion

I use Play 2.3 with Hibernate.
On starting up the application the first time, I want to have some data inserted into the database as default values.
In my case I have an entity class "Studycourse". All tables are created through JPA on first run.
I use DB evolution (1.sql) to insert the default data, e.g.:
INSERT INTO studycourse (id, title) VALUES (1, 'Computer Science');
This works when using the normal "activator run" command. But if I do "activator test" and start a simple integration test with inMemoryDatabase(), I get following error:
[error] play - Table "STUDYCOURSE" not found; SQL statement: INSERT INTO studycourse (id, title) VALUES (1, 'Computer Science')
I guess, that the initial JPA setup is not done in the in-memory DB.
Question: Is there a best practice on how to do this?
The integration test looks like:
public class IntegrationTest {
#Test
public void test() {
running(testServer(3333, fakeApplication(inMemoryDatabase())), HTMLUNIT, new Callback<TestBrowser>() {
public void invoke(TestBrowser browser) {
browser.goTo("http://localhost:3333");
assertThat(browser.pageSource()).contains("Your new application is ready.");
}
});
}
}
Thanks in advance.

Your original question was essentially asking "How can I execute my JPA initialization steps in a test environment so that my in-memory database is populated when I run integration tests involving my database?".
My answer will not directly address that but it will summarize how we solve the same underlying issues you're trying to solve.
My interpretation of your objectives is that you want to:
Establish a good practice for database schema migration
Establish a common practice for database integration testing
Database Schema Migration
As I mentioned in my comment above, we use http://flywaydb.org for database schema migrations and it has been an outstanding tool. Flyway has an SBT plugin so you can run flywayClean and flywayMigrate right from activator to delete and re-initialize your database instantly.
Flyway supports sophisticated file name versions so that you can execute SQL scripts like v1.1.0.sql, v1.1.1.sql, and v1.2.0.sql. Flyway will also complain if you try to execute migration scripts that are not a pure improvement on the existing state of the database. This means we're using flyway to push database schema migrations to production, resting confident that this will fail if we've done something silly. Of course, we always take a DB backup right before the migration just to be safe.
Finally, Flyway will even let you execute java programs to populate the database in case you want to use service methods instead of just raw SQL.
Anyway, your choices here are basically Play evolutions, Flyway, or Liquibase.
In-Memory Database vs. Dev-Database
On this issue, I've seen two primary positions:
Never test on an in-memory database because then your tests won't
reveal the subtle differences between your in-memory database and
your production database, or
Use an in-memory database for local testing, but at least have your
build server use a dev database.
You can see, for example, the comments at the end of http://blog.jooq.org/2014/06/26/stop-unit-testing-database-code/.
Option #1 gives you higher speed overall, but delays the feedback time between writing bad code and getting a failing integration test.
Option #2 gives you slightly lower speed overall, but gives you immediate feedback on the real database.
As with most things in engineering, there is no "best" solution, just a set of tradeoffs which make the most sense for your team.
Choosing an ORM Layer
We initially began with Hibernate but eventually switched to http://jooq.org. See http://www.vertabelo.com/blog/technical-articles/jooq-a-really-nice-alternative-to-hibernate for a jOOQ-positive overview, and http://blog.jooq.org/2012/04/21/jooq-and-hibernate-a-discussion/ for a good discussion on the two.
Hibernate seemed attractive to us because it was so mature and so popular, but when we began running into classic SQL vs. Object-Oriented impedance mismatches like how to handle inheritance, Hibernate required a learning curve and some setup overhead.
We reasoned that, if we're going to incur that overhead at all, why not just use SQL directly to do the mappings? So, we switched to JOOQ and have been able to write some very clean, elegant, and testable code. If you're not too far down the hibernate path, I would encourage you to take a look at jOOQ.
If you're already deep into Hibernate, and it's working well for you, there's probably little value to switching.
Best Practices for Database Integration Testing
I wondered this exact question and posted about it at https://groups.google.com/forum/#!topic/jooq-user/GkBW5ZGdgwQ. Lukas, the author of jOOQ responded with some general remarks.
At this point, we integration test most of our DAO's and service classes. Our tests are run after flywayClean and flywayMigrate have been run. Then, each test is written to clean up after itself. The biggest issue is performance, which so far is not a problem, but may be an issue later.
I also posted on that and received a helpful answer. See https://groups.google.com/d/msg/play-framework/BgOCIgz_9q0/jBy8zxejPEkJ.
Disclaimer: we are close to launching our app but not yet running it in production, so others may have additional best practices to add.

in-memory DBs evaluation

I am trying to increase the overall Integration test execution time and I am currently evaluating various in-memory db solutions. The idea is to have DAOs hit in-mem db during the tests as opposed to hitting a real DB. This is a java app using Hibernate for persistence.
I'd be interested to see your experience with one of these products H2, Derby, HSQLDB, Oracle Berkeley DB.
Some of my concerns are: will in-mem DBs be able to execute stored procedures, custom native sql? Can you selectively choose which one of your services should hit real DB vs in mem DB?
And overall, since this approach involves DB bootstrapping(pre-load/pre-create all tables with data) I am now thinking if it'd be simply easier to just mock out the DAO layer and not even worry about all the unknown problems that in mem DB may bring...
thanks.

My suggestion is to test everything, including the DAO layer as you mention. But see if you can test it in pieces. Services, DAOs, UI.
For service layer testing, mock out the DAOs. That way the service layer tests are independent of whether the DAOs are working. If the service layer tests are using DAOs and using a real database then I'd argue that it's not really a Unit test but an Integration test. Although those are valuable too, if they fail it doesn't pinpoint the problem like a Unit test.
For our DAO layer tests we use DbUnit with HSQLDB. (Using Unitils helps if you are using Spring/Hibernate/DbUnit to tie it all together.) Our DAO tests execute nice and quickly (which is important when you have 500+ tests). The memory db schema is built from our schema creation scripts so as a side effect we are testing those as well. We load/refresh a known set of data from some flat files into the memory database. (Compared to when we were using the DEV database and some data would get removed which then broke tests). This solution is working great for us and I would recommend it to anyone.
Note, however, that we are not able to test the DAO that uses a stored proc this way (but we only have one). I disagree somewhat with the poster who mentioned that using different databases is "bad" -- just be aware of the differences and know the implications of doing so.
You didn't mention if you are using Hibernate or not -- that is one important factor in that it abstracts us away from modifying any SQL that may be specific to Oracle or SQLServer or HSQLDB which another poster mentioned.

Mock out the DAO layer.
Despite what some claim unless you are just using trivial sql the subtle implementation differences and differing feature sets between databases will limit what you can do (stored procedures, views etc.) and also to some extent invalidate the tests.
My personal mocking framework of choice is Mockito. But there are lots that do the job and mocking out the DAO is standard practice so you'll find lots of documentation.

It is bad idea to have different databases for unit-testing and for production.
BTW, testing in real database should be fast, probably you are doing something wrong in your tests.

I just came across Oracle Times Ten in mem db.
http://www.oracle.com/technology/products/timesten/index.html
This may seem like possibly the most painless solution. Since no additional mocking/configuration is required. You still have all of your Integration tests intact hitting the DB but now the data is delivered faster. What do you guys think ?

Testing SQL query on Oracle which includes a remote database

Our development databases (Oracle 9i) use a remote database link to a remote shared database.
This decision was made years ago when it wasn't practical to put some of the database schemas on a development machine - they were too big.
We have certain schemas on the development machines and we make the remote schemas look local by using Oracle's database links, together with some synonyms on the development machines.
The problem I have is that I would like to test a piece of SQL which joins tables in schemas on either side of the database link.
e.g. (a simplified case):
select a.col, b.col
from a, b
where a.b_id = b.id
a is on the local database
b is on the remove database
I have a synonymn on the locale DB so that 'b' actually points at b#remotedb.
Running the query takes ages in the development environment because of the link. The queries run fine in production (I don't think the Oracle cost based optimiser can cope very well with database links).
We have not been very good at writing unit tests for these types of queries in the past - probably due to the due to the poor performance - so I'd like to start creating some tests for them.
Does anyone have any strategies for writing a unit test for such a query, so as to avoid the performance problems of using the database link?
I'd normally be looking at ways of trying to mock out remote service, but since all this is in a SQL query, I can't see anyway of easily mocking out the remove database.

You should create exact copies of all the schema you need from production on development but without all the data. You should populate the schema with enough data so you can do a proper test. You can also manipulate the optimizer to behave on the test system to be like production by exporting the statistics from the production server and importing them to the development database for the schemas you are duplicating. That way the query will run with the data set you've made but the query will optimize with plans that is similar to that of production. Then you can estimate theoretically how it will scale on production.

Copy the relevant data into your development database and create the tables locally.
Ideally, just build a test case which tells you:
The SQL is correct (it parses)
It operates correctly with a few rows of test data
Don't fall for the "let's copy everything" because that means you'll have no idea what you're testing anymore (and what you're missing).
If in doubt, create a table b with just a single record. If you get an error in this area, add more rows as you learn where it can fail.
If you want to take this to the edge, create the test table (with all data) in a unit test. This way, you can document the test data you're using.
[EDIT] What you need is a test database. Don't run tests against a database which can change. Ideally, the tests should tear down the whole database and recreate it from scratch (tables, indexes, data, everything) as the first step.
In this test database, only keep well defined test data that only changes by defining new tests (and not by someone "just doing something"). If you can, try to run your tests against an in-memory database.

I would suggest materialized views. These are views that store remote data locally.

In theory to do the unit-testing you can work with any set of controlled data created and designed based on your test-cases. It doesn't have to be your live or development system. That's assuming your unit is portable enough. You would test it with your current databases/application when you come to integration testing, which might as well be on the live system anyway (so no db links will be required - I understand your live databases are in one place).
What I'm trying to say, is that you can/should test your unit (i.e. your component, query or whatever you define as a unit) on a controlled set of data that would simulate different 'use cases' and once you complete your testing to satisfactory results, then you can proceed to integration + running integration tests.
Integration tests - you could run this in the live environment, but only after you've proved by unit-testing that your component is 'bullet-proof' (if that's OK with your company's approach/philosophy :) - sys admin's reaction:"Are you flippin creazy?!")
If you are trying to go back in time and test already implemented units, then why bother? If they've been in a production use for some time without any incidents then I would argue that they're OK. However, there's always a chance that your unit/query might have some 'slowly ticking time bomb' effect on the side (cumulative effect over time). Well, analyse the impact is the answer.