I am developing a very large scale J2EE application and we chose to use Derby as an embedded database for junit testing since hitting the actual prod database will slow down our tests. When I bootstrap my application, the Derby DB will create all the tables so I can run JDBC queries against it. It works fine but the drawback is I cannot actually query any of the tables except through JDBC calls at runtime, so if I need to make changes to my queries, I need to stop the app, modiify my query statements, then restart the application and run in debug. This process makes it very difficult when it comes to analyzing complex queries. Does anyone know of some kind of Derby plugin that can help me to query the DB without doing it through my java code?
If you are using Maven for your build, you can use the derby-maven-plugin, which I wrote and is available on GitHub and via Maven Central. It will take care of starting and stopping the database for you before your tests. You will need to populate this database, yourself of course. You will also have the database in your target/derby folder after the tests execute, so you can always query the data yourself afterwards. This will help you work in a separate development environment which doesn't affect the production database.
You can check here for my answer to a similar question.
Related
I am using Spring 4.1.6, and I have my service working fine with Hibernate. In the root of the project I've got my schema.sql which is being run every time I run the server. The problem is that first time I ran the server, I put some data in db, and when I restarted it, the script was executed again and I lost all that data that I loaded before restart.
So, I think that I have two options two solve this problem:
Edit sql script to execute all queries just in case they do not exist (which would be more laborious since I have to edit the script every time I export my db)
Tell hibernate, by some way, to execute sql script just in some cases. That would be great if there existed some config that executes the script just in case the data base doesn't exist.
Do you know if this is even possible? Thanks in advance.
It sounds like this is the perfect use-case for a tool called Liquibase. This is basically a version control tool for your database which allows you to define changes to your schema and/or data and ensures that these changes are only applied once.
It's incredibly useful if multiple people are changing the same database schema and ensures that your database is always valid for the version of the code that you have checked out/released etc.
I read a lot of posts like:
querying embedded database in netbeans using derby
But still I'm having trouble to understand embedded databases.
1) I create a Derby database on Netbeans and I can create tables, link the database to a form and submit the data and update the records with no problem.
2) The problem arises when I want to make the program portable. I apply Clean and Build, then copy the dist folder and also copy the libraries, database, etc ... but when running the program does not recognize the database
3) I read in several places that it is appropriate that the database is created by code using something like
String host = "jdbc: derby: // localhost: 1527 / EmployeesCreateTrue; create = true"
and not creating the database on Netbeans Service...
If I do this procedure with code the database is created but it does not appear or does not allow me to connect from NetBeans and I wish I could fix it to create tables from NetBeans and not from code.
4) I read manuals "how to import a database from Derby to NetBeans" and it doesn't work...
Question: What is the best way to create a database, tables and connect to NetBeans for the final application to be easily portable?
1) Create the database on Netbeans with the wizzard?
or
2) Just plain code on the application?
I don't understand precisely what you mean by "the database is created but it does not appear."
I think if you were to explain that precisely, the community could probably help you.
There are three common reasons for "table does not exist" when you think you've created the tables; I've explained those cases in this answer: Is it necessary to create tables each time you connect the derby database?
Please let us know more information about your situation so that we can help you better understand the behavior of your application.
I'm not 100% sure if this is your problem, but a lot of problems people seem to have with Netbeans and Derby seems to come from the fact that they don't set derby.system.home explicitly. When you don't, Derby stores databases in the current directory, and that is likely different when working in the IDE, either in the Services tab, or your own code, than when you execute your app's jar as a standalone program. So the advice (which you will also find in the manual) is: always set derby.system.home. An alternative would be to use full paths to the databases, but that rarely works well for a real application that is deployed on different machines.
I had the same problem --had the derby db in the services but the netbeans coded programs didn't access it. I solved it by adding the derby database (copy paste) to the package in the Files section. I use Windows 7. Once I did that, I was able use multiple tables (before netbeans just ignored secondary tables and only allowed me to use the primary table).
This question is extracted from a comment I posted here:
What's the best strategy for unit-testing database-driven applications?
So I have a huge database schema for a legacy application (with quite an old code base) that has many tables, synonyms, triggers, and dblinks. We and we have (finally) started to test some part of the application.
Our tests are already using mocks, but in order to test the queries that we are using we have decided to use an in-memory db with short-lived test dataset.
But the setup of the in-memory database requires a specific SQL script for the db schema setup. The script is not the real DDL we have in production because we can not import it directly.
To make things harder, the database contains functions and procedures that needs to be implemented in Java (we use the h2 db, and that is the way to declare procedures).
I'm afraid that our test won't break the day the real db will change and we will spot the problem only at runtime, potentially in production.
I know that our tests are quite at the border between integration and unit. However with the current architecture it is quite hard to insulate the test from the db. And we want to have proper tests for the db queries (no ORM inside).
What would be solution to have a DDL as close as possible of the real one and without the need to manually maintain it ?
If your environments are Dockerized I would highly suggest checking out Testcontainers (https://www.testcontainers.org/modules/databases/). We have used it to replace in-memory databases in our tests with database instances created from production DDL scripts.
Additionally, you can use tmpfs mounting to get performance levels similar to in-memory databases. This is nicely explained in following post from Vlad Mihalcea: https://vladmihalcea.com/how-to-run-integration-tests-at-warp-speed-with-docker-and-tmpfs/.
This combination works great for our purposes (especially when combined with Hibernate auto-ddl option) and I recommend that you check it out.
For a project I am working on(Spring/struts 2/hibernate), we decided to use h2 for unit testing with MySQL for the production store and manage the scheme in liquibase, pretty standard fare, but the issue we keep on running into is that h2 and MySQL differ in a lot of ways, for example how they handle timestamps and triggers. It's getting to the point that I am starting to regret using h2 as the extra headaches the mis-matches are causing are starting to outweigh its benefits. My question is this, is there any other in-memory/local file database that behaves more like MySQL? Obviously for integration testing we will still use MySQL, but being able to do unit testing without either making the liquibase files into a giant hack or having to ensure the local MySQL db is running would be nice.
I don't think there is another in-memory Java database that is more compatible to MySQL than H2. If you have a lot of code that only works with MySQL, then you should probably also use MySQL for testing.
Just be aware that it will be difficult in the future to switch to another database. Relying too much on features of one product will result in a "vendor lock in". In case of MySQL at least you have the option to switch to MariaDB, so it's not all that bad.
You may use a ram drive, copy your testing tables and datas into that drive, and start your mysql configured to load from that drive, all that in a script at boot time.
Then your unit tests will run insanely faster. We used it for developpers workstations and the level of frustation went three steps down.
I think that as of right now the correct approach is to use MySQL as a Docker image.
Once you create the image you can easily spin it up from your tests, and it's going to take seconds. Your hibernate will dynamically initialize DB schema and there you go!
The only issue is that CI servers need to have Docker installed.
i have a java project with mysql database
i am using advance installer to create a setup file...
i can embed jre to run the software(Without installing java in the system).
like wise,i want to embed the mysql database (system doesn't contains mysql )...
.There is any software to embed mysql database in my project setup...
MySQL is very difficult to embed correctly and there are a number of failure states that might occur if it is not shut down using the proper procedure. SQLite is a much better engine for this sort of thing and is used by a number of applications as a persistent backing store. While not as powerful as MySQL, it is much more resilient. It also has the advantage of not requiring a separate process.
SQLite's storage method is to persist things into a file that can be copied, moved, or backed-up without any issues. MySQL involves many such files, some of which are in an inconsistent state unless the correct FLUSH is called.
The best you can do with MySQL is bundle it, not embed it, but then you'll be responsible for setting it up on the host system, configuring it correctly, running the appropriate maintenance procedures, and providing some kind of back-up facility for the database itself.