Bonjour,
I am working on changing me Java application from using postgres to an embedded database. I would like the application to deploy with an initial set of data in the database. In the past during installation I have executed an sql script to fully generate the schema and insert the data in to my tables.
Ideally (becasue I don't really want to work out how to connect to the embedded database to generate it) I want to let JPA create my schema for the first time, and when it does I then want to be able to run my SQL to insert the data.
My search has turned up the obvious hibernate and JPA properties that allow running of an SQL script.
Firstly I found when using "hibernate.hbm2ddl.auto" you can define an import.sql file this made me very happy for a day until I realised it only works with create and not with update. My application when using postgres had this set to update. And what i would really like is for it to know if it's had to create the schema and if it has then run the import.sql. No Joy though.
I then moved on to using "javax.persistence.schema-generation.database.action" set to "create" I figured using the JPA specification was probably wiser anyway and so I defined "javax.persistence.sql-load-script-source" the spec says for "create"
The provider will create the database artifacts on application
deployment. The artifacts will remain unchanged after application
redeployment.
This lead me to believe it would do exactly what I wanted, only create the tables "on application deployment" however when I ran my tests using this, each test (creating a new spring context) tried to just create all the tables again and obviously failed, which made me realise application deployment didn't mean what i thought it meant (wishful thinking) and now I realise that JPA doesn't seem to even have an equivalent of Hibernates "update" property, so it's always going to generate the tables?
What I want is to have my tables and data generated when you first spin up the app and for subsequent executions to know the data is there and use it, I am assuming it's too much to hope for that this exists, but i'm sure that this must be a common requirement? So my question is what is the general recommended way to achieve the goal of allowing JPA to create my schema but being able to insert some data in to a db that persists between executions?
The answer is flyway. It is a database migration library, and if you are using Spring boot it is seamlessly integrated, with regular Spring you have to create a bean, which get a reference to the connection pool, creates a connection and does the migration.
Flyway creates a table so it keeps track of which scripts has already been applied to the database, and the scripts are simply part of the resources.
We normally use JPA to generate the initial script. This script becomes V1__initial.sql, if we need to add some data we can add V2__addUsers.sql and V3__addCustomers.sql etc.
Later when we need to rename columns or add additional tables, we simply add new scripts as part of the War file, and when the application is loaded Flyway looks at it's internal table, to see the current version, and then applies any new scripts to bring it up to de desired version.
In Spring the code would look like this
private void performFlywayMigration(DataSource dataSource) {
Flyway flyway = new Flyway();
flyway.setLocations("db/migration");
flyway.setDataSource(dataSource);
log.debug("Starting database migration.");
flyway.migrate();
log.debug("Database migration completed.");
MigrationInfo current = flyway.info().current();
if (current.getState() == MigrationState.FUTURE_SUCCESS) {
log.warn("The Database schema is version " + current.getVersion() + ", this application expects version " + flyway.getBaselineVersion().getVersion());
}
}
In general you should not JPA to create tables directly. because you sometimes need to modify the scripts, for instance on Sybase Varchar(255) means 255 bytes, so if you are storing 2 or 3 byte Unicode chars, you need more space - JPA implementation does not account for that (last time I checked).
Related
Is there any way to automatically create database functions using jpa/hibernate when i use spring.jpa.hibernate.ddl-auto=create ?
I've a number of custom functions (defined in a separated script file in the project) I use on my custom #Query in my repositories, but those functions needed to be create manually
You might think "what is the problem just run the script after created the database" but this must be done for every environment of the system: dev, test, prod... also it makes impossible to automatically test using in memory database (since when i hit run junit creates the database, hibernate creates the tables in the database, but nobody creates the functions so the repository wont work properly)
So how can I automatically create custom database functions using jpa/hibernate
You can add an import.sql file in your classpath.
Hibernate will create the schema and then run it on the selected db.
You can find more details in the documentation.
Note though that the main purpose for schema generation in Hibernate is prototyping or testing.
We use liquibase to specify the database layout and the changes of it. In development we use liquibase integrated in our Java application to perform the update if necessary.
Because our application does not have the rights to alter the schema in production, liquibase does not run in production. Instead we use liquibase to generate the SQL scripts. These scripts are then executed manually before our application is deployed.
We would like to make sure that the database layout matches to the changelog that corresponds to the application. The SQL scripts create the DATABASECHANGELOG table and insert the rows like the update command of liquibase would do, so the information about the applied changesets is stored in the database.
However I could not find a suitable liquibase method that only checks if the application's changesets and the database layout/DATABASECHANGELOG are equal. This method must not attempt to fix that, it should only return true/false. Is there such a method available in the liquibase Java API?
After digging through the liquibase source code, I found the method that I was looking for:
liquibase.listUnrunChangeSets(null, null);
This is exactly what I wanted it seems to be working fine.
I am working on a project in Java (using Spring Boot, Thymeleaf, Hibernate, JPA, MySql). Every time I create a new Model Class, I have to create a table in the database or if I make any change in the Model class I have to alter the table by myself. Is there any way to avoid this database related stuff. For example I will make Model classes and declare their relationships my Database tables will be generated automatically. In future if I make any changes to my classes they will be applied to the database automatically without loosing any data.
Previously I worked on PHP, Laravel. There all I needed to do is 1) run command php artisan make:migration create_posts_table, 2) declare columns like $table->string('title');, $table->foreign('user_id')->references('id')->on('users'); and then 3) run command php artisan migrate. That's it. No SQL scripts needed. I was wondering if Java, Spring has something like this.
Sure you can do it.
Use spring.jpa.hibernate.ddl-auto=update in your application.properties.
You can also use more advanced tools like https://www.liquibase.org/
Ideal way
spring.jpa.hibernate.ddl-auto=none
In my opinion, the ideal way is to create one SQL file which will create the schema at the startup for us.
To let Spring Boot to create it for you
spring.jpa.hibernate.ddl-auto=update
spring.jpa.hibernate.ddl-auto= # DDL mode. This is actually a shortcut for the "hibernate.hbm2ddl.auto" property. Defaults to "create-drop" when using an embedded database and no schema manager was detected. Otherwise, defaults to "none".
Other Possible values: create, create-drop, validate
More Detailed Explanation
You can do migration using Flyway, it's similar to Laravel migration.
Add the dependency and put your migration SQL files to classpath:db/migration. Flyway will automatically check the sql files version and apply any pending migrations.
https://flywaydb.org/documentation/plugins/springboot
For development and deployment of my WAR application I use the drop-and-create functionality. Basically erasing everything from the database and then automatically recreating all the necessary tables and fields according to my #Entity-classes.
Obviously, for production the drop-and-create functionality is out of question. How would I have to create the database tables and fields?
The nice thing about #Entity-classes is that due to OQL and the use of EntityManager all the database queries are generated, hence the WAR application gets database independent. If I now had to create the queries by hand in SQL and then let the application execute them, then I would have to decide in which sql dialect they are (i.e. MySQL, Oracly, SQL Server, ...). Is there a way to create the tables database independently? Is there a way to run structural database updates as well database independently (i.e. for database version 1 to database version 2)? Like altering field or table names, adding tables, droping tables, etc.?
Thank you #Qwerky for mentioning Liquibase. This absolutely is a solution and perfect for my case as I won't have to worry about versioning anymore. Liquibase is very easy to understand and studied in minutes.
For anyone looking for database versioning / scheme appliance:
Liquibase
I'm using H2 with Hibernate to generate in-memory DB on the fly for unit-testing.
I managed to create the DB successfully, and everything is working ok.
But I have an issue I don't know how to approach.
I need to load reference data to the DB for testing prior to the execution of the tests.
I have this data sored as a SQL insert's file which I need to run only once in real time envirnemnt, however, because the DB is generated every time from scratch I need to figure out how to insert the data on runtime.
The data is quite simple, it's countries lists, states list, etc.
Whats the best way to do it ?
btw, everything is working underneath Spring framework.
For your tests you could execute an init script on creation of the connection.
http://www.h2database.com/html/features.html#execute_sql_on_connection
From the question tags I see you're using Hibernate. You can add a file named "import.sql" to your classpath (i.e. in src/main/resources if you're using a Maven project layout).
From Spring documentation
In addition, a file named import.sql in the root of the classpath will
be executed on startup if Hibernate creates the schema from scratch
(that is if the ddl-auto property is set to create or create-drop).
This can be useful for demos and for testing if you are careful, but
probably not something you want to be on the classpath in production.
It is a Hibernate feature (nothing to do with Spring).