I don't quite understand what I am supposed to do when a migration fails using Flyway in a Spring Boot project.
I activated Flyway by simply adding the Flyway dependency in my pom.xml. And everything works fine. My database scripts are migrated when I launch the Spring Boot app.
But I had an error in one of my scripts and my last migration failed. Now when I try to migrate, there is a "Migration checksum mismatch". Normally, I would run mvn flyway:repair, but since I am using Spring Boot, I am not supposed to use the Flyway Maven plug-in. So what am I supposed to do?
there are several ways to perform a repair on the database. I personally prefer the simple SQL statement.
SQL Statement:
Just delete the row with the failed migration. After that you can run the migration again.
Run flyway directly
You can install Flyway local and run flyway repair in the console
Use the Flyway Maven Plugin
Add the Flyway Maven Plugin to your pom and run mvn flyway:repair. I don't think this contradict with the Spring Boot concept.
Extend Spring Boot
Spring Boot will call Flyway.migrate() to perform the database migration. If you would like more control, provide a #Bean that implements FlywayMigrationStrategy.
In the FlywayMigrationStrategy you can call the migrate or repair method from flyway. More Information is available in the Spring Boot Reference Guide.
I don't think the FlywayMigrationStrategy in the application is the right place to repair the database. A failed migration is a exception and should be handle outside the application.
You can do it through code by declaring the following bean.
#Bean
public FlywayMigrationStrategy cleanMigrateStrategy() {
return flyway -> {
flyway.repair();
flyway.migrate();
};
}
Flyway Maven Plugin
Just to add this info to #Daniel's answer
1.
...
<plugin>
<groupId>org.flywaydb</groupId>
<artifactId>flyway-maven-plugin</artifactId>
<version>4.1.0</version>
<configuration>
<url>jdbc:mysql://localhost:3306</url>
<user>root</user>
<password>root</password>
<schemas>
<schema>[your_schema]</schema>
</schemas>
</configuration>
</plugin>
...
2.
mvn flyway:clean
3.
mvn flyway:repair
PS.: if the step 2 and 3 don't work change the order.
More info on maven goals: https://flywaydb.org/documentation/maven/
When database migration fails, the migration is marked as failed in the schema history table (i.e flyway_schema_history) indicating manual database cleanup may be required. But if database supports DDL transactions, the migration is rolled back automatically and nothing is recorded in the schema history table. PostgreSQL, Amazon Redshift, MS SQL are few of the databases which support DDL transactions whereas Oracle Database, MySQL, MariaDB, Amazon Aurora does not support DDL transactions.
In case of failed migration entries, there are several options to repair it (only applicable for databases that do NOT support DDL transactions) as described by #daniel-käfer. I want to add another (may be easier way) to deal with failed migrations.
There are several callbacks supported by flyway, afterMigrateError is one of them. If we add a sql file with name afterMigrateError.sql then, it will be executed after each failed migrate runs. Therefore, we can simply create a file afterMigrateError.sql on default location of database migration folder (resources/db/migration) with sql command to remove failed migrations from flyway_schema_history table.
The sql command afterMigrateError.sql can be as mentioned below:
DELETE IGNORE FROM flyway_schema_history WHERE success=0;
This command looks for the table flyway_schema_history if it exists otherwise it will do no changes. Then it simply looks for the rows which has success column with 0 entry (actually this happen if migration fails , all successful migration will have value 1 in success column), then delete such entries. Now, we can simply change our latest migration file and correct it and run again.
Install flyway locally as said above, change directory into the installation then run (example for H2):
./flyway -url=jdbc:h2:/Users/mugo/dev/h2/das-boot -user=sa -password= repair
Related
I have integrated liquibase with maven plugin (in pom.xml) along with jOOQ generator that reads from db connection after liquibase migration.
The liquibase migration -> jooq generator runs on running mvn clean compile (Compile and build step).
I am wondering how will the liquibase change-management work for different deployments or different prod environments?
Will it create tables automatically for all the prod environments? Please guide.
This seems like the same ask as your earlier question.
If it's not the same, then the first place to look is Liquibase Documentation for Maven and then the jOOQ manual for Code generation from Liquibase XML, YAML, JSON files.
I have an app that uses Spring Boot app and I want to add a DB migration with flyway to it but for some reason is not working.
In my gradle file I have added this dependency:
implementation 'org.flywaydb:flyway-core:5.2.4'
Then in my yml configuration file, I add the following:
spring:
flyway.url: jdbc:postgresql://localhost:5432/my_database
flyway.user: postgres
flyway.password: postgres
In addition to this, I have my .sql scripts under resources/db/migration.
They have the right naming format(V1__base.sql).
I run the app using gradle bootrun
According to the documentation that I am following, this should just work out of the box. My database is running locally in a docker container and I don't see any error in the terminal. What am I missing?
The instructions I am following are from here: https://flywaydb.org/documentation/plugins/springboot
Having some issues when uploading my jhipster generated app to heroku, after some research I found out that some entity modifications I did while developing are not done in production db so some fields are missing.
I also added a field to User entity, so now I cannot even register/login while that field is missing from the table.
Would like to know how exacly the database generation works in prod mode so I can make it work. Could it be related to smth missing in liquibase changelog?(I'm absolutely new to liquibase so I'm a bit lost).
UPDATE:
The problem seems to be a missing column from "jhi_user" table. I need to create a database changelog to add that missing column, so I ran
"mvn liquibase:generateChangeLog"
I saw the correct changelog poping in the console, but no changelog was created under my changelogs folder. What am i doing wrong?
First, check that there is a migration file corresponding to your entity in the directory:
src/main/resources/config/liquibase/changelog/
Then confirm that the Heroku app has a Database by running:
heroku addons
Then run Maven to repackage your WAR file:
mvn install -Pprod -DskipTests
Then deploy again:
heroku deploy:jar target/*.war
JHipster will run migrations at boot time. If you are using Postgresql, you can inspect the tables by running heroku pg:psql and using commands like \d
The problem was changelogs were not beeing generated when i created entities from yo jhipster:entity command.
Solved the issue doing:
mvn liquibase:diff
Plus some hand-done changesets and adding them to master.xml.
After that:
mvn liquibase:dropAll
mvn liquibase:update
Then deployed to heroku and every table was created.
How can I start flyway manually with Spring Boot? Because I have 2 dbs in my application, so after Springboot load the properties and before connect to DB, I need to run the flyway from my 2 dbs.
If I understand you correctly, you don't want to start flyway manually, but run it on two databases. Here is how:
create two Datasources and mark one as #Primary, the other (should be a bean) as #FlywayDataSource.
When flyway runs, it will automatically run on both data sources. see https://docs.spring.io/spring-boot/docs/current-SNAPSHOT/reference/htmlsingle/#howto-execute-flyway-database-migrations-on-startup
You can use flyway maven plugin to populate your databases.
You can either have two properties file for each database and execute flyway using mvn -Dflyway.configFile=myConfig.properties
Or
you can provide the db properties while executing maven like mvn -Dflyway.user=myUser -Dflyway.schemas=schema1,schema2 -Dflyway.placeholders.keyABC=valueXYZ
But please note that you have to execute flyway maven task twice. Each time for one database. Like,
$mvn -Dflyway.configFile=db1.properties compile migrate
$mvn -Dflyway.configFile=db2.properties compile migrate
Refer https://flywaydb.org/documentation/maven/ for more information.
If you are fine with this maven approach you don't need spring to do flyway migration.
I use MyBatis, Java, Maven, MySQL and Eclipse IDE.
I have 2 dabases:
db1 and db2.
db1 - is my default database.
When in Maven I make migration:up it updates the database (schema) - db1.
How can I make migration:up with MyBatis to update database db2 too?
Create another environment properties in environment folder and specify details of db2 there.
When you run migration:up this uses default environment from migration repository which is development (or in your case some other environment may be specified in mybatis migration maven plugin configuration). You need to create new file in migration repository for example db2.properties and then use it when running maven like this
mvn migration:up -Dmigration.env=db2