I have an app that uses Spring Boot app and I want to add a DB migration with flyway to it but for some reason is not working.
In my gradle file I have added this dependency:
implementation 'org.flywaydb:flyway-core:5.2.4'
Then in my yml configuration file, I add the following:
spring:
flyway.url: jdbc:postgresql://localhost:5432/my_database
flyway.user: postgres
flyway.password: postgres
In addition to this, I have my .sql scripts under resources/db/migration.
They have the right naming format(V1__base.sql).
I run the app using gradle bootrun
According to the documentation that I am following, this should just work out of the box. My database is running locally in a docker container and I don't see any error in the terminal. What am I missing?
The instructions I am following are from here: https://flywaydb.org/documentation/plugins/springboot
Related
I am building a Spring Boot application using Bamboo. Is there any way I can read maven build related information like:
buildNumber and buildTimeStamp?
I have tried reading properties like below but they are not working.
#Value("${bamboo.buildNumber}")
private String buildNumber;
and
#Value("${buildNumber}")
private String buildNumber;
I think the bit I missing can be any of below:
Bamboo is not writing properties to Jar exported to Artifactory.
I am reading properties with wrong keys.
I am missing some POM configuration.
Any help is much appreciated.
The #Value notation reads values from a properties file when you run the spring application. There is no way that properties which existed in the Bamboo environment at the time that your spring app was built will be available when you run your spring app, unless you take steps to make them available.
You will need to build a .properties file during the Bamboo build, and have this packaged into your spring boot application.
According to the Spring Boot Devtools docs, devtools won't run in "production mode"; that is, if you execute your Spring Boot app with java -jar .., then it won't use devtools' built-in JVM magic. However, the Spring Boot starter docs only show you one way of running your Spring Boot application...via java -jar....
So first I'm wondering: How do I run my Spring Boot app in non-production mode? I know you can run your app with Spring Boot CLI (e.g. spring run), but is that the only way?
Also, the same devtools docs mention you can explicitly remove the devtools jar from your production binary by using some excludeDevtools option, but they never explain where/how to use this. Any examples/ideas here?
1) One example of running a Spring Boot app in non-production mode (i.e. not from the JAR) is from an IDE (most specifically Eclipse or IntelliJ), where you can launch the application from the main method, or using the Spring Boot launch support. This is the most practical use of Dev Tools as it allows to continuously watch the changes in your application without restarting.
2) excludeDevTools is an option of the Spring Boot Maven Plugin http://docs.spring.io/spring-boot/docs/current-SNAPSHOT/maven-plugin/repackage-mojo.html or the Gradle plugin - see http://docs.spring.io/spring-boot/docs/current/reference/html/build-tool-plugins-gradle-plugin.html#build-tool-plugins-gradle-repackage-configuration.
So in your case it would be
bootRepackage {
mainClass = 'demo.Application'
excludeDevtools = true
}
To start with Spring boot and devtools made development easier. Spring boot uses embedded servlet container ( tomcat,undertow,jetty) with this you can have production ready code even in non-production environment.
Coming to your question how to run Spring-boot application in non-production environments, we can run application directly from Spring tool suite IDE by right click -> run as Spring boot if it is local environment.
If the build tool used is maven then from command prompt mvn boot:run , if it is gradle then gradle bootRun from root folder of project location in command prompt. If you are packing your application as uber jar then Java -jar .., also if your environment is Linux you can start as a process.
Regarding ignoring devtools in production environment, Spring boot intelligently ignore devtools jar if we specify Spring-boot-devtools as optional scope in dependencies of your build tool(maven, gradle) . In gradle Spring boot gradle plugin takes care of ignoring devtools for final deployed jar or war
Update ::
To debug a Spring Boot application run with Gradle, as seen at http://docs.spring.io/spring-boot/docs/current-SNAPSHOT/reference/htmlsingle/#howto-remote-debug-gradle-run596
Add this to your build.gradle file:
applicationDefaultJvmArgs = [
"-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=5005" ]
In IntelliJ Idea IDE or STS, setup a new Debug configuration and in the options select "Remote". The defaults should be fine, but pay attention to the port number and transport type ("socket"). Have fun!
I don't quite understand what I am supposed to do when a migration fails using Flyway in a Spring Boot project.
I activated Flyway by simply adding the Flyway dependency in my pom.xml. And everything works fine. My database scripts are migrated when I launch the Spring Boot app.
But I had an error in one of my scripts and my last migration failed. Now when I try to migrate, there is a "Migration checksum mismatch". Normally, I would run mvn flyway:repair, but since I am using Spring Boot, I am not supposed to use the Flyway Maven plug-in. So what am I supposed to do?
there are several ways to perform a repair on the database. I personally prefer the simple SQL statement.
SQL Statement:
Just delete the row with the failed migration. After that you can run the migration again.
Run flyway directly
You can install Flyway local and run flyway repair in the console
Use the Flyway Maven Plugin
Add the Flyway Maven Plugin to your pom and run mvn flyway:repair. I don't think this contradict with the Spring Boot concept.
Extend Spring Boot
Spring Boot will call Flyway.migrate() to perform the database migration. If you would like more control, provide a #Bean that implements FlywayMigrationStrategy.
In the FlywayMigrationStrategy you can call the migrate or repair method from flyway. More Information is available in the Spring Boot Reference Guide.
I don't think the FlywayMigrationStrategy in the application is the right place to repair the database. A failed migration is a exception and should be handle outside the application.
You can do it through code by declaring the following bean.
#Bean
public FlywayMigrationStrategy cleanMigrateStrategy() {
return flyway -> {
flyway.repair();
flyway.migrate();
};
}
Flyway Maven Plugin
Just to add this info to #Daniel's answer
1.
...
<plugin>
<groupId>org.flywaydb</groupId>
<artifactId>flyway-maven-plugin</artifactId>
<version>4.1.0</version>
<configuration>
<url>jdbc:mysql://localhost:3306</url>
<user>root</user>
<password>root</password>
<schemas>
<schema>[your_schema]</schema>
</schemas>
</configuration>
</plugin>
...
2.
mvn flyway:clean
3.
mvn flyway:repair
PS.: if the step 2 and 3 don't work change the order.
More info on maven goals: https://flywaydb.org/documentation/maven/
When database migration fails, the migration is marked as failed in the schema history table (i.e flyway_schema_history) indicating manual database cleanup may be required. But if database supports DDL transactions, the migration is rolled back automatically and nothing is recorded in the schema history table. PostgreSQL, Amazon Redshift, MS SQL are few of the databases which support DDL transactions whereas Oracle Database, MySQL, MariaDB, Amazon Aurora does not support DDL transactions.
In case of failed migration entries, there are several options to repair it (only applicable for databases that do NOT support DDL transactions) as described by #daniel-käfer. I want to add another (may be easier way) to deal with failed migrations.
There are several callbacks supported by flyway, afterMigrateError is one of them. If we add a sql file with name afterMigrateError.sql then, it will be executed after each failed migrate runs. Therefore, we can simply create a file afterMigrateError.sql on default location of database migration folder (resources/db/migration) with sql command to remove failed migrations from flyway_schema_history table.
The sql command afterMigrateError.sql can be as mentioned below:
DELETE IGNORE FROM flyway_schema_history WHERE success=0;
This command looks for the table flyway_schema_history if it exists otherwise it will do no changes. Then it simply looks for the rows which has success column with 0 entry (actually this happen if migration fails , all successful migration will have value 1 in success column), then delete such entries. Now, we can simply change our latest migration file and correct it and run again.
Install flyway locally as said above, change directory into the installation then run (example for H2):
./flyway -url=jdbc:h2:/Users/mugo/dev/h2/das-boot -user=sa -password= repair
How can I start flyway manually with Spring Boot? Because I have 2 dbs in my application, so after Springboot load the properties and before connect to DB, I need to run the flyway from my 2 dbs.
If I understand you correctly, you don't want to start flyway manually, but run it on two databases. Here is how:
create two Datasources and mark one as #Primary, the other (should be a bean) as #FlywayDataSource.
When flyway runs, it will automatically run on both data sources. see https://docs.spring.io/spring-boot/docs/current-SNAPSHOT/reference/htmlsingle/#howto-execute-flyway-database-migrations-on-startup
You can use flyway maven plugin to populate your databases.
You can either have two properties file for each database and execute flyway using mvn -Dflyway.configFile=myConfig.properties
Or
you can provide the db properties while executing maven like mvn -Dflyway.user=myUser -Dflyway.schemas=schema1,schema2 -Dflyway.placeholders.keyABC=valueXYZ
But please note that you have to execute flyway maven task twice. Each time for one database. Like,
$mvn -Dflyway.configFile=db1.properties compile migrate
$mvn -Dflyway.configFile=db2.properties compile migrate
Refer https://flywaydb.org/documentation/maven/ for more information.
If you are fine with this maven approach you don't need spring to do flyway migration.
I have a spring boot application that uses a database. I have the local configuration in the application.properties file. I understand it is possible to change the values when I build the project with maven. I want to set the production values (db on AWS).
Here is application.properties
spring.datasource.url=${db.url:jdbc:postgresql://localhost:5432/mydb}
spring.datasource.username=${db.username:localusername}
spring.datasource.password=${db.password:localpassword}
I would like to change the values when I run. This will create a docker image.
mvn package docker:build
Is it possible?