I'm new to liquibase (I'm using Spring Liquibase)... Can someone please explain whether it's possible to have a manual db change get reflected in the changeset file of liquibase... I mean if we have a table A at the first time and then after I removed a column from it, how to get it automatically updated in the change set file...
Thanx
In other words, I'm looking for a liquibase diff operation in spring liquibase or from java
Thanx again
The key concept of liquibase is that you don't make manual DB changes (or at least, only when testing) outside of the liquibase changelog. You should add a new changeset that reflects the change you want to be made. If you're confused about what the changeset should look like, read the docs.
Related
I have the following liquibase configuration for the database in the spring boot application.
Initially, these YAML scripts were executed when the application started and the database was created, now I want to update the datatype for one column so do I need to update the existing create-tables.YAML with column configuration or need to create another file with a different name and add the entry in "db.changelog-master.yaml" file.
Please suggest, Thanks
You have to create a different change log and needs to be added to the master file
Best Practices Using Liquibase
Organizing Changelogs: Create a master changelog file that does not have actual changeSets but includes other changelogs (only YAML, JSON, and XML support using include, SQL does not). Doing so allows us to organize our changeSets in different changelog files. Every time we add a new feature to the application that requires a database change, we can create a new changelog file, add it to version control, and include it in the master changelog.
One Change per ChangeSet: Have only one change per changeSet, as this allows easier rollback in case of a failure in applying the changeSet.
Don’t Modify a ChangeSet: Never modify a changeSet once it has been executed. Instead, add a new changeSet if modifications are needed for the change that has been applied by an existing changeSet. Liquibase keeps track of the checksums of the changeSets that it already executed. If an already run changeSet is modified, Liquibase by default will fail to run that changeSet again, and it will not proceed with the execution of other changeSets.
Sample Setup
Currently, I am using SQL format liquibase changelog but you can use XML, YAML or any other format.
test_schema.sql
--liquibase formatted sql
--changeset test_user:test_schema splitStatements:true endDelimiter:; dbms:postgresql runOnChange:true runInTransaction:true context:dev, qa, prod logicalFilePath:schema/test_schema.sql
--preconditions onFail:HALT onError:HALT
--comment test_schema is added
CREATE TABLE test_data
(
test_id bigint PRIMARY KEY,
test_data character varying(100),
created_time timestamp with time zone,
last_modified_time timestamp with time zone
);
changelog-master.xml
<?xml version="1.0" encoding="UTF-8"?>
<databaseChangeLog
xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog
http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-4.3.xsd">
<include file="schemas/test_schema.sql" relativeToChangelogFile="true"/>
</databaseChangeLog>
directory setup
[![directory_setup][1]][1]
Soure : https://reflectoring.io/database-migration-spring-boot-liquibase/
As you can read here ,
Liquibase provides version control for your database
, the main reason Liquibase became so popular was that it allowed some type of version control to be applied on databases.
For this to be able to happen you need to always record your changes in database into changelogs.
So an example of Liquibase project will have
create-initial-tables-changelog.xml
update-schema-10-5-22.changelog.xml
update-data-15-5-22.changelog.xml
update-schema-10-6-22.changelog.xml
update-data-15-6-22.changelog.xml
Each changelog file with changes may be named according to a representation of what the main changes are (dates are used here only for simplifying).
Then the user can use a version control system (like git) and when for example the user checkouts a commit of the past (ex commit A on 15/05/2022) then he is able to view the database on that version as it existed on 15/05/2022, since liquibase will execute only the scripts that existed on that commit, namely create-initial-tables-changelog.xml, update-schema-10-5-22.changelog.xml, update-data-15-5-22.changelog.xml.
Also all changelog files need to be referenced in the master changelog file for a specific version, because this file is used for executing scripts in database when asked for.
The master changelog file works as a configuration file that will hold
all the references to all your other changelogs. Your master changelog
file must be in an XML, YAML, or JSON format.
From docs
Having said all the above, now I can answer your question
So do I need to update the existing create-tables.YAML with column
configuration or need to create another file with a different name
Probably the main reason you use Liquibase, is to have a version control of your database. If you wish to respect this reason, then you must create a different changelog file which would be a snapshot of a different version of database than the initial version that existed.
Also if for example you had the initial create-tables-changelog.xml and then you also had 3 more changelog.xml files that applied changes in database, you could not afford to make changes only in the create-tables-changelog.xml since this would risk that the execution of the 3 next files changelog1.xml, changelog2.xml, changelog3.xml would break.
We use liquibase to specify the database layout and the changes of it. In development we use liquibase integrated in our Java application to perform the update if necessary.
Because our application does not have the rights to alter the schema in production, liquibase does not run in production. Instead we use liquibase to generate the SQL scripts. These scripts are then executed manually before our application is deployed.
We would like to make sure that the database layout matches to the changelog that corresponds to the application. The SQL scripts create the DATABASECHANGELOG table and insert the rows like the update command of liquibase would do, so the information about the applied changesets is stored in the database.
However I could not find a suitable liquibase method that only checks if the application's changesets and the database layout/DATABASECHANGELOG are equal. This method must not attempt to fix that, it should only return true/false. Is there such a method available in the liquibase Java API?
After digging through the liquibase source code, I found the method that I was looking for:
liquibase.listUnrunChangeSets(null, null);
This is exactly what I wanted it seems to be working fine.
Is there a way to tell flyway to recreate the flyway-table without applying the migrations. Eg, look into the migration-folder for scripts and assume that they all have been applied and simply make sure that the flyway table contains all of them.
Our scenario is that we are not allowed to run flyway in production and for (ISO;Banking;certifications) reasons. The rules says that we need to remove the tables completely. So when we reset our test-environments from a copy of production we need to recreate the flyway-table. Now we are copy and pasting from an existing test-environment but sometime that isn't in sync with production and all kind of problems occur.
So, we would like to setup our production-copy with the same version as in production and then recreate the tables from that making sure that everything are in sync. But to my understanding the repair-option in flyway doesn't recreate the non-applied scripts...
It looks like what you're describing is called a baseline:
You tell flyway that the database you're working on is at a version number so all scripts previous to this version will be ignored during migrations.
https://flywaydb.org/documentation/commandline/baseline
Is it possible to use Liquibase just to check the consistency of the database?
We have several java application modules using the same database. We decided that only one of the modules is responsible for the execution of database migrations, while the other modules (several batch jobs) include the scripts as a dependency. For the batch job modules we want to prevent the migration of the database schema, but we need to be sure that the code base uses the same version as the database.
Is it possible to configure liquibase in a way to perform the validation but not the migration?
We want to try this approach because the migration of two modules starting at the same time caused conflicts that prevented the application from starting.
It's possible to use Liquibase for DB schema validation, however it wasn't quite designed for this.
So, for instance, if you want to always check weather certain table exists or not, you can do the following:
<changeSet author="author_name" id="changeset_id" runAlways="true">
<preConditions onFail="HALT">
<tableExists tableName="foo_bar"/>
</preConditions>
<comment>Table "foo_bar" exists.</comment>
</changeSet>
This changeSet doen't do anything but checking that foo_bar table exists.
runAlways="true" attribute will tell Liquibase to execute the changeSet every time the application starts.
onFail="HALT" will throw an error if foo_bar table doesn't exist, hence if preConditions weren't met.
You can use the liquibase status command to check that all changesets listed in the changelog have been applied, and add the --verbose flag to that to see which changesets have not been applied.
This does not ensure that no drift has happened though - it is certainly possible for someone to manually alter the database and make changes that cause the status command to be inaccurate. As long as you are generally confident that schema changes are always made through liquibase, the status command should be sufficient.
I have small project written in Spring. For database migration and seeds I use liquibase.
After some time I have request to downgrade my database to previous version. Is this possible to do with liquibase and what workflow do you recommend? I can do that with packaging new war file and run some pure sql scripts but that is not a good way for me. I just want to do that with liquibase - maybe some rollback to previous version.
Thank you in advance.
Liquibase can not simply (automatically) rollback already existent updates. Only thing you could do - write additional changesets, where you will manually describe needed changes in the DB, which will return structure to needed state.
You can also describe rollback actions in advance in order to make this process more nice, it is can be done via rollback section in each changeset.
remember that db rollback is not something that is in general feasible. for example in v1 you have a column A full of data (to make discussion easier: with not null constrain and without a default value). in v2 you delete column A. how do you want to automatically perform rollback / downgrade?
i suggest to add another migration that will migrate your db to state v3 that looks exactly like / similar to v1. inside this migration you can handle all the missing data etc