I have been trying for quite some time to figure out a solution for my problem, to no avail.
Anyway, i have a bunch of integration tests (in a nonstandard directory testRegression parallel to the standard test directory).
These integration tests use an h2 in memory database. In production as well as for testing i am using liquibase to simulate the schema evolution.
My properties (in application-testRegession.properties) look as follows:
spring.liquibase.enabled=true
spring.liquibase.user=sa
spring.liquibase.password=
spring.liquibase.change-log=classpath:/liquibase/changelog-master.xml
spring.datasource.url=jdbc:p6spy:h2:mem:testdb;MODE=PostgreSQL;DB_CLOSE_DELAY=-1;DATABASE_TO_UPPER=FALSE;INIT=CREATE SCHEMA IF NOT EXISTS nmc\\;CREATE SCHEMA IF NOT EXISTS mkt\\;CREATE SCHEMA IF NOT EXISTS cdb\\;CREATE SCHEMA IF NOT EXISTS pg_temp
spring.datasource.driver-class-name=com.p6spy.engine.spy.P6SpyDriver
spring.datasource.username=sa
spring.datasource.password=
The error i consistenly keep getting is:
2020-07-21 15:57:34.173 INFO [liquibase.lockservice.StandardLockService] [Test worker:13]: Successfully acquired change log lock
2020-07-21 15:57:34.303 INFO [liquibase.changelog.StandardChangeLogHistoryService] [Test worker:13]: Creating database history table with name: PUBLIC.DATABASECHANGELOG
2020-07-21 15:57:34.305 INFO [liquibase.executor.jvm.JdbcExecutor] [Test worker:13]: CREATE TABLE PUBLIC.DATABASECHANGELOG (ID VARCHAR(255) NOT NULL, AUTHOR VARCHAR(255) NOT NULL, FILENAME VARCHAR(255) NOT NULL, DATEEXECUTED TIMESTAMP NOT NULL, ORDEREXECUTED INT NOT NULL, EXECTYPE VARCHAR(10) NOT NULL, MD5SUM VARCHAR(35), DESCRIPTION VARCHAR(255), COMMENTS VARCHAR(255), TAG VARCHAR(255), LIQUIBASE VARCHAR(20), CONTEXTS VARCHAR(255), LABELS VARCHAR(255), DEPLOYMENT_ID VARCHAR(10))
2020-07-21 15:57:34.307 INFO [liquibase.lockservice.StandardLockService] [Test worker:13]: Successfully released change log lock
2020-07-21 15:57:34.309 WARN [org.springframework.boot.web.servlet.context.AnnotationConfigServletWebServerApplicationContext] [Test worker:13]: Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'liquibase' defined in class path resource [org/springframework/boot/autoconfigure/liquibase/LiquibaseAutoConfiguration$LiquibaseConfiguration.class]: Invocation of init method failed; nested exception is liquibase.exception.DatabaseException: Table "DATABASECHANGELOG" already exists; SQL statement:
CREATE TABLE PUBLIC.DATABASECHANGELOG (ID VARCHAR(255) NOT NULL, AUTHOR VARCHAR(255) NOT NULL, FILENAME VARCHAR(255) NOT NULL, DATEEXECUTED TIMESTAMP NOT NULL, ORDEREXECUTED INT NOT NULL, EXECTYPE VARCHAR(10) NOT NULL, MD5SUM VARCHAR(35), DESCRIPTION VARCHAR(255), COMMENTS VARCHAR(255), TAG VARCHAR(255), LIQUIBASE VARCHAR(20), CONTEXTS VARCHAR(255), LABELS VARCHAR(255), DEPLOYMENT_ID VARCHAR(10)) [42101-197] [Failed SQL: (42101) CREATE TABLE PUBLIC.DATABASECHANGELOG (ID VARCHAR(255) NOT NULL, AUTHOR VARCHAR(255) NOT NULL, FILENAME VARCHAR(255) NOT NULL, DATEEXECUTED TIMESTAMP NOT NULL, ORDEREXECUTED INT NOT NULL, EXECTYPE VARCHAR(10) NOT NULL, MD5SUM VARCHAR(35), DESCRIPTION VARCHAR(255), COMMENTS VARCHAR(255), TAG VARCHAR(255), LIQUIBASE VARCHAR(20), CONTEXTS VARCHAR(255), LABELS VARCHAR(255), DEPLOYMENT_ID VARCHAR(10))]
2020-07-21 15:57:34.309 INFO [com.zaxxer.hikari.HikariDataSource] [Test worker:13]: HikariPool-3 - Shutdown initiated...
2020-07-21 15:57:34.324 INFO [com.zaxxer.hikari.HikariDataSource] [Test worker:13]: HikariPool-3 - Shutdown completed.
2020-07-21 15:57:34.326 INFO [org.apache.catalina.core.StandardService] [Test worker:13]: Stopping service [Tomcat]
2020-07-21 15:57:34.342 INFO [org.springframework.boot.autoconfigure.logging.ConditionEvaluationReportLoggingListener] [Test worker:13]:
Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
2020-07-21 15:57:34.345 ERROR [org.springframework.boot.SpringApplication] [Test worker:13]: Application run failed
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'liquibase' defined in class path resource [org/springframework/boot/autoconfigure/liquibase/LiquibaseAutoConfiguration$LiquibaseConfiguration.class]: Invocation of init method failed; nested exception is liquibase.exception.DatabaseException: Table "DATABASECHANGELOG" already exists; SQL statement:
So how can i get around this issue? My basic understanding is that each test class creates its own ApplicationContext. For that it creates and loads a liquibase bean into it.
However, this problem occurs only for 2 out of 42 tests.
I would really like to get to the bottom of this and understand whats going on.
Can anyone shed light on my problem?
ADDITIONALLY
The test all run fine individually, but when run as a group they fail.
UPDATE 1
The relevant properties are as follows:
spring.main.allow-bean-definition-overriding=true
spring.datasource.url=jdbc:p6spy:h2:mem:testdb;MODE=PostgreSQL;DB_CLOSE_DELAY=-1;DATABASE_TO_UPPER=FALSE;INIT=CREATE SCHEMA IF NOT EXISTS nmc\\;CREATE SCHEMA IF NOT EXISTS mkt\\;CREATE SCHEMA IF NOT EXISTS cdb\\;CREATE SCHEMA IF NOT EXISTS pg_temp
spring.datasource.driver-class-name=com.p6spy.engine.spy.P6SpyDriver
spring.datasource.username=sa
spring.datasource.password=
spring.datasource.hikari.connectionTimeout=10000
spring.datasource.hikari.idleTimeout=60000
spring.datasource.hikari.maxLifetime=180000
spring.datasource.hikari.maximumPoolSize=50
spring.h2.console.enabled=true
spring.h2.console.path=/h2-console
My configuration is:
#Configuration
#ComponentScan(
basePackages = {
"com.aareal.nmc"
},
excludeFilters = {
#ComponentScan.Filter(type = FilterType.ASSIGNABLE_TYPE, value = CommandLineRunner.class)
})
#EnableTransactionManagement
#Profile("testRegression")
#SpringBootApplication(exclude = SecurityAutoConfiguration.class)
#EnableConfigurationProperties(LiquibaseProperties.class)
public class RegressionTestConfig {
My two tests are annotated as:
#RunWith(SpringRunner.class)
#SpringBootTest(
classes = {
RegressionTestConfig.class
},
//webEnvironment = SpringBootTest.WebEnvironment.DEFINED_PORT)
webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
Thanks
I had the same issue, and it seems to have been caused by case sensitive checking of the database table name. That is, the table was created as 'DATABASECHANGELOG', but Liquibase was checking for the existence of 'databasechangelog'.
The fix (at least for an H2 database) is to specify case insensitive identifiers in the database URL. For example:
jdbc:h2:mem:~/mydb;CASE_INSENSITIVE_IDENTIFIERS=TRUE
Explanation: The Spring test process fires up one or more instances of the Spring container to run tests on. If it thinks the config is exactly the same for two tests it will re-use an instance, otherwise it will start a new one. The instances are shared to avoid needing to start a whole new Springboot application for each test. But, the problem is that the instances may share some resources, such as databases and network ports. Therefore errors might occur from trying to start multiple instances at the same time. In this case, the test suite is starting two instances using the same database, but the second one is trying to re-run the whole Liquibase setup because the case sensitive issue means it doesn't see the table has already been created.
For my special case (that is for internal testing only, not production) what i have the following:
src
|-- main
|-- test
|-- testRegression
Workaround
Decide on the version of liquibase to use (i chose 4.0.0, which is the most recent at this point)
Create a file "src/testRegression/java/liquibase/changelog/StandardChangeLogHistoryService.java"
Open the original liquibase file "StandardChangeLogHistoryService.java" (mine is in ~//.gradle/caches/modules-2/files-2.1/org.liquibase/liquibase-core/4.0.0/23a5317eb5005b4765cd85e6f3a2cc4bb55c0daa/liquibase-core-4.0.0-sources.jar which i copied and unzipped)
and copy its contents 1:1 into the newly created file in 2.
Add a catch block by changing code (around line 396) from
if (SqlGeneratorFactory.getInstance().supports(sql, database)) {
executor.execute(sql);
getDatabase().commit();
} else {
to
if (SqlGeneratorFactory.getInstance().supports(sql, database)) {
try {
executor.execute(sql);
getDatabase().commit();
} catch (DatabaseException excptn) {
Scope.getCurrentScope()
.getLog(getClass())
.warning(
"Table '"
+ getDatabase()
.escapeTableName(
getLiquibaseCatalogName(),
getLiquibaseSchemaName(),
getDatabaseChangeLogTableName())
+ "' already exists.");
}
} else {
This is simply a workaround since there could be legitimate reasons for the ChangeLogTable table creation to fail. However, it already existing should not be cause for a major failure in my opinion.
My current view is that this is something which ought to be addressed/fixed in the official liquibase code base.
The following post(s) were helpful:
https://github.com/liquibase/liquibase-cache/issues/1
Related
I upgraded liquibase version in my Spring Boot App from 3.10.2 to 4.17.2 and I'm getting error
Suppressed: com.orientechnologies.orient.core.sql.OCommandSQLParsingException: Error parsing query: CREATE TABLE DATABASECHANGELOGLOCK (ID INT NOT NULL, LOCKED BOOLEAN NOT NULL, LOCKGRANTED datetime, LOCKEDBY VARCHAR(255), CONSTRAINT PK_DATABASECHANGELOGLOCK PRIMARY KEY (ID)) Encountered " <CREATE> "CREATE "" at line 1, column 1.Was expecting one of:
<LET> ...
<EXPLAIN> ...
DB name="mdp"
Error Code="1"
at com.orientechnologies.orient.core.sql.parser.OStatementCache.throwParsingException(OStatementCache.java:155)
at com.orientechnologies.orient.core.sql.parser.OStatementCache.parse(OStatementCache.java:147)
at com.orientechnologies.orient.core.sql.parser.OStatementCache.get(OStatementCache.java:86)
at com.orientechnologies.orient.core.sql.parser.OStatementCache.get(OStatementCache.java:65)
at com.orientechnologies.orient.core.sql.OSQLEngine.parse(OSQLEngine.java:80)
at com.orientechnologies.orient.core.db.document.ODatabaseDocumentEmbedded.command(ODatabaseDocumentEmbedded.java:650)
at com.orientechnologies.orient.server.OConnectionBinaryExecutor.executeQuery(OConnectionBinaryExecutor.java:1489)
at com.orientechnologies.orient.client.remote.message.OQueryRequest.execute(OQueryRequest.java:143)
at com.orientechnologies.orient.server.network.protocol.binary.ONetworkProtocolBinary.sessionRequest(ONetworkProtocolBinary.java:355)
at com.orientechnologies.orient.server.network.protocol.binary.ONetworkProtocolBinary.execute(ONetworkProtocolBinary.java:239)
at com.orientechnologies.common.thread.OSoftThread.run(OSoftThread.java:67)
As I can see liquibase can not create his own tables. I started getting this error after version update.
Can somebody tell me what can be a reason and how can I fix it. I can provide any information about my app whatever you need. Thanks!
I'm using Spring 2.6.2 and have tried both:
testImplementation("com.h2database:h2:2.1.214")
and
testImplementation("com.h2database:h2:1.4.200")
I create database schema's using raw SQL in JDBC connections to db during app startup e.g.
CREATE SCHEMA IF NOT EXISTS "001"
CREATE SCHEMA IF NOT EXISTS "002schema"
CREATE SCHEMA IF NOT EXISTS "3_schema"
Then Liquibase performs database migration.
When run against PostgreSQL, this works fine and schemas are created using the literal String provided. Tables inside those schemas are all lowercase too, as specified in the Liquibase changelogs.
When run against H2 in unit tests, an exception is thrown when performing database migration for 002schema. It looks like Liquibase then expects all schemas to have been upper-cased causing failure.
Exception during application start-up:
Initializing Liquibase for schema 001
Creating database history table with name: "001".DATABASECHANGELOG
Liquibase ran for schema 001
..
Initializing Liquibase for schema 002schema
Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'liquibaseMultiTenant' defined in class path resource [com/company/app/data/liquibase/LiquibaseConfiguration.class]: Invocation of init method failed; nested exception is liquibase.exception.LockException: liquibase.exception.DatabaseException: Schema "002SCHEMA" not found; SQL statement:
CREATE TABLE "002SCHEMA".DATABASECHANGELOGLOCK (ID INT NOT NULL, LOCKED BOOLEAN NOT NULL, LOCKGRANTED TIMESTAMP, LOCKEDBY VARCHAR(255), CONSTRAINT PK_DATABASECHANGELOGLOCK PRIMARY KEY (ID)) [90079-214] [Failed SQL: (90079) CREATE TABLE "002SCHEMA".DATABASECHANGELOGLOCK (ID INT NOT NULL, LOCKED BOOLEAN NOT NULL, LOCKGRANTED TIMESTAMP, LOCKEDBY VARCHAR(255), CONSTRAINT PK_DATABASECHANGELOGLOCK PRIMARY KEY (ID))]
I've tried the following options in H2 URL to prevent everything being made uppercase but none are working.
Any ideas?
spring:
datasource:
platform: h2
url: jdbc:h2:mem:test;MODE=PostgreSQL;DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=false;DATABASE_TO_UPPER=false;
driver-class-name: org.h2.Driver
username: sa
password:
Also tried:
spring:
datasource:
platform: h2
url: jdbc:h2:mem:test;MODE=PostgreSQL;DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=false;DATABASE_TO_LOWER=true;CASE_INSENSITIVE_IDENTIFIERS=true;
driver-class-name: org.h2.Driver
username: sa
password:
If the name of the schema is double quoted like "002SCHEMA" or "002schema" then it is case sensitive, it is not sensitive otherwise without the quotes:
Are PostgreSQL column names case-sensitive?
maybe you can solve the problem by changing this:
CREATE SCHEMA IF NOT EXISTS "002schema"
to this:
CREATE SCHEMA IF NOT EXISTS "002SCHEMA"
or somehow make liquibase not use quotes on schema name
In my project I have scripts that create tables, make changes, and so on, now when I add a new migration in a new file in the changeLog, I get an error. In the last migration that I added, there are no errors, because if you delete all tables from the schema and run the project, then everything will work without problems. But this option is not suitable, as a rule. The error occurs precisely when adding a new migration on top of the old one.
[ERROR] Error setting up or running Liquibase:
[ERROR] ??????: ????????? "databasechangelog" ??? ?????????? [Failed SQL: (0) CREATE TABLE databasechangelog (ID VARCHAR(255) NOT NULL, AUTHOR VARCHAR(255) NOT NULL, FILENAME VARCHAR(2
55) NOT NULL, DATEEXECUTED TIMESTAMP WITHOUT TIME ZONE NOT NULL, ORDEREXECUTED INTEGER NOT NULL, EXECTYPE VARCHAR(10) NOT NULL, MD5SUM VARCHAR(35), DESCRIPTION VARCHAR(255), COMMENTS V
ARCHAR(255), TAG VARCHAR(255), LIQUIBASE VARCHAR(20), CONTEXTS VARCHAR(255), LABELS VARCHAR(255), DEPLOYMENT_ID VARCHAR(10))]
I use spring boot, postgres in my project. What do you think might be the problem? thanks in advance.
left it in the form of a screen too, suddenly it will be more convenient for someone)
If you already have those spring configurations in your property file, It will create the liquibase tables. Make sure you are not creating it manually again in other liquibase script.
spring.liquibase.database-change-log-lock-table = DATABASECHANGELOGLOCK
spring.liquibase.database-change-log-table = DATABASECHANGELOG
I am new to flyway and struggling a bit to create tables using flyway in H2 database
I have created tables in my oracle Database using flyway script.
Now I want to perform some Junit test and want to create tables using Flyway script but not in
oracle Database instead I want to create this in H2 database (For unit test).
Flyway script used in my case is different both script have slightly different table structure.
Below is my project structure and configuration used
Below is the configuration present in src/main/resources folder
V1.0.0.0.001__tables.sql
CREATE TABLE COMMAND_ACTION
(COMMAND_TYPE_CD VARCHAR2(50 BYTE) NOT NULL,
ACTION VARCHAR2(50 BYTE) NOT NULL,
ACTION_ORDER NUMBER(*,0) NOT NULL,
CONSTRAINT COMMAND_ACTION_PK PRIMARY KEY (COMMAND_TYPE_CD, ACTION_ORDER),
CONSTRAINT COMMAND_ACTION_FK FOREIGN KEY (ACTION) REFERENCES ACTION_DEFINITION(ACTION)
);
CREATE TABLE COMMAND_ACTION_CONFIGURATION
( COMMAND_TYPE_CD VARCHAR2(50 BYTE) NOT NULL,
ACTION VARCHAR2(50 BYTE) NOT NULL,
CONFIGURATION VARCHAR2(1024 BYTE) NOT NULL);
application.properties
spring.profiles.active=cca
spring.flyway.table=CCA_SCHEMA_VERSION
spring.flyway.baseline-on-migrate=true
spring.flyway.schemas=CCA
spring.flyway.user=cca
spring.flyway.password=cca
application-cca.properties
spring.datasource.url=jdbc:oracle:thin:#linux:8080:devdb
spring.datasource.jdbc-url=jdbc:oracle:thin:#linux:8080:devdb
spring.datasource.username=cca
spring.datasource.password=cca
spring.datasource.driver-class-name=oracle.jdbc.OracleDriver
Below is the configuration present in src/test/resources folder
V1.0.0.0.001__tables.sql
CREATE TABLE COMMAND_ACTION
(COMMAND_TYPE_CD VARCHAR2(50 BYTE) NOT NULL,
ACTION VARCHAR2(50 BYTE) NOT NULL,
ACTION_ORDER NUMBER(10,0) NOT NULL,
CONSTRAINT COMMAND_ACTION_PK PRIMARY KEY (COMMAND_TYPE_CD, ACTION_ORDER),
CONSTRAINT COMMAND_ACTION_FK FOREIGN KEY (ACTION) REFERENCES ACTION_DEFINITION(ACTION)
);
CREATE TABLE COMMAND_ACTION_CONFIGURATION
( COMMAND_TYPE_CD VARCHAR2(50 BYTE) NOT NULL,
ACTION VARCHAR2(50 BYTE) NOT NULL,
CONFIGURATION VARCHAR2(1024 BYTE) NOT NULL);
test.properties
spring.profiles.active=test
spring.flyway.table=CCA_SCHEMA_VERSION
spring.flyway.baseline-on-migrate=true
spring.flyway.user=
spring.flyway.password=
application-test.properties
spring.datasource.url=jdbc:h2:mem:DATABASE
spring.datasource.jdbc-url=jdbc:h2:mem:DATABASE
spring.datasource.username=
spring.datasource.password=
spring.database.driverClassName=org.h2.Driver
spring.flyway.locations=classpath:db/migration
spring.jpa.hibernate.ddl-auto=none
# Enabling H2 Console
spring.h2.console.enabled=true
I want that my junit should work independent of my src/resources but not able to achieve instead getting
below error whenever I run maven install
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_66]
2020-02-27T12:45:22,000 DEBUG CommandService [pool-3-thread-1] Timer is started.
2020-02-27T12:45:22,001 ERROR CommandService [pool-3-thread-1] Error in processing command. Error :{}
org.h2.jdbc.JdbcSQLSyntaxErrorException: Table "COMMAND_ACTION" not found; SQL statement:
select ca.command_type_cd, ca.action, cac.CONFIGURATION from command_action ca
left outer join command_action_configuration cac
on ca.command_type_cd = cac.command_type_cd and ca.action = cac.action
order by ca.command_type_cd, ca.action_order [42102-200]
Either some time I get checksum related error.
I am using Spring Boot 2 with Data JPA and Flyway on a Postgres database. Everything works fine in production. Now I'm trying to write tests that will run on an H2 Embedded Database for testing purposes. But, the tests encounter a SQL syntax error. But, I don't understand what's wrong with the syntax:
CREATE TABLE mysite_user (
id int8 NOT NULL,
thirdparty_user_id varchar(255) NULL,
email varchar(255) NULL,
first_name varchar(255) NULL,
PRIMARY KEY (id)
);
CREATE INDEX mysite_user_thirdparty_user_id_idx ON mysite_user USING btree (thirdparty_user_id) ;
This is the error:
Migration V1__Initial.sql failed
--------------------------------
SQL State : 42001
Error Code : 42001
Message : Syntax error in SQL statement "CREATE INDEX MYSITE_USER_THIRDPARTY_USER_ID_IDX ON MYSITE_USER USING[*] BTREE (THIRDPARTY_USER_ID) "; expected "., COMMENT, ("; SQL statement:
CREATE INDEX mysite_user_thirdparty_user_id_idx ON mysite_user USING btree (thirdparty_user_id) [42001-199]
Location : db/migration/V1__Initial.sql (/Users/me/Development/mysite-website/target/classes/db/migration/V1__Initial.sql)
Line : 20
Statement : CREATE INDEX mysite_user_thirdparty_user_id_idx ON mysite_user USING btree (thirdparty_user_id)
What am I doing wrong?
Arguably, what you're doing wrong is using different databases between prod and test. As you've seen, your tests end up having to be coded around the differences between the databases, as well as potentially overlooking subtle edge cases which work in one db and not the other.
If you want to go down this route it is possible - see Best way for "database specific" sql scripts with Flyway