In my project I have scripts that create tables, make changes, and so on, now when I add a new migration in a new file in the changeLog, I get an error. In the last migration that I added, there are no errors, because if you delete all tables from the schema and run the project, then everything will work without problems. But this option is not suitable, as a rule. The error occurs precisely when adding a new migration on top of the old one.
[ERROR] Error setting up or running Liquibase:
[ERROR] ??????: ????????? "databasechangelog" ??? ?????????? [Failed SQL: (0) CREATE TABLE databasechangelog (ID VARCHAR(255) NOT NULL, AUTHOR VARCHAR(255) NOT NULL, FILENAME VARCHAR(2
55) NOT NULL, DATEEXECUTED TIMESTAMP WITHOUT TIME ZONE NOT NULL, ORDEREXECUTED INTEGER NOT NULL, EXECTYPE VARCHAR(10) NOT NULL, MD5SUM VARCHAR(35), DESCRIPTION VARCHAR(255), COMMENTS V
ARCHAR(255), TAG VARCHAR(255), LIQUIBASE VARCHAR(20), CONTEXTS VARCHAR(255), LABELS VARCHAR(255), DEPLOYMENT_ID VARCHAR(10))]
I use spring boot, postgres in my project. What do you think might be the problem? thanks in advance.
left it in the form of a screen too, suddenly it will be more convenient for someone)
If you already have those spring configurations in your property file, It will create the liquibase tables. Make sure you are not creating it manually again in other liquibase script.
spring.liquibase.database-change-log-lock-table = DATABASECHANGELOGLOCK
spring.liquibase.database-change-log-table = DATABASECHANGELOG
Related
I upgraded liquibase version in my Spring Boot App from 3.10.2 to 4.17.2 and I'm getting error
Suppressed: com.orientechnologies.orient.core.sql.OCommandSQLParsingException: Error parsing query: CREATE TABLE DATABASECHANGELOGLOCK (ID INT NOT NULL, LOCKED BOOLEAN NOT NULL, LOCKGRANTED datetime, LOCKEDBY VARCHAR(255), CONSTRAINT PK_DATABASECHANGELOGLOCK PRIMARY KEY (ID)) Encountered " <CREATE> "CREATE "" at line 1, column 1.Was expecting one of:
<LET> ...
<EXPLAIN> ...
DB name="mdp"
Error Code="1"
at com.orientechnologies.orient.core.sql.parser.OStatementCache.throwParsingException(OStatementCache.java:155)
at com.orientechnologies.orient.core.sql.parser.OStatementCache.parse(OStatementCache.java:147)
at com.orientechnologies.orient.core.sql.parser.OStatementCache.get(OStatementCache.java:86)
at com.orientechnologies.orient.core.sql.parser.OStatementCache.get(OStatementCache.java:65)
at com.orientechnologies.orient.core.sql.OSQLEngine.parse(OSQLEngine.java:80)
at com.orientechnologies.orient.core.db.document.ODatabaseDocumentEmbedded.command(ODatabaseDocumentEmbedded.java:650)
at com.orientechnologies.orient.server.OConnectionBinaryExecutor.executeQuery(OConnectionBinaryExecutor.java:1489)
at com.orientechnologies.orient.client.remote.message.OQueryRequest.execute(OQueryRequest.java:143)
at com.orientechnologies.orient.server.network.protocol.binary.ONetworkProtocolBinary.sessionRequest(ONetworkProtocolBinary.java:355)
at com.orientechnologies.orient.server.network.protocol.binary.ONetworkProtocolBinary.execute(ONetworkProtocolBinary.java:239)
at com.orientechnologies.common.thread.OSoftThread.run(OSoftThread.java:67)
As I can see liquibase can not create his own tables. I started getting this error after version update.
Can somebody tell me what can be a reason and how can I fix it. I can provide any information about my app whatever you need. Thanks!
I recently upgraded h2 version from 1.4.200 to 2.0.206. Some of the queries that used to work in the older version are not working properly after the upgrade.
CREATE TABLE SOMETABLE (
ID INT(11) NOT NULL AUTO_INCREMENT,
SOURCE_ID VARCHAR(255) NOT NULL,
MESSAGE VARCHAR(255) NOT NULL,
PRIMARY KEY (`ID`)
);
CREATE TABLE IF NOT EXISTS SOMEOTHERTABLE (
ID VARCHAR(255) NOT NULL,
NAME VARCHAR(255) NOT NULL,
CREATED_TIME TIMESTAMP NOT NULL,
LAST_MODIFIED TIMESTAMP NOT NULL,
HAS_FILE BOOLEAN(1) NOT NULL,
PRIMARY KEY (ID)
);
For both these, I get similar errors
org.h2.jdbc.JdbcSQLSyntaxErrorException: Syntax error in SQL statement " CREATE TABLE SOMETABLE ( ID INT([*]11) NOT NULL AUTO_INCREMENT, SOURCE_ID VARCHAR(255) NOT NULL, MESSAGE VARCHAR(255) NOT NULL, PRIMARY KEY (`ID`) )"; expected "ARRAY, INVISIBLE, VISIBLE, NOT, NULL, AS, DEFAULT, GENERATED, ON, NOT, NULL, AUTO_INCREMENT, DEFAULT, NULL_TO_DEFAULT, SEQUENCE, SELECTIVITY, COMMENT, CONSTRAINT, COMMENT, PRIMARY, UNIQUE, NOT, NULL, CHECK, REFERENCES, AUTO_INCREMENT, ., )";
org.h2.jdbc.JdbcSQLSyntaxErrorException: Syntax error in SQL statement " CREATE TABLE IF NOT EXISTS SOMEOTHERTABLE ( ID VARCHAR(255) NOT NULL, NAME VARCHAR(255) NOT NULL, CREATED_TIME TIMESTAMP NOT NULL, LAST_MODIFIED TIMESTAMP NOT NULL, HAS_FILE BOOLEAN([*]1) NOT NULL, PRIMARY KEY (ID) )"; expected "ARRAY, INVISIBLE, VISIBLE, NOT, NULL, AS, DEFAULT, GENERATED, ON, NOT, NULL, AUTO_INCREMENT, DEFAULT, NULL_TO_DEFAULT, SEQUENCE, SELECTIVITY, COMMENT, CONSTRAINT, COMMENT, PRIMARY, UNIQUE, NOT, NULL, CHECK, REFERENCES, AUTO_INCREMENT, ., )";
It seems that in both these cases, having INT(11) and BOOLEAN(1) is the issue. Are those not allowed anymore in the new version? If so, how should I change those? Any help regarding this is appreciated.
I was facing the same issue when updating the h2 version from 1.4.200 to 2.0.206. The project is based on Spring Boot and uses Hibernate.
In my case the problem was, that i had an entity with a field referencing a table column called "VALUE".
#Column(name = "VALUE")
private BigDecimal value;
According to the docs https://h2database.com/html/advanced.html#keywords, VALUE is a reserved keyword. Changing the column reference "VALUE" to something like "VALUE1" solved the issue.
Why do you have such definitions? Documentation of H2 1.4.200 doesn't allow any parameters for these data types.
INT(11) is allowed only in MySQL and MariaDB compatibility modes, but the specified precision is ignored by H2. This definition is rejected in all other compatibility modes in H2 2.0, you need to use INT or INTEGER.
BOOLEAN(1) is not allowed at all, if it worked in 1.4.200, it was a bug in the parser. You need to use BOOLEAN.
AUTO_INCREMENT clause also should normally be used only in MySQL and MariaDB compatibility modes, but it works in Regular mode too. The proper clause is GENERATED BY DEFAULT AS IDENTITY and explicit NOT NULL constraint isn't required for primary key and identity columns, you can remove it. Constraints also should normally be specified after all other clauses, NOT NULL before identity options is actually accepted by H2, but this wrong order of clauses isn't documented and isn't supported.
Had a similar issue with Hibernate and Spring Boot and as #Plumstone mentioned it was due to a reserved keyword (in my case constraint). Adding backticks to the name, solved the issue:
#Column(name = "`constraint`")
private String constraint;
I have been trying for quite some time to figure out a solution for my problem, to no avail.
Anyway, i have a bunch of integration tests (in a nonstandard directory testRegression parallel to the standard test directory).
These integration tests use an h2 in memory database. In production as well as for testing i am using liquibase to simulate the schema evolution.
My properties (in application-testRegession.properties) look as follows:
spring.liquibase.enabled=true
spring.liquibase.user=sa
spring.liquibase.password=
spring.liquibase.change-log=classpath:/liquibase/changelog-master.xml
spring.datasource.url=jdbc:p6spy:h2:mem:testdb;MODE=PostgreSQL;DB_CLOSE_DELAY=-1;DATABASE_TO_UPPER=FALSE;INIT=CREATE SCHEMA IF NOT EXISTS nmc\\;CREATE SCHEMA IF NOT EXISTS mkt\\;CREATE SCHEMA IF NOT EXISTS cdb\\;CREATE SCHEMA IF NOT EXISTS pg_temp
spring.datasource.driver-class-name=com.p6spy.engine.spy.P6SpyDriver
spring.datasource.username=sa
spring.datasource.password=
The error i consistenly keep getting is:
2020-07-21 15:57:34.173 INFO [liquibase.lockservice.StandardLockService] [Test worker:13]: Successfully acquired change log lock
2020-07-21 15:57:34.303 INFO [liquibase.changelog.StandardChangeLogHistoryService] [Test worker:13]: Creating database history table with name: PUBLIC.DATABASECHANGELOG
2020-07-21 15:57:34.305 INFO [liquibase.executor.jvm.JdbcExecutor] [Test worker:13]: CREATE TABLE PUBLIC.DATABASECHANGELOG (ID VARCHAR(255) NOT NULL, AUTHOR VARCHAR(255) NOT NULL, FILENAME VARCHAR(255) NOT NULL, DATEEXECUTED TIMESTAMP NOT NULL, ORDEREXECUTED INT NOT NULL, EXECTYPE VARCHAR(10) NOT NULL, MD5SUM VARCHAR(35), DESCRIPTION VARCHAR(255), COMMENTS VARCHAR(255), TAG VARCHAR(255), LIQUIBASE VARCHAR(20), CONTEXTS VARCHAR(255), LABELS VARCHAR(255), DEPLOYMENT_ID VARCHAR(10))
2020-07-21 15:57:34.307 INFO [liquibase.lockservice.StandardLockService] [Test worker:13]: Successfully released change log lock
2020-07-21 15:57:34.309 WARN [org.springframework.boot.web.servlet.context.AnnotationConfigServletWebServerApplicationContext] [Test worker:13]: Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'liquibase' defined in class path resource [org/springframework/boot/autoconfigure/liquibase/LiquibaseAutoConfiguration$LiquibaseConfiguration.class]: Invocation of init method failed; nested exception is liquibase.exception.DatabaseException: Table "DATABASECHANGELOG" already exists; SQL statement:
CREATE TABLE PUBLIC.DATABASECHANGELOG (ID VARCHAR(255) NOT NULL, AUTHOR VARCHAR(255) NOT NULL, FILENAME VARCHAR(255) NOT NULL, DATEEXECUTED TIMESTAMP NOT NULL, ORDEREXECUTED INT NOT NULL, EXECTYPE VARCHAR(10) NOT NULL, MD5SUM VARCHAR(35), DESCRIPTION VARCHAR(255), COMMENTS VARCHAR(255), TAG VARCHAR(255), LIQUIBASE VARCHAR(20), CONTEXTS VARCHAR(255), LABELS VARCHAR(255), DEPLOYMENT_ID VARCHAR(10)) [42101-197] [Failed SQL: (42101) CREATE TABLE PUBLIC.DATABASECHANGELOG (ID VARCHAR(255) NOT NULL, AUTHOR VARCHAR(255) NOT NULL, FILENAME VARCHAR(255) NOT NULL, DATEEXECUTED TIMESTAMP NOT NULL, ORDEREXECUTED INT NOT NULL, EXECTYPE VARCHAR(10) NOT NULL, MD5SUM VARCHAR(35), DESCRIPTION VARCHAR(255), COMMENTS VARCHAR(255), TAG VARCHAR(255), LIQUIBASE VARCHAR(20), CONTEXTS VARCHAR(255), LABELS VARCHAR(255), DEPLOYMENT_ID VARCHAR(10))]
2020-07-21 15:57:34.309 INFO [com.zaxxer.hikari.HikariDataSource] [Test worker:13]: HikariPool-3 - Shutdown initiated...
2020-07-21 15:57:34.324 INFO [com.zaxxer.hikari.HikariDataSource] [Test worker:13]: HikariPool-3 - Shutdown completed.
2020-07-21 15:57:34.326 INFO [org.apache.catalina.core.StandardService] [Test worker:13]: Stopping service [Tomcat]
2020-07-21 15:57:34.342 INFO [org.springframework.boot.autoconfigure.logging.ConditionEvaluationReportLoggingListener] [Test worker:13]:
Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
2020-07-21 15:57:34.345 ERROR [org.springframework.boot.SpringApplication] [Test worker:13]: Application run failed
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'liquibase' defined in class path resource [org/springframework/boot/autoconfigure/liquibase/LiquibaseAutoConfiguration$LiquibaseConfiguration.class]: Invocation of init method failed; nested exception is liquibase.exception.DatabaseException: Table "DATABASECHANGELOG" already exists; SQL statement:
So how can i get around this issue? My basic understanding is that each test class creates its own ApplicationContext. For that it creates and loads a liquibase bean into it.
However, this problem occurs only for 2 out of 42 tests.
I would really like to get to the bottom of this and understand whats going on.
Can anyone shed light on my problem?
ADDITIONALLY
The test all run fine individually, but when run as a group they fail.
UPDATE 1
The relevant properties are as follows:
spring.main.allow-bean-definition-overriding=true
spring.datasource.url=jdbc:p6spy:h2:mem:testdb;MODE=PostgreSQL;DB_CLOSE_DELAY=-1;DATABASE_TO_UPPER=FALSE;INIT=CREATE SCHEMA IF NOT EXISTS nmc\\;CREATE SCHEMA IF NOT EXISTS mkt\\;CREATE SCHEMA IF NOT EXISTS cdb\\;CREATE SCHEMA IF NOT EXISTS pg_temp
spring.datasource.driver-class-name=com.p6spy.engine.spy.P6SpyDriver
spring.datasource.username=sa
spring.datasource.password=
spring.datasource.hikari.connectionTimeout=10000
spring.datasource.hikari.idleTimeout=60000
spring.datasource.hikari.maxLifetime=180000
spring.datasource.hikari.maximumPoolSize=50
spring.h2.console.enabled=true
spring.h2.console.path=/h2-console
My configuration is:
#Configuration
#ComponentScan(
basePackages = {
"com.aareal.nmc"
},
excludeFilters = {
#ComponentScan.Filter(type = FilterType.ASSIGNABLE_TYPE, value = CommandLineRunner.class)
})
#EnableTransactionManagement
#Profile("testRegression")
#SpringBootApplication(exclude = SecurityAutoConfiguration.class)
#EnableConfigurationProperties(LiquibaseProperties.class)
public class RegressionTestConfig {
My two tests are annotated as:
#RunWith(SpringRunner.class)
#SpringBootTest(
classes = {
RegressionTestConfig.class
},
//webEnvironment = SpringBootTest.WebEnvironment.DEFINED_PORT)
webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
Thanks
I had the same issue, and it seems to have been caused by case sensitive checking of the database table name. That is, the table was created as 'DATABASECHANGELOG', but Liquibase was checking for the existence of 'databasechangelog'.
The fix (at least for an H2 database) is to specify case insensitive identifiers in the database URL. For example:
jdbc:h2:mem:~/mydb;CASE_INSENSITIVE_IDENTIFIERS=TRUE
Explanation: The Spring test process fires up one or more instances of the Spring container to run tests on. If it thinks the config is exactly the same for two tests it will re-use an instance, otherwise it will start a new one. The instances are shared to avoid needing to start a whole new Springboot application for each test. But, the problem is that the instances may share some resources, such as databases and network ports. Therefore errors might occur from trying to start multiple instances at the same time. In this case, the test suite is starting two instances using the same database, but the second one is trying to re-run the whole Liquibase setup because the case sensitive issue means it doesn't see the table has already been created.
For my special case (that is for internal testing only, not production) what i have the following:
src
|-- main
|-- test
|-- testRegression
Workaround
Decide on the version of liquibase to use (i chose 4.0.0, which is the most recent at this point)
Create a file "src/testRegression/java/liquibase/changelog/StandardChangeLogHistoryService.java"
Open the original liquibase file "StandardChangeLogHistoryService.java" (mine is in ~//.gradle/caches/modules-2/files-2.1/org.liquibase/liquibase-core/4.0.0/23a5317eb5005b4765cd85e6f3a2cc4bb55c0daa/liquibase-core-4.0.0-sources.jar which i copied and unzipped)
and copy its contents 1:1 into the newly created file in 2.
Add a catch block by changing code (around line 396) from
if (SqlGeneratorFactory.getInstance().supports(sql, database)) {
executor.execute(sql);
getDatabase().commit();
} else {
to
if (SqlGeneratorFactory.getInstance().supports(sql, database)) {
try {
executor.execute(sql);
getDatabase().commit();
} catch (DatabaseException excptn) {
Scope.getCurrentScope()
.getLog(getClass())
.warning(
"Table '"
+ getDatabase()
.escapeTableName(
getLiquibaseCatalogName(),
getLiquibaseSchemaName(),
getDatabaseChangeLogTableName())
+ "' already exists.");
}
} else {
This is simply a workaround since there could be legitimate reasons for the ChangeLogTable table creation to fail. However, it already existing should not be cause for a major failure in my opinion.
My current view is that this is something which ought to be addressed/fixed in the official liquibase code base.
The following post(s) were helpful:
https://github.com/liquibase/liquibase-cache/issues/1
I am using Spring Boot 2 with Data JPA and Flyway on a Postgres database. Everything works fine in production. Now I'm trying to write tests that will run on an H2 Embedded Database for testing purposes. But, the tests encounter a SQL syntax error. But, I don't understand what's wrong with the syntax:
CREATE TABLE mysite_user (
id int8 NOT NULL,
thirdparty_user_id varchar(255) NULL,
email varchar(255) NULL,
first_name varchar(255) NULL,
PRIMARY KEY (id)
);
CREATE INDEX mysite_user_thirdparty_user_id_idx ON mysite_user USING btree (thirdparty_user_id) ;
This is the error:
Migration V1__Initial.sql failed
--------------------------------
SQL State : 42001
Error Code : 42001
Message : Syntax error in SQL statement "CREATE INDEX MYSITE_USER_THIRDPARTY_USER_ID_IDX ON MYSITE_USER USING[*] BTREE (THIRDPARTY_USER_ID) "; expected "., COMMENT, ("; SQL statement:
CREATE INDEX mysite_user_thirdparty_user_id_idx ON mysite_user USING btree (thirdparty_user_id) [42001-199]
Location : db/migration/V1__Initial.sql (/Users/me/Development/mysite-website/target/classes/db/migration/V1__Initial.sql)
Line : 20
Statement : CREATE INDEX mysite_user_thirdparty_user_id_idx ON mysite_user USING btree (thirdparty_user_id)
What am I doing wrong?
Arguably, what you're doing wrong is using different databases between prod and test. As you've seen, your tests end up having to be coded around the differences between the databases, as well as potentially overlooking subtle edge cases which work in one db and not the other.
If you want to go down this route it is possible - see Best way for "database specific" sql scripts with Flyway
so I'm trying to use symmetricDS for replicating java h2 database to postgres. I'm using the zip file simple configuration. Here is what happen. I followed the getting started guide, download the symmetricds, and try the demo, then I tried with my own configuration with some table in the trigger. But:
If I replicate the table without varchar field in h2 it works perfectly fine.
If I have a table that has varchar field in it, it crash during creating the table.
JdbcSqlTemplate - ERROR: length for type varchar cannot exceed 10485760
Position: 161. Failed to execute: CREATE TABLE "asset"(
"db_id" BIGINT NOT NULL DEFAULT nextval('"asset_db_id_seq"'),
"id" BIGINT NOT NULL,
"account_id" BIGINT NOT NULL,
"name" VARCHAR(2147483647) NOT NULL,
"description" VARCHAR(2147483647),
"quantity" BIGINT NOT NULL,
"decimals" SMALLINT NOT NULL,
"initial_quantity" BIGINT NOT NULL,
"height" INTEGER NOT NULL,
"latest" BOOLEAN DEFAULT 'TRUE' NOT NULL,
PRIMARY KEY ("db_id")
)
indeed a clear error saying the varchar should not exceed 255, but that's how the source database is, is there anyway to force any varchar to TEXT type? or are there any other way around this? or is this a bug in symmetricds has yet to be solved?
Thanks.
I managed to go way around this by creating the table on target database manually. Here is what I did before running bin/sym.
generate query for table I want to create using dbexport by bin/dbexport --engine corp-000 --compatible=postgres --no-data table_a table_b > samples/create_asset_and_trade.sql
modify the flaw in generated query file samples/create_asset_and_trade.sql. in my case it's the length of the varchar.
after fixing that, run the generated query to fill in the target database using dbimport. bin/dbimport --engine store-001 samples/create_asset_and_trade.sql.
running bin/sym should be okay now, it'll detect that the table is already created, and skip the table creation step.
This is not the ideal way, but it should work for now.