Import CSV File to H2 before Migration - java

I'm trying to use Flyway to migrate an in-memory h2 database. Creating the table works fine the traditional way, but I would also like to use h2's import csv as described here. I can't find anything in the Flyway documentation that says this won't work. Is this supported?
UPDATE: This is the V02__createTables.sql that I'm trying to migrate. This script is in db.migration.test and Flyway knows about the directory. The csv file an_products.csv is also in this folder, hence being referenced by h2's CSVREAD method below.
CREATE TABLE an_products (
id int(10) PRIMARY KEY NOT NULL,
name varchar(100) NOT NULL DEFAULT '',
price double NOT NULL DEFAULT '0.00',
taxable tinyint(4) NOT NULL DEFAULT '0',
notes varchar(255) DEFAULT NULL,
isPercent tinyint(3) NOT NULL DEFAULT '0',
min double DEFAULT NULL,
period char(1) NOT NULL DEFAULT '',
qty int(10) NOT NULL DEFAULT '0',
cost double NOT NULL DEFAULT '0.00'
) AS SELECT * FROM CSVREAD('an_products.csv');

There is no explicit support, nor is there a restriction. Flyway will execute all statements through JDBC. So whether this will work depends on whether H2 supports this statement through JDBC or not.

First, be sure the relative path works, maybe FlyWay has a different working directory, maybe just make the path absolute.
Or Use a debugger and tell it to break on Exception throws, this way, you know where it occurs.

Related

org.h2.jdbc.JdbcSQLSyntaxErrorException after H2 version upgrade

I recently upgraded h2 version from 1.4.200 to 2.0.206. Some of the queries that used to work in the older version are not working properly after the upgrade.
CREATE TABLE SOMETABLE (
ID INT(11) NOT NULL AUTO_INCREMENT,
SOURCE_ID VARCHAR(255) NOT NULL,
MESSAGE VARCHAR(255) NOT NULL,
PRIMARY KEY (`ID`)
);
CREATE TABLE IF NOT EXISTS SOMEOTHERTABLE (
ID VARCHAR(255) NOT NULL,
NAME VARCHAR(255) NOT NULL,
CREATED_TIME TIMESTAMP NOT NULL,
LAST_MODIFIED TIMESTAMP NOT NULL,
HAS_FILE BOOLEAN(1) NOT NULL,
PRIMARY KEY (ID)
);
For both these, I get similar errors
org.h2.jdbc.JdbcSQLSyntaxErrorException: Syntax error in SQL statement " CREATE TABLE SOMETABLE ( ID INT([*]11) NOT NULL AUTO_INCREMENT, SOURCE_ID VARCHAR(255) NOT NULL, MESSAGE VARCHAR(255) NOT NULL, PRIMARY KEY (`ID`) )"; expected "ARRAY, INVISIBLE, VISIBLE, NOT, NULL, AS, DEFAULT, GENERATED, ON, NOT, NULL, AUTO_INCREMENT, DEFAULT, NULL_TO_DEFAULT, SEQUENCE, SELECTIVITY, COMMENT, CONSTRAINT, COMMENT, PRIMARY, UNIQUE, NOT, NULL, CHECK, REFERENCES, AUTO_INCREMENT, ., )";
org.h2.jdbc.JdbcSQLSyntaxErrorException: Syntax error in SQL statement " CREATE TABLE IF NOT EXISTS SOMEOTHERTABLE ( ID VARCHAR(255) NOT NULL, NAME VARCHAR(255) NOT NULL, CREATED_TIME TIMESTAMP NOT NULL, LAST_MODIFIED TIMESTAMP NOT NULL, HAS_FILE BOOLEAN([*]1) NOT NULL, PRIMARY KEY (ID) )"; expected "ARRAY, INVISIBLE, VISIBLE, NOT, NULL, AS, DEFAULT, GENERATED, ON, NOT, NULL, AUTO_INCREMENT, DEFAULT, NULL_TO_DEFAULT, SEQUENCE, SELECTIVITY, COMMENT, CONSTRAINT, COMMENT, PRIMARY, UNIQUE, NOT, NULL, CHECK, REFERENCES, AUTO_INCREMENT, ., )";
It seems that in both these cases, having INT(11) and BOOLEAN(1) is the issue. Are those not allowed anymore in the new version? If so, how should I change those? Any help regarding this is appreciated.
I was facing the same issue when updating the h2 version from 1.4.200 to 2.0.206. The project is based on Spring Boot and uses Hibernate.
In my case the problem was, that i had an entity with a field referencing a table column called "VALUE".
#Column(name = "VALUE")
private BigDecimal value;
According to the docs https://h2database.com/html/advanced.html#keywords, VALUE is a reserved keyword. Changing the column reference "VALUE" to something like "VALUE1" solved the issue.
Why do you have such definitions? Documentation of H2 1.4.200 doesn't allow any parameters for these data types.
INT(11) is allowed only in MySQL and MariaDB compatibility modes, but the specified precision is ignored by H2. This definition is rejected in all other compatibility modes in H2 2.0, you need to use INT or INTEGER.
BOOLEAN(1) is not allowed at all, if it worked in 1.4.200, it was a bug in the parser. You need to use BOOLEAN.
AUTO_INCREMENT clause also should normally be used only in MySQL and MariaDB compatibility modes, but it works in Regular mode too. The proper clause is GENERATED BY DEFAULT AS IDENTITY and explicit NOT NULL constraint isn't required for primary key and identity columns, you can remove it. Constraints also should normally be specified after all other clauses, NOT NULL before identity options is actually accepted by H2, but this wrong order of clauses isn't documented and isn't supported.
Had a similar issue with Hibernate and Spring Boot and as #Plumstone mentioned it was due to a reserved keyword (in my case constraint). Adding backticks to the name, solved the issue:
#Column(name = "`constraint`")
private String constraint;

SymmetricDS length for type varchar cannot exceed N

so I'm trying to use symmetricDS for replicating java h2 database to postgres. I'm using the zip file simple configuration. Here is what happen. I followed the getting started guide, download the symmetricds, and try the demo, then I tried with my own configuration with some table in the trigger. But:
If I replicate the table without varchar field in h2 it works perfectly fine.
If I have a table that has varchar field in it, it crash during creating the table.
JdbcSqlTemplate - ERROR: length for type varchar cannot exceed 10485760
Position: 161. Failed to execute: CREATE TABLE "asset"(
"db_id" BIGINT NOT NULL DEFAULT nextval('"asset_db_id_seq"'),
"id" BIGINT NOT NULL,
"account_id" BIGINT NOT NULL,
"name" VARCHAR(2147483647) NOT NULL,
"description" VARCHAR(2147483647),
"quantity" BIGINT NOT NULL,
"decimals" SMALLINT NOT NULL,
"initial_quantity" BIGINT NOT NULL,
"height" INTEGER NOT NULL,
"latest" BOOLEAN DEFAULT 'TRUE' NOT NULL,
PRIMARY KEY ("db_id")
)
indeed a clear error saying the varchar should not exceed 255, but that's how the source database is, is there anyway to force any varchar to TEXT type? or are there any other way around this? or is this a bug in symmetricds has yet to be solved?
Thanks.
I managed to go way around this by creating the table on target database manually. Here is what I did before running bin/sym.
generate query for table I want to create using dbexport by bin/dbexport --engine corp-000 --compatible=postgres --no-data table_a table_b > samples/create_asset_and_trade.sql
modify the flaw in generated query file samples/create_asset_and_trade.sql. in my case it's the length of the varchar.
after fixing that, run the generated query to fill in the target database using dbimport. bin/dbimport --engine store-001 samples/create_asset_and_trade.sql.
running bin/sym should be okay now, it'll detect that the table is already created, and skip the table creation step.
This is not the ideal way, but it should work for now.

SQLException when specifying length along with default value for varchar

I'm trying to initialise an inmemory H2 database with Spring using something like the following statement:
===Throws SQLException===
CREATE TABLE config (
id INTEGER NOT NULL,
module_id VARCHAR(10) NOT NULL DEFAULT 'global',
...
);
Every time I try to run the project an SQLException is thrown saying the syntax is wrong. After quite a bit of experimenting I have noted that when I remove the DEFAULT keyword, there is no exception thrown.
===This works===
CREATE TABLE config (
id INTEGER NOT NULL,
module_id VARCHAR(10) NOT NULL,
...
);
What is even more interesting is that when I removed the length on the VARCHAR and kept the DEFAULT keyword as it is also it worked.
===This also works===
CREATE TABLE config (
id INTEGER NOT NULL,
module_id VARCHAR NOT NULL DEFAULT 'global',
...
);
So the question is, is there something preventing both of these being specified together?
According to a strict reading of the syntax diagram in documentation for the SQL grammar...
the default clause has to come before the NOT NULL.

MYSQL Records Disappear (Java, c3p0)

I have a JavaFX application with modules, interacting with MySQL database in different aspects. Dozens of modules communicate it successfully. But the module I test adds records to a table successfully (with INSERT IGNORE ... ON DUPLICATE KEY UPDATE), then the same record is updated (UPDATE successful), and 15 minutes later this record disappears from database. id autoincrement doesn't roll back.
I noticed that records, that were inserted and never experienced my update query, don't disappear. Other old records in this table remain untouched. The problematic record is affected with this query:
UPDATE `order`
SET
`create_date` = '2014-03-20 00:00:00',
`discount_coupon_id` = 0,
`discount_percent_value` = 5,
`comment` = '',
`comment_length` = 0,
`delivery_address` = 'Perm',
`pay_date` = DEFAULT,
`shipped_date` = DEFAULT,
`delivery_date` = DEFAULT
WHERE id = 407
After this query was executed, I've closed application and exited IDE, checked that record 407 is present and is updated (in HeidiSQL DB Editor). 15 minutes later I've checked for records' state and saw record 407 disappeared.
Details on table updated:
CREATE TABLE `order` (
`id` INT(10) UNSIGNED NOT NULL AUTO_INCREMENT,
`create_date` DATETIME NULL DEFAULT NULL,
`customer_id` INT(10) UNSIGNED NOT NULL,
`discount_coupon_id` INT(10) UNSIGNED NULL DEFAULT NULL,
`discount_percent_value` FLOAT NULL DEFAULT NULL,
`comment` TEXT NOT NULL,
`comment_length` INT(10) UNSIGNED NOT NULL DEFAULT '0',
`delivery_address` CHAR(255) NOT NULL,
`pay_date` DATETIME NULL DEFAULT NULL,
`shipped_date` DATETIME NULL DEFAULT NULL,
`delivery_date` DATETIME NULL DEFAULT NULL,
PRIMARY KEY (`id`)
)
COLLATE='utf8_general_ci'
ENGINE=MyISAM
AUTO_INCREMENT=408;
I use c3p0 connection manager with default setup. Query is called like this:
try (
Connection c = m.cpdsSayt.getConnection();
Statement stmt = c.createStatement();) {
stmt.executeUpdate(SQL);
}
I've studied similar questions like this, but found no clue to my case, as I use no PROCEDURE, ROLLBACK or TRANSACTION. Though it looks like something is corrupted or unfinished from mysql's point of view and it throws out suspicious changes.
Application is cleint side and database is connected remotely, based on Hetzner virtual server.

com.mysql.jdbc.MysqlDataTruncation: Data truncation: Incorrect date value: '' for column 'Date_Of_Birth' at row 1

I'm trying to update the personal detail of a user through a java panel.Panel has fields like user_id(autogenerated),name,date of birth.
problem is when i enter nothing to the date of birth field in java panel and then save it. It gives me the above mentioned error.
i tried to verify it by inserting null to the date of birth(Date datatype) field directly using the mysql database.There it gives no error.
Why is it not taking null string when i insert through java panel but is taking when insert directly using mysql.
CREATE TABLE `userpersonaldetail` (
`User_Id` int(10) unsigned NOT NULL auto_increment,
`Name` varchar(45) default NULL,
`Date_Of_Birth` date default NULL,
`Address` varchar(300) default NULL,
`Phone` varchar(20) default NULL,
`Mobile` varchar(20) default NULL,
`Email` varchar(50) default NULL,
PRIMARY KEY (`User_Id`),
CONSTRAINT `FK_userpersonaldetail_1` FOREIGN KEY (`User_Id`) REFERENCES `usermaster` (`User_Id`)
)
And the portion of the code where exception occurs is:
try
{
con=(Connection) DBConnection.getConnection();
pstmt=(PreparedStatement) con.prepareStatement("Update userpersonaldetail set "+
"name=?,date_of_birth=?,address=?,phone=?,mobile=?,email=? where user_id=?");
pstmt.setInt(7,perBean.getUserId());
pstmt.setString(1,perBean.getName());
pstmt.setString(2,perBean.getDateOfBirth());
pstmt.setString(3,perBean.getAddress());
pstmt.setString(4,perBean.getPhone());
pstmt.setString(5,perBean.getMobile());
pstmt.setString(6,perBean.getEmail());
int i=pstmt.executeUpdate();
}
here perBean is the javaBean which retrieves values from the gui.In one of the test case i kept the date_of_birth text box null which is giving error while storing in DB.
My initial guess would be the field has been defined as 'NOT NULL' which means it will force you to enter a value...
If you were to do a mysql dump of the table (or view it in some tool) you'll probably find it defined such as:
`someDT` datetime NOT NULL
I replaced older version of my-sql-connector jar (to be found in lib folder of the server) with the latest. That solved my problem.
Ahh, i see now, you can't insert '' as a date. you will need to pass a date.
If an attribute does not have a value but you have it mentioned in the column list, forcing you to give something there, you need to use
statement.setNull(index, datatype)
to it. Setting to "" is not the same thing as setting to null.

Categories

Resources