I have a production and a QA instance of my application into which I'm integrating Liquibase. This means DDL and data already exists (or not if on development box). I have to create a changeLog which records everything as RAN on the non-empty DBs but execute actually on empty DBs.
I'm on a good way but I'm a bit stuck with creating the foreign keys. (the database is Oracle).
(In general I'm creating preconditions which expects various objects to NOT exists and on fail MARK_RAN the change).
I find difficulties writing a correct precondition when I don't know the exact name of foreign keys, which may or may not exist.
There is <foreignKeyConstraintExists> tag in liquibase (precondition) but it takes only schemaName and foreignKeyName attributes (and they are required). I don't know the foreign key names for sure in these instances as they are out of my control.
You can write custom SQL in preconditions like:
<changeSet id="1" author="bob">
<preConditions onFail="WARN">
<sqlCheck expectedResult="0">select count(*) from oldtable</sqlCheck>
</preConditions>
<dropTable tableName="oldtable"/>
</changeSet>
So I only have to create a custom SQL query which can check if a column on table A has foreign key referencing table B and use the result as a precondition.
This is where my problem is because you can do it in Oracle but it's quite bloat:
SELECT a.table_name, a.column_name, a.constraint_name, c.owner,
c.r_owner, c_pk.table_name r_table_name
FROM all_cons_columns a
JOIN all_constraints c ON a.owner = c.owner
AND a.constraint_name = c.constraint_name
JOIN all_constraints c_pk ON c.r_owner = c_pk.owner
AND c.r_constraint_name = c_pk.constraint_name
WHERE c.constraint_type = 'R' AND a.table_name = 'MY_TABLE'
AND a.column_name = 'MY_COLUMN'
AND c_pk.table_name = 'MY_OTHER_TABLE';
This prints a row if a foreign key exists on MY_COLUMN of MY_TABLE which references to MY_OTHER_TABLE. After rewriting it to COUNT you can check if there's foreign key without knowing it's name.
My question:
I have dozens of foreign keys, do I really have to write this big SQL such dozens of times? Any suggestions, like outsourcing this to some function? Thanks!
Would it worth asking Liquibase developers to make <foreignKeyConstraintExists> 's name attribute optional and introduce the referenced table attribute alogn with local column name?
There is one more possibility: implementing the interface http://www.liquibase.org/javadoc/liquibase/precondition/CustomPrecondition.html and use it as a custom precondition. More info: http://www.liquibase.org/documentation/preconditions.html
Here is the implementation (verified):
import liquibase.database.Database;
import liquibase.exception.CustomPreconditionErrorException;
import liquibase.exception.CustomPreconditionFailedException;
import liquibase.precondition.CustomPrecondition;
import liquibase.snapshot.SnapshotGeneratorFactory;
import liquibase.structure.core.ForeignKey;
import liquibase.structure.core.Schema;
import liquibase.structure.core.Table;
import liquibase.util.StringUtils;
/**
* {#link CustomPrecondition} implementation that checks if a column on a table
* has a foreign key constraint for some other table.
*/
public final class CheckForeignKey implements CustomPrecondition {
/**
* Schema.
*/
private String schemaName;
/**
* Table name (that has the column).
*/
private String tableName;
/**
* Column (that might have the foreign key).
*/
private String columnName;
/**
* Referenced table of the foreign key.
*/
private String foreignTableName;
#Override
public void check(final Database db)
throws CustomPreconditionFailedException,
CustomPreconditionErrorException {
try {
// The fkey we are looking for
final ForeignKey fKey = new ForeignKey();
// Schema, base table
fKey.setForeignKeyTable(new Table());
if (StringUtils.trimToNull(getTableName()) != null) {
fKey.getForeignKeyTable().setName(getTableName());
}
final Schema schema = new Schema();
schema.setName(getSchemaName());
fKey.getForeignKeyTable().setSchema(schema);
// Base column
fKey.addForeignKeyColumn(getColumnName());
// Referenced table
fKey.setPrimaryKeyTable(new Table());
if (StringUtils.trimToNull(getForeignTableName()) != null) {
fKey.getPrimaryKeyTable().setName(getForeignTableName());
}
if (!SnapshotGeneratorFactory.getInstance().has(fKey, db)) {
throw new CustomPreconditionFailedException(
String.format(
"Error fkey not found schema %s table %s column %s ftable %s",
getSchemaName(), getTableName(),
getColumnName(), getForeignTableName()));
}
} catch (final CustomPreconditionFailedException e) {
throw e;
} catch (final Exception e) {
throw new CustomPreconditionErrorException("Error", e);
}
}
public String getSchemaName() {
return schemaName;
}
public void setSchemaName(final String schemaName) {
this.schemaName = schemaName;
}
public String getTableName() {
return tableName;
}
public void setTableName(final String tableName) {
this.tableName = tableName;
}
public String getColumnName() {
return columnName;
}
public void setColumnName(final String columnName) {
this.columnName = columnName;
}
public String getForeignTableName() {
return foreignTableName;
}
public void setForeignTableName(final String foreignTableName) {
this.foreignTableName = foreignTableName;
}
}
I think that you have to do it like you suggested if you dont know foreign key constraint names.
But if you can modify database then you could prepare sql script which prepare another sql script which renames all FK to well known names. Something like that:
BEGIN
FOR cur IN (
SELECT
c_list.CONSTRAINT_NAME as FK_NAME,
'FK_' || c_dest.TABLE_NAME || '_' || substr(c_dest.COLUMN_NAME, 1, 20) as NEW_FK_NAME,
c_src.TABLE_NAME as SRC_TABLE,
c_src.COLUMN_NAME as SRC_COLUMN,
c_dest.TABLE_NAME as DEST_TABLE,
c_dest.COLUMN_NAME as DEST_COLUMN
FROM ALL_CONSTRAINTS c_list, ALL_CONS_COLUMNS c_src, ALL_CONS_COLUMNS c_dest
WHERE c_list.CONSTRAINT_NAME = c_src.CONSTRAINT_NAME
AND c_list.R_CONSTRAINT_NAME = c_dest.CONSTRAINT_NAME
AND c_list.CONSTRAINT_TYPE = 'R'
AND c_src.TABLE_NAME IN ('<your-tables-here>')
GROUP BY c_list.CONSTRAINT_NAME, c_src.TABLE_NAME, c_src.COLUMN_NAME, c_dest.TABLE_NAME, c_dest.COLUMN_NAME;
) LOOP
-- Generate here SQL commands (by string concatenation) something like:
-- alter table SRC_TABLE rename constraint FK_NAME to NEW_FK_NAME;
-- then paste this sql commands to some other script and run it
END LOOP;
END;
This is one time migration.
After this migration you know whats your FK constraint names are and you can use <foreignKeyConstraintExists> precondition in your changesets.
Related
I want to embed date information in the primary key, for a table that will be partitioned (monthly) in a PostgreSQL database. This should in theory speed up the process on finding out in which partition to look for the data. I followed this article to embed the date in a date into the serial.
Now, I am however facing the problem that I can't get the Id been used by Hibernate.
c.f. the sql that should give an idea of the attempted approach.
CREATE SEQUENCE test_serial START 1;
CREATE OR REPLACE FUNCTION gen_test_key() RETURNS BIGINT AS $$
DECLARE
new_id bigint;
BEGIN
new_id = (nextval('public.test_serial'::regclass)::bigint % 10000000000000::bigint
+ ( (EXTRACT(year from now())-2000)::bigint * 10000::bigint
+ EXTRACT(month from now())::bigint * 100::bigint
+ EXTRACT(day from now())::bigint
)::bigint * 10000000000000::bigint
)::bigint;
RETURN new_id;
END;
$$ LANGUAGE plpgsql;
CREATE TABLE test
( id bigint primary key default gen_test_key(),
something text,
tstamp timestamp default now()
) PARTITION BY RANGE (id);
CREATE TABLE test_2022_10 PARTITION OF test
FOR VALUES FROM (2210100000000000000::bigint ) TO (2211010000000000000::bigint);
I came across a similar question, where it was suggested to use a stored procedure. Unfortunately only functions are allowed as default in the table definition and therefore stored procedures, seam not to work for me.
I think what you need here is a subtype of SequenceStyleGenerator that overrides determineBulkInsertionIdentifierGenerationSelectFragment to run the code of this function. You should be able to configure this generator on your entity with #GenericGenerator. I understand the desire to use this concept when you don't want to change your existing queries, but are you sure that partitioning will help you in your use case?
Also, be careful and do not rely on the date information in the primary key, because with pooled optimizers, it might happen that a value is generated way before it actually is used as primary key for a row.
So this is a solution that worked out in the end as suggested #ChristianBeikov here the entity with the annotations pointing to the CustomIdGenerator.
public class Test {
#Id
#GenericGenerator(name = "CustomIdGenerator", strategy = "nl.test.components.CustomIdGenerator")
#GeneratedValue(generator = "CustomIdGenerator")
private Long id;
private String something;
private OffsetDateTime tstamp;
}
As explained by #Mr_Thorynque it is similarly possible to call a stored function as a procedure. Just replace "CALL gen_test_key()" with "SELECT gen_test_key()" and don't pass it to the wrong method for stored procedures connection.prepareCall(CALL_STORE_PROC);, but instead connection.prepareStatement(STORED_FUNCTION); So, this is the CustomIdGenerator.
public class CustomIdGenerator implements IdentifierGenerator {
private static final String STORED_FUNCTION = "select gen_test_key()";
#Override
public Serializable generate(SharedSessionContractImplementor session, Object object) throws HibernateException {
Long result = null;
try {
Connection connection = session.connection();
PreparedStatement pstmt = connection.prepareStatement(STORED_FUNCTION);
ResultSet resultSet = pstmt.executeQuery();
if (resultSet.next()) {
result = resultSet.getLong(1);
System.out.println("Generated Id: " + result);
}
} catch (SQLException sqlException) {
throw new HibernateException(sqlException);
}
return result;
}
}
I need to use the database Firebird and for this I use the Jaybird 2.2.9.
When I used the MySQL driver, to converter of ResultSet to Object this way:
empresa.setBairro(rs.getString("empresa.bairro")); // (Table.Column)
empresa.setCep(rs.getString("empresa.cep")); // (Table.Column)
empresa.setCidade(rs.getString("empresa.cidade")); // (Table.Column)
But with Jaybird the resultSet don't return rs.getString("Table.Column")
I need this way when I have inner join in SQL.
Anyone help me?
This is my full code
public ContaLivros converterContaLivros(ResultSet rs, Integer linha) throws Exception {
if (rs.first()) {
rs.absolute(linha);
ContaLivros obj = new ContaLivros();
obj.setId(rs.getLong("cad_conta.auto_id"));
obj.setNome(rs.getString("cad_conta.nome"));
if (contain("cad_banco.auto_id", rs)) {
obj.setBancoLivros(converterBancoLivros(rs, linha));
} else {
obj.setBancoLivros(new BancoLivros(rs.getLong("cad_conta.banco"), null, null, null));
}
obj.setAgencia(rs.getInt("cad_conta.agencia"));
obj.setAgenciaDigito(rs.getInt("cad_conta.agencia_digito"));
obj.setConta(rs.getInt("cad_conta.conta"));
obj.setContaDigito(rs.getInt("cad_conta.conta_digito"));
obj.setLimite(rs.getDouble("cad_conta.limite"));
obj.setAtivo(rs.getString("cad_conta.ativo"));
return obj;
} else {
return null;
}
}
You can't. Jaybird retrieves the columns by its label as specified in JDBC 4.2, section 15.2.3. In Firebird the column label is either the original column name, or the AS alias, the table name isn't part of this. The extension of MySQL that you can prefix the table name for disambiguation is non-standard.
Your options are to specify aliases in the query and retrieve by this aliasname, or to process the result set metadata to find the right indexes for each column and retrieve by index instead.
However note that in certain queries (for example UNION), the ResultSetMetaData.getTableName cannot return the table name, as Firebird doesn't "know" it (as you could be applying a UNION to selects from different tables).
The name in jdbc will not have the table in it.
You can either
work with positional parameters ( getString (1); and so on )
Or
define column name alias in your select (select a.name namefroma from tableone a )
Or
simply do rs.getString ("column"); without the table prefix if name is unambigous
I want to get all database table names that ends with _tbl like xyz_tbl, pqr_tbl,etc..
in mysql using java.pls help me.. currently my query retreives all tablename but i jst want table names that ends with _tbl.
My code is...
public List selectTable() {
List tableNameList= new ArrayList();
try {
DatabaseMetaData dbm = c.conn.getMetaData();
String[] types = {"TABLE"};
c.rs = dbm.getTables(null, null, "%", types);
while (c.rs.next()) {
tableNameList.add(c.rs.getString("TABLE_NAME"));
}
} catch(Exception e) {
System.out.println(e.toString());
}
return tableNameList;
}
Did you try using a different table name pattern?
You can try this: -
c.rs = dbm.getTables(null, null, "%_tbl", types);
You can use mysql query
show tables from tablename like '%_tbl';
I am unable to reply to Rohit's post. his answer looks correct.
If you do to JDK documentation for DatabaseMetaData's getTables method following is the signature and documentation comment.
ResultSet getTables(String catalog, String schemaPattern, String tableNamePattern,
String[] types) throws SQLException
Parameters:
catalog - a catalog name; must match the catalog name as it is stored
in the database; "" retrieves those without a catalog; null means that
the catalog name should not be used to narrow the search
schemaPattern
- a schema name pattern; must match the schema name as it is stored in the database; "" retrieves those without a schema; null means that the
schema name should not be used to narrow the search tableNamePattern -
a table name pattern; must match the table name as it is stored in the
database types - a list of table types, which must be from the list of
table types returned from getTableTypes(),to include; null returns all
types
In this case using %_tbl should work.
Use the String.endsWith() method to check if the table name ends with "_tbl".
For example inside your while loop:
while (c.rs.next())
{
String tableName = c.rs.getString("TABLE_NAME");
if(tableName.endsWith("_tbl"))
{
tableNameList.add(c.rs.getString("TABLE_NAME"));
}
}
I just created a custom hibernate ID generator, and since I'm not an hibernate expert I would like to get some feedback on my code. The generated ID is select max(id) from table, +1.
public class MaxIdGenerator implements IdentifierGenerator, Configurable {
private Type identifierType;
private String tableName;
private String columnName;
#Override
public void configure(Type type, Properties params, Dialect dialect) {
identifierType = type;
tableName = (String) params.getProperty("target_table");
columnName = (String) params.getProperty("target_column");
}
#Override
public synchronized Serializable generate(SessionImplementor session,
Object object) {
return generateHolder(session).makeValue();
}
protected IntegralDataTypeHolder generateHolder(SessionImplementor session) {
Connection connection = session.connection();
try {
IntegralDataTypeHolder value = IdentifierGeneratorHelper
.getIntegralDataTypeHolder(identifierType
.getReturnedClass());
String sql = "select max(" + columnName + ") from " + tableName;
PreparedStatement qps = connection.prepareStatement(sql);
try {
ResultSet rs = qps.executeQuery();
if (rs.next())
value.initialize(rs, 1);
else
value.initialize(1);
rs.close();
} finally {
qps.close();
}
return value.copy().increment();
} catch (SQLException e) {
throw new IdentifierGenerationException(
"Can't select max id value", e);
}
}
}
I'd like to know:
How can I make this multi-transaction-safe? (ie if two concurrent transactions insert data, how can I safely assume that I will not end-up having twice the same ID?) -- I guess the only solution here would be to prevent two concurrent hibernate transaction to run at the same time if they use the same generator, is this possible?
If the code could be improved: I feel wrong having to use hard-coded "select", "target_column", etc...
To guarantee point 1), I can fallback if necessary on synchronizing inserts on my java client code.
Please do not comment on the reasons why I'm using this kind of generator: legacy code still insert data onto the same database and uses this mechanism... and can't be modified. And yes, I know, it sucks.
I think the easiest way to achive a transaction-safe behaviour is to put the code you use to retrive the maximum id and do the insert statement, in a transactional block.
Something like:
Transaction transaction = session.beginTransaction();
//some code...
transaction.commit();
session.close()
I also recommend to use HQL (Hibernate Query Language) to create the query, instead of native sql where possible. Moreover, from your description, I have understood that you expect from the query a unique result, the maximum id. So, you could use uniqueResult() method over your query instead of executeQuery.
you can use AtomicInteger for generating ID. That can be used by many threads concurrently.
If you are free to use any other provider of ID then i will suggest to use UUID class for generating random ID.
UUID.randomUUID();
You can refer to link which contain some other ways to generate ID.
I'm writing a Java application that uses SQLite via JDBC for persistence.
What I need to do when the application starts is:
If the DB doesn't exists, create it with the right schema
If the DB exists, check if the DB has the right schema
In order to create the DB I thought of externalizing the SQL commands needed to create the structure (e.g. CREATE TABLE ...) in an external file, load it at runtime and execute it.
I thought of using BufferedReader with .readLine() and then feed each SQL command to a Statement.executeUpdate(). Is there some smarter way to do this?
Concerning the startup check of the database schema, while I think that this should be considered "good design", I don't know if in practice it could be overkill or useless.
One solution I came up with but has the disadvantage of being "vendor dependent" is this:
Create the DB by using a known prefix for all the structures' names (e.g. nwe_)
Query the sqlite_master table with WHERE name LIKE 'nwe_%'
Compare the content of the SQL column with the SQL command file I used for creating the DB
Again, is there some smarter way to do this? (maybe not "vendor dependent", but this at the time is not much of an issue for me)
Thanks.
I'm not a professional programmer and this is my first java application, but I decided to do
database verification too.
Fortunately, sqlite databases has a system table with sql statements for each table in database.
So there is a fast and dirty way to check database structure:
public class Model {
private Map<String, String> tableSql;
Model(){
// declare right database's tables syntax
tableSql = new HashMap<String, String>();
tableSql.put("settings", "CREATE TABLE [settings] ( [id] INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT, [item] ... )");
tableSql.put("scheta", "CREATE TABLE [scheta] ( [id] INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT, [sname] ...)");
tableSql.put("nomera", "CREATE TABLE [nomera] ( [id] INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT, [nvalue] ...)");
tableSql.put("corr", "CREATE TABLE [corr] ( [id] INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT, [cname] ...)");
tableSql.put("category", "CREATE TABLE [category] ( [id] INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT, [cname] ...)");
tableSql.put("price", "CREATE TABLE [price] ( [id] INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT, [cdate] ...)");
}
public void Connect( String path){
File DBfile = new File (path);
boolean DBexists = DBfile.exists();
Statement stmt = null;
ResultSet rs;
try {
Class.forName("org.sqlite.JDBC");
c = DriverManager.getConnection("jdbc:sqlite:" + path);
c.setAutoCommit(true);
stmt = c.createStatement();
if( DBexists ){
// check database structure
for (String tableName : tableSql.keySet()) {
rs = stmt.executeQuery( "SELECT sql FROM sqlite_master WHERE type = 'table' AND name = '" + tableName + "'");
if(rs.isBeforeFirst()){
rs.next();
// table and field names may be inside square brackets or inside quotes...
String table_schema_there = rs.getString(1).replaceAll("\\s+"," ").replaceAll("[\\[\\]'`]", "\"");
String table_schema_here = tableSql.get(tableName).replaceAll("\\s+"," ").replaceAll("[\\[\\]'`]", "\"");;
if( ! table_schema_there.equals(table_schema_here) ){
notifyListeners( new ModelResponse( false, "Structure error. Wrong structure of table " + tableName));
System.exit(0);
}
}
else{
notifyListeners( new ModelResponse( false, "Structure error. The table is missing: " + tableName ));
System.exit(0);
}
}
}
else{
// empty DB file created during connection so we need to create schema
for (String tableName : tableSql.keySet()) {
stmt.executeUpdate(tableSql.get(tableName));
}
}
}
catch ( Exception e ) {
notifyListeners( new ModelResponse( false, e.getMessage()));
System.exit(0);
}
}
}
You can write up your own verification of the database by looking in the metdata tables. This is not impossible, but it is a lot of code to maintain. You can also write up a lot of DDL statements to construct the database structures, again not impossible, but a lot of code to maintain.
If you follow such a path, I recommend a higher-level logic of
if (!checkDatabase()) {
try {
if (wantsToInstall()) {
installDatabase();
}
} catch (Exception e) {
exit();
}
}
or a specific installer (to reduce the chance of installing over existing items). Extensions that provide deleting data items can be done, but there are differing opinions as to what to do if encountering an unexpected pre-existing object during install / upgrade.
Now there are a few libraries that will do this for you; but by the time they add this feature, such a library probably takes care of much more than simple database structure maintenance. Look to JDO and a few others, I know that DataNucleus's environment will gladly create database structures tweaked to work well in just over 20 different databases (remember they all have subtle differences in how they do things).