I want to embed date information in the primary key, for a table that will be partitioned (monthly) in a PostgreSQL database. This should in theory speed up the process on finding out in which partition to look for the data. I followed this article to embed the date in a date into the serial.
Now, I am however facing the problem that I can't get the Id been used by Hibernate.
c.f. the sql that should give an idea of the attempted approach.
CREATE SEQUENCE test_serial START 1;
CREATE OR REPLACE FUNCTION gen_test_key() RETURNS BIGINT AS $$
DECLARE
new_id bigint;
BEGIN
new_id = (nextval('public.test_serial'::regclass)::bigint % 10000000000000::bigint
+ ( (EXTRACT(year from now())-2000)::bigint * 10000::bigint
+ EXTRACT(month from now())::bigint * 100::bigint
+ EXTRACT(day from now())::bigint
)::bigint * 10000000000000::bigint
)::bigint;
RETURN new_id;
END;
$$ LANGUAGE plpgsql;
CREATE TABLE test
( id bigint primary key default gen_test_key(),
something text,
tstamp timestamp default now()
) PARTITION BY RANGE (id);
CREATE TABLE test_2022_10 PARTITION OF test
FOR VALUES FROM (2210100000000000000::bigint ) TO (2211010000000000000::bigint);
I came across a similar question, where it was suggested to use a stored procedure. Unfortunately only functions are allowed as default in the table definition and therefore stored procedures, seam not to work for me.
I think what you need here is a subtype of SequenceStyleGenerator that overrides determineBulkInsertionIdentifierGenerationSelectFragment to run the code of this function. You should be able to configure this generator on your entity with #GenericGenerator. I understand the desire to use this concept when you don't want to change your existing queries, but are you sure that partitioning will help you in your use case?
Also, be careful and do not rely on the date information in the primary key, because with pooled optimizers, it might happen that a value is generated way before it actually is used as primary key for a row.
So this is a solution that worked out in the end as suggested #ChristianBeikov here the entity with the annotations pointing to the CustomIdGenerator.
public class Test {
#Id
#GenericGenerator(name = "CustomIdGenerator", strategy = "nl.test.components.CustomIdGenerator")
#GeneratedValue(generator = "CustomIdGenerator")
private Long id;
private String something;
private OffsetDateTime tstamp;
}
As explained by #Mr_Thorynque it is similarly possible to call a stored function as a procedure. Just replace "CALL gen_test_key()" with "SELECT gen_test_key()" and don't pass it to the wrong method for stored procedures connection.prepareCall(CALL_STORE_PROC);, but instead connection.prepareStatement(STORED_FUNCTION); So, this is the CustomIdGenerator.
public class CustomIdGenerator implements IdentifierGenerator {
private static final String STORED_FUNCTION = "select gen_test_key()";
#Override
public Serializable generate(SharedSessionContractImplementor session, Object object) throws HibernateException {
Long result = null;
try {
Connection connection = session.connection();
PreparedStatement pstmt = connection.prepareStatement(STORED_FUNCTION);
ResultSet resultSet = pstmt.executeQuery();
if (resultSet.next()) {
result = resultSet.getLong(1);
System.out.println("Generated Id: " + result);
}
} catch (SQLException sqlException) {
throw new HibernateException(sqlException);
}
return result;
}
}
Related
I am planning to use Redis with Redisson as a caching layer between my Java app and a PostgreSQL DB.
I have a table called Nodes looking like this:
CREATE TABLE nodes
(
node_id bigint GENERATED BY DEFAULT AS IDENTITY(START WITH 1 INCREMENT BY 1),
node_name varchar(100) NOT NULL,
PRIMARY KEY (node_id)
)
I want to use Redisson RMap persistence to cache this structure.
My goal is to have an rmap looking like this:
Rmap<Integer, Node>
where the key is the PK, and the value is the node itself.
I want to use read-through and write-trhough strategies for caching this Rmap, by using the MapLoader and the MapWriter.
Then, I want to have a Java method which should create and persist a node.
public void createNode(String nodeName) {
Node node = new Node();
node.setName(nodeName);
// how can I put elements in the rmap since,
// the PK will be available after the INSERT statement will run?
rmap.put(?, node);
}
And here comes the problem. Since the PK is auto-generated from Postgres, how can I use the RMapWriter to insert a node, since, for putting elements in the RMap I need the key, which I don't have until the insert statement will run?
You can get generated Keys from postgres using prepared Statement.
ps = connection.prepareStatement(sql, Statement.RETURN_GENERATED_KEYS) ;
ps.execute();
ResultSet rs = ps.getGeneratedKeys();
I'm aware this is an older post but answering because I came across the same issue.
For me, the solution was to use the MapLoader rather than the MapWriter, using a CallableStatement (rather than a PreparedStatement) backed by a stored procedure in the SELECT_or_INSERTthenSELECT mould.
#Override
public Integer load(Node node) {
// second parameter to the procedure is an OUTPUT param
try(CallableStatement callableStatement = conn.prepareCall("{CALL INSERT_or_SELECT_THEN_INSERT (?, ?)}"))
{
callableStatement.setString(1, node.getName());
callableStatement.registerOutParameter(2, java.sql.Types.BIGINT);
callableStatement.executeUpdate();
System.out.println("Debug nodeName: " + node.getName() + ", id: " + callableStatement.getLong(2));
return callableStatement.getLong(2);
} catch (Exception e) {
throw new IllegalStateException(e);
}
}
Now you can just call map.get(node) and the load() override will be called if the node is not in the map.
I need to use the database Firebird and for this I use the Jaybird 2.2.9.
When I used the MySQL driver, to converter of ResultSet to Object this way:
empresa.setBairro(rs.getString("empresa.bairro")); // (Table.Column)
empresa.setCep(rs.getString("empresa.cep")); // (Table.Column)
empresa.setCidade(rs.getString("empresa.cidade")); // (Table.Column)
But with Jaybird the resultSet don't return rs.getString("Table.Column")
I need this way when I have inner join in SQL.
Anyone help me?
This is my full code
public ContaLivros converterContaLivros(ResultSet rs, Integer linha) throws Exception {
if (rs.first()) {
rs.absolute(linha);
ContaLivros obj = new ContaLivros();
obj.setId(rs.getLong("cad_conta.auto_id"));
obj.setNome(rs.getString("cad_conta.nome"));
if (contain("cad_banco.auto_id", rs)) {
obj.setBancoLivros(converterBancoLivros(rs, linha));
} else {
obj.setBancoLivros(new BancoLivros(rs.getLong("cad_conta.banco"), null, null, null));
}
obj.setAgencia(rs.getInt("cad_conta.agencia"));
obj.setAgenciaDigito(rs.getInt("cad_conta.agencia_digito"));
obj.setConta(rs.getInt("cad_conta.conta"));
obj.setContaDigito(rs.getInt("cad_conta.conta_digito"));
obj.setLimite(rs.getDouble("cad_conta.limite"));
obj.setAtivo(rs.getString("cad_conta.ativo"));
return obj;
} else {
return null;
}
}
You can't. Jaybird retrieves the columns by its label as specified in JDBC 4.2, section 15.2.3. In Firebird the column label is either the original column name, or the AS alias, the table name isn't part of this. The extension of MySQL that you can prefix the table name for disambiguation is non-standard.
Your options are to specify aliases in the query and retrieve by this aliasname, or to process the result set metadata to find the right indexes for each column and retrieve by index instead.
However note that in certain queries (for example UNION), the ResultSetMetaData.getTableName cannot return the table name, as Firebird doesn't "know" it (as you could be applying a UNION to selects from different tables).
The name in jdbc will not have the table in it.
You can either
work with positional parameters ( getString (1); and so on )
Or
define column name alias in your select (select a.name namefroma from tableone a )
Or
simply do rs.getString ("column"); without the table prefix if name is unambigous
I have a table with unique index to eliminate duplicates (simplified example)
CREATE TABLE `domain` (
`id` INT(10) UNSIGNED NOT NULL AUTO_INCREMENT,
`subdomain` VARCHAR(255) NOT NULL,
`domain` VARCHAR(63) NOT NULL,
`zone` VARCHAR(63) NOT NULL,
PRIMARY KEY (`id`),
UNIQUE INDEX `UNIQUE` (`subdomain` ASC, `domain` ASC, `zone` ASC),
ENGINE = InnoDB;
I insert a lot of rows and i need to get primary keys returned (for other one-to-many inserts).
My problem is, that I insert a lot of duplicates and I need those keys returned too.
This is my solution which works, but isn't there more simple solution? With this I cannot use batch inserts and I want this to be most efficient.
PreparedStatement selectDomain = connection.prepareStatement("SELECT id FROM domain WHERE subdomain = ? AND domain = ? AND zone = ?");
PreparedStatement insertDomain = connection.prepareStatement("INSERT INTO domain(subdomain, domain, zone) VALUES (?,?,?)", Statement.RETURN_GENERATED_KEYS);
public int insertDomain(String subdomain, String domain, String zone) throws SQLException {
int domainId = 0;
selectDomain.setString(1, subdomain);
selectDomain.setString(2, domain);
selectDomain.setString(3, zone);
ResultSet resultSet = selectDomain.executeQuery();
if (resultSet.next()) {
domainId = resultSet.getInt(1);
} else {
insertDomain.setString(1, subdomain);
insertDomain.setString(2, subdomain);
insertDomain.setString(3, subdomain);
insertDomain.executeUpdate();
resultSet = insertDomain.getGeneratedKeys();
if (resultSet.next()) {
domainId = resultSet.getInt(1);
}
}
selectDomain.clearParameters();
insertDomain.clearParameters();
}
As I understand its not so easy approach for using batch execution. Your apporach is the best way to get the auto generated keys. There are few limitations of JDBC driver and it varies version to version, where getGeneratedKeys() works for single entry.
Please look into below links, it may help you :-
How to get generated keys from JDBC batch insert in Oracle?
http://docs.oracle.com/database/121/JJDBC/jdbcvers.htm#JJDBC28099
You could modify your INSERT to be something like this:
INSERT INTO domain (subdomain, domain, zone)
SELECT $subdomain, $domain, $zone
FROM domain
WHERE NOT EXISTS(
SELECT subdomain, domain, zone
FROM domain d
WHERE d.subdomain= $subdomain and d.domain=$domain and d.zone=$zone
)
LIMIT 1
Where $subdomain, $domain, $zone are the tag (properly quoted or as a placeholder of course) that you want to add if it isn't already there. This approach won't even trigger an INSERT (and the subsequent autoincrement wastage) if the tag is already there. You could probably come up with nicer SQL than that but the above should do the trick.
If your table is properly indexed then the extra SELECT for the existence check will be fast and the database is going to have to perform that check anyway.
I have a production and a QA instance of my application into which I'm integrating Liquibase. This means DDL and data already exists (or not if on development box). I have to create a changeLog which records everything as RAN on the non-empty DBs but execute actually on empty DBs.
I'm on a good way but I'm a bit stuck with creating the foreign keys. (the database is Oracle).
(In general I'm creating preconditions which expects various objects to NOT exists and on fail MARK_RAN the change).
I find difficulties writing a correct precondition when I don't know the exact name of foreign keys, which may or may not exist.
There is <foreignKeyConstraintExists> tag in liquibase (precondition) but it takes only schemaName and foreignKeyName attributes (and they are required). I don't know the foreign key names for sure in these instances as they are out of my control.
You can write custom SQL in preconditions like:
<changeSet id="1" author="bob">
<preConditions onFail="WARN">
<sqlCheck expectedResult="0">select count(*) from oldtable</sqlCheck>
</preConditions>
<dropTable tableName="oldtable"/>
</changeSet>
So I only have to create a custom SQL query which can check if a column on table A has foreign key referencing table B and use the result as a precondition.
This is where my problem is because you can do it in Oracle but it's quite bloat:
SELECT a.table_name, a.column_name, a.constraint_name, c.owner,
c.r_owner, c_pk.table_name r_table_name
FROM all_cons_columns a
JOIN all_constraints c ON a.owner = c.owner
AND a.constraint_name = c.constraint_name
JOIN all_constraints c_pk ON c.r_owner = c_pk.owner
AND c.r_constraint_name = c_pk.constraint_name
WHERE c.constraint_type = 'R' AND a.table_name = 'MY_TABLE'
AND a.column_name = 'MY_COLUMN'
AND c_pk.table_name = 'MY_OTHER_TABLE';
This prints a row if a foreign key exists on MY_COLUMN of MY_TABLE which references to MY_OTHER_TABLE. After rewriting it to COUNT you can check if there's foreign key without knowing it's name.
My question:
I have dozens of foreign keys, do I really have to write this big SQL such dozens of times? Any suggestions, like outsourcing this to some function? Thanks!
Would it worth asking Liquibase developers to make <foreignKeyConstraintExists> 's name attribute optional and introduce the referenced table attribute alogn with local column name?
There is one more possibility: implementing the interface http://www.liquibase.org/javadoc/liquibase/precondition/CustomPrecondition.html and use it as a custom precondition. More info: http://www.liquibase.org/documentation/preconditions.html
Here is the implementation (verified):
import liquibase.database.Database;
import liquibase.exception.CustomPreconditionErrorException;
import liquibase.exception.CustomPreconditionFailedException;
import liquibase.precondition.CustomPrecondition;
import liquibase.snapshot.SnapshotGeneratorFactory;
import liquibase.structure.core.ForeignKey;
import liquibase.structure.core.Schema;
import liquibase.structure.core.Table;
import liquibase.util.StringUtils;
/**
* {#link CustomPrecondition} implementation that checks if a column on a table
* has a foreign key constraint for some other table.
*/
public final class CheckForeignKey implements CustomPrecondition {
/**
* Schema.
*/
private String schemaName;
/**
* Table name (that has the column).
*/
private String tableName;
/**
* Column (that might have the foreign key).
*/
private String columnName;
/**
* Referenced table of the foreign key.
*/
private String foreignTableName;
#Override
public void check(final Database db)
throws CustomPreconditionFailedException,
CustomPreconditionErrorException {
try {
// The fkey we are looking for
final ForeignKey fKey = new ForeignKey();
// Schema, base table
fKey.setForeignKeyTable(new Table());
if (StringUtils.trimToNull(getTableName()) != null) {
fKey.getForeignKeyTable().setName(getTableName());
}
final Schema schema = new Schema();
schema.setName(getSchemaName());
fKey.getForeignKeyTable().setSchema(schema);
// Base column
fKey.addForeignKeyColumn(getColumnName());
// Referenced table
fKey.setPrimaryKeyTable(new Table());
if (StringUtils.trimToNull(getForeignTableName()) != null) {
fKey.getPrimaryKeyTable().setName(getForeignTableName());
}
if (!SnapshotGeneratorFactory.getInstance().has(fKey, db)) {
throw new CustomPreconditionFailedException(
String.format(
"Error fkey not found schema %s table %s column %s ftable %s",
getSchemaName(), getTableName(),
getColumnName(), getForeignTableName()));
}
} catch (final CustomPreconditionFailedException e) {
throw e;
} catch (final Exception e) {
throw new CustomPreconditionErrorException("Error", e);
}
}
public String getSchemaName() {
return schemaName;
}
public void setSchemaName(final String schemaName) {
this.schemaName = schemaName;
}
public String getTableName() {
return tableName;
}
public void setTableName(final String tableName) {
this.tableName = tableName;
}
public String getColumnName() {
return columnName;
}
public void setColumnName(final String columnName) {
this.columnName = columnName;
}
public String getForeignTableName() {
return foreignTableName;
}
public void setForeignTableName(final String foreignTableName) {
this.foreignTableName = foreignTableName;
}
}
I think that you have to do it like you suggested if you dont know foreign key constraint names.
But if you can modify database then you could prepare sql script which prepare another sql script which renames all FK to well known names. Something like that:
BEGIN
FOR cur IN (
SELECT
c_list.CONSTRAINT_NAME as FK_NAME,
'FK_' || c_dest.TABLE_NAME || '_' || substr(c_dest.COLUMN_NAME, 1, 20) as NEW_FK_NAME,
c_src.TABLE_NAME as SRC_TABLE,
c_src.COLUMN_NAME as SRC_COLUMN,
c_dest.TABLE_NAME as DEST_TABLE,
c_dest.COLUMN_NAME as DEST_COLUMN
FROM ALL_CONSTRAINTS c_list, ALL_CONS_COLUMNS c_src, ALL_CONS_COLUMNS c_dest
WHERE c_list.CONSTRAINT_NAME = c_src.CONSTRAINT_NAME
AND c_list.R_CONSTRAINT_NAME = c_dest.CONSTRAINT_NAME
AND c_list.CONSTRAINT_TYPE = 'R'
AND c_src.TABLE_NAME IN ('<your-tables-here>')
GROUP BY c_list.CONSTRAINT_NAME, c_src.TABLE_NAME, c_src.COLUMN_NAME, c_dest.TABLE_NAME, c_dest.COLUMN_NAME;
) LOOP
-- Generate here SQL commands (by string concatenation) something like:
-- alter table SRC_TABLE rename constraint FK_NAME to NEW_FK_NAME;
-- then paste this sql commands to some other script and run it
END LOOP;
END;
This is one time migration.
After this migration you know whats your FK constraint names are and you can use <foreignKeyConstraintExists> precondition in your changesets.
I just created a custom hibernate ID generator, and since I'm not an hibernate expert I would like to get some feedback on my code. The generated ID is select max(id) from table, +1.
public class MaxIdGenerator implements IdentifierGenerator, Configurable {
private Type identifierType;
private String tableName;
private String columnName;
#Override
public void configure(Type type, Properties params, Dialect dialect) {
identifierType = type;
tableName = (String) params.getProperty("target_table");
columnName = (String) params.getProperty("target_column");
}
#Override
public synchronized Serializable generate(SessionImplementor session,
Object object) {
return generateHolder(session).makeValue();
}
protected IntegralDataTypeHolder generateHolder(SessionImplementor session) {
Connection connection = session.connection();
try {
IntegralDataTypeHolder value = IdentifierGeneratorHelper
.getIntegralDataTypeHolder(identifierType
.getReturnedClass());
String sql = "select max(" + columnName + ") from " + tableName;
PreparedStatement qps = connection.prepareStatement(sql);
try {
ResultSet rs = qps.executeQuery();
if (rs.next())
value.initialize(rs, 1);
else
value.initialize(1);
rs.close();
} finally {
qps.close();
}
return value.copy().increment();
} catch (SQLException e) {
throw new IdentifierGenerationException(
"Can't select max id value", e);
}
}
}
I'd like to know:
How can I make this multi-transaction-safe? (ie if two concurrent transactions insert data, how can I safely assume that I will not end-up having twice the same ID?) -- I guess the only solution here would be to prevent two concurrent hibernate transaction to run at the same time if they use the same generator, is this possible?
If the code could be improved: I feel wrong having to use hard-coded "select", "target_column", etc...
To guarantee point 1), I can fallback if necessary on synchronizing inserts on my java client code.
Please do not comment on the reasons why I'm using this kind of generator: legacy code still insert data onto the same database and uses this mechanism... and can't be modified. And yes, I know, it sucks.
I think the easiest way to achive a transaction-safe behaviour is to put the code you use to retrive the maximum id and do the insert statement, in a transactional block.
Something like:
Transaction transaction = session.beginTransaction();
//some code...
transaction.commit();
session.close()
I also recommend to use HQL (Hibernate Query Language) to create the query, instead of native sql where possible. Moreover, from your description, I have understood that you expect from the query a unique result, the maximum id. So, you could use uniqueResult() method over your query instead of executeQuery.
you can use AtomicInteger for generating ID. That can be used by many threads concurrently.
If you are free to use any other provider of ID then i will suggest to use UUID class for generating random ID.
UUID.randomUUID();
You can refer to link which contain some other ways to generate ID.