I'm having some trouble using mySQL and Spring JDBCTemplate. I have an INSERT ... ON DUPLICATE KEY UPDATE which increments a counter, but uses the LAST_INSERT_ID() trick to return the new value in the same query through the last generated id. I'm using JDCTemplate.update() with the PreparedStatementCreator and GeneratedKeyHolder, but I'm getting multiple entries in the KeyHolder.getKeyList(). When I run it manually in the mySQL client, I'm getting 2 rows affected (when it hits the DUPLICATE KEY UPDATE), and then the SELECT LAST_INSERT_ID(); gives me the correct value.
UPDATE: It looks like the 3 entries in the keyList all have 1 entry each, and they are all incrementing values. So keyList.get(0).get(0) has the value that should be returned, and then keyList.get(0).get(1) is the previous value +1, and then keyList.get(0).get(2) is the previous value +1. So for example, if 4 is the value that should be returned, it gives me 4, 5, 6 in that order.
The relevant code is:
CREATE TABLE IF NOT EXISTS `ClickChargeCheck` (
uuid VARCHAR(50),
count INTEGER Default '0',
CreatedOn Timestamp DEFAULT '0000-00-00 00:00:00',
UpdatedOn Timestamp DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`uuid`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
String CLICK_CHARGE_CHECK_QUERY = "INSERT INTO ClickChargeCheck (uuid,count,CreatedOn) VALUES(?,LAST_INSERT_ID(1),NOW()) ON DUPLICATE KEY UPDATE count = LAST_INSERT_ID(count+1)";
...
...
GeneratedKeyHolder keyHolder = new GeneratedKeyHolder();
centralStatsJdbcTemplate.update(new PreparedStatementCreator() {
#Override
public PreparedStatement createPreparedStatement(Connection con) throws SQLException {
PreparedStatement ps = con.prepareStatement(CLICK_CHARGE_CHECK_QUERY, Statement.RETURN_GENERATED_KEYS);
ps.setString(1, uuid);
return ps;
}}, keyHolder);
int count = keyHolder.getKey().intValue();
For me, I ran into the same problem. For INSERT...ON DUPLICATE KEY UPDATE, GeneratedKeyHolder is throwing an exception because it expects there to only be one returned key, which is not the case with MySQL. I basically had to implement a new implementation of KeyHolder which can handle multiple returned key.
public class DuplicateKeyHolder implements KeyHolder {
private final List<Map<String, Object>> keyList;
/* Constructors */
public Number getKey() throws InvalidDataAccessApiUsageException, DataRetrievalFailureException {
if (this.keyList.isEmpty()) {
return null;
}
Iterator<Object> keyIter = this.keyList.get(0).values().iterator();
if (keyIter.hasNext()) {
Object key = keyIter.next();
if (!(key instanceof Number)) {
throw new DataRetrievalFailureException(
"The generated key is not of a supported numeric type. " +
"Unable to cast [" + (key != null ? key.getClass().getName() : null) +
"] to [" + Number.class.getName() + "]");
}
return (Number) key;
} else {
throw new DataRetrievalFailureException("Unable to retrieve the generated key. " +
"Check that the table has an identity column enabled.");
}
}
public Map<String, Object> getKeys() throws InvalidDataAccessApiUsageException {
if (this.keyList.isEmpty()) {
return null;
}
return this.keyList.get(0);
}
public List<Map<String, Object>> getKeyList() {
return this.keyList;
}
}
If you just use this class where you would otherwise use GeneratedKeyHolder, everything works. And I found that you don't even need to add ... ON DUPLICATE KEY UPDATE id = LAST_INSERT_ID(id) ... in your insert statement.
Turns out that mySQL's PreparedStatement runs that query, and says it returns 3 rows. So the getGeneratedKeys() call (delegated to StatementImpl.getGeneratedKeysInternal()) which, since it sees 3 rows returned, grabs the first LAST_INSERT_ID() value, and then increments from there up to the number of rows. So, I'll have to grab the GeneratedKeyHolder.getKeyList() and grab the first entry, and get it's first value and that'll be the value I need.
Related
I want to embed date information in the primary key, for a table that will be partitioned (monthly) in a PostgreSQL database. This should in theory speed up the process on finding out in which partition to look for the data. I followed this article to embed the date in a date into the serial.
Now, I am however facing the problem that I can't get the Id been used by Hibernate.
c.f. the sql that should give an idea of the attempted approach.
CREATE SEQUENCE test_serial START 1;
CREATE OR REPLACE FUNCTION gen_test_key() RETURNS BIGINT AS $$
DECLARE
new_id bigint;
BEGIN
new_id = (nextval('public.test_serial'::regclass)::bigint % 10000000000000::bigint
+ ( (EXTRACT(year from now())-2000)::bigint * 10000::bigint
+ EXTRACT(month from now())::bigint * 100::bigint
+ EXTRACT(day from now())::bigint
)::bigint * 10000000000000::bigint
)::bigint;
RETURN new_id;
END;
$$ LANGUAGE plpgsql;
CREATE TABLE test
( id bigint primary key default gen_test_key(),
something text,
tstamp timestamp default now()
) PARTITION BY RANGE (id);
CREATE TABLE test_2022_10 PARTITION OF test
FOR VALUES FROM (2210100000000000000::bigint ) TO (2211010000000000000::bigint);
I came across a similar question, where it was suggested to use a stored procedure. Unfortunately only functions are allowed as default in the table definition and therefore stored procedures, seam not to work for me.
I think what you need here is a subtype of SequenceStyleGenerator that overrides determineBulkInsertionIdentifierGenerationSelectFragment to run the code of this function. You should be able to configure this generator on your entity with #GenericGenerator. I understand the desire to use this concept when you don't want to change your existing queries, but are you sure that partitioning will help you in your use case?
Also, be careful and do not rely on the date information in the primary key, because with pooled optimizers, it might happen that a value is generated way before it actually is used as primary key for a row.
So this is a solution that worked out in the end as suggested #ChristianBeikov here the entity with the annotations pointing to the CustomIdGenerator.
public class Test {
#Id
#GenericGenerator(name = "CustomIdGenerator", strategy = "nl.test.components.CustomIdGenerator")
#GeneratedValue(generator = "CustomIdGenerator")
private Long id;
private String something;
private OffsetDateTime tstamp;
}
As explained by #Mr_Thorynque it is similarly possible to call a stored function as a procedure. Just replace "CALL gen_test_key()" with "SELECT gen_test_key()" and don't pass it to the wrong method for stored procedures connection.prepareCall(CALL_STORE_PROC);, but instead connection.prepareStatement(STORED_FUNCTION); So, this is the CustomIdGenerator.
public class CustomIdGenerator implements IdentifierGenerator {
private static final String STORED_FUNCTION = "select gen_test_key()";
#Override
public Serializable generate(SharedSessionContractImplementor session, Object object) throws HibernateException {
Long result = null;
try {
Connection connection = session.connection();
PreparedStatement pstmt = connection.prepareStatement(STORED_FUNCTION);
ResultSet resultSet = pstmt.executeQuery();
if (resultSet.next()) {
result = resultSet.getLong(1);
System.out.println("Generated Id: " + result);
}
} catch (SQLException sqlException) {
throw new HibernateException(sqlException);
}
return result;
}
}
I am planning to use Redis with Redisson as a caching layer between my Java app and a PostgreSQL DB.
I have a table called Nodes looking like this:
CREATE TABLE nodes
(
node_id bigint GENERATED BY DEFAULT AS IDENTITY(START WITH 1 INCREMENT BY 1),
node_name varchar(100) NOT NULL,
PRIMARY KEY (node_id)
)
I want to use Redisson RMap persistence to cache this structure.
My goal is to have an rmap looking like this:
Rmap<Integer, Node>
where the key is the PK, and the value is the node itself.
I want to use read-through and write-trhough strategies for caching this Rmap, by using the MapLoader and the MapWriter.
Then, I want to have a Java method which should create and persist a node.
public void createNode(String nodeName) {
Node node = new Node();
node.setName(nodeName);
// how can I put elements in the rmap since,
// the PK will be available after the INSERT statement will run?
rmap.put(?, node);
}
And here comes the problem. Since the PK is auto-generated from Postgres, how can I use the RMapWriter to insert a node, since, for putting elements in the RMap I need the key, which I don't have until the insert statement will run?
You can get generated Keys from postgres using prepared Statement.
ps = connection.prepareStatement(sql, Statement.RETURN_GENERATED_KEYS) ;
ps.execute();
ResultSet rs = ps.getGeneratedKeys();
I'm aware this is an older post but answering because I came across the same issue.
For me, the solution was to use the MapLoader rather than the MapWriter, using a CallableStatement (rather than a PreparedStatement) backed by a stored procedure in the SELECT_or_INSERTthenSELECT mould.
#Override
public Integer load(Node node) {
// second parameter to the procedure is an OUTPUT param
try(CallableStatement callableStatement = conn.prepareCall("{CALL INSERT_or_SELECT_THEN_INSERT (?, ?)}"))
{
callableStatement.setString(1, node.getName());
callableStatement.registerOutParameter(2, java.sql.Types.BIGINT);
callableStatement.executeUpdate();
System.out.println("Debug nodeName: " + node.getName() + ", id: " + callableStatement.getLong(2));
return callableStatement.getLong(2);
} catch (Exception e) {
throw new IllegalStateException(e);
}
}
Now you can just call map.get(node) and the load() override will be called if the node is not in the map.
I'm trying to add designation and its salary using a JDBC transaction.
The problem is this throws an exception about duplicate key.
This is the first time I put some invalid data in salary columns and after that everything is correct. It shows duplicate key exception for designation id
but the designation id is not stored already and not even for first attempt.
The first transaction that is invalid is rolled back, but storing on next time it shows duplicate key exception.
Below is my code:-
public boolean addDesignation(ObservableList nodeList) throws SQLException {
Connection demo = getConnection();
demo.setAutoCommit(false);
Savepoint savePoint = demo.setSavepoint("savePoint");
try {
PreparedStatement addDesig = demo.prepareStatement(
"INSERT INTO `designation`(`desig_id`,`dept_id`,`desig_name`,`desig_desc`) VALUES (?,?,?,?)");
PreparedStatement addSal = demo.prepareStatement("INSERT INTO `salary` "
+ "(`desig_id`, `basic`, `house_rent`, `conveyance`, `medical`, `dearness`,`others_allowances`,"
+ " `income_tax`, `pro_tax`, `emp_state_insu`, `absence_fine`, `others_deductions`, `month`)"
+ " VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?)");
addDesig.setString(1 , nodeList.get(0).toString());
addDesig.setString(2, nodeList.get(1).toString());
addDesig.setString(3, nodeList.get(2).toString());
addDesig.setString(4, nodeList.get(3).toString());
addDesig.executeUpdate();
addSal.setString(1, nodeList.get(0).toString());
addSal.setInt(2, Integer.parseInt(nodeList.get(4).toString()));
addSal.setInt(3, Integer.parseInt(nodeList.get(5).toString()));
addSal.setInt(4, Integer.parseInt(nodeList.get(6).toString()));
addSal.setInt(5, Integer.parseInt(nodeList.get(7).toString()));
addSal.setInt(6,Integer.parseInt(nodeList.get(8).toString()));
addSal.setInt(7,Integer.parseInt(nodeList.get(9).toString()));
addSal.setInt(8, Integer.parseInt(nodeList.get(10).toString()));
addSal.setInt(9, Integer.parseInt(nodeList.get(11).toString()));
addSal.setInt(10, Integer.parseInt(nodeList.get(12).toString()));
addSal.setInt(11, Integer.parseInt(nodeList.get(13).toString()));
addSal.setInt(12, Integer.parseInt(nodeList.get(14).toString()));
addSal.setString(13, nodeList.get(15).toString());
addSal.executeUpdate();
demo.commit();
return true;
} catch (SQLException ex) {
demo.rollback(savePoint);
Logger.getLogger(DatabaseHandler.class.getName()).log(Level.SEVERE, null, ex);
}
return false;
}
these are two tables and im trying to add that data in my first attempt failed but not store due roll back
There are 2 INSERT statements in your code.
The 1st for designation table:
"INSERT INTO `designation`(`desig_id`,`dept_id`,`desig_name`,`desig_desc`) VALUES (?,?,?,?)"
here it looks like desig_id is the primary key (maybe autoincrement in which case you must not supply a value at all).
Are you sure the value that you supply for this column does not already exist in the table?
The 2nd for the salary table:
"INSERT INTO `salary` " + "(`desig_id`, `basic`, `house_rent`, `conveyance`, `medical`, `dearness`,`others_allowances`," + " `income_tax`, `pro_tax`, `emp_state_insu`, `absence_fine`, `others_deductions`, `month`)" + " VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?)"
in this case it is not clear since you did not post the CREATE statement of the table, which is the primary key.
So you have to check, if the value (or values if it's a multi column key), violate the uniqueness of the key.
Is it possible to get the ##identity from the SQL insert on a Spring jdbc template call? If so, how?
The JDBCTemplate.update method is overloaded to take an object called a GeneratedKeyHolder which you can use to retrieve the autogenerated key. For example (code taken from here):
final String INSERT_SQL = "insert into my_test (name) values(?)";
final String name = "Rob";
KeyHolder keyHolder = new GeneratedKeyHolder();
jdbcTemplate.update(
new PreparedStatementCreator() {
public PreparedStatement createPreparedStatement(Connection connection) throws SQLException {
PreparedStatement ps =
connection.prepareStatement(INSERT_SQL, new String[] {"id"});
ps.setString(1, name);
return ps;
}
},
keyHolder);
// keyHolder.getKey() now contains the generated key
How about SimpleJdbcInsert.executeAndReturnKey? It takes two forms, depending on the input:
(1) The input is a Map
public java.lang.Number executeAndReturnKey(java.util.Map<java.lang.String,?> args)
Description copied from interface: SimpleJdbcInsertOperations
Execute the insert using the values passed in and return the generated key.
This requires that the name of the columns with auto generated keys have been specified. This method will always return a KeyHolder but the caller must verify that it actually contains the generated keys.
Specified by:
executeAndReturnKey in interface SimpleJdbcInsertOperations
Parameters:
args - Map containing column names and corresponding value
Returns:
the generated key value
(2) The input is a SqlParameterSource
public java.lang.Number executeAndReturnKey(SqlParameterSourceparameterSource)
Description copied from interface: SimpleJdbcInsertOperations
Execute the insert using the values passed in and return the generated key.
This requires that the name of the columns with auto generated keys have been specified. This method will always return a KeyHolder but the caller must verify that it actually contains the generated keys.
Specified by:
executeAndReturnKey in interface SimpleJdbcInsertOperations
Parameters:
parameterSource - SqlParameterSource containing values to use for insert
Returns:
the generated key value.
Adding detailed notes/sample code to todd.pierzina answer
jdbcInsert = new SimpleJdbcInsert(jdbcTemplate);
jdbcInsert.withTableName("TABLE_NAME").usingGeneratedKeyColumns(
"Primary_key");
Map<String, Object> parameters = new HashMap<>();
parameters.put("Column_NAME1", bean.getval1());
parameters.put("Column_NAME2", bean.getval2());
// execute insert
Number key = jdbcInsert.executeAndReturnKey(new MapSqlParameterSource(
parameters));
// convert Number to Int using ((Number) key).intValue()
return ((Number) key).intValue();
I don't know if there is a "one-liner" but this seems to do the trick (for MSSQL at least):
// -- call this after the insert query...
this._jdbcTemplate.queryForInt( "select ##identity" );
Decent article here.
I'm using iBatis/Java and Postgres 8.3.
When I do an insert in ibatis i need the id returned.
I use the following table for describing my question:
CREATE TABLE sometable ( id serial NOT NULL, somefield VARCHAR(10) );
The Sequence sometable_id_seq gets autogenerated by running the create statement.
At the moment i use the following sql map:
<insert id="insertValue" parameterClass="string" >
INSERT INTO sometable ( somefield ) VALUES ( #value# );
<selectKey keyProperty="id" resultClass="int">
SELECT last_value AS id FROM sometable_id_seq
</selectKey>
</insert>
It seems this is the ibatis way of retrieving the newly inserted id. Ibatis first runs a INSERT statement and afterwards it asks the sequence for the last id.
I have doubts that this will work with many concurrent inserts.
Could this cause problems? Like returning the id of the wrong insert?
( See also my related question about how to get ibatis to use the INSERT .. RETURING .. statements )
This is definitely wrong. Use:
select currval('sometable_id_seq')
or better yet:
INSERT INTO sometable ( somefield ) VALUES ( #value# ) returning id
which will return you inserted id.
Here is simple example:
<statement id="addObject"
parameterClass="test.Object"
resultClass="int">
INSERT INTO objects(expression, meta, title,
usersid)
VALUES (#expression#, #meta#, #title#, #usersId#)
RETURNING id
</statement>
And in Java code:
Integer id = (Integer) executor.queryForObject("addObject", object);
object.setId(id);
I have another thought. ibatis invokes the insert method delegate the Class: com.ibatis.sqlmap.engine.impl.SqlMapExecutorDelegate,with the code:
try {
trans = autoStartTransaction(sessionScope, autoStart, trans);
SelectKeyStatement selectKeyStatement = null;
if (ms instanceof InsertStatement) {
selectKeyStatement = ((InsertStatement) ms).getSelectKeyStatement();
}
// Here we get the old value for the key property. We'll want it later if for some reason the
// insert fails.
Object oldKeyValue = null;
String keyProperty = null;
boolean resetKeyValueOnFailure = false;
if (selectKeyStatement != null && !selectKeyStatement.isRunAfterSQL()) {
keyProperty = selectKeyStatement.getKeyProperty();
oldKeyValue = PROBE.getObject(param, keyProperty);
generatedKey = executeSelectKey(sessionScope, trans, ms, param);
resetKeyValueOnFailure = true;
}
StatementScope statementScope = beginStatementScope(sessionScope, ms);
try {
ms.executeUpdate(statementScope, trans, param);
}catch (SQLException e){
// uh-oh, the insert failed, so if we set the reset flag earlier, we'll put the old value
// back...
if(resetKeyValueOnFailure) PROBE.setObject(param, keyProperty, oldKeyValue);
// ...and still throw the exception.
throw e;
} finally {
endStatementScope(statementScope);
}
if (selectKeyStatement != null && selectKeyStatement.isRunAfterSQL()) {
generatedKey = executeSelectKey(sessionScope, trans, ms, param);
}
autoCommitTransaction(sessionScope, autoStart);
} finally {
autoEndTransaction(sessionScope, autoStart);
}
You can see that the insert and select operator are in a Transaction. So I think there is no concureency problem with the insert method.