Get generated key value while inserting in partitioned table - java

Hi I have a table in Postgres, say email_messages. It is partitioned so whatever the inserts i do it using my java application will not effect the master table, as the data is actually getting inserted in the child tables. Here is the problem, I want to get an auto generated column value (say email_message_id which is of big serial type). Postgres is returning it as null since there is no insert being done on master table. For oracle I used GeneratedKeyHolder to get that value. But I'm unable to do the same for partitioned table in postgres. Please help me out.
Here is the code snippet we used for oracle
public void createAndFetchPKImpl(final Entity pEntity,
final String pStatementId, final String pPKColumnName) {
final SqlParameterSource parameterSource =
new BeanPropertySqlParameterSource(pEntity);
final String[] columnNames = new String[]{"email_message_id"};
final KeyHolder keyHolder = new GeneratedKeyHolder();
final int numberOfRowsEffected = mNamedParameterJdbcTemplate.update(
getStatement(pStatementId), parameterSource, keyHolder, columnNames);
pEntity.setId(ConversionUtil.getLongValue(keyHolder.getKey()));
}

When you use trigger-based partitioning in PostgreSQL it is normal for the JDBC driver to report that zero rows were affected/inserted. This is because the original SQL UPDATE, INSERT or DELETE didn't actually take effect, no rows were changed on the main table. Instead operations were performed on one or more sub-tables.
Basically, partitioning in PostgreSQL is a bit of a hack and this is one of the more visible limitations of it.
The workarounds are:
INSERT/UPDATE/DELETE directly against the sub-table(s), rather than the top-level table;
Ignore the result rowcount; or
Use RULEs and INSERT ... RETURNING instead (but they have their own problems) (Won't work for partitions)

Related

Get a specific column of the updated rows

I need to know how to execute an update in JDBC with Oracle database backend and retrieve values for a specific column of the records that have been updated. The column that I am interested in is part of a composite primary key, e.g. COL_NAME in the example below.
I have tried the following:
String query = "UPDATE T1 SET COL_ABC = 'A'"; // Simplified
statement = conn.prepareStatement(query.toString(), new String[] { "COL_NAME" });
ResultSet rs = statement.getResultSet();
while (rs.next()) {
rs.getLong("COL_NAME");
}
But statement comes back as null.
I am not sure how to utilize RETURNING INTO in this case unless converting this whole thing into an anonymous PL/SQL block, if it is indeed a possible solution.
Please note that I need a list of the values from this column from all the records that have been updated.

Flyway Migration: NamedParameterJdbcTemplate

Is there anyway to create a flyway migration class utilizing the NamedParameterJdbcTemplate rather than the standard JdbcTemplate that comes across from the implementation of SpringJdbcMigration?
I have an upgrade I need to run where I need to convert a column type from text to integer (Replacing a string value with an internal id associated with that value.)
The way I'm doing this is temporarily storing the string values for a reverse lookup, deleting the column and re-adding it as the proper type, and then running an UPDATE call to add in the appropriate ID to the records. I have code similar to the following I want to execute as part of the migration:
String sql = "UPDATE my_table SET my_field = :my_field WHERE my_id IN (:my_ids)";
MapSqlParameterSource source = new MapSqlParameterSource();
source.addValue("my_field", someIntValue); // the internal id of the string I want to use.
source.addValue("my_ids", someListOfPKIds); // List of PK ids.
namedTemplate.update(sql,source); //namedTemplate is a NamedParameterJdbcTemplate
However, it seems as if I can't take advantage of the NamedParameterJdbcTemplate. Am I incorrect in this?
According to Flyway sources they create a new JdbcTemplate in SpringJdbcMigrationExecutor
However you can try creating a new NamedParameterJdbcTemplate in your migration given the classic JdbcTemplate. Check this constructor. E.g. new NamedParameterJdbcTemplate(jdbcTemplate)

Issue retrieving generated keys with SimpleJdbcInsert and Sybase

I'm having an odd problem with SimpleJdbcInsert.executeAndReturnKey with Sybase (jTDS driver) and certain data.
Take the following example:
SimpleJdbcInsert insert = new SimpleJdbcInsert(jdbcTemplate)
.withTableName("TABLE_NAME")
.usingGeneratedKeyColumns("ID");
List<String> columns = new ArrayList<String>();
columns.add("SOME_NUMERIC_DATA");
columns.add("SOME_STRING_DATA");
Map<String, Object> params = new HashMap<String, Object>();
params.put("SOME_NUMERIC_DATA", 10.02);
params.put("SOME_STRING_DATA", "AAAA");
Number insertId = insert.executeAndReturnKey(params);
The above will fail with
DataIntegrityViolationException: Unable to retrieve the generated key for the insert
The insert itself is fine as if I do an insert.execute(params) the insert will work correctly (but I need the generated column value).
If I insert null instead of 10.02 for the SOME_NUMERIC_DATA column then it will work correctly and return the generated column value. Also if all of the fields are VARCHAR/String then it will work correctly.
Can anyone see anything here that might be causing this with a combination of string and numeric fields.
I should also add that when I use the exact same code with an H2 database it works all of the time - this seems to be related to Sybase/jTDS
I had the same problem with SQL Server and fixed it by calling this configuration method right before the call to executeAndReturnKey():
mySimpleJdbcInsert.setAccessTableColumnMetaData(false);
I suspect the error has to do with database metadata : as explained in the spring reference http://docs.spring.io/spring-framework/docs/current/spring-framework-reference/html/jdbc.html, SimpleJdbcInsert uses database metadata to construct the actual insert statement.
One could also use the SQL OUTPUT clause such as
INSERT INTO myTable (Name, Age)
OUTPUT Inserted.Id
VALUES (?,?)
And use some more generic JdbcTemplate.execute() to handle the insert.

getting all the data from mysql database table is not working in java

i am trying to get all the data from a certain table.but it's only giving me only the first row twice a result (as i have two rows in the database)
here is my code
String data[]=new String[10];
String[] result;
Product p= new Product();
int serial=0;
try{
String sql="select * from product";
rslt=st.executeQuery(sql); //where private static Statement st, private static ResultSet rslt;
while(rslt.next()){
data[1]=rslt.getString("p_code");
data[2]=rslt.getString("p_name");
/* data[3]=rslt.getString("description");
data[4]=rslt.getString("measurement");
data[5]=Integer.toString(p.RemainProduct(data[1]));
data[6]=p.getSellPrice(data[1]);
serial+=1;
data[0]=Integer.toString(serial);
DTB.addRow(data); */
System.out.println("code :"+data[1]+" "+"Name :"+data[2]);
}
}catch(Exception ex){
System.out.println("ERROR :"+ex);
}
my table has two data, here is my database table data
and here is the result after i run the program.
i don't know where is the actual problem. the same code works fine on the other method but why i can't get it here. i am very new to java, please help me fixing this problem
Your code looks correct to me. I think this is a case of "what you are looking at is not what is broken". Perhaps you are not querying the table you think you are querying, or perhaps your table view is cached and out of date. Refresh your table view, and try printing out the PK (id column in the table) in your program output as a double check that the table your are querying is the correct table. Also, double check your JDBC URL and verify you are querying the correct database.
Also, your comments indicate that the same code is giving correct results in a different method. Maybe there is more going on in your actual code that we don't have insight into? Perhaps your code (not shown in your example) is:
Updating the table so that row 1 and 2 have the same data.
Querying the database (as shown in your example), and returning same data in row 1 and 2.
Update the table so that row 1 and 2 now have different data which shows up when you manually query the database.
Finally, make sure some of your other team members or an automated test process isn't updating the table without your knowledge.

Return Timestamp With Prepared Statement

I have an auto generated timestamp that is created each time a record is inserted or updated in a mysql table. Is there a way to return this timestamp in a way similar to how I would use a keyholder to return a newly created id?
KeyHolder keyHolder = new GeneratedKeyHolder();
JdbcTemplate jdbcTemplate = new JdbcTemplate(dataSource);
//Insert Contact
jdbcTemplate.update(new PreparedStatementCreator() {
#Override
public PreparedStatement createPreparedStatement(Connection connection) throws SQLException {
PreparedStatement preparedStatement = connection.prepareStatement(SQL_ADD, Statement.RETURN_GENERATED_KEYS);
preparedStatement.setString(1, contact.getFirstName());
preparedStatement.setString(2, contact.getLastName());
preparedStatement.setInt(3, contact.getOrganizationId());
preparedStatement.setString(4, contact.getType());
preparedStatement.setInt(5, contact.getUserId());
return preparedStatement;
}
}, keyHolder);
//Use keyholder to obtain newly created id
contact.setId(keyHolder.getKey().intValue());
Is there some way to also return the new timestamp without having to requery the table? I have been looking for ways to return it along with the id as a key in the keyholder, but it doesn't seem to be returned as a key?
Not very satisfying, but I think "no" is the answer to your question. I don't know any of the Spring stuff, but I think this is due to the basic JDBC that it's wrapping. See http://docs.oracle.com/javase/6/docs/api/java/sql/Statement.html#getGeneratedKeys%28%29
You only option would be to create a stored procedure on MySQL that has an out parameter and call that. See http://dev.mysql.com/doc/refman/5.0/en/call.html.
There are few options for solving this issue on the MySQL database server side. You could start with creating a TRIGGER on the table. As TRIGGER has a restriction and cannot return data, you can set the TIMESTAMP value to a variable:
DEMILITER //
CREATE TRIGGER ai_tbl_name AFTER INSERT ON tbl_name
FOR EACH ROW
BEGIN
SET #TimeStamp = NEW.timestamp_column;
END;//
DELIMITER ;
To retrieve this timestamp value, run the following command:
SELECT #TimeStamp;
Since the variables are stored in the memory, there will be no need to open any tables again.
You go even further. You could create a STORED PROCEDURE in MySQL to automate all the above (sample code, as I do not know your table's details):
DELIMITER //
DROP PROCEDURE IF EXISTS sp_procedure_name //
CREATE PROCEDURE sp_procedure_name (IN col1_val VARCHAR(25),
IN col2_val VARCHAR(25),
IN col3_val INT)
BEGIN
INSERT INTO tbl_name (col1, col2, col3)
VALUES (col1_val, col2_val, col3_val);
SELECT #TimeStamp;
END; //
DELIMITER ;
You can run this procedure with the following code:
CALL sp_procedure_name(col1_val, col2_val, col3_val);
As I'm not familiar with the Java, you'll need to finish it up with your side of code.
It seems that the variable contact is an instance for the newly inserted record. As it contains the newly generated id (primary key) field value, you can execute a new query to return the required timestamp field value for this new id.
The query may look like this:
select timestamp_field from my_table where id=?
Use PreparedStatement to input new id value and execute it to fetch required timestamp field value.
GeneratedKeyHolder also has two methods: getKeyList() that returns Map<String,Object> of generated fields; and getKeyList() that produces a list of generated keys for all affected rows.
See java doc of GeneratedKeyHolder and Spring tutorial of auto generated keys
In addition Spring's SimpleJdbcInsert has methods for generated key retrieval. See also method SimpleJdbcInsert#usingGeneratedKeyColumns
There are 2 methods in java.sql.Connection class causing PreparedStatement execution to return selected key columns :
PreparedStatement prepareStatement(String sql,
int[] columnIndexes)
throws SQLException
PreparedStatement prepareStatement(String sql,
String[] columnNames)
throws SQLException
You don't need to use Spring KeyHolder & JDBCTemplate to do this.
The give hope you could number/name your timestamp column. But the javadoc doesn't require or suggest that any JDBC implementation can return non-key columns, so your out of luck with this approach:
Creates a default PreparedStatement object capable of returning the auto-generated keys
designated by the given array. This array contains the names of the columns in the target
table that contain the auto-generated keys that should be returned.
As suggested in another answer, can switch to a stored procedure that does exactly what you want (CallableStatement is actually a PreparedStatement that executes storedprocedures - i.e. a subclass).
Can populate the timestamp column within the prepared statement via new Timestamp(new Date()) - but you should have in place a mechanism to sync times across your various servers (which is often used in windows and *nix environments). Your trigger could set the timestamp only if a value wasn't already provided.
As part of your app & DB design, you need to commit to a philosophy of where certain operations occur. If the DB derives needed data, the app needs to refresh data - you must pay the price of separate query executions or a combined stored proc that inserts & retrieves.

Categories

Resources