Is there anyway to create a flyway migration class utilizing the NamedParameterJdbcTemplate rather than the standard JdbcTemplate that comes across from the implementation of SpringJdbcMigration?
I have an upgrade I need to run where I need to convert a column type from text to integer (Replacing a string value with an internal id associated with that value.)
The way I'm doing this is temporarily storing the string values for a reverse lookup, deleting the column and re-adding it as the proper type, and then running an UPDATE call to add in the appropriate ID to the records. I have code similar to the following I want to execute as part of the migration:
String sql = "UPDATE my_table SET my_field = :my_field WHERE my_id IN (:my_ids)";
MapSqlParameterSource source = new MapSqlParameterSource();
source.addValue("my_field", someIntValue); // the internal id of the string I want to use.
source.addValue("my_ids", someListOfPKIds); // List of PK ids.
namedTemplate.update(sql,source); //namedTemplate is a NamedParameterJdbcTemplate
However, it seems as if I can't take advantage of the NamedParameterJdbcTemplate. Am I incorrect in this?
According to Flyway sources they create a new JdbcTemplate in SpringJdbcMigrationExecutor
However you can try creating a new NamedParameterJdbcTemplate in your migration given the classic JdbcTemplate. Check this constructor. E.g. new NamedParameterJdbcTemplate(jdbcTemplate)
Related
I have a jOOQ MockConnection/DSL set up for unit tests to be able to do insert, but in at least one instance of my testing I have to also implement a MockResult for a subsequent select statement.
My question is, why does jOOQ execute a select statement in org.jooq.impl.AbstractDMLQuery#executeReturningGeneratedKeysFetchAdditionalRows -> selectReturning for my insert?
My insert is a simple myRecord.insert(), and the mocked DSL looks something like this:
// Simplified
var connection = new MockConnection(ctx -> {
var sql = ctx.sql();
if (sql.startsWith("insert")) {
return mockResultCount(1); // Impl elsewhere
}
return null;
});
var dsl = DSL.using(connection, SQLDialect.MYSQL);
[...]
var myRecord = new MyRecord();
myRecord.setX(...).setY(...);
dsl.attach(myRecord);
// Why does this require a mocked insert result AND a mocked select result?
myRecord.insert();
And my test fails because jOOQ needs the DSL to return a result for a select on something like SELECT ID_COLUMN, UNIQUE_KEY_COLUMN WHERE UNIQUE_KEY_COLUMN = ?
The only thing I can think of is that this table has a unique key?
Anyone know why a simple record.insert(); requires a select statement to be executed?
This depends on a variety of factors.
First off, the dialect. MySQL, for example cannot fetch values other than the identity values via JDBC's Statement.getGeneratedKeys(). This is the main reasons why there might be an additional query at all.
Then, for example, your Settings.returnAllOnUpdatableRecord configuration might cause this behaviour. If you have turned this on, then a separate query is required in MySQL to fetch the non-identity values.
Or, if your identity column (in MySQL, the AUTO_INCREMENT column) does not coincide with your primary key, which seems to be the case given your logged SQL statement, where you distinguish between an ID_COLUMN and a UNIQUE_KEY_COLUMN.
The reason for this fetching is that jOOQ assumes that such values may be generated (e.g. by triggers). I guess that the special case where
The identity and the primary key do not coincide
The primary key has been supplied fully by the user
We can attempt not to fetch the possibly generated primary key value, and fetch only the identity value. I've created a feature request for this: https://github.com/jOOQ/jOOQ/issues/9125
I'm having an odd problem with SimpleJdbcInsert.executeAndReturnKey with Sybase (jTDS driver) and certain data.
Take the following example:
SimpleJdbcInsert insert = new SimpleJdbcInsert(jdbcTemplate)
.withTableName("TABLE_NAME")
.usingGeneratedKeyColumns("ID");
List<String> columns = new ArrayList<String>();
columns.add("SOME_NUMERIC_DATA");
columns.add("SOME_STRING_DATA");
Map<String, Object> params = new HashMap<String, Object>();
params.put("SOME_NUMERIC_DATA", 10.02);
params.put("SOME_STRING_DATA", "AAAA");
Number insertId = insert.executeAndReturnKey(params);
The above will fail with
DataIntegrityViolationException: Unable to retrieve the generated key for the insert
The insert itself is fine as if I do an insert.execute(params) the insert will work correctly (but I need the generated column value).
If I insert null instead of 10.02 for the SOME_NUMERIC_DATA column then it will work correctly and return the generated column value. Also if all of the fields are VARCHAR/String then it will work correctly.
Can anyone see anything here that might be causing this with a combination of string and numeric fields.
I should also add that when I use the exact same code with an H2 database it works all of the time - this seems to be related to Sybase/jTDS
I had the same problem with SQL Server and fixed it by calling this configuration method right before the call to executeAndReturnKey():
mySimpleJdbcInsert.setAccessTableColumnMetaData(false);
I suspect the error has to do with database metadata : as explained in the spring reference http://docs.spring.io/spring-framework/docs/current/spring-framework-reference/html/jdbc.html, SimpleJdbcInsert uses database metadata to construct the actual insert statement.
One could also use the SQL OUTPUT clause such as
INSERT INTO myTable (Name, Age)
OUTPUT Inserted.Id
VALUES (?,?)
And use some more generic JdbcTemplate.execute() to handle the insert.
Hi I have a table in Postgres, say email_messages. It is partitioned so whatever the inserts i do it using my java application will not effect the master table, as the data is actually getting inserted in the child tables. Here is the problem, I want to get an auto generated column value (say email_message_id which is of big serial type). Postgres is returning it as null since there is no insert being done on master table. For oracle I used GeneratedKeyHolder to get that value. But I'm unable to do the same for partitioned table in postgres. Please help me out.
Here is the code snippet we used for oracle
public void createAndFetchPKImpl(final Entity pEntity,
final String pStatementId, final String pPKColumnName) {
final SqlParameterSource parameterSource =
new BeanPropertySqlParameterSource(pEntity);
final String[] columnNames = new String[]{"email_message_id"};
final KeyHolder keyHolder = new GeneratedKeyHolder();
final int numberOfRowsEffected = mNamedParameterJdbcTemplate.update(
getStatement(pStatementId), parameterSource, keyHolder, columnNames);
pEntity.setId(ConversionUtil.getLongValue(keyHolder.getKey()));
}
When you use trigger-based partitioning in PostgreSQL it is normal for the JDBC driver to report that zero rows were affected/inserted. This is because the original SQL UPDATE, INSERT or DELETE didn't actually take effect, no rows were changed on the main table. Instead operations were performed on one or more sub-tables.
Basically, partitioning in PostgreSQL is a bit of a hack and this is one of the more visible limitations of it.
The workarounds are:
INSERT/UPDATE/DELETE directly against the sub-table(s), rather than the top-level table;
Ignore the result rowcount; or
Use RULEs and INSERT ... RETURNING instead (but they have their own problems) (Won't work for partitions)
I am working on a project that requires JDBC Calls to an Oracle Database. I have set up UCP pooling to work with SpringJDBC. I have a fairly simple query that I am executing like the following...
SELECT * FROM TABLE WHERE ID IN (:ids)
my java code to set this query up looks like the following...
NamedParameterJdbcTemplate template = new NamedParameterJdbcTemplate(datasource);
Map<String,Object> paramMap = new HashMap<String,Object>();
paramMap.put("ids", Arrays.asList(idArray));
List<Result> results = template.query("SELECT * FROM TABLE WHERE ID IN (:ids)",
paramMap, new ResultRowMapper());
This all performs fine as long as there is only 1 id in the array. When I add a 2nd ID the query takes nearly 5 minutes to run. If I take the exact query and execute it in SQLDeveloper, it takes .093 seconds.
Something must be going terribly wrong with my code or configuration... Does anyone have any ideas?
EDIT:
I stripped out the usage of the Spring NamedParameterJdbcTemplate and went with just straight Jdbc and everything seems to perform great. What is it that NamedParameterJdbcTemplate is doing differently?
Well, I found the difference between my straight jdbc solution and my spring-jdbc solution in this situation... It appears that as #Annjawn below explained, it is a bind variable issue not a spring-jdbc issue. My spring-jdbc issue was trying to bind a variable to an index (that doesn't exist) thus doing a table scan...
My straight JDBC solution ends up just doing a string replacement and executing as is thus no table scan...
The below link explains the difference.
http://bytes.com/topic/oracle/answers/65559-jdbc-oracle-beware-bind-variables
I don't understand how to get auto generated keys with commons-dbutils?
You can use QueryRunner#insert(). Below is an example. Given a table called users, which has an auto generated primary key column and a varchar column called username, you can do something like this:
DataSource dataSource = ... // however your app normally gets a DataSource
QueryRunner queryRunner = new QueryRunner(dataSource);
String sql = "insert into users (username) values (?)";
long userId = queryRunner.insert(sql, new ScalarHandler<Long>(), "test");
As a matter of fact I think it cannot be done with the current version of common-dbutils. A few months ago, when I was working for another company, I extented the QueryRunner with my own implementation.
The request has been submitted to the DbUtils project, and there you can even find a viable implementation which I guess you could copy if you really need it.
https://issues.apache.org/jira/browse/DBUTILS-54