I'm using jdbi (but would prepared to use raw jdbc if needed). My DB is currently Oracle. I have an update that updates the first row matching certain criteria. I want to get the primary key of the updated row from the same statement. Is this possible?
I tried
Integer rowNum = handle
.createUpdate(sqlFindUpdate)
.bind("some var", myVal)
.executeAndReturnGeneratedKeys("id")
.mapTo(Integer.class)
.findOnly();
but I guess this isn't a generated key, as it doesn't find it (illegal state exception, but the update succeeds).
Basically, I have a list of items in the DB that I need to process. So, I want to get the next and mark it as "in progress" at the same time. I'd like to be able to support multiple worker threads, so it needs to be a single statement - I can't do the select after (the status has changed so it won't match anymore) and doing it before introduces a race condition.
I guess I could do a stored procedure that uses returning into but can I do it directly from java?
I'm answering my own question, but I don't think it's a good answer :) What I'm doing is kind of a hybrid. It is possible to dynamically run PL/SQL blocks from jdbi. Technically, this is from Java as I had asked, not via a stored procedure. However, it's kind of a hack, in my opinion - in this case why not just create the stored procedure (as I probably will, if I don't find a better solution). But, for info, instead of:
String sql = "update foo set status = 1 where rownr in (select rownr from (select rownr from foo where runid = :runid and status = 0 order by rownr) where rownum = 1)";
return jdbi.withHandle((handle) -> {
handle
.createUpdate(sql)
.bind("runid", runId)
.executeAndReturnGeneratedKeys("rownr")
.mapTo(Integer.class)
.findOnly();
});
you can do
String sql = "declare\n" +
"vRownr foo.rownr%type;\n" +
"begin\n" +
"update foo set status = 1 where rownr in (select rownr from (select rownr from foo where runid = :runid and status = 0 order by rownr) where rownum = 1) returning rownr into vRownr;\n" +
":rownr := vRownr;\n" +
"end;";
return jdbi.withHandle((handle) -> {
OutParameters params = handle
.createCall(sql)
.bind("runid", runId)
.registerOutParameter("rownr", Types.INTEGER)
.invoke();
return params.getInt("rownr");
});
Like I said, it's probably better to just create the procedure in this case, but it does give you the option to still build the SQL dynamically in java if you need to I guess.
Based on this question, as linked by #APC in the comments, it is possible to use the OracleReturning class without the declare/begin/end.
String sql = "update foo set status = 1 where rownr in (select rownr from (select rownr from foo where runid = ? and status = 0 order by rownr) where rownum = 1) returning rownr into ?";
return jdbi.withHandle((handle) -> {
handle
.createUpdate(sql)
.bind(0, runId)
.addCustomizer(OracleReturning.returnParameters().register(1, OracleTypes.INTEGER))
.execute(OracleReturning.returningDml())
.mapTo(Integer.class)
.findOnly();
});
However, OracleReturning doesn't support named parameters, so you have to use positionals. Since my main reason for using JDBI over plain JDBC is to get named parameter support, that's important to me, so I'm not sure which way I'll go
Pretty hard dependency on it being an Oracle DB you're calling...
Update: enhancement for named parameters in OracleReturning was merged to master, and will be included in 3.1.0 release. Kudos to #qualidafial for the patch
Related
I'm altering a Postgres column's type from UUID to TEXT with
ALTER TABLE my_table ALTER COLUMN id TYPE TEXT;
But I want to ensure my code works with both column types. i.e. As it's difficult to sync up the alteration of the db with the code which runs on a server.
I'm testing this locally by switching between the types. This can be done by moving back to UUID (while the data is in the correct format) with
ALTER TABLE my_table ALTER COLUMN id TYPE UUID USING id::uuid;
This works. But when switched to a TEXT and running my Java code I noticed some failures in the tests. Specifically whenever a WHERE statement makes use of the switched id. e.g.
UPDATE my_table SET name = 'new_name' WHERE id = '00000000-0000-0000-0001-000000000000'::uuid;
generates
ERROR: operator does not exist: text = uuid
That makes sense to me as the namedParameterJdbcTemplate class I'm using is adding a ::uuid to the end of the id in the SQL query. I'm not certain why it bothers doing that but it can be worked around by converting the passed in parameter from a UUID to a string.
However if I switch the column back to UUID I get the following error with
UPDATE my_table SET name = 'new_name' WHERE id = '00000000-0000-0000-0001-000000000000';
generating
ERROR: operator does not exist: text = uuid
I can accept the first update query causing issues because I can run it on the psql command line and get the same error. However this does not happen with the second command in psql.
Furthermore if I stop using the namedParameterJdbcTemplate's parameters and bake the values into the sql string it works.
String sqlStatement = "UPDATE my_table SET name = :v_new_name WHERE id = :v_id";
MapSqlParameterSource sqlParameterMap = new MapSqlParameterSource();
sqlParameterMap.addValue("v_new_name", "New Name");
sqlParameterMap.addValue("v_id", id.toString());
namedParameterJdbcTemplate.update(sqlStatement, sqlParameterMap);
Generates
ERROR: operator does not exist: uuid = character varying
thrown from SQLStateSQLExceptionTranslator.java.
versus
String sqlStatement = "UPDATE my_table SET name = 'New Name' WHERE id = '" + id.toString() + "'";
MapSqlParameterSource sqlParameterMap = new MapSqlParameterSource();
namedParameterJdbcTemplate.update(sqlStatement, sqlParameterMap);
which works.
Why is that the case? It feels like the namedParameterJdbcTemplate is doing some extra type checking that I cannot find.
Frustrated with the idea of having to bake the variables into the SQL statement I took a guess and tried the following
String sqlStatement = "UPDATE my_table SET name = :v_new_name WHERE id = :v_id";
MapSqlParameterSource sqlParameterMap = new MapSqlParameterSource();
sqlParameterMap.addValue("v_new_name", "New Name");
sqlParameterMap.addValue("v_id", id.toString(), java.sql.Types.OTHER);
namedParameterJdbcTemplate.update(sqlStatement, sqlParameterMap);
To my surprised it worked. Before I had experimented with the differences between Types.BINARY and Types.VARCHAR. But I guess I needed to be less specific. The definition of OTHER states
The constant in the Java programming language that indicates that the SQL type is database-specific and gets mapped to a Java object that can be accessed via the methods getObject and setObject.
Which sounds like an appropriate default value but it seems the library does not use it as such.
I do have to note that this only works with id.toString(). Using id by itself leads to a type error.
Thanks to some insights from #a_horse_with_no_name which helped.
I'm attempting to convert an Oracle MERGE statement to a MySQL Update statement. This particular MERGE statement only does an update (no inserts), so am unclear why the previous engineer used a MERGE statement.
Regardless, I know need to convert this to MySQL and am not clear as to how this is done. (side note, I'm doing this within a JAVA App)
Here is the MERGE statement :
MERGE INTO table1 a
USING
(SELECT DISTINCT(ROWID) AS ROWID FROM table2
WHERE DATETIMEUTC >= TO_TIMESTAMP('
formatter.format(dWV.getTime())
','YYYY-MM-DD HH24:MI:SS')) b
ON(a.ROWID = b.ROWID and
a.STATE = 'WV' and a.LAST_DTE = trunc(SYSDATE))
WHEN MATCHED THEN UPDATE SET a.THISIND = 'S';
My attempts goes something like this :
UPDATE table1 a
INNER JOIN table2 b ON (a.ROWID = b.ROWID
and a.STATE = 'WV'
and a.LAST_DTE = date(sysdate()))
SET a.THISIND = 'S'
WHERE DATETIMEUTC >= TO_TIMESTAMP('formatter.form(dWV.getTime())', 'YYYY-MM-DD HH24:MI:SS')
However, I'm unclear if this is actually doing the same thing or not?
As noted by you, the original Oracle MERGE statement only performs updates, no inserts.
The general syntax of your MySQL query looks ok compared to the Oracle version. Here is an updated version :
UPDATE table1 a
INNER JOIN table2 b
ON a.ROWID = b.ROWID
AND b.DATETIMEUTC >= 'formatter.form(dWV.getTime())'
SET a.THISIND = 'S'
WHERE
a.STATE = 'WV'
AND a.LAST_DTE = CURDATE()
Changes :
current date can be obtained with function CURDATE()
'YYYY-MM-DD HH24:MI:SS' is the default format for MySQL dates, hence you do not need to convert it, you may just pass it as is (NB1 : it is unclear what 'formatter.form(dWV.getTime())' actually means - NB2 : if you ever need to translate a string to date, STR_TO_DATE is your friend)
the filter conditions on table a are better placed in the WHERE clause, while those on table b would better belong in the INNER JOIN
It feels like I'm close, but I cannot figure out how to do something like the below in jOOq.
MERGE INTO USER_ASSIGNMENTS ua
USING (
SELECT core_object_id
FROM core_objects
WHERE exists(SELECT *
FROM LKU_CODE lc JOIN LKU_CODE_TYPE lct
ON lc.LKU_CODE_TYPE_ID = lct.LKU_CODE_TYPE_ID AND lct.CODE_TYPE = 'OBJECT_TYPE' AND
lc.CODE = 'PORTFOLIOS'
WHERE lc.LKU_CODE_ID = core_objects.OBJECT_TYPE_ID) AND object_id = 83
) "co"
ON (ua.CORE_OBJECT_ID = "co".CORE_OBJECT_ID AND USER_ID = 24 AND SECTION = 1)
WHEN MATCHED THEN UPDATE
SET create_date = sysdate, created_by = '24', capabilities = 12
WHERE capabilities <> 12
WHEN NOT MATCHED THEN INSERT
(CAPABILITIES, CORE_OBJECT_ID, CREATE_DATE, CREATED_BY, SECTION, USER_ID)
VALUES (5, "co".CORE_OBJECT_ID, sysdate, '24', 1, 24);
The big thing to note is that I'm trying to use the value returned by USING, so I have to alias it and .values() has to accept a field call. I think I can get around the .values() issue using the .values(Collection<?>) call, bundling things, including that field, into a Collection, so I think that I have that part. What concerns me is that I cannot do an .as() call after .using(). If I make the USING query a "table" via .asTable(), supplying an alias, will that let me call the field? Here's kind of what I have at the moment:
Table<Record1<BigDecimal>> usingStatement = readContext
.select(_co.CORE_OBJECT_ID)
.from(_co)
.where(DSL.exists(readContext.select(_lc.fields()).from(
_lc.join(_lct).onKey(Keys.LC_LCT___FK)
.and(_lc.CODE.equal(capability.getObjectTypeCode()))
.and(_lct.CODE_TYPE.equal(LkuCodeTypeLookup.OBJECT_TYPE))))).asTable("sdf");
...
return writeContext
.mergeInto(_ua)
.using(usingStatement)
.on(sectionalConditions.and(_ua.CORE_OBJECT_ID.equal(coidField)))
.whenMatchedThenUpdate()
.set(_ua.CREATE_DATE, time)
.set(_ua.CREATED_BY, creator)
.set(_ua.CAPABILITIES, capabilities)
.where(_ua.CAPABILITIES.notEqual(capabilities))
.whenNotMatchedThenInsert(_ua.CAPABILITIES, _ua.CORE_OBJECT_ID, _ua.CREATE_DATE,
_ua.CREATED_BY, _ua.SECTION, _ua.USER_ID)
.values(capabilities, gcoid, time, creator, section, uuid).execute();
A "straight merge" using dual is simple in jOOq, but I'd like to try to combine that select into the merge to save queries and let the DB do what it does best, so I'm trying not to have to get core_object_id in another query, if possible.
The aliasing really happens on the table (i.e. the select), not on some artefact returned by the USING clause. At least, that's how jOOQ models it. You have already correctly aliased your usingStatement variable. Now all you have to do is dereference the desired column from it, e.g.:
usingStatement.field(_co.CORE_OBJECT_ID);
This will look for the column named CORE_OBJECT_ID in the usingStatement table.
I have a spring application which reads data from the Database and sends it to system 'X'. I am using task executors to spin up threads, so there are like 5 threads which are reading the database for rows at the same time. For each thread I need to make sure that unique records are selected.
To achieve this I am using JdbcTemplate and "select for update"
I have written the code but in the logs I am able to see 2 threads picking up the same rows. I am not able to figure out the root cause of this issue.
Does anyone has a suggestion
try {
List<Map<String, Object>> rows = getJdbcTemplate().queryForList(
SELECT_FOR_UPDATE,
new Object[] {a,b,c,d});
for (Map<String,Object> row : rows) {
Header a = new Header();
a.setMailID(((BigDecimal)row.get("mailID")).intValue());
a.setVersion(((BigDecimal)row.get("version")).intValue());
// some other parameters to get
getJdbcTemplate().update(UPDATE_MSG_STATE_VERSION_N_ORIG_MSG_STAT,
x,
a.getVersion()+1,
y),
a.getMailID(),
a.getVersion());
headers.add(a);
}
}
UPDATE_MSG_STATE_VERSION_N_ORIG_MSG_STAT = update MESSAGE set MSG_STAT_CD = ?, VERSION_NBR = ?, ORIG_MSG_STAT_CD=?, LAST_UPD_TS=SYSTIMESTAMP where MESSAGE.MAIL_ID = ? and VERSION_NBR = ?
String SELECT_FOR_UPDATE = "select m.MAIL_ID mailID, m.VERSION_NBR version, m.MSG_STAT_CD state,"
+ "from message m "
+ "and m.MSG_STAT_CD in ('Nerwerw')"
+ " and m.create_ts > (sysdate - ?)"
+ " and mod(mail_id,?) = ?"
+ " and ROWNUM <= ?"
+ " order by mt.MSG_PRIORITY FOR UPDATE";
You need to annotate your class with #Repostitory tag and the #Transactional tag to make sure that all the actions in the same call are handled in one transaction.
If they are not handled in the same transaction then each SELECT_FOR_UPDATE will happen on a different transaction and thus your threads queries will not be syncronized and your select_for_update does not matter.
Have you had transaction control properly set up?
If not, the transaction will only happen for the duration of the update statement, and will be committed automatically (You are using Oracle I believe, base on your syntax).
That means, although you acquired the lock of those records, they are released right-away.
Do you have access to modify the database? If I understand your question correctly I recently had a similar problem and implemented a scheme like this:
Add a new column to your database like "thread_number" or something like that. Set it to some default value like 0. Give each thread a unique identifier. Then you "claim" a record in the database by updating its "thread_number" to the identifier of the thread processing it. Then the other threads will not find it when querying if you include "where thread_number = 0" in the SQL.
I know it's kind of broad, but I hope it helps.
I have a SQL statement which works when executed in MS Server Management Studio and works when submitted from C# but which does not work when submitted from Java (1.6, using sqljdbc4.jar).
The basic problem seems to be selecting into a table variable. Following up on the first comment I've completely re-written this question using simpler examples to show what works and does not work.
The following query:
DECLARE #IsLoadRaw as INT = ?
DECLARE #PrimaryID as varchar(1000) = ?
--Declare temporary table to hold message IDs and fill according to Primary ID.
DECLARE #MessageIDs TABLE ( MessageID BIGINT )
SELECT MessageID FROM Messages WHERE PrimaryID = #PrimaryID
works both in SQL Management Studio and when submitted from Java. In both cases it returns a result set with two MessageIDs (correct for the given PrimaryID I'm using to test).
The following query:
DECLARE #IsLoadRaw as INT = ?
DECLARE #PrimaryID as varchar(1000) = ?
--Declare temporary table to hold message IDs and fill according to Primary ID.
DECLARE #MessageIDs TABLE ( MessageID BIGINT );
INSERT #MessageIDs SELECT MessageID FROM Messages WHERE PrimaryID = #PrimaryID;
SELECT * FROM #MessageIDs;
works in SQL Management Studio where it returns a result set with the same two MessageIDs. When submitted from Java it does not return any result set.
The complete statement, which makes us of #MessageIDs, works when submitted from C# via ADO.NET. I assume the second sample here would work as well. The problem is isolated to Java and seems to relate to using a table variable. Since the code appears correct and runs under SQL Management Studio I'm perplexed as to how to debug this.
Any idea why this is not working from Java? What tools can I use to understand what the server is doing with this query WHEN SUBMITTED FROM Java?
I did some more excavation and found the answer:
INSERT #MessageIDs SELECT MessageID FROM Messages WHERE PrimaryID = #PrimaryID;
when submitted from Java returns an update count. When submitted from C# or from SQL Management Console it does not return an update count. I was not expecting this so it took some digging to find.
The java API for stepping through results of an execute() this is confusing, there are not may examples, and at least one that I found was not fully correct. I'll explain how I understand this to work.
Since most statements are simple, one change or one select, there are convenience execute methods on Statement, such as executeQuerry(), which returns a result set. Most cases use these and that is the end of the story.
If you have a more complex statement which does several things you call execute() and get back a list of things. INSERT, UPDATE, and DELETE (I believe) return a count of records modified. SELECT returns a result set. The result of executing a complex statement is a list of update counts and result sets, in the order they were executed. You then write code that steps through this list, processing each item.
The statement
DECLARE #MessageIDs TABLE ( MessageID BIGINT )
INSERT #MessageIDs SELECT MessageID FROM Messages WHERE PrimaryID = #PrimaryID;
SELECT * FROM Messages WHERE MessageID IN (SELECT MessageID FROM #MessageIDs) ORDER BY MessageID;
SELECT * FROM Attrs WHERE MessageID IN (SELECT MessageID FROM #MessageIDs) ORDER BY MessageID;
returns 2 result sets. In java, and only in java for reasons I don't know, the INSERT #MessageIDs... statement returns an update count, which is the first item in the list.
The java API for this is confusing. Statement.execute() and Statement.getMoreResults() return:
true if the next result is a ResultSet
false if the next result is an update count OR there are no more results
false has two meanings and can not be interpreted to be the end of results. You have to also check for a non-zero update count.
The final, functioning code ended up looking like this:
List<DtaMessage> msgList = new ArrayList<DtaMessage>();
boolean isResult = stmt.execute();
// Skip over update counts.
while (!isResult) {
if (stmt.getUpdateCount() == 0)
// End of results.
return msgList;
isResult = stmt.getMoreResults();
}
// Process first result set.
ResultSet rs = stmt.getResultSet();
while (rs.next())
{
DtaMessage msg = PopulateMessage(rs, isLoadRaw);
msgList.add(msg);
}
rs.close();
// Skip over update counts.
isResult = stmt.getMoreResults();
while (!isResult) {
if (stmt.getUpdateCount() == 0)
// end of results.
return msgList;
isResult = stmt.getMoreResults();
}
// Process second result set.
rs = stmt.getResultSet();
while (rs.next())
{
// process.
}
rs.close();
return msgList;
Though my sample SQL does nothing that would generate an update count between the two result sets this method will process results from several different SQL statements so I added the code to skip over up date counts which may show up in some cases.