I have a following query:
SELECT
date, userId, value
FROM
tbl_table
WHERE
date = to_date(:date, 'YYYY-MM-DD')
AND
userId = :userId
It allows to request for a single value like this:
MapSqlParameterSource args = new MapSqlParameterSource();
args.addValue("date", date, Types.VARCHAR);
args.addValue("userId", userId, Types.VARCHAR);
SqlRowSet rowSet = jdbcTemplate.queryForRowSet(SQL_SELECT, args);
jdbcTemplate.queryForRowSet(SQL_SELECT_MARKET_VALUE, args);
This is totally ok, but extremelly slow in case you have to query value for many date/userId pairs.
I would like to optimize it using multicolumn IN clause, but how do I handle multicolumn list via JDBC (or better question: is it possible using JDBC)?
Oracle supports multiple columns in "in" predicate:
SELECT
date, userId, value
FROM
tbl_table
WHERE
(date, userId) IN ((to_date(:date1, 'YYYY-MM-DD'), :userId1), (to_date(:date2, 'YYYY-MM-DD'), :userId2))
However JDBC doesn't provide a decent support of in-statement parameters - you will have to build the query using StringBuilder or use some of workarounds described here
It depends of details. If user/date filter is quite persistent (should be user more than once) temporary table will be the best decision. You can fill it once, you can edit it, and you can use it several times without reloading.
If you need of quite large number of pairs, I'd recommend you to use a table type. It would be something like this:
create type DateUserPair as object (dt date, userid integer);
create type DateUserPairs as table of DateUserPair;
....
SELECT
date, userId, value
FROM
tbl_table src,
table(cast :filter as DateUserPairs) flt
WHERE
src.date = flt.dt and
src.userId = flt.userId;
If filter would be small, filtering by (date, userId) in ((?,?), (?,?), ...) would be simple and clever.
Btw, your approach
date = to_date(:date, 'YYYY-MM-DD')
isn't good practise. Such conversions should be done by client, not by server. Use
date = :date
and assign it as date instead.
If what you want is to pass JDBC a list of date/userId pairs, or a list of dates and a list of userIds, I think it will not work.
A possible workaround in Oracle would be using a global temporary table with ON COMMIT DELETE ROWS. Your would have:
-- DDL for the workaround
CREATE GLOBAL TEMPORARY TABLE admin_work_area
(d DATE,
userId VARCHAR2(10))
ON COMMIT DELETE ROWS;
...
-- Start of query method pseudo-code
...
-- You should be able to JDBC-batch these for better performance
INSERT INTO temp_multicolumn_filter (d, userId) VALUES (date1, userId1);
INSERT INTO temp_multicolumn_filter (d, userId) VALUES (date2, userId2);
...
-- Query using temp_multicolumn_filter
SELECT date, userId, value
FROM tbl_table
WHERE
(date, userId) in (select d, userId from temp_multicolumn_filter);
...
-- End of query method pseudo-code
As the temporary table has the ON COMMIT DELETE ROWS, each transaction will only see its own date/userId pairs. Just remember that if you use the temporary table more than once in the same transaction, you might need to clear it before using it.
UPDATE:
Another option would be using a PIPELINED PL/SQL function to "build" your table inside the query:
-- DDL for this workaround
CREATE TYPE date_userid_pair AS OBJECT (
d DATE,
userId VARCHAR2(10));
CREATE TYPE date_userid_dataset IS TABLE OF date_userid_pair;
CREATE FUNCTION decode_date_userid_pairs(dates in varchar2, userIds in varchar2)
RETURN date_userid_dataset PIPELINED IS
result_row date_userid_pair;
BEGIN
WHILE there are more "rows" in the parameters LOOP
result_row.d := -- Decode next date from dates
result_row.userId := -- Decode next userId from userIds
PIPE ROW(result_row);
END LOOP;
END;
// Start of query method pseudo-code
...
// This is Java code: encodeList encodes a List of elements into a String.
encodedDates = encodeList(listOfDates);
encodedUserIds = encodeList(listOfUserIds);
...
// Query using temp_multicolumn_filter
SELECT date, userId, value
FROM tbl_table
WHERE
(date, userId) in (
select date, userId
from TABLE(decode_date_userid_pair(:encodedDates, :encodedUserIds));
...
// End of query method pseudo-code
But this is more hacky, and if you don't have privileges to create a temporary table, then you probably won't have CREATE TYPE either (you might not even have CREATE FUNCTION privilege).
Related
I have a table with some data, there are record id, userIds, timestamp and data columns. Receive from the client a list of userIds, initially, I just had to fetch data by the same timestamp range for all useIds, just using userId IN (list). However, now I'm required to get data by different timestamp ranges for each userId, let's say userID 1 needs data from 1643580000000 to 1646431200000 and userId 2 from 1626418800000 to 1647500400000 (utcTimestamp in mills).
Most probably I'll receive a list of [userId, startTime, endTime], so, I was considering to loop the main query for each userId with its respective timestamp range (I've seen it's a bad idea due to performance, but if I have to, I have to), but I also found out about cursors (not much experience here).
I wanted to know if it's possible to get what I want without loops or cursors, and if not, best way with each one.
Thanks in advance!
Notes: using MariaDB. SQL query will be used as nativeQuery in a Java service repository.
You can run a query laike this.
Of course you have to add for every user his own time range
SELECT * FROM mytable
WHERE (userID = 1 AND `timestamp` BETWEEN 1643580000000 AND 1646431200000)
OR (userId = 2 AND `timestamp`BETWEEN 1626418800000 AND 1647500400000)
If you have a lot of rows and not that many id to process, you can do it with
SELECT * FROM mytable
WHERE (userID = 1 AND `timestamp` BETWEEN 1643580000000 AND 1646431200000)
UNION
SELECT * FROM mytable
WHERE (userId = 2 AND `timestamp`BETWEEN 1626418800000 AND 1647500400000)
Here also you need to add for every id another UNION
I have an update/insert SQL query that I created using a MERGE statement. Using either JdbcTemplate or NamedParameterJdbcTemplate, does Spring provide a method that I can use to update a single record, as opposed to a Batch Update?
Since this query will be used to persist data from a queue via a JMS listener, I'm only dequeuing one record at a time, and don't have need for the overhead of a batch update.
If a batch is the only way to do it through Spring JDBC, that's fine... I just want to make certain I'm not missing something simpler.
You can use a SQL MERGE statment using only a one row query containing your parameters.
For example if you have a table COMPANYcontaing IDas a key and NAMEas an attribute, the MERGE statement would be:
merge into company c
using (select ? id, ? name from dual) d
on (c.id = d.id)
when matched then update
set c.name = d.name
when not matched then insert (c.id, c.name)
values(d.id, d.name)
If your target table contains the parametrised key, the name will be updated, otherwise a new record will be inserted.
With JDBCTemplate you use the update method to call the MERGEstatement, as illustrated below (using Groovy script)
def id = 1
def name = 'NewName'
String mergeStmt = """merge into company c
using (select ? id, ? name from dual) d
on (c.id = d.id)
when matched then update
set c.name = d.name
when not matched then insert (c.id, c.name)
values(d.id, d.name)""";
def updCnt = jdbcTemplate.update(mergeStmt, id, name);
println "merging ${id}, name ${name}, merged rows ${updCnt}"
Just use one of update methods, for example this one: JdbcTemplate#update instead of BatchUpdate.
Update updates a single record, batchUpdate updates multiple records using JDBC batch
It feels like I'm close, but I cannot figure out how to do something like the below in jOOq.
MERGE INTO USER_ASSIGNMENTS ua
USING (
SELECT core_object_id
FROM core_objects
WHERE exists(SELECT *
FROM LKU_CODE lc JOIN LKU_CODE_TYPE lct
ON lc.LKU_CODE_TYPE_ID = lct.LKU_CODE_TYPE_ID AND lct.CODE_TYPE = 'OBJECT_TYPE' AND
lc.CODE = 'PORTFOLIOS'
WHERE lc.LKU_CODE_ID = core_objects.OBJECT_TYPE_ID) AND object_id = 83
) "co"
ON (ua.CORE_OBJECT_ID = "co".CORE_OBJECT_ID AND USER_ID = 24 AND SECTION = 1)
WHEN MATCHED THEN UPDATE
SET create_date = sysdate, created_by = '24', capabilities = 12
WHERE capabilities <> 12
WHEN NOT MATCHED THEN INSERT
(CAPABILITIES, CORE_OBJECT_ID, CREATE_DATE, CREATED_BY, SECTION, USER_ID)
VALUES (5, "co".CORE_OBJECT_ID, sysdate, '24', 1, 24);
The big thing to note is that I'm trying to use the value returned by USING, so I have to alias it and .values() has to accept a field call. I think I can get around the .values() issue using the .values(Collection<?>) call, bundling things, including that field, into a Collection, so I think that I have that part. What concerns me is that I cannot do an .as() call after .using(). If I make the USING query a "table" via .asTable(), supplying an alias, will that let me call the field? Here's kind of what I have at the moment:
Table<Record1<BigDecimal>> usingStatement = readContext
.select(_co.CORE_OBJECT_ID)
.from(_co)
.where(DSL.exists(readContext.select(_lc.fields()).from(
_lc.join(_lct).onKey(Keys.LC_LCT___FK)
.and(_lc.CODE.equal(capability.getObjectTypeCode()))
.and(_lct.CODE_TYPE.equal(LkuCodeTypeLookup.OBJECT_TYPE))))).asTable("sdf");
...
return writeContext
.mergeInto(_ua)
.using(usingStatement)
.on(sectionalConditions.and(_ua.CORE_OBJECT_ID.equal(coidField)))
.whenMatchedThenUpdate()
.set(_ua.CREATE_DATE, time)
.set(_ua.CREATED_BY, creator)
.set(_ua.CAPABILITIES, capabilities)
.where(_ua.CAPABILITIES.notEqual(capabilities))
.whenNotMatchedThenInsert(_ua.CAPABILITIES, _ua.CORE_OBJECT_ID, _ua.CREATE_DATE,
_ua.CREATED_BY, _ua.SECTION, _ua.USER_ID)
.values(capabilities, gcoid, time, creator, section, uuid).execute();
A "straight merge" using dual is simple in jOOq, but I'd like to try to combine that select into the merge to save queries and let the DB do what it does best, so I'm trying not to have to get core_object_id in another query, if possible.
The aliasing really happens on the table (i.e. the select), not on some artefact returned by the USING clause. At least, that's how jOOQ models it. You have already correctly aliased your usingStatement variable. Now all you have to do is dereference the desired column from it, e.g.:
usingStatement.field(_co.CORE_OBJECT_ID);
This will look for the column named CORE_OBJECT_ID in the usingStatement table.
I've a requirement where I need to pull out data from database.
The query is-
SELECT e.Data AS EntityBlob, f.Data AS FpmlBlob
FROM [Trades.InventoryRecord] ir, EntityBlob e, FpmlBlob f
WHERE %s AND uid = e.uid AND uid = f.uid
Here %s is the predicate after where clause which user will input from an html form.
User input will be in this form :
1. TradeDate = '2013-04-05' AND IsLatest = 'TRUE'
2. StreamId= 'IA0015'
3. The query may have IN clause also
Now when this query is rendered I get exception ambigous column streamId or ambigous column IsLatest, as these columns exists in more than one table with same name. So to remove this ambiguity I need to modify the query as - ir.IsLatest or ir.StreamId
To do so by java code, I need to first parse the predicate after where clause, extract column names and insert table name alias- 'ir' before each column name so that the query becomes -
SELECT e.Data AS EntityBlob, f.Data AS FpmlBlob
FROM [Trades.InventoryRecord] ir, EntityBlob e, FpmlBlob f
WHERE ir.TradeDate = '2013-04-05' AND ir.IsLatest = 'TRUE' AND uid = e.uid AND uid = f.uid
what is the best way to parse this predicate, or if there is any other way I can achieve the same result?
My answer to this question is to not parse the user input - there is far too much that can go wrong. It would be a lot better to have a UI with drop downs and buttons for selecting equality, inequality, ranges, in statements, etc. It may seem like more work, but protecting yourself from a SQL injection attack is even more. And even if you are not concerned about malicious SQL injection, then the user still has to get every thing exactly right, or the statement fails.
I have an auto generated timestamp that is created each time a record is inserted or updated in a mysql table. Is there a way to return this timestamp in a way similar to how I would use a keyholder to return a newly created id?
KeyHolder keyHolder = new GeneratedKeyHolder();
JdbcTemplate jdbcTemplate = new JdbcTemplate(dataSource);
//Insert Contact
jdbcTemplate.update(new PreparedStatementCreator() {
#Override
public PreparedStatement createPreparedStatement(Connection connection) throws SQLException {
PreparedStatement preparedStatement = connection.prepareStatement(SQL_ADD, Statement.RETURN_GENERATED_KEYS);
preparedStatement.setString(1, contact.getFirstName());
preparedStatement.setString(2, contact.getLastName());
preparedStatement.setInt(3, contact.getOrganizationId());
preparedStatement.setString(4, contact.getType());
preparedStatement.setInt(5, contact.getUserId());
return preparedStatement;
}
}, keyHolder);
//Use keyholder to obtain newly created id
contact.setId(keyHolder.getKey().intValue());
Is there some way to also return the new timestamp without having to requery the table? I have been looking for ways to return it along with the id as a key in the keyholder, but it doesn't seem to be returned as a key?
Not very satisfying, but I think "no" is the answer to your question. I don't know any of the Spring stuff, but I think this is due to the basic JDBC that it's wrapping. See http://docs.oracle.com/javase/6/docs/api/java/sql/Statement.html#getGeneratedKeys%28%29
You only option would be to create a stored procedure on MySQL that has an out parameter and call that. See http://dev.mysql.com/doc/refman/5.0/en/call.html.
There are few options for solving this issue on the MySQL database server side. You could start with creating a TRIGGER on the table. As TRIGGER has a restriction and cannot return data, you can set the TIMESTAMP value to a variable:
DEMILITER //
CREATE TRIGGER ai_tbl_name AFTER INSERT ON tbl_name
FOR EACH ROW
BEGIN
SET #TimeStamp = NEW.timestamp_column;
END;//
DELIMITER ;
To retrieve this timestamp value, run the following command:
SELECT #TimeStamp;
Since the variables are stored in the memory, there will be no need to open any tables again.
You go even further. You could create a STORED PROCEDURE in MySQL to automate all the above (sample code, as I do not know your table's details):
DELIMITER //
DROP PROCEDURE IF EXISTS sp_procedure_name //
CREATE PROCEDURE sp_procedure_name (IN col1_val VARCHAR(25),
IN col2_val VARCHAR(25),
IN col3_val INT)
BEGIN
INSERT INTO tbl_name (col1, col2, col3)
VALUES (col1_val, col2_val, col3_val);
SELECT #TimeStamp;
END; //
DELIMITER ;
You can run this procedure with the following code:
CALL sp_procedure_name(col1_val, col2_val, col3_val);
As I'm not familiar with the Java, you'll need to finish it up with your side of code.
It seems that the variable contact is an instance for the newly inserted record. As it contains the newly generated id (primary key) field value, you can execute a new query to return the required timestamp field value for this new id.
The query may look like this:
select timestamp_field from my_table where id=?
Use PreparedStatement to input new id value and execute it to fetch required timestamp field value.
GeneratedKeyHolder also has two methods: getKeyList() that returns Map<String,Object> of generated fields; and getKeyList() that produces a list of generated keys for all affected rows.
See java doc of GeneratedKeyHolder and Spring tutorial of auto generated keys
In addition Spring's SimpleJdbcInsert has methods for generated key retrieval. See also method SimpleJdbcInsert#usingGeneratedKeyColumns
There are 2 methods in java.sql.Connection class causing PreparedStatement execution to return selected key columns :
PreparedStatement prepareStatement(String sql,
int[] columnIndexes)
throws SQLException
PreparedStatement prepareStatement(String sql,
String[] columnNames)
throws SQLException
You don't need to use Spring KeyHolder & JDBCTemplate to do this.
The give hope you could number/name your timestamp column. But the javadoc doesn't require or suggest that any JDBC implementation can return non-key columns, so your out of luck with this approach:
Creates a default PreparedStatement object capable of returning the auto-generated keys
designated by the given array. This array contains the names of the columns in the target
table that contain the auto-generated keys that should be returned.
As suggested in another answer, can switch to a stored procedure that does exactly what you want (CallableStatement is actually a PreparedStatement that executes storedprocedures - i.e. a subclass).
Can populate the timestamp column within the prepared statement via new Timestamp(new Date()) - but you should have in place a mechanism to sync times across your various servers (which is often used in windows and *nix environments). Your trigger could set the timestamp only if a value wasn't already provided.
As part of your app & DB design, you need to commit to a philosophy of where certain operations occur. If the DB derives needed data, the app needs to refresh data - you must pay the price of separate query executions or a combined stored proc that inserts & retrieves.