To query a PostgreSQL 10.11 database, I am using jOOQ 3.12.4, which comes bundled with Spring Boot 2.2.
Let's assume I have built a query using jOOQ like this:
final String[] ids = ...;
final var query = dslContext.selectFrom(MY_TABLE).where(MY_TABLE.ID.in(ids));
final Map<String, List<MyTable>> changeDomains = query.fetch().intoGroups(MY_TABLE.ID, MyTable.class);
This code runs fine and produces the expected results. But when I refactor my query and introduce a named parameter (to reuse the query in multiple parts of my code), like this:
final String[] ids = ...;
final var query = dslContext.selectFrom(MY_TABLE).where(MY_TABLE.ID.in(param("ids")));
final Map<String, List<MyTable>> changeDomains = query.bind("ids", ids).fetch().intoGroups(MY_TABLE.ID, MyTable.class);
I suddenly start to get the following error:
org.springframework.jdbc.BadSqlGrammarException: jOOQ; bad SQL grammar ...; nested exception is org.postgresql.util.PSQLException: ERROR: operator does not exist: text = character varying[]
Hinweis: No operator matches the given name and argument type(s). You might need to add explicit type casts.
Edit: I get the same error when I use
MY_TABLE.ID.in(param("ids", String[].class))
instead.
How can I solve or work around this problem?
A better solution to your code reuse approach
But when I refactor my query and introduce a named parameter (to reuse the query in multiple parts of my code)
While you could use jOOQ this way (be careful, when mutating and reusing jOOQ queries in a non-threadsafe way!), it is generally recommended to use jOOQ in a more functional way, see e.g.:
https://blog.jooq.org/2017/01/16/a-functional-programming-approach-to-dynamic-sql-with-jooq/
https://www.jooq.org/doc/latest/manual/sql-building/dynamic-sql/
You don't gain much by re-using a jOOQ query, specifically, there's hardly any performance gain.
So, instead of this:
final var query = dslContext.selectFrom(MY_TABLE)
.where(MY_TABLE.ID.in(param("ids")));
final Map<String, List<MyTable>> changeDomains = query
.bind("ids", ids).fetch().intoGroups(MY_TABLE.ID, MyTable.class);
Write this:
public ResultQuery<MyTableRecord> query(String[] ids) {
return dslContext.selectFrom(MY_TABLE).where(MY_TABLE.ID.in(ids));
}
// And then:
final Map<String, List<MyTable>> changeDomains = query(ids)
.fetch().intoGroups(MY_TABLE.ID, MyTable.class);
The actual problem you ran into:
jOOQ, JDBC, and SQL don't support single bind value IN lists. While it seems useful to write this:
SELECT * FROM t WHERE c IN (:bind_value)
And passing an array or list as a single bind value, this is not supported in SQL. Some APIs might pretend that this is supported (but behind the scenes replace the single bind value by multiple ?, ?, ..., ?
PostgreSQL supports the = ANY (:bind_value) operator with arrays
SELECT * FROM t WHERE c = ANY (:bind_value)
You could use it in jOOQ using
dslContext.selectFrom(MY_TABLE).where(MY_TABLE.ID.eq(any(ids)));
That way, you could call the bind() method to replace the array prior to execution. However, I still recommend you write functions returning queries dynamically.
Related
I need to copy the records of one table to another table based on some condition.
String query = "insert into public.ticket_booking_archive select * from public.ticket_booking where ticketId in (:ticketIds)";
So here the :ticketIds are dynamic, where i need to pass ticketIds to make sure whether it satisfies the condition. So it may be the matching and non matching ticket id's here at runtime.
The values of ticketIds are something like this
('f1fa3a42-5837-11ec-bf63-0242ac130002','516fd14d-3c9d-4b4b-91a0-b684d8592dfe','c9652f86-734c-4df5-8ef9-d407cb3eaf7a','df7f2812-b445-45b4-b731-da23c36d7738','f1fa3a42-5837-11ec-bf63-0242ac130002'). And this is just an example. And the list might goes on.
Since it is of type UUID, I'm storing it into a Set<UUID>
Set<UUID> tktIds = new HashSet<UUID>();
for(int i=0 ; i<ticketIds.size(); i++) {
String ticketId = ticketIds[i];
tktIds.add(UUID.fromString(ticketId));
}
Map<String, Object> params = new HashMap<>();
params.put("ticketIds", tktIds);
SqlParameterSource namedParameters =
new MapSqlParameterSource().addValue("ticketIds",params.get("ticketIds"));
Since I'm using NamedParameterJdbcTemplate, so I'm using like below
int res = writeNamedJdbcTemplate.update(query, namedParameters);
res = 3 when executed programmatically.
Here the problem is, as soon as it finds the first matching value in the IN clause it executes. And it is not considering the other matching values (ticketIds here)
But if I execute the same query in pgadmin it works fine
insert into public.ticket_booking_archive select * from public.ticket_booking where ticketId in ('f1fa3a42-5837-11ec-bf63-0242ac130002','516fd14d-3c9d-4b4b-91a0-b684d8592dfe','c9652f86-734c-4df5-8ef9-d407cb3eaf7a','df7f2812-b445-45b4-b731-da23c36d7738','f1fa3a42-5837-11ec-bf63-0242ac130002');
result is 6. Working as expected.
writeNamedJdbcTemplate.queryForObject(query, namedParameters, Integer.class); //. throws an error
Can anyone please assist? I'm really not sure where I'm making a mistake
I am not quite sure whether you are using the appropriate JDBC template for the named parameters, but you can do the following:
you can consult this article to use the right template and employ proper SQL query composition,
for string passing you can wrap the parameter mapping as shown here
after all your named parameter should work
I have a query that returns a Postgres array of UUIDs:
SELECT e.id, e.date,
ARRAY
(
SELECT cs.api_id FROM event_condition_set ecs
JOIN condition_set cs on cs.id = ecs.condition_set_id
WHERE ecs.event_id = e.id
) AS condition_set_ids,
...
And then create and run this query as a native query: Query query = entityManager.createNativeQuery(queryString);
Since Hibernate can normally not deal with these Postgres arrays, I use Vlad's Hibernate Types library.
However, currently I need to register this UUIDArrayType globally in my application:
public class PostgreSQL95CustomDialect extends PostgreSQL95Dialect {
public PostgreSQL95CustomDialect() {
super();
this.registerHibernateType(Types.ARRAY, UUIDArrayType.class.getName());
}
}
Aside from the fact this is a bit ugly, it also leaves no room for other types of arrays.
(Note I also tried registering a generic ListArrayType but this throws a NullPointerException during execution of the query.)
I have also tried registering it as a scalar type:
query.unwrap(org.hibernate.query.NativeQuery.class)
.addScalar("condition_set_ids", UUIDArrayType.INSTANCE);
But this makes the entire query only return a single UUID, which is very strange and seems bugged.
Is there a way to ONLY use this UUIDArrayType specifically in this query?
(Please don't suggest using array_agg because the performance is terrible for this case)
you can call native queries using custom Hibernate types as follows:
String myJsonbData = ...;
String[] myStringArr = ...;
final String queryStr = "select your_function(?, ?, ...)"; // ? for each param
entityManager
.createNativeQuery(queryStr)
.setParameter(1, new TypedParameterValue(JsonBinaryType.INSTANCE, myJsonbData))
.setParameter(2, new TypedParameterValue(StringArrayType.INSTANCE, myStringArr));
This is just an example, but as a rule of thumb, you need to instantiate a new TypedParameterValue.
Answering my own question here. After waiting for a while and updating to the most recent library version (I'm on 2.19.2 right now) I don't have any issues anymore with the scalar types registration as I mentioned in my question, i.e.:
query.unwrap(org.hibernate.query.NativeQuery.class)
.addScalar("condition_set_ids", UUIDArrayType.INSTANCE);
So it appears to just have been a bug and I can now avoid the global registration in favor of using scalars.
Using the JOOQ parser API, I'm able to parse the following query and get the parameters map from the resulting Query object. From this, I can tell that there is one parameter, and it's name is "something".
However, I haven't been able to figure out how to determine that the parameter "something" is assigned to a column named "BAZ" and that column is part of the table "BAR".
Does the parser API have a way to get the table/column metadata associated to each parameter?
String sql = "SELECT A.FOO FROM BAR A WHERE A.BAZ = :something";
DSLContext context = DSL.using...
Parser parser = context.parser();
Query query = parser.parseQuery(sql);
Map<String, Param<?>> params = query.getParams();
Starting from jOOQ 3.16
jOOQ 3.16 introduced a new, experimental (as of 3.16) query object model API, which can be traversed, see:
The manual
A blog post about traversing jOOQ expression trees
Specifically, you can write:
List<QueryPart> parts = query.$traverse(
Traversers.findingAll(q -> q instanceof Param)
);
Or, to conveniently produce exactly the type you wanted:
Map<String, Param<?>> params = query.$traverse(Traversers.collecting(
Collectors.filtering(q -> q instanceof Param,
Collectors.toMap(
q -> ((Param<?>) q).getParamName(),
q -> (Param<?>) q
)
)
));
The Collectors.toMap() call could include a mergeFunction, in case you have the same param name twice.
Pre jOOQ 3.16
As of jOOQ 3.11, the SPI that can be used to access the internal expression tree is the VisitListener SPI, which you have to attach to your context.configuration() prior to parsing. It will then be invoked whenever you traverse that expression tree, e.g. on your query.getParams() call.
However, there's quite a bit of manual plumbing that needs to be done. For example, the VisitListener will only see A.BAZ as a column reference without knowing directly that A is the renamed table BAR. You will have to keep track of such renaming yourself when you visit the BAR A expression.
i have a nasty SQl that i want transform in JOOQ
Here are the Query:
SELECT
SUM(dpr.dpr_bruttopraemie_prt + dpr.dpr_sofortrabatt_prt)
, MAX(TO_NUMBER(DP1.dp_wert))
FROM deckungen deck, deckungspraemien dpr,
(SELECT dp.dp_id, dp.dp_wert
FROM textbausteine txb, druckparameter dp
WHERE dp.dp_txb_id = txb.txb_id
) DP1
WHERE DP1.dp_id = :druckparameter_id;
As you can see, i need to make alias from a select with two Fields.
dp.dp_id, dp.dp_wert
that im going to used it on some other parts.
How i can i get it done?
i've seen
.asField()
Funktion but it only make alias for one column.
PS: The actual Query are a lot more complicated. So i wrote a simpler one.
With hoping that it's satisfied the SQL ORACLE Dialect.
I'm assuming that you're using the code generator, so you have generated objects available for your tables like DECKUNGEN. I'm also assuming you're using these static imports:
import static org.jooq.impl.DSL.*; // The jOOQ API
import static com.example.myapp.generated.Tables.*; // Your generated tables
You can then write:
Deckungen deck = DECKUNGEN.as("deck");
Deckungspraemien dpr = DECKUNGSPRAEMIEN.as("dpr");
Textbausteine txb = TEXTBAUSTEINE.as("txb");
Druckparameter dp = DRUCKPARAMETER.as("dp");
Table<?> dp1 = table(
select(dp.DP_ID, dp.DP_WERT)
.from(txb, dp)
.where(dp.DP_TXB_ID.eq(txb.TXB_ID))
).as("dp1");
Record2<BigDecimal, BigDecimal> result =
using(configuration)
.select(
sum(dpr.DPR_BRUTTOPRAEMIE_PRT.plus(dpr.DPR_SOFORTRABATT_PRT)),
max(field("to_number({0})", BigDecimal.class, dp1.field(dp.DP_WERT))))
.from(deck, dpr, dp1)
.where(dp1.field(dp.DP_ID).eq(druckparameterId))
.fetchOne();
Some explanations
jOOQ currently doesn't have a built-in TO_NUMBER() function, but you can easily roll your own using DSL.field(String) and similar overloads. For more detail, refer to the manual's section about plain SQL
A derived table can be created most easily by using the DSL.table(Select) operator.
Columns from a derived table can be dereferenced using Table.field() methods, in particular the Table.field(Field), which tries to find a field by the same name as the argument field, retaining the argument field's type information.
Side-note
I don't think your query is correct as you're simply creating cartesian products between your tables deck, dpr, and dp1. Specifically, the SUM() is quite likely to be wrong, whereas MAX() is simply calculated inefficiently.
Spring Data Cassandra 1.5.0 comes with a streaming API in CassandraTemplate. I'm using spring-data-cassandra 1.5.1. I have a code like:
String tableName = cassandraTemplate.getTableName(MyEntity.class).toCql();
Select select = QueryBuilder.select()
.all()
.from(tableName);
// In real world, WHERE statement is much more complex
select.where(eq(ENTITY_FIELD_NAME, expectedField))
List<MyEntity> result = cassandraTemplate.select(select, MyEntity.class);
and want to replace this code with iterable or Java 8 Stream in order to avoid fetching a big list of results to memory at once.
What I'm looking for is a method signature like CassandraOperations.stream(Select query, Class<T> entityClass), but it is not available.
The only available method in CassandraOperations accepts query string: stream(String query, Class<T> entityClass). I tried to pass here a string generated by Select like
cassandraTemplate.stream(select.getQueryString(), MyEntity.class)
But that fails with InvalidQueryException: Invalid amount of bind variables, because getQueryString() returns query with question mark placeholders instead of variables.
I see 3 options to get what I want, but every option looks bad:
Use Spring Query creation mechanism with Stream/Iterator expected return type (good only for simple queries) http://docs.spring.io/spring-data/cassandra/docs/current/reference/html/#repositories.query-methods.query-creation
Use raw CQL query and not to use QueryBuilder
Call select.getQueryString() and then substitute parameters again via BoundStatement
Is there any better way to stream selection results?
Thanks.
So, as of now the answer on my question is to wait until stable version of spring-data-cassandra 2.0.0 comes out:
https://github.com/spring-projects/spring-data-cassandra/blob/2.0.x/spring-data-cassandra/src/main/java/org/springframework/data/cassandra/core/CassandraTemplate.java#L208