Develop Reference

How to extract used table from SelectConditionStep<Record>

I'm extending on my last question I asked about jOOQ. In the Hibernate models the #Filter annotation gets used, and I want to apply this same 'default filter' to the jOOQ queries. As I'm passing a jOOQ query to the nativeQuery(org.jooq.Query query, Class<E> type) I was wondering if it's possible to extract the table (TableImpl<?,?>) used from the FROM clause in the jOOQ query (org.jooq.Query).
This is what I've tried:
private static <E> SelectConditionStep<Record> applyDefaultFilters(Class<E> type, SelectConditionStep<Record> query)
{
if (BaseOrganizationModel.class.isAssignableFrom(type)) {
query
.getQuery()
.addConditions(
query
.getQuery()
.asTable()
.field("organization_id", Long.class)
.eq(currentOrganization().id));
if (SoftDeletableModel.class.isAssignableFrom(type)) {
query
.getQuery()
.addConditions(query.getQuery().asTable().field("deleted", Boolean.class).eq(false));
}
}
return query;
}
The result is this SQL, which is not what I want. I want it to filter the corresponding table.
select distinct `EventGroup`.*
from `EventGroup`
where (
...
and `alias_100341773`.`organization_id` = ?
and `alias_17045196`.`deleted` = ?
)
I want this
select distinct `EventGroup`.*
from `EventGroup`
where (
...
and `EventGroup`.`organization_id` = ?
and `EventGroup`.`deleted` = ?
)
Is this possible at all? And if not, what possible other routes are there? (aside from the obvious passing the table to the function)

Using jOOQ 3.16 query object model API
jOOQ 3.16 introduced a new, experimental (as of 3.16) query object model API, which can be traversed.
On any Select, just call Select.$from() to access an unmodifiable view of the contained table list.
An alternative, dynamic SQL approach for the ad-hoc case
Every time you're trying to mutate an existing query, ask yourself, is there a more elegant way to do this using a more functional, immutable approach do dynamic SQL? Rather than appending your additional predicates to the query, why not produce predicates from a function?
private static Condition defaultFilters(Class<?> type, Table<?> table) {
Condition result = noCondition();
if (BaseOrganizationModel.class.isAssignableFrom(type)) {
result = result.and(table.field("organization_id", Long.class)
.eq(currentOrganization().id));
if (SoftDeletableModel.class.isAssignableFrom(type))
result = result.and(not(table.field("deleted", Boolean.class)))
}
return result;
}
And now, when you construct your query, you can add the filters:
ctx.select(T.A, T.B)
.from(T)
.where(T.X.eq(1))
.and(defaultFilters(myType, T))
.fetch();
A generic way to transform your SQL
If you really want to mutate your query (e.g. in a utility for all queries), then a transformation approach might be better suited. There are different ways to approach this.
Using views
Some RDBMS can access session variables in views. In Oracle, you'd be setting some SYS_CONTEXT variable to your organization_id inside of a view, and then query only the (possibly updatable) views instead of the tables directly. MySQL unfortunately can't do the equivalent thing, see Is there any equivalent to ORACLE SYS_CONTEXT('USERENV', 'OS_USER') in MYSQL?
I've described this approach here in this blog post. The advantage of this approach is that you will never forget to set the predicate (you can validate your view source code with CI/CD tests), and if you ever forget to set the session context variable, the view will just not return any data, so it's quite a secure approach.
Together with the WITH CHECK OPTION clause, you can even prevent insertions into wrong organization_id, which improves security.
Using a VisitListener in jOOQ
This is the most powerful approach to do this in jOOQ, and exactly what you want, but also quite a tricky one to get right for all edge cases. See this post about implementing row level security in jOOQ. Starting from jOOQ 3.16, there will be better ways to transform your SQL via https://github.com/jOOQ/jOOQ/issues/12425.
Note, it won't work for plain SQL templates that do not use any jOOQ query parts, nor for JDBC based queries or other queries that you may have in your system, so be careful with this approach as you might leak data from other organisations.
Of course, you could implement this step also on the JDBC layer, using jOOQ's ParsingConnection or ParsingDataSource, that way you can intercept also third party SQL and append your predicates.
This can work for all DML statements, including UPDATE, DELETE. It's a bit harder for INSERT, as you'd have to transform INSERT .. VALUES into INSERT .. SELECT, or throw an exception if someone wants to insert into the wrong organization_id.
Using a ExecuteListener in jOOQ
A bit more hackish than the above VisitListener approach, but generally easier to get right, just regex-replace the WHERE clause of all your statements by WHERE organization_id = ... AND in an ExecuteListener.
To play it safe, you could reject all queries without a WHERE clause, or do some additional trickery to add the WHERE clause at the right place in case there isn't already one.
Using jOOQ's equivalent of Hibernate's #Filter
jOOQ's equivalent of Hibernate's #Filter is the Table.where(Condition) clause. It's not an exact equivalent, you'd have to prevent direct access to T in your code base and make sure users access T only via a method that replaces T by T.where(defaultFilters(myType, T)) instead.
This approach currently loses type safety of the T table, see: https://github.com/jOOQ/jOOQ/issues/8012

JOOQ SUM in Kotlin with custom types

currently I have a converter to work with value objects in JOOQ with Kotlin.
My database has a table called transactions with a field type decimal(10, 2) called billing _amount and on my Kotlin code I have a simple ValueObject from DDD to wrap it up, called BillingAmount defined as following
data class BillingAmount(val value: BigDecimal)
As for my jooq custom converter I have the following code:
public final TableField<TransactionsRecord, BillingAmount> BILLING_AMOUNT =
createField(
DSL.name("billing_amount"),
org.jooq.impl.SQLDataType.NUMERIC(10, 2).nullable(false),
this,
"",
org.jooq.Converter.ofNullable(
java.math.BigDecimal.class,
okano.dev.jooqtesting.BillingAmount.class,
t -> new okano.dev.jooqtesting.BillingAmount(t),
u -> u.getValue()
)
);
On my repository, I'm just trying to retrieve a sum a billing amounts, but jooq is complaining that BillingAmount doesn't extend a Number class. I know that is a Java generic validation that is preventing my code from working, but there's any way around, except by extending the Number class, for solving this problem? I thought that the converter should be enough, but for sure I'm wrong about this.
Here's the simple query I'm trying to achieve:
// jooq is an instance of DSLContext
return jooq.select(sum(TABLE.BILLING_AMOUNT))
.from(TABLE)
.fetchSingle()
Any thoughts on this question? Thanks in advance.

I assume that problems you're experiencing is just due to Java's type system.
If so, you can simply coerce the field to a different type for a query, like so (admittedly, muddying the query defining code):
BillingAmount sum = jooq
.select(sum(TABLE.BILLING_AMOUNT.coerce(BigDecimal.class)))
.from(TABLE)
.fetchSingle(TABLE.BILLING_AMOUNT);
It's not quite the same type of result as in original query, though, because it materializes BillingAmount directly, and not a Record<BillingAmount> (because after coercion the query return type would be Record<BigDecimal>.

As a workaround, you can always resort to using plain SQL templating in jOOQ
fun alternativeSum(field: Field<*>) = DSL.field("{0}", field.getDataType(), field)

jOOQ Subquery in Order By

I'm using a subquery in order by like this on MySQL 8 database:
select * from series
order by (select max(competition.competition_date) from competition
where competition.series_id = series.id) desc
But I didn't find a way to do that with jOOQ.
I tried the following query but this does not compile:
dsl
.selectFrom(SERIES)
.orderBy(dsl.select(DSL.max(COMPETITION.COMPETITION_DATE))
.from(COMPETITION).where(COMPETITION.SERIES_ID.eq(SERIES.ID)).desc())
.fetch()
Are subqueries not supported in order by?

Select<R> extends Field<R>
There's a pending feature request #4828 to let Select<R> extend Field<R>. This seems tempting because jOOQ already supports nested records to some extent for those dialects that support it.
But I have some doubts whether this is really a good idea in this case, because no database I'm aware of (i.e. where I tried this) supports scalar subqueries that project more than one column. It's possible to use such subqueries in row value expression predicates, e.g.
(a, b) IN (SELECT x, y FROM t)
But that's a different story, because it's limited to predicates, and not arbitrary column expressions. And it is already supported in jOOQ, via the various DSL.row() overloads, e.g.
row(A, B).in(select(T.X, T.Y).from(T))
Select<Record1<T>> extends Field<T>
This is definitely desireable, because a SELECT statement that projects only one column of type T really is a Field<T> in SQL, i.e. a scalar subquery. But letting Select<Record1<T>> extend Field<T> is not possible in Java. There is no way to express this using Java's generics. If we wanted to do this, we'd have to "overload" the Select type itself and create
Select1<T1> extends Select<Record1<T1>>
Select2<T1, T2> extends Select<Record2<T1, T2>>
etc.
In that case, Select1<T1> could be a special case, extending Field<T1>, and the other ones would not participate in such a type hierarchy. But in order to achieve this, we'd have to duplicate the entire Select DSL API per projection degree, i.e. copy it 22 times, which is probably not worth it. There are already 67 Select.*Step types in the jOOQ API, as of jOOQ 3.13. This makes it difficult to justify the enhancement even only for scalar subqueries, i.e. for Select1.
Using DSL.field(Select<Record1<T>>) and related API
You've already found the right answer. While Select<Record1<T>> cannot extend Field<T>, we can accept Select<? extends Record1<T>> in plenty of API, as an overload to the usual T|Field<T> overloads. This has been done occasionally, and might be done more thoroughly throughout the API: https://github.com/jOOQ/jOOQ/issues/7240.
It wouldn't help you, because you want to call .desc() on a column expression (the Select), rather than wrap pass it to a method, so we're back at Java's limitation mentioned before.
Kotlin and other languages
If you're using Kotlin or other languages that have some way of providing "extension functions", however, you could use this approach:
inline fun <T> Select<Record1<T>>.desc(): SortField<T> {
return DSL.field(this).desc();
}
jOOQ might provide these out of the box in the future: https://github.com/jOOQ/jOOQ/issues/6256

Turning the subquery into a Field works:
dsl.selectFrom(SERIES)
.orderBy(DSL.field(dsl.select(DSL.max(COMPETITION.COMPETITION_DATE)).from(COMPETITION)
.where(COMPETITION.SERIES_ID.eq(SERIES.ID))).desc())
.fetch()

jOOQ - join with nested subquery

Let's say I want to find out who wrote CLRS in a book db (tables BOOK, AUTHOR with a junction table BOOK_AUTHOR).
SelectConditionStep<Record1<String>> query = create
.select(AUTHOR.LASTNAME.as("AuthorName"))
.from(
(
BOOK.leftOuterJoin(BOOK_AUTHOR).on(BOOK.ID.eq(BOOK_AUTHOR.BOOKID))
).leftOuterJoin(AUTHOR).on(AUTHOR.ID.eq(BOOK_AUTHOR.AUTHORID))
)
.where(BOOK.TITLE.eq(CLRS_title))
;
A bit inefficient to match the entire table, just to select a single book. I now want to select that book prior to the match.
The jOOQ doc on this matter led me to believe that could look something like this:
Table<Record1<Integer>> clrs = create
.select(BOOK.ID.as("bookID"))
.from(BOOK)
.where(BOOK.TITLE.eq(CLRS_title))
.asTable()
;
SelectJoinStep<Record1<String>> query = create
.select(AUTHOR.LASTNAME.as("AuthorName"))
.from(
(
clrs.leftOuterJoin(BOOK_AUTHOR).on(clrs.field("bookID").eq(BOOK_AUTHOR.BOOKID))
).leftOuterJoin(AUTHOR).on(AUTHOR.ID.eq(BOOK_AUTHOR.AUTHORID))
)
;
However, that fails to compile because
Cannot resolve method 'eq(org.jooq.TableField<ch.cypherk.bookdb.public_.tables.records.BookAuthorRecord,java.lang.Integer>)'
in the join condition.
What's the correct way to write this join?

The problem you're having
You're dereferencing a column from your derived table using Table.field(String):
clrs.field("bookID")
The type you're getting back is Field<?>, with a wildcard. Like with any generic type, once you have a wild card, a lot of operations (but not all) will no longer be possible on that type. Take List<?>, for example. You can still call List<?>.get() to retrieve an Object, but not List<?>.add(? element). In Field<?>, you can no longer call eq(), unless you cast the argument to a raw type.
You can also coerce your field's <T> type to the type you already know, e.g. by using Table.field(String, DataType<T>)
clrs.field("bookID", BOOK.ID.getDataType())
Study your various options and you might discover the one(s) you might find most useful
A better solution to your query
You don't really need to
Assign your subquery to a local variable
Use a derived table for your problem
Often with jOOQ, if you're having issues with derived tables as above, ask yourself is there really not an easier query I could write instead?
What you really need here is a semi join. Write:
// Assuming this static import
import static org.jooq.impl.DSL.*;
ctx.select(AUTHOR.LASTNAME)
.from(AUTHOR)
.where(AUTHOR.ID.in(
select(BOOK_AUTHOR.AUTHORID)
.from(BOOK_AUTHOR)
.join(BOOK).on(BOOK.ID.eq(BOOK_AUTHOR.BOOKID))
.where(BOOK.TITLE.eq(clrsTitle))
)
.fetch();

What is fieldIndex about in JOOQ

I read JOOQ API documentation about following APIs, but still couldn't understand what is fieldIndex about.
fetchAny(int fieldIndex, java.lang.Class<? extends T> type)
For example, in the following code I already knows which column to select, why do we need filedIndex of 0? What does 0 mean?
String name = getDslContext().select(TESTB.STU_NAME)
.from(TESTB)
.where(TESTB.ID.eq(studentId))
.fetchAny(0, String.class);

jOOQ mostly operates on Record types. For instance, when you create the following query:
.select(TESTB.STU_NAME)
.from(TESTB)
.where(TESTB.ID.eq(studentId));
You're really operating on a ResultQuery<Record1<String>>. By calling fetchAny(0, String.class) on that type, you're telling jOOQ that you want to fetch any record, and from that record, you want to get only the value at index 0, converted to String.
This may feel like repeating the same information twice (using TESTB.STU_NAME, and fetching the column at index 0). This is because fetchign a single value is quite a special case in the jOOQ API. Unfortunately, the ResultQuery API doesn't really "know" that you're selecting only one column even if that type information is present via generics.
One alternative would be to use DSLContext.fetchValue(ResultQuery):
String name = getDslContext().fetchValue(
DSL.select(TESTB.STU_NAME)
.from(TESTB)
.where(TESTB.ID.eq(studentId))
);