I want to implement oracle row level security kind of feature in Java using JOOQ library
Here is an example JOOQ query code:
Result<Record> result = dslContext.select().from(Employee.EMPLOYEE).fetch();
The code above will generate SQL as below:
select [dbo].[Employee].Id,... from [dbo].[Employee]
I want to add a where clause to filter data specific to user security as below:
select [dbo].[Employee].Id,... from [dbo].[Employee] WHERE [dbo].[Employee].Security IN (1,2)
Explicit predicates
Unless I'm missing some nice SQL Server feature where rows / records contain a pseudo-column called .Security to implement row level security, you should be able to simply write
dslContext.select()
.from(EMPLOYEE)
.where(EMPLOYEE.SECURITY.in(1, 2))
.fetch();
For more info about jOOQ predicate building, see the manual here:
http://www.jooq.org/doc/latest/manual/sql-building/conditional-expressions
And in particular, the IN predicate:
http://www.jooq.org/doc/latest/manual/sql-building/conditional-expressions/in-predicate
General solution using jOOQ's ExecuteListener
Given your comments, you're looking for a general way to patch all SQL statements with an additional predicate, no matter what the particular programmer is typing.
You can do this with jOOQ, but beware of the fact that this will just help you enforcing the predicate, not guarantee it, if programmers bypass jOOQ. What you can do is set up an ExecuteListener, intercepting the renderStart() event in order to patch / replace the query being executed. Something along these lines:
#Override
public void renderStart(ExecuteContext ctx) {
if (ctx.query() instanceof Select) {
// Operate on jOOQ's internal query model
SelectQuery<?> select = null;
// Check if the query was constructed using the "model" API
if (ctx.query() instanceof SelectQuery) {
select = (SelectQuery<?>) ctx.query();
}
// Check if the query was constructed using the DSL API
else if (ctx.query() instanceof SelectFinalStep) {
select = ((SelectFinalStep<?>) ctx.query()).getQuery();
}
if (select != null) {
// Use a more appropriate predicate expression
// to form more generic predicates which work on all tables
select.addConditions(EMPLOYEE.SECURITY.in(1, 2));
}
}
}
Of course, there's room for improvement to the above. Feel free to discuss use-cases on the user group
General solution using jOOQ's VisitListener
If you're willing to go deep into jOOQ's internals, you an also try to implement a VisitListener and actually transform jOOQ's AST representation of your query. This is documented here:
http://www.jooq.org/doc/latest/manual/sql-building/queryparts/custom-sql-transformation/
General solution using views
While the above works, I would personally suggest you use views for this and hide the actual tables from developers. Example:
CREATE VIEW v_employee AS
SELECT a, b, c, ...
FROM t_employee
WHERE t_employee.security in (1, 2)
With appropriate grants, you can hide the tables from the developers, making sure they will only ever use the views with your desired predicate always in place
Related
Currently I'm using RQL (https://github.com/persvr/rql) as rest query language. Previously, Query DSL was used with the meta model for the SQL queries.
It is very easy to navigate over the metamodel to create a Predicate.
It is enough to use the root entity and was able to work your way through all the relationships and find out the Predicate
Example:
class Foo{
private Set<Bar> bars
}
class Bar{
private String name;
}
rql: filter=eq(bars.name, "test)
Query DSL is: QFoo.bars --> SetPath and then navigate to bars.name is a SimpleExpression then you can call SimpleExpression.eq returns the Predicate.
Is there also this possibility with jOOQ to create the condition dynamically?
I can't find way with the statically generated fields because they don't contain any relationships, do they?
What is the best way to create the condition using the root resource? Hints?
Or can I simply create a condition from the predicate?
As of jOOQ 3.17, to-many path expressions aren't yet available, see https://github.com/jOOQ/jOOQ/issues/13639
But you don't need them anyway, they're just syntax sugar for an IN or EXISTS predicate. E.g.
FOO.ID.in(select(BAR.FOO_ID).from(BAR).where(BAR.NAME.eq("test")))
It's a bit more verbose, but straightforward, I think? Note that every jOOQ query is a dynamic SQL query. You're not required to write the predicate like this. You can compose it from its individual parts completely dynamically.
I'm extending on my last question I asked about jOOQ. In the Hibernate models the #Filter annotation gets used, and I want to apply this same 'default filter' to the jOOQ queries. As I'm passing a jOOQ query to the nativeQuery(org.jooq.Query query, Class<E> type) I was wondering if it's possible to extract the table (TableImpl<?,?>) used from the FROM clause in the jOOQ query (org.jooq.Query).
This is what I've tried:
private static <E> SelectConditionStep<Record> applyDefaultFilters(Class<E> type, SelectConditionStep<Record> query)
{
if (BaseOrganizationModel.class.isAssignableFrom(type)) {
query
.getQuery()
.addConditions(
query
.getQuery()
.asTable()
.field("organization_id", Long.class)
.eq(currentOrganization().id));
if (SoftDeletableModel.class.isAssignableFrom(type)) {
query
.getQuery()
.addConditions(query.getQuery().asTable().field("deleted", Boolean.class).eq(false));
}
}
return query;
}
The result is this SQL, which is not what I want. I want it to filter the corresponding table.
select distinct `EventGroup`.*
from `EventGroup`
where (
...
and `alias_100341773`.`organization_id` = ?
and `alias_17045196`.`deleted` = ?
)
I want this
select distinct `EventGroup`.*
from `EventGroup`
where (
...
and `EventGroup`.`organization_id` = ?
and `EventGroup`.`deleted` = ?
)
Is this possible at all? And if not, what possible other routes are there? (aside from the obvious passing the table to the function)
Using jOOQ 3.16 query object model API
jOOQ 3.16 introduced a new, experimental (as of 3.16) query object model API, which can be traversed.
On any Select, just call Select.$from() to access an unmodifiable view of the contained table list.
An alternative, dynamic SQL approach for the ad-hoc case
Every time you're trying to mutate an existing query, ask yourself, is there a more elegant way to do this using a more functional, immutable approach do dynamic SQL? Rather than appending your additional predicates to the query, why not produce predicates from a function?
private static Condition defaultFilters(Class<?> type, Table<?> table) {
Condition result = noCondition();
if (BaseOrganizationModel.class.isAssignableFrom(type)) {
result = result.and(table.field("organization_id", Long.class)
.eq(currentOrganization().id));
if (SoftDeletableModel.class.isAssignableFrom(type))
result = result.and(not(table.field("deleted", Boolean.class)))
}
return result;
}
And now, when you construct your query, you can add the filters:
ctx.select(T.A, T.B)
.from(T)
.where(T.X.eq(1))
.and(defaultFilters(myType, T))
.fetch();
A generic way to transform your SQL
If you really want to mutate your query (e.g. in a utility for all queries), then a transformation approach might be better suited. There are different ways to approach this.
Using views
Some RDBMS can access session variables in views. In Oracle, you'd be setting some SYS_CONTEXT variable to your organization_id inside of a view, and then query only the (possibly updatable) views instead of the tables directly. MySQL unfortunately can't do the equivalent thing, see Is there any equivalent to ORACLE SYS_CONTEXT('USERENV', 'OS_USER') in MYSQL?
I've described this approach here in this blog post. The advantage of this approach is that you will never forget to set the predicate (you can validate your view source code with CI/CD tests), and if you ever forget to set the session context variable, the view will just not return any data, so it's quite a secure approach.
Together with the WITH CHECK OPTION clause, you can even prevent insertions into wrong organization_id, which improves security.
Using a VisitListener in jOOQ
This is the most powerful approach to do this in jOOQ, and exactly what you want, but also quite a tricky one to get right for all edge cases. See this post about implementing row level security in jOOQ. Starting from jOOQ 3.16, there will be better ways to transform your SQL via https://github.com/jOOQ/jOOQ/issues/12425.
Note, it won't work for plain SQL templates that do not use any jOOQ query parts, nor for JDBC based queries or other queries that you may have in your system, so be careful with this approach as you might leak data from other organisations.
Of course, you could implement this step also on the JDBC layer, using jOOQ's ParsingConnection or ParsingDataSource, that way you can intercept also third party SQL and append your predicates.
This can work for all DML statements, including UPDATE, DELETE. It's a bit harder for INSERT, as you'd have to transform INSERT .. VALUES into INSERT .. SELECT, or throw an exception if someone wants to insert into the wrong organization_id.
Using a ExecuteListener in jOOQ
A bit more hackish than the above VisitListener approach, but generally easier to get right, just regex-replace the WHERE clause of all your statements by WHERE organization_id = ... AND in an ExecuteListener.
To play it safe, you could reject all queries without a WHERE clause, or do some additional trickery to add the WHERE clause at the right place in case there isn't already one.
Using jOOQ's equivalent of Hibernate's #Filter
jOOQ's equivalent of Hibernate's #Filter is the Table.where(Condition) clause. It's not an exact equivalent, you'd have to prevent direct access to T in your code base and make sure users access T only via a method that replaces T by T.where(defaultFilters(myType, T)) instead.
This approach currently loses type safety of the T table, see: https://github.com/jOOQ/jOOQ/issues/8012
I'm using a subquery in order by like this on MySQL 8 database:
select * from series
order by (select max(competition.competition_date) from competition
where competition.series_id = series.id) desc
But I didn't find a way to do that with jOOQ.
I tried the following query but this does not compile:
dsl
.selectFrom(SERIES)
.orderBy(dsl.select(DSL.max(COMPETITION.COMPETITION_DATE))
.from(COMPETITION).where(COMPETITION.SERIES_ID.eq(SERIES.ID)).desc())
.fetch()
Are subqueries not supported in order by?
Select<R> extends Field<R>
There's a pending feature request #4828 to let Select<R> extend Field<R>. This seems tempting because jOOQ already supports nested records to some extent for those dialects that support it.
But I have some doubts whether this is really a good idea in this case, because no database I'm aware of (i.e. where I tried this) supports scalar subqueries that project more than one column. It's possible to use such subqueries in row value expression predicates, e.g.
(a, b) IN (SELECT x, y FROM t)
But that's a different story, because it's limited to predicates, and not arbitrary column expressions. And it is already supported in jOOQ, via the various DSL.row() overloads, e.g.
row(A, B).in(select(T.X, T.Y).from(T))
Select<Record1<T>> extends Field<T>
This is definitely desireable, because a SELECT statement that projects only one column of type T really is a Field<T> in SQL, i.e. a scalar subquery. But letting Select<Record1<T>> extend Field<T> is not possible in Java. There is no way to express this using Java's generics. If we wanted to do this, we'd have to "overload" the Select type itself and create
Select1<T1> extends Select<Record1<T1>>
Select2<T1, T2> extends Select<Record2<T1, T2>>
etc.
In that case, Select1<T1> could be a special case, extending Field<T1>, and the other ones would not participate in such a type hierarchy. But in order to achieve this, we'd have to duplicate the entire Select DSL API per projection degree, i.e. copy it 22 times, which is probably not worth it. There are already 67 Select.*Step types in the jOOQ API, as of jOOQ 3.13. This makes it difficult to justify the enhancement even only for scalar subqueries, i.e. for Select1.
Using DSL.field(Select<Record1<T>>) and related API
You've already found the right answer. While Select<Record1<T>> cannot extend Field<T>, we can accept Select<? extends Record1<T>> in plenty of API, as an overload to the usual T|Field<T> overloads. This has been done occasionally, and might be done more thoroughly throughout the API: https://github.com/jOOQ/jOOQ/issues/7240.
It wouldn't help you, because you want to call .desc() on a column expression (the Select), rather than wrap pass it to a method, so we're back at Java's limitation mentioned before.
Kotlin and other languages
If you're using Kotlin or other languages that have some way of providing "extension functions", however, you could use this approach:
inline fun <T> Select<Record1<T>>.desc(): SortField<T> {
return DSL.field(this).desc();
}
jOOQ might provide these out of the box in the future: https://github.com/jOOQ/jOOQ/issues/6256
Turning the subquery into a Field works:
dsl.selectFrom(SERIES)
.orderBy(DSL.field(dsl.select(DSL.max(COMPETITION.COMPETITION_DATE)).from(COMPETITION)
.where(COMPETITION.SERIES_ID.eq(SERIES.ID))).desc())
.fetch()
I have some code that looks like this:
Record record = jooq
.selectCount()
.from(USERS)
.fetchOne();
Currently I'm doing the following to get the count:
Integer count = (Integer) record.getValue(0);
But it seems like there must be a better solution (that's type-safe...since that's the whole point of using jooq). Any suggestions?
Unfortunately, for this particular query, there aren't many "better" ways to typesafely get the count() value. What you could do, to add type-safety, is this:
Field<Integer> f = count();
Integer count = jooq.
.select(f) // Or selectCount(). Replaced it to illustrate the case
.from(USERS)
.fetchOne(f);
The problem is that most of the type information about the projection has been "lost" to the Java compiler, by the time the fetch() methods are "reached". There is no way that a ResultQuery.fetchXXX() method could recover it from the SELECT clause, and produce it to you.
On the jOOQ user group, some users have argued to move the projection into the fetch() methods, entirely, the way C#'s LINQ, or Scala's SLICK do it. This would greatly complicate the expression of more advanced SELECT statements. An more elaborate explanation is documented here.
With jOOQ 3.0, additional record-level typesafety has been introduced. In jOOQ 3.3, it will thus be possible to fetch a single value as such (has been registered as #2246):
<T> T fetchValue(Select<Record1<T>> select);
For instance if I do something like:
Criteria c = session.createCriteria(Book.class)
.add(Expression.ge("release",reDate);
.add(Expression.ge("price",price);
.addOrder( Order.asc("date") )
.setFirstResult(0)
.setMaxResults(10);
c.list();
How can I use the same criteria instance, but remove (for example) the second criterion?
I'm trying to build a dynamic query in which I'd like to let the user remove a filter, without the backend having to reconstruct the criteria from scratch.
Thank you
As far as I know, there is no way to remove things (restrictions, ordering, etc) from the criteria query, once you create it. I'm not knowledgeable enough about the internals of the Criteria API, but I know there is nothing in the exposed interface. You could try manipulating the objects that you are passing in to add or addOrder, but that sounds like more work than it is worth, especially when there are cleaner alternatives.
Criteria queries have certainly been one-shot uses in every application that I have seen.
Now, what you can do is store your restrictions, orderings and limits in a custom format (e.g., Collection), and then build your query quite easily from that stored format. This would probably make more sense to your user interface since you certainly need fine-grained control from there.
Not the answer you are looking for, I'm sure, but it is exactly what I have done in the past.
HTH
How can I use the same criteria instance, but remove (for example) the second criterion? I'm trying to build a dynamic query in which I'd like to let the user remove a filter, without the backend having to reconstruct the criteria from scratch.
You can't, you'll have to resend the whole (updated) set of parameters used to build the dynamic query.
You can remove criterions in this way:
public static void List<CriterionType> removeCriterions(Criteria criteria, Class<? extends Criterion> type) {
Iterator<CriterionEntry> criterionIterator = ((CriteriaImpl) criteria).iterateExpressionEntries();
while (criterionIterator.hasNext()) {
CriterionEntry criterionEntry = criterionIterator.next();
if (criterionEntry.getCriteria() == criteria) {
Criterion criterion = criterionEntry.getCriterion();
if (null == type || criterion.getClass() == type) {
criterionIterator.remove();
}
}
}
}