Develop Reference

Is it possible to alter the SELECT/WHERE of jOOQ DSL query

I would like manipulate a jOOQ DSL query changing its SELECT columns and WHERE conditions.
For example:
DSLContext ctx = ...;
SelectHavingStep query = ctx.select(MyEntity.MY_ENTITY.ZIP, DSL.count(MyEntity.MY_ENTITY.ZIP))
.from(MyEntity.MY_ENTITY)
.where(MyEntity.MY_ENTITY.ID.gt("P"))
.groupBy(MyEntity.MY_ENTITY.ZIP);
Use case 1:
I would like to pass the above query to a utility class that will produce the same query just with with a different SELECT, for example:
ctx.select(DSL.count())
.from(MyEntity.MY_ENTITY)
.where(MyEntity.MY_ENTITY.ID.gt("P"))
.groupBy(MyEntity.MY_ENTITY.ZIP);
This particular example is to be able to create paginated results showing the total number of rows of the query.
Use case 2:
I would like to pass the above query to a utility class that will produce the same query just with with a modified WHERE clause, for example:
SelectHavingStep query =
ctx.select(MyEntity.MY_ENTITY.ZIP, DSL.count(MyEntity.MY_ENTITY.ZIP))
.from(MyEntity.MY_ENTITY)
.where(
MyEntity.MY_ENTITY.ID.gt("P")
.and(MyEntity.MY_ENTITY.ZIP.in("100", "200", "300"))
)
.groupBy(MyEntity.MY_ENTITY.ZIP);
This particular example is to further restrict a query based on some criteria (i.e. data visibility based on the user doing the query).
Is this possible?
Currently I'm using helper classes to do this at query construction time in the application code. I would like to move the responsibility to a library so it can be enforced transparently to the app.
Thanks.

You shouldn't try to alter jOOQ objects, instead you should try to create them dynamically in a functional way. There are different ways to achieve your use-cases, e.g.
Use case 1:
An approach to generic paginated querying can be seen here: https://blog.jooq.org/calculating-pagination-metadata-without-extra-roundtrips-in-sql/
Ideally, you would avoid the extra round trip for the COUNT(*) query and use a COUNT(*) OVER () window function. If that's not available in your SQL dialect, then you could do this, instead:
public ResultQuery<Record> mySelect(
boolean count,
Supplier<List<Field<?>>> select,
Function<? super SelectFromStep<Record>, ? extends ResultQuery<Record>> f
) {
return f.apply(count ? ctx.select(count()) : ctx.select(select.get()));
}
And then use it like this:
mySelect(false,
() -> List.of(MY_ENTITY.ZIP),
q -> q.from(MY_ENTITY)
.where(MY_ENTITY.ID.gt("P"))
.groupBy(MY_ENTITY.ZIP)
).fetch();
This is just one way to do it. There are many others, see the below link.
Use case 2:
Just take the above example one step further and extract the logic used to create the WHERE clause in yet another function, e.g.
public Condition myWhere(Function<? super Condition, ? extends Condition> f) {
return f.apply(MY_ENTITY.ID.gt("P"));
}
And now use it as follows:
mySelect(false,
() -> List.of(MY_ENTITY.ZIP),
q -> q.from(MY_ENTITY)
.where(myWhere(c -> c.and(MY_ENTITY.ZIP.in("100", "200", "300")))
.groupBy(MY_ENTITY.ZIP)
).fetch();
Again, there are many different ways to solve this, depending on what is the "common part", and what is the "user-defined part". You can also abstract over your MY_ENTITY table and pass around functions that produce the actual table.
More information
See also these resources:
https://www.jooq.org/doc/latest/manual/sql-building/dynamic-sql/
https://blog.jooq.org/a-functional-programming-approach-to-dynamic-sql-with-jooq/

with jooq is there a generic way to 'select by id'?

I'm pretty new to using jooq and I'm trying to implement the usual CRUD operations that us Java guys like to have in our DAOs/repositories. I have the following code for selecting a record by id:
public class JooqRepository<ID, E extends BaseObject<ID>, T extends Table<R>, R extends Record> {
...
private final T table; // would be coming from constructor to concrete reference in the generated classes
...
protected Optional<E> findById(ID id) {
final TableField<R, ID> idField = (TableField<R, ID>) table.getIdentity().getField();
return dsl.fetchOptional(table, idField.eq(id)).map(toEntity()); // conversion method omitted here
}
...
}
My question is firstly would this approach work for all kinds of tables/records or only ones that use identity/auto-increment?
What if I use a DBMS that doesn't have this feature (e.g. Oracle)?
What if a table has a composite key?
And lastly: Is it even recommended to use jooq in that way or should we explicitly craft dedicated queries for every table?

While it is possible to use jOOQ as a Spring repository implementation, you could also just use jOOQ's out of the box DAO support, which works in a similar way. The main difference is that jOOQ DAOs are unopinionated auxiliary tools, that do not impose DDD as a modeling paradigm, they just simplify the most common CRUD operations on each of your tables.
You can subclass the generated DAOs in order to add more functionality, and inject them to your services like Spring's repositories.

Capture generated SQL from Hibernate

I have a requirement to capture changes to certain data. I am looking to capture the following details:
Table Name the change occurred in
Column Changed
Previous Value
Updated Value
I suggested to our Technical lead that this can be accomplished easily with DB Triggers. I was told we do not control the DB and do not have a way of adding Triggers.
I am currently using Spring AspectJ with a custom annotation and wrapping my services to try and capture the generated SQL (I figure parsing SQL is much easier that trying to capture with Objects) that's executed after the 'save' method is called, however I have not found a way to trap the generated SQL.
I tried p6Spy and was able to view the SQL and print it to the console, but was told we cannot wrap our db drivers in our PROD environment.
Is there a Spring class I am missing to make this easier?
EDIT : We're using Spring Repositories to save the data.
EDIT 2: I'm looking into EventListeners, however I cannot seem to get them to listen to my events.
#Component
public class EventListner implements PreInserEventListener, PreUpdateEventListener {
#Override
#EventListener
public boolean onPreUpdate(PreUpdateEvent event){
// do something
return false;
}
#Override
#EventListener
public boolean onPreInsert(PreUpdateEvent event){
// do something
}
return false
}
I have break points around my Listener, however they're never reached.
This question looks like it might address my issue

If you are absolutely sure that every insert/update/delete is done through JPA without any bulk SQL then you should take a look at envers
With that you can use #Audited on the entities(all column) OR columns of entities you wish to keep history.
This will create 1 revision table with timestamp change (and other data you want like uid of the user that made the change), and 1 table per entity with the old value of the modified data.
For every data you can then retrieve the history and previous value.
one other way is to add instrument on your DB like a Change Data Capture (CDC) tools and push those event to another data repository.
On the + side everything will be detected (even native SQL run directly on the DB). Drawback, your DB need to support this kind of tools correctly. For instance tools like kafka-connect can work like you want, but some implementations are too simple (like with SAP hana for instance where the process is to do a select * from xxx).

I finally settled on using Hibernate interceptors to do the trick. Works perfectly!
public class TableInterceptor extends EmptyInterceptor {
#Override
public boolean onFlushDirty(Object entity, Serializable id,
Object[] currentState, Object[] previousState,
String[] propertyNames, Type[] types){
// do comparison logic on fields and save to DB
}
}
I bootstrapped the interceptor injecting it into my LocalContainerEntityManagerFactoryBean

Use Spring Data JPA, QueryDSL to update a bunch of records

I'm refactoring a code base to get rid of SQL statements and primitive access and modernize with Spring Data JPA (backed by hibernate). I do use QueryDSL in the project for other uses.
I have a scenario where the user can "mass update" a ton of records, and select some values that they want to update. In the old way, the code manually built the update statement with an IN statement for the where for the PK (which items to update), and also manually built the SET clauses (where the options in SET clauses can vary depending on what the user wants to update).
In looking at QueryDSL documentation, it shows that it supports what I want to do. http://www.querydsl.com/static/querydsl/4.1.2/reference/html_single/#d0e399
I tried looking for a way to do this with Spring Data JPA, and haven't had any luck. Is there a repostitory interface I'm missing, or another library that is required....or would I need to autowire a queryFactory into a custom repository implementation and very literally implement the code in the QueryDSL example?

You can either write a custom method or use #Query annotation.
For custom method;
public interface RecordRepository extends RecordRepositoryCustom,
CrudRepository<Record, Long>
{
}
public interface RecordRepositoryCustom {
// Custom method
void massUpdateRecords(long... ids);
}
public class RecordRepositoryImpl implements RecordRepositoryCustom {
#Override
public void massUpdateRecords(long... ids) {
//implement using em or querydsl
}
}
For #Query annotation;
public interface RecordRepository extends CrudRepository<Record, Long>
{
#Query("update records set someColumn=someValue where id in :ids")
void massUpdateRecords(#Param("ids") long... ids);
}
There is also #NamedQuery option if you want your model class to be reusable with custom methods;
#Entity
#NamedQuery(name = "Record.massUpdateRecords", query = "update records set someColumn=someValue where id in :ids")
#Table(name = "records")
public class Record {
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
private Long id;
//rest of the entity...
}
public interface RecordRepository extends CrudRepository<Record, Long>
{
//this will use the namedquery
void massUpdateRecords(#Param("ids") long... ids);
}
Check repositories.custom-implementations, jpa.query-methods.at-query and jpa.query-methods.named-queries at spring data reference document for more info.

This question is quite interesting for me because I was solving this very problem in my current project with the same technology stack mentioned in your question. Particularly we were interested in the second part of your question:
where the options in SET clauses can vary depending on what the user
wants to update
I do understand this is the answer you probably do not want to get but we did not find anything out there :( Spring data is quite cumbersome for update operations especially when it comes to their flexibility.
After I saw your question I tried to look up something new for spring and QueryDSL integration (you know, maybe something was released during past months) but nothing was released.
The only thing that brought me quite close is .flush in entity manager meaning you could follow the following scenario:
Get ids of entities you want to update
Retrieve all entities by these ids (first actual query to db)
Modify them in any way you want
Call entityManager.flush resulting N separate updates to database.
This approach results N+1 actual queries to database where N = number of ids needed to be updated. Moreover you are moving the data back and forth which is actually not good too.
I would advise to
autowire a queryFactory into a custom repository
implementation
Also, have a look into spring data and querydsl example. However you will find only lookup examples.
Hope my pessimistic answer helps :)

Cannot use QueryBuilder with column name = "`GROUP`" in ormlite

The code
QueryBuilder<MyClass, String> builder = mnDao.queryBuilder();
builder.where().eq("`GROUP`", someGroup));
throws exception
"Unknown column name '`GROUP`' in table MyClassTable"
I've tried to use escapeColumnName() from UpdateBuilder but the result is the same.
I could use raw queries, but they are not safe, because they don't provide placeholders.
The database is H2.
It looks like a bug in ormlite.
Any suggestions?

The right answer is to not do any quoting at all. ORMLite automatically quotes all of the field and table names. This is very database dependent for sure but H2 is my primary test database so I'm sure it works well in all cases that I know about for it. As to why it didn't work for you I can't comment on. If you post the query generated by ORMLite I may see the problem.
The pattern that is recommended is to define any columns that are to be used in queries as public strings. For example:
protected static class Reserved {
public static final String FIELD_NAME_GROUP = "group";
...
#DatabaseField(columnName = FIELD_NAME_GROUP)
String group;
...
}
Then when you query, you then use the FIELD_NAME_GROUP without any quoting and do:
QueryBuilder<Reserved, Integer> sb = dao.queryBuilder();
sb.where().eq(Reserved.FIELD_NAME_GROUP, "something");
...
I have good coverage of this in a couple of unit tests. See the following test that I run across all of my database types that are supported. Look for testCreateReserverdTable() and testCreateReserverdFields() methods and their associated classes in the JdbcBaseDaoImplTest.java unit test.

I don't know ormlite, and it ought to have a way of quoting a name in a way that allows you to use odd table names, but the easiest fix for this sort of thing is to avoid using sql keywords as table names or column names.
If you can change the table name to something sql doesn't use for something else, it'll probably just work.