Develop Reference

How can I project all database columns of a fast changing database?

I often have the situation that the generated jooq code doesn't match the database in production (columns get added all the time).
How can I fetch a weakly typed record, that contains all the database columns?
dsl.select(asterisk())
.from(PERSON)
.where(PERSON.PERSON_NO.eq(id))
.fetch()
Only returns the columns known at code generation.

A quick hack would be to make sure jOOQ doesn't know your tables by using plain SQL templating in your from clause. That way, jOOQ cannot resolve the asterisk and will try to discover the projection from the actual query results. For example:
dsl.select(asterisk())
.from("{0}", PERSON)
.where(PERSON.PERSON_NO.eq(id))
.fetch();
This has been a re-occurring request, I guess we can turn this into a feature: https://github.com/jOOQ/jOOQ/issues/10182
Note though, that it is usually better to make sure jOOQ knows the exact production schema and keep generated code up to date. A future jOOQ will support versioned generated meta data so that the same code can work with different production schema versions more easily:
https://github.com/jOOQ/jOOQ/issues/4232

Just use plain SQL: https://www.jooq.org/doc/3.14/manual-single-page/#query-vs-resultquery
If that won't work for you, explaining why not might help someone formulate a more suitable answer.

jOOQ using one database for generating code and another for executing it

We are changing databases, from one that supports an 8 bit int to one that does not. Our code breaks when Liquibase creates a DB that causes jOOQ to generate "short" variables but our code uses byte/Byte - this breaks code signatures.
Rather than recode, somebody suggested that we continue to use the previous database (HSQLDB) to generate the code and it "should" run with the new database. There are dissenting opinions, I cannot find anything definitive beyond intuition and it seems to be counter to what jOOQ was designed for. Has anyone done this successfully?

There is obviously no absolute yes/no answer to such a question, but there are several solutions/workarounds:
Use the previous database product to generate code
This will work for a short period of time, e.g. right now, but as you move on, it will be an extremely limiting factor for your schema design. You will continue tailoring your DDL and some other design decisions around what HSQLDB can do, and you won't be able to leverage other features of your new database product. This can be especially limiting when migrating data, as ALTER TABLE statements are quite different between dialects.
I would recommend this approach only for a very short period of time, e.g. if you can't thoroughly fix this right away.
Use jOOQ's <forcedType/> mechanism to rewrite your data types
jOOQ's code generator allows for rewriting data types prior to loading the meta data of your schema into the code generator. This way, you can pretend your byte types are TINYINT on your new database product, even if your new database product doesn't support TINYINT.
This is a thorough solution that you may want to implement regardless of what product you're using, as it will give you a way to re-define parts of your schema just for jOOQ's code generator, independently of how you're generating your code.
The feature is documented here:
https://www.jooq.org/doc/latest/manual/code-generation/codegen-advanced/codegen-config-database/codegen-database-forced-types
This is definitely a more long term solution for your case.
Notice, a future jOOQ will be able to use CHECK constraints as input meta data to decide whether to apply such a <forcedType/>. I would imagine that you will place a CHECK (my_smallint BETWEEN -128 AND 127) constraint on every such column, so you could easily recognise which columns to apply the <forcedType/> to: https://github.com/jOOQ/jOOQ/issues/8843
Until that feature is available, you can implement it yourself via programmatic code generator configuration:
https://www.jooq.org/doc/latest/manual/code-generation/codegen-programmatic/
Or, starting with jOOQ 3.12, by using a SQL expression to produce the regular expression that <forcedType/> matches. E.g. in Oracle:
<forcedType>
<name>TINYINT</name>
<sql>
select listagg(owner || '.' || table_name || '.'
|| regexp_replace(search_condition_vc, ' between.*', ''), '|')
from user_constraints
where constraint_type = 'C'
and regexp_like(search_condition_vc, '.* between -128 and 127');
</sql>
</forcedType>
You could use a file based meta data source
jOOQ doesn't have to connect to a live database instance to reverse engineer your schema. You can also pass DDL code to jOOQ, or XML files:
https://www.jooq.org/doc/latest/manual/code-generation/codegen-ddl/
https://www.jooq.org/doc/latest/manual/code-generation/codegen-xml/
This is not really solving your problem directly, but maybe, it might make solving it a bit easier. However, there are other limitations to these approaches, e.g. stored procedures aren't currently (jOOQ 3.12) supported, so I'm just adding this for completeness' sake here, not to suggest you use it right now.

Hibernate 6: What is SQM?

In the Hibernate 6.0 Roadmap (https://github.com/hibernate/hibernate-orm/wiki/Roadmap6.0) SQM is mentioned as upcoming.
What is SQM?
In this roadmap the following short words describe it:
SQM integration: Improved performance for SQL generation and execution (smaller SQL, position-based extraction of results rather than name(alias)-based); Unified approach for HQL, JPQL and Criteria queries.
This is all I've found about SQM. Could someone please explain it a little more in detail? What exactly is it, how will it look like when it comes to coding, which benefits will it have?

SQM stands for Semantic Query Model, and it is the new entity query parser that addresses both JPQL and Criteria API.
The new parser is much more flexible, and it provides better SQL translation of entity queries.
From a user perspective, SQM provides more features like Window Functions, CTE (Common Table Expressions), Lateral Joins, etc.
SQM provides better performance as well since Criteria API is parsed directly to SQL.

When Hibernate does not generate optimal query. How to make hibernate generate optimal query

While ORM frameworks like hibernate does simplify the activities of accessing and modifying data in RDBMS. And it's features like lazy loading, caching etc also help in better application performance when used properly.
But what I am thinking of is if the query generator or optimizer is not using the best query - say not following the proper ordering of tables during a join and not using the most suitable join strategy etc.
How can i optimize the query or say hibernate to generate a better query which is more optimal than the one it is generating by default.
(probably if I am not wrong, this question applies to other ORM frameworks like iBATIS etc also)
EDIT following answer from S. Bhattacharjee - I would also hope that Hibernate will generate the best optimal query. But on the other side even when using Oracle SQL executor in Oracle GUI, it's some time the case that the Optimizer DOES NOT use optimal plan. But I can influence it by writing the query more specifically. (don't ask me how it's done, if some one knows please indicate. I am no SQL expert. I need to analyse this case to ensure the systems flexibility and optimization's are not limited because of frameworks). So
CASE 1 - consider hibernate is not generating optimal query. So how to influence it to generate optimal query.
CASE 2 - hibernate is generating optimal query. But SQL engine is using a plan that is not optimal. Query and Plan (or execution) are different from my understanding. I am not sure if this is a valid case or not. So ignore if not applicable. But if anyone think this is a valid case even if they can't answer, plz leave a comment to bolster this point.
Next speaking about Hibernate query generator - another question pops in to my mind. Does Hibernate know about the Logical Structure of DB (I don't expect it to know the physical design anyway and may not be relevant to consider it here).
a) If it does know logical db design, how does it know about it.
b) if it does not know logical db design, then how can it be sure about the query being optimal.

Well, in my opinion hibernate generates a good and optimal query. Please be more clear on what grounds you are saying so, may be with an example? However if you want you may write your own SQL queries. Below is a very small example
Query query = session.createSQLQuery("select c.codename from
codes c where c.code = :paramcode")
.setParameter("paramcode", "900");
List result = query.list();

Java: JOOQ persistence framework performance and feed back [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
I've stumbled over a nice SQL builder framework, called JOOQ. BTW, in Russian JOOQ sounds like noun meaning "bug" (as an insect), "beetle" ;)
If you have any feedback about JOOQ, it's performance and such, please share. Links to blogs about JOOQ also welcome.

I think I should answer here also because I started using jooq one and a half months ago so I have some experience with it.
I wanted to use tool like jooq because:
ORM is an overkill in my current project (distributed calculations platform for cluster) since I need to read and write only separate fields from db, not complete table rows and some of my queries are complex enough not to be executed by simple and lightweight ORMs.
I wanted syntax autocomplete for my queries so that I don't need to keep my whole DB in mind
I wanted to be able to write queries directly in Java so that compiler could check basic query syntax on build.
I wanted my queries to be type-safe so that I couldn't accidentally pass a variable of one type, where another one is expected.
I wanted SQL, but I wanted it very convenient and easy to use
Well, with jooq I was able to achieve all that. My main requirement was for jooq to handle complex enough queries (nested, with grouping etc.). That was fulfilled.
I also wanted to be able to run queries using as few lines of code as possible and was able to reach this with jooq fluent API which allows jquery-like calls to perform SELECTs.
On my way using of jooq I reported a one or two bugs and I must say, that they were fixed surprisingly fast.
I also missed some features and again I must say, that I already have almost all of them.
What I liked very much, is that jooq now uses SLF4J for reporting some very interesting data about it's performance as well as for outputting the actual queries it has built. It really helped me with debugging.
Jooq even generate Java artifacts for stored procedures, UDFs and updatable recordsets, which I don't use currently, though.
What's important, jooq transparently supports DB2, Derby, H2, HSQLDB, MySQL, Oracle, PostGreSQL, SQLite, SQL Server, Sybase SQL Anywhere. Pretty extensive list, I think.
Jooq has support forum in Google groups where Lukas is day and night ready to answer even the stupidest of my questions.
Jooq supports Maven and that's a great relief for me since all my Java projects are Maven-based. We still miss Maven plugin for generator, but that's not important since running generator is a piece of cake.
Writting my queries with jooq I suddenly discovered, that they became really portable because I almost never used any MySQL-specific feature in the code since jooq tries to be as portable as possible. For those who can't live with such peculiarities, as I know support for SQL extensions is under the way also.
What does jooq lack for a moment from my point of view?
Well, there is no fluent API for statements other than SELECT. This complicates code a little and makes UPDATE/DELETE statements a little more complicated to write. But I think this will be added soon. Just implemented in 1.5.9! Ha! Too quick for me ;)
And one more thing. Jooq has a good manual, but... I don't know. May be I just don't understand it's structure or architecture... When I started using jooq for the first time, I opened one page after another looking for a feature I need. For example, try to guess, where in jooq manual UPDATE and DELETE statements are described, looking at contents... But that's really subjective, I believe. I also cannot even explain, what's wrong with manual from my point of view. When I can, I will post a ticket or two ;)
Manual also is not really well-navigable since Trac has no automatic "here, there and back"-like links.
Well, for me in Moscow (Russia) Trac pages doesn't open fast also so reading manual is a little boring.
Manual also misses a good architecture description of jooq for contributors. Jooq follows design-by-contract principle it seems and when I wanted to learn how certain feature is implemented inside by using my usual Ctrl-Click on some method name in IDE, I ended up inside a dull interface with no implementation ;) Not that I'm too smart to start improving jooq right away, but certainly I would enjoy understanding how exactly jooq is architectured from ground to up.
It's a pity also that we cannot contribute to jooq manual. I expected it to be in some kind of wiki.
What I would also want to improve, is the way news are reported. I would prefer link to manual there or examples how this or that new feature works.
Release notes link in manual is really just a roadmap. I think, I will do that one myself tomorrow...
Jooq also have relatively small community currently, but I am glad to report that it doesn't affect code quality or the way new features are introduced.
Jooq is really a good project. I will stick to it for my future projects as well. I really like it.

You can also take a look on MentaBean, a lightweight ORM and SQL Builder that lets you be as close as possible to SQL offering a lot of help with the boilerplate code. Here is an example:
Programmatic Configuration:
private BeanConfig getUserBeanConfig() {
// programmatic configuration for the bean... (no annotation or XML)
BeanConfig config = new BeanConfig(User.class, "Users");
config.pk("id", DBTypes.AUTOINCREMENT);
config.field("username", DBTypes.STRING);
config.field("birthdate", "bd", DBTypes.DATE); // note that the database column name is different
config.field("status", new EnumValueType(User.Status.class));
config.field("deleted", DBTypes.BOOLEANINT);
config.field("insertTime", "insert_time", DBTypes.TIMESTAMP).defaultToNow("insertTime");
return config;
}
// create table Users(id integer primary key auto_increment,
// username varchar(25), bd datetime, status varchar(20),
// deleted tinyint, insert_time timestamp)
A simple SQL join query:
Post p = new Post(1);
StringBuilder query = new StringBuilder(256);
query.append("select ");
query.append(session.buildSelect(Post.class, "p"));
query.append(", ");
query.append(session.buildSelect(User.class, "u"));
query.append(" from Posts p join Users u on p.user_id = u.id");
query.append(" where p.id = ?");
stmt = conn.prepareStatement(query.toString());
stmt.setInt(1, p.getId());
rset = stmt.executeQuery();
if (rset.next()) {
session.populateBean(rset, p, "p");
u = new User();
session.populateBean(rset, u, "u");
p.setUser(u);
}

If you are looking for only a SQL builder solution. I have one project which is an ORM framework for Java but it is still premature and in continuous development however handles many primitive usages of databases. https://github.com/ahmetalpbalkan/orman
There is no documentation in this stage however it can build safe queries using only Java chain methods and can handle many SQL operations. It can also map classes-fields to tables-columns respectively.
Here's a sample query building operation for query
SELECT COUNT(*) FROM sailors WHERE
rating>4 AND rating<9 GROUP BY rating HAVING AVG(age)>20;
Java code:
QueryBuilder qb = QueryBuilder.getBuilder(QueryType.SELECT);
System.out.println(qb
.from("sailors")
.where(
C.and(
C.gt("rating", 5),
C.lt("rating", 9)))
.groupBy("rating")
.having(
C.gt(
new OperationalField(QueryFieldOperation.AVG,
"age").toString(), 20)
).getQuery());
(LOL just give up developing that framework!)
Most probably that won't work for you but just wanted to announce my project :P