I often have the situation that the generated jooq code doesn't match the database in production (columns get added all the time).
How can I fetch a weakly typed record, that contains all the database columns?
dsl.select(asterisk())
.from(PERSON)
.where(PERSON.PERSON_NO.eq(id))
.fetch()
Only returns the columns known at code generation.
A quick hack would be to make sure jOOQ doesn't know your tables by using plain SQL templating in your from clause. That way, jOOQ cannot resolve the asterisk and will try to discover the projection from the actual query results. For example:
dsl.select(asterisk())
.from("{0}", PERSON)
.where(PERSON.PERSON_NO.eq(id))
.fetch();
This has been a re-occurring request, I guess we can turn this into a feature: https://github.com/jOOQ/jOOQ/issues/10182
Note though, that it is usually better to make sure jOOQ knows the exact production schema and keep generated code up to date. A future jOOQ will support versioned generated meta data so that the same code can work with different production schema versions more easily:
https://github.com/jOOQ/jOOQ/issues/4232
Just use plain SQL: https://www.jooq.org/doc/3.14/manual-single-page/#query-vs-resultquery
If that won't work for you, explaining why not might help someone formulate a more suitable answer.
Related
When storing a record with jooq it generates different SQL for h2 and MSSQL, which makes it impossible to use quickperfs #ExpectInsert/#ExpectSelect annotations in test, since it counts differently. The generated SQL looks for H2 like:
select "ID" from final table (insert into...
and for MSSQL:
declare #result table ([ID] bigint); insert into ...
so in the first case quickperf does not count an insert but an select. So, why is the generated SQL so different? What are my options to solve this? Any hints are much appreciated.
I used the jooq store() method and insertInto() with no difference in the generated SQL.
The expectation of jOOQ (or any ORM) producing the exact same SQL for different SQL dialects is wrong. The whole point of abstracting over SQL is, in parts, to be able to generate SQL that "does the job".
In this case, it seems you're running some sort of statement that tries to implement RETURNING behaviour, where generated keys are to be fetched from the RDBMS after insertion. Both H2 and SQL Server have means to do this:
H2 implements the standard SQL data change delta table (FINAL TABLE (...))
SQL Server has an OUTPUT clause, but it has some limitations, including not being able to produce trigger generated values, so jOOQ generates a statement batch which fetches results into a in-memory temporary table.
You can find more details about this topic in this blog post about the many ways of returning data from SQL DML statements, on the jOOQ blog, which explains what jOOQ does behind the scenes, and why.
Your question "What are my options to solve this?" is harder to answer, because I'm not convinced your motivation here is reasonable: That of expecting identical SQL for both RDBMS.
Perhaps a much better question is: Do you really need H2 (as a test database product)? Here's jOOQ's take on this.
Currently I am working on a jooq project where I need to perform schema validation of the columns.
Whats the best way to get Table schema using jooq using table name.
DSLContext.meta() is taking so much time to get schema .
Thanks in advance
By default, DSLContext.meta() queries your entire database with all the schemas and all the tables, even if you only consume parts of it.
You can use Meta.filterSchemas() (and possibly even Meta.filterTables()) to filter out content prior to querying it.
We are changing databases, from one that supports an 8 bit int to one that does not. Our code breaks when Liquibase creates a DB that causes jOOQ to generate "short" variables but our code uses byte/Byte - this breaks code signatures.
Rather than recode, somebody suggested that we continue to use the previous database (HSQLDB) to generate the code and it "should" run with the new database. There are dissenting opinions, I cannot find anything definitive beyond intuition and it seems to be counter to what jOOQ was designed for. Has anyone done this successfully?
There is obviously no absolute yes/no answer to such a question, but there are several solutions/workarounds:
Use the previous database product to generate code
This will work for a short period of time, e.g. right now, but as you move on, it will be an extremely limiting factor for your schema design. You will continue tailoring your DDL and some other design decisions around what HSQLDB can do, and you won't be able to leverage other features of your new database product. This can be especially limiting when migrating data, as ALTER TABLE statements are quite different between dialects.
I would recommend this approach only for a very short period of time, e.g. if you can't thoroughly fix this right away.
Use jOOQ's <forcedType/> mechanism to rewrite your data types
jOOQ's code generator allows for rewriting data types prior to loading the meta data of your schema into the code generator. This way, you can pretend your byte types are TINYINT on your new database product, even if your new database product doesn't support TINYINT.
This is a thorough solution that you may want to implement regardless of what product you're using, as it will give you a way to re-define parts of your schema just for jOOQ's code generator, independently of how you're generating your code.
The feature is documented here:
https://www.jooq.org/doc/latest/manual/code-generation/codegen-advanced/codegen-config-database/codegen-database-forced-types
This is definitely a more long term solution for your case.
Notice, a future jOOQ will be able to use CHECK constraints as input meta data to decide whether to apply such a <forcedType/>. I would imagine that you will place a CHECK (my_smallint BETWEEN -128 AND 127) constraint on every such column, so you could easily recognise which columns to apply the <forcedType/> to: https://github.com/jOOQ/jOOQ/issues/8843
Until that feature is available, you can implement it yourself via programmatic code generator configuration:
https://www.jooq.org/doc/latest/manual/code-generation/codegen-programmatic/
Or, starting with jOOQ 3.12, by using a SQL expression to produce the regular expression that <forcedType/> matches. E.g. in Oracle:
<forcedType>
<name>TINYINT</name>
<sql>
select listagg(owner || '.' || table_name || '.'
|| regexp_replace(search_condition_vc, ' between.*', ''), '|')
from user_constraints
where constraint_type = 'C'
and regexp_like(search_condition_vc, '.* between -128 and 127');
</sql>
</forcedType>
You could use a file based meta data source
jOOQ doesn't have to connect to a live database instance to reverse engineer your schema. You can also pass DDL code to jOOQ, or XML files:
https://www.jooq.org/doc/latest/manual/code-generation/codegen-ddl/
https://www.jooq.org/doc/latest/manual/code-generation/codegen-xml/
This is not really solving your problem directly, but maybe, it might make solving it a bit easier. However, there are other limitations to these approaches, e.g. stored procedures aren't currently (jOOQ 3.12) supported, so I'm just adding this for completeness' sake here, not to suggest you use it right now.
Good Afternoon,
We are bulding a web application and as part of it building a search functionality, have a design question on "Search Functionality"
The field names on the UI vs DB are different .i.e. a field on the UI called as "Number" the same is called Text10 in the DB. following are the two issues
How to generate a SQL as user gives the UI field names, we have a table in the DB where we r maintaining configuration(UI name to DB Name)?
User selects the columns which he wants to search, say for example there fields are selected "Number, Description, Price" and once the sql is generated, how to know what data corresponds to what column? Do we have to maintain an index capturing position or a bean?
what is the better way to gather the data based on the resultset?
Thanks
A solution that promotes commonality between UI and database column names would be nice but probably not feasible.
Some sort of mapping table that captures the following will work:
META-DB-TABLE-NAME
META-DB-COLUMN-NAME
META-UI-COLUMN-NAME
Personally I would prefer to keep this mapping meta-data as close to the database as possible.
User-defined meta data is nicely described here from an Oracle perspective:
http://docs.oracle.com/cd/B28359_01/appdev.111/b28369/xdb_repos_meta.htm
Do some research on this and keep us informed with what you find. Very interesting question!
In such a dynamic SQL scenario, query builders like jOOQ really shine. See for example the jOOQ manual section about dynamic SQL.
In your specific case, assuming you're using generated code in jOOQ (which isn't a must, but certainly recommended), you'll be maintaining some sort of lookup between UI fields and SQL fields, such as:
Map<UIField, Field<?>> lookup = ...
lookup.put(UI.NUMBER, TABLE.NUMBER);
lookup.put(UI.DESCRIPTION, TABLE.DESCRIPTION);
lookup.put(UI.PRICE, TABLE.PRICE);
You can then construct your query dynamically according to user needs:
List<UIField> userRequestedFields = ...
List<Field<?>> queryFields = userRequestedFields
.stream()
.map(lookup::get)
.toList();
And then:
ctx.select(queryFields)
.from(TABLE)
.where(...)
.fetch();
There are other query builders, even JPA has the criteria API for these purposes. You could also roll your own, though you'll be re-inventing a lot of wheels.
Disclaimer: I work for the company behind jOOQ.
I've been using H2 on the functional tests part of a MySQL based application with Hibernate. I was finally fed up with it and I decided to usq jOOQ mostly so I could still abstract myself from the underlying database.
My problem is that I don't like this code generation thing jOOQ does at all since I'm yet to see an example with it properly set up in multiple profiles, also don't like connecting to the database as part of my build. It's overall quite a nasty set-up I don't want to spend a morning doing to realise is very horrible and I don't want it in the project.
I'm using tableByName() and fieldByName() instead which I thought was a good solution, but I'm getting problems with H2 putting everything in uppercase.
If I do something like Query deleteInclusiveQuery = jooqContext.delete(tableByName("inclusive_test"))... I get table inclusive_test not found. Note this has nothing to do with the connection delay or closing configuration.
I tried changing the connection to use ;DATABASE_TO_UPPER=false but then I get field not found (I thought it would translate all schema).
I'm not sure if H2 is either unable to create non-upper cased schemas or I'm failing at that. If the former then I'd expect jOOQ to also upper case the table and field names in the query.
example output is:
delete from "inclusive_test" where "segment_id" in (select "id" from "segment" where "external_taxonomy_id" = 1)
which would be correct if H2 schema would have not been created like this, however the query I'm creating the schema with specifically puts it in lowercase, yet in the end it ends up being upper cased, which Hibernate seems to understand or solve, but not jOOQ
Anyway, I'm asking if there is a solution because I'm quite disappointed at the moment and I'm considering just dropping the tests where I can't use Hibernate.
Any solution that is not using the code generation feature is welcome.
My problem is that I don't like this code generation thing jOOQ does at all since I'm yet to see an example with it properly set up in multiple profiles, also don't like connecting to the database as part of my build. It's overall quite a nasty set-up I don't to spend a morning doing to realise is very horrible and I don't want it in the project.
You're missing out on a ton of awesome jOOQ features if you're going this way. See this very interesting discussion about the rationale of why having a DB-connection in the build isn't that bad:
https://groups.google.com/d/msg/jooq-user/kQO757qJPbE/UszW4aUODdQJ
In any case, don't get frustrated too quickly. There are a couple of reasons why things have been done the way they are. DSL.fieldByName() creates a case-sensitive column. If you provide a lower-case "inclusive_test" column, then jOOQ will render the name with quotes and in lower case, by default.
You have several options:
Consistently name your MySQL and H2 tables / columns, explicitly specifying the case. E.g. `inclusive_test` in MySQL and "inclusive_test" in H2.
Use jOOQ's Settings to override the rendering behaviour. As I said, by default, jOOQ renders everything with quotes. You can override this by specifying RenderNameStyle.AS_IS
Use DSL.field() instead of DSL.fieldByName() instead. It will allow you to keep full control of your SQL string.
By the way, I think we'll change the manual to suggest using DSL.field() instead of DSL.fieldByName() to new users. This whole case-sensitivity has been causing too many issues in the past. This will be done with Issue #3218