I need to parse sql statements and get column names and table names. I tried with sample code. I got table names but I am stucked to get column names for each table.
If you really really need to do this then you should look at using a proper parser toolkit like ANTLR. It is quite a steep learning curve but there are grammars around for SQL that others have already built.
Hand-cranking your own parser will lead you down a mess of bugs for anything but the most basic of queries.
What you need is an SQL parser.
You can try JSQLParser, or ZQL.
I have used both successfully for the same thing the you are trying to do.
You can try also sqlparser. This is commercial however and I have no experience using it.
If you are having ResultSet object then, you can read Column names and Table Names of fired Query.
by using ResultSetMetaData to get MetaData of your query (like table names and column names).
example
You will need to create a parser for your SQL grammar.
ANTLR allows to create such parsers but it can be quite difficult to create a grammar.
Here is a list of exisiting grammar for ANTLR http://www.antlr3.org/grammar/list.html
It contains grammar for mysql, oracle SQL that should work for you.
Checkout Foundation Parser http://foundationdb.github.io/sql-parser/
Also note JSQLParser is no longer on SourceForge but maintain at https://github.com/JSQLParser/JSqlParser/wiki
Related
I often have the situation that the generated jooq code doesn't match the database in production (columns get added all the time).
How can I fetch a weakly typed record, that contains all the database columns?
dsl.select(asterisk())
.from(PERSON)
.where(PERSON.PERSON_NO.eq(id))
.fetch()
Only returns the columns known at code generation.
A quick hack would be to make sure jOOQ doesn't know your tables by using plain SQL templating in your from clause. That way, jOOQ cannot resolve the asterisk and will try to discover the projection from the actual query results. For example:
dsl.select(asterisk())
.from("{0}", PERSON)
.where(PERSON.PERSON_NO.eq(id))
.fetch();
This has been a re-occurring request, I guess we can turn this into a feature: https://github.com/jOOQ/jOOQ/issues/10182
Note though, that it is usually better to make sure jOOQ knows the exact production schema and keep generated code up to date. A future jOOQ will support versioned generated meta data so that the same code can work with different production schema versions more easily:
https://github.com/jOOQ/jOOQ/issues/4232
Just use plain SQL: https://www.jooq.org/doc/3.14/manual-single-page/#query-vs-resultquery
If that won't work for you, explaining why not might help someone formulate a more suitable answer.
I'm trying to parse a SELECT statement in Java. I'm familiar with JOOQ, and was hoping to use that. I know it's not explicitly designed as an SQL parser—it's actually a lot more than that, so I was thinking there might be a way to use its internal parsers to parse SELECT queries.
I saw some information on how to access some of JOOQ's internals using the Visitor pattern, but I need to navigate inside the query using a tree-like structure that will allow access to each part of the query individually. I don't want to use the Visitor pattern for all use cases.
Is this possible? How would I go about doing it?
Yes, you can. jOOQ has a parser that can be used:
Programmatically
As a CLI
Online, as a SQL dialect translator
As of jOOQ 3.17, there's an experimental model API which can be used to traverse your expression tree externally, e.g. using pattern matching, or internally using the new Traverser API. It is also still possible to traverse the expression tree using a VisitListener when rendering the expression tree back to SQL.
A full-fledged SQL parser is available from DSLContext.parser() and from DSLContext.parsingConnection() (see the manual's section about parsing connections for the latter).
The SQL Parsing API page gives this trivial example:
ResultQuery<?> query =
DSL.using(configuration)
.parser()
.parseResultQuery("SELECT * FROM (VALUES (1, 'a'), (2, 'b')) t(a, b)");
parseResultQuery is the method you need for a single SELECT query, use parse(String) if you may have multiple queries.
I have a use case where in I need to read rows from a file, transform them using an engine and then write the output to a database (that can be configured).
While I could write a query builder of my own, I was interested in knowing if there's already an available solution (library).
I searched online and could find jOOQ library but it looks like it is type-safe and has a code-gen tool so is probably suited for static database schema's. In the use case that I have db's can be configured dynamically and the meta-data is programatically read and made available for write-purposes (so a list of tables would be made available, user can select the columns to write and the insert script for these column needs to be dynamically created).
Is there any library that could help me with the use case?
If I understand correctly you need to query the database structure, display the result to via a GUI and have the user map data from a file to that structure?
Assuming this is the case, you're not looking for a 'library', you're looking for an ETL tool.
Alternatively, if you're set on writing something yourself, the (very) basic way to do this is:
the structure of a database using Connection.getMetaData(). The exact usage can vary between drivers so you'll need to create an abstraction layer that meets your needs - I'd assume you're just interested in the table structure here.
the format of the file needs to be mapped to a similar structure to the tables.
provide a GUI that allows the user to connect elements from the file to columns in the table including any type mapping that is needed.
create a parametrized insert statement based on file element to column mapping - this is just a simple bit of string concatenation.
loop throw the rows in the file performing a batch insert for each.
My advice, get an ETL tool, this sounds like a simple problem, but it's full of idiosyncrasies - getting even an 80% solution will be tough and time consuming.
jOOQ (the library you referenced in your question) can be used without code generation as indicated in the jOOQ manual:
http://www.jooq.org/doc/latest/manual/getting-started/use-cases/jooq-as-a-standalone-sql-builder
http://www.jooq.org/doc/latest/manual/sql-building/plain-sql
When searching through the user group, you'll find other users leveraging jOOQ in the way you intend
The setps you need to do is:
read the rows
build each row into an object
transform the above object to target object
insert the target object into the db
Among the above 4 steps, the only thing you need to do is step 3.
And for the above purpose, you can use Transmorph, EZMorph, Commons-BeanUtils, Dozer, etc.
I have an issue.
I have SQL that I need to append different type of "restrictions" or even do a join. This depends on user's search criteria.
This SQL will involve different table as it can search one-to-many relationship, therefore hibernate ORM can't support my requirement.
May I know if there is a design pattern to help construct such SQL statements?
The design pattern that fits to the problem of representing a language statement is the Interpreter pattern. But before you start to code your SQL parser, take a look to ANTLR.
And what is more important, ask yourself two questions:
Are the number of different SQL's justify the effort of develop a general SQL interpreter solutions instead of programming (just if-else statements) my 5-10 different queries?
Have I reviewed in detail the Hibernate reference manual?
I exactly have a similar requirement where I have a context-free language to define the search criteria, parsed to ParseEntry objects in a ParseTree which are analogous to the Restrictions. I use a SQLQueryGeneratorVisitor to visit the parse table and generate the SQL query, similary a HibernateCriteriaGeneratorVisitor if the criteria needs to be generated for a single entity. So, I essentially used the Visitor pattern making the parse tree and the entries visitable so that different types of criteria can be generated (SQL/Hibernate or something else in future).
I want to get the index of main where clause of database query in Java ?
How can I handle this with Regex?
For example in this query I want to get Second where clause:
select u.id, (select x.id from XEntity where x.id = 200)
from UserEntity u
**where** u.id in (select g.id from AnotherEntity g where g.id = 100)
I think the main where is which that number of "(" characters and ")" characters is equal after it.
but I don't know how can I get this with regex.
With Best Regards
What Toote and David Brabant said is absolutely correct. Parsing SQL, especially complex SQL using only regex is a very hard problem.
In terms of parsing SQL in Java, which seems to be the thrust of your question, there's a very good (if apparently un-maintained) library called JSQLParser. A more up-to-date version of this library can be found on Github here (disclaimer: I made a very small contribution to this). The main page shows an example of visitor designed to consume the output of the AST here.
There is also a grammar for ANTLR available in it's grammar list. Or, if you're feeling adventurous, the H2 database supports a rather extensive range of SQL, including some proprietary features of, e.g., MySQL. You could modify it's Parser to generate an appropriate structure for extracting the information you need.
Regular Expressions are not very good at recognizing such complex structures as SQL queries can be. Mainly because SQL is not context-free, which is exactly the issue you are running into: the WHERE can appear in a lot of places and you want one in particular that depends on the overall structure of the query.
What you will need is an appropriate parser. The only JavaScript SQL parser I could find is not too complete, but you can always help develop it by making sure it fits your needs.