I'm trying to parse a SELECT statement in Java. I'm familiar with JOOQ, and was hoping to use that. I know it's not explicitly designed as an SQL parser—it's actually a lot more than that, so I was thinking there might be a way to use its internal parsers to parse SELECT queries.
I saw some information on how to access some of JOOQ's internals using the Visitor pattern, but I need to navigate inside the query using a tree-like structure that will allow access to each part of the query individually. I don't want to use the Visitor pattern for all use cases.
Is this possible? How would I go about doing it?
Yes, you can. jOOQ has a parser that can be used:
Programmatically
As a CLI
Online, as a SQL dialect translator
As of jOOQ 3.17, there's an experimental model API which can be used to traverse your expression tree externally, e.g. using pattern matching, or internally using the new Traverser API. It is also still possible to traverse the expression tree using a VisitListener when rendering the expression tree back to SQL.
A full-fledged SQL parser is available from DSLContext.parser() and from DSLContext.parsingConnection() (see the manual's section about parsing connections for the latter).
The SQL Parsing API page gives this trivial example:
ResultQuery<?> query =
DSL.using(configuration)
.parser()
.parseResultQuery("SELECT * FROM (VALUES (1, 'a'), (2, 'b')) t(a, b)");
parseResultQuery is the method you need for a single SELECT query, use parse(String) if you may have multiple queries.
Related
Is there any interface to Lucene from redquery builder (https://github.com/salk31/RedQueryBuilder)? I have a Lucene indexed system and would like to support complex Lucene queries using its AND and NOT operators.
Alternatively, are there other visual query builders out there for Lucene that support "advanced search" functionality that include arbitrary numbers of AND/OR clauses?
As far as I know there isn't one that exists already.
I can think of two options:
* Use a SQL parser and serialiser to transform SQL into Lucence syntax (or query objects on the server)
* Use the 0.8.0 custom serialiser in RQB to produce Lucence query string. NB This is very experimental feature.
I should declare that I started the RQB project and that I love Lucence. Maybe worth raising a ticket for a feature? It could be a good way to test the serialisation if it produces Lucence queries rather than SQL?
NB If you want to store/re-load queries RQB is only designed to use SQL. It was chosen as a standard language to load/save even if never gets executed.
Is there easy way to convert SQL string like:
SELECT u.name, u.role
FROM users
WHERE user_id = ?
to any of SQL builders like JOOQ, QueryDsl just to get ability to modify query — add joins, additional "where" clauses, LIMIT-OFFSET?
Another option, I've used JSQLParser in the past for a project like this. It wasn't very difficult to use.
jOOQ's Parser
jOOQ's more recent versions have introduced a parser, which parses arbitrary SQL from all currently supported vendor dialects into the jOOQ expression tree. Once you have that expression tree, you can manipulate it to whatever you want using the VisitListener SPI.
For historic context:
For jOOQ, there exists a third-party contribution by Gudu Software called SQL 2 jOOQ, which can parse SQL SELECT statements written in either MySQL or PostgreSQL dialects and generate jOOQ code as output.
I need to parse sql statements and get column names and table names. I tried with sample code. I got table names but I am stucked to get column names for each table.
If you really really need to do this then you should look at using a proper parser toolkit like ANTLR. It is quite a steep learning curve but there are grammars around for SQL that others have already built.
Hand-cranking your own parser will lead you down a mess of bugs for anything but the most basic of queries.
What you need is an SQL parser.
You can try JSQLParser, or ZQL.
I have used both successfully for the same thing the you are trying to do.
You can try also sqlparser. This is commercial however and I have no experience using it.
If you are having ResultSet object then, you can read Column names and Table Names of fired Query.
by using ResultSetMetaData to get MetaData of your query (like table names and column names).
example
You will need to create a parser for your SQL grammar.
ANTLR allows to create such parsers but it can be quite difficult to create a grammar.
Here is a list of exisiting grammar for ANTLR http://www.antlr3.org/grammar/list.html
It contains grammar for mysql, oracle SQL that should work for you.
Checkout Foundation Parser http://foundationdb.github.io/sql-parser/
Also note JSQLParser is no longer on SourceForge but maintain at https://github.com/JSQLParser/JSqlParser/wiki
I have an issue.
I have SQL that I need to append different type of "restrictions" or even do a join. This depends on user's search criteria.
This SQL will involve different table as it can search one-to-many relationship, therefore hibernate ORM can't support my requirement.
May I know if there is a design pattern to help construct such SQL statements?
The design pattern that fits to the problem of representing a language statement is the Interpreter pattern. But before you start to code your SQL parser, take a look to ANTLR.
And what is more important, ask yourself two questions:
Are the number of different SQL's justify the effort of develop a general SQL interpreter solutions instead of programming (just if-else statements) my 5-10 different queries?
Have I reviewed in detail the Hibernate reference manual?
I exactly have a similar requirement where I have a context-free language to define the search criteria, parsed to ParseEntry objects in a ParseTree which are analogous to the Restrictions. I use a SQLQueryGeneratorVisitor to visit the parse table and generate the SQL query, similary a HibernateCriteriaGeneratorVisitor if the criteria needs to be generated for a single entity. So, I essentially used the Visitor pattern making the parse tree and the entries visitable so that different types of criteria can be generated (SQL/Hibernate or something else in future).
I want to get the index of main where clause of database query in Java ?
How can I handle this with Regex?
For example in this query I want to get Second where clause:
select u.id, (select x.id from XEntity where x.id = 200)
from UserEntity u
**where** u.id in (select g.id from AnotherEntity g where g.id = 100)
I think the main where is which that number of "(" characters and ")" characters is equal after it.
but I don't know how can I get this with regex.
With Best Regards
What Toote and David Brabant said is absolutely correct. Parsing SQL, especially complex SQL using only regex is a very hard problem.
In terms of parsing SQL in Java, which seems to be the thrust of your question, there's a very good (if apparently un-maintained) library called JSQLParser. A more up-to-date version of this library can be found on Github here (disclaimer: I made a very small contribution to this). The main page shows an example of visitor designed to consume the output of the AST here.
There is also a grammar for ANTLR available in it's grammar list. Or, if you're feeling adventurous, the H2 database supports a rather extensive range of SQL, including some proprietary features of, e.g., MySQL. You could modify it's Parser to generate an appropriate structure for extracting the information you need.
Regular Expressions are not very good at recognizing such complex structures as SQL queries can be. Mainly because SQL is not context-free, which is exactly the issue you are running into: the WHERE can appear in a lot of places and you want one in particular that depends on the overall structure of the query.
What you will need is an appropriate parser. The only JavaScript SQL parser I could find is not too complete, but you can always help develop it by making sure it fits your needs.