Basically I need to be able to parse a couple of SQL commands but I am not really sure of a good way.
Here is an example of the SQL commands
CREATE TABLE DEPARTMENT (
deptid INT CHECK(deptid > 0 AND deptid < 100),
dname CHAR(30),
location CHAR(10),
PRIMARY KEY(deptid)
);
INSERT INTO STUDENT VALUES (16711,'A.Smith',22,'A',20);
I am coding in java so would split be the best way? Or should I write my own parser? If so, can someone give me an example of how to parse it specifically Strings that are surrounded by ' ' but might contain a ' inside. Also for the CREATE TABLE I need to some how separate the CHECK parameters
If you are working in Java, I'd suggest looking into a Java-based parser generator with an available SQL grammar. ANTLR seems like a good choice for this.
If you're OK with using third parties, there are numerous parsers out there, including:
jOOQ, which has a parser
JsqlParser
General SQL Parser
You could parse your code like this:
Queries queries = ctx.parser().parse("CREATE TABLE ...");
And then access the expression tree, e.g. to find the check constraint
ctx.parser().parse("CREATE TABLE t (i int check (i > 0))")
.forEach(q -> {
if (q instanceof CreateTable ct) {
for (TableElement e : ct.$tableElements()) {
if (e instanceof Constraint c) {
println(c);
}
}
}
});
Note that as of jOOQ 3.17, the traversal is still experimental
Disclaimer: I work for the company behind jOOQ
Related
How to make long queries more readable?
For example I have this one:
String query = "SELECT CASE WHEN EXISTS (SELECT * FROM users WHERE username = 'username' AND user_password = crypt('password', user_password)) THEN 'match' ELSE 'differ' END";
And it's completely unreadable, are there any ways to beautify it?
Since Java 15, you can use text blocks:
String query = """
SELECT CASE
WHEN
EXISTS (
SELECT *
FROM users
WHERE
username = 'username'
AND user_password = crypt('password', user_password)
)
THEN 'match'
ELSE 'differ'
END
""";
In cases when you don't wont to blend SQL and JAVA you can put SQL queries in an .sql file. And get this text when needed.
public class QueryUtil {
static public String getQuery(String fileName) throws IOException {
Path path = Paths.get("src/test/resources//" + fileName + ".sql");
return Files.readAllLines(path).get(0);
}
}
If you can mix SQL and JAVA then starting from JDK15 you can use text blocks for this.
Also you can generates Java code from your database by using JOOQ, it gives many benefits.
Assuming that you can't move to a newer-than-8 version of Java (or even if you can), by far the best solution is to use an ORM. For Java it pretty much comes down to Hibernate, or jOOQ. jOOQ (and possibly Hibernate, I haven't used it so can't say, sorry) allows you to use a fluent programming interface, which is very much in keeping with existing Java code style and patterns.
Another specific advantage of using an ORM is that you can very easily change which DB engine you use without having to change the Java code that you've written beyond changing the SQL dialect in your setup functions. See https://www.jooq.org/javadoc/latest/org.jooq/org/jooq/SQLDialect.html.
You can use JOOQ and get multiple other benefits like type safety, auto-complete, easy mapping and great support.
Have used it for several projects so far and also competition like Kotlin Exposed but always came back to JOOQ.
Move to Java 13+. There are Text Blocks for this.
Or use some ORM library.
I need to parse a query which a user enters, say in a text box, and then what I need, is that I want to encrypt all the values in query leaving the query keywords. To convert it into an equivalent query that can be performed on an encrypted database.
Such as,
select name from employee where salary = 10000
I need an equivalent query as,
select name_enc from employee_enc where salary_enc = 10000_enc
where name_enc,employee_enc, salary_enc and 10000_enc are the encrypted values of name, employee, salary and 10000. I need to do this in java and the the database I'm using is MySQL Server where the table Employee is already encrypted.
Please provide any necessary help. Thanks in advance.
You may want to consider using code from Alibaba's Druid project. Although designed as a sophisticated connection pooling library, this project supports a very advanced parser and AST for ANSI SQL and non-ANSI dialects such as MySQL, Oracle, SQL Server, etc. The project is open source and bears the very liberal Apache License Version 2.0.
The main entry points into this part of the library is SQLUtils.java. You can use values returned from SQLUtils.parseStatements to access a typed model of the statements:
List<SQLStatement> statements = SQLUtils.parseStatements(sql, JdbcConstants.MYSQL);
for (SQLStatement statement : statements) {
if (statement instanceof SQLSelectStatement) {
SQLSelectStatement createTable = (SQLSelectStatement) statement;
// Use methods like: createTable.getSelect().getQuery().
}
}
If you don't need to do it manually use SQL's included encryption and encoding operations.
If you need to do it manually split your SQL query string by spaces and ignore SQL key words as you loop to encrypt. Remember to encode your cipher results with base 64 or hex to ensure data integrity.
private String encryptSQLQuery(String plainSQLQuery){
StringBuilder cipherQuery = new StringBuilder();
String plainQuery = plainSQLQuery;
String[] splitQuery = plainQuery.split("\\s+");
for(String queryWord : splitQuery){
if(!isSQLKeyWord(queryWord))
queryWord = cryptoObject.encryptAndEncode(queryWord);
cipherQuery.append(queryWord);
cipherQuery.append(" ");
}
return cipherQuery.toString();
}
Note that you will have to implement the isSQLKeyWord() and CryptoObject.encryptAndEncode() methods.
I am using spring, hibernate and postgreSQL.
Let's say I have a table looking like this:
CREATE TABLE test
(
id integer NOT NULL
name character(10)
CONSTRAINT test_unique UNIQUE (id)
)
So always when I am inserting record the attribute id should be unique
I would like to know what is better way to insert new record (in my spring java app):
1) Check if record with given id exists and if it doesn't insert record, something like this:
if(testDao.find(id) == null) {
Test test = new Test(Integer id, String name);
testeDao.create(test);
}
2) Call straight create method and wait if it will throw DataAccessException...
Test test = new Test(Integer id, String name);
try{
testeDao.create(test);
}
catch(DataAccessException e){
System.out.println("Error inserting record");
}
I consider the 1st way appropriate but it means more processing for DB. What is your opinion?
Thank you in advance for any advice.
Option (2) is subject to a race condition, where a concurrent session could create the record between checking for it and inserting it. This window is longer than you might expect because the record might be already inserted by another transaction, but not yet committed.
Option (1) is better, but will result in a lot of noise in the PostgreSQL error logs.
The best way is to use PostgreSQL 9.5's INSERT ... ON CONFLICT ... support to do a reliable, race-condition-free insert-if-not-exists operation.
On older versions you can use a loop in plpgsql.
Both those options require use of native queries, of course.
Depends on the source of your ID. If you generate it yourself you can assert uniqueness and rely on catching an exception, e.g. http://docs.oracle.com/javase/1.5.0/docs/api/java/util/UUID.html
Another way would be to let Postgres generate the ID using the SERIAL data type
http://www.postgresql.org/docs/8.1/interactive/datatype.html#DATATYPE-SERIAL
If you have to take over from an untrusted source, do the prior check.
I am trying to write a simple application which reads the a database and produces a set of functions with which to access it; so far so good. Now, what I have come across is that some of the columns in my database are defined as MySQL enum types (e.g. ENUM('red','green','violet')) and I would like to validate the stuff I send to the database rather than receive an error from the driver when an unacceptable value is given, so I was wondering if there is a way to retrieve the possible values for the enum from within clojure.
I am using [clojure.java.jdbc "0.3.0-alpha5"] and [mysql/mysql-connector-java "5.1.25"]. In order to get the metadata for the table I am currently using java.sql.DatabaseMetaData, but trying .getPseudoColumns just gives me nil every time.
Turns out there is no straight forward way to do this using libraries. My own solution is:
(defn- parse-enum
"Parses an enum string and returns it's components"
[enum-str]
; "enum('temp','active','canceled','deleted')"
(map (comp keyword #(.replace % "'" ""))
(-> enum-str
(.replaceFirst "^[^\\(]+\\(([^\\)]+)\\)$" "$1")
(.split "'?,'?"))))
(defn get-enum-value
"Returns the values for an enum in a table.column"
[table column]
(jdbc/with-connection db
(jdbc/with-query-results rs
[(str "show columns from " table " where field = ?") column]
((comp set parse-enum :type first) rs))))
I am using oracle 10g and hibernate 3.3.2. I have used regular expression in sql before, now for the first time I am using it in HQL.
Query query = getSession().createQuery("From Company company
where company.id!=:companyId and
regexp_like(upper(rtrim(ltrim(company.num))), '^0*514619915$' )");
This is my hql, when i run it without regex_like function it runs as expected. But I am not able to execute it with regex_like expression.
It says..
nested exception is org.hibernate.hql.ast.QuerySyntaxException:
unexpected AST node: ( near line 1, column 66.....
Kindly help, how can I use regex_like in hibernate native query? OR some other alternative to do so.
Actually, you can't compare the result of REGEXP_LIKE to anything except in conditional statements in PL/SQL.
Hibernate seems to not accept a custom function without a returnType, as you always need to compare the output to something, i.e:
REGEXP_LIKE('bananas', 'a', 'i') = 1
As Oracle doesn't allow you to compare this function's result to nothing, I came up with a solution using case condition:
public class Oracle10gExtendedDialect extends Oracle10gDialect {
public Oracle10gExtendedDialect() {
super();
registerFunction(
"regexp_like", new SQLFunctionTemplate(StandardBasicTypes.BOOLEAN,
"(case when (regexp_like(?1, ?2, ?3)) then 1 else 0 end)")
);
}
}
And your HQL should look like this:
REGEXP_LIKE('bananas', 'a', 'i') = 1
It will work :)
You can most definitely use any type of database-specific function you wish with Hibernate HQL (and JPQL as long as Hibernate is the provider). You simply have to tell Hibernate about those functions. In 3.3 the only option for that is to provide a custom Dialect and register the function from the Dialect's constructor. If you take a look at the base Dialect class you will see lots of examples of registering functions. Usually best to extend the exact Dialect you currently use and simply provide your extensions (here, registering the function).
An interesting note is that Oracle does not classify regexp_like as a function. They classify it as a condition/predicate. I think this is mostly because Oracle SQL does not define a BOOLEAN datatype, even though their PL/SQL does and I would bet regexp_like is defined as a PL/SQL function returning BOOLEAN...
Assuming you currently use Oracle10gDialect, you would do:
public class MyOracle10gDialect extends Oracle10gDialect {
public Oracle10gDialect() {
super();
registerFunction(
"regexp_like",
new StandardSQLFunction( "regexp_like", StandardBasicTypes.BOOLEAN )
);
}
}
I cannot remember if the HQL parser likes functions returning booleans however in terms of being a predicate all by itself. You may instead have to convert true/false to something else and check against that return:
public class MyOracle10gDialect extends Oracle10gDialect {
public Oracle10gDialect() {
super();
registerFunction(
"regexp_like",
new StandardSQLFunction( "regexp_like", StandardBasicTypes.INTEGER ) {
#Override
public String render(
Type firstArgumentType,
List arguments,
SessionFactoryImplementor factory) {
return "some_conversion_from_boolean_to_int(" +
super.render( firstArgumentType, arguments, factory ) +
")";
}
}
);
}
}
You can't access specific database functions unless JPAQL/HQL provide a way to do so, and neither provide anything for regular expressions. So you'll need to write a native SQL query to use regexes.
On another, and very important point, a few colleagues (Oracle DBAs) told me to never use regexes in oracle, as they can't be indexed, which ends up in the DB performing a full DB scan. If the table has a few entries, then it's ok, but if it has lots of rows, it might cripple the performance.
For those using Hibernate criterion with sqlRestriction (Hibernate Version 4.2.7)
Criterion someCriterion = Restrictions.sqlRestriction("regexp_like (column_name, ?, 'i')", "(^|\\s)"+searchValue+"($|\\s|.$)", StringType.INSTANCE);
Or another option is to create similar function in oracle which will return numeric value based on operation result. Something like that
CREATE OR REPLACE FUNCTION MY_REGEXP_LIKE(text VARCHAR2, pattern VARCHAR2)
RETURN NUMBER
IS function_result NUMBER;
BEGIN
function_result := CASE WHEN REGEXP_LIKE(text, pattern)
THEN 1
ELSE 0
END;
RETURN(function_result);
END MY_REGEXP_LIKE;
and you will be able to use
MY_REGEXP_LIKE('bananas', 'a') = 1
You can use Specification.
Specification<YourEntity> specification = (root, query, builder) -> builder.equal(builder.selectCase()
.when(builder.function("regexp_like", Boolean.class, root.get("your_field"), builder.literal("^0*514619915$")), 1)
.otherwise(0), 1);
List<YourEntity> yourEntities = yourEntityRepository.findAll(specification);
i found the solution Accessing REGEXP_LIKE function in CriteriaBuilder useful for this. Add the dialect based on your Oracle version.