Develop Reference

Dynamically build SQL where clause based on presence of attributes in DTO

I have a screen where users can search a table based on 1 or more of its columns. I am passing the columns the users would like to filter by as a DTO to the backend.
There I am building a SQL to query by checking for the presence of these fields in the DTO...
Object transactionSearch (TransactionSearchDTO dto ){
StringBuilder transactionQuery = new StringBuilder()
transactionQuery.append("SELECT id, delivery_date, order_date, customer_name FROM transaction where 1=1 ")
if (dto.order_id) {
transactionQuery.append("and order_id = ${dto.order_id}")
}
if (delivery_date) {
transactionQuery.append("and order_id = ${delivery_date}")
}
if (customer_name) {
transactionQuery.append("and order_id = ${customer_name}")
}
}
What I am ending up with is a ton of if statements for each column.
Is there a way to achieve this without having an if block per filter?
I found some ideas here but I need a solution in Groovy or Java.

i suggest to use GString instead of StringBuilder to pass parameters correctly to groovy.sql.Sql
class DTO{
int id
String name
Date created
}
def a = new DTO(id:123, created:new Date())
GString f(DTO a){
GString q = GString.EMPTY + 'select ' + a.getProperties().collect{k,v-> k}.join(',')+' from T'
a.getProperties().each{k,v->
if(k!='class' && v!=null){
q = q + ( q.getValueCount()==0 ? ' WHERE ' : ' AND ' )
q = q + k + " = ${v}"
}
}
return q
}
def s = f(a)
println s
println s.getValues()
println s.getStrings()

Loop MySQL run with java

I have 3 tables having the following content :
Author
idAuthor INT
name VARCHAR
Publication
idPublication INT
Title VARCHAR
Date YEAR
Type VARCHAR
Conference
author_has_publication
author_idAuthor INT
publication_idPublication INT
I am trying to do relational schema on the authors. The objectif is to show the number of publication they have in common. The authors name are parameters, I can have up to 8 names. My code is giving the number of common publication between 2 authors, so i have to loop it. I am currently using a Java loop and SQL statement to do that. Here is the SQL part
private int runQuery(String a1, String a2){ // a1 author 1 and a2 author 2
try {
auth1 = new ArrayList<String>();
Class.forName("com.mysql.jdbc.Driver");
Connection connection = DriverManager.getConnection(
"jdbc:mysql://localhost:3306/mydb", "root", "ROOT");
Statement stmt = connection.createStatement();
long start = System.currentTimeMillis();
String queryUpdate1 = "DROP TABLE IF EXISTS temp1;";
String queryUpdate2 = "DROP TABLE IF EXISTS temp2;";
String queryUpdate3 = "CREATE TEMPORARY TABLE IF NOT EXISTS temp1 AS (SELECT Author.name, Publication.idPublication, Publication.title FROM Author INNER JOIN Author_has_Publication ON Author_has_Publication.author_idAuthor=author.idAuthor INNER JOIN Publication ON Author_has_Publication.publication_idPublication=publication.idPublication WHERE Author.name='"+ a1+"');";
String queryUpdate4 = "CREATE TEMPORARY TABLE IF NOT EXISTS temp2 AS (SELECT Author.name, Publication.idPublication, Publication.title FROM Author INNER JOIN Author_has_Publication ON Author_has_Publication.author_idAuthor=author.idAuthor INNER JOIN Publication ON Author_has_Publication.publication_idPublication=publication.idPublication WHERE Author.name='"+ a2+"');";
String query = "SELECT COUNT(*) FROM (SELECT temp1.title from temp1 INNER JOIN temp2 on temp1.idPublication = temp2.idPublication) as t;";
stmt.executeUpdate(queryUpdate1);
stmt.executeUpdate(queryUpdate2);
stmt.executeUpdate(queryUpdate3);
stmt.executeUpdate(queryUpdate4);
ResultSet rs = stmt.executeQuery(query);
int result = -1;
while (rs.next()) {
result = rs.getInt(1);
}
System.out.println("result = " + result);
long end = System.currentTimeMillis() - start;
queryTimeLabel.setText("Query Execution Time :"+end);
connection.close();
return result;
} catch (Exception e) {
System.out.println(e);
}
return -1;
}
Here is the loop part (to repeat the SQL when there are more than 2 authors given) and generate the graph :
public void actionPerformed(ActionEvent e) {
graph = new mxGraph();
Object parent = graph.getDefaultParent();
authVertex = getAuthors();
// ///////////////////////////////////
// CREATES GRAPH, Graph only shows up after you resize the window
graph.getModel().beginUpdate();
try {
int i = 0;
for(String a: authVertex.keySet()){
int j = 0;
for(String b: authVertex.keySet()){
if(j > i) {
graph.insertEdge(parent, null, String.valueOf(runQuery(a,b)), authVertex.get(a), authVertex.get(b)); // loop the SQL statement 2 by 2.
}
j++;
}
i++;
}
} finally {
graph.getModel().endUpdate();
}
graphComponent = new mxGraphComponent(graph);
graphPan.removeAll();
graphPan.add(graphComponent);
setVisible(true);
// /////////////////////////////////////////
}
My code is currently working, but I would like to know if it was possible to increase the performance by passing everything into MySQL, that means that I enter the authors name in parameter and the loop is hangled by MySQL, I check the MySQL procedure but my issue is how to handle the authors names parameter as it is a variable.

One way, in a single statement:
SELECT COUNT(*)
FROM Author_has_Publication AS ap1
JOIN Author_has_Publication AS ap2 ON ap1.publication_idPublication =
ap2.publication_idPublication
JOIN Author AS a1 ON ap1.author_idAuthor = a1.id_Author
JOIN Author AS a2 ON ap2.author_idAuthor = a2.id_Author
WHERE a1.name = '...'
AND a2.name = '...'
Another way may be
SELECT COUNT(*)
FROM
(
SELECT ahp.publication_idPublication, COUNT(*)
FROM Author_has_Publication AS ahp
JOIN Author AS a ON a.id_Author = ahp.author_idAuthor
WHERE a.name IN ('...', '...')
GROUP BY ahp.publication_idPublication
HAVING COUNT(*) = 2 -- Number of authors
) x
Composite indexes needed:
Author_has_Publication: (author_idAuthor, publication_idPublication)
Author_has_Publication: (publication_idPublication, author_idAuthor)
Author: (name, id)
Note: Each technique can be rather easily extended to more than 2 authors. The second query could even be adapted to "at least 3 of these 5 authors": 5 names in IN and HAVING COUNT(*) >= 3.

How to make recursive query in SQLite?

if my data structure is like this
parentA
-------parentAA
--------------parentAAA
---------------------childA
if i can get "childA.name" . how can i know all the parent name till the top level.
so it will be like this > parentA/parentAA/parentAAA/childA
what is the best way to do this ?
i'm working with SQLite and JAVA/android .. thanks in adv.
____________ EDIT
Okay guys, thanks for all of u . so i just make it by repeating "select query". BOTTOM-UP this is the method i create
public String getPath(int id, int type) {
StringBuilder pathBuilder = new StringBuilder();
String sql = null;
int parentId = 0;
if (id == 0) {
pathBuilder.insert(0, "/root/");
return pathBuilder.toString();
}
if (type == LayerManagementActivity.PARENT) {
do {
sql = "SELECT id, name, parent_id from parents_table where id="
+ id;
Cursor c = mDatabase.rawQuery(sql, null);
if (c.moveToFirst()) {
parentId = c.getInt(2);
id = c.getInt(0);
pathBuilder.insert(0, "/" + c.getString(1));
c.close();
}
id = parentId;
} while (parentId != 0);
pathBuilder.insert(0, "/root");
pathBuilder.append("/");
} else if (type == LayerManagementActivity.CHILD) {
sql = "SELECT id, name, folder_id FROM childs_table WHERE id=" + id;
Cursor c = mDatabase.rawQuery(sql, null);
if (c.moveToFirst()) {
pathBuilder.append(c.getString(1));
id = c.getInt(0);
int folderId = c.getInt(2);
String path = getPath(folderId, LayerManagementActivity.PARENT);
pathBuilder.insert(0, path);
}
c.close();
}
Log.d("crumb", pathBuilder.toString());
return pathBuilder.toString();
}

In this SQLite Release 3.8.3 On 2014-02-03 has been added support for CTEs. Here is documentation WITH clause
Example:
WITH RECURSIVE
cnt(x) AS (
SELECT 1
UNION ALL
SELECT x+1 FROM cnt
LIMIT 1000000
)
SELECT x FROM cnt;

I have a table called project with a column named rates.
The rates column is a string that holds a JSON array.
To split this string into rows that I can use in an IN statement to get rows from the related table, I use this for the IN part
WITH
split(s, p) AS (
SELECT substr(printf("%s%s", ss, ","), instr(ss, ",")+1), trim(substr(ss, 0, instr(ss, ","))) from ( select replace(replace(rates,"[",""), "]","") ss from project where rowid = 1)
UNION ALL
SELECT substr(s, instr(s, ",")+1), trim(substr(s, 0, instr(s, ","))) FROM split
where p!=""
)
select p from split where p!=""

You can use nested set model. Nested sets have big advantage that they can be implemented in most SQL engines using simple, non-recursive SQL queries.

SQLite doesn't support recursive CTEs (or CTEs at all for that matter),
there is no WITH in SQLite. Since you don't know how deep it goes, you can't use the standard JOIN trick to fake the recursive CTE. You have to do it the hard way and implement the recursion in your client code:
Grab the initial row and the sub-part IDs.
Grab the rows and sub-part IDs for the sub-parts.
Repeat until nothing comes back.

how to generate sql query dynamically having specific colums

I have several tables. I have a query also. My problem is to generate the SQL query dynamically using Java.
I have the following fields in a separate table:
Collumn name status
po_number, Y
unit_cost, Y
placed_date , Y
date_closed, Y
scheduled_arrival_date Y
date_closed Y
order_quantity Y
roll_number N
product_sku N
product_category_name N
rec_vendor_quantity Y
vendor_name Y
et_conversion_unit_quantity Y
from which i have to generate a query when the status is Y, the problem here is some time the above columns
The following query is the out put of the above :
here i have inculded all the columns but i have to exculde the column which has the status of N, please help me to construt the query using java.
select
pi.po_number,poi.unit_cost,pi.placed_date CreateDate,
case when isnull(pi.date_closed) then pi.scheduled_arrival_date
else pi.date_closed end as ReceviedDate,
poi.order_quantity,poi.roll_number,p.product_sku product_name,
pc.product_category_name,poi.rec_vendor_quantity,pv.vendor_name,p.et_conversion_unit_quantity,pi.note
from
purchase_order as pi,
purchase_order_inventory as poi,
product_vendors as pv,
products AS p,
product_categories AS pc
where
pi.purchase_order_id=poi.purchase_order_id and
pc.product_category_id=p.product_category_id and
poi.product_id = p.product_id and
poi.product_category_id=pc.product_category_id and
pi.vendor_id=pv.product_vendor_id and
( ( pi.date_closed >= '2012-01-01' and pi.date_closed <='2012-09-05 23:59:59' )
or ( pi.scheduled_arrival_date >= '2012-01-01' and pi.scheduled_arrival_date <='2012-09-05 23:59:59') ) and
pi.po_type=0
and pi.status_id = 0 and poi.transaction_type = 0
order by pi.po_number
UPDATE :
QUERY : STEP 1:
SELECT rcm.id,rcm.tablename,rcm.columnname,rcm.size,rcm.displayorder,rcm.isactive FROM report_customise_master rcm where rcm.tablename !='employee' and rcm.isactive='Y' order by rcm.displayorder;
STEP 2 :
Java method to construct the query :
public Map getComplexReportQuery() {
String query = "SELECT rcm.id,rcm.tablename,rcm.columnname,rcm.size,rcm.displayorder,rcm.isactive FROM report_customise_master rcm where rcm.tablename !='employee' and rcm.isactive='Y' order by rcm.displayorder;";
String tableName = "", from = "", select = "";
StringBuffer sb = new StringBuffer();
Map<String, List<String>> resultsMap = new LinkedHashMap<String, List<String>>();
Map<String, String> displayOrderMap = new LinkedHashMap<String, String>();
Map queryMap = new LinkedHashMap();
if (!query.isEmpty() || query.length() > 0) {
sb.append(query);
}
Connection connection = getConnection();
if (connection != null) {
try {
PreparedStatement reportQueryPS = connection.prepareStatement(sb.toString());
ResultSet reportQuery_rst = reportQueryPS.executeQuery();
List<String> tables = new ArrayList<String>();;
if (reportQuery_rst != null) {
StringBuilder selectQuery = new StringBuilder(" SELECT ");
StringBuilder fromQuery = new StringBuilder(" FROM ");
while (reportQuery_rst.next()) {
tableName = reportQuery_rst.getString("tablename");
List<String> columns = resultsMap.get(tableName);
if (columns == null) {
columns = new ArrayList<String>();
resultsMap.put(tableName, columns);
}
columns = resultsMap.get(tableName);
String columnName = reportQuery_rst.getString("columnname");
columns.add(columnName);
}
tableName = "";
for (Entry<String, List<String>> resultEntry : resultsMap.entrySet()) {
tableName = resultEntry.getKey();
List<String> columns = resultEntry.getValue();
int i = 0;
for (String column : columns) {
selectQuery.append(tableName + "." + column);
if (i != columns.size()) {
selectQuery.append(",");
} else {
selectQuery.append("");
}
i++;
}
if (!tables.contains(tableName)) {
tables.add(tableName);
}
}
//to remove comma at the end of line
select = selectQuery.toString().replaceAll(",$", "");
tableName = "";
int i = 0;
for (String table : tables) {
fromQuery.append(table);
fromQuery.append(" ");
fromQuery.append(table);
if (i != tables.size()) {
fromQuery.append(",");
} else {
fromQuery.append("");
}
i++;
}
from = fromQuery.toString().replaceAll(",$", "");
queryMap.put("query", select + from);
}
//from = from+"ORDER BY "+orderbyColumn+" "+sort+" ";
} catch (Exception ex) {
ex.printStackTrace();
} finally {
try {
closeConnection(connection, null, null);
} catch (Exception ex) {
ex.printStackTrace();
}
}
} else {
System.out.println("Connection not Established. Please Contact Vendor");
}
return queryMap;// return the map/ list which contains query and sory and display order
}
STEP 3 : Result query
{query= SELECT purchase_order.po_number,purchase_order.placed_date,purchase_order.date_closed,purchase_order.scheduled_arrival_date,purchase_order_inventory.unit_cost,purchase_order_inventory.order_quantity,purchase_order_inventory.roll_number,purchase_order_inventory.rec_vendor_quantity,products.product_sku,products.et_conversion_unit_quantity,product_categories.product_category_name ,product_vendors.vendor_name FROM purchase_order purchase_order,purchase_order_inventory purchase_order_inventory,products products,product_categories product_categories,product_vendors product_vendors}
but this not what i wanted, Please help me to construct the query i have given.

Two queries
You need to make two queries:
Query which fields are enabled
Build the second query string (the one you want to build dinamically)
It's this way because a SQL query has to tell which columns will be included before querying any data. In fact it will be used to build the internal DB query plan, it is, the way the DB motor will use to retrieve and organize the data you ask.
Query all columns
Is it necesary to query only that fields? Can't you query everything and use the relevant data?
Joins
Looking at the updated question I guess you need to dynamically add where conditions to join tables correctly. What I should do is have a reference telling me what coindition to add when a table is present.
There are at least two options:
Based on table pairs present (by example: "if A and B are present then add A.col1 = B.col2")
Based on tables present ("if B is present, then add A.col1 = B.col2; A should be present"
Based on your example I think the second option is more suitable (and easy to implement).
So I should have some static Map<String, JoinInfo> where JoinInfo has at least:
JoinInfo
+ conditionToAdd // by example "A.col1 = B.col2"
+ dependsOnTable // by example "A" to indicate that A must be present when B is present
So you can use:
that info to add tables that should be (by example: even if A has no selected cols, must be present to join with B)
include the conditionToAdd to the where clause
Anyway... I think you are getting into much trouble. Why so dynamic?

You have to approach the thing step by step.
Firstly you have to create a query that will return all rows that have status='Y'
Then you will put the COLUMN_NAME in a list of Strings.
List<String> list = new List<String>();
while(rs.next()){
list.add(rs.getString(columnNumber));
}
And then you have to loop the List and generate dynamically your second sql statement
String sqlSelect = "SELECT ";
String sqlFrom = " FROM SOME_OTHER_TABLE "
String sqlWhere = " WHERE SOME_CONDITION = 'SOME_VALUE' "
for(String x : list){
sqlFrom += x +" , "+;
}
//here make sure that you remove the last comma from sqlFrom because you will get an SQLException
String finalSql = sqlSelect + sqlFrom + sqlWhere ;

Hibernate query building based on conditions

I have a form where user can select search criteria.
The criterias are say:
Product Name: Input field
Name Option: Radio button group - begins with (default selected)/ is/ contains
Country: dropdown of country
Status: All, Active, Blocked
Type: All, One, Two, Three
Only Product Name is mandatory. Other dropdowns are optional.
So if country is not given, I should find products for all countries.
If active is not given, I should find both active and blocked products.
If Type is not given, I should return all the three types products.
I am building hibernate query as below:
String productName = searchCriteria.getValue("productName");
String productNameCriteria = searchCriteria.getValue("productNameCriteria");
String country = searchCriteria.getValue("country");
String status = searchCriteria.getValue("status");
String type = searchCriteria.getValue("type");
Query prodQuery = null;
String prodSql = "select count(*) from Product p where";
// is
if (productNameCriteria.equalsIgnoreCase("IS")){
prodSql += "p.productName = '"+productName+"'";
}
// begins with
else if (productNameCriteria.equalsIgnoreCase("BEGINS WITH")){
prodSql += "p.productName = '"+productName+"%'";
}
// contains
else (productNameCriteria.equalsIgnoreCase("BEGINS WITH")){
prodSql += "p.productName = '%"+productName+"%'";
}
if(!country.equalsIgnoreCase("0")){
prodSql += " and p.country = '"+country+"'";
}
if(!status.equalsIgnoreCase("ALL")){
if(status.equalsIgnoreCase("active"))
prodSql += " and p.status = 'active'";
else
prodSql += " and p.status = 'blocked'";
}
if(!type.equalsIgnoreCase("ALL")){
if(type.equalsIgnoreCase("one"))
prodSql += " and p.type = 'one'";
else if(type.equalsIgnoreCase("two"))
prodSql += " and p.type = 'two'";
else
prodSql += " and p.type = 'three'";
}
prodQuery = this.em.createQuery(prodSql);
List<Object[]> results = prodQuery.getResultList();
Am I doing query building the right way ? Or is there any other efficient method ???
Thanks for reading!!

Try looking at Criteria Query
Criteria crit = sess.createCriteria(Product.class);
if (productNameCriteria.equalsIgnoreCase("IS"))
crit.add( Restrictions.eq("productName", productName);
else if (productNameCriteria.equalsIgnoreCase("BEGINS WITH"))
crit.add( Restrictions.like("productName", productName + "%")
// etc
If you absolutely must build a string query then you should be using a StringBuilder
StringBuilder sb = new StringBuilder();
sb.append("select count(*) from Product p where ");
if (productNameCriteria.equalsIgnoreCase("IS"))
sb.append("p.productName = '").append(productName).append("'");
// etc
String query = sb.toString();
Using a StringBuilder reduces the number of instances created at runtime.
You could also look into using query parameters, which would reduce some of the query complexity, though I don't know what the runtime query performance effects are.
"select count(*) from Product p where p.productName = :productName"
"select count(*) from Product p where p.productName = ?"
You can then use Query#setParameter (or one of the other variants like setString) to define the values in the query. This is also a much, much better way of building the query because it's going to automatically manage quoting and escaping of values you're receiving from the UI. Use query parameters and not string concatenation, regardless of how you build the query string.

Yes .It will work if you build the query dynamically in this way .But the code will become tedious and noisy as it involves string manipulating of the where-condition clause .
For this kind of query 's use case , which is a search that allows users to specify a range of different property values to be matched by the returned result set , using Query By Example(QBE) is more efficient and elegant.
The idea of QBE is that you provide an instance of the queried class with some properties initialized, and the query will returns the records with matching property values.
Reference
Example JavaDocs
YouTube Hibernate Tutorial - Projections and Query By Example