Develop Reference

sql query to check existing records

I have data in following format:
HashMap<PageID, Set<SubscriberIDS>>
What I need to check is how many SubscriberIDS for each of the PageIDs do not exist in a MySQL table already. MySQL table has PageID and SubscriberID columns.
This is what I have so far:
String NEW_SUBSCRIBER_COUNT = "SELECT ? - COUNT(*) as new_subscribers from FB_SUBSCRIPTIONS WHERE PAGEID=? AND SUBSCRIBERID IN (?)";
First parameter being numberOFSubscriberIDs, Second being PageId and Third being SubscriberIds
but this will need to be hit for each pageId. How do I modify it to give me number of new subscribers for each PageID using single query.

Is there any specific need to do it in one query? Because while it can, it might actually be more readable to use your original solution and invoke a query for each page id. In any case, what you want can't be done in a single line, so you need to expect to loop at a given point.
// Build dynamic query
StringBuilder whereClause = new StringBuilder();
Iterator<PageID> it = yourMap.keySet().iterator();
while(it.hasNext()){
PageID key = it.next();
Set<SubscriberIDS> value = yourMap.get(key);
// You need to fill the 'IN' clause with multiple parameters, one for each subscriber id
StringBuilder inClause = new StringBuilder();
for(SubscriberIDS subId : value){
if(inClause.length > 0){
inClause.append(", ");
}
inClause.append("?");
preparedStatement.setInt(paramIndex++, subId.getId());
}
// For each page id we append a new 'OR' to our query
if(whereClause.lenght > 0){
whereClause.append(" OR ");
}
whereClause.append("(PAGEID=? AND SUBSCRIBERID IN (").append(inClause.toString()).append("))");
}
String query = "SELECT PAGEID, COUNT(SUBSCRIBERID) AS SUBSCRIBERS FROM FB_SUBSCRIPTIONS WHERE " + whereClause.toString() + " GROUP BY PAGEID";
// Create prepared statement and set parameters
PreparedStatement preparedStatement = connection.prepareStatement(query);
int paramIndex = 0;
it = yourMap.keySet().iterator();
while(it.hasNext()){
PageID key = it.next();
Set<SubscriberIDS> value = yourMap.get(key);
preparedStatement.setInt(paramIndex++, key.getId());
for(SubscriberIDS subId : value){
preparedStatement.setInt(paramIndex++, subId.getId());
}
}
// Execute query, loop over result and calculate new subscriptions
ResultSet rs = preparedStatement.executeQuery();
while(rs.next()){
int pageId = rs.getInt("PAGEID");
int newSubscriptions = yourMap.get(pageId).size() - rs.getInt("SUBSCRIBERS");
System.out.println(pageId + ", " + newSubscriptions);
}
Given following data in your map:
PAGEID SUBSCRIBERIDS
1 1,3,4,5,9
2 3,4,5,6,8,9
3 2,5,6
And following data in the DB:
PAGEID SUBSCRIBERIDS
1 3,4,10,11
2 1,2,5,7
3 1,2,5,6,7,8,9
This should give following output:
1,3
2,6
3,0
I haven't actually ran the code, so it might need some adjustments, but it gives you the general idea...

JDBC dynamic query, only put value to string if not -1

I have a problem with JDBC and java.
I have a Query like this:
String updateSql = "UPDATE league SET season=?, playedMatches=?, percentHomeWins=?, percentDraws=?, percentAwayWins=?, averageGoalsPerGame=?, averageGoalsHomePerGame=?, averageGoalsAwayPerGame=?, percentOverOne=?, percentOverTwo=?, percentOverThree=?, percentBothTeamsScored=?, scoredGoalsTotal=? " + whereClause + " and country='" + l.getCountry() + "'";
all values after "season" can either be a number >= 0 or -1. -1 means, that there is no value. the values come from a class that holds data (like an object model).
I want only the values in my query, which are >= 0. The other one should not be in the query, because they replace data in the database, which they should not.
Can anyone help me archiving this?

Use a StringBuilder to dynamically build the SQL statement, e.g.
StringBuilder sql = new StringBuilder("UPDATE league SET season=?");
List<Integer> numValues = new ArrayList<>();
if (l.getPlayedMatches() != -1) {
sql.append(", playedMatches=?");
numValues.add(l.getPlayedMatches());
}
if (l.getPercentHomeWins() != -1) {
sql.append(", percentHomeWins=?");
numValues.add(l.getPercentHomeWins());
}
// ... more code ...
sql.append(whereClause)
.append(" and country=?");
try (PreparedStatement stmt = conn.prepareStatement(sql.toString())) {
int paramIdx = 0;
stmt.setInt(++paramIdx, l.getSeason());
for (Integer numValue : numValues)
stmt.setInt(++paramIdx, numValue);
stmt.setString(++paramIdx, l.getCountry());
stmt.executeUpdate();
}

Java - Set generic table with SQL results

Below you can find the code that I'm using to retrieve a table from a generic SQL statement (the SQL code is inputed by the user in another part of the code).
Since I'll be storing more than one table in the future, and the retrieved table will have some format functions applied to it, I'm storing the values in a:
Map<String, Map<String,List<Object>>> tables = new HashMap<String, Map<String,List<Object>>>();
The first Map is a Table, the second is a Column and finally the list hold each line of data.
The column names/order in:
Map<String, TreeMap<Integer, String>> order = new HashMap<String, TreeMap<Integer, String>>();
The Map is tha table, the TreeMap are the column number - column name
The code to create de generic table is:
PreparedStatement stmt = conn.prepareStatement(sql);
ResultSet rs = stmt.executeQuery();
ResultSetMetaData rsmd = rs.getMetaData();
for(int i = 1; i <= rsmd.getColumnCount(); i++){
String name = rsmd.getColumnLabel(i); //Label vs name. Label is also what's defined by the user in: "as VENDA"
answer.put(name, null);
singleOrder.put(i, name);
}
while(rs.next()){
Iterator answerIt = answer.entrySet().iterator();
while (answerIt.hasNext()) { //Get Columns in index order to compare with body HashMap (unordered)
Map.Entry columnNameValue = (Map.Entry)answerIt.next(); //key = columnName, value = List
String columnName = (String) columnNameValue.getKey();
List<Object> tmp = answer.get(columnName);
if (tmp == null) {
tmp = new ArrayList<Object>();
answer.put(columnName, tmp);
}
Object item = rs.getObject(columnName);
if(item instanceof Integer){
item = ((Integer) item).doubleValue();
} else if (item instanceof Long){
item = ((Long) item).doubleValue();
} else if (item instanceof BigDecimal){
item = ((BigDecimal) item).doubleValue();
}
tmp.add(item);
}
}
Efficiency will be key for this part of my code. How can I improve the table creation?
(I think this is a important question for Java coders, so a clear response can serve a pattern for future programmers that have the same difficulty that I'm having now)
Update 1:
Why I'm asking: I couldn't find any example of how to handle a flexible query in Java. I've created my own solution, but I fear it's some kind of monster compared to how to handle such a case properly in Java.
As example:
Lets say I'll name the result of a first query as "table1".
The query will be: Select STORE, SALES, GOAL, EXTRA from ... ;
The order variable will be like:
Map<String, TreeMap<Integer, String>> order = new HashMap<String, TreeMap<Integer, String>>();
table1,
1, "STORE"
2, "SALES"
3, "GOAL"
4, "EXTRA"
The tables variable:
Map<String, Map<String,List<Object>>> tables = new HashMap<String, Map<String,List<Object>>>();
"table1",
"STORE",
"unit1"
"unit2"
"unit3"
"unit4"
"SALES",
1312
126
1361
6823
"GOAL"
1300
160
1200
6000
"EXTRA"
"info1"
"info2"
"info3"
"info4"

insert into DB table java

I want to fill a table using insert statement. To get the data I used arrays that contain the data I need.
I want to insert the id and some other values in this format.
id w0 w1 w2 w3
1 0.0 1.0 0.54 0.0
2 1.0 0.5 0.0 0.8
and others
id ---> read from a normal array
the other numbers ---> read from a 2d array
and I want for each new row a new id.
The problem I have is the rows are moving and incrementing while the id is stuck in 0, so each new row is added with the same id which is wrong since the id is a primary key.
String sqlselect=new String("select k_id from keywordsTable");
ResultSet rss = stmt1.executeQuery(sqlselect);
kId=new String[numOfFields];
String f="";
String ff="";
String ffff="";
for (int i=0;i<kId.length;i++) {
while(rss.next())
{
kId[i]=rss.getString(1);
f="w"+kId[i]+ " varchar(20)";
ff+="w"+kId[i]+", ";
ffff+="w"+kId[i]+", ";
//fff+="w"+kId[i]+ " varchar(20), ";
String sqlalter=new String("ALTER TABLE "+ tableName +" add "+f+"");
//System.out.println(sqlalter);
stmt1.executeUpdate(sqlalter);
}
}
ff = ff.replaceAll(", $","");
ffff = ffff.replaceAll(", $","");
String sqlselectF=new String("select f_id from filesTable");
ResultSet rssF = stmt1.executeQuery(sqlselectF);
FId=new String[numOfFiles];
String matInsert = null;
String g="";
String seperator = "";
for (int s=0;s<FId.length;s++) {
while(rssF.next())
{
FId[s]=rssF.getString(1);
g=seperator+FId[s];
for (int k=0;k<di.mat.length;k++) { //row
String m="";
String sep = "";
for (int j=0;j<di.mat[k].length;j++) { //col
m+= (sep+di.mat[k][j]);
sep = " ,";
matInsert=new String("INSERT INTO "+ tableName +"(id,"+ffff+")"+"values" +"("+g+","+m+")");
} //col
System.out.println("ff="+g);
System.out.println(matInsert);
stmt1.executeUpdate(matInsert);
} //row
} //while
} //for 1
seperator=" ,";

You shouldn't insert IDs, configure the column as AUTO_INCREMENT and let MySQL assign the IDs upon each INSERT.
Second, the nesting of while loop inside a for loop is probably a logical mistake.
Third, ffff is not declared anywhere in the code that you posted.
Forth, the INSERT query is missing whitespaces in a few places, it will come out (if all the input is correct) as something like:
INSERT INTO tableName(id,ffff)values(x,y)

how to generate sql query dynamically having specific colums

I have several tables. I have a query also. My problem is to generate the SQL query dynamically using Java.
I have the following fields in a separate table:
Collumn name status
po_number, Y
unit_cost, Y
placed_date , Y
date_closed, Y
scheduled_arrival_date Y
date_closed Y
order_quantity Y
roll_number N
product_sku N
product_category_name N
rec_vendor_quantity Y
vendor_name Y
et_conversion_unit_quantity Y
from which i have to generate a query when the status is Y, the problem here is some time the above columns
The following query is the out put of the above :
here i have inculded all the columns but i have to exculde the column which has the status of N, please help me to construt the query using java.
select
pi.po_number,poi.unit_cost,pi.placed_date CreateDate,
case when isnull(pi.date_closed) then pi.scheduled_arrival_date
else pi.date_closed end as ReceviedDate,
poi.order_quantity,poi.roll_number,p.product_sku product_name,
pc.product_category_name,poi.rec_vendor_quantity,pv.vendor_name,p.et_conversion_unit_quantity,pi.note
from
purchase_order as pi,
purchase_order_inventory as poi,
product_vendors as pv,
products AS p,
product_categories AS pc
where
pi.purchase_order_id=poi.purchase_order_id and
pc.product_category_id=p.product_category_id and
poi.product_id = p.product_id and
poi.product_category_id=pc.product_category_id and
pi.vendor_id=pv.product_vendor_id and
( ( pi.date_closed >= '2012-01-01' and pi.date_closed <='2012-09-05 23:59:59' )
or ( pi.scheduled_arrival_date >= '2012-01-01' and pi.scheduled_arrival_date <='2012-09-05 23:59:59') ) and
pi.po_type=0
and pi.status_id = 0 and poi.transaction_type = 0
order by pi.po_number
UPDATE :
QUERY : STEP 1:
SELECT rcm.id,rcm.tablename,rcm.columnname,rcm.size,rcm.displayorder,rcm.isactive FROM report_customise_master rcm where rcm.tablename !='employee' and rcm.isactive='Y' order by rcm.displayorder;
STEP 2 :
Java method to construct the query :
public Map getComplexReportQuery() {
String query = "SELECT rcm.id,rcm.tablename,rcm.columnname,rcm.size,rcm.displayorder,rcm.isactive FROM report_customise_master rcm where rcm.tablename !='employee' and rcm.isactive='Y' order by rcm.displayorder;";
String tableName = "", from = "", select = "";
StringBuffer sb = new StringBuffer();
Map<String, List<String>> resultsMap = new LinkedHashMap<String, List<String>>();
Map<String, String> displayOrderMap = new LinkedHashMap<String, String>();
Map queryMap = new LinkedHashMap();
if (!query.isEmpty() || query.length() > 0) {
sb.append(query);
}
Connection connection = getConnection();
if (connection != null) {
try {
PreparedStatement reportQueryPS = connection.prepareStatement(sb.toString());
ResultSet reportQuery_rst = reportQueryPS.executeQuery();
List<String> tables = new ArrayList<String>();;
if (reportQuery_rst != null) {
StringBuilder selectQuery = new StringBuilder(" SELECT ");
StringBuilder fromQuery = new StringBuilder(" FROM ");
while (reportQuery_rst.next()) {
tableName = reportQuery_rst.getString("tablename");
List<String> columns = resultsMap.get(tableName);
if (columns == null) {
columns = new ArrayList<String>();
resultsMap.put(tableName, columns);
}
columns = resultsMap.get(tableName);
String columnName = reportQuery_rst.getString("columnname");
columns.add(columnName);
}
tableName = "";
for (Entry<String, List<String>> resultEntry : resultsMap.entrySet()) {
tableName = resultEntry.getKey();
List<String> columns = resultEntry.getValue();
int i = 0;
for (String column : columns) {
selectQuery.append(tableName + "." + column);
if (i != columns.size()) {
selectQuery.append(",");
} else {
selectQuery.append("");
}
i++;
}
if (!tables.contains(tableName)) {
tables.add(tableName);
}
}
//to remove comma at the end of line
select = selectQuery.toString().replaceAll(",$", "");
tableName = "";
int i = 0;
for (String table : tables) {
fromQuery.append(table);
fromQuery.append(" ");
fromQuery.append(table);
if (i != tables.size()) {
fromQuery.append(",");
} else {
fromQuery.append("");
}
i++;
}
from = fromQuery.toString().replaceAll(",$", "");
queryMap.put("query", select + from);
}
//from = from+"ORDER BY "+orderbyColumn+" "+sort+" ";
} catch (Exception ex) {
ex.printStackTrace();
} finally {
try {
closeConnection(connection, null, null);
} catch (Exception ex) {
ex.printStackTrace();
}
}
} else {
System.out.println("Connection not Established. Please Contact Vendor");
}
return queryMap;// return the map/ list which contains query and sory and display order
}
STEP 3 : Result query
{query= SELECT purchase_order.po_number,purchase_order.placed_date,purchase_order.date_closed,purchase_order.scheduled_arrival_date,purchase_order_inventory.unit_cost,purchase_order_inventory.order_quantity,purchase_order_inventory.roll_number,purchase_order_inventory.rec_vendor_quantity,products.product_sku,products.et_conversion_unit_quantity,product_categories.product_category_name ,product_vendors.vendor_name FROM purchase_order purchase_order,purchase_order_inventory purchase_order_inventory,products products,product_categories product_categories,product_vendors product_vendors}
but this not what i wanted, Please help me to construct the query i have given.

Two queries
You need to make two queries:
Query which fields are enabled
Build the second query string (the one you want to build dinamically)
It's this way because a SQL query has to tell which columns will be included before querying any data. In fact it will be used to build the internal DB query plan, it is, the way the DB motor will use to retrieve and organize the data you ask.
Query all columns
Is it necesary to query only that fields? Can't you query everything and use the relevant data?
Joins
Looking at the updated question I guess you need to dynamically add where conditions to join tables correctly. What I should do is have a reference telling me what coindition to add when a table is present.
There are at least two options:
Based on table pairs present (by example: "if A and B are present then add A.col1 = B.col2")
Based on tables present ("if B is present, then add A.col1 = B.col2; A should be present"
Based on your example I think the second option is more suitable (and easy to implement).
So I should have some static Map<String, JoinInfo> where JoinInfo has at least:
JoinInfo
+ conditionToAdd // by example "A.col1 = B.col2"
+ dependsOnTable // by example "A" to indicate that A must be present when B is present
So you can use:
that info to add tables that should be (by example: even if A has no selected cols, must be present to join with B)
include the conditionToAdd to the where clause
Anyway... I think you are getting into much trouble. Why so dynamic?

You have to approach the thing step by step.
Firstly you have to create a query that will return all rows that have status='Y'
Then you will put the COLUMN_NAME in a list of Strings.
List<String> list = new List<String>();
while(rs.next()){
list.add(rs.getString(columnNumber));
}
And then you have to loop the List and generate dynamically your second sql statement
String sqlSelect = "SELECT ";
String sqlFrom = " FROM SOME_OTHER_TABLE "
String sqlWhere = " WHERE SOME_CONDITION = 'SOME_VALUE' "
for(String x : list){
sqlFrom += x +" , "+;
}
//here make sure that you remove the last comma from sqlFrom because you will get an SQLException
String finalSql = sqlSelect + sqlFrom + sqlWhere ;