Develop Reference

Looking for the optimization of the below code

I have found below code buggy as it degrades the performance of extjs3 grid, i am looking for possibilities of optimization at query or code level, as per my analysis, if we extract out the query there are two nested inner queries which are responding slow, in addition, the code inside while loop trying to find the unique id, can't we have distinct in query, or joins rather than inner queries.
Please suggest me the best practice to follow in order to achieve optimization.
public boolean isSCACreditOverviewGridVisible(String sessionId) {
Connection conn = null;
ResultSet rs = null;
PreparedStatement ps = null;
boolean result = false;
try {
CommonUtility commUtil = new CommonUtility();
List<String> hmIds = new ArrayList<String>();
Map<String, String> tmStockMap = new TreeMap<String, String>();
Set<String> setRecentCertificate = new HashSet<String>();
String managerAccountId = sessionInfo.getMembershipAccount();
String stockQuery = " select memberId , RootCertficateId from stockposition sp where sp.stocktype = 'TR' and sp.memberId "
+ " IN ( select hm2.accountId from "
DATALINK
+ ".holdingmembers hm2 "
+ " where hm2.holdingId = ( select holdingId from "
DATALINK
+ ".holdingmembers hm1 where hm1.accountId = ? )) "
+ " order by sp.createdDate desc ";
conn = getChildDBConnection();
if (null != conn) {
ps = conn.prepareStatement(stockQuery);
ps.setString(1, managerAccountId);
rs = ps.executeQuery();
if (null != rs) {
while (rs.next()) {
String memberId = rs.getString("memberId");
String rootCertficateId = rs
.getString("RootCertficateId");
if (tmStockMap.containsKey(rootCertficateId)) {
continue;
}
hmIds.add(memberId);
tmStockMap.put(rootCertficateId, memberId);
}
}
rs.close();
ps.close();
if (null != hmIds && !hmIds.isEmpty()) {
String inIds = commUtil.getInStateParam(hmIds);
String mostRecentLicense = "Select RootCertificateId , memberaccountid from "
+ OctopusSchema.octopusSchema
+ ".certificate c where c.memberaccountid IN ("
+ inIds
+ ") and c.isrootcertificate=0 and c.certificationstatusid > 1 order by c.modifieddate desc";
ps = conn.prepareStatement(mostRecentLicense);
rs = ps.executeQuery();
if (null != rs) {
while (rs.next()) {
String rootCertficateId = rs
.getString("RootCertificateId");
String memberaccountid = rs
.getString("memberaccountid");
if (setRecentCertificate.contains(memberaccountid)) {
continue;
}
setRecentCertificate.add(memberaccountid);
if (tmStockMap.containsKey(rootCertficateId)) {
result = true;
break;
}
}
}
rs.close();
ps.close();
} else {
result = false;
}
}
} catch (Exception e) {
LOGGER.error(e);
} finally {
closeDBReferences(conn, ps, null, rs);
}
return result;
}
QUERY:
select RootCertficateId,memberId from stockposition sp where sp.stocktype = 'TR' and sp.memberId
IN ( select hm2.accountId from
DATALINK.holdingmembers hm2
where hm2.holdingId = ( select holdingId from
DATALINK.holdingmembers hm1 where hm1.accountId = '4937' ))
order by sp.createdDate DESC;

One quick approach would be a substition of your IN by EXISTS. If your inner queryes return a lot of rows, it would be a lot more efficient. It depends if your subquery returns a lot of results.
SQL Server IN vs. EXISTS Performance

Program with lots of queries just stops, no exceptions

I'm running a program that is making a query for several thousand individuals. About 2/3 of the way through the list, it just stops...no exception, nothing. It just won't continue.
I'm not sure exactly what is going on here, why it just stops. I don't see anything wrong with the data (which would generate an exception anyway). Am I doing too many queries in a row?
Thanks in advance for any suggestions.
File inputFile = new File(datafile);
BufferedReader br = new BufferedReader(new FileReader(inputFile));
List <WRLine> empList = new ArrayList<>();
String s;
int counter = 0;
while ((s = br.readLine()) != null) {
String[] sLine = s.split(",");
if (sLine.length > 3) {
try {
//if it's a number, it's not a name. Skip the line.
int i = Integer.parseInt(sLine[0].trim());
} catch (Exception e) {
//if it's not a number and not blank, add it to the list
if (!sLine[2].equals("")) {
try {
int q = Integer.parseInt(sLine[2].trim());
WRLine wr = new WRLine(sLine[0], sLine[2], sLine[3]);
empList.add(wr);
} catch (Exception ex) {
//continue
}
}
}
}
}
//empList contains 1,998 items
Map<String, Integer> resultMap = new HashMap<>();
Iterator i = empList.iterator();
try {
String connectionURL = "jdbc:mysql://" + ip + ":" + port + "/" + dbName + "?user=" + userName + "&password=" + pw;
Class.forName("com.mysql.jdbc.Driver");
Connection conn = DriverManager.getConnection(connectionURL);
PreparedStatement ps = null;
ResultSet rs = null;
String query = "";
while (i.hasNext()) {
WRLine wr = (WRLine) i.next();
System.out.println("Searching " + wr.getName() + "...");
query = "Select count(*) as APPLIED from request where (requestDate like '%2017%' or requestDate like '%2018%') AND officer=(select id from officer where employeenumber=?)";
ps = conn.prepareStatement(query);
ps.setString(1, wr.getEmployeeNum());
rs = ps.executeQuery();
while (rs.next()) {
int queryResult = rs.getInt("APPLIED");
//if the division is already in there
if (resultMap.containsKey(wr.getDivision())) {
Integer tmp = resultMap.get(wr.getDivision());
tmp = tmp + queryResult;
resultMap.put(wr.getDivision(), tmp);
} else {
resultMap.put(wr.getDivision(), queryResult);
}
}
}
rs.close();
ps.close();
conn.close();
} catch (Exception e) {
e.printStackTrace();
}
//report by division

Summarizing what others have said in the comments, your problem could be due to improper JDBC resource handling. With Java 7 and above, you should use the try-with-resources statement, which frees resources automatically. Also, as of JDBC 4, you don't need to call Class.forName() explicitly. Finally, you should never prepare a PreparedStatement inside a loop when the only thing that changes is the bind variable.
Putting this together, the data access part could be rewritten as
String connectionURL = "jdbc:mysql://" + ip + ":" + port + "/" + dbName
+ "?user=" + userName + "&password=" + pw;
String query = "Select count(*) as APPLIED from request where "
+ "(requestDate like '%2017%' or requestDate like '%2018%') "
+ "AND officer=(select id from officer where employeenumber=?)";
try (Connection conn = DriverManager.getConnection(connectionURL);
PreparedStatement ps = conn.prepareStatement(query)) {
while (i.hasNext()) {
WRLine wr = (WRLine) i.next();
System.out.println("Searching " + wr.getName() + "...");
ps.setString(1, wr.getEmployeeNum());
// the result set is wrapped in its own try-with-resources
// so that it gets properly deallocated after reading
try (ResultSet rs = ps.executeQuery()) {
// SQL count is a scalar function so we can just use if instead of while
if (rs.next()) {
int queryResult = rs.getInt("APPLIED");
//if the division is already in there
if (resultMap.containsKey(wr.getDivision())) {
Integer tmp = resultMap.get(wr.getDivision());
tmp = tmp + queryResult;
resultMap.put(wr.getDivision(), tmp);
} else {
resultMap.put(wr.getDivision(), queryResult);
}
}
}
}
} catch (SQLException e) {
// consider wrapping as a RuntimeException and rethrowing instead of just logging
// because these are usually caused by
// programming errors or fatal problems with the DB
e.printStackTrace();
}

Resultset always empty when doing executeQuery

I am currently writing a simple Java app that reads information from an XLS file and then enters it in the database. Since that XLS does have duplicated records, I do a simple check if the entry in the XLS file already exists in the database. Here is my code:
public static void addResult(ArrayList<ArrayList<String>> listResults)
{
try
{
openDatabase();
stmt = c.createStatement();
for (int i = 0; i < listResults.size(); i++)
{
PreparedStatement stm = c.prepareStatement("SELECT player_name FROM results WHERE player_name=?;");
stm.setString(1, listResults.get(i).get(ReadResultsFile.NAME));
System.out.println(stm);
ResultSet rs = stm.executeQuery();
if (rs.getRow() <= 0)
{
String typeOfPlay = new String();
if (listResults.get(i).get(ReadResultsFile.TYPE).equals("Simple"))
{
typeOfPlay = "single";
}
else if (listResults.get(i).get(ReadResultsFile.TYPE).equals("Double"))
{
typeOfPlay = "double";
}
stm = c.prepareStatement("INSERT INTO results (player_name, school_id, " + typeOfPlay + ", tournament_id) "
+ "VALUES(?,?,?,?);");
stm.setString(1, listResults.get(i).get(ReadResultsFile.NAME));
stm.setString(2, listResults.get(i).get(ReadResultsFile.SCHOOL_ID));
stm.setInt(3, Integer.parseInt(listResults.get(i).get(ReadResultsFile.SCORE)));
stm.setString(4, "1");
stm.executeUpdate();
}
else
{
String typeOfPlay = new String();
if (listResults.get(i).get(ReadResultsFile.TYPE).equals("Simple"))
{
typeOfPlay = "single";
}
else if (listResults.get(i).get(ReadResultsFile.TYPE).equals("Double"))
{
typeOfPlay = "double";
}
stm = c.prepareStatement("UPDATE results SET " + typeOfPlay + "=? WHERE player_name=?;");
stm.setString(1, typeOfPlay);
stm.setString(2, listResults.get(i).get(ReadResultsFile.SCORE));
stm.setString(1, listResults.get(i).get(ReadResultsFile.NAME));
System.out.println(stm);
stm.executeUpdate();
}
}
closeDatabase();
}
catch (Exception e)
{
System.err.println(e.getClass().getName() + ": " + e.getMessage());
System.exit(0);
}
}
The problem that arises is that the rs.getRow() function always returns -1. I tried running the SELECT query directly in the database tool and the query returns the player_name column if there is already a similar entry existing. It unfortunately do the same in Java.
I am unsure what to do at this point.
Thank you for any hint!

getRow will not work as per the javadocs
Retrieves the current row number. The first row is number 1, the second number 2, and so on.
and
A ResultSet cursor is initially positioned before the first row; the
first call to the method next makes the first row the current row
Usually use
while (rs.next ()) {....

How to generalize resultset queries?

at the moment I'm working on a script that reads several values from different tables of one database. Every time I start a request, I have to open a statement and create a new resultset which leads to horrible, repetative code. What would be a good way of generalizing this and how can this be done?
Some elements from my code. At the moment there's just one statement and the closing has to be inserted. One of the primary reasons I ask this question.
public static void main(String[] args) throws Exception
{
Connection c = null;
Statement stmt = null;
try
{
//set up database connection
Class.forName("org.sqlite.JDBC");
c = DriverManager.getConnection("jdbc:sqlite:/nfs/home/mals/p/pu2002/workspace/Database2");
c.setAutoCommit(false);
stmt = c.createStatement();
//end
//get task id to work with
String Task_id = null;
if(args.length != 0) //if an argument was passed, Task_id will be the first element of the array args (arguments)
{
Task_id = args[0];
}
else if(args.length == 0) //if no arguments were passed, the highest number in the column id from tasks_task will be selected and set as Task_id
{
ResultSet TTask_id = stmt.executeQuery("SELECT max(id) FROM tasks_task");
int t_id = TTask_id.getInt(1);
Task_id = String.valueOf(t_id);
TTask_id.close();
}
//end
//get solution IDs from taks_ids
ArrayList<Integer> List_solIDs = new ArrayList<Integer>(); //create an empty array list
ResultSet SSolution_task_id = stmt.executeQuery("SELECT id FROM solutions_solution WHERE task_id ="+Task_id + " AND final = 1;"); //Sqlite3-Ausdruck SELECT..., Task IDs verändern pro Aufgabe - "SELECT * FROM solutions_solution where task_id ="+Task_id +";"
while (SSolution_task_id.next()) //loops through all elements of SSolution_task_id
{
List_solIDs.add(SSolution_task_id.getInt("id")); //adds all elements of the resultset SSolution_task_id to the list List_solIDs
}
SSolution_task_id.close();
//end
//get logs according to content type
int count = List_solIDs.size();
String log_javaBuilder = null;
List<String> log_JunitChecker = new ArrayList<String>();
for (int i = 0; i < count; i++)
{
boolean sol_id_valid = false;
String solID = String.valueOf(List_solIDs.get(i));
try
{
ResultSet AAttestation_sol_id = stmt.executeQuery("SELECT * FROM attestation_attestation WHERE solution_id =" +solID+";");
int Returned = AAttestation_sol_id.getInt("final_grade_id");
}
catch(Exception e)
{
sol_id_valid = true;
}
if(sol_id_valid ==true)
{
try
{
ResultSet CCresult_javaBuilder = stmt.executeQuery("SELECT log FROM checker_checkerresult WHERE solution_id = " +solID+ " AND content_type_id = 22;"); //"SELECT id FROM checker_checkerresult where solution_id = " +List_solIDs.get(i)+ ";"
log_javaBuilder = CCresult_javaBuilder.getString("log");
CCresult_javaBuilder.close();
ResultSet CCresult_Junit_checker = stmt.executeQuery("SELECT log FROM checker_checkerresult WHERE solution_id = " +solID+ " AND content_type_id = 24;");
while (CCresult_Junit_checker.next())
{
log_JunitChecker.add(CCresult_Junit_checker.getString("log"));
}
CCresult_Junit_checker.close();
}
catch (Exception e)
{
log_JunitChecker.add(null);
}
//end
All types of potential improvements will be welcome.
P.S.: Tried googling.

Seems you want to look at using some ORM layer e.g. http://hibernate.org/orm/
What you're looking for is probably a higher-level layer which
abstracts you from the underlying lower-level JDBC type of coding.

Better than writing generic method by yourself it is always better to use some framework, There are many JPA implementations out there which solve not only this issue but also takes care of multiple persistence layer boiler plate code. Start JPA from Here. You can also use Spring JDBC template as well to solve problem mentioned above Spring JDBC Documentation.
Now, if you really don't want any framework dependency and finish this code quite fast, You can define your own JDBCTemplate class which takes query and parameter map and return ResultSet. This class can handle open connection, query execution and closing connection etc.

What if you try to use generics on methods? this is a quick example, just for illustration, you must improve all this :)
resource: official docs
public static <T> List<T> getSingleValueList(ResultSet rs, Class<T> clazz, String colName) throws Exception {
ArrayList<T> list = new ArrayList<T>();
while (rs.next()) {//loops through all elements of generic list
list.add((T) rs.getObject(colName)); //adds all elements of the resultset rs to the list
}
rs.close();
return list;
}
public static <T> T getSingleValue(ResultSet rs, Class<T> clazz, String colName) throws Exception {
try {
if (rs.next()) {//loops through all elements of generic list
return (T) rs.getObject(colName);
} else {
throw new Exception("no value found.");
}
} finally {
rs.close();
}
}
public static void main(String[] args) throws Exception {
Connection c = null;
Statement stmt = null;
try {
//set up database connection
Class.forName("org.sqlite.JDBC");
c = DriverManager.getConnection("jdbc:sqlite:/nfs/home/mals/p/pu2002/workspace/Database2");
c.setAutoCommit(false);
stmt = c.createStatement();
//end
//get task id to work with
String Task_id = null;
if (args.length != 0) //if an argument was passed, Task_id will be the first element of the array args (arguments)
{
Task_id = args[0];
} else if (args.length == 0) //if no arguments were passed, the highest number in the column id from tasks_task will be selected and set as Task_id
{
ResultSet TTask_id = stmt.executeQuery("SELECT max(id) FROM tasks_task");
int t_id = TTask_id.getInt(1);
Task_id = String.valueOf(t_id);
TTask_id.close();
}
//end
//get solution IDs from taks_ids
ResultSet SSolution_task_id = stmt.executeQuery("SELECT id FROM solutions_solution WHERE task_id =" + Task_id + " AND final = 1;"); //Sqlite3-Ausdruck SELECT..., Task IDs verändern pro Aufgabe - "SELECT * FROM solutions_solution where task_id ="+Task_id +";"
List<Integer> List_solIDs = getSingleValueList(SSolution_task_id, Integer.class, "id"); //create an empty array list
//end
//get logs according to content type
int count = List_solIDs.size();
String log_javaBuilder = null;
List<String> log_JunitChecker = new ArrayList<String>();
List<String> tmplog_JunitChecker;
for (int i = 0; i < count; i++) {
boolean sol_id_valid = false;
String solID = String.valueOf(List_solIDs.get(i));
try {
ResultSet AAttestation_sol_id = stmt.executeQuery("SELECT * FROM attestation_attestation WHERE solution_id =" + solID + ";");
Integer Returned = getSingleValue(AAttestation_sol_id, Integer.class, "final_grade_id");
} catch (Exception e) {
sol_id_valid = true;
}
if (sol_id_valid == true) {
try {
ResultSet CCresult_javaBuilder = stmt.executeQuery("SELECT log FROM checker_checkerresult WHERE solution_id = " + solID + " AND content_type_id = 22;"); //"SELECT id FROM checker_checkerresult where solution_id = " +List_solIDs.get(i)+ ";"
log_javaBuilder = getSingleValue(CCresult_javaBuilder, String.class, "log");
ResultSet CCresult_Junit_checker = stmt.executeQuery("SELECT log FROM checker_checkerresult WHERE solution_id = " + solID + " AND content_type_id = 24;");
tmplog_JunitChecker = getSingleValueList(CCresult_Junit_checker, String.class, "log");
log_JunitChecker.addAll(tmplog_JunitChecker);
} catch (Exception e) {
log_JunitChecker.add(null);
}
//end
}
}
} catch (Exception eeee) {
//handle it
}
}
I hope I gave you a light.
Anyway, frameworks in almost all cases help a lot.

Result set returns 3 rows but i am only able to print 2?

The code below gets the information i require from my database but is not printing out all of the information. Firstly i know it is getting all of the correct information from the table because i have tried the query in sql developer.
public static void main(String[] args) {
Connection conn = null;
Statement stmt = null;
ResultSet rs = null;
try {
conn = getConnection();
String query = "SELECT menu.menu_id, menu_title, dish.dish_id, dish_name, dish_description, dish_price, menu.week_no "
+ "FROM menu, dish, menu_allocation "
+ "WHERE menu.active = '1' "
+ "AND menu.menu_id = menu_allocation.menu_id "
+ "AND dish.dish_id = menu_allocation.dish_id "
+ "AND menu.week_no IN (09, 10, 11)";
stmt = conn.createStatement();
rs = stmt.executeQuery(query);
MenuList list = null;
while (rs.next()) {
list = new MenuList(rs);
System.out.println(rs.getRow());
}
for (int pos = 0; pos < list.size(); pos++) {
Menu menu = list.getMenuAt(pos);
System.out.println(menu.getDescription());
}
} catch (Exception e) {
e.printStackTrace();
} finally {
try {
rs.close();
stmt.close();
conn.close();
} catch (SQLException e) {
}
}
}
The output from the terminal is as follows:
3 //Number of rows
Fish and Chips //3rd row
Chocolate Cake //2nd row
//Here should be 1st row
BUILD SUCCESSFUL (total time: 2 seconds)
Even though it says there are three rows it has only printed the two. Can anybody see if there is a problem with the above?

It's hard to be sure without seeing the code for the MenuList class but I don't think you need to loop over the ResultSet as MenuList does that for you.
As the MenuList constructor takes the ResultSet in rs as a parameter it probably loops over the ResultSet to create its entries. As you've already called rs.next() in the while of your loop the MenuList misses the first result.
I think you should replace all this:
MenuList list = null;
while (rs.next()) {
list = new MenuList(rs);
System.out.println(rs.getRow());
}
With:
MenuList list = new MenuList(rs);

I would suggest you use a debugger so you understand what your progam is doing.
You appear to be only keeping the last row loaded, so while you have 3 rows, you only keep the last. It appears you are getting two values from the last row.

public static void main(String[] args) {
Connection conn = null;
Statement stmt = null;
ResultSet rs = null;
try {
conn = getConnection();
String query = "SELECT menu.menu_id, menu_title, dish.dish_id, dish_name, dish_description, dish_price, menu.week_no "
+ "FROM menu, dish, menu_allocation "
+ "WHERE menu.active = '1' "
+ "AND menu.menu_id = menu_allocation.menu_id "
+ "AND dish.dish_id = menu_allocation.dish_id "
+ "AND menu.week_no IN (09, 10, 11)";
stmt = conn.createStatement();
rs = stmt.executeQuery(query);
MenuList[3] list = null;
int idx = 0; //Add index
while (rs.next()) {
list[idx] = new MenuList(rs); //use index
idx++; //increment index
System.out.println(rs.getRow());
}
for (int pos = 0; pos < list.size(); pos++) {
Menu menu = list.getMenuAt(pos);//Don't know that
//get menu by index
System.out.println(menu.getDescription());
}
} catch (Exception e) {
e.printStackTrace();
} finally {
try {
rs.close();
stmt.close();
conn.close();
} catch (SQLException e) {
}
}
}