Develop Reference

Program with lots of queries just stops, no exceptions

I'm running a program that is making a query for several thousand individuals. About 2/3 of the way through the list, it just stops...no exception, nothing. It just won't continue.
I'm not sure exactly what is going on here, why it just stops. I don't see anything wrong with the data (which would generate an exception anyway). Am I doing too many queries in a row?
Thanks in advance for any suggestions.
File inputFile = new File(datafile);
BufferedReader br = new BufferedReader(new FileReader(inputFile));
List <WRLine> empList = new ArrayList<>();
String s;
int counter = 0;
while ((s = br.readLine()) != null) {
String[] sLine = s.split(",");
if (sLine.length > 3) {
try {
//if it's a number, it's not a name. Skip the line.
int i = Integer.parseInt(sLine[0].trim());
} catch (Exception e) {
//if it's not a number and not blank, add it to the list
if (!sLine[2].equals("")) {
try {
int q = Integer.parseInt(sLine[2].trim());
WRLine wr = new WRLine(sLine[0], sLine[2], sLine[3]);
empList.add(wr);
} catch (Exception ex) {
//continue
}
}
}
}
}
//empList contains 1,998 items
Map<String, Integer> resultMap = new HashMap<>();
Iterator i = empList.iterator();
try {
String connectionURL = "jdbc:mysql://" + ip + ":" + port + "/" + dbName + "?user=" + userName + "&password=" + pw;
Class.forName("com.mysql.jdbc.Driver");
Connection conn = DriverManager.getConnection(connectionURL);
PreparedStatement ps = null;
ResultSet rs = null;
String query = "";
while (i.hasNext()) {
WRLine wr = (WRLine) i.next();
System.out.println("Searching " + wr.getName() + "...");
query = "Select count(*) as APPLIED from request where (requestDate like '%2017%' or requestDate like '%2018%') AND officer=(select id from officer where employeenumber=?)";
ps = conn.prepareStatement(query);
ps.setString(1, wr.getEmployeeNum());
rs = ps.executeQuery();
while (rs.next()) {
int queryResult = rs.getInt("APPLIED");
//if the division is already in there
if (resultMap.containsKey(wr.getDivision())) {
Integer tmp = resultMap.get(wr.getDivision());
tmp = tmp + queryResult;
resultMap.put(wr.getDivision(), tmp);
} else {
resultMap.put(wr.getDivision(), queryResult);
}
}
}
rs.close();
ps.close();
conn.close();
} catch (Exception e) {
e.printStackTrace();
}
//report by division

Summarizing what others have said in the comments, your problem could be due to improper JDBC resource handling. With Java 7 and above, you should use the try-with-resources statement, which frees resources automatically. Also, as of JDBC 4, you don't need to call Class.forName() explicitly. Finally, you should never prepare a PreparedStatement inside a loop when the only thing that changes is the bind variable.
Putting this together, the data access part could be rewritten as
String connectionURL = "jdbc:mysql://" + ip + ":" + port + "/" + dbName
+ "?user=" + userName + "&password=" + pw;
String query = "Select count(*) as APPLIED from request where "
+ "(requestDate like '%2017%' or requestDate like '%2018%') "
+ "AND officer=(select id from officer where employeenumber=?)";
try (Connection conn = DriverManager.getConnection(connectionURL);
PreparedStatement ps = conn.prepareStatement(query)) {
while (i.hasNext()) {
WRLine wr = (WRLine) i.next();
System.out.println("Searching " + wr.getName() + "...");
ps.setString(1, wr.getEmployeeNum());
// the result set is wrapped in its own try-with-resources
// so that it gets properly deallocated after reading
try (ResultSet rs = ps.executeQuery()) {
// SQL count is a scalar function so we can just use if instead of while
if (rs.next()) {
int queryResult = rs.getInt("APPLIED");
//if the division is already in there
if (resultMap.containsKey(wr.getDivision())) {
Integer tmp = resultMap.get(wr.getDivision());
tmp = tmp + queryResult;
resultMap.put(wr.getDivision(), tmp);
} else {
resultMap.put(wr.getDivision(), queryResult);
}
}
}
}
} catch (SQLException e) {
// consider wrapping as a RuntimeException and rethrowing instead of just logging
// because these are usually caused by
// programming errors or fatal problems with the DB
e.printStackTrace();
}

Get a result from First SQL Query to use it at Second SQL Query which is inside First

How to make this one work? Obviously I don't know some very basic staff about SQL queries inside other SQL queries in Java but searching around didn't help!
Thank you in advance
try (Connection con = L2DatabaseFactory.getInstance().getConnection())
{
PreparedStatement stm = con.prepareStatement("SELECT count,owner_id FROM items WHERE item_id=57 order by count desc limit 10");
ResultSet rSet = stm.executeQuery();
while (rSet.next())
{
int owner_id = rSet.getInt("owner_id");
int count = rSet.getInt("count");
if (count == 0)
{
continue;
}
PreparedStatement stm1 = con.prepareStatement("SELECT char_name,accesslevel,online FROM characters WHERE obj_Id=" + owner_id);
ResultSet rSet1 = stm1.executeQuery();
while (rSet1.next())
{
int accessLevel = rSet.getInt("accesslevel");
if (accessLevel > 0)
{
continue;
}
String pl = rSet.getString("char_name");
int online = rSet.getInt("online");
String status = online == 1 ? "<font color=\"00FF00\">Online</font>" : "<font color=\"FF0000\">Offline</font>";
sb.append("<tr><td>"+ pl +"</td><td>"+ count +"</td><td>"+ status +"</td></tr>");
}
}
}
catch (Exception e)
{
_log.log(Level.SEVERE, "Error", e);
}

It looks like you are trying to join two tables using Java code. This is not such a great idea and not good for performance. Let the database do the joins for you - it is an expert at that. Do not code "inner joins" in Java.
Apart from that: the prepared statements are not being closed and this will sooner or later cause you trouble with OS resources.
My suggestion would be to create one single query with an inner join or a select in statement and also close all prepared statements using try with resources. Something along these lines:
private String test() throws SQLException {
StringBuilder sb = new StringBuilder();
int count = 0;
try (Connection con = L2DatabaseFactory.getInstance().getConnection()) {
try (PreparedStatement stm1 = con.prepareStatement(
"SELECT char_name,accesslevel,online FROM characters WHERE obj_Id in (SELECT owner_id FROM items WHERE item_id=57 order by count desc limit 10)")) {
ResultSet rSet = stm1.executeQuery();
while (rSet.next()) {
count++;
int accessLevel = rSet.getInt("accesslevel");
if (accessLevel > 0) {
continue;
}
String pl = rSet.getString("char_name");
int online = rSet.getInt("online");
String status = online == 1 ? "<font color=\"00FF00\">Online</font>" : "<font color=\"FF0000\">Offline</font>";
sb.append("<tr><td>" + pl + "</td><td>" + count + "</td><td>" + status + "</td></tr>");
}
}
} catch (Exception e) {
Logger.getLogger("test").log(Level.SEVERE, "Error", e);
}
return sb.toString();
}

How to pass multiple parameters to a PreparedStatement in java?

I am using the below code to retrive the order data from db2 and it works fine when i am passing only the BranchNumber and used the getWildcards() function since sometime i am passing multiple branch numbers .
public List<Order> getallorders(List<Branch> BranchNumber) throws SQLException {
List<Order> orders = new ArrayList<Order>();
try {
StringBuilder sb = new StringBuilder();
sb.append("SELECT ORDER_NUMBER as ordernumber,SERVICE_TYPE as service"
+ "FROM ORDER WHERE "
+ "BRANCH IN(");
sb.append(getWildCards(BranchNumber.size())).append(")").append(" WITH UR");
String query = sb.toString();
PreparedStatement statement = connection.prepareStatement(query);
for(int i=0 ; i<BranchNumber.size() ;i++)
{
statement.setInt(i+1,BranchNumber.get(i).getBranch());
}
ResultSet resultSet = statement.executeQuery();
{
while (resultSet .next()) {
Order order1 = new Order();
order1.setOrdernumber(resultSet.getInt("ordernumber"));
orders.add(order1);
}
}
}
catch (SQLException e) {
e.printStackTrace();
}
return orders;
}
private String getWildCards(int size) {
// TODO Auto-generated method stub
StringBuilder sb = new StringBuilder();
for(int i =0 ; i<size ; i++)
{
sb = (i == 0) ? sb.append("?")
: sb.append(",").append("?");
}
return sb.toString();
}
Now i need to pass the startDate and endDate inside the function to retrieve the data but the preparedstatement is not formatting the select query with the passed value .
public List<Order> getallorders(List<Branch> BranchNumber,String startDate,String endDate) throws SQLException {
List<Order> orders = new ArrayList<Order>();
try {
StringBuilder sb = new StringBuilder();
sb.append("SELECT ORDER_NUMBER as ordernumber,SERVICE as service"
+ "FROM ORDER WHERE "
+ "BRANCH IN(");
sb.append(getWildCards(BranchNumber.size())).append(")");
sb.append("AND ORDERDATE BETWEEN ? and ? WITH UR");
String query = sb.toString();
PreparedStatement statement =
connection.prepareStatement(query);
for(int i=0 ; i<BranchNumber.size() ;i++)
{
statement.setInt(i+1,BranchNumber.get(i).getBranch());
}
ResultSet resultSet = statement.executeQuery();
{
while (resultSet .next()) {
Order order1 = new Order();
order1.setOrdernumber(resultSet.getInt("ordernumber"));
orders.add(order1);
}
}
}
catch (SQLException e) {
e.printStackTrace();
}
return orders;
}
Can someone please explain me what i am doing wrong here and how i can get the expected preparedstatement,below is the formatted query coming in my log and error message recorded,
SELECT ORDER_NUMBER as ordernumber,SERVICE_TYPE as service FROM .ORDER WHERE
BRANCH_NUMBER IN(?) + AND ORDERDATE BETWEEN ? AND ? WITH UR
com.ibm.db2.jcc.am.SqlSyntaxErrorException: DB2 SQL Error: SQLCODE=-104,
SQLSTATE=42601, SQLERRMC=ORDER DATE BETWEEN ? AND;H_NUMBER IN(?) + AND;
<order_siblings_by>, DRIVER=3.63.75
at com.ibm.db2.jcc.am.fd.a(fd.java:679)

Each ? in the PrepareStatement should be assigned a value. Here is an example adopted from here :
String updateString =
"update " + dbName + ".COFFEES " +
"set SALES = ? where COF_NAME = ?";
PreparedStatement updateSales = con.prepareStatement(updateString);
updateSales.setInt(1, 500); //set value to first `?`
updateSales.setString(2, "roasted"); //set value to second `?`

How to generalize resultset queries?

at the moment I'm working on a script that reads several values from different tables of one database. Every time I start a request, I have to open a statement and create a new resultset which leads to horrible, repetative code. What would be a good way of generalizing this and how can this be done?
Some elements from my code. At the moment there's just one statement and the closing has to be inserted. One of the primary reasons I ask this question.
public static void main(String[] args) throws Exception
{
Connection c = null;
Statement stmt = null;
try
{
//set up database connection
Class.forName("org.sqlite.JDBC");
c = DriverManager.getConnection("jdbc:sqlite:/nfs/home/mals/p/pu2002/workspace/Database2");
c.setAutoCommit(false);
stmt = c.createStatement();
//end
//get task id to work with
String Task_id = null;
if(args.length != 0) //if an argument was passed, Task_id will be the first element of the array args (arguments)
{
Task_id = args[0];
}
else if(args.length == 0) //if no arguments were passed, the highest number in the column id from tasks_task will be selected and set as Task_id
{
ResultSet TTask_id = stmt.executeQuery("SELECT max(id) FROM tasks_task");
int t_id = TTask_id.getInt(1);
Task_id = String.valueOf(t_id);
TTask_id.close();
}
//end
//get solution IDs from taks_ids
ArrayList<Integer> List_solIDs = new ArrayList<Integer>(); //create an empty array list
ResultSet SSolution_task_id = stmt.executeQuery("SELECT id FROM solutions_solution WHERE task_id ="+Task_id + " AND final = 1;"); //Sqlite3-Ausdruck SELECT..., Task IDs verändern pro Aufgabe - "SELECT * FROM solutions_solution where task_id ="+Task_id +";"
while (SSolution_task_id.next()) //loops through all elements of SSolution_task_id
{
List_solIDs.add(SSolution_task_id.getInt("id")); //adds all elements of the resultset SSolution_task_id to the list List_solIDs
}
SSolution_task_id.close();
//end
//get logs according to content type
int count = List_solIDs.size();
String log_javaBuilder = null;
List<String> log_JunitChecker = new ArrayList<String>();
for (int i = 0; i < count; i++)
{
boolean sol_id_valid = false;
String solID = String.valueOf(List_solIDs.get(i));
try
{
ResultSet AAttestation_sol_id = stmt.executeQuery("SELECT * FROM attestation_attestation WHERE solution_id =" +solID+";");
int Returned = AAttestation_sol_id.getInt("final_grade_id");
}
catch(Exception e)
{
sol_id_valid = true;
}
if(sol_id_valid ==true)
{
try
{
ResultSet CCresult_javaBuilder = stmt.executeQuery("SELECT log FROM checker_checkerresult WHERE solution_id = " +solID+ " AND content_type_id = 22;"); //"SELECT id FROM checker_checkerresult where solution_id = " +List_solIDs.get(i)+ ";"
log_javaBuilder = CCresult_javaBuilder.getString("log");
CCresult_javaBuilder.close();
ResultSet CCresult_Junit_checker = stmt.executeQuery("SELECT log FROM checker_checkerresult WHERE solution_id = " +solID+ " AND content_type_id = 24;");
while (CCresult_Junit_checker.next())
{
log_JunitChecker.add(CCresult_Junit_checker.getString("log"));
}
CCresult_Junit_checker.close();
}
catch (Exception e)
{
log_JunitChecker.add(null);
}
//end
All types of potential improvements will be welcome.
P.S.: Tried googling.

Seems you want to look at using some ORM layer e.g. http://hibernate.org/orm/
What you're looking for is probably a higher-level layer which
abstracts you from the underlying lower-level JDBC type of coding.

Better than writing generic method by yourself it is always better to use some framework, There are many JPA implementations out there which solve not only this issue but also takes care of multiple persistence layer boiler plate code. Start JPA from Here. You can also use Spring JDBC template as well to solve problem mentioned above Spring JDBC Documentation.
Now, if you really don't want any framework dependency and finish this code quite fast, You can define your own JDBCTemplate class which takes query and parameter map and return ResultSet. This class can handle open connection, query execution and closing connection etc.

What if you try to use generics on methods? this is a quick example, just for illustration, you must improve all this :)
resource: official docs
public static <T> List<T> getSingleValueList(ResultSet rs, Class<T> clazz, String colName) throws Exception {
ArrayList<T> list = new ArrayList<T>();
while (rs.next()) {//loops through all elements of generic list
list.add((T) rs.getObject(colName)); //adds all elements of the resultset rs to the list
}
rs.close();
return list;
}
public static <T> T getSingleValue(ResultSet rs, Class<T> clazz, String colName) throws Exception {
try {
if (rs.next()) {//loops through all elements of generic list
return (T) rs.getObject(colName);
} else {
throw new Exception("no value found.");
}
} finally {
rs.close();
}
}
public static void main(String[] args) throws Exception {
Connection c = null;
Statement stmt = null;
try {
//set up database connection
Class.forName("org.sqlite.JDBC");
c = DriverManager.getConnection("jdbc:sqlite:/nfs/home/mals/p/pu2002/workspace/Database2");
c.setAutoCommit(false);
stmt = c.createStatement();
//end
//get task id to work with
String Task_id = null;
if (args.length != 0) //if an argument was passed, Task_id will be the first element of the array args (arguments)
{
Task_id = args[0];
} else if (args.length == 0) //if no arguments were passed, the highest number in the column id from tasks_task will be selected and set as Task_id
{
ResultSet TTask_id = stmt.executeQuery("SELECT max(id) FROM tasks_task");
int t_id = TTask_id.getInt(1);
Task_id = String.valueOf(t_id);
TTask_id.close();
}
//end
//get solution IDs from taks_ids
ResultSet SSolution_task_id = stmt.executeQuery("SELECT id FROM solutions_solution WHERE task_id =" + Task_id + " AND final = 1;"); //Sqlite3-Ausdruck SELECT..., Task IDs verändern pro Aufgabe - "SELECT * FROM solutions_solution where task_id ="+Task_id +";"
List<Integer> List_solIDs = getSingleValueList(SSolution_task_id, Integer.class, "id"); //create an empty array list
//end
//get logs according to content type
int count = List_solIDs.size();
String log_javaBuilder = null;
List<String> log_JunitChecker = new ArrayList<String>();
List<String> tmplog_JunitChecker;
for (int i = 0; i < count; i++) {
boolean sol_id_valid = false;
String solID = String.valueOf(List_solIDs.get(i));
try {
ResultSet AAttestation_sol_id = stmt.executeQuery("SELECT * FROM attestation_attestation WHERE solution_id =" + solID + ";");
Integer Returned = getSingleValue(AAttestation_sol_id, Integer.class, "final_grade_id");
} catch (Exception e) {
sol_id_valid = true;
}
if (sol_id_valid == true) {
try {
ResultSet CCresult_javaBuilder = stmt.executeQuery("SELECT log FROM checker_checkerresult WHERE solution_id = " + solID + " AND content_type_id = 22;"); //"SELECT id FROM checker_checkerresult where solution_id = " +List_solIDs.get(i)+ ";"
log_javaBuilder = getSingleValue(CCresult_javaBuilder, String.class, "log");
ResultSet CCresult_Junit_checker = stmt.executeQuery("SELECT log FROM checker_checkerresult WHERE solution_id = " + solID + " AND content_type_id = 24;");
tmplog_JunitChecker = getSingleValueList(CCresult_Junit_checker, String.class, "log");
log_JunitChecker.addAll(tmplog_JunitChecker);
} catch (Exception e) {
log_JunitChecker.add(null);
}
//end
}
}
} catch (Exception eeee) {
//handle it
}
}
I hope I gave you a light.
Anyway, frameworks in almost all cases help a lot.

Result set returns 3 rows but i am only able to print 2?

The code below gets the information i require from my database but is not printing out all of the information. Firstly i know it is getting all of the correct information from the table because i have tried the query in sql developer.
public static void main(String[] args) {
Connection conn = null;
Statement stmt = null;
ResultSet rs = null;
try {
conn = getConnection();
String query = "SELECT menu.menu_id, menu_title, dish.dish_id, dish_name, dish_description, dish_price, menu.week_no "
+ "FROM menu, dish, menu_allocation "
+ "WHERE menu.active = '1' "
+ "AND menu.menu_id = menu_allocation.menu_id "
+ "AND dish.dish_id = menu_allocation.dish_id "
+ "AND menu.week_no IN (09, 10, 11)";
stmt = conn.createStatement();
rs = stmt.executeQuery(query);
MenuList list = null;
while (rs.next()) {
list = new MenuList(rs);
System.out.println(rs.getRow());
}
for (int pos = 0; pos < list.size(); pos++) {
Menu menu = list.getMenuAt(pos);
System.out.println(menu.getDescription());
}
} catch (Exception e) {
e.printStackTrace();
} finally {
try {
rs.close();
stmt.close();
conn.close();
} catch (SQLException e) {
}
}
}
The output from the terminal is as follows:
3 //Number of rows
Fish and Chips //3rd row
Chocolate Cake //2nd row
//Here should be 1st row
BUILD SUCCESSFUL (total time: 2 seconds)
Even though it says there are three rows it has only printed the two. Can anybody see if there is a problem with the above?

It's hard to be sure without seeing the code for the MenuList class but I don't think you need to loop over the ResultSet as MenuList does that for you.
As the MenuList constructor takes the ResultSet in rs as a parameter it probably loops over the ResultSet to create its entries. As you've already called rs.next() in the while of your loop the MenuList misses the first result.
I think you should replace all this:
MenuList list = null;
while (rs.next()) {
list = new MenuList(rs);
System.out.println(rs.getRow());
}
With:
MenuList list = new MenuList(rs);

I would suggest you use a debugger so you understand what your progam is doing.
You appear to be only keeping the last row loaded, so while you have 3 rows, you only keep the last. It appears you are getting two values from the last row.

public static void main(String[] args) {
Connection conn = null;
Statement stmt = null;
ResultSet rs = null;
try {
conn = getConnection();
String query = "SELECT menu.menu_id, menu_title, dish.dish_id, dish_name, dish_description, dish_price, menu.week_no "
+ "FROM menu, dish, menu_allocation "
+ "WHERE menu.active = '1' "
+ "AND menu.menu_id = menu_allocation.menu_id "
+ "AND dish.dish_id = menu_allocation.dish_id "
+ "AND menu.week_no IN (09, 10, 11)";
stmt = conn.createStatement();
rs = stmt.executeQuery(query);
MenuList[3] list = null;
int idx = 0; //Add index
while (rs.next()) {
list[idx] = new MenuList(rs); //use index
idx++; //increment index
System.out.println(rs.getRow());
}
for (int pos = 0; pos < list.size(); pos++) {
Menu menu = list.getMenuAt(pos);//Don't know that
//get menu by index
System.out.println(menu.getDescription());
}
} catch (Exception e) {
e.printStackTrace();
} finally {
try {
rs.close();
stmt.close();
conn.close();
} catch (SQLException e) {
}
}
}