I had a very sophisticated class that performed DB queries, the problem is it wasnt using try-with-resource statements so i had to .close() manually. To be safer, I tried to re-design it with try-with-resource. My question is if these resources will close properly given how I'm referencing them outside the objects containing those resources. For example, this class DBQuery i use to create queries and resources related to those queries
public class DBQuery {
private String _query;
private PreparedStatement _stmt;
private ResultSet _rs;
// constructor
public DBQuery (String query) {
_query = query;
}
public PreparedStatement execPreparedStatement() throws SQLException {
_stmt = DB.getCon().prepareStatement(_query, Statement.RETURN_GENERATED_KEYS);
return _stmt;
}
public ResultSet getRecordSet() throws SQLException {
_rs = _stmt.executeQuery();
return _rs;
}
public void setInt(int paramNum, int setVal) throws SQLException {
_stmt.setInt(paramNum, setVal);
}
public void setString(int paramNum, String setVal) throws SQLException {
_stmt.setString(paramNum, setVal);
}
}
Then this would be example usage of the class. loadActiveCompany given a companyId retrieves the company from the database and creates some objects. My question is two-fold:
will the resources close properly when loadActiveCompany completes.
is there any problem with how I'm using the try-catch blocks.
Thank you
// loads the active company into the view
public void loadActiveCompany(int companyId) {
boolean loadFailed = false;
// we are passed the company id
_activeCompany.setCompanyId(companyId);
DBQuery qComps = new DBQuery("SELECT comp_name FROM comps WHERE id=?");
try ( PreparedStatement stmtComps = qComps.execPreparedStatement() ) {
DB.getCon().rollback();
qComps.setInt(1, companyId);
try ( ResultSet rsComps = qComps.getRecordSet() ) {
rsComps.next();
String companyName = rsComps.getString("comp_name");
_activeCompany.setCompanyName(companyName);
}
} catch (SQLException e) {
System.out.println("Couldn't find company!");
loadFailed = true;
} finally {
if (loadFailed)
return;
}
DBQuery qGroups = new DBQuery("SELECT id, group_name FROM comps_groups WHERE comp_id=? ORDER BY sort_order ASC");
try ( PreparedStatement stmtGroups = qGroups.execPreparedStatement() ) {
qGroups.setInt(1, companyId);
try ( ResultSet rsGroups = qGroups.getRecordSet() ) {
while (rsGroups.next()) {
int groupId = rsGroups.getInt("id");
Group thisGroup = new Group();
thisGroup.setGroupId(groupId);
thisGroup.setGroupName(rsGroups.getString("group_name"));
DBQuery qAnchors = new DBQuery("SELECT id, anchor_name, anchor_type FROM comps_groups_anchors WHERE group_id=? ORDER BY sort_order ASC");
try ( PreparedStatement stmtAnchors = qAnchors.execPreparedStatement() ) {
qAnchors.setInt(1, groupId);
try ( ResultSet rsAnchors = qAnchors.getRecordSet() ) {
while (rsAnchors.next()) {
int anchorId = rsAnchors.getInt("id");
Anchor thisAnchor = new Anchor();
thisAnchor.setAnchorId(anchorId);
thisAnchor.setAnchorName(qAnchors.getRS().getString("anchor_name"));
thisAnchor.setAnchorType(qAnchors.getRS().getInt("anchor_type"));
thisGroup.addGroupAnchor(thisAnchor);
}
}
}
_activeCompany.getCompanyGroups().add(thisGroup);
}
}
DB.getCon().commit();
} catch (SQLException e) {
System.out.println("Could not load company!");
loadFailed = true;
} finally {
if (loadFailed)
return;
}
// print out the active company
_activeCompany.printStatus();
}
Short answers:
Yes, the PreparedStatements are all closed.
Not directly problems, but easier ways.
Here my BUTs:
The name execPreparedStatement is totally misleading, as it is not (in DB terms) executing anything, just creating the PreparedStatement. A better name would be createPreparedStatement or - lol - preparePreparedStatement
Why do you call DB.getCon().rollback();? I do not think this will lead to someplace good...
the way you use DBQuery at the moment, it will only bring pain. Basicall this is just a container that saves additional infos (_stmt + _rs) which makes it SEVERELY state and sequence dependent, prone to hit you with lots of NPEs
so the actions you call on DBQuery you could simply also call on the PreparedStatement, reducing complexity and taking away a few pitfalls
so either completely remove DBQuery
or remodel the DBQuery
to be Closeable/AutoCloseable,
add some checks to the other functions,
create the PreparedStatement right away (CTOR, query string as CTOR parameter),
keep it private, do not expose it
use it inside getRecordSet,
do not store any other references unless you REALLY need them
and in the close method close the PS,
Your loadFailed = true; and if (loadFailed) return; seems overly convoluted and error-prone. Why not directly call return; right where you currently have loadFailed = true; lines?
I would - personal preference - put those 2 whole try-catch blocks into their own methods, signaling failure with a boolean or something => more methods with each less code and better variable scope (for example no re-use of loadFailed, but better re-usability of the two methods)
You actually do NOT need the inner try-resource on the ResultSets, but it's good if you (can) keep em. Just be careful there, as closing a ResultSet might have an impact on its creator PreparedStatement. So if you test it (in a situation where you re-use the PreparedStatement, that what it's actually made for) and get a 'closed' Exception when reusing the PreparedStatement, then you remove the try-resource blocks around the ResultSets.
Related
I am having a problem with a ResultSet being closed. What confuses me is that it works for a portion of the data and then closes. At first I thought it might be because of connection timeout but that doesn't seem the case.
This portion of the program pertains to comparing an .xlsx workbook to an already present SQL database and for lack of a better term merges/updates it.
First, in my CompareDatabase class I am calling a search function that searches an SQLite database for a specific string every 6 iterations.
int columnCount = 6;
dataPoint = dataPoint.replaceAll("Detail", "");
String[] temp = dataPoint.trim().split("\\s+");
System.out.println(Arrays.toString(temp));
for (String tempDataPoint : temp) {
if ( columnCount == 6) {
System.out.println(search(tempDataPoint, connection));
}
columnCount = 0;
} else {
columnCount++;
}
}
This search function (also in the CompareDatabase class is then supposed to search for the value and return a String (was originally a Boolean but I wanted to see the output).
private String search (String searchValue, Connection connection) throws SQLException {
PreparedStatement pStatement = null;
pStatement = connection.prepareStatement("SELECT * FROM lotdatabase where (Vehicle) = (?)");
pStatement.setString(1, searchValue);
try (ResultSet resultSet = pStatement.executeQuery()){
return resultSet.getString(1);
}finally {
close(pStatement);
}
}
At the end you can see that the PreparedStatement is closed. The ResultSet should also be closed automatically (I read somewhere) but JDBC could possibly be being unreliable.
The Connection however is still open as it will be searching some 200+ strings and opening and closing that many times did not seem like a good idea.
These functions are called by my main class here:
One is commented out since it will error out because of primary key violation.
public static void main(String[] args) {
SQLDatabase sqlDatabase = new SQLDatabase();
//sqlDatabase.convertToSQL("Database1.xlsx");
sqlDatabase.compare("Database2.xlsx");
}
I have a suspicion that I am going about a bunch of this wrong (on the aspect of managing connections an such) and I would appreciate a reference to where I can learn to do it properly.
Also, being that PreparedStatement can only handle one ResultSet I don't see that being my issue since I close it every iteration in the for loop.
If more code or explanation is required please let me know and I will do my best to assist.
Thank you for taking the time to read this.
So after a bit more Googling and sleeping on it here is what worked for me.
The search function in compareDatabase changed to this:
private Boolean search (String searchValue, Connection connection) {
PreparedStatement ps = null;
try {
ps = connection.prepareStatement("SELECT * FROM lotdatabase where " +
"(Vehicle) = (?)");
ps.setString(1, searchValue);
ResultSet resultSet = ps.executeQuery();
//The following if statement checks if the ResultSet is empty.
if (!resultSet.next()){
resultSet.close();
ps.close();
return false;
}else{
resultSet.close();
ps.close();
return true;
}
} catch (SQLException e) {
e.printStackTrace();
}
return false;
}
And in the other function within compareDatabase I call the search function like this:
if (search(tempDataPoint, connection)) {
System.out.println("MATCH FOUND: " + tempDataPoint);
}else {
System.out.println("NOT FOUND: " + tempDataPoint);
}
This allows me to check the ResultSet and also be sure that it is closed.
try {
Connection lig = DriverManager.getConnection(
"jdbc:mysql://localhost/gym", "root", "0000");
PreparedStatement inst = lig
.prepareStatement("SELECT * FROM produtos_has_historico WHERE Produtos_idProdutos AND Historico_idHistorico");
ResultSet a = inst.executeQuery();
while (a.next()){
DefaultListModel model = new DefaultListModel();
model.addElement(a);
}
} catch( Exception e ){}
In this Select, I get the id historic and the id product, but I wanted to get also the name and price of the products that are in my other table "products" to add to my Jlist, can I use two selects ? Thank you.
It's not a problem to have two selects in a single try-catch block.
If you want to handle the generated exception the same way regardless of where it occurred, then I don't see a problem with enclosing multiple statements in the same try block.
Yes, it's fine to put multiple calls that may fail into a single try/catch block — in fact, that's a big part of why we have try/catch blocks (and exception handling in general), to move the code that handles exceptional things (failures) out of the way of the main code, so it's easier to understand the main code.
Compare these bits of pseudo-code
result = doThis();
if (result != success) {
handleTheFailure(result)
} else {
result = doThat();
if (result != success) {
handleThefailure(result);
} else {
result = doTheOther();
if (result != success) {
handleTheFailure(result);
}
}
}
vs.
try {
doThis();
doThat();
doTheOther();
}
catch (failure) {
handleFailure(failure);
}
More in the Java tutorial on exceptions.
No problem. I also recommend you use try-with-resources instead and be more specific about the exceptions you want to catch:
try (Connection conn = DriverManager.getConnection("...")) {
PreparedStatement ps1 = conn.prepareStatement("...");
try (ResultSet rs1 = ps1.executeQuery()) {
/* Parse first result */
}
PreparedStatement ps2 = conn.prepareStatement("...");
try (ResultSet rs2 = ps2.executeQuery()) {
/* Parse second result */
}
} catch (SQLException ex) {
for (Throwable t : ex) {
t.printStackTrace();
}
}
Instead of going for two selects - you can write a SQL query having a join in it .
You will be able to perform the join if you have common columns in your tables.
Then you can write a query using join and then the same code you can use to perform the operations and you can fetch all the values that you want.
For SQL joins basic - you can visit this site -
http://www.w3schools.com/sql/sql_join.asp
I have a thread that executes and updates a database with some values. I commented out all the operations done to the data and just left the lines you can see below. While running the program with the lines you see bellow I get a memory leak.
This is what it looks like in VisualVM
Mysql Class
public ResultSet executeQuery(String Query) throws SQLException {
statement = this.connection.createStatement();
resultSet = statement.executeQuery(Query);
return resultSet;
}
public void executeUpdate(String Query) throws SQLException {
Statement tmpStatement = this.connection.createStatement();
tmpStatement.executeUpdate(Query);
tmpStatement.close();
tmpStatement=null;
}
Thread file
public void run() {
ResultSet results;
String query;
int id;
String IP;
int port;
String SearchTerm;
int sleepTime;
while (true) {
try {
query = "SELECT * FROM example WHERE a='0'";
results = this.database.executeQuery(query);
while (results.next()) {
id = results.getInt("id");
query = "UPDATE example SET a='1' WHERE id='"
+ id + "'";
SearchTerm=null;
this.database.executeUpdate(query);
}
results.close();
results = null;
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
The problem has happened with many other people after researching the web,
https://forum.hibernate.org/viewtopic.php?f=1&t=987128
Is bone cp or mysql.jdbc.JDBC4Connection known for leaking?
and a few more if you google "jdbc4resultset memory leak"
The leak is yours, not MySQL's. You aren't closing the statement.
The design of your method is poor. It should close the statement, and it should return something that will survive closing of the statement, such as a CachedRowSet.
Or else it should not exist at all. It's only three lines, and it doesn't support query parameters, so it isn't really much use. I would just delete it.
You also appear to have statement as an instance member, which is rarely if ever correct. It should be local to the method. At present your code isn't even thread-safe.
You should also be closing the ResultSet in a finally block to ensure it gets closed. Ditto the Statement.
Make sure that you are explicitly closing the database connections.
I've started creating a toDoList and I like to create a "DataMapper" to fire queries to my Database.
I created this Datamapper to handle things for me but I don't know if my way of thinking is correct in this case. In my Datamapper I have created only 1 method that has to execute the queries and several methods that know what query to fire (to minimalize the open and close methods).
For example I have this:
public Object insertItem(String value) {
this.value = value;
String insertQuery = "INSERT INTO toDoList(item,datum) " + "VALUES ('" + value + "', CURDATE())";
return this.executeQuery(insertQuery);
}
public Object removeItem(int id) {
this.itemId = id;
String deleteQuery = "DELETE FROM test WHERE id ='" + itemId + "'";
return this.executeQuery(deleteQuery);
}
private ResultSet executeQuery(String query) {
this.query = query;
Connection con = null;
Statement st = null;
ResultSet rs = null;
try {
con = db.connectToAndQueryDatabase(database, user, password);
st = con.createStatement();
st.executeUpdate(query);
}
catch (SQLException e1) {
e1.printStackTrace();
}
finally {
if (rs != null) {
try {
rs.close();
} catch (SQLException e2) { /* ignored */}
}
if (st != null) {
try {
st.close();
} catch (SQLException e2) { /* ignored */}
}
if (con != null) {
try {
con.close();
} catch (SQLException e2) { /* ignored */}
}
System.out.println("connection closed");
}
return rs;
}
So now I don't know if it's correct to return a ResultSet like this. I tought of doing something like
public ArrayList<ToDoListModel> getModel() {
return null;
}
To insert every record returned in a ArrayList. But I feel like I'm stuck a little bit. Can someone lead me to a right way with an example or something?
It depends on the way the application works. If you have a lot of databases hits in a short time it would be better to bundle them and use the same database connection for all querys to reduce the overhead of the connection establishment and cleaning.
If you only have single querys in lager intervals you could do it this way.
You should also consider if you want to seperate the database layer and the user interface (if existing).
In this case you should not pass the ResultSet up to the user interface but wrap the data in an independent container and pass this through your application.
If I understand your problem correctly!, you need to pass a list of ToDoListModel objects
to insert into the DB using the insertItem method.
How you pass your object to insert items does not actually matter, but what you need to consider is how concurrent this DataMapper works, if it can be accessed by multiple threads at a time, you will end up creating multiple db connections which is little expensive.Your code actually works without any issue in sequential access.
So you can add a synchronized block to connection creation and make DataMapper class singleton.
Ok in that case what you can do is, create a ArrayList of hashmap first. which contains Key, Value as Column name and Column value. After that you can create your model.
public List convertResultSetToArrayList(ResultSet rs) throws SQLException{
ResultSetMetaData mdata = rs.getMetaData();
int columns = mdata.getColumnCount();
ArrayList list = new ArrayList();
while (rs.next()){
HashMap row = new HashMap(columns);
for(int i=1; i<=columns; ++i){
row.put(md.getColumnName(i),rs.getObject(i));
}
list.add(row);
}
return list;
}
There are thee tables inside my database. One is employee, the second is employee_Project, and the third is employee_Reporting. Each table has a common employee_Number as its primary key, and there is a one to many relationship among them such that an employee has many projects and reporting dates.
I have run select * from employee, select * from employee_project, select * from employee_reporting in three data holder classes which have methods fillResultSet(Result set) and List<T> getData(). This is based on a SqlDbEngine class with a runQuery(PreparedStatement,DataHolder) method, and the implementation has been completed.
Now I have to design a getAllEmployee() method along with project and reporting detail with optimal code in java using JDBC. I have used an iterator but this solution is not acceptable; now I have to use a foreach loop.
This is what I have done:
public List<Employee> getAllEmployees() {
EmployeeDataHolderImpl empdataholder = new EmployeeDataHolderImpl();
List<Employee> list_Employee_Add = null;
try {
Connection connection = mySqlDbConnection.getConnection();
PreparedStatement preparedStatement = connection
.prepareStatement(GET_ALL_EMPLOYEE_DETAILS);
mySqlDBEngineImpl.runQuery(preparedStatement, empdataholder);
} catch (SQLException e) {
e.printStackTrace();
}
for (Employee employee : empdataholder.getData()) {
new EmployeeDAOImpl().getProject(employee);
new EmployeeDAOImpl.getReport(employee);
}
list_Employee_Add = empdataholder.getData();
return list_Employee_Add;
}
and make another method
public void getProject(Employee emp) {
EmployeeProjectDataHolderImpl employeeProjectHolder = new EmployeeProjectDataHolderImpl();
try {
Connection connection = mySqlDbConnection.getConnection();
PreparedStatement preparedStatement = connection
.prepareStatement(GET_ALL_PROJECT_DETAILS);
mySqlDBEngineImpl
.runQuery(preparedStatement, employeeProjectHolder);
} catch (SQLException e) {
e.printStackTrace();
}
for (EmployeeProject employee_Project : employeeProjectHolder.getData()) {
if (employee_Project.getEmployeeNumber() == emp.getEmpNumber()) {
emp.getProjects().add(employee_Project);
}
}
}
public void getReport(Employee emp) {
EmployeeReportDataHolderImpl employeeReportHolder = new EmployeeReportDataHolderImpl();
try {
Connection connection = mySqlDbConnection.getConnection();
PreparedStatement preparedStatement = connection
.prepareStatement(GET_ALL_REPORT_DETAILS);
mySqlDBEngineImpl
.runQuery(preparedStatement, employeeReportHolder);
} catch (SQLException e) {
e.printStackTrace();
}
for (EmployeeReport employee_Report : employeeReportHolder.getData()) {
if (employee_Report.getEmployeeNumber() == emp.getEmpNumber()) {
emp.getProjects().add(employee_Project);
}
}
}
}
and same for Employee Reporting but doing, this performance is going to decrease.no body worry about closing connection i will do it
Please tell me how I could improve my solution..
There are some issue with your code.
1.you are initializing EmployeeDAOImpl everytime, rather you can just keep one instance and call the operations over it.
new EmployeeDAOImpl().getProject(employee); new
EmployeeDAOImpl.getReport(employee);
2.I don't see where you close your connection after performing an SQL operation.
You should be having
try {
--code statements
}
catch(SQLException e){
e.printStackTrace();
}
finally{
-- close your connection and preparedStatement
}
Closing database connections is very vital.
If you use your actual code, you will have 3 impacts in your code:
You're opening a connection to get the employee's data.
For every employee, you open (and close) a new connection to get his projects.
For every employee, you open (and close) a new connection to get his reports.
Note that opening a new connection is a performance hit on your application. It doesn't matter if you use an enhanced for-loop or an Iterator, there would be many hits that can slow down your application.
Two ways to solve this problem:
Open a single connection where you run all your select statements. This will be better than opening/closing lot of connections.
Create a single SQL statement to retrieve the employees and the data you need for every employee. It will have better performance for different reasons:
A single connection to the database.
A single query instead of lot of queries to the database (a single I/O operation).
If your rdbms allows it, the query will be optimized for future requests (a single query instead of multiple queries).
I would prefer to go with the second option. For this, I tend to use a method that executes any SQL select statement and return a ResultSet. I'll post a basic example (note, the provided code can be improved depending on your needs), this method could be in your SqlDbEngine class:
public ResultSet executeSQL(Connection con, String sql, List<Object> arguments) {
PreparedStatement pstmt = null;
ResultSet rs = null;
try {
pstmt = con.prepareStatement(sql);
if (arguments != null) {
int i = 1;
for(Object o : arguments) {
pstmt.setObject(i++, o);
}
}
//method to execute insert, update, delete statements...
rs = pstmt.execute();
} catch(SQLException e) {
//handle the error...
}
return rs;
}
And this other method to handle all the query operation
public List<Employee> getAllEmployee() {
Connection con = null;
ResultSet rs = null;
List<Employee> lstEmployee = new ArrayList<Employee>();
try {
con = mySqlDbConnection.getConnection();
//write the sql to retrieve all the data
//I'm assuming these can be your columns, it's up to you
//this can be written using JOINs...
String sql = "SELECT E.EMPLOYEE_ID, E.EMPLOYEE_NAME, P.PROJECT_NAME, R.REPORT_NAME FROM EMPLOYEE E, PROJECT P, REPORT R WHERE E.EMPLOYEE_ID = P.EMPLOYEE_ID AND E.EMPLOYEE_ID = R.EMPLOYEE_ID";
//I guess you don't need parameters for this...
rs = SqlDbEngine.executeSQL(con, sql, null);
if (rs != null) {
Employee e;
int employeeId = -1, lastEmployeeId = -1;
while (rs.next()) {
//you need to make sure to create a new employee only when
//reading a new employee id
employeeId = rs.getInt("EMPLOYEE_ID");
if (lastEmployeeId != employeeId) {
e = new Employee();
lastEmployeeId = employeeId;
lstEmployee.add(e);
}
Project p = new Project();
Report r = new Report();
//fill values of p...
//fill values of r...
//you can fill the values taking advantage of the column name in the resultset
//at last, link the project and report to the employee
e.getProjects().add(p);
e.getReports().add(r);
}
}
} catch (Exception e) {
//handle the error...
} finally {
try {
if (rs != null) {
Statement stmt = rs.getStatement();
rs.close();
stmt.close();
}
if (con != null) {
con.close();
}
} catch (SQLException e) {
//handle the error...
}
}
return lstEmployee;
}
Note that the second way can be harder to code but it will give you the best performance. It's up to you to improve the provided methods, some advices:
Create a class that receives a ResultSet and builds a Project instance using the columns name of the ResultSet (similar for Report and Employee).
Create a method that handles the ResultSet and its Statement close.
As a best practice, never use select * from mytable, it's preferable to write the needed columns.
If I understand correctly, your code first loads all EmployeeReport rows and then filters them according to getEmployeeNumber(). You can let your database do this by modifying your SQL query.
Since you didn't show your SQL queries (I assume they're in GET_ALL_REPORT_DETAILS), I'll just make a guess... Try executing SQL like:
select *
from employee_reporting
where employeeNumber = ?
If you put this in a PreparedStatement, and then set the parameter value, your database will only return the data you need. For example:
PreparedStatement pstmt = con.prepareStatement(GET_ALL_REPORT_DETAILS);
pstmt.setInt(1, employee.getEmployeeNumber());
That should return only the EmployeeReport records having the desired employeeNumber. In case performance is still an issue, you could consider adding an index to your EmployeeReport table, but that's a different story...