void insert(ArrayList myList){
conn = openDBConnection(); //Database Connection
Iterator iterator = myList.iterator();
while(iterator.hasNext()){
insertIntoDB((myClass)iterator.next(),conn);
}
closeDBConnection();
}
void insertIntoDB(myClass myObject, Connection conn){
String query = "insert into myTable values(?,?)";
PreparedStatement myStatement = conn.prepareStatement(query);
myStatement.setInt(1,myObject.getMyKey());
myStatement.setInt(2,myObject.getMyValue());
myStatement.execute();
}
In the above code myList is an arrayList of objects with more than 1.2M entries. I am getting the following error after inserting around 1000 records:
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at com.mysql.jdbc.PreparedStatement.<init>(PreparedStatement.java:437)
at com.mysql.jdbc.Connection.clientPrepareStatement(Connection.java:2187)
at com.mysql.jdbc.Connection.prepareStatement(Connection.java:4829)
at com.mysql.jdbc.Connection.prepareStatement(Connection.java:4734)
at com.att.research.space.SpaceDaoImpl.insertMapping(SpaceDaoImpl.java:99)
at com.att.research.space.ElementMappingLoader.insertMappingData(ElementMappingLoader.java:68)
at com.att.research.space.CorrelationEngine.loadMappingFiles(CorrelationEngine.java:69)
at com.att.research.space.CorrelationEngine.main(CorrelationEngine.java:25)
I tried using System.gc() inside the iterator loop. But I don't think it is a good way to code and also it is consuming lot of CPU cycle.
The above code is a sample code format of my original code.
You run out of memory because you did not close the statement. This is called a resource leak.
Using try-with-resources (Java 7+):
void insertIntoDB(myClass myObject, Connection conn){
String query = "insert into myTable values(?,?)";
try (PreparedStatement myStatement = conn.prepareStatement(query)) {
myStatement.setInt(1,myObject.getMyKey());
myStatement.setInt(2,myObject.getMyValue());
myStatement.execute();
}
}
Pre-Java 7:
void insertIntoDB(myClass myObject, Connection conn){
String query = "insert into myTable values(?,?)";
PreparedStatement myStatement = conn.prepareStatement(query);
try {
myStatement.setInt(1,myObject.getMyKey());
myStatement.setInt(2,myObject.getMyValue());
myStatement.execute();
} finally {
myStatement.close();
}
}
As others have suggested, you should re-use your statement, but it's the missing close() that's the main issue.
The whole point of PreparedStatement is to create it once and bind variables.
Here's how I'd suggest that you write it:
private static final String INSERT_QUERY = "insert into myTable values(?,?)";
public int insert(Connection c, List<MyClass> myList) {
int numRows = 0;
PreparedStatement ps = null;
try {
ps = c.prepareStatement(INSERT_QUERY);
for (MyClass x : myList) {
ps.setInt(1, x.getMyKey());
ps.setInt(2, x.getMyValue());
numRows += ps.executeUpdate();
}
} finally {
close(ps);
}
return numRows;
}
I left some details for you to figure out (e.g. that close method).
A word of advice: Lose that affection for naming everything "MyFoo". Your code is unreadable with such a naming convention. Think more carefully about how you name things.
move the statement out of the loop and user addBatch() and executeBatch instead
Avoiding memory leaks with finally block should definitely be addressed first; however if its just a very large resultset; I found this useful:
http://benjchristensen.com/2008/05/27/mysql-jdbc-memory-usage-on-large-resultset/
namely adding
stmt.setFetchSize(Integer.MIN_VALUE);
Before runing stmt.executeQuery();
Related
I am having a problem with a ResultSet being closed. What confuses me is that it works for a portion of the data and then closes. At first I thought it might be because of connection timeout but that doesn't seem the case.
This portion of the program pertains to comparing an .xlsx workbook to an already present SQL database and for lack of a better term merges/updates it.
First, in my CompareDatabase class I am calling a search function that searches an SQLite database for a specific string every 6 iterations.
int columnCount = 6;
dataPoint = dataPoint.replaceAll("Detail", "");
String[] temp = dataPoint.trim().split("\\s+");
System.out.println(Arrays.toString(temp));
for (String tempDataPoint : temp) {
if ( columnCount == 6) {
System.out.println(search(tempDataPoint, connection));
}
columnCount = 0;
} else {
columnCount++;
}
}
This search function (also in the CompareDatabase class is then supposed to search for the value and return a String (was originally a Boolean but I wanted to see the output).
private String search (String searchValue, Connection connection) throws SQLException {
PreparedStatement pStatement = null;
pStatement = connection.prepareStatement("SELECT * FROM lotdatabase where (Vehicle) = (?)");
pStatement.setString(1, searchValue);
try (ResultSet resultSet = pStatement.executeQuery()){
return resultSet.getString(1);
}finally {
close(pStatement);
}
}
At the end you can see that the PreparedStatement is closed. The ResultSet should also be closed automatically (I read somewhere) but JDBC could possibly be being unreliable.
The Connection however is still open as it will be searching some 200+ strings and opening and closing that many times did not seem like a good idea.
These functions are called by my main class here:
One is commented out since it will error out because of primary key violation.
public static void main(String[] args) {
SQLDatabase sqlDatabase = new SQLDatabase();
//sqlDatabase.convertToSQL("Database1.xlsx");
sqlDatabase.compare("Database2.xlsx");
}
I have a suspicion that I am going about a bunch of this wrong (on the aspect of managing connections an such) and I would appreciate a reference to where I can learn to do it properly.
Also, being that PreparedStatement can only handle one ResultSet I don't see that being my issue since I close it every iteration in the for loop.
If more code or explanation is required please let me know and I will do my best to assist.
Thank you for taking the time to read this.
So after a bit more Googling and sleeping on it here is what worked for me.
The search function in compareDatabase changed to this:
private Boolean search (String searchValue, Connection connection) {
PreparedStatement ps = null;
try {
ps = connection.prepareStatement("SELECT * FROM lotdatabase where " +
"(Vehicle) = (?)");
ps.setString(1, searchValue);
ResultSet resultSet = ps.executeQuery();
//The following if statement checks if the ResultSet is empty.
if (!resultSet.next()){
resultSet.close();
ps.close();
return false;
}else{
resultSet.close();
ps.close();
return true;
}
} catch (SQLException e) {
e.printStackTrace();
}
return false;
}
And in the other function within compareDatabase I call the search function like this:
if (search(tempDataPoint, connection)) {
System.out.println("MATCH FOUND: " + tempDataPoint);
}else {
System.out.println("NOT FOUND: " + tempDataPoint);
}
This allows me to check the ResultSet and also be sure that it is closed.
My DB is Oracle.
I know Statement can mix SQL sentences(insert or delete or update) into one single batch. Here is my code.
DBConnection db = new DBConnection();
Connection c = db.getConn();
Statement s = null ;
try
{
String sql = "insert into t1(id, name) values ('10', 'apple')";
String sql1 = "insert into t1(id, name) values ('14', 'pie')";
String sql2 = "delete from t1 where id = '10'";
s = c.createStatement();
s.addBatch(sql);
s.addBatch(sql1);
s.addBatch(sql2);
int[] re = s.executeBatch();...
My question is can PreparedStatement do this? and how?
You can create a batch by PreparedStatement.addBatch() and you can execute it by PreparedStatement.executeBatch
For more about PreparedStatement you can look into documentation
Now if i am not wrong you want to do something like this:
public void save(List<Entity> elements) throws SQLException {
Connection connection = null;
PreparedStatement statement = null;
try {
connection = database.getConnection();
statement = connection.prepareStatement(SQL_INSERT);
for (int i = 0; i < elements.size(); i++) {
Element element= elements.get(i);
statement.setString(1, element.getProperty1());
statement.setString(2, element.getProperty2());
.....
statement.addBatch();
if ((i + 1) % 200 == 0) {
statement.executeBatch(); // Execute every 200 items.
}
}
statement.executeBatch();
} finally {
if (statement != null) try { statement.close(); } catch (SQLException e) { //}
if (connection != null) try { connection.close(); } catch (SQLException e) {//}
}
}
In this case i am executing every 200 items, if you wish you can set your own. But do test it because it also depends on drivers limitation on batch operations.
Statement:
Use for general-purpose access to your database. Useful when you are using static SQL statements at runtime. The Statement interface cannot accept parameters.
PreparedStatement:
Use when you plan to use the SQL statements many times. The PreparedStatement interface accepts input parameters at runtime.
CallableStatement:
Use when you want to access database stored procedures. The CallableStatement interface can also accept runtime input parameters.
I have my Java program and I need to get data from my MYSQL DB,
I wrote this one out but its just sysout so getting data from my class and not using the Prepared Statement (I can delete the first 3 lines and it will work the same )
Could use some help to figure out how to get data from my DB and print it out
public void viewClientDetails(ClientsBean client) {
try {
PreparedStatement ps = connect.getConnection().prepareStatement(
"SELECT * FROM mbank.clients WHERE client_id = ?");
ps.setLong(1, client.getClient_id());
System.out.println(client.getClient_id());
System.out.println(client.getName());
System.out.println(client.getType());
System.out.println(client.getPhone());
System.out.println(client.getAddress());
System.out.println(client.getEmail());
System.out.println(client.getComment());
} catch (SQLException e) {
e.printStackTrace();
JOptionPane.showMessageDialog(null,
"Problem occurs while trying to see client details");
}
}
Well you're not actually executing the prepared statement... you're just preparing it. You should call PreparedStatement.executeQuery and use the ResultSet it returns:
// ...code as before...
try (ResultSet results = ps.executeQuery()) {
while (results.next()) {
// Use results.getInt etc
}
}
(You should use a try-with-resources statement to close the PreparedStatement too - or a manual try/finally block if you're not using Java 7.)
You need to do executeQuery on the preparedstatement to get a result set back of the query you performed.
You are simply not executing the query. Add a PreparedStatement.executeQuery() call. And fetch the results from the returned ResultSet.
For example:
PreparedStatement ps = connect.getConnection().prepareStatement("SELECT * FROM mbank.clients WHERE client_id = ?");
ps.setLong(1, client.getClient_id());
ResultSet rs = ps.executeQuery();
while (rs.next()) {
String userid = rs.getString("id");
String username = rs.getString("name");
}
As #Jon Skeet pointed out, the declaration of ResultSet in Java 7 is updated to:
public interface ResultSet extends Wrapper, AutoCloseable
It is AutoClosable now, which means that you can and should use the try-with-resource pattern.
You can do the below.
PreparedStatement ps = connect.getConnection().prepareStatement(
"SELECT * FROM mbank.clients WHERE client_id = ?");
resultSet = ps.executeQuery();
while (resultSet.next()) {
String user = resultSet.getString("<COLUMN_1>");
String website = resultSet.getString("<COLUMN_2>");
String summary = resultSet.getString("<COLUMN_3>");
}
If I do something like
try (
Connection conn = Database.getConnection();
PreparedStatement ps = conn.prepareStatement("SELECT * FROM table WHERE something = ? LIMIT 1");
) {
ps.setString(1, "hello world");
ResultSet results = ps.executeQuery();
if(results.next()) {
// blah
}
} catch(SQLException e) {
e.printStackTrace();
}
Will the ResultSet still be closed when the PreparedStatement is closed, or will I still have to explicitly close the ResultSet also?
As per javax.sql.Statement.close() method's JavaDoc:
Note:When a Statement object is closed, its current ResultSet object, if one exists, is also closed.
So, answering your question - yes, ResultSet will be automatically closed in your case, because related Statement is closed in try-with-resources block.
However, please note that explicitly closing ResultSets is a good practice which is recommended to follow, so your modified code following good practices would look like:
try (
Connection conn = Database.getConnection();
PreparedStatement ps = prepareStatement(conn, "SELECT * FROM table WHERE something = ? LIMIT 1", param);
ResultSet results = ps.executeQuery();
) {
if(results.next()) {
// blah
}
} catch(SQLException e) {
e.printStackTrace();
}
private static PreparedStatement prepareStatement(Connection connection, String sql, String param) throws SQLException {
final PreparedStatement ps = conn.prepareStatement(sql);
ps.setString(1, param);
return ps;
}
Always As a good practice, try to close your ResultSets and PreparedStatements. In a finally block. Every single time , managing exceptions, so you won't leave resources unattended (is a common source of leaks).
Unless you inject them to the method, hence the calling method probably needs them.
EDIT: Stand corrected. If resultset was created as try-with-resource, will die with your PS.
I'm using Java for a web application, and I'm working with a MySql database. I need to escape the query before execute it. This is my actual code :
db_result=mydb.selectQuery("SELECT nickname FROM users WHERE nickname='"+log_check_user+"' AND password='"+log_check_pass+"'");
public Vector selectQuery(String query) {
Vector v = null;
String [] record;
int colonne = 0;
try {
Statement stmt = db.createStatement();
ResultSet rs = stmt.executeQuery(query);
v = new Vector();
ResultSetMetaData rsmd = rs.getMetaData();
colonne = rsmd.getColumnCount();
while(rs.next()) {
record = new String[colonne];
for (int i=0; i<colonne; i++) record[i] = rs.getString(i+1);
v.add( (String[]) record.clone() );
}
rs.close();
stmt.close();
} catch (Exception e) { e.printStackTrace(); errore = e.getMessage(); }
return v;
}
I need this, as you can believe, to avoid the SQL Injection problem! How can I do it?
Use a prepared statement:
Sometimes it is more convenient to use a PreparedStatement object for sending SQL statements to the database. This special type of statement is derived from the more general class, Statement...
If you want to execute a Statement object many times, it usually reduces execution time to use a PreparedStatement object instead.
The main feature of a PreparedStatement object is that, unlike a Statement object, it is given a SQL statement when it is created. The advantage to this is that in most cases, this SQL statement is sent to the DBMS right away, where it is compiled. As a result, the PreparedStatement object contains not just a SQL statement, but a SQL statement that has been precompiled. This means that when the PreparedStatement is executed, the DBMS can just run the PreparedStatement SQL statement without having to compile it first...