Develop Reference

iterative update to a jdbc derby using a ResultSet [duplicate]

This question already has an answer here:
Why do I get java.sql.SQLException: ResultSet not open. Operation 'next' not permitted. java derby database?
(1 answer)
Closed 6 years ago.
In connection with another programming project, I am prototyping a JDBC project (with Netbeans, Java, and a Derby database). My program needs to iteratively update all the rows in a database table as follows:
There are three columns in the table: famousName, famousQuote, hashKey.
Originally, the famousQuote column contains a verbatim quote. I want to go down that column using a while loop, get the checksum of the ascii letters, bitwise "AND" with the hashKey value, and then replace the verbatim quote with an "encrypted" value.
As of right now, I try to extract the verbatim famousQuote using a ResultSet object, perform necessary encryption, and then an SQL statement that updates the value. All of this takes place in a while(rs.next()) loop as follows:
(pseudo-code): //all necessary database connections and variable declared here.
String sqlStatement = "Select * FROM mainTable ORDER BY famousName";
ResultSet rs = stmt.executeQuery(sqlStatement);
while(rs.next()){
tempString1 = rs.getString("famousQuote");
tempString2 = rs.getString("hashKey");
tempString3 = EncryptionAlgorithm.EncryptStatement(tempString1, tempString2);
sqlStatement = "UPDATE maintable SET famousQuote=tempString3 WHERE hashKey=tempString2";
(note, there is a bit of pseudo-code regarding the WHERE part,
but I'm sure it's immaterial to the error message I'm getting.)
stmt.executeUpdate(sqlStatement);
}
This seemed like a good idea until the program started throwing errors such as:
ResultSet not open. Operation 'next' not permitted. Verify that autoCommit is off
I later read in the documentation (concerning ResultSet):
A ResultSet object is automatically closed when the Statement object that generated it
is closed, re-executed, or used to retrieve the next result from a sequence of multiple results.
Lastly, I guess I could try moving the ResultSet rs declaration inside the while loop, so that it would instantiate a new ResultSet object, but I'm pretty sure this would lose my place in the database (re-updating the first row perpetually).
I'm now at a standstill on how to iterate my way down the table rows, executing my EncryptionAlgorithm on each quote, and then updating the column values in place.

I apologize in advance if my JDBC is a little rusty, but something like this might do the trick:
// 'conn' is your JDBC connection
Statement stmt = conn.createStatement();
PreparedStatement update = conn.prepareStatement(
"UPDATE maintable SET famousQuote=tempString3 WHERE hashKey=tempString2";
String sqlStatement = "Select * FROM mainTable ORDER BY famousName";
ResultSet rs = stmt.executeQuery(sqlStatement);
while(rs.next()){
tempString1 = rs.getString("famousQuote");
tempString2 = rs.getString("hashKey");
tempString3 = EncryptionAlgorithm.EncryptStatement(tempString1, tempString2);
update.setObject(1, tempString3);
update.setObject(2, tempString2);
update.executeUpdate(sqlStatement);
}

Why doesn't JDBC support bulk fetching of data?

JDBC has been supporting bulk updates for a long time using addBatch and executeBatch. Why isn't there any support for adding a bunch of prepared statements and getting an array of result sets as response?
For example, if I wanted to load customer details, basic account details, basic card details, basic loan details etc. for a single view, I would prefer to create a bunch of prepared statements and append the prepared statements to an ArrayList and execute them as a batch. I would then loop through the result sets and process the data. Hopefully, several network round trips would be saved (assuming my queries are performant).
Sample bunch of queries:
SELECT custid, first, last, age FROM Customer where custid = ?
SELECT custid, acno, accountname, accounttype, status FROM Account where custid = ?
SELECT custid, cardno, cardname, cardtype, status FROM CreditCard where custid = ?
SELECT custid, loanno, principal, rate FROM Loan where custid = ?
I can imagine several hypothetical reasons why it could be a bad idea. But, I am not sure which is most likely true in the real world.
Hypothetical reasons against having bulk-fetch:
There is some fundamental networking/db stack/memory related issue
which prevents a bunch of select queries to be executed on the same
connection and result-sets kept open.
Response handling code would be too cumbersome, as there could be exceptions at call level and individual statement level. And, several statements would have to be closed correctly.
There is no significant performance gain in reducing the number of network-calls. Query execution is the main bottleneck and network round-trip cost is insignificant.
There could be misuse of such a feature. A single non-performant query batched up like this with other queries could cause application to perform poorly.
The reason I ask this is because often I see a lot of Join queries which merge parent-child relationships into a single SQL query, just for the sake of completing the loading in a single call.
However, as the number of tables grows, the query becomes complex. Also, the parent table information is repeated in every row of every child. So, there is huge amount of data redundancy in the single join-ed result set.
Sample join query:
SELECT custid, first, last, age, acno, accountname, accounttype, a.status, cardno, cardname, cardtype, c.status, loanno, principal, rate
FROM Customer cc, Account a, CreditCard c, Loan l
WHERE a.custid=CC.custid(+) and c.custid=CC.custid(+) and l.custid=CC.custid(+)

The JDBC API does support this.
Statement.getMoreResults() can tell you if the SQL statement you executed through execute() produced more than one ResultSet
Quote from the JavaDocs for getMoreResults():
Moves to this Statement object's next result, returns true if it is a ResultSet object, and implicitly closes any current ResultSet object(s) obtained with the method getResultSet.
There are no more results when the following is true:
// stmt is a Statement object<br>
((stmt.getMoreResults() == false) && (stmt.getUpdateCount() == -1))
However it depends on the backend DBMS and the JDBC driver if you can use this. Some JDBC driver simply reject to run more than one statement with a single execute() call (mainly as a means to prevent SQL injenction), others don't.
So in e.g. Postgres you can do something like this:
boolean hasResult = stmt.execute(
"select * from table_1;\n" +
"select * from table_2;");
while (hasResult)
{
rs = stmt.getResultSet();
while (rs.next())
{
// process the result set
}
hasResult = stmt.getMoreResults();
}
This even allows mixing SELECT and e.g. UPDATE statements if you also check for getUpdateCount()
As far as I know you can also do this with SQL Server. It does not work with Oracle.
I haven't tried this with a PreparedStatement though. But as getMoreResults() is defined for Statement it is available for a PreparedStatement as well.

How about to put queries to a procedure and then use CallableStatement to execute that procedure?
A CallableStatement can return one ResultSet object or multiple
ResultSet objects. Multiple ResultSet objects are handled using
operations inherited from Statement.
try
{
CallableStatement stmt = con.prepareCall(/* call procedure */);
boolean results = stmt.execute();
int rsCount = 0;
while (results)
{
ResultSet rs = stmt.getResultSet();
while (rs.next())
{
}
rs.close();
results = stmt.getMoreResults();
}
stmt.close();
}
catch (Exception e) {
e.printStackTrace();
}

Relational databases are designed and optimized for retrieving data through SQL queries that JOIN data from multiple tables. Executing a single query that (correctly) JOINs data is likely always more efficient than getting the same data with separate queries.
When a single query gets too complex, it should be refactored into a VIEW -- from which you can query, joining data from other TABLEs and VIEWs, if required.
Given the above, I don't see a need for bulk queries.

I get the feeling you don't understand what a prepared statement is.
A prepared statement is an object you declare once, then reuse it all the time with different supplied parameters to it.
You're not telling me that you recreate a prepared statement from scratch each time you wish to execute it again?
Say you have four loops. before executing your loops you do this:
PreparedStatement statement1, statement2, statement3,statement4;
try {
con.setAutoCommit(false);//only needed when also doing updates/inserts
statement1 = con.prepareStatement("SELECT custid, first, last, age FROM Customer where custid = ?");
statement2 = con.prepareStatement("SELECT custid, acno, accountname, accounttype, status FROM Account where custid = ?");
// etc....
for (Map.Entry<String, Integer> e : customers.entrySet()) {
statement1.setInt(1, e.getValue().intValue());
ResultSet rs = statement1.executeQuery();
// do what you need to do
statement2.setInt(1, e.getValue().intValue());
ResultSet rs2 = statement2.executeQuery();
// do what you need to do
}
con.commit();//only needed when also doing updates/inserts
}
}
There is no need to recreate the prepared statements. That is why its calleda prepared statement. You just feed it the new values it needs to query.
This way you can add it to lists, itereate it the way you want to itereate it, etc.. and it's all optimised since the database engine will remember the query plans and the optimisations it makes for it. What you do with the prepared statement object is up to you.
It also does this if you recreate the objects constantly because it will remember the query, but you save the overhead of createing new objects over and over and the memory issues that come with that.
So, without a clearer question this is the best answer I can give you.

Execute multiple queries using a single JDBC Statement object

In JDBC, can I use single Statement object to call executeQuery("") multiple times? Is it safe? Or should I close the statement object after each query, and create new object for executing another query.
E.G:
Connection con;
Statement s;
ResultSet rs;
ResultSet rs2;
try
{
con = getConnection();
// Initially I was creating the Statement object in an
// incorrect way. It was just intended to be a pseudocode.
// But too many answerers misinterpretted it wrongly. Sorry
// for that. I corrected the question now. Following is the
// wrong way, commented out now.
// s = con.prepareStatement();
// Following is the way which is correct and fits for my question.
s = con.createStatement();
try
{
rs = s.executeQuery(".......................");
// process the result set rs
}
finally
{
close(rs);
}
// I know what to do to rs here
// But I am asking, should I close the Statement s here? Or can I use it again for the next query?
try
{
rs2 = s.executeQuery(".......................");
// process the result set rs2
}
finally
{
close(rs2);
}
}
finally
{
close(s);
close(con);
}

Yes you can re-use a Statement(specifically a PreparedStatement) and should do so in general with JDBC. It would be inefficient & bad style if you didn't re-use your statement and immediately created another identical Statement object. As far as closing it, it would be appropriate to close it in a finally block, just as you are in this snippet.
For an example of what you're asking check out this link: jOOq Docs

I am not sure why you are asking. The API design and documentation show it is perfectly fine (and even intended) to reuse a Statement object for multiple execute, executeUpdate and executeQuery calls. If it wouldn't be allowed that would be explicitly documented in the Java doc (and likely the API would be different).
Furthermore the apidoc of Statement says:
All execution methods in the Statement interface implicitly close a statment's [sic] current ResultSet object if an open one exists.
This is an indication that you can use it multiple times.
TL;DR: Yes, you can call execute on single Statement object multiple times, as long as you realize that any previously opened ResultSet will be closed.
Your example incorrectly uses PreparedStatement, and you cannot (or: should not) be able to call any of the execute... methods accepting a String on a PreparedStatement:
SQLException - if [...] the method is called on a PreparedStatement or CallableStatement
But to answer for PreparedStatement as well: the whole purpose of a PreparedStatement is to precompile a statement with parameter placeholders and reuse it for multiple executions with different parameter values.

I can't find anything in the API docs that would state, that you shouldn't call executeQuery() on a given PreparedStatement instance more than once.
However your code does not close the PreparedStatement - a call to executeQuery() would throw a SQLException in that case - but the ResultSet that is returned by executeQuery(). A ResultSet is automatically closed, when you reexecute a PreparedStatement. Depending on your circumstances you should close it, when you don't need it anymore. I would close it, because i think it's bad style not to do so.
UPDATE Upps, I missed your comment between the two try blocks. If you close your PreparedStatement at this point, you shouldn't be able to call executeQuery() again without getting a SQLException.

A Prepared Statement tells the database to remember your query and to be prepared to accept parameterized variables to execute in that query. It's a lot like a stored procedure.
Prepared Statement accomplishes two main things:
It automatically escapes your query variables to help guard against SQL Injection.
It tells the database to remember the query and be ready to take variables.
Number 2 is important because it means the database only has to interpret your query once, and then it has the procedure ready to go. So it improves performance.
You should not close a prepared statement and/or the database connection in between execute calls. Doing so is incredibly in-efficient and it will cause more overhead than using a plain old Statement since you instruct the database each time to create a procedure and remember it. Even if the database is configured for "hot spots" and remembers your query anyways even if you close the PreparedStatement, you still incur network overhead as well as small processing time.
In short, keep the Connection and PreparedStatement open until you are done with them.
Edit: To comment on not returning a ResultSet from the execution, this is fine. executeQuery will return the ResultSet for whatever query just executed.

Firstly I am confused about your code
s = con.prepareStatement();
Is it work well?I can't find such function in JAVA API,at least one parameter is needed.Maybe you want to invoke this function
s = con.createStatement();
I just ran my code to access DB2 for twice with one single Statement instance without close it between two operation.It's work well.I think you can try it yourself too.
String sql = "";
String sql2 = "";
String driver = "com.ibm.db2.jcc.DB2Driver";
String url = "jdbc:db2://ip:port/DBNAME";
String user = "user";
String password = "password";
Class.forName(driver).newInstance();
Connection conn = DriverManager.getConnection(url, user, password);
Statement statement = conn.createStatement();
ResultSet resultSet = statement.executeQuery(sql);
int count = 0;
while (resultSet.next()) {
count++;
}
System.out.println("Result row count of query number one is: " + count);
count = 0;
resultSet = statement.executeQuery(sql2);
while (resultSet.next()) {
count++;
}
System.out.println("Result row count of query number two is: " + count);

Which way is better for update database record?

I have two method for update:
String query = "update mytable set name = 'new_value' where id ='20' ";
Connection conn;
PreparedStatement pState;
try {
conn = DriverManager.getConnection(dbUrl, "root", "2323");
pState = conn.prepareStatement(query);
pState.executeUpdate();
} catch (SQLException sql) {
sql.printStackTrace();
}
OR:
String query = "update mytable set name = ?" + "where id = ?";
Connection conn;
PreparedStatement pState;
int s;
try {
conn = DriverManager.getConnection(dbUrl, "root", "2323");
pState = conn.prepareStatement(query);
pState.setStringt(1, "new_value");
pState.setString(2, "20");
s = pState.executeUpdate(); // if s = 1 then update done successfully
} catch (SQLException sql) {
sql.printStackTrace();
}
Both methods update database record correctly, Which is better?

Second approach is good practice to avoid SQL Injection attacks.
And following is enough to construct query String, another + concatenation is not required.
String query = "update mytable set name = ? where id = ?";

I would say the second approach.
You aren't returning anything, so why create a result set and go down that path?
Edit:
Even after your comment, I would still use the second template. It's more flexible. Additionally, it's faster. The PreparedStatement is pre-compiled in the database which allows the database to execute a parametric query using the statement faster than a normal query. This won't happen if you use string concatenation (like in your first example).
See: http://docs.oracle.com/javase/tutorial/jdbc/basics/prepared.html
Additionally, from that page:
The main feature of a PreparedStatement object is that, unlike a
Statement object, it is given a SQL statement when it is created. The
advantage to this is that in most cases, this SQL statement is sent to
the DBMS right away, where it is compiled. As a result, the
PreparedStatement object contains not just a SQL statement, but a SQL
statement that has been precompiled. This means that when the
PreparedStatement is executed, the DBMS can just run the
PreparedStatement SQL statement without having to compile it first.
Although PreparedStatement objects can be used for SQL statements with
no parameters, you probably use them most often for SQL statements
that take parameters. The advantage of using SQL statements that take
parameters is that you can use the same statement and supply it with
different values each time you execute it.

The second way is more faster if you use frequently the same query. Depends of the database vendor, the query is cached and the efficiency is higher than that using flat sentences. But all that depends on the implementation of the JDBC driver and the services provided by the database.
See more in Using Prepared Statements in the The Java Tutorials.

How to add records to databse via sql in Java

I am working a Airsoft application.
I'm trying to add records to a MS Access Database via SQL in Java. I have established a link to the database, with the following:
try
{
//String Driver = "sun.java.odbc.JdbcOdbcDriver";
Class.forName("net.ucanaccess.jdbc.UcanaccessDriver");
Connection conn = DriverManager.getConnection("jdbc:ucanaccess://" + URL,"","");
Statement stmt = conn.createStatement();
System.out.println("Connection Established!");
ResultSet rs = stmt.executeQuery("SELECT * FROM AirsoftGunRentals");
tblRent.setModel(DbUtils.resultSetToTableModel(rs));
}
catch(Exception ex)
{
JOptionPane.showMessageDialog(null, "Error");
}
I am using Ucanaccess to access my MS database. It is reading the database and is displaying to a JTable. However, I need to create three JButtons to add, delete and update the table. I have tried to code the add button, and I have tried to add a record, but it crashes and gives me errors.
try
{
//String Driver = "sun.java.odbc.JdbcOdbcDriver";
Class.forName("net.ucanaccess.jdbc.UcanaccessDriver");
Connection conn = DriverManager.getConnection("jdbc:ucanaccess://" + URL,"","");
Statement stmt = conn.createStatement();
System.out.println("Connection Established!");
String Query= "INSERT INTO AirsoftGunRentals(NameOfGun, Brand, TypeOfGuns, NumberOfMagazines,Extras,NumberAvailable,UnitRent)"+
"VALUES('"+pName+"','"+pBrand+"','"+pTypeOfGun+"','"+pNumMags+"','"+pExtras+"','"+pNumberAvail+"','"+pRent+"');";
ResultSet rs = stmt.executeQuery(Query);
JOptionPane.showMessageDialog(null, "Success!");
}
catch(Exception ex)
{
JOptionPane.showMessageDialog(null, "Error");
}
I have attempted all three, hoping for a result. But am still getting big errors. The only difference between the buttons is that one adds, one deletes and one updates the table. Other then that, the code is the same, minus variables.

As Brahim mentionned it, you should use stmt.executeUpdate(Query) whenever you update / insert or delete data. Also with this particular query, given your String concatenation (see end of line), there is no space between the ")" and the "VALUES" which probably causes a malformed query.
However, I can see from your code that you are not very experienced with such use-cases, and I'd like to add some pointers before all hell breaks loose in your project :
Use PreparedStatement instead of Statement and replace variables by placeholders to prevent SQL Injection.
The code that you are using here is extremely prone to SQL injection - if any user has any control over any of the variables, this could lead to a full database dump (theft), destruction of data (vandalism), or even in machine takeover if other conditions are met.
A good advice is to never use the Statement class, better be safe than sorry :)
Respect Java Conventions (or be coherent).
In your example you define the String Query, while all the other variables start with lower-case (as in Java Conventions), instead of String query. Overtime, such little mistakes (that won't break a build) will lead to bugs due to mistaking variables with classnames etc :)
Good luck on your road to mastering this wonderful language ! :)

First add a space before the quotation marks like this :
String Query= "INSERT INTO AirsoftGunRentals(NameOfGun, Brand, TypeOfGuns, NumberOfMagazines,Extras,NumberAvailable,UnitRent) "+
" VALUES('"+pName+"','"+pBrand+"','"+pTypeOfGun+"','"+pNumMags+"','"+pExtras+"','"+pNumberAvail+"','"+pRent+"');";
And use stmt.executeUpdate(Query); instead of : stmt.executeQuery(Query);in your insert, update and delete queries. For select queries you can keep it.

I managed to find an answer on how to add, delete and update records to a MS Access DB. This is what I found, after I declared the connection, and the prepped statement. I will try to explain to the best I can. I had to add values individually using this:
(pstmt = Prepped Statement Variable)
pstmt.setWhatever(1,Variable);
And it works fine now. I use the same method to delete and update records.
This is the basic query format:
String SQLInsert = "INSERT INTO Tbl VALUES(NULL,?,?,?,?)";
The NULL in the statement is the autonumber in the table. and .setWhatever() clause replaces the question marks with the data types. Thus manipulating the database.
Thank you everyone for all your contributions. It helped a lot, and made this section a lot more understandable.