Develop Reference

JDBC INSERT ... SELECT Prepared Statement quietly failing to update database

Very much a beginner, this relates to a college project and as such I understand if you are unwilling to solve the problem for me, but I would appreciate being pointed in the right direction if possible.
The below code is a snippet from my database management class for a project.
I'm keeping an ArrayList of users within the java project, loading it if possible from the database at runtime and saving back to the database when closed. Any changes to the users are made in the app (just a JavaFX GUI) and only reflected in the database at shutdown.
For this reason, I need it to be a "INSERT IF NOT EXISTS" type of statement.
I tried this originally without the complex INSERT...SELECT, it just plain inserted the user, causing duplicates. This caused duplicates, but otherwise worked, updating and reading from the database as expected with no errors, even on the ArrayList to byte array to BLOB part, which was very much stretching my Java ability.
I also understand that there are some examples of poor encapsulation in my code, but I am more concerned with the fact that the code runs, no errors, and even in the debugger the prepared statement seems to be getting formed correctly, but the database fails to be updated, with or without duplicates.
Any help, whether a solution, work around or suggestion as to where to go looking for a cause, would be much appreciated.
public static void saveUsers(ArrayList<User> users){
makeConn();
try{
for (User u : carbonapp.CarbonApp.userList.users){
String n = u.getName();
String p = u.getPass();
boolean a = u.getIsAdmin();
UserActivityList l = u.list;
byte[] m = null;
try {
m = toByteArray(l.getUserActivities());
} catch (IOException ex) {
Logger.getLogger(dbStor.class.getName()).log(Level.SEVERE, null, ex);
}
Blob ualblob = new SerialBlob(m);
String sql = "INSERT INTO root.person (name,pass,isAdmin,ual)\n" +
"SELECT CAST(? as VARCHAR(50)),CAST(? as VARCHAR(50)),CAST(? as BOOLEAN),CAST(? as BLOB) " +
"FROM root.person WHERE NOT EXISTS (SELECT name FROM root.person WHERE name = ? AND pass = ?)";
pstmt = conn.prepareStatement(sql);
pstmt.setString(1, n);
pstmt.setString(2, p);
pstmt.setBoolean(3, a);
pstmt.setBlob(4, ualblob);
pstmt.setString(5, n);
pstmt.setString(6, p);
pstmt.executeUpdate();
conn.commit();
}}catch(SQLException se){se.printStackTrace();
}try{conn.close();
}catch(SQLException se){se.printStackTrace();}
}
FYI, I'm using a derby database, working in Netbeans. The makeConn(); function creates the connection and works in other cases, it sets a static Connection conn which is used in the shown code.
Again, no error messages, no crashes or other issues there, just an empty table "person", the desired result being that the table gets updated with the values, in the case that those values don't exists already.

JDBC Instrumentation and ORA-01000: maximum open cursors exceeded

I am trying to better instrument which web applications make use of Oracle (11g) connections in our Tomcat JDBC connection pool when a connection is created and closed; this way, we can see what applications are using connections by monitoring the V$SESSION table. This is working, but since adding this "instrumentation" I am seeing ORA-01000: maximum open cursors exceeded errors being logged and noticing some connections being dropped out of the pool during load testing (which is probably fine as I have testOnBorrow enabled, so I'm assuming the connection is being flagged as invalid and dropped from the pool).
I have spent the better part of the week scouring the internet for possible answers. Here is what I have tried (all result in the open cursors error after a period of time)...
The below methods are all called the same way...
On Create
We obtain a connection from the pool
We call a method that executes the below code, passing in the context name of the web application
On Close
We have the connection being closed (returned to the pool)
Before we issue close() on the connection, we call a method that executes the code below, passing in "Idle" as the name to store in V$SESSION
Method 1:
CallableStatement cs = connection.prepareCall("{call DBMS_APPLICATION_INFO.SET_MODULE(?,?)}");
try {
cs.setString(1, appId);
cs.setNull(2, Types.VARCHAR);
cs.execute();
log.trace(">>> Executed Oracle DBMS_APPLICATION_INFO.SET_MODULE with module_name of '" + appId + "'");
} catch (SQLException sqle) {
log.error("Error trying to call DBMS_APPLICATION_INFO.SET_MODULE('" + appId + "')", sqle);
} finally {
cs.close();
}
Method 2:
I upgraded to the 12c OJDBC driver (ojdbc7) and used the native setClientInfo method on the connection...
// requires ojdbc7.jar and oraclepki.jar to work (setEndToEndMetrics is deprecated in ojdbc7)
connection.setClientInfo("OCSID.CLIENTID", appId);
Method 3:
I'm currently using this method.
String[] app_instrumentation = new String[OracleConnection.END_TO_END_STATE_INDEX_MAX];
app_instrumentation[OracleConnection.END_TO_END_CLIENTID_INDEX] = appId;
connection.unwrap(OracleConnection.class).setEndToEndMetrics(app_instrumentation, (short)0);
// in order for this to be sent, a query needs to be sent to the database - this works fine when a
// connection is created, but when it is closed, we need a little something to get the change into the db
// try using isValid()
connection.isValid(1);
Method 4:
String[] app_instrumentation = new String[OracleConnection.END_TO_END_STATE_INDEX_MAX];
app_instrumentation[OracleConnection.END_TO_END_CLIENTID_INDEX] = appId;
connection.unwrap(OracleConnection.class).setEndToEndMetrics(app_instrumentation, (short)0);
// in order for this to be sent, a query needs to be sent to the database - this works fine when a
// connection is created, but when it is closed, we need a little something to get the change into the db
if ("Idle".equalsIgnoreCase(appId)) {
Statement stmt = null;
ResultSet rs = null;
try {
stmt = connection.createStatement();
rs = stmt.executeQuery("select 1 from dual");
} finally {
if (rs != null) {
rs.close();
}
if (stmt != null) {
stmt.close();
}
}
}
When I query for open cursors, I notice the following SQL being returned on the account being used in the pool (for each connection in the pool)...
select NULL NAME, -1 MAX_LEN, NULL DEFAULT_VALUE, NULL DESCR
This does not explicitly exist anywhere in our code, so I can only assume it is coming from the pool when running the validation query (select 1 from dual) or from the setEndToEndMetrics method (or from the DBMS_APPLICATION_INFO.SET_MODULE proc, or from the isValid() call). I tried to be explicit in creating and closing Statement (CallableStatement) and ResultSet objects in methods 1 and 4, but they made no difference.
I don't want to increase the number of allowed cursors, as this will only delay the inevitable (and we have never had this issue until I added in the "instrumentation").
I've read through the excellent post here (java.sql.SQLException: - ORA-01000: maximum open cursors exceeded), but I must still be missing something. Any help would be greatly appreciated.

So Mr. Poole's statement: "that query looks like it's getting fake metadata" set off a bell in my head.
I started to wonder if it was some unknown remnant of the validation query being run on the testOnBorrow attribute of the pool's datasource (even though the validation query is defined as select 1 from dual). I removed this from the configuration, but it had no effect.
I then tried removing the code that sets the client info in V$SESSION (Method 3 above); Oracle continued to show that unusual query and after only a few minutes, the session would hit the maximum open cursors limit.
I then found that there was a "logging" method in our DAO class that logged some metadata from the connection object (values for settings like current auto commit, current transaction isolation level, JDBC driver version, etc.). Within this logging was the use of the getClientInfoProperties() method on the DatabaseMetaData object. When I looked at the JavaDocs for this method, it became crystal clear where that unusual query was coming from; check it out...
ResultSet java.sql.DatabaseMetaData.getClientInfoProperties() throws SQLException
Retrieves a list of the client info properties that the driver supports. The result set contains the following columns
1. NAME String=> The name of the client info property
2. MAX_LEN int=> The maximum length of the value for the property
3. DEFAULT_VALUE String=> The default value of the property
4. DESCRIPTION String=> A description of the property. This will typically contain information as to where this property is stored in the database.
The ResultSet is sorted by the NAME column
Returns:
A ResultSet object; each row is a supported client info property
You can clearly see that unusual query (select NULL NAME, -1 MAX_LEN, NULL DEFAULT_VALUE, NULL DESCR) matches what the JavaDocs say about the DatabaseMetaData.getClientInfoProperties() method. Wow, right!?
This is the code that was performing the function. As best as I can tell, it looks correct from a "closing of the ResultSet" standpoint - not sure what was happening that would keep the ResultSet open - it clearly being closed in the finally block.
log.debug(">>>>>> DatabaseMetaData Client Info Properties (jdbc driver)...");
ResultSet rsDmd = null;
try {
boolean hasResults = false;
rsDmd = dmd.getClientInfoProperties();
while (rsDmd.next()) {
hasResults = true;
log.debug(">>>>>>>>> NAME = '" + rsDmd.getString("NAME") + "'; DEFAULT_VALUE = '" + rsDmd.getString("DEFAULT_VALUE") + "'; DESCRIPTION = '" + rsDmd.getString("DESCRIPTION") + "'");
}
if (!hasResults) {
log.debug(">>>>>>>>> DatabaseMetaData Client Info Properties was empty (nothing returned by jdbc driver)");
}
} catch (SQLException sqleDmd) {
log.warn("DatabaseMetaData Client Info Properties (jdbc driver) not supported or no access to system tables under current id");
} finally {
if (rsDmd != null) {
rsDmd.close();
}
}
Looking at the logs, when an Oracle connection was used, the >>>>>>>>> DatabaseMetaData Client Info Properties was empty (nothing returned by jdbc driver) line was logged, so an exception wasn't being thrown, but no record was being returned either. I can only assume that the ojdbc6 (11.2.0.x.x) driver doesn't properly support the getClientInfoProperties() method - it is weird (I think) that an exception wasn't being thrown, as the query itself is missing the FROM keyword (it won't run when executed in TOAD for example). And no matter what, the ResultSet should have at least been getting closed (the connection itself would still be in use though - maybe this causes Oracle to not release the cursors even though the ResultSet was closed).
So all of the work I was doing was in a branch (I mentioned in a comment to my original question that I was working in trunk - my mistake - I was in a branch that was already created thinking it was based on trunk code and not modified - I failed to do my due diligence here), so I checked the SVN commit history and found that this additional logging functionality was added by a fellow teammate a couple of weeks ago (fortunately it hasn't been promoted to trunk or to higher environments - note this code works fine against our Sybase database). My update from the SVN branch brought in his code, but I never really paid attention to what got updated (my bad). I spoke with him about what this code was doing against Oracle, and we agreed to remove the code from the logging method. We also put in place a check to only log the connection metadata when in our development environment (he said he added this code to help troubleshoot some driver version and auto commit questions he had). Once this was done, I was able to run my load tests without any open cursor issues (success!!!).
Anyway, I wanted to answer this question because when I searched for select NULL NAME, -1 MAX_LEN, NULL DEFAULT_VALUE, NULL DESCR and ORA-01000 open cursors no credible hits were returned (the majority of the hits returned were to make sure you are closing your connection resources, i.e., ResultSets, Statements, etc.). I think this shows it was the database metadata query through JDBC against Oracle was the culprit of the ORA-01000 error. I hope this is useful to others. Thanks.

Fetching data from Oracle database does not work first time

We are trying to fetch data from Oracle DB using a PreparedStatement. It keeps fetching zero records while the same runs and fetches data when run from PL/SQL developer.
We found the root cause while trying to debug. While debugging the code fetched the two records properly.
We did a temporary fix by placing this piece of code.
ResultSet rs = ps.executeQuery();
while(!rs.hasNext()){
ps.executeQuery();}
This works. But this is not the best solution since it results in an unwanted DB hit.It clearly looks like a time issue. We also explicitly committed earlier transactions since they can affect the result of this query.
What could be causing this. What's the best way to solve this?
The method is quite big: I'll just post some parts here:
private static boolean loadCommission(Member member){
Connection conn = getConnection("schema1"); //obtained through connection pool
//insertion into table
conn.close();
Conn conn2 = getConnection("schema2"); //obtained through connection pool
PreparedStatement ps = conn2.prepareStatement(sql);
//this sql combines data from schema1
// and 2 with DB links
ResultSet rs = ps.executeQuery();
//business logic
conn2.close();
return true;
}
Thanks
We tried a few more things yesterday. We replaced the second connection code with direct jdbc connection like so
Connection conn = DriverManager.getConnection(URL, USER, PASS);
This too works. Now we are not sure if the delay is in getting connection from pool or in completing previous transaction like we thought earlier.

If your query selects from a materialized view, then there may be some elapsed time before it will yield results (as materialized views do not necessarily refresh instantly after a commit, depending upon how they've been created).
If this is the case, then you can resolve the problem by either selecting directly from the base table (or equivalent non-materialized views), or forcing the materialized view to refresh.

Java ResultSet working with postgresql

I have a table on my database where it holds 2 columns: uid1 - someone's id, uid2 - his friend.
I want to create a list where I someone's friend - till 5 depth connection.
So I built the following recursive method:
private void recursive(int theUid,ResultSet rs,ArrayList<Integer> friends,int count,int next,PreparedStatement pstmt) throws SQLException{
if(count>=1 && next==theUid){
return;
}
else if(count>=DEPTH_OF_CONNECTION){
return;
}
else{
pstmt.setInt(1,next);
rs=pstmt.executeQuery();
count++;
while(rs.next()) {
friends.add(rs.getInt(1));
recursive(theUid,rs,friends,count,rs.getInt(1),pstmt);
}
}
}
}
And I got the following error:
Exception in thread "main" org.postgresql.util.PSQLException: This
ResultSet is closed. at
org.postgresql.jdbc2.AbstractJdbc2ResultSet.checkClosed(AbstractJdbc2ResultSet.java:2654) at
org.postgresql.jdbc2.AbstractJdbc2ResultSet.next(AbstractJdbc2ResultSet.java:1786)
Can you help me find what is the problem?

Javadoc for jdbc Statement says
By default, only one ResultSet object per Statement object can be open at the same time.
So my guess is you are trying to open too many resultsets for the same PreparedStatement at the same time.
edit:
Postgres jdbc driver doc seems to confirm it:
You can use a single Statement instance as many times as you want. You could create one as soon as you open the connection and use it for the connection's lifetime. But you have to remember that only one ResultSet can exist per Statement or PreparedStatement at a given time.

From what I can tell, you are using the ResultSet object to store your statement results and then passing it as a parameter to the recursive function call. So basically you are overwriting the variable and losing the reference to it. You should either write your sql statement to retrieve the data you need or not pass the same ResultSet object to the recursive call.

How to add records to databse via sql in Java

I am working a Airsoft application.
I'm trying to add records to a MS Access Database via SQL in Java. I have established a link to the database, with the following:
try
{
//String Driver = "sun.java.odbc.JdbcOdbcDriver";
Class.forName("net.ucanaccess.jdbc.UcanaccessDriver");
Connection conn = DriverManager.getConnection("jdbc:ucanaccess://" + URL,"","");
Statement stmt = conn.createStatement();
System.out.println("Connection Established!");
ResultSet rs = stmt.executeQuery("SELECT * FROM AirsoftGunRentals");
tblRent.setModel(DbUtils.resultSetToTableModel(rs));
}
catch(Exception ex)
{
JOptionPane.showMessageDialog(null, "Error");
}
I am using Ucanaccess to access my MS database. It is reading the database and is displaying to a JTable. However, I need to create three JButtons to add, delete and update the table. I have tried to code the add button, and I have tried to add a record, but it crashes and gives me errors.
try
{
//String Driver = "sun.java.odbc.JdbcOdbcDriver";
Class.forName("net.ucanaccess.jdbc.UcanaccessDriver");
Connection conn = DriverManager.getConnection("jdbc:ucanaccess://" + URL,"","");
Statement stmt = conn.createStatement();
System.out.println("Connection Established!");
String Query= "INSERT INTO AirsoftGunRentals(NameOfGun, Brand, TypeOfGuns, NumberOfMagazines,Extras,NumberAvailable,UnitRent)"+
"VALUES('"+pName+"','"+pBrand+"','"+pTypeOfGun+"','"+pNumMags+"','"+pExtras+"','"+pNumberAvail+"','"+pRent+"');";
ResultSet rs = stmt.executeQuery(Query);
JOptionPane.showMessageDialog(null, "Success!");
}
catch(Exception ex)
{
JOptionPane.showMessageDialog(null, "Error");
}
I have attempted all three, hoping for a result. But am still getting big errors. The only difference between the buttons is that one adds, one deletes and one updates the table. Other then that, the code is the same, minus variables.

As Brahim mentionned it, you should use stmt.executeUpdate(Query) whenever you update / insert or delete data. Also with this particular query, given your String concatenation (see end of line), there is no space between the ")" and the "VALUES" which probably causes a malformed query.
However, I can see from your code that you are not very experienced with such use-cases, and I'd like to add some pointers before all hell breaks loose in your project :
Use PreparedStatement instead of Statement and replace variables by placeholders to prevent SQL Injection.
The code that you are using here is extremely prone to SQL injection - if any user has any control over any of the variables, this could lead to a full database dump (theft), destruction of data (vandalism), or even in machine takeover if other conditions are met.
A good advice is to never use the Statement class, better be safe than sorry :)
Respect Java Conventions (or be coherent).
In your example you define the String Query, while all the other variables start with lower-case (as in Java Conventions), instead of String query. Overtime, such little mistakes (that won't break a build) will lead to bugs due to mistaking variables with classnames etc :)
Good luck on your road to mastering this wonderful language ! :)

First add a space before the quotation marks like this :
String Query= "INSERT INTO AirsoftGunRentals(NameOfGun, Brand, TypeOfGuns, NumberOfMagazines,Extras,NumberAvailable,UnitRent) "+
" VALUES('"+pName+"','"+pBrand+"','"+pTypeOfGun+"','"+pNumMags+"','"+pExtras+"','"+pNumberAvail+"','"+pRent+"');";
And use stmt.executeUpdate(Query); instead of : stmt.executeQuery(Query);in your insert, update and delete queries. For select queries you can keep it.

I managed to find an answer on how to add, delete and update records to a MS Access DB. This is what I found, after I declared the connection, and the prepped statement. I will try to explain to the best I can. I had to add values individually using this:
(pstmt = Prepped Statement Variable)
pstmt.setWhatever(1,Variable);
And it works fine now. I use the same method to delete and update records.
This is the basic query format:
String SQLInsert = "INSERT INTO Tbl VALUES(NULL,?,?,?,?)";
The NULL in the statement is the autonumber in the table. and .setWhatever() clause replaces the question marks with the data types. Thus manipulating the database.
Thank you everyone for all your contributions. It helped a lot, and made this section a lot more understandable.