JDBC Instrumentation and ORA-01000: maximum open cursors exceeded - java

I am trying to better instrument which web applications make use of Oracle (11g) connections in our Tomcat JDBC connection pool when a connection is created and closed; this way, we can see what applications are using connections by monitoring the V$SESSION table. This is working, but since adding this "instrumentation" I am seeing ORA-01000: maximum open cursors exceeded errors being logged and noticing some connections being dropped out of the pool during load testing (which is probably fine as I have testOnBorrow enabled, so I'm assuming the connection is being flagged as invalid and dropped from the pool).
I have spent the better part of the week scouring the internet for possible answers. Here is what I have tried (all result in the open cursors error after a period of time)...
The below methods are all called the same way...
On Create
We obtain a connection from the pool
We call a method that executes the below code, passing in the context name of the web application
On Close
We have the connection being closed (returned to the pool)
Before we issue close() on the connection, we call a method that executes the code below, passing in "Idle" as the name to store in V$SESSION
Method 1:
CallableStatement cs = connection.prepareCall("{call DBMS_APPLICATION_INFO.SET_MODULE(?,?)}");
try {
cs.setString(1, appId);
cs.setNull(2, Types.VARCHAR);
cs.execute();
log.trace(">>> Executed Oracle DBMS_APPLICATION_INFO.SET_MODULE with module_name of '" + appId + "'");
} catch (SQLException sqle) {
log.error("Error trying to call DBMS_APPLICATION_INFO.SET_MODULE('" + appId + "')", sqle);
} finally {
cs.close();
}
Method 2:
I upgraded to the 12c OJDBC driver (ojdbc7) and used the native setClientInfo method on the connection...
// requires ojdbc7.jar and oraclepki.jar to work (setEndToEndMetrics is deprecated in ojdbc7)
connection.setClientInfo("OCSID.CLIENTID", appId);
Method 3:
I'm currently using this method.
String[] app_instrumentation = new String[OracleConnection.END_TO_END_STATE_INDEX_MAX];
app_instrumentation[OracleConnection.END_TO_END_CLIENTID_INDEX] = appId;
connection.unwrap(OracleConnection.class).setEndToEndMetrics(app_instrumentation, (short)0);
// in order for this to be sent, a query needs to be sent to the database - this works fine when a
// connection is created, but when it is closed, we need a little something to get the change into the db
// try using isValid()
connection.isValid(1);
Method 4:
String[] app_instrumentation = new String[OracleConnection.END_TO_END_STATE_INDEX_MAX];
app_instrumentation[OracleConnection.END_TO_END_CLIENTID_INDEX] = appId;
connection.unwrap(OracleConnection.class).setEndToEndMetrics(app_instrumentation, (short)0);
// in order for this to be sent, a query needs to be sent to the database - this works fine when a
// connection is created, but when it is closed, we need a little something to get the change into the db
if ("Idle".equalsIgnoreCase(appId)) {
Statement stmt = null;
ResultSet rs = null;
try {
stmt = connection.createStatement();
rs = stmt.executeQuery("select 1 from dual");
} finally {
if (rs != null) {
rs.close();
}
if (stmt != null) {
stmt.close();
}
}
}
When I query for open cursors, I notice the following SQL being returned on the account being used in the pool (for each connection in the pool)...
select NULL NAME, -1 MAX_LEN, NULL DEFAULT_VALUE, NULL DESCR
This does not explicitly exist anywhere in our code, so I can only assume it is coming from the pool when running the validation query (select 1 from dual) or from the setEndToEndMetrics method (or from the DBMS_APPLICATION_INFO.SET_MODULE proc, or from the isValid() call). I tried to be explicit in creating and closing Statement (CallableStatement) and ResultSet objects in methods 1 and 4, but they made no difference.
I don't want to increase the number of allowed cursors, as this will only delay the inevitable (and we have never had this issue until I added in the "instrumentation").
I've read through the excellent post here (java.sql.SQLException: - ORA-01000: maximum open cursors exceeded), but I must still be missing something. Any help would be greatly appreciated.

So Mr. Poole's statement: "that query looks like it's getting fake metadata" set off a bell in my head.
I started to wonder if it was some unknown remnant of the validation query being run on the testOnBorrow attribute of the pool's datasource (even though the validation query is defined as select 1 from dual). I removed this from the configuration, but it had no effect.
I then tried removing the code that sets the client info in V$SESSION (Method 3 above); Oracle continued to show that unusual query and after only a few minutes, the session would hit the maximum open cursors limit.
I then found that there was a "logging" method in our DAO class that logged some metadata from the connection object (values for settings like current auto commit, current transaction isolation level, JDBC driver version, etc.). Within this logging was the use of the getClientInfoProperties() method on the DatabaseMetaData object. When I looked at the JavaDocs for this method, it became crystal clear where that unusual query was coming from; check it out...
ResultSet java.sql.DatabaseMetaData.getClientInfoProperties() throws SQLException
Retrieves a list of the client info properties that the driver supports. The result set contains the following columns
1. NAME String=> The name of the client info property
2. MAX_LEN int=> The maximum length of the value for the property
3. DEFAULT_VALUE String=> The default value of the property
4. DESCRIPTION String=> A description of the property. This will typically contain information as to where this property is stored in the database.
The ResultSet is sorted by the NAME column
Returns:
A ResultSet object; each row is a supported client info property
You can clearly see that unusual query (select NULL NAME, -1 MAX_LEN, NULL DEFAULT_VALUE, NULL DESCR) matches what the JavaDocs say about the DatabaseMetaData.getClientInfoProperties() method. Wow, right!?
This is the code that was performing the function. As best as I can tell, it looks correct from a "closing of the ResultSet" standpoint - not sure what was happening that would keep the ResultSet open - it clearly being closed in the finally block.
log.debug(">>>>>> DatabaseMetaData Client Info Properties (jdbc driver)...");
ResultSet rsDmd = null;
try {
boolean hasResults = false;
rsDmd = dmd.getClientInfoProperties();
while (rsDmd.next()) {
hasResults = true;
log.debug(">>>>>>>>> NAME = '" + rsDmd.getString("NAME") + "'; DEFAULT_VALUE = '" + rsDmd.getString("DEFAULT_VALUE") + "'; DESCRIPTION = '" + rsDmd.getString("DESCRIPTION") + "'");
}
if (!hasResults) {
log.debug(">>>>>>>>> DatabaseMetaData Client Info Properties was empty (nothing returned by jdbc driver)");
}
} catch (SQLException sqleDmd) {
log.warn("DatabaseMetaData Client Info Properties (jdbc driver) not supported or no access to system tables under current id");
} finally {
if (rsDmd != null) {
rsDmd.close();
}
}
Looking at the logs, when an Oracle connection was used, the >>>>>>>>> DatabaseMetaData Client Info Properties was empty (nothing returned by jdbc driver) line was logged, so an exception wasn't being thrown, but no record was being returned either. I can only assume that the ojdbc6 (11.2.0.x.x) driver doesn't properly support the getClientInfoProperties() method - it is weird (I think) that an exception wasn't being thrown, as the query itself is missing the FROM keyword (it won't run when executed in TOAD for example). And no matter what, the ResultSet should have at least been getting closed (the connection itself would still be in use though - maybe this causes Oracle to not release the cursors even though the ResultSet was closed).
So all of the work I was doing was in a branch (I mentioned in a comment to my original question that I was working in trunk - my mistake - I was in a branch that was already created thinking it was based on trunk code and not modified - I failed to do my due diligence here), so I checked the SVN commit history and found that this additional logging functionality was added by a fellow teammate a couple of weeks ago (fortunately it hasn't been promoted to trunk or to higher environments - note this code works fine against our Sybase database). My update from the SVN branch brought in his code, but I never really paid attention to what got updated (my bad). I spoke with him about what this code was doing against Oracle, and we agreed to remove the code from the logging method. We also put in place a check to only log the connection metadata when in our development environment (he said he added this code to help troubleshoot some driver version and auto commit questions he had). Once this was done, I was able to run my load tests without any open cursor issues (success!!!).
Anyway, I wanted to answer this question because when I searched for select NULL NAME, -1 MAX_LEN, NULL DEFAULT_VALUE, NULL DESCR and ORA-01000 open cursors no credible hits were returned (the majority of the hits returned were to make sure you are closing your connection resources, i.e., ResultSets, Statements, etc.). I think this shows it was the database metadata query through JDBC against Oracle was the culprit of the ORA-01000 error. I hope this is useful to others. Thanks.

Related

Error or wrong type of code while creating connection & execute SQL query statements using mysql in java

I'm using MySQL 5.7 with Java in Eclipse, and the connection statement below code below is causing an error when I try to connect:
try
{
//1. Get a connection to database
jdbc:mysql://localhost:3306/databaseName?autoReconnect=true;useSSL=false;
// 2. Create a statement
Statement myStmt=myConn.createStatement();
// 3. Execute SQL query
ResultSet myRs=myStmt.executeQuery("select * from employee");
//4. Process the result set
while(myRs.next())
{
System.out.println(myRs.getString("last_name")+","+myRs.getString("first_name"));
}
}
catch(Exception exc){
exc.printStackTrace();
}
First things first.
Code will only be used to validate the error. So you must paste the error fired by your program.
Since we don't have enough information to the problem, I will just cover basic troubleshooting.
Basic trouble shooting:
Do you have the driver? if not, you can download it here.
Next, Do you have the driver on your project class path? If not yet, you must add it. see how here
Did you load the driver to the program? if not, Class.forName("com.mysql.jdbc.Driver"); // Load Driver like that before doing anything.
Did you establish the connection? if not, Connection con = DriverManager.getConnection("jdbc:mysql://127.0.0.1:3306/DATABASE","USERNAME","PASSWORD");//3306 or port number depends on you config, same with DATABASE, USERNAME, PASSWORD
After the connection were established, so you should create a statement object like Statement s = con.createStatement(); // Create Statement. This will be used to execute sql commands.
finally, you can execute the commands like s.execute("select * from employee"); // Execute Query NOTE that s here is the variable created on number 5.
If all of the above were properly done but still gets an error, check if your have the database server running. In you case, mysql. Make sure there were not other installation of mysql prior to your current mysql. Sometimes, it will mess up your database. Troubleshooting your mysql, see mysql official doc here
While possible error is the datatype of mysql to your java code or getting a column that does not exist on your query or worse the column does not exist on your table.
Hope that help you and other who needs it.

Sqlite problems: database file is locked

I ve got a sqlite database with several tables. One of them called studentsession. Inside my code I am calling plenty times the table for select, insert, updates. After a while I am receiving the message:
java.sql.SQLException: [SQLITE_BUSY] The database file is locked (database is locked)
One instance of the calling is the following:
String query3 = "select * from studentssession where id= ? and math= ? and level = ?";
PreparedStatement pst__ = connectionUsers.prepareStatement(query3);
pst__.setString(1, x);
pst__.setString(2, x1);
pst__.setString(3, x2);
ts2 = pst__.executeQuery();
I am trying to figure out if I have or not to close every time the prepared statement, and if there is a case that this is causing my problems.
EDIT: Is it possible to have a check for possible open references in the database, using for example a javafxbutton?
EDIT: Is there a way that I can check in my code whether there is a problem in the references to the table and locate and possible close them?
It's probably due to you having multiple open references to the sqlite database.
I'd start by closing your PreparedStatement in a finally block inside your while loop.
PreparedStatement pst__ = null;
try{
pst__ = connectionUsers.prepareStatement(query3);
pst__.setString(1, x);
pst__.setString(2, x1);
pst__.setString(3, x2);
ts2 = pst__.executeQuery();
}finally{
if(pst__ != null) {
pst__.close();
}
}
You should also close the database connection at the end of everything.
Also it is a bad practice to use multiple connections when connecting to SQLite. See
http://touchlabblog.tumblr.com/post/24474398246/android-sqlite-locking
Set your poolsize maxactive to 1 and try out.

Unable to obtain SQL execution plan in Java for Prepared- and CallableStatement with sqljdbc4.jar

We have recently implemented Microsoft Type 4 JDBC Driver (sqljdbc4.jar of Microsoft JDBC Driver 4.2 for SQL Server)in favor of jTDS. We are now having problems when trying to retrieve estimated- and actual execution plans for both Callable- and PreparedStatement objects from within our Java code.
The only lead I've been able to locate is:
Get the query plan using jdbc PreparedStatement on sql server
but no concrete answer have been provided yet.
The database is SQL Server 2008R2.
Example code is:
Connection cn = null;
PreparedStatement stmt = null;
Statement execplan_stmt = null;
ResultSet rs = null;
try
{
cn = getConnection();
execplan_stmt = cn.createStatement();
execplan_stmt.executeUpdate("SET SHOWPLAN_XML ON");
stmt = cn.prepareStatement("SELECT value FROM cfg_system_properties WHERE
property = ?");
stmt.execute();
rs = stmt.getResultSet();
while(rs != null && rs.next())
{
Object obj = rs.getObject(1);
System.out.println("Query plan {} " + obj);
}
execplan_stmt.execute("SET SHOWPLAN_XML OFF");
}
catch (SQLException e)
{
e.printStackTrace();
}
Even replacing the
while(rs != null && rs.next())
loop statement with
if(rs.next())
{
String exec_plan = rs.getString(1);
}
does not work, as rs.next() is false.
In this example the sql contains a placeholder character '?', which firstly causes a "java.sql.SQLException: Parameter #1 has not been set." error, which is unwanted. I do not want to execute the query, I want to obtain the execution plan, be it the actual or estimated query plan.
Replacing the placeholder with an actual value doesn't break it anymore, but no execution plan is returned.
Activating SQL profiler indicates a execution plan should be returned, but as the linked article correctly states, it appears that the actual result is returned and not the execution plan.
Is the driver perhaps throwing the execution plan away?
Are there any specific configurations required to be set when obtaining the database connection that need to be set in order to be able to do this?
Has anybody successfully done this?
SQL Server does not return estimated execution plans for the execution of prepared statements. jTDS has a quirk with executes Java PreparedStatement’s more like plain Statement’s causing SQL Server to return an estimated execution plan. The MS driver does not have this quirk and so no estimated execution plan is ever returned when executing a Java Prepared- or CallableStatement. When using jTDS, the estimated execution plan might be returned, but it isn’t guaranteed for all PreparedStatement’s.

different results from query in java code and on oracle db server: but no connection error

I have java code that connects to a remote oracle 11g EE db server. If i run a particular query in sqlplus it returns one result
SQL> SELECT COURSENAME from COURSES where skillID=1;
COURSENAME
--------------------
basic
But if I run the same query from the java code below it returns no results. I can copy the query syntax out of the query variable in the java debugger and running it on oracle so I know there is no syntax issue with the query. Also, it is not SQL exceptions or class not found exceptions so it seems to be running the query successfully -- just returning zero results.
What might be going on?
private String getCourseForSkill(int skillID){
try{
Class.forName("oracle.jdbc.OracleDriver");
String query="SELECT COURSENAME from COURSES where skillID=" + skillID ;
con = DriverManager.getConnection(url, user, password);
Statement stmt = con.createStatement();
rs = stmt.executeQuery(query);
rs.next();
return rs.getString("COURSENAME");
}
catch (ClassNotFoundException ex){
System.out.println(ex.getMessage());
}
catch (SQLException ex) {
System.out.println(ex.getMessage());
}
return null;
}
I think you're connecting to different Oracle instances, or more likely, as different Oracle users in the two cases
#GreyBeardedGeek the URL looks like "jdbc:oracle:thin:#website:port:orcl I get to the manual query by doing ssh#website, authenticating and then running command=sqlplus
Safer to run sqlplus <username>/<password>#<orainstancename>, because you can explicitly specify the oracle instance ID. In your case, it seems your program is using jdbc connection jdbc:oracle:thin:#website:port:orcl, so your orainstancename would be 'orcl' - just ensure that your tnsnames.ora file has the instance 'orcl' with the same 'port' as used by the jdbc connection
How to debug a little more
Run the following code:
con = DriverManager.getConnection(url, user, password);
con.setAutoCommit(false);
String insert="INSERT INTO COURSES (SKILLID, COURSE)"+ // can add other columns
"values (?, ?) );" // add ? for other columns
PreparedStatement ps = con.createPreparedStatement();
ps.setInt(1, 999);
ps.setString(2, "Complete Bullwarks");
// can set other columns
ps.executeUpdate(insert);
con.commit();
NOW connect manually, re-run your original select statement & see if the added row is there. If no error in java and no new row in Oracle: extremely likely you're using 2 different Oracle instances/schemas.
ALSO rerun your original java select code, but with SkillID=999 - extremely likely it will work.
Cheers
I had to do a commit to add the rows. When I typed commit; into the sql plus terminal then the remote jdbc connection could 'see' the rows. I am used to SQL server where you don't have to explicitly do these kinds of commits when using linq-to-sql or sql management studio.
It can be three issues.
1) skillID <> 1 in your Java code. Add debug and check.
2a) You are connecting to another database.
2b) You are connecting to the same database but SELECTING from a table in another schema.
To check 2a and 2b:
select user from dual; -- connect username
select name from v$database; -- database name
select host_name from v$instance; -- host name database is running on
This query returns all three into one result.
select user || '' || d.name || '' || i.host_name
from v$database d, v$instance i;
Assuming you are actually connecting to the same database this is caused by not committing the INSERT in the sql*plus connection.
Oracle by default does not run in auto-commit mode when connecting via OCI (which sql*plus uses to connect). Any DML(INSERT ...) executed in sql*plus will not be visible to any other session until it is committed. This is because Oracle provides a read committed isolation level by default. The only thing visible to other users across sessions are write locks.
It doesn't matter if you connect the second connection via JDBC or OCI, it won't see the changes till you commit the first connection.
To test this out try opening 2 sql*plus connections and run the following:
-- Executing DDL in Oracle causes an implicit commit before and after the
-- command so the second connection will see the existence of this table:
CREATE TABLE foobar ( x VARCHAR(1) );
Execute this in connection #1 - you should get zero (we haven't inserted anything yet):
SELECT COUNT(*) FROM foobar;
Execute this in connection #2:
INSERT INTO foobar ( x ) VALUES ( 'A' );
Execute this in connection #1 - you should still get zero (INSERT is not committed so connection #1 cannot see it):
SELECT COUNT(*) FROM foobar;
Execute this in connection #2:
COMMIT;
Execute this in connection #1 - you should get 1 (it's committed now):
SELECT COUNT(*) FROM foobar;

What is the best way to 'ping' a database via JDBC?

I'm trying to determine the best way to ping a database via JDBC. By 'best' I mean fast and low overhead. For example, I've considered executing this:
"SELECT 1 FROM DUAL"
but I believe the DUAL table is Oracle-specific, and I need something more generic.
Note that Connection has an isClosed() method, but the javadoc states that this cannot be used to test the validity of the connection.
With JDBC 4 you can use isValid(int) (JavaDoc) from the Connection Interface. This basically does the trial statement for you.
Some driver implement this by sending the correct dummy SQL to the database and some directly uses low level operations which reduces the parsing overhead.
However beware of the timeout, some drivers (DB/400 and Oracle Thin) do spawn a new time thread for each invocation, which is not really acceptable for most Pool validation scenarios). And Oracle also does not seem to use a prepared statement, so it’s kind of relying on the implicit cache.
Yes, that would be Oracle-only, but there is no generic way to do this in JDBC.
Most connection pool implementations have a configuration parameter where you can specify the SQL that will be used for ping, thus pushing the responsiblity to figure out how to do it to the user.
That seems like the best approach unless someone comes up with a little helper tool for this (of course, it precludes using potentially even faster non-SQL-based methods like Oracle's internal ping function)
MySQL has a nice mechanism, documented in this SO answer. From the answer:
"/* ping */ SELECT 1"
This will actually cause the driver send a ping to the server and return a fake, light-weight, result set.
Having said that, #eckes answer is the best (using JDBC 4's Connection.isValid(int)).
I'm not aware of a generic solution, either. For IBM's UDB on iSeries (and perhaps other DB2 systems) it would be
select 1 from SYSIBM.SYSDUMMY1;
You could try to get the db name from the connection meta data and execute a matching sql staement. E.g.
Connection con = null;
Statement st = null;
ResultSet rs = null;
try {
con = dataSource.getConnection();
String dbProductName = con.getMetaData().getDatabaseProductName();
Statement st = con.createStatement();
if ( "PostgreSQL".equalsIgnoreCase(dbProductName) ) {
rs = st.executeQuery("select version();");
} else if ( "Oracle".equalsIgnoreCase(dbProductName) ) {
rs = st.executeQuery("select 1 from dual");
} else {
...
}
} catch ( Exception ex ) {
System.out.prinln("DB not reachable");
} finally {
// close statement, connection etc.
...
}
I may be out to lunch on this one, but could you simply execute some non-sense query, such as:
SELECT * FROM donkey_giraffe_87
I don't know very much about JDBC's error handling, but perhaps you could check to see if the database is at least telling you that the table does not exist. If JDBC's error codes are vendor-specific, the Spring Framework has some utilities for mapping these codes to more meaningful exceptions.

Categories

Resources