I'm trying to determine the best way to ping a database via JDBC. By 'best' I mean fast and low overhead. For example, I've considered executing this:
"SELECT 1 FROM DUAL"
but I believe the DUAL table is Oracle-specific, and I need something more generic.
Note that Connection has an isClosed() method, but the javadoc states that this cannot be used to test the validity of the connection.
With JDBC 4 you can use isValid(int) (JavaDoc) from the Connection Interface. This basically does the trial statement for you.
Some driver implement this by sending the correct dummy SQL to the database and some directly uses low level operations which reduces the parsing overhead.
However beware of the timeout, some drivers (DB/400 and Oracle Thin) do spawn a new time thread for each invocation, which is not really acceptable for most Pool validation scenarios). And Oracle also does not seem to use a prepared statement, so it’s kind of relying on the implicit cache.
Yes, that would be Oracle-only, but there is no generic way to do this in JDBC.
Most connection pool implementations have a configuration parameter where you can specify the SQL that will be used for ping, thus pushing the responsiblity to figure out how to do it to the user.
That seems like the best approach unless someone comes up with a little helper tool for this (of course, it precludes using potentially even faster non-SQL-based methods like Oracle's internal ping function)
MySQL has a nice mechanism, documented in this SO answer. From the answer:
"/* ping */ SELECT 1"
This will actually cause the driver send a ping to the server and return a fake, light-weight, result set.
Having said that, #eckes answer is the best (using JDBC 4's Connection.isValid(int)).
I'm not aware of a generic solution, either. For IBM's UDB on iSeries (and perhaps other DB2 systems) it would be
select 1 from SYSIBM.SYSDUMMY1;
You could try to get the db name from the connection meta data and execute a matching sql staement. E.g.
Connection con = null;
Statement st = null;
ResultSet rs = null;
try {
con = dataSource.getConnection();
String dbProductName = con.getMetaData().getDatabaseProductName();
Statement st = con.createStatement();
if ( "PostgreSQL".equalsIgnoreCase(dbProductName) ) {
rs = st.executeQuery("select version();");
} else if ( "Oracle".equalsIgnoreCase(dbProductName) ) {
rs = st.executeQuery("select 1 from dual");
} else {
...
}
} catch ( Exception ex ) {
System.out.prinln("DB not reachable");
} finally {
// close statement, connection etc.
...
}
I may be out to lunch on this one, but could you simply execute some non-sense query, such as:
SELECT * FROM donkey_giraffe_87
I don't know very much about JDBC's error handling, but perhaps you could check to see if the database is at least telling you that the table does not exist. If JDBC's error codes are vendor-specific, the Spring Framework has some utilities for mapping these codes to more meaningful exceptions.
Related
I work on some really old code (10 years+). And its (probably) practically impossible to get The Query that is going to be Executed later.
But I need to log the Queries in case of an error.
I tried to read the Metadata of the statement. But that does not do the job when I try to get the exact query.
System.out.println(preparedStatement); is also not working because we use a really old JDBC Driver, which does not implement a nice "toString" for the prepared Statement and I am not allowed to change the driver.
public void doQuery(Connection conn) throws SQLException{
PreparedStatement st=null;
ResultSet result=null;
st=createStatement(conn);
result = st.executeQuery();
...
}
public abstract PreparedStatement createStatement(Connection conn) throws SQLException;
the "createStatement" is an abstract method with 46 (magical) implementations, which are all different.
That's what makes it really hard for me to figure out how the "Statement" is created at all.
I only want to have something like
String query = preparedStatement.getQueryString();
Much time ago I used p6spy: a proxy for the real JDBC driver that logs real queries before passing them to the real driver.
Another solution is datasource-proxy that, as the name implies, proxies a DataSource, not the low-level driver. I've never tried it.
I am trying to better instrument which web applications make use of Oracle (11g) connections in our Tomcat JDBC connection pool when a connection is created and closed; this way, we can see what applications are using connections by monitoring the V$SESSION table. This is working, but since adding this "instrumentation" I am seeing ORA-01000: maximum open cursors exceeded errors being logged and noticing some connections being dropped out of the pool during load testing (which is probably fine as I have testOnBorrow enabled, so I'm assuming the connection is being flagged as invalid and dropped from the pool).
I have spent the better part of the week scouring the internet for possible answers. Here is what I have tried (all result in the open cursors error after a period of time)...
The below methods are all called the same way...
On Create
We obtain a connection from the pool
We call a method that executes the below code, passing in the context name of the web application
On Close
We have the connection being closed (returned to the pool)
Before we issue close() on the connection, we call a method that executes the code below, passing in "Idle" as the name to store in V$SESSION
Method 1:
CallableStatement cs = connection.prepareCall("{call DBMS_APPLICATION_INFO.SET_MODULE(?,?)}");
try {
cs.setString(1, appId);
cs.setNull(2, Types.VARCHAR);
cs.execute();
log.trace(">>> Executed Oracle DBMS_APPLICATION_INFO.SET_MODULE with module_name of '" + appId + "'");
} catch (SQLException sqle) {
log.error("Error trying to call DBMS_APPLICATION_INFO.SET_MODULE('" + appId + "')", sqle);
} finally {
cs.close();
}
Method 2:
I upgraded to the 12c OJDBC driver (ojdbc7) and used the native setClientInfo method on the connection...
// requires ojdbc7.jar and oraclepki.jar to work (setEndToEndMetrics is deprecated in ojdbc7)
connection.setClientInfo("OCSID.CLIENTID", appId);
Method 3:
I'm currently using this method.
String[] app_instrumentation = new String[OracleConnection.END_TO_END_STATE_INDEX_MAX];
app_instrumentation[OracleConnection.END_TO_END_CLIENTID_INDEX] = appId;
connection.unwrap(OracleConnection.class).setEndToEndMetrics(app_instrumentation, (short)0);
// in order for this to be sent, a query needs to be sent to the database - this works fine when a
// connection is created, but when it is closed, we need a little something to get the change into the db
// try using isValid()
connection.isValid(1);
Method 4:
String[] app_instrumentation = new String[OracleConnection.END_TO_END_STATE_INDEX_MAX];
app_instrumentation[OracleConnection.END_TO_END_CLIENTID_INDEX] = appId;
connection.unwrap(OracleConnection.class).setEndToEndMetrics(app_instrumentation, (short)0);
// in order for this to be sent, a query needs to be sent to the database - this works fine when a
// connection is created, but when it is closed, we need a little something to get the change into the db
if ("Idle".equalsIgnoreCase(appId)) {
Statement stmt = null;
ResultSet rs = null;
try {
stmt = connection.createStatement();
rs = stmt.executeQuery("select 1 from dual");
} finally {
if (rs != null) {
rs.close();
}
if (stmt != null) {
stmt.close();
}
}
}
When I query for open cursors, I notice the following SQL being returned on the account being used in the pool (for each connection in the pool)...
select NULL NAME, -1 MAX_LEN, NULL DEFAULT_VALUE, NULL DESCR
This does not explicitly exist anywhere in our code, so I can only assume it is coming from the pool when running the validation query (select 1 from dual) or from the setEndToEndMetrics method (or from the DBMS_APPLICATION_INFO.SET_MODULE proc, or from the isValid() call). I tried to be explicit in creating and closing Statement (CallableStatement) and ResultSet objects in methods 1 and 4, but they made no difference.
I don't want to increase the number of allowed cursors, as this will only delay the inevitable (and we have never had this issue until I added in the "instrumentation").
I've read through the excellent post here (java.sql.SQLException: - ORA-01000: maximum open cursors exceeded), but I must still be missing something. Any help would be greatly appreciated.
So Mr. Poole's statement: "that query looks like it's getting fake metadata" set off a bell in my head.
I started to wonder if it was some unknown remnant of the validation query being run on the testOnBorrow attribute of the pool's datasource (even though the validation query is defined as select 1 from dual). I removed this from the configuration, but it had no effect.
I then tried removing the code that sets the client info in V$SESSION (Method 3 above); Oracle continued to show that unusual query and after only a few minutes, the session would hit the maximum open cursors limit.
I then found that there was a "logging" method in our DAO class that logged some metadata from the connection object (values for settings like current auto commit, current transaction isolation level, JDBC driver version, etc.). Within this logging was the use of the getClientInfoProperties() method on the DatabaseMetaData object. When I looked at the JavaDocs for this method, it became crystal clear where that unusual query was coming from; check it out...
ResultSet java.sql.DatabaseMetaData.getClientInfoProperties() throws SQLException
Retrieves a list of the client info properties that the driver supports. The result set contains the following columns
1. NAME String=> The name of the client info property
2. MAX_LEN int=> The maximum length of the value for the property
3. DEFAULT_VALUE String=> The default value of the property
4. DESCRIPTION String=> A description of the property. This will typically contain information as to where this property is stored in the database.
The ResultSet is sorted by the NAME column
Returns:
A ResultSet object; each row is a supported client info property
You can clearly see that unusual query (select NULL NAME, -1 MAX_LEN, NULL DEFAULT_VALUE, NULL DESCR) matches what the JavaDocs say about the DatabaseMetaData.getClientInfoProperties() method. Wow, right!?
This is the code that was performing the function. As best as I can tell, it looks correct from a "closing of the ResultSet" standpoint - not sure what was happening that would keep the ResultSet open - it clearly being closed in the finally block.
log.debug(">>>>>> DatabaseMetaData Client Info Properties (jdbc driver)...");
ResultSet rsDmd = null;
try {
boolean hasResults = false;
rsDmd = dmd.getClientInfoProperties();
while (rsDmd.next()) {
hasResults = true;
log.debug(">>>>>>>>> NAME = '" + rsDmd.getString("NAME") + "'; DEFAULT_VALUE = '" + rsDmd.getString("DEFAULT_VALUE") + "'; DESCRIPTION = '" + rsDmd.getString("DESCRIPTION") + "'");
}
if (!hasResults) {
log.debug(">>>>>>>>> DatabaseMetaData Client Info Properties was empty (nothing returned by jdbc driver)");
}
} catch (SQLException sqleDmd) {
log.warn("DatabaseMetaData Client Info Properties (jdbc driver) not supported or no access to system tables under current id");
} finally {
if (rsDmd != null) {
rsDmd.close();
}
}
Looking at the logs, when an Oracle connection was used, the >>>>>>>>> DatabaseMetaData Client Info Properties was empty (nothing returned by jdbc driver) line was logged, so an exception wasn't being thrown, but no record was being returned either. I can only assume that the ojdbc6 (11.2.0.x.x) driver doesn't properly support the getClientInfoProperties() method - it is weird (I think) that an exception wasn't being thrown, as the query itself is missing the FROM keyword (it won't run when executed in TOAD for example). And no matter what, the ResultSet should have at least been getting closed (the connection itself would still be in use though - maybe this causes Oracle to not release the cursors even though the ResultSet was closed).
So all of the work I was doing was in a branch (I mentioned in a comment to my original question that I was working in trunk - my mistake - I was in a branch that was already created thinking it was based on trunk code and not modified - I failed to do my due diligence here), so I checked the SVN commit history and found that this additional logging functionality was added by a fellow teammate a couple of weeks ago (fortunately it hasn't been promoted to trunk or to higher environments - note this code works fine against our Sybase database). My update from the SVN branch brought in his code, but I never really paid attention to what got updated (my bad). I spoke with him about what this code was doing against Oracle, and we agreed to remove the code from the logging method. We also put in place a check to only log the connection metadata when in our development environment (he said he added this code to help troubleshoot some driver version and auto commit questions he had). Once this was done, I was able to run my load tests without any open cursor issues (success!!!).
Anyway, I wanted to answer this question because when I searched for select NULL NAME, -1 MAX_LEN, NULL DEFAULT_VALUE, NULL DESCR and ORA-01000 open cursors no credible hits were returned (the majority of the hits returned were to make sure you are closing your connection resources, i.e., ResultSets, Statements, etc.). I think this shows it was the database metadata query through JDBC against Oracle was the culprit of the ORA-01000 error. I hope this is useful to others. Thanks.
I am running a Neo4j server (2.0.3) and am accessing it via Cypher queries through the Neo4j JDBC. It works well for most of the uses I have had so far.
I currently want to run a query that will return a result set that is of the type - TYPE_SCROLL_SENSITIVE and not the default TYPE_FORWARD_ONLY.
Here is the code I use to create my Statement and return the ResultSet:
Statement all = con.createStatement(ResultSet.TYPE_SCROLL_SENSITIVE, ResultSet.CONCUR_READ_ONLY);
results = all.executeQuery(query);
However when I try and use the method beforeFirst() on the ResultSet I get an SQLException. Which is a result of either a closed ResultSet or one that is TYPE_FORWARD_ONLY. I have used the isClosed() method to ensure it is not closed.
Within the Neo4j JDBC Driver there is the following createStatement method:
#Override
public Statement createStatement(int resultSetType, int resultSetConcurrency) throws SQLException
{
return debug(new Neo4jStatement(this));
}
So from what I can see, it is simply ignoring the type and concurrency constants that are passed to it. I realize not all JDBC drivers allow for different type result sets. I guess I am asking why the Neo4j one does not? And if not, are there any plans to implement support for this? I would find it useful, and thought others might as well?
I found out that Java doesn't let ResultSet to be return as a data type. And would like to ask whether is there any better solution to return results after executing query?
try {
Class.forName("com.mysql.jdbc.Driver");
Connection con = DriverManager.getConnection( "jdbc:mysql://localhost:3306/shopping mall?zeroDateTimeBehavior=convertToNull", "root", "" );
java.sql.Statement stmt = con.createStatement() ;
String query = "SELECT * FROM shops WHERE SHOPNAME LIKE \'%" + shopName + "%\' ORDER BY shopname ASC"; // Select and sort using user input
ResultSet rs = stmt.executeQuery(query) ;
// How to return the result after executing the query?
}
catch ( SQLException err ) {
return null;
}
Don't mind the return null; part. Was testing something. Basically this is my current code for now. How can I return the result after executing query?
You need to provide better context for the error. Is this, for example, a JAX-WS web service endpoint? Anyway, as stated in the trace, your error is a web service error--not a JDBC error. This error can happen for many reasons--usually related to something wrong with the way you are defining the API to your service.
You are certainly allowed to return a ResultSet from a method even if that is a very bad idea, especially from a web service endpoint. A ResultSet can't be serialized into a SOAP message. More generally, to return a ResultSet betrays implementation details to the callers of the method. Why should they know you are using JDBC? Or even that you are talking to a relational (or any) database at all?
The better approach is to populate a model object relevant to your domain with the data in the ResultSet, and that object will be serialized to SOAP via JAXB or whatever you use. Or maybe you just return some text from the database, in which case JAX-WS knows what to do.
Also, make sure you do something with SQLException so you can trace the cause of your actual JDBC errors.
You write a method to retrieve info from a database, where should the data be processed and put into a model class? If you want code that is loosely coupled, then 98% of the time you would process the result set within the same method.
What happens if the query needs to change? You want the changes to be localized into a small of a subset of code as possible.
Take a look at Spring-JDBC JdbcTemplate. This method
List<Map<String, Object>> queryForList(String sql)
returns a List that contains a Map per row. You can use it or make something similar
How can I get SQL query that was received by MySQL server when it raises exception?
I have a code similar to the following:
Connection con = null;
PreparedStatement ps = null;
try {
con = DbConnectionManager.getConnection();
ps = con.prepareStatement(query);
setStatementParameters(ps, params);
ps.executeUpdate();
} catch (SQLExeption e) {
// How to get wrong query here?
} finally {
DbConnectionManager.closeConnection(ps, con);
}
Where query variable is like "INSERT into someTable (qwe, asd) VALUES (?, ?)"
The question is how can I get query string in the catch block?
The SQLException may or may not have the query string contained in its exception message. You can't depend on it. If you just want to see it for debugging purposes, though, then that's probably your best bet. However, in this particular example that's not a problem because you have direct access to the query variable that you used to set up the query in the first place.
I've run across another solution to this problem.
The MySQL JDBC driver overrides the toString of PreparedStatement to display the query as it is sent to the database. This is implementation dependent so it may not the best thing to rely on, but it's very simple to get at. I'm now using this to dump query text to a log file for debugging purposes. While there are probably other solutions that are more portable and future-proof, this has the advantage of getting exactly the string that the MySQL driver says it's sending to the database.
The string comes back with an object ID, then a colon, then the SQL string. You can split it on the colon to get just the SQL.
The type com.mysql.jdbc.PreparedStatement also exposes a protected method call asSql(). You could override the class with your own implementation that gives public access to this method. From looking at the disassembly of the class's toString() method, it seems to be using asSql() to get the actual SQL string. This approach adds the problem of how to instantiate your subclass, though; the simplest approach is just to use the toString that you already have access to, without even having to downcast your PreparedStatement to a MySQL-specific subtype.
Again, just be aware that the maintainers of the MySQL API probably don't consider this part of the public interface to their software (JDBC defines the standard interface), so they may make changes later that would break this mechanism. For the time being, though, it will get the job done.
This is true for the version of the MySQL driver I'm currently using, which is 5.1.7.