Develop Reference

JDBC Pool - How to properly use

I use JDBC pool for my Tomcat web applications.
As I read in docs, I need get connection, execute query and close.
After close, connection returns to pool.
I think Way 1 better when query DB depend on external event and for the task which running every 5sec Way 2 more correct.
Could someone explain which way use for task which repeat every 5 sec?
PS: I skip extra checks in code to make code looks readable.
Way #1 Gets connection from pool and close every 5 sec
Connection c = null;
Statement s = null;
ResultSet rs = null;
DataSource ds = ... Get DataSource ...
while(running) {
try {
c = ds.getConnection();
s = c.createStatement();
rs = s.executeQuery('SELECT data FROM my_table');
... do something with result ...
} catch (SQLException sec) {
... print exception ...
} finally {
try {
rs.close();
s.close();
c.close();
} catch (SQLException sec) { ... print exception ... }
... Thread sleep 5 seconds and repeat ...
}
}
Way #2 Get connection before loop and close after, reconnect inside loop
Connection c = null;
Statement s = null;
ResultSet rs = null;
DataSource ds = ... Get DataSource ...
c = ds.getConnection();
while(running) {
try {
s = c.createStatement();
rs = s.executeQuery('SELECT data FROM my_table');
... do something with result ...
} catch (SQLException sec) {
... print exception ...
... if connection lost, try reconnect and execute query again ...
} finally {
try {
rs.close();
s.close();
} catch (SQLException sec) {
... print exception ...
}
... Thread sleep 5 seconds and repeat ...
}
}
c.close();
Pool config
<Resource name="jdbc/pg_mega" auth="Container"
type="javax.sql.DataSource" driverClassName="org.postgresql.Driver"
url="jdbc:postgresql://127.0.0.1:6432/db"
factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
username="***" password="****"
defaultAutoCommit="true"
initialSize="1"
maxActive="300"
maxTotal="300"
maxIdle="20"
minIdle="5"
maxWait="10000"
validationQuery="select 1"
validationInterval="30000"
testWhileIdle="false"
testOnBorrow="true"
testOnReturn="false"
timeBetweenEvictionRunsMillis="30000"
minEvictableIdleTimeMillis="30000"
/>

I think the most common pattern is this:
Connection conn = null;
PreparedStatement stmt = null;
ResultSet res = null;
try {
conn = ds.getConnection();
stmt = conn.prepareStatement(sqlStatement);
//
// ....
res = stmt.executeQuery();
// use the resultset
conn.commit();
} catch (SQLException e) {
// Manage the exception
try {
conn.rollback();
} catch (SQLException e1) {
// SWALLOW
}
} finally {
close(res);
close(stmt);
close(conn);
}
I use these helper functions to safely close without too much boilerplate, from java 7 on you can autoclose so these helper are no use anymore.
public static void close(Connection conn) {
try {
if (conn != null)
conn.close();
} catch (SQLException e) {
// SWALLOW
}
}
public static void close(Statement stmt) {
try {
if (stmt != null)
stmt.close();
} catch (SQLException e) {
// SWALLOW
}
}
public static void close(ResultSet res) {
try {
if (res != null)
res.close();
} catch (SQLException e) {
// SWALLOW
}
}
You should really be sure to close the connection in the finally statement, bad things happens if you do not close the connection, and if (for example) the rs is null in your example (not so difficult) you won't close the connection.
Getting and releasing the connection from the pool is not a performance problem, it happens in microseconds , thousands of times faster then any possible query.
The reason why you do not release the connection eagerly is transactions, you want to keep the same connection for the whole transaction (no way around this).
When you commit (or rollback) then you don't need that peculiar connection anymore, so just release it.
Another hint, close the connection in finally, even if you catch SQL Exceptions, because there are always Runtime Exceptions and even Errors (which you will not catch), but finally will be executed even in the face of OutOfMemoryError or ClassDefNotFoundError or any other, and the connection will be returned to the pool.
Last but not least is the pool that will try to reconnect in case of disconnection, in fact the pool will just ditch invalid connection and create a fresh batch when needed.
You should choose a good policy of connection validation, a bad choice will lead to much extra time in getting the connection thus hitting hard on performance, or to exceptions caused by invalid connection acquired from the pool.
Optimizing the pool is like many other performance tuning tasks: HARD .
For example:
testOnBorrow="true" will hit the DB before acquiring the connection, it is safe, but it will cost, tens or hundreds of time slower that not checking on borrow.
testWhileIdle="true" instead is less safe (you could get an invalid connection) but is much faster an has the advantage of keeping alive the connections.
You have to choose considering how you use connection, how you deal with errors, where is the DB (on the same machine, on a lan, on a wan) and many other factors.

Way #2 is not correct when you use a pool. If you use a pool, you should always try to keep the connection out of the pool ("leased") for as short as possible to get the most out of pool usage. If you do not use a pool, you have to consider the cost of creating and destroying connections and manage the connection's life cycle.
If you use a pool, a leased connection must always be returned to the pool (a connection is returned to the pool when you close the connection). If connections are not returned to the pool (i.e. connections are leaked), the pool will be empty soon and your application will stop working. This is especially important when things go wrong (e.g. in your code example, when rs is null due to a query error, the connection will be leaked). To prevent connections from leaking, consider using a tool like Sql2o which has built-in protection against connection leakage.
Also reduce the number of connections in the pool. Start with minIndle="1" and maxActive="4". Use stress-testing to determine the upper-limit of the pool-size (more connections in a pool usually do more harm than good, see also About Pool Sizing from HikariCP which has more good articles about database connection pools).

Insert Data In Oracle DataBase

//in context listener
Statement stmt;
try {
Class.forName("oracle.jdbc.OracleDriver");
Connection conn = DriverManager.getConnection("jdbc:oracle:thin:#127.0.0.1:1521:orcl","abc", "abc");
stmt = conn.createStatement();
sce.getServletContext().setAttribute("stmt", stmt);
} catch (ClassNotFoundException e) {
} catch (SQLException e) {
}
//in Servlet page
try {
Statement stmts=(Statement) getServletContext().getAttribute("stmt");
stmts.executeUpdate("INSERT INTO COMPANYS values(1,'ahmed',5,'t012','t345','email#eamil','adressadress')");
System.out.println("connection succeed");
} catch (SQLException e) {
System.out.println("connection fail");
}
in Servlet page "try code" is execute and "connection succeed" is appearing but in oracle database there is no data inserted >> why???

use con.setAutoCommit(false); after conn has created. Once the transaction completes i.e.,after executeUpdate(...) call con.commit(); It will solve your issue if their are no errors displayed on the console.

Try it again with below steps
Set auto commit as false
Commit after updating the record to make it persisted in the database
Rollback if there is any exception while inserting the record
Don't forget to clean-up the environment such as ResultSet, Statement etc.
Code in finally block to close the resources
Don't keep connection opened for long time.
Must read
Is it mandatory to close all ResultSets, Statements and Connections?
Java Tutorial - Using Transactions
Find a sample code HERE with detailed inline comments.

When my app loses connection, how should I recover it?

I have an app that I'm connecting to a MySQL database. It loses connection in the middle of the night and then spouts about null connections and JDBC hasn't received messages in X seconds.
I call getConnection() before I do anything that requires communication with the SQL server.
This is my getConnection() method:
private Connection getConnection() {
try {
if (connection != null) {
if (connection.isClosed() || !connection.isValid(10000)) {
this.initializeRamsesConnection();
}
} else {
this.initializeRamsesConnection();
}
} catch (Exception e) {
debug("Connection failed: " + e);
}
return connection;
}
In the initializeRamsesConnection() method I put the password and so on information into a string and then I create the connection in the standard JDBC way.
Then I call this method:
private Connection getConnectionFromConnectionString() {
Connection con = null;
String driver = "com.mysql.jdbc.Driver";
try {
Class.forName(driver);//jdbc sorcery
//if there is no connection string
if (getConnectionString() == null) {
HMIDatabaseAdapter.debug("No connection string");
}
//makes a string out of the values of db/host
String str = getConnectionString();
//if there is no driver
if (driver == null) {
debug("" + ": " + "No driver");
}
//Tries to make a connection from the connection string, username, and the password.
con = DriverManager.getConnection(str, username, password);
//if for some reason the connection is null
if (con == null) {
HMIDatabaseAdapter.debug("CONNECTION IS NULL, WHAT?");
}
} catch (Exception ex) {
HMIDatabaseAdapter.debug("getConnection() " + ex);
}
return con;
}
What can I change in either of these methods to accommodate losing connection?

This is not the correct way of retrieving a connection. You're retrieving the connection and assigning it as an instance (or worse, static) variable of the class. Basically, you're keeping the connection open forever and reusing a single connection for all queries. This may end up in a disaster if the queries are executed by different threads. Also, when it's been kept open for too long, the DB will reclaim it because it assumes that it's dead/leaked.
You should acquire and close the connection in the shortest possible scope. I.e. in the very same try block as where you're executing the query. Something like this:
public Entity find(Long id) throws SQLException {
Entity entity = null;
try (
Connection connection = dataSource.getConnection(); // This should return a NEW connection!
PreparedStatement statement = connection.prepareStatement(SQL_FIND);
) {
statement.setLong(1, id);
try (ResultSet resultSet = preparedStatement.executeQuery()) {
if (resultSet.next()) {
entity = new Entity(
resultSet.getLong("id"),
resultSet.getString("name"),
resultSet.getInt("value")
);
}
}
}
return entity;
}
If you worry about connecting performance and want to reuse connections, then you should be using a connection pool. You could homegrow one, but I strongly discourage this as you seem to be pretty new to the stuff. Just use an existing connection pool like BoneCP, C3P0 or DBCP. Note that you should not change the JDBC idiom as shown in the above example. You still need to acquire and close the connection in the shortest possible scope. The connection pool will by itself worry about actually reusing, testing and/or closing the connection.
See also:
Am I Using JDBC Connection Pooling?
JDBC MySql connection pooling practices to avoid exhausted connection pool

Where in your code are the errors on losing connection coming from? This would probably be the best place to start.
Off the top of my head (and I may be wrong), JDBC connections will only close on an actual fatal error, so you won't know they've failed until you try to do something.
What I've done in the past is to invalidate the connection at the point of failure and retry periodically.

Maybe this is what you are looking for:
http://dev.mysql.com/doc/refman/5.0/en/auto-reconnect.html
For java see autoReconnect:
http://dev.mysql.com/doc/refman/5.0/en/connector-j-reference-configuration-properties.html

Java JDBC connection status

I am (successfully) connecting to a database using the following:
java.sql.Connection connect = DriverManager.getConnection(
"jdbc:mysql://localhost/some_database?user=some_user&password=some_password");
What should I be checking to see if the connection is still open and up after some time?
I was hoping for something like connect.isConnected(); available for me to use.

Your best chance is to just perform a simple query against one table, e.g.:
select 1 from SOME_TABLE;
Oh, I just saw there is a new method available since 1.6:
java.sql.Connection.isValid(int timeoutSeconds):
Returns true if the connection has not been closed and is still valid.
The driver shall submit a query on the connection or use some other
mechanism that positively verifies the connection is still valid when
this method is called. The query submitted by the driver to validate
the connection shall be executed in the context of the current
transaction.

Nothing. Just execute your query. If the connection has died, either your JDBC driver will reconnect (if it supports it, and you enabled it in your connection string--most don't support it) or else you'll get an exception.
If you check the connection is up, it might fall over before you actually execute your query, so you gain absolutely nothing by checking.
That said, a lot of connection pools validate a connection by doing something like SELECT 1 before handing connections out. But this is nothing more than just executing a query, so you might just as well execute your business query.

Use Connection.isClosed() function.
The JavaDoc states:
Retrieves whether this Connection object has been closed. A
connection is closed if the method close has been called on it or if
certain fatal errors have occurred. This method is guaranteed to
return true only when it is called after the method Connection.close
has been called.

You also can use
public boolean isDbConnected(Connection con) {
try {
return con != null && !con.isClosed();
} catch (SQLException ignored) {}
return false;
}

If you are using MySQL
public static boolean isDbConnected() {
final String CHECK_SQL_QUERY = "SELECT 1";
boolean isConnected = false;
try {
final PreparedStatement statement = db.prepareStatement(CHECK_SQL_QUERY);
isConnected = true;
} catch (SQLException | NullPointerException e) {
// handle SQL error here!
}
return isConnected;
}
I have not tested with other databases. Hope this is helpful.

The low-cost method, regardless of the vendor implementation, would be to select something from the process memory or the server memory, like the DB version or the name of the current database. IsClosed is very poorly implemented.
Example:
java.sql.Connection conn = <connect procedure>;
conn.close();
try {
conn.getMetaData();
} catch (Exception e) {
System.out.println("Connection is closed");
}

Here is a simple solution if you are using JDBC to get the default connection
private Connection getDefaultConnection() throws SQLException, ApiException {
Connection connection = null;
try {
connection = dataSource.getConnection ();
}catch (SQLServerException sqlException) {
// DB_UNAVAILABLE EXCEPTION
}
return connection;
}

Connection Retry using JDBC

I am using JDBC to connect to a DB. Since the network was slow, i could get the connection after some 2 or 3 retry manually. Is it possible to retry the connection automatically if connection fails? I am using SQLServer 2008 database.
Thanks

A bit decent connection pool is already configureable to do so, for example BoneCP. Most do even do it by default. If you don't use a connection pool but just the basic DriverManager#getConnection() approach, then you have to re-execute it yourself in a while loop as long as the Connection is null.
Here's a basic kickoff example:
Connection connection = null;
while (connection == null) {
try {
connection = DriverManager.getConnection(url, username, password);
} catch (SQLException e) {
logger.info("Connecting failed, retrying...");
}
}
return connection;
This can of course be optimized more with bit longer pauses in between and by setting a maximum retry count, etcetera.

Here the code by which it re-try it 3 times to connect,you can also change the cont instead of 3 this can be changed to any number of times(5,10 etc)
Connection connection = null;
while (connection == null && count<3){
try {
String Connection="jdbc:sqlserver://"+serverip+';'+"database="+dbName+';'+"user=" +Username+';'+"password=" +Password;
Class.forName(classname);
connection = DriverManager.getConnection(Connection);
}catch (SQLException e){
count++;
System.err.println("Connecting failed, retrying...");
}
}
count=0;
return connection;
}