So I was looking into the c3p0 API to debug one of our production issues which was resulting in a stack overflow error while checking out a connection.
I found below comments in BasicResourcePool class's checkoutResource method:
/*
* This function recursively calls itself... under nonpathological
* situations, it shouldn't be a problem, but if resources can never
* successfully check out for some reason, we might blow the stack...
*
* by the semantics of wait(), a timeout of zero means forever.
*/
I want to know what might be the reasons the resources from this pool can never get successfully checked out.
The answer might help me look into what might be going possibly wrong in my application.
So, although it's a reasonable guess, mere pool exhaustion (what happens if you leak or forget to close() Connections) won't lead to the stack overflow.
The stack overflow occurs when checkoutResource(...)
finds a Connection available to check out, and "preliminarily" checks it out; then
something goes wrong, indicating that the preliminarily checked-out Connection is not usable; so
the function goes "back to the well", recursively calling itself to try again with a fresh Connection
The mystery is in the "something goes wrong" part. There are really two things that can go wrong:
(most likely!) You have testConnectionOnCheckout set to true and all Connections are failing their Connection tests
The Connection happened to be removed (e.g. expired for exceeding maxIdleTime or maxConnectionAge) from the pool during the checkout procedure
If you are seeing this, the first thing to examine is whether there is a problem with your Connection or your Connection testing regime. Try...
Log com.mchange.v2.resourcepool.BasicResourcePool at DEBUG or FINE and look for Exceptions indicating inability to checkout. You can grep for A resource could not be refurbished for checkout. Alternatively, switch Connection testing regimes to testing idle Connections and on Connection check-in rather than on check-out, and watch the problem show-up in a perhaps less disruptive way.
If you are doing something that would force the pool to really churn Connections, setting very short timeouts or something, it's imaginable that the race condition is biting. Check your values for configuration properties maxConnectionAge, maxIdleTime, and maxIdleTimeExcessConnections and make sure that they are reasonable or not set (i.e. left at reasonable defaults).
Related
I am working on mongodb connection pooling & I came across this option we can set in mongo client : "MaxConnectionIdleTime".
It basically means that connection will die after this time when sitting idle.
The default value is zero & it's stated that in case of zero, there's no limit.
Does it mean that once a connection has been created it won't die at all & will be kept in pool forever ?
Assuming there's space for new connections to be created in the pool. like min connections=10 & max connections =1000. & also, the max connection time isn't set.
If you can suggest a method to test it out on my own, that'll be really helpful too.
Please let me know if there's any way I can improve the question.
Thanks!
"Die" is a sloppy term and it is not clear what you mean by it.
If no idle time is set, the connection will not be proactively closed by the driver upon that idle time elapsing. That's all.
Connections may be closed by the driver in the following other circumstances:
It experiences a network error or a timeout.
Another connection to the server that this connection is associated with experiences a network error, causing the server to be marked unusable by the driver.
A connection may become unusable because of a network error. The connection may be unusable without the driver knowing about it. The driver often detects unusability of a connection when trying to use it (i.e. write or read something) which means a connection may be unusable (one might say dead) while the driver thinks it is perfectly fine, for a long time.
I asked this question (How do I call java.sql.Connection::abort?) and it led me to another question.
With
java.sql.Connection conn = ... ;
What is the difference between
conn.close();
and
conn.abort(...);
?
You use Connection.close() for a normal, synchronous, close of the connection. The abort method on the other hand is for abruptly terminating a connection that may be stuck.
In most cases you will need to use close(), but close() can sometimes not complete in time, for example it could block if the connection is currently busy (eg executing a long running query or update, or maybe waiting for a lock).
The abort method is for that situation: the driver will mark the connection as closed (hopefully) immediately, the method returns, and the driver can then use the provided Executor to asynchronously perform the necessary cleanup work (eg making sure the statement that is stuck gets aborted, cleaning up other resources, etc).
I hadn't joined the JSR-221 (JDBC specification) Expert Group yet when this method was defined, but as far as I'm aware, the primary intended users for this method is not so much application code, but connection pools, transaction managers and other connection management code that may want to forcibly end connections that are in use too long or 'stuck'.
That said, application code can use abort as well. It may be faster than close (depending on the implementation), but you won't get notified of problems during the asynchronous clean up, and you may abort current operations in progress.
However keep in mind, an abort is considered an abrupt termination of the connection, so it may be less graceful than a close, and it could lead to unspecified behaviour. Also, I'm not sure how well it is supported in drivers compared to a normal close().
Consulting the java docs seems to indicate that abort is more thorough than close, which is interesting.
abort...
Terminates an open connection. Calling abort results in: The
connection marked as closed Closes any physical connection to the
database Releases resources used by the connection Insures that any
thread that is currently accessing the connection will either progress
to completion or throw an SQLException.
close...
Releases this Connection object's database and JDBC resources
immediately instead of waiting for them to be automatically released.
Calling the method close on a Connection object that is already closed
is a no-op.
So it seems if you are only concerned with releasing the objects, use close. If you want to make sure it's somewhat more "thread safe", using abort appears to provide a more graceful disconnect.
Per Mark Rotteveel's comment (which gives an accurate summary of the practical difference), my interpretation was incorrect.
Reference: https://docs.oracle.com/javase/8/docs/api/java/sql/Connection.html#close--
Everywhere we see that the Statement.close() is 'handled' by eating up the exception that it throws. What are the cases where it can throw an exception in the first place? And what does it mean for the connection with which this statement was created?
In other words, when does statement.close() throw an exception and would the connection still be 'healthy' to be used for creating new statements?
Also, what happens if resultset.close() throws?
First, consider what the close() method might need to do, and what might cause an exception.
E.g. a PreparedStatement might have created a stored procedure, which needs to be deleted by the close() method. executeQuery() may have opened a cursor, which is used by the ResultSet, and close() needs to close that cursor.
Exception could of course be an internal error, but is most likely a communication error, preventing the close operation from succeeding.
So, what does that mean? It means that resources are not being explicitly cleaned up. Since your operation is already complete, it's generally ok to ignore those close() exceptions, since resources will be reclaimed eventually anyway.
However, since the cause is probably a communication error, the connection is likely broken, which means that you'll just get another exception on whatever you try next, making it even less likely that your handling of the close() exception matters.
To be safe, an exception means that something is wrong, and unless you examine the exception to understand how bad it is, you should abort whatever you're doing. A new connection should be established if you want to try again.
But, as already mentioned, ignoring close() exceptions aren't really a big issue. It may lead to resource leaks, but if the problem is bad, you're just going to get another exception on your next action anyway.
Simplest case for such an exception: the connection, that handled the statement is closed before you are trying to close the statement or if the statement was closed -somehow- already. speaking for jdbc, the connection should be healthy.
In general: As Peter stated, if the documentation of the driver does not contain any recommendations how to handle such an exception, you can only log or debug it. maybe you could re-establish the connection to be sure it is healthy.
When you close a statement, a lot of things can happen. These basic things can happen when closing a statement:
The open result set - if any - is closed, which may require communication to the database
The statement handle on the database server is released, which requires communication to the database
Given this involves communication to the database, all kinds of errors can occur: file system errors, network connection problems, etc. These may be safe to ignore, but could also indicated something very wrong with your application or database.
A secondary effect of a statement close can be a transaction completion (a commit or rollback). This could - for example - happen in auto-commit mode when you execute a data-modifying statement that produces a result set: the transaction ends when the result set is closed by the close of the statement. If this transaction commit fails, and you ignore it, your application may have just had a data-loss event (because the data was not persisted), and you just went ahead and ignored it.
In other words: you should not just ignore or swallow exceptions from Statement.close() unless you are absolutely sure there will be no detrimental effects. At minimum log them so you can trace them in your logs (and maybe define alerts on the number of exceptions logged), but always consider if you need to wrap them in application-specific exceptions and throw them higher up the call chain for handling, or - for commit failures - if you need to retry anything.
We have seen connection droughts in our system every once in a while, and the problem seems to be that Sessions are not being returned to the connection pool quick enough. I wrote a test that seems to confirm using Session.disconnect() on the sessions (after being done with one) will solve this problem. However, I also timed these calls, and it seems like using disconnect is increasing running time by 3 times.
According to the docs (http://docs.jboss.org/hibernate/core/3.5/api/org/hibernate/Session.html#disconnect() ), disconnect should be returning it to the connection pool. However, the doc also says it "closes" the connection. I'm not sure what it means because I know for a fact that Session.close() does more than disconnect, and what good would a connection pool be if you close the connection before returning it?
In any case, I'm wondering why a method that returns the session to the connection pool would be anything but instantaneous and essentially free. Surely thats the whole point of a connection pool, right?
Any ideas would be appreciated.
I not very good in java.
I have made a website for a client but am continuously getting an error like Server connection failure during transaction. Due to underlying exception: 'com.mysql.jdbc.exceptions.MySQLSyntaxErrorException: User root already has more than 'max_user_connections' active connections'.
The website hosting that I am using provides only 10 max_user_connections. But if I continuously use that site, I get this error because of continuously hits on the webserver.
What can be the reason behind this?
Am I not closing the connections right?
I have closed all the connections using con.close().
Please help
Regards Apurv
To open the connection I have used
Connection con=null;
Statement st=null;
Class.forName("com.mysql.jdbc.Driver").newInstance();
String useq="root";
String paq="manager";
String url="jdbc:mysql://localhost:3306/jayna?autoReconnect=true";
con=DriverManager.getConnection(url,useq,paq);
st=con.createStatement();
To close the connection I have used
if(rs!=null){
rs.close();
}
if(st!=null){
st.close();
}
if(con!=null){
con.close();
}
I haven't used a database pool but what can be the use of that when I am closing each of the connection properly??
This seems like a case of connection leak.
Are you sure, you have closed all the connections?
Conneciton.close() throws IOException. Check if it is successfully able to close connections.
Use netstat or other tools to find whether connections are really closed or are in WAITING state or something.
I think you should increase number of connections. Because if there are 10 slow query and you get 11 requests to your site 11 request couldn't be processed and you get this error.
So:
Try increase a number of connections.
Try to find slow queries(using slow query log) and optimize them
Connection pooling is a technique to provide a set of ready-to-use connections, one of the advantages being that you save the creation/opening time on each call. Another advantage is that the connection pooler can help detect abandoned connections, ie connections that the application forgot to close.
There's a standard connection pool in Tomcat, for 5.x version Tomcats look here for some info, for version 6 look here and for version 7 info can be found here. Its removeAbandoned and logabandoned features can help you determine whether your app really forgets to close connections, or 10 just isn't enough - see Andrej's suggestion, you should profile your queries.
As others have said, you either are not getting connections closed or your application simply needs more than 10 concurrent connections under some load conditions.
If the error always occurs on the 11th request, it's likely you're never getting the connections closed.
If it occurs sometime later, unpredictably, and goes away on its own, it's more likely 10 simply isn't sufficient for certain load scenarios.
If it occurs later, unpredictably, but never goes away on its own, it's possible you're failing to close connections only in specific cases that aren't hit every time.
If 10 is too small for some load scenarios (option 2), you should both check your queries and code logic to ensure you're not holding connections way longer than necessary and you should probably try to move to a Connection Pool, as others have suggested. Among other things, creating new Connections from scratch has more overhead than reusing them from a pool, so that could be causing individual accesses to take much longer than necessary.
This problem can be solved by using a Singleton class structure for initializing connection objects.
Using the Singleton pattern, whenever a connection object is initialized, rather than creating a new object, it will look for existing instance of connection object and use that one, if it exists.