How would you detect a lost connection using JDBC in a reliable and an efficient way? Currently I'm using a Vertica JDBC driver. I don't think I can use connection pools as I need to control to which node I'm connected at all times. Things I thought of so far:
Assume SQLException is a lost connection. Seems unreliable.
Check ex.getCause and look for a socket exception. Not sure if checking cause is a good practice.
Check error code which I know is a case for a lost connection. Also seems a bit unreliable.
Edit:
I've tried isValid and isClosed. isValid throws an exception and isClosed always returns false regardless.
From version 1.6, you can theoretically use the Connection.isValid method but I am not sure if every vendor supports it. An other alternative is using a connection pool which manages connections and you get the connection from them and release it when you don't need it. Connection pools take the responsiblity of maintaining connections and they always provide valid connections.
Related
I am working on mongodb connection pooling & I came across this option we can set in mongo client : "MaxConnectionIdleTime".
It basically means that connection will die after this time when sitting idle.
The default value is zero & it's stated that in case of zero, there's no limit.
Does it mean that once a connection has been created it won't die at all & will be kept in pool forever ?
Assuming there's space for new connections to be created in the pool. like min connections=10 & max connections =1000. & also, the max connection time isn't set.
If you can suggest a method to test it out on my own, that'll be really helpful too.
Please let me know if there's any way I can improve the question.
Thanks!
"Die" is a sloppy term and it is not clear what you mean by it.
If no idle time is set, the connection will not be proactively closed by the driver upon that idle time elapsing. That's all.
Connections may be closed by the driver in the following other circumstances:
It experiences a network error or a timeout.
Another connection to the server that this connection is associated with experiences a network error, causing the server to be marked unusable by the driver.
A connection may become unusable because of a network error. The connection may be unusable without the driver knowing about it. The driver often detects unusability of a connection when trying to use it (i.e. write or read something) which means a connection may be unusable (one might say dead) while the driver thinks it is perfectly fine, for a long time.
So I have a Java process that runs indefinitely as a TCP server (receives messages from another process, and has onMsg handlers).
One of the things I want to do with the message in my Java program is to write it to disk using a database connection to postgres. Right now, I have one single static connection object which I call every time a message comes in. I do NOT close and reopen the connection for each message.
I am still a bit new to Java, I wanted to know 1) whether there are any pitfalls or dangers with using one connection object open indefinitely, and 2) Are there performance benefits to never closing the connection, as opposed to reopening/closing every time I want to hit the database?
Thanks for the help!
I do NOT close and reopen the connection for each message.
Yes you do... at least as far as the plain Connection object is concerned. Otherwise, if you ever end up with a broken connection, it'll be broken forever, and if you ever need to perform multiple operations concurrently, you'll have problems.
What you want is a connection pool to manage the "real" connections to the database, and you just ask for a connection from the pool for each operation and close it when you're done with it. Closing the "logical" connection just returns the "real" connection to the pool for another operation. (The pool can handle keeping the connection alive with a heartbeat, retiring connections over time etc.)
There are lots of connection pool technologies available, and it's been a long time since I've used "plain" JDBC so I wouldn't like to say where the state of the art is at the moment - but that's research you can do for yourself :)
Creating a database connection is always a performance hit. Only a very naive implementation would create and close a connection for each operation. If you only needed to do something once an hour, then it would be acceptable.
However, if you have a program that performs several database accesses per minute (or even per second for larger apps), you don't want to actually close the connection.
So when do you close the connection? Easy answer: let a connection pool handle that for you. You ask the pool for a connection, it'll give you an open connection (that it either has cached, or if it really needs to, a brand new connection). When you're finished with your queries, you close() the connection, but it actually just returns the connection to the pool.
For very simple programs setting up a connection pool might be extra work, but it's not very difficult and definitely something you'll want to get the hang of. There are several open source connection pools, such as DBCP from Apache and 3CPO.
We have seen connection droughts in our system every once in a while, and the problem seems to be that Sessions are not being returned to the connection pool quick enough. I wrote a test that seems to confirm using Session.disconnect() on the sessions (after being done with one) will solve this problem. However, I also timed these calls, and it seems like using disconnect is increasing running time by 3 times.
According to the docs (http://docs.jboss.org/hibernate/core/3.5/api/org/hibernate/Session.html#disconnect() ), disconnect should be returning it to the connection pool. However, the doc also says it "closes" the connection. I'm not sure what it means because I know for a fact that Session.close() does more than disconnect, and what good would a connection pool be if you close the connection before returning it?
In any case, I'm wondering why a method that returns the session to the connection pool would be anything but instantaneous and essentially free. Surely thats the whole point of a connection pool, right?
Any ideas would be appreciated.
Because none of the JDBC operations have a concept of timeout, is it advisable to try to close JDBC connection of the overdue operation from the other thread if timeout is exceeded?
I am aware of Statement.cancel method, however it does not seem to work in all cases. Also, the driver that I am using does not respond to interrupts.
I think Connection.close is, unfortunately, the lowest common denominator that can be used to prevent resource leakage in case the database operation had ran amok.
I am aware that when using connection pooling I have to close the real connection object.
For derby it should be ok:
http://db.apache.org/derby/docs/10.8/devguide/cdevconcepts23499.html
According to the JDBC spec you can share Connections and Statements with multiple threads
The best answer is, it depends on what driver you are using. But from the jdbc stack point of view, it should be ok.
I'd wrap that operation in a ThreadPoolExecutor with a timeout. It'll give you more control over what's happening.
I not very good in java.
I have made a website for a client but am continuously getting an error like Server connection failure during transaction. Due to underlying exception: 'com.mysql.jdbc.exceptions.MySQLSyntaxErrorException: User root already has more than 'max_user_connections' active connections'.
The website hosting that I am using provides only 10 max_user_connections. But if I continuously use that site, I get this error because of continuously hits on the webserver.
What can be the reason behind this?
Am I not closing the connections right?
I have closed all the connections using con.close().
Please help
Regards Apurv
To open the connection I have used
Connection con=null;
Statement st=null;
Class.forName("com.mysql.jdbc.Driver").newInstance();
String useq="root";
String paq="manager";
String url="jdbc:mysql://localhost:3306/jayna?autoReconnect=true";
con=DriverManager.getConnection(url,useq,paq);
st=con.createStatement();
To close the connection I have used
if(rs!=null){
rs.close();
}
if(st!=null){
st.close();
}
if(con!=null){
con.close();
}
I haven't used a database pool but what can be the use of that when I am closing each of the connection properly??
This seems like a case of connection leak.
Are you sure, you have closed all the connections?
Conneciton.close() throws IOException. Check if it is successfully able to close connections.
Use netstat or other tools to find whether connections are really closed or are in WAITING state or something.
I think you should increase number of connections. Because if there are 10 slow query and you get 11 requests to your site 11 request couldn't be processed and you get this error.
So:
Try increase a number of connections.
Try to find slow queries(using slow query log) and optimize them
Connection pooling is a technique to provide a set of ready-to-use connections, one of the advantages being that you save the creation/opening time on each call. Another advantage is that the connection pooler can help detect abandoned connections, ie connections that the application forgot to close.
There's a standard connection pool in Tomcat, for 5.x version Tomcats look here for some info, for version 6 look here and for version 7 info can be found here. Its removeAbandoned and logabandoned features can help you determine whether your app really forgets to close connections, or 10 just isn't enough - see Andrej's suggestion, you should profile your queries.
As others have said, you either are not getting connections closed or your application simply needs more than 10 concurrent connections under some load conditions.
If the error always occurs on the 11th request, it's likely you're never getting the connections closed.
If it occurs sometime later, unpredictably, and goes away on its own, it's more likely 10 simply isn't sufficient for certain load scenarios.
If it occurs later, unpredictably, but never goes away on its own, it's possible you're failing to close connections only in specific cases that aren't hit every time.
If 10 is too small for some load scenarios (option 2), you should both check your queries and code logic to ensure you're not holding connections way longer than necessary and you should probably try to move to a Connection Pool, as others have suggested. Among other things, creating new Connections from scratch has more overhead than reusing them from a pool, so that could be causing individual accesses to take much longer than necessary.
This problem can be solved by using a Singleton class structure for initializing connection objects.
Using the Singleton pattern, whenever a connection object is initialized, rather than creating a new object, it will look for existing instance of connection object and use that one, if it exists.