I have lots of method in my database class that all of them have one statement, when should I close the connection? In each method or at the end of database class?
You should close the connection when you are finished your transaction. Since we don't know the contents of the class or its usage, it's impossible to tell when your access of the connection begins or ends.
Actually, that may not even be true if the connection is dedicated for a specific usage and not in a pool. You may want to keep it open for the duration of your application.
We've found that the best policy is to get a connection from the connection pool, execute a single transaction, and then put the connection back into the pool immediately. This way you don't have a connection being held onto for long blocks of logic, thus preventing other threads from using it - which is an issue for scalability.
As a best practice, you should close the connection in the logical place after you are done - right after all of database activity for that task is done.
Generally, you should close a connection in the same method that opens it. Closing and opening connections isn't an arduous task, since modern DB servers keep even closed connections on "hot standby", so they are quickly accessed through a connection pool. Leaving them open though...that can get you in trouble and can be a nightmare to debug.
use lombok and it will handle both try/catch and conn.close() for you
public void doSomething() throws SQLException {
#Cleanup Connection connection = database.getConnection();
}
lombok
It depends when and how repeatedly you are using these methods. If they are sequential you should close the connection only in the end, instead of open and close often
This largely depends on what your Database class does. If your methods are called individually at various times then the methods should be responsible for opening and closing the connection. However, if the class does some kind of big processing operation that calls many methods then you may want to open and close the connection outside of the individual methods.
The most important thing is that wherever you open the connection, you also close the connection. Otherwise you get into the business of making assumptions about the state of the connection which can get you into trouble.
Close the connection (and statement and resultset!) in the same method block as you've acquired it, in the finally block of the try block where they are opened.
The general idiom is:
public void doSomething() throws SQLException {
Connection connection = null;
try {
connection = database.getConnection();
} finally {
if (connection != null) try { connection.close(); } catch (SQLException logOrIgnore) {}
}
}
Closing of connection, statement and resultset should happen in reversed order as they're opened.
If your actual concern is performance, then consider using a connection pool to improve connecting performance, for example C3P0. This by the way does not change the general idiom. Just continue writing the same idiom, the connection pooling implementation will worry itself further under the hoods.
Also see this article for more practices/examples.
Always close the connection when you finish executing your transaction. It is a good practice to obtain and close your connection in the same method. If you have separate transactions that are tightly coupled you might execute them with the same connection, but for best practices, I try to execute one transaction per connection.
If it is an option, don't explicitly mess with connections at all. Use a full blown orm framework like hibernate or use something significantly more lightweight like spring jdbc templates.
It depends somewhat on the context your application operates in. If it's a web app you need to be careful to open your connection, do whatever work is needed, and close the connection quickly. However, in a C/S or batch environment it may be better to acquire the connection and hold onto it as long as the user is interacting "frequently" (for whatever value of "frequently" you choose), especially if the user has expectations of rapid response time and it's expensive (in terms of time or resources) to acquire the connection to your particular variety of database.
I like to set a timer every time the user has the app go to the database. If/when the timer expires, close the connection, then re-open the next time he/she/it wants to hit the database again. Timer expiration can be somewhere between 1 and 20 minutes. Just so long as it's less than the database's "inactivity disconnect" time.
Share and enjoy.
Related
I asked this question (How do I call java.sql.Connection::abort?) and it led me to another question.
With
java.sql.Connection conn = ... ;
What is the difference between
conn.close();
and
conn.abort(...);
?
You use Connection.close() for a normal, synchronous, close of the connection. The abort method on the other hand is for abruptly terminating a connection that may be stuck.
In most cases you will need to use close(), but close() can sometimes not complete in time, for example it could block if the connection is currently busy (eg executing a long running query or update, or maybe waiting for a lock).
The abort method is for that situation: the driver will mark the connection as closed (hopefully) immediately, the method returns, and the driver can then use the provided Executor to asynchronously perform the necessary cleanup work (eg making sure the statement that is stuck gets aborted, cleaning up other resources, etc).
I hadn't joined the JSR-221 (JDBC specification) Expert Group yet when this method was defined, but as far as I'm aware, the primary intended users for this method is not so much application code, but connection pools, transaction managers and other connection management code that may want to forcibly end connections that are in use too long or 'stuck'.
That said, application code can use abort as well. It may be faster than close (depending on the implementation), but you won't get notified of problems during the asynchronous clean up, and you may abort current operations in progress.
However keep in mind, an abort is considered an abrupt termination of the connection, so it may be less graceful than a close, and it could lead to unspecified behaviour. Also, I'm not sure how well it is supported in drivers compared to a normal close().
Consulting the java docs seems to indicate that abort is more thorough than close, which is interesting.
abort...
Terminates an open connection. Calling abort results in: The
connection marked as closed Closes any physical connection to the
database Releases resources used by the connection Insures that any
thread that is currently accessing the connection will either progress
to completion or throw an SQLException.
close...
Releases this Connection object's database and JDBC resources
immediately instead of waiting for them to be automatically released.
Calling the method close on a Connection object that is already closed
is a no-op.
So it seems if you are only concerned with releasing the objects, use close. If you want to make sure it's somewhat more "thread safe", using abort appears to provide a more graceful disconnect.
Per Mark Rotteveel's comment (which gives an accurate summary of the practical difference), my interpretation was incorrect.
Reference: https://docs.oracle.com/javase/8/docs/api/java/sql/Connection.html#close--
So I have a Java process that runs indefinitely as a TCP server (receives messages from another process, and has onMsg handlers).
One of the things I want to do with the message in my Java program is to write it to disk using a database connection to postgres. Right now, I have one single static connection object which I call every time a message comes in. I do NOT close and reopen the connection for each message.
I am still a bit new to Java, I wanted to know 1) whether there are any pitfalls or dangers with using one connection object open indefinitely, and 2) Are there performance benefits to never closing the connection, as opposed to reopening/closing every time I want to hit the database?
Thanks for the help!
I do NOT close and reopen the connection for each message.
Yes you do... at least as far as the plain Connection object is concerned. Otherwise, if you ever end up with a broken connection, it'll be broken forever, and if you ever need to perform multiple operations concurrently, you'll have problems.
What you want is a connection pool to manage the "real" connections to the database, and you just ask for a connection from the pool for each operation and close it when you're done with it. Closing the "logical" connection just returns the "real" connection to the pool for another operation. (The pool can handle keeping the connection alive with a heartbeat, retiring connections over time etc.)
There are lots of connection pool technologies available, and it's been a long time since I've used "plain" JDBC so I wouldn't like to say where the state of the art is at the moment - but that's research you can do for yourself :)
Creating a database connection is always a performance hit. Only a very naive implementation would create and close a connection for each operation. If you only needed to do something once an hour, then it would be acceptable.
However, if you have a program that performs several database accesses per minute (or even per second for larger apps), you don't want to actually close the connection.
So when do you close the connection? Easy answer: let a connection pool handle that for you. You ask the pool for a connection, it'll give you an open connection (that it either has cached, or if it really needs to, a brand new connection). When you're finished with your queries, you close() the connection, but it actually just returns the connection to the pool.
For very simple programs setting up a connection pool might be extra work, but it's not very difficult and definitely something you'll want to get the hang of. There are several open source connection pools, such as DBCP from Apache and 3CPO.
I have been unable to find an exact answer to this question. I'm using C3P0's ComboPooledDataSource. Which of these methodologies is better practice:
dataSource = connectionClass.getDataSource();
conn = dataSource.getConnection;
executeQuery(query1, conn);
executeQuery(query2, conn);
...
executeQuery(finalQuery, conn);
conn.close();
OR
executeQuery(query1);
executeQuery(query2);
...
executeQuery(finalQuery);
where executeQuery:
conn = dataSource.getConnection;
st = conn.createStatement();
rs = executeQuery(query);
conn.closed();
In short, I have to do a decent amount of queries every so often. Is it better to go with the first design, which gets the connection once for each batch and passes it as an argument. Or is it better to go with the second approach and just get a connection each time I call my executeQuery method. If I was using DriverManager I would obviously choose the first (only get the connection once), but when using the C3P0 package I am not sure if doing that is the right way to go or not. Or does it not matter with such a package?
With a connection pool, the difference is neglectible, because even if you use the second approach, bringing back a pooled connection takes little time. Still, using the first approach is the better way to go, because
It avoids the additional (little) overhead of getting a connection from the pool.
If you later need to introduce transactions (do all of your changes or, in case of an error, conveniently and securely roll back your changes), then the first approach is your only option.
Some comments/suggestions
If you application is single threaded (unless you mention), it does not matter. It even does not matter whether you use connection pool or not. Just use a single connection and pass the same to function where you need it.
Connection pools are useful when the use case involves multiple database connections simultaneously.
Since your application is a batch and single threaded, it does not warrant use of connection pool.
Regarding your application, both the approaches are equivalent. When you call connection.close() on pooled datasource connection, its not actually closed but returned to pool.
I not very good in java.
I have made a website for a client but am continuously getting an error like Server connection failure during transaction. Due to underlying exception: 'com.mysql.jdbc.exceptions.MySQLSyntaxErrorException: User root already has more than 'max_user_connections' active connections'.
The website hosting that I am using provides only 10 max_user_connections. But if I continuously use that site, I get this error because of continuously hits on the webserver.
What can be the reason behind this?
Am I not closing the connections right?
I have closed all the connections using con.close().
Please help
Regards Apurv
To open the connection I have used
Connection con=null;
Statement st=null;
Class.forName("com.mysql.jdbc.Driver").newInstance();
String useq="root";
String paq="manager";
String url="jdbc:mysql://localhost:3306/jayna?autoReconnect=true";
con=DriverManager.getConnection(url,useq,paq);
st=con.createStatement();
To close the connection I have used
if(rs!=null){
rs.close();
}
if(st!=null){
st.close();
}
if(con!=null){
con.close();
}
I haven't used a database pool but what can be the use of that when I am closing each of the connection properly??
This seems like a case of connection leak.
Are you sure, you have closed all the connections?
Conneciton.close() throws IOException. Check if it is successfully able to close connections.
Use netstat or other tools to find whether connections are really closed or are in WAITING state or something.
I think you should increase number of connections. Because if there are 10 slow query and you get 11 requests to your site 11 request couldn't be processed and you get this error.
So:
Try increase a number of connections.
Try to find slow queries(using slow query log) and optimize them
Connection pooling is a technique to provide a set of ready-to-use connections, one of the advantages being that you save the creation/opening time on each call. Another advantage is that the connection pooler can help detect abandoned connections, ie connections that the application forgot to close.
There's a standard connection pool in Tomcat, for 5.x version Tomcats look here for some info, for version 6 look here and for version 7 info can be found here. Its removeAbandoned and logabandoned features can help you determine whether your app really forgets to close connections, or 10 just isn't enough - see Andrej's suggestion, you should profile your queries.
As others have said, you either are not getting connections closed or your application simply needs more than 10 concurrent connections under some load conditions.
If the error always occurs on the 11th request, it's likely you're never getting the connections closed.
If it occurs sometime later, unpredictably, and goes away on its own, it's more likely 10 simply isn't sufficient for certain load scenarios.
If it occurs later, unpredictably, but never goes away on its own, it's possible you're failing to close connections only in specific cases that aren't hit every time.
If 10 is too small for some load scenarios (option 2), you should both check your queries and code logic to ensure you're not holding connections way longer than necessary and you should probably try to move to a Connection Pool, as others have suggested. Among other things, creating new Connections from scratch has more overhead than reusing them from a pool, so that could be causing individual accesses to take much longer than necessary.
This problem can be solved by using a Singleton class structure for initializing connection objects.
Using the Singleton pattern, whenever a connection object is initialized, rather than creating a new object, it will look for existing instance of connection object and use that one, if it exists.
Hello i am trying to implement a database-object("connection") pooler for BerkeleyDB...
I decided to use a singleton EJB propably or ENUM singleton implementation for this..
A final concurrenthash map would store database objects with a timestamp...
the method getConnection() would use double check locking as long as the value from map is volatile. - No performance issues i believe..(Java Connection Pooler getConnection is synchronized!!)
The database is spread into 100 files + the daily ones.. (application designed in mid seventies 1976)..
So far everything is fine... But i want to close daily unused handles.
So i decided to use a Timer to run every 24 hours a cleanup routine..
The problem is that how can i ensure that during cleanup a connection to be closed isnt requested ?
Pseudo algorithm
cleanup(){
for(Database db in map){
if(db.getLastAccess - now >24hours) {
res=map.remove("key",db);
db.close();
}
}
}
i know that the above isnt thread safe..How could i block getconnection ? Because many things could go wrong... "If condition" may be true but before removing db obj getLastAccess could be changed! Cleanup would be called by single thread though..
Is there any solution to block getconnection somehow so cleanup to work or anyother solution?
I am not sure if you currently do this, but if you have a way to determine if a connection is in use this would make this slightly easier. One thing that you can do, is iterate over the connections in your pool. When you find one that matches your criteria for being closed, try to mark it as being in use (assuming that a connection that is in use will not be returned as a open connection). If you succeed, close it. Otherwise, check it until it becomes free and you are able to mark it as being in use. Once you have been able to do this, you should be able to close it.
Each connection would have a lock associated with it, in order for the connection to be returned by the getConnection method, the correct lock would have to be acquired. The cleanup method would also need to acquire the lock before closing a connection. Take a look at the java.util.concurrent.lock package.
Maybe a Semaphore is a better solution. Follow the link for an example.
I've never worked with BerkeleyDB, but I assume it has a JDBC interface. Can't you use an out of the box solution like DBCP or c3p0? Also check the Pool Component, it is a generic pool interface.