I would like to ask, if I am going to use a single connection in the whole program, do I need to use a connection pool? Does using a connection pool give me better performance?
For example, I may open a static connection and use it for multiple sql query, then finally when the program finish, I close the connection. Most of the program I am dealing with, are not multi-threaded, so I don't see the benefit of using connection pool in such situation. Would anyone can answer my question, thank you.
Thanks in advance.
For example, I may open a static
connection and use it for multiple sql
query, then finally when the program
finish, I close the connection. Most
of the program I am dealing with, are
not multi-threaded
Sounds like you don't need a connection pool.
You don't ever need a connection pool.
It can be useful if you require enforcement on having a limit number of connections.
Related
So I have a Java process that runs indefinitely as a TCP server (receives messages from another process, and has onMsg handlers).
One of the things I want to do with the message in my Java program is to write it to disk using a database connection to postgres. Right now, I have one single static connection object which I call every time a message comes in. I do NOT close and reopen the connection for each message.
I am still a bit new to Java, I wanted to know 1) whether there are any pitfalls or dangers with using one connection object open indefinitely, and 2) Are there performance benefits to never closing the connection, as opposed to reopening/closing every time I want to hit the database?
Thanks for the help!
I do NOT close and reopen the connection for each message.
Yes you do... at least as far as the plain Connection object is concerned. Otherwise, if you ever end up with a broken connection, it'll be broken forever, and if you ever need to perform multiple operations concurrently, you'll have problems.
What you want is a connection pool to manage the "real" connections to the database, and you just ask for a connection from the pool for each operation and close it when you're done with it. Closing the "logical" connection just returns the "real" connection to the pool for another operation. (The pool can handle keeping the connection alive with a heartbeat, retiring connections over time etc.)
There are lots of connection pool technologies available, and it's been a long time since I've used "plain" JDBC so I wouldn't like to say where the state of the art is at the moment - but that's research you can do for yourself :)
Creating a database connection is always a performance hit. Only a very naive implementation would create and close a connection for each operation. If you only needed to do something once an hour, then it would be acceptable.
However, if you have a program that performs several database accesses per minute (or even per second for larger apps), you don't want to actually close the connection.
So when do you close the connection? Easy answer: let a connection pool handle that for you. You ask the pool for a connection, it'll give you an open connection (that it either has cached, or if it really needs to, a brand new connection). When you're finished with your queries, you close() the connection, but it actually just returns the connection to the pool.
For very simple programs setting up a connection pool might be extra work, but it's not very difficult and definitely something you'll want to get the hang of. There are several open source connection pools, such as DBCP from Apache and 3CPO.
What are best practices for closing or caching sql connections?
I see things saying to always close it in finally block.
Though I also see things talking about caching them for future use.
How expensive is it to open new connections for each task?
I just started to work on a java data warehousing app someone else programmed. (and I don't have DB experience) It is a single threaded app that loads files into the DB, and does some aggregation and analysis. We have had some issues with deadlocks, which I wouldn't think should be a problem in a single threaded app. I see exceptions are swallowed and not logged all over the class that handles the DB connections, so I am hoping adding logging to those should provide some insight.
But I am hoping for some guidance on best practices for dealing with DB connections in the meantime.
Regardless of whether or not you are using connection pooling, any database code should follow this form:
try (
Connection connection = dataSource.getConnection();
PreparedStatement preparedStatement = ...
)
{
...
}
The dataSource could be a pool or a simple connection factory, to your code it shouldn't matter, it's simply where it gets connections from.
This said, I'd use Spring's JdbcTemplate class, which does pretty much what the code above does.
Regarding the question 'how expensive is it to open new connections for each task?' - the answer is very, at least in comparison to grabbing one from a pool. You also have to consider what happens if large numbers of tasks are run - most databases will be configured to limit the number of connections.
The bottom line is, in anything but the most simple application, you should use a connection pool like c3po and size it according you your needs
Regarding your deadlock, this is most likely occurring in the database where there are a variety of locks that can be made when data is updated depending on how the database is configured.
Why should I prefer using a connection pool instead of static variable from a static class in Tomcat to hold the database connection?
This seems to me equivalent of using a connection pool having the capacity to store just one connection. So, a related question is: why the capacity of a connection pool needs to be bigger than one connection?
Thank's in advance.
With a pool, you can have multiple threads using different connections. Do you really want to limit your web application to handling one db-related request at a time? :) (And adding the complication of synchronizing to make sure that one thread doesn't try to use that single connection while another request is doing so...)
It would be generally a very bad idea to have a connection pool with a capacity of 1 - but at least if you did so, you could later increase the capacity without changing any other code, of course.
(EDIT: And as noted in other answers, connections in a pool can be closed and reopened if they become stale or broken in some way.)
The reason is to increase scalability, robustness and speed.
If you're creating a web application, there can be many concurrent HTTP requests coming in, each served by a different thread.
If you have only one static connection to the database, you need synchronization around that connection. You can't share a connection between several threads, That means each HTTP request have to wait for someone else using the database. And you need to fix up/reconnect that connection if something goes wrong with it at one point or another.
You could open a connection at the start of each HTTP request - however opening a new database connection can be expensive, and you don't get much control over how many database connections you have. Databases can be overwhelmed by having too many connections.
A connection pool solves this, as you have a pool of already opened connections that can be handed out to serve an HTTP request, or to different parts of the applications that needs to do database operations, and is returned to the pool when the database operation is finished, ready to be used again by something else.
A connection pool of just 1 connection rarely makes sense though - however connection pools take care of many other things as well, such a closing the connection and opening a new one if a connection goes stale or is otherwise in a bad state, as well as it takes care of the synchronization when there is are no more connections to hand out at a particular time.
If you're using a connection pool with only one connection it is equivalent to have one static connection - like you mentioned and there's no advantage for connection pool in regards.
The strength of a connection pool is when you're using multiple connections (multiple threads) - it saves you the effort of managing the connections (open/close connections, boilerplate-code, smart resource handling etc).
Using a connection pool for one connection only is kind of like paving a 10-lane road that will be used by one car only - lot of overhead with (almost) no gain.
Using a connection pool is not just about sharing connections: it is about leveraging years of experience with broken JDBC drivers and all the weird ways in which a Connection can become unusable. Using a single static connection is not only a bottleneck, but a very fragile solution. It will cause your application to break on a regular basis, and even restarting the application will not clean up the problems: you may get a connection leak.
We have seen connection droughts in our system every once in a while, and the problem seems to be that Sessions are not being returned to the connection pool quick enough. I wrote a test that seems to confirm using Session.disconnect() on the sessions (after being done with one) will solve this problem. However, I also timed these calls, and it seems like using disconnect is increasing running time by 3 times.
According to the docs (http://docs.jboss.org/hibernate/core/3.5/api/org/hibernate/Session.html#disconnect() ), disconnect should be returning it to the connection pool. However, the doc also says it "closes" the connection. I'm not sure what it means because I know for a fact that Session.close() does more than disconnect, and what good would a connection pool be if you close the connection before returning it?
In any case, I'm wondering why a method that returns the session to the connection pool would be anything but instantaneous and essentially free. Surely thats the whole point of a connection pool, right?
Any ideas would be appreciated.
Because none of the JDBC operations have a concept of timeout, is it advisable to try to close JDBC connection of the overdue operation from the other thread if timeout is exceeded?
I am aware of Statement.cancel method, however it does not seem to work in all cases. Also, the driver that I am using does not respond to interrupts.
I think Connection.close is, unfortunately, the lowest common denominator that can be used to prevent resource leakage in case the database operation had ran amok.
I am aware that when using connection pooling I have to close the real connection object.
For derby it should be ok:
http://db.apache.org/derby/docs/10.8/devguide/cdevconcepts23499.html
According to the JDBC spec you can share Connections and Statements with multiple threads
The best answer is, it depends on what driver you are using. But from the jdbc stack point of view, it should be ok.
I'd wrap that operation in a ThreadPoolExecutor with a timeout. It'll give you more control over what's happening.