I am using a connection pool in my project. I want to know how many connections are opened?
I have given 20 connections as max no of connections. What will happen if it exceeds 20? Will the pool manage this or it will through error?
Its more of a configuration related problem. But in general the pool will throw exception when the new request for a connection results in the number of connection exceeding the max connection setting.
Must depend on the particular connection pool implementation.
Conceptually it would be possible to wait, fail or victimise.
Waiting indefinitely is potentially a very bad thing, in a badly configured system with antisocial clients keeping connections for too long the number of waiters could grow very large.
Arbitrarily victimising some existing client and taking their connection is usually not reasonable (or indeed possible) for DB connections. So that's not likely to be seen.
Which leaves a failure - you'll get an exception on the lines of "No Connections Available". Some connection pools actually wait a while (for a configurable period of time) before throwing that exception - this deals better with cases of occasional peaks in demand.
To know the number of connections opened in Oracle you can write a query like this.
SELECT s.program, s.server, p.spid, s.username FROM v$session s, v$process p
WHERE s.type = 'USER'
and s.username != 'sys';
The query assumes that your connected to Oracle in a dedicated server mode also the query counts connections made other than the user sys.
Then you can use the result in your application. And the second question, "What will happen if it exceeds 20?" depends on your implementation of connection pool and I won't say more since it has been answered.
Hope this is helpful.
Related
I am working on mongodb connection pooling & I came across this option we can set in mongo client : "MaxConnectionIdleTime".
It basically means that connection will die after this time when sitting idle.
The default value is zero & it's stated that in case of zero, there's no limit.
Does it mean that once a connection has been created it won't die at all & will be kept in pool forever ?
Assuming there's space for new connections to be created in the pool. like min connections=10 & max connections =1000. & also, the max connection time isn't set.
If you can suggest a method to test it out on my own, that'll be really helpful too.
Please let me know if there's any way I can improve the question.
Thanks!
"Die" is a sloppy term and it is not clear what you mean by it.
If no idle time is set, the connection will not be proactively closed by the driver upon that idle time elapsing. That's all.
Connections may be closed by the driver in the following other circumstances:
It experiences a network error or a timeout.
Another connection to the server that this connection is associated with experiences a network error, causing the server to be marked unusable by the driver.
A connection may become unusable because of a network error. The connection may be unusable without the driver knowing about it. The driver often detects unusability of a connection when trying to use it (i.e. write or read something) which means a connection may be unusable (one might say dead) while the driver thinks it is perfectly fine, for a long time.
I am building a Java application which queries my SQL server once a minute. Right now, the application is using a connection pool with a single connection (min pool size 1, max pool size 1).
I figured that a pool size of 1 would be enough because of the infrequent queries (once a minute, as mentioned before).
Do I need a connection pool at all, and if the answer is yes, is 1 connection enough? Or should I just not use a pool and open a new connection every minute?
using a connection pool with a single connection (min pool size 1, max
pool size 1).
In such case don't see any need or benefit of using connection pool since at any point in time there will be only one connection object and if it's in use then other request has to wait (or) create a non-pooled connection object.
Connection pooling is generally used to save time/resource from creation/tearing-up the connection object.
In your specific case, you can probably create a connection instance and dispose it off once done with your work.
I would like to give some reason to use the connection pool, potentially with multiple connections. Not sure if you are considering this negative case.
In real world, a query might run more than 1 minute with various reason.
Do you want the application to wait for the hung connection? Or what is your expected behavior for this?
Also if you use connection pool, DB connection initialize process(time and resource consuming) is done while the pool is generated. When you actually use the DB connection pool, some of initialization step should be already done so it reduce repetitive overhead when the application is running query.
If you're accessing the database that rarely, then in practice having a connection pool will not provide any significant benefit. However, I'd keep it if it's already written. Maybe one day your application will grow and you'll find the pool useful. It's not a bad thing to have.
I have a scenario in production for a web app, where when a form is submitted the data gets stored in 3 tables in Oracle DB through JDBC. Sometimes I am seeing connection time out errors in logs while the app is trying to connect to Oracle DB through Java code. This is intermittent.
Below is the exception:
SQL exception while storing data in table
java.sql.SQLRecoverableException: IO Error: Connection timed out
Most of the times the web app is able to connect to data base and insert values in it but some times and I am getting this time out error and unable to insert data in it. I am not sure why am I getting this intermittent issue. When I checked the connections pool config in my application, I noticed the following things:
Pool Size (Maximum number of Connections that this pool can open) : 10
Pool wait (Maximum wait time, in milliseconds, before throwing an Exception if all pooled Connections are in use) : 1000
Since the pool size is just 10 and if there are multiple users trying to connect to data base will this connection time out issue occur ?
Also since there are 3 tables where the data insertion occurs we are doing the whole insertion in just one connection itself. We are not opneing each DB connection for each individual table.
NOTE: This application is deployed on AEM (Content Management system) server and connections pool config is provided by them.
Update: I tried setting the validation query in the connections pool but still I am getting the connection time out error. I am not sure whether the connections pool has checked the validation query or not. I have attached the connections pool above for reference.
I would try two things:
Try setting a validation query so each time the pool leases a connection, you're sure it's actually available. select 1 from dual should work. On recent JDBC drivers that should not be required but you might give it a go.
Estimate the concurrency of your form. A 10 connections pool is not too small depending on the complexity of your work on DB. It seems you're saving a form so it should not be THAT complex. How many users per day do you expect? Then, on peak time, how many users do you expect to be using the form at the same time? A 10 connections pool often leases and retrieves connections quite fast so it can handle several transactions per second. If you expect more, increase the size slightly (more than 25-30 actually degrades DB performance as more queries compete for resources there).
If nothing seems to work, it would be good to check what's happening on your DB. If possible, use Enterprise Manager to see if there are latches while doing stuff on those three tables.
I give this answer from programming point of view. There are multiple possibilities for this problem. These are following and i have added appropriate solution for it. As connection timeout occurs, means your new thread do not getting database access within mentioned time and it is due to:
Possibility I: Not closing connection, there should be connection leakage somewhere in your application Solution
You need to ensure this thing and need to check for this leakage and close the connection after use.
Possibility II: Big Transaction Solution
i. Is these insertion synchronized, if it is so then use it very carefully. Use it at block level not method level. And your synchronized block size should be minimum as much as possible.
What happen is if we have big synchronized block, we give connection, but it will be in waiting state as this synchronized block needs too much time for execution. so other thread waiting time increases. Suppose we have 100 users, each have 100 threads for that operation. 1st is executing and it takes too long time. and others are waiting. So there may be a case where 80th 90th,etc thread throw timeout. And For some thread this issue occurs.
So you must need to reduce size of the synchronized block.
ii. And also for this case also check If the transaction is big, then try to cut the transaction into smaller ones if possible:-
For an example here, for one insertion one small transaction. for second other small transaction, like this. And these three small transaction completes operation.
Possibility III: Pool size is not enough if usability of application is too high Solution
Need to increase the pool size. (It is applicable if you properly closes all the connection after use)
You can use Java Executor service in this case .One thread One connection , all asynchronous .Once transaction completed , release the connection back to pool.That way , you can get rid of this timeout issue.
If one connection is inserting the data in 3 tables and other threads trying to make connection are waiting, timeout is bound to happen.
I'm having an issue with the jdbc Connection Pool on glassfish handing out dead database connections. I'm running Glassfish 3.1.2.2 using jconn3 (com.sybase.jdbc3) to connect to Sybase 12.5. Our organization has a nightly reboot process during which time we restart the Sybase server. My issue manifests itself when an attempt to use a database connection during the reboot occurs. Here are the order of operations to produce my issue:
Sybase is down for restart.
Connection is requested from the pool.
DB operation fails as expected.
Connection is returned to the pool in a closed state.
Sybase is back up.
Connection is requested from the pool.
DB operation fails due to "Connection is already closed" exception.
Connection is returned to the pool
I've implemented a database recovery singleton that attempts to recover from this scenario. Any time a database exception occurs I make a jmx call to pause all queue's and execute a flushConnectionPool operation on the JDBC Connection Pool. If the database connection is still not up the process sets up a timer to retry in 10 minutes. While this process works, it's not without flaws.
I realize there's a setting on the pool so that you can require validation on the database connection prior to handing it out but I've shied away from this for performance reasons. My process performs approximately 5 million database transactions a day.
My question is, does anyone know of a way to avoid returning a dead connection back to the pool in the first place?
You've pretty well summed up your options. We had that problem, the midnight DB going down. For us, we turned on connection validation, but we don't have your transaction volume.
Glassfish offers a custom validation option, with which a class can be specified to do the validation.
By default, all the classes provided by Glassfish do (You'll see them offered as options in the console) is a SQL statement like this:
SELECT 1;
The syntax varies a bit between databases, SQL Server is uses '1', whereas for Postgres, it just uses 1. But the intent is the same.
The net is that it will cost you an extra DB hit every time you try to get a connection, but it's a really, really cheap hit. But still, it's a hit.
But you could implement your own version. It could do the check, say, every 10th request, or even less frequent. Roll a random number from 1 to N (N = 10, 20, 100...), if you get a '1', do the select (and fail, if it fails), otherwise return "true". But at the same time, configure it so that if you do detect an error, purge the entire pool. Clearly tweak this so you have a good chance of it happening when your db goes down at night (dunno how busy your system is at night) vs peak processing.
You could even "lower the odds" during peak processing. "if time between 6am and 6pm then odds = 1000 else odds = 100; if (random(odds) == 1) { do select... }"
A random option removes the need to maintain a thread safe counter.
In the end, it doesn't really matter, you just want a timely note that the DB is down so you can ask GF to abort the pool.
I can definitely see it thrashing a bit at the very beginning as the DB comes up, possibly refreshing the pool more than once, but that should be harmless.
Different ways you could play with that, but that's an avenue to consider.
Why should I prefer using a connection pool instead of static variable from a static class in Tomcat to hold the database connection?
This seems to me equivalent of using a connection pool having the capacity to store just one connection. So, a related question is: why the capacity of a connection pool needs to be bigger than one connection?
Thank's in advance.
With a pool, you can have multiple threads using different connections. Do you really want to limit your web application to handling one db-related request at a time? :) (And adding the complication of synchronizing to make sure that one thread doesn't try to use that single connection while another request is doing so...)
It would be generally a very bad idea to have a connection pool with a capacity of 1 - but at least if you did so, you could later increase the capacity without changing any other code, of course.
(EDIT: And as noted in other answers, connections in a pool can be closed and reopened if they become stale or broken in some way.)
The reason is to increase scalability, robustness and speed.
If you're creating a web application, there can be many concurrent HTTP requests coming in, each served by a different thread.
If you have only one static connection to the database, you need synchronization around that connection. You can't share a connection between several threads, That means each HTTP request have to wait for someone else using the database. And you need to fix up/reconnect that connection if something goes wrong with it at one point or another.
You could open a connection at the start of each HTTP request - however opening a new database connection can be expensive, and you don't get much control over how many database connections you have. Databases can be overwhelmed by having too many connections.
A connection pool solves this, as you have a pool of already opened connections that can be handed out to serve an HTTP request, or to different parts of the applications that needs to do database operations, and is returned to the pool when the database operation is finished, ready to be used again by something else.
A connection pool of just 1 connection rarely makes sense though - however connection pools take care of many other things as well, such a closing the connection and opening a new one if a connection goes stale or is otherwise in a bad state, as well as it takes care of the synchronization when there is are no more connections to hand out at a particular time.
If you're using a connection pool with only one connection it is equivalent to have one static connection - like you mentioned and there's no advantage for connection pool in regards.
The strength of a connection pool is when you're using multiple connections (multiple threads) - it saves you the effort of managing the connections (open/close connections, boilerplate-code, smart resource handling etc).
Using a connection pool for one connection only is kind of like paving a 10-lane road that will be used by one car only - lot of overhead with (almost) no gain.
Using a connection pool is not just about sharing connections: it is about leveraging years of experience with broken JDBC drivers and all the weird ways in which a Connection can become unusable. Using a single static connection is not only a bottleneck, but a very fragile solution. It will cause your application to break on a regular basis, and even restarting the application will not clean up the problems: you may get a connection leak.