Would anyone care to elaborate how the HikariCP handles connections in the pool? How do you put a new connection in the pool, and how can you call on it / retrieve it later?
This is my current code:
HikariConfig config = new HikariConfig();
config.setMaximumPoolSize(100);
config.setDataSourceClassName("com.mysql.jdbc.jdbc2.optional.MysqlDataSource");
config.addDataSourceProperty("serverName", "localhost");
config.addDataSourceProperty("port", "8889");
config.addDataSourceProperty("databaseName", "XXX");
config.addDataSourceProperty("user", "XXX");
config.addDataSourceProperty("password", "XXX");
System.out.print("qq");
HikariDataSource ds = new HikariDataSource(config);
ds.setConnectionTimeout(800);
With a pool, you don't add a connection to the pool to retrieve it later. You do the exact inverse: you get a connection from the pool when you need one, and close the connection when you're done with it to give it back to the pool. The HikariDataSource, as its name indicates, is a DataSource. A DataSource is an object from which you get connections.
The pool handles the opening of the connection for you. It puts you in a waiting queue if no connections are available automatically, etc.
Depending on the properties of the pool, the pool can open the connections immediately or on demand, keep a given number of connections always opened, shrink the pool size after given amount of unused time, etc.
That's all very well documented: https://github.com/brettwooldridge/HikariCP#user-content-configuration-knobs-baby
Example code (Java 7 and later):
try (Connection connection = ds.getConnection()) {
// use the connection
}
Example code (Before Java 7):
Connection connection = ds.getConnection();
try {
// use the connection
}
finally {
connection.close();
}
Related
I have an application where user can choose any type of database(relational) provide URL, username, password and create datasource connect to database, then run SELECT queries. To connect to databases I am using HikariCp for my connection pool(before I used DBCP2 as connection timeout didn't work I changed to HikariCp). Here is my configuration
HikariConfig config = new HikariConfig();
config.setDriverClassName(StringUtils.trimToEmpty(driver));
config.setJdbcUrl(StringUtils.trimToEmpty(url));
config.setUsername(username);
config.setPassword(password);
config.setMaximumPoolSize(100);
config.setMinimumIdle(3);
config.setConnectionTestQuery(validationQuery);
config.setConnectionTimeout(TimeUnit.SECONDS.toMillis(jdbcTimeout));
config.setIdleTimeout(TimeUnit.MINUTES.toMillis(10));
config.setMaxLifetime(TimeUnit.MINUTES.toMillis(11));
config.addDataSourceProperty("closeMethod", "close");
HikariDataSource dataSource = new HikariDataSource(config);
Connection connection = dataSource.getConnection();
jdbcTimeout- is configurable it can be 0 -30
Running tests which using application creates queries and runs those queries, after some time I am getting an error that too many clients.
What properties should I add to return abandoned connections to the pool?
I have this code that uses HikariCP Connection Pool:
config.setMaximumPoolSize(100);
config.setDataSourceClassName("com.mysql.jdbc.jdbc2.optional.MysqlDataSource");
config.addDataSourceProperty("serverName", hostname);
config.addDataSourceProperty("port", portnumber);
config.addDataSourceProperty("databaseName", dbname);
config.addDataSourceProperty("user", username);
config.addDataSourceProperty("password", password);
config.setConnectionTimeout(30000);
config.setInitializationFailFast(false);
pooledDataSource = new HikariDataSource(config);
I monitor connections in mysql by issuing command "Show Processlist" and I see that 100 connections is created after line:
pooledDataSource = new HikariDataSource(config);
...is run. I'm sure this is not meant to happen, right? It should create connections later when I do pooledDataSource.getConnection().
What am I doing wrong? Why is it creating 100 connections immediately??
By default HikariCP runs as a fixed-sized pool. You need to set minimumIdle. That's it.
From the documentation for minimumIdle:
This property controls the minimum number of idle connections that
HikariCP tries to maintain in the pool. If the idle connections dip
below this value, HikariCP will make a best effort to add additional
connections quickly and efficiently. However, for maximum performance
and responsiveness to spike demands, we recommend not setting this
value and instead allowing HikariCP to act as a fixed size connection
pool. Default: same as maximumPoolSize
I'm using JBoss AS 7.1 as a server and I have my DataSource configured with pooling. I'm quite new to this so please excuse any rookie mistakes... after all I'm here to learn.
When a client logs-in it gets a connection to the database and I need to keep that connection(from the pool) open until the user logs-out or the HttpSession expires. This is an absolute requirement coming from our DB Admin. who says that he needs the DB session variables. I am using a servlet for all this.
Playing with the possibilities I have encountered 2 major problems:
As far as I see JBoss automatically closes unused connections => my opened connection returns to the pool. So this might not be the right path.
If I try to store/recall the Connection object like this:
private Hashtable<String, Connection> connections = new Hashtable<String, Connection>();
try {
String strDSName1 = "java:/OracleDSJNDI";
ctx = new InitialContext();
ds1 = (javax.sql.DataSource) ctx.lookup(strDSName1);
System.out.println("Got 1'st ds.");
} catch (Exception e) {
System.out.println("ERROR getting 1'st DS : " + e);
}
connection = ds1.getConnection();
connections.put(session.getId(), connection);
conn = (Connection) connections.get(sessionID);
it throws this exception:
java.sql.SQLException: Connection is not associated with a managed
connection.org.jboss.jca.adapters.jdbc.jdk6.WrappedConnectionJDK6#dee1f37
My question is: How do I properly keep my connection opened?
Thanks
How do I properly keep my connection opened?
You must not do that, let the connection pool handle this.
Behind the scenes, the connection pool will keep a bunch of database connections to the database engine (MySQL, Oracle, SQL Server... depends how you configure it) in SLEEPING state. When you execute this code:
//avoiding all the particular exceptions just for code simplicity purposes...
//in real world applications, you must handle each of these exceptions
public Connection getConnection() throws Exception {
ctx = new InitialContext();
ds1 = (javax.sql.DataSource) ctx.lookup(strDSName1);
return ds1.getConnection();
}
You're asking to the connection pool to retrieve one of these connections available. The connection pool will give you a database connection (if available) and let you use it as long as you want. Then you use it wherever you want/need and close it:
public void foo() throws Exception {
Connection connection = getConnection();
//do what you want/need...
//in the end, you close the connection
//this is A MUST!
connection.close();
}
When executing connection.close() from a connection retrieved by the connection pool, you're not closing the physical database connection but notifying the connection pool this specific database connection must return to the SLEEPING state.
Some advices from the explanation:
You must not try to keep the connection alive, that's connection pool's job.
You must not try to store the connections in any cache-like structure, that's connection pool's job.
You must retrieve a java.sql.Connection in the shortest scope you will need it. Once you have used it, close it.
Your DBA is basically requiring you to avoid connection pooling by making the database connection equivalent to the user's session.
So one option is to not use the connection pool, and instead roll your own functionality that opens/closes the database connection around the user's session. That seems complicated and unusual though.
Another option is to examine the DBA's requirement. The DBA may have to adapt to the idea that he'll need to track state in a different way, e.g. by using a key related to the session to store the state he needs in a table, instead of storing state in the connection layer.
Generally speaking storing state in some component's session handling is adding indirect complexity, because you start having to care about how the component handles expiry and uniqueness, as you're finding here where the HTTP session state handles this differently from the database session.
Referring to Tomcat JBDC connection pool, I see in the standalone java example given there, one gets the connection using datasource.getConnection()which is cool. But in the finally block, it says con.close().
Question: When I implement this, it seems obvious that the con I get from datasource will be closed every time in the finally. When this is closed, will the connection pooling mechanism acquire a new connection and adds it to the pool?
I presume there should be a method call like releaseConnection() that will let the pool take its own decision whether to close it or let it be open for some other use.
I've also tried doing this ConnectionPool aPool = datasource.createPool();
But I see there is nothing like release connection on this aPool.
I think I'm missing something here?
Appreciate your help.
Code snippet from Tomcat JBDC connection pool:
DataSource datasource = new DataSource();
datasource.setPoolProperties(p);
Connection con = null;
try {
con = datasource.getConnection();
Statement st = con.createStatement();
ResultSet rs = st.executeQuery("select * from user");
int cnt = 1;
while (rs.next()) {
System.out.println((cnt++)+". Host:" +rs.getString("Host")+
" User:"+rs.getString("User")+" Password:"+rs.getString("Password"));
}
rs.close();
st.close();
} finally {
if (con!=null) try {con.close();}catch (Exception ignore) {}
}
Since you call the close() on a method obtained by the pool it is up to the pool what to do inside this method call. It does not neccessarily have to close the pooled database connection - it may do some cleanup and then add the connetion back to the pool.
This is already answered in Closing JDBC Connections in Pool
OK, my bad, that I did not see the implementation of DataSource.
It extends DataSourceProxy that internally creates a pool before returning a Connectionbased on the PoolProperties
I understand, its upto this DataSource to handle the connections, even though I close the con in finally, DataSource may take necessary action.
Do add a comment/reply if anybody thinks otherwise.
That example only shows how to create and use a data source. For connection pool on Tomcat you may configure JNDI.
// Sample
public static Connection getConnectionFromPool() {
Context initCtx = new InitialContext();
Context envCtx = (Context) initCtx.lookup("java:comp/env");
DataSource ds = (DataSource) envCtx.lookup("jdbc/TestDB");
return ds.getConnection();
...
Quote from How connection pooling works in Java and JDBC:
A connection pool operates by performing the work of creating
connections ahead of time, In the case of a JDBC connection pool, a
pool of Connection objects is created at the time the application
server (or some other server) starts. These objects are then managed
by a pool manager that disperses connections as they are requested by
clients and returns them to the pool when it determines the client is
finished with the Connection object. A great deal of housekeeping is
involved in managing these connections.
When the connection pool server starts, it creates a predetermined
number of Connection objects. A client application would then perform
a JNDI lookup to retrieve a reference to a DataSource object that
implements the ConnectionPoolDataSource interface. The client
application would not need make any special provisions to use the
pooled data source; the code would be no different from code written
for a nonpooled DataSource.
I have created a mysqlDatasource connection using the following code:
MysqlDataSource d = new MysqlDataSource();
d.setUser("user");
d.setPassword("pass");
d.setServerName("hostname.com");
d.setDatabaseName("db");
Connection c = d.getConnection();
If Im running my application and the connections are disconnected because mysql restarted or for some other reason, the remaining operations will fail even if the mysql server instance is running.
In that case I want to recreate a connection? Is this possible? How do I go about doing this?
In most cases when you're creating multiple connections and juggling them, it's better to use a connection pool, where the Connection objects are just that, objects, and are multiplexed to actual socket connections handled by an underlying implementation. This means that connections are created automatically when you need them and you don't need to worry about reclaiming resources and creating an appropriate number of connections.
Two prominent examples are BoneCP and C3P0.
The other option, in addition to Mr Mao's, is to define the url explicitly using setURL to allow for automatic reconnection, using the autoReconnect parameter, do note, though, that this approach is not recommended. The answer is provided here only for completeness.
Just take an help of DBCP CONNECTION and create your Connection pool like below code. You can see there is validation query which pings database at regular interval and refreshes the pool. There are other property like validateConnection you can set this property to true.
BasicDataSource dataSource = new BasicDataSource();
dataSource.setDriverClassName("com.mysql.jdbc.Driver");
dataSource.setUsername("username");
dataSource.setPassword("password");
dataSource.setUrl("jdbc:mysql://<host>:<port>/<database>");
dataSource.setMaxActive(50);
dataSource.setMaxIdle(5);
dataSource.setInitialSize(5);
dataSource.setValidationQuery("SELECT 1");
Try this.
String driver = "com.microsoft.jdbc.sqlserver.SQLServerDriver";
Class.forName(driver);
String url = "jdbc:microsoft:sqlserver://host:1433/database";
Connection conn = DriverManager.getConnection(url, "username", "password");