I have this code that uses HikariCP Connection Pool:
config.setMaximumPoolSize(100);
config.setDataSourceClassName("com.mysql.jdbc.jdbc2.optional.MysqlDataSource");
config.addDataSourceProperty("serverName", hostname);
config.addDataSourceProperty("port", portnumber);
config.addDataSourceProperty("databaseName", dbname);
config.addDataSourceProperty("user", username);
config.addDataSourceProperty("password", password);
config.setConnectionTimeout(30000);
config.setInitializationFailFast(false);
pooledDataSource = new HikariDataSource(config);
I monitor connections in mysql by issuing command "Show Processlist" and I see that 100 connections is created after line:
pooledDataSource = new HikariDataSource(config);
...is run. I'm sure this is not meant to happen, right? It should create connections later when I do pooledDataSource.getConnection().
What am I doing wrong? Why is it creating 100 connections immediately??
By default HikariCP runs as a fixed-sized pool. You need to set minimumIdle. That's it.
From the documentation for minimumIdle:
This property controls the minimum number of idle connections that
HikariCP tries to maintain in the pool. If the idle connections dip
below this value, HikariCP will make a best effort to add additional
connections quickly and efficiently. However, for maximum performance
and responsiveness to spike demands, we recommend not setting this
value and instead allowing HikariCP to act as a fixed size connection
pool. Default: same as maximumPoolSize
Related
I use Hikary connection pool with following settings:
HikariDataSource dataSource = new HikariDataSource();
dataSource.setMinimumIdle(0);
dataSource.setMaximumPoolSize(Integer.MAX_VALUE);
dataSource.setJdbcUrl(jdbcConnectionString);
dataSource.setConnectionTestQuery("select 1");
dataSource.setIdleTimeout(TimeUnit.SECONDS.toMillis(60));
dataSource.getConnection();
After getConnection() hikari try to get 2 connections to instance, but put in connection pool just one connection. How can I fix it? The hikari version is 3.4.0
I found the answer. Hikari creates first connection in checkFailFast method. I update this comment when find how to disable this method. The checkFailFast doesn't work if initializationFailTimeout<0. It helps me
After getConnection() hikari try to get 2 connections to instance, but put in connection pool just one connection. How can I fix it?
There is nothing to fix in this behavior. It simple means, that two conenctions were opened and one of them was closed.
The reason why the second connection was closed is that you set setMinimumIdle(0), i.e. no idle connection is maintainend in the pool and all idle connection are closed.
If you want to see both connection in the pool, simple set setMinimumIdle(1). After calling DataSource.getConnection() there will be two connection in the pool - one yours and one idle.
If you don't want to open the second connection at all, set
config.setMinimumIdle( 1 );
config.setMaximumPoolSize( 1 );
But think twice, why do you use a connection pool with only one connection.
You may anyway increase both parameters later, while the pool is running.
HikariConfigMXBean bn = DataSource.ds.getHikariConfigMXBean()
bn.setMaximumPoolSize(10)
bn.setMinimumIdle(10)
This will (not instantly) open 9 additional connections to the database.
Note that while setting the MaximumPoolSize == MinimumIdle the number of connection in the pool remains stable, no connections are opened or closed, which is probably the thing you want to observe.
Tested with Hicari 3.4.0 and Oracle 12.2
I'm developing a web app on Tomcat 8 with Maven, I'm using c3p0 to handle connections on the main thread and on 2 other concurrent threads, my connection manager class is asking a DataSource singleton class I've implemented for synchronized connections, like so
public synchronized Connection getConnection() {
try {
return cpds.getConnection();
} catch (SQLException ex) {
logger.error("Error while issuing a pooled connection", ex);
}
return null;
}
, but when I'm trying to use these connections they start to either interrupt
09:47:17.164 [QuartzScheduler_Worker-4] ERROR com.myapp.providers.DataSource - Error while issuing a pooled connection
java.sql.SQLException: An SQLException was provoked by the following failure: java.lang.InterruptedException
at com.mchange.v2.sql.SqlUtils.toSQLException(SqlUtils.java:106) ~[c3p0-0.9.1.2.jar:0.9.1.2]
at com.mchange.v2.sql.SqlUtils.toSQLException(SqlUtils.java:65) ~[c3p0-0.9.1.2.jar:0.9.1.2]
at com.mchange.v2.sql.SqlUtils.toSQLException(SqlUtils.java:62) ~[c3p0-0.9.1.2.jar:0.9.1.2]
at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool.checkoutPooledConnection(C3P0PooledConnectionPool.java:531) ~[c3p0-0.9.1.2.jar:0.9.1.2]
at com.mchange.v2.c3p0.impl.AbstractPoolBackedDataSource.getConnection(AbstractPoolBackedDataSource.java:128) ~[c3p0-0.9.1.2.jar:0.9.1.2]
or close in mid transaction and breaking any statements and result sets that are being used at that time
I'm configuring the DataSource object like so
cpds = new ComboPooledDataSource();
cpds.setDriverClass(oracle.jdbc.driver.OracleDriver);
cpds.setJdbcUrl(jdbc:oracle:thin:#xx.xxx.xxx.xxx:1521:XE);
cpds.setUser("username");
cpds.setPassword("password");
// database connection properties
cpds.setInitialPoolSize(10);
cpds.setAcquireIncrement(3);
cpds.setMaxPoolSize(100);
cpds.setMinPoolSize(15);
cpds.setMaxStatements(75);
// connection pool preferences
cpds.setIdleConnectionTestPeriod(60);
cpds.setMaxIdleTime(30000);
cpds.setAutoCommitOnClose(false);
cpds.setPreferredTestQuery("SELECT 1 FROM DUAL");
cpds.setTestConnectionOnCheckin(false);
cpds.setTestConnectionOnCheckout(false);
cpds.setAcquireRetryAttempts(30);
cpds.setAcquireRetryDelay(1000);
cpds.setBreakAfterAcquireFailure(false);
I've also written a small test method that runs in a loop and queries the database for n times but that works fine.
c3p0-0.9.1.2 is very, very old; please consider upgrading to 0.9.5.1, the current production version.
The problem is both clear and not so clear. The clear part is that something is calling interrupt() on client Threads that are waiting to acquire Connections. The not-so-clear part is who is doing that and why.
A guess is that Tomcat itself is doing that because the client Threads are hung too long. If the Threads are hanging at getConnection(), that could be due to a Connection leak and pool exhaustion. We see above how you acquire Connections. Are you vigilant about ensuring that they are reliably close()ed in finally blocks?
A thing you might try is to set a checkoutTimeout, e.g.
cpds.setCheckoutTimeout( 5000 ); // 5 secs
This won't actually solve the problem if Connection checkouts are hanging. But rather than a problem provoked by mysterious interrupts, you'll see c3p0 TimeoutExceptions instead. That will verify that the issue is long hangs on checkout, though, which would most likely be due to pool exhaustion, either from a Connection leak (missing calls to close()), or simply from a maxPoolSize value too low for your load.
If there does seem to be a Connection leak, please see unreturnedConnectionTimeout and debugUnreturnedConnectionStackTraces for help tracking it down. See also "Configuring to Debug and Workaround Broken Client Applications"
I have a Java application which is connecting to an Oracle database, but my application is losing its connection after 20 minutes. It will works after closing and logging:
My poolingDatasourceExample.java is given below:
Properties props = new Properties();
props.put("user", uname);
props.put("password", pass);
props.put("password", pass);
props.put("testWhileIdle", "true");
props.put("testOnBorrow", "true");
props.put("testOnReturn", "false");
props.put("validationQuery", "SELECT 1");
props.put("validationInterval", "30000");
props.put("timeBetweenEvictionRunsMillis", "5000");
props.put("maxActive", "100");
props.put("minIdle", "10");
props.put("maxWait", "10000");
props.put("initialSize", "10");
props.put("removeAbandonedTimeout", "60");
props.put("removeAbandoned", "true");
props.put("logAbandoned", "true");
props.put("minEvictableIdleTimeMillis", "30000");
ConnectionFactory connectionFactory = new DriverManagerConnectionFactory(connectURI, props);
Are you storing the java.sql.Connection object, and seeing it stop working after 20 minutes? That's quite normal, particularly if it's inactive in between (see, for example, this post). Inactive connections will be closed by the database and quite possibly by the network, to protect against zombie processes hogging all the resources. They're also bad at error recovery - once the Connection breaks, you need to recreate it. They're not designed for sharing either, although they are technically thread-safe.
Rather than keeping the Connection, you should keep the factory and request a new Connection when needed. Better still, use a connection pool - these typically test connections for disconnects and recreate them when needed. They also allow your application to share connections efficiently. Connection pooling is available in most app servers, but if your deployment doesn't allow that it's possible to create one of your own.
Would anyone care to elaborate how the HikariCP handles connections in the pool? How do you put a new connection in the pool, and how can you call on it / retrieve it later?
This is my current code:
HikariConfig config = new HikariConfig();
config.setMaximumPoolSize(100);
config.setDataSourceClassName("com.mysql.jdbc.jdbc2.optional.MysqlDataSource");
config.addDataSourceProperty("serverName", "localhost");
config.addDataSourceProperty("port", "8889");
config.addDataSourceProperty("databaseName", "XXX");
config.addDataSourceProperty("user", "XXX");
config.addDataSourceProperty("password", "XXX");
System.out.print("qq");
HikariDataSource ds = new HikariDataSource(config);
ds.setConnectionTimeout(800);
With a pool, you don't add a connection to the pool to retrieve it later. You do the exact inverse: you get a connection from the pool when you need one, and close the connection when you're done with it to give it back to the pool. The HikariDataSource, as its name indicates, is a DataSource. A DataSource is an object from which you get connections.
The pool handles the opening of the connection for you. It puts you in a waiting queue if no connections are available automatically, etc.
Depending on the properties of the pool, the pool can open the connections immediately or on demand, keep a given number of connections always opened, shrink the pool size after given amount of unused time, etc.
That's all very well documented: https://github.com/brettwooldridge/HikariCP#user-content-configuration-knobs-baby
Example code (Java 7 and later):
try (Connection connection = ds.getConnection()) {
// use the connection
}
Example code (Before Java 7):
Connection connection = ds.getConnection();
try {
// use the connection
}
finally {
connection.close();
}
I have created a mysqlDatasource connection using the following code:
MysqlDataSource d = new MysqlDataSource();
d.setUser("user");
d.setPassword("pass");
d.setServerName("hostname.com");
d.setDatabaseName("db");
Connection c = d.getConnection();
If Im running my application and the connections are disconnected because mysql restarted or for some other reason, the remaining operations will fail even if the mysql server instance is running.
In that case I want to recreate a connection? Is this possible? How do I go about doing this?
In most cases when you're creating multiple connections and juggling them, it's better to use a connection pool, where the Connection objects are just that, objects, and are multiplexed to actual socket connections handled by an underlying implementation. This means that connections are created automatically when you need them and you don't need to worry about reclaiming resources and creating an appropriate number of connections.
Two prominent examples are BoneCP and C3P0.
The other option, in addition to Mr Mao's, is to define the url explicitly using setURL to allow for automatic reconnection, using the autoReconnect parameter, do note, though, that this approach is not recommended. The answer is provided here only for completeness.
Just take an help of DBCP CONNECTION and create your Connection pool like below code. You can see there is validation query which pings database at regular interval and refreshes the pool. There are other property like validateConnection you can set this property to true.
BasicDataSource dataSource = new BasicDataSource();
dataSource.setDriverClassName("com.mysql.jdbc.Driver");
dataSource.setUsername("username");
dataSource.setPassword("password");
dataSource.setUrl("jdbc:mysql://<host>:<port>/<database>");
dataSource.setMaxActive(50);
dataSource.setMaxIdle(5);
dataSource.setInitialSize(5);
dataSource.setValidationQuery("SELECT 1");
Try this.
String driver = "com.microsoft.jdbc.sqlserver.SQLServerDriver";
Class.forName(driver);
String url = "jdbc:microsoft:sqlserver://host:1433/database";
Connection conn = DriverManager.getConnection(url, "username", "password");