We have a spring boot application with spring JPA and Sybase ASE database. We have used Hikari as the connection pool.
We have 1 stored procedure written which gets called every 4 minute.
Hikari configuration is:
datasource:
type: com.zaxxer.hikari.HikariDataSource
data-source-class-name: com.sybase.jdbc4.jdbc.SybDataSource
driver-class-name: com.sybase.jdbc4.jdbc.SybDriver
url: xxx-yyy-zzz
username: xxx-yyy-zzz
password: xxx-yyy-zzz
hikari:
maximum-pool-size: 10
minimum-idle: 1
The problem is, I am getting below error after every 10 calls to the stored procedure as I have configured pool size 10.
> Caused by: java.sql.SQLTransientConnectionException: HikariPool-2 -
> Connection is not available, request timed out after 30007ms. at
> com.zaxxer.hikari.pool.HikariPool.createTimeoutException(HikariPool.java:669)
>
> org.springframework.dao.DataAccessResourceFailureException: Unable to
> acquire JDBC Connection; nested exception is
> org.hibernate.exception.JDBCConnectionException: Unable to acquire
> JDBC Connection
Code is written as follows:
#Autowired
#Lazy
Smsxyz smsxyzImpl;
#PostConstruct
#Scheduled(fixedRate = 240000)
public void completeStatus() {
try {
smsxyzImpl.checkStatus(Constants.STATUS_PER_SCAN, Constants.STATUS_EXPIRY_DURATION, -1L);
} catch (Exception e) {
e.printStackTrace();
}
}
#Autowired
private EntityManager entityManager;
#Override
//#Transactional
public Integer checkStatus(Integer count, Integer expiry, Long userId) {
StoredProcedureQuery query = entityManager.createNamedStoredProcedureQuery("status.check");
query.setParameter("#status_count", count);
query.setParameter("#status_expiry", expiry);
query.setParameter("#user_id", userId);
Integer outputValue= (Integer) query.getOutputParameterValue("#return_status");
return outputValue;
}
If I remove #Transactional then it runs for 10 times (as hikari pool size is 10) and after that it throws an exception as mentioned above
If I keep #Transactional then it runs fine with no exception but it does not update the database.
If I run the stored procedure manually directly from the database, then it works perfectly fine that means no issue in the Stored procedure
My suspect is, it is not releasing the connection. So I closed the connection by entitymanager.close() but it didn't work
I spent many days for this issue but no luck. Could anyone let me know what I am doing wrong here?
I have an application where user can choose any type and version of database(relational) provide URL, username, password and create datasource, connect to database, then run SELECT queries. To connect database I am using HikariCp for my connection pool(before I used DBCP2 as connection timeout didn't work I changed to HikariCp). Here is my configuration
HikariConfig config = new HikariConfig();
config.setDriverClassName(StringUtils.trimToEmpty(driver));
config.setJdbcUrl(StringUtils.trimToEmpty(url));
config.setUsername(username);
config.setPassword(password);
config.setMaximumPoolSize(POOL_MAX_ACTIVE);
config.setMinimumIdle(POOL_INITIAL_SIZE);
config.setConnectionTestQuery(validationQuery);
config.setConnectionTimeout(TimeUnit.SECONDS.toMillis(jdbcTimeout));
HikariDataSource dataSource = new HikariDataSource(config);
Connection connection = dataSource.getConnection();
When I am trying to connect MYsql database I am getting the following error:
HikariPool-8 - Starting...
2019-04-10 18:45:49 WARN com.zaxxer.hikari.pool.PoolBase:472 - ReqId:60a79ef60413424e96a4fa73010fc861 - HikariPool-8 - Default transaction isolation level detection failed ((conn=14) Unknown system variable 'tx_isolation').
2019-04-10 18:45:49 ERROR com.zaxxer.hikari.pool.HikariPool:574 - ReqId:60a79ef60413424e96a4fa73010fc861 - HikariPool-8 - Exception during pool initialization.
java.sql.SQLException: (conn=14) Unknown system variable 'tx_isolation'
at org.mariadb.jdbc.internal.util.exceptions.ExceptionMapper.get(ExceptionMapper.java:198)
at org.mariadb.jdbc.internal.util.exceptions.ExceptionMapper.getException(ExceptionMapper.java:110)
What is wrong in the configuration? with DBCP2 it works fine
I am using mariadb-java-client, version: '2.2.1' driver to connect to Mysql database.
I have a small Java application for testing purposes. I have moved to hikari recently. What I notice is that I keep getting this error.
java.sql.SQLTransientConnectionException: HikariPool-1 - Connection is not available, request timed out after 30000ms.
java.sql.SQLTransientConnectionException: HikariPool-1 - Connection is not available, request timed out after 30000ms.
at com.zaxxer.hikari.pool.HikariPool.createTimeoutException(HikariPool.java:602)
at com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:195)
at com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:145)
at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:85)
Below is my settings for the hikari initially.
HikariConfig config = new HikariConfig();
config.setJdbcUrl("jdbc:mysql://localhost:3306/****");
config.setUsername("***");
config.setPassword("*****");
config.setMaximumPoolSize(20);
Hardly its being used my two devices and I ensure towards the end I do close it. So I don't know why it keep getting the error? What could be the issue or is there some settings which I need to change?
My hikari version is HikariCP-2.6.1.jar.
Your database is not obtaining connection within (30000 milliseconds that is default connectionTimeout property) because of network latency or some of the queries which are taking too long to execute(more than 30000 milliseconds).
Please try to increase value of property connectionTimeout.
YML configuration example:
spring:
datasource:
hikari:
minimumIdle: 2
maximumPoolSize: 10
idleTimeout: 120000
connectionTimeout: 300000
leakDetectionThreshold: 300000
Java Config example:
HikariConfig config = new HikariConfig();
config.setMaximumPoolSize(20);
config.setConnectionTimeout(300000);
config.setConnectionTimeout(120000);
config.setLeakDetectionThreshold(300000);
I am using spring boot and I was facing the same problem, and my solution was to get the connection like this "DataSourceUtils.getConnection(dataSource)". So I change from dataSource.getConnection() to DataSourceUtils.getConnection(dataSource).
In my case the code wasn't closing the connections.
Try-with-resources fixed it:
try (
Connection connection = dataSource.getConnection();
Statement statement = …
) {
…
}
In my case I was using JPA and hence using EntityManagerFactory for persistence and query for my springBoot project and got the same error.
The reason was in any CRUD operation I was not closing EntityManager once the operation is done hence exhausting the resources.
Hope this helps!!
EntityManager em = emf.createEntityManager();
Customer c = em.find(Customer.class , id);
em.close();
request timeout is not something that you can fix by increasing the timeout. Perhaps you'd need to evaluate all the queries from your service and implement indexing if it's needed
This can also happen if the client app is requesting lot of open connections and the database server setting has a max limit on number of pool connections. So the client app is unable to get any more connections from the database server. Check the database server connections pool to see if the max is exceeded during the time period of the errors.
Took forever to figure it out... In my case I used solution similar to #Andres Rincon:
try (Connection connection = DataSourceUtils.getConnection(jdbcTemplate.getDataSource())) {
// some code here
}
What fixed the issue in my case was to add proper indexing in the proper db tables. Take a look at the queries / transactions you're making to the db.
In my case the statement that was causing the latency was an UPDATE statement, e.g.
UPDATE table_name WHERE column1 = value1, column2 = value2;
What fixed the issue for me in this case was to add an index in that table for those two columns like:
CREATE INDEX index_name ON table_name (column1, column2);
Another good reason could be that you're not closing out your connections. You can close the connections with a try-with-resource statement like:
try( Connection connection = datasource.getConnection() ){
//your code
}
In my opinion, increasing the timeout as Girdhar Singh Rathore suggested is not ideal. It could temporarily fix the issue, but at some point you'll need to take care of proper indexing and closing connections management.
Hope this helps.
Generally opened and unclosed connections cause this problem.There is a limit of application servers to connect database and if you over this limit it will be crash your environment.
Connection must be stand on singleton pattern but if you really need to open a datasource or connect external datasource like reports you must close your connection in your finally block where you open connection block
connection.getConnection().rollback();
connection.getConnection().close();
You must also close if you are using PersistenceJpa without singleton
persistenceJPAConfig.dataSource().getConnection().rollback();
persistenceJPAConfig.dataSource().getConnection().close();
If you are using some stress test tools via creating threads to test your methods you probably get this error on your queries which take long time.It will be lead the way optimizing your queries or service instance size.
In my case a:
o.h.engine.jdbc.spi.SqlExceptionHelper: HikariPool-1 - Connection is not available, request timed out after 30019ms.
i.s.commons.web.error.ExceptionLogger: Internal Server Error
org.springframework.transaction.CannotCreateTransactionException: Could not open JPA EntityManager for transaction; nested exception is org.hibernate.exception.JDBCConnectionException: Unable to acquire JDBC Connection
Was caused by a too low spring.hikari.maximumPoolSize in the application properties, increasing from 5 to 20 solved the issue.
The log message is kind of miss-leading.
In my case I used solution similar to #Andres Rincon:
try (Connection conn = connectionManager.getDataConnection()) {
Statement stmt = conn.createStatement();
...
conn.close();
} catch (Exception e) {
e.printStackTrace();
}
I've fixed my issue using:
increase the minIdle and maxPool
spring.datasource.hikari.minimumIdle=20
spring.datasource.hikari.maximumPoolSize=30
spring.datasource.hikari.connectionTimeout=50000
To debug the issue/check if the values are ok, enable the logging for Hikari:
logging.level.com.zaxxer.hikari.HikariConfig=DEBUG
logging.level.com.zaxxer.hikari=TRACE
The logs will look like:
DEBUG 2023-01-06T16:12:31.932018849Z HikariPool-1 - Before cleanup stats (total=17, active=0, idle=17, waiting=0)
DEBUG 2023-01-06T16:12:31.932665522Z HikariPool-1 - After cleanup stats (total=17, active=0, idle=17, waiting=0)
DEBUG 2023-01-06T16:12:31.932733949Z HikariPool-1 - Fill pool skipped, pool is at sufficient level.
DEBUG 2023-01-06T16:12:32.495269726Z HikariPool-1 - After adding stats (total=17, active=0, idle=17, waiting=0)
DEBUG 2023-01-06T16:12:38.309953158Z HikariPool-1 - Fill pool skipped, pool is at sufficient level.
DEBUG 2023-01-06T16:12:39.200246897Z HikariPool-1 - Fill pool skipped, pool is at sufficient level.
DEBUG 2023-01-06T16:12:44.812065268Z HikariPool-1 - Before cleanup stats (total=18, active=0, idle=18, waiting=0)
DEBUG 2023-01-06T16:12:44.812822113Z HikariPool-1 - After cleanup stats (total=18, active=0, idle=18, waiting=0)
Good Luck ! :)
I'm having issues making a connection to an AS400 database inside of Play!.
My application.conf looks like:
db.default.driver="com.ibm.as400.access.AS400JDBCDriver"
db.default.url="jdbc:as400://SERVER;libraries=A,B,C;toolbox trace=all;trace=true"
db.default.username="user"
db.default.password="password"
I've set up jt400 in the classpath, and I can see under "external libraries" that it shows up and is available. But essentially I get an error message about failing to connect (on user/password I know works) and failure to execute isValid(), which is a function that can not be found inside of AS400JDBCConnection class.
[error] c.z.h.p.PoolBase - HikariPool-1 - Failed to execute isValid() for connection, configure connection test query. (com.ibm.as400.access.AS400JDBCConnection.isValid(I)Z)
[error] application -
! #72265nf0a - Internal server error, for (GET) [/] ->
play.api.Configuration$$anon$1: Configuration error[Cannot connect to database [default]]
at play.api.Configuration$.configError(Configuration.scala:154)
at play.api.Configuration.reportError(Configuration.scala:806)
at play.api.db.DefaultDBApi$$anonfun$connect$1.apply(DefaultDBApi.scala:48)
at play.api.db.DefaultDBApi$$anonfun$connect$1.apply(DefaultDBApi.scala:42)
at scala.collection.immutable.List.foreach(List.scala:381)
at play.api.db.DefaultDBApi.connect(DefaultDBApi.scala:42)
at play.api.db.DBApiProvider.get$lzycompute(DBModule.scala:72)
at play.api.db.DBApiProvider.get(DBModule.scala:62)
at play.api.db.DBApiProvider.get(DBModule.scala:58)
at com.google.inject.internal.ProviderInternalFactory.provision(ProviderInternalFactory.java:81)
Caused by: play.api.Configuration$$anon$1: Configuration error[Failed to initialize pool: com.ibm.as400.access.AS400JDBCConnection.isValid(I)Z]
at play.api.Configuration$.configError(Configuration.scala:154)
at play.api.PlayConfig.reportError(Configuration.scala:996)
at play.api.db.HikariCPConnectionPool.create(HikariCPModule.scala:70)
at play.api.db.PooledDatabase.createDataSource(Databases.scala:199)
at play.api.db.DefaultDatabase.dataSource$lzycompute(Databases.scala:123)
at play.api.db.DefaultDatabase.dataSource(Databases.scala:121)
at play.api.db.DefaultDatabase.getConnection(Databases.scala:142)
at play.api.db.DefaultDatabase.getConnection(Databases.scala:138)
at play.api.db.DefaultDBApi$$anonfun$connect$1.apply(DefaultDBApi.scala:44)
at play.api.db.DefaultDBApi$$anonfun$connect$1.apply(DefaultDBApi.scala:42)
Caused by: com.zaxxer.hikari.pool.HikariPool$PoolInitializationException: Failed to initialize pool: com.ibm.as400.access.AS400JDBCConnection.isValid(I)Z
at com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:512)
at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:105)
at com.zaxxer.hikari.HikariDataSource.<init>(HikariDataSource.java:71)
at play.api.db.HikariCPConnectionPool$$anonfun$1.apply(HikariCPModule.scala:58)
at play.api.db.HikariCPConnectionPool$$anonfun$1.apply(HikariCPModule.scala:54)
at scala.util.Try$.apply(Try.scala:192)
at play.api.db.HikariCPConnectionPool.create(HikariCPModule.scala:54)
at play.api.db.PooledDatabase.createDataSource(Databases.scala:199)
at play.api.db.DefaultDatabase.dataSource$lzycompute(Databases.scala:123)
at play.api.db.DefaultDatabase.dataSource(Databases.scala:121)
Caused by: java.lang.AbstractMethodError: com.ibm.as400.access.AS400JDBCConnection.isValid(I)Z
at com.zaxxer.hikari.pool.PoolBase.checkDriverSupport(PoolBase.java:400)
at com.zaxxer.hikari.pool.PoolBase.setupConnection(PoolBase.java:375)
at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:346)
at com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:506)
at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:105)
at com.zaxxer.hikari.HikariDataSource.<init>(HikariDataSource.java:71)
at play.api.db.HikariCPConnectionPool$$anonfun$1.apply(HikariCPModule.scala:58)
at play.api.db.HikariCPConnectionPool$$anonfun$1.apply(HikariCPModule.scala:54)
at scala.util.Try$.apply(Try.scala:192)
at play.api.db.HikariCPConnectionPool.create(HikariCPModule.scala:54)
I'm able to connect in other java-based projects using something like:
try {
Class.forName("com.ibm.as400.access.AS400JDBCDriver");
Connection con = DriverManager.getConnection("jdbc:as400://" +
ApplicationAuthentication.server + "/" +
ApplicationAuthentication.library,
ApplicationAuthentication.user,
ApplicationAuthentication.password
);
} catch (Exception e) {
System.err.println(e);
throw new WebApplicationException(genericError, Response.Status.UNAUTHORIZED);
}
Guessing from the stacktrace, it appears that the connection returned from your driver is not playing well with the connection Hikari Connection Pool. Hikari is default connection pool in playframework.
Specifically, your exception trace shows that the Hikari CP is attempting to call isValid method on the connection object returned by your JDBC driver and then failing with java.lang.AbstractMethodError.
You can try switching to BoneCP connection pool and see if it helps. You can also check comments on this issue on hikari github issue list
Try adding the following to application.config
db.default.hikaricp.connectionTestQuery="SELECT 1"
Not tested in Play Framework but I had similar issue on spring framework and solved in that way.
Use liquibase datasource like below with connection-test-query
#Bean
#LiquibaseDataSource
public DataSource liquibaseDataSource() {
HikariDataSource dataSource = (HikariDataSource) DataSourceBuilder.create().url("url")
.username("username")
.password("password")
.type(HikariDataSource.class).build();
dataSource.setConnectionTestQuery("select 1 from sysibm.sysdummy1");
return dataSource;
}
After getting fed up with c3p0's constant locking I'm turning to BoneCP for an alternative connection pool for my database. I have a server app that processes around 7,000 items per minute and needs to log those items into our MySQL database. I currently have 100 worker threads and have set up my pool like such:
BoneCPConfig config = new BoneCPConfig();
config.setJdbcUrl("jdbc:mysql://"+Settings.MYSQL_HOSTNAME+"/"+Settings.MYSQL_DATABASE+"?autoReconnectForPools=true" );
config.setUsername(Settings.MYSQL_USERNAME);
config.setPassword(Settings.MYSQL_PASSWORD);
config.setMinConnectionsPerPartition(5);
config.setMaxConnectionsPerPartition(10);
config.setPartitionCount(5);
config.setAcquireIncrement(5);
connectionPool = new BoneCP(config); // setup the connection pool
Are those acceptable settings for such an app? I'm asking because after a minute or two into running I was getting BoneCP exceptions when trying to call getConnection on the pool. Thanks for the help.
Here is the code I was using for the db calls in my worker threads, it can't fail on the dbConn = this.dbPool.getConnection() line. Am I not closing connections properly?
private void insertIntoDb() {
try {
Connection dbConn = this.dbPool.getConnection();
try {
PreparedStatement ps3 = dbConn.prepareStatement("INSERT IGNORE INTO test_table1 SET test1=?, test2=?, test3=?");
ps3.setString(1, "some string");
ps3.setString(2, "some other string");
ps3.setString(3, "more strings");
ps3.execute();
ps3.close();
PreparedStatement ps4 = dbConn.prepareStatement("INSERT IGNORE INTO test_table2 SET test1=?, test2=?, test3=?");
ps4.setString(1, "some string");
ps4.setString(2, "some other string");
ps4.setString(3, "more strings");
ps4.execute();
ps4.close();
} catch(SQLException e) {
logger.error(e.getMessage());
} finally {
try {
dbConn.close();
} catch (SQLException e) {
logger.error(e.getMessage());
}
}
} catch(SQLException e) {
logger.error(e.getMessage());
}
}
This is the error I was seeing:
[java] WARN [com.google.common.base.internal.Finalizer] (ConnectionPartition.java:141) - BoneCP detected an unclosed connection and will now attempt to close it for you. You should be closing this connection in your application - enable connectionWatch for additional debugging assistance.
[java] WARN [com.google.common.base.internal.Finalizer] (ConnectionPartition.java:141) - BoneCP detected an unclosed connection and will now attempt to close it for you. You should be closing this connection in your application - enable connectionWatch for additional debugging assistance.
[java] WARN [com.google.common.base.internal.Finalizer] (ConnectionPartition.java:141) - BoneCP detected an unclosed connection and will now attempt to close it for you. You should be closing this connection in your application - enable connectionWatch for additional debugging assistance.
[java] WARN [com.google.common.base.internal.Finalizer] (ConnectionPartition.java:141) - BoneCP detected an unclosed connection and will now attempt to close it for you. You should be closing this connection in your application - enable connectionWatch for additional debugging assistance.
[java] WARN [com.google.common.base.internal.Finalizer] (ConnectionPartition.java:141) - BoneCP detected an unclosed connection and will now attempt to close it for you. You should be closing this connection in your application - enable connectionWatch for additional debugging assistance.
[java] WARN [com.google.common.base.internal.Finalizer] (ConnectionPartition.java:141) - BoneCP detected an unclosed connection and will now attempt to close it for you. You should be closing this connection in your application - enable connectionWatch for additional debugging assistance.
[java] WARN [com.google.common.base.internal.Finalizer] (ConnectionPartition.java:141) - BoneCP detected an unclosed connection and will now attempt to close it for you. You should be closing this connection in your application - enable connectionWatch for additional debugging assistance.
ERROR pool-2-thread-39 2010-09-04 13:36:19,798 com.test.testpackage.MyTask - null
java.sql.SQLException
at com.jolbox.bonecp.BoneCP.getConnection(BoneCP.java:381)
Are those acceptable settings for such an app? I'm asking because after a minute or two into running I was getting boneCP exceptions when trying to call getConnection on the pool. thanks for the help.
If you have 100 workers, why do you limit the pool to 50 connections (number of partitions x max number of connections per partition i.e. 5 x 10 in your case)?
Am I not closing connections properly?
Looks ok (but maybe enable connectionWatch as hinted to see what the warning is about exactly). Personally, I close all the resources I use, including statement and result sets. Just in case, here is the idiom I use:
Connection conn = null;
PreparedStatement pstmt = null;
ResultSet rs = null;
try {
conn = pool.getConnection();
pstmt = conn.prepareStatement(SOME_SQL);
pstmt.setFoo(1, foo);
...
rs = pstmt.executeQuery();
...
} finally {
if (rs != null) try { rs.close(); } catch (SQLException quiet) {}
if (pstmt != null) try { pstmt.close(); } catch (SQLException quiet) {}
if (conn != null) try { conn.close(); } catch (SQLException quiet) {}
}
You could group the above calls in a static method of a utility class.
Or you could use DbUnit.closeQuietly(Connection, Statement, ResultSet) from Commons DbUtils that already does this.