i have threaded system. too many (thousands)
Sometimes we have had a database bottleneck.
For analyse this bottleneck i had opened log on debug mode. And i saw this :
Hikari loaded...
springHikariCP - configuration:
allowPoolSuspension.............false
autoCommit......................true
catalog.........................none
connectionInitSql...............none
connectionTestQuery............."SELECT 1"
connectionTimeout...............30000
dataSource......................none
dataSourceClassName.............none
dataSourceJNDI..................none
dataSourceProperties............{user=blabla, url=jdbc:jtds:sqlserver:
driverClassName................."net.sourceforge.jtds.jdbc.Driver"
healthCheckProperties...........{}
healthCheckRegistry.............none
idleTimeout.....................600000
initializationFailFast..........true
initializationFailTimeout.......1
isolateInternalQueries..........false
jdbc4ConnectionTest.............false
jdbcUrl........................."jdbc:jtds:sqlserver://blabla"
leakDetectionThreshold..........0
maxLifetime.....................1800000
maximumPoolSize.................3000
metricRegistry..................none
metricsTrackerFactory...........none
minimumIdle.....................0
password........................<masked>
poolName........................"springHikariCP"
readOnly........................false
registerMbeans..................false
scheduledExecutor...............none
scheduledExecutorService........internal
threadFactory...................internal
transactionIsolation............default
username........................none
validationTimeout...............5000
springHikariCP - Starting...
and there is hundered 0 statistics until end of the log...
Pool stats (total=0, active=0, idle=0, waiting=0)
Pool stats (total=0, active=0, idle=0, waiting=0)
Pool stats (total=0, active=0, idle=0, waiting=0)
.....
How is possible it? There is too many db operations update,delete,select,insert executing. What means Total=0 connections?
Hadn't we set-load Hikari configuration properly?
Related
I have a spring boot application using hikari and a Mysql database.
There is my hikari configuration :
HikariPool-1 - configuration:
allowPoolSuspension.............false
autoCommit......................true
catalog.........................none
connectionInitSql...............none
connectionTestQuery.............none
connectionTimeout...............300000
dataSource......................none
dataSourceClassName.............none
dataSourceJNDI..................none
dataSourceProperties............{password=<masked>}
driverClassName................."org.mariadb.jdbc.Driver"
exceptionOverrideClassName......none
healthCheckProperties...........{}
healthCheckRegistry.............none
idleTimeout.....................600000
initializationFailTimeout.......1
isolateInternalQueries..........false
jdbcUrl.........................jdbc:mysql://mydatabase?rewriteBatchedStatements=true
leakDetectionThreshold..........0
maxLifetime.....................27000000
maximumPoolSize.................70
metricRegistry..................none
metricsTrackerFactory...........none
minimumIdle.....................20
password........................<masked>
poolName........................"HikariPool-1"
readOnly........................false
registerMbeans..................false
scheduledExecutor...............none
schema..........................none
threadFactory...................internal
transactionIsolation............default
username........................"user_name"
validationTimeout...............10000
There is my database configuration :
wait_timeout 28800
interactive_timeout 28800
innodb_lock_wait_timeout 180
connect_timeout 10
I keep getting this error and I don't understand why :
2022-10-20 14:39:37,465 WARN c8c20a5b-7877-4830-bac8-ece994ae84e5 c.z.h.p.ProxyConnection: HikariPool-1 - Connection org.mariadb.jdbc.MariaDbConnection#5737d116 marked as broken because of SQLSTATE(08000), ErrorCode(0)
java.sql.SQLNonTransientConnectionException: (conn=10076858) Connexion terminée par expiration du délai d'attente (Read failed)
at org.mariadb.jdbc.internal.util.exceptions.ExceptionFactory.createException(ExceptionFactory.java:73)
at org.mariadb.jdbc.internal.util.exceptions.ExceptionFactory.create(ExceptionFactory.java:153)
at org.mariadb.jdbc.MariaDbStatement.executeExceptionEpilogue(MariaDbStatement.java:274)
at org.mariadb.jdbc.MariaDbStatement.executeInternal(MariaDbStatement.java:363)
at org.mariadb.jdbc.MariaDbStatement.execute(MariaDbStatement.java:501)
at org.mariadb.jdbc.MariaDbConnection.commit(MariaDbConnection.java:721)
at com.zaxxer.hikari.pool.ProxyConnection.commit(ProxyConnection.java:387)
at com.zaxxer.hikari.pool.HikariProxyConnection.commit(HikariProxyConnection.java)
at org.hibernate.resource.jdbc.internal.AbstractLogicalConnectionImplementor.commit(AbstractLogicalConnectionImplementor.java:86)
at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl$TransactionDriverControlImpl.commit(JdbcResourceLocalTransactionCoordinatorImpl.java:282)
at org.hibernate.engine.transaction.internal.TransactionImpl.commit(TransactionImpl.java:101)
at org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:562)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:743)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:711)
at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:654)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:407)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:692)
The connection 5737d116 referenced in the stacktrace has been created just 2 hours before :
2022-10-20 12:37:30,333 DEBUG c.z.h.p.HikariPool: HikariPool-1 - Added connection org.mariadb.jdbc.MariaDbConnection#5737d116
And here is the state of the pool of connection just before the exception occured :
2022-10-20 14:39:00,600 DEBUG c.z.h.p.HikariPool: HikariPool-1 - Before cleanup stats (total=21, active=1, idle=20, waiting=0)
2022-10-20 14:39:00,601 DEBUG c.z.h.p.HikariPool: HikariPool-1 - After cleanup stats (total=21, active=1, idle=20, waiting=0)
2022-10-20 14:39:00,601 DEBUG c.z.h.p.HikariPool: HikariPool-1 - Fill pool skipped, pool is at sufficient level.
2022-10-20 14:39:02,197 DEBUG c.z.h.p.HikariPool: HikariPool-2 - Before cleanup stats (total=20, active=0, idle=20, waiting=0)
2022-10-20 14:39:02,197 DEBUG c.z.h.p.HikariPool: HikariPool-2 - After cleanup stats (total=20, active=0, idle=20, waiting=0)
2022-10-20 14:39:02,197 DEBUG c.z.h.p.HikariPool: HikariPool-2 - Fill pool skipped, pool is at sufficient level.
2022-10-20 14:39:30,601 DEBUG c.z.h.p.HikariPool: HikariPool-1 - Before cleanup stats (total=21, active=1, idle=20, waiting=0)
2022-10-20 14:39:30,601 DEBUG c.z.h.p.HikariPool: HikariPool-1 - After cleanup stats (total=21, active=1, idle=20, waiting=0)
2022-10-20 14:39:30,601 DEBUG c.z.h.p.HikariPool: HikariPool-1 - Fill pool skipped, pool is at sufficient level.
2022-10-20 14:39:32,198 DEBUG c.z.h.p.HikariPool: HikariPool-2 - Before cleanup stats (total=20, active=0, idle=20, waiting=0)
2022-10-20 14:39:32,198 DEBUG c.z.h.p.HikariPool: HikariPool-2 - After cleanup stats (total=20, active=0, idle=20, waiting=0)
Thanks in advance for your help
I recently updated my spring-boot/spring-batch application from spring-boot-starter 1.5.9.RELEASE (uses spring-version 4.3.13.RELEASE) => 2.2.4.RELEASE (uses spring-version 5.2.3.RELEASE. The 1.5.9.RELEASE was using Hikari 2.6.3; I did try this version also in my migrated/updated 2.2.4.RELEASE version of my app. 2.2.4.RELEASE uses Hikari 3.4.2 Neither seem to work. I am using postgresql 42.2.9
This is how the Hikari logs to my console when it works (when running the 1.5.9.RELEASE spring-boot version of my app):
3c22fb53ed64 2021-04-09 05:10:58.223 WARN CorrelationId=
[main] com.zaxxer.hikari.HikariConfig - HikariPool-1 - maxLifetime is less than 30000ms, setting to default 1800000ms.
3c22fb53ed64 2021-04-09 05:10:58.223 DEBUG CorrelationId=
[main] com.zaxxer.hikari.HikariConfig - HikariPool-1 - configuration:
3c22fb53ed64 2021-04-09 05:10:58.224 DEBUG CorrelationId=
[main] com.zaxxer.hikari.HikariConfig - allowPoolSuspension.............false
3c22fb53ed64 2021-04-09 05:10:58.224 DEBUG CorrelationId=
[main] com.zaxxer.hikari.HikariConfig - autoCommit......................false
3c22fb53ed64 2021-04-09 05:10:58.224 DEBUG CorrelationId=
[main] com.zaxxer.hikari.HikariConfig - catalog.........................none
3c22fb53ed64 2021-04-09 05:10:58.224 DEBUG CorrelationId=
[main] com.zaxxer.hikari.HikariConfig - connectionInitSql...............none
3c22fb53ed64 2021-04-09 05:10:58.224 DEBUG CorrelationId=
[main] com.zaxxer.hikari.HikariConfig - connectionTestQuery.............none
3c22fb53ed64 2021-04-09 05:10:58.224 DEBUG CorrelationId=
[main] com.zaxxer.hikari.HikariConfig - connectionTimeout...............60000
3c22fb53ed64 2021-04-09 05:10:58.224 DEBUG CorrelationId=
[main] com.zaxxer.hikari.HikariConfig - dataSource......................none
3c22fb53ed64 2021-04-09 05:10:58.224 DEBUG CorrelationId=
[main] com.zaxxer.hikari.HikariConfig - dataSourceClassName.............none
3c22fb53ed64 2021-04-09 05:10:58.224 DEBUG CorrelationId=
[main] com.zaxxer.hikari.HikariConfig - dataSourceJNDI..................none
3c22fb53ed64 2021-04-09 05:10:58.225 DEBUG CorrelationId=
[main] com.zaxxer.hikari.HikariConfig - dataSourceProperties............{password=<masked>}
3c22fb53ed64 2021-04-09 05:10:58.225 DEBUG CorrelationId=
[main] com.zaxxer.hikari.HikariConfig - driverClassName.................none
3c22fb53ed64 2021-04-09 05:10:58.225 DEBUG CorrelationId=
[main] com.zaxxer.hikari.HikariConfig - healthCheckProperties...........{}
3c22fb53ed64 2021-04-09 05:10:58.225 DEBUG CorrelationId=
[main] com.zaxxer.hikari.HikariConfig - healthCheckRegistry.............none
3c22fb53ed64 2021-04-09 05:10:58.225 DEBUG CorrelationId=
[main] com.zaxxer.hikari.HikariConfig - idleTimeout.....................600000
3c22fb53ed64 2021-04-09 05:10:58.225 DEBUG CorrelationId=
[main] com.zaxxer.hikari.HikariConfig - initializationFailFast..........true
3c22fb53ed64 2021-04-09 05:10:58.225 DEBUG CorrelationId=
[main] com.zaxxer.hikari.HikariConfig - initializationFailTimeout.......1
3c22fb53ed64 2021-04-09 05:10:58.225 DEBUG CorrelationId=
[main] com.zaxxer.hikari.HikariConfig - isolateInternalQueries..........false
3c22fb53ed64 2021-04-09 05:10:58.225 DEBUG CorrelationId=
[main] com.zaxxer.hikari.HikariConfig - jdbc4ConnectionTest.............false
3c22fb53ed64 2021-04-09 05:10:58.225 DEBUG CorrelationId=
[main] com.zaxxer.hikari.HikariConfig - jdbcUrl........................."jdbc:postgresql://mydatabaseurl.mywebsite.com:5432/supercooldatabase"
3c22fb53ed64 2021-04-09 05:10:58.225 DEBUG CorrelationId=
[main] com.zaxxer.hikari.HikariConfig - leakDetectionThreshold..........0
3c22fb53ed64 2021-04-09 05:10:58.225 DEBUG CorrelationId=
[main] com.zaxxer.hikari.HikariConfig - maxLifetime.....................1800000
3c22fb53ed64 2021-04-09 05:10:58.225 DEBUG CorrelationId=
[main] com.zaxxer.hikari.HikariConfig - maximumPoolSize.................5
3c22fb53ed64 2021-04-09 05:10:58.225 DEBUG CorrelationId=
[main] com.zaxxer.hikari.HikariConfig - metricRegistry..................none
3c22fb53ed64 2021-04-09 05:10:58.225 DEBUG CorrelationId=
[main] com.zaxxer.hikari.HikariConfig - metricsTrackerFactory...........none
3c22fb53ed64 2021-04-09 05:10:58.225 DEBUG CorrelationId=
[main] com.zaxxer.hikari.HikariConfig - minimumIdle.....................5
3c22fb53ed64 2021-04-09 05:10:58.225 DEBUG CorrelationId=
[main] com.zaxxer.hikari.HikariConfig - password........................<masked>
3c22fb53ed64 2021-04-09 05:10:58.225 DEBUG CorrelationId=
[main] com.zaxxer.hikari.HikariConfig - poolName........................"HikariPool-1"
3c22fb53ed64 2021-04-09 05:10:58.225 DEBUG CorrelationId=
[main] com.zaxxer.hikari.HikariConfig - readOnly........................false
3c22fb53ed64 2021-04-09 05:10:58.225 DEBUG CorrelationId=
[main] com.zaxxer.hikari.HikariConfig - registerMbeans..................false
3c22fb53ed64 2021-04-09 05:10:58.225 DEBUG CorrelationId=
[main] com.zaxxer.hikari.HikariConfig - scheduledExecutor...............none
3c22fb53ed64 2021-04-09 05:10:58.225 DEBUG CorrelationId=
[main] com.zaxxer.hikari.HikariConfig - scheduledExecutorService........internal
3c22fb53ed64 2021-04-09 05:10:58.225 DEBUG CorrelationId=
[main] com.zaxxer.hikari.HikariConfig - threadFactory...................internal
3c22fb53ed64 2021-04-09 05:10:58.225 DEBUG CorrelationId=
[main] com.zaxxer.hikari.HikariConfig - transactionIsolation............default
3c22fb53ed64 2021-04-09 05:10:58.226 DEBUG CorrelationId=
[main] com.zaxxer.hikari.HikariConfig - username........................"mktgoffers_ddladmin"
3c22fb53ed64 2021-04-09 05:10:58.226 DEBUG CorrelationId=
[main] com.zaxxer.hikari.HikariConfig - validationTimeout...............5000
3c22fb53ed64 2021-04-09 05:10:58.226 INFO CorrelationId=
[main] com.zaxxer.hikari.HikariDataSource - HikariPool-1 - Starting...
3c22fb53ed64 2021-04-09 05:10:59.377 DEBUG CorrelationId=
[main] com.zaxxer.hikari.pool.HikariPool - HikariPool-1 - Added connection org.postgresql.jdbc.PgConnection#14ed7ddf
3c22fb53ed64 2021-04-09 05:10:59.379 INFO CorrelationId=
[main] com.zaxxer.hikari.HikariDataSource - HikariPool-1 - Start completed.
3c22fb53ed64 2021-04-15 01:04:40.021 INFO CorrelationId=
[main] org.hibernate.dialect.Dialect - HHH000400: Using dialect: org.hibernate.dialect.PostgreSQL82Dialect
You can see it configures Hikari with my connection information and shows the jdbc url and even adds the postgresql connection.
This is how it looks when it does NOT work, after I updated to spring-boot 2.2.4.RELEASE:
2021-04-15 01:14:11.681 INFO 75096 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Starting...
2021-04-15 01:14:12.748 INFO 75096 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Start completed.
2021-04-15 01:14:12.760 INFO 75096 --- [ main] org.hibernate.dialect.Dialect : HHH000400: Using dialect: org.hibernate.dialect.PostgreSQL82Dialect
As you can see, its different. Normally I wouldn't care about logs, but when any of my JpaRepository methods get called like .saveAll(), the application hangs/stops doing anything and never writes to database. I was perplexed after first, and thought it was my Transaction/Entity manager setup in Spring-batch, but as I said before, all my code/configuration actually works perfectly fine (writes to database) on 1.5.9.RELEASE spring boot. Now, I just discovered this database config in the log, and I'm 99% sure my database connection isn't even initializing properly.
Here is my application yaml (this is a snippet, but has all relevant info to database connection):
spring:
application:
name: My Super Cool App
profiles:
active: "myLocal,myLocalDev"
batch.job.enabled: false
datasource:
hikari:
maximumPoolSize: 5
connectionTimeout: 60000
max-lifetime: 20000
auto-commit: false
jpa:
database: POSTGRESQL
database-platform: org.hibernate.dialect.PostgreSQL82Dialect
show-sql: false
hibernate:
ddl-auto: none
properties:
hibernate:
jdbc.batch_size: 5000
order_inserts: true
flushMode: COMMIT
---
spring:
profiles: myLocal
datasource:
url: jdbc:postgresql://database-main-url.mywebsite.com:5432/cooldbname
username: mysuperkewluser
password: hahasupersecretpw
And my definition for my database configuration, DatabaseConfiguration.java:
#Configuration
#EnableTransactionManagement
public class DatabaseConfiguration {
#Autowired
private Environment envVar;
#Autowired
private KeyVault keyProvider;
#Bean(name = "dataSource")
#ConfigurationProperties(
prefix = "spring.datasource.hikari"
)
public DataSource dataSource(DataSourceProperties dataSourceProperties) {
log.debug("Configuring Datasource");
String dbPassword = keyProvider.getKeyProvider(dataSourceProperties.getPassword());
return DataSourceBuilder
.create(dataSourceProperties.getClassLoader())
.type(HikariDataSource.class)
.driverClassName(dataSourceProperties.getDriverClassName())
.url(dataSourceProperties.getUrl())
.username(dataSourceProperties.getUsername())
.password(dbPassword)
.build();
}
}
I am using this #Autowiring this dataSource in my Spring Batch JobRepository, which is defined in my BatchConfiguration.java class
#Configuration
#EnableBatchProcessing
public class BatchConfiguration {
// Autowiring of the JobBuilderFactory, StepBuilderFactory here
#Bean
public JobRepository jobRepository(PlatformTransactionManager transactionManager, #Qualifier("dataSource") DataSource dataSource) throws Exception {
JobRepositoryFactoryBean jobRepositoryFactoryBean = new JobRepositoryFactoryBean();
jobRepositoryFactoryBean.setDataSource(dataSource); // dataSource
jobRepositoryFactoryBean.setTransactionManager(transactionManager);
jobRepositoryFactoryBean.setDatabaseType("POSTGRES");
return jobRepositoryFactoryBean.getObject();
// more business logic code here
}
I am facing this problem for 2 hours.
The problem is I have other classes that are working well. I don't know why this error is happening for the current class.
org.hibernate.exception.JDBCConnectionException: unable to obtain isolated JDBC connection
Caused by: java.sql.SQLTransientConnectionException: HikariPool-1 - Connection is not available, request timed out after 351ms.
But the code works for other tables(other classes).
It was working so fine until today. Is there anyone that faced this problem before?
2020-03-13 13:52:25,392 [main] WARN com.zaxxer.hikari.HikariConfig -ScraperPool - idleTimeout is less than 10000ms, setting to default 600000ms.
2020-03-13 13:52:25,400 [main] DEBUG com.zaxxer.hikari.HikariConfig -ScraperPool - configuration:
2020-03-13 13:52:25,406 [main] DEBUG com.zaxxer.hikari.HikariConfig -allowPoolSuspension.............false
2020-03-13 13:52:25,407 [main] DEBUG com.zaxxer.hikari.HikariConfig -autoCommit......................true
2020-03-13 13:52:25,407 [main] DEBUG com.zaxxer.hikari.HikariConfig -catalog.........................none
2020-03-13 13:52:25,408 [main] DEBUG com.zaxxer.hikari.HikariConfig -connectionInitSql...............none
2020-03-13 13:52:25,408 [main] DEBUG com.zaxxer.hikari.HikariConfig -connectionTestQuery.............none
2020-03-13 13:52:25,409 [main] DEBUG com.zaxxer.hikari.HikariConfig -connectionTimeout...............10000
2020-03-13 13:52:25,409 [main] DEBUG com.zaxxer.hikari.HikariConfig -dataSource......................none
2020-03-13 13:52:25,409 [main] DEBUG com.zaxxer.hikari.HikariConfig -dataSourceClassName.............none
2020-03-13 13:52:25,410 [main] DEBUG com.zaxxer.hikari.HikariConfig -dataSourceJNDI..................none
2020-03-13 13:52:25,411 [main] DEBUG com.zaxxer.hikari.HikariConfig -dataSourceProperties............{password=<masked>, prepStmtCacheSqlLimit=2048, cachePrepStmts=true, prepStmtCacheSize=250, leakDetectionThreshold=15000}
2020-03-13 13:52:25,412 [main] DEBUG com.zaxxer.hikari.HikariConfig -driverClassName.................none
2020-03-13 13:52:25,412 [main] DEBUG com.zaxxer.hikari.HikariConfig -healthCheckProperties...........{}
2020-03-13 13:52:25,412 [main] DEBUG com.zaxxer.hikari.HikariConfig -healthCheckRegistry.............none
2020-03-13 13:52:25,413 [main] DEBUG com.zaxxer.hikari.HikariConfig -idleTimeout.....................600000
2020-03-13 13:52:25,413 [main] DEBUG com.zaxxer.hikari.HikariConfig -initializationFailTimeout.......1
2020-03-13 13:52:25,413 [main] DEBUG com.zaxxer.hikari.HikariConfig -isolateInternalQueries..........false
2020-03-13 13:52:25,414 [main] DEBUG com.zaxxer.hikari.HikariConfig -jdbcUrl.........................jdbc:mysql://**********?verifyServerCertificate=false&useSSL=true&serverTimezone=UTC
2020-03-13 13:52:25,414 [main] DEBUG com.zaxxer.hikari.HikariConfig -leakDetectionThreshold..........10000
2020-03-13 13:52:25,414 [main] DEBUG com.zaxxer.hikari.HikariConfig -maxLifetime.....................1800000
2020-03-13 13:52:25,414 [main] DEBUG com.zaxxer.hikari.HikariConfig -maximumPoolSize.................100
2020-03-13 13:52:25,415 [main] DEBUG com.zaxxer.hikari.HikariConfig -metricRegistry..................none
2020-03-13 13:52:25,415 [main] DEBUG com.zaxxer.hikari.HikariConfig -metricsTrackerFactory...........none
2020-03-13 13:52:25,415 [main] DEBUG com.zaxxer.hikari.HikariConfig -minimumIdle.....................4
2020-03-13 13:52:25,415 [main] DEBUG com.zaxxer.hikari.HikariConfig -password........................<masked>
2020-03-13 13:52:25,416 [main] DEBUG com.zaxxer.hikari.HikariConfig -poolName........................"ScraperPool"
2020-03-13 13:52:25,416 [main] DEBUG com.zaxxer.hikari.HikariConfig -readOnly........................false
2020-03-13 13:52:25,416 [main] DEBUG com.zaxxer.hikari.HikariConfig -registerMbeans..................true
2020-03-13 13:52:25,417 [main] DEBUG com.zaxxer.hikari.HikariConfig -scheduledExecutor...............none
2020-03-13 13:52:25,417 [main] DEBUG com.zaxxer.hikari.HikariConfig -schema..........................none
2020-03-13 13:52:25,417 [main] DEBUG com.zaxxer.hikari.HikariConfig -threadFactory...................internal
2020-03-13 13:52:25,417 [main] DEBUG com.zaxxer.hikari.HikariConfig -transactionIsolation............default
2020-03-13 13:52:25,418 [main] DEBUG com.zaxxer.hikari.HikariConfig -username........................"xxxxxxxxxxx"
2020-03-13 13:52:25,418 [main] DEBUG com.zaxxer.hikari.HikariConfig -validationTimeout...............5000
2020-03-13 13:52:25,420 [main] INFO com.zaxxer.hikari.HikariDataSource -ScraperPool - Starting...
2020-03-13 13:52:25,452 [main] DEBUG com.zaxxer.hikari.util.DriverDataSource -Loaded driver with class name com.mysql.cj.jdbc.Driver for jdbcUrl=jdbc:mysql://*********?verifyServerCertificate=false&useSSL=true&serverTimezone=UTC
2020-03-13 13:52:27,099 [main] DEBUG com.zaxxer.hikari.pool.HikariPool -ScraperPool - Added connection com.mysql.cj.jdbc.ConnectionImpl#e84a8e1
2020-03-13 13:52:27,107 [main] INFO com.zaxxer.hikari.HikariDataSource -ScraperPool - Start completed.
2020-03-13 13:52:27,211 [ScraperPool housekeeper] DEBUG com.zaxxer.hikari.pool.HikariPool -ScraperPool - Before cleanup stats (total=1, active=0, idle=1, waiting=0)
2020-03-13 13:52:27,212 [ScraperPool housekeeper] DEBUG com.zaxxer.hikari.pool.HikariPool -ScraperPool - After cleanup stats (total=1, active=0, idle=1, waiting=0)
2020-03-13 13:52:27,280 [ScraperPool connection adder] DEBUG com.zaxxer.hikari.pool.HikariPool -ScraperPool - Added connection com.mysql.cj.jdbc.ConnectionImpl#58b0aed5
2020-03-13 13:52:27,346 [ScraperPool connection adder] DEBUG com.zaxxer.hikari.pool.HikariPool -ScraperPool - Added connection com.mysql.cj.jdbc.ConnectionImpl#285edfdd
2020-03-13 13:52:27,414 [ScraperPool connection adder] DEBUG com.zaxxer.hikari.pool.HikariPool -ScraperPool - Added connection com.mysql.cj.jdbc.ConnectionImpl#79a25ba1
2020-03-13 13:52:27,415 [ScraperPool connection adder] DEBUG com.zaxxer.hikari.pool.HikariPool -ScraperPool - After adding stats (total=4, active=0, idle=4, waiting=0)
I got this line before the error occured
DEBUG com.zaxxer.hikari.pool.HikariPool -HikariPool-1 - Timeout failure stats (total=1, active=1, idle=0, waiting=0)
Try increasing the max limit of your pool. Below code tells you that pool reached max connection limit set in the config.
DEBUG com.zaxxer.hikari.pool.HikariPool -HikariPool-1 - Timeout failure stats (total=1, active=1, idle=0, waiting=0)
My application starts in INFO: Server startup in [337,971] milliseconds
I have tried setting hibernate.temp.use_jdbc_metadata_defaults to false.
I have tried setting hibernate.dialect to org.hibernate.dialect.PostgreSQLDialect
I can connect using pgadmin to Postgres server in ms and also can query in short period of time.
The problem is in application.
jdbc.prefetchsize=10
jdbc.maximumPoolSize=1
jdbc.minimumIdle=0
jdbc.idleTimeout=30000
<property name="jpaProperties">
<props>
<prop key="hibernate.dialect">org.hibernate.dialect.PostgreSQLDialect</prop>
<prop key="hibernate.show_sql">true</prop>
<prop key="hibernate.format_sql">false</prop>
<prop key="hibernate.generate_statistics">false</prop>
<prop key="hibernate.auditable">true</prop>
<prop key="org.hibernate.envers.audit_table_prefix">Z_</prop>
<prop key="hibernate.temp.use_jdbc_metadata_defaults">false</prop>
</props>
</property>
Here is the console log for the problematic part:
16:41:07.395 [HikariPool-1 housekeeper] DEBUG com.zaxxer.hikari.pool.HikariPool - HikariPool-1 - Before cleanup stats (total=1, active=0, idle=1, waiting=0)
16:41:07.396 [HikariPool-1 housekeeper] DEBUG com.zaxxer.hikari.pool.HikariPool - HikariPool-1 - After cleanup stats (total=1, active=0, idle=1, waiting=0)
16:41:37.401 [HikariPool-1 housekeeper] DEBUG com.zaxxer.hikari.pool.HikariPool - HikariPool-1 - Before cleanup stats (total=1, active=0, idle=1, waiting=0)
16:41:37.401 [HikariPool-1 housekeeper] DEBUG com.zaxxer.hikari.pool.HikariPool - HikariPool-1 - After cleanup stats (total=0, active=0, idle=0, waiting=0)
16:41:37.401 [HikariPool-1 connection closer] DEBUG com.zaxxer.hikari.pool.PoolBase - HikariPool-1 - Closing connection org.postgresql.jdbc.PgConnection#4bfc28ec: (connection has passed idleTimeout)
16:42:07.401 [HikariPool-1 housekeeper] DEBUG com.zaxxer.hikari.pool.HikariPool - HikariPool-1 - Before cleanup stats (total=0, active=0, idle=0, waiting=0)
16:42:07.402 [HikariPool-1 housekeeper] DEBUG com.zaxxer.hikari.pool.HikariPool - HikariPool-1 - After cleanup stats (total=0, active=0, idle=0, waiting=0)
16:42:37.403 [HikariPool-1 housekeeper] DEBUG com.zaxxer.hikari.pool.HikariPool - HikariPool-1 - Before cleanup stats (total=0, active=0, idle=0, waiting=0)
16:42:37.403 [HikariPool-1 housekeeper] DEBUG com.zaxxer.hikari.pool.HikariPool - HikariPool-1 - After cleanup stats (total=0, active=0, idle=0, waiting=0)
16:43:07.404 [HikariPool-1 housekeeper] DEBUG com.zaxxer.hikari.pool.HikariPool - HikariPool-1 - Before cleanup stats (total=0, active=0, idle=0, waiting=0)
16:43:07.404 [HikariPool-1 housekeeper] DEBUG com.zaxxer.hikari.pool.HikariPool - HikariPool-1 - After cleanup stats (total=0, active=0, idle=0, waiting=0)
16:43:37.407 [HikariPool-1 housekeeper] DEBUG com.zaxxer.hikari.pool.HikariPool - HikariPool-1 - Before cleanup stats (total=0, active=0, idle=0, waiting=0)
16:43:37.408 [HikariPool-1 housekeeper] DEBUG com.zaxxer.hikari.pool.HikariPool - HikariPool-1 - After cleanup stats (total=0, active=0, idle=0, waiting=0)
16:44:07.408 [HikariPool-1 housekeeper] DEBUG com.zaxxer.hikari.pool.HikariPool - HikariPool-1 - Before cleanup stats (total=0, active=0, idle=0, waiting=0)
16:44:07.409 [HikariPool-1 housekeeper] DEBUG com.zaxxer.hikari.pool.HikariPool - HikariPool-1 - After cleanup stats (total=0, active=0, idle=0, waiting=0)
Jul 16, 2019 4:44:29 PM com.etstur.cs.backend.datasrvds.impl.RuleContainer getMap
INFO: Rule map refilled in 219127ms
16:44:30.078 [HikariPool-1 connection adder] DEBUG com.zaxxer.hikari.pool.HikariPool - HikariPool-1 - Added connection org.postgresql.jdbc.PgConnection#110cb9bc
Hibernate: select autocomple0_.id as id1_13_, autocomple0_.status as status2_13_, autocomple0_.text as text3_13_ from AutoCompleteWhiteList autocomple0_ where autocomple0_.status='ACTIVE'
Hibernate: select hotel_id from black_friday_hotels
Initializing ExecutorService 'taskExecutor'
Root WebApplicationContext initialized in 324837 ms
I am getting connection is not available error msg with below config xml mapping.Please suggest me what i am doing wrong in beolw code or this is some another issue. As per my investigation this is an intregation issue with HikariCP, Hibernate and Spring Batch.
2.0.3
5.1.6
4.0.0.RELEASE
3.0.0.RELEASE
3.0.0.RELEASE
0.5
2.3.2
4.3.5.Final
Below Configuration
#Bean
public DataSource dataSource() {
final HikariConfig config = new HikariConfig();
config.setMaximumPoolSize(1);
System.out.println("Pool size is "+config.getMaximumPoolSize());
config.setDriverClassName(environment.getRequiredProperty(PROPERTY_NAME_DATABASE_DRIVER));
config.setJdbcUrl(environment.getRequiredProperty(PROPERTY_NAME_DATABASE_URL));
config.setUsername(environment.getRequiredProperty(PROPERTY_NAME_DATABASE_USERNAME));
config.setPassword(environment.getRequiredProperty(PROPERTY_NAME_DATABASE_PASSWORD));
final HikariDataSource dataSource = new HikariDataSource(config);
return dataSource;
}
Error:--
DEBUG com.zaxxer.hikari.pool.HikariPool - HikariPool-1 - Timeout failure stats (total=1, active=1, idle=0, waiting=0)
2017-01-03T11:10:30.33+0000 [App/0] OUT PDC::fre-staging/0: 2017-01-03 11:10:30.335 [task-scheduler-6] WARN o.h.e.jdbc.spi.SqlExceptionHelper - SQL Error: 0, SQLState: null
2017-01-03T11:10:30.33+0000 [App/0] OUT PDC::fre-staging/0: 2017-01-03 11:10:30.335 [task-scheduler-6] ERROR o.h.e.jdbc.spi.SqlExceptionHelper - HikariPool-1 - Connection is not available, request timed out after 30000ms.
2017-01-03T11:10:30.34+0000 [App/0] OUT In finally If
2017-01-03T11:10:43.89+0000 [App/0] OUT PDC::fre-staging/0: 2017-01-03 11:10:43.896 [HikariPool-1 housekeeper] DEBUG com.zaxxer.hikari.pool.HikariPool - HikariPool-1 - Pool stats (total=1, active=1, idle=0, waiting=1)
2017-01-03T11:11:00.34+0000 [App/0] OUT PDC::fre-staging/0: 2017-01-03 11:11:00.340 [task-scheduler-6] DEBUG com.zaxxer.hikari.pool.HikariPool - HikariPool-1 - Timeout failure stats (total=1, active=1, idle=0, waiting=0)
2017-01-03T11:11:00.34+0000 [App/0] OUT PDC::fre-staging/0: 2017-01-03 11:11:00.341 [task-scheduler-6] WARN o.h.e.jdbc.spi.SqlExceptionHelper - SQL Error: 0, SQLState: null
2017-01-03T11:11:00.34+0000 [App/0] OUT PDC::fre-staging/0: 2017-01-03 11:11:00.341 [task-scheduler-6] ERROR o.h.e.jdbc.spi.SqlExceptionHelper - HikariPool-1 - Connection is not available, request timed out after 30000ms.
2017-01-03T11:11:00.35+0000 [App/0] OUT PDC::fre-staging/0: 2017-01-03 11:11:00.354 [task-scheduler-6] ERROR o.s.batch.core.step.AbstractStep - Encountered an error executing step emisTopupStep in job emisTopupJob
2017-01-03T11:11:00.35+0000 [App/0] OUT javax.persistence.PersistenceException: org.hibernate.exception.JDBCConnectionException: Could not open connection.
Please help me out.
You are setting the pool max size to 1 with the line
config.setMaximumPoolSize(1);
My guess is that is your problem. Try increasing it, just don't over do it. The defaults is 10 which is good for most cases.