I have an application in Tomcat 8 Engine which have both: JDBC and Hibernate connections to two different databases. The Hibernate connection fails after 3-4 hours of user inactivity
also I have c3p0 properties in hibernate.cfg.xml
hibernate.c3p0.min_size 1
hibernate.c3p0.idle_test_period 100
the exception stacktrace:
STACKTRACE:
com.mysql.jdbc.CommunicationsException: Communications link failure due to underlying exception:
** BEGIN NESTED EXCEPTION **
java.net.SocketException
MESSAGE: Broken pipe
STACKTRACE:
java.net.SocketException: Broken pipe
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:109)
at java.net.SocketOutputStream.write(SocketOutputStream.java:153)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
at com.mysql.jdbc.MysqlIO.send(MysqlIO.java:2637)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1554)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:1665)
at com.mysql.jdbc.Connection.execSQL(Connection.java:3176)
at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1153)
at com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:1266)
at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeQuery(NewProxyPreparedStatement.java:116)
at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.extract(ResultSetReturnImpl.java:80)
at org.hibernate.loader.Loader.getResultSet(Loader.java:2065)
at org.hibernate.loader.Loader.executeQueryStatement(Loader.java:1862)
at org.hibernate.loader.Loader.executeQueryStatement(Loader.java:1838)
at org.hibernate.loader.Loader.doQuery(Loader.java:909)
at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:354)
at org.hibernate.loader.Loader.doList(Loader.java:2551)
at org.hibernate.loader.Loader.doList(Loader.java:2537)
at org.hibernate.loader.Loader.listIgnoreQueryCache(Loader.java:2367)
at org.hibernate.loader.Loader.list(Loader.java:2362)
at org.hibernate.loader.hql.QueryLoader.list(QueryLoader.java:496)
at org.hibernate.hql.internal.ast.QueryTranslatorImpl.list(QueryTranslatorImpl.java:387)
at org.hibernate.engine.query.spi.HQLQueryPlan.performList(HQLQueryPlan.java:229)
at org.hibernate.internal.SessionImpl.list(SessionImpl.java:1260)
at org.hibernate.internal.QueryImpl.list(QueryImpl.java:103)
at DAO.CarsDAOimpl.getData(CarsDAOimpl.java:121)
does anybody has an idea what is going on and how to fix that?
UPD: here is about memory state:
Runtime runtime = Runtime.getRuntime();
long freeMemory = runtime.freeMemory();
...
free memory: 183787192
Yes, I have had the same problem. Try to add a validationQuery to your DataSource.
Here is how I have defined mine:
<Resource auth="Container"
type="javax.sql.DataSource"
name="jdbc/mydb"
driverClassName="com.mysql.jdbc.Driver"
url="jdbc:mysql://localhost/mydb"
maxActive="10"
maxIdle="5"
validationQuery="SELECT 1"
testOnBorrow="true"
testWhileIdle="true"
timeBetweenEvictionRunsMillis="10000"
minEvictableIdleTimeMillis="60000"
username="..." password="..."/>
Related
I set up the redis-cluster in docker in aws ec2. They use the docker swarm net. The ip is from 10.0.0.6:6379 to 10.0.0.11:6379 and is mapped to the ec2 static address like (public address:5001 to public address:5006) I can use the client tool like TablePlus to access them using public address successfully. But when I use the spring boot(version 2.1.1) and StringRedisTemplate to set value in redis,(the configuration file is
spring.redis.database=0
spring.redis.password=xxxxxx
spring.redis.cluster.nodes=public address:5001,public address:5002,public address:5003,public address:5004,public address:5005,public address:5006
spring.redis.timeout=5000
The pom is
<!-- redis -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-redis</artifactId>
</dependency>
<dependency>
<groupId>redis.clients</groupId>
<artifactId>jedis</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.data</groupId>
<artifactId>spring-data-redis</artifactId>
</dependency>
, I always got the error message like below:
2019-01-14 08:15:28,570 [http-nio-8081-exec-2] [org.springframework.web.servlet.FrameworkServlet.initServletBean(FrameworkServlet.java:524)] - [INFO] Initializing Servlet 'dispatcherServlet'
2019-01-14 08:15:28,577 [http-nio-8081-exec-2] [org.springframework.web.servlet.FrameworkServlet.initServletBean(FrameworkServlet.java:546)] - [INFO] Completed initialization in 7 ms
2019-01-14 08:15:52,589 [http-nio-8081-exec-8] [com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:110)] - [INFO] HikariPool-1 - Starting...
2019-01-14 08:15:56,831 [http-nio-8081-exec-8] [com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:123)] - [INFO] HikariPool-1 - Start completed.
2019-01-14 08:15:58,734 [http-nio-8081-exec-8] [io.lettuce.core.EpollProvider.<clinit>(EpollProvider.java:68)] - [INFO] Starting without optional epoll library
2019-01-14 08:15:58,736 [http-nio-8081-exec-8] [io.lettuce.core.KqueueProvider.<clinit>(KqueueProvider.java:70)] - [INFO] Starting without optional kqueue library
2019-01-14 08:16:09,935 [lettuce-nioEventLoop-4-1] [io.lettuce.core.cluster.topology.ClusterTopologyRefresh.lambda$getConnections$2(ClusterTopologyRefresh.java:228)] - [WARN] Unable to connect to 10.0.0.7:6379
java.util.concurrent.CompletionException: io.netty.channel.ConnectTimeoutException: connection timed out: /10.0.0.7:6379
at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292)
at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:308)
at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:593)
at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1977)
at io.lettuce.core.AbstractRedisClient.lambda$initializeChannelAsync0$4(AbstractRedisClient.java:329)
at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:511)
at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:504)
at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:483)
at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:424)
at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:121)
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe$1.run(AbstractNioChannel.java:269)
at io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38)
at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:127)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:466)
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:897)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
Caused by: io.netty.channel.ConnectTimeoutException: connection timed out: /10.0.0.7:6379
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe$1.run(AbstractNioChannel.java:267)
... 8 more
2019-01-14 08:16:09,935 [lettuce-nioEventLoop-4-9] [io.lettuce.core.cluster.topology.ClusterTopologyRefresh.lambda$getConnections$2(ClusterTopologyRefresh.java:228)] - [WARN] Unable to connect to 10.0.0.10:6379
java.util.concurrent.CompletionException: io.netty.channel.ConnectTimeoutException: connection timed out: /10.0.0.10:6379
at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292)
at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:308)
at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:593)
at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1977)
at io.lettuce.core.AbstractRedisClient.lambda$initializeChannelAsync0$4(AbstractRedisClient.java:329)
at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:511)
at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:504)
at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:483)
at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:424)
at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:121)
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe$1.run(AbstractNioChannel.java:269)
at io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38)
at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:127)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:466)
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:897)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
Caused by: io.netty.channel.ConnectTimeoutException: connection timed out: /10.0.0.10:6379
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe$1.run(AbstractNioChannel.java:267)
... 8 more
2019-01-14 08:16:09,935 [lettuce-nioEventLoop-4-10] [io.lettuce.core.cluster.topology.ClusterTopologyRefresh.lambda$getConnections$2(ClusterTopologyRefresh.java:228)] - [WARN] Unable to connect to 10.0.0.6:6379
java.util.concurrent.CompletionException: io.netty.channel.ConnectTimeoutException: connection timed out: /10.0.0.6:6379
at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292)
at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:308)
at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:593)
at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1977)
at io.lettuce.core.AbstractRedisClient.lambda$initializeChannelAsync0$4(AbstractRedisClient.java:329)
at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:511)
at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:504)
at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:483)
at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:424)
at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:121)
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe$1.run(AbstractNioChannel.java:269)
at io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38)
at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:127)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:466)
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:897)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
Caused by: io.netty.channel.ConnectTimeoutException: connection timed out: /10.0.0.6:6379
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe$1.run(AbstractNioChannel.java:267)
... 8 more
2019-01-14 08:16:09,935 [lettuce-nioEventLoop-4-11] [io.lettuce.core.cluster.topology.ClusterTopologyRefresh.lambda$getConnections$2(ClusterTopologyRefresh.java:228)] - [WARN] Unable to connect to 10.0.0.11:6379
java.util.concurrent.CompletionException: io.netty.channel.ConnectTimeoutException: connection timed out: /10.0.0.11:6379
at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292)
at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:308)
at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:593)
at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1977)
at io.lettuce.core.AbstractRedisClient.lambda$initializeChannelAsync0$4(AbstractRedisClient.java:329)
at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:511)
at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:504)
at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:483)
at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:424)
at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:121)
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe$1.run(AbstractNioChannel.java:269)
at io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38)
at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:127)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:466)
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:897)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
Caused by: io.netty.channel.ConnectTimeoutException: connection timed out: /10.0.0.11:6379
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe$1.run(AbstractNioChannel.java:267)
... 8 more
2019-01-14 08:16:09,935 [lettuce-nioEventLoop-4-12] [io.lettuce.core.cluster.topology.ClusterTopologyRefresh.lambda$getConnections$2(ClusterTopologyRefresh.java:228)] - [WARN] Unable to connect to 10.0.0.9:6379
java.util.concurrent.CompletionException: io.netty.channel.ConnectTimeoutException: connection timed out: /10.0.0.9:6379
at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292)
at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:308)
at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:593)
at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1977)
at io.lettuce.core.AbstractRedisClient.lambda$initializeChannelAsync0$4(AbstractRedisClient.java:329)
at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:511)
at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:504)
at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:483)
at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:424)
at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:121)
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe$1.run(AbstractNioChannel.java:269)
at io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38)
at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:127)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:466)
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:897)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
Caused by: io.netty.channel.ConnectTimeoutException: connection timed out: /10.0.0.9:6379
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe$1.run(AbstractNioChannel.java:267)
... 8 more
2019-01-14 08:16:09,935 [lettuce-nioEventLoop-4-5] [io.lettuce.core.cluster.topology.ClusterTopologyRefresh.lambda$getConnections$2(ClusterTopologyRefresh.java:228)] - [WARN] Unable to connect to 10.0.0.8:6379
java.util.concurrent.CompletionException: io.netty.channel.ConnectTimeoutException: connection timed out: /10.0.0.8:6379
at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292)
at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:308)
at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:593)
at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1977)
at io.lettuce.core.AbstractRedisClient.lambda$initializeChannelAsync0$4(AbstractRedisClient.java:329)
at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:511)
at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:504)
at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:483)
at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:424)
at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:121)
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe$1.run(AbstractNioChannel.java:269)
at io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38)
at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:127)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:466)
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:897)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
Caused by: io.netty.channel.ConnectTimeoutException: connection timed out: /10.0.0.8:6379
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe$1.run(AbstractNioChannel.java:267)
... 8 more
2019-01-14 08:16:15,509 [lettuce-nioEventLoop-4-7] [io.lettuce.core.cluster.RedisClusterClient.lambda$connect$13(RedisClusterClient.java:621)] - [WARN] io.netty.channel.ConnectTimeoutException: connection timed out: /10.0.0.7:6379
2019-01-14 08:16:25,511 [lettuce-nioEventLoop-4-8] [io.lettuce.core.cluster.RedisClusterClient.lambda$connect$13(RedisClusterClient.java:621)] - [WARN] io.netty.channel.ConnectTimeoutException: connection timed out: /10.0.0.10:6379
2019-01-14 08:16:35,518 [lettuce-nioEventLoop-4-9] [io.lettuce.core.cluster.RedisClusterClient.lambda$connect$13(RedisClusterClient.java:621)] - [WARN] io.netty.channel.ConnectTimeoutException: connection timed out: /10.0.0.11:6379
2019-01-14 08:16:45,520 [lettuce-nioEventLoop-4-10] [io.lettuce.core.cluster.RedisClusterClient.lambda$connect$13(RedisClusterClient.java:621)] - [WARN] io.netty.channel.ConnectTimeoutException: connection timed out: /10.0.0.9:6379
2019-01-14 08:16:55,527 [lettuce-nioEventLoop-4-11] [io.lettuce.core.cluster.RedisClusterClient.lambda$connect$13(RedisClusterClient.java:621)] - [WARN] io.netty.channel.ConnectTimeoutException: connection timed out: /10.0.0.8:6379
2019-01-14 08:17:05,534 [lettuce-nioEventLoop-4-12] [io.lettuce.core.cluster.RedisClusterClient.lambda$connect$13(RedisClusterClient.java:621)] - [WARN] io.netty.channel.ConnectTimeoutException: connection timed out: /10.0.0.6:6379
Jan 14, 2019 8:17:05 AM org.apache.catalina.core.StandardWrapperValve invoke
SEVERE: Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is org.springframework.data.redis.RedisConnectionFailureException: Unable to connect to Redis; nested exception is io.lettuce.core.RedisConnectionException: Unable to connect] with root cause
io.netty.channel.ConnectTimeoutException: connection timed out: /10.0.0.6:6379
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe$1.run(AbstractNioChannel.java:267)
at io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38)
at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:127)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:466)
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:897)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
2019-01-14 08:21:56,268 [HikariPool-1 housekeeper] [com.zaxxer.hikari.pool.HikariPool$HouseKeeper.run(HikariPool.java:758)] - [WARN] HikariPool-1 - Retrograde clock change detected (housekeeper delta=29s293ms), soft-evicting connections from pool.
Looks like it directly accesses the ip address in docker swarm net not the exposed public ip address. I take a long time try to find a solution but still can't fix this. Can anyone help to figure out how to solve this problem? Thank you in advance.
I am trying to change the configuration in my application to use the datasource into the Apache Tomcat server with Oracle DB 11g, but it is throwing an error about SYS.AUD$ tablespace. Follow my configuration:
$TOMCAT_HOME/conf/context.xml
<?xml version="1.0" encoding="UTF-8"?>
<Context>
<Resource auth="Container"
name="jdbc/oracleds"
type="javax.sql.DataSource"
driverClassName="oracle.jdbc.OracleDriver"
url="jdbc:oracle:thin:#HOSTNAME:PORT/SERVICENAME"
username="${db.user}"
password="${db.password}"
maxActive="20"
maxIdle="10"
maxWait="10000"
validationQuery="SELECT SYSDATE FROM DUAL"
/>
</Context>
While the server is loading the following exception is displayed into the console:
Jul 24, 2018 11:17:37 AM org.apache.tomcat.jdbc.pool.ConnectionPool init
SEVERE: Unable to create initial connections of the pool.
java.sql.SQLException: ORA-00604: error occurred at recursive SQL level 1
ORA-01653: unable to extend table SYS.AUD$ by 1024 in tablespace SYSTEM
ORA-02002: error while writing to audit trail
ORA-00604: error occurred at recursive SQL level 1
ORA-01653: unable to extend table SYS.AUD$ by 1024 in tablespace SYSTEM
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:445)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:389)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:382)
at oracle.jdbc.driver.T4CTTIfun.processError(T4CTTIfun.java:600)
at oracle.jdbc.driver.T4CTTIoauthenticate.processError(T4CTTIoauthenticate.java:445)
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:450)
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:192)
at oracle.jdbc.driver.T4CTTIoauthenticate.doOAUTH(T4CTTIoauthenticate.java:380)
at oracle.jdbc.driver.T4CTTIoauthenticate.doOAUTH(T4CTTIoauthenticate.java:760)
at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:401)
at oracle.jdbc.driver.PhysicalConnection.<init>(PhysicalConnection.java:546)
at oracle.jdbc.driver.T4CConnection.<init>(T4CConnection.java:236)
at oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:32)
at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:521)
at org.apache.tomcat.jdbc.pool.PooledConnection.connectUsingDriver(PooledConnection.java:278)
at org.apache.tomcat.jdbc.pool.PooledConnection.connect(PooledConnection.java:182)
at org.apache.tomcat.jdbc.pool.ConnectionPool.createConnection(ConnectionPool.java:730)
at org.apache.tomcat.jdbc.pool.ConnectionPool.borrowConnection(ConnectionPool.java:664)
at org.apache.tomcat.jdbc.pool.ConnectionPool.init(ConnectionPool.java:482)
at org.apache.tomcat.jdbc.pool.ConnectionPool.<init>(ConnectionPool.java:156)
at org.apache.tomcat.jdbc.pool.DataSourceProxy.pCreatePool(DataSourceProxy.java:116)
at org.apache.tomcat.jdbc.pool.DataSourceProxy.createPool(DataSourceProxy.java:103)
at org.apache.tomcat.jdbc.pool.DataSourceFactory.createDataSource(DataSourceFactory.java:560)
at org.apache.tomcat.jdbc.pool.DataSourceFactory.getObjectInstance(DataSourceFactory.java:245)
at org.apache.naming.factory.ResourceFactory.getObjectInstance(ResourceFactory.java:141)
at javax.naming.spi.NamingManager.getObjectInstance(NamingManager.java:321)
at org.apache.naming.NamingContext.lookup(NamingContext.java:848)
at org.apache.naming.NamingContext.lookup(NamingContext.java:159)
at org.apache.naming.NamingContextBindingsEnumeration.nextElementInternal(NamingContextBindingsEnumeration.java:117)
at org.apache.naming.NamingContextBindingsEnumeration.next(NamingContextBindingsEnumeration.java:71)
at org.apache.naming.NamingContextBindingsEnumeration.next(NamingContextBindingsEnumeration.java:34)
at org.apache.catalina.mbeans.GlobalResourcesLifecycleListener.createMBeans(GlobalResourcesLifecycleListener.java:138)
at org.apache.catalina.mbeans.GlobalResourcesLifecycleListener.createMBeans(GlobalResourcesLifecycleListener.java:145)
at org.apache.catalina.mbeans.GlobalResourcesLifecycleListener.createMBeans(GlobalResourcesLifecycleListener.java:110)
at org.apache.catalina.mbeans.GlobalResourcesLifecycleListener.lifecycleEvent(GlobalResourcesLifecycleListener.java:82)
at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:117)
at org.apache.catalina.util.LifecycleBase.fireLifecycleEvent(LifecycleBase.java:90)
at org.apache.catalina.util.LifecycleBase.setStateInternal(LifecycleBase.java:388)
at org.apache.catalina.util.LifecycleBase.setState(LifecycleBase.java:333)
at org.apache.catalina.core.StandardServer.startInternal(StandardServer.java:751)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145)
at org.apache.catalina.startup.Catalina.start(Catalina.java:694)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:294)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:428)
Try replacing the oracle.jdbc.OracleDriver with oracle.jdbc.driver.OracleDriver
RESOLVED I've discovered that my database FREE SPACE was very low (less than 100 MB) so I've asked the support team to fix it (increasing the memory available) and now the initial configuration has worked as expected.
Giving a look at another resolved question here I've found some interesting information about how to configure the Connection Pool in Tomcat server:
org.apache.tomcat.dbcp.dbcp.SQLNestedException: Cannot create PoolableConnectionFactory ( 'v$session.osuser' , value is 'x')
I'm encountering a ORA-12519, TNS:no appropriate service handler found error when attempting to integrate Tomcat's JDBC Pool into my web application, using Oracle.
I typically see this error appear intermittently, after several minutes of running integration tests against the application.
The configuration I have is:
oracle driver and tomcat-dbcp jars in the tomcat/lib directory
two webapps, both using the same Resources. Configuration done in Spring:
<jee:jndi-lookup id="webDS" jndi-name="jdbc/web"
expected-type="javax.sql.DataSource" />
DataSource Resources defined in conf/context.xml, as follows:
<Resource name="jdbc/web" auth="Container"
type="javax.sql.DataSource"
factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
testWhileIdle="true"
testOnBorrow="true"
testOnReturn="false"
validationQuery="SELECT 1"
timeBetweenEvictionRunsMillis="30000"
maxActive="20"
maxIdle="10"
minIdle="5"
removeAbandonedTimeout="60"
removeAbandoned="false"
logAbandoned="true"
minEvictableIdleTimeMillis="30000"
jdbcInterceptors="org.apache.tomcat.jdbc.pool.interceptor.ConnectionState;org.apache.tomcat.jdbc.pool.interceptor.StatementFinalizer"
username="${database.user}"
password="${database.password}"
driverClassName="${database.driver}"
url="${database.url}" />
I've tried various suggestions made elsewhere on SO such as increasing or decreasing the maxActive size, but am having no luck thus far. I was previously using c3p0 to manage the pool of connections. With the switch, I'm wondering if there is some additional configuration I need to do that pertains to the closing of connections, because it seems like I'm leaking them.
The stack trace:
Caused by: org.hibernate.exception.GenericJDBCException: Could not open connection
at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:54)
at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:124)
at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:109)
at org.hibernate.engine.jdbc.internal.LogicalConnectionImpl.obtainConnection(LogicalConnectionImpl.java:221)
at org.hibernate.engine.jdbc.internal.LogicalConnectionImpl.getConnection(LogicalConnectionImpl.java:157)
at org.hibernate.engine.jdbc.internal.StatementPreparerImpl.connection(StatementPreparerImpl.java:56)
at org.hibernate.engine.jdbc.internal.StatementPreparerImpl$5.doPrepare(StatementPreparerImpl.java:159)
at org.hibernate.engine.jdbc.internal.StatementPreparerImpl$StatementPreparationTemplate.prepareStatement(StatementPreparerImpl.java:183)
at org.hibernate.engine.jdbc.internal.StatementPreparerImpl.prepareQueryStatement(StatementPreparerImpl.java:157)
at org.hibernate.loader.Loader.prepareQueryStatement(Loader.java:1881)
at org.hibernate.loader.Loader.executeQueryStatement(Loader.java:1858)
at org.hibernate.loader.Loader.executeQueryStatement(Loader.java:1838)
at org.hibernate.loader.Loader.doQuery(Loader.java:906)
at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:348)
at org.hibernate.loader.Loader.doList(Loader.java:2550)
at org.hibernate.loader.Loader.doList(Loader.java:2536)
at org.hibernate.loader.Loader.listIgnoreQueryCache(Loader.java:2366)
at org.hibernate.loader.Loader.list(Loader.java:2361)
at org.hibernate.loader.hql.QueryLoader.list(QueryLoader.java:495)
at org.hibernate.hql.internal.ast.QueryTranslatorImpl.list(QueryTranslatorImpl.java:357)
at org.hibernate.engine.query.spi.HQLQueryPlan.performList(HQLQueryPlan.java:198)
at org.hibernate.internal.SessionImpl.list(SessionImpl.java:1230)
at org.hibernate.internal.QueryImpl.list(QueryImpl.java:101)
at org.hibernate.ejb.QueryImpl.getResultList(QueryImpl.java:268)
... 112 more
Caused by: java.sql.SQLException: Listener refused the connection with the following error:
ORA-12519, TNS:no appropriate service handler found
at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:489)
at oracle.jdbc.driver.PhysicalConnection.<init>(PhysicalConnection.java:553)
at oracle.jdbc.driver.T4CConnection.<init>(T4CConnection.java:254)
at oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:32)
at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:528)
at org.apache.tomcat.jdbc.pool.PooledConnection.connectUsingDriver(PooledConnection.java:278)
at org.apache.tomcat.jdbc.pool.PooledConnection.connect(PooledConnection.java:182)
at org.apache.tomcat.jdbc.pool.PooledConnection.reconnect(PooledConnection.java:315)
at org.apache.tomcat.jdbc.pool.ConnectionPool.borrowConnection(ConnectionPool.java:803)
at org.apache.tomcat.jdbc.pool.ConnectionPool.borrowConnection(ConnectionPool.java:628)
at org.apache.tomcat.jdbc.pool.ConnectionPool.getConnection(ConnectionPool.java:187)
at org.apache.tomcat.jdbc.pool.DataSourceProxy.getConnection(DataSourceProxy.java:128)
at org.hibernate.ejb.connection.InjectedDataSourceConnectionProvider.getConnection(InjectedDataSourceConnectionProvider.java:70)
at org.hibernate.internal.AbstractSessionImpl$NonContextualJdbcConnectionAccess.obtainConnection(AbstractSessionImpl.java:301)
at org.hibernate.engine.jdbc.internal.LogicalConnectionImpl.obtainConnection(LogicalConnectionImpl.java:214)
... 132 more
Caused by: oracle.net.ns.NetException: Listener refused the connection with the following error:
ORA-12519, TNS:no appropriate service handler found
at oracle.net.ns.NSProtocol.connect(NSProtocol.java:399)
at oracle.jdbc.driver.T4CConnection.connect(T4CConnection.java:1140)
at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:340)
... 146 more
Your validation query (SELECT 1) will not work in Oracle. It has to be select 1 from dual.
Seems that when using a wrong validation query the application is not able to identify healthy connections and and mark all of them as invalid.
We have several jobs running in springbatch. We restarted the server this morning and afterwards we started having issues with two of the jobs. The BATCH_STEP_EXECUTION table shows the following exit message for each failed execution:
org.springframework.transaction.TransactionSystemException: Could not roll back JDBC transaction; nested exception is com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Connection.close() has already been called. Invalid operation in this state.
at org.springframework.jdbc.datasource.DataSourceTransactionManager.doRollback(DataSourceTransactionManager.java:286)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processRollback(AbstractPlatformTransactionManager.java:846)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.rollback(AbstractPlatformTransactionManager.java:823)
at org.springframework.transaction.support.TransactionTemplate.rollbackOnException(TransactionTemplate.java:162)
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:135)
at org.springframework.batch.core.step.tasklet.TaskletStep$2.doInChunkContext(TaskletStep.java:267)
at org.springframework.batch.core.scope.context.StepContextRepeatCallback.doInIteration(StepContextRepeatCallback.java:77)
at org.springframework.batch.repeat.support.RepeatTemplate.getNextResult(RepeatTemplate.java:368)
at org.springframework.batch.repeat.support.RepeatTemplate.executeInternal(RepeatTemplate.java:215)
at org.springframework.batch.repeat.support.RepeatTemplate.iterate(RepeatTemplate.java:144)
at org.springframework.batch.core.step.tasklet.TaskletStep.doExecute(TaskletStep.java:253)
at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:195)
at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:141)
at org.springframework.batch.core.job.flow.JobFlowExecutor.executeStep(JobFlowExecutor.java:64)
at org.springframework.batch.core.job.flow.support.state.StepState.handle(StepState.java:60)
at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:151)
at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:130)
at org.springframework.batch.core.job.flow.FlowJob.doExecute(FlowJob.java:135)
at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:301)
at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:134)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.ja
This appears to be in relation to the mysql connection the application is making, however I'm not sure what could have caused this as the application was functioning fine before the reboot. Has anyone stumbled across this before?
EDIT
(caused by message that appears in the logs)
Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Connection.close() has already been called. Invalid operation in this state.
Datasource
javax.sql.DataSource
Connection Pool
Configured in context.xml file with the following format:
<?xml version="1.0" encoding="UTF-8"?>
<Context docBase="webapps/appname" mapperContextRootRedirectEnabled="true" mapperDirectoryRedirectEnabled="true" path="/appname" reloadable="false" >
<Resource name="jdbc/springbatchmeta"
auth="Container"
type="javax.sql.DataSource"
username="${jobmeta.username}"
password="${jobmeta.password}"
driverClassName="${mysql.driverClass}"
url="${jobmeta.url}"
initialSize="2"
maxTotal="20"
maxIdle="10"
minIdle="2"/>
<Resource name="jdbc/firstjob"
auth="Container"
type="javax.sql.DataSource"
username="${firstjob.username}"
password="${firstjob.password}"
driverClassName="${mysql.driverClass}"
url="${firstjob.url}"
initialSize="2"
maxTotal="20"
maxIdle="10"
minIdle="2"/>
<Resource name="jdbc/secondjob"
auth="Container"
type="javax.sql.DataSource"
username="${secondjob.username}"
password="${secondjob.password}"
driverClassName="${secondjob.driverClass}"
url="${secondjob.url}"
initialSize="2"
maxTotal="20"
maxIdle="10"
minIdle="2"/>
</Context>
This issue was caused by the wait-timeout option in mysql being set to 60 seconds, as most of the jobs we are running take over a minute to complete.
I am just configuring tomcat for JNDI using the externalized. I have done some thing wrong, but no clue.
In Context.xml
<Resource name="jdbc/AppDB" auth="Container" type="javax.sql.DataSource"
driverClassName="org.postgresql.Driver"
factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
url="${app.env.database.url}"
username="${app.env.database.user}"
password="${app.env.database.password}"
maxActive="100"
maxIdle="50"
minIdle="10"
suspectTimeout="60"
timeBetweenEvictionRunsMillis="30000"
minEvictableIdleTimeMillis="60000"
validationQuery="select 1"
validationInterval="30000"
testOnBorrow="true"
removeAbandoned="true"
removeAbandonedTimeout="60"
abandonWhenPercentageFull="10"
maxWait="10000"
jdbcInterceptors="ResetAbandonedTimer;StatementFinalizer"
/>
While running the Tomcat , I am setting this
-DappProperties=c:/user/abc/app.properties
On Starting the server, I am getting below error.
Caused by: javax.naming.NamingException: Driver:org.postgresql.Driver#d8cf794 returned null for URL:${app.env.database.url}
at org.apache.naming.NamingContext.lookup(NamingContext.java:858)
at org.apache.naming.NamingContext.lookup(NamingContext.java:152)
at org.apache.naming.NamingContext.lookup(NamingContext.java:829)
at org.apache.naming.NamingContext.lookup(NamingContext.java:152)
at org.apache.naming.NamingContext.lookup(NamingContext.java:829)
at org.apache.naming.NamingContext.lookup(NamingContext.java:152)
at org.apache.naming.NamingContext.lookup(NamingContext.java:829)
at org.apache.naming.NamingContext.lookup(NamingContext.java:166)
at org.apache.naming.SelectorContext.lookup(SelectorContext.java:157)
at javax.naming.InitialContext.lookup(InitialContext.java:411)
at org.springframework.jndi.JndiTemplate$1.doInContext(JndiTemplate.java:155)
at org.springframework.jndi.JndiTemplate.execute(JndiTemplate.java:87)
at org.springframework.jndi.JndiTemplate.lookup(JndiTemplate.java:152)
at org.springframework.jndi.JndiTemplate.lookup(JndiTemplate.java:179)
at org.springframework.jndi.JndiLocatorSupport.lookup(JndiLocatorSupport.java:95)
at org.springframework.jndi.JndiObjectLocator.lookup(JndiObjectLocator.java:106)
at org.springframework.jndi.JndiObjectFactoryBean.lookupWithFallback(JndiObjectFactoryBean.java:231)
at org.springframework.jndi.JndiObjectFactoryBean.afterPropertiesSet(JndiObjectFactoryBean.java:217)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1625)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1562)
... 31 more
Can any one please help.
Please note : application is working good if I change ${app.env.} with actual values.
Thank you for your help
Are you using Maven? You need to use resource filtering plugin so maven will transform your property file values into actual values. Something like:
<build>
...
<resources>
<resource>
<directory>src/main/resources</directory>
</resource>
...
</resources>
...
Where the directory contains your xml file should work