Tomcat DBCP Running out of connections - java

I'm encountering a ORA-12519, TNS:no appropriate service handler found error when attempting to integrate Tomcat's JDBC Pool into my web application, using Oracle.
I typically see this error appear intermittently, after several minutes of running integration tests against the application.
The configuration I have is:
oracle driver and tomcat-dbcp jars in the tomcat/lib directory
two webapps, both using the same Resources. Configuration done in Spring:
<jee:jndi-lookup id="webDS" jndi-name="jdbc/web"
expected-type="javax.sql.DataSource" />
DataSource Resources defined in conf/context.xml, as follows:
<Resource name="jdbc/web" auth="Container"
type="javax.sql.DataSource"
factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
testWhileIdle="true"
testOnBorrow="true"
testOnReturn="false"
validationQuery="SELECT 1"
timeBetweenEvictionRunsMillis="30000"
maxActive="20"
maxIdle="10"
minIdle="5"
removeAbandonedTimeout="60"
removeAbandoned="false"
logAbandoned="true"
minEvictableIdleTimeMillis="30000"
jdbcInterceptors="org.apache.tomcat.jdbc.pool.interceptor.ConnectionState;org.apache.tomcat.jdbc.pool.interceptor.StatementFinalizer"
username="${database.user}"
password="${database.password}"
driverClassName="${database.driver}"
url="${database.url}" />
I've tried various suggestions made elsewhere on SO such as increasing or decreasing the maxActive size, but am having no luck thus far. I was previously using c3p0 to manage the pool of connections. With the switch, I'm wondering if there is some additional configuration I need to do that pertains to the closing of connections, because it seems like I'm leaking them.
The stack trace:
Caused by: org.hibernate.exception.GenericJDBCException: Could not open connection
at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:54)
at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:124)
at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:109)
at org.hibernate.engine.jdbc.internal.LogicalConnectionImpl.obtainConnection(LogicalConnectionImpl.java:221)
at org.hibernate.engine.jdbc.internal.LogicalConnectionImpl.getConnection(LogicalConnectionImpl.java:157)
at org.hibernate.engine.jdbc.internal.StatementPreparerImpl.connection(StatementPreparerImpl.java:56)
at org.hibernate.engine.jdbc.internal.StatementPreparerImpl$5.doPrepare(StatementPreparerImpl.java:159)
at org.hibernate.engine.jdbc.internal.StatementPreparerImpl$StatementPreparationTemplate.prepareStatement(StatementPreparerImpl.java:183)
at org.hibernate.engine.jdbc.internal.StatementPreparerImpl.prepareQueryStatement(StatementPreparerImpl.java:157)
at org.hibernate.loader.Loader.prepareQueryStatement(Loader.java:1881)
at org.hibernate.loader.Loader.executeQueryStatement(Loader.java:1858)
at org.hibernate.loader.Loader.executeQueryStatement(Loader.java:1838)
at org.hibernate.loader.Loader.doQuery(Loader.java:906)
at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:348)
at org.hibernate.loader.Loader.doList(Loader.java:2550)
at org.hibernate.loader.Loader.doList(Loader.java:2536)
at org.hibernate.loader.Loader.listIgnoreQueryCache(Loader.java:2366)
at org.hibernate.loader.Loader.list(Loader.java:2361)
at org.hibernate.loader.hql.QueryLoader.list(QueryLoader.java:495)
at org.hibernate.hql.internal.ast.QueryTranslatorImpl.list(QueryTranslatorImpl.java:357)
at org.hibernate.engine.query.spi.HQLQueryPlan.performList(HQLQueryPlan.java:198)
at org.hibernate.internal.SessionImpl.list(SessionImpl.java:1230)
at org.hibernate.internal.QueryImpl.list(QueryImpl.java:101)
at org.hibernate.ejb.QueryImpl.getResultList(QueryImpl.java:268)
... 112 more
Caused by: java.sql.SQLException: Listener refused the connection with the following error:
ORA-12519, TNS:no appropriate service handler found
at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:489)
at oracle.jdbc.driver.PhysicalConnection.<init>(PhysicalConnection.java:553)
at oracle.jdbc.driver.T4CConnection.<init>(T4CConnection.java:254)
at oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:32)
at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:528)
at org.apache.tomcat.jdbc.pool.PooledConnection.connectUsingDriver(PooledConnection.java:278)
at org.apache.tomcat.jdbc.pool.PooledConnection.connect(PooledConnection.java:182)
at org.apache.tomcat.jdbc.pool.PooledConnection.reconnect(PooledConnection.java:315)
at org.apache.tomcat.jdbc.pool.ConnectionPool.borrowConnection(ConnectionPool.java:803)
at org.apache.tomcat.jdbc.pool.ConnectionPool.borrowConnection(ConnectionPool.java:628)
at org.apache.tomcat.jdbc.pool.ConnectionPool.getConnection(ConnectionPool.java:187)
at org.apache.tomcat.jdbc.pool.DataSourceProxy.getConnection(DataSourceProxy.java:128)
at org.hibernate.ejb.connection.InjectedDataSourceConnectionProvider.getConnection(InjectedDataSourceConnectionProvider.java:70)
at org.hibernate.internal.AbstractSessionImpl$NonContextualJdbcConnectionAccess.obtainConnection(AbstractSessionImpl.java:301)
at org.hibernate.engine.jdbc.internal.LogicalConnectionImpl.obtainConnection(LogicalConnectionImpl.java:214)
... 132 more
Caused by: oracle.net.ns.NetException: Listener refused the connection with the following error:
ORA-12519, TNS:no appropriate service handler found
at oracle.net.ns.NSProtocol.connect(NSProtocol.java:399)
at oracle.jdbc.driver.T4CConnection.connect(T4CConnection.java:1140)
at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:340)
... 146 more

Your validation query (SELECT 1) will not work in Oracle. It has to be select 1 from dual.
Seems that when using a wrong validation query the application is not able to identify healthy connections and and mark all of them as invalid.

Related

Error when configuring Datasource Oracle 11g into Apache Tomcat 7 (7.0.82)

I am trying to change the configuration in my application to use the datasource into the Apache Tomcat server with Oracle DB 11g, but it is throwing an error about SYS.AUD$ tablespace. Follow my configuration:
$TOMCAT_HOME/conf/context.xml
<?xml version="1.0" encoding="UTF-8"?>
<Context>
<Resource auth="Container"
name="jdbc/oracleds"
type="javax.sql.DataSource"
driverClassName="oracle.jdbc.OracleDriver"
url="jdbc:oracle:thin:#HOSTNAME:PORT/SERVICENAME"
username="${db.user}"
password="${db.password}"
maxActive="20"
maxIdle="10"
maxWait="10000"
validationQuery="SELECT SYSDATE FROM DUAL"
/>
</Context>
While the server is loading the following exception is displayed into the console:
Jul 24, 2018 11:17:37 AM org.apache.tomcat.jdbc.pool.ConnectionPool init
SEVERE: Unable to create initial connections of the pool.
java.sql.SQLException: ORA-00604: error occurred at recursive SQL level 1
ORA-01653: unable to extend table SYS.AUD$ by 1024 in tablespace SYSTEM
ORA-02002: error while writing to audit trail
ORA-00604: error occurred at recursive SQL level 1
ORA-01653: unable to extend table SYS.AUD$ by 1024 in tablespace SYSTEM
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:445)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:389)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:382)
at oracle.jdbc.driver.T4CTTIfun.processError(T4CTTIfun.java:600)
at oracle.jdbc.driver.T4CTTIoauthenticate.processError(T4CTTIoauthenticate.java:445)
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:450)
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:192)
at oracle.jdbc.driver.T4CTTIoauthenticate.doOAUTH(T4CTTIoauthenticate.java:380)
at oracle.jdbc.driver.T4CTTIoauthenticate.doOAUTH(T4CTTIoauthenticate.java:760)
at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:401)
at oracle.jdbc.driver.PhysicalConnection.<init>(PhysicalConnection.java:546)
at oracle.jdbc.driver.T4CConnection.<init>(T4CConnection.java:236)
at oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:32)
at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:521)
at org.apache.tomcat.jdbc.pool.PooledConnection.connectUsingDriver(PooledConnection.java:278)
at org.apache.tomcat.jdbc.pool.PooledConnection.connect(PooledConnection.java:182)
at org.apache.tomcat.jdbc.pool.ConnectionPool.createConnection(ConnectionPool.java:730)
at org.apache.tomcat.jdbc.pool.ConnectionPool.borrowConnection(ConnectionPool.java:664)
at org.apache.tomcat.jdbc.pool.ConnectionPool.init(ConnectionPool.java:482)
at org.apache.tomcat.jdbc.pool.ConnectionPool.<init>(ConnectionPool.java:156)
at org.apache.tomcat.jdbc.pool.DataSourceProxy.pCreatePool(DataSourceProxy.java:116)
at org.apache.tomcat.jdbc.pool.DataSourceProxy.createPool(DataSourceProxy.java:103)
at org.apache.tomcat.jdbc.pool.DataSourceFactory.createDataSource(DataSourceFactory.java:560)
at org.apache.tomcat.jdbc.pool.DataSourceFactory.getObjectInstance(DataSourceFactory.java:245)
at org.apache.naming.factory.ResourceFactory.getObjectInstance(ResourceFactory.java:141)
at javax.naming.spi.NamingManager.getObjectInstance(NamingManager.java:321)
at org.apache.naming.NamingContext.lookup(NamingContext.java:848)
at org.apache.naming.NamingContext.lookup(NamingContext.java:159)
at org.apache.naming.NamingContextBindingsEnumeration.nextElementInternal(NamingContextBindingsEnumeration.java:117)
at org.apache.naming.NamingContextBindingsEnumeration.next(NamingContextBindingsEnumeration.java:71)
at org.apache.naming.NamingContextBindingsEnumeration.next(NamingContextBindingsEnumeration.java:34)
at org.apache.catalina.mbeans.GlobalResourcesLifecycleListener.createMBeans(GlobalResourcesLifecycleListener.java:138)
at org.apache.catalina.mbeans.GlobalResourcesLifecycleListener.createMBeans(GlobalResourcesLifecycleListener.java:145)
at org.apache.catalina.mbeans.GlobalResourcesLifecycleListener.createMBeans(GlobalResourcesLifecycleListener.java:110)
at org.apache.catalina.mbeans.GlobalResourcesLifecycleListener.lifecycleEvent(GlobalResourcesLifecycleListener.java:82)
at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:117)
at org.apache.catalina.util.LifecycleBase.fireLifecycleEvent(LifecycleBase.java:90)
at org.apache.catalina.util.LifecycleBase.setStateInternal(LifecycleBase.java:388)
at org.apache.catalina.util.LifecycleBase.setState(LifecycleBase.java:333)
at org.apache.catalina.core.StandardServer.startInternal(StandardServer.java:751)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145)
at org.apache.catalina.startup.Catalina.start(Catalina.java:694)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:294)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:428)
Try replacing the oracle.jdbc.OracleDriver with oracle.jdbc.driver.OracleDriver
RESOLVED I've discovered that my database FREE SPACE was very low (less than 100 MB) so I've asked the support team to fix it (increasing the memory available) and now the initial configuration has worked as expected.
Giving a look at another resolved question here I've found some interesting information about how to configure the Connection Pool in Tomcat server:
org.apache.tomcat.dbcp.dbcp.SQLNestedException: Cannot create PoolableConnectionFactory ( 'v$session.osuser' , value is 'x')

Java Spring Tomcat: Custom DataSource Factory Being Ignored

I am working on a project that requires a database password to be resolved at run time. I am effectively doing this locally by specifying a DataSourceFactory within the Tomcat configuration. This process prevents any credential being stored in a plain text config file and gives us the custom control to resolve passwords at runtime. I have great success with this locally, but when I move my solution to a real/non-embedded Tomcat instance on AWS, I see this error:
Could not invoke the static newInstance method on the named factory
class "org.apache.openjpa.jdbc.kernel.JDBCBrokerFactory".
To start, there are a few fishy details around this error message. Number one (as you'll see below), my factory class is being ignored in the tomcat config file despite the other credentials being valid (e.g. username, password, etc).
The code for my DataSourceFactory needs to reference a class to resolve a value.
Here is the xml configuration that I have for tomcat:
<?xml version='1.0' encoding='UTF-8'?>
<Context useHttpOnly="false">
<Resource auth="Container" name="mail/Session" type="javax.mail.Session"/>
<Resource name="jdbc/jndi"
auth="Container"
scope="Shareable"
type="javax.sql.DataSource"
useLocalSessionState="true"
cacheServerConfiguration="true"
useServerPrepStmts="true"
cachePrepStmts="true"
cacheCallableStmts="true"
elideSetAutoCommits="true"
alwaysSendSetIsolation="false"
enableQueryTimeouts="false"
prepStmtCacheSize="250"
prepStmtCacheSqlLimit="2048"
maxActive="100"
maxIdle="20"
maxWait="10000"
removeAbandoned="true"
driverClassName="com.mysql.jdbc.Driver"
url="${myURL}"
username="${myUsername}"
password="${myPasswordToResolve}"
factory="my.class.CustomDatabaseConfigurationFactory"
testOnBorrow="true"
testWhileIdle="true"
timeBetweenEvictionRunsMillis="20000"
poolPreparedStatements="true"
maxOpenPreparedStatements="50"
validationQuery="select 1"
validationInterval="60000"
defaultTransactionIsolation="READ_COMMITTED"
/>
<Valve className="org.apache.catalina.valves.CometConnectionManagerValve" />
<Loader loaderClass="org.apache.catalina.loader.ParallelWebappClassLoader" />
<Resource auth="Container"
name="jms/JMSConnectionFactory"
type="org.apache.activemq.pool.PooledConnectionFactory"
description="JMS Connection Factory"
factory="org.apache.activemq.jndi.JNDIReferenceFactory"
brokerURL="my.URL"
maxConnections="100"
maximumActiveSessionPerConnection="50"
expiryTimeout="10000"
/>
</Context>
I will not add the code of my my.class.CustomDatabaseConfigurationFactory because I know it works locally. For some reason, when I launch this using a remote configuration and setup, Tomcat completely ignores the custom factory I have put in place for the JNDI resource. Is there anything that I have to add to force Tomcat to look for my custom factory and not ignore it? I have almost the same configuration working on an embedded tomcat instance locally. On top of that, I don't see any clues in the logs that are related to a misconfigured file.
Here is the complete stack trace:
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'entityManagerFactory' defined in class path resource [spring/dataaccess/openjpa/openjpa.xml]: Invocation of ini\
t method failed; nested exception is <openjpa-2.4.0-ep2.1-runknown fatal user error> org.apache.openjpa.persistence.ArgumentException: Could not invoke the static newInstance method on the named fact\
ory class "org.apache.openjpa.jdbc.kernel.JDBCBrokerFactory".
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1514)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:521)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:458)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:293)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:223)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:290)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:191)
at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1119)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:924)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:479)
at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:410)
at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:306)
at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:112)
at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:5118)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5634)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145)
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:899)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:875)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:652)
at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:679)
at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1966)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java)
at java.lang.Thread.run(Thread.java:748)
Realistically, this problem should never occur when proper passwords are being resolved by a custom data factory.
A few things for anyone else who is having this issue:
On top of that, I don't see any clues in the logs that are related to
a misconfigured file.
This was a dead giveaway that something was not being loaded or loggers were not being set up properly. After I realized that log4j wasn't configured properly, I was able to see that my factory was in fact being loaded. Later on, after seeing the results, I was able to see that the passwords being set for the database were incorrect.
Although the log error was too high level and without enough detailed information, this issue was fixable with basic debugging methods and can be avoided with attention to logging.

Broken Pipe in Hibernate application

I have an application in Tomcat 8 Engine which have both: JDBC and Hibernate connections to two different databases. The Hibernate connection fails after 3-4 hours of user inactivity
also I have c3p0 properties in hibernate.cfg.xml
hibernate.c3p0.min_size 1
hibernate.c3p0.idle_test_period 100
the exception stacktrace:
STACKTRACE:
com.mysql.jdbc.CommunicationsException: Communications link failure due to underlying exception:
** BEGIN NESTED EXCEPTION **
java.net.SocketException
MESSAGE: Broken pipe
STACKTRACE:
java.net.SocketException: Broken pipe
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:109)
at java.net.SocketOutputStream.write(SocketOutputStream.java:153)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
at com.mysql.jdbc.MysqlIO.send(MysqlIO.java:2637)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1554)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:1665)
at com.mysql.jdbc.Connection.execSQL(Connection.java:3176)
at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1153)
at com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:1266)
at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeQuery(NewProxyPreparedStatement.java:116)
at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.extract(ResultSetReturnImpl.java:80)
at org.hibernate.loader.Loader.getResultSet(Loader.java:2065)
at org.hibernate.loader.Loader.executeQueryStatement(Loader.java:1862)
at org.hibernate.loader.Loader.executeQueryStatement(Loader.java:1838)
at org.hibernate.loader.Loader.doQuery(Loader.java:909)
at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:354)
at org.hibernate.loader.Loader.doList(Loader.java:2551)
at org.hibernate.loader.Loader.doList(Loader.java:2537)
at org.hibernate.loader.Loader.listIgnoreQueryCache(Loader.java:2367)
at org.hibernate.loader.Loader.list(Loader.java:2362)
at org.hibernate.loader.hql.QueryLoader.list(QueryLoader.java:496)
at org.hibernate.hql.internal.ast.QueryTranslatorImpl.list(QueryTranslatorImpl.java:387)
at org.hibernate.engine.query.spi.HQLQueryPlan.performList(HQLQueryPlan.java:229)
at org.hibernate.internal.SessionImpl.list(SessionImpl.java:1260)
at org.hibernate.internal.QueryImpl.list(QueryImpl.java:103)
at DAO.CarsDAOimpl.getData(CarsDAOimpl.java:121)
does anybody has an idea what is going on and how to fix that?
UPD: here is about memory state:
Runtime runtime = Runtime.getRuntime();
long freeMemory = runtime.freeMemory();
...
free memory: 183787192
Yes, I have had the same problem. Try to add a validationQuery to your DataSource.
Here is how I have defined mine:
<Resource auth="Container"
type="javax.sql.DataSource"
name="jdbc/mydb"
driverClassName="com.mysql.jdbc.Driver"
url="jdbc:mysql://localhost/mydb"
maxActive="10"
maxIdle="5"
validationQuery="SELECT 1"
testOnBorrow="true"
testWhileIdle="true"
timeBetweenEvictionRunsMillis="10000"
minEvictableIdleTimeMillis="60000"
username="..." password="..."/>

Springbatch failing | Could not roll back JDBC transaction

We have several jobs running in springbatch. We restarted the server this morning and afterwards we started having issues with two of the jobs. The BATCH_STEP_EXECUTION table shows the following exit message for each failed execution:
org.springframework.transaction.TransactionSystemException: Could not roll back JDBC transaction; nested exception is com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Connection.close() has already been called. Invalid operation in this state.
at org.springframework.jdbc.datasource.DataSourceTransactionManager.doRollback(DataSourceTransactionManager.java:286)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processRollback(AbstractPlatformTransactionManager.java:846)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.rollback(AbstractPlatformTransactionManager.java:823)
at org.springframework.transaction.support.TransactionTemplate.rollbackOnException(TransactionTemplate.java:162)
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:135)
at org.springframework.batch.core.step.tasklet.TaskletStep$2.doInChunkContext(TaskletStep.java:267)
at org.springframework.batch.core.scope.context.StepContextRepeatCallback.doInIteration(StepContextRepeatCallback.java:77)
at org.springframework.batch.repeat.support.RepeatTemplate.getNextResult(RepeatTemplate.java:368)
at org.springframework.batch.repeat.support.RepeatTemplate.executeInternal(RepeatTemplate.java:215)
at org.springframework.batch.repeat.support.RepeatTemplate.iterate(RepeatTemplate.java:144)
at org.springframework.batch.core.step.tasklet.TaskletStep.doExecute(TaskletStep.java:253)
at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:195)
at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:141)
at org.springframework.batch.core.job.flow.JobFlowExecutor.executeStep(JobFlowExecutor.java:64)
at org.springframework.batch.core.job.flow.support.state.StepState.handle(StepState.java:60)
at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:151)
at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:130)
at org.springframework.batch.core.job.flow.FlowJob.doExecute(FlowJob.java:135)
at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:301)
at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:134)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.ja
This appears to be in relation to the mysql connection the application is making, however I'm not sure what could have caused this as the application was functioning fine before the reboot. Has anyone stumbled across this before?
EDIT
(caused by message that appears in the logs)
Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Connection.close() has already been called. Invalid operation in this state.
Datasource
javax.sql.DataSource
Connection Pool
Configured in context.xml file with the following format:
<?xml version="1.0" encoding="UTF-8"?>
<Context docBase="webapps/appname" mapperContextRootRedirectEnabled="true" mapperDirectoryRedirectEnabled="true" path="/appname" reloadable="false" >
<Resource name="jdbc/springbatchmeta"
auth="Container"
type="javax.sql.DataSource"
username="${jobmeta.username}"
password="${jobmeta.password}"
driverClassName="${mysql.driverClass}"
url="${jobmeta.url}"
initialSize="2"
maxTotal="20"
maxIdle="10"
minIdle="2"/>
<Resource name="jdbc/firstjob"
auth="Container"
type="javax.sql.DataSource"
username="${firstjob.username}"
password="${firstjob.password}"
driverClassName="${mysql.driverClass}"
url="${firstjob.url}"
initialSize="2"
maxTotal="20"
maxIdle="10"
minIdle="2"/>
<Resource name="jdbc/secondjob"
auth="Container"
type="javax.sql.DataSource"
username="${secondjob.username}"
password="${secondjob.password}"
driverClassName="${secondjob.driverClass}"
url="${secondjob.url}"
initialSize="2"
maxTotal="20"
maxIdle="10"
minIdle="2"/>
</Context>
This issue was caused by the wait-timeout option in mysql being set to 60 seconds, as most of the jobs we are running take over a minute to complete.

logback and mysql Connections could not be acquired

I am using logback and mysql like this:
<appender name="db" class="ch.qos.logback.classic.db.DBAppender">
<connectionSource class="ch.qos.logback.core.db.DataSourceConnectionSource">
<dataSource class="com.mchange.v2.c3p0.ComboPooledDataSource">
<driverClass>com.mysql.jdbc.Driver</driverClass>
<jdbcUrl>${jdbc.url}</jdbcUrl>
<user>${jdbc.username}</user>
<password>${jdbc.password}</password>
</dataSource>
</connectionSource>
</appender>
However I get:
11:51:45,851 |-WARN in ch.qos.logback.core.db.DataSourceConnectionSource#4c1be833 - Could not get a connection to discover the dialect to use. java.sql.SQLException: Connections could not be acquired from the underlying database!
at java.sql.SQLException: Connections could not be acquired from the underlying database!
at at com.mchange.v2.sql.SqlUtils.toSQLException(SqlUtils.java:106)
at at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool.checkoutPooledConnection(C3P0PooledConnectionPool.java:529)
at at com.mchange.v2.c3p0.impl.AbstractPoolBackedDataSource.getConnection(AbstractPoolBackedDataSource.java:128)
at at ch.qos.logback.core.db.DataSourceConnectionSource.getConnection(DataSourceConnectionSource.java:72)
at at ch.qos.logback.core.db.DataSourceConnectionSource.start(DataSourceConnectionSource.java:46)
at at ch.qos.logback.core.joran.action.NestedComplexPropertyIA.end(NestedComplexPropertyIA.java:167)
at at ch.qos.logback.core.joran.spi.Interpreter.callEndAction(Interpreter.java:318)
at at ch.qos.logback.core.joran.spi.Interpreter.endElement(Interpreter.java:197)
at at ch.qos.logback.core.joran.spi.Interpreter.endElement(Interpreter.java:183)
at at ch.qos.logback.core.joran.spi.EventPlayer.play(EventPlayer.java:62)
at at ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:147)
at at ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:133)
at at ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:96)
at at ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:55)
at at ch.qos.logback.classic.util.ContextInitializer.configureByResource(ContextInitializer.java:75)
at at ch.qos.logback.classic.util.ContextInitializer.autoConfig(ContextInitializer.java:148)
at at org.slf4j.impl.StaticLoggerBinder.init(StaticLoggerBinder.java:84)
at at org.slf4j.impl.StaticLoggerBinder.<clinit>(StaticLoggerBinder.java:54)
at at org.slf4j.LoggerFactory.getSingleton(LoggerFactory.java:223)
at at org.slf4j.LoggerFactory.bind(LoggerFactory.java:120)
at at org.slf4j.LoggerFactory.performInitialization(LoggerFactory.java:111)
at at org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:269)
at at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:242)
at at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:255)
at at com.grroo.manager.impl.AmazonEmailManagerImpl.<init>(AmazonEmailManagerImpl.java:42)
at at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:126)
at at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:74)
at at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateBean(AbstractAutowireCapableBeanFactory.java:958)
at at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:911)
at at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:485)
at at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456)
at at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:291)
at at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
at at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:288)
at at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:190)
at at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:580)
at at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:895)
at at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:425)
at at org.springframework.web.context.ContextLoader.createWebApplicationContext(ContextLoader.java:276)
at at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:197)
at at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:47)
at at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4723)
at at org.apache.catalina.core.StandardContext$1.call(StandardContext.java:5226)
at at org.apache.catalina.core.StandardContext$1.call(StandardContext.java:5221)
at at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at at java.lang.Thread.run(Thread.java:662)
Caused by: com.mchange.v2.resourcepool.CannotAcquireResourceException: A ResourcePool could not acquire a resource from its primary factory or source.
at at com.mchange.v2.resourcepool.BasicResourcePool.awaitAvailable(BasicResourcePool.java:1319)
at at com.mchange.v2.resourcepool.BasicResourcePool.prelimCheckoutResource(BasicResourcePool.java:557)
at at com.mchange.v2.resourcepool.BasicResourcePool.checkoutResource(BasicResourcePool.java:477)
at at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool.checkoutPooledConnection(C3P0PooledConnectionPool.java:525)
at ... 51 common frames omitted
11:51:45,851 |-WARN in ch.qos.logback.core.db.DataSourceConnectionSource#4c1be833 - Connection does not support GetGeneratedKey method and could not discover the dialect.
11:51:45,854 |-ERROR in ch.qos.logback.core.joran.spi.Interpreter#14:13 - RuntimeException in Action for tag [appender] java.lang.IllegalStateException: DBAppender cannot function if the JDBC driver does not support getGeneratedKeys method *and* without a specific SQL dialect
at java.lang.IllegalStateException: DBAppender cannot function if the JDBC driver does not support getGeneratedKeys method *and* without a specific SQL dialect
at at ch.qos.logback.core.db.DBAppenderBase.start(DBAppenderBase.java:62)
at at ch.qos.logback.classic.db.DBAppender.start(DBAppender.java:96)
at at ch.qos.logback.core.joran.action.AppenderAction.end(AppenderAction.java:96)
What is wrong?
I am using mysql-connector-5.0.08 and the database is mysql 5.0.51
EDITED:
I tried to add <sqlDialect class="ch.qos.logback.core.db.dialect.MySQLDialect" /> like this:
<appender name="db" class="ch.qos.logback.classic.db.DBAppender">
<connectionSource class="ch.qos.logback.core.db.DataSourceConnectionSource">
<dataSource class="com.mchange.v2.c3p0.ComboPooledDataSource">
<driverClass>com.mysql.jdbc.Driver</driverClass>
<jdbcUrl>${jdbc.url}</jdbcUrl>
<user>${jdbc.username}</user>
<password>${jdbc.password}</password>
</dataSource>
</connectionSource>
<sqlDialect class="ch.qos.logback.core.db.dialect.MySQLDialect" />`
</appender>
But with no success!
I got this error
java.lang.IllegalStateException: DBAppender cannot function if the
JDBC driver does not support getGeneratedKeys method and without a
specific SQL dialect
The reason was, I have seted a wrong connection URL for DriverManagerConnectionSource. There was no any DB server with that URL. Unfortunately the logback is confusing us with a wrong error message
The problem was that I missed logback-examples-1.0.0.jar
i had a similar error pop up, and it was simply a configuration error.... The "driverClass" was wrong in my xml config file.... I didn't notice right off because I took it from an existing project where I assumed it worked...
So long story short, make sure you're driverClass is:
com.mysql.jdbc.Driver
if you're trying to connect to mysql!
The root cause is right at the bottom of your log snippet:
ERROR in ch.qos.logback.core.joran.spi.Interpreter#14:13 - RuntimeException in Action for tag [appender] java.lang.IllegalStateException: DBAppender cannot function if the JDBC driver does not support getGeneratedKeys method *and* without a specific SQL dialect
You'll have to specify a MySQL dialect in the configuration for DBAppender.
See this: http://logback.qos.ch/manual/appenders.html#DBAppender

Categories

Resources