I am working on a project that requires a database password to be resolved at run time. I am effectively doing this locally by specifying a DataSourceFactory within the Tomcat configuration. This process prevents any credential being stored in a plain text config file and gives us the custom control to resolve passwords at runtime. I have great success with this locally, but when I move my solution to a real/non-embedded Tomcat instance on AWS, I see this error:
Could not invoke the static newInstance method on the named factory
class "org.apache.openjpa.jdbc.kernel.JDBCBrokerFactory".
To start, there are a few fishy details around this error message. Number one (as you'll see below), my factory class is being ignored in the tomcat config file despite the other credentials being valid (e.g. username, password, etc).
The code for my DataSourceFactory needs to reference a class to resolve a value.
Here is the xml configuration that I have for tomcat:
<?xml version='1.0' encoding='UTF-8'?>
<Context useHttpOnly="false">
<Resource auth="Container" name="mail/Session" type="javax.mail.Session"/>
<Resource name="jdbc/jndi"
auth="Container"
scope="Shareable"
type="javax.sql.DataSource"
useLocalSessionState="true"
cacheServerConfiguration="true"
useServerPrepStmts="true"
cachePrepStmts="true"
cacheCallableStmts="true"
elideSetAutoCommits="true"
alwaysSendSetIsolation="false"
enableQueryTimeouts="false"
prepStmtCacheSize="250"
prepStmtCacheSqlLimit="2048"
maxActive="100"
maxIdle="20"
maxWait="10000"
removeAbandoned="true"
driverClassName="com.mysql.jdbc.Driver"
url="${myURL}"
username="${myUsername}"
password="${myPasswordToResolve}"
factory="my.class.CustomDatabaseConfigurationFactory"
testOnBorrow="true"
testWhileIdle="true"
timeBetweenEvictionRunsMillis="20000"
poolPreparedStatements="true"
maxOpenPreparedStatements="50"
validationQuery="select 1"
validationInterval="60000"
defaultTransactionIsolation="READ_COMMITTED"
/>
<Valve className="org.apache.catalina.valves.CometConnectionManagerValve" />
<Loader loaderClass="org.apache.catalina.loader.ParallelWebappClassLoader" />
<Resource auth="Container"
name="jms/JMSConnectionFactory"
type="org.apache.activemq.pool.PooledConnectionFactory"
description="JMS Connection Factory"
factory="org.apache.activemq.jndi.JNDIReferenceFactory"
brokerURL="my.URL"
maxConnections="100"
maximumActiveSessionPerConnection="50"
expiryTimeout="10000"
/>
</Context>
I will not add the code of my my.class.CustomDatabaseConfigurationFactory because I know it works locally. For some reason, when I launch this using a remote configuration and setup, Tomcat completely ignores the custom factory I have put in place for the JNDI resource. Is there anything that I have to add to force Tomcat to look for my custom factory and not ignore it? I have almost the same configuration working on an embedded tomcat instance locally. On top of that, I don't see any clues in the logs that are related to a misconfigured file.
Here is the complete stack trace:
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'entityManagerFactory' defined in class path resource [spring/dataaccess/openjpa/openjpa.xml]: Invocation of ini\
t method failed; nested exception is <openjpa-2.4.0-ep2.1-runknown fatal user error> org.apache.openjpa.persistence.ArgumentException: Could not invoke the static newInstance method on the named fact\
ory class "org.apache.openjpa.jdbc.kernel.JDBCBrokerFactory".
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1514)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:521)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:458)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:293)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:223)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:290)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:191)
at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1119)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:924)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:479)
at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:410)
at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:306)
at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:112)
at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:5118)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5634)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145)
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:899)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:875)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:652)
at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:679)
at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1966)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java)
at java.lang.Thread.run(Thread.java:748)
Realistically, this problem should never occur when proper passwords are being resolved by a custom data factory.
A few things for anyone else who is having this issue:
On top of that, I don't see any clues in the logs that are related to
a misconfigured file.
This was a dead giveaway that something was not being loaded or loggers were not being set up properly. After I realized that log4j wasn't configured properly, I was able to see that my factory was in fact being loaded. Later on, after seeing the results, I was able to see that the passwords being set for the database were incorrect.
Although the log error was too high level and without enough detailed information, this issue was fixable with basic debugging methods and can be avoided with attention to logging.
Related
I have a very specific Tomcat configuration that I don't want to change and I am looking for a solution for this specific configuration.
I have declared multiple contexts (<Context>) for deployment inside the conf/server.xml file in Tomcat, something I know is not recommended. My problem is that if one of those deployments fail, the whole Tomcat process stops which terminates the rest of the applications that have successfully started. Is this behavior expected? Is there an attribute or a way to prevent this and emulate the behavior of distinct context declarations akin to the usage of context.xml file?
The error I'm getting is the following:
ConfigServletWebServerApplicationContext : Exception encountered during context initialization
with this stacktrace (for Tomcat version 8.5.24):
SEVERE [Catalina-startStop-1] org.apache.catalina.core.ContainerBase.startInternal A child container failed during start
java.util.concurrent.ExecutionException: org.apache.catalina.LifecycleException: Failed to start component [StandardEngine[Catalina].StandardHost[localhost].StandardContext[/auth-service]]
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:939)
at org.apache.catalina.core.StandardHost.startInternal(StandardHost.java:872)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1419)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1409)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.catalina.LifecycleException: Failed to start component [StandardEngine[Catalina].StandardHost[localhost].StandardContext[/auth-service]]
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:167)
... 6 more
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'authenticationController': Injection of autowired dependencies failed; nested exception is java.lang.IllegalArgumentException: Could not resolve placeholder 'app.client-html.context' in value "${app.client-html.context}"
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessProperties(AutowiredAnnotationBeanPostProcessor.java:405)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1415)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:608)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:531)
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:944)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:925)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:588)
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:144)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:767)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:759)
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:426)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:326)
at org.springframework.boot.web.servlet.support.SpringBootServletInitializer.run(SpringBootServletInitializer.java:173)
at org.springframework.boot.web.servlet.support.SpringBootServletInitializer.createRootApplicationContext(SpringBootServletInitializer.java:153)
at org.springframework.boot.web.servlet.support.SpringBootServletInitializer.onStartup(SpringBootServletInitializer.java:95)
at org.springframework.web.SpringServletContainerInitializer.onStartup(SpringServletContainerInitializer.java:174)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5196)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
... 6 more
Caused by: java.lang.IllegalArgumentException: Could not resolve placeholder 'app.client-html.context' in value "${app.client-html.context}"
at org.springframework.util.PropertyPlaceholderHelper.parseStringValue(PropertyPlaceholderHelper.java:178)
at org.springframework.util.PropertyPlaceholderHelper.replacePlaceholders(PropertyPlaceholderHelper.java:124)
at org.springframework.core.env.AbstractPropertyResolver.doResolvePlaceholders(AbstractPropertyResolver.java:239)
at org.springframework.core.env.AbstractPropertyResolver.resolveRequiredPlaceholders(AbstractPropertyResolver.java:210)
at org.springframework.context.support.PropertySourcesPlaceholderConfigurer.lambda$processProperties$0(PropertySourcesPlaceholderConfigurer.java:175)
at org.springframework.beans.factory.support.AbstractBeanFactory.resolveEmbeddedValue(AbstractBeanFactory.java:931)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1308)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1287)
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:640)
at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:119)
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessProperties(AutowiredAnnotationBeanPostProcessor.java:399)
... 27 more
Sample configuration:
<Host name="localhost" unpackWARs="true" autoDeploy="true">
<Context docBase="../../app-that-works" path="/itworks"/>
<Context docBase="../../app-that-fails" path="/itfails"/>
</Host>
IMPORTANT: The failing application is a spring-boot application that is packaged as a .war file and running inside a vanilla tomcat, not the embedded tomcat way. This is important because it seems that these applications have this behavior...
DISCLAIMER: I am referring to vanilla Tomcat, not spring boot or embedded tomcat etc. I am not looking for such solutions, simply a solution to this specific problem.
Basically:
an exception during the start phase (cf. LifeCycle) of any component configured in server.xml causes the server to exit. However some exceptions (like a failed ServletContextListener) don't propagate.
an error of a component added during runtime, does not cause the server to exit.
If you don't want the whole server to stop, when a context fails to start up, use auto-deployment: for each context defined in server.xml create a file $CATALINA_BASE/conf/<engine_name>/<host_name>/<context_path>.xml with content:
<Context docBase="/path/to/application" />
In your case you need to create $CATALINA_BASE/conf/Catalina/localhost/itworks.xml and $CATALINA_BASE/conf/Catalina/localhost/itfails.xml. See Naming for Tomcat's naming convention for XML descriptors.
Edit: Your stack trace suggests that we might be dealing with a Tomcat or Spring bug:
On one side the StandardContext expects a ServletContainerInitializer to throw only ServletExceptions (cf. source code) and let's all unchecked exceptions through. Remark that at the same time all exceptions from ServletContextListeners are caught and just logged (cf. source code).
On the other hand Spring's SpringServletContainerInitializer
should probably wrap all unchecked exceptions into a ServletException.
What happens in your case is that an unchecked exception makes it all the way through to the StandardContext and is rethrown as a LifecycleException stopping the server. Anyway, setting:
<Context throwOnFailure="false" ... />
should stop Tomcat from propagating the exception.
When I want to deploy multiple Spring-Boot applications on the same tomcat I face multiple problems.
1) Both applications have to run independent while sharing the same tomcat. When deploying two Spring-Boot application on the same tomcat I get the following Exception:
Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
2019-03-06 17:31:01 ERROR o.s.boot.SpringApplication - Application run failed
org.springframework.jmx.export.UnableToRegisterMBeanException: Unable to register MBean [HikariDataSource (HikariPool-2)] with key 'dataSource'; nested exception is javax.management.InstanceAlreadyExistsException: com.zaxxer.hikari:name=dataSource,type=HikariDataSource
at org.springframework.jmx.export.MBeanExporter.registerBeanNameOrInstance(MBeanExporter.java:625)
at org.springframework.jmx.export.MBeanExporter.lambda$registerBeans$2(MBeanExporter.java:551)
at java.util.HashMap.forEach(HashMap.java:1289)
at org.springframework.jmx.export.MBeanExporter.registerBeans(MBeanExporter.java:551)
at org.springframework.jmx.export.MBeanExporter.afterSingletonsInstantiated(MBeanExporter.java:434)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:863)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:863)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:546)
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:142)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:775)
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:397)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:316)
at org.springframework.boot.web.servlet.support.SpringBootServletInitializer.run(SpringBootServletInitializer.java:157)
at org.springframework.boot.web.servlet.support.SpringBootServletInitializer.createRootApplicationContext(SpringBootServletInitializer.java:137)
at org.springframework.boot.web.servlet.support.SpringBootServletInitializer.onStartup(SpringBootServletInitializer.java:91)
at org.springframework.web.SpringServletContainerInitializer.onStartup(SpringServletContainerInitializer.java:171)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5267)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:754)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:730)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:734)
at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:629)
at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1839)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
at java.util.concurrent.FutureTask.run(FutureTask.java)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: javax.management.InstanceAlreadyExistsException: com.zaxxer.hikari:name=dataSource,type=HikariDataSource
at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)
at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522)
at org.springframework.jmx.support.MBeanRegistrationSupport.doRegister(MBeanRegistrationSupport.java:137)
at org.springframework.jmx.export.MBeanExporter.registerBeanInstance(MBeanExporter.java:671)
at org.springframework.jmx.export.MBeanExporter.registerBeanNameOrInstance(MBeanExporter.java:615)
... 28 common frames omitted
2) I need to provide external configurations for both applications and it needs to be clear that each application uses the right configuration.
1) When two Spring-Boot applications run on the same tomcat you will normally face the issue that the instantiation of a datasource might fail, because an instance with the same name already exists. This is the exception you described in your question.
This can be solved by adding a unique name to each Spring-Boot application e.g. in the
application.yml
spring:
application:
name: application-name-1
jmx:
default-domain: application-name-1
2) Providing an external configuration for each Spring-Boot application can be done by tomcat context configurations for every application individually. Assuming that all applications are deployed as .war e.g. app1.war, app2.war we need to configure the context for both applications as the following:
create the following files (and directories if missing)
tomcat-base-dir
/conf
/catalina
/localhost #must be the same as specified in the Host tag in the server.xml
app1.xml #must have the same name as the .war application file
app2.xml
Content for the app1.xml
<?xml version="1.0" encoding="UTF-8"?>
<!-- docBase must contain be the name of the application to be configured -->
<Context docBase="app1.war">
<Parameter name="spring.config.location" value="${catalina.base}/conf/app1.yml" />
</Context>
This will configure the application app1.war to use the file app1.yml for configuration. Do the same for app2. The actual configuration file app1.yml can be located at any path specified in the value.
I'm encountering a ORA-12519, TNS:no appropriate service handler found error when attempting to integrate Tomcat's JDBC Pool into my web application, using Oracle.
I typically see this error appear intermittently, after several minutes of running integration tests against the application.
The configuration I have is:
oracle driver and tomcat-dbcp jars in the tomcat/lib directory
two webapps, both using the same Resources. Configuration done in Spring:
<jee:jndi-lookup id="webDS" jndi-name="jdbc/web"
expected-type="javax.sql.DataSource" />
DataSource Resources defined in conf/context.xml, as follows:
<Resource name="jdbc/web" auth="Container"
type="javax.sql.DataSource"
factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
testWhileIdle="true"
testOnBorrow="true"
testOnReturn="false"
validationQuery="SELECT 1"
timeBetweenEvictionRunsMillis="30000"
maxActive="20"
maxIdle="10"
minIdle="5"
removeAbandonedTimeout="60"
removeAbandoned="false"
logAbandoned="true"
minEvictableIdleTimeMillis="30000"
jdbcInterceptors="org.apache.tomcat.jdbc.pool.interceptor.ConnectionState;org.apache.tomcat.jdbc.pool.interceptor.StatementFinalizer"
username="${database.user}"
password="${database.password}"
driverClassName="${database.driver}"
url="${database.url}" />
I've tried various suggestions made elsewhere on SO such as increasing or decreasing the maxActive size, but am having no luck thus far. I was previously using c3p0 to manage the pool of connections. With the switch, I'm wondering if there is some additional configuration I need to do that pertains to the closing of connections, because it seems like I'm leaking them.
The stack trace:
Caused by: org.hibernate.exception.GenericJDBCException: Could not open connection
at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:54)
at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:124)
at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:109)
at org.hibernate.engine.jdbc.internal.LogicalConnectionImpl.obtainConnection(LogicalConnectionImpl.java:221)
at org.hibernate.engine.jdbc.internal.LogicalConnectionImpl.getConnection(LogicalConnectionImpl.java:157)
at org.hibernate.engine.jdbc.internal.StatementPreparerImpl.connection(StatementPreparerImpl.java:56)
at org.hibernate.engine.jdbc.internal.StatementPreparerImpl$5.doPrepare(StatementPreparerImpl.java:159)
at org.hibernate.engine.jdbc.internal.StatementPreparerImpl$StatementPreparationTemplate.prepareStatement(StatementPreparerImpl.java:183)
at org.hibernate.engine.jdbc.internal.StatementPreparerImpl.prepareQueryStatement(StatementPreparerImpl.java:157)
at org.hibernate.loader.Loader.prepareQueryStatement(Loader.java:1881)
at org.hibernate.loader.Loader.executeQueryStatement(Loader.java:1858)
at org.hibernate.loader.Loader.executeQueryStatement(Loader.java:1838)
at org.hibernate.loader.Loader.doQuery(Loader.java:906)
at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:348)
at org.hibernate.loader.Loader.doList(Loader.java:2550)
at org.hibernate.loader.Loader.doList(Loader.java:2536)
at org.hibernate.loader.Loader.listIgnoreQueryCache(Loader.java:2366)
at org.hibernate.loader.Loader.list(Loader.java:2361)
at org.hibernate.loader.hql.QueryLoader.list(QueryLoader.java:495)
at org.hibernate.hql.internal.ast.QueryTranslatorImpl.list(QueryTranslatorImpl.java:357)
at org.hibernate.engine.query.spi.HQLQueryPlan.performList(HQLQueryPlan.java:198)
at org.hibernate.internal.SessionImpl.list(SessionImpl.java:1230)
at org.hibernate.internal.QueryImpl.list(QueryImpl.java:101)
at org.hibernate.ejb.QueryImpl.getResultList(QueryImpl.java:268)
... 112 more
Caused by: java.sql.SQLException: Listener refused the connection with the following error:
ORA-12519, TNS:no appropriate service handler found
at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:489)
at oracle.jdbc.driver.PhysicalConnection.<init>(PhysicalConnection.java:553)
at oracle.jdbc.driver.T4CConnection.<init>(T4CConnection.java:254)
at oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:32)
at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:528)
at org.apache.tomcat.jdbc.pool.PooledConnection.connectUsingDriver(PooledConnection.java:278)
at org.apache.tomcat.jdbc.pool.PooledConnection.connect(PooledConnection.java:182)
at org.apache.tomcat.jdbc.pool.PooledConnection.reconnect(PooledConnection.java:315)
at org.apache.tomcat.jdbc.pool.ConnectionPool.borrowConnection(ConnectionPool.java:803)
at org.apache.tomcat.jdbc.pool.ConnectionPool.borrowConnection(ConnectionPool.java:628)
at org.apache.tomcat.jdbc.pool.ConnectionPool.getConnection(ConnectionPool.java:187)
at org.apache.tomcat.jdbc.pool.DataSourceProxy.getConnection(DataSourceProxy.java:128)
at org.hibernate.ejb.connection.InjectedDataSourceConnectionProvider.getConnection(InjectedDataSourceConnectionProvider.java:70)
at org.hibernate.internal.AbstractSessionImpl$NonContextualJdbcConnectionAccess.obtainConnection(AbstractSessionImpl.java:301)
at org.hibernate.engine.jdbc.internal.LogicalConnectionImpl.obtainConnection(LogicalConnectionImpl.java:214)
... 132 more
Caused by: oracle.net.ns.NetException: Listener refused the connection with the following error:
ORA-12519, TNS:no appropriate service handler found
at oracle.net.ns.NSProtocol.connect(NSProtocol.java:399)
at oracle.jdbc.driver.T4CConnection.connect(T4CConnection.java:1140)
at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:340)
... 146 more
Your validation query (SELECT 1) will not work in Oracle. It has to be select 1 from dual.
Seems that when using a wrong validation query the application is not able to identify healthy connections and and mark all of them as invalid.
We have several jobs running in springbatch. We restarted the server this morning and afterwards we started having issues with two of the jobs. The BATCH_STEP_EXECUTION table shows the following exit message for each failed execution:
org.springframework.transaction.TransactionSystemException: Could not roll back JDBC transaction; nested exception is com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Connection.close() has already been called. Invalid operation in this state.
at org.springframework.jdbc.datasource.DataSourceTransactionManager.doRollback(DataSourceTransactionManager.java:286)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processRollback(AbstractPlatformTransactionManager.java:846)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.rollback(AbstractPlatformTransactionManager.java:823)
at org.springframework.transaction.support.TransactionTemplate.rollbackOnException(TransactionTemplate.java:162)
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:135)
at org.springframework.batch.core.step.tasklet.TaskletStep$2.doInChunkContext(TaskletStep.java:267)
at org.springframework.batch.core.scope.context.StepContextRepeatCallback.doInIteration(StepContextRepeatCallback.java:77)
at org.springframework.batch.repeat.support.RepeatTemplate.getNextResult(RepeatTemplate.java:368)
at org.springframework.batch.repeat.support.RepeatTemplate.executeInternal(RepeatTemplate.java:215)
at org.springframework.batch.repeat.support.RepeatTemplate.iterate(RepeatTemplate.java:144)
at org.springframework.batch.core.step.tasklet.TaskletStep.doExecute(TaskletStep.java:253)
at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:195)
at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:141)
at org.springframework.batch.core.job.flow.JobFlowExecutor.executeStep(JobFlowExecutor.java:64)
at org.springframework.batch.core.job.flow.support.state.StepState.handle(StepState.java:60)
at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:151)
at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:130)
at org.springframework.batch.core.job.flow.FlowJob.doExecute(FlowJob.java:135)
at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:301)
at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:134)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.ja
This appears to be in relation to the mysql connection the application is making, however I'm not sure what could have caused this as the application was functioning fine before the reboot. Has anyone stumbled across this before?
EDIT
(caused by message that appears in the logs)
Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Connection.close() has already been called. Invalid operation in this state.
Datasource
javax.sql.DataSource
Connection Pool
Configured in context.xml file with the following format:
<?xml version="1.0" encoding="UTF-8"?>
<Context docBase="webapps/appname" mapperContextRootRedirectEnabled="true" mapperDirectoryRedirectEnabled="true" path="/appname" reloadable="false" >
<Resource name="jdbc/springbatchmeta"
auth="Container"
type="javax.sql.DataSource"
username="${jobmeta.username}"
password="${jobmeta.password}"
driverClassName="${mysql.driverClass}"
url="${jobmeta.url}"
initialSize="2"
maxTotal="20"
maxIdle="10"
minIdle="2"/>
<Resource name="jdbc/firstjob"
auth="Container"
type="javax.sql.DataSource"
username="${firstjob.username}"
password="${firstjob.password}"
driverClassName="${mysql.driverClass}"
url="${firstjob.url}"
initialSize="2"
maxTotal="20"
maxIdle="10"
minIdle="2"/>
<Resource name="jdbc/secondjob"
auth="Container"
type="javax.sql.DataSource"
username="${secondjob.username}"
password="${secondjob.password}"
driverClassName="${secondjob.driverClass}"
url="${secondjob.url}"
initialSize="2"
maxTotal="20"
maxIdle="10"
minIdle="2"/>
</Context>
This issue was caused by the wait-timeout option in mysql being set to 60 seconds, as most of the jobs we are running take over a minute to complete.
Everything works fine in eclipse on my test machine, but when I deploy the WAR to the server I receive "Cannot locate JNDI Resource" errors on each attempt to connect with the database.
I am using a resource within WEB-INF/context.xml. The mysql-connector-java-5.1.25-bin.jar is located within the lib folder of the app. I've tried everything I can think of. I'm assuming its a tomcat specific problem because it works fine on the tomcat installation in eclipse. Is there a setting I am missing?
The context.xml
<?xml version="1.0" encoding="UTF-8"?>
<Context>
<Resource
name="jdbc/appointmentree" type="javax.sql.DataSource"
maxActive="100" maxIdle="30" maxWait="10000"
url="jdbc:mysql://1.1.1.1/appointmentree"
driverClassName="com.mysql.jdbc.Driver"
username="appointmentree" password="password"
/>
</Context>
The resulting error:
fm.sprout.java.db.DatabaseConnectionException: appointmentree is missing in JNDI.
fm.sprout.java.db.DBConnection.<init>(DBConnection.java:17)
fm.sprout.java.db.MySqlDBHandler.find(MySqlDBHandler.java:108)
com.appointmentree.db.DBUser.findAll(DBUser.java:173)
com.appointmentree.db.DBUser.find(DBUser.java:146)
com.appointmentree.db.DBUser.find(DBUser.java:142)
com.appointmentree.obj.User.getInstance(User.java:39)
com.appointmentree.seeds.Login$1.handle(Login.java:39)
fm.sprout.java.forms.FormHandler.init(FormHandler.java:53)
fm.sprout.java.views.SeedBuilder.post(SeedBuilder.java:103)
com.appointmentree.seeds.Login.onLoad(Login.java:33)
fm.sprout.java.views.SeedPlanter.sow(SeedPlanter.java:131)
fm.sprout.java.session.RequestHandler.doPost(RequestHandler.java:30)
javax.servlet.http.HttpServlet.service(HttpServlet.java:646)
javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
com.appointmentree.session.UserSessionFilter.doFilter(UserSessionFilter.java:72)
root cause
javax.naming.NamingException: com.mysql.jdbc.Driver
org.apache.naming.NamingContext.lookup(NamingContext.java:859)
org.apache.naming.NamingContext.lookup(NamingContext.java:153)
org.apache.naming.NamingContext.lookup(NamingContext.java:830)
org.apache.naming.NamingContext.lookup(NamingContext.java:153)
org.apache.naming.NamingContext.lookup(NamingContext.java:830)
org.apache.naming.NamingContext.lookup(NamingContext.java:153)
org.apache.naming.NamingContext.lookup(NamingContext.java:830)
org.apache.naming.NamingContext.lookup(NamingContext.java:167)
org.apache.naming.SelectorContext.lookup(SelectorContext.java:156)
javax.naming.InitialContext.lookup(InitialContext.java:411)
fm.sprout.java.db.DBConnection.<init>(DBConnection.java:15)
fm.sprout.java.db.MySqlDBHandler.find(MySqlDBHandler.java:108)
com.appointmentree.db.DBUser.findAll(DBUser.java:173)
com.appointmentree.db.DBUser.find(DBUser.java:146)
com.appointmentree.db.DBUser.find(DBUser.java:142)
com.appointmentree.obj.User.getInstance(User.java:39)
com.appointmentree.seeds.Login$1.handle(Login.java:39)
fm.sprout.java.forms.FormHandler.init(FormHandler.java:53)
fm.sprout.java.views.SeedBuilder.post(SeedBuilder.java:103)
com.appointmentree.seeds.Login.onLoad(Login.java:33)
fm.sprout.java.views.SeedPlanter.sow(SeedPlanter.java:131)
fm.sprout.java.session.RequestHandler.doPost(RequestHandler.java:30)
javax.servlet.http.HttpServlet.service(HttpServlet.java:646)
javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
com.appointmentree.session.UserSessionFilter.doFilter(UserSessionFilter.java:72)
The resource in web.xml
<resource-env-ref>
<resource-env-ref-name>jdbc/appointmentree</resource-env-ref-name>
<resource-env-ref-type>javax.sql.DataSource</resource-env-ref-type>
</resource-env-ref>
Try using this:
<resource-ref>
<res-ref-name>jdbc/appointmentree</res-ref-name>
<res-ref-type>javax.sql.DataSource</res-ref-type>
<res-auth>Container</res-auth>
</resource-ref>
I got this information from here: http://tomcat.apache.org/tomcat-7.0-doc/config/context.html#Resource_Definitions
Turns out I hadn't included the mysql connector jar in the CATALINA_HOME/lib folder. I was under the impression that having the connector in the lib folder of the application was sufficient but I was wrong.
From another stackoverflow question/answer: Tomcat does not recognize the MySQL .jar library
It's not going to work in WEB-INF/lib since the container needs access to the library, and, that is putting it in the classloader of one web app, not the container. While I would have imagined the CLASSPATH approach would work, it's not the standard way to do this. Perhaps there is some other snag that's preventing it from working.