How to handle Broken pipe error with Spring, (JPA) Hibernate, MySQL - java

I have an web application using Spring JPA (Hibernate) and MySQL running in Tomcat.
After period of inactivity I always get this exception:
Caused by: javax.persistence.PersistenceException: org.hibernate.TransactionException: JDBC begin transaction failed:
at org.hibernate.jpa.spi.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1763) ~[hibernate-entitymanager-4.3.5.Final.jar:4.3.5.Final]
at org.hibernate.jpa.spi.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1677) ~[hibernate-entitymanager-4.3.5.Final.jar:4.3.5.Final]
at org.hibernate.jpa.spi.AbstractEntityManagerImpl.throwPersistenceException(AbstractEntityManagerImpl.java:1771) ~[hibernate-entitymanager-4.3.5.Final.jar:4.3.5.Final]
at org.hibernate.jpa.internal.TransactionImpl.begin(TransactionImpl.java:64) ~[hibernate-entitymanager-4.3.5.Final.jar:4.3.5.Final]
at org.springframework.orm.jpa.DefaultJpaDialect.beginTransaction(DefaultJpaDialect.java:67) ~[spring-orm-4.0.3.RELEASE.jar:4.0.3.RELEASE]
at org.springframework.orm.jpa.JpaTransactionManager.doBegin(JpaTransactionManager.java:380) ~[spring-orm-4.0.3.RELEASE.jar:4.0.3.RELEASE]
... 38 common frames omitted
Caused by: org.hibernate.TransactionException: JDBC begin transaction failed:
at org.hibernate.engine.transaction.internal.jdbc.JdbcTransaction.doBegin(JdbcTransaction.java:76) ~[hibernate-core-4.3.5.Final.jar:4.3.5.Final]
at org.hibernate.engine.transaction.spi.AbstractTransactionImpl.begin(AbstractTransactionImpl.java:162) ~[hibernate-core-4.3.5.Final.jar:4.3.5.Final]
at org.hibernate.internal.SessionImpl.beginTransaction(SessionImpl.java:1435) ~[hibernate-core-4.3.5.Final.jar:4.3.5.Final]
at org.hibernate.jpa.internal.TransactionImpl.begin(TransactionImpl.java:61) ~[hibernate-entitymanager-4.3.5.Final.jar:4.3.5.Final]
... 40 common frames omitted
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: The last packet successfully received from the server was 48,764,262 milliseconds ago. The last packet sent successfully to the server wa
s 48,764,262 milliseconds ago. is longer than the server configured value of 'wait_timeout'. You should consider either expiring and/or testing connection validity before use in your application, increasing
the server configured values for client timeouts, or using the Connector/J connection property 'autoReconnect=true' to avoid this problem.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[na:1.7.0_55]
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) ~[na:1.7.0_55]
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[na:1.7.0_55]
at java.lang.reflect.Constructor.newInstance(Constructor.java:526) ~[na:1.7.0_55]
at com.mysql.jdbc.Util.handleNewInstance(Util.java:409) ~[mysql-connector-java-5.1.30.jar:na]
at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1127) ~[mysql-connector-java-5.1.30.jar:na]
at com.mysql.jdbc.MysqlIO.send(MysqlIO.java:3983) ~[mysql-connector-java-5.1.30.jar:na]
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2596) ~[mysql-connector-java-5.1.30.jar:na]
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2776) ~[mysql-connector-java-5.1.30.jar:na]
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2832) ~[mysql-connector-java-5.1.30.jar:na]
at com.mysql.jdbc.ConnectionImpl.setAutoCommit(ConnectionImpl.java:5357) ~[mysql-connector-java-5.1.30.jar:na]
at org.apache.tomcat.dbcp.dbcp.DelegatingConnection.setAutoCommit(DelegatingConnection.java:373) ~[commons-dbcp-eap6.jar:na]
at org.apache.tomcat.dbcp.dbcp.PoolingDataSource$PoolGuardConnectionWrapper.setAutoCommit(PoolingDataSource.java:330) ~[commons-dbcp-eap6.jar:na]
at org.hibernate.engine.transaction.internal.jdbc.JdbcTransaction.doBegin(JdbcTransaction.java:72) ~[hibernate-core-4.3.5.Final.jar:4.3.5.Final]
... 43 common frames omitted
Caused by: java.net.SocketException: Broken pipe
at java.net.SocketOutputStream.socketWrite0(Native Method) ~[na:1.7.0_55]
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:113) ~[na:1.7.0_55]
at java.net.SocketOutputStream.write(SocketOutputStream.java:159) ~[na:1.7.0_55]
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82) ~[na:1.7.0_55]
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140) ~[na:1.7.0_55]
at com.mysql.jdbc.MysqlIO.send(MysqlIO.java:3964) ~[mysql-connector-java-5.1.30.jar:na]
... 50 common frames omitted
Now, I do understand this has to do with MySQL closing the connection after 8 hours and Hibernate trying to use this connection and that my connection pool might not be configured properly or configured at all but I do not know how to do it.
I even added autoReconnect=true to mysql url connection configuration but it did not help.
This is what is in server.xml of the Tomcat configuration.
<Resource name="jdbc/mysqldb" auth="Container"
type="javax.sql.DataSource"
...
url="jdbc:mysql://${env.OPENSHIFT_MYSQL_DB_HOST}:${env.OPENSHIFT_MYSQL_DB_PORT}/${env.OPENSHIFT_APP_NAME}?autoReconnect=true"/>
What and where should be configured to get this working?

Try doing something like this in hibernate configuration:
<property name="connection.autoReconnect"> true </ property>
<property name="connection.autoReconnectForPools"> true </ property>
<property name="connection.is-connection-validation-required"> true </ property>

Try adding testOnBorrow="true" and validationQuery="SELECT 1" validationInterval="60000"
This happens when DB server terminates connections. After adding these parameters, the connection pool will validate the connection before giving it to the application. If it is not valid, the connection will be discarded and a new one will be created.

Related

Unable to initiate metastore in Hive with Hodoop

I have Hadoop 3.3.2, Hive 3.1.3, MySql 8.0.29(apt install mysql-server) installed. MySql is running properly with mysql -h 127.0.0.1 -u debian-sys-maint -p command. But unable to run any command in Hive. I'm getting FAILED: HiveException java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient exception.
Then in SO, I found this question on SO, which says to execute hive --service metastore. But, when I execute this command, I get this error:
2022-07-24 17:15:57: Starting Hive Metastore Server
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/hdoop/apache-hive-3.1.3-bin/lib/log4j-slf4j-impl-2.17.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/hdoop/hadoop-3.3.2/share/hadoop/common/lib/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
MetaException(message:Error creating transactional connection factory)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:84)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:93)
at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:8672)
at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:8667)
at org.apache.hadoop.hive.metastore.HiveMetaStore.startMetaStore(HiveMetaStore.java:8937)
at org.apache.hadoop.hive.metastore.HiveMetaStore.main(HiveMetaStore.java:8854)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:323)
at org.apache.hadoop.util.RunJar.main(RunJar.java:236)
Caused by: MetaException(message:Error creating transactional connection factory)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:208)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:80)
... 11 more
Caused by: javax.jdo.JDOFatalInternalException: Error creating transactional connection factory
NestedThrowables:
java.lang.reflect.InvocationTargetException
at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:671)
at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:830)
at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:334)
at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:213)
at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
at java.security.AccessController.doPrivileged(Native Method)
at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:651)
at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:694)
at org.apache.hadoop.hive.metastore.ObjectStore.initializeHelper(ObjectStore.java:484)
at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:421)
at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:376)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:79)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:139)
at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:59)
at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:67)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStoreForConf(HiveMetaStore.java:720)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMSForConf(HiveMetaStore.java:698)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:692)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:775)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:540)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
... 13 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.GeneratedConstructorAccessor76.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:606)
at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:330)
at org.datanucleus.store.AbstractStoreManager.registerConnectionFactory(AbstractStoreManager.java:203)
at org.datanucleus.store.AbstractStoreManager.<init>(AbstractStoreManager.java:162)
at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:285)
at sun.reflect.GeneratedConstructorAccessor75.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:606)
at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
at org.datanucleus.NucleusContextHelper.createStoreManagerForProperties(NucleusContextHelper.java:133)
at org.datanucleus.PersistenceNucleusContextImpl.initialise(PersistenceNucleusContextImpl.java:422)
at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:817)
... 43 more
Caused by: org.datanucleus.exceptions.NucleusException: Attempt to invoke the "HikariCP" plugin to create a ConnectionPool gave an error : Failed to initialize pool: Client does not support authentication protocol requested by server; consider upgrading MySQL client
at org.datanucleus.store.rdbms.ConnectionFactoryImpl.generateDataSources(ConnectionFactoryImpl.java:232)
at org.datanucleus.store.rdbms.ConnectionFactoryImpl.initialiseDataSources(ConnectionFactoryImpl.java:117)
at org.datanucleus.store.rdbms.ConnectionFactoryImpl.<init>(ConnectionFactoryImpl.java:82)
... 59 more
Caused by: com.zaxxer.hikari.pool.HikariPool$PoolInitializationException: Failed to initialize pool: Client does not support authentication protocol requested by server; consider upgrading MySQL client
at com.zaxxer.hikari.pool.HikariPool.throwPoolInitializationException(HikariPool.java:544)
at com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:536)
at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:112)
at com.zaxxer.hikari.HikariDataSource.<init>(HikariDataSource.java:72)
at org.datanucleus.store.rdbms.connectionpool.HikariCPConnectionPoolFactory.createConnectionPool(HikariCPConnectionPoolFactory.java:176)
at org.datanucleus.store.rdbms.ConnectionFactoryImpl.generateDataSources(ConnectionFactoryImpl.java:213)
... 61 more
Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Client does not support authentication protocol requested by server; consider upgrading MySQL client
at sun.reflect.GeneratedConstructorAccessor81.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:406)
at com.mysql.jdbc.Util.getInstance(Util.java:381)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:984)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:956)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3558)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3490)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:919)
at com.mysql.jdbc.MysqlIO.secureAuth411(MysqlIO.java:3996)
at com.mysql.jdbc.MysqlIO.doHandshake(MysqlIO.java:1284)
at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2137)
at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:776)
at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:46)
at sun.reflect.GeneratedConstructorAccessor78.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:406)
at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:352)
at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:284)
at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:95)
at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:101)
at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:356)
at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:199)
at com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:444)
at com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:515)
... 65 more
Exception in thread "main" MetaException(message:Error creating transactional connection factory)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:84)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:93)
at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:8672)
at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:8667)
at org.apache.hadoop.hive.metastore.HiveMetaStore.startMetaStore(HiveMetaStore.java:8937)
at org.apache.hadoop.hive.metastore.HiveMetaStore.main(HiveMetaStore.java:8854)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:323)
at org.apache.hadoop.util.RunJar.main(RunJar.java:236)
Caused by: MetaException(message:Error creating transactional connection factory)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:208)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:80)
... 11 more
Caused by: javax.jdo.JDOFatalInternalException: Error creating transactional connection factory
NestedThrowables:
java.lang.reflect.InvocationTargetException
at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:671)
at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:830)
at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:334)
at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:213)
at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
at java.security.AccessController.doPrivileged(Native Method)
at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:651)
at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:694)
at org.apache.hadoop.hive.metastore.ObjectStore.initializeHelper(ObjectStore.java:484)
at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:421)
at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:376)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:79)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:139)
at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:59)
at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:67)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStoreForConf(HiveMetaStore.java:720)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMSForConf(HiveMetaStore.java:698)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:692)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:775)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:540)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
... 13 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.GeneratedConstructorAccessor76.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:606)
at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:330)
at org.datanucleus.store.AbstractStoreManager.registerConnectionFactory(AbstractStoreManager.java:203)
at org.datanucleus.store.AbstractStoreManager.<init>(AbstractStoreManager.java:162)
at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:285)
at sun.reflect.GeneratedConstructorAccessor75.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:606)
at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
at org.datanucleus.NucleusContextHelper.createStoreManagerForProperties(NucleusContextHelper.java:133)
at org.datanucleus.PersistenceNucleusContextImpl.initialise(PersistenceNucleusContextImpl.java:422)
at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:817)
... 43 more
Caused by: org.datanucleus.exceptions.NucleusException: Attempt to invoke the "HikariCP" plugin to create a ConnectionPool gave an error : Failed to initialize pool: Client does not support authentication protocol requested by server; consider upgrading MySQL client
at org.datanucleus.store.rdbms.ConnectionFactoryImpl.generateDataSources(ConnectionFactoryImpl.java:232)
at org.datanucleus.store.rdbms.ConnectionFactoryImpl.initialiseDataSources(ConnectionFactoryImpl.java:117)
at org.datanucleus.store.rdbms.ConnectionFactoryImpl.<init>(ConnectionFactoryImpl.java:82)
... 59 more
Caused by: com.zaxxer.hikari.pool.HikariPool$PoolInitializationException: Failed to initialize pool: Client does not support authentication protocol requested by server; consider upgrading MySQL client
at com.zaxxer.hikari.pool.HikariPool.throwPoolInitializationException(HikariPool.java:544)
at com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:536)
at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:112)
at com.zaxxer.hikari.HikariDataSource.<init>(HikariDataSource.java:72)
at org.datanucleus.store.rdbms.connectionpool.HikariCPConnectionPoolFactory.createConnectionPool(HikariCPConnectionPoolFactory.java:176)
at org.datanucleus.store.rdbms.ConnectionFactoryImpl.generateDataSources(ConnectionFactoryImpl.java:213)
... 61 more
Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Client does not support authentication protocol requested by server; consider upgrading MySQL client
at sun.reflect.GeneratedConstructorAccessor81.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:406)
at com.mysql.jdbc.Util.getInstance(Util.java:381)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:984)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:956)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3558)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3490)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:919)
at com.mysql.jdbc.MysqlIO.secureAuth411(MysqlIO.java:3996)
at com.mysql.jdbc.MysqlIO.doHandshake(MysqlIO.java:1284)
at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2137)
at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:776)
at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:46)
at sun.reflect.GeneratedConstructorAccessor78.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:406)
at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:352)
at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:284)
at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:95)
at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:101)
at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:356)
at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:199)
at com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:444)
at com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:515)
... 65 more
I am unable to resolve this error. $HIVE_HOME/conf/hive-site.xml looks like this:
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://127.0.0.1/hive_metastore?createDatabaseIfNotExist=true</value>
</property>
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
</property>
<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>hiveuser</value>
</property>
<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>hivepassword</value>
</property>
<property>
<name>datanucleus.autoCreateSchema</name>
<value>true</value>
</property>
<property>
<name>datanucleus.fixedDatastore</name>
<value>true</value>
</property>
<property>
<name>datanucleus.autoCreateTables</name>
<value>True</value>
</property>
</configuration>
I also have mysql-connector-java-5.1.9.jar in $HIVE_HOME/lib. I am not getting what I am missing. Any help would be appreciated.
Thanks to suggestions by #ggordon in comments under the question. I followed the steps in this website with some small modifications.
When I executed mysql -u debian-sys-maint -p, I was getting ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (13) error. So, I changed it to use loopback address: mysql -h 127.0.0.1 -u debian-sys-maint -p.
CREATE USER 'hiveuser'#'%' IDENTIFIED BY 'hivepassword' has to be changed to CREATE USER 'hiveuser'#'127.0.0.1' IDENTIFIED BY 'hivepassword', since that is what we are using. (I missed this step in question)
GRANT all on . to hiveuser was not working for me. So, after some googling, I changed it to GRANT all on hive_metastore.* to 'hiveuser'#'127.0.0.1'.
javax.jdo.option.ConnectionURL property in $HIVE_HOME/conf/hive-site.xml was set to jdbc:mysql://localhost/hive_metastore?createDatabaseIfNotExist=true. It has to be changed to jdbc:mysql://127.0.0.1/hive_metastore?createDatabaseIfNotExist=true.
With these changes, I followed steps in the above linked site, and it is working fine now.

Communications link failure Caused by: java.net.SocketException: Connection reset.

I am trying to connect to MySql db using hibernate from my java application. When i deploy the war on one of our servers, i am getting the below exception though it works when i deploy it locally on my machine. Network connectivity is also not the issue since i successfully tried pinging the db from the server.
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet successfully received from the server was 600,902 milliseconds ago. The last packet sent successfully to the server was 600,902 milliseconds ago.
at sun.reflect.GeneratedConstructorAccessor230.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1117)
at com.mysql.jdbc.MysqlIO.send(MysqlIO.java:3851)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2471)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2651)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2728)
at com.mysql.jdbc.ConnectionImpl.setAutoCommit(ConnectionImpl.java:5075)
at com.mchange.v2.c3p0.impl.NewProxyConnection.setAutoCommit(NewProxyConnection.java:881)
at org.hibernate.service.jdbc.connections.internal.C3P0ConnectionProvider.getConnection(C3P0ConnectionProvider.java:89)
at org.hibernate.internal.AbstractSessionImpl$NonContextualJdbcConnectionAccess.obtainConnection(AbstractSessionImpl.java:292)
at org.hibernate.engine.jdbc.internal.LogicalConnectionImpl.obtainConnection(LogicalConnectionImpl.java:297)
... 30 more
Caused by: java.net.SocketException: Connection reset
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:118)
at java.net.SocketOutputStream.write(SocketOutputStream.java:159)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
at com.mysql.jdbc.MysqlIO.send(MysqlIO.java:3832)
... 38 more
at com.shc.ecom.rx.batch.scheduler.RunSchedulerPoc.run(RunSchedulerPoc.java:63)
at sun.reflect.GeneratedMethodAccessor180.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:64)
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:53)
at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:81)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
The answers so far on this question doesn't seem to solve my problem. Any help is appreciated.
It worked after i added the following property to my hibernate configuration, it was missing earlier :
hibernate.c3p0.idle_test_period=300
You can read about it on :
https://www.mkyong.com/hibernate/how-to-configure-the-c3p0-connection-pool-in-hibernate/

jboss wildfly datasource connection failing - No managed connections available within configured blocking timeout

I have a jboss wildfly server connected to a mysql database via a datasource configured on the app server.
It hapens some time that the mysql database is not accessible for an amount of time (netework problem etc...), when the mysql is not accessible I receive in logs the message :
09:42:30,015 ERROR [org.jboss.seam.async.AsynchronousExceptionHandler] (4pm_services_QuartzScheduler_Worker-4) Exception thrown whilst executing asynchronous call: java.
lang.RuntimeException: javax.persistence.PersistenceException: org.hibernate.exception.GenericJDBCException: Cannot open connection
this is expected as thy mysql is not available at that moment, the porblem is that when the mysql server comes online again the error still persists, and the jboss server is unable to get a valid from the connection pool.
I have:
wildfly 8.1
mysql 5.5
use JPA for comunicating with database
here is the datasource configuration:
<datasource jta="true" jndi-name="java:jboss/services_DS" pool-name="fourPmDSServices" enabled="true" use-java-context="true">
<connection-url>jdbc:mysql://192.168.10.110:3306/FOUR_PM__SERVICES?useUnicode=true&characterEncoding=UTF-8</connection-url>
<driver>mysql</driver>
<new-connection-sql>select 1</new-connection-sql>
<transaction-isolation>TRANSACTION_READ_COMMITTED</transaction-isolation>
<pool>
<min-pool-size>1</min-pool-size>
<max-pool-size>5</max-pool-size>
<prefill>true</prefill>
<use-strict-min>false</use-strict-min>
</pool>
<security>
<user-name>*********</user-name>
<password>*********</password>
</security>
<validation>
<valid-connection-checker class-name="org.jboss.jca.adapters.jdbc.extensions.mysql.MySQLValidConnectionChecker"/>
<check-valid-connection-sql>select 1</check-valid-connection-sql>
<stale-connection-checker class-name="org.jboss.jca.adapters.jdbc.extensions.mysql.MySQLStaleConnectionChecker"/>
<exception-sorter class-name="org.jboss.jca.adapters.jdbc.extensions.mysql.MySQLExceptionSorter"/>
</validation>
<statement>
<prepared-statement-cache-size>10</prepared-statement-cache-size>
<share-prepared-statements>true</share-prepared-statements>
</statement>
</datasource>
And here is the complete stack trace of the error:
00:00:00,017 WARN [org.hibernate.util.JDBCExceptionReporter] (4pm_services_QuartzScheduler_Worker-2) SQL Error: 0, SQLState: null
00:00:00,018 ERROR [org.hibernate.util.JDBCExceptionReporter] (4pm_services_QuartzScheduler_Worker-2) javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:jboss/services_DS
00:00:00,018 ERROR [org.jboss.seam.async.AsynchronousExceptionHandler] (4pm_services_QuartzScheduler_Worker-2) Exception thrown whilst executing asynchronous call: java.lang.RuntimeException: javax.persistence.PersistenceException: org.hibernate.exception.GenericJDBCException: Cannot open connection
at si.arctur.fourPmServices.scheduleJobs.QuartzJobBase.executeInternal(QuartzJobBase.java:91) [4pm_services-ejb.jar:]
at si.arctur.fourPmServices.scheduleJobs.QuartzJobBase.schedule(QuartzJobBase.java:76) [4pm_services-ejb.jar:]
at sun.reflect.GeneratedMethodAccessor98.invoke(Unknown Source) [:1.6.0_32]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.6.0_32]
at java.lang.reflect.Method.invoke(Method.java:622) [rt.jar:1.6.0_32]
at org.jboss.seam.util.Reflections.invoke(Reflections.java:22) [jboss-seam.jar:2.2.2.Final]
at org.jboss.seam.intercept.RootInvocationContext.proceed(RootInvocationContext.java:32) [jboss-seam.jar:2.2.2.Final]
at org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:56) [jboss-seam.jar:2.2.2.Final]
at org.jboss.seam.transaction.RollbackInterceptor.aroundInvoke(RollbackInterceptor.java:28) [jboss-seam.jar:2.2.2.Final]
at org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:68) [jboss-seam.jar:2.2.2.Final]
at org.jboss.seam.core.BijectionInterceptor.aroundInvoke(BijectionInterceptor.java:77) [jboss-seam.jar:2.2.2.Final]
at org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:68) [jboss-seam.jar:2.2.2.Final]
at org.jboss.seam.core.MethodContextInterceptor.aroundInvoke(MethodContextInterceptor.java:44) [jboss-seam.jar:2.2.2.Final]
at org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:68) [jboss-seam.jar:2.2.2.Final]
at org.jboss.seam.async.AsynchronousInterceptor.aroundInvoke(AsynchronousInterceptor.java:52) [jboss-seam.jar:2.2.2.Final]
at org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:68) [jboss-seam.jar:2.2.2.Final]
at org.jboss.seam.intercept.RootInterceptor.invoke(RootInterceptor.java:107) [jboss-seam.jar:2.2.2.Final]
at org.jboss.seam.intercept.JavaBeanInterceptor.interceptInvocation(JavaBeanInterceptor.java:185) [jboss-seam.jar:2.2.2.Final]
at org.jboss.seam.intercept.JavaBeanInterceptor.invoke(JavaBeanInterceptor.java:103) [jboss-seam.jar:2.2.2.Final]
at si.arctur.fourPmServices.scheduleJobs.emailSender.MailChecker_$$_javassist_seam_2.schedule(MailChecker_$$_javassist_seam_2.java) [4pm_services-ejb.jar:]
at sun.reflect.GeneratedMethodAccessor104.invoke(Unknown Source) [:1.6.0_32]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.6.0_32]
at java.lang.reflect.Method.invoke(Method.java:622) [rt.jar:1.6.0_32]
at org.jboss.seam.util.Reflections.invoke(Reflections.java:22) [jboss-seam.jar:2.2.2.Final]
at org.jboss.seam.util.Reflections.invokeAndWrap(Reflections.java:144) [jboss-seam.jar:2.2.2.Final]
at org.jboss.seam.async.AsynchronousInvocation$1.process(AsynchronousInvocation.java:62) [jboss-seam.jar:2.2.2.Final]
at org.jboss.seam.async.Asynchronous$ContextualAsynchronousRequest.run(Asynchronous.java:80) [jboss-seam.jar:2.2.2.Final]
at org.jboss.seam.async.AsynchronousInvocation.execute(AsynchronousInvocation.java:44) [jboss-seam.jar:2.2.2.Final]
at org.jboss.seam.async.QuartzDispatcher$QuartzJob.execute(QuartzDispatcher.java:243) [jboss-seam.jar:2.2.2.Final]
at org.quartz.core.JobRunShell.run(JobRunShell.java:202) [quartz.jar:1.6.0]
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:529) [quartz.jar:1.6.0]
Caused by: javax.persistence.PersistenceException: org.hibernate.exception.GenericJDBCException: Cannot open connection
at org.hibernate.ejb.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1389) [hibernate-core.jar:3.6.10.Final]
at org.hibernate.ejb.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1317) [hibernate-core.jar:3.6.10.Final]
at org.hibernate.ejb.QueryImpl.getResultList(QueryImpl.java:255) [hibernate-core.jar:3.6.10.Final]
at si.arctur.fourPmServices.scheduleJobs.emailSender.MailChecker.checkQueue(MailChecker.java:37) [4pm_services-ejb.jar:]
at si.arctur.fourPmServices.scheduleJobs.emailSender.MailChecker.execute(MailChecker.java:26) [4pm_services-ejb.jar:]
at si.arctur.fourPmServices.scheduleJobs.QuartzJobBase$TransactionalWork.work(QuartzJobBase.java:144) [4pm_services-ejb.jar:]
at org.jboss.seam.util.Work.workInTransaction(Work.java:61) [jboss-seam.jar:2.2.2.Final]
at si.arctur.fourPmServices.scheduleJobs.QuartzJobBase.executeInternal(QuartzJobBase.java:89) [4pm_services-ejb.jar:]
... 30 more
Caused by: org.hibernate.exception.GenericJDBCException: Cannot open connection
at org.hibernate.exception.SQLStateConverter.handledNonSpecificException(SQLStateConverter.java:140) [hibernate-core.jar:3.6.10.Final]
at org.hibernate.exception.SQLStateConverter.convert(SQLStateConverter.java:128) [hibernate-core.jar:3.6.10.Final]
at org.hibernate.exception.JDBCExceptionHelper.convert(JDBCExceptionHelper.java:66) [hibernate-core.jar:3.6.10.Final]
at org.hibernate.exception.JDBCExceptionHelper.convert(JDBCExceptionHelper.java:52) [hibernate-core.jar:3.6.10.Final]
at org.hibernate.jdbc.ConnectionManager.openConnection(ConnectionManager.java:449) [hibernate-core.jar:3.6.10.Final]
at org.hibernate.jdbc.ConnectionManager.getConnection(ConnectionManager.java:167) [hibernate-core.jar:3.6.10.Final]
at org.hibernate.jdbc.AbstractBatcher.prepareQueryStatement(AbstractBatcher.java:161) [hibernate-core.jar:3.6.10.Final]
at org.hibernate.loader.Loader.prepareQueryStatement(Loader.java:1700) [hibernate-core.jar:3.6.10.Final]
at org.hibernate.loader.Loader.doQuery(Loader.java:801) [hibernate-core.jar:3.6.10.Final]
at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:274) [hibernate-core.jar:3.6.10.Final]
at org.hibernate.loader.Loader.doList(Loader.java:2542) [hibernate-core.jar:3.6.10.Final]
at org.hibernate.loader.Loader.listIgnoreQueryCache(Loader.java:2276) [hibernate-core.jar:3.6.10.Final]
at org.hibernate.loader.Loader.list(Loader.java:2271) [hibernate-core.jar:3.6.10.Final]
at org.hibernate.loader.hql.QueryLoader.list(QueryLoader.java:459) [hibernate-core.jar:3.6.10.Final]
at org.hibernate.hql.ast.QueryTranslatorImpl.list(QueryTranslatorImpl.java:365) [hibernate-core.jar:3.6.10.Final]
at org.hibernate.engine.query.HQLQueryPlan.performList(HQLQueryPlan.java:196) [hibernate-core.jar:3.6.10.Final]
at org.hibernate.impl.SessionImpl.list(SessionImpl.java:1268) [hibernate-core.jar:3.6.10.Final]
at org.hibernate.impl.QueryImpl.list(QueryImpl.java:102) [hibernate-core.jar:3.6.10.Final]
at org.hibernate.ejb.QueryImpl.getResultList(QueryImpl.java:246) [hibernate-core.jar:3.6.10.Final]
... 35 more
Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:jboss/services_DS
at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137)
at org.hibernate.ejb.connection.InjectedDataSourceConnectionProvider.getConnection(InjectedDataSourceConnectionProvider.java:71) [hibernate-core.jar:3.6.10.Final]
at org.hibernate.jdbc.ConnectionManager.openConnection(ConnectionManager.java:446) [hibernate-core.jar:3.6.10.Final]
... 49 more
Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:jboss/services_DS
at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390)
at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368)
at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464)
at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129)
... 51 more
Caused by: javax.resource.ResourceException: IJ000655: No managed connections available within configured blocking timeout (30000 [ms])
at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:377)
at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getTransactionNewConnection(AbstractPool.java:495)
at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:374)
at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329)
... 54 more
It looks like you need to define a flush strategy of you failed connections.
check the "flush-strategy" param.
A good point to start is here:
https://docs.jboss.org/author/display/WFLY8/DataSource+configuration
They reference this XSD:
http://www.ironjacamar.org/doc/schema/datasources_1_1.xsd
And the XSD decription is here:
http://www.ironjacamar.org/doc/userguide/1.2/en-US/html_single/index.html#deployingds
In your case I would try to figure out the network issues.
Independently of this. I would in your case (with bad connection issues) flush my complete connection pool:
flush-strategy => EntirePool
good luck on fixing such an odd issue
What Does The Message No Managed Connection Available mean?
It means that all possible connections to the db are in use and/or that a thread has timed-out waiting for a connection to open up.
Possible solutions:
Increase your connection pool size
Add
<max-pool-size>whatever</max-pool-size>
to your -ds.xml file to increase
the number of connections available. The default is 20
There is an attribute
MaxConnectionsInUse
viewable on the jmx-console against the
ManagedConnectionPool
that can help you understand how many connections you really need.
Tell your threads to wait longer for a connection to open up
Add
<blocking-timeout-millis>whatever</blocking-timeout-millis>
to your -ds.xml file to increase the length of time to wait. The default is 30000 (5000 before JBoss-3.2.4) milli-seconds.
Make sure you are properly closing open connections
I encountered this issue as well. However, my connection worked briefly and then stopped working after a number of requests. It turned out to be an unclosed connection to the database, which caused connections to be consumed and eventually exhausting the connection pool. Simply increasing the connection pool size will only delay the eventual error. You'll need to identify where you not correctly closing your connection in order to resolve this issue.
debug="true" error="true" in the JCA in standalone-ha.xml
Verify that exists in the jca subsystem and set debug="true".
...
...
Then in server.log it will write if any connections are not closing with stack trace of the code.
I had a similar issue that I fixed with the answer that #Harsh said. I increased the maxpoolsize from 20 to 100 and the problem is gone.
Caused by: javax.resource.ResourceException: IJ000655: No managed connections available within configured blocking timeout (30000 [ms])
This message says, that configured pool cannot provide any connection within configured timeout of 3sec (3000msec). Either your pool is too low or your connections are not closed after usage, perhaps increasing the pool size will work.
Also check the connection close from JBoss.
Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:jboss/services_DS
It seem that problem is related to some database table and java configuration like hbm.xml. For each Table Type parameter the java api opens a connection to database or due to some other environment configuration.
Please refer following link here you will get same problem with solution.
https://developer.jboss.org/thread/211940
http://www.coderanch.com/t/556721/JBoss/Jboss-Datasource-configured-correctly-unable

grails mysql packet failed

I've been performing some database opreations select and insert for more than 12 hours using Grails and Gorm. I pool the connections too.
I suddenly get the following error and had to restart MySQL. Why?:
java.net.SocketException: Software caused connection abort: recv failed
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:150)
at java.net.SocketInputStream.read(SocketInputStream.java:121)
at com.mysql.jdbc.util.ReadAheadInputStream.fill(ReadAheadInputStream.java:114)
at com.mysql.jdbc.util.ReadAheadInputStream.readFromUnderlyingStreamIfNecessary(ReadAheadInputStream.java:161)
at com.mysql.jdbc.util.ReadAheadInputStream.read(ReadAheadInputStream.java:189)
at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:2537)
at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2990)
at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2979)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3520)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1990)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2151)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2625)
at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2119)
at com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:2281)
at grails.orm.HibernateCriteriaBuilder.invokeMethod(HibernateCriteriaBuilder.java:1167)
at indiaweatherstation.PopulatedataService$_crawlAWS_closure2.doCall(PopulatedataService.groovy:89)
at indiaweatherstation.PopulatedataService.crawlAWS(PopulatedataService.groovy:20)
at indiaweatherstation.PopulatedataService$crawlAWS.call(Unknown Source)
at indiaweatherstation.CrawlerController$_closure1.doCall(CrawlerController.groovy:16)
at indiaweatherstation.CrawlerController$_closure1.doCall(CrawlerController.groovy)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
Set the wait_timeout and interactive_timeout in mysql to a higher value

Strange Error using Hibernate with MySQL in Java over JDBC

I get the following exception after my program proceeding database oparations for around one minute:
WARN : org.hibernate.util.JDBCExceptionReporter - SQL Error: 0, SQLState: 08S01
ERROR: org.hibernate.util.JDBCExceptionReporter - Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
org.springframework.dao.DataAccessResourceFailureException: Cannot open connection; nested exception is org.hibernate.exception.JDBCConnectionException: Cannot open connection
at org.springframework.orm.hibernate3.SessionFactoryUtils.convertHibernateAccessException(SessionFactoryUtils.java:631)
at org.springframework.orm.jpa.vendor.HibernateJpaDialect.translateExceptionIfPossible(HibernateJpaDialect.java:104)
at org.springframework.orm.jpa.AbstractEntityManagerFactoryBean.translateExceptionIfPossible(AbstractEntityManagerFactoryBean.java:403)
at org.springframework.dao.support.ChainedPersistenceExceptionTranslator.translateExceptionIfPossible(ChainedPersistenceExceptionTranslator.java:58)
at org.springframework.dao.support.DataAccessUtils.translateIfNecessary(DataAccessUtils.java:213)
at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:163)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
at $Proxy36.getBankResultsForDate(Unknown Source)
at com.***.service.ScanResultDataServiceImpl.getBankResultsForDate(ScanResultDataServiceImpl.java:138)
at com.***.CalculateScore.main(CalculateScore.java:63)
Caused by: org.hibernate.exception.JDBCConnectionException: Cannot open connection
at org.hibernate.exception.SQLStateConverter.convert(SQLStateConverter.java:99)
at org.hibernate.exception.JDBCExceptionHelper.convert(JDBCExceptionHelper.java:66)
at org.hibernate.exception.JDBCExceptionHelper.convert(JDBCExceptionHelper.java:52)
at org.hibernate.jdbc.ConnectionManager.openConnection(ConnectionManager.java:449)
at org.hibernate.jdbc.ConnectionManager.getConnection(ConnectionManager.java:167)
at org.hibernate.jdbc.AbstractBatcher.prepareQueryStatement(AbstractBatcher.java:161)
at org.hibernate.loader.Loader.prepareQueryStatement(Loader.java:1596)
at org.hibernate.loader.Loader.doQuery(Loader.java:717)
at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:270)
at org.hibernate.loader.Loader.loadCollection(Loader.java:2062)
at org.hibernate.loader.collection.CollectionLoader.initialize(CollectionLoader.java:62)
at org.hibernate.persister.collection.AbstractCollectionPersister.initialize(AbstractCollectionPersister.java:628)
at org.hibernate.event.def.DefaultInitializeCollectionEventListener.onInitializeCollection(DefaultInitializeCollectionEventListener.java:83)
at org.hibernate.impl.SessionImpl.initializeCollection(SessionImpl.java:1853)
at org.hibernate.collection.AbstractPersistentCollection.forceInitialization(AbstractPersistentCollection.java:476)
at org.hibernate.engine.StatefulPersistenceContext.initializeNonLazyCollections(StatefulPersistenceContext.java:899)
at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:275)
at org.hibernate.loader.Loader.doList(Loader.java:2294)
at org.hibernate.loader.Loader.listIgnoreQueryCache(Loader.java:2172)
at org.hibernate.loader.Loader.list(Loader.java:2167)
at org.hibernate.loader.hql.QueryLoader.list(QueryLoader.java:448)
at org.hibernate.hql.ast.QueryTranslatorImpl.list(QueryTranslatorImpl.java:363)
at org.hibernate.engine.query.HQLQueryPlan.performList(HQLQueryPlan.java:196)
at org.hibernate.impl.SessionImpl.list(SessionImpl.java:1258)
at org.hibernate.impl.QueryImpl.list(QueryImpl.java:102)
at org.hibernate.ejb.QueryImpl.getResultList(QueryImpl.java:241)
at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.springframework.orm.jpa.SharedEntityManagerCreator$DeferredQueryInvocationHandler.invoke(SharedEntityManagerCreator.java:310)
at $Proxy34.getResultList(Unknown Source)
at org.springframework.data.jpa.repository.query.JpaQueryExecution$CollectionExecution.doExecute(JpaQueryExecution.java:77)
at org.springframework.data.jpa.repository.query.JpaQueryExecution.execute(JpaQueryExecution.java:54)
at org.springframework.data.jpa.repository.query.AbstractJpaQuery.doExecute(AbstractJpaQuery.java:93)
at org.springframework.data.jpa.repository.query.AbstractJpaQuery.execute(AbstractJpaQuery.java:83)
at org.springframework.data.repository.core.support.RepositoryFactorySupport$QueryExecutorMethodInterceptor.invoke(RepositoryFactorySupport.java:343)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:110)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:155)
... 5 more
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1116)
at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:344)
at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2333)
at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2370)
at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2154)
at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:792)
at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:47)
at sun.reflect.GeneratedConstructorAccessor37.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:381)
at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:305)
at java.sql.DriverManager.getConnection(DriverManager.java:579)
at java.sql.DriverManager.getConnection(DriverManager.java:190)
at org.springframework.jdbc.datasource.DriverManagerDataSource.getConnectionFromDriverManager(DriverManagerDataSource.java:173)
at org.springframework.jdbc.datasource.DriverManagerDataSource.getConnectionFromDriver(DriverManagerDataSource.java:164)
at org.springframework.jdbc.datasource.AbstractDriverBasedDataSource.getConnectionFromDriver(AbstractDriverBasedDataSource.java:149)
at org.springframework.jdbc.datasource.AbstractDriverBasedDataSource.getConnection(AbstractDriverBasedDataSource.java:119)
at org.hibernate.ejb.connection.InjectedDataSourceConnectionProvider.getConnection(InjectedDataSourceConnectionProvider.java:71)
at org.hibernate.jdbc.ConnectionManager.openConnection(ConnectionManager.java:446)
... 41 more
Caused by: java.net.NoRouteToHostException: Die angeforderte Adresse kann nicht zugewiesen werden
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
at java.net.Socket.connect(Socket.java:579)
at java.net.Socket.connect(Socket.java:528)
at java.net.Socket.<init>(Socket.java:425)
at java.net.Socket.<init>(Socket.java:241)
at com.mysql.jdbc.StandardSocketFactory.connect(StandardSocketFactory.java:257)
at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:294)
... 60 more
ERROR: com.***.CalculateScore - org.springframework.dao.DataAccessResourceFailureException: Cannot open connection; nested exception is org.hibernate.exception.JDBCConnectionException: Cannot open connection
My program is using Hibernate with MySQL in Java over JDBC.
The operation system is Ubuntu 12.04. A reinstallation of the system did't fix the problem.
Do someone have an idea what's going wrong?
This is due to NoRouteToHostException. That case for the network configuration state.
Signals that an error occurred while attempting to connect a socket to a remote address and port. Typically, the remote host cannot be reached because of an intervening firewall, or if an intermediate router is down.
Check How to avoid a NoRouteToHostException.

Categories

Resources