We are using camunda with RDS/MySql as DB. It works fine but then sometimes it says DB is closed and so throws the ProcessEngine Exception.
Here is what I understood from our config and logs:
We have 5active connections at any time in our pool (Specified in datasource config)
There was a scenario where it was closed.
We saw error like:
Request received Context path: /engine-rest Request received Path
Info: /user PathInfo: /user ExceptionHandler:
org.camunda.bpm.engine.ProcessEngineException: Process engine
persistence exception at
org.camunda.bpm.engine.impl.interceptor.CommandInvocationContext.rethrow(CommandInvocationContext.java:148)
at org.camunda.bpm.engine.impl.interceptor.CommandContext.close(CommandContext.java:173)
at org.camunda.bpm.engine.impl.interceptor.CommandContextInterceptor.execute(CommandContextInterceptor.java:113)
at org.camunda.bpm.engine.impl.interceptor.ProcessApplicationContextInterceptor.execute(ProcessApplicationContextInterceptor.java:66)
at org.camunda.bpm.engine.impl.interceptor.LogInterceptor.execute(LogInterceptor.java:30)
...... Caused by: org.apache.ibatis.exceptions.PersistenceException:
Error querying database. Cause:
com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException:
No operations allowed after connection closed. The error may exist in
org/camunda/bpm/engine/impl/mapping/entity/User.xml The error may
involve
org.camunda.bpm.engine.impl.persistence.entity.UserEntity.selectUserByQueryCriteria
The error occurred while executing a query SQL: select distinct RES.*
from ACT_ID_USER RES
order by RES.ID_ asc LIMIT ? OFFSET ? Cause:
com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException:
No operations allowed after connection closed.
Our tomcat props specify: minIdle = 5;
My best guess: Its closed on server but we are maintaining locally due to the above property.
Per tomcat doc (https://tomcat.apache.org/tomcat-8.0-doc/jdbc-pool.html):
testOnBorrow = true;
validationQuery = "select 1";
The two props should fix it as it validates the connection.
Question I am trying to figure out:
How can I repro this issue? Apart from keeping connection it idle for several hours, the scenario where this happened.
Does the AWS RDS server close the connection? If so, can we control it?
As mentioned by #Zelldon, the connection timeout can be reduced and then we could try it. It works as expected.
Just to be sure, I ran two instances of camunda, one with this fix and other without. Could see that the fix worked.
Regarding RDS, it does close the connection but I could not find any documentation on it.
Related
Using Rational Asset Manager 7.5.4.1 version with Oracle 11.g on WAS 8.5.5.13 I am getting following oracle error?
While we see the logs we get following SQL error code.
SQL Error Code is 17008 SQL State is :08003
Other log trace says like below.
java.nio.channels.ClosedByInterruptException
java.sql.SQLRecoverableException: Closed Connection
com.ibm.websphere.ce.cm.StaleConnectionException: Closed Connection
com.ibm.websphere.ce.cm.StaleConnectionException: IO Error: Socket read interrupted
What could be the reason?
The error usually is caused due to network issue or firewall issues.
To root cause and eliminate those:
1. You can check AWR and other connectivity reports
2. Ping works between app server and DB host
3. Required configurations for sqlnet.ora parameters time out are set correctly.
If none of them fix the issue, it worth to check jdbc jar file and its version.
In this case i think solution is to use jdbc6.jar and checking if there is any mismatch in the version of the jar that is presently used.
I have a working Spring Boot app (1.2) that uses Postgres. Today I am trying to switch it to Oracle, but when I try to connect I get an exception that says:
java.sql.SQLRecoverableException: IO Error: The Network Adapter could not establish the connection
And below that,
Caused by: java.net.ConnectException: Connection refused
So of course that looks like bad credentials, but I know they are good, and they are working in Oracle SQL Developer just fine. I'm baffled. Here are my properties file entries:
# Properties for Hibernate and Oracle
spring.datasource.driverClassName=oracle.jdbc.driver.OracleDriver
spring.datasource.url=jdbc:oracle:thin:#//earth-db-11:5121/stardev
spring.datasource.username=ops$abcdefg
spring.datasource.password=mypassword
spring.jpa.database-platform=org.hibernate.dialect.Oracle10gDialect
The only idea I have is that there is a $ in the user name, and I have tried escaping it and putting double quotes around it.
Any ideas?
Thanks...
UPDATE:
Many thanks to BonanzaOne, I did have the port number wrong. Correcting that results in a new error:
java.sql.SQLRecoverableException: Listener refused the connection with the following error:
ORA-12514, TNS:listener does not currently know of service requested in connect descriptor
I looked it up of course, but I don't follow what its telling me:
ORA-12514: TNS:listener does not currently know of service requested in connect descriptor
Cause: The listener received a request to establish a connection to a database
or other service. The connect descriptor received by the listener specified a
service name for a service (usually a database service) that either has not yet
dynamically registered with the listener or has not been statically configured
for the listener. This may be a temporary condition such as after the listener
has started, but before the database instance has registered with the listener.
Still, SQL Explorer connects fine.
That exception means that the Oracle listener is not up, or you are trying to connect to a listener that don't exist/not accessible.
My guess is that you trying the wrong port "5121". Oracle default port is 1521.
Can you try with that and see what happens?
From the FAQ there are basically two ways of composing your JDBC string URL:
Old syntax
jdbc:oracle:thin:#[HOST][:PORT]:SID
New syntax
jdbc:oracle:thin:#//[HOST][:PORT]/SERVICE
My guess is that you are using the wrong syntax-SID/Service name combination, in other words, you are using the new syntax that requires the SERVICE name, but you are using the SID name to do it.
Try this: jdbc:oracle:thin:#earth-db-11:1521:stardev
Or maybe find out the Service name and apply it to the new syntax that you are using, instead of the SID name.
I added setMaxActive(8) on org.apache.tomcat.jdbc.pool.PoolProperties. Every time the DB restarts, the application is unusable because the established connections remain. I get the following error:
org.postgresql.util.PSQLException: This connection has been closed
I've tried using some other settings on the pool to no avail...
Thank you for help!
Use the validationQuery property which will check if the connection is valid before returning the connection.
Ref: Tomcat 6 JDBC Connection Pool
This property is available on latest tomcat versions.
Look at this link:
Postgres connection has been closed error in Spring Boot
Very valid question and this problem is usually faced by many. The
exception generally occurs, when network connection is lost between
pool and database (most of the time due to restart). Looking at the
stack trace you have specified, it is quite clear that you are using
jdbc pool to get the connection. JDBC pool has options to fine-tune
various connection pool settings and log details about whats going on
inside pool.
You can refer to to detailed Apache documentation on pool
configuration to specify abandon timeout
Check for removeAbandoned, removeAbandonedTimeout, logAbandoned parameters
Additionally you can make use of additional properties to further
tighten the validation
Use testXXX and validationQuery for connection validity.
My own $0.02: use these two parameters:
validationQuery=<TEST SQL>
testOnBorrow=true
I am getting the Error in Production as per below. Can any one give some input to eliminate this error.
java.sql.SQLException: Closed Connection
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:113)
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:147)
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:209)
at oracle.jdbc.driver.OracleStatement.ensureOpen(OracleStatement.java:3550)
at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3396)
at oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3460)
at com.IBS.trade.order.netPortfolio.getClientPortfolioDetails(netPortfolio.java:556)
at core.ClientThreadInteractive.getNetPortFolioData(ClientThreadInteractive.java:14403)
at core.ClientThreadInteractive.netPortfolioSingle(ClientThreadInteractive.java:14370)
at core.ClientThreadInteractive.run(ClientThreadInteractive.java:1895)
Track shows that your connection has been closed when you are trying to use. There are few possibilities depending on your connection management.
1- Your invoked close().
2- You exceed the maximum idle timeout so database has closed your connection.
Basically it depends how you manage your connections if using per thread then may be possible that somewhere it closed during your thread that wasn't working.
-- Some details about your connection management can explain it better
Somewhere you are closing connection.
con.close();
And after that trying to access the database.
Simply restarting the Application worked for me. This error is due to unavailability of resources or unable to connect to DB for some reason in JDBC.
Today I managed to recreate the farms with Scalr.net and apparently after a few times restarting tomcat and fixing issues, I get this error once again. The thing is I was using MySQL with a clean install on the entire server, that includes Java 6.1_24, Tomcat 5.5.33, Sakai 2.7.1. The issue I keep running into is user denied when the fact that I have this user in the MySQL Instance, as well giving it complete remote access with sakai#% and even this is not working when it was working about an hour ago since this post was made.
... Continued from above log, everything before logs just fine
2011-03-31 18:31:14,120 WARN main org.springframework.jdbc.datasource.LazyConnectionDataSourceProxy - Could not retrieve default auto-commit and transaction isolation settings
org.apache.commons.dbcp.SQLNestedException: Error preloading the connection pool
... continued over 400+ lines...
Here is another error in regards to the access denied error...
2011-03-31 18:31:16,854 WARN main org.hibernate.cfg.SettingsFactory - Could not obtain connection metadata
java.sql.SQLException: Access denied for user 'sakai'#'ec2-50-17-184-70.compute-1.amazonaws.com' (using password: YES)
.... continued....
I now get this error whenever I startup, this is with a fresh install of tomcat/sakai
SEVERE: Unable to set localhost. This prevents creation of a GUID. Cause was: ec2-72-44-56-167.compute-1.amazonaws.com: ec2-72-44-56-167.compute-1.amazonaws.com
java.net.UnknownHostException: ec2-72-44-56-167.compute-1.amazonaws.com: ec2-72-44-56-167.compute-1.amazonaws.com
(This most recent error (Localhost) was simply fixed by restarting the amazon aws instance. Thankfully) Although I keep getting the same errors even with a fresh install... Almost as if the information is being refreshed from a cache... Or something
As with the last question you posted on this topic, the error message seems very clear: the user 'sakai'#... does not have access to login to the database you have set it up to. I recommend taking a look at the Mysql documentation to understand how to administer the user accounts to find out if you've missed a setting somewhere to allow this account to have access.
I believe I may have figured out how to fix this problem. It has nothing to do with mysql, or the apache server itself. It has to do with the failure of Scalr.net not Initializing the IP or something of that sort. After doing some research I found some issues with the HostInit issues such as....
Cannot deliver message 'HostInit' (message_id: af9dcfdb-a09e-4971-bdb7-7871b3f7e21c) via REST to server '50.17.135.98' (server_id: e49cfec9-5bcb-44d1-bbc5-fde32450fc89). Error: 0 Timeout was reached; connect() timed out! (http://50.17.135.98:8013/control)
Cannot deliver message 'BlockDeviceAttached' (message_id: a153d83f-3d96-4d53-920a-ccb80701675a) via REST to server '50.17.135.98' (server_id: e49cfec9-5bcb-44d1-bbc5-fde32450fc89). Error: 0 Timeout was reached; connect() timed out! (http://50.17.135.98:8013/control)
Cannot deliver message 'HostUp' (message_id: 1adde27c-9982-4551-b266-c3c432d1dd44) via REST to server '50.17.135.98' (server_id: e49cfec9-5bcb-44d1-bbc5-fde32450fc89). Error: 0 Timeout was reached; connect() timed out! (http://50.17.135.98:8013/control)
Cannot deliver message 'HostInit' (message_id: f1aa4b14-ef57-4361-ae56-87702d674b11) via REST to server '50.17.135.98' (server_id: e49cfec9-5bcb-44d1-bbc5-fde32450fc89). Error: 0 Timeout was reached; connect() timed out! (http://50.17.135.98:8013/control)
So what I did was I made a snapshot image of the apache server/mysql etc. and terminated them allowing the recreation of the instance and this managed to solve the problem in one manner.