Debugging database connection leak in concurrent environment - java

I'm currently working on a project which was originally not build for high load.
My problem atm is that at some point during the stress test (30 users) the application seems to "get stuck" and when it releases it spits out a lot of exceptions. Unable to get managed connection for [MY_DS]
When I run only one user, it works like a charm so it has something to do with the concurrency.
I also checked if there were any unclosed DB connections at the end of one run and there were none, so at normal usage, there are no connection leaks.
My suspicion goes out to my open and close methods (because they are static). Here are the methods:
public static Connection getConnection() {
if (logger.isDebugEnabled()) logger.debugLog("getConnection()");
try {
return DSUtils.getDefaultDataSource().getConnection();
} catch (SQLException se) {
logger.errorLog("SQLException", se);
throw new ApplicationRuntimeException(MessageCodesConstants.ERROR_SQL_EXCEPTION, se);
}
}
public static void closeConnection(Connection con) {
if (logger.isDebugEnabled()) logger.debugLog("closeConnection");
try {
if (con != null) {
con.close();
}
} catch (SQLException se) {
logger.warnLog("SQLException while closing connection");
}
}
It is an EE application running on JBoss EAP 6.2.0 backed-up by a SQL Server 2008.
Can somebody point me in the right direction to find out where the solution can be found?

Related

ORA-01017 error when attempting to connect to Oracle XE from servlet

I'm trying to write a servlet application for learning purposes that connects to an Oracle database, queries some data and then prints it to the browser. Simple!
However, I'm experiencing an ORA-01017: invalid username/password when attempting to connect to a locally installed and running version of Oracle XE (19c). For the sake of testing the connection, I'm connection with the system user. Here's my code:
// http://localhost:8080/demo/
public class DemoServ extends HttpServlet {
public void doGet(HttpServletRequest req,HttpServletResponse res)
throws ServletException,IOException {
try{
Class.forName("oracle.jdbc.driver.OracleDriver");
Connection con = DriverManager.getConnection("jdbc:oracle:thin:#localhost:1523:xe", "system", "SYSTEM");
con.close();
} catch (Exception e) {
e.printStackTrace();
}
}
}
The user that I'm using absolutely does exist, and I can connect using SQL Developer without issue.
I would be willing to put this down to my own ignorance of Java, but if I run the following code independently of any servlet, I can connect and execute the sample query!
public class DataReader {
public static void main (String [] args) {
try {
Class.forName("oracle.jdbc.driver.OracleDriver");
Connection con = DriverManager.getConnection("jdbc:oracle:thin:#localhost:1523:xe", "system", "SYSTEM");
Statement statement = con.createStatement();
ResultSet rs = statement.executeQuery("SELECT count(*) num FROM dual");
if (rs.next()) {
int i = rs.getInt("num"); // get first column returned
System.out.println("number: " + i);
}
rs.close();
statement.close();
con.close();
}
catch (Exception e) {
System.out.println(e);
}
}
}
I've been searching Google for solutions to this, but I have been unable to find a solution, so here I am.
I'm working on Windows 10, using Java 1.8 and Oracle 19c XE.
Any help would be great. Thanks
Okay, I finally go this to work, but I cannot explain why.
Oracle 19c is case sensitive, which I knew. I attempted to disable this, but as it's a depreciated feature, this seemed expeditious. I altered the password for the system user to be "system", and I can connect successfully. "SYSTEM" as a password continues to fail.
What strikes me as odd about this is that I'm sure that I tried to use the "system" (lowercase) password in the past. :(
Anyway, I probably was doing something daft, but at least I'm got over the hump. Phew!
Thank you to everyone!!

Can't connect to oracle db most of the times

Good evening everyone, I'm getting confused with some troubles I'm incurring opening a connection with my db.
The code I've listed down, works fine establishing the connection at work, but at home it's working funny. I mean, at first I used my ip instead of localhost and it was working fine. After restarting my computer, I got the " The Network Adapter could not establish the connection" error, and after 1 hour of efforts and simply putting localhost instead of my ip, it magically started working again, but guess what. Just after another restart, again with the same error.
I'm using eclipse neon the latest version, tomcat 9.0 on windows 7 64 bit and
I have bitdefender as antivirus.
Can someone help me? I don't understand what's happening.
public class Connessione {
private String driver="oracle.jdbc.OracleDriver";
private static Connection connessione;
private final String url="jdbc:oracle:thin:#localhost:1521:ORCL";
private final String userName="xxxxx";
private final String password="xxxxx";
public Connection getConnessione() {
return connessione;
}
public Connessione() {
try {
Class.forName(driver);
connessione= DriverManager.getConnection(url,userName,password);
} catch (ClassNotFoundException | SQLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
public boolean chiudiConnessione(){
try {
connessione.close();
return true;
} catch (SQLException e) {
return false;
}
}

Connection Closed even when the connection is closed below the code

I have a basic code snippet below but it is not working.What may be the problem with it.
public List<String> getStores() throws SQLException{
List<String> store_id=new ArrayList<>();
String query="select distinct(store_id) from stores";
Connection con=ConnectDatabase.getDb2ConObj();
Statement stmt=con.createStatement();
java.sql.ResultSet rsResultSet=stmt.executeQuery(query);
while(rsResultSet.next()){
store_id.add(rsResultSet.getString(1));
}
con.close();
return store_id;
}
It is throwing the below exception
com.mysql.jdbc.exceptions.MySQLNonTransientConnectionException: No operations allowed after connection closed.
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:888)
at com.mysql.jdbc.Connection.checkClosed(Connection.java:1931)
at com.mysql.jdbc.Connection.createStatement(Connection.java:3087)
at com.mysql.jdbc.Connection.createStatement(Connection.java:3069)
at com.dao.StoreDao.getStores(StoreDao.java:52)
at org.apache.jsp.adminViewAllStore_jsp._jspService(adminViewAllStore_jsp.java:119)
The code for ConnectDatabse is
public class ConnectDatabase {
static Connection con=null;
static String connectionString="jdbc:mysql://localhost:3306/ayurveda";
static String username="root";
static String password="";
public static Connection getDb2ConObj(){
if(con==null){
try{
Class.forName("com.mysql.jdbc.Driver");
con=DriverManager.getConnection(connectionString,username,password);
}
catch(ClassNotFoundException | SQLException e)
{
System.out.println("Connect initialized with error"+e.getMessage());
e.printStackTrace();
}
}
return con;
}
I cannot understand the reason for the same.What may be the problem.Since I am closing the connection after I am done with it.
It worked after I enclosed it in a try catch finally block.Changed the code as given below
public List<String> getStores() throws SQLException{
List<String> store_id=new ArrayList<>();
Connection con=ConnectDatabase.getDb2ConObj();
try{
String query="select distinct(store_id) from stores";
Statement stmt=con.createStatement();
java.sql.ResultSet rsResultSet=stmt.executeQuery(query);
while(rsResultSet.next()){
store_id.add(rsResultSet.getString(1));
}
}catch(Exception e){
}finally{
con.close();
}
return store_id;
}
Thanks.
You can use this type of code...for solving your problem
public void actionPerformed(ActionEvent ae)
{
String u=t1.getText();
String p=t2.getText();
if(ae.getSource()==b1)
{
try{
Class.forName("sun.jdbc.odbc.JdbcOdbcDriver");
Connection con=DriverManager.getConnection("jdbc:odbc:newdsn");
String stp="SELECT * FROM reg";
Statement sa=con.createStatement();
rs=sa.executeQuery(stp);
while(rs.next())
{
String du=rs.getString(2);
String dp=rs.getString(3);
if(du.equals(u)&&dp.equals(p))
{
a=0;
break;
}else{ a=1;}
}
if(a==0){
JOptionPane.showMessageDialog(this,"LOGIN PAGE","Login is successful",1);
}
if(a==1){
JOptionPane.showMessageDialog(this,"LOGIN PAGE","Login is not successful",1);
}}
catch(Exception e){}
}}
if even the it is throwing exception then you check the system 32 bit or 64 bit..you should try if 64bit then please make your dsn in 32 bit and anduse ms access 2002-2003 version
then you get tour solution .....thank u
Use Java 7 -The try-with-resources Statement
According to the oracle documentation, you can combine a try-with-resources block with a regular try block
The typical Java application manipulates several types of resources such as files, streams, sockets, and database connections. Such resources must be handled with great care, because they acquire system resources for their operations. Thus, you need to ensure that they get freed even in case of errors.
Indeed, incorrect resource management is a common source of failures in production applications, with the usual pitfalls being database connections and file descriptors remaining opened after an exception has occurred somewhere else in the code. This leads to application servers being frequently restarted when resource exhaustion occurs, because operating systems and server applications generally have an upper-bound limit for resources.
sample code:
try(Connection con = getConnection()) {
...
}
Read more Java 7 Automatic Resource Management JDBC
Close Statement and ResultSet as well.
Don't load driver class every time when connection is needed. Just load it once in static initialization block.
static {
try {
Class.forName("com.mysql.jdbc.Driver");
} catch (ClassNotFoundException e) {
e.printStackTrace();
}
}
I suggest you to use JNDI and DataSource to keep username and password outside the Java code to make it more manageable. Keep the database configuration in a separate xml/properties file instead of hard-coding in Java file.
See Java Tutorial on Connecting with DataSource Objects
I have already posted a nice ConnectionUtil class to manage all the connections in a single class for whole application.

Singleton in JSP, how to properly tidy up on close?

I am just getting started with jsp and my question is this - when I have a singleton class, how do I tidy up after it?
In particular:
public class DBConnection {
private static Connection connection = null;
private static Statement statement = null;
public static ResultSet executeQuery(String query){
if (connection == null) { /*initConnection*/ }
if (statement == null) { /*initStatement*/ }
// do some stuff
}
}
Now, I use this class in several pages to get results from jdbc. However, I need to eventually call statement.close(); and connection.close(); - when should I call those?
I am using singleton, because it felt wrong to call for connection to a database over and over whenever I needed to make a query.
The Connection must be closed always, and after you have executed all your database statements for the desired operations. Two examples:
Case 1: You must show a list of products to user filtered by criteria from database. Solution: get a connection, retrieve a list of products using the filter criteria, close the connection.
Case 2: The client selects some of these products and updates the minimum stock to get an alert and restock them. Solution: get a connection, update all the products, close the connection.
Based on these cases, we can learn lot of things:
You can execute more than a single statement while having/maintaining a single connection open.
The connection should live only in the block where it is used. It should not live before or after that.
Both cases can happen at the same time since they are in a multi threaded environment. So, a single database connection must not be available to be used by two threads at the same time, in order to avoid result problems. For example, user A searches the products that are in category Foo and user B searches the products that are in category Bar, you don't want to show the products in category Bar to user A.
From last sentence, each database operation ((or group of similar operations like Case 2) should be handled in an atomic operation. To assure this, the connection must not be stored in a singleton object, instead it must be live only in the method being used.
In consequence:
Do not declare the Connection nor the Statement nor the ResultSet nor other JDBC resource as static. It will simply fail. Instead, declare only the Connection as field of your DBConnection class. Let each method decide to handle each Statement (or PreparedStatement) and ResultSet and specific JDBC resources.
Since you must close the connection after its usage, then add two more methods: void open() and void close(). These methods will handle the database connection retrieval and closing that connection.
Additional, since the DBConnection looks like a wrapper class for Connection class and database connection operations, I would recommend to have at least three more methods: void setAutoCommit(boolean autoCommit), void commit() and void rollback(). These methods will be plain wrappers for Connection#setAutoCommit Connection#close and Connection#rollback respectively.
Then you can use the class in this way:
public List<Product> getProducts(String categoryName) {
String sql = "SELECT id, name FROM Product WHERE categoryName = ?";
List<Product> productList = new ArrayList<Product>();
DBConnection dbConnection = new DBConnection();
try {
dbConnection.open();
ResultSet resultSet = dbConnection.executeSelect(sql, categoryName); //execute select and apply parameters
//fill productList...
} catch (Exception e) {
//always handle your exceptions
...
} finally {
//don't forget to also close other resources here like ResultSet...
//always close the connection
dbConnection.close();
}
}
Note that in this example the PreparedStatement is not in the getProducts method, it will be a local variable of the executeSelect method.
Additional notes:
When working in an application server, you should not open connections naively e.g. using Class.forName("..."), instead use a database connection pool. You can roll on some database connection pooling libraries like C3P0 as explained here: How to establish a connection pool in JDBC?. Or configure one in your application server, as I explain here: Is it a good idea to put jdbc connection code in servlet class?
If this is for learning purposes, then roll on your own classes to handle the communication with your database. In real world applications, this is not recommended (doesn't mean you should not do it). Instead, use a database connectivity framework like ORMs e.g. JPA (Java official ORM framework) or Hibernate; there are no ORM frameworks that handles database communication like Spring JDBC and MyBatis. The choice is yours.
More info:
Should a database connection stay open all the time or only be opened when needed?
How do servlets work? Instantiation, sessions, shared variables and multithreading. Not directly related to your question, but it will help you understand why to not maintain state in resources that are used in multithreaded environments.
Define connection resource in mywebapp/META-INF/context.xml file
<Resource name="jdbc/mydb" auth="Container" type="javax.sql.DataSource"
maxActive="10" maxIdle="2" maxWait="20000"
driverClassName="com.mysql.jdbc.Driver"
username="myuser" password="mypwd"
url="jdbc:mysql://localhost:3306/mydb?useUnicode=true&characterEncoding=utf8"
validationQuery="SELECT 1" />
Create DB.java helper class to minimize code in other parts of app
import java.sql.*;
import javax.sql.DataSource;
import javax.naming.Context;
import javax.naming.InitialContext;
public class DB {
public static Connection createConnection() throws SQLException {
try {
Context ctx = new InitialContext();
DataSource ds = (DataSource)ctx.lookup("java:comp/env/jdbc/mydb");
return ds.getConnection();
} catch (SQLException ex) {
throw ex;
} catch (Exception ex) {
SQLException sqex = new SQLException(ex.getMessage());
sqex.initCause(ex);
throw sqex;
}
}
public static void close(ResultSet rs, Statement stmt, Connection conn) {
if (rs != null) try { rs.close(); } catch (Exception e) { }
if (stmt != null) try { stmt.close(); } catch (Exception e) { }
if (conn != null) try { conn.close(); } catch (Exception e) { }
}
public static void close(ResultSet rs, boolean closeStmtAndConn) {
if (rs==null) return;
try {
Statement stmt = rs.getStatement();
close(rs, stmt, stmt!=null ? stmt.getConnection() : null);
} catch (Exception ex) { }
}
}
And somewhere in your app DAO code use DB helper.
public List<MyBean> getBeans() throws SQLException {
List<MyBean> list = new ArrayList<MyBean>();
ResultSet rs=null;
try {
Connection con = DB.createConnection();
String sql = "Select * from beantable where typeid=?";
PreparedStatement stmt = con.prepareStatement(sql, Statement.NO_GENERATED_KEYS);
stmt.setInt(1, 101);
rs = stmt.executeQuery();
while(rs.next()
list.add( createBean(rs) );
} finally {
DB.close(rs, true); // or DB.close(rs, stmt, conn);
}
return list;
}
private MyBean createBean(ResultSet rs) throws SQLException {
MyBean bean = new MyBean();
bean.setId( rs.getLong("id") );
bean.setName( rs.getString("name" );
bean.setTypeId( rs.getInt("typeid") );
return bean;
}
I would add two methods to the class:
public static void open() throws SomeException;
public static void close() throws SomeException;
then your calling code looks something like this{
try {
DBConnection.open();
... code to use the connection one or more times ...
} finally {
DBConnection.close();
}
Wrap all your database calls inside that and it will take care of closing whether there is an exception thrown or not.
Of course, this isn't much different than having a regular class, which I might recommend:
try {
DBConnection conn = new DBConnection();
conn.open();
... all the code to use the database (but you pass 'conn' around) ...
} finally {
conn.close();
}
And you might want to look at the java.lang.AutoCloseable and java.io.Closeable to see if that helps you.
2
If you are keeping it open across page loads, there isn't any place to put the try ... finally stuff so you can open it and close it when the servlet closes or the server closes or something like that.
If you are going to leave it open, you need to make sure and add code to verify it doesn't close when you aren't looking. A short network glitch, for example, could close it down. In that case, you need to reopen it when it gets closed. Otherwise, all database access from that point will fail.
You might want to look into the concept of a DataBase Pool. Apache has one -- DBCP. Tomcat has its own that's quite good. Other containers, like JBOSS, WebSphere, WebLogic all have them. There's a couple that can be used with the Spring Framework. What it does is manage one or more database connections. Your code asks it for one and it returns an open one, unless none is available and then it opens one and returns it. You call close when your code gets through with it but it doesn't really close the connection, it just returns it to the pool.
You can usually configure the pool to check for shut down connections and reopen if needed.

H2 on Tomcat shutdown hanging, memory leak when using Tomcat Datasource

This is related to these 2 posts:
What is the proper way to close H2?
Tomcat doesn't stop. How can I debug this?
Basically H2 keeps a lock on the database, even when all connections are closed, and so when stopping Tomcat it hangs waiting on a thread, the process is still running.
The only way I managed to get H2 to not lock the database is by issueing the statement SHUTDOWN IMMEDIATELY command (the vanilla or compact did not release the lock).
This is performed in my ServletContextListener class in the contextDestroyed like this (I have omitted comments and log lines):
ServletContext ctx = servletContextEvent.getServletContext();
DataSource closeDS = databaseConnection.getDatasource();
Connection closeConn = null;
PreparedStatement closePS = null;
try {
closeConn = closeDS.getConnection();
closePS = closeConn.prepareStatement("SHUTDOWN IMMEDIATELY");
closePS.execute();
} catch (Exception ex) {
} finally {
if (closePS != null) {
try { closePS.close(); } catch (SQLException ex) {}
}
if (closeConn != null) {
try { closeConn.close(); } catch (SQLException ex) {}
}
}
try {
databaseConnection.close();
databaseConnection = null;
ctx.setAttribute("databaseConnection", null);
} catch(Exception e) {
}
Enumeration<Driver> drivers = DriverManager.getDrivers();
while (drivers.hasMoreElements()) {
Driver driver = drivers.nextElement();
try {
DriverManager.deregisterDriver(driver);
} catch (Exception e) {
}
}
Now the lock is released Tomcat stops (although I still get the severe memory leak messages in the logs) but now I receive also a number of error stacks in the logs thus:
INFO: Illegal access: this web application instance has been stopped already.
Could not load java.lang.ThreadGroup.
The eventual following stack trace is caused by an error thrown
for debugging purposes as well as to attempt to terminate
the thread which caused the illegal access, and has no functional impact.
java.lang.IllegalStateException
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1531)
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1491)
at org.h2.engine.DatabaseCloser.reset(DatabaseCloser.java:43)
at org.h2.engine.Database.close(Database.java:1155)
at org.h2.engine.DatabaseCloser.run(DatabaseCloser.java:80)
10-sep-2013 13:31:41 org.apache.catalina.loader.WebappClassLoader loadClass
The question is: how can I shut down the database without causing illegal state exceptions. Is there something wrong in my code to call the shutdown command?
Why is this such an issue with H2? I do not have this issue with JBoss or Websphere where the application also runs using datasources provided by the container.

Categories

Resources