I am just getting started with jsp and my question is this - when I have a singleton class, how do I tidy up after it?
In particular:
public class DBConnection {
private static Connection connection = null;
private static Statement statement = null;
public static ResultSet executeQuery(String query){
if (connection == null) { /*initConnection*/ }
if (statement == null) { /*initStatement*/ }
// do some stuff
}
}
Now, I use this class in several pages to get results from jdbc. However, I need to eventually call statement.close(); and connection.close(); - when should I call those?
I am using singleton, because it felt wrong to call for connection to a database over and over whenever I needed to make a query.
The Connection must be closed always, and after you have executed all your database statements for the desired operations. Two examples:
Case 1: You must show a list of products to user filtered by criteria from database. Solution: get a connection, retrieve a list of products using the filter criteria, close the connection.
Case 2: The client selects some of these products and updates the minimum stock to get an alert and restock them. Solution: get a connection, update all the products, close the connection.
Based on these cases, we can learn lot of things:
You can execute more than a single statement while having/maintaining a single connection open.
The connection should live only in the block where it is used. It should not live before or after that.
Both cases can happen at the same time since they are in a multi threaded environment. So, a single database connection must not be available to be used by two threads at the same time, in order to avoid result problems. For example, user A searches the products that are in category Foo and user B searches the products that are in category Bar, you don't want to show the products in category Bar to user A.
From last sentence, each database operation ((or group of similar operations like Case 2) should be handled in an atomic operation. To assure this, the connection must not be stored in a singleton object, instead it must be live only in the method being used.
In consequence:
Do not declare the Connection nor the Statement nor the ResultSet nor other JDBC resource as static. It will simply fail. Instead, declare only the Connection as field of your DBConnection class. Let each method decide to handle each Statement (or PreparedStatement) and ResultSet and specific JDBC resources.
Since you must close the connection after its usage, then add two more methods: void open() and void close(). These methods will handle the database connection retrieval and closing that connection.
Additional, since the DBConnection looks like a wrapper class for Connection class and database connection operations, I would recommend to have at least three more methods: void setAutoCommit(boolean autoCommit), void commit() and void rollback(). These methods will be plain wrappers for Connection#setAutoCommit Connection#close and Connection#rollback respectively.
Then you can use the class in this way:
public List<Product> getProducts(String categoryName) {
String sql = "SELECT id, name FROM Product WHERE categoryName = ?";
List<Product> productList = new ArrayList<Product>();
DBConnection dbConnection = new DBConnection();
try {
dbConnection.open();
ResultSet resultSet = dbConnection.executeSelect(sql, categoryName); //execute select and apply parameters
//fill productList...
} catch (Exception e) {
//always handle your exceptions
...
} finally {
//don't forget to also close other resources here like ResultSet...
//always close the connection
dbConnection.close();
}
}
Note that in this example the PreparedStatement is not in the getProducts method, it will be a local variable of the executeSelect method.
Additional notes:
When working in an application server, you should not open connections naively e.g. using Class.forName("..."), instead use a database connection pool. You can roll on some database connection pooling libraries like C3P0 as explained here: How to establish a connection pool in JDBC?. Or configure one in your application server, as I explain here: Is it a good idea to put jdbc connection code in servlet class?
If this is for learning purposes, then roll on your own classes to handle the communication with your database. In real world applications, this is not recommended (doesn't mean you should not do it). Instead, use a database connectivity framework like ORMs e.g. JPA (Java official ORM framework) or Hibernate; there are no ORM frameworks that handles database communication like Spring JDBC and MyBatis. The choice is yours.
More info:
Should a database connection stay open all the time or only be opened when needed?
How do servlets work? Instantiation, sessions, shared variables and multithreading. Not directly related to your question, but it will help you understand why to not maintain state in resources that are used in multithreaded environments.
Define connection resource in mywebapp/META-INF/context.xml file
<Resource name="jdbc/mydb" auth="Container" type="javax.sql.DataSource"
maxActive="10" maxIdle="2" maxWait="20000"
driverClassName="com.mysql.jdbc.Driver"
username="myuser" password="mypwd"
url="jdbc:mysql://localhost:3306/mydb?useUnicode=true&characterEncoding=utf8"
validationQuery="SELECT 1" />
Create DB.java helper class to minimize code in other parts of app
import java.sql.*;
import javax.sql.DataSource;
import javax.naming.Context;
import javax.naming.InitialContext;
public class DB {
public static Connection createConnection() throws SQLException {
try {
Context ctx = new InitialContext();
DataSource ds = (DataSource)ctx.lookup("java:comp/env/jdbc/mydb");
return ds.getConnection();
} catch (SQLException ex) {
throw ex;
} catch (Exception ex) {
SQLException sqex = new SQLException(ex.getMessage());
sqex.initCause(ex);
throw sqex;
}
}
public static void close(ResultSet rs, Statement stmt, Connection conn) {
if (rs != null) try { rs.close(); } catch (Exception e) { }
if (stmt != null) try { stmt.close(); } catch (Exception e) { }
if (conn != null) try { conn.close(); } catch (Exception e) { }
}
public static void close(ResultSet rs, boolean closeStmtAndConn) {
if (rs==null) return;
try {
Statement stmt = rs.getStatement();
close(rs, stmt, stmt!=null ? stmt.getConnection() : null);
} catch (Exception ex) { }
}
}
And somewhere in your app DAO code use DB helper.
public List<MyBean> getBeans() throws SQLException {
List<MyBean> list = new ArrayList<MyBean>();
ResultSet rs=null;
try {
Connection con = DB.createConnection();
String sql = "Select * from beantable where typeid=?";
PreparedStatement stmt = con.prepareStatement(sql, Statement.NO_GENERATED_KEYS);
stmt.setInt(1, 101);
rs = stmt.executeQuery();
while(rs.next()
list.add( createBean(rs) );
} finally {
DB.close(rs, true); // or DB.close(rs, stmt, conn);
}
return list;
}
private MyBean createBean(ResultSet rs) throws SQLException {
MyBean bean = new MyBean();
bean.setId( rs.getLong("id") );
bean.setName( rs.getString("name" );
bean.setTypeId( rs.getInt("typeid") );
return bean;
}
I would add two methods to the class:
public static void open() throws SomeException;
public static void close() throws SomeException;
then your calling code looks something like this{
try {
DBConnection.open();
... code to use the connection one or more times ...
} finally {
DBConnection.close();
}
Wrap all your database calls inside that and it will take care of closing whether there is an exception thrown or not.
Of course, this isn't much different than having a regular class, which I might recommend:
try {
DBConnection conn = new DBConnection();
conn.open();
... all the code to use the database (but you pass 'conn' around) ...
} finally {
conn.close();
}
And you might want to look at the java.lang.AutoCloseable and java.io.Closeable to see if that helps you.
2
If you are keeping it open across page loads, there isn't any place to put the try ... finally stuff so you can open it and close it when the servlet closes or the server closes or something like that.
If you are going to leave it open, you need to make sure and add code to verify it doesn't close when you aren't looking. A short network glitch, for example, could close it down. In that case, you need to reopen it when it gets closed. Otherwise, all database access from that point will fail.
You might want to look into the concept of a DataBase Pool. Apache has one -- DBCP. Tomcat has its own that's quite good. Other containers, like JBOSS, WebSphere, WebLogic all have them. There's a couple that can be used with the Spring Framework. What it does is manage one or more database connections. Your code asks it for one and it returns an open one, unless none is available and then it opens one and returns it. You call close when your code gets through with it but it doesn't really close the connection, it just returns it to the pool.
You can usually configure the pool to check for shut down connections and reopen if needed.
Related
I am developing a Spring Boot web (REST) application where I need to serve many requests. Therefore I wanted my application to be able to handle requests concurrently. Since Spring Boot REST-Services are out-of-the-box concurrently usable, I only need to make the (PostgreSQL) database access concurrently accessible. For that I am using the HikariCP data source.
Since a lot of my statements are prepared statements, I collected them in one method where I call pstmt = connection.prepareStatement("SQLCODE"); once for every statemment. Those prepared statements are then used in various methods when user interaction from the REST service is processed.
Now, when I use the HikariCP I can't do that anymore, can I?
When I prepare a statement, this statement is bound to one connection. If I then try to access it concurrently, I can't because the connection is not shared.
Am I missing something? How can I solve this? Do I need to retrieve a connection from the pool, prepare the statement locally, execute my query, and close the connection? If so, what's the point of using a prepared statement then (other than preventing SQL injection)?
I know that the statements are cached by on the PostreSQL side. So would it be a good idea to keep the method where all prepared statements are prepared? To sent them to the database cache. And then just creating locally the same statements again. That way, one might still leverage the caching possibilities of the database. But on the other hand it would be really ugly code.
Im am using Spring: 5.3.10, Java: 11, PostgreSQL: 14.0
#RestController
public class RESTController {
/** The database controller. */
private DBController dbc;
/** The data source object serving as a connection pool. */
private HikariDataSource ds;
/** The logger object for this class. */
private static Logger logger = LoggerFactory.getLogger(RESTController.class);
public RESTController(DBController dbc, Config config) {
this.dbc = dbc;
// Create the database
if (!this.dbc.createDB(config)) {
logger.error("Couldn't create the database. The service will now exit.");
Runtime.getRuntime().halt(1);
}
// Create a connection pool
ds = new HikariDataSource();
ds.setJdbcUrl(config.getUrl());
ds.setUsername(config.getUser());
ds.setPassword(config.getPassword());
ds.addDataSourceProperty("cachePrepStmts", "true");
ds.addDataSourceProperty("prepStmtCacheSize", "250");
ds.addDataSourceProperty("prepStmtCacheSqlLimit", "2048");
// Create the necessary tables
if (!this.dbc.createTables(ds)) {
logger.error("Couldn't create the tables. The service will now exit.");
ds.close();
Runtime.getRuntime().halt(1);
}
// Prepare SQL statements
if (!this.dbc.prepareStatements(ds)) {
logger.error("Couldn't prepare the SQL statements. The service will now exit.");
ds.close();
Runtime.getRuntime().halt(1);
}
}
#PostMapping("/ID")
public ResponseEntity<String> createNewDomain(#RequestParam(name = "name", required = true) String name) {
// Do stuff ...
}
// [...]
}
#Component
public class DBController {
/** The logger object for this class. */
private static Logger logger = LoggerFactory.getLogger(DBController.class);
// Prepared Statements
private PreparedStatement stmt1, stmt2, stmt3;
public boolean prepareStatements(HikariDataSource ds) {
try {
// Get connection from the pool
Connection c = ds.getConnection();
// Prepare all the statements
stmt1 = c.prepareStatement("SQLCODE");
stmt2 = c.prepareStatement("SQLCODE1");
stmt2 = c.prepareStatement("SQLCODE1");
// [...]
} catch (SQLException e) {
logger.debug("Could not prepare the SQL statements: " + e.getMessage());
return false;
}
logger.debug("Successfully prepared the SQL statements.");
return true;
}
public boolean m1(int i) {
stmt1.setInt(i);
ResultSet rs = stmt1.executeQuery();
}
public boolean m2(int j) {
stmt1.setInt(j);
ResultSet rs = stmt1.executeQuery();
}
public boolean m3(String a) {
stmt2.setString(a);
ResultSet rs = stmt2.executeQuery();
}
// [...]
}
Thanks in advance.
pleae read the part Statement Cache at https://github.com/brettwooldridge/HikariCP
Many connection pools, including Apache DBCP, Vibur, c3p0 and others
offer PreparedStatement caching. HikariCP does not. Why?
So it does not cache. and if you read explanaition maybe you decide you don't need it to.
I need to use a connection pool in a standalone (as in non-web) Java application. Where I work, we are not allowed to use APIs without going through layers of security, and the job needs to be completed soon. Below is my attempt at creating this connection pool.
I have unit tested this code and tested it within the context of the overall application a hundred times and in all cases the tests passed with zero errors, and in addition the performance of each run is just under three thousand times faster than a simple connect, retrieve data, disconnect in serial approach; however, I still have nagging concerns that there could be issues with this approach that I simply haven't unearthed yet. I would appreciate any advice anyone has concerning the below code. This is my first post on this site; please let me know if I've made any errors in etiquette. I did search this site about this problem before posting. Please see below the code for an invocation example. Thanks. --JR
package mypackage;
import java.sql.Connection;
import java.sql.SQLException;
import java.util.Map;
import java.util.concurrent.ArrayBlockingQueue;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.TimeUnit;
/**
* Note: This class is only instantiated once per application run.
* Multiple instantiations, as specified in the release notes,
* are not supported.
*/
public class ConnectionManager {
// Use a blocking queue to store the database connections.
// The application will only be called once, by a single user,
// but within the application many threads will require
// a connection.
private BlockingQueue<Connection> connectionQueue = null;
// Load the connection queue with a user-defined number of connections.
// Params contains a map of all non hard-coded variables in the
// application.
public ConnectionManager(int howMany, Map<String, Object> params) {
Database database = new Database();
connectionQueue = new ArrayBlockingQueue<Connection>(howMany);
for(int i = 0; i < howMany; i++) {
connectionQueue.add(database.getConn(params));
}
}
// Return a connection from the queue, waiting up to 15 minutes to do so.
// 15 minutes is hard-coded because it is the standard time-out for all
// processes at our agency. This application must complete in less
// than fifteen minutes (is currently completing in thirty five seconds).
public Connection getConnection() {
Connection conn = null;
try {
conn = connectionQueue.poll(15, TimeUnit.MINUTES);
}
catch(InterruptedException e) {
e.printStackTrace();
}
catch(SQLException e) {
e.printStackTrace();
}
return conn;
}
// Returns a connection to the connection queue.
public void returnConnectionToManager(Connection conn) {
connectionQueue.add(conn);
}
// Called on the last line of the application program's dispatcher.
// Closes all active connections (which will only exist if there
// was a failure within one of the worker threads).
public void closeAllConnections() {
for(Connection conn : connectionQueue) {
try {
conn.close();
}
catch(SQLException e) {
e.printStackTrace();
}
}
}
}
Invocation example:
...
private ConnectionManager cm;
...
public Table(Map<String, Object> params, String method) {
...
cm = (ConnectionManager) params.get("cm");
}
// Execute a chunk of SQL code without requiring processing of a
// result set. Acquires connection from pool via cm.getConnection
// and releases connection via cm.returnConnectionToManager.
// (Database is just a helper class with simple methods for
// closing prepared statement, result sets, etc.)
private void execute(String sql) {
PreparedStatement ps = null;
Connection conn = null;
try {
conn = cm.getConnection();
ps = conn.prepareStatement(sql);
ps.execute();
}
catch (SQLException e) {
e.printStackTrace();
}
finally {
database.closePreparedStatement(ps);
cm.returnConnectionToManager(conn);
}
}
Your code looks good, but there is one serious problem, that clients of your API needs to take care of getting and releasing connection, one of them forget, and memory/resource leak is ready.
Make a one place in which you posts your queries to execute, in this place take connection, execute query and return the connection to the pool. It will secure you that the connections are returned. If you need to invoke multiple queries one after another in a single connection make the method accept an array or list of SQL queries to execute in order. The idea is to encapsulate each request to the db, so you manage all connections. It could be donethat you write an interface that has en execute(Connection conn) which you need to implement, and you could have then some Service that takes such object gives it a connection and then releases the resources back to connection pool.
Something like:
interface SqlWork {
execute(Connection conn);
}
SqlWork myWork = new SqlWork () {
execute(Connection conn) {
// do you work with the conn here
}
}
class SqlExecutionService {
ConnectionManager cm = ...;
public void execute(SqlWork sqlWork) {
Connection conn = null;
try {
conn = cm.getConnection();
sqlWork.execute(conn);
} catch (Your exceptions here) {
//serve or rethrow them
}
finally
{
if (conn!=null) {
cm.returnConnectionToManager(conn);
}
}
}
}
Example of use:
SqlExecutionService sqlExecService = ...;
sqlExecService.execute(myWork);
I was trying to write a separate database connection method in .java file which can be called upon by any servlet or jsp file needing the database connection. My code is
import java.sql.*;
import java.lang.*;
public class ConnectionClass {
private String username="root";
private String password="passwd";
/* Adjust the above two as per the username
* password combination of your MySql databse */
public Connection connect()
{
try
{
Class.forName("com.mysql.jdbc.Driver");
String url="jdbc:mysql://localhost/schooldatabase";
Connection con = DriverManager.getConnection(url,username,password);
return con;
}
catch(Exception e)
{
response.sendRedirect("studentserr.html");
out.println(e);
}
}
}
Now, the problem is that i'll be returning a Connection type so that all the servlets (which require database connection) may use it to execute various statements. However, in my code what should i return in the catch block ( which means that the connection to the database could not be established) ? Also, in case of a connection failure, I'm redirecting the user to the following page:
"studentserr.html"
This works fine if i use it in a servlet but not in .java class. What should i do for this ??
You should only be catching exceptions at exactly that moment where you can sensibly deal with them. You can't sensibly deal with them in the getConnection() method, so you should instead throw it so that the caller itself needs to deal with it.
Displaying an error page in case of a specific exception is however the responsibility of the servlet container itself. You normally configure the error page in web.xml as follows:
<error-page>
<exception-type>java.sql.SQLException</exception-type>
<location>/WEB-INF/errorpages/database.jsp</location>
</error-page>
You only need to change your code accordingly that you never catch the exception, or at least rethrow as ServletException where necessary.
Here's a minor rewrite:
public class Database {
private String url = "jdbc:mysql://localhost/schooldatabase";
private String username = "root";
private String password = "passwd";
static {
try {
Class.forName("com.mysql.jdbc.Driver"); // You don't need to load it on every single opened connection.
} catch (ClassNotFoundException) {
throw new ExceptionInInitializerError("MySQL JDBC driver missing in classpath", e);
}
}
public static Connection getConnection() throws SQLException {
return DriverManager.getConnection(url, username, password);
}
}
And here's how you should use it in your DAO classes:
public List<Student> list() throws SQLException {
List<Student> students = new ArrayList<Student>();
Connection connection = null;
// ...
try {
connection = Database.getConnection();
// ...
} finally { // Note: no catch block!
// ...
if (connection != null) try { connection.close(); } catch (SQLException ignore) {}
}
return students;
}
And here is how you should use the DAO class in your servlet's doGet() or doPost().
try {
List<Student> students = studentDAO.list();
request.setAttribute("students", students);
request.getRequestDispatcher("/WEB-INF/students.jsp").forward(request, response);
} catch (SQLException e) {
throw new ServletException(e);
}
It has to be rethrown as ServletException simply because you can't add SQLException to the throws clause of any HttpServlet method. The servletcontainer will unwrap the SQLException while locating the error page.
Return a null value in case an exception is thrown. In case of getting a null value from your method, handle the exception and don't perform any database operation.
Also, remember to separate the database logic (connection, statement execution) from the business logic and the presentation. In your actual method, this code
response.sendRedirect("studentserr.html");
should never be in the database logic, because is presentation logic.
More info:
Java 101: Layered Architecture
How to avoid Java Code in JSP-Files?
Try moving class "ConnectionClass" into some package. and then call using this package in your java class where you require this (seems to be classpath issue in java file) or in jsp or servlet page.
Example
You need this in Demo.java as
pkg1.ConnectionClass obj = new pkg1.ConnectionClass ();
It is recommended to have database connection class as singleton. So that through out the application only a single instance of connection will be created and shared.
Currently, I load the below custom driver (TestDriver.java), get a connection, create a Statement, execute a query, gets the results and close the connection. I open and close a connection for each query. Is this common practice or is there an standard way to share the open connections?
public static void main(String[] args) {
Class.forName("com.sql.TestDriver");
java.sql.Connection conn = DriverManager.getConnection("jdbc:test://8888/connectme", props);
Statement stmt = conn.createStatement;
ResultSet rs = stmt.executeQuery("select * from table");
//loop through rs and pull out needed data
conn.close();
}
public class TestDriver implements java.sql.Driver{
private final TestSchema schema;
private Properties props = null;
static {
try {
DriverManager.registerDriver(new TestDriver());
} catch (SQLException e) {
e.printStackTrace();
}
protected TestDriver() throws SQLException {
schema = TestSchemaFactory.getInstance().getDbSchemaFromFile(SCHEMA_FILE);
//loads in and parses a file containing tables, columns used for business logic
}
public Connection connect(String url, Properties info)
throws SQLException {
TestSqlConnection conn=null;
//connect logic here
return conn; //will return an instance of TestSqlConnection
}
#Override
public boolean jdbcCompliant() {
return false;
}
}
Yes, it's more common to use a database connection pool. This will allow connections to be reused without the overhead or closing/re-opening. Here's a link to DBCP which is one implementation of a database connection pool: http://commons.apache.org/dbcp/
Ideally you should write a separate factory class (can be static)
say ConnectionFactory which returns a connection object.
Also I see that you are not using try/catch/finally block while creating
connection.I strongly suggest to close the connection in finally
clause otherwise you program may suffer from connection leak if any
exception is raised and causes abrupt behavior.
Ideally you should close the connection after your operation is complete in finally
clause.In web based application if you are using connections pool
then closing connection will return the connection back to pool and
will be available for use.
Is it possible to store a database connection as a separate class, then call the database objects from a main code? ie;
public class main{
public static void main{
try{
Class.forName("com.jdbc.driver");
Database to = new Database(1,"SERVER1","DATABASE");
Database from = new Database(2,"SERVER2","DATABASE");
String QueryStr = String.format("SELECT * FROM TABLE WHERE Id = %i", to.id)
to.results = sql.executeQuery(QueryStr);
while (to.results.next()) {
String QueryStr = String.format("INSERT INTO Table (A,B) VALUES (%s,%s)",to.results.getString(1),to.results.getString(2));
from.sql.executeQuery("QueryStr");
}
to.connection.close()
from.connection.close()
} catch (Exception ex) {
ex.printStackTrace();
{ finally {
if (to.connection != null)
try {
to.connection.close();
} catch (SQLException x) {
}
if (from.connection != null)
try {
from.connection.close();
} catch (SQLException x) {
}
}
}
public static class Database {
public int id;
public String server;
public String database;
public Connection connection;
public ResultSet results;
public Statement sql;
public Database(int _id, String _server, String _database) {
id = _id;
server = _server;
database = _database;
String connectStr = String.format("jdbc:driver://SERVER=%s;port=6322;DATABASE=%s",server,database);
connection = DriverManager.getConnection(connectStr);
sql = connection.createStatement;
}
}
}
I keep getting a "Connection object is closed" error when I call to.results = sql.executeQuery("SELECT * FROM TABLE"); like the connection closes as soon as the Database is done initializing.
The reason I ask is I have multiple databases that are all about the same that I am dumping into a master database. I thought it would be nice to setup a loop to go through each from database and insert into each to database using the same class. Is this not possible? Database will also contain more methods than shown as well. I am pretty new to java, so hopefully this makes sense...
Also, my code is probably riddled with syntax errors as is, so try not to focus on that.
Connection object is closed doesn't mean that the connection is closed, but that the object relative to the connection is closed (it could be a Statement or a ResultSet).
It's difficult to see from your example, since it has been trimmed/re-arranged, but it looks like you may be trying to use a ResultSet after having re-used its corresponding Statement. See the documentation:
By default, only one ResultSet object per Statement object can be open
at the same time. Therefore, if the reading of one ResultSet object is
interleaved with the reading of another, each must have been generated
by different Statement objects. All execution methods in the Statement
interface implicitly close a statment's current ResultSet object if an
open one exists.
In your example, it may be because autoCommit is set to true by default. You can override this on the java.sql.Connection class. Better yet is to use a transaction framework if you're updating multiple tables.