Develop Reference

Tomcat doesn't sync with the mysql database

I'm currently working on a college project, and I'm creating a very simple e-commerce style website.
I'm using JDBC driver manager and connection pool for the connection to the db, while using Tomcat 9.0 as the container.
The problem is: when I modify some product through the website (let's say the amount available for example), the website doesn't always reflect the changes, while I can always see the data correctly in MySql Workbench.
It actually works one time out of two on the same query:
I run the query for the first time after the changes -> it shows the old value
I run the query for the second time after the changes -> it shows the new value
I run the query for the third time after the changes -> it shows the old value
And so on.
I've already tried to set caching off (from the query, using the SQL_NO_CACHE), but it didn't seem to solve the problem, I've tried to use Datasource instead, but it causes other problems that most likely I won't have the time to solve.
This is the connection pool file, which I think might be problem, I'm not that sure tho:
public class DriverManagerConnectionPool {
private static List<Connection> freeDbConnections;
static {
freeDbConnections = new LinkedList<Connection>();
try {
Class.forName("com.mysql.cj.jdbc.Driver");
} catch (ClassNotFoundException e) {
System.out.println("DB driver not found:"+ e.getMessage());
}
}
private static synchronized Connection createDBConnection() throws SQLException {
Connection newConnection = null;
String ip = "localhost";
String port = "3306";
String db = "storage";
String username = "root";
String password = "1234";
newConnection = DriverManager.getConnection("jdbc:mysql://"+ ip+":"+ port+"/"+db+"?useUnicode=true&useJDBCCompliantTimezoneShift=true&useLegacyDatetimeCode=false&serverTimezone=UTC", username, password);
newConnection.setAutoCommit(false);
return newConnection;
}
public static synchronized Connection getConnection() throws SQLException {
Connection connection;
if (!freeDbConnections.isEmpty()) {
connection = (Connection) freeDbConnections.get(0);
freeDbConnections.remove(0);
try {
if (connection.isClosed())
connection = getConnection();
} catch (SQLException e) {
connection.close();
connection = getConnection();
}
} else {
connection = createDBConnection();
}
return connection;
}
public static synchronized void releaseConnection(Connection connection) throws SQLException {
if(connection != null) freeDbConnections.add(connection);
}
}
I really hope you can help me, I haven't found any solution online!

I guess it is because of auto-commit is disabled. Please try using #Transactional or set auto-commit to true. You can also try to use db.commit after each statement.

As per your connection pool implementation, all connection in your pool seems to be auto committed false.
Please check you have properly committed the connection after executing the query or not.
So it might be the case that, when executing the query after changes with same connection it reflects those changes, done earlier and on other connections, old values are might get returned.

Tomcat connection pool & idle connections

We are developing a website using
Tomcat 7
JDBC
PostgreSQL 9.2
We've had some connection leaks and think we corrected them (the database no longer stops responding), but the behaviour of the connection pool still seems leaky, as we have a number of idle connections greater than the maxIdle set in context.xml. I'd like to be sure the problem is fixed.
For testing purposes, I'm using the following context.xml :
<Resource
auth="Container"
name="jdbc/postgres"
factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
type="javax.sql.DataSource"
username="admin"
password="..."
driverClassName="org.postgresql.Driver"
url="jdbc:postgresql://127.0.0.1:5432/..."
initialSize="1"
maxActive="50"
minIdle="0"
maxIdle="3"
maxWait="-1"
minEvictableIdleTimeMillis="1000"
timeBetweenEvictionRunsMillis="1000"
/>
If I understand correctly, we should have 1 idle connection on startup and from 0 to 3 depending on the load, right ?
What is happening is : 1 connection on startup, up to 3 idle connections if the load is low, and more than 3 idle connections after a high load. Then these connections are not closed immediatly, and we don't know when/if they will be closed (sometime some of them are closed).
So the question is : is this behaviour normal, or not ?
Thanks for your help
EDIT : added factory attribute, didn't change the problem
EDIT 2 : using removeAbandoned & removeAbandonedTimeout make the idle connexions being closed every removeAbandonedTimeout. So we probably still have some connection leaks. Here are some pieces of code we are using to connect to the database and execute requests :
PostgreSQLConnectionProvider, just a static class to provide a connection :
public class PostgreSQLConnectionProvider {
public static Connection getConnection() throws NamingException, SQLException {
String dsString = "java:/comp/env/jdbc/postgres";
Context context = new InitialContext();
DataSource ds = (DataSource) context.lookup(dsString);
Connection connection = ds.getConnection();
return connection;
}
}
DAO abstract class :
public abstract class DAO implements java.lang.AutoCloseable {
// Private attributes :
private Connection _connection;
// Constructors :
public DAO() {
try { _connection = PostgreSQLConnectionProvider.getConnection(); }
catch (NamingException | SQLException ex) {
Logger.getLogger(DAO.class.getName()).log(Level.SEVERE, null, ex);
}
}
// Getters :
public Connection getConnection() { return _connection; }
// Closeable :
#Override
public void close() throws SQLException {
if(!_connection.getAutoCommit()) {
_connection.rollback();
_connection.setAutoCommit(true);
}
_connection.close();
}
}
UserDAO, a small DAO subclass (we have several DAO sublasses to request the database) :
public class UserDAO extends DAO {
public User getUserWithId(int id) throws SQLException {
PreparedStatement ps = null;
ResultSet rs = null;
User user = null;
try {
String sql = "select * from \"USER\" where id_user = ?;";
ps = getConnection().prepareStatement(sql);
ps.setInt(1, id);
rs = ps.executeQuery();
rs.next();
String login = rs.getString("login");
String password = rs.getString("password");
String firstName = rs.getString("first_name");
String lastName = rs.getString("last_name");
String email = rs.getString("email");
user = new User(id, login, password, firstName, lastName, email);
}
finally {
if(rs != null) rs.close();
if(ps != null) ps.close();
}
return user;
}
}
An example of a DAO subclass use :
try(UserDAO dao = new UserDAO()) {
try {
User user = dao.getUserWithId(52);
}
catch (SQLException ex) {
// Handle exeption during getUserWithId
}
}
catch (SQLException ex) {
// Handle exeption during dao.close()
}

Looking at the code it appears the connection is grabbed for the lifetime of the DAO, not the lifetime of the statement, which is the usual expectation. Normally, you would grab a connection from the pool just as you're about to execute the statement, and call close() on it when you're done in order to return it to the pool.
Additionally, in your finally clause, both rs.close() and ps.close() can throw exceptions resulting in missing the last call against the prepared statement.
In Java 7 you can also use a try with resources statement that will close both the prepared statement and the connection for you. According to the spec, the driver is supposed to close the result for you when the statement is closed.

How to use connection pooling

I am new in connection pooling.I have a created a connection pool in mysql that adds five connections.Now i want to know what is the application of connection pooling,i.e after creating that pool how to use that.. i am pasting my code here
import java.sql.DriverManager;
import java.sql.SQLException;
import java.util.Vector;
import com.mysql.jdbc.Connection;
class ConnectionPoolManager {
String databaseUrl = "jdbc:mysql://localhost:3306/homeland";
String userName = "root";
String password = "root";
Vector connectionPool = new Vector();
public ConnectionPoolManager() {
initialize();
}
public ConnectionPoolManager(
// String databaseName,
String databaseUrl, String userName, String password) {
this.databaseUrl = databaseUrl;
this.userName = userName;
this.password = password;
initialize();
}
private void initialize() {
// Here we can initialize all the information that we need
initializeConnectionPool();
}
private void initializeConnectionPool() {
while (!checkIfConnectionPoolIsFull()) {
System.out
.println("Connection Pool is NOT full. Proceeding with adding new connections");
// Adding new connection instance until the pool is full
connectionPool.addElement(createNewConnectionForPool());
}
System.out.println("Connection Pool is full.");
}
private synchronized boolean checkIfConnectionPoolIsFull() {
final int MAX_POOL_SIZE = 5;
// Check if the pool size
if (connectionPool.size() < 5) {
return false;
}
return true;
}
// Creating a connection
private Connection createNewConnectionForPool() {
Connection connection = null;
try {
Class.forName("com.mysql.jdbc.Driver");
connection = (Connection) DriverManager.getConnection(databaseUrl,
userName, password);
System.out.println("Connection: " + connection);
} catch (SQLException sqle) {
System.err.println("SQLException: " + sqle);
return null;
} catch (ClassNotFoundException cnfe) {
System.err.println("ClassNotFoundException: " + cnfe);
return null;
}
return connection;
}
public synchronized Connection getConnectionFromPool() {
Connection connection = null;
// Check if there is a connection available. There are times when all
// the connections in the pool may be used up
if (connectionPool.size() > 0) {
connection = (Connection) connectionPool.firstElement();
connectionPool.removeElementAt(0);
}
// Giving away the connection from the connection pool
return connection;
}
public synchronized void returnConnectionToPool(Connection connection) {
// Adding the connection from the client back to the connection pool
connectionPool.addElement(connection);
}
public static void main(String args[]) {
new ConnectionPoolManager();
}
}
can any one help?

The purpose of connection pooling is to maintain a number of open connections to a database so that when your application requires a connection it does not have to go through the potentially resource and time intensive process of opening a new connection.
When an application requires a database connection it 'borrows' one from the pool. When it's done, it gives it back and that connection may be reused at some later point.
Once you have obtained a connection, you use it in your application through the JDBC (Java Database Connectivity) API.
Oracle's basic tutorial for using JDBC can be found at http://docs.oracle.com/javase/tutorial/jdbc/basics/index.html
Another thing to keep in mind is that alot of work has gone into developing connection pools already, and it probably is not necessary to reinvent the wheel, except perhaps as a learning excercise. Apache Tomcat's connection pool implementation can be used outside of Tomcat (for example, in a standalone Java application) and is fairly flexible and easy to configure. It can be found at https://people.apache.org/~fhanik/jdbc-pool/jdbc-pool.html

I would say the code is pretty self explanatory.
You create an instance of the pool, personally, I prefer to use a singleton, but that's another topic
ConnectionPoolManager connectionPoolManager = new ConnectionPoolManager();
Now, every body that wants a connection, is going to need a reference to this manager. When you need to, you request a free connection from the pool...
public void queryDatabaseForStuff(ConnectionPoolManager cpm) throws SQLException {
Connection con = cpm.getConnectionFromPool();
//....
Once you're finished with the connection, you pass it back to the manager...
try {
//...
} finally {
cmp.returnConnectionToPool(con);
}
Now. You might like to investigating a blocking process that will block the current call to getConnectionFromPool while the pool is empty, meaning that it will either throw an exception (if you want to include a time out feature) or a valid connection.
When re-pooling a Connection, you might like to check to see if the Connection has been closed or not and have some kind of revival process to ensure that the pool is awlays close to capcaity...

Please check this link for getting detailed answer - https://examples.javacodegeeks.com/core-java/sql/jdbc-connection-pool-example/
You don't need to recreate your Connection object pool , instead please use the libraries provided by Apache . Please be clear of the following :
1 - Why and what made you think of connection pool ?
2 - Use the following Apache commons-dbcp lib in your Maven project and then use the classes as per documentation .
3. Does this solve all your problems ?

ITs Better to perform the connection pooling via in built API
Like
DBCP or this.
Its always better let these API perform the connection pooling and programmatically creating and maintaining the connection pooling always painful activity.

Singleton in JSP, how to properly tidy up on close?

I am just getting started with jsp and my question is this - when I have a singleton class, how do I tidy up after it?
In particular:
public class DBConnection {
private static Connection connection = null;
private static Statement statement = null;
public static ResultSet executeQuery(String query){
if (connection == null) { /*initConnection*/ }
if (statement == null) { /*initStatement*/ }
// do some stuff
}
}
Now, I use this class in several pages to get results from jdbc. However, I need to eventually call statement.close(); and connection.close(); - when should I call those?
I am using singleton, because it felt wrong to call for connection to a database over and over whenever I needed to make a query.

The Connection must be closed always, and after you have executed all your database statements for the desired operations. Two examples:
Case 1: You must show a list of products to user filtered by criteria from database. Solution: get a connection, retrieve a list of products using the filter criteria, close the connection.
Case 2: The client selects some of these products and updates the minimum stock to get an alert and restock them. Solution: get a connection, update all the products, close the connection.
Based on these cases, we can learn lot of things:
You can execute more than a single statement while having/maintaining a single connection open.
The connection should live only in the block where it is used. It should not live before or after that.
Both cases can happen at the same time since they are in a multi threaded environment. So, a single database connection must not be available to be used by two threads at the same time, in order to avoid result problems. For example, user A searches the products that are in category Foo and user B searches the products that are in category Bar, you don't want to show the products in category Bar to user A.
From last sentence, each database operation ((or group of similar operations like Case 2) should be handled in an atomic operation. To assure this, the connection must not be stored in a singleton object, instead it must be live only in the method being used.
In consequence:
Do not declare the Connection nor the Statement nor the ResultSet nor other JDBC resource as static. It will simply fail. Instead, declare only the Connection as field of your DBConnection class. Let each method decide to handle each Statement (or PreparedStatement) and ResultSet and specific JDBC resources.
Since you must close the connection after its usage, then add two more methods: void open() and void close(). These methods will handle the database connection retrieval and closing that connection.
Additional, since the DBConnection looks like a wrapper class for Connection class and database connection operations, I would recommend to have at least three more methods: void setAutoCommit(boolean autoCommit), void commit() and void rollback(). These methods will be plain wrappers for Connection#setAutoCommit Connection#close and Connection#rollback respectively.
Then you can use the class in this way:
public List<Product> getProducts(String categoryName) {
String sql = "SELECT id, name FROM Product WHERE categoryName = ?";
List<Product> productList = new ArrayList<Product>();
DBConnection dbConnection = new DBConnection();
try {
dbConnection.open();
ResultSet resultSet = dbConnection.executeSelect(sql, categoryName); //execute select and apply parameters
//fill productList...
} catch (Exception e) {
//always handle your exceptions
...
} finally {
//don't forget to also close other resources here like ResultSet...
//always close the connection
dbConnection.close();
}
}
Note that in this example the PreparedStatement is not in the getProducts method, it will be a local variable of the executeSelect method.
Additional notes:
When working in an application server, you should not open connections naively e.g. using Class.forName("..."), instead use a database connection pool. You can roll on some database connection pooling libraries like C3P0 as explained here: How to establish a connection pool in JDBC?. Or configure one in your application server, as I explain here: Is it a good idea to put jdbc connection code in servlet class?
If this is for learning purposes, then roll on your own classes to handle the communication with your database. In real world applications, this is not recommended (doesn't mean you should not do it). Instead, use a database connectivity framework like ORMs e.g. JPA (Java official ORM framework) or Hibernate; there are no ORM frameworks that handles database communication like Spring JDBC and MyBatis. The choice is yours.
More info:
Should a database connection stay open all the time or only be opened when needed?
How do servlets work? Instantiation, sessions, shared variables and multithreading. Not directly related to your question, but it will help you understand why to not maintain state in resources that are used in multithreaded environments.

Define connection resource in mywebapp/META-INF/context.xml file
<Resource name="jdbc/mydb" auth="Container" type="javax.sql.DataSource"
maxActive="10" maxIdle="2" maxWait="20000"
driverClassName="com.mysql.jdbc.Driver"
username="myuser" password="mypwd"
url="jdbc:mysql://localhost:3306/mydb?useUnicode=true&characterEncoding=utf8"
validationQuery="SELECT 1" />
Create DB.java helper class to minimize code in other parts of app
import java.sql.*;
import javax.sql.DataSource;
import javax.naming.Context;
import javax.naming.InitialContext;
public class DB {
public static Connection createConnection() throws SQLException {
try {
Context ctx = new InitialContext();
DataSource ds = (DataSource)ctx.lookup("java:comp/env/jdbc/mydb");
return ds.getConnection();
} catch (SQLException ex) {
throw ex;
} catch (Exception ex) {
SQLException sqex = new SQLException(ex.getMessage());
sqex.initCause(ex);
throw sqex;
}
}
public static void close(ResultSet rs, Statement stmt, Connection conn) {
if (rs != null) try { rs.close(); } catch (Exception e) { }
if (stmt != null) try { stmt.close(); } catch (Exception e) { }
if (conn != null) try { conn.close(); } catch (Exception e) { }
}
public static void close(ResultSet rs, boolean closeStmtAndConn) {
if (rs==null) return;
try {
Statement stmt = rs.getStatement();
close(rs, stmt, stmt!=null ? stmt.getConnection() : null);
} catch (Exception ex) { }
}
}
And somewhere in your app DAO code use DB helper.
public List<MyBean> getBeans() throws SQLException {
List<MyBean> list = new ArrayList<MyBean>();
ResultSet rs=null;
try {
Connection con = DB.createConnection();
String sql = "Select * from beantable where typeid=?";
PreparedStatement stmt = con.prepareStatement(sql, Statement.NO_GENERATED_KEYS);
stmt.setInt(1, 101);
rs = stmt.executeQuery();
while(rs.next()
list.add( createBean(rs) );
} finally {
DB.close(rs, true); // or DB.close(rs, stmt, conn);
}
return list;
}
private MyBean createBean(ResultSet rs) throws SQLException {
MyBean bean = new MyBean();
bean.setId( rs.getLong("id") );
bean.setName( rs.getString("name" );
bean.setTypeId( rs.getInt("typeid") );
return bean;
}

I would add two methods to the class:
public static void open() throws SomeException;
public static void close() throws SomeException;
then your calling code looks something like this{
try {
DBConnection.open();
... code to use the connection one or more times ...
} finally {
DBConnection.close();
}
Wrap all your database calls inside that and it will take care of closing whether there is an exception thrown or not.
Of course, this isn't much different than having a regular class, which I might recommend:
try {
DBConnection conn = new DBConnection();
conn.open();
... all the code to use the database (but you pass 'conn' around) ...
} finally {
conn.close();
}
And you might want to look at the java.lang.AutoCloseable and java.io.Closeable to see if that helps you.
2
If you are keeping it open across page loads, there isn't any place to put the try ... finally stuff so you can open it and close it when the servlet closes or the server closes or something like that.
If you are going to leave it open, you need to make sure and add code to verify it doesn't close when you aren't looking. A short network glitch, for example, could close it down. In that case, you need to reopen it when it gets closed. Otherwise, all database access from that point will fail.
You might want to look into the concept of a DataBase Pool. Apache has one -- DBCP. Tomcat has its own that's quite good. Other containers, like JBOSS, WebSphere, WebLogic all have them. There's a couple that can be used with the Spring Framework. What it does is manage one or more database connections. Your code asks it for one and it returns an open one, unless none is available and then it opens one and returns it. You call close when your code gets through with it but it doesn't really close the connection, it just returns it to the pool.
You can usually configure the pool to check for shut down connections and reopen if needed.

Java Postgres connection limit exceeded

I am using Java 1.7 and Postgres via the Postgres JDBC drivers. The database connection will be used from a Web Service. In testing, I got the following error:
FATAL: connection limit exceeded for non-superusers
I solved the error by making my connection static, and, only creating once. My question is, is a static connection safe? Is this the right way to do this?
I am using the connection via a ConnectionFactory that looks something like this:
public class ConnectionFactory
{
String driverClassName = "org.postgresql.Driver";
String connectionUrl = "jdbc:postgresql://localhost:5432/dbName";
String dbUser = "user";
String dbPwd = "password";
private static ConnectionFactory connectionFactory = null;
private static Connection conn = null;
private ConnectionFactory()
{
try
{
Class.forName(driverClassName);
}
catch (ClassNotFoundException e)
{
e.printStackTrace();
}
}
public Connection getConnection() throws SQLException
{
if (conn == null)
{
conn = DriverManager.getConnection(connectionUrl, dbUser, dbPwd);
}
return conn;
}
public static ConnectionFactory getInstance()
{
if (connectionFactory == null)
{
connectionFactory = new ConnectionFactory();
}
return connectionFactory;
}
}

Postgres JDBC driver is documented as thread safe and the connection can be used by multiple threads if so required. If a thread attempts to use the connection while another one is using it, it will wait until the other thread has finished its current operation.
Connection pooling may be used anyway for performance reasons.

Sorry, original post didn't look at your code carefully. Wow. I still can't read. Anyway, third time's the charm. If your code is single threaded - then your fine. If it's multi-threaded, use something like the Commons connection pools to manage your connections. It looks like the driver is thread safe but the connection shouldn't be viewed as thread safe. So, once the driver is loaded you can safely call getConnection on the driver from multiple threads, but the connection shouldn't be shared across threads.