Develop Reference

How to use a HikariCP connection pool with prepared statements?

I am developing a Spring Boot web (REST) application where I need to serve many requests. Therefore I wanted my application to be able to handle requests concurrently. Since Spring Boot REST-Services are out-of-the-box concurrently usable, I only need to make the (PostgreSQL) database access concurrently accessible. For that I am using the HikariCP data source.
Since a lot of my statements are prepared statements, I collected them in one method where I call pstmt = connection.prepareStatement("SQLCODE"); once for every statemment. Those prepared statements are then used in various methods when user interaction from the REST service is processed.
Now, when I use the HikariCP I can't do that anymore, can I?
When I prepare a statement, this statement is bound to one connection. If I then try to access it concurrently, I can't because the connection is not shared.
Am I missing something? How can I solve this? Do I need to retrieve a connection from the pool, prepare the statement locally, execute my query, and close the connection? If so, what's the point of using a prepared statement then (other than preventing SQL injection)?
I know that the statements are cached by on the PostreSQL side. So would it be a good idea to keep the method where all prepared statements are prepared? To sent them to the database cache. And then just creating locally the same statements again. That way, one might still leverage the caching possibilities of the database. But on the other hand it would be really ugly code.
Im am using Spring: 5.3.10, Java: 11, PostgreSQL: 14.0
#RestController
public class RESTController {
/** The database controller. */
private DBController dbc;
/** The data source object serving as a connection pool. */
private HikariDataSource ds;
/** The logger object for this class. */
private static Logger logger = LoggerFactory.getLogger(RESTController.class);
public RESTController(DBController dbc, Config config) {
this.dbc = dbc;
// Create the database
if (!this.dbc.createDB(config)) {
logger.error("Couldn't create the database. The service will now exit.");
Runtime.getRuntime().halt(1);
}
// Create a connection pool
ds = new HikariDataSource();
ds.setJdbcUrl(config.getUrl());
ds.setUsername(config.getUser());
ds.setPassword(config.getPassword());
ds.addDataSourceProperty("cachePrepStmts", "true");
ds.addDataSourceProperty("prepStmtCacheSize", "250");
ds.addDataSourceProperty("prepStmtCacheSqlLimit", "2048");
// Create the necessary tables
if (!this.dbc.createTables(ds)) {
logger.error("Couldn't create the tables. The service will now exit.");
ds.close();
Runtime.getRuntime().halt(1);
}
// Prepare SQL statements
if (!this.dbc.prepareStatements(ds)) {
logger.error("Couldn't prepare the SQL statements. The service will now exit.");
ds.close();
Runtime.getRuntime().halt(1);
}
}
#PostMapping("/ID")
public ResponseEntity<String> createNewDomain(#RequestParam(name = "name", required = true) String name) {
// Do stuff ...
}
// [...]
}
#Component
public class DBController {
/** The logger object for this class. */
private static Logger logger = LoggerFactory.getLogger(DBController.class);
// Prepared Statements
private PreparedStatement stmt1, stmt2, stmt3;
public boolean prepareStatements(HikariDataSource ds) {
try {
// Get connection from the pool
Connection c = ds.getConnection();
// Prepare all the statements
stmt1 = c.prepareStatement("SQLCODE");
stmt2 = c.prepareStatement("SQLCODE1");
stmt2 = c.prepareStatement("SQLCODE1");
// [...]
} catch (SQLException e) {
logger.debug("Could not prepare the SQL statements: " + e.getMessage());
return false;
}
logger.debug("Successfully prepared the SQL statements.");
return true;
}
public boolean m1(int i) {
stmt1.setInt(i);
ResultSet rs = stmt1.executeQuery();
}
public boolean m2(int j) {
stmt1.setInt(j);
ResultSet rs = stmt1.executeQuery();
}
public boolean m3(String a) {
stmt2.setString(a);
ResultSet rs = stmt2.executeQuery();
}
// [...]
}
Thanks in advance.

pleae read the part Statement Cache at https://github.com/brettwooldridge/HikariCP
Many connection pools, including Apache DBCP, Vibur, c3p0 and others
offer PreparedStatement caching. HikariCP does not. Why?
So it does not cache. and if you read explanaition maybe you decide you don't need it to.

How does the postgresql (set role user) command use in SSM projects?

Now the project is using springmvc+ spring + mybatis + druid + postgresql
The users in the project correspond to the users in the database, so each time you run SQL, you switch the users with the (set role user) command and then perform the crud operations of the database.
My question:
Because there are many connections in the connection pool, the first step is to get the connection of the database, then switch users, and then perform the operation of business SQL on the database. But I don't know which part of the project this logic should be processed, because the connection of the connection pool and the execution of SQL are implemented by the underlying code. Do you have any good plans?
Can you provide me with a complete demo, such as the following operations:
Step 1, get the user's name from spring security (or shiro).
Step 2, Get the connection currently using the database from the connection pool.
Step 3, execute SQL (set role user) to switch roles.
Step 4, perform crud operation.
Step 5, Reset the database connection(reset role)

Here is a simple way to do what you need with the help of mybatis-spring.
Unless you already use mybatis-spring the first step would be to change the configuration of your project so that you obtain SqlSessionFactory using org.mybatis.spring.SqlSessionFactoryBean provided by mybatis-spring.
The next step is the implementation of setting/resetting the user role for the connection. In mybatis the connection lifecycle is controlled by the class implementing org.apache.ibatis.transaction.Transaction interface. The instance of this class is used by the query executor to get the connection.
In a nutshell you need to create your own implementation of this class and configure mybatis to use it.
Your implementation can be based on the SpringManagedTransaction from mybatis-spring and would look something like:
import org.springframework.security.core.Authentication;
class UserRoleAwareSpringManagedTransaction extends SpringManagedTransaction {
public UserRoleAwareSpringManagedTransaction(DataSource dataSource) {
super(dataSource);
}
#Override
public Connection getConnection() throws SQLException {
Connection connection = getCurrentConnection();
setUserRole(connection);
return connection;
}
private Connection getCurrentConnection() {
return super.getConnection();
}
#Override
public void close() throws SQLException {
resetUserRole(getCurrentConnection());
super.close();
}
private void setUserRole(Connection connection) {
Authentication authentication = SecurityContextHolder.getContext().getAuthentication();
String username = authentication.getName();
Statement statement = connection.createStatement();
try {
// note that this direct usage of usernmae is a subject for SQL injection
// so you need to use the suggestion from
// https://stackoverflow.com/questions/2998597/switch-role-after-connecting-to-database
// about encoding of the username
statement.execute("set role '" + username + "'");
} finally {
statement.close();
}
}
private void resetUserRole(Connection connection) {
Statement statement = connection.createStatement();
try {
statement.execute("reset role");
} finally {
statement.close();
}
}
}
Now you need to configure mybatis to use you Transaction implementation. For this you need to implement TransactionFactory similar to org.mybatis.spring.transaction.SpringManagedTransactionFactory provided by mybatis-spring:
public class UserRoleAwareSpringManagedTransactionFactory implements TransactionFactory {
#Override
public Transaction newTransaction(DataSource dataSource, TransactionIsolationLevel level, boolean autoCommit) {
return new UserRoleAwareSpringManagedTransaction(dataSource);
}
#Override
public Transaction newTransaction(Connection conn) {
throw new UnsupportedOperationException("New Spring transactions require a DataSource");
}
#Override
public void setProperties(Properties props) {
}
}
And then define a bean of type UserRoleAwareSpringManagedTransactionFactory in your spring context and inject it into transactionFactory property of the SqlSessionFactoryBeen in your spring context.
Now every time mybatis obtains a Connection the implementation of Transaction will set the current spring security user to set the role.

Best practice is that database users are applications. Application users' access to particular data/resource should be controlled in the application. Applications should not rely on database to restrict data/resource access. Therefore, application users should not have different roles in database. An application should use only a single database user account.
Spring is manifestation of best practices. Therefore, Spring does not implement this functionality. If you want such functionality, you need to hack.
Referring to this, your best bet is to:
#Autowired JdbcTemplate jdbcTemplate;
// ...
public runPerUserSql() {
jdbcTemplate.execute("set role user 'user_1';");
jdbcTemplate.execute("SELECT 1;");
}
I still do not have much confidence in this. Unless you are writing a pgAdmin webapp for multiple users, you should re-consider your approach and design.

Using local MySQL Database in JPA driven Java Application [duplicate]

This question already has answers here:
What is a NullPointerException, and how do I fix it?
(12 answers)
Closed 4 years ago.
I'm trying to create a Java application that will run on a hypothetical client machine, where members of staff can both view or add customer details from a local MySQL database.
I'm trying to use JPA to do so, with query methods being in this form:
public class DataManagerImpl implements DataManager{
#PersistenceContext
private EntityManager em;
public List<Customer> AllCustomers(){
TypedQuery<Customer> query = em.createNamedQuery("Customer.findAll", Customer.class);
return query.getResultList();
} }
I've got a DBConnection class:
public class MyDBConn implements DBConnectivity {
#Resource(mappedName="jdbc:mysql://localhost:3306/solsoft_DB") DataSource dataSource;
Connection myConn = null;
public Connection open_Connection() {
String user = "root";
String pass = "password";
try {
Class.forName("com.mysql.jdbc.Driver");
myConn = dataSource.getConnection(user, pass);
return myConn;
} catch (Exception exc) {
exc.printStackTrace();
return myConn;
}
}}
And then in my main method:
DataManagerImpl dm = new DataManagerImpl();
List<Customer> allCustomers = dm.AllCustomers();
for(Customer c : allCustomers){
String cust = "" + c.getForename() + " " + c.getSurname();
System.out.println(cust);
}
I'd really appreciate if anyone could point my in the right direction on how to actually go about getting some information from the DB using JPA in this way.

The application will be running in a server? What server?
Or is an standalone application?
My guess (the best I can do as there are many things not pointed in the question) is that you are trying to supply the connection that PersistenceContext should use.
If its like this and you are using JPA you should register an EntityManagerFactory with the required properties for connection and get your PersistenceContext from that factory. (See an example here)
Another way to go would be to edit your persistence.xml file defining this properties inside the file like this and just let your context handle the logic for database connection.

Is Spring JdbcTemplate thread safe?

I'm using Spring JdbcTemplate on one of my projects and now, when there are really very much requests with it - I started to face this exception:
org.springframework.jdbc.UncategorizedSQLException: CallableStatementCallback;
uncategorized SQLException for SQL [{? = call API.get_data_for_nb(?, ?)}];
SQL state [99999]; error code [17009]; Closed Statement;
nested exception is java.sql.SQLException: Closed Statement
So the Closed Statement exception is received when you try to execute statement that is already closed, but in my case I don't close it by myself - I use JdbcTemplate exactly for that. So, firstly, what could be the reason for that?
The JdbcTemplate object itself is contained in #Stateless EJB in this way:
#Stateless(name = "NbEdwServiceEJB")
public class NbEdwServiceBean implements NbEdwServiceLocal, NbEdwServiceRemote {
#Resource(mappedName = JNDI)
private DataSource dataSource;
private static volatile JdbcTemplate jdbcTemplate;
#PostConstruct
protected void construct() {
synchronized (NbEdwServiceBean.class) {
if (jdbcTemplate == null) {
jdbcTemplate = new JdbcTemplate(dataSource);
jdbcTemplate.setResultsMapCaseInsensitive(true);
}
}
}
private String getDataFromDB(final String request, final int isDigitalSignVerified) {
String response = null;
try {
response = jdbcTemplate.execute(SQL_GET_DATA, new CallableStatementCallback<String>() {
public String doInCallableStatement(CallableStatement cs) throws SQLException, DataAccessException {
cs.registerOutParameter(1, Types.VARCHAR);
cs.setInt(2, isDigitalSignVerified);
cs.setString(3, request);
cs.executeUpdate();
return cs.getString(1);
}
});
} catch (DataAccessException ex) {
LOGGER.error("getDataFromDB()", ex);
}
return response;
}
}
I know that this is maybe not the strictly right way to do it, I could just create instance of JdbcTemplate for every stateless bean - so I might do just that. So, secondly, why is this ever happening? My suppose was that JdbcTemplate's execute method isn't thread safe, but can someone give the full explanation on what is going on?
I have JEE version 5 running on WebLogic 10.3.5 if it's matter.

#Tolegen Izbassar I'm sorry that you're stuck with EE5.
Concerning the Singleton and EE5 there were some alternatives out there. One is vendor specific extensions, for example JBoss 5.x had service beans providing Singleton+JMX. A second solution is to use a earlier version of Jboss Seam compatible with EE5. A third alternative is to use the ServerContext from the Servlet API.
What you're trying to do in #PostConstuct is definitely not good. Non final statics in SLSB is a no go.
I suggest to have a look, at section 29.3 from Spring framework reference which describes EJB - Spring integration, an example from that section:
#Stateless
#Interceptors(SpringBeanAutowiringInterceptor.class)
public class MyFacadeEJB implements MyFacadeLocal {
// automatically injected with a matching Spring bean
#Autowired
private MyComponent myComp;
// for business method, delegate to POJO service impl.
public String myFacadeMethod(...) {
return myComp.myMethod(...);
}

Singleton in JSP, how to properly tidy up on close?

I am just getting started with jsp and my question is this - when I have a singleton class, how do I tidy up after it?
In particular:
public class DBConnection {
private static Connection connection = null;
private static Statement statement = null;
public static ResultSet executeQuery(String query){
if (connection == null) { /*initConnection*/ }
if (statement == null) { /*initStatement*/ }
// do some stuff
}
}
Now, I use this class in several pages to get results from jdbc. However, I need to eventually call statement.close(); and connection.close(); - when should I call those?
I am using singleton, because it felt wrong to call for connection to a database over and over whenever I needed to make a query.

The Connection must be closed always, and after you have executed all your database statements for the desired operations. Two examples:
Case 1: You must show a list of products to user filtered by criteria from database. Solution: get a connection, retrieve a list of products using the filter criteria, close the connection.
Case 2: The client selects some of these products and updates the minimum stock to get an alert and restock them. Solution: get a connection, update all the products, close the connection.
Based on these cases, we can learn lot of things:
You can execute more than a single statement while having/maintaining a single connection open.
The connection should live only in the block where it is used. It should not live before or after that.
Both cases can happen at the same time since they are in a multi threaded environment. So, a single database connection must not be available to be used by two threads at the same time, in order to avoid result problems. For example, user A searches the products that are in category Foo and user B searches the products that are in category Bar, you don't want to show the products in category Bar to user A.
From last sentence, each database operation ((or group of similar operations like Case 2) should be handled in an atomic operation. To assure this, the connection must not be stored in a singleton object, instead it must be live only in the method being used.
In consequence:
Do not declare the Connection nor the Statement nor the ResultSet nor other JDBC resource as static. It will simply fail. Instead, declare only the Connection as field of your DBConnection class. Let each method decide to handle each Statement (or PreparedStatement) and ResultSet and specific JDBC resources.
Since you must close the connection after its usage, then add two more methods: void open() and void close(). These methods will handle the database connection retrieval and closing that connection.
Additional, since the DBConnection looks like a wrapper class for Connection class and database connection operations, I would recommend to have at least three more methods: void setAutoCommit(boolean autoCommit), void commit() and void rollback(). These methods will be plain wrappers for Connection#setAutoCommit Connection#close and Connection#rollback respectively.
Then you can use the class in this way:
public List<Product> getProducts(String categoryName) {
String sql = "SELECT id, name FROM Product WHERE categoryName = ?";
List<Product> productList = new ArrayList<Product>();
DBConnection dbConnection = new DBConnection();
try {
dbConnection.open();
ResultSet resultSet = dbConnection.executeSelect(sql, categoryName); //execute select and apply parameters
//fill productList...
} catch (Exception e) {
//always handle your exceptions
...
} finally {
//don't forget to also close other resources here like ResultSet...
//always close the connection
dbConnection.close();
}
}
Note that in this example the PreparedStatement is not in the getProducts method, it will be a local variable of the executeSelect method.
Additional notes:
When working in an application server, you should not open connections naively e.g. using Class.forName("..."), instead use a database connection pool. You can roll on some database connection pooling libraries like C3P0 as explained here: How to establish a connection pool in JDBC?. Or configure one in your application server, as I explain here: Is it a good idea to put jdbc connection code in servlet class?
If this is for learning purposes, then roll on your own classes to handle the communication with your database. In real world applications, this is not recommended (doesn't mean you should not do it). Instead, use a database connectivity framework like ORMs e.g. JPA (Java official ORM framework) or Hibernate; there are no ORM frameworks that handles database communication like Spring JDBC and MyBatis. The choice is yours.
More info:
Should a database connection stay open all the time or only be opened when needed?
How do servlets work? Instantiation, sessions, shared variables and multithreading. Not directly related to your question, but it will help you understand why to not maintain state in resources that are used in multithreaded environments.

Define connection resource in mywebapp/META-INF/context.xml file
<Resource name="jdbc/mydb" auth="Container" type="javax.sql.DataSource"
maxActive="10" maxIdle="2" maxWait="20000"
driverClassName="com.mysql.jdbc.Driver"
username="myuser" password="mypwd"
url="jdbc:mysql://localhost:3306/mydb?useUnicode=true&characterEncoding=utf8"
validationQuery="SELECT 1" />
Create DB.java helper class to minimize code in other parts of app
import java.sql.*;
import javax.sql.DataSource;
import javax.naming.Context;
import javax.naming.InitialContext;
public class DB {
public static Connection createConnection() throws SQLException {
try {
Context ctx = new InitialContext();
DataSource ds = (DataSource)ctx.lookup("java:comp/env/jdbc/mydb");
return ds.getConnection();
} catch (SQLException ex) {
throw ex;
} catch (Exception ex) {
SQLException sqex = new SQLException(ex.getMessage());
sqex.initCause(ex);
throw sqex;
}
}
public static void close(ResultSet rs, Statement stmt, Connection conn) {
if (rs != null) try { rs.close(); } catch (Exception e) { }
if (stmt != null) try { stmt.close(); } catch (Exception e) { }
if (conn != null) try { conn.close(); } catch (Exception e) { }
}
public static void close(ResultSet rs, boolean closeStmtAndConn) {
if (rs==null) return;
try {
Statement stmt = rs.getStatement();
close(rs, stmt, stmt!=null ? stmt.getConnection() : null);
} catch (Exception ex) { }
}
}
And somewhere in your app DAO code use DB helper.
public List<MyBean> getBeans() throws SQLException {
List<MyBean> list = new ArrayList<MyBean>();
ResultSet rs=null;
try {
Connection con = DB.createConnection();
String sql = "Select * from beantable where typeid=?";
PreparedStatement stmt = con.prepareStatement(sql, Statement.NO_GENERATED_KEYS);
stmt.setInt(1, 101);
rs = stmt.executeQuery();
while(rs.next()
list.add( createBean(rs) );
} finally {
DB.close(rs, true); // or DB.close(rs, stmt, conn);
}
return list;
}
private MyBean createBean(ResultSet rs) throws SQLException {
MyBean bean = new MyBean();
bean.setId( rs.getLong("id") );
bean.setName( rs.getString("name" );
bean.setTypeId( rs.getInt("typeid") );
return bean;
}

I would add two methods to the class:
public static void open() throws SomeException;
public static void close() throws SomeException;
then your calling code looks something like this{
try {
DBConnection.open();
... code to use the connection one or more times ...
} finally {
DBConnection.close();
}
Wrap all your database calls inside that and it will take care of closing whether there is an exception thrown or not.
Of course, this isn't much different than having a regular class, which I might recommend:
try {
DBConnection conn = new DBConnection();
conn.open();
... all the code to use the database (but you pass 'conn' around) ...
} finally {
conn.close();
}
And you might want to look at the java.lang.AutoCloseable and java.io.Closeable to see if that helps you.
2
If you are keeping it open across page loads, there isn't any place to put the try ... finally stuff so you can open it and close it when the servlet closes or the server closes or something like that.
If you are going to leave it open, you need to make sure and add code to verify it doesn't close when you aren't looking. A short network glitch, for example, could close it down. In that case, you need to reopen it when it gets closed. Otherwise, all database access from that point will fail.
You might want to look into the concept of a DataBase Pool. Apache has one -- DBCP. Tomcat has its own that's quite good. Other containers, like JBOSS, WebSphere, WebLogic all have them. There's a couple that can be used with the Spring Framework. What it does is manage one or more database connections. Your code asks it for one and it returns an open one, unless none is available and then it opens one and returns it. You call close when your code gets through with it but it doesn't really close the connection, it just returns it to the pool.
You can usually configure the pool to check for shut down connections and reopen if needed.