This question already has answers here:
Closing JDBC Connections in Pool
(3 answers)
Best approach for returning connection objects to HikariCP pool
(1 answer)
Closed 4 years ago.
I have a standalone java application that gets streams of messages, batches and inserts them to a SQL Server database using Hikaricp.
Currently what I do is the following:
Get a connection from the pool.
Create a prepared statement to insert using the connection
Execute the batch insert.
Note that I never close the connection! And once I reach the maxPoolSize (20), I get an error when I tried to get a new connection.
Should I be closing the connection after every batch insert?
Get a connection from the pool.
Create a prepared statement to insert using the connection
Execute the batch insert.
Close the prepared statement & connection.
However, this means that I am incurring the cost of getting a connection from the pool + creating a new prepared statement after every batch insert.
Is this the recommended approach or are there any alternatives which can reduce this extra cost?
You need to close connection. usually pools return connections wrapped into another object. And when you call close on wrapper it simply marks internal database connection as free and returns it to the pool.
Also you can reuse existing PreparedStatement objects if they are the same. If each of your tasks has to use its unique PreparedStatement (due to your business logic) then there is no other way and you have to create new PreparedStatement for each task.
Related
In my application I have multithreads that needs to access database and I am using apache.tomcat.jdbc.pool.DataSource as a JDBC connection pool.
In some cases users execute stored procedures that might affect the database session context/variables before executing another query to retrieve some data.
After a thread is finished, connection is closed but the way the pooled connections work, the connection is not actually closed but rather returned to the connection pool for reuse. The problem is similar to this question question.
The problem is that these variables usually affect the data retrieved from the database. So when another thread acquire a connection where these variables/context where set and then query the database the data retrieved is affected by the context/variables.
What is the best way to handle such an issue? I have outlined some solutions but not sure which is best practise and not how to implement them all?
Execute a Procedure / Statement that reset session / variables before releasing the connection back to the pool:
This solution could work but the issue is my application use different databases. For now MySQL and Oracle are supported. For Oracle we have RESET_PACKAGE Procedure, but there is no equivalent for MySQL. Is there a way to do it in MySQL? also what if new databases are supported?
Is there a way to enforce or explicitly close the actual/physical connection instead of just returning it to the pool? or Is there a property to enforce pool to close the connection?
Does rollback to a savepoint revert db session variables / context?
Savepoint savepoint = connection.setSavepoint();
// execute some procedure connection that affect session
connection.rollback(savepoint);
connection.close();
Or is there any other way to enforce clearing of session or restart/close the actual connection?
My question is related to this question.
On MySQL you might perform the sequence:
BEGIN;
COMMIT RELEASE;
as your validationQuery. The RELEASE part disconnects the current session and creates a new one (cf. MySQL Documentation).
I have three problems for which I am not finding a solution. We are using connection pool in our project. We see in our project the statements are closed after the connection is closed. I know that in case of a connection pool, after a connection is closed the physical connection to the database is not closed but returns to the pool for reuse. So my questions are:
What will happen if statements are closed after a connection is closed? Will the statements be closed properly/will closing the connection close all the statements and closing the statements are redundant/the statements are open and though the connection returns to the pool, it is not reusable because of open statements? (We are using both Statement and PreparedStatement).
Can I use the same Statement/PreparedStatement object for multiple queries? I know we can reuse the same PreparedStatement for executing the same query mane times with multiple inputs, but my question is can I use it for executing two different queries at different points in time? Like:
PreparedStatement pstme = null;
// Do Stuff
pstmt = con.prepareStatement(sql1);
//Do Stuff
pstmt = con.prepareStatement(sql2);
//Do stuff
pstmt.close();
con.close();
If there any way to check for connection leaks in application and avoid them, instead of going through each line of code?
I'm creating a server-side Java task that executes the same SQL UPDATE every 60-seconds forever so it is ideal for using a java.sql.PreparedStatement.
I would rather re-connect to the database every 60-seconds than assume that a single connection will still be working months into the future. But if I have to re-generate a new PreparedStatement each time I open a new connection, it seems like it is defeating the purpose.
My question is: since the PreparedStatement is created from a java.sql.Connection does it mean that the connection must be maintained in order to use the PreparedStatement efficiently or is the PreparedStatement held in the database and not re-compiled with each new connection? I'm using postgresql at the present, but may not always.
I suppose I could keep the connection open and then re-open only when an exception occurs while attempting an update.
Use a database connection pool. This will maintain the connections alive in sleep mode even after closing them. This approach also saves performance for your application.
Despite the connection that created the PreparedStatement, the SQL statement will be cached by the database engine and there won't be any problems when recreating the PreparedStatement object.
Set your connection timeout to the SQL execution time+few minutes.
Now, you can take 2 different approaches here -
Check before executing the update, if false is returned then open new Connection
if( connection == null || !connection.isValid(0)) {
// open new connection and prepared statement
}
Write a stored procedure in the Db, and call it passing necessary params. This is an alternate approach.
Regarding you approach of closing and opening db connection every 60 seconds for the same prepared statement, it does not sound like a good idea.
This question already has answers here:
Must JDBC Resultsets and Statements be closed separately although the Connection is closed afterwards?
(12 answers)
Closed 9 years ago.
I know the safe pattern in Java is to close your ResultSet, Statement, and Connection in order in a finally block.
If you close connection and then try to close statement(doesnt throw exception). But if you try to call any method from statement an exception is thrown.
I was wondering does closing connection automatically close all the statement objects created out of that connection?
Update:
I am using DatabaseProductVersion: Oracle Database 11g Release 11.1.0.0.0
DriverName: Oracle JDBC driver
DriverVersion: 10.2.0.4.0
Yes it does, Connection.close API says "Releases this Connection object's database and JDBC resources immediately instead of waiting for them to be automatically released". The problem is that applications typically use database connection pools and these may simply return Connection to pool on Connection.close.
In any case, it's a good practice to always close ResultSet and Statement explicitly and not to rely on Connection.close.
Besides, it's not the best idea to work with JDBC directly. You can use Spring JDBC instead and forget about releasing resources problem.
The details are ultimately down to each JDBC driver implementation; however, once a connection to the database is closed, everything related to it is disposed at the DB side, so there is nothing much the client side can do but auto-close the objects representing these resources.
You never know in what ways the databeses/drivers could be broken (there may be resource leaks, for example), therefore the best practice recommendation is to close everything explicitly.
In JDBC can we say the transaction begins as soon as we get the connection and finishes
as we close the connection. IS this right? If yes can we say In different requests sharing
the same connection, even all the uncommitted transactions will be visible to all
all requests?
#BalusC - This is not correct really. By default autocommit is set to true, which means that transaction begins before any JDBC operation and finishes RIGHT AFTER that single operation - not when connection is closed.
But you are right that sharing connection is indeed not good, If you want to multi-thread your DB it's best to handle it in the way that you have thread pool (look for ThreadPoolExecutor in java.util.concurrent) and for each thread you get a separate connection. ConnectionPool is also a good one, but I would rather limit that through ThreadPool - this way there is never a thread waiting for a connection from the pool.
That's right. That's the default behaviour. You can take over this control by setting the auto commit to false by connection.setAutoCommit(false) after retrieving the connection and committing the transaction by connection.commit() after doing all queries.
However, sharing a connection between different requests (threads) is at its own already a bad design. Your application is this way not threadsafe. You do not want to share the same connection among different threads. If all you want is eliminating the cost of connecting a database, then you should consider using a connection pool.
first rule when you access the database.
Every nontransactional operation should:
1.open connection, if have connection pool then get connection from the pool
2. Create execute statement
3. if is read query then map the result set.
4. close the result set.
5. close the statement.
6. close the connection.
if you want your operation to be in transaction then you should consider this approach:
Operation 1:
1. getSharedConnection
2.create/execute statement
3. if is read query then map the result set.
4. close resultSEt
5. close statement
operation 2:
Same as operation 1.
and the transaction:
public void updateInTransaction(){
Connection conn=pool.getConnection();//or you can create a new connection
conn.setAutocommit(false);
operation1(conn);
operation2(conn);
conn.close;
}
This is just basics for small applications.
If you are developing bigger applicatoin you should using same framework like JDBCTemplates from Springsoruce.