I'm creating a server-side Java task that executes the same SQL UPDATE every 60-seconds forever so it is ideal for using a java.sql.PreparedStatement.
I would rather re-connect to the database every 60-seconds than assume that a single connection will still be working months into the future. But if I have to re-generate a new PreparedStatement each time I open a new connection, it seems like it is defeating the purpose.
My question is: since the PreparedStatement is created from a java.sql.Connection does it mean that the connection must be maintained in order to use the PreparedStatement efficiently or is the PreparedStatement held in the database and not re-compiled with each new connection? I'm using postgresql at the present, but may not always.
I suppose I could keep the connection open and then re-open only when an exception occurs while attempting an update.
Use a database connection pool. This will maintain the connections alive in sleep mode even after closing them. This approach also saves performance for your application.
Despite the connection that created the PreparedStatement, the SQL statement will be cached by the database engine and there won't be any problems when recreating the PreparedStatement object.
Set your connection timeout to the SQL execution time+few minutes.
Now, you can take 2 different approaches here -
Check before executing the update, if false is returned then open new Connection
if( connection == null || !connection.isValid(0)) {
// open new connection and prepared statement
}
Write a stored procedure in the Db, and call it passing necessary params. This is an alternate approach.
Regarding you approach of closing and opening db connection every 60 seconds for the same prepared statement, it does not sound like a good idea.
Related
PreparedStatment ps = null;
public void executeQueries(){
try{
ps = conn.prepareStatement(Query1);
// Execute Query1 here and do the processing.
ps = conn.prepareStatement(Query2);
// Execute Query2 here and do the processing.
//... more queries
}catch(){}
finally{
ps.close(); // At this point would the caching of queries in DB be lost?
}
}
In my Application, I call the method executeQueries() frequently.
My question is, If I close the PreparedStatement in the finally block inside the method (that I use frequently), would the database system remove the caching? If YES, can I make a global PreparedStatement for the entire application as there are loads of JAVA CLASSES in my application that query the database.
Thank you!
Update : The question has been marked duplicate but the linked thread does not answer my question at all. AFAIK, the database system stores the executed queries in the cache memory. It also stores their execution plan. This is where PreparedStatement perfoms better than Statement. However, I am not very sure if the information related to the query is removed once the PreparedStatement is closed.
Specifically with regard to MySQL, according to
8.10.3 Caching of Prepared Statements and Stored Programs
The server maintains caches for prepared statements and stored programs on a per-session basis. Statements cached for one session are not accessible to other sessions. When a session ends, the server discards any statements cached for it.
So closing a PreparedStatement would not remove the statement(s) from the cache, but closing the Connection presumably would.
... unless the application uses a connection pool, in which case closing the Connection may not necessarily end the database session; it may keep the session open and just return the connection to the pool.
Then there's also the question of whether the statements are actually being PREPAREd on the server. That is controlled by the useServerPrepStmts connection string attribute. IIRC, by default, server-side prepared statements are not enabled.
This question already has answers here:
Closing JDBC Connections in Pool
(3 answers)
Best approach for returning connection objects to HikariCP pool
(1 answer)
Closed 4 years ago.
I have a standalone java application that gets streams of messages, batches and inserts them to a SQL Server database using Hikaricp.
Currently what I do is the following:
Get a connection from the pool.
Create a prepared statement to insert using the connection
Execute the batch insert.
Note that I never close the connection! And once I reach the maxPoolSize (20), I get an error when I tried to get a new connection.
Should I be closing the connection after every batch insert?
Get a connection from the pool.
Create a prepared statement to insert using the connection
Execute the batch insert.
Close the prepared statement & connection.
However, this means that I am incurring the cost of getting a connection from the pool + creating a new prepared statement after every batch insert.
Is this the recommended approach or are there any alternatives which can reduce this extra cost?
You need to close connection. usually pools return connections wrapped into another object. And when you call close on wrapper it simply marks internal database connection as free and returns it to the pool.
Also you can reuse existing PreparedStatement objects if they are the same. If each of your tasks has to use its unique PreparedStatement (due to your business logic) then there is no other way and you have to create new PreparedStatement for each task.
I have been unable to find an exact answer to this question. I'm using C3P0's ComboPooledDataSource. Which of these methodologies is better practice:
dataSource = connectionClass.getDataSource();
conn = dataSource.getConnection;
executeQuery(query1, conn);
executeQuery(query2, conn);
...
executeQuery(finalQuery, conn);
conn.close();
OR
executeQuery(query1);
executeQuery(query2);
...
executeQuery(finalQuery);
where executeQuery:
conn = dataSource.getConnection;
st = conn.createStatement();
rs = executeQuery(query);
conn.closed();
In short, I have to do a decent amount of queries every so often. Is it better to go with the first design, which gets the connection once for each batch and passes it as an argument. Or is it better to go with the second approach and just get a connection each time I call my executeQuery method. If I was using DriverManager I would obviously choose the first (only get the connection once), but when using the C3P0 package I am not sure if doing that is the right way to go or not. Or does it not matter with such a package?
With a connection pool, the difference is neglectible, because even if you use the second approach, bringing back a pooled connection takes little time. Still, using the first approach is the better way to go, because
It avoids the additional (little) overhead of getting a connection from the pool.
If you later need to introduce transactions (do all of your changes or, in case of an error, conveniently and securely roll back your changes), then the first approach is your only option.
Some comments/suggestions
If you application is single threaded (unless you mention), it does not matter. It even does not matter whether you use connection pool or not. Just use a single connection and pass the same to function where you need it.
Connection pools are useful when the use case involves multiple database connections simultaneously.
Since your application is a batch and single threaded, it does not warrant use of connection pool.
Regarding your application, both the approaches are equivalent. When you call connection.close() on pooled datasource connection, its not actually closed but returned to pool.
I'm using c3p0. I set up a pooled as follows,
cpds = new ComboPooledDataSource();
cpds.setJdbcUrl(...);
/* connection setup */
spds.setMaxStatements(200);
I have an object that prepares several prepared statements on initialization. In order to do that, I grab a connection (con = getConnection()) from the PooledDataSource and then prepare a statement (e.g., PreparedStatement stmt = con.preparedStatemet(/*sql*/)). The prepared statements are stored as private variables in the object and the current connection is closed at the end of initialization (con.close()). The prepared statements are used in methods of the object.
For prepared statements that update the database, this works just fine. However, when I call a method that uses a prepared statement (stmt.executeQuery()) to query the database, I get the following SQLException
java.sql.SQLException: You can't operate on a closed Statement!!!
at com.mchange.v2.sql.SqlUtils.toSQLException(SqlUtils.java:118)
at com.mchange.v2.sql.SqlUtils.toSQLException(SqlUtils.java:77)
at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeQuery(NewProxyPreparedStatement.java:127)
Did I get something wrong concerning the usage of c3p0?
Many thanks in advance!
Edit: Obviously, my question is partly based on my lack of understanding. As was pointed out in the definite answer, a PreparedStatement belongs to a connection and whenever the connection is closed, the associated statements should be closed, as well. But if that is the case, I don't understand what the use of c3p0's statement cache is.
you should get the same Exception calling executeUpdate(). JDBC Connection and Statement pooling is designed to be transparent: the same API that works for unpooled DataSources should be used for pooled versions too. There will be a dramatic difference in performance, but the code should be semantically interchangeable.
in an unpooled environment, it should be obvious why your approach fails: a Statement, prepared or otherwise, is a child of a Connection, without which it can't function. you are hoping that in the pooled environment, even though the Connection has been "closed", it should still exist in the pool, so hey, those Statements might be good. but that's a very bad idea (and if your attempts to do updates really are succeeding after the parent Connection has been close()ed, again, that'd be a bug, a bad one.) once a Connection has been "closed" it goes back in the pool, but not forever. other clients will check it out, and start performing transaction work that shouldn't be interrupted by your stale Statements. eventually Connections will be expired out of the pool. what should happened to your retained PreparedStatements then?
c3p0 pools Statements transparently, meaning you should use exactly the same API you would have used with no pooling. Call prepareStatement(...) on your Connection, every time. if you've enabled Statement pooling in c3p0 (as you have), then internally c3p0 will check to see whether the Statement has already been prepared, and if so it will quietly use the cached version rather than forwarding the request to the dbms.
i hope this helps!
In the tutorial "Using Prepared Statements" it states that they should always be closed. Suppose I have a function
getPrice() {
}
that I expect to be called multiple times per second. Should this method be opening and closing the PreparedStatement with every single method call? This seems like a lot of overhead.
First of all, PreparedStatement are never opened. It's just a prepared Statement that is executed. The statement is sent to the RDBMS that executes the SQL statement compiled by the PreparedStatement. The connection to the SQL statement should be opened during the duration of the SQL querying and closed when no other RDMS calls is needed.
You can send many Statement/PreparedStatement as you require provided that you finally close its ResultSet and PreparedStatement once you're completed with them and then close the RDBMS connection.
Should this method be opening and closing the PreparedStatement with every single method call?
If you are creating the PreparedStatement object within the method, then you must close it, once you are done with it. You may reuse the PreparedStatement object for multiple executions, but once you are done with it, you must close it.
This is because, although all Statement objects (including PreparedStatements) are supposed to be closed on invoking Connection.close(), it is rarely the case. In certain JDBC drivers, especially that of Oracle, the driver will be unable to close the connection if the connection has unclosed ResultSet and Statement objects. This would mean that, on these drivers:
You should never lose a reference to a PreparedStatement object. If you do, then the connection will not be closed, until garbage collection occurs. If you are reusing PreparedStatement instances for different SQL statements, it is easy to forget this.
You should close the PreparedStatement once you no longer need it. Only then can the Connection.close() actually tear down the physical connection.
As the example in the tutorial shows you should close it after all your queries have been performed.
Once the statement is closed the RDMS may release all resources associated with your statement. Thus to use it further you'd have to re-prepare the very same statement.
I think that, after every database interaction, every component like statement, resultset must be closed, except for connection, if u tend to perform more operation.
And there is no need to worry, if you are creting the prepared statement again and again, because as you will be using the same statement again and again, there wont be any performannce issue.
Yes..No issues are there if you are creating the prepared statement n number of times, because as you will be using the same statement at all the places. No need to have any observation here regarding performance
Thanks