I am implementing a server with MySQL server.
I have come to a conclusion that all my queries will be sent using prepared statements.
I am going to make multiple singleton prepared statements for different purposes(For example, select token, insert/replace token, insert phone numbers ...).
Here are my questions.
For singleton prepared statements to perform well, should I close it every time I execute a query? (close only the ResultSet, not prepared statements or the connection)
Will singleton prepared statement be better than creating a statement every time?
Simple answer - create and close statement every time.
If you are not implementing a VERY specialized batch functionality, then creating and closing prepared statement will be an insignificant overhead.
Also having a singleton connection is not a good idea for server application. Only a single thread can work with one connection at any time.
Related
Let's say we have a class that writes in a database a log message. This class is called from different parts of the code and executes again and again the same INSERT statement. It seems that is calling to use a PreparedStatement.
However I am wondering what is the right usage of it. Do I still get the benefit of using it, like the DBMS using the same execution path each time it is executed, even if I create a new PreparedStatement each time the method is called or should I have a PreparedStatement as a class member and never close it in order to re use it and get benefit from it?
Now, if the only way to obtain benefit using the PreparedStatement in this scenario is to keeping it opened as class member, may the same connection have different PreparedStatement's (with different queries) opened at the same time? What happens when two of these PreparedStatements are executed at the same time? Does the JDBC driver queue the execution of the PreparedStatements?
Thanks in advance,
Dani.
For all I know and experienced, statements don't run in parallel on one connection. And as you observed correctly, PreparedStatements are bound to the Connection they were created on.
As you probably don't want to synchronize your logging call (one insert at a time plus locking overhead), you'd have to keep the connections reserved for this logging statement.
But having a dedicated pool for only one statement seems very wasteful - don't want to do that as well.
So what options are left?
prepare the statement for every insert. As you'll have I/O operations to send data to the db, the overhead of preparing is relatively small.
prepare the statement inside your pool on creating a new connection and build a Map <Connection,PreparedStatement> to reference them later. Makes creating new connections a bit slower but allowes to recycle the statement.
Use some async way to queue your logs (JMS) and do the Insert as batch inside a message driven bean or similar
Probably some more options - but that's all I could think of right now.
Good luck with that.
I'm using c3p0. I set up a pooled as follows,
cpds = new ComboPooledDataSource();
cpds.setJdbcUrl(...);
/* connection setup */
spds.setMaxStatements(200);
I have an object that prepares several prepared statements on initialization. In order to do that, I grab a connection (con = getConnection()) from the PooledDataSource and then prepare a statement (e.g., PreparedStatement stmt = con.preparedStatemet(/*sql*/)). The prepared statements are stored as private variables in the object and the current connection is closed at the end of initialization (con.close()). The prepared statements are used in methods of the object.
For prepared statements that update the database, this works just fine. However, when I call a method that uses a prepared statement (stmt.executeQuery()) to query the database, I get the following SQLException
java.sql.SQLException: You can't operate on a closed Statement!!!
at com.mchange.v2.sql.SqlUtils.toSQLException(SqlUtils.java:118)
at com.mchange.v2.sql.SqlUtils.toSQLException(SqlUtils.java:77)
at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeQuery(NewProxyPreparedStatement.java:127)
Did I get something wrong concerning the usage of c3p0?
Many thanks in advance!
Edit: Obviously, my question is partly based on my lack of understanding. As was pointed out in the definite answer, a PreparedStatement belongs to a connection and whenever the connection is closed, the associated statements should be closed, as well. But if that is the case, I don't understand what the use of c3p0's statement cache is.
you should get the same Exception calling executeUpdate(). JDBC Connection and Statement pooling is designed to be transparent: the same API that works for unpooled DataSources should be used for pooled versions too. There will be a dramatic difference in performance, but the code should be semantically interchangeable.
in an unpooled environment, it should be obvious why your approach fails: a Statement, prepared or otherwise, is a child of a Connection, without which it can't function. you are hoping that in the pooled environment, even though the Connection has been "closed", it should still exist in the pool, so hey, those Statements might be good. but that's a very bad idea (and if your attempts to do updates really are succeeding after the parent Connection has been close()ed, again, that'd be a bug, a bad one.) once a Connection has been "closed" it goes back in the pool, but not forever. other clients will check it out, and start performing transaction work that shouldn't be interrupted by your stale Statements. eventually Connections will be expired out of the pool. what should happened to your retained PreparedStatements then?
c3p0 pools Statements transparently, meaning you should use exactly the same API you would have used with no pooling. Call prepareStatement(...) on your Connection, every time. if you've enabled Statement pooling in c3p0 (as you have), then internally c3p0 will check to see whether the Statement has already been prepared, and if so it will quietly use the cached version rather than forwarding the request to the dbms.
i hope this helps!
I have made a java class which runs in it's own thread that works through a queue of mysql queries so that it doesn't block the main application thread. I want to use prepared statements, however if I keep reusing the same prepared statement then if there's two or more of that same prepared statement in the queue (I use the same preparedstatement object for each query of that type), it will have the wrong params. If I make a new preparedstatement each time will it recompile the prepared statement each time it is run or will it detect that it's already been compiled and just execute?
I think you really wont be able to utilize Prepared Statement Pooling feature available in Java connection pooling implementations like Apache DBCP. This is because you are storing prepared statement objects in the queue. If you could store your SQL and paramenters in a custom class instead, and create / execute PreparedStatements in execution thread you can get the benefit of pooling by using something like DBCP
See DBCP configurations docs on how to enable statement pooling (poolPreparedStatements)
will it recompile the prepared statement each time it is run or will
it detect that it's already been compiled and just execute?
It's up to the DBMS, not Java, but the general idea is that it can detect a reuse of an existing compiled statement and pools them under the hood, either in the driver or at the server. See the JDBC 4.0 Specification, #11.6, "Reuse of Statements by Pooled
Connections".
From the java tutorial
If you want to execute a Statement object many times, it usually reduces execution time to use a PreparedStatement object instead.
The main feature of a PreparedStatement object is that, unlike a Statement object, it is given a SQL statement when it is created. The advantage to this is that in most cases, this SQL statement is sent to the DBMS right away, where it is compiled. As a result, the PreparedStatement object contains not just a SQL statement, but a SQL statement that has been precompiled. This means that when the PreparedStatement is executed, the DBMS can just run the PreparedStatement SQL statement without having to compile it first.
http://docs.oracle.com/javase/tutorial/jdbc/basics/prepared.html#supply_values_ps
In the tutorial "Using Prepared Statements" it states that they should always be closed. Suppose I have a function
getPrice() {
}
that I expect to be called multiple times per second. Should this method be opening and closing the PreparedStatement with every single method call? This seems like a lot of overhead.
First of all, PreparedStatement are never opened. It's just a prepared Statement that is executed. The statement is sent to the RDBMS that executes the SQL statement compiled by the PreparedStatement. The connection to the SQL statement should be opened during the duration of the SQL querying and closed when no other RDMS calls is needed.
You can send many Statement/PreparedStatement as you require provided that you finally close its ResultSet and PreparedStatement once you're completed with them and then close the RDBMS connection.
Should this method be opening and closing the PreparedStatement with every single method call?
If you are creating the PreparedStatement object within the method, then you must close it, once you are done with it. You may reuse the PreparedStatement object for multiple executions, but once you are done with it, you must close it.
This is because, although all Statement objects (including PreparedStatements) are supposed to be closed on invoking Connection.close(), it is rarely the case. In certain JDBC drivers, especially that of Oracle, the driver will be unable to close the connection if the connection has unclosed ResultSet and Statement objects. This would mean that, on these drivers:
You should never lose a reference to a PreparedStatement object. If you do, then the connection will not be closed, until garbage collection occurs. If you are reusing PreparedStatement instances for different SQL statements, it is easy to forget this.
You should close the PreparedStatement once you no longer need it. Only then can the Connection.close() actually tear down the physical connection.
As the example in the tutorial shows you should close it after all your queries have been performed.
Once the statement is closed the RDMS may release all resources associated with your statement. Thus to use it further you'd have to re-prepare the very same statement.
I think that, after every database interaction, every component like statement, resultset must be closed, except for connection, if u tend to perform more operation.
And there is no need to worry, if you are creting the prepared statement again and again, because as you will be using the same statement again and again, there wont be any performannce issue.
Yes..No issues are there if you are creating the prepared statement n number of times, because as you will be using the same statement at all the places. No need to have any observation here regarding performance
Thanks
We all know that we should rather reuse a JDBC PreparedStatement than creating a new instance within a loop.
But how to deal with PreparedStatement reuse between different method invocations?
Does the reuse-"rule" still count?
Should I really consider using a field for the PreparedStatement or should I close and re-create the prepared statement in every invocation (keep it local)?
(Of course an instance of such a class would be bound to a Connection which might be a disadvantage in some architectures)
I am aware that the ideal answer might be "it depends".
But I am looking for a best practice for less experienced developers that they will do the right choice in most of the cases.
Of course an instance of such a class would be bound to a Connection which might be a disadvantage
Might be? it would be a huge disadvantage. You'd either need to synchronize access to it, which would kill your multi-user performance stone-dead, or create multiple instances and keep them in a pool. Major pain in the ass.
Statement pooling is the job of the JDBC driver, and most, if not all, of the current crop of drivers do this for you. When you call prepareStatement or prepareCall, the driver will handle re-use of existing resource and pre-compiled statements.
Statement objects are tied to a connection, and connections should be used and returned to the pool as quickly as possible.
In short, the standard practice of obtaining a PreparedStatement at the start of the method, using it repeatedly within a loop, then closing it at the end of the method, is best practice.
Many database workloads are CPU-bound, not IO-bound. This means that the database ends up spending more time doing work such as parsing SQL queries and figuring out how to handle them (doing the 'execution plan'), than it spends accessing the disk. This is more true of 'transactional' workloads than 'reporting' workloads, but in both cases the time spent preparing the plan may be more than you expect.
Thus it is always a good idea, if the statement is going to be executed frequently and the hassle of making (correct) arrangements to cache PreparedStatements 'between method invocations' is worth your developer time. As always with performance, measurement is key, but if you can do it cheaply enough, cache your PreparedStatement out of habit.
Some JDBC drivers and/or connection pools offer transparent 'prepared statement caching', so that you don't have to do it yourself. So long as you understand the behaviour of your particular chosen transparent caching strategy, it's fine to let it keep track of things ... what you really want to avoid is the hit on the database.
Yes it can be reused, but I believe this only counts if the same Connection object is being used and if you are using a Database Connection Pool (from within a Web Application, for example) then the Connection objects will be potentially different each time.
I always recreate the PreparedStatement before each use within a Web Application for this reason.
If you aren't using a Connection Pool then you are golden!
I don't see the difference: If I execute the same statement repeatedly against the same connection, why not reuse the PreparedStatement in any way? If multiple methods execute the same statement, then maybe that statement needs to be encapsulated in its own method (or even its own class). That way you wouldn't need to pass around a PreparedStatement.