Develop Reference

Correct usage of PreparedStatement

Let's say we have a class that writes in a database a log message. This class is called from different parts of the code and executes again and again the same INSERT statement. It seems that is calling to use a PreparedStatement.
However I am wondering what is the right usage of it. Do I still get the benefit of using it, like the DBMS using the same execution path each time it is executed, even if I create a new PreparedStatement each time the method is called or should I have a PreparedStatement as a class member and never close it in order to re use it and get benefit from it?
Now, if the only way to obtain benefit using the PreparedStatement in this scenario is to keeping it opened as class member, may the same connection have different PreparedStatement's (with different queries) opened at the same time? What happens when two of these PreparedStatements are executed at the same time? Does the JDBC driver queue the execution of the PreparedStatements?
Thanks in advance,
Dani.

For all I know and experienced, statements don't run in parallel on one connection. And as you observed correctly, PreparedStatements are bound to the Connection they were created on.
As you probably don't want to synchronize your logging call (one insert at a time plus locking overhead), you'd have to keep the connections reserved for this logging statement.
But having a dedicated pool for only one statement seems very wasteful - don't want to do that as well.
So what options are left?
prepare the statement for every insert. As you'll have I/O operations to send data to the db, the overhead of preparing is relatively small.
prepare the statement inside your pool on creating a new connection and build a Map <Connection,PreparedStatement> to reference them later. Makes creating new connections a bit slower but allowes to recycle the statement.
Use some async way to queue your logs (JMS) and do the Insert as batch inside a message driven bean or similar
Probably some more options - but that's all I could think of right now.
Good luck with that.

Why is it impossible to serve connections from pool according to the prepared statement already executed with them?

I've been researching all around the web the most efficient way to design a connection pool and tried to analyze into details the available libraries (HikariCP, BoneCP, etc.).
Our application is a heavy-load consumer webapp and most of the time the users are working on similar business objects (thus the underlying SQL queries executed are the often the same, but still there are numerous).
It is designed to work with different DBMS (Oracle and MS SQL Server especially).
So a simplified use case would be :
User goes on a particular JSP page (e.g. Enterprise).
A corresponding Bean is created.
Each time it realizes an action (e.g. getEmployees(), computeTurnover()), the Bean asks the pool for a connection and returns it back when done.
If we want to take advantage of the Prepared Statement caching of the underlying JDBC driver (as PStatements are attached to a connection - jTDS doc.), from what I understand an optimal way of doing it would be :
Analyze what kind of SQL query a particular Bean want to execute before providing it an available connection from the pool.
Find a connection where the same prepared statement has already been executed if possible.
Serve the connection accordingly (and use the benefits of the cache/precompiled statement).
Return the connection to the pool and start over.
Am I missing an important point here (like JDBC drivers capable of reusing cached statements regardless of the connection) or is my analysis correct ?
The different sources I found state it is not possible, but why ?

For your scheme to work, you'd need to be able to get the connection that already has that statement prepared.
This falls foul on two points:
In JDBC you obtain the connection first,
Cached prepared statements (if a driver or connection pool even supports that) aren't exposed in a standardized way (if at all) nor would you be able to introspect them.
The performance overhead of finding the right connection (and the subsequent contention on the few connections that already have it prepared) would probably undo any benefit of reusing the prepared statement.
Also note that some database systems also have a serverside cache for prepared statements (meaning that it already has the plan etc available), limiting the overhead from a new prepare from the client.
If you really think the performance benefit is big enough, you should consider using a data source specific for this functionality (so it is almost guaranteed that the connection will have the statement in its cache).

A solution could be for a connection pool implementation to delay retrieving the connection from the pool until the Connection.prepareStatement() is called. At that time a connection pool would look up available connections by the SQL statement text and then play forward all the calls made before Connection.prepareStatement(). This way it would be possible to get a connection with a ready PreparedStatement without the issues other guys suggested.
In other words, when you request a connection from the pool, it would return a wrapper that logs everything until the first operation requiring DB access (such as prepareStatement() is requested.
You'd need to ask a vendor of your connection pool functionality to add this feature.
I've logged this request with C3P0:
https://github.com/swaldman/c3p0/issues/55
Hope this helps.

Java SQL PreparedStatement and maintaining connection

I'm creating a server-side Java task that executes the same SQL UPDATE every 60-seconds forever so it is ideal for using a java.sql.PreparedStatement.
I would rather re-connect to the database every 60-seconds than assume that a single connection will still be working months into the future. But if I have to re-generate a new PreparedStatement each time I open a new connection, it seems like it is defeating the purpose.
My question is: since the PreparedStatement is created from a java.sql.Connection does it mean that the connection must be maintained in order to use the PreparedStatement efficiently or is the PreparedStatement held in the database and not re-compiled with each new connection? I'm using postgresql at the present, but may not always.
I suppose I could keep the connection open and then re-open only when an exception occurs while attempting an update.

Use a database connection pool. This will maintain the connections alive in sleep mode even after closing them. This approach also saves performance for your application.
Despite the connection that created the PreparedStatement, the SQL statement will be cached by the database engine and there won't be any problems when recreating the PreparedStatement object.

Set your connection timeout to the SQL execution time+few minutes.
Now, you can take 2 different approaches here -
Check before executing the update, if false is returned then open new Connection
if( connection == null || !connection.isValid(0)) {
// open new connection and prepared statement
}
Write a stored procedure in the Db, and call it passing necessary params. This is an alternate approach.
Regarding you approach of closing and opening db connection every 60 seconds for the same prepared statement, it does not sound like a good idea.

c3p0 Prepared Statement closed for no apparent reason

I'm using c3p0. I set up a pooled as follows,
cpds = new ComboPooledDataSource();
cpds.setJdbcUrl(...);
/* connection setup */
spds.setMaxStatements(200);
I have an object that prepares several prepared statements on initialization. In order to do that, I grab a connection (con = getConnection()) from the PooledDataSource and then prepare a statement (e.g., PreparedStatement stmt = con.preparedStatemet(/*sql*/)). The prepared statements are stored as private variables in the object and the current connection is closed at the end of initialization (con.close()). The prepared statements are used in methods of the object.
For prepared statements that update the database, this works just fine. However, when I call a method that uses a prepared statement (stmt.executeQuery()) to query the database, I get the following SQLException
java.sql.SQLException: You can't operate on a closed Statement!!!
at com.mchange.v2.sql.SqlUtils.toSQLException(SqlUtils.java:118)
at com.mchange.v2.sql.SqlUtils.toSQLException(SqlUtils.java:77)
at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeQuery(NewProxyPreparedStatement.java:127)
Did I get something wrong concerning the usage of c3p0?
Many thanks in advance!
Edit: Obviously, my question is partly based on my lack of understanding. As was pointed out in the definite answer, a PreparedStatement belongs to a connection and whenever the connection is closed, the associated statements should be closed, as well. But if that is the case, I don't understand what the use of c3p0's statement cache is.

you should get the same Exception calling executeUpdate(). JDBC Connection and Statement pooling is designed to be transparent: the same API that works for unpooled DataSources should be used for pooled versions too. There will be a dramatic difference in performance, but the code should be semantically interchangeable.
in an unpooled environment, it should be obvious why your approach fails: a Statement, prepared or otherwise, is a child of a Connection, without which it can't function. you are hoping that in the pooled environment, even though the Connection has been "closed", it should still exist in the pool, so hey, those Statements might be good. but that's a very bad idea (and if your attempts to do updates really are succeeding after the parent Connection has been close()ed, again, that'd be a bug, a bad one.) once a Connection has been "closed" it goes back in the pool, but not forever. other clients will check it out, and start performing transaction work that shouldn't be interrupted by your stale Statements. eventually Connections will be expired out of the pool. what should happened to your retained PreparedStatements then?
c3p0 pools Statements transparently, meaning you should use exactly the same API you would have used with no pooling. Call prepareStatement(...) on your Connection, every time. if you've enabled Statement pooling in c3p0 (as you have), then internally c3p0 will check to see whether the Statement has already been prepared, and if so it will quietly use the cached version rather than forwarding the request to the dbms.
i hope this helps!

Multithreading - MySQL java connector prepared statement use

I have made a java class which runs in it's own thread that works through a queue of mysql queries so that it doesn't block the main application thread. I want to use prepared statements, however if I keep reusing the same prepared statement then if there's two or more of that same prepared statement in the queue (I use the same preparedstatement object for each query of that type), it will have the wrong params. If I make a new preparedstatement each time will it recompile the prepared statement each time it is run or will it detect that it's already been compiled and just execute?

I think you really wont be able to utilize Prepared Statement Pooling feature available in Java connection pooling implementations like Apache DBCP. This is because you are storing prepared statement objects in the queue. If you could store your SQL and paramenters in a custom class instead, and create / execute PreparedStatements in execution thread you can get the benefit of pooling by using something like DBCP
See DBCP configurations docs on how to enable statement pooling (poolPreparedStatements)

will it recompile the prepared statement each time it is run or will
it detect that it's already been compiled and just execute?
It's up to the DBMS, not Java, but the general idea is that it can detect a reuse of an existing compiled statement and pools them under the hood, either in the driver or at the server. See the JDBC 4.0 Specification, #11.6, "Reuse of Statements by Pooled
Connections".

From the java tutorial
If you want to execute a Statement object many times, it usually reduces execution time to use a PreparedStatement object instead.
The main feature of a PreparedStatement object is that, unlike a Statement object, it is given a SQL statement when it is created. The advantage to this is that in most cases, this SQL statement is sent to the DBMS right away, where it is compiled. As a result, the PreparedStatement object contains not just a SQL statement, but a SQL statement that has been precompiled. This means that when the PreparedStatement is executed, the DBMS can just run the PreparedStatement SQL statement without having to compile it first.
http://docs.oracle.com/javase/tutorial/jdbc/basics/prepared.html#supply_values_ps