I have a severe problem with my database connection in my web application. Since I use a single database connection for the whole application from singleton Database class, if i try concurrent db operations (two users) the database rollsback the transactions.
This is my static method used:
All threads/servlets call static Database.doSomething(...) methods, which in turn call the the below method.
private static /* synchronized*/ Connection getConnection(final boolean autoCommit) throws SQLException {
if (con == null) {
con = new MyRegistrationBean().getConnection();
}
con.setAutoCommit(true); //TODO
return con;
}
What's the recommended way to manage this db connection/s I have, so that I don't incurr in the same problem.
Keeping a Connection open forever is a very bad idea. It doesn't have an endless lifetime, your application may crash whenever the DB times out the connection and closes it. Best practice is to acquire and close Connection, Statement and ResultSet in the shortest possible scope to avoid resource leaks and potential application crashes caused by the leaks and timeouts.
Since connecting the DB is an expensive task, you should consider using a connection pool to improve connecting performance. A decent applicationserver/servletcontainer usually already provides a connection pool feature in flavor of a JNDI DataSource. Consult its documentation for details how to create it. In case of for example Tomcat you can find it here.
Even when using a connection pool, you still have to write proper JDBC code: acquire and close all the resources in the shortest possible scope. The connection pool will on its turn worry about actually closing the connection or just releasing it back to pool for further reuse.
You may get some more insights out of this article how to do the JDBC basics the proper way. As a completely different alternative, learn EJB and JPA. It will abstract away all the JDBC boilerplate for you into oneliners.
Hope this helps.
See also:
Is it safe to use a static java.sql.Connection instance in a multithreaded system?
Am I Using JDBC Connection Pooling?
How should I connect to JDBC database / datasource in a servlet based application?
When is it necessary or convenient to use Spring or EJB3 or all of them together?
I've not much experience with PostgreSql, but all the web applications I've worked on have used a single connection per set of actions on a page, closing it and disposing it when finished.
This allows the server to pool connections and stops problems such as the one that you are experiencing.
Singleton should be the JNDI pool connection itself; Database class with getConnection(), query methods et al should NOT be singleton, but can be static if you prefer.
In this way the pool exists indefinitely, available to all users, while query blocks use dataSource.getConnection() to draw a connection from the pool; exec the query, and then close statement, result set, and connection (to return it to the pool).
Also, JNDI lookup is quite expensive, so it makes sense to use a singleton in this case.
Related
How to reset a JDBC Connection object (that is, a java.sql.Connection object)?
I have explored the concept of connection pooling. When a connection pool is used, a Connection object can be recycled. But how can a connection pool recycle a Connection object? I think that a connection needs to be "reset" (for example, if it is in a transaction, then perhaps rollback it, but there may be more things to reset) before it can be reused. But I cannot find such a "reset" method in the Java documentation about the class java.sql.Connection.
If you are talking about what you as a user of a connection should do, then that is simple: call close() on the connection. Closing the logical connection will signal to the connection pool that the connection is available for reuse, and the connection pooling manager is then responsible for performing the necessary reset, invalidation of statement handles, etc.
If you are talking about what you as the implementer of a connection pooling manager should do, that is where things become complicated.
Historically, JDBC provides no way to 'reset' a java.sql.Connection, other than by your own code (or your third-party connection pool) remembering the initial configuration, and restoring it after use, and keeping track of objects like statements and result sets and closing them when the connection is returned to the pool.
Originally, the intended way to reset connections in JDBC was for an application server to use a driver's javax.sql.ConnectionPoolDataSource as a factory for javax.sql.PooledConnection objects. These PooledConnection objects serve as handles for physical connections to be held in a connection pool (to be clear, ConnectionPoolDataSource is not a connection pool, it is a data source for a connection pool). The application server would then expose a javax.sql.DataSource handing out logical Connection objects, where on Connection.close(), the driver specific implementation of PooledConnection would take care of any necessary reset of a connection (though JDBC underspecifies what is 'necessary').
However, in practice this route is hardly ever used because support in drivers was (and often still is) spotty, inconsistent or downright incorrect, and JDBC wasn't clear enough on exactly what needed to be done when the logical connection was closed. The world has also shifted to using third-party connection pool libraries that do not use ConnectionPoolDataSource.
JDBC 4.3 (Java 9 and higher) introduced the methods Connection.beginRequest() and Connection.endRequest() to be called by connection pooling managers, which would seem to fit such pattern, but unfortunately JDBC 4.3 doesn't actually specify what kind of things an implementation should or should not do in response to beginRequest and endRequest.
In short, there is no real general way to reset a connection in JDBC.
Do you try to close() method and reinitialize Connection object again.
close()
Releases this Connection object's database and JDBC resources immediately instead of waiting for them to be automatically released
I am really getting confused between these 2 now as below :
1. is the returning of only a Singleton instance to a db connection during the entire run time of a JAVA app
2. is the concept of Thread pool connections in general... I mean if we are planning to have only a singleton instance to a db connection obj, why even have a concept of pools, though i do get what exactly a pool is used for
Are these 2 not very opposite concepts, or am i mixing up anything here...?
is the returning of only a Singleton instance to a db connection during the entire run time of a JAVA app?
you may not want to return a Singleton object for a database connection. You can choose to do if database concurrency not required. In a multi threaded environment best option is to go for Connection pool.
is the concept of Thread pool connections in general..
Establishing a database connection is a very resource-intensive process and involves a lot of overhead. Moreover, in a multi-threaded environment, opening and closing a connection can worsen the situation greatly.
Creating JNDI in server and use it in your web app.
Context context=new InitialContext();
DataSource dataSource=(DataSource)
context.lookup("jdbc/test_jndi");
so when DataSource uses connection pooling, the lookup return a connection from the pool of available connection objects. If there is no available connection, the lookup creates a new connection.
Connection connection=dataSource.getConnection
("testuser","testpwd");
// ...
connection.close();
Once the application is done with database processing, it explicitly closes the connection. This makes the connection available again for reuse again. The closing event of the pooled connection signals the pooling module to restore back to the connection pool.
Any resource shared requires overhead of handling concurrent access so you want to reduce that by not having singletons always. Moreover to reduce resource-intensive process connection pooling is preferred.
I've been researching all around the web the most efficient way to design a connection pool and tried to analyze into details the available libraries (HikariCP, BoneCP, etc.).
Our application is a heavy-load consumer webapp and most of the time the users are working on similar business objects (thus the underlying SQL queries executed are the often the same, but still there are numerous).
It is designed to work with different DBMS (Oracle and MS SQL Server especially).
So a simplified use case would be :
User goes on a particular JSP page (e.g. Enterprise).
A corresponding Bean is created.
Each time it realizes an action (e.g. getEmployees(), computeTurnover()), the Bean asks the pool for a connection and returns it back when done.
If we want to take advantage of the Prepared Statement caching of the underlying JDBC driver (as PStatements are attached to a connection - jTDS doc.), from what I understand an optimal way of doing it would be :
Analyze what kind of SQL query a particular Bean want to execute before providing it an available connection from the pool.
Find a connection where the same prepared statement has already been executed if possible.
Serve the connection accordingly (and use the benefits of the cache/precompiled statement).
Return the connection to the pool and start over.
Am I missing an important point here (like JDBC drivers capable of reusing cached statements regardless of the connection) or is my analysis correct ?
The different sources I found state it is not possible, but why ?
For your scheme to work, you'd need to be able to get the connection that already has that statement prepared.
This falls foul on two points:
In JDBC you obtain the connection first,
Cached prepared statements (if a driver or connection pool even supports that) aren't exposed in a standardized way (if at all) nor would you be able to introspect them.
The performance overhead of finding the right connection (and the subsequent contention on the few connections that already have it prepared) would probably undo any benefit of reusing the prepared statement.
Also note that some database systems also have a serverside cache for prepared statements (meaning that it already has the plan etc available), limiting the overhead from a new prepare from the client.
If you really think the performance benefit is big enough, you should consider using a data source specific for this functionality (so it is almost guaranteed that the connection will have the statement in its cache).
A solution could be for a connection pool implementation to delay retrieving the connection from the pool until the Connection.prepareStatement() is called. At that time a connection pool would look up available connections by the SQL statement text and then play forward all the calls made before Connection.prepareStatement(). This way it would be possible to get a connection with a ready PreparedStatement without the issues other guys suggested.
In other words, when you request a connection from the pool, it would return a wrapper that logs everything until the first operation requiring DB access (such as prepareStatement() is requested.
You'd need to ask a vendor of your connection pool functionality to add this feature.
I've logged this request with C3P0:
https://github.com/swaldman/c3p0/issues/55
Hope this helps.
What are best practices for closing or caching sql connections?
I see things saying to always close it in finally block.
Though I also see things talking about caching them for future use.
How expensive is it to open new connections for each task?
I just started to work on a java data warehousing app someone else programmed. (and I don't have DB experience) It is a single threaded app that loads files into the DB, and does some aggregation and analysis. We have had some issues with deadlocks, which I wouldn't think should be a problem in a single threaded app. I see exceptions are swallowed and not logged all over the class that handles the DB connections, so I am hoping adding logging to those should provide some insight.
But I am hoping for some guidance on best practices for dealing with DB connections in the meantime.
Regardless of whether or not you are using connection pooling, any database code should follow this form:
try (
Connection connection = dataSource.getConnection();
PreparedStatement preparedStatement = ...
)
{
...
}
The dataSource could be a pool or a simple connection factory, to your code it shouldn't matter, it's simply where it gets connections from.
This said, I'd use Spring's JdbcTemplate class, which does pretty much what the code above does.
Regarding the question 'how expensive is it to open new connections for each task?' - the answer is very, at least in comparison to grabbing one from a pool. You also have to consider what happens if large numbers of tasks are run - most databases will be configured to limit the number of connections.
The bottom line is, in anything but the most simple application, you should use a connection pool like c3po and size it according you your needs
Regarding your deadlock, this is most likely occurring in the database where there are a variety of locks that can be made when data is updated depending on how the database is configured.
What is the best way in Java to create a singleton?
Should a DB connection be a singleton (being a singleton it's automatically thread-safe)? Because theoretical the DB can't be accessed by many users in the same time.
A DB connection should not normally be a Singleton.
Two reasons:
many DB drivers are not thread safe. Using a singleton means that if you have many threads, they will all share the same connection. The singleton pattern does not give you thread saftey. It merely allows many threads to easily share a "global" instance.
Personally, I think Singleton often leads to bad design: See this post (by somebody else) http://tech.puredanger.com/2007/07/03/pattern-hate-singleton/
Instead of doing this consider a database pool. The pool is shared (and could be a singleton if you wanted). When you need to do database work your code does this:
getConnectioFromPool();
doWork()
closeConnection() // releases back to pool
Sample Pool Libraries:
http://commons.apache.org/dbcp/
http://jolbox.com/
The best way to create a singleton (as of today) is the enum singleton pattern (Java enum singleton)
I doubt, that a Singleton is necessary or of any value for a database connection. You probably want some lazy creation: a connection is created upon first demand and cached, further requests will be fullfilled with the cached instance:
public ConnectionProvider {
private Connection conn;
public static Connection getConnection() {
if (conn == null || conn.isClosed()) {
conn = magicallyCreateNewConnection();
}
return conn;
}
}
(not thread safe - synchronize, if needed)
What is the best way in Java to create
a singleton?
Follow the design pattern creation guidelines. i.e private constructor, etc.
Should a DB connection be a singleton
(being a singleton it's automatically
thread-safe)?
creating a DB connection as a singleton might be a poor design choice in many scenarios. Use it only if you are sure that you don't need DB concurrency. If you have multiple users logged in at the same time, or even if your single user spawns many threads that need to access the DB, then a DB Connection pool is a better choice. You can use either apache or tomcat db connection pools. THese classes are defined for example in the package
org.apache.commons.dbcp.*;
org.apache.tomcat.dbcp.dbcp.*;
where dbcp stands for database connection pooling.
The biggest reason for using a connection pool is that on average the time it takes for the DB access (DML etc) is much smaller than the time it takes to create a connection and then close the connection. Additionally, don't forget to close your ResultSet, PreparedStatement and Connection variables after the transaction is done.
Because theoretical the DB can't be
accessed by many users in the same
time.
Why not? DB in most cases is meant to be used concurrently. You have these DB isolation levels - READ_COMMITTED, READ_UNCOMMITTED, SERIALIZED etc.
SERIALIZED is the case where your DB becomes single user access.
Singletons are a pattern - there's no explicit way to create one, you just follow the design practice.
So, if you're using a database that can handle concurrent reads/writes (i.e. MySQL), you don't need to worry so much about thread safety. If you're using a DB that's doesn't do concurrent writes well (SQLite), then a singleton should theoretically work.