I am developing an WebApp and would like to know if there is any pros/cons of sharing JDBC connection object between methods
So the situation is
It is a WebApp and multiuser environment (aprox 1000 user using at a same time)
I am using a container that supports connection pooling (Weblogic Server)
I use Transactions in my JDBC calls
Most of my JDBC calls are single record insert/Update but select/delete can happen in bulk
Now i have two approach
Approach 1
Open the connection object once and relay it across the methods
// Just a Pseudo Code
Cn = OpenJDBConnection() // This will open up the connection
obj.Task1(Cn, Param1, Param2);
obj.Task2(Cn, Param1, Param2);
obj.Task3(Cn, Param1, Param2);
Cn.close();
Approach 2
Open and close the connection in each method
I am leaning on Approach 1 as that way i will be able to avoid some boilerplate code. but i am unsure will it be threadsafe ? i have a poolsize of 100 connection which i think is okay for 1000 users active at a given time
is there anything else too that i should be considering before adopting one of the approach
Consider that transactions are managed at the connection level.
For your simple use-case, approach #1 may work, but it's easy to get into a situation where, for example, you have transaction pending on the connection, and call another method which does a select on the same connection, which will cause your transaction to commit (earlier than you expected).
With properly-configured connection pooling, the overhead of releasing and re-obtaining a connection should be minmal, so I'd suggest approach #2. When pooled, closing the connection doesn' really close it, it leaves it open and returns it to the pool.
Related
I have a severe problem with my database connection in my web application. Since I use a single database connection for the whole application from singleton Database class, if i try concurrent db operations (two users) the database rollsback the transactions.
This is my static method used:
All threads/servlets call static Database.doSomething(...) methods, which in turn call the the below method.
private static /* synchronized*/ Connection getConnection(final boolean autoCommit) throws SQLException {
if (con == null) {
con = new MyRegistrationBean().getConnection();
}
con.setAutoCommit(true); //TODO
return con;
}
What's the recommended way to manage this db connection/s I have, so that I don't incurr in the same problem.
Keeping a Connection open forever is a very bad idea. It doesn't have an endless lifetime, your application may crash whenever the DB times out the connection and closes it. Best practice is to acquire and close Connection, Statement and ResultSet in the shortest possible scope to avoid resource leaks and potential application crashes caused by the leaks and timeouts.
Since connecting the DB is an expensive task, you should consider using a connection pool to improve connecting performance. A decent applicationserver/servletcontainer usually already provides a connection pool feature in flavor of a JNDI DataSource. Consult its documentation for details how to create it. In case of for example Tomcat you can find it here.
Even when using a connection pool, you still have to write proper JDBC code: acquire and close all the resources in the shortest possible scope. The connection pool will on its turn worry about actually closing the connection or just releasing it back to pool for further reuse.
You may get some more insights out of this article how to do the JDBC basics the proper way. As a completely different alternative, learn EJB and JPA. It will abstract away all the JDBC boilerplate for you into oneliners.
Hope this helps.
See also:
Is it safe to use a static java.sql.Connection instance in a multithreaded system?
Am I Using JDBC Connection Pooling?
How should I connect to JDBC database / datasource in a servlet based application?
When is it necessary or convenient to use Spring or EJB3 or all of them together?
I've not much experience with PostgreSql, but all the web applications I've worked on have used a single connection per set of actions on a page, closing it and disposing it when finished.
This allows the server to pool connections and stops problems such as the one that you are experiencing.
Singleton should be the JNDI pool connection itself; Database class with getConnection(), query methods et al should NOT be singleton, but can be static if you prefer.
In this way the pool exists indefinitely, available to all users, while query blocks use dataSource.getConnection() to draw a connection from the pool; exec the query, and then close statement, result set, and connection (to return it to the pool).
Also, JNDI lookup is quite expensive, so it makes sense to use a singleton in this case.
I've been researching all around the web the most efficient way to design a connection pool and tried to analyze into details the available libraries (HikariCP, BoneCP, etc.).
Our application is a heavy-load consumer webapp and most of the time the users are working on similar business objects (thus the underlying SQL queries executed are the often the same, but still there are numerous).
It is designed to work with different DBMS (Oracle and MS SQL Server especially).
So a simplified use case would be :
User goes on a particular JSP page (e.g. Enterprise).
A corresponding Bean is created.
Each time it realizes an action (e.g. getEmployees(), computeTurnover()), the Bean asks the pool for a connection and returns it back when done.
If we want to take advantage of the Prepared Statement caching of the underlying JDBC driver (as PStatements are attached to a connection - jTDS doc.), from what I understand an optimal way of doing it would be :
Analyze what kind of SQL query a particular Bean want to execute before providing it an available connection from the pool.
Find a connection where the same prepared statement has already been executed if possible.
Serve the connection accordingly (and use the benefits of the cache/precompiled statement).
Return the connection to the pool and start over.
Am I missing an important point here (like JDBC drivers capable of reusing cached statements regardless of the connection) or is my analysis correct ?
The different sources I found state it is not possible, but why ?
For your scheme to work, you'd need to be able to get the connection that already has that statement prepared.
This falls foul on two points:
In JDBC you obtain the connection first,
Cached prepared statements (if a driver or connection pool even supports that) aren't exposed in a standardized way (if at all) nor would you be able to introspect them.
The performance overhead of finding the right connection (and the subsequent contention on the few connections that already have it prepared) would probably undo any benefit of reusing the prepared statement.
Also note that some database systems also have a serverside cache for prepared statements (meaning that it already has the plan etc available), limiting the overhead from a new prepare from the client.
If you really think the performance benefit is big enough, you should consider using a data source specific for this functionality (so it is almost guaranteed that the connection will have the statement in its cache).
A solution could be for a connection pool implementation to delay retrieving the connection from the pool until the Connection.prepareStatement() is called. At that time a connection pool would look up available connections by the SQL statement text and then play forward all the calls made before Connection.prepareStatement(). This way it would be possible to get a connection with a ready PreparedStatement without the issues other guys suggested.
In other words, when you request a connection from the pool, it would return a wrapper that logs everything until the first operation requiring DB access (such as prepareStatement() is requested.
You'd need to ask a vendor of your connection pool functionality to add this feature.
I've logged this request with C3P0:
https://github.com/swaldman/c3p0/issues/55
Hope this helps.
I have been unable to find an exact answer to this question. I'm using C3P0's ComboPooledDataSource. Which of these methodologies is better practice:
dataSource = connectionClass.getDataSource();
conn = dataSource.getConnection;
executeQuery(query1, conn);
executeQuery(query2, conn);
...
executeQuery(finalQuery, conn);
conn.close();
OR
executeQuery(query1);
executeQuery(query2);
...
executeQuery(finalQuery);
where executeQuery:
conn = dataSource.getConnection;
st = conn.createStatement();
rs = executeQuery(query);
conn.closed();
In short, I have to do a decent amount of queries every so often. Is it better to go with the first design, which gets the connection once for each batch and passes it as an argument. Or is it better to go with the second approach and just get a connection each time I call my executeQuery method. If I was using DriverManager I would obviously choose the first (only get the connection once), but when using the C3P0 package I am not sure if doing that is the right way to go or not. Or does it not matter with such a package?
With a connection pool, the difference is neglectible, because even if you use the second approach, bringing back a pooled connection takes little time. Still, using the first approach is the better way to go, because
It avoids the additional (little) overhead of getting a connection from the pool.
If you later need to introduce transactions (do all of your changes or, in case of an error, conveniently and securely roll back your changes), then the first approach is your only option.
Some comments/suggestions
If you application is single threaded (unless you mention), it does not matter. It even does not matter whether you use connection pool or not. Just use a single connection and pass the same to function where you need it.
Connection pools are useful when the use case involves multiple database connections simultaneously.
Since your application is a batch and single threaded, it does not warrant use of connection pool.
Regarding your application, both the approaches are equivalent. When you call connection.close() on pooled datasource connection, its not actually closed but returned to pool.
In JDBC can we say the transaction begins as soon as we get the connection and finishes
as we close the connection. IS this right? If yes can we say In different requests sharing
the same connection, even all the uncommitted transactions will be visible to all
all requests?
#BalusC - This is not correct really. By default autocommit is set to true, which means that transaction begins before any JDBC operation and finishes RIGHT AFTER that single operation - not when connection is closed.
But you are right that sharing connection is indeed not good, If you want to multi-thread your DB it's best to handle it in the way that you have thread pool (look for ThreadPoolExecutor in java.util.concurrent) and for each thread you get a separate connection. ConnectionPool is also a good one, but I would rather limit that through ThreadPool - this way there is never a thread waiting for a connection from the pool.
That's right. That's the default behaviour. You can take over this control by setting the auto commit to false by connection.setAutoCommit(false) after retrieving the connection and committing the transaction by connection.commit() after doing all queries.
However, sharing a connection between different requests (threads) is at its own already a bad design. Your application is this way not threadsafe. You do not want to share the same connection among different threads. If all you want is eliminating the cost of connecting a database, then you should consider using a connection pool.
first rule when you access the database.
Every nontransactional operation should:
1.open connection, if have connection pool then get connection from the pool
2. Create execute statement
3. if is read query then map the result set.
4. close the result set.
5. close the statement.
6. close the connection.
if you want your operation to be in transaction then you should consider this approach:
Operation 1:
1. getSharedConnection
2.create/execute statement
3. if is read query then map the result set.
4. close resultSEt
5. close statement
operation 2:
Same as operation 1.
and the transaction:
public void updateInTransaction(){
Connection conn=pool.getConnection();//or you can create a new connection
conn.setAutocommit(false);
operation1(conn);
operation2(conn);
conn.close;
}
This is just basics for small applications.
If you are developing bigger applicatoin you should using same framework like JDBCTemplates from Springsoruce.
I have lots of method in my database class that all of them have one statement, when should I close the connection? In each method or at the end of database class?
You should close the connection when you are finished your transaction. Since we don't know the contents of the class or its usage, it's impossible to tell when your access of the connection begins or ends.
Actually, that may not even be true if the connection is dedicated for a specific usage and not in a pool. You may want to keep it open for the duration of your application.
We've found that the best policy is to get a connection from the connection pool, execute a single transaction, and then put the connection back into the pool immediately. This way you don't have a connection being held onto for long blocks of logic, thus preventing other threads from using it - which is an issue for scalability.
As a best practice, you should close the connection in the logical place after you are done - right after all of database activity for that task is done.
Generally, you should close a connection in the same method that opens it. Closing and opening connections isn't an arduous task, since modern DB servers keep even closed connections on "hot standby", so they are quickly accessed through a connection pool. Leaving them open though...that can get you in trouble and can be a nightmare to debug.
use lombok and it will handle both try/catch and conn.close() for you
public void doSomething() throws SQLException {
#Cleanup Connection connection = database.getConnection();
}
lombok
It depends when and how repeatedly you are using these methods. If they are sequential you should close the connection only in the end, instead of open and close often
This largely depends on what your Database class does. If your methods are called individually at various times then the methods should be responsible for opening and closing the connection. However, if the class does some kind of big processing operation that calls many methods then you may want to open and close the connection outside of the individual methods.
The most important thing is that wherever you open the connection, you also close the connection. Otherwise you get into the business of making assumptions about the state of the connection which can get you into trouble.
Close the connection (and statement and resultset!) in the same method block as you've acquired it, in the finally block of the try block where they are opened.
The general idiom is:
public void doSomething() throws SQLException {
Connection connection = null;
try {
connection = database.getConnection();
} finally {
if (connection != null) try { connection.close(); } catch (SQLException logOrIgnore) {}
}
}
Closing of connection, statement and resultset should happen in reversed order as they're opened.
If your actual concern is performance, then consider using a connection pool to improve connecting performance, for example C3P0. This by the way does not change the general idiom. Just continue writing the same idiom, the connection pooling implementation will worry itself further under the hoods.
Also see this article for more practices/examples.
Always close the connection when you finish executing your transaction. It is a good practice to obtain and close your connection in the same method. If you have separate transactions that are tightly coupled you might execute them with the same connection, but for best practices, I try to execute one transaction per connection.
If it is an option, don't explicitly mess with connections at all. Use a full blown orm framework like hibernate or use something significantly more lightweight like spring jdbc templates.
It depends somewhat on the context your application operates in. If it's a web app you need to be careful to open your connection, do whatever work is needed, and close the connection quickly. However, in a C/S or batch environment it may be better to acquire the connection and hold onto it as long as the user is interacting "frequently" (for whatever value of "frequently" you choose), especially if the user has expectations of rapid response time and it's expensive (in terms of time or resources) to acquire the connection to your particular variety of database.
I like to set a timer every time the user has the app go to the database. If/when the timer expires, close the connection, then re-open the next time he/she/it wants to hit the database again. Timer expiration can be somewhere between 1 and 20 minutes. Just so long as it's less than the database's "inactivity disconnect" time.
Share and enjoy.