In JDBC can we say the transaction begins as soon as we get the connection and finishes
as we close the connection. IS this right? If yes can we say In different requests sharing
the same connection, even all the uncommitted transactions will be visible to all
all requests?
#BalusC - This is not correct really. By default autocommit is set to true, which means that transaction begins before any JDBC operation and finishes RIGHT AFTER that single operation - not when connection is closed.
But you are right that sharing connection is indeed not good, If you want to multi-thread your DB it's best to handle it in the way that you have thread pool (look for ThreadPoolExecutor in java.util.concurrent) and for each thread you get a separate connection. ConnectionPool is also a good one, but I would rather limit that through ThreadPool - this way there is never a thread waiting for a connection from the pool.
That's right. That's the default behaviour. You can take over this control by setting the auto commit to false by connection.setAutoCommit(false) after retrieving the connection and committing the transaction by connection.commit() after doing all queries.
However, sharing a connection between different requests (threads) is at its own already a bad design. Your application is this way not threadsafe. You do not want to share the same connection among different threads. If all you want is eliminating the cost of connecting a database, then you should consider using a connection pool.
first rule when you access the database.
Every nontransactional operation should:
1.open connection, if have connection pool then get connection from the pool
2. Create execute statement
3. if is read query then map the result set.
4. close the result set.
5. close the statement.
6. close the connection.
if you want your operation to be in transaction then you should consider this approach:
Operation 1:
1. getSharedConnection
2.create/execute statement
3. if is read query then map the result set.
4. close resultSEt
5. close statement
operation 2:
Same as operation 1.
and the transaction:
public void updateInTransaction(){
Connection conn=pool.getConnection();//or you can create a new connection
conn.setAutocommit(false);
operation1(conn);
operation2(conn);
conn.close;
}
This is just basics for small applications.
If you are developing bigger applicatoin you should using same framework like JDBCTemplates from Springsoruce.
Related
In my application I have multithreads that needs to access database and I am using apache.tomcat.jdbc.pool.DataSource as a JDBC connection pool.
In some cases users execute stored procedures that might affect the database session context/variables before executing another query to retrieve some data.
After a thread is finished, connection is closed but the way the pooled connections work, the connection is not actually closed but rather returned to the connection pool for reuse. The problem is similar to this question question.
The problem is that these variables usually affect the data retrieved from the database. So when another thread acquire a connection where these variables/context where set and then query the database the data retrieved is affected by the context/variables.
What is the best way to handle such an issue? I have outlined some solutions but not sure which is best practise and not how to implement them all?
Execute a Procedure / Statement that reset session / variables before releasing the connection back to the pool:
This solution could work but the issue is my application use different databases. For now MySQL and Oracle are supported. For Oracle we have RESET_PACKAGE Procedure, but there is no equivalent for MySQL. Is there a way to do it in MySQL? also what if new databases are supported?
Is there a way to enforce or explicitly close the actual/physical connection instead of just returning it to the pool? or Is there a property to enforce pool to close the connection?
Does rollback to a savepoint revert db session variables / context?
Savepoint savepoint = connection.setSavepoint();
// execute some procedure connection that affect session
connection.rollback(savepoint);
connection.close();
Or is there any other way to enforce clearing of session or restart/close the actual connection?
My question is related to this question.
On MySQL you might perform the sequence:
BEGIN;
COMMIT RELEASE;
as your validationQuery. The RELEASE part disconnects the current session and creates a new one (cf. MySQL Documentation).
I have a scenario in production for a web app, where when a form is submitted the data gets stored in 3 tables in Oracle DB through JDBC. Sometimes I am seeing connection time out errors in logs while the app is trying to connect to Oracle DB through Java code. This is intermittent.
Below is the exception:
SQL exception while storing data in table
java.sql.SQLRecoverableException: IO Error: Connection timed out
Most of the times the web app is able to connect to data base and insert values in it but some times and I am getting this time out error and unable to insert data in it. I am not sure why am I getting this intermittent issue. When I checked the connections pool config in my application, I noticed the following things:
Pool Size (Maximum number of Connections that this pool can open) : 10
Pool wait (Maximum wait time, in milliseconds, before throwing an Exception if all pooled Connections are in use) : 1000
Since the pool size is just 10 and if there are multiple users trying to connect to data base will this connection time out issue occur ?
Also since there are 3 tables where the data insertion occurs we are doing the whole insertion in just one connection itself. We are not opneing each DB connection for each individual table.
NOTE: This application is deployed on AEM (Content Management system) server and connections pool config is provided by them.
Update: I tried setting the validation query in the connections pool but still I am getting the connection time out error. I am not sure whether the connections pool has checked the validation query or not. I have attached the connections pool above for reference.
I would try two things:
Try setting a validation query so each time the pool leases a connection, you're sure it's actually available. select 1 from dual should work. On recent JDBC drivers that should not be required but you might give it a go.
Estimate the concurrency of your form. A 10 connections pool is not too small depending on the complexity of your work on DB. It seems you're saving a form so it should not be THAT complex. How many users per day do you expect? Then, on peak time, how many users do you expect to be using the form at the same time? A 10 connections pool often leases and retrieves connections quite fast so it can handle several transactions per second. If you expect more, increase the size slightly (more than 25-30 actually degrades DB performance as more queries compete for resources there).
If nothing seems to work, it would be good to check what's happening on your DB. If possible, use Enterprise Manager to see if there are latches while doing stuff on those three tables.
I give this answer from programming point of view. There are multiple possibilities for this problem. These are following and i have added appropriate solution for it. As connection timeout occurs, means your new thread do not getting database access within mentioned time and it is due to:
Possibility I: Not closing connection, there should be connection leakage somewhere in your application Solution
You need to ensure this thing and need to check for this leakage and close the connection after use.
Possibility II: Big Transaction Solution
i. Is these insertion synchronized, if it is so then use it very carefully. Use it at block level not method level. And your synchronized block size should be minimum as much as possible.
What happen is if we have big synchronized block, we give connection, but it will be in waiting state as this synchronized block needs too much time for execution. so other thread waiting time increases. Suppose we have 100 users, each have 100 threads for that operation. 1st is executing and it takes too long time. and others are waiting. So there may be a case where 80th 90th,etc thread throw timeout. And For some thread this issue occurs.
So you must need to reduce size of the synchronized block.
ii. And also for this case also check If the transaction is big, then try to cut the transaction into smaller ones if possible:-
For an example here, for one insertion one small transaction. for second other small transaction, like this. And these three small transaction completes operation.
Possibility III: Pool size is not enough if usability of application is too high Solution
Need to increase the pool size. (It is applicable if you properly closes all the connection after use)
You can use Java Executor service in this case .One thread One connection , all asynchronous .Once transaction completed , release the connection back to pool.That way , you can get rid of this timeout issue.
If one connection is inserting the data in 3 tables and other threads trying to make connection are waiting, timeout is bound to happen.
I've been researching all around the web the most efficient way to design a connection pool and tried to analyze into details the available libraries (HikariCP, BoneCP, etc.).
Our application is a heavy-load consumer webapp and most of the time the users are working on similar business objects (thus the underlying SQL queries executed are the often the same, but still there are numerous).
It is designed to work with different DBMS (Oracle and MS SQL Server especially).
So a simplified use case would be :
User goes on a particular JSP page (e.g. Enterprise).
A corresponding Bean is created.
Each time it realizes an action (e.g. getEmployees(), computeTurnover()), the Bean asks the pool for a connection and returns it back when done.
If we want to take advantage of the Prepared Statement caching of the underlying JDBC driver (as PStatements are attached to a connection - jTDS doc.), from what I understand an optimal way of doing it would be :
Analyze what kind of SQL query a particular Bean want to execute before providing it an available connection from the pool.
Find a connection where the same prepared statement has already been executed if possible.
Serve the connection accordingly (and use the benefits of the cache/precompiled statement).
Return the connection to the pool and start over.
Am I missing an important point here (like JDBC drivers capable of reusing cached statements regardless of the connection) or is my analysis correct ?
The different sources I found state it is not possible, but why ?
For your scheme to work, you'd need to be able to get the connection that already has that statement prepared.
This falls foul on two points:
In JDBC you obtain the connection first,
Cached prepared statements (if a driver or connection pool even supports that) aren't exposed in a standardized way (if at all) nor would you be able to introspect them.
The performance overhead of finding the right connection (and the subsequent contention on the few connections that already have it prepared) would probably undo any benefit of reusing the prepared statement.
Also note that some database systems also have a serverside cache for prepared statements (meaning that it already has the plan etc available), limiting the overhead from a new prepare from the client.
If you really think the performance benefit is big enough, you should consider using a data source specific for this functionality (so it is almost guaranteed that the connection will have the statement in its cache).
A solution could be for a connection pool implementation to delay retrieving the connection from the pool until the Connection.prepareStatement() is called. At that time a connection pool would look up available connections by the SQL statement text and then play forward all the calls made before Connection.prepareStatement(). This way it would be possible to get a connection with a ready PreparedStatement without the issues other guys suggested.
In other words, when you request a connection from the pool, it would return a wrapper that logs everything until the first operation requiring DB access (such as prepareStatement() is requested.
You'd need to ask a vendor of your connection pool functionality to add this feature.
I've logged this request with C3P0:
https://github.com/swaldman/c3p0/issues/55
Hope this helps.
In Hibernate API, there is a property hibernate.connection.autocommit which can be set to true.
But in the API, they have mentioned that it is not recommended to set it like so:
Enables autocommit for JDBC pooled connections (it is not
recommended).
Why is it not recommended ?
What are the ill-effects of setting this property to true ?
All database statements are executed within the context of a physical transaction, even when we don’t explicitly declare transaction boundaries (BEGIN/COMMIT/ROLLBACK).
If you don't declare the transaction boundaries, then each statement will have to be executed in a separate transaction. This may even lead to opening and closing one connection per statement.
Declaring a service as #Transactional will give you one connection for the whole transaction duration, and all statements will use that single isolation connection. This is way better than not using explicit transactions in the first place. On large applications you may have many concurrent requests and reducing the database connection acquiring request rate is definitely improving your overall application performance.
So the rule of thumb is:
If you have read-only transactions that only execute one query, you can enable auto-commit for those.
If you have transactions containing more than one statement, you need to disable auto-commit, since you want all operations to execute in a single unit-of-work and you don't want to put extra pressure on your connection pool.
By default the autocommit value is false, therefore the transaction needs to be commited explicitly. This might be the reason why the changes not getting reflected in database, else can try flush to force the changes before commit.
When you close the session, then it will get commited in database implicitly [depends on the implementation].
When you have cascading transactions & needs to rollback for atomicity, you need to have control over transactions & in that case, autocommit should be false.
Either set autocommit as true or handle transactions explicitly.
Here is a good explanation on it.
Hibernate forum related to this.
Stackoverflow question on it.
My understanding is that if Hibernate autocommits, then a flush that fails part way through won't be rolled back. You'll have an incomplete/broken object graph.
If you want a connection with autocommit on for something, you can always unwrap a newly created Session to get the underlying JDBC Connection, setAutocommit(true) on it, do your work via JDBC APIs, setAutocommit(false), and close the session. I would not recommend doing this on a Session that's already done anything.
Do not use the session-per-operation antipattern: do not open and close a Session for every simple database call in a single thread. The same is true for database transactions. Database calls in an application are made using a planned sequence; they are grouped into atomic units of work. This also means that auto-commit after every single SQL statement is useless in an application as this mode is intended for ad-hoc SQL console work. Hibernate disables, or expects the application server to disable, auto-commit mode immediately. Database transactions are never optional. All communication with a database has to occur inside a transaction. Auto-commit behavior for reading data should be avoided, as many small transactions are unlikely to perform better than one clearly defined unit of work. The latter is also more maintainable and extensible.
find more information on this topic
Connection.setTransactionIsolation(int) warns:
Note: If this method is called during a transaction, the result is implementation-defined.
This bring up the question: how do you begin a transaction in JDBC? It's clear how to end a transaction, but not how to begin it.
If a Connection starts inside in a transaction, how are we supposed to invoke Connection.setTransactionIsolation(int) outside of a transaction to avoid implementation-specific behavior?
Answering my own question:
JDBC connections start out with auto-commit mode enabled, where each SQL statement is implicitly demarcated with a transaction.
Users who wish to execute multiple statements per transaction must turn auto-commit off.
Changing the auto-commit mode triggers a commit of the current transaction (if one is active).
Connection.setTransactionIsolation() may be invoked anytime if auto-commit is enabled.
If auto-commit is disabled, Connection.setTransactionIsolation() may only be invoked before or after a transaction. Invoking it in the middle of a transaction leads to undefined behavior.
See JDBC Tutorial by Oracle.
JDBC implicitly demarcates each query/update you perform on the connection with a transaction. You can customize this behavior by calling setAutoCommit(false) to turn off the auto-commit mode and call the commit()/rollback() to indicate the end of a transaction. Pesudo code
try
{
con.setAutoCommit(false);
//1 or more queries or updates
con.commit();
}
catch(Exception e)
{
con.rollback();
}
finally
{
con.close();
}
Now, there is a type in the method you have shown. It should be setTransactionIsolation(int level) and is not the api for transaction demarcation. It manages how/when the changes made by one operation become visible to other concurrent operations, the "I" in ACID (http://en.wikipedia.org/wiki/Isolation_(database_systems))
I suggest you read this you'll see
Therefore, the first call of
setAutoCommit(false) and each call of
commit() implicitly mark the start of
a transaction. Transactions can be
undone before they are committed by
calling
Edit:
Check the official documentation on JDBC Transactions
When a connection is created, it is in auto-commit mode. This means
that each individual SQL statement is treated as a transaction and is
automatically committed right after it is executed. (To be more
precise, the default is for a SQL statement to be committed when it is
completed, not when it is executed. A statement is completed when all
of its result sets and update counts have been retrieved. In almost
all cases, however, a statement is completed, and therefore committed,
right after it is executed.)
The way to allow two or more statements to be grouped into a
transaction is to disable the auto-commit mode. This is demonstrated
in the following code, where con is an active connection:
con.setAutoCommit(false);
Source: JDBC Transactions
Startingly, you can manually run a transaction, if you wish to leave your connection in "setAutoCommit(true)" mode but still want a transaction:
try (Statement statement = conn.createStatement()) {
statement.execute("BEGIN");
try {
// use statement ...
statement.execute("COMMIT");
}
catch (SQLException failure) {
statement.execute("ROLLBACK");
}
}
Actually, this page from the JDBC tutorial would be a better read.
You would get your connection, set your isolation level and then do your updates amd stuff and then either commit or rollback.
You can use these methods for transaction:
you must create the connection object like con
con.setAutoCommit(false);
your queries
if all is true con.commit();
else con.rollback();
Maybe this will answer your question:
You can only have one transaction per connection.
If autocommit is on (default), every select, update, delete will automatically start and commit (or rollback) a transaction.
If you set autocommit off, you start a "new" transaction (means commit or rollback won't happen automatically). After some statements, you can call commit or rollback, which finishes current transaction and automatically starts a new one.
You cannot have two transactions actively open on one JDBC connection on pure JDBC.
Using one connection for multiple transactions (reuse, pooling or chaining) some weird problems can lurk creating problems people have to live by since they usually cant identify the causes.
The following scenarios come to mind:
(Re-)Using a connection with an ongoing / uncommitted transaction
Flawed connection pool implementations
Higher isolation level implementations in some databases (especially the distributed SQL and the NoSQL one)
Point 1 is straight forward and understandable.
Point 2 basically leads to either point 1 or (and) point 3.
Point 3 is all about a system where a new transaction has begun before the first statement is issued. From a database perspective such a transaction might have started long before the 'first' real statement was issued. If the concurrency model is based on the snapshot idea where one reads only states/values that were valid at the point the transaction begins but no change that has changed later on, it is very important that on commit the full read set of the current transaction is also validated.
Since NoSQL and certain isolation levels like MS SQL-Server Snapshot often do not validate the read-set (in the right way), all bets are usually off to what to expect. While this is a problem always being present, it is way worse when one is dealing with transactions that start on the last commit or when the connection was pooled rather than the connection being actually used, it is usually important to make sure the transaction actually starts when it is expected to start. (Also very important if one uses a rollback-only read-only transaction).
I use the following rules when dealing with JDBC in JAVA:
Always rollback a JDBC connection before using it (scraps everyting and starts a new transaction), if the company uses plain JDBC in conjunction with any pooling mechanism
Use Hibernate for Transaction handling even if only using a session managed JDBC connection for plain SQL. Never had a problem with transactions till now.
Use BEGIN / COMMIT / ROLLBACK as SQL-Statements (like already mentioned). Most implementations will fail if you issue a BEGIN statement during an active transaction (test it for your database and remember the test database is not the production database and JDBC Driver and JDBC Server-side implementations can differ in behaviro than running a SQL console on the actual server).
Use 3 inside one's own wrapper for a JDBC connection instances. This way transaction handling is always correct (if no reflection is used and the connection pooling is not flawed).
3+4 I only use if response time is critical or if Hibernate is not available.
4 allows for using some more advanced performance (response time) improvement patterns for special cases