I've got some Java code uses JDBC to connect to a MySQL database, then the code does some read operations then a single update, all using the same connection. If there is an exception, then connection.rollback() is called; if there's no exception, connection.commit() is called. At this stage, the connection is newly created each time I run my test (i.e. it does not come from a pool). My code only ever creates one connection, and it is used throughout the test.
The connection being used has connection.setAutoCommit(false) called immediately after the connection instance is created.
For some reason, when there is an exception and connection.rollback() is called, it turns out that my update has been committed rather than rolled back.
Via debugging, I have confirmed the following,
After calling connection.setAutoCommit(false), connection.getAutoCommit() returns a value of false, as expected. Also, "Select ##session.autocommit" returns a value of 0, indicating that auto commit is off, as expected.
Immediately prior to calling connection.rollback(), the same checks show that auto commit is off, as expected.
connection.commit() is definitely not being called and connection.rollback() is definitely being called.
I have also tried explicitly running the statement "rollback;" but it does not solve my issue. I have also tried explicitly running the statement "Set AUTOCOMMIT = 0;" after creating the connection.
All of my tables use a storage engine of InnoDB. Via SQL Workbench, with auto commit off, rollbacks and commits work as expected.
I am using MySQL version '5.0.91-community-nt'. The MySQL jdbc driver version is 5.1.19. I'm using Java 5.
Has anyone got any suggestions as to why my update is getting committed even though auto-commit is off, commit is never called, and rollback is explicitly called?
Cheers.
The code I refer to in the OP is not in a single block, it is dispersed across classes. To satisfy the queries above for examples of the code, I prepared a single block of code to illistrate the issue. Once the block of code was completed, I tested it to ensure I could replicate the issue. Well, I couldn't replicate the issue - the rollback worked just fine. This caused me to go through the production code to work out all the things it was doing beyond the block of code I had put together.
It turns out that the production code both creates and drops temporary tables. Most relevant, it drops the temporary tables after performing the update, and it drops the temporary tables whether or not there is an exception. It turns out that dropping a table in MySQL issues an implicit commit call. Hence, in between my code throwing an exception and my code calling connection.rollback(), it drops tables which causes an implicit call to commit. Hence my 'auto commit' issue.
Related
In Hibernate API, there is a property hibernate.connection.autocommit which can be set to true.
But in the API, they have mentioned that it is not recommended to set it like so:
Enables autocommit for JDBC pooled connections (it is not
recommended).
Why is it not recommended ?
What are the ill-effects of setting this property to true ?
All database statements are executed within the context of a physical transaction, even when we don’t explicitly declare transaction boundaries (BEGIN/COMMIT/ROLLBACK).
If you don't declare the transaction boundaries, then each statement will have to be executed in a separate transaction. This may even lead to opening and closing one connection per statement.
Declaring a service as #Transactional will give you one connection for the whole transaction duration, and all statements will use that single isolation connection. This is way better than not using explicit transactions in the first place. On large applications you may have many concurrent requests and reducing the database connection acquiring request rate is definitely improving your overall application performance.
So the rule of thumb is:
If you have read-only transactions that only execute one query, you can enable auto-commit for those.
If you have transactions containing more than one statement, you need to disable auto-commit, since you want all operations to execute in a single unit-of-work and you don't want to put extra pressure on your connection pool.
By default the autocommit value is false, therefore the transaction needs to be commited explicitly. This might be the reason why the changes not getting reflected in database, else can try flush to force the changes before commit.
When you close the session, then it will get commited in database implicitly [depends on the implementation].
When you have cascading transactions & needs to rollback for atomicity, you need to have control over transactions & in that case, autocommit should be false.
Either set autocommit as true or handle transactions explicitly.
Here is a good explanation on it.
Hibernate forum related to this.
Stackoverflow question on it.
My understanding is that if Hibernate autocommits, then a flush that fails part way through won't be rolled back. You'll have an incomplete/broken object graph.
If you want a connection with autocommit on for something, you can always unwrap a newly created Session to get the underlying JDBC Connection, setAutocommit(true) on it, do your work via JDBC APIs, setAutocommit(false), and close the session. I would not recommend doing this on a Session that's already done anything.
Do not use the session-per-operation antipattern: do not open and close a Session for every simple database call in a single thread. The same is true for database transactions. Database calls in an application are made using a planned sequence; they are grouped into atomic units of work. This also means that auto-commit after every single SQL statement is useless in an application as this mode is intended for ad-hoc SQL console work. Hibernate disables, or expects the application server to disable, auto-commit mode immediately. Database transactions are never optional. All communication with a database has to occur inside a transaction. Auto-commit behavior for reading data should be avoided, as many small transactions are unlikely to perform better than one clearly defined unit of work. The latter is also more maintainable and extensible.
find more information on this topic
I have a web application that consists of a web service with two operations: createA and createB. An handler is registered for the endpoint. This handler opens a Session and start a transaction when the request is received. Then the code of the requested operation is executed. Before the response is sent back, the transaction is committed and the session is closed.
The code of createA consists of creating an entity of type A and persisting it using Session.save() method. In DEBUG mode, after Session.save() is called, I can see that there is one insertion in the ActionQueue of the session.
The code of createB consists of :
retrieving the previously created entity of type A
creating an Entity B that references the instance of A (B has a property that represents an associated A)
updating A to reference the new instance of B
call Session.save() for the new instance of B
call Session.update() for the new modified instance of A
However, in DEBUG mode, after calling Session.save() and Session.update(), the ActionQueue of the corresponding Session is empty. But, after the transaction commits, I can see the created entity in the database.
Operation createA and createB are invoked in this order without DEBUG. An error appears during the execution of the create B when it tries to retrieve the instance of A previously created using a criteria and the Session.list() method. The problem is that the instance of A is not found.
However, if I repeat the same sequence of operations in DEBUG or using Thread.sleep(15s) between invocations of the two operations, the instance of A can be found.
Thanks
EDIT: I forgot to precise that it works on certain machines but not on others. And I don't see any differences between these machines.
If you use the same Hibernate session for both createA and createB, then it'll work. You can store the Hibernate session in the Http session for achieving this (pay attention to synchronize the access to the session object, as requests from the same browser session can com in different threads).
Your problem is, Hibernate opens a new database connection for every session. Now your database seems not to synchronize the statements. It can happen in the database the select arrives before the insert is finished. Then it just depends of the speed of the involved computers if this condition happens or not. With the debug mode or the sleep() you make one computer slower so you don't have the problem any more.
If you want to continue with two different sessions for these two procedures, you can
look for the transaction mode of your database. Some databases have a dirty read where no correct locking or synchronization is done. Check if you accidentally used such a mode.
Check the JDBC parameters (they can be used in the hibernate connection.url) if there are parameters for your database which change timing and synchronization.
Check your connection pool (for the case you're using one).
The problem is that Hibernate does not save the entity to the database when you call Session.save(). It simply prepares the statement for execution later. This happens when you the transaction ends or when the you flush the session.
Your call to B is probably happening sometimes before the transaction ends for the A request. That is why it works if you wait a little while.
Try adding session.flush() after the save call. This will force Hibernate to persist the changes to the DB.
We are trying to fix some issues on a testing harness, and having issues with a particular test, which basically tests a feature that creates an entity, does some processing and stores it in a database (Yes, the C in CRUD).
In the tearDown section of the fitnesse test, we execute a delete statement on that record. However, nothing is being deleted.
We suspect that this may be because the tearDown is executed before the SUT commits its transaction. So consequently, there's nothing to be deleted.
To try and fix this, we are making a pollable jdbc delete:
java.sql.Statement statement;
/*creates a statement*/
do{
recordsDeleted = statement.executeUpdate("delete...");
Thread.sleep(INTERVAL);
}while(recordsDeleted == 0);
So here comes the questions:
When is a jdbc transaction commited?
In the code above, will the updates be executed on the same transaction, or will a new transaction be created for each iteration of the do-while loop? (I'm inclined to think that they will be executed in the same transaction, since the java.sql.Connection holds the commit, rollback, etc methods).
Can you suggest another solution for this problem? I would think that this is quite common, but my teammates have not found any solution online, just the "poll until its deleted or timeout" suggestion.
I don't see anything wrong with your loop initially. You are calling statement.close() somewhere right? I assume the SUT is in another thread or is remote? Are you sure your delete criteria matches the input? Can you examine the database in another process to see if the create is making it to the database?
In terms of transactions, it depends on the database but typically there are no transactions by default. Typically auto-commit is enabled so each individual statement is executed and committed immediately. If you want to enable transactions then you need to disable auto-commit and then call databaseConnection.setSavePoint(). A transaction is committed when commit() is called. If the connection is closed (or rollback() called) then the transaction is rolled back.
Connection.setTransactionIsolation(int) warns:
Note: If this method is called during a transaction, the result is implementation-defined.
This bring up the question: how do you begin a transaction in JDBC? It's clear how to end a transaction, but not how to begin it.
If a Connection starts inside in a transaction, how are we supposed to invoke Connection.setTransactionIsolation(int) outside of a transaction to avoid implementation-specific behavior?
Answering my own question:
JDBC connections start out with auto-commit mode enabled, where each SQL statement is implicitly demarcated with a transaction.
Users who wish to execute multiple statements per transaction must turn auto-commit off.
Changing the auto-commit mode triggers a commit of the current transaction (if one is active).
Connection.setTransactionIsolation() may be invoked anytime if auto-commit is enabled.
If auto-commit is disabled, Connection.setTransactionIsolation() may only be invoked before or after a transaction. Invoking it in the middle of a transaction leads to undefined behavior.
See JDBC Tutorial by Oracle.
JDBC implicitly demarcates each query/update you perform on the connection with a transaction. You can customize this behavior by calling setAutoCommit(false) to turn off the auto-commit mode and call the commit()/rollback() to indicate the end of a transaction. Pesudo code
try
{
con.setAutoCommit(false);
//1 or more queries or updates
con.commit();
}
catch(Exception e)
{
con.rollback();
}
finally
{
con.close();
}
Now, there is a type in the method you have shown. It should be setTransactionIsolation(int level) and is not the api for transaction demarcation. It manages how/when the changes made by one operation become visible to other concurrent operations, the "I" in ACID (http://en.wikipedia.org/wiki/Isolation_(database_systems))
I suggest you read this you'll see
Therefore, the first call of
setAutoCommit(false) and each call of
commit() implicitly mark the start of
a transaction. Transactions can be
undone before they are committed by
calling
Edit:
Check the official documentation on JDBC Transactions
When a connection is created, it is in auto-commit mode. This means
that each individual SQL statement is treated as a transaction and is
automatically committed right after it is executed. (To be more
precise, the default is for a SQL statement to be committed when it is
completed, not when it is executed. A statement is completed when all
of its result sets and update counts have been retrieved. In almost
all cases, however, a statement is completed, and therefore committed,
right after it is executed.)
The way to allow two or more statements to be grouped into a
transaction is to disable the auto-commit mode. This is demonstrated
in the following code, where con is an active connection:
con.setAutoCommit(false);
Source: JDBC Transactions
Startingly, you can manually run a transaction, if you wish to leave your connection in "setAutoCommit(true)" mode but still want a transaction:
try (Statement statement = conn.createStatement()) {
statement.execute("BEGIN");
try {
// use statement ...
statement.execute("COMMIT");
}
catch (SQLException failure) {
statement.execute("ROLLBACK");
}
}
Actually, this page from the JDBC tutorial would be a better read.
You would get your connection, set your isolation level and then do your updates amd stuff and then either commit or rollback.
You can use these methods for transaction:
you must create the connection object like con
con.setAutoCommit(false);
your queries
if all is true con.commit();
else con.rollback();
Maybe this will answer your question:
You can only have one transaction per connection.
If autocommit is on (default), every select, update, delete will automatically start and commit (or rollback) a transaction.
If you set autocommit off, you start a "new" transaction (means commit or rollback won't happen automatically). After some statements, you can call commit or rollback, which finishes current transaction and automatically starts a new one.
You cannot have two transactions actively open on one JDBC connection on pure JDBC.
Using one connection for multiple transactions (reuse, pooling or chaining) some weird problems can lurk creating problems people have to live by since they usually cant identify the causes.
The following scenarios come to mind:
(Re-)Using a connection with an ongoing / uncommitted transaction
Flawed connection pool implementations
Higher isolation level implementations in some databases (especially the distributed SQL and the NoSQL one)
Point 1 is straight forward and understandable.
Point 2 basically leads to either point 1 or (and) point 3.
Point 3 is all about a system where a new transaction has begun before the first statement is issued. From a database perspective such a transaction might have started long before the 'first' real statement was issued. If the concurrency model is based on the snapshot idea where one reads only states/values that were valid at the point the transaction begins but no change that has changed later on, it is very important that on commit the full read set of the current transaction is also validated.
Since NoSQL and certain isolation levels like MS SQL-Server Snapshot often do not validate the read-set (in the right way), all bets are usually off to what to expect. While this is a problem always being present, it is way worse when one is dealing with transactions that start on the last commit or when the connection was pooled rather than the connection being actually used, it is usually important to make sure the transaction actually starts when it is expected to start. (Also very important if one uses a rollback-only read-only transaction).
I use the following rules when dealing with JDBC in JAVA:
Always rollback a JDBC connection before using it (scraps everyting and starts a new transaction), if the company uses plain JDBC in conjunction with any pooling mechanism
Use Hibernate for Transaction handling even if only using a session managed JDBC connection for plain SQL. Never had a problem with transactions till now.
Use BEGIN / COMMIT / ROLLBACK as SQL-Statements (like already mentioned). Most implementations will fail if you issue a BEGIN statement during an active transaction (test it for your database and remember the test database is not the production database and JDBC Driver and JDBC Server-side implementations can differ in behaviro than running a SQL console on the actual server).
Use 3 inside one's own wrapper for a JDBC connection instances. This way transaction handling is always correct (if no reflection is used and the connection pooling is not flawed).
3+4 I only use if response time is critical or if Hibernate is not available.
4 allows for using some more advanced performance (response time) improvement patterns for special cases
I have EJB RESTEasy controller with CMT.
One critical method which creates some entities in DB works fine and quickly on single invocation.
But when i try to invoke it simultaneously by 10 users it works very slowly.
I've tracked time in logs and the most expanded place vs. single invocation is
lag between exit from RESTeasy controller and enter into MainFilter.
So this lag grows from 0-1 ms for single invocation to 8 sec. for 10 simultaneously invocations!
I need ideas what could be a reason and how can I speed up it.
My immediate reaction is that it's a database locking problem. Can you tell if the lag occurs as the flow of control passes across the transaction boundary? Try the old technique of littering your code with print statements to see when things stop.
Lock contention over resteasy threads? database? It is very difficult to predict where the bottleneck is.
As per some of the comments above, it does sound like it could be a database locking problem. From what you said, the "lag" occours between the Controller and the Filter invoking the controller. Presumably that is where the transaction commit is occuring.
You say however that the code creates some entities in the database, but you don't say if the code does any updates or selects. Just doing inserts wouldn't normally create a locking problem with most databases, unless there are associated updates or selects (i.e. select for update in Oracle).
Check and see if there are any resources like a table of keys or an parent record that are being updated that might be causing the problem.
Also check your JDBC documentation. Most JDBC drivers have logging levels that should allow you to see the operations being performed on the database. While this may generate a sizeable log, if you include a thread identifier in the log, you should be able to see where problems are occuring.