I'm having problem trying to push changes made within a Hibernate transaction to the database for DbUnit to work properly in my test case. It seems like DbUnit is not seeing the changes made by Hibernate because they are not committed at the end of the transaction yet... and I'm not sure how to restructure my test case to get this to work.
Here's my over-simplified test case to demonstrate my problem:-
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration(locations = {
"classpath:applicationContext-test.xml"
})
#TransactionConfiguration(transactionManager = "transactionManager")
#Transactional
public class SomeTest {
#Autowired
protected DataSource dataSource;
#Autowired
private SessionFactory sessionFactory;
#Test
public void testThis() throws Exception {
Session session = sessionFactory.getCurrentSession();
assertEquals("initial overlayType count", 4, session.createQuery("from OverlayType").list().size());
//-----------
// Imagine this block is an API call, ex: someService.save("AAA");
// But for the sake of simplicity, I do it this way
OverlayType overlayType = new OverlayType();
overlayType.setName("AAA");
session.save(overlayType);
//-----------
// flush has no effect here
session.flush();
assertEquals("new overlayType count", 5, session.createQuery("from OverlayType").list().size());
// pull the data from database using dbunit
IDatabaseConnection connection = new DatabaseConnection(dataSource.getConnection());
connection.getConfig().setProperty(DatabaseConfig.PROPERTY_DATATYPE_FACTORY, new MySqlDataTypeFactory());
QueryDataSet partialDataSet = new QueryDataSet(connection);
partialDataSet.addTable("resultSet", "select * from overlayType");
ITable actualTable = partialDataSet.getTable("resultSet");
// FAIL: Actual row count is 4 instead of 5
assertEquals("dbunit's overlayType count", 5, actualTable.getRowCount());
DataSourceUtils.releaseConnection(connection.getConnection(), dataSource);
}
}
My whole idea in using DbUnit is to:-
Call someService.save(...) that saves data into several tables.
Use DbUnit to get expected table from XML.
Use DbUnit to get actual table from database.
Do Assertion.assertEquals(expectedTable, actualTable);.
But, at this point, I'm not able to get DbUnit to see the changes made by Hibernate within the transaction.
How should I change to get DbUnit to work nicely with Hibernate transaction?
Thanks.
I have never worked with DbUnit, but it seems like TransactionAwareDataSourceProxy will do the trick. Basically you need to wrap your original data source with this proxy and use it instead, so that this code:
new DatabaseConnection(dataSource.getConnection())
actually goes through the proxy and uses the same transaction and connection as Hibernate.
I found Transaction aware datasource (use dbunit & hibernate in spring) blog post explaining this.
Another approach would be to skip transactional tests altogether and cleanup the database instead manually. Check out my transactional tests considered harmful artcle.
Looks like that test case needs two transactions: one for putting data into the database, and a second one to retrieve it.
What I would do is:
Use a memory database so the data is cleaned when the unit test ends.
Remove the transactional annotations and use the beginTransaction and commit methods of the session directly.
The initial overlaytype count would be 0, and after the session is saved, it should be 1.
Related
The following SQL if run in MSSQL will insert the 1st and 3rd rows successfully:
BEGIN TRAN
INSERT ... -- valid data
INSERT ... -- invalid data (e.g. over column width)
INSERT ... -- valid data
COMMIT
Even though the second row fails within the transaction, you can still see the two rows with some valid data after the commit in the table.
However, when trying something similar in Hibernate, it rollbacks the whole transaction. Is there a way to tell Hibernate not to rollback on failed rows and commit the rest as same as how MSSQL does it?
e.g.
EntityTransaction transaction = em.getTransaction();
transaction.begin();
em.persist(new MyEntity("good"));
em.persist(new MyEntity("too long"));
em.persist(new MyEntity("good"));
transaction.commit();
This is not possible within the same transaction. Hibernate simply doesn't allow this. An error in a statement leads to an exception, which Hibernate cannot recover from. From the manual:
If the JPA EntityManager or the Hibernate-specific Session throws an exception, including any JDBC SQLException, you have to immediately rollback the database
transaction and close the current EntityManager or Session.
Certain methods of the JPA EntityManager or the Hibernate Session will not leave the Persistence Context in a consistent state. As a rule of thumb, no exception thrown by Hibernate can be treated as recoverable. Ensure that the Session will be closed by calling the close() method in a finally block.
Now this is a restriction (design decision) of Hibernate and not of the underlying JDBC or database stack. So what you want is perfectly possible using JDBC directly. If it is really important for you to get that behaviour, you might consider using JDBC calls for this section of the code. There you can do it exactly like in the SQL client: open transaction, issue statements, catching any exceptions manually and "ignoring" them, and at the end committing the transaction.
Example code:
Session session = em.unwrap(Session.class);
session.doWork(connection -> {
// manual commit mode
connection.setAutoCommit(false);
executeInsertIgnoringError(connection, new Object[]{123, null, "abc"});
executeInsertIgnoringError(connection, new Object[]{....});
...
connection.commit();
});
private void executeInsertIgnoringError(Connection connection, Object[] values) {
try (PreparedStatement stmt =
connection.prepareStatement("INSERT INTO MY_ENTITY VALUES (?, ?, ?, ...)")) {
for (int i = 0; i < values.length; i++) {
// PreparedStatement is indexed from 1
stmt.setObject(i+1, values[i]);
}
stmt.executeUpdate();
} catch (Exception e) {
log.warn("Error occurred, continuing.");
}
}
The way i did it is to divide your logic into diferent functions, and open the transaction inside the persisting function instead of the main one.
The main problem I see in your code is that you're defining a block transaction insead of opening a transaction for each operation.
Here's my snippet:
persistEntity(new MyEntity("good"));
persistEntity(new MyEntity("bad"));
persistEntity(new MyEntity("good"));
...
private void persistEntity(MyEntity entity){
EntityTransaction transaction = em.getTransaction();
transaction.begin();
em.persist(entity);
transaction.commit();
}
This way it will rollback just for the bad entity and keep going with the other. You can also add a try catch inside the persistEntity method, if you want to log the exception.
Fun fact, If you're using Spring you could create another #Component for the persist operations and only add #Transactional to the persisting method, this way you don't have to manage the transactions yourself.
Don't do so, that is idiomatically wrong, at first just review the real scope of your transactions.
You could write the code to run one statement at a time with autocommit on and not use #Transactional... Then perhaps catch any exceptions and throw them away as you go. But pretty much everything in that sentence is troublesome to even think about as a responsible developer and it would affect your entire app. Flavius's post would be a little more granular in doing something similar with explicitly smaller transactions and is a good way to go about it too.
As others have been commenting it's not a long term great plan and goes against so many ways to write programs correctly and the benefits and purpose of transactions. Perhaps if you plan to only use this as a one off data ingestion plan you could but again be very wary of using these patterns in a production grade app.
Having been sufficiently alarmed, you can read more about auto commit here and also be sure to read through the post links on why you probably shouldn't use it.
Spring JPA - No transaction set autocommit 'true'
You can do that by adding below property in hibernate config xml file
<property name="hibernate.connection.autocommit" value="true"/>
If you could use #Transactional annotation then
#Transactional(dontRollbackOn={SQLException.class, NOResultException.class})
Then I would suggest one some change in your code. It's better if you add your entities in a loop and catch exception on each transaction.
I'm using JDBI for preparing data for test scenarios (setting up preconditions) in a separate project (rest-assured stuff). Now I have to use those JDBI-powered Daos for integration tests in a production project.
An example data setup Dao looks like this:
class DocumentDao(jdbi: Jdbi) : DefaultCrudRepository<Document>(jdbi) {
/* ... */
override fun save(entity: Document, handle: Handle): Int {
return handle.createUpdate(
"""
INSERT INTO documents (
/* ... */
)
VALUES (
/* ... */
)
"""
)
.bindBean(entity)
.execute()
}
/* ... */
}
And now I must use it together with a production persistence framework, because each scenario in integration tests is performed in a single transaction. So both precondition insertion and production data modification in DB must happen in a single transaction and be rolled back. Like this:
#Test
public void LoadDocument_001() throws Exception
{
// Start a transaction
startTx();
Integer id = testDocumentDao.nextIdFromSequence();
// Insert fake document into DB as a test precondition
test.document.Document document = new test.document.Document(/* ... */);
testDocumentDao.save(document);
// Production Dao (subject) is called
Document loadedDocument = subject.load(id.toString());
// Rollback the transaction
rollbackTx();
}
startTx() and rollbackTx() are methods I can use to manually control a transaction by communicating with a connection directly. BUT! I can't control transactions for those JDBI-powered Daos. Every operation that is using JDBI happens to be automatically committed by JDBI. So Document I've created as a precondition is left in a DB after test finishes.
Note: of course, I'm using the same connection instance in JDBI and in production persistence layer.
So I have two questions:
How do I do manual transaction management for JDBI-only operations? For instance, I have two Daos (DocumentDao and CustomerDao) built like the example above. I want to call methods on both of them, but I want all changes to happen in a single transaction. And then I want to roll back that transaction.
How do I do manual transaction management for JDBI operation mixed with other non-JDBI code? E.g. I'm using DocumentDao and also some production code which is changing DB state too. How can I control a transaction using JDBI and/or communicating with a connection directly? Do I have to somehow disable automatic transaction management in JDBI? I tried doing it using fake TransactionHandler with empty methods, but it doesn't take any effect.
I am to write a JUnit to check that version is being maintained or not(on an event). Here is what I did using JUnit:
#Test
Public void testAudit() {
try {
//create Dao code
dao.save(); //This will create entry in AUD- and REVINFO-tables perfectly
SomeObject obj = SomeHelper.getAuditData(dao));
/*Method to be tested which generates audit message using envers i.e(dao created)*/
//Some logic to check if output is as expected
}
catch(Exception e) {
Assert.fail();
}
finally {
dao.delete(); //delete the data saved by JUnit (Problem starts here )
}
}
Calling the delete for dao would cause
UnsupportedOperationException: Can't write to a readonly object
I use Ehcache for caching. I googled for the problem and came to know that it might be because of CacheConcurrencyStrategy wrongly set for domain object which I want to delete. I checked.
For domain object there was no CacheConcurrencyStrategy. But nested object had CacheConcurrencyStrategy set as READ_WRITE (This might be real culprit).
But I don't want to change existing domain and existing code. Is it any way to bypass CacheConcurrencyStrategy for JUnit? If not, is there any possible way out without changing the existing code?
The ENVERs data is written post the transactions commit, so your code will never access the audit record, because one does not exist yet. If you want to test ENVERs, you need to manage the transactions yourself. Here is an example;
#Before
public void setup() {
// Envers audit information is written via post-event listeners and therefore the transaction needs to be
// committed.
PlatformTransactionManager txMgr = applicationContext.getBean(PlatformTransactionManager.class);
TransactionStatus status = txMgr.getTransaction(new DefaultTransactionDefinition(TransactionDefinition.PROPAGATION_REQUIRES_NEW));
Account account = accountDAO.getByUsername(UPDATE);
if (account != null) {
accountDAO.delete(account);
}
account = createAccount();
account.setUsername(INITIAL);
accountDAO.update(account);
txMgr.commit(status);
status = txMgr.getTransaction(new DefaultTransactionDefinition(TransactionDefinition.PROPAGATION_REQUIRES_NEW));
account.setUsername(UPDATE);
accountDAO.update(account);
txMgr.commit(status);
}
Then in your test, you can query out the audit information anyway you want (raw SQL, via the AuditReader, etc).
Have a JUNIT test set up as such
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration({ "/applicationContext.xml", "/applicationContext-security.xml" })
#TransactionConfiguration(defaultRollback = true)
#Transactional
public class BlahIntegrationTests{
#Test
public void testMappingsOfHugeObjectGraph(){
}
}
I'm attempting to test that my hibernate mappings (annotation driven and JPA based) are correct and when run like above my test passes (just asserts that an ID is created).
If I take the #Transactional away, I get errors with some of my relationships which I was expecting. Anyone have thoughts on why it's not failing when it's #Transactional?
EDIT: To Clarify, the exception that was thrown was regarding bad hibernate mappings (it's a very large object structure and I had borked some of them) upon saving of the object
If you remove #Transactional your test DB won't be empty for another test and therefore tests won't be isolated.
If you have it there then some tests may pass even though they should fail (as you described, or another example is if you insert entity wich duplicates some unique constraint).
Solution is to have #Transactional in its place and inject
#PersistenceContext
private EntityManager em;
and do following before you extract your data from database
em.flush();
em.clear();
The first line will cause synchronization between session and database (your provider usually waits till the end of the transaction).
The second line will remove all entities from session so all queries will go to the database.
And after the test everything is still rolled back so you have your database in original state.
Hope it helps.
If you take away #Transactional the Hibernate factory is running in its equivalent of auto-commit mode where each new access you make generates an entire new Session. So once you've retrieved an object it is immediately no longer associated with an open session and ineligible for Lazy Loading.
I'm doing a test by using JUnit.
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration(locations = {"classpath:com/mt/sm/application-context.xml", "classpath:com/mt/sm/security-context.xml"})
#TransactionConfiguration(transactionManager = "transactionManager", defaultRollback = true)
#Transactional
public class LocationPathServiceImplTest { /* Class code here */ }
The test method declaration is quite simple:
#Test
public void testRefresh() { /* Method body */}
I've created save an obj in the setup() and save to db.
While in #Test I run the refresh() from DAO (the refresh() method just calls EntityManager .refresh()), but it causes me the error below
org.hibernate.HibernateException: this instance does not yet exist as a row in the database
javax.persistence.PersistenceException: org.hibernate.HibernateException: this instance does not yet exist as a row in the database
I have no idea how to fix it. Did anybody come across this before?
All suggestions would be appreciated.
At no point I commit the changes to the database nor I call .flush(). For my best understanding, they are within the current transaction.
Without more code I'd say, you need to flush your DAO so the instance gets persisted. refresh is only object level while flush does an actual transaction on database level (hence the rollback = true so it gets rolled back after the test)
I'm not sure the other answers saying you should flush() are correct as this will not commit anything to the database. See Hibernate docs. Flushing the Session simply gets the data that is currently in the session synchronized with what is in the database. So your exception makes sense if you have not called myobject.save() in your setUp() method.
I don't think you want to call commit() anywhere either becasue you want everything to rollback after your test completes. Use these annotations on your Class
#RunWith(SpringJUnit4ClassRunner.class)
#TransactionConfiguration(transactionManager = "transactionManager", defaultRollback = true)
#Transactional
You can then add #before on your setUp() method although if your class extends TestCase this will be the same. Thor84no is correct in that the #before method will execute in the same transaction as your #Test method. If you actually wanted to seed the database with commited data you can use a method annotated with #beforeTransaction instead.
[EDIT]
Based on your updated question, it sounds like you have not called persist() or similar on the object you say you have created in setup() and it's considered to be dettached (i.e. not persisted to the database within your transaction).
I would also flush / close / reopen session to force actual writing into database.