How should I test a service method that is transactional for its simultaneous use (it updates a database row by decreasing a value)?
I have setup a JUnit test class with SpringJunit4ClassRunner and components are #autowired.
Just spawning threads which would call the method doesn't seem to work. I'm not sure whether this has something to do with the Spring proxy mechanism.
What I would like to achieve is to create a situation where simultaneously two threads are "inside" the tested method and other one will fail and rollback. e.g. The row value is 3 and both method calls try to decrease the value by 2; if the method wouldn't work, the value would be -1, which is illegal. But I want that either both of the calls fail and rollback, or failing the one that tries to update it an instant later than the other.
Is this even possible?
The first problem is that the transaction context is bound to one thread (with a thread local). So you have to start a transaction in each of your threads. (I think there is no support for this in spring. You can start transaction programmatically with the transaction manager.)
The code you described: read, decrement, write does only work with the right isolation level (serialized and repeatable read would work).
After this setup is done, you can test the behavior by blocking one thread while he has the database lock. You can use a Latch for this.
The thread without database lock will now still not rollback. It will block until the database lock is available again. The scheme you're describing is quite similiar to Optimistic concurrency control so maybe this is already implemented.
Related
I have 2 questions related to each other
Q1 What exactly is a transaction boundary in hibernate/Spring Data JPA.
I am new to JPA , so please give a very basic example so I can understand as I tried to read multiple blogs but still not very clear.
Q2 And on top of it, what does this mean-
In hibernate, persist() method guarantees that it will not execute an INSERT statement if it is called outside of transaction boundaries, save() method does not guarantee the same.
What is outside and inside of a transaction boundary and how executions are performed outside boundaries?
A transaction is a unit of work that is either executed completely or not at all.
Transactions are fairly simple to use in a typical relational database.
You start a transaction by modifying some data. Every modification starts a transaction, you typically can't avoid it. You end the transaction by executing a commit or rollback.
Before your transaction is finished your changes can't be seen in other transactions (there are exceptions, variations and details). If you rollback your transaction all your changes in the database are undone.
If you commit your changes your changes become visible to other transactions, i.e. for other users connected to the same database. Implementations vary among many other things if changes become visible only for new transactions or also for already running transactions.
A transaction in JPA is a database transaction plus additional stuff.
You can start and end a transaction by getting a Transaction object and calling methods on it. But nobody does that anymore since it is error prone. Instead you annotate methods with #Transaction and entering the method will start a transaction and exiting the method will end the transaction.
The details are taken care of by Spring.
The tricky part with JPAs transactions is that within a JPA transaction JPA might (and will) choose to delay or even avoid read and write operations as long as possible. For example when you load an entity, and load it again in the same JPA transaction, JPA won't load it from the database but return the same instance it returned during the first load operation. If you want to learn more about this I recommend looking into JPAs first level cache.
A transaction boundary it's where the transaction starts or is committed/rollbacked.
When a transaction is started, the transaction context is bound to the current thread. So regardless of how many endpoints and channels you have in your Message flow your transaction context will be preserved as long as you are ensuring that the flow continues on the same thread. As soon as you break it by introducing a Pollable Channel or Executor Channel or initiate a new thread manually in some service, the Transactional boundary will be broken as well.
some other people ask about it - look it up.
If you do not understand something write to me again more accurately and I will explain.
I really hope I helped!
In a spring application when we receive message #Service persist bean is calling the database operation to insert in to database & parallel #Service to parse & process message. In this case persist is using #Transactional. In order to make the flow in parallel, is it advised to add #Async for persist.
Additionally there is #Aspect on each save method called by persist service for logging & audit.
Is #Async advisable for database operations?
Does #Async create table locks?
All that #Async does is cause the methods of the annotated component to be executed on another thread, where it gets the thread from a pool (which can be specified, so you can choose for some operations to have a dedicated pool).
#Async itself doesn’t do anything to lock database tables, or anything else database-related. If you want database-level locking you will have to implement that through some other means. If you want the call to use a transaction you have to use the #Transactional annotation on the component being called asynchronously. The transaction will be separate from the caller's transaction. Of course the transaction can possibly cause database locking depending on the isolation level and database implementation.
It’s tricky to use #Async with database work. One pitfall occurs with jpa persistent entities passed across threads, when they have a lazy property that gets realized in the new thread (where the proxy is now invalid because it can’t get to the entityManager from the old thread). It’s safer if the things passed between threads are immutable.
#Async adds complexity and is hard to reason about. There are opportunities for race conditions and deadlocks where if you don’t get it exactly right then bad things can happen, and you can’t count on testing to uncover the issues. It’s working without a net, if you want any infrastructure to help with exception handling, retries, or other recovery you will have to provide it yourself.
So no, I wouldn’t necessarily call it advisable. It's a good capability to have in your toolbox that might be helpful for a few isolated cases, but pervasive usage would seem like a bad thing. There are alternatives if you’re looking for ways to persist data without blocking.
I have a DAO class that uses Spring JDBC to access an SQLite database. I have declared transactions on the DAO methods themselves since my service layer never combines queries in a transaction.
Since I use a few worker threads in parallel but only one thread can update an SQLite DB at the same time, I use synchronized to serialize access to the DAO.
At first, I synchronized externally from my service class, for example:
synchronized (dao) {
dao.update(...);
}
Then, I figured I might as well get rid of the external synchronization and put synchronized on the DAO method itself:
public synchronized void update(...) {
// Spring JDBC calls here
}
The strange thing is: my queries now take twice the time they used to!
Why?
Well, one difference is obvious:
synchronized (dao) {
// here you are synchronizing on the transactional proxy
}
public synchronized void update(...) {
// and here you are synchronizing on the target class, *inside* the proxy
}
What the implications of this are depends on your other code, but that's the obvious difference.
My guess is your update method or entire class is annotated with Transactional or wrapped by transactional proxy through other means. This means whenever you call dao's method, the transactional proxy retrieves db connection from the pool, opens a transaction and then calls the real method.
In your first scenario you synchronize before even reaching the proxy, thus no connection and transaction magic happens. In the second scenario you do the waiting call after that.
If there are multiple threads trying to perform simultaneous updates there will be only one doing the update and the rest will be first opening new connections and then waiting for dao access. As a consequence instead of one connection being constantly reused you will have multiple connections in use. I can only guess how this really affects the performance but you can experiment with different pool size starting with one.
We have a service method which does approximately the following:
#Transactional
public void serviceMethod(...){
for(Item i : bunchOfItems){
webServices.webServiceCall(...);
dao.daoUpdateMethod(...);
}
}
The problem is that as soon as an update occurs the DB is holding a lock on the table for the duration of the Transaction (webservice calls average 5 sec each). Any exception in a webservice call or DAO call should, of course, cause a full rollback.
What's the best approach to this situation?
If the web service call doesn't depend on what you might have updated in a previous iteration, you could make all your web service calls in a first pass and collect the results in memory, and then start a transaction for all your updates. This would make your transaction much shorter and, since I assume the web service call isn't transactional anyway, it wouldn't affect the coherence of your data.
Because I assume the webservice call is not transactional in any way, you can do all webservice calls before you start the transaction to store something.
You can do the whole stuff in different ways:
sequential - 2 loops, one transaction and a bit memory: loop trough all webservice invocation store the results in an array, open the transaction and then loop trough all results and store them
sequential - one loop, and n transaction: in the loop, first call the web service for one item, then start a new transaction and store it (loop end)
in parallel - do the web service invokation in parallel - you can combine it with the two ways mentioned above like JB Nizet suggested his answer
By employing MVCC mode in the database I can avoid locking on updates altogether. After doing this I can perform the same test without any lock contention.
MVCC mode allows reads to occur while an uncommitted update is still in progress.
I am using CommonsHttpSolrServer in a Web Application. Is it safe to reuse the CommonsHttpSolrServer over multiple requests or should I instantiate a new object for each request? Could not find the answer in the API docs.
According to the documentation and the source comments, SolrJ is thread safe.
However, be careful when you update solr. According to this post, the transactions are implemented per instance, not per queue. This means that each thread does not have it's own isolated transaction to work with. Rollback will rollback every call (regardless of originating thread) to the last commit.
Overall, this means that you should be safe to query (using the same CommonsHttpSolrServer) with as many threads as you like. However, if you wish to take advantage of rollback, you will need to ensure only one thread is updating your solr instance at a time (regardless of object distribution).