Canceling a transaction through API - java

I'm developing a system that contains multiple microservices operating on the same database, which isn't ideal but is done for the sake of legacy support. As we refactored the old system we split old, transactional functionality into separate microservices, which led us to have distributed transactions now that must be handled in some way. We're using Spring boot.
As it usually goes, we have a microservice A calling microservices B and then C, we need a way to rollback transaction B if transaction C throws an error.
I've read on 2PC and Saga approaches, I was wondering if there is a way to implement a somewhat simpler variation of 2PC approach. It would need to support the following functionality.
1)Microservice endpoint is called, it calls a #Transactional service which creates a transaction. A special parameter is passed that tells the TransactionManager to keep the transaction "hanging" for some time (say, 5 seconds)
2)In those 5 seconds, another call can be made that either confirms or rolls back the transaction
3)If the time elapses (times out), default handling for the transaction is applied (either rollback or commit)
4)If the special parameter isn't passed in, transactions behave as if they are not distributed
5)API calls are still asynchronous
Simple example:
1)Service A endpoint "createResource" is called with parameter "hangTransaction=true"
2)Service A returns status 200
3)Service B endpoint "registerResource" is called
4)Service B returns status 500
5)Service A endpoint "createResource" is called with parameter "transactionValid=false"
6)Service A rolls the transaction back and returns status 200
7)There are no changes to DB after this
There would of course be additional parameters sent (transaction ID for example), the example is simplified.
Is there some way to control the TransactionManager in a way that allows the transaction to persist between API calls?

Related

What is a transaction boundary in hibernate

I have 2 questions related to each other
Q1 What exactly is a transaction boundary in hibernate/Spring Data JPA.
I am new to JPA , so please give a very basic example so I can understand as I tried to read multiple blogs but still not very clear.
Q2 And on top of it, what does this mean-
In hibernate, persist() method guarantees that it will not execute an INSERT statement if it is called outside of transaction boundaries, save() method does not guarantee the same.
What is outside and inside of a transaction boundary and how executions are performed outside boundaries?
A transaction is a unit of work that is either executed completely or not at all.
Transactions are fairly simple to use in a typical relational database.
You start a transaction by modifying some data. Every modification starts a transaction, you typically can't avoid it. You end the transaction by executing a commit or rollback.
Before your transaction is finished your changes can't be seen in other transactions (there are exceptions, variations and details). If you rollback your transaction all your changes in the database are undone.
If you commit your changes your changes become visible to other transactions, i.e. for other users connected to the same database. Implementations vary among many other things if changes become visible only for new transactions or also for already running transactions.
A transaction in JPA is a database transaction plus additional stuff.
You can start and end a transaction by getting a Transaction object and calling methods on it. But nobody does that anymore since it is error prone. Instead you annotate methods with #Transaction and entering the method will start a transaction and exiting the method will end the transaction.
The details are taken care of by Spring.
The tricky part with JPAs transactions is that within a JPA transaction JPA might (and will) choose to delay or even avoid read and write operations as long as possible. For example when you load an entity, and load it again in the same JPA transaction, JPA won't load it from the database but return the same instance it returned during the first load operation. If you want to learn more about this I recommend looking into JPAs first level cache.
A transaction boundary it's where the transaction starts or is committed/rollbacked.
When a transaction is started, the transaction context is bound to the current thread. So regardless of how many endpoints and channels you have in your Message flow your transaction context will be preserved as long as you are ensuring that the flow continues on the same thread. As soon as you break it by introducing a Pollable Channel or Executor Channel or initiate a new thread manually in some service, the Transactional boundary will be broken as well.
some other people ask about it - look it up.
If you do not understand something write to me again more accurately and I will explain.
I really hope I helped!

JPA Stop Executing Code in Transaction if Exception Occurs

I have a JPA transaction like the following (Using controller advice to catch exceptions)
#Transactional
public void save(MyObj myObj) {
// Attempt to save the object
this.myRepo.save(myObj)
// After it saves, call my audit log service to record the change
this.myAuditLogService.logChange(myObj)
}
Works fine, but the problem is if the save fails and throws an exception, it still calls the audit log service, and then throws an exception afterwards. Causing erroneous audit entries to be created.
Expected Flow
Call save function
Save fails
Transaction stops and rolls back
Controller advice catches the exception
Actual Flow
Call save function
Save fails
Audit log service is called
Transaction rolls back
Controller advice catches the exception
This is a common problem in Computer Science in Distributed Systems.
Basically what you want to achieve is to have atomic operation across multiple systems.
Your transaction spans only your local (or first) database and that's all.
When the REST call to the second system is initiated and successful but the first save results in crash you want to have rollback on the first system (first save) and rollback on the second system as well. There are multiple problems with that and it's really hard to have atomic-like consistency across multiple systems.
You could use Database supported technologies for such cases:
What you probably need is a 2PC / 3PC or change the processing of your request somehow.
The trade-off of course will be that you'll have to sacrifice immediate results to have eventual consistency.
You could use eventual-consistency
For example send message to some storage for later processing -> make both systems read the message:
System1 reads from storage this message and will save myObj
System2 reads from storage this message and will log change
This will of course happen "eventually" - there will never be a guarantee that either system is up at the time of the processing or even later on (e.g. somebody killed the server or deployed code with bug and the server restarts indefinitely).
Moreover you'll sacrifice read-after-write consistency.
You could use in case of a failure a Compensating transaction.
I recommend reading more on the topic of Distributed Systems in:
(Fallacies of distributed computing)[https://en.wikipedia.org/wiki/Fallacies_of_distributed_computing]
(Designing Data Intensive Applications)[https://www.amazon.com/Designing-Data-Intensive-Applications-Reliable-Maintainable/dp/1449373321]
(CAP theorem)[https://en.wikipedia.org/wiki/CAP_theorem]
(Consistency models)[https://en.wikipedia.org/wiki/Consistency_model]

Subset of changes from transaction sometimes not visible shortly after commit

Let's consider the following context:
2 spring integration channels, they are each in separate database transactions. At the end of the first transaction, a message is put into the second channel. In the first channel, elements are created in the database that is later consumed by the corresponding message that has been sent from the first channel to the second channel.
To make sure that the transaction from channel 1 is fully committed before the second channel is triggered our subclass of the JpaTransactionManager is registering a TransactionSynchronization in the prepareForCommit method it overrides from the JpaTransactionManager
The flow (channel 1) looks like this:
Do all the message processing and database handling
Last step of the flow registers a TransactionSynchronization that does a MessageChannel.send in the afterCommit phase to send the message to channel 2
My understanding is that at the time the message is sent to the second channel (in afterCommit) all changes that have been done in the database transaction of channel 1 are flushed and committed.
Now the second channel does some work (like an MQ PUT) and later updates an entry that was created in the first flow. We have now observed that the repository method returned no entry in the database, but it is visible in the table at a later point. Other entries that were also created in the transaction of the first channel however are visible. This happens only once every few thousand messages, normally they are there but sometimes they are not visible for the second channel a few milliseconds after the transaction has been committed by channel 1.
I have created an image that should illustrate it:
The Chain 1 is the first chain that consists of multiple ServiceActivators that perform database work, a splitter that generates more messages and then another ServiceActivator that I named SENDER which registers the TransactionSynchronization that (so my understanding) should send the for example 3 generated messages to chain 2 after the red transaction is fully committed and therefore before the blue transaction begins.
One thing I have noticed is that the entries that were sometimes present and sometimes not are all in the one method that (not on purpose) uses javax.transaction.Transactional instead of org.springframework.transaction.annotation.Transactional. However, we are using spring core 5.0.8.RELEASE and in other questions I have seen that this should make 0 difference since spring 4.2.x.
I don't think the afterCommit is the right place to send messages downstream.
There should be just enough to have a Service Activator for POJO method marked with the #Transactional. This way a transaction is going to start and finish exactly around this method call. The result of the method is going to be sent to the output channel already, exactly after that transaction is committed.
UPDATE
The best way to achieve your requirements is a <gateway> around your Chain1. This way the TX is going to be committed over there before producing reply to the Chain2 from the gateway.
With the TransactionSynchronization::afterCommit there is no guarantee that TX is going to be committed on DB when QueueChannel is ready for polling messages. Although you can use JdbcChannelMessageStore for transactional storage of messages. This way they are not going to be visible until TX commit in DB.
See more about <gateway> in Docs: https://docs.spring.io/spring-integration/docs/current/reference/html/messaging-routing-chapter.html#_calling_a_chain_from_within_a_chain

Webservice calls within the same JDBC transaction are causing DB lock timeouts

I'm using the H2 database embedded within a Spring-MVC app.
I have declare Transactions at my service level. In particular I have a case where we do the folllowing:
Start a transaction
Make a webservices call
Make the subsequent database calls
Upon rollback (an exception), the webservices call is manually rolled back, and the Spring Transaction Manager handles calling a rollback on the DB transaction.
But I realized that the existence of the webservices call within the DB transaction is causing a table-wide lock, which causes other users to receive errors (I can produce this on a 1-user system with 2 browsers).
I am trying to plot my best course of action.
Should I be changing the transaction isolation level in Spring, will that affect the DB locking?
Is it just bad form to have a webservice and DB call in the same transaction?
Should I enable row level locking within the H2 DB because of this?
Am I not thinking of other solutions?
I should add that my service method serviceA() calls two other methods webServiceX() and daoMethodY(). serviceA() is encompassed in a transaction because any exception needs to rollback both the webServiceX() (a function I provide), and rollback the daoMethodY() database operations (a function of the DataSourceTransactionManager).
I think your approach is reasonable, and you should definitely try row-level locking if it's possible.
You might want to reconsider your design. Is the database duplicating state which really comes from the web service? In that case, you might want to consider caching the web service calls instead. But that depends on your application.
Alternatively, you might just have to roll your own transaction management. As long as it's generic, it shouldn't be too much trouble. We've done this in our project where we're not using Spring's transactions. Something like
performTransaction() {
doWSCall();
// no need to worry about WS call exception, because DB call won't happen
try {
doDbCall()
} catch (Exception ex) {
rollbackWSCall()
// rethrow ex
}
}
where all the methods are abstract.
I wouldn't have a web service call mingled into a database call. Your method is breaking the rule that says "do one thing well".
Have your service call other web services and the DAO.

Spring: Best way to handle long-running web-service calls with transactional DB methods?

We have a service method which does approximately the following:
#Transactional
public void serviceMethod(...){
for(Item i : bunchOfItems){
webServices.webServiceCall(...);
dao.daoUpdateMethod(...);
}
}
The problem is that as soon as an update occurs the DB is holding a lock on the table for the duration of the Transaction (webservice calls average 5 sec each). Any exception in a webservice call or DAO call should, of course, cause a full rollback.
What's the best approach to this situation?
If the web service call doesn't depend on what you might have updated in a previous iteration, you could make all your web service calls in a first pass and collect the results in memory, and then start a transaction for all your updates. This would make your transaction much shorter and, since I assume the web service call isn't transactional anyway, it wouldn't affect the coherence of your data.
Because I assume the webservice call is not transactional in any way, you can do all webservice calls before you start the transaction to store something.
You can do the whole stuff in different ways:
sequential - 2 loops, one transaction and a bit memory: loop trough all webservice invocation store the results in an array, open the transaction and then loop trough all results and store them
sequential - one loop, and n transaction: in the loop, first call the web service for one item, then start a new transaction and store it (loop end)
in parallel - do the web service invokation in parallel - you can combine it with the two ways mentioned above like JB Nizet suggested his answer
By employing MVCC mode in the database I can avoid locking on updates altogether. After doing this I can perform the same test without any lock contention.
MVCC mode allows reads to occur while an uncommitted update is still in progress.

Categories

Resources