I have the following application setup:
#SpringBootApplication
#EnableTransactionManagement
public class MyApp extends SpringBootServletInitializer {
...
}
with a class which has the following:
public class DoStaff {
public void doStaffOnAll(List<MyObject> myObjects) {
for (int i=0; i<myObjects.size(); i++) {
try {
doStaffOnSingle(myObjects.get(i), i);
} catch (Exception e) {
e.printStrackTrace();
}
}
}
#Transactional
public void doStaffOnSingle(MyObject myObject, int i) {
repository.save(myObject);
if (i%2==0) {
throw new RuntimeException();
}
}
}
So if I call DoStaff.doStaffOnAll with a list of MyObjects, the code saves all element from the list but also throws a runtime exception for every second element.
Since the doStaffOnSingle has #Transactional annotation, I would expect that every second element will be rolled back.
But if I run this code, every element is saved in the DB successfully. Why is that? What am I doing wrong?
Quoting Spring Documentation:
In proxy mode (which is the default), only external method calls coming in through the proxy are intercepted. This means that self-invocation (in effect, a method within the target object calling another method of the target object) does not lead to an actual transaction at runtime even if the invoked method is marked with #Transactional. Also, the proxy must be fully initialized to provide the expected behavior, so you should not rely on this feature in your initialization code (that is, #PostConstruct).
Move the doStaffOnAll() to a different Spring component, and it'll work.
Or change to aspectj mode.
I would recommend moving the method, and design the code so transaction boundaries are clear and distinct, i.e. all public methods on the class starts a transaction, or no methods on the class starts a transaction.
It should always be very clear where your transaction boundaries are, e.g. in a layered design, you would normally make the #Service layer also be the transaction layer, i.e. any call from a higher layer to the service layer is an atomic transaction.
#Transactional annotation is able to do the magic because of a proxy object.
Since you call the method directly you don't get that magic. In doStaffOnAll method you are directly invoking doStaffOnSingle method. So, nothing of Transactional behaviour gets added.
Try invoking the method using self invocation.
#Service
public class DoStaff {
#Autowired
private DoStaff doStaff;
public void doStaffOnAll(List<MyObject> myObjects) {
for (int i=0; i<myObjects.size(); i++) {
doStaff.doStaffOnSingle(..) // invoke like this
}
}
#Transactional
public void doStaffOnSingle(MyObject myObject, int i) {
}
}
Since the doStaffOnSingle has #Transactional annotation, I would
expect that every second element will be rolled back.
The default Transactional mode will commit everything or nothing. I think you would want to use REQUIRES_NEW Propagation.
Look here for supported propagation types.
https://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/transaction/annotation/Propagation.html#REQUIRED
Related
I have multiple methods in my codebase annotated with Spring's #transactional with different propgation levels (lets ignore the idea behind choosing the propagation levels). Example -
public class X {
#Transactional(Propagation.NOT_SUPPORTED)
public void A() { do_something; }
#Transactional(Propagation.REQUIRED)
public void B() { do_something; }
#Transactional(Propagation.REQUIRES_NEW)
public void C() { do_something; }
}
Now I have a new use case where I want to perform all these operations in a single transaction (for this specific use case only, without modifying existing behavior), overriding any annotated propagation levels. Example -
public class Y {
private X x;
// Stores application's global state
private GlobalState globalState;
#Transactional
public void newOperation() {
// Set current operation as the new operation in the global state,
// in case this info might be required somewhere
globalState.setCurrentOperation("newOperation");
// For this new operation A, B, C should be performed in the current
// transaction regardless of the propagation level defined on them
x.A();
x.B();
x.C();
}
}
Does Spring provide some way to achieve this ? Is this not possible ?
One way I could think of is to split the original methods
#Transactional(Propagation.NOT_SUPPORTED)
public void A() { A_actual(); }
// Call A_actual from A and newOperation
public void A_actual() { do_something; }
But this might not be as simple to do as this example (there can be a lot of such methods and doing this might not scale). Also it does not look much clean.
Also the use case might also appear counter intuitive, but anyway let's keep that out of scope of this question.
I do believe the only option is to replace TransactionInterceptor via BeanPostProcessor, smth. like:
public class TransactionInterceptorExt extends TransactionInterceptor {
#Override
public Object invoke(MethodInvocation invocation) throws Throwable {
// here some logic determining how to proceed invocation
return super.invoke(invocation);
}
}
public class TransactionInterceptorPostProcessor implements BeanFactoryPostProcessor, BeanPostProcessor, BeanFactoryAware {
#Setter
private BeanFactory beanFactory;
#Override
public void postProcessBeanFactory(#NonNull ConfigurableListableBeanFactory beanFactory) throws BeansException {
beanFactory.addBeanPostProcessor(this);
}
#Override
public Object postProcessBeforeInitialization(#NonNull Object bean, #NonNull String beanName) throws BeansException {
if (bean instanceof TransactionInterceptor) {
TransactionInterceptor interceptor = (TransactionInterceptor) bean;
TransactionInterceptor result = new TransactionInterceptorExt();
result.setTransactionAttributeSource(interceptor.getTransactionAttributeSource());
result.setTransactionManager(interceptor.getTransactionManager());
result.setBeanFactory(beanFactory);
return result;
}
return bean;
}
}
#Configuration
public class CustomTransactionConfiguration {
#Bean
//#ConditionalOnBean(TransactionInterceptor.class)
public static BeanFactoryPostProcessor transactionInterceptorPostProcessor() {
return new TransactionInterceptorPostProcessor();
}
}
However, I would agree with #jim-garrison suggestion to refactor your spring beans.
UPD.
But you favour refactoring the beans instead of following this approach. So for the sake of completeness, can you please mention any issues/shortcomings with this
Well, there are a plenty of things/concepts/ideas in spring framework which were implemented without understanding/anticipating consequences (I believe the goal was to make framework attractive to unexperienced developers), and #Transactional annotation is one of such things. Let's consider the following code:
#Transactional(Propagation.REQUIRED)
public void doSomething() {
do_something;
}
The question is: why do we put #Transactional(Propagation.REQUIRED) annotation above that method? Someone might say smth. like this:
that method modifies multiple rows/tables in DB and we would like to avoid inconsistencies in our DB, moreover Propagation.REQUIRED does not hurt anything, because according to the contract it either starts new transaction or joins to the exisiting one.
and that would be wrong:
#Transactional annotation poisons stacktraces with irrelevant information
in case of exception it marks existing transaction it joined to as rollback-only - after that caller side has no option to compensate that exception
In the most cases developers should not use #Transactional(Propagation.REQUIRED) - technically we just need a simple assertion about transaction status.
Using #Transactional(Propagation.REQUIRES_NEW) is even more harmful:
in case of existing transaction it acquires another one JDBC-connection from connection pool, and hence you start getting 2+ connections per thread - this hurts performance sizing
you need to carefully watch for data you are working with - data corruptions and self-locks are the consequences of using #Transactional(Propagation.REQUIRES_NEW), cause now you have two incarnations of the same data within the same thread
In the most cases #Transactional(Propagation.REQUIRES_NEW) is an indicator that you code requires refactoring.
So, the general idea about #Transactional annotation is do not use it everywhere just because we can, and your question actually confirms this idea: you have failed to tie up 3 methods together just because developer had some assumptions about how those methods should being executed.
Need some help here, I'm not able to understand why my transactions are not getting rolled back in an event of exception.
I will try to put my code as close to as It is on the project (cannot share on the internet)
This is my Service
#Service
#SL4j
#Transactional(propagation = propagation.SUPPORTS, readOnly=true)
public class PublicationServiceImpl implements PublicationService{
#Autowired
private PartnerRepo partnerRepo;
#Autowired
private FlowRepo flowRepo;
#Autowired
private PubRepo pubRepo;
#Override
#Transactional(propagation = propagation.REQUIRED, rollbackFor=Exception.class)
public int save(Request request) {
try{
int pk_id_partner = partnerRepo.save(request);
int pk_id_flow = flowRepo.save(request);
String publicationCode = generatePubCode(request);
int publicationCode= pubRepo.save(pk_id_partner, pk_id_flow, request);
}
catch(Exception e){
log.error("Exception in saving");
}
return 0;
}
}
This is my Repository (example of 1 , all 3 repos follow same coding standards)
#Repository
#Slf4j
public class PartnerRepo implemets PartnerRepo{
#Autowired
private NamedParamaterJDBCTemplate namedParamaterJDBCTemplate;
//String Declarations .....
private MapSqlParameterSource sqlParameterSource;
#Override
public int save(Request request){
sqlParamatersSource = new MapSqlParameterSource();
//sqlParamatersSource.addValue(.....)
//sqlParamatersSource.addValue(.....)
//sqlParamatersSource.addValue(.....)
return executeQuery();
}
private int executeQuery(){
try{
keyHolder = new GenerateKeyHolder();
namedParamaterJDBCTemplate.update(getInsertQuery(), sqlParamaterSource , kekHolder, new String[]{"pk_id"})
return keyHolder.getKey().intValue();
}catch(Exception e){
log.error("Exception while saving");
return 0;
}
}
}
So the problem is , Consider there is an exception in the method generatePubCode(request); , ideally since I have used #Transactional at class level and method level , The previous 2 repo transactions () should be rolled back right? However it isn't happening, Even After the code is finished execution I can see the records in DB (Postgres DB v10).
Please help figure out this issue , Am I doing something fundamentally wrong ?
Please do let me know in case you need further information that might help here!
P.S: I have tried all permutations of #Transactional , nothing works : ONLY having this in the catch block works! TransactionAspectSupport.currentTransactionStatus().setRollbackOnly();
I wonder If its the right approach for a springBoot project
Thanks in advance for the help!
Edit: as per suggestion made the PublicationServiceSaverImpl.save() public
Best reagards,
Bhargav.
There are several things that break proper transactions in Spring
Your service method is private
You are catching and swallowing exceptions
private method
The fact that your PublicationServiceImpl save method is private basically makes the #Transactional on that method useless. As a private method cannot be proxied, no transactions will apply. Even if it would be public it wouldn't work as you are calling the method from within the same object, hence the transactionality of that method applies.
To fix, make your method public and call the save method from an other class (or make the actual method that is calling save have the proper #Transactional.
The fact that is doesn't work is due to the type op AOP being used, by default Spring will use proxies and this is a drawback of using proxy based AOP.
Another solution to make it work with private methods is to switch to full-blown AspectJ with either compile-time or load-time weaving of the classes. Both require additional setup and that can be tedious.
Catch and swallow exceptions
You have in both your repository as well as your service a try/catch block. Each of those catches and swallows the exceptions (they are logged but not re-thrown).
For transactions to work properly it needs to see the exceptions. The fact that you are catching and swallowing them, makes the transaction aspect not see them and instead of doing a rollback, do a commit. For the transaction aspect everything is ok because there was no exception.
To fix, remove either the try/catch or rethrow the exceptions.
Annotations in general never work on methods called from the same class because of how proxies are created in Spring.
It has nothing to do with #Transaction in particular but with the fact that your methods is private and called from the within same object.
Please make the method public and move the #Transactional method in a separate class annoted with #Service and called it from outside of the instance of the class
#Service
public class PublicationServiceSaverImpl {
#Transactional
**public** int save(Request request) {
...
}
}
You must call the save method from outside of the class PublicationServiceSaverImpl, maybe from PublicationServiceImpl.
The method PublicationServiceImpl.save must be public if you want to use #Transactional.
As per Spring Documentation:
When you use transactional proxies with Spring’s standard configuration, you should apply the #Transactional annotation only to methods with public visibility. If you do annotate protected, private, or package-visible methods with the #Transactional annotation, no error is raised, but the annotated method does not exhibit the configured transactional settings.
First of all: make your method public.
Second: you have to throw the exception. If you catch and not rethrow it, how do you expect the transactional processing to know that an error occured and then rollback?
You have two options: throw the Exception instead of catching it, or catch, do some further processing and then rethrow it.
So in your repository, just add a throws keyword and then rethrow the exception after the log statement:
public int executeQuery() throws Exception {
try {
keyHolder = new GenerateKeyHolder();
namedParamaterJDBCTemplate.update(getInsertQuery(), sqlParamaterSource, kekHolder, new String[] {
"pk_id"
})
return keyHolder.getKey().intValue();
} catch(Exception e) {
log.error("Exception while saving");
throw e;
}
}
Now, for your service:
Example 1 - use the throws keyword to propagate the checked exception:
#Override
#Transactional(propagation = propagation.REQUIRED, rollbackFor = Exception.class)
public int save(Request request) throws Exception {
int pk_id_partner = partnerRepo.save(request);
int pk_id_flow = flowRepo.save(request);
String publicationCode = generatePubCode(request);
int publicationCode = pubRepo.save(pk_id_partner, pk_id_flow, request);
return 0;
}
Example 2 - catch and rethrow it as an RuntimeException, which is unchecked.
#Override
#Transactional(propagation = propagation.REQUIRED)
public int save(Request request) {
try {
int pk_id_partner = partnerRepo.save(request);
int pk_id_flow = flowRepo.save(request);
String publicationCode = generatePubCode(request);
int publicationCode = pubRepo.save(pk_id_partner, pk_id_flow, request);
} catch(Exception ex) {
throw new RuntimeException(ex);
}
return 0;
}
Note that the second example doesn't need the rollbackFor argument to the #Transactional. By default, a transaction is rolled back if a unchecked exception occurs, so there's no need to explicitly use rollbackFor in cases of RuntimeExceptions.
Another verification that would have to be done if the solution does not work. It is to verify that the database tables allow the rollback. For this, the engine has to be in InnoDB and not in MyISAM and others.
In my case adding #EnableTransactionManagement annotation on Application class resolved the issue
In stead of #Transactional(propagation = propagation.REQUIRED provide #Transactional(propagation = propagation.REQUIRED_NEW
If you use the latter, it will use the parent transaction boundary, which is at class level.
And you don't need explicitly state rollbackFor=Exception.class. By default spring will roll back on exception
And do change private to public
Try this out
My question is given below. Pseudocode Code is given below:
public Object rollBackTestMainMethod(List<Object> list) {
List<Object> responseList = new ArrayList<>();
for(Object item:list){
try{
Boolean isOperationSuccess = rollBackTestSubMethod(item);
if (isOperationSuccess==null || !isOperationSuccess){
item.addError("Operation failed");
item.addSuccess(false);
} else {
item.addError(null);
item.addSuccess(true);
}
} catch(Exception exception) {
item.addError(exception.getMessage());
item.addSuccess(false);
}
responseList.add(item);
}
return responseList;
}
#Transactional(rollbackFor = {Exception.class, SQLException.class})
private Boolean rollBackTestSubMethod(Object listItem){
Long value1=save(listItem.getValue1());
if(value1==null){
throw new Exception("Error during save 1");
}
Long value2=save(listItem.getValue2());
if(value2==null){
throw new Exception("Error during save 2");
}
Long value3=save(listItem.getValue3());
if(value3==null){
throw new Exception("Error during save 3");
}
return Boolean.TRUE;
}
What I am doing here:
Iterate a list in rollBackTestMainMethod(). Sending one list item in rollBackTestSubMethod() and performing a 3 save operation.
If all save complete then returning true response, otherwise throwing an exception.
In rollBackTestMainMethod(), after getting response or exception, it is adding error or successful value on each item.
It is adding this item in new list named responseList. After all operations it is sending this back as response.
My questions:
After throwing from rollBackTestSubMethod() it will not be rolled back because it is calling from a try catch block.
If I want to forcefully roll back via TransactionAspectSupport.currentTransactionStatus().setRollbackOnly(); then it will be rolled back all item for any throw/exception.
Here I want rollback only for throw item not all item.
This method's are in a spring bean
I am saving data into my relational database via spring data jpa
My imports:
import org.springframework.transaction.annotation.Transactional;
import org.springframework.transaction.interceptor.TransactionAspectSupport;
It's because you're invoking #Transactional method from within same bean.
#Transactional only works on methods invoked on proxies created by spring. It means, that when you create a #Service or other bean, method called from the outside will be transactional. If invoked from within bean, nothing will happen, as it doesn't pass through proxy object.
The easiest solution would be to move the method to another #Service or bean. If you really want to keep it within same component, then you need to invoke it, so that it gets wrapped in proxy by spring AOP. You can do this like that:
private YourClass self;
#Autowired
private ApplicationContext applicationContext;
#PostConstruct
public void postContruct(){
self = applicationContext.getBean(YourClass.class);
}
Then invoking method on self would result in opening a transaction.
Marking a non-public method #Transactional is both useless and misleading because Spring doesn't "see" non-public methods, and so makes no provision for their proper invocation. Nor does Spring make provision for the methods invoked by the method it called.
Therefore marking a private method, for instance, #Transactional can only result in a runtime error or exception if the method is actually written to be #Transactional.
I developed a typical enterprise application that is responsible for provisioning customer to a 3rd party system. This system has a limitation, that only one thread can work on a certain customer. So we added a simple locking mechanism that consists of #Singleton which contains a Set of customerIds currently in progress. Whenever a new request comes for provisioning, it first checks this Set. If cusotomerId is present, it waits otherwise it adds it to the Set and goes into processing.
Recently it was decided, that this application will be deployed in cluster which means that this locking approach is no longer valid. We came up with a solution to use DB for locking. We created a table with single column that will contain customerIds (it also has a unique constraint). When a new provisioning request comes we start a transaction and try and lock the row with customerId with SELECT FOR UPDATE (if customerId does not exist yet, we insert it). After that we start provisioning customer and when finished, we commit transaction.
Concept works but I have problems with transactions. Currently we have a class CustomerLock with add() and remove() methods that take care of adding and removing customerIds from Set. I wanted to convert this class to a stateless EJB that has bean-managed transactions. add() method would start a transaction and lock the row while remove() method would commit transaction and thus unlocked the row. But it seems that start and end of transaction has to happen in the same method. Is there a way to use the approach I described or do I have to modify the logic so the transaction starts and ends in the same method?
CustomerLock class:
#Stateless
#TransactionManagement(TransactionManagementType.BEAN)
public class CustomerLock {
#Resource
private UserTransaction tx;
public void add(String customerId) throws Exception {
try {
tx.begin();
dsApi.lock()
} catch (Exception e) {
throw e;
}
}
public void remove(String customerId) throws Exception {
try {
tx.commit();
} catch (Exception e) {
throw e
}
}
}
CustomerProvisioner class excerpt:
public abstract class CustomerProvisioner {
...
public void execute(String customerId) {
try {
customerLock.add(customerId);
processing....
customerLock.remove(customerId);
} catch (Exception e) {
logger.error("Error", e);
}
}
...
}
StandardCustomerProvisioner class:
#Stateless
public class StandardCustomerProvisioner extends CustomerProvisioner {
...
public void provision(String customerId) {
// do some business logic
super.execute(customerId);
}
}
As #Gimby noted, you should not mix container-managed and bean-managed transactions. Since your StandardCustomerProvisioner has no annotation like "#TransactionManagement(TransactionManagementType.BEAN)" - it uses container-managed transactions, and REQUIRED by default.
You have 2 options to make it work:
1) To remove "#TransactionManagement(TransactionManagementType.BEAN)" with UserTransaction calls and run CMT
2) Add this annotation ("#TransactionManagement(TransactionManagementType.BEAN)") to StandardCustomerProvisioner and use transaction markup calls from this method, so all the invoked methods use the same transactional context. Markup calls from CustomerLock should be removed anyway.
All MyService methods are transactional. The junit test below, gets count of items, saves a new item, and gets count of items to make sure that counts has been incremented by 1.
public class MyTest extends ServiceTest{
1. int countBefore = myService.getCount(); //return n
2. myService.add(item); //item is really added to DB
3. int countAfter = myService.getCount(); //return n (sometimes n+1)
}
#Transactional(propagation=Propagation.REQUIRES_NEW, isolation=Isolation.READ_COMMITTED)
getCount(){…}
#Transactional(propagation=Propagation.REQUIRES_NEW, isolation=Isolation.SERIALIZABLE)
add(){…}
#Ignore
#ContextConfiguration(locations = { "file:src/main/resources/xxx-context.xml",
"file:src/main/resources/xxx-data.xml",
"file:src/main/resources/xxx-services.xml" })
#TransactionConfiguration(transactionManager = "txManager", defaultRollback = false)
#TestExecutionListeners( { DependencyInjectionTestExecutionListener.class,
DirtiesContextTestExecutionListener.class,
TransactionalTestExecutionListener.class,
TestListener.class})
public class ServiceTest extends AbstractUT{
#Ignore
#RunWith(SpringJUnit4ClassRunner.class)
#TestExecutionListeners( {TestListener.class})
public class AbstractUT{
When debugging (3.) returns n+1 which is what I want. But when running the test without debug I get n.
Even sometimes when running the test I get n+1 and next time I get n and when comparing the std output between the two execution, it looks exactly the same. I have enabled log4j.logger.org.springframework.transaction=TRACE and I can see:
Initializing transaction synchronization
Getting transaction for MyService.getCount
...
Completing transaction for MyService.getCount
Clearing transaction synchronization
...
Initializing transaction synchronization
Getting transaction for MyService.add
...
Completing transaction for MyService.add
Clearing transaction synchronization
...
Initializing transaction synchronization
Getting transaction for MyService.getCount
...
Completing transaction for MyService.getCount
Clearing transaction synchronization
So transactions are being executed one after the other, but how is possible that (3.) don't see the saved item?
Transaction managment is setup in my test class as per: https://stackoverflow.com/a/28657650/353985
How can I find what is going wrong?
Thanks!
Had similar issue, but in my case it did not rollback. It seems that you forgot to add #Transactional. From documentation (link)
Transaction management
In the TestContext framework, transactions are managed by the
TransactionalTestExecutionListener which is configured by default,
even if you do not explicitly declare #TestExecutionListeners on your
test class. To enable support for transactions, however, you must
configure a PlatformTransactionManager bean in the ApplicationContext
that is loaded via #ContextConfiguration semantics (further details
are provided below). In addition, you must declare Spring’s
#Transactional annotation either at the class or method level for your
tests.
Here is example form the link above.
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration
#TransactionConfiguration(transactionManager="txMgr", defaultRollback=false)
#Transactional
public class FictitiousTransactionalTest {
#BeforeTransaction
public void verifyInitialDatabaseState() {
// logic to verify the initial state before a transaction is started
}
#Before
public void setUpTestDataWithinTransaction() {
// set up test data within the transaction
}
#Test
// overrides the class-level defaultRollback setting
#Rollback(true)
public void modifyDatabaseWithinTransaction() {
// logic which uses the test data and modifies database state
}
#After
public void tearDownWithinTransaction() {
// execute "tear down" logic within the transaction
}
#AfterTransaction
public void verifyFinalDatabaseState() {
// logic to verify the final state after transaction has rolled back
}
}
A solution I found till now to pass test is to put assert in afterTransaction method
public class MyTest extends ServiceTest{
#Test
public void test(){
1. int countBefore = myService.getCount(); //return n
2. myService.add(item); //item is really added to DB
}
#AfterTransaction
public void verifyFinalDatabaseState() {
3. int countAfter = myService.getCount(); //return n (sometimes n+1)
//Now always return n+1
}
I would have asked this in a comment but since my reputation does not allow it, I would just try to provide an answer.
It is possible that your are using and ORM that caches the results of count query. Depending how your add/getCount methods are implemented and the configurations of the ORM and datasource, on your second invocation of getCount, you might get a cached value obtained during first invocation of getCount.
This does not explain however why in debug mode you always get the correct result.
Because the current running transaction is set at the test method level, you have two options:
You either remove the #Transactional from the test method and rely on your service method #Transactional boundaries. This way when you call:
int countBefore = myService.getCount();
myService.add(item);
int countAfter = myService.getCount();
Each service call will run in an isolated transaction, just like it happens in the run-time production call.
You flush the Hibernate Session, just after adding the item:
int countBefore = myService.getCount();
myService.add(item);
transactionTemplate.execute(new TransactionCallback<Void>() {
#Override
public Company doInTransaction(TransactionStatus transactionStatus) {
entityManager.flush();
return null;
}
});
int countAfter = myService.getCount();
A HQL/JPQL count query should trigger a flush in AUTO flush mode, but an SQL native query doesn't flush the Session.