This question already has answers here:
How to flush data into db inside active spring transaction?
(4 answers)
Closed 7 years ago.
I'm using Spring / Spring-data-JPA and find myself needing to manually force a commit in a unit test. My use case is that I am doing a multi-threaded test in which I have to use data that is persisted before the threads are spawned.
Unfortunately, given that the test is running in a #Transactional transaction, even a flush does not make it accessible to the spawned threads.
#Transactional
public void testAddAttachment() throws Exception{
final Contract c1 = contractDOD.getNewTransientContract(15);
contractRepository.save(c1);
// Need to commit the saveContract here, but don't know how!
em.getTransaction().commit();
List<Thread> threads = new ArrayList<>();
for( int i = 0; i < 5; i++){
final int threadNumber = i;
Thread t = new Thread( new Runnable() {
#Override
#Transactional
public void run() {
try {
// do stuff here with c1
// sleep to ensure that the thread is not finished before another thread catches up
Thread.sleep(1000);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
});
threads.add(t);
t.start();
}
// have to wait for all threads to complete
for( Thread t : threads )
t.join();
// Need to validate test results. Need to be within a transaction here
Contract c2 = contractRepository.findOne(c1.getId());
}
I've tried using the entity manager to, but get an error message when I do:
org.springframework.dao.InvalidDataAccessApiUsageException: Not allowed to create transaction on shared EntityManager - use Spring transactions or EJB CMT instead; nested exception is java.lang.IllegalStateException: Not allowed to create transaction on shared EntityManager - use Spring transactions or EJB CMT instead
at org.springframework.orm.jpa.EntityManagerFactoryUtils.convertJpaAccessExceptionIfPossible(EntityManagerFactoryUtils.java:293)
at org.springframework.orm.jpa.aspectj.JpaExceptionTranslatorAspect.ajc$afterThrowing$org_springframework_orm_jpa_aspectj_JpaExceptionTranslatorAspect$1$18a1ac9(JpaExceptionTranslatorAspect.aj:33)
Is there any way to commit the transaction and continue it? I have been unable to find any method that allows me to call a commit().
I had a similar use case during testing hibernate event listeners which are only called on commit.
The solution was to wrap the code to be persistent into another method annotated with REQUIRES_NEW. (In another class) This way a new transaction is spawned and a flush/commit is issued once the method returns.
Keep in mind that this might influence all the other tests! So write them accordingly or you need to ensure that you can clean up after the test ran.
Why don't you use spring's TransactionTemplate to programmatically control transactions? You could also restructure your code so that each "transaction block" has it's own #Transactional method, but given that it's a test I would opt for programmatic control of your transactions.
Also note that the #Transactional annotation on your runnable won't work (unless you are using aspectj) as the runnables aren't managed by spring!
#RunWith(SpringJUnit4ClassRunner.class)
//other spring-test annotations; as your database context is dirty due to the committed transaction you might want to consider using #DirtiesContext
public class TransactionTemplateTest {
#Autowired
PlatformTransactionManager platformTransactionManager;
TransactionTemplate transactionTemplate;
#Before
public void setUp() throws Exception {
transactionTemplate = new TransactionTemplate(platformTransactionManager);
}
#Test //note that there is no #Transactional configured for the method
public void test() throws InterruptedException {
final Contract c1 = transactionTemplate.execute(new TransactionCallback<Contract>() {
#Override
public Contract doInTransaction(TransactionStatus status) {
Contract c = contractDOD.getNewTransientContract(15);
contractRepository.save(c);
return c;
}
});
ExecutorService executorService = Executors.newFixedThreadPool(5);
for (int i = 0; i < 5; ++i) {
executorService.execute(new Runnable() {
#Override //note that there is no #Transactional configured for the method
public void run() {
transactionTemplate.execute(new TransactionCallback<Object>() {
#Override
public Object doInTransaction(TransactionStatus status) {
// do whatever you want to do with c1
return null;
}
});
}
});
}
executorService.shutdown();
executorService.awaitTermination(10, TimeUnit.SECONDS);
transactionTemplate.execute(new TransactionCallback<Object>() {
#Override
public Object doInTransaction(TransactionStatus status) {
// validate test results in transaction
return null;
}
});
}
}
I know that due to this ugly anonymous inner class usage of TransactionTemplate doesn't look nice, but when for some reason we want to have a test method transactional IMHO it is the most flexible option.
In some cases (it depends on the application type) the best way to use transactions in Spring tests is a turned-off #Transactional on the test methods. Why? Because #Transactional may leads to many false-positive tests. You may look at this sample article to find out details. In such cases TransactionTemplate can be perfect for controlling transaction boundries when we want that control.
Related
I am new to Hibernate. As far as I understood all modifying actions with Hibernate are done inside entityManager.getTransaction().begin() and entityManager.getTransaction().commit(), hence Hibernate under the hood always does everything within transaction boundaries (except for read-only operations). If so, why do we need to explicitly use #Transactional annotation?
UPDATE
So basically I can write something like
void foo() {
try {
em.getTransaction().begin();
...
em.persist(a1); // 1 operation
...
em.persist(a2); // 2 operation
...
em.commit();
catch (Exception e) {
em.rollback();
throw new RuntimeException(e);
}
}
which is equivalent to
#Transactional
void foo() {
em.persist(a1);
...
em.persist(a2);
}
or if I want to split all my actions into more than one method, I create one wrapper method, annotate it with #Transactional and then call other methods within this wrapper method
#Transactional
void call() {
foo();
bar();
}
void foo() {
em.persist(a1);
}
void bar() {
em.persist(a2);
}
And both foo() and bar() will be executed in terms of one transaction, is that correct?
If you only ever make a single insert/update/delete per transaction, yes than using #Transactional is unnecessary. But if you want to make multiple insert/update/delete than #Transactional can be used to group all those to a single transaction.
I have Service.class with start() and asychronous() method :
public ResponseEntity<Object> start() throws APICommandConstructionException, APICommunicationException, APIReplyParseException,
APIErrorResponse, IOException {
List<Company> companiesList = dbHandler.retrieveCompaniesList();
Company company = null;
for (int i = 0; i < companiesList.size(); i++) {
asychronousMethod(companiesList, i, company);
}
return new ResponseEntity<Object>("Start method has Finished", HttpStatus.OK);
}
#Async("threadPoolTaskExecutor")
public void asychronousMethod(List<Company> companiesList, int i, Company company) throws APICommandConstructionException, APIReplyParseException, APICommunicationException, APIErrorResponse, IOException {
company = companiesList.get(i);
company = utils.websiteScrap(company);
companiesRepository.save(company);
}
Everything that is inside a loop doesn't run in parallel, but it starts second loop after first finished. Why is that? How to do it parallel?
In brief: you shouldn't call explicitly methods with Spring annotations.
More detailed:
Spring creates special proxies that on back-stage do 'magic' for you. So if you have async annotation, that means that (depending on compile- and runtime-configuration of Spring) there was some hidden part of code that is not executed when you invoke this.asynchronousMethod.
How to fix:
First of all method should match public CompletableFuture<Void> - such way you know when thread is complete.
Second instead of this you need resolve self-instance as Spring proxy.
The simplest way over #Autowired:
#Autowired
MyClass zhis;
.... //in for loop:
future = zhis.asychronousMethod(companiesList, i, company);
P.s please see good example at https://spring.io/guides/gs/async-method/
I'm having a problem when I trying to delete data from the DB using multiple threads with Hibernate.
Repo:
#Modifying
#Query("DELETE FROM Customer cus WHERE cus.customerId in :customerIds")
public void deleteByCustomerIds(#Param("customerIds") List<Long> customerIds);
Service:
public runDelete (List<Long> customerIds) {
List<List<Long>> partitions = Lists.partition(customerIds, 5000);
for(int i = 0; i < partitions.size(); i++ ) {
final int index = i;
Runnable thread = () -> deleteCustomersInBatches(partitions.get(index));
new Thread(thread).start();
}
}
#Transactional(propagation = Propagation.REQUIRES_NEW)
private void deleteCustomerInBatches(List<Long> customerIds) {
for (List<Long> batch : Lists.partition(oldCalcIds, 1000)) {
customerRepo.deleteByCustomerIds(batch);
}
}
This is how code looks like, I have the #Transactional tag on the service layer where the repo call is being made.
at java.lang.Thread.run(Thread.java:748) Caused by:
javax.persistence.TransactionRequiredException: Executing an
update/delete query
at org.hibernate.jpa.spi.AbstractQueryImpl.executeUpdate(AbstractQueryImpl.java:54)
I keep getting this error. Any help is appreciated.
It's because you're invoking #Transactional method from within same bean.
#Transactional only works on methods invoked on proxies created by spring. It means, that when you create a #Service or other bean, method called from the outside will be transactional. If invoked from within bean, nothing will happen, as it doesn't pass through proxy object.
The easiest solution would be to move the method to another bean. If you really want to keep it within same component, then you need to invoke it, so that it gets wrapped in proxy by spring AOP. You can do this like that:
private YourClass self;
#Autowired
private ApplicationContext applicationContext;
#PostConstruct
public void postContruct(){
self = applicationContext.getBean(YourClass.class);
}
Then invoking method on self would result in opening a transaction.
The title might be incorrect, but I will try to explain my issue. My project is a Spring Boot project. I have services which do calls to external REST endpoints.
I have a service method which contains several method calls to other services I have. Every individual method call can be successful or not. Every method call is done to a REST endpoint and there can be issues that for example the webservice is not available or that it throws an unknown exception in rare cases. What ever happens, I need to be able to track which method calls were successful and if any one of them fails, I want to rollback to the original state as if nothing happened, see it a bit as #Transactional annotation. All REST calls are different endpoints and need to be called separately and are from an external party which I don't have influence on. Example:
public MyServiceImpl implements MyService {
#Autowired
private Process1Service;
#Autowired
private Process2Service;
#Autowired
private Process3Service;
#Autowired
private Process4Service;
public void bundledProcess() {
process1Service.createFileRESTcall();
process2Service.addFilePermissionsRESTcall();
process3Service.addFileMetadataRESTcall(); <-- might fail for example
process4Service.addFileTimestampRESTcall();
}
}
If for example process3Service.addFileMetadataRESTcall fails I want to do something like undo (in reverse order) for every step before process3:
process2Service.removeFilePermissionsRESTcall();
process1Service.deleteFileRESTcall();
I read about the Command pattern, but that seems to be used for Undo actions inside an application as a sort of history of actions performed, not inside a Spring web application. Is this correct for my use case too or should I track per method/webservice call if it was successful? Is there a best practice for doing this?
I guess however I track it, I need to know which method call failed and from there on perform my 'undo' method REST calls. Although in theory even these calls might also fail of course.
My main goal is to not have files being created (in my example) which any further processes have not been performed on. It should either be all successful or nothing. A sort of transactional.
Update1: improved pseudo implementation based on comments:
public Process1ServiceImpl implements Process1Service {
public void createFileRESTcall() throws MyException {
// Call an external REST api, pseudo code:
if (REST-call fails) {
throw new MyException("External REST api failed");
}
}
}
public class BundledProcessEvent {
private boolean createFileSuccess;
private boolean addFilePermissionsSuccess;
private boolean addFileMetadataSuccess;
private boolean addFileTimestampSuccess;
// Getters and setters
}
public MyServiceImpl implements MyService {
#Autowired
private Process1Service;
#Autowired
private Process2Service;
#Autowired
private Process3Service;
#Autowired
private Process4Service;
#Autowired
private ApplicationEventPublisher applicationEventPublisher;
#Transactional(rollbackOn = MyException.class)
public void bundledProcess() {
BundleProcessEvent bundleProcessEvent = new BundleProcessEvent();
this.applicationEventPublisher.publishEvent(bundleProcessEvent);
bundleProcessEvent.setCreateFileSuccess = bundprocess1Service.createFileRESTcall();
bundleProcessEvent.setAddFilePermissionsSuccess = process2Service.addFilePermissionsRESTcall();
bundleProcessEvent.setAddFileMetadataSuccess = process3Service.addFileMetadataRESTcall();
bundleProcessEvent.setAddFileTimestampSuccess = process4Service.addFileTimestampRESTcall();
}
#TransactionalEventListener(phase = TransactionPhase.AFTER_ROLLBACK)
public void rollback(BundleProcessEvent bundleProcessEvent) {
// If the last process event is successful, we should not
// be in this rollback method even
//if (bundleProcessEvent.isAddFileTimestampSuccess()) {
// remove timestamp
//}
if (bundleProcessEvent.isAddFileMetadataSuccess()) {
// remove metadata
}
if (bundleProcessEvent.isAddFilePermissionsSuccess()) {
// remove file permissions
}
if (bundleProcessEvent.isCreateFileSuccess()) {
// remove file
}
}
Your operation looks like a transaction, so you can use #Transactional annotation. From your code I can't really tell how you are managing HTTP response calls for each of those operations, but you should consider having your service methods to return them, and then do a rollback depending on response calls. You can create an array of methods like so, but how exactly you want your logic to be is up to you.
private Process[] restCalls = new Process[] {
new Process() { public void call() { process1Service.createFileRESTcall(); } },
new Process() { public void call() { process2Service.addFilePermissionsRESTcall(); } },
new Process() { public void call() { process3Service.addFileMetadataRESTcall(); } },
new Process() { public void call() { process4Service.addFileTimestampRESTcall(); } },
};
interface Process {
void call();
}
#Transactional(rollbackOn = Exception.class)
public void bundledProcess() {
restCalls[0].call();
... // say, see which process returned wrong response code
}
#TransactionalEventListener(phase = TransactionPhase.AFTER_ROLLBACK)
public void rollback() {
// handle rollback according to failed method index
}
Check this article. Might come in handy.
The answer to this question is quite broad. There are various ways to do distributed transactions to go through them all here. However, since you are using Java and Spring, your best bet is to use something like JTA (Java Transaction API), which enables a distributed transactions across multiple services/instances/etc.. Fortunately, Spring Boot supports JTA using either Atomikos or Bitronix. You can read the doc here.
One approach to enable distributed transactions is through a message broker such as JMS, RabbitMQ, Kafka, ActiveMQ, etc. and use a protocol like XA transactions (two-phase commit). In the case of external services that do not support distributed, one approach is to write a wrapper service that understands XA transactions to that external service.
I developed a typical enterprise application that is responsible for provisioning customer to a 3rd party system. This system has a limitation, that only one thread can work on a certain customer. So we added a simple locking mechanism that consists of #Singleton which contains a Set of customerIds currently in progress. Whenever a new request comes for provisioning, it first checks this Set. If cusotomerId is present, it waits otherwise it adds it to the Set and goes into processing.
Recently it was decided, that this application will be deployed in cluster which means that this locking approach is no longer valid. We came up with a solution to use DB for locking. We created a table with single column that will contain customerIds (it also has a unique constraint). When a new provisioning request comes we start a transaction and try and lock the row with customerId with SELECT FOR UPDATE (if customerId does not exist yet, we insert it). After that we start provisioning customer and when finished, we commit transaction.
Concept works but I have problems with transactions. Currently we have a class CustomerLock with add() and remove() methods that take care of adding and removing customerIds from Set. I wanted to convert this class to a stateless EJB that has bean-managed transactions. add() method would start a transaction and lock the row while remove() method would commit transaction and thus unlocked the row. But it seems that start and end of transaction has to happen in the same method. Is there a way to use the approach I described or do I have to modify the logic so the transaction starts and ends in the same method?
CustomerLock class:
#Stateless
#TransactionManagement(TransactionManagementType.BEAN)
public class CustomerLock {
#Resource
private UserTransaction tx;
public void add(String customerId) throws Exception {
try {
tx.begin();
dsApi.lock()
} catch (Exception e) {
throw e;
}
}
public void remove(String customerId) throws Exception {
try {
tx.commit();
} catch (Exception e) {
throw e
}
}
}
CustomerProvisioner class excerpt:
public abstract class CustomerProvisioner {
...
public void execute(String customerId) {
try {
customerLock.add(customerId);
processing....
customerLock.remove(customerId);
} catch (Exception e) {
logger.error("Error", e);
}
}
...
}
StandardCustomerProvisioner class:
#Stateless
public class StandardCustomerProvisioner extends CustomerProvisioner {
...
public void provision(String customerId) {
// do some business logic
super.execute(customerId);
}
}
As #Gimby noted, you should not mix container-managed and bean-managed transactions. Since your StandardCustomerProvisioner has no annotation like "#TransactionManagement(TransactionManagementType.BEAN)" - it uses container-managed transactions, and REQUIRED by default.
You have 2 options to make it work:
1) To remove "#TransactionManagement(TransactionManagementType.BEAN)" with UserTransaction calls and run CMT
2) Add this annotation ("#TransactionManagement(TransactionManagementType.BEAN)") to StandardCustomerProvisioner and use transaction markup calls from this method, so all the invoked methods use the same transactional context. Markup calls from CustomerLock should be removed anyway.