I have the following setup:
#Component
public class Scheduler {
Logger logger = LoggerFactory.getLogger(this.getClass());
#Autowired
BatchService batchService;
#Scheduled(cron = "0 */1 * ? * *")
void tick() {
logger.info("Beginning of a batch tick");
batchService.refundNotAssignedVisits();
logger.info("End of the batch tick");
}
}
With BatchService containing the following:
#Service
public class BatchServiceImpl implements BatchService {
Logger logger = LoggerFactory.getLogger(this.getClass());
#Autowired
VisitService visitService;
#Override
#Transactional
public void refundNotAssignedVisits() {
logger.info("Start automatic refund of past visits being assigned");
Set<Visit> visits = visitService.findRefundableVisits();
if(visits != null && visits.size() != 0) {
logger.info("Found " + visits.size() + " visits to refund with IDs: " + visits.stream().map(x -> x.getId().toString()).collect(Collectors.joining(", ")));
visits.forEach(x -> {
logger.info("Refunding visit with ID: " + x.getId());
try {
visitService.cancel(x);
logger.info("Visit successfully refunded!");
}
catch(Exception e) {
logger.error("Error while refunding visit...", e);
}
});
}
else {
logger.info("Found no visit to refund.");
}
logger.info("End of automatic refund");
}
}
And the cancel method defined like this:
#Override
#Transactional(propagation = Propagation.REQUIRES_NEW)
public Visit cancel(Visit visit) throws Exception {
// Some business logic
}
I need the cancel method to have one transaction per call, for business purposes, and at the moment, the refundNotAssignedVisits is #Transactional in order to enable a Hibernate session so I am able to use lazy loading with related entities in the cancel method.
This causes problems such as duplicate commits and I'm wondering what's a good pattern to achieve what I want: have a #Scheduled method that enables a Hibernate session in order to make multiple calls to another method with one transaction per call.
#Transactional 's REQUIRES_NEW will create another new Hibernate session , so the session inside cancel() will be different from the session that is used to load the entities which seems like awkward to me. Normally , we use the same session for loading and managing the same entity within a transaction.
I will refactor the codes into the followings :
VisitService:
//Cannel by visitorId and load the Visitor by Id in a new transaction
#Override
#Transactional(propagation = Propagation.REQUIRES_NEW)
public Visit cancel(Integer visitorId) throws Exception {
Visit visit= session.get(Visit.class , visitorId);
cancel(visit);
}
#Override
public Visit cancel(Visit visit) throws Exception {
// Some business logic
}
//Add method to return the IDs only
#Transactional(readOnly=true)
public Set<Integer> findRefundableVisitId(){
}
BatchService:
//#Transactional (Do not require anymore)
public void refundNotAssignedVisits() {
logger.info("Start automatic refund of past visits being assigned");
Set<Integer> refundVisitIds = visitService.findRefundableVisitId();
refundVisitIds.forEach( id-> {
try {
visitService.refund(id);
logger.info("Visit successfully refunded!");
}
catch(Exception e) {
logger.error("Error while refunding visit...", e);
}
});
}
In this way , each refund is executed in their own transaction and the transaction that is used to load the refund visitors do not need to wait for all refund complete in order to commit and no more "duplicate commits".
Related
Bank.java
#Stateless
#Local
public class Bank implements IBank {
#EJB
IConfigBean iConfigBean;
#EJB
IDbs iDBS;
#EJB
IPosb iPosb;
#Override
public void doTransaction() {
System.out.println("--Bank Transaction Started--");
try {
Config config1 = getConfig(1);
iConfigBean.create(config1);
iDBS.doDBSTransaction();
Config config3 = getConfig(3);
iConfigBean.create(config3);
iPosb.doPOSBTransaction();
Config config5 = getConfig(5);
iConfigBean.create(config5);
} catch (Exception e) {
e.printStackTrace();
System.out.println("---Bank Exception--");
}
System.out.println("--Bank Transaction End--");
}
#Override
public Config getConfig(int inserttionOrderNo) {
Config config = new Config();
config.setType("EJBTransactionTESTING - " + inserttionOrderNo);
return config;
}
}
DBS.java
#Stateless
#Local
public class DBS implements IDbs {
#EJB
IConfigBean iConfigBean;
#Override
public void doDBSTransaction() {
System.out.println("--DBS Transaction Started--");
try {
Config config2 = getConfig(2);
iConfigBean.create(config2);
} catch (Exception e) {
e.printStackTrace();
System.out.println("--DBS Exception--");
}
System.out.println("--DBS Transaction End--");
}
#Override
public Config getConfig(int inserttionOrderNo) {
Config config = new Config();
config.setType("EJBTransactionTESTING - " + inserttionOrderNo);
return config;
}
}
POSB.java
#Stateless
#Local
public class POSB implements IPosb {
#EJB
IConfigBean iConfigBean;
#Override
public void doPOSBTransaction() {
System.out.println("--POSB Transaction Started--");
try {
Config config4 = getConfig(4);
iConfigBean.create(config4);
if (true) {
//For Test 1
//throw new NullPointerException();
}
} catch (Exception e) {
e.printStackTrace();
System.out.println("--POSB Exception--");
}
if (true) {
//For Test 2
// throw new NullPointerException();
}
System.out.println("--POSB Transaction End--");
}
#Override
public Config getConfig(int inserttionOrderNo) {
Config config = new Config();
config.setType("EJBTransactionTESTING - " + inserttionOrderNo);
return config;
}
}
I am new to Stack Overflow and Its my new question so correct me If I am wrong.
Environment is..
Windows 10
Java 1.8
Eclipse
Tomcat 8.5
EJB3
I have Three stateless bean, Please look at the Sequence Diagram of the Transaction flow.
I purposely making NullPointer Exception at two places during the transaction to know the difference and I have marked with Lightening Bold symbol in sequence diagram.
I am not using any #TransactionAttribute to any methods.
Test 1 - Null Pointer in Inside the try block (Lightening Bold symbol with Green)
When I start the testing, Got Null pointer exception and all the transaction are not marked for roll back and data also got inserted in db.
I can only see Null pointer exception in the console log.
Test 2 - Null Pointer in Outside the try - catch method (Lightening Bold symbol with Red)
When I start the testing, Got Null pointer exception plus EJBTransactionRolledbackException and all the transaction marked for roll back and no data inserted in db.
I can see NullPointer and EJBTransactionRolledback Exception in the console log.
Question here is,
Why EJB transaction is not marked for roll back If I made Null pointer inside try block
Why EJB transaction is roll back happens If I made null pointer outside try block
Thanks in advance.
Keep in mind EJB calls, all the "magic" made by container happens there, including transaction markup. It's possible due to the fact that EJB calls are not direct, but always go through proxy.
You have such calls in your code:
iPosb.doPOSBTransaction();
So, if unchecked exception (NPE for example) is thrown in this method and not caught - it's ultimately caught by container due to EJB proxy wrapping the call above. In this case transaction only could be rolled back.
Adding a call to a method of the same bean in your method (without using #EJB reference), does not change that:
#Override
public void doPOSBTransaction() {
try {
Config config4 = getConfig(4);
iConfigBean.create(config4);
if (true) {
newMethod();
}
} catch (Exception e) {
e.printStackTrace();
System.out.println("--POSB Exception--");
}
if (true) {
newMethod();
}
}
private void newMethod(){
throw new RuntimeException();
}
You can easily check that commit/rollback behaviour is just the same in this case, no matter that a method is added to call stack.
So the important thing you have to remember is that all container tricks work only on #EJB calls. So, for example, it's pointless to place transactional annotation on a private method - it won't be used ever.
Another important point is about checked exceptions. By default these do not cause transaction rollback indeed. But it's still possible to annotate your checked exception like below to make it rollback ongoing transaction:
#ApplicationException(rollback = true)
I have a method that is annotated with
#Transactional(isolation = Isolation.SERIALIZABLE, propagation = Propagation.REQUIRES_NEW, rollbackFor = Exception.class)
and calls several repository methods. Now when one repository tries to alter a DB-row that is locked and not rolled back by another instance of the method, spring correctly throws
org.springframework.orm.jpa.JpaSystemException: could not execute statement [...] Caused by: java.sql.SQLException: transaction could not be serialized and rolls back the failed transaction.
Now I want to keep all this behaviour but additionally handle the exception and start a retry. Here's a code snippet:
#Transactional(isolation = Isolation.SERIALIZABLE, propagation = Propagation.REQUIRES_NEW, rollbackFor = Exception.class)
#Lock(LockModeType.PESSIMISTIC_WRITE)
public void messageReceived(Card card) {
this.request = request;
this.numberOfAttempts = reset ? 0 : this.numberOfAttempts++;
this.reset = true;
LOGGER.info("Message received from Queue: " + card);
TransactionDebugUtils.transactionRequired("MessageReceivedController.messageReceived");
try {
[...]
repository1.createKonto(card);
repository2.doStuff(card);
} catch (JpaSystemException e) {
//This is obviously never invoked
LOGGER.error("TRANSACTION FAILED!!!");
} catch (Exception e) {
LOGGER.error("Error mapping json request to data model", message, e);
}
}
#ExceptionHandler(JpaSystemException.class)
//This is also never invoked
public void handleJpaSystemException(JpaSystemException ex) {
this.messageReceived(this.request);
this.reset = false;
}
I had this issue recently. As it is a method level #Transactional annotation, Transaction commit occurs after finishing method execution.
When you are using this annotation 2 concepts should be considered
persistence context
database transaction
After messageReceived() method is executed, those 2 things will happen and JPA exception is thrown at #Transactional level which means you need to handle this exception from where you are calling this method(controller; if you are calling from a controller).
More regarding #Transactional can be found in this link.
I have the following scheduled piece of code in my Spring Boot Application:
#Scheduled(fixedDelay = DELAY_SECONDS)
private void processJobQueue() {
BlockingQueue<ReportDeliverable> jobQueue = JobQueueFactory.getJobQueueInstance();
while (!jobQueue.isEmpty()) {
//do stuff
if (rCount == 0) {
status = send(reportDeliverable);
if (status == TransferStatus.FAILURE) {
populateQueue(reportDeliverable);
}
if (status == TransferStatus.SUCCESS) { //write the metadata to database
int i = dbMetadataWriter.writeMetadata(reportDeliverable);
}
} else if (rCount == -1) {
populateQueue(reportDeliverable);
} else
logger.info("Record exists in MetaData for {}. Ignoring the File transfer....", reportDeliverable.getFilePath());
}
}
In my DBMetadataWriter component, the writeMetadataWriter() looks something like this:
#Component
public class DBMetadataWriter {
public int writeMetadata(final ReportDeliverable reportDeliverable) {
int nbInserted = 0;
try {
nbInserted = jdbcTemplate.update(PORTAL_METADATA_INSERT, insertDataValues);
} catch (Exception e) {
logger.error("Could not insert metadata for {}, Exception: {} ", reportDeliverable.toString(), e.getMessage());
}
return nbInserted;
}
In some cases, when writing the insert to the database, I get table space issues with the database at which point I think it would be wise for me to shut down the spring boot application until table space problems are resolved.
What would be the correct way to handle these rare cases? What technique can I use to gracefully shutdown the spring boot application and how can I do it in the above code?
My entry point class where I initially validate all my database connections before processing etc has the following...
#Component
public class RegisterReportSchedules implements ApplicationListener<ContextRefreshedEvent> {
#Autowired
private ApplicationContext applicationContext;
#Override
public void onApplicationEvent(ContextRefreshedEvent contextRefreshedEvent) {
}
private void shutdownApplication() {
int exitCode = SpringApplication.exit(applicationContext, (ExitCodeGenerator) () -> 0);
System.exit(exitCode);
}
}
You have exit() method on SpringApplication class, which can be used for exiting Spring boot application gracefully.
It requires 2 paramerter:
ApplicationContext
ExitCodeGenerator
For further reading:
https://docs.spring.io/spring-boot/docs/current/api/org/springframework/boot/SpringApplication.html#exit-org.springframework.context.ApplicationContext-org.springframework.boot.ExitCodeGenerator...-
Code Example:
#Autowired
public void shutDown(ExecutorServiceExitCodeGenerator exitCodeGenerator) {
SpringApplication.exit(applicationContext, exitCodeGenerator);
}
Call this method when you get exception for No Table space
#Retryable doesn't seem to be working on 2nd level of methods as in sphRemoteCall below. I see that a proxy is created but it is never retried on failures.
Once I moved #Retryable to the 1st level of methods like getSubscriberAccount, it's started working.
Example below:
#Service
public class SphIptvClient extends WebServiceGatewaySupport {
//Works over here
#Retryable(maxAttempts=3, backoff=#Backoff(delay=100))
public GetSubscriberAccountResponse getSubscriberAccount(String loginTocken, String billingServId) {
GetSubscriberAccountResponse response = (GetSubscriberAccountResponse) sphRemoteCall(sphIptvEndPoint, getSubAcc, "xxxxx");
return response;
}
/*
* Retryable is not working on the 2nd level methods in the bean.
* It works only with methods which are called directly from outside
* if there is 2nd level method, like this, Retryable is not working.
*/
//#Retryable
private Object sphRemoteCall(String uri, Object requestPayload, String soapAction) {
log.debug("Calling the sph for uri:{} and soapAction:{}", uri, soapAction);
return getWebServiceTemplate().marshalSendAndReceive(uri, requestPayload, new SoapActionCallback(soapAction));
}
}
#Configuration
#EnableRetry
public class SphClientConfig {
#Bean
public SphIptvClient sphIptvClient() {
SphIptvClient client = new SphIptvClient();
return client;
}
}
So this is a super late answer, but since I've just come here and confronted the same problem (again, after years ago wrestling with transactions) I'll furnish a little more fleshed out solution and hopefully someone will find it useful. Suffice to say that #M. Deinum's diagnosis is correct.
In the above case, and to paraphrase Understanding AOP proxies, any place where SphIptvClient gets autowired will be given a reference to a proxy which Spring Retry will create when #EnableRetry is handled:
"The #EnableRetry annotation creates proxies for #Retryable beans" - Declarative Retry - Spring Retry
Once getSubscriberAccount has been invoked and execution has passed through the proxy and into the #Service instance of the object, no reference to the proxy is known. As a result sphRemoteCall is called as if there were no #Retryable at all.
You could work with the framework by shuffling code around in such a way as to allow getSubscriberAccount to call a proxy-ed sphRemoteCall, which requires a new interface and class implementation.
For example:
public interface SphWebService {
Object sphRemoteCall(String uri, Object requestPayload, String soapAction);
}
#Component
public class SphWebServiceImpl implements SphWebService {
#Retryable
public Object sphRemoteCall(String uri, Object requestPayload, String soapAction) {
log.debug("Calling the sph for uri:{} and soapAction:{}", uri, soapAction);
return getWebServiceTemplate().marshalSendAndReceive(uri, requestPayload, new SoapActionCallback(soapAction));
}
}
#Service
public class SphIptvClient extends WebServiceGatewaySupport {
#Autowired
SphWebService sphWebService;
#Retryable(maxAttempts=3, backoff=#Backoff(delay=100))
public GetSubscriberAccountResponse getSubscriberAccount(String loginTocken, String billingServId) {
GetSubscriberAccountResponse response = (GetSubscriberAccountResponse) this.sphWebService.sphRemoteCall(sphIptvEndPoint, getSubAcc, "xxxxx");
return response;
}
}
#Configuration
#EnableRetry
public class SphClientConfig {
// the #Bean method was unnecessary and may cause confusion.
// #Service was already instantiating SphIptvClient behind the scenes.
}
#Retryable only works on the methods when called directly from other classes.
If you will try to invoke one method with #Retryable annotation from some other method of the same class, it will eventually not work.
// any call from this method to test method will not invoke the retry logic.
public void yetAnotherMethod() {
this.test();
}
// it will work
#Retryable(value = {RuntimeException.class}, backoff = #Backoff(delay = 1500))
public void test() {
System.out.println("Count: " + count++);
throw new RuntimeException("testing");
}
#Recover
public void recover() {
System.out.println("Exception occured.");
}
So, the output if test method is called, will be:
Count: 0
Count: 1
Count: 2
Exception occured.
But, if the yetAnotherMethod is called, output will be:
Count: 0
And a Runtime exception will be thrown.
Suppose you have a method which calls certain API - callAPI() and you want to implement retry logic over it, you can try use a do while, as it will execute only once, if successful.
Method to hit the external API
public int callAPI() {
return 1;
}
Method to implement retry logic
public int retrylogic() throws InterruptedException {
int retry = 0;
int status = -1;
boolean delay = false;
do {
// adding a delay, if you want some delay between successive retries
if (delay) {
Thread.sleep(2000);
}
// Call the actual method, and capture the response,
// and also catch any exception which occurs during the call.
// (Network down/ endpoint not avaliable
try {
status = callAPI();
}
catch (Exception e) {
System.out.println("Error occured");
status = -1;
}
finally {
switch (status) { //now based on error response or any exception you retry again
case HTTPStatus.OK:
System.out.println("OK");
return status;
default:
System.out.println("Unknown response code");
break;
}
retry++;
System.out.println("Failed retry " + retry + "/" + 3);
delay = true;
}
}while (retry < 3);
return status;
}
Setup: Spring application deployed on Weblogic 12c, using JNDI lookup to get a datasource to the Oracle Database.
We have multiple services which will be polling the database regularly for new jobs. In order to prevent two services picking the same job we are using a native SELECT FOR UPDATE query in a CrudRepository. The application then takes the resulting job and updates it to PROCESSING instead of WAITING using the CrusRepository.save() method.
The problem is that I can't seem to get the save() to work within the FOR UPDATE transaction (at least this is my current working theory of what goes wrong), and as a result the entire polling freezes until the default 10 minute timeout occurs. I have tried putting #Transactional (with various propagation flags) basically everywhere, but I'm not able to get it to work (#EnableTransactionManagement is activated and working).
Obviously there must be some basic knowledge I'm missing. Is this even a possible setup? Unfortunately, just using #Transactional with a non-native CrudRepository SELECT query is not possible, as it apparently first makes a SELECT to see if the row is locked or not, and only then makes a new SELECT that locks it. Another service could very well pick up the same job in the meanwhile, which is why we need it to lock immediately.
Update in relation to #M. Deinum's comment.: I should perhaps also mention that it's a setup wherein the central component that's doing the polling is a library used by all the other services (therefore the library has #SpringBootApplication, as does each service using it, so double component scanning is certainly present). Furthermore, the service has two separate classes for polling depending on the type of service, with a lot of common code, shared in an AbstractTransactionHelper class. Below I've aggregated some code for the sake of brevity.
The library's main class:
#SpringBootApplication
#EnableTransactionManagement
#EnableJpaRepositories
public class JobsMain {
public static void initializeJobsMain(){
PersistenceProviderResolverHolder.setPersistenceProviderResolver(new PersistenceProviderResolver() {
#Override
public List<PersistenceProvider> getPersistenceProviders() {
return Collections.singletonList(new HibernatePersistenceProvider());
}
#Override
public void clearCachedProviders() {
//Not quite sure what this should do...
}
});
}
#Bean
public JtaTransactionManager transactionManager(){
return new WebLogicJtaTransactionManager();
}
public DataSource dataSource(){
final JndiDataSourceLookup dsLookup = new JndiDataSourceLookup();
dsLookup.setResourceRef(true);
DataSource dataSource = dsLookup.getDataSource("Jobs");
return dataSource;
}
}
The repository (we're returning a set with only one job as we had some other issues when returning a single object):
public interface JobRepository extends CrudRepository<Job, Integer> {
#Query(value = "SELECT * FROM JOB WHERE JOB.ID IN "
+ "(SELECT ID FROM "
+ "(SELECT * FROM JOB WHERE "
+ "JOB.STATUS = :status1 OR "
+ "JOB.STATUS = :status2 "
+ "ORDER BY JOB.PRIORITY ASC, JOB.CREATED ASC) "
+ "WHERE ROWNUM <= 1) "
+ "FOR UPDATE", nativeQuery = true)
public Set<Job> getNextJob(#Param("status1") String status1, #Param("status2") String status2);
The transaction handling class:
#Service
public class JobManagerTransactionHelper extends AbstractTransactionHelper{
#Transactional
#Override
public QdbJob getNextJobToProcess(){
Set<Job> jobs = null;
try {
jobs = jobRepo.getNextJob(Status.DONE.name(), Status.FAILED.name());
} catch (Exception ex) {
logger.error(ex);
}
return extractSingleJobFromSet(jobs);
}
Update 2: Some more code.
AbstractTransactionHelper:
#Service
public abstract class AbstractTransactionHelper {
#Autowired
QdbJobRepository jobRepo;
#Autowired
ArchivedJobRepository archive;
protected Job extractSingleJobFromSet(Set<Job> jobs){
Job job = null;
if(jobs != null && !jobs.isEmpty()){
for(job job : jobs){
if(this instanceof JobManagerTransactionHelper){
updateJob(job);
}
job = job;
}
}
return job;
}
protected void updateJob(Job job){
updateJob(job, Status.PROCESSING, null);
}
protected void updateJob(Job job, Status status, String serviceMessage){
if(job != null){
if(status != null){
job.setStatus(status);
}
if(serviceMessage != null){
job.setServiceMessage(serviceMessage);
}
saveJob(job);
}
}
protected void saveJob(Job job){
jobRepo.save(job);
archive.save(Job.convertJobToArchivedJob(job));
}
Update 4: Threading. newJob() is implemented by each service that uses the library.
#Service
public class JobManager{
#Autowired
private JobManagerTransactionHelper transactionHelper;
#Autowired
JobListener jobListener;
#Autowired
Config config;
protected final AtomicInteger atomicThreadCounter = new AtomicInteger(0);
protected boolean keepPolling;
protected Future<?> futurePoller;
protected ScheduledExecutorService pollService;
protected ThreadPoolExecutor threadPool;
public boolean start(){
if(!keepPolling){
ThreadFactory pollServiceThreadFactory = new ThreadFactoryBuilder()
.setNamePrefix(config.getService() + "ScheduledPollingPool-Thread").build();
ThreadFactory threadPoolThreadFactory = new ThreadFactoryBuilder()
.setNamePrefix(config.getService() + "ThreadPool-Thread").build();
keepPolling = true;
pollService = Executors.newSingleThreadScheduledExecutor(pollServiceThreadFactory);
threadPool = (ThreadPoolExecutor)Executors.newFixedThreadPool(getConfig().getThreadPoolSize(), threadPoolThreadFactory);
futurePoller = pollService.scheduleWithFixedDelay(getPollTask(), 0, getConfig().getPollingFrequency(), TimeUnit.MILLISECONDS);
return true;
}else{
return false;
}
}
protected Runnable getPollTask() {
return new Runnable(){
public void run(){
try{
while(atomicThreadCounter.get() < threadPool.getMaximumPoolSize() &&
threadPool.getActiveCount() < threadPool.getMaximumPoolSize() &&
keepPolling == true){
Job job = transactionHelper.getNextJobToProcess();
if(job != null){
threadPool.submit(getJobHandler(job));
atomicThreadCounter.incrementAndGet();//threadPool.getActiveCount() isn't updated fast enough the first loop
}else{
break;
}
}
}catch(Exception e){
logger.error(e);
}
}
};
}
protected Runnable getJobHandler(final Job job){
return new Runnable(){
public void run(){
try{
atomicThreadCounter.decrementAndGet();
jobListener.newJob(job);
}catch(Exception e){
logger.error(e);
}
}
};
}
As it turns out, the problem was the WeblogicJtaTransactionManager. My guess is that the FOR UPDATE resulted in a JPA transaction, but upon updating the object in the database, the WeblogicJtaTransactionManager was used, which failed to find an ongoing JTA transaction. Since we're deploying on Weblogic we wrongly assumed we had to use the WeblogicJtaTransactionManager.
Either way, exchanging the TransactionManager to a JpaTransactionManager (and explicitly setting the EntityManagerFactory and DataSource on it) basically solved all problems.
#Bean
public PlatformTransactionManager transactionManager() {
JpaTransactionManager jpaTransactionManager = new JpaTransactionManager(entityManagerFactory().getObject());
jpaTransactionManager.setDataSource(dataSource());
jpaTransactionManager.setJpaDialect(new HibernateJpaDialect());
return jpaTransactionManager;
}
Assuming you also have added an EntityManagerFactoryBean which is needed if you want to use multiple datasources in the same project (which we're doing, but not within single transactions, so no need for JTA).
#Bean
public LocalContainerEntityManagerFactoryBean entityManagerFactory() {
HibernateJpaVendorAdapter vendorAdapter = new HibernateJpaVendorAdapter();
LocalContainerEntityManagerFactoryBean factoryBean = new LocalContainerEntityManagerFactoryBean();
factoryBean.setDataSource(dataSource());
factoryBean.setJpaVendorAdapter(vendorAdapter);
factoryBean.setPackagesToScan("my.model");
return factoryBean;
}