In our spring boot application we have a paging loop over the database which will send a JMS message for every page using a JMSTemplate. The method containing the loop is #Transactional. The JMSTemplate is created with the transacted flag set to true.
I have been browsing through the source code of JMSTemplate and as far as I can see it will not commit a transacted session if there is already a outer transaction going on but will put it into that transaction.
Let's now consider the following code:
#Transactional
public void loopThroughPages(String destination, String from, String to) {
Pageable pageRequest = PageRequest.of(0, maxPageSize);
Page<Event> eventPage;
do {
eventPage = eventRepo.getEventsInTimeRangeForDestination(from, to, destination, pageRequest);
if(eventPage.hasContent()) {
Message<String> eventMessage = buildEventMessage(eventPage.getContent());
JmsTemplate template = queueService.createTemplate(destination);
template.send(eventMessage);
pageRequest = eventPage.nextPageable();
}
} while(pageRequest != null && eventPage.hasNext());
}
The createTemplate creates a DynamicJmsTemplate using a CachingConnectionFactory and setSessionTransacted to true
I am now not entirely sure how this translates to the transactions. My understanding is that all N page messages are sent in the transaction created from the loopThroughPages and once the loopThroughPages method finishes it will commit all N messages and not after each message has been sent. That also implies that the transaction on MQ side will stay open until the last page has been processed. Is this understanding correct?
The key point here is the Transastion managment.
If you use a XA datasource and configure it on your spring boot app you will have a distributed transaction and the commit/rollback for your transaction will be managed by spring since that you have a method annotated with #Transactional otherwise you will have a local transaction managment and the transaction of your database and messaging system will be not syncronized.
For sending message you can by properties configure if the message are persisted or not that it means if your messages will be persisted by your messaging system, while for listener perspective you can configure the acknowledged mode.
By the way the my advice is let to spring managed the transaction and all will be fine but the real point of attention is if you want a distributed transaction between database and jms system you have only configure it atomikos can be an available option, otherwise it is sufficient do not manage the transaction by hand and let to Spring manage it for you.
I hope that it can help you
Related
I have a channel named "creationChannel" which is backed up with MongoMessageStore like this:
#Bean
ChannelMessageStore messageStore() {
return new MongoDbChannelMessageStore(mongoDatabaseFactory);
}
#Bean
PollableChannel creationChannel(ChannelMessageStore messageStore) {
return MessageChannels.queue("creationChannel", messageStore, "create").get();
}
And I want to use it in my flow here, but I want to be sure, that message from there will be read-only if "createOrderHandler" worked fine (the same applies to "updateOrderHandler", but with the different channel).
...some code here...
.<HeadersElement, OperationType>route(
payload -> route(payload),
spec -> spec
.transactional(transactionHandleMessageAdvice)
.subFlowMapping(
OperationType.New,
sf -> sf
.channel("creationChannel")
.log(Level.DEBUG, "Creation of a new order", Message::getPayload)
.transform(Mapper::mapCreate)
.handle(createOrderHandler,
handlerSpec -> handlerSpec.advice(retryOperationsInterceptor))
)
.subFlowMapping(
OperationType.Update,
sf -> sf
.channel("updateChannel")
.log(Level.DEBUG, "Update for existing order", Message::getPayload)
.transform(Mapper::mapUpdate)
.handle(updateOrderHandler,
handlerSpec -> handlerSpec.advice(retryOperationsInterceptor))
)
)
...some code here...
I tried to configure "transactionHandleMessageAdvice" like this:
#Bean
TransactionHandleMessageAdvice transactionHandleMessageAdvice(MongoTransactionManager transactionManager) {
return new TransactionHandleMessageAdvice(transactionManager, new Properties());
}
But messages are still being deleted from the database after the handler fails with an exception.
Maybe I should configure Poller for subflows and configure it with MongoTransactionManager somehow?
Maybe I should configure Poller for subflows and configure it with MongoTransactionManager somehow?
That's correct assumption. As fast as you have a thread shifting in the flow (like yours PollableChannel creationChannel), the current transaction is committed at the moment the message is placed into the store. Nothing more happens in the current thread and, therefore, current transaction which you have started with that .transactional(transactionHandleMessageAdvice).
To make reading transactional, you indeed have to configure a Poller on the .transform(Mapper::mapCreate) endpoint. So, every poll from that queue channel is going to be transactional until you shift to different thread again.
There is just no way (and must not be) to have the whole async flow transactional since transactions are tied to the ThreadLocal and at the moment when call stack comes back to the transaction initiator, it is committed or rolled back. With an async logic we just intend to "send-and-forget" from the producer side and let consumer to deal with data whenever it is ready. That is not what transactions have been designed for.
I have applied spring transaction on service layer of application. There is one method which performing following two operations
1) send message to SQS.
2) And logs that entry in DB.
So, while adding log in DB if any exception occurs then operation (1) will roll backed ? OR Spring will apply transaction on non DB operations ?
Rollback in case of exception is applied to anything that is managed by that transaction. Sending a message to the SQS is not managed by the database transaction, therefore it will not be rolledback.
To achieve this you would need to make a hook into the rollback and do the rollback equivalent of the sending the message to SQS.
I struggeling with JTA, two-phase-commit, JMS- and JDBC-transactions. The idea is (in short) to
receive a message on a queue
perform some database operations
acknowledge the message, when the db operations have been successful
So I got the XAQueueConnectionFactory, create the XAQueueSession, create a receiver from the session and set a message listener.
Inside the listener, in the onMessage method, I begin my user transaction, do the jdbc stuff and commit the transaction or do a rollback, if something went wrong. Now I expected (aka "hoped") that the message would be acknowledged, when the user transaction commits.
But that doesn't happen, the messages are still on the queue and get redelivered again and again.
What am I missing? I double-checked the session and the acknowledge mode really is "SESSION_TRANSACTED" and getTransacted returns true.
I don't have a Java EE container, no spring, no message driven beans. I use the standalone JTA bitronix.
You don't really need XA for this. Just following your algorithm: receive the message, perform the DB operations, then acknowledge the message... Literally, that's the solution. (And instead a transacted session, you probably would just choose explicit CLIENT_ACKNOWLEDGE.) If your application should fail while performing the DB operations, don't ack the JMS msg and it will be redelivered. If your app fails after the DB txn and before the ack, then the message will be redelivered -- but you can detected this (redelivered flag will be set to true on the message), and you can decide to reprocess the message or not, based on the state of the database.
When you say that inside the listener, you begin your user transaction, this seems to hint that you are using Bean Managed Transaction (BMT). Is there a good reason for doing so?
If you used Container Managed Transaction (CMT), what you want would come for free.
As far as I remember, it is not possible with BMT, since the UserTransaction will not participate and will not be able to participate in the transaction created for the message. But you might want to double check with the Java EE spec.
Edit:
Sorry, I realized too late that you are not using a Java EE container.
Are you sure that the user transaction that you start inside the listener is part of the transaction started for the message? It seems that you start an independent transaction for the db work.
If you use no container, who provides the JMS implementation, i.e. XAQueueConnectionFactory etc?
I think with XA you shoulndn't use transacted session.
I have a Java SE(!) scenario with JMS and JPA where I might need distributed transactions as well as "regular" JDBC transactions. I have to listen to a queue which sends service requests, persist log on receiving, process the request and update the log after the request has been processed. The message shall only be acknowledged if the request has been processed successfully.
The first idea was to only use JTA (provided by Bitronix). But there I face two problems:
no log will be persisted if the request can't be processed
the request won't be processed if the log can't be updated (unlikely but yet possible)
So the other idea is to create and update the log with regular JDBC transactions. Only the entitymanager(s) for the request transaction(s) would join the user transactions and the entity managers for creating and updating the log would commit directly.
Is it possible to "mix" JTA and JPA on a single persistence unit? Or do we already have patterns for those kinds of JMS and JDBC transactions?
I actually solved my problem with a slightly different approach. Instead of "mixing" JTA and JDBC transactions I used suspend and resume to work with different user transaction.
The task is still the same: I start a (JTA) user transaction that contains some JMS and JDBC transactions (receiving a message, performing some database operations). And in the middle of that workflow, I want to write a message log but that logging shall not be rolled back when the "outer" transaction fails.
So the solution is, in pseudo code:
transactionManager.begin()
doSomeJdbcStuff();
Transaction main = transactionManager.suspend();
// do the logging
transactionManager.begin() // <- now a new transaction is created and active!
doSomeLogging();
transactionManager.commit()
// continue
transactionManager.resume(main);
doSomeMoreJdbcStuff();
transactionManager.commit();
Does anyone have a good tutorial or some advice on how to implement one's own XAResource? I need Spring's MailSender to be transactional, so that the mail will only be sent once the transaction commits, and it seems there isn't any existing transactional wrapper.
If you just need to wait for the commit, as you say in a comment, you can investigate using TransactionSynchronizationManager.registerSynchronization() to trigger email sending on commit.
You can use a TransactionSynchronizationManager.registerSynchronization (like gpeche mentioned) with a TransactionSynchronizationAdapter which has a variety of methods that are called at various stages of the current transaction. I think the most suitable method for the question is the afterCommit.
TransactionSynchronizationManager.registerSynchronization(new TransactionSynchronizationAdapter() {
#Override
public void afterCommit() {
super.afterCommit();
sendEmail();
}
});
I doubt that it's possible to implement true XAResource for SMTP. There should be transaction support on the resource manager (SMTP server in this case) and I don't believe there are any. I would say your best bet is 'Last resource commit' pattern - which allows one non XA resource participate in XA transaction. Search Google, there are plenty of info. Most Java EE servers supports this.
One other option next to the one mentioned by gpeche, is sending a transactional JMS message from within the transaction. Then let the message listener (like e.g. a MDB bean) send the email.
Another trick in EJB is scheduling a timer from within a transaction. The timer is also transactional and will only be started when the transaction commits. Simply use a timer with timeout = 0, so it will start immediately after the transaction commits.