My project connects to a database using hibernate, getting connections from a connection pool on JBoss. I want to replace some of the reads/writes to tables with publish/consume from queues. I built a working example that uses OracleAQ, however, I am connecting to the DB using:
AQjmsFactory.getQueueConnectionFactory followed by createQueueConnection,
then using createQueueSession to get a (JMS) QueueSession on which I can call createProducer and createConsumer.
So I know how to do what I want using a jms.QueueSession. But using hibernate, I get a hibernate.session, which doesn't have those methods.
I don't want to open a new connection every time I perform an action on a queue - which is what I am doing now in my working example. Is there a way to perform queue operations from a hibernate.session? Only with SQL queries?
I think you're confusing a JMS (message queue) session with a Hibernate (database) session. The Hibernate framework doesn't have any overlap with JMS, so it can't be used to do both things.
You'll need 2 different sessions for this to work:
A Hibernate Session (org.hibernate.Session) for DB work
A JMS Session (javax.jms.Session) to to JMS/queue work
Depending on your use case, you may also want an XA transaction manager to do a proper two-phase commit across both sessions and maintain transactional integrity.
I was also looking for some "sane" way how to use JMS connection to manipulate database data. There is not any. Dean is right, you have to use two different connections to the same data and have distributed XA transaction between them.
This solution opens a world of various problems never seen before. In real life distributed transactions can really be non-trivial. Surprisingly in some situations Oracle can detect that two connections are pointing into the same database and then two-phase commit can be bypassed - even when using XA.
Related
I'm trying to understand the use of Java XA Datasource.
But I can't still figure when to use it, and when not to use it.
I read that XA Datasource used when we use two databases.
But I'm not sure what is the meaning of two database.
For example:
I had two layer of classes (Service and DAO)
A method in service layer annotated as a transaction, invoke two methods in DAO.
Each method in DAO open new connection to database and close it in the end of the method.
If I use one instance of database, and each method in DAO write to different table, do I have to use XA Datasource? since transaction occured in service layer but only in one instance database
Systems such as databases, but also for example queueing systems that you use through JMS have the concept of transactions. A transaction is treated as a unit of work; at the end of doing the work, such as inserting, updating or deleting records in the database, you commit the transaction and then the database definitively does the work; or you rollback the transaction and then everything done in the transaction is cancelled.
In some cases, your software has to perform operations over multiple different systems. For example, you might need to insert data into multiple databases, or insert something in the database and put a message on a queue.
If you want to do such combinations of operations as if they are in one transaction, then you need a distributed transaction system - a system that can combine the transactions of the different systems into one. That way you can write your code as if it's running inside a single transaction; the distributed transaction system automatically commits or rolls back the transactions in the underlying systems.
To make it more concrete: Suppose that you insert a record in a database, and put a message on a queue, and you want to do this inside one transaction. When something goes wrong with putting the message on the queue, you also want the database transaction to be rolled back, so that you don't have a record in the database but not a corresponding message on the queue. Instead of manually keeping track of the transactions of the database and the queue system (including handling all combinations of possible errors), you can use a distributed transaction system.
XA is a standard for working with distributed transactions. You can work with XA transactions in Java through the Java Transaction API (JTA). Java EE servers have support for this built-in. If you're not using a Java EE server, then you can use a separate library that implements JTA such as Narayana or Atomikos.
Each method in DAO open new connection to database and close it in the end of the method.
Normally this isn't how you should write DAOs. Opening a database connection is a relatively slow operation; if you open a new database connection for every method that you call in a DAO, your program is most like going to run slowly. You should at least use a connection pool, that manages a number of database connections and allows you to reuse already open connections.
If I use one instance of database, and each method in DAO write to different table, do I have to use XA Datasource? since transaction occured in service layer but only in one instance database
If your DAO methods each open their own connection, then they will run in separate transactions. Whether this is a problem or not depends on what your application needs to do. An XA datasource is not the solution to make them run in one transaction. Instead, you should both let them use the same connection and transaction.
XA transactions are really only useful if you are using multiple database systems or other systems, and you want to be able to perform transactions that span across these systems.
In my Spring Boot application I have a following scenario:
Read message from JMS queue
Do some things with the data.
Do a JDBC insert to Oracle database
If at any stage an exception occurs, I would like to be sure that the message isn't lost and is placed in the Backout Queue.
Do I need an XA transaction between XA-enabled JMS connection factory and XA-enabled Oracle db driver? Or is it sufficient to use for example DefaultMessageListenerContainer with sessionTransacted flag set to true?
And if the second one is the right answer, what would be a good situation to use XA transaction?
You can avoid XA by using Spring to synchronize the two transactions and coding your app to handle the (small) possibility of a duplicate delivery (when the DB commits but the JMS rolls back due to, say, a lost connection between the DB and JMS commit). I suggest you read the excellent Distributed transactions in Spring, with and without XA by Dave Syer from the Spring team.
If javaworld moves that link sometime in future, google syer xa spring and you should find it.
I am working on a stand-alone application that uses both JMS and Hibernate.
The documentation suggests JTA has to be used if I want to have transactions across both resources.
However, right now with a #Transaction annotated DAO method (and HibernateTransactionManager), this already seems to work. When I call send() on the JmsTemplate, the message is not immediately sent, but rather the JMS session is committed with the Hibernate session as the method returns.
I didn't know how this is possible without the JtaTransactionManager, so I checked the source code. It turns out both the wrapper for Hibernate and JmsTemplate registers the sessions with TransactionSynchronizationManager and the JMS session will be committed when the Hibernate session commits.
What's the different between this and a JTA transaction. Can I use this to replace the latter??
In short no, you can't get support for 2-phase commit without a JTATransactionManager and XA aware datasources.
What you are witnessing is a co-ordination of two Local Transactions supporting 1-phase commit only. Roughly performing this sequence of events...
Start JMS Transaction
Read JMS message
Start JDBC Transaction
Write to database
Commit JDBC Transaction
Commit/Acknowledge JMS
The JMS transaction will be started first wrapping the nested JDBC transaction, so that the JMS queue will rollback if the Hibernate/JDBC commit fails. Your JMS Listener Container should be setup not to acknowledge="auto" and instead wait for the Hibernate transaction to complete before sending the acknowledgement.
If you only have these two resources then the issue you will have to consider is when Hibernate succeeds in persiting then you get an Exception before you can acknowledge the JMS server. Not a big issue as the JMS message is not lost and you will read it again.
However
You must write your MessageListener to handle duplicate messages from the server
You must also handle a message that cannot be processed due to bad data and ending up in an infinite loop of trying to comsume it. In this case the server may be configured to move the message to a "dead message queue", or you deal with this yourself in the MessageListener
Other options and further reading
If your JMS server does not support XA (global) transactions this is pretty much your only solution.
If JMS server does support XA transactions but JDBC doesn't then you can use a JTATransactionManager and use the LastResourceCommitOptimisation. There are open source JTATransactionManagers you can use like JOTM
This JavaWorld article goes into more detail on your problem space.
Although this has been answered in detail by Brad, i would like to address a very specific part of your query:-
I didn't know how this is possible without the JtaTransactionManager
From the spring documentation:-
When a JTA environment is detected, Spring’s JtaTransactionManager will be used to manage transactions
https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-jta.html
I wan to know how the transaction is internally implemented in EJB. I want to know the logic they use to create a transaction. if you could point out some articles that would be helpful
Hibernate doesn't implement transactions, it relies on and wraps JDBC transactions or JTA transactions (either container managed or application managed).
Regarding EJBs, if you want to understand the details of a JTA Transaction Manager, you'll need to be fluent with the JTA interfaces UserTransaction, TransactionManager, and XAResource which are described in the JTA specification. The JDBC API Tutorial and Reference, Third Edition will also be useful to understand the XA part of a JDBC driver.
Then, get the sources of an EJB container (like JBoss) or of a standalone JTA Transaction Manager (like Atomikos) to analyze the TM part. And good luck.
This question could have answers at many levels.
A general discussion of what's going on can be found here
My summary goes like this ... First, somewhere there must be a transaction coordinator, the EJB container will know about the coordinator - typically that's part of the application server. So all the EJB container has to do is to call
someobject.BeginTransaction()
that's it. The actual API the EJB container uses is JTA. EJBs can actually use Bean Managed transaction transaction or Container managed transactions. In the Bean Managed case the implementer nhas to make the JTA calls. More usually we use Container Managed transactions (CMT). In which case the container has logic which is run before the implementation is reached. For example:
if ( we're not already in a transaction )
begin transaction
call the EJB implementation
and later the container has logic
if ( finished processing request )
commit transaction
with other paths to abort the transaction if errors have happened.
Now that logic is more complex because CMT EJBs are annotated with transaction control statements. For example you can say things "if we already have a transaction, use it" So if one EJB calls another only a single transaction is used. Read up the EJB spec for that.
However all that's pretty obvious in any write-up of Java EE EJBs. So I suspect that you're asking moe about what happens inside the JTA calls, how the transaction manager is implemented and its relationship to the transactional resource managers (eg. Databases). That's a huge topic. You've actually go implementations of the XA distributed transaction protocol down there. Frankly I doubt that you really need to need to know this. At some point you have trust the APIs you're using. However there is one key detail: your Transaction Manager (typically the App Server itself) must be able to tell the REsource Managers the fate of any given transaction, and that information must survive restart of the App Server, hence some persistent store of transaction information must be kept. You will find transaction logs somewhere, and in setting up the App Server you need to make sure those logs are well looked after.
From EJB in Action book
The protocol commonly used to achieve multiple resource is the two-phase commit. The two-phase commit protocol performs an additional preparatory step before the final commit. Each resource manager involved is asked if the current transaction can be successfully committed. If any of the resource managers indicate that the transaction cannot be committed if attempted, the entire transaction is abandoned (rolled back). Otherwise, the transaction is allowed to proceed and all resource managers are asked to commit.
A resource manager can be a database, for instance. Others examples includes a Message Service. The component which coordinates transactions is called Transaction manager.
Suppose you have an application which involves two distincts databases. How does Transaction manager performs its work by using Two phase commit protocol ?
Transaction Manager ask database 1 if it can commit the current transaction
If so, ask database 2 if it can commit the current transaction
Transaction Manager ask database 1 to commit
Transaction Manager ask database 2 to commit
Hibernate is built on top of the JDBC API. It just coordinates one database. So if you call
session.commit();
Behind the scenes, it call
connection.commit();
If you really want to study Transaction internals, my advice is Java Transaction Processing book.
Hibernate has TransactionFactory:
An abstract factory for Transaction instances. Concrete implementations are specified by hibernate.transaction.factory_class.
It has implementations: JDBCTransactionFactory, JTATransactionFactory, CMTTransactionFactory. These factories create an instance of Transaction - for example JDBCTransaction.
Then I can't tell you what happens for JTA and CMT, but for JDBC it's as simple as setting the auto-commit to false (when you call begin a transaction):
connection.setAutoCommit(false);
And respectively on transaction.commit(): connection.commit()
If any exception occurs when operating with the session, it invokes connection.rollback()
Another good read would be the JTS articles by Brian Goetz; links:
http://www.ibm.com/developerworks/java/library/j-jtp0305.html
http://www.ibm.com/developerworks/java/library/j-jtp0410/index.html
http://www.ibm.com/developerworks/java/library/j-jtp0514.html
I have two level access to database: the first with Hibernate, the second with JDBC. The JDBC level work with nontransactional tables (I use MyISAM for speed). I want make both levels works within transaction. I read about JTA which can manage distributed transactions. But there is lack information in the internet about how to implement and use custom resource.
Does any one have experience with using custom XAResources?
I want make both levels works within transaction.
Then you'll have to change your storage engine for InnoDB, MyISAM tables do not support transactions (technically, you won't get an error but a rollback won't rollback anything).
Does any one have experience with using custom XAResources?
I'm not sure to understand what you're talking about. The only XA resource I see here is your database and you don't need to implement anything custom. What you need to do is to use XA connections very likely obtained from two XA DataSources (which are supported by MySQL Connector/J 5.0.0+), use the JTA API and let the Transaction Manager do its job .
But to be honest, you should really clarify your requirements. There might be other (and easier) options than using XA. And if all the above sounds like Chinese, then I have the feeling that don't use XA would be a good advice here.
Connection are obtained via a DataSource that can be configured to support distributed transaction or not. To use multiple connections in a distributed transaction, you must configure the multiple DataSource to support XA and return XA connections.
That said, you need several physical connections only if you connects to different database, which doesn't seem to be your case (that's not clear in the question).
A DataSource can be smart enough to make sure that the same physical connection is used as long as you are in the same thread; each time you ask for a connection, it actually returns a "handle" to the same physical connection, and the physical connection returns to the pool when all handles have been closed. (But that depends on the DataSource implementation).
Hibernate per se is not an XA resource: it uses underlying a connection obtained via a DataSource. But it hooks itself in the transaction manager via JTA, in particular to flush all pending changes before the distributed transaction commits.
You can most of the time obtain the underlying connection used by the EntityManager using implementation specific API (it's at least possible with Hibernate). This means that you can maybe fulfill your requirement without using JTA and XA at all: use the underlying connection of the EntityManager for your JDBC stuffs.
In summary:
No need to mess with XAResource
Switch to InnoDB
You can try to switch to a XA DataSource and obtain a connection with DataSource.getConnection()
You can try to switch to a XA DataSource and obtain the underlying EntityManager connection
You can try to stick with a non-XA DataSource and obtain the underlying EntityManager connection
Hope I understood your question right and that it helps.