I need to do transaction management for remote SOAP calls. What is the best possible
strategy in Java?
Using ACID transactions for long-running remote calls might not be a good idea. Mark Little and Bruce Martin discuss the key ideas here, which might provide some good ideas on how to use compensating transactions.
You could use something like JTA or roll your own approach, though this can be somewhat painful, since you have to ensure your webservices can support transactional semantics (e.g. rollback) and there is some risk in that if the calling application controls the transactions (most common) then you run some risk if that application/session dies.
Related
I have a requirement to migrate a legacy CORBA system to any latest java technology. The main problem I am facing is to provide long lived transaction(db) in the proposed system. Currently the client(Swing App) retain the CORBA service object and perform multiple db txn before actually committing/rolling back all the txn. Service layer keep the state of connection object through out to complete transaction.
I wanted to reproduce this mechanism in my new system(REST/WS) so that either Swing client/Web(future) can work in the same as is.
eg:
try {
service1.updateXXData(); // --> insert in to table XX
service2.updateUUData() //--> insert in to table UU
service1.updateZZData(); // --> insert in to table ZZ
service2.updateAAData(); // --> insert in to table AA
service1.commit(); // con.commmit();
service2.commit(); // con.commmit();
}
exception(){
service1.rollback(); // con.rollback();
service2.rollback(); // con.rollback();
}
Now I wanted to migrate CORBA to any modern technolgy, but still I am at large to find a solution for this. ( the concern is client do not want to make any change to service layer or db layer) , they just wanted to remove CORBA.
couple of options available for me are
Migrate CORBA to RMI --> so that changes required to current system are minimal, but transaction management,connection pooling, retaining state need to do my self.
Migrate CORBA to Stateful EJB --> Compare RMI more changes required, but better since I can use container managed connection pooling, maintain state in a better way.
Migrate CORBA to Stateful Webservice(SOAP) --> More futuristic, but lot of changes required - How ever I can convert IDL to WSDL, and delegate the call to implementation layer
Migrate CORBA to REST --> Most desired if possible - but the amount of time required to migrate is huge , Code changes would require from UI layer to service layer.
Thank you very much in advance
The order in which I would choose the options, from best to worst, would be 4, 3, 2, and 1, however I'd avoid stateful beans or services if humanly possible to do so.
I'll go over the implementation details of what you'll have to do in detail.
For any of these solutions, you'll have to use XA-compliant data sources and transactions so you can guarantee ACID compliance, preferably from an application server so you don't have to generate the transaction yourself. This should be an improvement from your existing application as it almost certainly can't guarantee that, but be advised that in my experience, people put loads of hacks in to essentially reinvent JTA, so watch out for that.
For 4, you'll want to use container-managed transactions with XA. You might do this by injecting a #PersistenceContext backed by a JTA connection. Yes, this costs a ton of time, testing, and effort, but it has two bonuses: First, moving to the web will be a lot easier, and it sounds like that time is coming. Second, those that come after you are more likely to be well-versed in newer web service technologies than bare CORBA and RMI.
For 3, you'll also want to use container-managed transactions with XA. SOAP would not be my first choice as it uses very verbose messages and REST is more popular, but it could be done. If it's stateful, though, you'll have to use bean-managed transactions instead and then hang on to resources across web service calls. This is dangerous, as it could potentially deadlock the whole system.
For 2, you can go two ways, either using container-managed transactions with XA by using a stateless session facade for a stateful EJB. You can use a client JAR for your EJB and package that with the Swing app. Using the stateless facade is preferable, as it will reduce the load on your application server. Keep in mind that you can generate web services from stateless EJB beans too, essentially turning this into #3.
For 1... well, good luck. It is possible to use RMI to interface with EJB's, and generate your own stub and tie, though this is not recommended, and for very good reason. This hasn't been a popular practice for years, may require the stubs and ties to be regenerated periodically, and may require an understanding of the low-level functions of the app server. Even here, you'll want XA transactions. You don't want to handle the transaction management yourself, if possible.
Ultimately, as I'm sure everyone will agree, the choice is yours on what to do, and there's no "right" or "wrong" way, despite the opinions stated above. If it were me (and it's not), I'd ask two important questions of myself and my customer:
Is this for a contract or temporary engagement, and if so what is the term? Do I get first pick at another contract for this same system later when they want additional updates? (In other words, how much money am I going to get out of this vs. how much time am I spending? If it's going to be a long term, then I would go with 4 or 3, otherwise 3 or 2 would be better.)
Why get rid of CORBA? "Because it's old" is an honest answer, but what's the impetus of getting rid of the "old hotness?" Do they plan on expanding usage of this system in the future? Is there some license about to expire and they just want to keep the lights on? Is it because they don't want to dump this on some younger programmer who might not know how to deal with low-level stuff like this? What do you want the system to do in two years, five years, or longer?
(OK, so that's more than two questions :D)
I have a web application which receives requests to save orders in a database. I want to write to 2 different databases - one Cassandra instance and one PostgreSQL instance. I am using plain Java and JDBC (with apache DBUtis) with a lightweight web application library at the front.
What I am unsure about is how to implement transactionality across the two databases, i.e. if a write to one of the databases fails, then rollback the other write and put an error message in the error log.
Are there any mechanisms in Java to implement this? I know of such a thing as two phase commit, is that what I would be looking for here? Are there any alternatives?
Both Cassandra & PostgreSQL support linearizability and compare-and-set (CAS), so you can implement transactions on the client side.
If you want Serializable Isolation level then you should take a look on the Percolator's transactions. The Percolator's transactions are quite known in the industry and have been used in the Amazon's DynamoDB transaction library, in the CockroachDB database and in the Google's Pecolator system itself. A step-by-step visualization of the Percolator's transactions may help you to understand it.
If you expect contention and can deal with Read Committed isolation level then RAMP transactions by Peter Bailis may suit you. I also created a step-by-step RAMP visualization.
The third approach is to use compensating transactions also known as the saga pattern. It was described in the late 80s in the Sagas paper but became more actual with the raise of distributed systems. Please see the Applying the Saga Pattern talk for inspiration.
If we have a web application which has
heavy UI (Spring MVC + JQuery with JSON)
Hibernate with JPA annotations being the domain model
extend Spring-provided DAO to code DAO layer
JBOSS being the app server with Oracle as backend
Datasource (JNDI) based connection pooling (Not an XA rather Local data source)
also has access to multiple data sources (dealing with multiple DB)
Behaviorally, lot of Data retrieval (70%) and update of data being 30%
What would be the best practices for the following to effectively consume DB connections and also see to that there is no much leakage at connection usage?
would it be better to opt for Hibernate template based DAOs?
What kind of transaction manager would be suggest-able and should we go for AOP-based transaction managementWhere
where to instantiate session and and where to close the sessions to effectively consume connections from connection pooling.
It is true that we need to handle transactions from Service layer but what happens to sessions would they be waiting for longer time (we are not using any opensessioninviewFilter)
which layer is better to handle the checked exceptions (business exceptions) and runtime exceptions.
Sorry for this being bit lengthier question, however I see that this is being a common query and I tried consolidating it. Appreciate your patience and guidance. Thanks for your help.
This sounds like a pretty typical Spring/Hibernate application, so I would recommend following current best practices, which I recently outlined in another answer. Specifically:
Do not extend Spring DAO support classes or use HibernateTemplate. Use the #Repository annotation combined with component scanning, and directly inject the SessionFactory into your DAO.
Use Spring's HibernateTransactionManager, and definitely use declarative transaction management via #Transactional as your default approach.
Let Spring manage that. It will open sessions just in time for transactions by default, but prefer the open session in view pattern enabled by Spring's OpenSessionInViewFilter.
See #3.
Always handle exceptions where they should be handled--in other words, this is a design decision. Note, however, that the Spring transaction framework by default rolls back on unchecked exceptions, but not checked, to match the behavior of the EJB spec. Make sure to set the proper rollback rules (see previous link) anywhere you use checked exceptions.
Additionally, use a connection pool, obviously. Apache Commons DBCP is a great choice. "Not much leakage in connection usage" isn't enough. You have to have zero connection leakage. Depending on Spring to manage your resources will help ensure this. As for any other performance issues, don't try to optimize prematurely. Wait until you see where your problem areas are, and then figure out the best way to solve each one individually. Since your bottlenecks will most likely be database-related, check out the performance chapter of the Hibernate reference to get an idea what you're up against. It covers the important concepts of caching and fetching strategies.
Use JPA EntityManager directly in your DAOs. Be sure not to mark it as Extended
Prefer <tx:annotation-driven /> and #Transactional - only on the service layer
The transaction manager also opens and closes sessions (if one doesn't exist already in the thread). Here it is good to know that sessions are session-per-request. Each request(=thread) has a separate session instance. But a database connection is created only if one is needed, so even if there is a transaction manager around all methods, needless connections won't be opened.
read-only transactions - use #Transactional(readOnly=true) in cases when there is only data retrieval
caching - utilize hibernate 2nd level cache to put entities in memory (instead of fetching them from the database each time)
avoid OpenSessionInView and lazy collections. This is subjective, but in my opinion all objects that leave the service layer must be initialized. For small collections (for ex. list of roles) you can have eager collections. For bigger collections use HQL queries.
We are in the process of re-thinking our tech stack and below are our choices (We can't live without Spring and Hibernate due to the complexity etc of the app). We are also moving from J2EE 1.4 to Java EE 5.
Technology stack
Java EE 5
JPA 2.0 (I know Java EE 5 only supports
JPA 1.0 but we want to use Hibernate
as the JPA provider)
Hibernate 3.6.0 (We already have
lots of hbm files with custom types
etc. so we doesn't want to migrate
them at this time to JPA. This means
we want both jpa/hbm mappings work
together and hence the Hibernate as
the JPA provider instead of using
the default that comes with App
Server)
Now the problems is that I want to stick with local transactions but other team members want to use JTA. I have been working with J2EE for last 9 years and I've heard time and again people suggesting to stick with local transactions if I doesn't need two phase commits. This is not only for performance reasons but debugging/troubleshooting a local transaction is lot easier than a JTA (even if JTA only does single phase commit when required).
My suggestion is to use spring declarative transaction management + local transactions (HibernateTransactionManager) instead of container JTA
I want to make sure if I am being paranoid or I have a valid point. I'd like to hear what the rest of the Java EE world thinks. Or please point me an appropriate article.
As Duffy already mentioned, JTA is not synonymous with 2 phase commit, which is something done via the XA protocol.
In JBoss AS for example, you can explicitly choose whether you want a given data source to be an xa-datasource or a tx-datasource. In both cases, transactions are managed via JTA.
In some cases you might already have been using JTA without knowing it. If you send a JMS message transactionally, or update a transactional cache in the same transaction where you modify something in a database, the transaction manager automatically switches to XA mode. The datasource representing your DB may not be XA, but in an XA transaction 1 resource is allowed to be non-XA. Updates to this resource then happens via the last resource commit optimization.
Although you should always calculate the risks and test for your self, I do want to warn against unfounded fear. XA seems to be one of those things we as developers have been brought up to fear. There was an interesting discussion on the JBoss forum about this recently: when to use xa-datasource.
The thing is that XA might have been a complex technology with sub-par implementations in the past, but almost a decade and a half since this FUD this might not be the case anymore. What was complex big enterprise stuff in 1995 is your common run of the mill technology in 2011.
Compare this with the fear we were once brought up with for EJB, which is now completely irrelevant anymore, or the fear for virtual machines (obviously not a problem for Java programmers), or when you're really participating in this industry for a long time, the fear for doing something as basic as function calls ;)
JTA doesn't mean two phase commits. I think it's the combination of JTA and XA drivers that makes two phase commits possible.
I'd still recommend using JTA and declarative transactions over embedding transaction logic in code. Transactions are best done in aspect oriented fashion, a la Spring.
UPDATE:
With the additional information you've posted, I agree with your argument. I'd recommend using Spring declarative transactions and the HibernateTransactionManager class.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
Is there a viable alternative to Hibernate? Preferably something that doesn't base itself on JPA.
Our problem is that we are building a complex (as in, many objects refer to each other) stateful RIA system. It seems as Hibernate is designed to be used mainly on one-off applications - JSF and the like.
The problem is mainly that of lazy loading. Since there can be several HTTP requests between the initialization and actually loading lazy collections, a session per transaction is out of the question. A long-lived session (one per application) doesn't work well either, because once a transaction hits a snag and throws an exception, the whole session is invalidated, thus the lazy loaded objects break. Then there's all kinds of stuff that just don't work for us (like implicit data persisting of data from outside an initialized transaction).
My poor explanations aside, the bottom line is that Hibernate does magic we don't like. It seems like TopLink isn't any better, it also being written on top of EJB.
So, a stateless persistence layer (or even bright-enough object-oriented database abstraction layer) is what we would need the most.
Any thoughts, or am I asking for something that doesn't exist?
Edit: I'm sorry for my ambiguous terminology, and thank you all for your corrections and insightful answers. Those who corrected me, you are all correct, I meant JPA, not EJB.
If you're after another JPA provider (Hibernate is one of these) then take a look at EclipseLink. It's far more fully-featured than the JPA 1.0 reference implementation of TopLink Essentials. In fact, EclipseLink will be the JPA 2.0 reference implementation shipped with Glassfish V3 Final.
JPA is good because you can use it both inside and outside a container. I've written Swing clients that use JPA to good effect. It doesn't have the same stigma and XML baggage that EJB 2.0/2.1 came with.
If you're after an even lighter weight solution then look no further than ibatis, which I consider to be my persistence technology of choice for the Java platform. It's lightweight, relies on SQL (it's amazing how much time ORM users spend trying to make their ORM produce good SQL) and does 90-95% of what JPA does (including lazy loading of related entities if you want).
Just to correct a couple of points:
JPA is the peristence layer of EJB, not built on EJB;
Any decent JPA provider has a whole lot of caching going on and it can be hard to figure it all out (this would be a good example of "Why is Simplicity So Complex?"). Unless you're doing something you haven't indicatd, exceptions shouldn't be an issue for your managed objects. Runtime exceptions typically rollback transactions (if you use Spring's transaction management and who doesn't do that?). The provider will maintain cached copies of loaded or persisted objects. This can be problematic if you want to update outside of the entity manager (requiring an explicit cache flush or use of EntityManager.refresh()).
As mentioned, JPA <> EJB, they're not even related. EJB 3 happens to leverage JPA, but that's about it. We have a bunch of stuff using JPA that doesn't even come close to running EJB.
Your problem is not the technology, it's your design.
Or, I should say, your design is not an easy fit on pretty much ANY modern framework.
Specifically, you're trying to keep transactions alive over several HTTP requests.
Naturally, most every common idiom is that each request is in itself one or more transactions, rather than each request being a portion of a larger transaction.
There is also obvious confusion when you used the term "stateless" and "transaction" in the same discussion, as transactions are inherently stateful.
Your big issue is simply managing your transactions manually.
If you transaction is occurring over several HTTP requests, AND those HTTP requests happen to be running "very quicky", right after one another, then you shouldn't really be having any real problem, save that you WILL have to ensure that your HTTP requests are using the same DB connection in order to leverage the Databases transaction facility.
That is, in simple terms, you get a connection to the DB, stuff it in the session, and make sure that for the duration of the transaction, all of your HTTP requests go through not only that same session, but in such a way that the actual Connection is still valid. Specifically, I don't believe there is an off the shelf JDBC connection that will actually survive failover or load balancing from one machine to another.
So, simply, if you want to use DB transactions, you need to ensure that your using the same DB Connection.
Now, if your long running transaction has "user interactions" within it, i.e. you start the DB transaction and wait for the user to "do something", then, quite simply, that design is all wrong. You DO NOT want to do that, as long lived transactions, especially in interactive environments, are just simply Bad. Like "Crossing The Streams" Bad. Don't do it. Batch transactions are different, but interactive long lived transactions are Bad.
You want to keep your interactive transactions as short lived as practical.
Now, if you can NOT ensure you will be able to use the same DB connection for your transaction, then, congratulations, you get to implement your own transactions. That means you get to design your system and data flows as if you have no transactional capability on the back end.
That essentially means that you will need to come up with your own mechanism to "commit" your data.
A good way to do this would be where you build up your data incrementally into a single "transaction" document, then feed that document to a "save" routine that does much of the real work. Like, you could store a row in the database, and flag it as "unsaved". You do that with all of your rows, and finally call a routine that runs through all of the data you just stored, and marks it all as "saved" in a single transaction mini-batch process.
Meanwhile, all of your other SQL "ignores" data that is not "saved". Throw in some time stamps and have a reaper process scavenging (if you really want to bother -- it may well be actually cheaper to just leave dead rows in the DB, depends on volume), these dead "unsaved" rows, as these are "uncomitted" transactions.
It's not as bad as it sounds. If you truly want a stateless environment, which is what it sounds like to me, then you'll need to do something like this.
Mind, in all of this the persistence tech really has nothing to do with it. The problem is how you use your transactions, rather than the tech so much.
I think you should have a look at apache cayenne which is a very good alternative to "big" frameworks. With its decent modeler, the learning curve is shorten by a good documentation.
I've looked at SimpleORM last year, and was very impressed by its lightweight no-magic design. Now there seems to be a version 3, but I don't have any experience with that one.
Ebean ORM (http://www.avaje.org)
It is a simpler more intuitive ORM to use.
Uses JPA Annotations for Mapping (#Entity, #OneToMany etc)
Sessionless API - No Hibernate Session or JPA Entity Manager
Lazy loading just works
Partial Object support for greater performance
Automatic Query tuning via "Autofetch"
Spring Integration
Large Query Support
Great support for Batch processing
Background fetching
DDL Generation
You can use raw SQL if you like (as good as Ibatis)
LGPL licence
Rob.
BEA Kodo (formerlly Solarmetric Kodo) is another alternative. It supports JPA, JDO, and EJ3. It is highly configurable and can support agressive pre-fetching, detaching/attaching of objects, etc.
Though, from what you've described, Toplink should be able to handle your problems. Mostly, it sounds like you need to be able to attach/detach objects from the persistence layer as requests start and end.
Just for reference, why the OP's design is his biggest problem: spanning transactions across multiple user requests means you can have as many open transactions at a given time as there are users connected to your app - a transaction keeps the connection busy until it is committed/rolled back. With thousand of simultaneously connected users, this can potentially mean thousands of connections. Most databases don't support this.
Neither Hibernate nor Toplink (EclipseLink) is based on EJB, they are both POJO persistancy frameworks (ORM).
I agree with the previous answer: iBatis is a good alternative to ORM frameworks: full control over sql, with a good caching mechanism.
One other option is Torque, I am not saying it is better than any of the options mentioned above but just that it is another option to look at.
It is getting quite old now but may fit some of your requirements.
Torque
When I was myself looking for a replacement to Hibernate I stumbled upon DataNucleus Access Platform, which is an Apache2-licensed ORM. It isn't just ORM as it provides persistence and retrieval of data also in other datasources than RDBMS, like LDAP, DB4O and XML. I don't have any usage experience, but it looks interesting.
Consider breaking your paradigm completely with something like tox. If you need Java classes you could load the XML result into JDOM.