Develop Reference

Restful services reliabilty

I am developing Restful services where we will be inserting/updating new records into database.
Since REST uses HTTP for its communication and HTTP is not reliable, I am worried that, the request may not be sent to the server in case of connection failure.
One of the suggestions I found in the link was "if connection fails just retry again from the client side." But we don't have any control over the client applications.
Other solution was to implement messaging systems like RabbitMQ/JMS to ensure reliability.
I also found in the following link that adding session states improves reliability. I am not able to understand how this happen and more importantly doesn't a good restful service is always stateless?
So to summarize my questions:
To achieve reliability, is Messaging systems best possible approach?
How does session management help me in achieving reliability?

Messaging can help, as long as you don't do any processing when you receive a command to insert or update information, as you need to immediately put the command in a queue. This solution usually adds quite a bit of complexity as you need to notify your client asynchronously when you finish processing the command (was it successful or did it fail?... or did I fail to send the outcome?).
Session management? For reliability? Never heard of that :). Restful services are usually stateless... so no sessions here!
Another option (but depends how your clients integrate with you) is to allow your clients to generate the ids of the items you are going to be storing/updating, in this case, if they get an error back, but you have processed the command successfully, the client can retry, and the same update will happen. You can pair this with versioning to prevent stale updates arriving late.

How to do distributed transaction in java involving jdbc,jms and webservices

I was asked the following question in an interview and couldn't answer that.
How do you include a jdbc operation,a web service call and a JMS operation into one single transaction. That means if one of them fails all has to be roll backed.
I have heard about two-phase commit protocol and oracl XA in case of database transactions involving multiple databases. But not sure whether the same can be used here.

The critical factor is that the web services you connect to have been built using a web services framework that supports transactions. JBoss Narayana is one such web services framework. Once the web services endpoint you are connecting to is on such a framework, it's just a matter of configuring spring to use the appropriate client.
In the case of Narayana, the spring config (from http://bgshinhung.blogspot.ca/2012/10/integrating-spring-framework-jetty-and.html) for transactions with web services:

You are never going to be able to do this in a completely bomb-proof way as the systems are separate. A failure in one stage of the system (for example between the SQL commit and the JMS commit the power on your server gets turned off) will leave the SQL commit in place.
The only way to resolve that would be to keep some record of partial commits somewhere and scan that on startup to fix any resulting problems but now what happens if you have a failure processing or keeping that list.
Essentially the solution is to do your own implementation of the multiple-stage-commit and rollback process wrapping the three operations you need to make. If any of the operations fails then you need to reverse (preferably using an internal transaction mechanism, if not then by issuing reversing commands) any that have been done so far.
There are a lot of corner cases and potential ways for a system like this to fail though, so really the first approach should be to consider whether you can redesign the system so you don't need to do this at all!

It may be trick question and the right answer is "it can not be done".
But I would try to pseodo-code something like this:
try{
jdbc.startTransaction();
Savepoint saveJdbc = jdbc.setSavepoint();
JMS.startTransaction();
Savepoint saveJMS = JMS.setSavepoint();
jdbs.doSomeStuff();
JMS.doSomeStuff();
jdbc.commit();
JMS.commit();
if(webServise.doSomeStuff() == fail){throw new Exception();}
}
catch(Eception e){
jdbc.rollback(saveJdbc);
JMS.rollback(saveJMS);
}
You prepare one servise that has roll back. You prepare second servise that has roll back. You will try web servise and if web servise fail you will roll back those two which have rollback.
May be it is a way to implement rollback to your web servise.

We had same situation like web service will push the data, we have to read the xml stream and persist to db(oracle). implementation we followed is.
Web service send soap message and that will contain xml stream data.
all request soap messages pushed to jms.
respective listner will read the stream and persist the data into 'Temporary tables'.
if request processed successfully then move data from temp table to actual table.
if any error roll back.
hope above points may help.

To my mind, it looks like interviewer liked to understand your ability to think in terms of enterprise wide distribution. Few points:
JDBC is used for Database connectivity
WebService is probably a mechanism to send control command to a
server from any client.
JMS is mainly used for alerts of what is being happened in the
system.
My guess is your interviewer might be having a typical scenario with him that they wish to suffice the following situation:
Data is on one tier ( cluster, or machine )
Clients may be any kind, mobile, app, ios, objective c, browser pay, etc.
JMS is configured to listen to topics. Or is that he wishes he could do that.
Now probably the best approach is to write a JMS Subscriber which decides what to do in the onMessage() method. As an example, suppose a web service is initiated a payment request from client. This will initiate a JMS publisher to tell a DAO do the necessary internal connection to database and when transaction is in middle and when it finishes, one message will be published to subscriber. You will have full grain control of every step as that would be configured to be published through JMS. Though this is difficult to achieve, this could be your interviewer's expected approach from you. (This is Only my guess, and please note.)

How can I get a server to notify the client about changes using the spring Framework?

I'm starting to develop what should become a client-server Application using Hibernate, Spring and Eclipse RCP (for the client). This is the first time I'm designing an application from the beginning so I'm just making my first steps.
I have set up Spring on both client and server using RMI for remoting (but I wouldn't mind using something else if there was a clear advantage). So right now I'm able to call exposed services of the server from different clients to get information from the database. What I haven't done is get any kind of authentication in place, so basically the server just answers to the different clients without knowing anything about them, there is not concept of a session yet. Of course this has to change since I need different user to have different roll and so on, but right now the problem I'm facing is getting the server to notify the client when certain thing happen.
My idea to solve this problem was to have a queue of events at the Server and have the clients get them every 3 second or so. The server would then identify the client by it's session token and send the appropriate events. Yet my partner in this project is concerned that this technique (polling) might waste too much bandwidth unnecessarily.
So to bring it to the point. What are the standard techniques for a server to notify a client about changes using Spring? Please notice that I'm not developing a web application and that this is only intended to be used withing a private network. That is one of the difficulties I've been facing: every single tutorial about Spring security or remoting assumes you are making a web application, but I really don't want to get lost into the details of Spring MVC and web applications in general.
Any resources would be appreciated. A good and long tutorial on the matter would be great.
EDIT: Hmm, it looks like JMS might be what I'm looking for.

As I understand, the issues you are facing is identifying a client in request and correlate different client request i.e. have something like a session.
Spring also support RMI over HTTP protocol (Using Hessian and its own HTTP Invokers). Check out this link (Section 17.3). Now once you have transport as HTTP, it has inherent Basic Authentication and session which can be leveraged to get around the issues you are facing.
This is just a pointer. I would be curious to know how eventually you resolved your problem.

Asynchronous email processing in Java web application

I would like to implement asynchronous email sending in my web application when users register for a new account. This is so that if there is a problem or delay in sending the email message (e.g. the mail server is down or the network connection to the mail server is slow) the user won't be kept waiting for the sending to complete.
My web app is built using Spring and Hibernate's implementation of JPA.
What would be the best and most reliable way for me to implement asynchronous email processing in this web application?
I am thinking about persisting the email information in a database table which is then regularly polled by a Quartz (http://www.opensymphony.com/quartz/) scheduled job for updates and when it finds new unsent emails, it attempts to send them.
Is this a reasonable way of implementing what I want?
Thanks.
Edit:
The most voted on response is to leave the sending of mail as a synchronous call but what's triggered my thinking that an asynchronous approach might be best is that I'm currently using GMail as my outbound mail server (this is for testing while developing) and I'm experiencing a 25 second delay in response from when my app tries to send the email to when the call to the mail send function returns. What do you think?

I would suggest that you don't bother. Most Unix-style MTAs invented and perfected deferred sending decades ago, and you shouldn't be reinventing the wheel. You'll do it poorly (in comparison to sendmail or postfix), and you'll miss something. My best advice is to use the Java Mail APIS javax.mail and let the MTA deal with the asynchronous part.

You can implement queueing by hand, using MySQL or some other persistent mechanism, but you could also use JMS for queueing. It's pretty much a perfect match for just this kind of situations.
In that case I would be tempted by splitting the mail component off from the main app, and let the two communicate using JMS. The main app puts a message in the JMS and the mailer app would subscribe to the queue and try to process the messages.
JMS can be made persistent (to e.g. MySQL) pretty easily by configuration.
The advantage of spliting the webapp is that you abstract away the notification mechanism and could in the future implement e.g. Google Wave or IRC or whatever without having to touch your main app.
Someone else suggested to use a postfix or sendmail and let them handle the queueing. That is also a great solution, especially if you put the postfix or sendmail on localhost and let it route the messages further. Do try to configure that mailer program such that only mail from localhost is allowed to be routed, to prevent creating an open mailer :)
EDIT clarified the use of JMS + comment on local mailer daemon

It is fairly reasonable, this is the kind of thing Quartz was build for.
However, note that you don't need to schedule through the database at all (unless the server downtime is a real issue). You can simply schedule a Quartz job without database access (the simplest Quartz examples show you how).
Otherwise, if you do choose the database access you have the possibility of sending the emails from another application entirely (a nice thing if you need it).

Java EE 6 makes this so easy with the #Asynchronous annotation.

Using JMS or ThreadPool to send email messages

I will like to know:
I have a scenario. If a user adds a product to the system (I'm developing), there's a listener that sends a notification to the user's client base notifying of a new product added by the user.
I've read this thread and (seeing I've never used JMS nor ThreadPool before) I was wondering whether I should use JMS or ThreadPooling.
I am using Tomcat 5.5 and higher and JBoss 5 and higher (depending on company last resort) to deploy my web application.
If I use JMS, do I use Apache ActiveMQ or JBoss Messaging? Are they both compatible to run on both platforms (Tomcat and JBoss)?
Thanks in advance.

For communicating between applications, JMS is a very good solution, especially for events and notifications. JMS allows for such notifications to be sent and received using what is known as asynchronous messaging whereby the sender and receiver have no knowledge of one another and no requirement to be available at the same time.
ActiveMQ is a very widely used message broker that provides client APIs for Java, C/C++, C#, Perl, PHP, Python, Ruby and more. This allows the use of JMS with applications written in Java and other languages.
I have implemented JMS messaging many, many times for a large variety of business situations to handle events and notifications. The vast majority of these times, I have recommended and/or used Spring JMS no matter what message broker is being used. Spring JMS is incredibly easy to use, extremely robust and highly scalable. Spring JMS removes the complexity of creating your own message producers and message consumers, which can save you a tremendous amount of time.
To see how easy it is to send messages using Spring JMS, check out a blog post I wrote recently titled Using the Spring JmsTemplate to Send JMS Messages. I'm also working on a blog post about receiving messages using Spring JMS.
If you have any further questions, let me know.
Bruce

I had a similar requirement once, and we used JMS. Then main problem was how to deal with errors because SMTP is indeed not transactional:
is it ok if some email are lost?
is it ok if some email are sent twice?
We decided it was better to send the message twice, and here is more or less the design we had:
We relied on container managed transaction and if for some reason the email can not be sent, we decided to rollback the JMS transaction; the message would be redelivered later by JMS and an new attempt to send the message was done.
If the JMS message delivery transaction failed after the email was sent (e.g. because of a problem with JMS), the transaction would be rolled back automatically and the message was redelivered later. In this case, the email was sent twice because STMP is not transactional.
Even if the email can be sent (from point of view of code), the SMTP server can still have problem later. In this case, the JMS have been delivered and consumed, so we had no way to know which email had been processed and how to re-send them manually.
But we were already using JMS. I would not introduce JMS just for that given that the main argument is that JMS is transactional, but SMTP isn't anyway.
I would go for something lighter -- possibly with a ThreadPool -- and store the state in a database to know which email need to be sent or has been sent. If there are some problem, you can look at the database and take the ad-hoc decisions.

I know that this reply is very late to this dicsussion, but I hope it will still be valuable for folks seeking info on integrating ActiveMQ and Tomcat.
I've had many people ask me for help with issues they have had integrating ActiveMQ and Tomcat so I decided to write some articles about it. Not only is this topic covered in ActiveMQ in Action (see chapter 8), but I also published a series of articles on it titled ActiveMQ and Tomcat: Perfect Partners. Hopefully people will find this helpful.

I would go for a persistent JMS (I have used only WLS JMS and Websphere MQ so can't compare AQ vs JBoss, whichever offers a better guarantee for delivery). Also, I would seriously consider making the email engine a completely separate application, depending on how much you expect the traffic to grow.