In Java in a web service, I have a requirement I want to return the response to the user after configured threshold time reaches and wants to continue processing after that.
Let's say I have a service it does step1, step 2, and the configured threshold is 1 second. Let's say step1 is completed at 1 second I want to return an acknowledgment response to the user and continue processing with step2 and wants to store response in DB or something like that.
Please let me know if anyone has any solutions or thoughts on this problem
There are multiple ways to achieve this
HTTP Layer
On HTTP layer, if the response comes back before the threshold, then I'd be tempted to send back a 200 Success.
However, if it takes more time than the threshold, you could use 202 Accepted
Looking at the RFC, its use case looks like this
6.3.3. 202 Accepted
The 202 (Accepted) status code indicates that the request has been
accepted for processing, but the processing has not been completed.
The request might or might not eventually be acted upon, as it might
be disallowed when processing actually takes place. There is no
facility in HTTP for re-sending a status code from an asynchronous
operation.
The 202 response is intentionally noncommittal. Its purpose is to
allow a server to accept a request for some other process (perhaps a
batch-oriented process that is only run once per day) without
requiring that the user agent's connection to the server persist
until the process is completed. The representation sent with this
response ought to describe the request's current status and point to
(or embed) a status monitor that can provide the user with an
estimate of when the request will be fulfilled.
Now, of course, instead of having a mix of 200 and 202, you could just return 202 everytime
Application Layer
In your application layer, you'll typically want to make use of asynchronous processing for this purpose.
There are multiple ways to leverage this way of working, you can:
Post a message on a queue/topic and let a message broker take care of dispatching it to another part of the app, or another app and let this part do the processing
Save the request inside of a database, and have another service poll the database for new requests, similar to queueing explained above, without JMS
If you're using Java EE, your EJB container allows you to work with #Asynchronous which will call a method asynchronously and return (so you'll be able to return 202)
If you're using Spring, it has an #Async annotation for the same purpose as hereabove
There are definitely other methods you could use to achieve this use case, but I think the ones I presented are the most common ones
let's say I have a REST endpoint that does this:
Receives a json body, do some mapping and then send the message via a messaging producer. (RabbitMQ)
The producer is async.
I have a consumer for the producer in 2 that will do some business logic and post a reply.
Now, I need to receive a reply after some interactions in my rest endpoint.
As the client of my rest call is expecting a reply, the solution that comes to mind is to have the endpoint listening on reply queue with a short timeout, so that I can return the response via REST.
Am I thinking the right way or should I just have a blocking producer and use RPC like it is stated here: https://www.rabbitmq.com/tutorials/tutorial-six-java.html
I want to find the most optimal solution.
Note: I'm not using Spring as I'm learning all these concepts to have a clear understanding.
Producing on queue from an HTTP call on server is fine.
Waiting for a queue and and replying for HTTP call on its response is not preferred way.
A reply queue may have many messages on it. What happens if your consume a message meant for different client call? Do you pass the message to appropriate consumer? How do you find out appropriate consumer? You may re-enqueue the message which is wastage of cycles. A message may stay on queue forever.
If request queue is flooded and you may have to poll many times to get your result. If you receive reply before client polls you would need to store reply so that you can pass it to client on next call.
These are just scratching the surface of the problems that may arise. If your client is really an HTTP client, you can use above to cover important bases of your application.
A better solution would be to write a consumer on reply queue which would call some endpoint on consumer. However it would still require you to consider cases like failures and retries both on server and client side.
I didn't answer your question but gave you some points to think about. Please don't mark this as accepted answer.
I know this is an old but if someone is still looking for this. This is mostly related to RPC in rabbitmq. One way could be to set Correlation ID on each produced message (on consumer you receive this Correlation ID and you can match it back), then check if that's the one you are expecting. You set it to unique value (UUID for example) on each request and compare to see if that's the response you are expecting back.
There is also a feature called Direct Reply-to explained nicely: https://www.rabbitmq.com/direct-reply-to.html
The producer consume from "amq.rabbitmq.reply-to" and produce with ReplyTo property set to "amq.rabbitmq.reply-to", while the consumer once received and processed message reply to the messages "ReplyTo" property as routing key and empty exchange.
Hope so this helps someone in future.
I have a Java web application, sitting in a Jetty container. I would like to know what happens if I submit 2 requests to the same URL one immediately after the other. Assume that the requests are simple GET requests and have no side effects.
I imagine what happens for each request is that a HTTP request is made to the URL, Jetty receives it and starts up a new thread to handle the request, then generates the response and sends it back over HTTP.
In the context of a browser - if I have sent off a second request before the first one returns, does the first response simply get discarded and not used? Is it effectively a wasted transaction?
In general, you have no way of knowing that the server isn't providing a different response to each get -- perhaps incrementing a hit counter, as one trivial example -- so unless the server, or your client, is explicitly set up to cache results each request gets processed independently.
What is the best technology solution (framework/approach) to have a Request Queue in front of a REST service.
so that i can increase the no of instances of REST service for higher availability and by placing Request queue in front to form a service/transaction boundary for the service client.
I need good and lightweight technology/framework choice for Request Queue (java)
Approach to implement a competing consumer with it.
There's a couple of issues here, depending on your goals.
First, it only promotes availability of the resources on the back end. Consider if you have 5 servers handling queue requests on the back end. If one of those servers goes down, then the queued request should fall back in to the queue, and be redelivered to one of the remaining 4 servers.
However, while those back end servers are processing, the front end servers are holding on to the actual, initiating requests. If one of those front end servers fails, then those connections are lost completely, and it will be up to the original client to resubmit the request.
The premise perhaps is that simpler front end systems are at a lower risk for failure, and that's certainly true for software related failure. But networks cards, power supplies, hard drives, etc. are pretty agnostic to such false hopes of man and punish all equally. So, consider this when talking about overall availability.
As to design, the back end is a simple process waiting upon a JMS message queue, and processing each message as they come. There are a multitude of examples of this available, and any JMS server will suit at a high level. All you need is to ensure that the message handling is transactional so that if a message processing fails, the message remains in the queue and can be redelivered to another message handler.
Your JMS queue's primary requirement is being clusterable. The JMS server itself is a single point of failure in the system. Lost the JMS server, and your system is pretty much dead in the water, so you'll need to be able to cluster the server and have the consumers and producers handle failover appropriately. Again, this is JMS server specific, most do it, but it's pretty routine in the JMS world.
The front end is where things get a little trickier, since the front end servers are the bridge from the synchronous world of the REST request to the asynchronous world of the back end processors. A REST request follows a typically RPC pattern of consuming the request payload from the socket, holding the connection open, processing the results, and delivering the results back down the originating socket.
To manifest this hand off, you should take a look at the Asynchronous Servlet handling the Servlet 3.0 introduced, and is available in Tomcat 7, the latest Jetty (not sure what version), Glassfish 3.x, and others.
In this case what you would do is when the request arrives, you convert the nominally synchronous Servlet call in to an Asynchronous call using HttpServletRequest.startAsync(HttpServletRequest request, HttpServletResponse response).
This returns an AsynchronousContext, and once started, allows the server to free up the processing thread. You then do several things.
Extract the parameters from the request.
Create a unique ID for the request.
Create a new back end request payload from your parameters.
Associate the ID with the AsyncContext, and retain the context (such as putting it in to a application wide Map).
Submit the back end request to the JMS queue.
At this point, the initial processing is done, and you simply return from doGet (or service, or whatever). Since you have not called AsyncContext.complete(), the server will not close out the connection to the server. Since you have the AsyncContext store in the map by the ID, it's handy for safe keeping for the time being.
Now, when you submitted the request to the JMS queue, it contained: the ID of the request (that you generated), any parameters for the request, and the identification of the actual server making the request. This last bit is important as the results of the processing needs to return to its origin. The origin is identified by the request ID and the server ID.
When your front end server started up, it also started a thread who's job it is to listen to a JMS response queue. When it sets up its JMS connection, it can set up a filter such as "Give me only messages for a ServerID of ABC123". Or, you could create a unique queue for each front end server and the back end server uses the server ID to determine the queue to return the reply to.
When the back end processors consume the message, they're take the request ID, and parameters, perform the work, and then take the result and put them on to the JMS response Queue. When it puts it the result back, it'll add the originating ServerID and the original Request ID as properties of the message.
So, if you got the request originally for Front End Server ABC123, the back end processor will address the results back to that server. Then, that listener thread will be notified when it gets a message. The listener threads task is to take that message and put it on to an internal queue within the front end server.
This internal queue is backed by a thread pool who's job is to send the request payloads back to the original connection. It does this by extracting the original request ID from the message, looking up the AsyncContext from that internal map discussed earlier, and then sending results down to the HttpServletResponse associated with the AsyncContext. At the end, it call AsyncContext.complete() (or a similar method) to tell the server that you're done and to allow it to release the connection.
For housekeeping, you should have another thread on the front end server who's job it is to detect when requests have been waiting in the map for too long. Part of the original message should have been a time the request started. This thread can wake up every second, scan the map for requests, and for any that have been there too long (say 30 seconds), it can put the request on to another internal queue, consumed by a collection of handlers designed to inform the client that the request timed out.
You want these internal queues so that the main processing logic isn't stuck waiting on the client to consume the data. It could be a slow connection or something, so you don't want to block all of the other pending requests to handle them one by one.
Finally, you'll need to account that you may well get a message from the response queue for a request that no longer exists in your internal map. For one, the request may have timed out, so it should not be there any longer. For another, that front end server may have stopped and been restarted, so it internal map of pending request will simply be empty. At this point, if you detect you have a reply for a request that no longer exists, you should simply discard it (well, log it, then discard it).
You can't reuse these requests, there's not such thing really as a load balancer going back to the client. If the client is allowing you to make callbacks via published end points, then, sure you can just have another JMS message handler make those requests. But that's not a REST kind of thing, REST at this level of discussion is more client/server/RPC.
As to which framework support Asynchronous Servlets at a higher level than a raw Servlet, (such as Jersey for JAX-RS or something like that), I can't say. I don't know what frameworks are supporting it at that level. Seems like this is a feature of Jersey 2.0, which is not out yet. There well may be others, you'll have to look around. Also, don't fixate on Servlet 3.0. Servlet 3.0 is simply a standardization of techniques used in individual containers for some time (Jetty notably), so you may want to look at container specific options outside of just Servlet 3.0.
But the concepts are the same. The big takeaway are the response queue listener with the filtered JMS connection, the internal request map to the AsyncContext, and the internal queues and thread pools to do the actual work within the application.
If you relax your requirement that it must be in Java, you could consider HAProxy. It's very lightweight, very standard, and does a lot of good things (request pooling / keepalives / queueing) well.
Think twice before you implement request queueing, though. Unless your traffic is extremely bursty it will do nothing but hurt your system's performance under load.
Assume that your system can handle 100 requests per second. Your HTTP server has a bounded worker thread pool. The only way a request pool can help is if you are receiving more than 100 requests per second. After your worker thread pool is full, requests start to pile up in your load balancer pool. Since they are arriving faster than you can handle them, the queue gets bigger ... and bigger ... and bigger. Eventually either this pool fills too, or you run out of RAM and the load balancer (and thus the entire system) crashes hard.
If your web server is too busy, start rejecting requests and get some additional capacity online.
Request pooling certainly can help if you can get additional capacity in time to handle the requests. It can also hurt you really badly. Think through the consequences before turning on a secondary request pool in front of your HTTP server's worker thread pool.
The design we use is a a REST interface receiving all the request and dispatching them to a message queue (i.e. Rabbitmq)
Then workers listen to the messages and execute them following certain rules. If everything goes down you would still have the request in the MQ and if you have a high number of request you can just add workers...
Check this keynote, it kind of shows the power of this concept!
http://www.springsource.org/SpringOne2GX2012
I want to make an AJAX call to my Java webapp. The Java webapp will in turn make an asynchronous return call elsewhere. The result of that call will then be returned as the result of AJAX request.
The crux of my question is what would I do with the HttpRequest whilst I'm waiting for the second call to return?
Do I just block and wait for the call within the AJAX handler method or do I store the request somewhere and wait for a callback? How would I handle errors / timeouts?
For those who care further information as to how I arrived at this situation follows:
This is part of an XMPP based instant messaging system. There is one global support user which is displayed as an icon on every page in our webapp. I also want to display the presence of this user, so, I could just use the IM system to request this users presence on every single page load for every user and eventually DDOS myself. Instead I want to have a single user query the presence from the webapp periodically and cache the result.
The AJAX call is therefore to the server which will then either return the cached presence or query the XMPP server asynchronously.
You shouldn't have to block and wait for the AJAX call. That is, don't make the call synchronously. What you should do on the Java side is figure out a way to block while you wait for the response to come back from your asynchronous call (i.e., figure out to a way to make the request synchronously. The performance hit will be on the first call for any new data. Subsequent calls will hit the cache, so you should be good). You can maintain a cache for this data, so you can check the cache first to see if the data exists. If it doesn't make the call and store the result in the cache. Otherwise, grab the data from the cache and send it back to the view. Since AJAX is asynchronous, your callback will be called as soon as the data comes back from the server.
here is what i would do:
when the page startup, init an job to retrieve data array you need for that specific page, you need to identify the job and the job result for later usage
use ajax from the page to poll for the job result, once the job is done, the poll finishes and returned with data
cache the entries you have requested as Vivin indicated
cache the job result on your server and give it a time-out option
HTTP requests, i.e. HttpServletRequest objects are not serializable. Therefore you cannot store them in a persistent store of any sort, for the duration of the call. It doesn't make sense anyway to store the request, for its life is limited to the duration of the HTTP request itself, given the stateless nature of the HTTP protocol.
This effectively means that you have to hold on to the HttpServletResponse object for the duration of the call. The HttpServletRequest object is no longer needed, once the parsing of the HTTP request is performed, and once all the data is available to your application; it is the response object that is of importance in your context.
The response could be populated with the cached copy of the user status. If the copy in the cache is stale, you might want to refresh it synchronously from the XMPP server (after all, it affects the performance of just one page load). You could query asynchronously from within the application server, but some result must be returned to the browser (so there might be a few edges cases that need to be taken care of).