I'm looking for opinion from you all. I have a web application that need to records data into another web application database. I not prefer to use HTTP request GET on 2nd application because of latency issue. I looking for fast way to save records on 2nd application quickly, I came across the idea of "fire and forget" , will JMS suit for this scenario? from my understanding JMS will guarantee message delivery, guarantee whether message will be 100% deliver is not important as long as can serve as many requests as possible. Let say I need to call at least 1000 random requests per seconds to 2nd application should I use JMS? HTTP request? or XMPP instead?
I think you're misunderstanding networking in general. There's positively no reason that a HTTP GET would have to be any slower than anything else, and if HTTP takes advantage of keep alives it's faster that most options.
JMX isn't a protocol, it's a specification that wraps many other protocols including, possibly, HTTP or XMPP.
In the end, at the levels where Java will operate, there's either UDP or TCP. TCP has more overhead by guarantees delivery (via retransmission) and ordering. UDP offers neither guaranteed delivery nor in-order delivery. If you can deal with UDP's limitations you'll find it "faster", and if you can't then any lightweight TCP wrapper (of which HTTP is one) is just about the same.
Your requirements seem to be:
one client and one server (inferred from your first sentence),
HTTP is mandatory (inferred from your talking about a web application database),
1000 or more record updates per second, and
individual updates do not need to be acknowledged synchronously (you are willing to use "fire and forget" approach.
The way I would approach this is to have the client threads queue the updates internally, and implement a client thread that periodically assembles queued updates into one HTTP request and sends it to the server. If necessary, the server can send a response that indicates the status for individual updates.
Batching eliminates the impact of latency on the client, and potentially allows the server to process the updates more efficiently.
The big difference between HTTP and JMS or XMPP is that JMS and XMPP allow asynchronous fire and forget messaging (where the client does not really know when and if a message will reach its destination and does not expect a response or an acknowledgment from the receiver). This would allow the first app to respond fast regardless of the second application processing time.
Asynchronous messaging is usually preferred for high-volume distributed messaging where the message consumers are slower than the producers. I can't say if this is exactly your case here.
If you have full control and the two web applications run in the same web container and hence in the same JVM, I would suggest using JNDI to allow both web applications to get access to a common data structure (a list?) which allows concurrent modification, namely to allow application A to add new entries and application B to consume the oldest entries simultaneously.
This is most likely the fastest way possible.
Note, that you should keep the information you put in the list to classes found in the JRE, or you will most likely run into class cast exceptions. These can be circumvented, but the easiest is most likely to just transfer strings in the common data structure.
Related
Is making REST based web service (POST) asynchronous is the best way to handle thousands of requests at one time (Keeping in mind that I have only single instance of server serving the request)?
Edited:
Jersey is wrongly tagged.
For eg: I have a rest based web service, which is supposed to be consumed by 100 thousand clients within a very short span of time (~60 seconds). I understand that if I am allowed to deploy multiple instance of the server, then I can use a load balancer to handle all my incoming request and delegate them accordingly. But I am restricted to use only single instance. What design could I opt within this restriction?
I could think of making the request asynchronous( which will not respond to client immediately ) in order to be able to let the server be free from this load and handle the requests at it's own pace.
For now we can ignore memory limitations.
Please let me know if this clarifies your doubt?
The term asynchronous could have different meanings in different places. For a web application code, it could refer to a Nonblocking I/O server such as Node or Netty/Akka which is a way for HTTP Requests to time multiplex on the same worker threads. If you're writing callbacks or using async or future constructs, it probably is non-blocking I/O which people sometimes refer to as asynchronous.
However, I could have REST API running on Node which implements non-blocking I/O, but the API or the overall architecture is still fully synchronous. For example, let's say I have an API endpoint POST /photos, which takes in a photo, creates image thumbnails, stores the URLs of the photo in a SQL Db and then stores the images in S3. The REST API could still block from the initial POST until after the image is processed and stored.
A second way is for the server to accept the photo process as a job and return immediately. Then the server could store the photo in a in memory or network based queue to be processed later by some other worker thread. In fact, I could even implement this async architecture even with a blocking server like some good old Java 7 and Jetty.
I am developing Restful services where we will be inserting/updating new records into database.
Since REST uses HTTP for its communication and HTTP is not reliable, I am worried that, the request may not be sent to the server in case of connection failure.
One of the suggestions I found in the link was "if connection fails just retry again from the client side." But we don't have any control over the client applications.
Other solution was to implement messaging systems like RabbitMQ/JMS to ensure reliability.
I also found in the following link that adding session states improves reliability. I am not able to understand how this happen and more importantly doesn't a good restful service is always stateless?
So to summarize my questions:
To achieve reliability, is Messaging systems best possible approach?
How does session management help me in achieving reliability?
Messaging can help, as long as you don't do any processing when you receive a command to insert or update information, as you need to immediately put the command in a queue. This solution usually adds quite a bit of complexity as you need to notify your client asynchronously when you finish processing the command (was it successful or did it fail?... or did I fail to send the outcome?).
Session management? For reliability? Never heard of that :). Restful services are usually stateless... so no sessions here!
Another option (but depends how your clients integrate with you) is to allow your clients to generate the ids of the items you are going to be storing/updating, in this case, if they get an error back, but you have processed the command successfully, the client can retry, and the same update will happen. You can pair this with versioning to prevent stale updates arriving late.
I'm creating a monitor application that monitors the activities of a user. There are four elements in my system:
EventCatcher: The EventCatcher is responsible for catching all the events that happen in a subsystem and pushes the data to the EventHandler. Based from observation, there is an average of 10 events per second that is being pushed to the EventHandler. Some events are UserLogin, UserLogout.
EventHandler: The EventHandler is a singleton class that handles all the incoming events from the EventCatcher. It also keeps track of all the logged in users in the system. So, whenever the EventHandler receives a UserLogin event, the User object is extracted from the event and is stored in a HashMap. When a UserLogout event is received, that User object will be remove from the HashMap. This class also maintains a Set of all active Websocket sessions because everytime an event has occurred, I would want to inform all the open sessions that a particular event happened.
Websocket Endpoint: This is just a simple Java class annotated with #ServerEndpoint.
Clients: The system I will be building is for internal (company) use only. At production, at most, there will only be around 5 - 10 clients. All the clients will be receiving the same information every time an event has occurred.
So right now I am trying to convince my supervisor that Websockets is the way to go, however, my supervisor finds it really unnecessary because a simple polling solution would do the trick.
His points are:
We don't really need up-to-date information by the millisecond. We can poll every second.
If I was to maintain a list of open WebSocket sessions, how would that work in a clustered environment (we use a load balancer)
If I plan to send information to the client every time an event (UserLogin, UserLogout) has occurred, I should be able to just send small updates to all WebSocket sessions - meaning, I can't be sending a whole JSON dump of everything. So that means, for every WebSocket instance, I would have to maintain another Set of Users and properly maintain it to mirror the Set contained in the EventHandler.
What my supervisor suggests is that I lose the WebSocket and just convert it to a simple Servlet and let the clients poll every second to receive the entire JSON dump.
In this scenario, should I stick with WebSockets? Or should I just poll?
The main advantage, as far as I've read, of Websockets vs. polling is that by using Websockets, you will have a persistent connection from client to server. HTTP is not really meant for real-time data.
Also, polling requires sending an HTTP request every time and every request comes with HTTP headers. If an HTTP request header contains 800 bytes, then that's 48kb sent per minute per client. With a WebSocket, this isn't problem.
But then again, we won't really have a lot of active clients. We're not concerned about third parties sniffing our requests because this system is for company use only - internal use! And I believe my supervisor wants something simple and reliable.
I am fine with either way. I just want to be sure whether I'm using the right tool for the job.
Additional question: If WebSockets is the way to go, is there any reason why I should consider polling?
The entire purpose of WebSocket is to efficiently support continuing connections between client and server.
I’m not clear on how you are implementing your app. If this is a web app running in a Servlet environment leveraging WebSocket support in the web server, be aware that you need to use recent versions of the Servlet container. For example, with Tomcat you must use either version 8 or the latest updates to version 7.
And of course the web browser must have support for WebSocket.
Be aware that WebSocket is still a new technology that has been changing and evolving in both the specs and the implementations.
Atmosphere
You may want to consider using the Atmosphere framework. Atmosphere supports multiple techniques of Push including WebSocket & Comet.
The Vaadin web-app framework leverages Atmosphere to provide automatic support for Push in your app. By default, WebSocket is automatically attempted first. If WebSocket is not available, Vaadin+Atmosphere falls back automatically to the other techniques including polling.
My application makes Web Service requests; there is a max rate of requests the provider will handle, so I need to throttle them down.
When the app ran on a single server, I used to do it at the application level: an object that keeps track of how many requests have been made so far, and waits if the current request makes it exceeds the maximum allowed load.
Now, we're migrating from a single server to a cluster, so there are two copies of the application running.
I can't keep checking for the max load at the application code, because the two nodes combined might exceed the allowed load.
I can't simply reduce the load on each server, because if the other node is idle, the first node can send out more requests.
This is a JavaEE 5 environment. What is the best way to throttle the requests the application sends out ?
Since you are already in a Java EE environment, you can create an MDB that handles all requests to the webservice based on a JMS queue. The instances of the application can simply post their requests to the queue and the MDB will recieve them and call the webservice.
The queue can actually be configured with the appropriate number of sessions that will limit the concurrent access to you webservice, thus your throttling is handled via the queue config.
The results can be returned via another queue (or even a queue per application instance).
The N nodes need to communicate. There are various strategies:
broadcast: each node will broadcast to everybody else that it's macking a call, and all other nodes will take that into account. Nodes are equal and maintain individial global count (each node know about every other node's call).
master node: one node is special, its the master and all other nodes ask permission from the master before making a call. The master is the only one that know the global count.
dedicated master: same as master, but the 'master' doesn't do calls on itslef, is just a service that keep track of calls.
Depending on how high do you anticipate to scale later, one or the other strategy may be best. For 2 nodes the simplest one is broadcast, but as the number of nodes increases the problems start to mount (you'll be spending more time broadcasting and responding to broadcats than actually doing WS requests).
How the nodes communicate, is up to you. You can open a TCP pipe, you can broadcats UDP, you can do a fully fledged WS for this purpose alone, you can use a file share protocol. Whatever you do, you are now no longer inside a process so all the fallacies of distributed computing apply.
Many ways of doing this: you might have a "Coordination Agent" which is responsible of handing "tokens" to the servers. Each "token" represents a permission to perform a task etc. Each application needs to request "tokens" in order to place calls.
Once an application depletes its tokens, it must ask for some more before proceeding to hit the Web Service again.
Of course, this all gets complicated when there are requirements with regards to the timing of each calls each application makes because of concurrency towards the Web Service.
You could rely on RabbitMQ as Messaging framework: Java bindings are available.
I recommend using beanstalkd to periodically pump a collection of requests (jobs) into a tube (queue), each with an appropriate delay. Any number of "worker" threads or processes will wait for the next request to be available, and if a worker finishes early it can pick up the next request. The down side is that there isn't any explicit load balancing between workers, but I have found that distribution of requests out of the queue has been well balanced.
This is an interesting problem, and the difficulty of the solution depends to a degree on how strict you want to be on the throttling.
My usual solution to this is JBossCache, partly because it comes packaged with JBoss AppServer, but also because it handles the task rather well. You can use it as a kind of distributed hashmap, recording the usage statistics at various degrees of granularity. Updates to it can be done asynchronously, so it doesn't slow things down.
JBossCache is usually used for heavy-duty distributed caching, but I rather like it for these lighter-weight jobs too. It's pure java, and requires no mucking about with the JVM (unlike Terracotta).
Hystrix was designed for pretty much the exact scenario you're describing. You can define a thread pool size for each service so you have a set maximum number of concurrent requests, and it queues up requests when the pool is full. You can also define a timeout for each service and when a service starts exceeding its timeout, Hystrix will reject further requests to that service for a short period of time in order to give the service a chance to get back on its feet. There's also real time monitoring of the entire cluster through Turbine.
I've a WEB application (with pure Java servlet) that have some heavy computational work, with database access, that can be done in asynchronous mode.
I'm planning to use a dedicated server to execute such batch jobs and I'm wondering which tools/techniques/protocols to use for communication between servlets in the WEB server and batch jobs in the new dedicated server.
I'm looking at JMS. Is it the right choice?
There are industry standard and/or widely adopted techniques?
I need also queue and priority handling for multiple simultaneous jobs.
JMS is a pretty standard solution. The high-end platforms (Sun's JCAPS, for example) makes heavy use of JMS to partition and manage the workload of web services.
There are many advantages to buying a high-end JMS implementation from Sun (or IBM or Microsoft). First, you get things like reliable message queues that are backed to the file system. No message can get lost. Second, you get some monitoring and management tools.
One cool thing is to have a JMS queue with (potentially) multiple subscribers to do workload balancing.
Another cool thing is to have JMS topic which has a logging process as well as the real work process subscribed. The logging process picks off the messages and simply records the essential stages of the job being started and stopped.
Messaging is one of the best options.
Make the messaging framework very generic so that it can handle any type of batch jobs.
One approach is to have an event/task manager where you put an event on the queue and the queue consumer processes the event and converts it into a set of tasks. The tasks can then be executed by separate task handlers. A task can also generate some more events that can be again put on the queues to provide a feedback loop. This way you can add work flow like features to the framework and allow your batch jobs to have dependencies on each other.
JMS would be the appropriate solution for sending your batch jobs from the servlet. It may not be the best solution for the batch server to communicate with the servlet though, as it cannot be a listener to messages.
As I don't know what the communication from the batch server to the servlet is supposed to entail, I can only say that there are probably several options you can use (yes JMS is one of them). But they all basically rely on polling calls to the servlet which will then check in some way to see if there is anything from the batch server waiting. This could simply be a servlet on the batch server or making receive calls to a JMS response queue. Other solutions are available, but the point is it is not asynchronous, unless you have the ability to push from the batch server all the way to you client end (a browser I am guessing) via something like AJAX.
Anyway, just something to keep in mind.
Another alternative for asynchronous processing is to have the web application store the request in the database, and have the batch process poll the database for new batch jobs to process. Since your application appears to be smaller (pure Java Servlets) this may be a simpler and lower cost solution.
Hope it helps.
We use JMS with web services:
Client requests computation via web service
Server writes JMS message, and creates an ID value which is stored in a database along with a status (initially "Pending"). Server returns the id to the client.
Server (can be separate server) reads JMS message, does computation, and when finished updates the status to "Completed" in the database
While the computation is ongoing, the client is polling the server to determine the status using another web service (along with the id). The server returns the status which is retrieved from the database. Once the server computation is completed, the client will see the "Completed" status and know that the computation is complete.