i'm developing an app that make requests to the Musicbrainz webservice. I read in the musicbrainz manual to not make more than one request per second to the webservice or the client IP will be blocked.
What architecture do you suggest in order to make this restriction transparent to the service client.
I would like to call a method (getAlbuns for example) and it should only make the request 1sec after the last request.
I also want to call 10 request at once and the service should handle the queueing, returning the results when avaiable (Non-blocking).
Thanks!
Because of the required delay between invocations, I'd suggest a java.util.Timer or java.util.concurrent.ScheduledThreadPoolExecutor. Timer is very simple, and perfectly adequate for this use case. But if additional scheduling requirements are identified later, a single Executor could handle all of them. In either case, use fixed-delay method, not a fixed-rate method.
The recurring task polls a concurrent queue for a request object. If there is a pending request, the task executes it, and returns the result via a callback. The query for the service and the callback to invoke are members of the request object.
The application keeps a reference to the shared queue. To schedule a request, simply add it to the queue.
Just to clarify, if the queue is empty when the scheduled task is executed, no request is made. The simple approach would be just to end the task, and the scheduler will invoke the task one second later to check again.
However, this means that it could take up to one second to start a task, even if no requests have been processed lately. If this unnecessary latency is intolerable, writing your own thread is probably preferable to using Timer or ScheduledThreadPoolExecutor. In your own timing loop, you have more control over the scheduling if you choose to block on an empty queue until a request is available. The built-in timers aren't guaranteed to wait a full second after the previous execution finished; they generally schedule relative to the start time of the task.
If this second case is what you have in mind, your run() method will contain a loop. Each iteration starts by blocking on the queue until a request is received, then recording the time. After processing the request, the time is checked again. If the time difference is less than one second, sleep for the the remainder. This setup assumes that the one second delay is required between the start of one request and the next. If the delay is required between the end of one request and the next, you don't need to check the time; just sleep for one second.
One more thing to note is that the service might be able to accept multiple queries in a single request, which would reduce overhead. If it does, take advantage of this by blocking on take() for the first element, then using poll(), perhaps with a very short blocking time (5 ms or so), to see if the application is making any more requests. If so, these can be bundled up in a single request to the service. If queue is a BlockingQueue<? extends Request>, it might look something like this:
Collection<Request> bundle = new ArrayList<Request>();
bundle.add(queue.take());
while (bundle.size() < BUNDLE_MAX) {
Request req = queue.poll(EXTRA, TimeUnit.MILLISECONDS);
if (req == null)
break;
bundle.add(req);
}
/* Now make one service request with contents of "bundle". */
You need to define a local "proxy service" which your local clients will call.
The local proxy will receive requests and pass it on to the real service. But only at the rate of one message per second.
How you do this depends very much on the tecnoligy available to you.
The simplest would be a mutithreaded java service with a static and synchronised LastRequestTime long;" timestamp variable. (Although you would need some code acrobatics to keep your requests in sequence).
A more sophisticated service could have worker threads receiving the requests and placing them on a queue with a single thread picking up the requests and passing them on to the real service.
Related
We are using Akka actors in our application. Actors are invoking services and in some cases those services are calling rest API exposed by third party. Now the third party APIs are taking a very long time to respond. Due to this during peak time , the system through put is getting impacted as threads are blocked as they wait for client API to respond.
Sometimes during peak time , because threads are waiting , messages just keep waiting in akka mail box for long time and are picked up, once blocked threads are made available.
I am looking for some solution where in i can improve the throughput of the system and can free up threads so that actor messages can start processing.
I am thinking to change the rest API call from blocking to non blocking call using Future and creating a dedicated actor for such kind of rest API invocations . Actor will periodically check if future is completed and then sends a completion message and then rest of process can continue.This way the number of blocking threads will reduce and they will be available for actor message processing.
Also would like to have separate execution context for actors which are doing blocking operations. As of now we have global execution context.
Need further inputs on this.
Firstly, you are correct that you should not block when processing a message, so you need to start an asynchronous process and return immediately from the message handler. The Actor can then send another message when the process completes.
You don't say what technology you are using for the REST API calls but libraries like Akka HTTP will return a Future when making an HTTP request so you shouldn't need to add your own Future around it. E.g:
def getStatus(): Future[Status] =
Http().singleRequest(HttpRequest(uri = "http://mysite/status")).flatMap {
res =>
Unmarshal(res).to[Status]
}
If for some reason you are using a synchronous library, wrap the call in Future(blocking{ ??? }) to notify the execution context that it needs to create a new thread. You don't need a separate execution context unless you need to control the scheduling of threads for this operation.
You also don't need to poll the Future, simply add an onComplete handler that sends an internal message back to your actor which then processes the result of the API call.
Context:
1) We have a scheduler which picks up jobs and process them by calling another rest-call in a blocking manner.
2) Scheduler thread needs to wait for the rest-call to complete and in-turn do some another task based upon the result.
3) There is no constraint for this to be real time.
Problem Statement:
1) What we want is to free scheduler threads as soon as an external call is made as external call takes significant time to complete.
2) We should be informed about the result received from the external call as we need to do some processing based on the result.
Idea in my mind:
1) Rather than calling the external system using synchronous Http call, we
can push the event to the queue.
2) Api consumer of another system will read the event from the queue and do the long running task. And post processing push the result back to the queue on a different topic.
3) Our system now can read the response from the queue(second topic) and do the necessary actions.
This is one of the design approach that comes to my
I need advice on whether we can improve the design somehow.
1) Can this be done without introduction of queue ?
2) Is there any better way to achieve the asynchronous processing ?
If you want to avoid using a queue, I can think of 2 other alternatives, for example:
1) Rather than calling the external system using synchronous Http call, we can push the event to the queue.
alternative a)
you do a synchronous HTTP GET to tell the other system that you want certain job to be executed (the other system replies quickly with a "200 OK" to confirm that it received the request).
alternative b)
you do a synchronous HTTP GET to tell the other system that you want certain job to be executed (the other system replies quickly with a "200 OK" and a unique ID to identify the job to be executed)
2) Api consumer of another system will read the event from the queue and do the long running task. And post processing push the result back to the queue on a different topic.
3) Our system now can read the response from the queue(second topic) and do the necessary actions.
alternative a)
upon receiving the request, the other system performs the long running computation and then when it is ready it makes a synchronous HTTP call to your original system to inform that the job is ready.
alternative b)
upon receiving the request, the other system performs the long running computation.
the original system doesn't know if the job is done, so it polls at certain times (doing a synchronous HTTP GET to a different REST API) providing the JOB ID, to find out if the job is ready.
In the gwt client I can cancel a request by letting the method in the Async interface return a Request object which contains a cancel() method I can call.
So far so good, but is there a way to detect this in the java code running on the server?
My use-case is that I have a rpc call which take a long time to complete on the server, and where the user has a "cancel" button to stop the request.
And If the user cancel the request, the server should stop processing, but there seems to be no way to detect that the client have closed the connection.
It is usually not a good idea to use server's request threads for long running tasks. You need to redesign them to be executed asynchronously (if you still have not done it). You can utilize java.util.concurent tools like FutureTask and Executors to achieve this. You will also need to use thread pools to make sure to control a max number of concurrent tasks.
When you submit a request for a long task from the client, you need to return a reference key (e.g. UUID or some unique string) to your FutureTask as soon as you schedule it for execution. Then to cancel the task, you need pass the key from the client and look up you task and cancel it:
See javadoc for more details:
http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/FutureTask.html
Because of browser compatibility issues, I have decided to use long polling for a real time syncing and notification system. I use Java on the backend and all of the examples I've found thus far have been PHP. They tend to use while loops and a sleep method. How do I replicate this sort of thing in Java? There is a Thread.sleep() method, which leads me to...should I be using a separate thread for each user issuing a poll? If I don't use a separate thread, will the polling requests be blocking up the server?
[Update]
First of all, yes it is certainly possible to do a straightforward, long polling request handler. The request comes in to the server, then in your handler you loop or block until the information you need is available, then you end the loop and provide the information. Just realize that for each long polling client, yes you will be tying up a thread. This may be fine and perhaps this is the way you should start. However - if your web server is becoming so popular that the sheer number of blocking threads is becoming a performance problem, consider an asynchronous solution where you can keep a large numbers of client requests pending - their request is blocking, that is not responding until there is useful data, without tying up one or more threads per client.
[original]
The servlet 3.0 spec provides a standard for doing this kind asynchronous processing. Google "servlet 3.0 async". Tomcat 7 supports this. I'm guessing Jetty does also, but I have not used it.
Basically in your servlet request handler, when you realize you need to do some "long" polling, you can call a method to create an asynchronous context. Then you can exit the request handler and your thread is freed up, however the client is still blocking on the request. There is no need for any sleep or wait.
The trick is storing the async context somewhere "convenient". Then something happens in your app and you want to push data to the client, you go find that context, get the response object from it, write your content and invoke complete. The response is sent back to the client without you having to tie up a thread for each client.
Not sure this is the best solution for what you want but usually if you want to do this at period intervals in java you use the ScheduleExecutorService. There is a good example at the top of the API document. The TimeUnit is a great enum as you can specify the period time easily and clearly. So you can specify it to run every x minutes, hours etc
I need to wait for a condition in a Spring MVC request handler while I call a third party service to update some entities for a user.
The wait averages about 2 seconds.
I'm calling Thread.sleep to allow the remote call to complete and for the entities to be updated in the database:
Thread.currentThread().sleep(2000);
After this, I retrieve the updated models from the database and display the view.
However, what will be the effect on parallel requests that arrive for processing at this controller/request handler?
Will parallel requests also experience a wait?
Or will they be spawned off into separate threads and so not be affected by the delay experienced by the current request?
What are doing may work sometimes, but it is not a reliable solution.
The Java Future interface, along with a configured ExecutorService allows you to begin some operation and have one or more threads wait until the result is ready (or optionally until a certain amount of time has passed).
You can find documentation for it here:
http://download.oracle.com/javase/6/docs/api/java/util/concurrent/Future.html