I'm trying to wrap my head around Hystrix and after reading their docs, still have a question about its usage patterns.
For one, I don't understand the use case for when to use their Asynchronous execution vs. their Reactive execution. The only difference I can see is that Asynchronous execution is always non-blocking, whereas Reactive can either be blocking or non-blocking. So I guess my real question is:
What's the difference between Synchronous and Blocking Reactive execution?; and
What's the difference between Asynchronous and Non-Blocking Reactive execution?
Let's assume you have wrapped two service calls A and B as a HystrixCommand. You now have three options:
use .execute(): pure synchronous call. You call the method and continue your program once the result has arrived. Your program's total execution time is the sum both calls. The main flow of your program is very linear.
use .queue(): receive a Future immediately for both commands. Both service calls are executed in parallel. Then use .get() to retrieve the results. These calls with block until the result is there. Your total execution time is faster than before: your execution time will be the length of the longest service call. Use this when you i.e. want to combine the results of the two services. The main flow of your program is still linear, although both calls are executed in parallel.
use .subscribe(): receive a Observable immediately for both commands. Both service calls are executed in parallel. Then use .subscribe() to register a call-back to act on the result once it is available. This is very useful if you don't want to combine the results and want to react independently on the results of service A and B once they arrive.
The main flow of your program is no linear, but reactive: the flow of the program will continue inside the callback for each command.
I hope it helps.
Related
I'm attempting to implement a spring integration flow that requires a multithreaded call if an input variable in true. If this variable is true then the flow with execute a multithreaded call and the main thread will continue it's flow.
Then at the end it will be required to wait for both flows to finish before returning a response.
I've been successful at implementing a multithreaded spring integration flow using a splitter, but the splitter results in all of the messages going to the same channel, this is different since the multithreaded call requires calling a different channel than the main thread of execution.
Is there a way to set up a splitter to send to different channels based on if the parameter is true or not? Or how would I set up an executor channel to spawn a new thread if that value is true while continuing the main flow at the same time.
As for waiting for both of the flows to finish execution would a spring integration barrier or an aggregator be a better approach for this use case?
Consider to use a PublishSubscribeChannel with an Executor configuration to let the same message to be sent to different parallel flows. This way you really can continue your main flow with one subscriber and do something else with other subscribers. With an Executor all of them are going to be executed in parallel.
Docs: https://docs.spring.io/spring-integration/docs/current/reference/html/core.html#channel-implementations-publishsubscribechannel
If you still insist that main flow must be executed only on the same thread, then consider to use a RecipientListRouter, where one of the recipients could be an unconditional next direct channel in a main flow. The other recipient could be conditional on your boolean variable and it can be an ExecutorChannel to let its subscriber to be invoked in parallel.
Docs: https://docs.spring.io/spring-integration/docs/current/reference/html/message-routing.html#router-implementations-recipientlistrouter
For waiting for both flows it is up to you to decide - barrier, an aggregator or more sophisticated scatter-gather. All of them will work for your "wait-for-all" requirements. Or you may implement some custom solution based on a CountDownLatch in some header. So, every time your parallel flow is done, you count it down. And this way you even will be able to determine the number according your boolean value. So, if no parallel, then just 1 and only main flow is going to be executed and only this one is going to count down that latch you are waiting for on original request.
I am new to this concept and want to have a great understanding of this topic.
To make my point clear I want to take an analogy.
Let's take a scenario of Node JS which is single-threaded and provide fast IO operation using an event loop. Now that makes sense since It is single-threaded and is not blocked for any task.
While studying reactive programming in Java using reactor. I came to a situation where the main thread is blocked when an object subscribes and some delay event took place.
Then I came to know the concept of subscribeOn.boundedElastic and many more pipelines like this.
I got it that they are trying to make it asynchronous by moving those subscribers to other threads.
But if it occurs like this then why is the asynchronous. Is it not thread-based programming?
If we are trying to achieve the async behaviour of Node JS then according to my view it should be in a single thread.
Summary of my question is:
So I don't get the fact of using or calling reactive programming as asynchronous or functional programming because of two reason
Main thread is blocked
We can manage the thread and can run it in another pool. Runnable service/ callable we can also define.
First of all you can't compare asynchronous with functional programming. Its like comparing a rock with a banana. Its two separate things.
Functional programming is compared to other types of programming, like object oriented programming or procedural programming etc. etc.
Reactor is a java library, and java is an object oriented programming language with functional features.
Asynchronous i will explain with what wikipedia says
Asynchrony, in computer programming, refers to the occurrence of events independent of the main program flow and ways to deal with such events.
So basically how to handle stuff "around" your application, that is not a part of the main flow of your program.
In comparison to Blocking, wikipedia again:
A process that is blocked is one that is waiting for some event, such as a resource becoming available or the completion of an I/O operation.
A traditional servlet application works by assigning one thread per request.
So every time a request comes in, a thread is spawned, and this thread follows along the request until the request returns. If there is something blocking during this request, for instance reading a file from the operating system, or making a request to another service. The assigned thread will block and wait until the reading of the file is completed, or the request has returned etc.
Reactive works with subscribers and producers and makes heavy use of the observer pattern. Which means that as soon as some thing blocks, reactor can take that thread and use it for something else. And then it is un-blocked any thread can pick up where it left off. This makes sure that every thread is always in use, and utilized at 100%.
All things processed in reactor is done by the event loop the event loop is a single threaded loop that just processes events as quick as possible. Schedulers schedule things to be processed on the event loop, and after they are processed a scheduler picks up the result and carries on.
If you just run reactor you get a default scheduler that will schedule things for you completely automatically.
But lets say you have something blocking. Well then you will stop the event loop. And everything needs to wait for that thing to finish.
When you run a fully reactive application you usually get one event loop per core during startup. Which means lets say you have 4 cores, you get 4 event loops and you block one, then during that period of blockages your application runs 25% slower.
25% slower is a lot!
Well sometimes you have something that is blocking that you can't avoid. For instance an old database that doesn't have a non-blocking driver. Or you need to read files from the operating system in a blocking manor. How do you do then?
Well the reactor team built in a fallback, so that if you use onSubscribe in combination with its own elastic thread pool, then you will get the old servlet behaviour back for that single subscriber to a specific say endpoint etc.
This makes sure that you can run fully reactive stuff side by side with old legacy blocking things. So that maybe some reaquests usese the old servlet behaviour, while other requests are fully non-blocking.
You question is not very clear so i am giving you a very unclear answer. I suggest you read the reactor documentation and try out all their examples, as most of this information comes from there.
I'm combing over the JAX-RS spec. Everything seems simple enough. But I'm left with the question. Why wouldn't I do this everywhere? In otherwords, what are the downsides? Since most resource requests end up hitting the DB, wouldn't you want all of them to be Async?
What are the downsides that I should be aware of? Is there a bunch of extra overhead? I haven't been able to find anything that says why you shouldn't do this, just how to do this.
I think it is not about Async feature of JAX-RS, it is about synchronous vs asynchronous operations.
As you know synchronous operations use single thread to execute the task where as asynchronous operations require creation of additional thread to execute the task. Now, this is the overhead - creation of a new thread, executing task in new thread and then destroying that thread. This overhead results in performance.
Although, you can reuse already created threads instead of always creating new thread. But, still additional thread exists which impacts performance.
For example, a synchronous web service which processes around 1000 parallel requests. So, for handling these requests, 1000 threads will get created. Now, consider the same situation in asynchronous environment. For handling 1000 parallel requests, 2000 threads will get created - 1000 for handling requests and 1000 for asynchronous operation processing.
Now, you can understand the load on the operating system. High load will impact the performance. If the task is expensive i.e. takes time to execute, overall turn around time will decrease compare to synchronous execution. However, if the task is not expensive like db operations. overall turn around time will increase compare to synchronous execution. This is due to overhead of handling additional threads.
Another reason is complexity, asynchronous is always complex, requires additional code for making things asynchronous. However, framework like Jersey hides this complexity up to some extent.
So, it is always recommended to process expensive operations i.e the operations whose execution will take time, in asynchronous way so that overall turn around time can be decreased. Asynchronous execution for each operation is not recommended.
Today I did some experiments on AsyncRestTemplate. Below is a piece of sample code:
ListenableFuture<ResponseEntity<MyObject[]>> result
= asyncRestTemplate.getForEntity(uri, MyObject[]);
List<MyObject> objects = Arrays.asList(result.get().getBody());
To my surprise, the request was not sent to uri in first line (i.e. after calling getForEntity) but sent after result.get() is called.
Isn't it a synchronous way of doing stuff?
The whole idea of doing async request is that either you do not want to wait for the async task to start/complete OR you want the main thread to do some other task before asking for the result from the Future instance. Internally, the AsyncRestTemplate prepares an AsyncRequest and calls executeAsync method.
AsyncClientHttpRequest request = createAsyncRequest(url, method);
if (requestCallback != null) {
requestCallback.doWithRequest(request);
}
ListenableFuture<ClientHttpResponse> responseFuture = request.executeAsync();
There are two different implementations - HttpComponentsAsyncClientHttpRequest ( which uses high performant async support provided in Apache http component library ) and SimpleBufferingAsyncClientHttpRequest (which uses facilities provided by J2SE classes). In case of HttpComponentsAsyncClientHttpRequest, internally it has a thread factory (which is not spring managed AFAIK) whereas in SimpleBufferingAsyncClientHttpRequest, there is a provision of Spring managed AsyncListenableTaskExecutor. The whole point is that in all cases there is some ExecutorService of some kind to be able to run the tasks asynchronously. Of course as is natural with these thread pools, the actual starting time of task is indeterminate and depends upon lots of factor like load, available CPU etc. and should not be relied upon.
When you call future.get() you're essentially turning an asynchronous operation into a synchronous one by waiting for the result.
It doesn't matter when the actual request is performed, the important thing is that since it's asynchronous, you don't need to worry about it unless/until you need the result.
The advantage is obvious when you need to perform other work before processing the result, or when you're not waiting for a result at all.
i'm developing an app that make requests to the Musicbrainz webservice. I read in the musicbrainz manual to not make more than one request per second to the webservice or the client IP will be blocked.
What architecture do you suggest in order to make this restriction transparent to the service client.
I would like to call a method (getAlbuns for example) and it should only make the request 1sec after the last request.
I also want to call 10 request at once and the service should handle the queueing, returning the results when avaiable (Non-blocking).
Thanks!
Because of the required delay between invocations, I'd suggest a java.util.Timer or java.util.concurrent.ScheduledThreadPoolExecutor. Timer is very simple, and perfectly adequate for this use case. But if additional scheduling requirements are identified later, a single Executor could handle all of them. In either case, use fixed-delay method, not a fixed-rate method.
The recurring task polls a concurrent queue for a request object. If there is a pending request, the task executes it, and returns the result via a callback. The query for the service and the callback to invoke are members of the request object.
The application keeps a reference to the shared queue. To schedule a request, simply add it to the queue.
Just to clarify, if the queue is empty when the scheduled task is executed, no request is made. The simple approach would be just to end the task, and the scheduler will invoke the task one second later to check again.
However, this means that it could take up to one second to start a task, even if no requests have been processed lately. If this unnecessary latency is intolerable, writing your own thread is probably preferable to using Timer or ScheduledThreadPoolExecutor. In your own timing loop, you have more control over the scheduling if you choose to block on an empty queue until a request is available. The built-in timers aren't guaranteed to wait a full second after the previous execution finished; they generally schedule relative to the start time of the task.
If this second case is what you have in mind, your run() method will contain a loop. Each iteration starts by blocking on the queue until a request is received, then recording the time. After processing the request, the time is checked again. If the time difference is less than one second, sleep for the the remainder. This setup assumes that the one second delay is required between the start of one request and the next. If the delay is required between the end of one request and the next, you don't need to check the time; just sleep for one second.
One more thing to note is that the service might be able to accept multiple queries in a single request, which would reduce overhead. If it does, take advantage of this by blocking on take() for the first element, then using poll(), perhaps with a very short blocking time (5 ms or so), to see if the application is making any more requests. If so, these can be bundled up in a single request to the service. If queue is a BlockingQueue<? extends Request>, it might look something like this:
Collection<Request> bundle = new ArrayList<Request>();
bundle.add(queue.take());
while (bundle.size() < BUNDLE_MAX) {
Request req = queue.poll(EXTRA, TimeUnit.MILLISECONDS);
if (req == null)
break;
bundle.add(req);
}
/* Now make one service request with contents of "bundle". */
You need to define a local "proxy service" which your local clients will call.
The local proxy will receive requests and pass it on to the real service. But only at the rate of one message per second.
How you do this depends very much on the tecnoligy available to you.
The simplest would be a mutithreaded java service with a static and synchronised LastRequestTime long;" timestamp variable. (Although you would need some code acrobatics to keep your requests in sequence).
A more sophisticated service could have worker threads receiving the requests and placing them on a queue with a single thread picking up the requests and passing them on to the real service.