One Spring boot application sends many calls in parallel to another and in one time i faced with 29 Too Many Requests
How can avoid it?
You should stop spamming your server and rethink your architecture. The 429 status is not an error. It is more a hint to stop spamming:
The 429 status code indicates that the user has sent too many
requests in a given amount of time ("rate limiting").
https://www.rfc-editor.org/rfc/rfc6585#page-3
You need K8s with clustering of your micro service and to think how you want
to scale your pods horizontally.
On top of that you need to use a load balancer and maybe to use the ribbon + Hysterix + Feign client out of the stack of spring cloud solution.
And you also have too check that you don't receive too many auto-retry technical requests of your underlying HTTP client.
Your application sends many calls in parallel to another, I unable to get what is another?,
If another is db call then you can increase connection size in db then you wouldn't be get in future.
or else if another is API call then you should go with batch statement it will helpful instead of parallel call.
This is standard use case for Pub-Sub pattern. Use any message queue like, SQS, Kafka etc.
First spring boot instance should push messages to this queue. Another spring application (or any application) should poll messages at the rate suitable for it self.
Messages will be hold in the queue until app2 picks it up.
Related
We are in the process of designing the migration of our monolithic Java application to microservices to meet various client requirements such as scalability, high availability, etc. The core function of our application is data processing, i.e. retrieve data from a source, put it through 0 or more transformations, and finally push the result to a destination. For this reason, we are looking at Spring Cloud Data Flow running on Kubernetes and Kafka to do the heavy lifting for us, with a few custom built stream applications to handle our business logic.
One thing we have not yet figured out though is how it could handle synchronous responses to requests being sent in via an HTTP source - specifically when some processing is required before responding. For example, let us say that a request is received containing two different amounts in a JSON packet. We then pass this on to a custom "addition" transformer that outputs the sum of these amounts, and needs to return the result back to the calling party. However, because the transformer is a completely separate process that received the data by consuming it from a Kafka topic, it has no access to the original HTTP connection to respond.
Is this something that is possible with Spring Cloud Data Flow, perhaps by combining it with something like Spring Cloud Gateway to manage the HTTP connection? Or are we barking up the wrong tree completely?
It is not easy combine Async Flows (Spring Cloud Data Flow) with a Sync HTTP Flow (HTTP Requests have a timeout and Async Flow processing time is not known upfront). What you can do is return an id as response for your HTTP source and have another HTTP Endpoint to check the status of the initial request and get the result back using that. This is a polling approach.
please give me some advice about the best pattern of the task solution. My task is this:
User makes a request to the Camunda processor through own rest
controller
BPMN schema on a backend side consists of a chain of
several asynchronous services
Data will be ready for response to
the User only when one final service on BPMN makes it.
Every chain works not greater than 10-15 secs. And users sessions count is less than 500 an hour.
How to organize the work of the rest controller? Is it acceptable to force controller waiting of result in the same call? Where a bottleneck?
Can you use some server push technology? If it were just a couple of seconds, I'd say go for waiting in the rest controller.
Being 15 seconds and thinking about scalability, I'd say to follow some kind of asynchronous pattern with the client to.
Client sends a request to do something
The controller delegates the work to some external process and returns to the client ok
The process ends and a response is ready.
If the other side is a browser, use some kind of server push technology to notify it. If it is an application use some kind of rpc, polling or any other inter process mechanism to communicate.
Note that depending on the hosting technology, there are different limits on concurrent connections. Check Spring Boot - Limit on number of connections created for tomcat.
I have a Java application which exposes REST.
I have a master and 2 slaves. (jboss7.1.1)
Apache-modcluster takes care of load balancing.
I have queues maintained in my application which gets filled up at times, when there are no consumers. Hence I get OutOfMemory and the slave node goes down.
Is there any configuration in apache modcluster to check the load and block incoming requests, inspite of just routing?
Or any different suggestion?
As far as I am aware you can only route requests based on load (using a LoadMetric), not block them
I would suggest you should design you API to return a different response code (509 or 503 perhaps) when your queues are full and the jboss node is unable to service the request, you can then instruct the API consumers to try again after a timeout if they encounter one of these responses
It is possible to 'rate-limit' apache using something like mod_security (see: https://johnleach.co.uk/words/1073/rate-limiting-with-apache-and-mod-security), however I think that controlling this from within your application code would be better
i am going to integrate some applications using RabbitMQ. Now i am facing the design issue. Right now i am having one application producing message and one application consuming it (in future more are possible). Both applications have access to some database. Application A is some kind of registration application when it receives registration request it sends message to on rabitmq. Now application b receives this message and its task is to load the registration data to elasticsearch server. Now i have some options
consumer will read the message and id from q and load the data and send it to the elastic search server
fastest throughput. Because things will move in asynchronous way. other process which may be running on separate
server will loading the data and sending to elastic server
consumer will read the message and id from the q and then call the rest service to load the company data.
will take more time for processing each request as it will be having network overhead.although it will save time to data load
but will add network delay. And it will by pass the ESB(Message Broker) also. (i personally think if i am using esb in my application
it is not necessary that i use it for every single method call)
send all the registration data in the message. consumer will receive it and just upload it to elasticsearch server.
which approach i should follow?
Apparently there are many components to your application set up that is hard to take into account and suggest a straightforward answer. I would suggest that you should look into each design and identify I/O points, calls over the network and data volume exchanged over the network. Then depending on the load you expect and the volume of data you expect to store over time I would suggest you hierarchize these bottlenecks giving a higher score depending on the severity of it. Identify the one solution that has the lowest score and go with that.
I would suggest you should benchmark the difference between sending only the iq or sending the whole object. I would expect that the difference is negligible.
One suggestion. Make your objects immutable. It is not directly relevant with what you are describing but in situations like yours, where components are operating "blindly" you will find that knowing that an object has not changed state is a big assurance.
I work with a large Java web application which uses a service to return information. My key goal is to retrieve 100+ individual results as quick as possible. We've noticed that sending 100+ requests for information doesn't give us the best performance on the replies of data. Our solution was to break the 100+ requests into small batches (~15,25) and assemble them once all have been received.
I'm looking for a suggestion in Java to make 1 or 50 or 200 requests from the application, to the service, get back info to the application and perform another batch if there are more requests. If no requests left, assemble into on list and return that full list.
Any suggestions of form are welcome, thanks.
I use Spring Integration for this kind of thing. You can set up a configurable message splitter that chops up your request and sends off many tiny ones, and a message aggregator that knows when it has received all responses and can then give you back a single result.
Spring also has a product called Spring Batch that might be a useful alternative, but that is more for heavy batch processing which it doesn't sound like you are doing.
If possible / feasible, extend the /service to support handle multiple logical requests in a single protocol request; e.g. HTTP request.
In theory, the both the client and server side should be able to do the work with less overheads if the server side gets a single request. There should be savings on both the client and server sides.