I have a Spring-Cloud based application.
I have two gateway servers behind an HTTP Load balancer. The gateway servers redirect calls to 3 types of backend servers (let's call them UI1, UI2, REST) by querying a Eureka server for an endpoint.
However, if I am understanding the Spring documentation correctly, once a connection has been made between the client and the endpoint, Eureka is no longer needed until a disaster occurs. Client-side load balancing means that the client now has knowledge of an endpoint, and as long as it is working, it will not fetch a new endpoint from Eureka.
This is good in general, but in my setup, the client is actually the Gateway server, not the client browser. The client browser connects to the HTTP load balancer. Everything else is pretty much managed by the gateway.
So, it appears that if I have 2 Gateways and 6 backend servers of each type - I'm not getting any benefits of scalability. Each gateway will take ownership of the first two backend servers per server type, and that's it. The other 4 servers will just hang there, waiting for some timeout or crash to occur on the first 2 servers, so they could serve the next requests.
This doesn't sound like the correct architecture to me. I would have liked Eureka/Client side load balancing to have the ability to do some sort of round-robin or other approach, that would distribute my calls evenly between all servers as much as possible.
Am I wrong?
Related
In our company, we are developing a brokerage web-based application in which customers can log in from client product and place order requests to the server.
The server has both WebSocket and rest endpoints. Further, it is deployed in JBOSS container. client applications push orders via WebSocket to the server at a high rate (approximately ~5000 requests per second).
And other inquiry services such as login/buying power validations are provided through rest services.
The complication arose when we were having a load test in different environments. We logged into the server via client product and push order requests at high rate via web socket,
at the same time we tried to log into the server with another client product. But all of sudden rest API became overwhelmed and unresponsive.
The server only prioritized on processing order requests but failed in response to a login request. Recommendations are highly appreciated in this scenario.
Are there any throttling mechanism available for JBOSS container or does precaution should be taken at the application level?
I'm using Spring Eureka as discovery server in my application which is implemented using microservices architecture. The services are mostly created with PHP, and they register themselves on start-up using Eureka REST endpoints and each one of them sends a heartbeat every 30 seconds and everything works well.
Now, imagine service A wants to talk to service B. How does the discovery happen?
Currently I think service A should send a GET request to http://localhost:8761/eureka/apps/service-B endpoint, retrieve the list of current instances of service B and choose between them. Is it the right approach?
What about load-balancing? Should I implement that in my services to ask for a different instance every time? Or choose between them randomly?
Any help would be greatly appreciated.
Update: Take a look at this library.
There is an easy way to do this with Spring Cloud Netflix Sidecar: http://cloud.spring.io/spring-cloud-static/Camden.SR7/#_polyglot_support_with_sidecar
If you want to continue to implementing this yourself you have several options. With client side load balancing you could retrieve all instances from Eureka and then choose one randomly at the consuming side. If you want server side load balancing you will be needing an extra component, like Zuul, and let it do the load balancing and routing. Zuul uses the eureka configuration so it is easy to integrate it.
I have separate application for client side which is in ReactJs and NodeJS (Express server) and Web Services in Java application running in tomcat.
My query is which is better approach in terms of making web service call.
One is making direct web service call from ReactJS and get the data.
Other one is calling web service in Express server. Request from client browser will go to Express and Express will make all web services call.
I know one issue in making direct call to web service will be cross domain policy which can be handle by setting configuration in Java server.
Apart from it what should be better approach.
From my experience it ended up better using direct calls from UI application and avoiding intermediate server.
Reason for doing this directly is that our servers ended up with a lot of restrictions based on IP addresses, and all requests have been coming from intermediate server (nodeJS server), so DDOS protection of end server had to have some exceptions (our node server which could be on ACS with dynamic IP addresses so it might be hard to manage).
Also If you want to pass and track end users IP addresses, you need to manage headers on Node server (to be sure you are passing it as it was in original request).
It is way simpler to manage this kind of situation if calls are comming from React app and simply set up CORS on java server.
Also its way easier to debug it on UI app directly, so you will be watching response logs on one place only. Other way around you could end up debugging your node server and UI app.
Hope this helps a bit.
Best way IMO is to create a router in Node js specifically for all your Java webservices and act as a proxy.
Suppose if your url pattern is like http://domain/java-ws/api then all these requests will be routed to your Java service.
If you have an Apache server directing requests to your node JS then configure url pattern proxy using proxy module.
Browsers are blocking CORS requests for a reason. You may get away by setting things on your server.
I have a GAE project and I would like to make requests to an external server. The server needs to whitelist the IP address but GAE works with dynamic IP addresses and the list is too long. I have been looking at 2 options:
1. Move the GAE project to Compute Engine
My biggest concern about this is the time it would take to move the project to Compute Engine. So far we've been using App Engine for free and we would like to avoid extra costs.
2. Use our domain to make the requests
We could white list the IPs ourselves and receive requests from the GAE project to our domain and then make requests from the domain's static host IP itself to their server.
For this option I'd also like to know if transferring the domain to GAE has any advantage. Will the outgoing requests from GAE through the transferred domain be made through the domain's IP?
Note:
This is largely based on assumption, I am highly inexperienced with networking so if there's anything that could be better expressed please tell me. I tried to search online for a way to do this but couldn't get to a satisfactory answer.
You don't need to think about domains in this context, because you're whitelisting IPs, not domains. And it's not necessary to move the whole project, you need only move that part that makes this requests.
Just create a proxy on any server with a static IP, like micro instance on Google Compute, and do all requests through this server.
As I understand you do all request to the same external server, right? In this case it could be even simpler. You don't need to install a full featured proxy, just install an Nginx on a micro instance (with SSL and some authentication, of course) that will proxy all requests to the target server.
I ended up using RabbitMQ to send messages from Google App Engine to Compute Engine, the compute engine then forward those messages as Http requests.
I was asked in Interview and really confused on answer for below Question.
If say some site is getting 500 requests per second and the mechanism they handle to solve large number of hits is through Load Balancer.
In above case, all the request will first go to Load Balancer, it is then Load Balancer responsibility to pass on request to actual server which will give the response.
So Load Balancer is the entry point for all request and it too have some capacity to handle the request?
So how much request Load Balancer can accept and if it accept upto some limit then how system works in such case?
I am sorry if my Question is meaningless, Please explain?
Thanks.
A load balancer is usually running a lightweight event based HTTP server (NginX, for example) that maintains a connection to a backend server for each incoming request.
A popular setup is:
NginX
| \
| \
Apache Apache ...
Since NginX can handle a lot more connections and has a fairly predictable memory usage pattern, it can usually be used as a frontend for several Apache backend servers (running PHP, Python, Ruby, etc).
Replace Apache with Tomcat/Jetty/insert your favorite servlet container here and the same logic applies.
The main point here is that your backend servers are usually doing something a lot more time consuming (Running some scripting language, querying a DB, etc) that it is more than likely that each backend server is not bottlenecked by its HTTP server component but rather by the application logic.
There is no one size fits all solution to these kinds of problems. Another solution (amongst many), once you outgrow the capacity of a single load balancer machine is to use DNS round-robin between several balancers supporting several backend servers.
I would say that if the amount of connections rise up to the limits of the Load Balancer entry point, then you can implement a DNS level balancing (If name resolution is used in the requests). Each request will be driven (e.g. "round-robined" ) to different entry point thanks to the DNS resolution which is the responsible of switching among different resolutions for the same name in the requests, and sending those requests to different Load Balancers or directly to different servers (latter one, would imply a different LB technique).
But the 500 requests per second shouldn't be a problem for a server responsible of balancing.
HAProxy can handle thousands of requests/second without problem, driven different sessions to different servers, or also keeping active sessions distributed in different servers.
HAProxy is a free, very fast and reliable solution offering high
availability, load balancing, and proxying for TCP and HTTP-based
applications. It is particularly suited for web sites crawling under
very high loads while needing persistence or Layer7 processing.
Supporting tens of thousands of connections is clearly realistic with
todays hardware.