It's difficult to simulate a real stress scenario when every micro-service is under stress. Generally in load test, there's only few micro-service which are under stress and not all the micro-services. Is there any way we can increase the response time of all the outgoing calls from my service, so that it's easy to estimate the actual response time of my service during a load test or any way how to implement that.
I havent tried anything because i dont have any clue about it.
If you have a complex application built on top of microservice architecture pattern be aware that it will always act on speed of its slowest component.
So your goal is not to stress all the microservices, but to determine what is the slowest one. Once done you can investigate the reason, suggest and/or implement the fix and repeat the test until you will be satisfied with the result.
The best approach would be going for stress testing of the whole integrated system, starting from 1 virtual user and increasing the load until transport starts queuing up at ingress of one of the microservices.
Once you figure out which is the slowest microservice you should be able to stress-test it separately and use monitoring or profiling tools in order to determine the bottleneck.
You could consider two of these options.
Gatling - https://www.baeldung.com/load-test-a-website-with-gatling
JMeter - https://octoperf.com/blog/2018/04/23/jmeter-rest-api-testing/
Related
Recently, I have faced one java interview question. It goes like this : "There are 3 Microservices (flow goes from 1st to 2nd to 3rd) which takes a minimum roundabout of 0.5sec to provide the response. But the web request should get response in 1sec itself. How to achieve this ?"
Any architecture design or pattern or any settings, need to do ?
It's a very vague question and there's no easy direct answer, it's more to identify in which direction you will go, what analisys options will you suggest. Its reliability engineering (SRE) which includes many tricks and approaches.
I would start by analyzing and clarifying what business process is implemented by this requests sequence, think about needless (it happens that not always Developers write correct code, hence some non-needed calls to services, DB etc)
Monitor network latency and identify where is a focus area. If the network takes significant time, then makes sense to improve network hardware or software, look for problems, bad packages when the Client resends data, "Package storm" issue etc. If the network is fine, focus on services.
Then consider caching data from downstream services (In-progress cache or Distributed depends on architecture and data type). This step should be done carefully with a full understanding of data nature, e.g Can it be cached, for which period, which way to use for refreshing/evicting data?
Pay attention to the possibility of code optimisation which is executed. It happens, that Developers don't keep in mind performance during implementation, hence can create functionality with unneeded operations (for example some sorting, filtering, synchronization (with locks), etc).
Part of 3., parallelize everything that is possible inside the code execution (no guarantee it helps), get rid of locks. For example, there can be some dependencies on DB or other sources before/after calls to downstream service, which may lead to unpredicted blockings, in such a situation make sense to do execution of tasks in parallel threads without blocking each other.
If no low-level tricks help, it can ring a bell to revisit the architecture of services, e.g if SLO 1s is very important, then maybe it makes sense to join 1+2 or 2+3 microservices into bigger service to reduce data transformation and transferring between service (need to calc before).
There are much more things to consider, depends how deep you would like to go.
I am using JMeter to do stress testing to my project.
I set the Constant Throughout Timer as 20/s, and last for 100s. But the actual is: the TPS most get 7/s, and never get 20/s. I don't know why is this.
I want to ask 2 questions:
1、As my understanding, the JMeter can simulate the stress so it can get TPS at 20/s, is there any config wrong of my JMeter?
2、If for other case, the Jmeter can simulate the stress only depend on the server, is it mean there is problem with my server lead to the TPS can't be improved?
Constant Throughput Timer can only pause the threads in order to limit JMeter to the defined number of requests per minute. Moreover, it's more or less precise only on "minute" level, if your test lasts less - you may not see the impact.
Another factor is application response time as JMeter waits for the previous response before sending the next request therefore if the application doesn't respond fast enough you will not be able to reach the desired number of requests per second
And last but not the least is that JMeter should have enough resources in order to send required number of requests per second because if JMeter is not properly configured or the machine where JMeter is running is overloaded - you will not be able to send requests fast enough even the application can handle more load.
So recommendations are:
Make sure you provide enough threads (virtual users) to conduct the required load in the Thread Group
Make sure to follow JMeter Best Practices
Make sure to monitor the baseline health metrics of the machine where JMeter is running (CPU, RAM, Network, Disk, etc.), it can be done using JMeter PerfMon Plugin. If you detect the lack of resources you will need to find another machine and go for Distributed Testing
You can consider Concurrency Thread Group and Throughput Shaping Timer combination instead of your current setup, they can be connected via Feedback Function so JMeter will be able to kick off more threads if current amount is not enough to create the necessary load
In Play's documention about ThreadPools it's recommended for highly synchronized Java web applications with a lot of blocking IO to 'use a very large number of threads in its pool'. In the example they use 200 to 300 parallel threads. My app reads files from a hard disk in most of the requests and nearly each request access a MySQL database via JPA, so I'd say it's pretty synchronized.
Now I did some stress test with Gatling on my development laptop and compared the Play's default thread pool with the one recommended for synchronized web applications. Surprisingly I couldn't find any difference between both.
Do I the wrong kind of stress test? What kind of stress test should I do to see the difference in the two configs? Or do I misunderstand Play's documentation?
Regarding your load test, make sure to you don't just run a few concurrent users that execute requests at fast pace. This way, you might get the number of requests per sec you want, but not the proper level of concurrency.
Make sure you run the proper number of concurrent users to match what you're expecting on your live system.
I have a legacy product in financial domain.Using tomcat 6. We get millions of request 10k of request in hour. I am wondering at high level
should i go for ditributed application where my mvc component is on one system and service/dao on another box(can use spring remote/EJB).
The reason i am planning to go in this direction so that load is distribute and get better performance With this it becomes scalable also.
I only see the positive side of it but somehow not able to figure out what can be the negative aspect of it?
If some expert can help
what is the criteria i should consider to go for distributed model and pros/cons of it? I also tried googling where i could get some stats
like how much load a given webserver (tomcat in my case)handle efiiciently with given hardware(16 gb ram, windows 7, processor ).
Yes i am going
to do POC where i will be measuring performance with distributed model vs without bit high level input will be highly appreciated?
It is impossible to answer this questions without more details - how long does it take to reply to one request on the current server? How many resources are allocated for one request?
having 10k requests per hour means ~3 requests per second. If performing the necessary operations and replying to a request, using 1 CPU takes ~300ms - one simple machine is totally fine. This is simple math, and doesn't always work. I guess you still have peaks within those 10k requests per hour and they aren't gradually distributed.
If we assume, one reply can take up to 1 second, than you can handle as many replies per second as your system has CPUs (given that a CPU would be the bottle neck) If the CPU isn't the bottle neck for your application server, there's probably something wrong. You should set up the database(s) on a different machine and only perform computation tasks on the application server machine.
Especially in the financial sector with a legacy software, I wouldn't try splitting a running product. How old is the current server? I believe that a new Server should be cheaper than rewriting an application. Unless you expect 50-100k requests per hour very soon, I don't think, splitting up such small parts makes sense.
Instead - run it on an up to date server hardware, split application server and data storage and you should be fine.
I am wondering at high level if should i go for ditributed application where my mvc component is on one system and service/dao on another box(can use spring remote/EJB).
I'm not sure what you mean for "system" in this context, but if it means that you are planning to run your application in two servers,
one dedicated to presentation and other dedicated to business layer, take in mind that a simpler approach (and probably more suitable for your app)
is build a co-located architecture.
Basically, the idea is to replicate your app in several servers (at least two) and put in front of them a load balancer that routes the incoming requests among the available servers.
All servers share the same database instance. This will give you vertical scalability and also will improve the availability of your system.
I only see the positive side of it but somehow not able to figure out what can be the negative aspect of it?
Distributing your business logic will probably involve a refactor of your application code, if the system is working well you will add some bugs for sure.
The necessary remote calls will add latency and the fact that you execute your business logic in several servers doesn't resolve the performance problems on the presentation tier.
In Expert One-on-One J2EE Development Without EJB (pag. 65), you can find a good reading about why not distribute your business logic.
How can we increase the performance of an application. My application is written using Java, Hibernate, Servlets, Wsdl i have used for web services. I have executed some of the tests on linux machine, so that i can get proper TPS of the execution.
but still , i am not satisfied by the performance.
So for this, what all steps i should try to increase the performance.
adding to above, i have executed code coverage and used find bugs in the code prominently for each and every test and every service i have written.
Individual suggestions are invited.
Thanks.
Profile your application, and remove all of your bottlenecks.
In addition, or better before, take a day or two and read as much from the Java Performance Tuning newsletters as you understand.
You should monitor your application with a tool like VisualVM, JProfiler etc. to determine the performance bottleneck(s). It is pointless to tune the application without knowing where the actual performance problems are located.
In a professional environment, I suggest dynaTrace that can show you performance bottlenecks along the execution path. The tool can show you exactly where the application spends its time.
Is the performance related to disk I/O or network I/O? In a high throughput system (from DB point of view) Hibernate might not be the best way to go. If you have a lot of writes I would recommend you use a different mechanism to write to database -- perhaps simply switching to simple JDBC might speed it up?
Secondly, is it the case that your webservices are taking too long to get back with results? SOAP is not the fastest protocols really -- have you looked at something like REST maybe coupled with JSON ?