I'm quite new to performance testing and am looking to be pointed in the right direction.
I have a Java project which contains two parts, deployed seperately:
A service-broker, published as a webservice; which has service and db wrappers.
A front end, which has a service-broker facade, business logic and a Spring MVC UI.
It is deployed on tomcat, which is running on a fresh install of Windows server 2008.
I need to do basic soak testing on this project, to highlight major memory leaks performance issues.
I've been told SOAP UI is the tool I need to do this.
Now for my questions:
Soap UI (Load UI) is only appropriate as the load-generator for testing the service-broker aspect of the project, right?
What additional tools would be helpful (Something to visualize garbage collection, memory use, heap/stack size etc?)
Can I use Load UI as a load generator for a Spring MVC Front end? If not, what's an appropriate alternative?
Thanks a lot.
Here is my opinions
SoapUI is good enough for microbenchmark test but not good at huge scale of testing. So i recommend to use other load testing tool. LoadUI can be a solution. But i want to recommend nGrinder. I used it, it works very well. Apache Jmeter is common tool. But it is JVM based so Jmeter itself needs a tuning.
To monitoring application during perfomance testing. easiest way is use VisualVM. It can monitor all that u mentioned. But it can show u a data in just Java Virtual machine perspective. I rather to recommend to use APM (Application Performance Monitoring). AppDynamic will be good solution.
About UX testing, big difference is, it needs record and play feature. U can do it by using Load UI. but nGrinder can also cover that by implementing HTTP resquest in coding. (It is a reason why we use such a expensive tools like LoadRunner etc).
I hope this will be useful to u.
Cheers
There are four sets of requirements you need to cover in a tool
Can it exercise my interface (Any HTTP test tool will do this for a web services application)
Can it monitor my infrastructure. Now you are getting into the details of if your underlying OS and Virtual Machine can be monitored in an integrated fashion. Not all tools allow for this and you need to be very explicit as to the level of detail you are interested in.
Will it report appropriate to my requirements and in a way which allows for easier identification of system bottlenecks? This is a mix of objective and subjective items. You have not indicated what level of reporting you need
Does my user community have the skills to use the tool? Get the top three right and miss this one and even a free as in beer tool goes to a negative ROI almost immediately.
It's time to button up the requirements or just hire a firm with a set of tools included to do the job.
Related
In the attempt to design & implement & test a distributed capabilities system, Remote Promises[1][2][3], bit identical between Squeak & Java, there are shortcomings. I am seeking work-arounds.
With Remote Promises, proxies can change state, which changes the class implementing the proxy. In Squeak this is done with #becomeForward:, while in Java, it requires a secondary proxy, one that can change it's implemention. This does work.
Exceptions should be non-blocking to allow the event loop to continue, yet also display the problem stack for debugging, out of a quarantine. This is good in Squeak but an open issue with Java. I suppose the answer is do all your logging and then close the exception, allowing the event loop to proceed: it is server-style log debugging.
Using a meta repository, it should be possible to demand load consumers of a particular event type. Dynamically load the latest released code into the consumer servers and spread out the load to speed up the throughput. Update the system at runtime for continuous, seemless operations. I suppose the solution here is to build a dynamic jar classLoader system. Are there any examples of this? An Apache project perhaps?
Remote Promises in Squeak
Cryptography in
Squeak
Remote Promises in
Java, called Raven
Use cloud technologies made for that kind of usecases
I would say that in today world, to get the latest version of a code, you don't use a class loader or any advanced capability of your programming langage. You would user likely some kind of cloud service.
That's may be serverless cloud implementation or a container/kubernetes (https://kubernetes.io/) implementation. You can then perfectly when the new release is loaded, control if you want to do Canary, Blue/Green or progressive rollout or even implement your own strategy.
Because it would work with containers, that would be fine whatever the langage be it C++, java, python, shell, Squeak or anything.
That layer would also provide auto scaling of your various services, redundancy and load balancing and distribute the workload on your cluster.
You can go to the next step with gitops. A PR merge in git automatically trigger the load of the new version in production (https://www.weave.works/technologies/gitops/)
Dynamically loading of jars in Java
Still for sure java thanks to its class loaded API allows to load classes dynamically. This is what web servers are doing and several implementations of that do exist like OSGI or check the response of dimo414.
Conclusion
It would seems that the java route make more sense for a generic plugin system like the one of Eclipse (OSGI), and that the containers solution make more sense for a globally distributed system, auto scaling & resiliance in clusters.
Kubernetes scale to thousand of nodes and provides a whole echosystem to deal with distributed system and it can scale and operate any linux or windows process. This is the de-facto standard pushed by Google and used by thousand of companies over the world.
demand load consumers of a particular event type.
This is typically done via the ServiceLoader API. See the AutoService project to simplify working with services.
This may not be what you need; your question is still very broad, and there are many plausible approaches. Searches for [dynamically load jars] finds existing posts like Load jar dynamically at runtime? that may be of interest.
Out of the box AI works very well when it comes to correlate the web requests and its dependencies, events etc.
But assuming I have a long running background job, I would like to split its tracking into smaller pieces, units of work or spans, all within the same parent (and this division could go further too). Just like that image
Can I somehow do it in code?
A new SDK called Microsoft.ApplicationInsights.WorkerService was released very recently which is best suited for these type of background jobs, and non-HTTP workloads like messaging, console applications etc. This is the blog announcing the Worker service SDK, with detailed instructions on onboarding different types of supported scenarios with examples here.
However, while this is a .Net core solution, I can check with our internal Teams about the available options for Java applications and get back as I have more information. To pursue this further, please help me with the following additional details about your app environment from your end:
What Azure services do you use in this application's ecosystem? (VMs / DBs / App Services?)
What is the framework used in code?
Are there any Load Balancers or PaaS services also involved? If yes, how is their performance measured? Where are their logs configured?
This information would certainly help us understand your setup better. Thanks!
I am working on salesforce UI’s.
How can I ensure that whatever UI’s created are able to handle a load.
Do I need to do performance testing?
Please suggest tools , process so that 500 user load / performance testing if any?
Tool choice depends on many factors, i.e.
ability to integrate flawlessly into your existing SDLC (i.e. if you have Java stack a Python-based tool would be an "alien)
support of all required use cases (nearly all load testing tools are suitable for web applications testing, however when it comes to more "exotic" tasks like communicating with database, WebSockets, handle AJAX requests, etc.) the choice might be less wide
bugdet (there are some proprietary solutions which cost millions of coins and free/open source options)
skills (the tool might require coding skills which you and your team may or may not have)
You can find quite a list for instance here: http://www.opensourcetesting.org/category/testing-tools-overview/performance/?menu-page=overview
Given you have java tag I would recommend to stick to one of the following:
The Grinder
Gatling
Apache JMeter
Check out Open Source Load Testing Tools: Which One Should You Use? article which highlights main features of the above tools followed by comparison matrix, sample scripts, reports, etc., hopefully it will help you to choose the right option.
if you needed to build a highly scalable web application using java, what framework would you use and why?
I'm just reading thinking-in-java, head first servlets and manning's spring framework book, but really I want to focus on highly scalable architectures etc.
would you use tomcat, hibernate, ehcache?
(just assume you have to design for scale, not looking for the 'worry about it when you get traffic type responses)
The answer depends on what we mean by "scalable". A lot depends on your application, not on the framework you choose to implement it with.
No matter what framework you choose, the fact is that the hardware you deploy it on will have an upper limit on the number of simultaneous requests it'll be able to handle. If you want to handle more traffic, you'll have to throw more hardware at it and include load balancing, etc.
The part that's pertinent in that case has to do with shared state. If you have a lot of shared state, you'll have to make sure that it's thread safe, "sticky" when it needs to be, replicated throughout a cluster, etc. All that has to do with the app server you deploy it to and the way you design your app, not the framework.
Tomcat's not a "framework", it's a servlet/JSP engine. It's got clustering capabilities, but so do most other Java EE app servers. You can use Tomcat if you've already chosen Spring, because it implies that you don't have EJBs. Jetty, Resin, WebLogic, JBOSS, Glassfish - any of them will do.
Spring is a good choice if you already know it well. I think following the Spring idiom will make it more likely that your app is layered and architecturally sound, but that's not the deciding factor when it comes to scalability.
Hibernate will make your development life easier, but the scalability of your database depends a great deal on the schema, indexes, etc. Hibernate isn't a guarantee.
"Scalable" is one of those catch-all terms (like "lightweight") that is easy to toss off but encompasses many considerations. I'm not sure that a simple choice of framework will solve the issue once and for all.
I would check out Apache Mina. From the home page:
Apache MINA is a network application
framework which helps users develop
high performance and high scalability
network applications easily. It
provides an abstract · event-driven ·
asynchronous API over various
transports such as TCP/IP and UDP/IP
via Java NIO.
It has an HTTP engine AsyncWeb built on top of it.
A less radical suggestion (!) is Jetty - a servlet container geared towards performance and a small footprint.
The two keywords I would mainly focus on are Asynchronous and Stateless. Or at least "as stateless as possible: Of course you need state but maybe, instead of going for a full fledged RDBMS, have a look at document centered datastores.
Have a look at AKKA concerning async and CouchDB or MongoDB as datastores...
Frameworks are more geared towards speeding up development, not performance. There will be some overhead with any framework because of use cases it handles that you don't need. Granted, the overhead may be low, and most frameworks will point you towards patterns that have been proven to scale, but those patterns can be used without the framework as well.
So I would design your architecture assuming 'bare metal', i.e. pure servlets (yes, you could go even lower level, but I'm assuming you don't want to write your own http socket layer), straight JDBC, etc. Then go back and figure out which frameworks best fit your architecture, speed up your development, and don't add too much overhead. Tomcat versus other containers, Hibernate versus other ORMs, Struts versus other web frameworks - none of that matters if you make the wrong decisions about the key performance bottlenecks.
However, a better approach might be to choose a framework that optimizes for development time and then find the bottlenecks and address those as they occur. Otherwise, you could spin your wheels optimizing prematurely for cases that never occur. But that probably falls in the category of 'worry about it when you get traffic'.
All popular modern frameworks (and "stacks") are well-written and don't pose any threat to performance and scaling, if used correctly. So focus on what stack will be best for your requirements, rather than starting with the scalability upfront.
If you have a particular requirement, then you can ask a question about it and get recommendations about what's best for handling it.
There is no framework that is magically going to make your web service scalable.
The key to scalability is replicating the functionality that is (or would otherwise be) a bottleneck. If you are serious about making your service, you need to start with a good understanding of the characteristics of your application, and hence an idea of where the bottlenecks are likely to be:
Is it a read-only service or do user requests cause primary data to change?
Do you have / need sessions, or is the system RESTful?
Are the requests normal HTTP requests with HTML responses, or are you doing AJAX or callbacks or something.
Are user requests computation intensive, I/O intensive, rendering intensive?
How big/complicated is your backend database?
What are the availability requirements?
Then you need to decide how scalable you want it to be. Do you need to support hundreds, thousands, millions of simultaneous users? (Different degrees of scalability require different architectures, and different implementation approaches.)
Once you have figured these things out, then you decide whether there is an existing framework that can cope with the level traffic that you need to support. If not, you need to design your own system architecture to be scalable in the problem areas.
If you are able to work with a commercial system, then I'd suggest taking a look at Jazz Foundation at http://jazz.net. It's the base for IBM Rational's new products. The project is led by the guys that developed Eclipse within IBM before it was open-sourced. It has pluggable DB layer as well as supporting multiple App Servers. It's designed to handle clustering and multi-site type deployments. It has nice capabilities like OAuth support and License management.
In addition to the above:
Take a good look at JMS (Java Message Service). This is a much under rated technology. There are vendor solutions such as TibCo EMS, Oracle etc. But there are also free stacks such as Active MQ.
JMS will allow you to build synch and asynch solutions using queues. You can choose to have persistent or non-persistent queues.
As others already have replied scalability isn't about what framework you use. Sure it is nice to squeeze out as much performance as possible from each node, but what you ideally want is that by adding another node you scale your app in a linear fashion.
The application should be architected in distinct layers so it is possible to add more power to different layers of the application without a rewrite and also to add different layered caching. Caching is key to archive speed.
One example of layers for a big webapp:
Load balancers (TCP level)
Caching reverse proxies
CDN for static content
Front end webservers
Appservers (business logic of the app)
Persistent storage (RDBMS, key/value, document)
I'm building a VERY simple web hosting service that will cater to the ma and pa type small business.
Now my dilemma is whether I should built it from scratch or use an existing CMS. The CMS needs to be customisable, as I wish to build my own client. I only want the user to have to put a title and content. Everything else will be hidden from the end user.
Later on I want to be able to associate my own custom built web applications to a given page, so the CMS needs to be able to cater for that.
The other requirement is that it needs to be Java based (Groovy acceptable).
Any ideas?
Even a "simple" CMS is a fairly complicated application once you start doing things like user authentication, security, scalability, etc. There are many many things you can get wrong. The major CMSes out there suffer from lots of maintenance problems and security bugs and there are a fair number of talented people working on them. If you think your CMS will be different, I suggest that it will not be. You will need to update and maintain it constantly. If you use an off-the-shelf solution you should be able to benefit from the work those other developers are doing. If you want to write your own code, consider customizing an off-the-shelf CMS or contributing new features or bug fixes.
Own CMS:
Total control over the features
Low cost
Easy for you to understand
Only you understand it and can fix it
No bug fixes from other developers
Off-the-shelf CMS:
You can simply install it, then concentrate on adding value
Steeper learning curve
Lots of hosted solutions and online help
Lots of people can admin it if you're not around
Bug fixes and security updates are released by the vendor
More limited in terms of customization, etc
Someone needs to keep on top of the updates and install them, or else the customer might be victim to a worm such as the WordPress worms. With your own CMS it's less likely that someone will create a worm just for you. (But your customers are still at risk of other security problems).
Consider the typical security issues that face every website faces: XSS, CSRF, SQL Injection, configuration errors, loose security, session hijacking, parameter validation errors, race conditions, etc. You need to handle all those cases, but the CMS vendors are already doing that for you.
As for your Java requirement, the Resin web server ships with a PHP interpreter which might allow you to deploy a PHP app in Java should you choose a PHP CMS. It should also be possible to port the PHP engine to another servlet container in a few hours (I think it's GPL).
You should definitely learn an existing CMS. As much as creating one from scratch will fit your needs. An existing CMS with a live community (such as drupal or django) will have sold foundations, based on the accumulated experience of others.
Doing so will help skip the common mistakes of creating a CMS, and will allow you to focus on functionality and making your product, versus building a framework.
Check out Alfresco Community Edition, it's java based and it's the first one to be compliant with CMIS 1.0. You can build up a web client/application on it. It has ftp, webdav, cifs, http interface.
Liferay? You'll be up and running in less than 5 minutes.
Java based, Supports Groovy
If you want to diy, take a look at CMIS. (Forget it, don't diy)
I can't imagine there isn't an existing CMS that would fit those requirements. Won't do the research for you though ;-)
If you want to figure out some Java EE technology aspects or advantages/disadvantages it is better to write your own CMS just for practice.
But before you do this go through some of the existing CMS to create list of pros and cons.
All existing CMS/frameworks/whatever can't fit well your own problem but instead they are often too big to cover as many cases as the can.
One more thing to think about. Do you want your CMS to work great at Google App Engine?