Develop Reference

API Perfromance testing tool - JMeter or JUnit?

i am working on some performance testing task. The main goal is to compare duration of old NCP protocol calls againts new REST API calls. I have this scenario:
Client has an authenticated session
Client access protected resource
I have to create two variants:
a) One-by-one variant: The question is: How long does it take to perform 2000 requests sent one by one?
b) Concurrent variant: The question is: How long does it take to perform 2000 Request which are sent concurrently (300 Threads ideal)
I dont know the best way to solve this problem. My idea is:
a) Creation of 2000 Http clients -> Each client sends HTTP Post with credentials in body -> Each client sends HTTP GET and get the response (I will measure the time between sending the GET request and getting a response for each iteration and Sum it.
b) Creation of 2000 Httpclients -> Use executor service with fixed thread pool (300) -> each thread will perform sending get request.
Is there any other way? I know that Jmeter is a great tool but i am not sure that this scenario could by performed on Jmeter. Thanks!

For the second variant: you need to determine what is you targeted throughput (TP). 2000 request per hour? Per minute? Per second? Once you get the TP, and a guesstimate for the scenario response time (RT), you could estimate the number of VUsers using the Little's Law. Alternatively, you can use a calculator to determine that number.
Jmeter provides a mechanism to submit this workload (scenarios) by using Arrivals Thread Group. This TG will instantiate the number of threads needed to sustain the targeted TP.
Be aware that there is possibility that you might not reach the TP goal due to:
the SUT does not have the capacity to handle the load
a bottleneck (resource saturation) somewhere in the environment
the client (JMeter) does have enough resources to produce the load

JUnit itself doesn't provide any multithreading logic, you will have to construct the HTTP requests yourself (or with a 3rd-party library like RestAssured) and then execute them using i.e. ExecutorService or jmh and then come up with something for results analysis.
JMeter has everything out of the box so you won't need to write a single line of code, reporting is also included, it might be not that CI friendly as JMeter .jmx scripts are XML but on the other hand you will get nice protocol metrics and ability to correlate increasing load with increasing response time

What's the proper heartbeat/keep-alive technology/layer for Java REST? Http? Tcp? Encoding: chunked?

The setup:
We have an https://Main.externaldomain/xmlservlet site, which is authenticating/validating/geo-locating and proxy-ing (slightly modified) requests to http://London04.internaldomain/xmlservlet for example.
There's no direct access to internaldomain exposed to end-users at all. The communication between the sites gets occasionally interrupted and sometimes the internaldomain nodes become unavailable/dead.
The Main site is using org.apache.http.impl.client.DefaultHttpClient (I know it's deprecated, we're gradually upgrading this legacy code) with readTimeout set to 10.000 milli-seconds.
The request and response have xml payload/body of variable length and the Transfer-Encoding: chunked is used, also the Keep-Alive: timeout=15 is used.
The problem:
Sometimes London04 actually needs more than 10 seconds (let's say 2 minutes) to execute. Sometimes it non-gracefully crashes. Sometimes other (networking) issues happen.
Sometimes during those 2 minutes - the portions of response-xml-data are being so gradually filled that there're no 10-second gaps between the portions and therefore the readTimeout is never exceeded,
sometimes there's a 10+ seconds gap and HttpClient times out...
We could try to increase the timeout on Main side, but that would easily bloat/overload the listener pool (just by regular traffic, not even being DDOSed yet).
We need a way to distinguish between internal-site-still-working-on-generating-the-response and the cases where it really crashed/network_lost/etc.
And a best thing feels to be some kind of heart-beat (every 5 seconds) during the communication.
We thought the Keep-Alive would save us, but it seems to only secure the gaps between the requests (not during the requests) and it seems to not do any heartbeating during the gap (just having/waiting_for the timeout).
We thought chunked-encoding may save us by sending some heartbeat (0-bytes-sized-chunks) to let other side aware, but there seems to be no such/default implementation of supporting any heartbeat this way and moreso it seems that 0-bytes-sized chunk is an EOD indicator itself...
Question(s):
If we're correct in assumptions that KeepAlive/ChunkedEncoding won't help us with achieving the keptAlive/hearbeat/fastDetectionOfDeadBackend then:
1) which layer such a heart-beat should be rather implemented at? Http? tcp?
2) any standard framework/library/setting/etc implementing it already? (if possible: Java, REST)
UPDATE
I've also looked into heartbeat-implementers for WADL/WSDL, though found none for REST, checked out the WebSockets...
Also looked into TCP-keepalives which seem to be the right feauture for the task:
https://en.wikipedia.org/wiki/Keepalive
http://tldp.org/HOWTO/TCP-Keepalive-HOWTO/usingkeepalive.html
Socket heartbeat vs keepalive
WebSockets ping/pong, why not TCP keepalive?
BUT according to those I'd have to set up something like:
tcp_keepalive_time=5
tcp_keepalive_intvl=1
tcp_keepalive_probes=3
which seems to be a counter-recommendation (2h is the recommended, 10min already presented as an odd value, is going to 5s sane/safe?? if it is - might be my solution upfront...)
also where should I configure this? on London04 alone or on Main too? (if I set it up on Main - won't it flood client-->Main frontend communication? or might the NATs/etc between sites ruin the keepalive intent/support easily?)
P.S. any link to an RTFM is welcome - I might just be missing something obvious :)

My advice would be don't use a heartbeat. Have your external-facing API return a 303 See Other with headers that indicates when and where the desired response might be available.
So you might call:
POST https://public.api/my/call
and get back
303 See Other
Location: "https://public.api/my/call/results"
Retry-After: 10
To the extent your server can guess how long a response will take to build, it should factor that into the Retry-After value. If a later GET call is made to the new location and the results are not yet done being built, return a response with an updated Retry-After value. So maybe you try 10, and if that doesn't work, you tell the client to wait another 110, which would be two minutes in total.
Alternately, use a protocol that's designed to stay open for long periods of time, such as WebSockets.

Take a look SSE
example code:
https://github.com/rsvoboda/resteasy-sse
or vertx event-bus:
https://vertx.io/docs/apidocs/io/vertx/core/eventbus/EventBus.html

How to make work HttpResponse faster?

I am working on a project in which "I have to get 4096 bytes of data to server" from "server" every "between 1-millisecond to 10-millisecond".But it's "taking too much time" i.e "around 300ms - 700ms" which causes my application to lose data.
I am using below snippet
HttpClient client = new DefaultHttpClient();
HttpPost request = new HttpPost("http://192.168.1.40/ping");
HttpResponse response = client.execute(request);
The HttpResponse is only taking too much time i.e around 300ms - 700ms.
How I can get response faster ?
Instead of this what else I can use to get a response from sever faster then this?
Please let me know any solution or way to solve it.
I have done google, gone through other ways like DataOutputStream and ByteOutputStream but no use of this, it also taking too much time then HttpResponse.
Help will be appreciated.

Before you can make the responses faster, you are going to need to investigate and understand why they are currently taking a long time. Roughly speaking, it could be:
the client side taking a long time to create the request and/or preocess the result (seems unlikely ...)
a slow android network protocol stack
a problem with your local networking (e.g. WiFi) or your telecoms provider
a congested / overloaded server or server-side network, or
something pessimal in the server implementation.
Do things like:
try the request from a web browser on a conventional PC and use the browser's web-developer stuff to try to tease out whether/why the request is taking a long time ...
look in the server-side logs and/or monitoring for request load and timing information
other suggestions please
Implementing SPDY might help, but it is unlikely to change response times in the order of 500ms to a couple of tens of milliseconds. The problem seems more fundamental than "HTTP is old and slow". And the same reasoning applies to all of the other suggestions that people have made.

This is not possible. You are recreating a connection every time.
You need to hold a persistent connection with the server. Try creating a persistent http connection.
If that doesn't work you can try sending raw udp packets (or anything else). It will be harder but it will take less time.

#sheldonCooper answer is right if the server enables SPDY. Also you can add Gzip compression. It has been added to all requests after GingerBread but you could add it for former SDK versions : http://android-developers.blogspot.fr/2011/09/androids-http-clients.html

Use SPDY protocol. This would improve your response time.
I think in your case you can use websockets so that you would not have to create a connection each time and the live connection is available every time.

Synchronous, Asynchronous and Command Client Requests with GWT and GAE

In designing my GWT/GAE app, it has become evident to me that my client-side (GWT) will be generating three types of requests:
Synchronous - "answer me right now! I'm important and require a real-time response!!!"
Asynchronous - "answer me when you can; I need to know the answer at some point but it's really not all that ugent."
Command - "I don't need an answer. This isn't really a request, it's just a command to do something or process something on the server-side."
My game plan is to implement my GWT code so that I can specify, for each specific server-side request (note: I've decided to go with RequestFactory over traditional GWT-RPC for reasons outside the scope of this question), which type of request it is:
SynchronousRequest - Synchronous (from above); sends a command and eagerly awaits a response that it then uses to update the client's state somehow
AsynchronousRequest - Asynchronous (from above); makes an initial request and somehow - either through polling or the GAE Channel API, is notified when the response is finally received
CommandRequest - Command (from above); makes a server-side request and does not wait for a response (even if the server fails to, or refuses to, oblige the command)
I guess my intention with SynchronousRequest is not to produce a totally blocking request, however it may block the user's ability to interact with a specific Widget or portion of the screen.
The added kicker here is this: GAE strongly enforces a timeout on all of its frontend instances (60 seconds). Backend instances have much more relaxed constraints for timeouts, threading, etc. So it is obvious to me that AsynchronousRequests and CommandRequests should be routed to backend instances so that GAE timeouts do not become an issue with them.
However, if GAE is behaving badly, or if we're hitting peak traffic, or if my code just plain sucks, I have to account for the scenario where a SynchronousRequest is made (which would have to go through a timeout-regulated frontend instance) and will timeout unless my GAE server code does something fancy. I know there is a method in the GAE API that I can call to see how many milliseconds a request has before its about to timeout; but although the name of it escapes me right now, it's what this "fancy" code would be based off of. Let's call it public static long GAE.timeLeftOnRequestInMillis() for the sake of this question.
In this scenario, I'd like to detect that a SynchronousRequest is about to timeout, and somehow dynamically convert it into an AsynchronousRequest so that it doesn't time out. Perhaps this means sending an AboutToTimeoutResponse back to the client, and force the client to decide about whether to resend as an AsynchronousRequest or just fail. Or perhaps we can just transform the SynchronousRequest into an AsynchronousRequest and push it to a queue where a backend instance will consume it, process it and return a response. I don't have any preferences when it comes to implementation, so long as the request doesn't fail or timeout because the server couldn't handle it fast enough (because of GAE-imposed regulations).
So then, here is what I'm actually asking here:
How can I wrap a RequestFactory call inside SynchronousRequest, AsynchronousRequest and CommandRequest in such a way that the RequestFactory call behaves the way each of them is intended? In other words, so that the call either partially-blocks (synchronous), can be notified/updated at some point down the road (asynchronous), or can just fire-and-forget (command)?
How can I implement my requirement to let a SynchronousRequest bypass GAE's 60-second timeout and still get processed without failing?
Please note: timeout issues are easily circumvented by re-routing things to backend instances, but backends don't/can't scale. I need scalability here as well (that's primarily why I'm on GAE in the first place!) - so I need a solution that deals with scalable frontend instances and their timeouts. Thanks in advance!

If the computation that you want GAE to do is going to take longer than 60 seconds, then don't wait for the results to be computed before sending a response. According to your problem definition, there is no way to get around this. Instead, clients should submit work orders, and wait for a notification from the server when the results are ready. Requests would consist of work orders, which might look something like this:
class ComputeDigitsOfPiWorkOrder {
// parameters for the computation
int numberOfDigitsToCompute;
// Used by the GAE app to contact the requester when results are ready.
ClientId clientId;
}
This way, your GAE app can respond as soon as the work order is saved (e.g. in Task Queue), and doesn't have to wait until it actually finishes calculating a billion digits of pi before responding. Your GWT client then waits for the result using the Channel API.
In order to give some work orders higher priority, you can use multiple task queues. If you want Task Queue work to scale automatically, you'll want to use push queues. Implementing priority using push queues is a little tricky, but you can configure high priority queues to have faster feed rate.
You could replace Channel API with some other notification solution, but that would probably be the most straightforward.

Measure Total Network Transfer Time from Servlets

How do i measure how long a client has to wait for a request.
On the server side it is easy, through a filter for example.
But if we want to take into accout the total time including latency and data transfer, it gets diffcult.
is it possible to access the underlying socket to see when the request is finished?
or is it neccessary to do some javascript tricks? maybe through clock synchronisation between browser and server? are there any premade javascripts for this task?

You could wrap the HttpServletResponse object and the OutputStream returned by the HttpServletResponse. When output starts writing you could set a startDate, and when it stops (or when it's flushed etc) you can set a stopDate.
This can be used to calculate the length of time it took to stream all the data back to the client.
We're using it in our application and the numbers look reasonable.
edit: you can set the start date in a ServletFilter to get the length of time the client waited. I gave you the length of time it took to write output to the client.

There's no way you can know how long the client had to wait purely from the server side. You'll need some JavaScript.
You don't want to synchronize the client and server clocks, that's overkill. Just measure the time between when the client makes the request, and when it finishes displaying its response.
If the client is AJAX, this can be pretty easy: call new Date().getTime() to get the time in milliseconds when the request is made, and compare it to the time after the result is parsed. Then send this timing info to the server in the background.
For a non-AJAX application, when the user clicks on a request, use JavaScript to send the current timestamp (from the client's point of view) to the server along with the query, and pass that same timestamp back through to the client when the resulting page is reloaded. In that page's onLoad handler, measure the total elapsed time, and then send it back to the server - either using an XmlHttpRequest or tacking on an extra argument to the next request made to the server.

If you want to measure it from your browser to simulate any client request you can watch the net tab in firebug to see how long it takes each piece of the page to download and the download order.

Check out Jiffy-web, developed by netflix to give them a more accurate view of the total page -> page rendering time

I had the same problem. But this JavaOne Paper really helped me to solve this problem. I would request you to go thru it and it basically uses javascript to calculate the time.

You could set a 0 byte socket send buffer (and I don't exactly recommend this) so that when your blocking call to HttpResponse.send() you have a closer idea as to when the last byte left, but travel time is not included. Ekk--I feel queasy for even mentioning it. You can do this in Tomcat with connector specific settings. (Tomcat 6 Connector documentation)
Or you could come up with some sort of javascript time stamp approach, but I would not expect to set the client clock. Multiple calls to the web server would have to be made.
timestamp query
the real request
reporting the data
And this approach would cover latency, although you still have have some jitter variance.
Hmmm...interesting problem you have there. :)