I´m trying to run JMeter tests directly from my Java code. The load should be produced via a JMeter server. In the code, I use a ClientJMeterEngine instance that connects to the JMeter daemon running on the server. This works so far, I can start and run my test. What I currently don´t understand is how to get the results from the test run after it has completed. My current code looks like this:
// create ClientJMeterEngine instance
ClientJMeterEngine jmeter = new ClientJMeterEngine("myHost:4000");
// load test
File file = new File("C:/myTest.jmx");
// configure test
HashTree testPlanTree = SaveService.loadTree(file);
jmeter.configure(testPlanTree);
// run test
jmeter.runTest();
If I run the test using the StandardJMeterEngine object, I use a ResultCollector to get the results from my test runs. This works as expected. When I try to run the tests using the described server-based approach, I get the following exception on the machine where the server is running:
2016/03/15 09:24:35 ERROR - jmeter.samplers.BatchSampleSender: testEnded(host) java.rmi.ConnectException: Connection refused to host: myHost; nested exception is:
java.net.ConnectException: Connection refused
at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:631)
at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:228)
at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:214)
at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:142)
at java.rmi.server.RemoteObjectInvocationHandler.invokeRemoteMethod(RemoteObjectInvocationHandler.java:238)
at java.rmi.server.RemoteObjectInvocationHandler.invoke(RemoteObjectInvocationHandler.java:190)
at com.sun.proxy.$Proxy1.processBatch(Unknown Source)
The server thus wants to call back to the client from which I start the test run. It fails, because I don´t have anything waiting for the remote call. I´ve worked my way through the JMeter source, but haven´t found an answer, what I need to do on my Java side to make this work.
EDIT
In the meantime I´ve made some progress. What I´ve found is the following:
The client thread must be kept in state running, so the ResultCollector on the client can receive the results from the server. I now do this by "busy waiting" in my client Java code.
The issue now is to know when the test execution on the server is finished, so I can also finish the client thread. The ResultCollector does have the capabilities to recognize the test is finished, but no public methods for this.
The solution would be to extend the ResultCollector and override the testEnded method: This method sets a public boolean variable "testFinished" to true.
Though this feels a little bit "hacky". Does anybody have a better solution here?
I had the same problem and tried your solution with overriding testEnded() method of ResultCollector class. My test was endlessly waiting for response from remote JMeter slave server.
The actual problem is that remote server - Slave - succeed in replying of requests from Master at early stage. However, the connection dropped at some point and the test finished without receiving any report from a Slave. Slave is unable to start a new TCP connection, and can't send results when it is ready with his testing report as its hostname (IP) hasn't been set/sent to Master correctly.
Therefore, the solution is to start a Slave with the correct parameter in command line or in the jmeter.properties.
I started my slave like this:
c:\JMeter_new\apache-jmeter-3.1\bin>jmeter-server -Djava.rmi.server.hostname=IP
where IP is of the Slave
PS If you run tests in GUI, run Master with the parameter as well:
D:\JMeter_new\apache-jmeter-3.1\bin>jmeterw -Djava.rmi.server.hostname=IP_of_master
Related
Jmeter I keep getting: 'java.net.ConnectException: Connection timed out: connect' ?
I have created a load test which tests a specific url at 200 users
when running the load test for x1 iteration i keep seem to be getting: Connection timed outs?
I have made the following changes listed here: https://msdn.microsoft.com/en-us/library/aa560610(v=bts.20).aspx
But the issue is still there:
[1
You most probably don't have access to the target host from where you test.
Did you configure proxy as your web browser is probably configured
http://jmeter.apache.org/usermanual/get-started.html#proxy_server
But if failure is partial, then you server might be overloaded and rejecting some requests.
My expectation is that "problematic" requests are simply not able to finish in 20 seconds (most probably you have modified Connect timeout and set this value in HTTP Request or HTTP Request Defaults)
20 seconds looks like a long response time to me so your finding indicates application under test performance problem.
Going forward if you would like to see more "human readable" message in the results file switch to Duration Assertion instead of setting timeouts on protocol level
See How to Use JMeter Assertions in Three Easy Steps article for more information on conditionally failing JMeter requests.
Please check client configuration from where you are running your tests. It might be like your client system is not able to handle 200 threads. Do the test iteration wise means try with 10, 50 , 70 and so on. Check from which iteration onwards you are getting the error. It is also advisable not to include the listeners during load testing.
Please check the best practices for load testing using jmeter.
http://jmeter.apache.org/usermanual/best-practices.html
For my tests I am using a test Zookeeper server but I would like to be able to wait until the server is fully started (since I am starting it as part of the test init process).
How is it possible to cleanly check that a (Test) Zookeeper server is correctly started using Curator? Some form of ping/etc?
I managed to find the answer and wanted to share.
Curator has a method blockUntilConnected which will wait until it gets a connection from Zookeeper.
CuratorFramework curator = CuratorFrameworkFactory.newClient("localhost:" + TestConstants.TEST_ZOOKEEPER_PORT, new RetryOneTime(100));
curator.start();
curator.blockUntilConnected();
I'm running a Java server (Jetty, to be specific) on an AWS EC2 instance using WebSocket to connect to a client's browser. When I do this locally (hosting the server on my computer, not AWS), it runs fine. However, when I move the code to an EC2 instance, I get the following error message on the client-side:
WebSocket connection to 'ws://Elastic_IP:8080/?username=name_of_user' failed: Error during WebSocket handshake: Unexpected response code: 500
I made sure that the EC2 instance will accept traffic on port 8080.
On the server-side, I'm getting many java.lang.NoClassDefFoundError when the connection is attempted. I do not get these error when I run it locally. Perhaps there's an issue when I'm compiling on the EC2 instance, however it does compile without error. I'm compiling and running the code using Eclipse locally, but I'm compiling and running the code on EC2 by hand (javac with lots of classpaths). It's likely that I made an error when compiling by hand, but I'm not sure what the error could be.
Any help would be greatly appreciated.
EDIT
After a little trouble-shooting on my own, I realized that JSON.ParseException was the source of issue. After I removed all calls to this class from the server code, the handshake completed and I was able to establish a connection between the server and the client.
However, I am now running into the following error when I receive a message from the client:
WARN:MyWebSocketHandler:qtp990368553-16: Unhandled Error (closing connection)
java.lang.RuntimeException: Cannot call method public void
MyWebSocketHandler#onMessage(org.eclipse.jetty.websocket.api.Session, java.lang.String) with args:
[org.eclipse.jetty.websocket.common.WebSocketSession, java.lang.String]
It seems that I defined the argument to be org.eclipse.jetty.websocket.api.Session, but during runtime the argument is actually org.eclipse.jetty.websocket.common.WebSocketSession. Any ideas on how this is happening or which one (Session vs WebSocketSession) I should use? The only capability I need is to send strings between the server and the client.
I figured out a possible solution to my problem. Through Eclipse, I can Export the Java project to a runnable jar ("packing" the libraries into the jar). Then running it with java -jar <jar_filename> will work on the server and function the same as on the local machine. However, I've notice some performance issues (slow start), so I do not think this is the best solution, however it is a solution.
I am using chrome driver 2.1 and Google chrome 28.
Selenium standalone server 2.31. I run a set of 350 test and at the end 25 chrome-driver instances are running.
I don't know what test framework are using, but for me it happened with Selenium and MSTest. The problem was that the [TestCleanup] attribute is not executed if something fails while the code from [TestInitialize] is running.
Because of this, since I was calling driver.Quit() in my [TestCleanup] I ended up having a lot of server instances opened, and even managed to deplete the server machine's memory after some huge UI changes in our app.
It's not just in case of chrome driver. It is the same with firefox driver.
If you forgot to use server.stop() or your program interrupted in between, you can use this port to start your server from next time
import org.browsermob.proxy.ProxyServer;
Import the above.
String sePortNumber = System.getProperty("WEBDRIVER_SERVER_PORT_NUMBER");
if (sePortNumber == null) {
sePortNumber="0";
}
ProxyServer server = new ProxyServer(Integer.parseInt(sePortNumber));
server.start();
You will get a random unused port every time.
Background/Setup:
I have a bunch of test classes in JUnit. All configured through Maven and Eclipse both. Each test class instantiates a RemoteWebDriver instance and quits it in the tearDown method after the test class has finished executing.
When i try to run all the tests by doing a "mvn clean install", some of the tests execute just fine but the rest of them fail because of the following exception (stacktrace):
Tests in error:
com.tagged.qa.selenium.tests.gifts.GiftsPageTest: Error communicating with the remote browser. It may have died.(..)
com.tagged.qa.selenium.tests.tags.TagsPageTest: Error communicating with the remote browser. It may have died.(..)
addFriendsTest(com.tagged.qa.selenium.tests.friends.FriendsTest): Error communicating with the remote browser. It may have died.(..)
deleteFriendsTest(com.tagged.qa.selenium.tests.friends.FriendsTest): Error communicating with the remote browser. It may have died.(..)
com.tagged.qa.selenium.tests.statusupdates.StatusUpdatesTest: Error communicating with the remote browser. It may have died.(..)
com.tagged.qa.selenium.tests.comments.CommentsTest: Error communicating with the remote browser. It may have died.(..)
com.tagged.qa.selenium.tests.search.SearchPageTest: Error communicating with the remote browser. It may have died.(..)
com.tagged.qa.selenium.tests.homepage.HomePageTest: Error communicating with the remote browser. It may have died.(..)
com.tagged.qa.selenium.tests.wink.WinkPageTest: Error communicating with the remote browser. It may have died.(..)
com.tagged.qa.selenium.tests.profile.ProfilePageTest: Error communicating with the remote browser. It may have died.(..)
com.tagged.qa.selenium.tests.footerpagestests.TermsOfServiceTest: Error communicating with the remote browser. It may have died.(..)
Tests run: 18, Failures: 0, Errors: 11, Skipped: 0
Checking the logs in target/surefire-reports/ for individual tests that have failed leads me to this:
org.openqa.selenium.remote.UnreachableBrowserException: Error communicating with the remote browser. It may have died.
Caused by: org.openqa.selenium.WebDriverException: Session ID may not be null.
While the test execution was going on, i noticed that the browser (firefox in my case) was trying to open but quitting in a second. Selenium server logs that there is no session id. It tries for a couple of times and quits trying. After which all other tests in the pipeline fail due to the same reason.
What's baffling is the fact that when i try to run these tests individually, this problem does not occur but when i'm trying to run all of them together using maven, this happens consistently. Help please?
I don't know if this will help, but I couldn't get selenium working via maven with any version of firefox beyond 3.x (from memory I used 3.18). I got your symptoms when using a later versions.
I tried everything, but conceded defeat and simply installed and lived with the older version.
At least our selenium tests worked and we got on with doing real work.
I'm not sure, but usually when I see things failing when I run a group of tests, and passing when I run a single test, the culprits are usually either:
1) threading problem. Are these tests multi-threaded? If so, maybe there is some kind of resource contention.
2) tearDown problem. Is there something happening in tearDown which is leaving the browser in a bad state such that the new browser can't start up cleanly?
Sorry I couldn't be of more help, I don't use Maven and Eclipse to drive my Selenium tests.
Had the same problem in this code of mine.
WebDriver augmentedDriver;
if(BrowserConfig.getHubURL().equalsIgnoreCase("none"))
augmentedDriver = getDriver();
else augmentedDriver = new Augmenter().augment(getDriver());
LOGGER.info("Just before capture: ");
LOGGER.info(augmentedDriver.toString());
byte[] screenshot = ((TakesScreenshot) augmentedDriver).getScreenshotAs(OutputType.BYTES);
Figured out that driver.quit() was being called in o flow before the getscreenshot call. This b