I am trying to run jenkins job for my automation. Here I am using cucumber Bdd. While running locally in single thread, test cases are working good. But while building the job in jenkins using multiThreading, test cases are failing randomly. Once one is passed , in the successive build it is getting failed. Most of them are having the below error. It seems like there is some server issue that page is not loading in the browser while running the test cases. Can you suggest any way out to run this successfully?
My whole test cases are so large in size that if not run in multiThread it will take 6 to 7 hrs . The server is not supporting such long time. It is getting socket timeout after half of the run(after 3 hrs.)
org.openqa.selenium.TimeoutException: Course wasn't loaded after 30
seconds. Build info:
version:'2.53.1',revision:'a36b8b1cd5757287168e54b817830adce9b0158d',time:
'2016-06-30 19:26:09' System info: host: 'jenkins-50a5c8e3',
ip:'100.64.111.138',os.name:'Linux',os.arch:'amd64',os.version:'2.6.32-5-xen-amd64',java.version:'1.8.0_40'Driver
info:driver.version:unknown
This question is a bit vague so the best I can do is give a vague answer of what you might want to look into.
Make sure your tests can run in isolation You mentioned it works in a single thread. If your tests don't each have isolated data (course, user, whatever) then it's really easy to have one test do something like remove a course while another test is trying to access it if you run them in parallel.
Timing issues If you have timing issues in your tests, you probably need more robust explicit waits.
Make sure you properly dispose of your driver Preferably in a finally block. You need to make sure that no matter what path your tests take, the driver will get closed. You might want to start a strategy as well to clean up stranded driver instances, restart nodes, etc...
Related
Why do some threads not close/ stop within my session of JMeter?
I have a simple https request to a URL, 1500users, 20second ramp up, 5 iteration
For some reason the threads freeze when ramping down, for example 1500users would ramp down but when reach around the 30mark the 30 threads freeze and dont stop, any ideas?
Don't ever run your JMeter tests in GUI mode, use GUI only for tests development and/or debugging. When it comes to load test execution make sure you run your test in command-line non-GUI mode.
Also make sure you are following JMeter Best Practices and recommendations from 9 Easy Solutions for a JMeter Load Test “Out of Memory” Failure article.
We have a weird problem. We are using an automatic test tool. The DSL was implemented in Scala. The system which we test with this tool was written in Java, and the interface between the two components is RMI. Indeed, the interface part of the automatic test tool is also Java (the rest is Scala). We have the full control of the source code of these components.
We already have at the magnitude of thousand test cases. We execute these test cases automatically once every night, using Jenkins on a Linux server. The problem is that we sporadically receive a java.lang.NoClassDefFoundError exception. This typically happens when trying to access a Java artifacts from a Scala code.
If we execute the same test case manually, or check the result of the next nightly run, then typically the problem solves automatically, but sometimes it happens again in a completely different place. In case of some runs no such problem appears at all. The biggest problem is that the error is not reproducible; furthermore, as it happens in case of an automatic run, we have hardly any information about the exact circumstances, just the test case and the log.
Has somebody already encountered with such a problem? Do you have any idea how to proceed? Any hint or piece of information would be helpful, not only the solution of the problem. Thank you!
I found the reason of the error (99% sure). We had the following 2 Jenkins jobs:
Job1: Performs a full clean build of the tested system, written in Java, then performs a full clean build of the DSL, and finally executes the test cases. This is a long running job (~5 hours).
Job2: Performs a full clean build of the tested system, and then executes something else on it. The DSL is not involved. This is a shorter job (~1 hour).
We have one single Maven repository for all the jobs. Furthermore, some parts of the tested component is part of the interface between the two components.
Considering the time stamps the following happened:
Job1 performed the full build of both components, and started a test suite containing several test cases, which execution lasts about half an hour.
The garbage collector might swept out the components not used yet.
Job2 started the build, and it also rebuilt the interface parts, including the one swept out by garbage collector of Job1.
Job1 reached a test case which uses an interface component already swept out.
The solution was the following: we moved Job2 to an earlier time; now it finishes the job before Job1 starts the tests.
I have been writing selenium java testing script for a little while and recently discovered some issue when i am writing script for some relatively complex webpages based for IE, Firefox and Chrome.
1) when script run in Firefox, browser very often will become 'not responding', and "A script on this page may be busy, or it may have stopped responding. You can stop the script now, or you can continue to see if the script will complete." dialog shows. This almost never happens when user is manually performing the same operation.
2) something i found very strange is that although in my script the explicit wait are all ~20sec, when i run the test in Eclipse, it hangs for a long time and fail the test at last, and when finally test exit, it states that it has only been doing an explicit wait for some element for 20 sec. How come?
3) from a previous post in stackoverflow (When to use explicit wait vs implicit wait in Selenium Webdriver?), it seems that implicit is bad, so now i try to only use explicit wait in my script. Is it a correct way to write script with no implicit wait and add explicit wait for elements only when necessary? or to add explicit wait for every webelement?
4) the last issue is related to a post i read about long time ago, it states that for each webdriver running instance at least 2G memory need to be allocated. Right now when i am running the scripts, most of timing issues happen when humming noise of my computer is at its loudest. Is there a basic requirement on computing power. memory and more on machine that runs selenium testing? how do i know i am not running low on resource to run Selenium?
Could some one comment on some of the issues if you run into them before? Thanks,
I have an issue with Silk Test's Open Agent (Silk Test 2011).
I am using Silk4J for acceptance tests written in Java for a C++ client. It starts the tests properly but does not stop after the tests are completed. I have to do it manually so the job is completed in Jenkins.
Silk Test 2011 with Hotfix3 is installed on my Windows 7 SP1 (64bit).
The Silk Test 2011 release notes mention turning off user access control and I have done so.
I am having a hard time isolating the problem and hope somebody may have some ideas.
Silk Test's Open Agent should not be blocking your tests from completion.
In the past I've seen similar issues by the Java VM leaking sockets to their child processes, which then occasionally causes another processes to think the JVM was still alive.
You could try
Upgrading the involved JVM(s)
Upgrading Silk Test (2011 is ancient)
Take a close look at the network sockets the various processes have open, if the first two steps don't solve your issue, this might help you find out why.
I have written tests for my windows based application in TestNG. Sometimes my tests will result in BSOD of windows, so the test will run waiting for the Windows to be up till timeout and fail.
I have written a Listener for BSOD detection and handling, so that whenever the test starts the BSOD detector will start as listener and handle if the BSOD occurs. But still I have no way to notify my test that it has to halt. It still continues to execute till timeout.
How can solve this problem? Is there a way to notify the test to terminate it from Listener?
I think you may be out of luck.
The discussion at https://groups.google.com/forum/#!topic/testng-users/BkIQeI0l0lc clearly indicates that, as of August 2012 there was no way to stop a running test suite.
That combined with a much older post (https://groups.google.com/forum/#!topic/testng-users/FhC3rqs1yDM) from February 2010 suggests it has been a shortcoming of TestNG for a while.
The current TestNG documentation does not describe any methods for stopping the suite.
From what I can see in the source code (although I have not traced through all paths) at https://github.com/cbeust/testng/blob/master/src/main/java/org/testng/TestNG.java, it simply creates new tasks for each test and runs them, providing no way to break early.
Perhaps you could modify your tests to have access to some flag that indicates if subsequent tests should be skipped (triggered by, e.g., the BSOD detector). Then all of your subsequent tests are responsible for indicating themselves as skipped if this flag is set. It might not be ideal, but one way to think about it is: Any test should be skipped if a BSOD (or other terminal event) was detected prior to that test.
You could request this feature on the TestNG Google Group (linked above). If you are so inclined, you could also perhaps customize the TestNG source.