How do I set up Selenium to kill the test browser page on occasions where the test fails?
Currently, when running Selenium test cases and a test fails, the browser page stays open and that causes problems when a large number of tests is failing. Interestingly enough, it isn't the case when the test passes.
Any suggestion??
You should call selenium.stop() of course :) It sounds like you need a try/finally block
Its not quite that simple, in my experience anyway (I'm new to selenium and JUnit). It depends how the failure occurs and what you do with it. JUnit should automatically call tearDown() when a test fails, and selenium.stop() should be in this. However sometimes tearDown() isn't called, for instance when the error occurs in setUp(), or if you're doing something sill when a failure occurs.
Related
I have test automation framework with a page object model.
Scenario:
In my testng.xml I mentioned 3 tests (Test1, Test2, Test3) which are in same package. Each Test contains 3 methods
I ran my testng.xml file and say like in Test1 only 3rd method failed. So when I'm running testng-failed.xml it tries to run only 3rd method.
But in my case I need to run complete Test i.e Test1 again.
I ran testng-failed.xml from test-output folder and verified the result but no luck.
Expected:
If any method fails (from a Test) then complete Test should be executed
Actual:
After running testng-failed.xml only failed method from a class is executed.
It sounds like you have created a test suite where the tests rely upon earlier ones to get into a certain state. i.e. you cannot run Test3 on its own as it relies on Test1 to do something first.
This is a bad pattern to follow.
Each of your tests should be able to be run independently. If there is setup required, it should be carried out as part of the life-cycle for that individual test, meaning in the test itself, or during BeforeEach.
It may take a little more time for the test suite to execute, but you'll avoid the issue you're currently facing.
I don't think there is a direct way of doing this in TestNG, but I think this will help you getting the idea.
Try implementing ITestListener and override onFinish(ITestContext context) method.
Probably you can check if #test is failing then change the status of all the passed methods and rerun again by overriding retry method of IRetryAnalyzer.
There are so many posts about running JUnit tests in a specific order and I fully understand:
Tests should not order specific
and that the creators did this with no 1 in mind
But I have test cases that creates a bunch out output files. I need to capability to have one final test that goes and collects these files, zip it and emails it of to someone.
Is there a way to group JUnit tests together for me to have a "wrap up" group that goes and do this? Or is there a better way of doing this?
I am running these from Jenkins as a maven job. I could create another job that does just that based on the previous jobs output but would prefer if I can do it all in one meaning I would be able to run it everywhere even from my IDE.
Maybe the #After and #AfterClass annotations are what you are looking for:
#AfterClass
void cleanupClass() {
//will run after all tests are finished
}
#After
void cleanup() {
//will run after every test
}
However, I would consider handling this through Jenkins if possible. In my opinion the annotations above are for cleaning up any kind of setup that was previously done in order to do the testing.
Sending these files through email does not sound like part of the testing and therefore I would be inclined to keep it separated.
I guess the real problem is that you want the results and output of the tests sent via email.
Your suggestion of using a test for this threw me on the wrong track.
Definitely use some sort of custom Jenkins post hook to do this. There are some fancy plugins that let you code groovy which will do the trick.
Do not abuse a unit test for this. These (should) also run locally as part of builds and you don't want that email being sent every time.
My question is -Is it possible to run a suite of testcases in single instance of browser ie only need to login to application once and also this window should not get close even if any of the testcase in the suite gets fail.If yes then how can i implement .Thanks in advance.
I am guessing - What you are trying to achieve is - scenario/flow testing (in my words) and selenium / JUnit testcases are not suitable for this.
The way we achieve it in our project is to design the flow / Scenario tests as java classes ( not test cases ) and then run them through a JUnit TestCase. This way you create a Single instance of the browser in your test method and run detailed scenarios on it.
Any test asserts that fail needs to be manually designed and you will need to rely on custom exceptions to be thrown to fail the test.
The other answer pointed to by Tim has a problem that you need to assume the sequence / order of tests which is not guaranteed and may cause your tests to fail.
I am a developer temporarily tasked with helping the QA team with test automation using JUnit 4 and Selenium WebDriver. I am new to testing and test automation.
Looking at various WebDriver examples, a common pattern is to instantiate an implementation of WebDriver (like FirefoxWebDriver) in the #Before method, use the instance in #Test method to interface with the browser and driver.quit() in #After.
So if there are 5 #Test methods, the browser will be opened, test app initialized and browser closed 5 times.
My question is why open, initialize and close required for each test case? My guess is to prevent one test case failure from having a negative side effect on other tests. Is this correct? Are there any other reasons?
Great Question!
Coming from the OTHER side, i'm a QA automation test engineer, working with the Dev team on automation...
As #Prateek and myself have pointed out, there are a couple reasons why.
The two most obvious:
The main purpose of testing is to localize the error. If all the tests would run in one go and it fails, you would be unable to tell where it failed. ~Prateek
Multithreading is a reason why as well. Picture your company having thousands of regression tests that need to be ran. Having each test isolated, makes it so they are completely independent of other tests, making you able to potentionally run them all at the same time (assuming your app can handle it) which .. say each test takes 1 minute to run, 1x1000=1000 minutes to run each test sequentially as opposed to finishing them all in just 1 minute!
So you are correct in your assumtion here -
My guess is to prevent one test case failure from having a negative side effect on other tests. Is this correct?
But of course, there are more reasons than that alone.
There is one more reason, each time WebDriver is instantiated, a new profile is created which clears browser cache..Currently there is no other way to clear browser cache..
How do you manage resource clean up in JUnit4 (in a JUnitRunner / in a TestCase) when you interrupt the test in debug mode ?
I use Selenium WebDriver by implementing JUnit tests. When I run the test in debug, if I interrupt the test by clicking on the stop button, then my navigator is still open. I would like to close this navigator by calling a method. But I don't know where calling this method.
In fact, I've overrided BlockJUnit4ClassRunner to have my own Runner and I also have implemented a WebDriverTestCase. So I can do a lot of things.
I tried to implement my own SecurityManager and override the method checkExit(int status) but, when I interrupt my test in debug mode, the SecurityManager is not called (with a System.exit(), it works correctly).
As far as I know, there is no way to execute anything, when you stop a debug session.
The only solution to this kind of problem is: Do the tear down in the set up!
This means, you should check in your #Before method, bevor you create a new WebDriver, if an old browser process is still running, and if you find one, kill it. (See How to get a list of current open windows/process with Java? and Killing a process using Java)
Furthermore you probably also want to delete the webdriver directories in your temp directory.