I'm really confused by the Selenium web site. At first they go to great lengths to explain Selenium IDE, but it outputs HTML test cases. But when you go to the Java documentation, it uses WebDriver which is pure java and your HTML test cases are useless. I found an export to JUnit feature from Selenium IDE but it doesn't use the WebDriver api, it wraps it inside the Selenium api, which doesn't even work. I get errors that I can only have one session at a time. I had only one test, made sure using netstat that I didn't have any other software listening on the port and disconnected the selenium instance. It just wouldn't work. Additionally the testcase extended from a deprecated class.
You cannot get back your test case to Selenium IDE from Java code so you can at that point throw the Selenium IDE away.
I converted the test case to pure WebDriver and I got it to work. So what is the recommended workflow for working with selenium and JUnit? Should I forget about Selenium IDE and recording actions in browser and just write everything in Java? Or is it still possible to use Selenium IDE somehow?
Having recently completed a project that used Selenium 2.0, I could find that the Selenium IDE is good only for prototying tests.
There are several drawbacks with the IDE that prevent it from being used to run Selenium tests. I could recall the following:
Typically you would want to run tests in a Suite. While the IDE does have this feature, I found that the IDE lacks a more important feature of running test setup and tear down scripts. This is trivial to achieve in JUnit/TestNG, but quite a pain with the recorded scripts in HTML. In short, the recorded tests aren't maintainable until you use a unit-testing library to run the tests from Java.
Data within the tests cannot be shared across tests; you will need to duplicate data in each test that requires it. This is expected when the tests are stored in a presentation language like HTML.
The default format of the exported tests does not use the page-object design pattern (which works very well for organizing Selenium tests). I didn't attempt creating my own format template for this, but this only convinced me that the best tests involving WebDriver and JUnit/TestNG are written by hand.
The optimal way of using the Selenium IDE is to create recordings of failed tests (by functional testers), instead of directly exporting the tests into your test suite. You could use the IDE to record the preliminary test so that the important aspects of the test (the assert/verify calls) are captured, and then rewrite it in your suite.
I use the IDE to create a "Work Flow Script", convert it into java code. Then I write everything from scratch in Java but with the info from the converted IDE script. That will have all ID's and so on but also in what order you have planned to "click" around, even some parts of it might be copied right off. Anyway it does speed things up a bit, but if you are using the webdriver it will complicate things a bit more and I have not yet moved over to the latest version.
Cheers
Stefan
The point is you can use either one, depending on your goal. In your case, WebDriver sounds like the way to go.
Selenium IDE is useful if you want to generate the HTML test cases.
Selenium WebDriver is useful for writing unit tests in Java (or other languages).
For a clear indication of this from the source, see the SeleniumHQ home page. It has a section on "Which part of Selenium is appropriate for me?", which answers your question.
Related
I have a website for testing and also need to Automate the website tests using selenium with java using BDD framework.
The website is built in English language and it supports 52 different languages too. The only difference will be URL change and text language changes.
What is the best way to Automate this type of website?
The requirement is, website should be automated and my selenium script should support the website when in all the languages. Is this a possible scenario?
Thought i have in my mind is put the list of URL's and texts for assertions of a website for all the languages,(Which is a gigantic process) in a repository and automate all the tests. Then Webdriver startup function will have a logic to identify for which language i am running the website tests for? and execute the tests accordingly. But finding for a better way
Couple of things.
Make different property files for each language. Actually if your site is localized it's already there, you just need to re-use that.
if you are using Maven you can choose this property file based on the variable you pass in maven command. which would be easy to configure in jenkins as well.
Use property files only for text verification for finding element use unique IDs, if element doesn't have that ask developers to use it rather than making any xpath/css which contains text.
I am developing a library that extends Selenium 2 with some custom commands. The library should be usable from Selenium's Java and Python bindings as well as in Selenium IDE. From my research, these three target bindings should cover at least 80% of all Selenium 2 scripts.
In order to implement my custom commands for Selenium IDE, I think I need to write a plugin for it in JavaScript.
My question is this: If I already have an implementation of my custom commands in JavaScript, is it safe to re-use this implementation for the Java- and Python bindings of my library?
I am thinking of an approach that injects the JavaScript implementation of my commands via WebDriver#executeScript. Here is a pseudocode implementation of what I am thinking of.
In Java:
public void fooJava() {
executeScript("Inject code.js");
executeScript("fooJavaScript();");
}
In code.js:
function fooJavaScript() {
// Implementation of command "foo" from Selenium IDE plugin.
}
So, to execute my custom command fooJava() in Java, my library's code.js would be injected into the browser via executeScript. This would contain a JavaScript implementation of foo, say fooJavaScript. In a next executeScript call, this fooJavaScript would then be called.
While this approach would prevent me from having to implement my custom commands three times (Java, Python, Selenium IDE), I have a few concerns:
When I inject my code.js, am I in danger of destroying global state of the web site?
To which extent can I rely on JavaScript? Will it work if an alert dialog is present? In practice, how many of the drivers used with Selenium do not support JavaScript? Eg. HtmlUnit?
Will this work in all major browsers (somewhat recent versions of IE, Chrome, Firefox, Safari)?
Your real-life experiences with this would be much appreciated.
Principle states you should not be using JS as your testing mechanism if you are just delivering a payload with WebDriver.
WebDriver = integration testsJS = if you want unit tests
I don't know your use cases exactly, but:
If you're trying to run integration tests, stick with WebDriver to best simulate user interaction. You also avoid cross-browser JS issues in the future by relying on the WebDriver hooks to interact with the page, as in, you are better off relying on the community to provide reliable basic DOM interaction APIs for each browser. If you can't trigger test conditions with browser interactions, you're getting into unit/code testing territory instead of integration testing.
If you are trying to run the JS for sake of essentially testing a single function or piece of code rather than an integrated interaction, you are trying to run a unit test. Unit tests are best done in JS with something like Jasmine (name any framework here).
Reasoning:
Integration tests should be written to be as implementation independent as possible. You should not need to know a function name to trigger an integration test, since someone might change the function name in the future or restructure the code.
Since you are filling a QE/tester role, you do not want to be responsible for breaking integration tests when code changes - if you use this and are responsible, then you will need to change a test every time there is a code restructure.
Sources: Experience as a QE in 10,000+ employee software co.
At present I have a framework which support firefox browser only. Now I am going to enhance my frame work so that can any give your ideas to me by suggesting your thoughts.
At present I am using Selenium Java, Hybrid framework, maven setup, Webdriver surefire reports.
So
1) If I want to run test cases on multiple browsers and multiple version of browsers parallel. what is the best approach?
Thanks
R
Well you need to setup the grid for parallel runs.
You need to make your browser launch code depend on a configurable parameter.
You can either take it as a parameter of the testng xml or make it as a system argument.
Depending on the value of this parameter, your listeners or your launch code (I am hoping that would be isolated since u mention a framework is in place) should be able to use this parameter to make a decision on what browser/node to launch.
For parallel runs, it would be better if you can use some framework. We use testng as the framework which supports parallel runs with good results.
I know you dont want to hear it, but best way actually is trial and error
Your site is going to behave differently in different browsers. So you will have to update your script for almost every browser and you will have to do it manually
Browsing speed is going to be different. Maybe invest in Implicit wait into your script
And yes, you are going to hate it.
My "best approach" is:
Implement the script for one most used browser and automate as much as possible
In other versions do manual shakedown of most crucial functions
Communicate with business to write somewhere that some obscure browsers (like IE 6) are no longer supported so you have more time to polish the script
I have used Selenium to program some web utilities in an embedded system, but after adding the Java runtime the size of the deliverable increases by 30Mb. Is there some other tool that can do the same thing with less?
I am quite happy with it, but the increase in the size feels wrong.
Update: After reading the replies I have realized that HttpClient or HtmlUnit is what I need. Selenium uses HtmlUnit and as I am not testing as such HtmlUnit may be enough.
Since the answer turns me in a different direction am I supposed to close it?
You shouldn't be including Selenium or your Java-based Selenium test cases in your final deliverable. Can you find a way to package your final application such that Selenium isn't included?
Check Watij (watir for Java).
But your argument against selenium sounds strange wre you using it for tests? If so, your runtime shouldn't change. If not, perhaps you are using the wrong thing. You can see HtmlUnit instead.
I have to admit that I fell in love with Selenium for its record-and-play feature as well as the testcase generation functionality for those recorded actions from the IDE. But I am still hesitated to advance to the implementation stage because of the incidental details (e.g, locating the events with DOM, xpath..etc) that are built into the testcase during the recording, which could make the testcase failure prone whenever there is a html change once it's imported to the RC. I fully understand that it's a part of testers' jobs to adjust the expected results from time to time as part of the regression test, but I also do not wish the time spent on this is larger than the time that takes to do the manual test.
As far as I know Selenium with Robot framework has the keywords form of testcases. My guess is it allows us to extract the incidental details into various keywords, which could make the testcases being adjusted easier and are more maintainable. (Please correct me if I am wrong)
It will be appreciated to hear suggestions on how an effective UI automation environment should be setup. Should I just use Selenium RC or Selenium with Robot framework? And why?
Thanks in advance
You are absolutely right that incidental and often changing details in the produced scripts is the biggest problem of record-and-playback automation. You can obviously remove the details from the scripts after recording, but in my opinion it's better to build reusable libraries and code scripts manually from the start.
A good alternative for coding scripts using "real" programming languages is using some higher level automation framework such as Robot Framework that you mentioned. As you speculated, Robot's reusable keywords and also variables make extracting details away from tests very easy. The test cases in SeleniumLibrary's demo illustrates this very well and the demo also shows how to use Selenium through Robot.
You also asked about Sikuli. I've never used it myself but it sure looks interesting. You might be interested on this great how-to that explains how to use it through Robot Framework.
Our company is using Fitnesse, not Robot, to control Selenium however, we have the same problem. We switched from making assumptions about the DOM to only accessing elements by ID. Since this is cumbersome in Fitnesse we are currently working to add a Selenium backend to our own Framework (which previously only had backends for Java and Smalltalk).
So, by requiring that elements with certain ID's are present in the DOM we will of course break our tests if someone removes the elements from the page; however, we found that this behavious is very useful as this enforces the contract the tests made with the implementation and it is a good thing we find missing elements as soon as someone broke the implementation.
In addition, it is good practice to keep UI automation skin-deep: Only test what is present on the page with Selenium and test the business-logic by calling the underlying functions directly.