Using the code below, i am trying to open a link page and then go to mobile section and sort the items on the basis of name order. now i want to check if the mobile devices are sorted by Name means alphabetically.
i tried to convert my List below to arraylist but not able to check if elements printed are in ascending order, kindly help
package selflearning;
import java.util.ArrayList;
import java.util.Collection;
import java.util.Collections;
import java.util.HashSet;
import java.util.List;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.support.ui.Select;
public class Guru99Ecommerce1 {
public static void main(String[] args) throws Exception {
System.setProperty("webdriver.gecko.driver","C:\\geckodriver\\geckodriver.exe");
WebDriver driver = new FirefoxDriver();
driver.get("http://live.guru99.com/index.php/");
String title=driver.getTitle();
String expectedTitle = "Home page";
System.out.println("The title of the webPage is " + title);
expectedTitle.equalsIgnoreCase(title);
System.out.println("Title is verified");
driver.findElement(By.xpath("//a[text()='Mobile']")).click();
String nextTitle = driver.getTitle();
System.out.println("The title of next page" + nextTitle);
String nextExpectedTitle = "pageMobile";
nextExpectedTitle.equalsIgnoreCase(nextTitle);
System.out.println("The next title is verified");
Select s = new Select(driver.findElement(By.xpath("//div[#class='category-products']//div/div[#class='sorter']/div/select[#title='Sort By']")));
s.selectByVisibleText("Name");
List<WebElement> element = driver.findElements(By.xpath("//div[#class='product-info']/h2/a"));
for(WebElement e: element)
{
String str = e.getText();
System.out.println("The items are " + str);
}
HashSet<WebElement> value = new
List<WebElement> list = new ArrayList<WebElement>(element);
list.addAll(element);
System.out.println("arrangement" + list);
}
}
The easiest way to do this is to just grab the list of products, loop through them, and see if the current product name (a String) is "greater" than the last product name using String#compareToIgnoreCase().
I would write some functions for the common tasks you are likely to repeat for this page.
public static void sortBy(String sortValue)
{
new Select(driver.findElement(By.cssSelector("select[title='Sort By']"))).selectByVisibleText(sortValue);
}
public static List<String> getProductNames()
{
List<String> names = new ArrayList<>();
List<WebElement> products = driver.findElements(By.cssSelector("ul.products-grid h2.product-name"));
for (WebElement product : products)
{
names.add(product.getText());
}
return names;
}
public static boolean isListSorted(List<String> list)
{
String last = list.get(0);
for (int i = 1; i < list.size(); i++)
{
String current = list.get(i);
if (last.compareToIgnoreCase(current) > 0)
{
return false;
}
last = current;
}
return true;
}
NOTE: You should be using JUnit or TestNG for your assertions instead of writing your own because it makes it much, much easier (and you don't have to write and debug your own which saves time). The code I wrote below is using TestNG. You can see how much shorter (and simpler) the code below is when using a library like TestNG.
String url = "http://live.guru99.com/index.php";
driver.navigate().to(url);
Assert.assertEquals(driver.getTitle(), "Home page");
driver.findElement(By.xpath("//nav[#id='nav']//a[.='Mobile']")).click();
Assert.assertEquals(driver.getTitle(), "Mobile");
sortBy("Name");
System.out.println(getProductNames());
System.out.println(isListSorted(getProductNames()));
Where getProductNames() returns
[IPHONE, SAMSUNG GALAXY, SONY XPERIA]
Related
Im trying to build a web crawler for my OOP class. The crawler needs to traverse 1000 wikipedia pages and collect the titles and words off the page. The current code I have will traverse a singular page and collect the required information but it also gives me the error code "java.lang.IllegalArgumentException: Must supply a valid URL:" Here is my crawlers code. Ive been using Jsoups libraries.
import java.util.HashMap;
import java.util.HashSet;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.jsoup.select.Elements;
public class crawler {
private static final int MAX_PAGES = 1000;
private final HashSet<String> titles = new HashSet<>();
private final HashSet<String> urlVisited = new HashSet<>();
private final HashMap<String, Integer> map = new HashMap<>();
public void getLinks(String startURL) {
if ((titles.size() < MAX_PAGES) && !urlVisited.contains(startURL)) {
urlVisited.add(startURL);
try {
Document doc = Jsoup.connect(startURL).get();
Elements linksFromPage = doc.select("a[href]");
String title = doc.select("title").first().text();
titles.add(title);
String text = doc.body().text();
CountWords(text);
for (Element link : linksFromPage) {
if(titles.size() <= MAX_PAGES) {
Thread.sleep(50);
getLinks(link.attr("a[href]"));
}
else {
System.out.println("URL couldn't visit");
System.out.println(startURL + ", " + urlVisited.size());
}
}
} catch (Exception e) {
e.printStackTrace();
}
}
}
public void PrintAllTitles() {
for (String t : titles) {
System.out.println(t);
}
}
public void PrintAllWordsAndCount() {
for (String key : map.keySet()) {
System.out.println(key + " : " + map.get(key));
}
}
private void CountWords(String text) {
String[] lines = text.split(" ");
for (String word : lines) {
if (map.containsKey(word)) {
int val = map.get(word);
val += 1;
map.remove(word);
map.put(word, val);
} else {
map.put(word, 1);
}
}
}
}
The Driver function just uses c.getLinks(https://en.wikipedia.org/wiki/Computer)
as the starting URL.
The issue is in this line:
getLinks(link.attr("a[href]"));
link.attr(attributeName) is a method for getting an element's attribute by name. But a[href] is a CSS selector. So that method call returns a blank String (as there is no attribute in the element named a[href]), which is not a valid URL, and so you get the validation exception.
Before you call connect, you should log the URL you are about to hit. That way you will see the error.
You should change the line to:
getLinks(link.attr("abs:href"));
That will get the absolute URL pointed to by the href attribute. Most of the hrefs on that page are relative, so it's important to make them absolute before they are made into a URL for connect().
You can see the URLs that the first a[href] selector will return here. You should also think about how to only fetch HTML pages and not images (e.g., maybe filter out by filetype).
There is more detail and examples of this area in the Working with URLs article of jsoup.
titles – is a string variable that contains the string values
List<String> titles = new WebDriverWait(driver, 20).until(ExpectedConditions.visibilityOfAllElementsLocatedBy(By.xpath("//div[#class='document-card__details']//h3/a"))).stream().map(element->element.getAttribute("innerHTML")).collect(Collectors.toList());
System.out.print("THE NAME ARE " + titles);
below is the x-path of a search box
driver.findElement(By.xpath("//div[#class='text-filter__input']//input[#placeholder='Search Title']"));
The action I want to perform – To take the random values from the titles[variable] and enter that in the search box.
Is there any way to do it?
Yes possible. Use java random class.
Random rand = new Random();
for (int i = 0; i < titles.size() ; i++) {
String titlesItem = titles.get(rand.nextInt(titles.size()));
driver.findElement(By.xpath("//div[#class='text-filter__input']//input[#placeholder='Search Title']")).sendKeys(titlesItem);
}
int length = 10;
boolean useLetters = true;
boolean useNumbers = false;
List<String> titles = new ArrayList<>();
titles.forEach(title -> {
String randomString = RandomStringUtils.random(length, useLetters, useNumbers);
driver.findElement(By.xpath("//div[#class='text-filter__input']//input[#placeholder='Search Title']")).sendKeys(randomString);
});
You can also utilize some alphanumeric strings if required.
Thank You.
For getting a random title and setting it in input field you can use this:
String randomTitle = titles.get(new Random().nextInt(titles.size()));
WebElement searchInput = driver.findElement(By.xpath("//div[#class='text-filter__input']//input[#placeholder='Search Title']"));
searchInput.sendKeys(randomTitle);
To take a random value from List<String> titles you have to use Random() class and then pass it within the element using sendKeys() you can use the following solution:
import java.util.Collections;
import java.util.Iterator;
import java.util.List;
import java.util.Random;
import org.openqa.selenium.By;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.WebDriverWait;
List<String> titles = new WebDriverWait(driver, 20).until(ExpectedConditions.visibilityOfAllElementsLocatedBy(By.xpath("//div[#class='document-card__details']//h3/a"))).stream().map(element->element.getAttribute("innerHTML")).collect(Collectors.toList());
String name = titles.get(new Random().nextInt(titles.size()));
driver.findElement(By.xpath("//div[#class='text-filter__input']//input[#placeholder='Search Title']")).sendKeys(name);
Optimizing your code:
List<String> titles = new WebDriverWait(driver, 20).until(ExpectedConditions.visibilityOfAllElementsLocatedBy(By.xpath("//div[#class='document-card__details']//h3/a"))).stream().map(element->element.getAttribute("innerHTML")).collect(Collectors.toList());
driver.findElement(By.xpath("//div[#class='text-filter__input']//input[#placeholder='Search Title']")).sendKeys(titles.get(new Random().nextInt(titles.size())));
My code pulls the links and adds them to the HashSet. I want the link to replace the original link and repeat the process till no more new links can be found to add. The program keeps running but the link isn't updating and the program gets stuck in an infinite loop doing nothing. How do I get the link to update so the program can repeat until no more links can be found?
package downloader;
import java.io.IOException;
import java.net.URL;
import java.util.HashSet;
import java.util.Scanner;
import java.util.Set;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.jsoup.select.Elements;
public class Stage2 {
public static void main(String[] args) throws IOException {
int q = 0;
int w = 0;
HashSet<String> chapters = new HashSet();
String seen = new String("/manga/manabi-ikiru-wa-fuufu-no-tsutome/i1778063/v1/c1");
String source = new String("https://mangapark.net" + seen);
// 0123456789
while( q == w ) {
String source2 = new String(source.substring(21));
String last = new String(source.substring(source.length() - 12));
String last2 = new String(source.substring(source.length() - 1));
chapters.add(seen);
for (String link : findLinks(source)) {
if(link.contains("/manga") && !link.contains(last) && link.contains("/i") && link.contains("/c") && !chapters.contains(link)) {
chapters.add(link);
System.out.println(link);
seen = link;
System.out.print(chapters);
System.out.println(seen);
}
}
}
System.out.print(chapters);
}
private static Set<String> findLinks(String url) throws IOException {
Set<String> links = new HashSet<>();
Document doc = Jsoup.connect(url)
.data("query", "Java")
.userAgent("Mozilla")
.cookie("auth", "token")
.timeout(3000)
.get();
Elements elements = doc.select("a[href]");
for (Element element : elements) {
links.add(element.attr("href"));
}
return links;
}
}
Your progamm didn't stop becouse yout while conditions never change:
while( q == w )
is always true. I run your code without the while and I got 2 links print twice(!) and the programm stop.
If you want the links to the other chapters you have the same problem like me. In the element
Element element = doc.getElementById("sel_book_1");
the links are after the pseudoelement ::before. So they will not be in your Jsoup Document.
Here is my questsion to this topic:
How can I find a HTML tag with the pseudoElement ::before in jsoup
I parsed a website with Jsoup and extracted the links. Now I tried to store just a part of that link in an ArrayList. Somehow I cannot store one link at a time.
I tried several String methods, Scanner and BufferedReader without success.
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
public class DatenImportUnternehmen {
public static void main(String[] args) throws IOException {
ArrayList<String> aktien = new ArrayList<String>();
String searchUrl = "https://www.ariva.de/aktiensuche/_result_table.m";
for(int i = 0; i < 1; i++) {
String searchBody = "page=" + Integer.toString(i) +
"&page_size=25&sort=ariva_name&sort_d=asc
&ariva_performance_1_year=_&ariva_per
formance_3_years=&ariva_performance_5_years=
&index=0&founding_year=&land=0&ind
ustrial_sector=0§or=0¤cy=0
&type_of_share=0&year=_all_years&sales=_&p
rofit_loss=&sum_assets=&sum_liabilities=
&number_of_shares=&earnings_per_share=
÷nd_per_share=&turnover_per_share=
&book_value_per_share=&cashflow_per_sh
are=&balance_sheet_total_per_share=
&number_of_employees=&turnover_per_employee
=_&profit_per_employee=&kgv=_&kuv=_&kbv=_÷nd
_yield=_&return_on_sales=_";
// post request to search URL
Document document =
Jsoup.connect(searchUrl).requestBody(searchBody).post();
// find links in returned HTML
for(Element link:document.select("a[href]")) {
String link1 = link.toString();
String link2 = link1.substring(link1.indexOf('/'));
String link3 = link2.substring(0, link2.indexOf('"'));
aktien.add(link3);
System.out.println(aktien);
}
}
}
}
My output looks like (just a part of it):
[/1-1_drillisch-aktie]
[/1-1_drillisch-aktie, /11_88_0_solutions-aktie]
[/1-1_drillisch-aktie, /11_88_0_solutions-aktie, /1st_red-aktie]
[/1-1_drillisch-aktie, /11_88_0_solutions-aktie, /1st_red-aktie, /21st-
_cent-_fox_b_new-aktie]
[/1-1_drillisch-aktie, /11_88_0_solutions-aktie, /1st_red-aktie, /21st-
_cent-_fox_b_new-aktie, /21st_century_fox-aktie]
[/1-1_drillisch-aktie, /11_88_0_solutions-aktie, /1st_red-aktie, /21st-
_cent-_fox_b_new-aktie, /21st_century_fox-aktie, /2g_energy-aktie]
[/1-1_drillisch-aktie, /11_88_0_solutions-aktie, /1st_red-aktie, /21st-
_cent-_fox_b_new-aktie, /21st_century_fox-aktie, /2g_energy-aktie,
/3i_group-aktie]
[/1-1_drillisch-aktie, /11_88_0_solutions-aktie, /1st_red-aktie, /21st-
_cent-_fox_b_new-aktie, /21st_century_fox-aktie, /2g_energy-aktie,
/3i_group-aktie, /3i_infrastructure-aktie]
What I want to achieve is:
[/1-1_drillisch-aktie]
[/11_88_0_solutions-aktie]
[/1st_red-aktie]
[/21st-_cent-_fox_b_new-aktie]
and so on.
I just don't now what the problem is at this stage.
Your problem is that you are printing the array whilst adding to it in the loop.
To resolve the issue you can print the array outside of the array to print everything in one go, or you can print link3 (which is what you are adding to the ArrayList), instead of the array in the loop.
Option 1:
for(Element link:document.select("a[href]")) {
String link1 = link.toString();
String link2 = link1.substring(link1.indexOf('/'));
String link3 = link2.substring(0, link2.indexOf('"'));
aktien.add(link3);
}
System.out.println(aktien);
Option 2:
for(Element link:document.select("a[href]")) {
String link1 = link.toString();
String link2 = link1.substring(link1.indexOf('/'));
String link3 = link2.substring(0, link2.indexOf('"'));
aktien.add(link3);
System.out.println(link3);
}
I'm very new to Selenium and I've been trying to make the test suite gather data from a table. I don't have the slightest clue on how to do this.
Here's the table I am working with:
http://i.imgur.com/vdITVug.jpg
New appointments (dates) are randomly added at random times of the day. I've created a test suite that will constantly refresh at this page. The next step, would be to save all the dates in the table, create a loop to compare if the dates after a refresh happen to be different the original stored dates.
If they are different, notify the user.
Here's a theoretical example of what I'm trying to accomplish.
//Navigate to the appointment page
//Store all the current dates from the table
for (until a new appointment pops up)
{
//Refresh the page
// Compare the dates to the stored dates
if (the dates =/ stored dates)
{
notify the user(me in this case)
}
}
I'm also trying to figure out how I can find the element ID of the table.
Here's a screenshot with some of the html code: http://i.imgur.com/GD4yOp9.png
The statement that is highlighted has the first date stored.
Any advice would be appreciated, thanks!
Tried replicating a similar HTML structure (in fact 2 of them, one after the refresh). Here is a quick solution for you to compare the HTML tables after refresh.
The key here is organizing your table data into a Map<String, List<String>> like data structure.
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.firefox.FirefoxDriver;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
public class CheckTables {
public WebDriver driver;
public static void main(String[] args) throws Exception {
CheckTables objTest = new CheckTables();
objTest.runTest();
}
public void runTest(){
driver = new FirefoxDriver();
driver.navigate().to("file:///D:/00_FX_WorkSpace/X_Hour/RoadTest_1.html");
Map<String, List<String>> objTable_1 = readTable();
System.out.println("TABLE:1" + objTable_1);
//event to refresh the table
driver.navigate().to("file:///D:/00_FX_WorkSpace/X_Hour/RoadTest_2.html");
Map<String, List<String>> objTable_2 = readTable();
System.out.println("TABLE:2" + objTable_2);
compareTables(objTable_1, objTable_2);
}
public Map<String, List<String>> readTable(){
Map<String, List<String>> objTable = new HashMap<>();
List<WebElement> objRows = driver.findElements(By.cssSelector("tr#data"));
for(int iCount=0; iCount<objRows.size(); iCount++){
List<WebElement> objCol = objRows.get(iCount).findElements(By.cssSelector("td.tableTxt"));
List<String> columns = new ArrayList<>();
for(int col=0; col<objCol.size(); col++){
columns.add(objCol.get(col).getText());
}
objTable.put(String.valueOf(iCount), columns);
}
return objTable;
}
public void compareTables(Map<String, List<String>> objTable1, Map<String, List<String>> objTable2){
for(int count=0; count<objTable1.size(); count++){
List<String> objList1 = objTable1.get(String.valueOf(count));
System.out.println(objList1);
List<String> objList2 = objTable2.get(String.valueOf(count));
System.out.println(objList2);
if(objList1.containsAll(objList2)){
System.out.println("Row [" + count + "] is SAME");
}
else{
//notify
System.out.println("Row [" + count + "] has CHANGED");
}
}
}
}
Here are the HTML snippets for RoadTest_1.html and RoadTest_2.html --
https://gist.github.com/anonymous/43c3b1f44817c69bd03d/