Develop Reference

HtmlUnit - getByXPath with unknown element type

I'm using HtmlUnit to scrape data and I'm getting used to the syntax of XPath.
However I've run into a problem.
I have an element that I need to pull that varies between pages, sometimes it is a "span" element and sometimes it is an "a" element (a link). The reason being simply sometimes the item I am scraping has a link and sometimes it is just plain text (to state the obvious).
What is the same however is an attribute called "data-reactid", which always has a set value of, let's just say 99.
I've been reading and messing around, and have been trying things like this:
HtmlElement element = (HtmlElement) myPage.getFirstByXPath("//#data-reactid='99'");
System.out.println(element.getTextContent());
I am getting the following error:
java.lang.ClassCastException: java.lang.Boolean cannot be cast to com.gargoylesoftware.htmlunit.html.HtmlElement
Why getFirstByXPath() is returning a boolean is beyond me.
So my question is, how can I access an element by a specified attribute and value, when I do not know what type the element will be?
Thanks!

It's giving you a boolean because your XPath is asking for a boolean. Your XPath,
//#data-reactid='99'
is asking the question "does there exist a data-reactid attribute anywhere in my document with a value of 99?"
What you want is a predicate -- that is, "select elements where this logical condition is true". For all elements (we'll use a * wildcard since we don't know the name) that have a #data-reactid of 99:
//*[#data-reactid = '99']

How to get count of Selenium XPath results

As of now I am getting the count of the number of matching results using listChanges.size() . How do I directly get the count without loading getChanges in the list?
By getChanges = By.xpath("//td[contains(#class,'blob-code blob-code-addition') or contains(#class,'blob-code blob-code-deletion')]");
List<WebElement> listChanges = driver.findElements(getChanges);
I found this(Count function in XPath) and I tried the below which does not work!
Integer getCount = By.xpath(count("//td[contains(#class,'blob-code blob-code-addition') or contains(#class,'blob-code blob-code-deletion')]"));
Looks like I have to do something like this.
Integer getCount = By.xpath("count(//td[contains(#class,'blob-code blob-code-addition') or contains(#class,'blob-code blob-code-deletion')])");
But the right hand side returns an object of type By

As alex says, size() is the way to go. However I do have another suggestion for you.
Even though the proper way to find the element counts is to use WebDriver api with findElements() as per my knowledge. Another way is to execute javascript by using executeScript() and with proper script. I am not sure if javascript and xpath can be mixed together to accomplish this since xpath execution through javascript is not multi-browser right now. See this. However, I do think using cssSelector with javascript can make it lot easier to accomplish. See the following code :
String cssQuery = ".blob-code-addition, .blob-code-deletion";
String script = "return document.querySelectorAll('" + cssQuery + "').length;";
Object count = ((JavascriptExecutor)driver).executeScript(script);
System.out.println(count);
Print
26

You cannot get the count using XPath, because an xpath expression in selenium has to correspond to an actual element on a page.
The way you are doing it via findElements() + size() is how you have to count elements using the selenium webdriver.

Is there a way to check all the components name in java using a loop?

the reason im asking this is because for example I have 3 textbox name(variable), text_1,text_2 and text_3. and i want to automatically write something in the textbox depending on which textbox name(variable) i have to write on. Using a loop I need to check if current textbox name is text_1, text_2 or text_3.
if i were to write it in pseudo code it will look like this:
loop:
if(component name == text_1)
text_1.setText = text;
else if(component name == text_2)
.... and so on..

This is what arrays are for. Then you can access them by index with
textFields[0].setText(text);
and doing something to all of them is just a matter of using a loop and an index.

Selenium won't read the current input value

I'm running Selenium on a site that changes the value of a disabled input text box using jquery. Looking at the HTML, the value of the input box continues to say "Not Available" even though the value is obviously changed.
I can get the current value using Firebug with
$("#inputid").val()
but I get the value "Not Available when I've used my selenium code:
driver.findElement(By.id("inputid")).getAttribute("value");
Any suggestions on how to get this value in Selenium? I want to avoid trying to use something like JavascriptExecutor but if that's the best solution it would be good to know.
I don't have access to the jQuery code so I can't help you there. Sorry :-/

If the value is changed by jQuery due to some DOM events, chances are your Selenium test is going to check for the new value too fast. You can get the value after it changes away from "Not Available" with something like this:
WebDriverWait wait = new WebDriverWait(driver,10);
String value = wait.until(new ExpectedCondition<String>() {
public String apply(WebDriver driver) {
String value = driver.findElement(By.id("inputid")).getAttribute("value");
if value.equals("Not Available")
return null;
return value;
}
});
(Disclaimer: It's been ages since I've written Java code so I may have goofed in the code above.) The wait.until call will run the apply method until it returns something else than null. It will wait for at most 10 seconds. The value returned by wait.until will be the value that was last returned by the apply that terminated the end. In other words, it will return the new value of the element.
You say
Looking at the HTML, the value of the input box continues to say "Not Available" even though the value is obviously changed.
Yes, that's a quirk of the DOM. When you change the value of the input field, the value attribute on the element that represent the input field does not change. What changes is the value property on the element. This is why you have to do $("#inputid").val(), not $("#inputid").attr('value').

How do you handle incomplete data in a Processing table?

I'm parsing a CSV using Processing's Table interface, but some rows are missing some data. I want to pull all the data available into my table, but I'm not sure how to handle the missing data--I keep getting NullPointerException when I loop over the table with dataTable.getInt on the missing values.
I don't have a background in statically typed languages, so I've no idea how to conditionally assign this data short of putting a separate try/catch around each assignment. Surely there's a better way?

Before calling dataTable.getInt method check if dataTable is not null like
if(dataTable != null) {
int my_nt = dataTable.getInt
}
//else skip since it is empty

Since your're using getInt--you should perform a regex search/replace ,<not numeric>, with ,<some int>,. In your case it may be as simple as replacing ,, with ,0,
Also, as Hassan suggests, double check that dataTable is not null.

Ok, so I figured out a way to do this:
First, call dataTable.makeNullEmpty(), which turns all the null values into empty strings.
Then, you can use a pattern like this:
String total_value = dataTable.getString(i, 4);
if(total_value.length() > 0) s.total_value = parseInt(total_value);
and you get assignment only if an int is there to be parsed.