I was having trouble getting just the href from a rows of table data. Although I was able to get it working, I am wondering if anyone has an explanation for why my code here works.
for (Element element : result.select("tr")) {
if (element.select("tr.header.left").isEmpty()) {
Elements tds = element.select("td");
//The line below is what I don't understand
String link = tds.get(0).getElementsByAttribute("href").first().attr("href");
String position = tds.get(1).text();
}
}
The line that I was using before, that did not work is below:
String link = tds.get(0).attr("href");
Why does this line return an empty string? I'm assuming it has to do with how I am iterating through the elements as I've selected by "tr". However, I'm not familiar with how Elements vs Element are structured.
Thanks for your help!
Elements is simply an ArrayList<Element>
The reason you're having to write that extra code is because <td> doesn't have an href attribute, so tds.get(0).attr("href"); won't work. You're presumably trying to capture the href from an <a> within the cell. The longer, working code is saying:
For the first cell in the row, get the first element with an #href attribute (i.e. a link), and get
its #href attribute
Try the following example (with example document) to show how to access the child links more clearly:
Element result = Jsoup.parse("<html><body><table><tr><td><a href=\"http://a.com\" /</td><td>Label1</td></tr><tr><td><a href=\"http://b.com\" /></td><td>Label2</td></tr></table></body></html>");
for (Element element : result.select("tr")) {
if (element.select("tr.header.left").isEmpty()) {
Elements tds = element.select("td");
String link = tds.get(0).getElementsByTag("a").attr("href");
String position = tds.get(1).text();
System.out.println(link + ", " + position);
}
}
Related
I am trying to get text between tags and save into some variable, for example:
Here I want to save value return which is between em tags. Also I need the rest of the text which is in p tags,
em tag value is assigned with return and
p tag value should return only --> an item, cancel an order, print a receipt, track your purchases or reorder items.
if some value is before em tag, even that value should be in different variable basically one p if it has multiple tags within then it should be split and save into different variables. If I know how can I get rest of text which are not in inner tags I can retrieve the rest.
I have written below: the below is returning just "return" which is in "'em' tags.
Here ep is basically doc.select(p), selecting p tag and then iterating, not sure if I am doing right way, any other approaches are highly appreciated.
String text ="\<p><em>return </em>an item, cancel an order, print a receipt, track your purchases or reorder items.</p>"
Elements italic_tags = ep.select("em");
for(Element em:italic_tags) {
if(em.tagName().equals("em")) {
System.out.println( em.select("em").text());
}
}
If you need to select each sub text and text enclosed by different tags you need to try selecting Node instead of Element. I modified your HTML to include more tags so the example is more complete:
String text = "<p><em>return </em>an item, <em>cancel</em> an order, <em>print</em> a receipt, <em>track</em> your purchases or reorder items.</p>";
Document doc = Jsoup.parse(text);
Element ep = doc.selectFirst("p");
List<Node> childNodes = ep.childNodes();
for (Node node : childNodes) {
if (node instanceof TextNode) {
// if it's a text, just display it
System.out.println(node);
} else {
// if it's another element, then display its first
// child which in this case is a text
System.out.println(node.childNode(0));
}
}
output:
return
an item,
cancel
an order,
print
a receipt,
track
your purchases or reorder items.
I've got some HTML that looks like this:
<tr>
<td>
Some text that is interrupted by an image here:
<img alt="imageName.png" src="linkhere" width="18" height="18">
and then continues here.
</td>
</tr>
and basically I just need a way to loop through the nodes here and add either the text or the image alt to a string with JSoup, maintaining the order of the nodes.
In the end it should look like this:
Some text that is interrupted by an image here: "imageName.png" and then continues here
So far I'm able to get the image by itself or the text by itself by using:
element.text();
//or
element.select("img").attr("alt")
but I'm having trouble getting both of them into an ordered list.
Any ideas?
The following code should give you the output string you are looking for. It basically loops through all the nodes in the document and determines whether or not they are text nodes or elements. If they are text nodes, it will add them to the output string. If they are elements, it will check for an image child and add the alt text to the string.
String test = "";
Element body = doc.getElementsByTag("body").first();
List<Node> childNodes = body.childNodes();
for(Node node : childNodes){
if(node instanceof TextNode){
// These are text nodes, lets see if they are empty or not and add them to the string.
String nodeString = node.toString();
if(nodeString != null && !nodeString.trim().isEmpty()){
test += nodeString;
}
} else if (node instanceof Element) {
// Here is an element, let's see if there is an image.
Element element = (Element)node;
Element image = element.children().select("img").first();
if(image != null)
{
test += image.attr("alt");
}
}
}
Trying to practice extracting data from tables using JSoup. Can't figure out why I can't pull the "Shares Outstanding" field from
https://finance.yahoo.com/q/ks?s=AAPL+Key+Statistics
Here's two attempts where 's' is AAPL:
public class YahooStatistics {
String sharesOutstanding = "Shares Outstanding:";
public YahooStatistics(String s) {
String keyStatisticsURL = ("https://finance.yahoo.com/q/ks?s="+s+"+Key+Statistics");
//Attempt 1
try {
Document doc = Jsoup.connect(keyStatisticsURL).get();
for (Element table : doc.select("table.yfnc_datamodoutline1")) {
for (Element row : table.select("tr")) {
Elements tds = row.select("td");
for (Element td : tds.select(sharesOutstanding)) {
System.out.println(td.ownText());
}
}
}
}
catch (IOException ex) {
ex.printStackTrace();
}
//Attempt 2
try {
Document doc = Jsoup.connect(keyStatisticsURL).get();
for (Element table : doc.select("table.yfnc_datamodoutline1")) {
for (Element row : table.select("tr")) {
Elements tds = row.select("td");
for (int j = 0; j < tds.size() - 1; j++) {
Element td = tds.get(j);
if ((td.ownText()).equals(sharesOutstanding)) {
System.out.println(tds.get(j+1).ownText());
}
}
}
}
}
catch(IOException ex) {
ex.printStackTrace();
}
The attempts return: BUILD SUCCESSFUL and nothing else.
I've disabled JavaScript on my browser and the table still shows, so I'm assuming this is not written in JavaScript but HTML.
Any suggestions are appreciated.
Notes about your source after the edit:
You should compare ownText() rather than text(). text() gives you the combined text of all the element and all its sub-elements. In this case the element contains Shares Outstanding<font size="-1"><sup>5</sup></font>:, so its combined text is "Shares Outstanding5:". If you use ownText it will just be "Shares Outstanding:".
Note the colon (:). Update the value in sharesOutstanding accordingly.
You are passing it the wrong URL. There should be a + following the AAPL.
Your current query (at least the second attempt) is returning the element twice, because there is a nested table so it finds the TDs twice.
You can either break from your loops once you found a match, go back to your original version (with corrections as above) - see note - or you can try using a more sophisticated query which will only match once:
Elements elems = doc.select("td.yfnc_tablehead1:containsOwn("+sharesOutstanding+") + td.yfnc_tabledata1");
if ( ! elems.isEmpty() ) {
System.out.println( elems.get(0).owntext() );
}
This selector gives you all the td elements whose class is yfnc_tabledata1, whose immediate preceding sibling is a td element whose class is yfnc_tablehead1 and whose own text contains the "Shares Outstanding:" string. This should basically select the exact TD you need.
Note: the previous version of this answer was a long rattle about the difference between Elements.select() and Element.select(). It turns out that I was dead wrong and your original version should have worked - if you had corrected the four points above. So to set the record straight: select() on an Elements actually does look inside each element and the resulting list may contain descendents of any of the elements in the original list that match the selection. Sorry about that.
I have managed to successfully grab the href links using JSoup. I have also managed to grab the relative value and absolute value of a href for a single link. As shown below:
//works perfectly, website: bbc.co.uk
Document document = Jsoup.connect(url).get();
Element link = document.select("a").last();
String relHref = testlink.attr("href");
String absHref = testlink.attr("abs:href");
System.out.println(relHref);
System.out.println(absHref);
//output:
relHref: /help/web/links/
absHref: http://www.bbc.co.uk/help/web/links/
I can even use Element link = document.select("a").first(); and this also works. However, when I try and add this in a loop to iterate through all of the grabbed links and print out each link, it doesn't give me the expected results. Here is my code:
//not working
Elements links = document.select("a");
for(int i=0; i<links.size(); i++){
String relHref = links.attr("href");
String absHref = links.attr("abs:href");
System.out.println(relHref);
System.out.println(absHref);
}
//output
http://m.bbc.co.uk
http://m.bbc.co.uk
http://m.bbc.co.uk
....
I know the links array of type Elements has the correct data, and if I try and print the elements in the links array it displays all of the href tags i.e.
for (Element link : links) {
System.out.println(link);
}
//output 116 links:
mobile site
<img src="http://static.bbci.co.uk/frameworks/barlesque/2.72.5/orb/4/img/bbc-blocks-dark.png" width="84" height="24" alt="BBC">
Skip to content
<a id="orb-accessibility-help" href="/accessibility/">Accessibility Help</a>
....
But how do I get the relHref and absHref for an array to work? Instead my code just prints out the first link over and over again. I've been going at this for hours, so I'm probably making a silly mistake somewhere but help is appreciated!
Thanks.
On this line:
String relHref = links.attr("href");
...how is it supposed to know you're talking about the ith link? (It doesn't: Elements#attr always returns the value for the first entry in the Elements collection.)
You want
String relHref = links.get(i).attr("href");
...which gets the specific link you're interested in via Elements#get, then uses Node#attr on it.
That said, though, I would just use the enhanced for loop:
for (Element link : document.select("a")) {
String relHref = link.attr("href");
String absHref = link.attr("abs:href");
System.out.println(relHref);
System.out.println(absHref);
}
...unless you need i for something.
You need to use the Elements method, get(int index) inside of your for loop to get each Element held by your Elements.
e.g.,
Elements links = document.select("a");
for(int i=0; i < links.size(); i++) {
Element ele = links.get(i);
/// use ele here to extract info from each Element
}
I'm using the library Jsoup, is that I have a string with two HTML components with the attribute ID to all, I want to do is capture the two IDs in an array.
String chain = "<div id='stylized' class='myform' style='margin:20px auto;'>
<div id='material_comprado' > </div> ";
I was trying to use this for, but failed.
int i = 0;
Elements values = doc.getElementsByAttribute("id");
String s[] = new String[values.size()];
for(Element el : values){
s[i++] = el.attr("id");
System.out.println("==> "+s[i]);
}
Anyone can help me.
Your JSoup code itself is fine.
You're incrementing the array index for s beyond its upper bound resulting in an ArrayIndexOutOfBoundsException when you attempt to display the element. Increment the index after youve finished accessing the array
for (Element el : values){
s[i] = el.attr("id");
System.out.println("==> " + s[i]);
i++; // now safe to increment
}