Develop Reference

JSoup Scraping based on custom attributes

So I am using JSoup to scrape a website that creates a bunch of divs with dynamic class names (they change every reload), but the same attribute names. E.g:
<div class="[random text here that changes] js_resultTile" data-listing-number="[some number]">
<div class="a12_regularTile js_rollover_container " itemscope itemtype="http://schema.org/Product" data-listing-number="[same number here]">
<a href...
I've tried multiple approaches to selecting those divs and saving them in elements, but I can't seem to get it right. I've tried by attribute:
Document doc = Jsoup.connect([theUrl]).get();
Elements myEls = doc.select("div[data-listing-number]");
I've tried by class:
Document doc = Jsoup.connect([theUrl]).get();
Elements myEls = doc.getElementsByClass("a12_regularTile")
And:
Document doc = Jsoup.connect([theUrl]).get();
Elements myEls = doc.select("div[class*=js_resultTile]")
I've tried another attribute method:
Document doc = Jsoup.connect([theUrl]).get();
Elements myEls = new Elements();
for (Element element : doc.getAllElements() )
{
for ( Attribute attribute : element.attributes() )
{
if ( attribute.getKey().equalsIgnoreCase("data-listing-number"))
{
myEls.add(element);
}
}
}
None of these work. I can select the doc that gets me all the HTML, but my myEls object is always empty. What can I use to select these elements?

Are you sure these elements are present in HTML returned by server? They may be added later by JavaScript. If JavaScript is involved in page presentation then you won't be able to use Jsoup. More details in my answer to similar question here: JSoup: Difficulty extracting a single element
And one more tip. Instead of using your for-for-if construction you can use this:
for (Element element : doc.getAllElements()) {
if (element.dataset().containsKey("listing-number")) {
myEls.add(element);
}
}

jSoup get data using td-class tags from webpage

I would like to get data from http://www.futbol24.com/Live/?__igp=1&LiveDate=20141104 using jSoup. I know how to use jSoup - but I am finding it difficult to pinpoint the data that I need.
I would like the Time, Home Team and Away Team from each row of the tbody table. So the output from the first row should be:
08:30 Persipura Jayapura Pelita Bandung Raya
I can see the td class of each of these elements as "status alt", "home" and "guest".
Currently I have tried the below, but it doesn't seem to output anything... what am I doing wrong?
matches = new ArrayList<Match>();
//getHistory
String website = "http://www.futbol24.com/Live/?__igp=1&LiveDate=20141104";
Document doc = Jsoup.connect(website).get();
Element tblHeader = doc.select("tbody").first();
List<Match> data = new ArrayList<>();
for (Element element1 : tblHeader.children()){
Match match = new Match();
match.setTimeOfMatch(element1.select("td.status.alt").text());
match.setAwayTeam(element1.select("td.home").text());
match.setHomeTeam(element1.select("td.guest").text());
data.add(match);
System.out.println(data.toString());
Does anybody know how I can use jSoup to get these elements from each row of the table?
Thanks,
Rob

The content of this site is generated via AJAX it seems. Jsoup can't handle this, since it is not a browser that interprets JavaScript. To solve this scraping problem you may need something like Selenium webdriver. I gave a longer answer to a generalized question about this before, so please look here:
Jsoup get dynamically generated HTML

JSoup with Wunderground Pollen data

I am currently scraping pollen data from wunderground since their API accessor doesn't offer pollen data, specifically the values attributed to each day.
I've navigated the HTML using Chrome Dev Tools and found the specific line that I want. Using the documentation offered by JSoup, I tried putting in my own custom CSS Selectors, but I am quite lost.
I was wondering if anyone would give me some insight on how to access that particular element.
For example, below is an example of what I have so far.
doc = Jsoup.connect("http://www.wunderground.com/DisplayPollen.asp?Zipcode=19104").get();
Element title = doc.getElementById("td");
Element tagName = doc.tagName("id");
System.out.println(tagName);

You don't want to use doc.getElementById("td") because <td> is not id attribute, but tag (also getElementById doesn't support CSS query).
What you want is to select first <td> with class levels. You can do it via
Element tag = doc.select("td.levels").first();
Also to get only text which will be generated with this tag (and not entire HTML) use text() method like
System.out.println(tag.text());

Document doc = Jsoup.connect("http://www.wunderground.com/DisplayPollen.asp?Zipcode=19104").get();
Elements days = doc.select("table.pollen-table").first().select("td.even-four");
for (Element day : days) {
System.out.println(day.text());
}
Elements levels = doc.select("td.levels");
for (Element level : levels) {
System.out.println(level.text());
}

Android parsing multiple HTML tables with Jsoup

I am trying to parse an HTML page on the internet to retrieve data from a table in it with Jsoup. But the page I want to parse contains more than one table. How can I do that? Is that possible?
Edit:
Here is the page I want to parse:
http://metudex.com/mobilepac/browse.php?SEARCH=calculus&kriter=X&Submit=Search
I want to retrieve data from the tables with book info.

Document doc = Jsoup.connect("http://metudex.com/mobilepac/browse.php?SEARCH=calculus&kriter=X&Submit=Search").get();
Elements els = doc.select("td:has(span.briefcitDetail)"); //gets every td that has a child span with class briefcitDetail
for(Element el : els) {
System.out.println("--" + el.text());
}

JSoup extracting data from within paragraph

I want to extract all the text there is between all paragraphs on an unknown site (meaning i do not know the structure of the site).
So far i've got:
Elements paragraphEmail = doc.select("p");
Where doc = Jsoup.connect(url).get();
for (Element e : paragraphEmail) {
}
How to achieve this?

doc.select("p") will give you all the paragraph elements as a collection Elements.
Use a for each loop to get the text:
for(Element e : paragraphEmail){
System.out.println(e.text());
}
I suggest you take a look at the Jsoup cookbook and the API reference to get more familiar with the methods in Jsoup.
Cookbook
API Reference