I have a html file like the following
...
<span itemprop="A">234</span>
...
<span itemprop="B">690</span>
...
In this i want to extract values as A and B.
Can u suggest any html parser library for java that can do this easily?
Personally, I favour JSoup over JTidy. It has CSS-like selectors, and the documentation is much better, imho. With JSoup, you can easily extract those values with the following lines:
Document doc = Jsoup.connect("your_url").get();
Elements spans = doc.select("span[itemprop]");
for (Element span : spans) {
System.out.println(span.text()); // will print 234 and 690
}
http://jsoup.org/
JSoup is the way to go.
JTidy is a confusingly named yet respected HTML parser.
Related
I am trying to select, using Jsoup, a <div> that has multiple classes:
<div class="content-text right-align bold-font">...</div>
The syntax for doing so, to the best of my understanding, should be:
document.select("div.content-text.right-align.bold-font");
However, for some reason, this doesn't work for me.
When I try the same exact syntax on JSFIDDLE, it works without a hitch.
Does multi-class selection work in Jsoup?
(I'd rather find out that this is a bug in my code than find out that this is a Jsoup limitation :)
UPDATE (thanks to the answer below): Jsoup works perfectly with the aforementioned syntax.
Works for me with latest Jsoup (1.5.2).
String html = "<div class=\"content-text right-align bold-font\">foo</div>";
Document document = Jsoup.parse(html);
Elements elements = document.select("div.content-text.right-align.bold-font");
System.out.println(elements.text()); // foo
So either you're possibly using an outdated version of Jsoup which exposes a bug related to this, or the actual HTML doesn't contain a <div> like that.
It would by helpfull for you in near future. Have fun.
Jsoup selectors,
jQuery selectors
I need to parse HTML and find corresponding CSS styles. I can parse HTML and CSS separataly, but I can't combine them. For example, I have an XHTML page like this:
<html>
<head>
<title></title>
</head>
<body>
<div class="abc">Hello World</div>
</body>
</html>
I have to search for "hello world" and find its class name, and after that I need to find its style from an external CSS file. Answers using Java, JavaScript, and PHP are all okay.
Use jsoup library in java which is a HTML Parser. You can see for example here
For example you can do something like this:
String html="<<your html content>>";
Document doc = Jsoup.parse(html);
Element ele=doc.getElementsContainingOwnText("Hello World").first.clone(); //get tag containing Hello world
HashSet<String>class=ele.classNames(); //gives you the classnames of element containing Hello world
You can explore the library further to fit your needs.
Similiar question Can jQuery get all CSS styles associated with an element?. Maybe css optimizers can do what you want, take a look at unused-css.com its online tool but also lists other tools.
As i understood you have chance to parse style sheet from external file and this makes your task easy to solve. First try to parse html file with jsoup which supports jquery like selector syntax that helps you parse complicated html files easier. then check this previous solution to parse css file. Im not going to full solution as i state with these libraries all task done internally and the only thing you should do is writing glue code to combine these two.
Using Java java.util.regex
String s = "<body>...<div class=\"abc\">Hello World</div></body>";
Pattern p = Pattern.compile("<div.+?class\\s*?=\\s*['\"]?([^ '\"]+).*?>Hello World</div>", Pattern.CASE_INSENSITIVE); Matcher m = p.matcher(s);
if (m.find()) {
System.out.println(m.group(1));
}
prints abc
Can anyone recommend me a java library to allow me XPath Queries over URLs?
I've tried JAXP without success.
Thank you.
There are several different approaches to this documented on the Web:
Using HtmlCleaner
HtmlCleaner / Java DOM parser - Using XPath Contains against HTML in Java (This is the way I recommend)
HtmlCleaner itself has a built in utility supporting XPath - See the javadocs http://htmlcleaner.sourceforge.net/doc/org/htmlcleaner/XPather.html or this example http://thinkandroid.wordpress.com/2010/01/05/using-xpath-and-html-cleaner-to-parse-html-xml/
Using Jericho
Jericho and Jaxen
http://sujitpal.blogspot.com/2009/04/xpath-over-html-using-jericho-and-jaxen.html
I have tried a few different variations of these approaches, i.e. HtmlParser plus the Java DOM parser, and JSoup plus Jaxen, but the combination that worked best is HtmlCleaner plus the Java DOM parser. The next best combination was Jericho plus Jaxen.
jsoup, Java HTML Parser Very similar to jQuery syntax way.
You could use TagSoup together with Saxon. That way you simply replace any XML SAX parser used with TagSoup and the XPath 2.0 or XSLT 2.0 or XQuery 1.0 implementation works as usual.
Use Xsoup. According to the docs, it's faster than HtmlCleaner. Example
#Test
public void testSelect() {
String html = "<html><div><a href='https://github.com'>github.com</a></div>" +
"<table><tr><td>a</td><td>b</td></tr></table></html>";
Document document = Jsoup.parse(html);
String result = Xsoup.compile("//a/#href").evaluate(document).get();
Assert.assertEquals("https://github.com", result);
List<String> list = Xsoup.compile("//tr/td/text()").evaluate(document).list();
Assert.assertEquals("a", list.get(0));
Assert.assertEquals("b", list.get(1));
}
Link to Xsoup - https://github.com/code4craft/xsoup
I've used JTidy to make HTML into a proper DOM, then used plain XPath to query the DOM.
If you want to do cross-document/cross-URL queries, better use JTidy with XQuery.
If I have HTML that looks like:
<td class="blah">&nbs;???? </td>
Could I get the ???? value using xpath?
What would it look like?
To use XPath you usually need XML not HTML, but some parsers (e.g. the one built into PHP) have a relaxed Mode which will parse most HTML, too.
If you want to find all <a> that are direct children of <td class="blah"> the XPath you need is
//td[#class = 'blah']/a
or
//td[#class = 'blah']/a[#href = 'http://...']
(depending on whether you only want the one url or all urls)
This will give you a Set of Nodes. You'll need to iterate through it and then check for the nodeType of the firstChild (supposed to be a text node) and the number of child nodes (supposed to be 1). Then the firstChild will contain the ????
Why would you use an XML parser to parse HTML?
I would suggest using a dedicated Java HTML parser, there are many, but I haven't tried any myself.
As for your question, would it work, I suspect it will not work, you will get an error when trying to parse it as HTML right at &nbs; if not earlier.
I'm looking for an HTML object model in Java, capable of parsing HTML (not required) and containing all HTML elements (and CSS as well) in an elegant object model.
I'm looking for a pure java version of the Groovy's HTML builder.
(I have no luck on google with this request.)
I want to be able to perform stuff like:
HTML html = new HTML();
Body body = html.body();
Table table body.addTable(myCssStyle);
Row row = table.addRow("a", "b", "c").withCss(cssRowStyle);
and so on...
Check out Jsoup:
Example:
(Building some html)
Document doc = Document.createShell("");
Element headline = doc.body().appendElement("h1").text("thats a headline");
Element pTag = doc.body().appendElement("p").text("some text ...");
Element span = pTag.prependElement("span").text("That's");
System.out.println(doc);
Output:
<html>
<head></head>
<body>
<h1>thats a headline</h1>
<p><span>That's</span>some text ...</p>
</body>
</html>
Documentation:
Codebook
API Documentation (JavaDoc)
Jakarta ECS might be able to do what you want.
Just an idea: you could take a look at the source code of xhtmlrenderer project.
http://code.google.com/p/flying-saucer//
It's not plain HTML (it's XHTML), but may be a good starting point, don't you think?