Develop Reference

How can I use Jsoup to turn unallowed html tag delimiter into entities where there are unallowed tags

Using Jsoup clean is it possible to convert this string:
Here is some <b>important</b> stuff that can't have
<script>javascript</script> or the following embed tag
<embed src="helloworld.swf" type="application/vnd.adobe.flash-movie"> movie
in the output
to this :
Here is some <b>important</b> stuff that can't have
<script>javascript</script> or the following embed tag
<embed src="helloworld.swf" type="application/vnd.adobe.flash-movie">
movie in the output
so it renders
Here is some important stuff that can't have
<script>javascript</script> or the following embed tag
<embed src="helloworld.swf" type="application/vnd.adobe.flash-movie">
movie in the output
Where the bold tag is allowed and left alone but the script and embed tags delimiters change from < > to < and > so they are treated as just text and not real html elements.
What settings are necessary to accomplish this? I have:
private static String limitHtml(String value) {
String result = value;
if (value != null && !value.isEmpty()) {
Document.OutputSettings settings = new Document.OutputSettings();
settings.prettyPrint(false);
// what other settings ???
Whitelist whitelist = Whitelist.none().addTags(ALLOWED_HTML_TAGS);
whitelist.addAttributes(":all", ALLOWED_HTML_ATTRIBUTES);
result = Jsoup.clean(value, "", whitelist, settings);
}
return result;
}
Is there a similar Java lib that can accomplish this if Jsoup doesn't.

Jsoup can definitively get your back here. The trick is to use a dummy document (transitional variable in the code) with a single pre element in it.
We will simply add each unallowed element found in this pre element.
Later, we replace the unallowed element in the initial value with its escaped html code.
CODE
// Comma separated list of allowed tags.
private static String ALLOWED_HTML_TAGS_CSS_QUERY = "b,span";
private static String limitHtml(String value) {
String result = value;
if (value != null && !value.isEmpty()) {
// Build a sided document. It will help us escape unallowed tags.
Document transitional = Jsoup.parse("<pre></pre>");
// Parse the actual value for finding unallowed tags
Document doc = Jsoup.parseBodyFragment(value, "");
Elements unallowedElements = doc.select("*:not("+ALLOWED_HTML_TAGS_CSS_QUERY+")");
for (Element e : unallowedElements) {
switch (e.tagName()) {
case "#root": case "html": case "head": case "body":
// Those tags are added automatically by Jsoup. Nothing to do...
break;
default:
// Load the unallowed element to escape its html code in the transitional document
Element pre = transitional.select("pre").first().text(e.outerHtml());
// Replace unallowed element with its escape html code
e.replaceWith(new TextNode(pre.text(), ""));
}
}
// Get the final sanitized value
Document.OutputSettings settings = new Document.OutputSettings();
settings.prettyPrint(false);
Whitelist whitelist = Whitelist.none().addTags(ALLOWED_HTML_TAGS);
whitelist.addAttributes(":all", ALLOWED_HTML_ATTRIBUTES);
result = Jsoup.clean(doc.body().html(), "", whitelist, settings);
}
return result;
}
SAMPLE USAGE
String unsanitizedHtml = "Here is some <b>important</b> stuff that can't have " + //
"<script>javascript</script> or the following embed tag " + //
"<embed src=\"helloworld.swf\" type=\"application/vnd.adobe.flash-movie\"> movie" + //
"in the output";
System.out.println("BEFORE:\n" + unsanitizedHtml);
System.out.println();
System.out.println("AFTER:\n" + limitHtml(unsanitizedHtml));
OUTPUT
BEFORE:
Here is some <b>important</b> stuff that can't have <script>javascript</script> or the following embed tag <embed src="helloworld.swf" type="application/vnd.adobe.flash-movie"> moviein the output
AFTER:
Here is some <b>important</b> stuff that can't have <script>javascript</script> or the following embed tag <embed src="helloworld.swf" type="application/vnd.adobe.flash-movie"> moviein the output

Java jsoup link ignore

I have the following code:
private static final Pattern FILE_FILTER = Pattern.compile(
".*(\\.(css|js|bmp|gif|jpe?g|png|tiff?|mid|mp2|mp3|mp4|wav|avi|mov|mpeg|ram|m4v|pdf" +
"|rm|smil|wmv|swf|wma|zip|rar|gz))$");
private boolean isRelevant(String url) {
if (url.length() < 1) // Remove empty urls
return false;
else if (FILE_FILTER.matcher(url).matches()) {
return false;
}
else
return TLSpecific.isRelevant(url);
}
I am using this part when i am parsing a web site to check whether it contains links that contains some of the patterns declared, but I dont know is there a way to do it directly through jsoup and optimize the code. For example given a web page how I can ignore all of them with jsoup?

how I can ignore all of them with jsoup?
Let's say we want any element not having jpg or jpeg extension in their hrefor src attribute.
String filteredLinksCssQuery = "[href]:not([href~=(?i)\\.jpe?g$]), " + //
"[src]:not([src~=(?i)\\.jpe?g$])";
String html = "<a href='foo.jpg'>foo</a>" + //
"<a href='bar.svg'>bar</a>" + //
"<script src='baz.js'></script>";
Document doc = Jsoup.parse(html);
for(Element e: doc.select(filteredLinksCssQuery)) {
System.out.println(e);
}
OUTPUT
bar
<script src="baz.js"></script>
[href] /* Select any element having an href attribute... */
:not([href~=(?i)\.jpe?g$]) /* ... but exclude those matching the regex (?i)\.jpe?g$ */
, /* OR */
[src] /* Select any element having a src attribute... */
:not([src~=(?i)\.jpe?g$]) /* ... but exclude those matching the regex (?i)\.jpe?g$ */
You can add more extensions to filter. You may want to write some code for generating filteredLinksCssQuery automatically because this CSS query can quickly become unmaintainable.

How to read the public URL in GWT?

I m new in GWT and I m generating a web application in which i have to create a public URL.
In this public URL i have to pass hashtag(#) and some parameters.
I am finding difficulty in achieving this task.
Extracting the hashtag from the URL.
Extracting the userid from the URL.
My public URL example is :: http://www.xyz.com/#profile?userid=10003

To access the URL in GWT you can use the History.getToken() method. It will give you the entire string that follows the hashtag ("#").
In your case (http://www.xyz.com/#profile?userid=10003) it will return a string "profile?userid=10003". After you have this you can parse it however you want. You can check if it contains("?") and u can split it by "?" or you can get a substring. How you get the information from that is really up to you.

I guess you already have the URL. I'm not that good at Regex, but this should work:
String yourURL = "http://www.xyz.com/#profile?userid=10003";
String[] array = yourURL.split("[\\p{Lower}\\p{Upper}\\p{Punct}}]");
int userID = 0;
for (String string : array) {
if (!string.isEmpty()) {
userID = Integer.valueOf(string);
}
}
System.out.println(userID);

To get the parameters:
String userId = Window.Location.getParameter("userid");
To get the anchor / hash tag:
I don't think there is something, you can parse the URL: look at the methods provided by Window.Location.

Parser JSoup change the tags to lower case letter

I did some research and it seems that is standard Jsoup make this change. I wonder if there is a way to configure this or is there some other Parser I can be converted to a document of Jsoup, or some way to fix this?

Unfortunately not, the constructor of Tag class changes the name to lower case:
private Tag(String tagName) {
this.tagName = tagName.toLowerCase();
}
But there are two ways to change this behavour:
If you want a clean solution, you can clone / download the JSoup Git and change this line.
If you want a dirty solution, you can use reflection.
Example for #2:
Field tagName = Tag.class.getDeclaredField("tagName"); // Get the field which contains the tagname
tagName.setAccessible(true); // Set accessible to allow changes
for( Element element : doc.select("*") ) // Iterate over all tags
{
Tag tag = element.tag(); // Get the tag of the element
String value = tagName.get(tag).toString(); // Get the value (= name) of the tag
if( !value.startsWith("#") ) // You can ignore all tags starting with a '#'
{
tagName.set(tag, value.toUpperCase()); // Set the tagname to the uppercase
}
}
tagName.setAccessible(false); // Revert to false

Here is a code sample (version >= 1.11.x):
Parser parser = Parser.htmlParser();
parser.settings(new ParseSettings(true, true));
Document doc = parser.parseInput(html, baseUrl);

There is ParseSettings class introduced in version 1.9.3.
It comes with options to preserve case for tags and attributes.

You must use xmlParser instead of htmlParser and the tags will remain unchanged. One line does the trick:
String html = "<camelCaseTag>some text</camelCaseTag>";
Document doc = Jsoup.parse(html, "", Parser.xmlParser());

I am using 1.11.1-SNAPSHOT version which does not have this piece of code.
private Tag(String tagName) {
this.tagName = tagName.toLowerCase();
}
So I checked ParseSettings as suggested above and changed this piece of code from:
static {
htmlDefault = new ParseSettings(false, false);
preserveCase = new ParseSettings(true, true);
}
to:
static {
htmlDefault = new ParseSettings(true, true);
preserveCase = new ParseSettings(true, true);
}
and skipped test cases while building JAR.

Logical help - java regex

I have a string which contains image tags, more than 1. Now I need to regex out the alt= tag. I tried it like this:
while (m3.find()) {
Pattern p4 = Pattern.compile("<!\\[CDATA\\[(.*?)\\]\\]>");
Matcher m4 = p4.matcher(m3.group());
while (m4.find()) {
if(m4.group().contains("<img src")) {
Pattern p6 = Pattern.compile("<img src(.*?)/>");
Matcher m6 = p6.matcher(m4.group());
while (m6.find()) {
Pattern p7 = Pattern.compile("alt=\"(.*?)\"");
Matcher m7 = p7.matcher(m6.group());
while (m7.find()) {
messages.add(m4.group().replace(m6.group(), m7.group().replace("alt=", "").replace("\"", "")).replace("<![CDATA[", "").replace("]]>", ""));
}
}
} else {
messages.add(m4.group().replace("<![CDATA[", "").replace("]]>", ""));
}
}
}
The problem is: there is more than 1 image tag. messages is an ArrayList. I need just 1 messages.add for ALL images in the actual message. The code as it is does something very different and I don't have any idea how to fix it or where my mistakes are :/ I just want replace the whole with the content of alt="...", but every the actual message contains. Can anyone help me?

Maybe you should use a third party library like jsoup, which allows parsing, extracting and modifying html documents in a jquery fashion. Using this library, changing the attributes of certain html elements (as explained here) should work like this:
Document doc = Jsoup.parse(html);
doc.select("div.comments a").attr("rel", "nofollow");