Develop Reference

Accessing unparsed entities in XSLT with a SAXTransformerFactory and TransformerHandlers

I have some trouble while retrieving unparsed entity URIs, with the XPath function unparsed-entity-uri().
I'm using a SAXTransformerFactory like in "Efficient XSLT pipeline in Java" question, because I need to perform a transformations chain (i.e. apply several XSLT transformations, and use the result of a transformation as input for the second transformation).
I discovered I'm unable to retrieve unparsed entity thank to the code below. Actually it works well with Xalan, but not with Saxon-HE (version 9.7.0) - but I need Saxon because I'd rather XSLT 2.0 (even if in the code below there's nothing specific to XSLT 2, it's only for the sake of providing an example). It also works with Saxon if I don't use a TransformerHandler, e.g. stf.newTransformer(new StreamSource("transfo.xsl")).transform(new StreamSource("input.xsl"), new StreamResult(System.out)) will produce the desired output.
Is there a configuration step that I forgot?
// use "org.apache.xalan.processor.TransformerFactoryImpl" for Xalan
String transformerFactoryClassName = "net.sf.saxon.TransformerFactoryImpl";
SAXTransformerFactory stf = (SAXTransformerFactory) TransformerFactory.newInstance(transformerFactoryClassName,
LaunchSimpleTransformationUnparsedEntities.class.getClassLoader());
try {
TransformerHandler thTransf = stf
.newTransformerHandler(new StreamSource("transfo.xsl"));
// output the result in console
thTransf.setResult(new StreamResult(System.out));
// Launch transformation of input.xml
Transformer t = stf.newTransformer();
t.transform(new StreamSource("input.xml"),
new SAXResult(thTransf));
} catch (TransformerConfigurationException e) {
e.printStackTrace();
} catch (TransformerException e) {
e.printStackTrace();
}
In input, I have (for input.xml):
<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE book
[<!ENTITY cover_hadrien SYSTEM "images/covers/cover_hadrien.jpg" NDATA jpeg>]>
<book>
<title>Les mémoires d'Hadrien</title>
<author>Marguerite Yourcenar</author>
<cover imgref="cover_hadrien" />
</book>
and a sample XSLT (for transfo.xsl):
<?xml version="1.0" encoding="utf-8"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="2.0">
<xsl:template match="cover">
<xsl:copy>
<xsl:value-of select="unparsed-entity-uri(#imgref)"/>
</xsl:copy>
</xsl:template>
<xsl:template match="#*|node()">
<xsl:copy>
<xsl:apply-templates select="#*|node()"/>
</xsl:copy>
</xsl:template>
</xsl:stylesheet>
as a result, I would expect something like:
<?xml version="1.0" encoding="UTF-8"?><book>
<title>Les mémoires d'Hadrien</title>
<author>Marguerite Yourcenar</author>
<cover>images/covers/cover_hadrien.jpg</cover>
</book>
but <cover> is empty when performing the transformation with Saxon.

Interesting observation. The issue in fact is not with Saxon's TransformerHandler, but rather with the "identity transformer" obtained using SAXTransformerFactory.newTransformer(): the identity transformer is not passing unparsed entities down the line. This is essentially because Saxon's identity transformer is reusing parts of the XSLT engine, and XSLT does not provide any way for a transformation to output unparsed entities in the result. If you sent the SAX parser output directly to the TransformerHandler, rather than going via an identity transformer, then I think it would all work.
As with all things JAXP-related, the specification of SAXTransformerFactory.newTransformer() is infuriatingly vague. All it says is that the returned Transformer performs a copy of the Source to the Result. i.e. the "identity transform". What exactly counts as a copy? I think Saxon's interpretation has been that it is equivalent to the effect of doing an XSLT identity transform - which would lose unparsed entities (as well as other things like CDATA sections, the DTD, etc).
Incidentally XSLT 2.0 specifies that the result of unparsed-entity-uri() should be an absolute URI (XSLT 1.0 doesn't say anything on the subject) so even if this is fixed, the Saxon output will be different.
Entered as a Saxon issue here: https://saxonica.plan.io/issues/3201 I think we need to be a bit careful about passing unparsed entities to a SAXResult if we don't pass all the other events expected by a SAX DTDHandler - and we're certainly not going to change the Saxon identity transformer to retain things (like DTD declarations) that aren't modelled in XDM.

Indeed, following #MichaelKay's details, launching the transformation that way works properly:
// launch transformation of input.xml
XMLReader reader = XMLReaderFactory.createXMLReader();
reader.setContentHandler(thTransf);
reader.setDTDHandler(thTransf);
reader.parse(new InputSource(input.xml"));
(this will replace the following line:
// Launch transformation of input.xml
Transformer t = stf.newTransformer();
t.transform(new StreamSource("input.xml"),
new SAXResult(thTransf));
that were used initially).

Java XML Programming - Extracting the Child nodes

I have an xml file like below. I need to extract all the Child nodes under logdata and all the sub-Child nodes under each of the Child nodes along with their values. How can i extract these
<logdata>
<Request RequestID="123" RequestType = "Read">
<Data Mode = "Read">
<Type>ReadWrite</Type>
</Data>
<Textdetails Eligible = "true">
<Code>1</Code
<Name>ABC</Name>
</Textdetails>
</Request>
<Request RequestID="456" RequestType = "Read">
<Data Mode = "Read">
<Type>ReadWrite</Type>
</Data>
<Textdetails Eligible = "true">
<Code>2</Code>
<Name>DEF</Name>
</Textdetails>
</Request>
</logdata>

Using the XOM Library this would be rather simple. All you would need is to build the Document from a Builder. Then get the root element (logdata) using getRootElement(). After that you can use getChildElements() to get all the child elements from logdata and any other Element.

Doing DOM Node-to-String transformation, but with namespace issues

So we have an XML Document with custom namespaces. (The XML is generated by software we don't control. It's parsed by a namespace-unaware DOM parser; standard Java7SE/Xerces stuff, but also outside our effective control.) The input data looks like this:
<?xml version="1.0" encoding="ISO-8859-1" standalone="no"?>
<MainTag xmlns="http://BlahBlahBlah" xmlns:CustomAttr="http://BlitherBlither">
.... 18 blarzillion lines of XML ....
<Thing CustomAttr:gibberish="borkborkbork" ... />
.... another 27 blarzillion lines ....
</MainTag>
The Document we get is usable and xpath-queryable and traversable and so on.
Converting this Document into a text format for writing out to a data sink uses the standard Transformer approach described in a hundred SO "how do I change my XML Document into a Java string?" questions:
Transformer transformer = TransformerFactory.newInstance().newTransformer();
transformer.setOutputProperty(OutputKeys.OMIT_XML_DECLARATION, "no");
transformer.setOutputProperty(OutputKeys.INDENT, "yes");
StringWriter stringwriter = new StringWriter();
transformer.transform (new DOMSource(theXMLDocument), new StreamResult(stringwriter));
return stringwriter.toString();
and it works perfectly.
But now I'd like to transform individual arbitrary Nodes from that Document into strings. A DOMSource constructor accepts Node pointers just the same as it accepts a Document (and in fact Document is just a subclass of Node, so it's the same API as far as I can tell). So passing in an individual Node in the place of "theXMLDocument" in the snippet above works great... until we get to the Thing.
At that point, transform() throws an exception:
java.lang.RuntimeException: Namespace for prefix 'CustomAttr' has not been declared.
at com.sun.org.apache.xml.internal.serializer.SerializerBase.getNamespaceURI(Unknown Source)
at com.sun.org.apache.xml.internal.serializer.SerializerBase.addAttribute(Unknown Source)
at com.sun.org.apache.xml.internal.serializer.ToUnknownStream.addAttribute(Unknown Source)
......
That makes sense. (The "com.sun.org.apache" is weird to read, but whatever.) It makes sense, because the namespace for the custom attribute was declared at the root node, but now the transformer is starting at a child node and can't see the declarations "above" it in the tree. So I think I understand the problem, or at least the symptom, but I'm not sure how to solve it though.
If this were a String-to-Document conversion, we'd be using a DocumentBuilderFactory instance and could call .setNamespaceAware(false), but this is going in the other direction.
None of the available properties for transformer.setOutputProperty() affect the namespaceURI lookup, which makes sense.
There is no such corresponding setInputProperty or similar function.
The input parser wasn't namespace aware, which is how the "upstream" code got as far as creating its Document to hand to us. I don't know how to hand that particular status flag on to the transforming code, which is what I really would like to do, I think.
I believe it's possible to (somehow) add a xmlns:CustomAttr="http://BlitherBlither" attribute to the Thing node, the same as the root MainTag had. But at that point the output is no longer identical XML to what was read in, even if it "means" the same thing, and the text strings are eventually going to be compared in the future. We wouldn't know if it were needed until the exception got thrown, then we could add it and try again... ick. For that matter, changing the Node would alter the original Document, and this really ought to be a read-only operation.
Advice? Is there some way of telling the Transformer, "look, don't stress your dimwitted little head over whether the output is legit XML in isolation, it's not going to be parsed back in on its own (but you don't know that), just produce the text and let us worry about its context"?

Given your quoted error message "Namespace for prefix 'CustomAttr' has not been declared.",
I'm assuming that your pseudo code is along the lines of:
<?xml version="1.0" encoding="ISO-8859-1" standalone="no"?>
<MainTag xmlns="http://BlahBlahBlah" xmlns:CustomAttr="http://BlitherBlither">
.... 18 blarzillion lines of XML ....
<Thing CustomAttr:attributeName="borkborkbork" ... />
.... another 27 blarzillion lines ....
</MainTag>
With that assumption, here's my suggestion:
So you want to extract the "Thing" node from the "big" XML. The standard approach is to use a little XSLT to do that. You prepare the XSL transformation with:
Transformer transformer = transformerFactory.newTransformer(new StreamSource(new File("isolate-the-thing-node.xslt")));
transformer.setOutputProperty(OutputKeys.OMIT_XML_DECLARATION, "no");
transformer.setParameter("elementName", stringWithCurrentThing); // parameterize transformation for each Thing
...
EDIT: #Ti, please note the parameterization instruction above (and below in the xslt).
The file 'isolate-the-thing-node.xslt' could be a flavour of the following:
<xsl:stylesheet
xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns:custom0="http://BlahBlahBlah"
xmlns:custom1="http://BlitherBlither"
version="1.0">
<xsl:param name="elementName">to-be-parameterized</xsl:param>
<xsl:output encoding="utf-8" indent="yes" method="xml" omit-xml-declaration="no" />
<xsl:template match="/*" priority="2" >
<!--<xsl:apply-templates select="//custom0:Thing" />-->
<!-- changed to parameterized selection: -->
<xsl:apply-templates select="custom0:*[local-name()=$elementName]" />
</xsl:template>
<xsl:template match="node() | #*" priority="1">
<xsl:copy>
<xsl:apply-templates select="node() | #*" />
</xsl:copy>
</xsl:template>
</xsl:stylesheet>
Hope that gets you over the "Thing" thing :)

I have managed to parse the provided document, get the Thing node and print it without issues.
Take a look at the Working Example:
Node rootElement = d.getDocumentElement();
System.out.println("Whole document: \n");
System.out.println(nodeToString(rootElement));
Node thing = rootElement.getChildNodes().item(1);
System.out.println("Just Thing: \n");
System.out.println(nodeToString(thing));
nodeToString:
private static String nodeToString(Node node) {
StringWriter sw = new StringWriter();
try {
Transformer t = TransformerFactory.newInstance().newTransformer();
t.setOutputProperty(OutputKeys.OMIT_XML_DECLARATION, "no");
t.setOutputProperty(OutputKeys.INDENT, "yes");
t.transform(new DOMSource(node), new StreamResult(sw));
} catch (TransformerException te) {
System.out.println("nodeToString Transformer Exception");
}
return sw.toString();
}
Output:
Whole document:
<?xml version="1.0" encoding="UTF-8"?><MainTag xmlns="http://BlahBlahBlah" xmlns:CustomAttr="http://BlitherBlither">
<Thing CustomAttr="borkborkbork"/>
</MainTag>
Just Thing:
<?xml version="1.0" encoding="UTF-8"?><Thing CustomAttr="borkborkbork"/>
When I try the same code with CustomAttr:attributeName as suggested by #marty it fails with the original exception, so it looks like somewhere in your original XML you are prefixing a attribute or node with that custom CustomAttr namespace.
In the latter case you can leverage the problem with setNamespaceAware(true), which will include the namespace information on the Thing node itself.
<?xml version="1.0" encoding="UTF-8"?><Thing xmlns:CustomAttr="http://BlitherBlither" CustomAttr:attributeName="borkborkbork" xmlns="http://BlahBlahBlah"/>

Using regexp in java to modify an xml

I'm trying to change an xml by using regular expressions in java, but I can't find the right way. I have an xml like this (simplified):
<ROOT>
<NODE ord="1" />
<NODE ord="3,2" />
</ROOT>
The xml actually shows a sentence with its nodes, chunks ... in two languages and has more attributes. Each sentence it's loaded in two RichTextAreas (one for the source sentence, and the other for the translated one).
What I need to do is add a style attribute to every node that has an specific value in its ord attribute (this style attribute will show correspondences between two languages, like Google Translate does when you mouse over a word). I know this could be done using DOM (getting all the NODE nodes and then seeing the ord attribute one by one), but I am looking for the fastest way to do the change as it is going to execute in the client side of my GWT app.
When that ord attribute has a single value (like in the first node) it is easy to do just taking the xml as a string and using the replaceAll() function . The problem is when the attribute has composed values (like in the second node).
For example, how could I do to add that attribute if the value I'm looking for is 2? I believe this could be done using regular expressions, but I can't find out how. Any hint or help would be appreciated (even if it doesn't use regexp and replaceAll function).
Thanks in advance.

XPath can do this for you. You could select:
/ROOT/NODE[contains(concat(',', #ord, ','), ',2,')]
Since you intend to use GWT on the client, you could give gwtxslt a try. With it you could specify an XSLT stylesheet to do the transformation (i.e. adding the attribute) for you:
XsltProcessor processor = new XsltProcessor();
processor.importStyleSheet(styleSheetText);
processor.importSource(sourceText);
processor.setParameter("ord", "2");
processor.setParameter("style", "whatever");
String resultString = processor.transform();
// do something with resultString
where styleSheetText could be an XSLT document along the lines of
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:param name="ord" select="''" />
<xsl:param name="style" select="''" />
<xsl:template match="node()|#*">
<xsl:copy>
<xsl:apply-templates select="node()|#*" />
</xsl:copy>
</xsl:template>
<xsl:template match="NODE">
<xsl:copy>
<xsl:apply-templates select="#*" />
<xsl:if test="contains(concat(',', #ord, ','), concat(',', $ord, ','))">
<xsl:attribute name="style">
<xsl:value-of select="$style" />
</xsl:attribute>
</xsl:if>
<xsl:apply-templates select="node()" />
</xsl:copy>
</xsl:template>
</xsl:stylesheet>
Note that I use concat() to prevent partial matches in the comma-separated list that the attribute value of #ord actually is.

String resultString = subjectString.replaceAll("<NODE ord=\"([^\"]*\\b2\\b[^\"]*)\" />", "<NODE ord=\"$1\" style=\"whatever\"/>");
will find any <NODE> tag that has a single ord attribute with a value of "2" (or "1,2" or "2,3" or "1,2,3" but not "12") and adds a style attribute.
This is quick and dirty, and rightfully advised against by many here, but for a one-off quick job it should be OK.
Explanation:
<NODE ord=" # Match <NODE ord:" verbatim
( # Match and capture...
[^"]* # any number of characters except "
\b2\b # "2" as a whole word (surrounded by non-alphanumerics)
[^"]* # any number of characters except "
) # End of capturing group
" /> # Match " /> verbatim

I'm trying to change an xml by using regular expressions in java, but I can't find the right way.
That's because there isn't a right way. Regular expressions are not the right way to manipulate XML. That's because XML is not a regular grammar (which is a technical term in computer science, not a generalized insult.)

It might sound like overkill, but I'd consider using the standard DOM parsers to read the fragment, modify it using setAttribute() calls, and then write it out again. I know you said that efficiency is important, but how long does this really take? Testing shows 60ms on my ageing 2GHz pentium.
This approach will be more robust against comments, things split across lines etc. It is also much more likely to give you well-formed XML. Also things like your requirement of only doing it if certain values are present will become trivial.
public class AddStyleExample {
public static void main(final String[] args) {
String input = "<ROOT> <NODE ord=\"1\" /> <NODE ord=\"3,2\" /> </ROOT>";
try {
final DocumentBuilderFactory factory = DocumentBuilderFactory
.newInstance();
factory.setValidating(false);
factory.setNamespaceAware(false);
DocumentBuilder builder;
builder = factory.newDocumentBuilder();
final Document doc = builder.parse(new InputSource(
new StringReader(input)));
NodeList tags = doc.getElementsByTagName("NODE");
for (int i = 0; i < tags.getLength(); i++) {
Element node = (Element) tags.item(i);
node.setAttribute("style", "example value");
}
StringWriter writer = new StringWriter();
final StreamResult result = new StreamResult(writer);
final Transformer t = TransformerFactory.newInstance()
.newTransformer();
t.setOutputProperty(OutputKeys.INDENT, "yes");
t.setOutputProperty(OutputKeys.OMIT_XML_DECLARATION, "yes");
t.transform(new DOMSource(doc), result);
System.out.println(writer.toString());
} catch (ParserConfigurationException e) {
e.printStackTrace();
} catch (TransformerException e) {
e.printStackTrace();
} catch (SAXException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
}

Java replace in XML-file

I have created my own XML-file on my Android phone, which looks similar to this
<?xml version="1.0" encoding="utf-8" ?>
<backlogs>
<issue id="1">
<backlog id="0" name="Linux" swid="100" />
<backlog id="0" name="Project Management" swid="101" />
</issue>
<issue id="2">
<backlog id="0" name="Tests" swid="110" />
<backlog id="0" name="Online test" swid="111" />
<backlog id="0" name="Test build" swid="112" />
<backlog id="0" name="Update" swid="113" />
</issue>
</backlogs>
I have then converted it into a String to replace inside the string using Regular Expression, but I have a problem with the Regular Expression. The Regular Expression I just created looks like this
([\n\r]*)<(.*)issue(.*)1(.*)([\n\r]*)(.*)([\n\r]*)(.*)([\n\r]*)(.*)<(.*)/(.*)issue(.*)
I need to replace the specific issue-tag (located with the specific ID) with another issue-tag in another String
The Regular Expression works fine for the tag with ID 1, but not with ID 2 as there is another amount of tags, but is there any way to get around the use of amount?
I hope you understand my question
I finally found a solution for my question, which is
([\n\r]*)<(.*)issue(.*)1[\S\s]*?<(.*)/(.*)issue(.*)

Do not use regex. Please. Use XML parser.
Do you know what is the highest voted SO answer

Use a SAX (or StAX) parser and writer at the same time.
As you read one event, detect whether to write the same event type to the writer without modification, or to do some modifications in the state you are currently in - like swapping an element name or attribute value. This will handle an unlimited amount of elements at the expense of CPU usage; in general it will be pretty light-weight.