Develop Reference

How to get (and output) namespace prefix declarations using staxmate

Xml documents may show namespace prefix declarations in their root element. As I am new to StaxMate I managed to process xml input events for elements and element attributes. However, I never got a Namespace event.
<?xml version="1.0" encoding="UTF-8"?>
<myRoot xmlns="http://myurl.com/myProject"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:mya="http://myurl.com/myAttributes"
xsi:schemaLocation="http://myurl.com/myProject ./../../main/xsd/mySchema.xsd ">
<myElement mya:myAttribute="attribute content">
<mySubElement>subelements content</original>
</myElement>
</myRoot>
When processing the element myRoot how to get the xmlns namespaces? E.g. in order to output some of them to the root element of the SMOutputDocument?

Found out by experimenting. Following is a - somewhat useless - operation to copy an XML document, including all namespace declarations. Its purpose here is to exemplify how to cope with namespaces in StaxMate.
It is called once with a SMOutputDocument as SMOutputContainer. The cursor points to root element for the output.
After that it recursively explores and copies all elements found.
private void processStartElement(SMInputCursor cursor, SMOutputContainer element) throws XMLStreamException {
SMOutputElement loe = element.addElement(cursor.getPrefixedName());
// add all namespace declarationss to the element
for (int i = 0; i < cursor.getStreamReader().getNamespaceCount(); i++) {
loe.predeclareNamespace(element.getNamespace(
cursor.getStreamReader().getNamespaceURI(i),
cursor.getStreamReader().getNamespacePrefix(i)));
}
for (int i = 0; i < cursor.getAttrCount(); i++) {
loe.addAttribute(
element.getNamespace(cursor.getAttrNsUri(i)),
cursor.getAttrLocalName(i),
cursor.getAttrValue(i));
}
SMInputCursor lc = cursor.childCursor();
while ((lc != null) && (lc.getNext() != null)) {
this.processStartElement(lc, loe);
}
}

How to get a specifc information for an XML file

I have a large XML file and below is an extract from it:
...
<LexicalEntry id="Ait~ifAq_1">
<Lemma partOfSpeech="n" writtenForm="اِتِّفاق"/>
<Sense id="Ait~ifAq_1_tawaAfuq_n1AR" synset="tawaAfuq_n1AR"/>
<WordForm formType="root" writtenForm="وفق"/>
</LexicalEntry>
<LexicalEntry id="tawaA&um__1">
<Lemma partOfSpeech="n" writtenForm="تَوَاؤُم"/>
<Sense id="tawaA&um__1_AinosijaAm_n1AR" synset="AinosijaAm_n1AR"/>
<WordForm formType="root" writtenForm="وأم"/>
</LexicalEntry>
<LexicalEntry id="tanaAgum_2">
<Lemma partOfSpeech="n" writtenForm="تناغُم"/>
<Sense id="tanaAgum_2_AinosijaAm_n1AR" synset="AinosijaAm_n1AR"/>
<WordForm formType="root" writtenForm="نغم"/>
</LexicalEntry>
<Synset baseConcept="3" id="tawaAfuq_n1AR">
<SynsetRelations>
<SynsetRelation relType="hyponym" targets="AinosijaAm_n1AR"/>
<SynsetRelation relType="hyponym" targets="AinosijaAm_n1AR"/>
<SynsetRelation relType="hypernym" targets="ext_noun_NP_420"/>
</SynsetRelations>
<MonolingualExternalRefs>
<MonolingualExternalRef externalReference="13971065-n" externalSystem="PWN30"/>
</MonolingualExternalRefs>
</Synset>
...
I want to extract specific information from it. For a given writtenForm whether from <Lemma> or <WordForm>, the programme takes the value of synset from <Sense> of that writtenForm (same <LexicalEntry>) and searches for all the value id of <Synset> that have the same value as the synset from <Sense>. After that, the programme gives us all the relations of that Synset, i.e it displays the value of relType and returns to <LexicalEntry> and looks for the value synset of <Sense> who have the same value of targets then displays its writtenForm.
I think it's a little bit complicated but the result should be like this:
اِتِّفاق hyponym تَوَاؤُم, اِنْسِجام
One of the solutions is the use of the Stream reader because of the memory consumption. but I don't how should I proceed to get what I want. help me please.

The SAX Parser is different from DOM Parser.It is looking only on the current item it can't see on the future items until they become the current item . It is one of the many you can use when XML file is extremely big . Instead of it there are many out there . To name a few:
SAX PARSER
DOM PARSER
JDOM PARSER
DOM4J PARSER
STAX PARSER
You can find for all them tutorials here.
In my opinion after learning it go straight to use DOM4J or JDOM for commercial product.
The logic of SAX Parser is that you have a MyHandler class which is extending DefaultHandler and #Overrides some of it's methods:
XML FILE:
<?xml version="1.0"?>
<class>
<student rollno="393">
<firstname>dinkar</firstname>
<lastname>kad</lastname>
<nickname>dinkar</nickname>
<marks>85</marks>
</student>
<student rollno="493">
<firstname>Vaneet</firstname>
<lastname>Gupta</lastname>
<nickname>vinni</nickname>
<marks>95</marks>
</student>
<student rollno="593">
<firstname>jasvir</firstname>
<lastname>singn</lastname>
<nickname>jazz</nickname>
<marks>90</marks>
</student>
</class>
Handler class:
import org.xml.sax.Attributes;
import org.xml.sax.SAXException;
import org.xml.sax.helpers.DefaultHandler;
public class UserHandler extends DefaultHandler {
boolean bFirstName = false;
boolean bLastName = false;
boolean bNickName = false;
boolean bMarks = false;
#Override
public void startElement(String uri,
String localName, String qName, Attributes attributes)
throws SAXException {
if (qName.equalsIgnoreCase("student")) {
String rollNo = attributes.getValue("rollno");
System.out.println("Roll No : " + rollNo);
} else if (qName.equalsIgnoreCase("firstname")) {
bFirstName = true;
} else if (qName.equalsIgnoreCase("lastname")) {
bLastName = true;
} else if (qName.equalsIgnoreCase("nickname")) {
bNickName = true;
}
else if (qName.equalsIgnoreCase("marks")) {
bMarks = true;
}
}
#Override
public void endElement(String uri,
String localName, String qName) throws SAXException {
if (qName.equalsIgnoreCase("student")) {
System.out.println("End Element :" + qName);
}
}
#Override
public void characters(char ch[],
int start, int length) throws SAXException {
if (bFirstName) {
System.out.println("First Name: "
+ new String(ch, start, length));
bFirstName = false;
} else if (bLastName) {
System.out.println("Last Name: "
+ new String(ch, start, length));
bLastName = false;
} else if (bNickName) {
System.out.println("Nick Name: "
+ new String(ch, start, length));
bNickName = false;
} else if (bMarks) {
System.out.println("Marks: "
+ new String(ch, start, length));
bMarks = false;
}
}
}
Main Class :
import java.io.File;
import javax.xml.parsers.SAXParser;
import javax.xml.parsers.SAXParserFactory;
import org.xml.sax.Attributes;
import org.xml.sax.SAXException;
import org.xml.sax.helpers.DefaultHandler;
public class SAXParserDemo {
public static void main(String[] args){
try {
File inputFile = new File("input.txt");
SAXParserFactory factory = SAXParserFactory.newInstance();
SAXParser saxParser = factory.newSAXParser();
UserHandler userhandler = new UserHandler();
saxParser.parse(inputFile, userhandler);
} catch (Exception e) {
e.printStackTrace();
}
}
}

XPath was designed for exactly this. Java provides support for it in the javax.xml.xpath package.
To do what you want, the code will look something like this:
List<String> findRelations(String word,
Path xmlFile)
throws XPathException {
String xmlLocation = xmlFile.toUri().toASCIIString();
XPath xpath = XPathFactory.newInstance().newXPath();
xpath.setXPathVariableResolver(
name -> (name.getLocalPart().equals("word") ? word : null));
String id = xpath.evaluate(
"//LexicalEntry[WordForm/#writtenForm=$word or Lemma/#writtenForm=$word]/Sense/#synset",
new InputSource(xmlLocation));
xpath.setXPathVariableResolver(
name -> (name.getLocalPart().equals("id") ? id : null));
NodeList matches = (NodeList) xpath.evaluate(
"//Synset[#id=$id]/SynsetRelations/SynsetRelation",
new InputSource(xmlLocation),
XPathConstants.NODESET);
List<String> relations = new ArrayList<>();
int matchCount = matches.getLength();
for (int i = 0; i < matchCount; i++) {
Element match = (Element) matches.item(i);
String relType = match.getAttribute("relType");
String synset = match.getAttribute("targets");
xpath.setXPathVariableResolver(
name -> (name.getLocalPart().equals("synset") ? synset : null));
NodeList formNodes = (NodeList) xpath.evaluate(
"//LexicalEntry[Sense/#synset=$synset]/WordForm/#writtenForm",
new InputSource(xmlLocation),
XPathConstants.NODESET);
int formCount = formNodes.getLength();
StringJoiner forms = new StringJoiner(",");
for (int j = 0; j < formCount; j++) {
forms.add(
formNodes.item(j).getNodeValue());
}
relations.add(
String.format("%s %s %s", word, relType, forms));
}
return relations;
}
Some basic XPath information:
XPath uses a single file-path-like string to match parts of an XML document. The parts can be any structural part of the document: text, elements, attributes, even things like comments.
A Java XPath expression can attempt to match exactly one part, or several parts, or can even concatenate all matched parts as a String.
In an XPath expression, a name by itself represents an element. For example, WordForm in XPath means any <WordForm …> element in the XML document.
A name starting with # represents an attribute. For example, #writtenForm refers to any writtenForm=… attribute in the XML document.
A slash indicates a parent and child in an XML document. LexicalEntry/Lemma means any <Lemma> element which is a direct child of a <LexicalEntry> element. Synset/#id means the id=… attribute of any <Synset> element.
Just as a path starting with / indicates an absolute (root-relative) path in Unix, an XPath starting with a slash indicates an expression relative to the root of an XML document.
Two slashes means a descendant which may be a direct child, a grandchild, a great-grandchild, etc. Thus, //LexicalEntry means any LexicalEntry in the document; /LexicalEntry only matches a LexicalEntry element which is the root element.
Square brackets indicate match qualifiers. Synset[#baseConcept='3'] matches any <Synset> element with an baseConcept attribute whose value is the string "3".
XPath can refer to variables, which are defined externally, using Unix-shell-like $ substitutions, like $word. How those variables are passed to an XPath expression depends on the engine. Java uses the setXPathVariableResolver method. Variable names are in a completely separate namespace from node names, so it is of no consequence if a variable name is the same as an element name or attribute name in the XML document.
So, the XPath expressions in the code mean:
//LexicalEntry[WordForm/#writtenForm=$word or Lemma/#writtenForm=$word]/Sense/#synset
Match any <LexicalEntry> element anywhere in the XML document which has either
a WordForm child with a writtenForm attribute whose value is equal to the word variable
a Lemma child with a writtenForm attribute whose value is equal to the word variable
and for every such <LexicalEntry> element, return the value of the synset attribute of any <Sense> element which is a direct child of the <LexicalEntry> element.
The word variable is defined externally, by an xpath.setXPathVariableResolver, right before the XPath expression is evaluated.
//Synset[#id=$id]/SynsetRelations/SynsetRelation
Match any <Synset> element anywhere in the XML document whose id attribute is equal to the id variable. For each such <Synset> element, look for any direct SynsetRelations child element, and return each of its direct SynsetRelation children.
The id variable is defined externally, by an xpath.setXPathVariableResolver, right before the XPath expression is evaluated.
//LexicalEntry[Sense/#synset=$synset]/WordForm/#writtenForm
Match any <LexicalEntry> element anywhere in the XML document which has a <Sense> child element which has a synset attribute whose value is identical to the synset variable. For each matched element, find any <WordForm> child element and return that element’s writtenForm attribute.
The synset variable is defined externally, by an xpath.setXPathVariableResolver, right before the XPath expression is evaluated.
Logically, what the above should amount to is:
Locate the synset value for the requested word.
Use the synset value to locate SynsetRelation elements.
Locate writtenForm values corresponding to the targets value of each matched SynsetRelation.

If this XML file is too large to represent in memory, use SAX.
You will want to write your SAX parser to maintain a location. To do this, I typically use a StringBuffer, but a Stack of Strings would work just as nicely. This portion will be important because it will permit you to keep track of the path back to the root of the document, which will allow you to understand where in the document you are at a given point in time (useful when trying to only extract a little information).
The main logic flow looks like:
1. When entering a node, add the node's name to the stack.
2. When exiting a node, pop the node's name (top element) off the stack.
3. To know your location, read your current branch of the XML from the bottom of the stack to the top of the stack.
4. When entering a region you care about, clear the buffer you will capture the characters into
5. When exiting a region you care about, flush the buffer into the data structure you will return back as your output.
This way you can efficiently skip over all the branches of the XML tree that you don't care about.

How to use Selenium get text from an element not including its sub-elements

HTML
<div id='one'>
<button id='two'>I am a button</button>
<button id='three'>I am a button</button>
I am a div
</div>
Code
driver.findElement(By.id('one')).getText();

I've seen this question pop up a few times in the last maybe year or so and I've wanted to try writing this function... so here you go. It takes the parent element and removes each child's textContent until what remains is the textNode. I've tested this on your HTML and it works.
/**
* Takes a parent element and strips out the textContent of all child elements and returns textNode content only
*
* #param e
* the parent element
* #return the text from the child textNodes
*/
public static String getTextNode(WebElement e)
{
String text = e.getText().trim();
List<WebElement> children = e.findElements(By.xpath("./*"));
for (WebElement child : children)
{
text = text.replaceFirst(child.getText(), "").trim();
}
return text;
}
and you call it
System.out.println(getTextNode(driver.findElement(By.id("one"))));

Warning: the initial solution (deep below) won't workI opened an enhancement request: 2840 against the Selenium WebDrive and another one against the W3C WebDrive specification - the more votes, the sooner they'll get enough attention (one can hope). Until then, the solution suggested by #shivansh in the other answer (execution of a JavaScript via Selenium) remains the only alternative. Here's the Java adaptation of that solution (collects all text nodes, discards all that are whitespace only, separates the remaining by \t):
WebElement e=driver.findElement(By.xpath("//*[#id='one']"));
if(driver instanceof JavascriptExecutor) {
String jswalker=
"var tw = document.createTreeWalker("
+ "arguments[0],"
+ "NodeFilter.SHOW_TEXT,"
+ "{ acceptNode: function(node) { return NodeFilter.FILTER_ACCEPT;} },"
+ "false"
+ ");"
+ "var ret=null;"
+ "while(tw.nextNode()){"
+ "var t=tw.currentNode.wholeText.trim();"
+ "if(t.length>0){" // skip over all-white text values
+ "ret=(ret ? ret+'\t'+t : t);" // if many, tab-separate them
+ "}"
+ "}"
+ "return ret;" // will return null if no non-empty text nodes are found
;
Object val=((JavascriptExecutor) driver).executeScript(jswalker, e);
// ---- Pass the context node here ------------------------------^
String textNodesTabSeparated=(null!=val ? val.toString() : null);
// ----^ --- this is the result you want
}
References:
TreeWalker - supported by all browsers
Selenium Javascript Executor
Initial suggested solution - not working - see enhancement request: 2840
driver.findElement(By.id('one')).find(By.XPath("./text()").getText();
In a single search
driver.findElement(By.XPath("//[#id=one]/text()")).getText();
See XPath spec/Location Paths the child::text() selector.

I use a function like below:
private static final String ALL_DIRECT_TEXT_CONTENT =
"var element = arguments[0], text = '';\n" +
"for (var i = 0; i < element.childNodes.length; ++i) {\n" +
" var node = element.childNodes[i];\n" +
" if (node.nodeType == Node.TEXT_NODE" +
" && node.textContent.trim() != '')\n" +
" text += node.textContent.trim();\n" +
"}\n" +
"return text;";
public String getText(WebDriver driver, WebElement element) {
return (String) ((JavascriptExecutor) driver).executeScript(ALL_DIRECT_TEXT_CONTENT, element);
}

var outerElement = driver.FindElement(By.XPath("a"));
var outerElementTextWithNoSubText = outerElement.Text.Replace(outerElement.FindElement(By.XPath("./*")).Text, "");

Similar solution to the ones given, but instead of JavaScript or setting text to "", I remove elements in the XML and then get the text.
Problem:
Need text from 'root element without children' where children can be x levels deep and the text in the root can be the same as the text in other elements.
The solution treats the webelement as an XML and replaces the children with voids so only the root remains.
The result is then parsed. In my cases this seems to be working.
I only verified this code in a environment with Groovy. No idea if it will work in Java without modifications. Essentially you need to replace the groovy libraries for XML with Java libraries and off you go I guess.
As for the code itself, I have two parameters:
WebElement el
boolean strict
When strict is true, then really only the root is taken into account. If strict is false, then markup tags will be left. I included in this whitelist p, b, i, strong, em, mark, small, del, ins, sub, sup.
The logic is:
Manage whitelisted tags
Get element as string (XML)
Parse to an XML object
Set all child nodes to void
Parse and get text
Up until now this seems to be working out.
You can find the code here: GitHub Code

Remove empty namespace in java using DOM

<Request>
<EMPId>?</EMPId>
</Request>
I know this is a repeated question, but i would like to post it again as i dint get a convincing answer from any of the threads i went through.
My ultimate aim is to add the XML given above as the Body content of a SOAP message.
You can have a look at the following link to see how i am doing it.
Namespace related error on creating SOAP Request
It worked fine when i was using the Websphere Application Server 7.0 library.JRE is also present, forgot to include in screen shot.
Since i have to export it as a jar and run it as a stand alone application, i have to remove the dependency of 'Websphere Application Server 7.0 library'. Because, by keeping this library, my jar size will go above 100MB. So i thought of taking only the library which i needed.
'com.ibm.ws.prereq.soap.jar'
Now the issue is, the Request tag of the generated SOAP request is coming in following format.
<Request xmlns="">
<EMPId>?</EMPId>
</Request>
I am able to create a 'org.w3c.dom.Document' representation for the generated SOAP message.
Now, can any one tell me how can I delete the xmlns="" from Request tag.

The simplest way what i found is:
first:
in child set nasmespace as in root:
second:
remove namespace
Document doc = new Document();
Namespace xmlns = Namespace.getNamespace("http://www.microsoft.com/networking/WLAN/profile/v1");
Element rootXML = new Element("WLANProfile", xmlns);
Element nameXML = new Element("name");
nameXML.addContent(name);
rootXML.addContent(nameXML);
//below solution
nameXML.setNamespace(xmlns);
nameXML.removeNamespaceDeclaration(xmlns);

Finally I found several solutions of the described problem.
First, you can remove all namespaces from all xml using this answer.
Second, if you do not need to remove all namespaces in Xml, but only empty ones, they arise due to the fact that some namespace is written in the root elements, which is not in the child. For example:
<ЭДПФР xmlns="http://пф.рф/КСАФ/2018-04-03"
xmlns:АФ4="xx"...>
<КСАФ xmlns="">
...
</КСАФ>
So you need to set the same namespace for all children of root elements. It can be done using this code (call setTheSameNamespaceForChildren(rootElement) for root element before saving):
private static final String namespaceKey = "xmlns";
private static String namespaceValue;
public static void setTheSameNamespaceForChildren(Element rootEl) {
namespaceValue = rootEl.getAttribute(namespaceKey);
NodeList list = rootEl.getChildNodes();
for (int i = 0; i < list.getLength(); i++) {
Node child = list.item(i);
setTheSameNamespaceRecursively(child);
}
}
private static void setTheSameNamespaceRecursively(Node node) {
if (node.getNodeType() == Node.ELEMENT_NODE) {
boolean isChanged = setTheSameNamespace((Element) node);
if (isChanged) {
NodeList list = node.getChildNodes();
for (int i = 0; i < list.getLength(); i++) {
Node child = list.item(i);
setTheSameNamespaceRecursively(child);
}
}
}
}
private static boolean setTheSameNamespace(Element node) {
String curValue = node.getAttribute(namespaceKey);
if (curValue.length() == 0) {
node.setAttribute(namespaceKey, namespaceValue);
return true;
}
return false;
}

jdom removes duplicate namespace declaration (xmloutputter)

jdom seems to remove duplicate namespace declarations. This is a problem when a XML document is embedded into another XML structure, such as for example in the OAI-PHM (open archive initiative). This can be a problem when the surrounding xml is only a container and the embedded document gets extracted later.
Here is some code. The embedded xml is contained in the string with the same name. It declares the xsi namespace. We construct a jdom container, also declaring the xsi namespace. We parse and embed the string. When we print the whole thing the inner xsi namepsace is gone.
public static final Namespace OAI_PMH= Namespace.getNamespace( "http://www.openarchives.org/OAI/2.0/");
public static final Namespace XSI = Namespace.getNamespace("xsi", "http://www.w3.org/2001/XMLSchema-instance");
public static final String SCHEMA_LOCATION = "http://www.openarchives.org/OAI/2.0/ http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd";
public static final String ROOT_NAME = "OAI-PMH";
String embeddedxml = "<?xml version=\"1.0\" encoding=\"UTF-8\"?> <myxml xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" xsi:schemaLocation=\""
+ "http://www.isotc211.org/2005/gmd"
+ " http://www.ngdc.noaa.gov/metadata/published/xsd/schema/gmd/gmd.xsd"
+ " http://www.isotc211.org/2005/gmx"
+ " http://www.ngdc.noaa.gov/metadata/published/xsd/schema/gmx/gmx.xsd\">\""
+ "</myxml>";
// loadstring omitted (parse embeddedxml into jdom)
Element xml = loadString(embeddedxml ,false);
Element root = new Element(ROOT_NAME, OAI_PMH);
root.setAttribute("schemaLocation", SCHEMA_LOCATION, XSI);
// insert embedded xml into container structure
root.addContent(xml);
XMLOutputter out = new XMLOutputter(Format.getPrettyFormat());
// will see that the xsi namespace declaration from embeddedxml is gone
out.output(root,System.out);
I think that XMLoutputter is responsible for this behaviour. Any hints how I can make it preserve the duplicate namepspace?
thanks
Kurt

Something is missing in your code: The declaration of final static String ROOT_NAME is not shown and Element xml ist not used after initialization.
If ROOT_NAME is initialized with "myxml" somewhere else, then the solution to your problem is, that you just don't add the xml element to your document, and the result looks as if you did so.