I;m using XPath to parse XML document returned by a URL, when i run my code with given inputs it works but when giving it inputs as a user input it throws an exception.
The Code:
class{
private String generalQuery = "//#*";
method(){
System.out.println("Enter URL");
url = scan.nextLine();
URL oracle = new URL(url);
InputStream is = oracle.openStream();
org.w3c.dom.Document doc = null;
DocumentBuilderFactory domFactory;
DocumentBuilder builder;
try {
domFactory = DocumentBuilderFactory.newInstance();
domFactory.setNamespaceAware(true);
builder = domFactory.newDocumentBuilder();
doc = builder.parse(is);
} catch (Exception ex) {
System.err.println("unable to load XML: " + ex);
}
Map <String, String> params = new HashMap<String, String> ();
XPathFactory factory = XPathFactory.newInstance();
XPath xpath = factory.newXPath();
xpath.setNamespaceContext(new NameSpaces(doc));
XPathExpression expr = xpath.compile(generalQuery);
Object result = expr.evaluate(doc, XPathConstants.NODESET); // exception thrown here
NodeList nl = (NodeList) result;
for (int i = 0 ; i < nl.getLength() ; i++){
Node n = (Node)nl.item(i);
params.put(n.getNodeName(), n.getNodeValue());
}
return params;
}
}
The Exception:
javax.xml.transform.TransformerException: Unable to evaluate expression using this context
The class NameSpaces :
import java.util.Iterator;
import javax.xml.XMLConstants;
import javax.xml.namespace.NamespaceContext;
import org.w3c.dom.Document;
public class NameSpaces implements NamespaceContext {
private Document sourceDocument;
public NameSpaces(Document document) {
sourceDocument = document;
}
#Override
public String getNamespaceURI(String prefix) {
if (prefix.equals(XMLConstants.DEFAULT_NS_PREFIX)) {
return sourceDocument.lookupNamespaceURI(null);
} else {
return sourceDocument.lookupNamespaceURI(prefix);
}
}
#Override
public String getPrefix(String namespaceURI) {
return sourceDocument.lookupPrefix(namespaceURI);
}
#Override
public Iterator<String> getPrefixes(String namespaceURI) {
return null;
}
}
The exception "Unable to evaluate expression using this context" may also result from a null document when trying to evaluate an XPath expression. (I had the same error and it took me a while to figure out I did not initialize my document properly).
In your code you have
try {
// load document
}
catch (Exception ex) {
System.err.println("unable to load XML: " + ex);
}
// happily continue
This is a call for trouble. If an exception happens during initialization you should STOP right there and you should not continue. If you have absolutely no idea how to handle the error, use catch(Exception e) { throw new Error(e); }. This will cause exceptions to bubble up and hopefully be handled by the default exception handler which prints a stack trace and exits.
As the reader of your question I don't even know where the exception was thrown. You should provide this information. Note that you can also use someException.printStackTrace(); to get the stack trace which points you to the correct line.
What you seem to be missing is a NameSpaceContext that you can implement yourself.
Also see this thread: NamespaceContext and using namespaces with XPath
Example:
class NamespaceResolver implements NamespaceContext {
private final Document document;
public NamespaceResolver(Document document) {
this.document = document;
}
public String getNamespaceURI(String prefix) {
if (prefix.equals(XMLConstants.DEFAULT_NS_PREFIX)) {
return document.lookupNamespaceURI(null);
} else {
return document.lookupNamespaceURI(prefix);
}
}
public String getPrefix(String namespaceURI) {
return document.lookupPrefix(namespaceURI);
}
#SuppressWarnings("rawtypes")
public Iterator getPrefixes(String namespaceURI) {
// not implemented
return null;
}
}
Then you initiate the XPath instance like this:
getXPath().setNamespaceContext(new NamespaceResolver(doc));
In my case this was not due to a null document, but due to an empty document with no root element. Appending the latter solved the issue.
Related
I publish some csv input file on a server and it gives me a xml file that looks like this:
<ns0:TransportationEvent xmlns:ns0="http://www.server.com/schemas/TransportationEvent.xsd">
<ns0:deviceId>4567289456</ns0:deviceId>
.....
.....
</ns0:TransportationEvent>
<ns0:TransportationEvent xmlns:ns0="http://www.server.com/schemas/TransportationEvent.xsd">
<ns0:deviceId>7965145741</ns0:deviceId>
.....
.....
</ns0:TransportationEvent>
<ns0:TransportationEvent xmlns:ns0="http://www.server.com/schemas/TransportationEvent.xsd">
<ns0:deviceId>2168744654</ns0:deviceId>
.....
.....
</ns0:TransportationEvent>
The TransportationEvent tag would be added again and again with the updated deviceId in it.
I am retrieving data from this xml using XpathFactory class and NamespaceContext class which is shown as below:
NamespaceContext ctx = new NamespaceContext() {
public String getNamespaceURI(String prefix) {
String uri;
if (prefix.equals("ns0"))
uri = "http://www.server.com/schemas/TransportationEvent.xsd";
else
uri = null;
return uri;
}
public Iterator getPrefixes(String val) {
return null;
}
// Dummy implementation - not used!
public String getPrefix(String uri) {
return null;
}
};
XPathFactory xpathFact = XPathFactory.newInstance();
XPath xpath = xpathFact.newXPath();
xpath.setNamespaceContext(ctx);
String strXpath = "//ns0:TransportationEvent/ns0:deviceId/text()";
String deviceId = xpath.evaluate(strXpath, doc);
The above code gives the value of deviceId as 4567289456. Basically it always take values from the first TransportationEvent tag.
I need to pick data from that "TransportationEvent" tag where the "deviceId" is equal to the deviceId of my choice. Something like this:
String strXpath = "//ns0:TransportationEvent[where ns0:deviceId = " + myDeviceId + "]/ns0:deviceId/text()";
I can perform this by using NodeList class and can iterate through all the "TransportationEvent" tags but then I would not be able to use the Xpath or NamespaceContext implementation. I am finding no connection between the NodeList class and the NamespaceContext class or the Xpath class.
I want to get the value of ctx which has the context of the desired TransportationEvent tag.
I know I am missing something. Could somebody help please?
You could fetch the parent node of the deviceId you are intrested in like this:
//ns0:deviceId[text()='7965145741']/parent::node()
private static final String NS0_NS = "http://www.server.com/schemas/TransportationEvent.xsd";
private static final String NS0 = "ns0";
private static final List<String> prefixes = Arrays.asList(NS0);
public void fromDocument(Document doc) throws XPathExpressionException, TransformerConfigurationException,
TransformerFactoryConfigurationError, TransformerException {
XPath xpath = XPathFactory.newInstance().newXPath();
xpath.setNamespaceContext(new NamespaceContext() {
#Override
public Iterator getPrefixes(String namespaceURI) {
return prefixes.iterator();
}
#Override
public String getPrefix(String namespaceURI) {
String res = namespaceURI.equals(NS0_NS)?NS0:null;
return res;
}
#Override
public String getNamespaceURI(String prefix) {
String res = prefix.equals(NS0)?NS0_NS:null;
return res;
}
});
XPathExpression devex = xpath.compile("//ns0:deviceId[text()='7965145741']/parent::node()");
Node node = (Node) devex.evaluate(doc,XPathConstants.NODE);
Transformer xformer = TransformerFactory.newInstance().newTransformer();
xformer.transform(new DOMSource(node),new StreamResult(System.out));
}
that's the output:
<ns0:TransportationEvent xmlns:ns0="http://www.server.com/schemas/TransportationEvent.xsd">
<ns0:deviceId>7965145741</ns0:deviceId>
</ns0:TransportationEvent>
I am trying to customize xml plugin of Sonarqube to check Json syntax. My logic to check Json is working fine when I test it by providing a particular xml file. But when in plugin,it fails.
Here is my code inside the package org.sonar.plugins.xml.checks
#Rule(
key = "JsonCheck",
name = "Json should be in correct format",
description = "This rule is for checking Json",
priority = Priority.CRITICAL,
tags = {"convention"})
#BelongsToProfile(title = CheckRepository.SONAR_WAY_PROFILE_NAME, priority = Priority.CRITICAL)
#SqaleSubCharacteristic(RulesDefinition.SubCharacteristics.READABILITY)
#SqaleConstantRemediation("1min")
#RuleTemplate
public class JsonCheck extends AbstractXmlCheck {
#RuleProperty(key = "Json", description = "This rule is for checking Json", type = "TEXT")
private static final String MESSAGE = "Please provide valid Json";
#Override
public void validate(XmlSourceCode xmlSourceCode) {
setWebSourceCode(xmlSourceCode);
Document document = getWebSourceCode().getDocument(true);
if (document.getDocumentElement() != null) {
validateJson(document.getDocumentElement(),document);
}
}
private void validateJson(Node node, Document document)
{
String JsonString = getJsonString(document);
boolean Json = ValidateJson(JsonString);
if (Json == false) {
createViolation(getWebSourceCode().getLineForNode(node), String.format(MESSAGE));
}
}
private String getJsonString(Document document)
{
XPathFactory xpathFactory = XPathFactory.newInstance();
XPath xpath = xpathFactory.newXPath();
XPathExpression expr;
try {
expr = xpath.compile("//Payload");
return (String) expr.evaluate(document, XPathConstants.STRING);
} catch (XPathExpressionException e) {
e.printStackTrace();
return "";
}
}
private boolean ValidateJson(String JsonString)
{
boolean bool;
Gson gson = new Gson();
try {
gson.fromJson(JsonString, Object.class);
bool = true;
return bool;
} catch(com.google.gson.JsonSyntaxException ex) {
bool = false;
return bool;
}
}
}
I am not able to understand how this plugin is iterating through all xml files in the project based on which I should change my logic? Or am I missing something here?
I am trying to read from xml file but I get a null pointer exception.
this is the xml file:
<war>
<missileLaunchers>
<launcher id="L101" isHidden="false">
<missile id="M1" destination="Sderot" launchTime="2" flyTime="12" damage="1500"/>
<missile id="M2" destination="Beer-Sheva" launchTime="5" flyTime="7" damage="2000"/>
</launcher>
<launcher id="L102" isHidden="true">
<missile id="M3" destination="Ofakim" launchTime="4" flyTime="3" damage="5000"/>
<missile id="M4" destination="Beer-Sheva" launchTime="9" flyTime="7" damage="1000"/>
</launcher>
</missileLaunchers>
<missileDestructors >
<destructor id="D201">
<destructdMissile id="M1" destructAfterLaunch="4"/>
<destructdMissile id="M3" destructAfterLaunch="7" />
<destructdMissile id="M4" destructAfterLaunch="2"/>
</destructor>
<destructor id="D202">
<destructdMissile id="M2" destructAfterLaunch="3"/>
</destructor>
</missileDestructors>
<missileLauncherDestructors >
<destructor type="plane" >
<destructedLanucher id="L101" destructTime="4"/>
</destructor>
<destructor type="ship">
<destructedLanucher id="L102" destructTime="8" />
<destructedLanucher id="L102" destructTime="12"/>
</destructor>
</missileLauncherDestructors>
</war>
and this is the code:
public class XmlReader
{
File fXmlFile=null;
DocumentBuilderFactory dbFactory=null;
DocumentBuilder dBuilder=null;
Document doc=null;
public XmlReader(String filePath) throws ClassNotFoundException
{
if(filePath!=null)
{
this.fXmlFile = new File(filePath);
dbFactory = DocumentBuilderFactory.newInstance();
try {
dBuilder = dbFactory.newDocumentBuilder();
} catch (ParserConfigurationException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
try {
doc = dBuilder.parse(fXmlFile);
doc.getDocumentElement().normalize();
} catch (SAXException | IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
else System.out.println("Xml file not found");
}
//gets value by tag name
private static String getTagValue(String tag, Element element) {
if(element.hasChildNodes())
{
NodeList nodeList = element.getElementsByTagName(tag).item(0).getChildNodes();
Node node = (Node) nodeList.item(0);
if(node==null)
return null;
return node.getNodeValue();
}
else return element.getNodeValue();
}
//launcher
public List<Launcher> readLauncher() throws Exception
{
List<Launcher> launcherList = new ArrayList<Launcher>();
try
{
NodeList nList = doc.getElementsByTagName("launcher");
for(int i=0;i<nList.getLength();i++)
{launcherList.add(getLauncher(nList.item(i)));}
}
catch (Exception e)
{
e.printStackTrace();
}
return launcherList;
}
//builds the object
private static Launcher getLauncher(Node node)
{
//XMLReaderDOM domReader = new XMLReaderDOM();
Launcher launcher = new Launcher();
if (node.getNodeType() == Node.ELEMENT_NODE)
{
Element element = (Element) node;
// launcher.setIsHidden(Boolean.parseBoolean(getTagValue("isHidden", element)));
// launcher.setId(getTagValue("id", element));
System.out.println("id = "+getTagValue("id", element));
System.out.println("ishidden = "+getTagValue("isHidden", element));
}
return launcher;
}
}
And this is the stack trace:
java.lang.NullPointerException
at XmlReader.getTagValue(XmlReader.java:56)
at XmlReader.getLauncher(XmlReader.java:96)
at XmlReader.readLauncher(XmlReader.java:78)
at Program.main(Program.java:27)
I can not change the format of the xml file.
It seems to fail when it tries to get the actual value of the node's fields or so I assume.
Though I don;t understand the reason...when I check the size of the node list it turns fine it does give me 2.
The problem is below line:
System.out.println("id = " + getTagValue("id", element));
where getTagValue("id", element) is calling
NodeList nodeList = element.getElementsByTagName(tag).item(0).getChildNodes();
Here element.getElementsByTagName("id") will return null
It should be get from attribute
// gets value by tag name
private static String getTagValue(String tag, Element element) {
return element.getAttributeNode(tag).getValue();
}
You are calling getElementsByTagName() in getTagValues, however you are trying to retrieve attributes of the tag. You may need to call getAttribute() instead. For Example:
element.getAttribute(attributeName)
where attributeName is "id" or "isHidden". This will return the value as a String and can be returned directly with no further processing.
My xml String is
Got message from Queue ==> <?xml version='1.0' encoding='UTF-8'?><soapenv:Envelope xmlns:soapenv="http://www.w3.org/2003
/05/soap-envelope"><soapenv:Body><ns1:PostPublicationResponse xmlns:ns1="http://www.openoandm.org/xml/ISBM/"><ns1:Messag
eID>urn:uuid:7d361fb0-bc54-48bd-bbd1-6e34960ef3f8</ns1:MessageID><ns1:MessageContent><MessageContent xmlns="http://www.o
penoandm.org/xml/ISBM/"><hi>k786</hi></MessageContent></ns1:MessageContent></ns1:PostPublicationResponse></soapenv:Body>
</soapenv:Envelope>
Now i have writtent a function that is trying to get Content of element MessageContent i.e <hi>k786</hi> but i am getting null value always.
My function to parse above xml is:
private String parseQueueMessage(String message)
throws ParserConfigurationException, SAXException, IOException,
XPathExpressionException {
String resultMsg = "";
DocumentBuilderFactory domFactory = DocumentBuilderFactory
.newInstance();
domFactory.setNamespaceAware(true);
DocumentBuilder builder = domFactory.newDocumentBuilder();
Document doc = builder.parse(new InputSource(new java.io.StringReader(
message)));
XPath xpath = XPathFactory.newInstance().newXPath();
// XPath Query for showing all nodes value
xpath.setNamespaceContext(new NamespaceContext() {
#SuppressWarnings("rawtypes")
#Override
public Iterator getPrefixes(String arg0) {
return null;
}
#Override
public String getPrefix(String arg0) {
return null;
}
#Override
public String getNamespaceURI(String arg0) {
if("xmlns:ns1".equals(arg0)) {
return "http://www.openoandm.org/xml/ISBM/";
}
return null;
}
});
XPathExpression expr = xpath.compile("//xmlns:ns1:MessageContent");
Object result = expr.evaluate(doc, XPathConstants.NODESET);
NodeList nodes = (NodeList) result;
for (int i = 0; i < nodes.getLength(); i++) {
System.out.println("The message obtained after parsing : "
+ nodes.item(i).getNodeValue());
resultMsg = nodes.item(i).getNodeValue();
}
return resultMsg;
}
What i have done wrong in here?
Thanks in advance
You need to define the name space URI first before selecting from XPATH. For example, first define the namespace URI as follows on the root;
element.setAttribute("xmlns:ns1", "http://www.openoandm.org/xml/ISBM/");
xpath.compile("//ns1:MessageContent");
//Try something like ...
XmlDocument doc = new XmlDocument();
doc.LoadXml("urn:uuid:7d361fb0-bc54-48bd-bbd1-6e34960ef3f8k786
");
XmlElement elem = (XmlElement) doc.DocumentElement.FirstChild;
Console.Write("{0}:{1} = {2}", elem.Prefix, elem.LocalName, elem.InnerText);
Console.WriteLine("\t namespaceURI=" + elem.NamespaceURI);
I have xml like below (Google API), but can't get gphoto:id element value. How to do that ? Notice: When i'm using domFactory.setNamespaceAware(true);, /feed/entry xpath stops working.
<?xml version="1.0" encoding="UTF-8"?>
<feed xmlns="http://www.w3.org/2005/Atom"
xmlns:gphoto="http://schemas.google.com/photos/2007"
xmlns:media="http://search.yahoo.com/mrss/"
xmlns:openSearch="http://a9.com/-/spec/opensearchrss/1.0/">
<entry>
<title type="text">Test</title>
<author>
<name>username</name>
<uri>https://picasaweb.google.com/113422203255202384532</uri>
</author>
<gphoto:id>57060151229174417</gphoto:id>
</entry>
</feed>
Java
NodeList nodes = (NodeList) path(body, "/feed/entry", XPathConstants.NODESET);
for (int i = 0; i < nodes.getLength(); i++) {
Node n = nodes.item(i);
XPath xpath = XPathFactory.newInstance().newXPath();
// empty :(
System.out.println(
xpath.evaluate("id[namespace-uri()='http://schemas.google.com/photos/2007']",n)
);
// empty too :(
System.out.println(
xpath.evaluate("gphoto:id",n)
);
// ok
System.out.println(
xpath.evaluate("author",n)
);
l.add(new Album("", "", ""));
}
path method
private Object path(String content, String path, QName returnType) {
try {
DocumentBuilderFactory domFactory = DocumentBuilderFactory.newInstance();
DocumentBuilder builder = domFactory.newDocumentBuilder();
Document doc = builder.parse(new InputSource(new StringReader(content)));
XPath xpath = XPathFactory.newInstance().newXPath();
XPathExpression expr = xpath.compile(path);
return expr.evaluate(doc, returnType);
} catch (Exception e) {
e.printStackTrace();
}
return null;
}
SOLVED according to #gioele answer path() method is now like below:
private Object path(String content, String path, QName returnType) {
try {
domFactory.setNamespaceAware(true);
DocumentBuilder builder = domFactory.newDocumentBuilder();
Document doc = builder.parse(new InputSource(new StringReader(content)));
XPath xpath = XPathFactory.newInstance().newXPath();
NamespaceContext nsContext = new NamespaceContext() {
#Override
public Iterator getPrefixes(String namespaceURI) {
return null;
}
#Override
public String getPrefix(String namespaceURI) {
return null;
}
#Override
public String getNamespaceURI(String prefix) {
if ("gphoto".equals(prefix))
return "http://schemas.google.com/photos/2007";
if ("media".equals(prefix))
return "http://search.yahoo.com/mrss/";
if("".equals(prefix))
return "http://www.w3.org/2005/Atom";
throw new IllegalArgumentException(prefix);
}
};
xpath.setNamespaceContext(nsContext);
XPathExpression expr = xpath.compile(path);
return expr.evaluate(doc, returnType);
} catch (Exception e) {
e.printStackTrace();
}
return null;
}
Before compiling your xpath you need to register a NamespaceContext.
Have a look at the code in https://github.com/gioele/xpathapi-jaxp/blob/master/src/main/java/it/svario/xpathapi/jaxp/NodeNamespaceContext.java.
If you want to avoid all these complications, you can use the XPathAPI library:
Map<String, String> nsMap = new HashMap<String, String>();
nsMap.put(XMLConstants.DEFAULT_NS_PREFIX, "http://www.w3.org/2005/Atom");
nsMap.put("gphoto", "http://schemas.google.com/photos/2007");
List<Node> entries = XPathAPI.selectListOfNodes(doc, "/feed/entry", nsMap);
for (Node entry : entries) {
String id = XPathAPI.selectSingleNodeAsString(entry, "gphoto:id", nsMap);
// or, if you prefer a Node
// Node id = XPathAPI.selectSingleNode(entry, "gphoto:id", nsMap);
}
Disclaimer: I am the creator of XPathAPI-JAXP.
A much easier way to deal with the namespace issue is just to redirect the call from the NamespaceContext to the document lookupNamespaceURI() method. This will return "http://search.yahoo.com/mrss/" when called with "media" etc...
xPath.setNamespaceContext(new NamespaceContext() {
#Override
public String getNamespaceURI(String prefix) {
return doc.lookupNamespaceURI(prefix);
}
#Override
public Iterator<?> getPrefixes(String arg0) {
return null;
}
#Override
public String getPrefix(String arg0) {
return null;
}
});