Develop Reference

Java DOM: getElementsByTagName is returning no elements?

I am trying to parse the XML document at http://web.mta.info/status/ServiceStatusSubway.xml and extract all the PtSituationElement elements with the following code:
DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
DocumentBuilder builder = factory.newDocumentBuilder();
Document subwayStatusDoc = builder.parse(new URL("http://web.mta.info/status/ServiceStatusSubway.xml").openStream());
NodeList situationList = subwayStatusDoc.getDocumentElement().getElementsByTagName("PtSituationElement");
System.out.println(situationList.item(0)); //prints null
What am I doing wrong here ?

The PtSituationElement tags contain child tags, so you need to go into those. Just printing .item(0) relies on the toString() method, and apparently it does not do a great job of explaining your nodes.
So add this to see some of the data in the child nodes:
Node item = situationList.item(0);
NodeList childNodes = item.getChildNodes();
for (int j = 0; j < childNodes.getLength(); j++) {
System.out.println(childNodes.item(j).getTextContent());
}
(I'm not sure what you want to do with the data in the xml structure, but this example shows how you can proceed with your work.)
Also, I noted that the LongDescription tags contain HTML that is not correct XML (<br clear=left> should be <br clear=left> etc). The parser could have a problem with that. It would be better if the HTML was escaped (see How to escape "&" in XML?).

XPath on DeferredDocumentImpl takes extremely long time to evaluate

In Java, I load an XML file from file like so which returns a DeferredDocumentImpl
private Document loadMasterFileXml(String path)
{
File masterFilePath = new File(path);
DocumentBuilderFactory masterDocBuilderFactory = DocumentBuilderFactory.newInstance();
DocumentBuilder masterDocBuilder = masterCbcCollBuilderFactory.newDocumentBuilder();
masterDocument = masterDocBuilder.parse(masterFilePath);
return masterDocument;
}
The XML file contains around 1000 elements like this:
<com.something.something.Collection xmi:version="2.0" xmlns:xmi="http://www.omg.org/XMI" xmlns:com.something.something.model="http://www.something.com/something.1.0.0" xmi:id="_HklwsJnWEeeaddrVFPWCMg" name="SOME_THING">
<signals xmi:id="_N0ir0ZnWEeeaddrVFPWCMg" id="10000">
<signal href="#_6M0edJhNEeeNvfntr9AQ8g"/>
</signals>
<signals xmi:id="_N0jS4JnWEeeaddrVFPWCMg" id="10001">
<signal href="#_6M1FgJhNEeeNvfntr9AQ8g"/>
</signals>
...
The first XPath operation on this document that is executed is as follows:
public long getMaximumSignalIdFromMasterDocument()
{
Integer errorCode=-1;
try
{
XPathFactory xPathfactory = XPathFactory.newInstance();
XPath xPath = xPathfactory.newXPath();
String expression = "//signals[not(#id < //signals/#id)]";
Node node = (Node) xPath.evaluate(expression, masterCbCollDocument, XPathConstants.NODE);
return Long.parseLong(node.getAttributes().getNamedItem("id").getNodeValue());
}
catch(Exception e)
{
return errorCode;
}
}
In debug mode the following line takes over 1 hour to execute.
Node node = (Node) xPath.evaluate(expression, masterCbCollDocument, XPathConstants.NODE);
Why is this?
Is it a problem with the XPath expression (usage of //)?
Is it due to the Document concrete implementation being deferred so there is too much file IO going on?
Can anyone suggest an alternative approach?

An alternative approach would be to refrain from using XPath. Although the hour-long computation seems more likely to be a trouble with the debugger/IDE you use, the XPath expression is not very efficient either (O(n^2)) and cannot be significantly optimized in XPath 1.0. Using Java directly in this case seems more appropriate. An approach could be:
NodeList signals = masterCbCollDocument.getElementsByTagName("signals");
long result = IntStream.range(0, signals.getLength()).mapToLong(i -> Long.parseLong(((Element)signals.item(i)).getAttribute("id"))).max().orElse(-1);
The masterCbCollDocument.getElementsByTagName does the same thing as //signals XPath expression in this case. The resulting signal elements within the NodeList are then mapped to their respective IDs and the maximum of them is returned.

Access data in xml as string

I am receiving a xml in string format. Is there any library to search for elements in the string?
<Version value="0"/>
<IssueDate>2017-12-15</IssueDate>
<Locale>en_US</Locale>
<RecipientAddress>
<Category>Primary</Category>
<SubCategory>0</SubCategory>
<Name>Vitsi</Name>
<Attention>Stowell Group Llc.</Attention>
<AddressLine1>511 6th St</AddressLine1>
<City>Lake Oswego</City>
<Country>United States</Country>
<PresentationValue>Lake Oswego OR 97034-2903</PresentationValue>
<State>OR</State>
<ZIPCode>97034</ZIPCode>
<ZIP4>2903</ZIP4>
</RecipientAddress>
<RecipientAddress>
<Category>Additional</Category>
<SubCategory>1</SubCategory>
<Name>Vitsi</Name>
<AddressLine1>Po Box 957</AddressLine1>
<City>Lake Oswego</City>
<Country>United States</Country>
<PresentationValue>Lake Oswego OR 97034-0104</PresentationValue>
<State>OR</State>
<ZIPCode>97034</ZIPCode>
<ZIP4>0104</ZIP4>
</RecipientAddress>
<SenderName>TMO</SenderName>
<SenderId>IL</SenderId>
<SenderAddress>
<Name>T-mobile</Name>
<AddressLine1>Po Box 790047</AddressLine1>
<City>St. Louis</City>
<PresentationValue>ST. LOUIS MO 63179-0047</PresentationValue>
<State>MO</State>
<ZIPCode>63179</ZIPCode>
.
.
.
.
I want to access the element RecipientAddress, which is a list. Is there any library to do that? Please note that what I receive is a string. It is an invoice and there will be many to process, so performance is important

Following options are available:
Convert xml string to java objects using JAXB.
Use .indexOf() in string method to retrieve specific parts of xml.
Use regular expression to retrieve specific parts of xml.
SAX/DOM/STAX parser for parsing and extraction from xml.
Xpath for fetching the specific values from xml.

You could use XPATH. Java has inbuilt support for XML querying without any thirdparty library,
Code piece would be,
String xmlInputStr = "<YOUR_XML_STRING_INPUT>"
String xpathExpressionStr = "<XPATH_EXPRESSION_STRING>"
DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
DocumentBuilder builder = factory.newDocumentBuilder();
Document doc = builder.parse(xmlInputStr);
XPathFactory xPathfactory = XPathFactory.newInstance();
XPath xpath = xPathfactory.newXPath();
XPathExpression expr = xpath.compile(xpathExpressionStr);
You can write your own expression string for querying. Typical example
"/RecipientAddress/Category"
Evaluate your xml against expression to retrieve list of nodes.
NodeList nodes = (NodeList) expr.evaluate(doc, XPathConstants.NODESET);
And iterate over nodes,
for (int i = 0; i < nodes.getLength(); i++) {
Node nNode = nodes.item(i);
...
}

There lot of pre-implemented api is available to convert xml to java object.
please look at that the xerces from Apache.
If you want extract only specified value the put whole in to string and use indexOf("string")

Why getting null node value while parsing XML

While parsing the below XML .First url-malformed-exception was coming while parsing so in the code instead of giving the xml String i used this code
Document doc=dBuilder.parse(newInputSource(newByteArrayInputStream(xmlResponse.getBytes("utf-8"))));
according to this link
java.net.MalformedURLException: no protocol
now i am getting the node value as null .How can i overcome this .In the code in for loop i have mentioned where the null value for node is coming
i am using following code:
try {
DocumentBuilderFactory dbFactory = DocumentBuilderFactory.newInstance();
DocumentBuilder dBuilder = dbFactory.newDocumentBuilder();
Document doc = dBuilder.parse(new InputSource(new ByteArrayInputStream(xmlResponse.getBytes("utf-8"))));
//read this - https://stackoverflow.com/questions/13786607/normalization-in-dom-parsing-with-java-how-does-it-work
doc.getDocumentElement().normalize();
System.out.println("Root element :" + doc.getDocumentElement().getNodeName());
XPath xPath = XPathFactory.newInstance().newXPath()
String expression = "/GetMatchingProductForIdResponse/GetMatchingProductForIdResult/Products/Product"
System.out.println(expression)
NodeList nodeList = (NodeList) xPath.compile(expression).evaluate(doc, XPathConstants.NODESET)
System.out.println("the size will be of the node list ${nodeList.getLength()}");
for (int i = 0; i < nodeList.getLength(); i++) {
System.out.println(nodeList.item(i).getNodeValue()+"the value coming will be "); // here i am getting value null for each node
}
} catch (Exception e) {
e.printStackTrace(System.out);
}
to parse the XML:
<?xml version="1.0"?>
<GetMatchingProductForIdResponse xmlns="http://mws.amazonservices.com/schema/Products/2011-10-01">
<GetMatchingProductForIdResult Id="H5-9OSH-9NZ7" IdType="SellerSKU" status="Success">
<Products xmlns="http://mws.amazonservices.com/schema/Products/2011-10-01" xmlns:ns2="http://mws.amazonservices.com/schema/Products/2011-10-01/default.xsd">
<Product>
<Identifiers>
<MarketplaceASIN>
<MarketplaceId>ATVPDKIKX0DER</MarketplaceId>
<ASIN>B004FQLAH2</ASIN>
</MarketplaceASIN>
</Identifiers>
<AttributeSets>
<ns2:ItemAttributes xml:lang="en-US">
<ns2:Binding>Office Product</ns2:Binding>
<ns2:Brand>Konica-Minolta</ns2:Brand>
<ns2:Color>Y</ns2:Color>
<ns2:CPUSpeed Units="MHz">200</ns2:CPUSpeed>
<ns2:Department>Printers</ns2:Department>
<ns2:Feature>Amp Up your Output - The magicolor 3730DN business color laser printer outputs at speeds up to 25 ppm in both color and B&W which means you can keep up in just about any business environment.</ns2:Feature>
<ns2:Feature>Unparalleled Image Quality - High resolution 2400 (equivalent) x 600 dpi printing for great color and clarity in both images and text.</ns2:Feature>
<ns2:Feature>Happy Planet, Outstanding Printing - Simitri HD Toner with Biomass allows for outstanding printing with the environment in mind.</ns2:Feature>
<ns2:Feature>Connect quicker - Why wait? Standard Ethernet and high-speed USB 2.0 gets you connected faster than ever before.Specifications</ns2:Feature>
<ns2:Feature>Type - Full-Color Laser Printer</ns2:Feature>
<ns2:ItemDimensions>
<ns2:Height Units="inches">13.62</ns2:Height>
<ns2:Length Units="inches">20.47</ns2:Length>
<ns2:Width Units="inches">16.50</ns2:Width>
<ns2:Weight Units="pounds">56.22</ns2:Weight>
</ns2:ItemDimensions>
<ns2:IsAutographed>false</ns2:IsAutographed>
<ns2:IsMemorabilia>false</ns2:IsMemorabilia>
<ns2:Label>Konica</ns2:Label>
<ns2:ListPrice>
<ns2:Amount>449.00</ns2:Amount>
<ns2:CurrencyCode>USD</ns2:CurrencyCode>
</ns2:ListPrice>
<ns2:Manufacturer>Konica</ns2:Manufacturer>
<ns2:Model>A0VD017</ns2:Model>
<ns2:NumberOfItems>1</ns2:NumberOfItems>
<ns2:OperatingSystem>Windows XP, Vista, 7</ns2:OperatingSystem>
<ns2:OperatingSystem>Mac X 10.2.8, 10.6+</ns2:OperatingSystem>
<ns2:PackageDimensions>
<ns2:Height Units="inches">19.00</ns2:Height>
<ns2:Length Units="inches">24.20</ns2:Length>
<ns2:Width Units="inches">22.00</ns2:Width>
<ns2:Weight Units="pounds">65.30</ns2:Weight>
</ns2:PackageDimensions>
<ns2:PackageQuantity>1</ns2:PackageQuantity>
<ns2:PartNumber>A0VD017</ns2:PartNumber>
<ns2:ProductGroup>CE</ns2:ProductGroup>
<ns2:ProductTypeName>PRINTER</ns2:ProductTypeName>
<ns2:Publisher>Konica</ns2:Publisher>
<ns2:SmallImage>
<ns2:URL>http://ecx.images-amazon.com/images/I/21qN3BU-BHL._SL75_.jpg</ns2:URL>
<ns2:Height Units="pixels">75</ns2:Height>
<ns2:Width Units="pixels">75</ns2:Width>
</ns2:SmallImage>
<ns2:Studio>Konica</ns2:Studio>
<ns2:Title>Konica Minolta Magicolor 3730DN Color Laser Printer 24PPM 2400X600DPI ENET USB 2.0</ns2:Title>
</ns2:ItemAttributes>
</AttributeSets>
<Relationships/>
<SalesRankings/>
</Product>
</Products>
</GetMatchingProductForIdResult>
<ResponseMetadata>
<RequestId>0b508338-3afe-4178-adc4-60c9c8448987</RequestId>
</ResponseMetadata>
</GetMatchingProductForIdResponse>

The getNodeValue method in the DOM is defined to always return null for element nodes (see the table at the top of the JavaDoc page for org.w3c.dom.Node for details). If you want the text inside the element then you should use getTextContent() instead.
You've added a second question in a comment to this answer asking how you can use an XPath to search for nodes that have a namespace prefix such as ns2:. The way XPath 1.0 handles namespaces is that unprefixed names always refer to nodes that are not in a namespace, and if you want to reference namespaced nodes then you have to provide a binding of namespace URIs to prefixes (which in javax.xml.xpath is the job of a NamespaceContext) and then use those prefixes in the expressions. The prefixes you use in the expression need not be the same ones as the original document used, as long as they bind to the right URIs.
Thus the original XPath you were using:
/GetMatchingProductForIdResponse/GetMatchingProductForIdResult/Products/Product
should not actually have matched anything, because the GetMatchingProductForIdResponse etc. elements in your document are in a namespace, but you got away with it because DocumentBuilderFactory is by default not namespace aware. The correct thing to do here is to use a namespace-aware parser, and provide a suitable namespace context to the XPath engine. There's no default implementation of NamespaceContext available in the core Java library, unfortunately, but Spring provides a convenient SimpleNamespaceContext implementation you can use if you don't want to roll your own.
DocumentBuilderFactory dbFactory = DocumentBuilderFactory.newInstance();
dbFactory.setNamespaceAware(true); // parse with namespaces
DocumentBuilder dBuilder = dbFactory.newDocumentBuilder();
Document doc = dBuilder.parse(new InputSource(new ByteArrayInputStream(xmlResponse.getBytes("utf-8"))));
doc.getDocumentElement().normalize();
XPath xPath = XPathFactory.newInstance().newXPath();
SimpleNamespaceContext nsCtx = new SimpleNamespaceContext();
xPath.setNamespaceContext(nsCtx);
nsCtx.bindNamespaceUri("prod", "http://mws.amazonservices.com/schema/Products/2011-10-01");
nsCtx.bindNamespaceUri("ns2", "http://mws.amazonservices.com/schema/Products/2011-10-01/default.xsd");
String expression = "/prod:GetMatchingProductForIdResponse/prod:GetMatchingProductForIdResult/prod:Products/prod:Product‌​/prod:AttributeSets/ns2:ItemAttributes/ns2:Binding";
// ...

How to program some XPath functions using Java Design Patterns

I need your help and your experience to realize the best java code using Design Patterns.
I must write some custom XPath functions that can:
Load a DOM document (I can use a mock object);
Check the validity of an user XPath expression;
Find and return the DOM node that satisfy the user expression.
I must evaluate only absolute expressions ( /... ) that can contain the path expression " .. " and predicates, embedded in square brackets, regarding attributes or leaf nodes, for examples:
/com/university/student/../exam
/com/university/exam[#tt = 'poo']/vote
/com/university/student/number[. = '1234']
I'll use the Composite pattern for the first step, the Chain of Resonsibility for the second step and a Visitor for the third step but I am not sure that this can be the best way to do this.
Can Chain of Resonsibility be usefull to check the validity?
All suggestions are welcome, thank you in advance for any help you can provide.

Isn't it a bit ... overcomplicated?
Create a DOM object for some XML input
Compile the user input - XPath will complain if it is not valid (XPathExpressionException)
Evalute the expression with the DOM object
Sample:
// #1 load document
DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();
dbf.setValidating(false);
DocumentBuilder db = dbf.newDocumentBuilder();
Document doc = db.parse(file);
// #2 - validate expression
XPathFactory factory = XPathFactory.newInstance();
XPath xpath = factory.newXPath();
XPathExpression expr = null;
try {
XPathExpression expr = xpath.compile(getExpression());
} catch (XPathExpressionException e) {
// ... handle & return <- invalid expression
}
// #3 evaluate expression
String result = expr.evaluate(doc);