Develop Reference

Unmarshal big xml files using JAXB

I have 1-2MB xml files and I would like to load them into the list of objects. I wanted to do it through JAXB because I have the appropriate XSD, only the problem appears when loading these files because it throws me errors saying that there is an unclosed tag somewhere, or something like that, when I check the file, there are no errors there. When I loaded files with a size of e.g. 40KB, there was no problem, everything was loading properly. So I understand there's a problem in jaxb when unmarshaling larger files. Is there any way to eat this? Another way of unmarshalling is unlikely to be included in the game because each xml file has a slightly different structure and objects that I have created with XSD.
The structure of xml files:
<Request>
<Header>
<Name> </Name>
<Id> </Id>
</Header>
<RequestItems>
<Request>
<Header>
<Name> </Name>
<Id> </Id>
</Header>
<ObjectName>
<City> </City>
<Street> </Street>
</ObjectName>
</Request>
</RequestItems>
</Request>
Inside the RequestItems tags there is a list of Request objects and in the Request ObjectName objects it is different depending on the file.
I do unmrashal in the simplest way and for smaller files it works fine, and the larger ones have strange syntax errors but I understand that it just cuts lines because it is too long and therefore there are syntactic errors.
JAXBContext jaxbContext = JAXBContext.newInstance (Request.class);
Unmarshaller jaxbUnmarshaller = jaxbContext.createUnmarshaller ();
Request request = (Request) jaxbUnmarshaller.unmarshal (xmlFile);

If I have understood your question correctly then this should work. I have used Moxy which is an extension of Jaxb and Project Lombok for Getter setter.
Imagine the following is the large XML with different structures request.xml:
<request>
<person>
<name>Batman</name>
<age>29</age>
<job>IT</job>
</person>
<animal>
<name>Tommy</name>
<age>5</age>
<type>Dog</type>
</animal>
<person>
<name>Superman</name>
<age>30</age>
<job>HR</job>
</person>
</request>
Following are the related classes and interface:
import jakarta.xml.bind.annotation.XmlSeeAlso;
#XmlSeeAlso({Person.class, Animal.class})
public interface XmlSupportExtension {
Object xmlSupport();
}
import jakarta.xml.bind.annotation.XmlAccessType;
import jakarta.xml.bind.annotation.XmlAccessorType;
import jakarta.xml.bind.annotation.XmlTransient;
import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.NoArgsConstructor;
import java.io.Serializable;
#Data
#NoArgsConstructor
#AllArgsConstructor
#XmlAccessorType(XmlAccessType.FIELD)
#XmlTransient
public class Common implements Serializable {
private String name;
private String age;
}
import lombok.*;
#Data
#EqualsAndHashCode(callSuper = true)
#ToString(callSuper = true)
#NoArgsConstructor
#AllArgsConstructor
public class Person extends Common implements XmlSupportExtension {
private String job;
#Override
public Person xmlSupport() {
return this;
}
}
import lombok.*;
#Data
#EqualsAndHashCode(callSuper = true)
#ToString(callSuper = true)
#NoArgsConstructor
#AllArgsConstructor
public class Animal extends Common implements XmlSupportExtension {
private String type;
#Override
public Animal xmlSupport() {
return this;
}
}
import jakarta.xml.bind.JAXBContext;
import jakarta.xml.bind.JAXBException;
import jakarta.xml.bind.Unmarshaller;
import javax.xml.stream.XMLInputFactory;
import javax.xml.stream.XMLStreamException;
import javax.xml.stream.XMLStreamReader;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.InputStream;
import java.util.Arrays;
public class MainStack {
public static void main(String[] args) throws JAXBException, XMLStreamException, FileNotFoundException {
final String[] EVENT_TYPES = new String[]{"person", "animal"};
InputStream inputStream = MainStack.class.getResourceAsStream("/request.xml");
//Create an instance of XMLStreamReader to read the events one-by-one
final XMLInputFactory inputFactory = XMLInputFactory.newInstance();
inputFactory.setProperty(XMLInputFactory.SUPPORT_DTD, Boolean.FALSE);
inputFactory.setProperty(XMLInputFactory.IS_SUPPORTING_EXTERNAL_ENTITIES, Boolean.FALSE);
final XMLStreamReader xmlStreamReader = inputFactory.createXMLStreamReader(inputStream);
//Create an instance of JAXBContext and Unmarshaller for unmarshalling the classes to respective event
final Unmarshaller unmarshaller = JAXBContext.newInstance(Person.class, Animal.class).createUnmarshaller();
//Navigate to next and start of the XML Elements
xmlStreamReader.next();
//Read Until the end of the file and unmarshall event-by-event
while (xmlStreamReader.hasNext()) {
//Check if the initial element is one of the elements from "EVENT_TYPES"
if (xmlStreamReader.isStartElement() && Arrays.asList(EVENT_TYPES).contains(xmlStreamReader.getLocalName())) {
//Get the event type
final String eventType = xmlStreamReader.getLocalName();
Object event = null;
System.out.println(eventType);
// Based on eventType make unmarshaller call to respective event class
switch (eventType) {
case "person":
//Unmarshal the Person
event = unmarshaller.unmarshal(xmlStreamReader, Person.class).getValue();
break;
case "animal":
//Unmarshal the Animal
event = unmarshaller.unmarshal(xmlStreamReader, Animal.class).getValue();
break;
default:
//If NONE of the event type matches then do not convert and make a note
System.out.println("XML event does not match any of the required event : " + event);
break;
}
System.out.println(" After Unmarhsalling : " + event.toString());
}
//Move to the next event/element in InputStream
xmlStreamReader.next();
}
}
}
This will produce the following output:
person
After Unmarhsalling : Person(super=Common(name=Batman, age=29), job=IT)
animal
After Unmarhsalling : Animal(super=Common(name=Tommy, age=5), type=Dog)

JAXB/XSD: Number instead of Element name

I have to create Java Objects from an xml file (that I can not edit before conversion), so I thought that it would probably be the best to create an XSD and then Unmarshall the file via JAXB.
The problem is, there are some elements in that XML that are formatted like this:
<parent-element>
<id-00001>...</id-00001>
<id-00002>...</id-00002>
...
</parent-element>
So basically I have a list of xml-elements, with the "name" of the element being the order inside of the list. There is a potentially unlimited number of elements in each of those lists.
Is it possible to unmarshall this with JAXB or do I HAVE TO use an XML-Reader and loop through the elements? The "id-XXXXX" elements are all formatted in the same way internally, so if it would be formatted like "" it would be possible without any issues.
Thanks in advance!

I guess that your XML payload looks like below:
<root>
<parent-element>
<id-00001>value 1</id-00001>
<id-00002>value 2</id-00002>
</parent-element>
</root>
To read nodes with dynamic names we need to write custom adapter (javax.xml.bind.annotation.adapters.XmlAdapter). Assume that model looks like below:
#XmlRootElement(name = "root")
#XmlAccessorType(XmlAccessType.FIELD)
class Root {
#XmlElement(name = "parent-element")
#XmlJavaTypeAdapter(IdsXmlAdapter.class)
private MultiItemList list;
// getters, setters, toString
}
class MultiItemList {
private List<String> ids;
// getters, setters, toString
}
For above XML and POJO model custom adapter could look like below:
class IdsXmlAdapter extends XmlAdapter<Object, MultiItemList> {
#Override
public MultiItemList unmarshal(Object v) {
Element element = (Element) v;
NodeList childNodes = element.getChildNodes();
List<String> ids = new ArrayList<>(childNodes.getLength());
for (int i = 0; i < childNodes.getLength(); i++) {
Node item = childNodes.item(i);
if (item.getNodeName().equals("#text")) {
continue;
}
ids.add(item.getTextContent());
}
MultiItemList multiItemList = new MultiItemList();
multiItemList.setIds(ids);
return multiItemList;
}
#Override
public Object marshal(MultiItemList v) throws Exception {
return null; // Implement if needed
}
}
Example usage:
import org.w3c.dom.Element;
import org.w3c.dom.Node;
import org.w3c.dom.NodeList;
import javax.xml.bind.JAXBContext;
import javax.xml.bind.annotation.XmlAccessType;
import javax.xml.bind.annotation.XmlAccessorType;
import javax.xml.bind.annotation.XmlElement;
import javax.xml.bind.annotation.XmlRootElement;
import javax.xml.bind.annotation.adapters.XmlAdapter;
import javax.xml.bind.annotation.adapters.XmlJavaTypeAdapter;
import java.io.File;
import java.io.FileReader;
import java.util.ArrayList;
import java.util.List;
public class JaxbApp {
public static void main(String[] args) throws Exception {
File xmlFile = new File("./resource/test.xml").getAbsoluteFile();
JAXBContext jaxbContext = JAXBContext.newInstance(Root.class);
Object unmarshal = jaxbContext.createUnmarshaller().unmarshal(new FileReader(xmlFile));
System.out.println(unmarshal);
}
}
prints:
Root{list=Root{ids=[value 1, value 2]}}
Notice, that you can not implement directly XmlAdapter<Element, MultiItemList> adapter because of exception:
Exception in thread "main" com.sun.xml.internal.bind.v2.runtime.IllegalAnnotationsException: 1 counts of IllegalAnnotationExceptions
org.w3c.dom.Element is an interface, and JAXB can't handle interfaces.
this problem is related to the following location:
at org.w3c.dom.Element
See also:
Read XML body element having multiple values based on different requests

Since your xml has potentially an unlimited number of "id-elements", it is tricky to create matching POJO structures. You can use #XmlAnyElement and an adapter. #XmlAnyElement will read the varying elements as dom nodes and you can process them in your adapter.
For example, with the xml given below:
<parent-element>
<id-00001>value 1</id-00001>
<id-00002>value 2</id-00002>
</parent-element>
Your root could look like this:
#XmlRootElement(name = "parent-element")
#XmlAccessorType(XmlAccessType.FIELD)
public class Root {
#XmlAnyElement
#XmlJavaTypeAdapter(MyAdapter.class)
private List<String> varyingElements;
}
and your adapter like this:
public class MyAdapter extends XmlAdapter<Element, String> {
#Override
public Element marshal(String arg0) throws Exception {
// do what you must
return null;
}
#Override
public String unmarshal(Element element) throws Exception {
return element.getChildNodes().item(0).getNodeValue();
}
}
Element as in org.w3c.dom.Element

Unmarshalling fails with no errors when setting namespace to #XmlRootElement

I have created a JAXB object and I am trying to unmarshal an xml string into it.
The problem that I am facing is that when I put the namespace property in the #XmlRootElement and in the xml document that I am sending, the JAXB object is getting created but it is empty. If I remove the namespace it works. So here is what I mean
My JAXB Object:
#XmlRootElement(name = "incident", namespace = "http://www.ba.com/schema/BAserviceDeskAPI/incident")
#XmlAccessorType(XmlAccessType.FIELD)
public class Incident {
#XmlElement
private String eventTitle;
public Incident() {
}
public String getEventTitle() {
return eventTitle;
}
public void setEventTitle(String eventTitle) {
this.eventTitle = eventTitle;
}
#Override
public String toString() {
StringBuilder builder = new StringBuilder();
builder.append("Incident [");
builder.append("eventTitle=");
builder.append(eventTitle);
builder.append("]");
return builder.toString();
}
}
My Main:
public static void main(String[] args) throws JAXBException {
String s = "<incident xmlns=\"http://www.ba.com/schema/BAserviceDeskAPI/incident\">"
+ "<eventTitle>Test Title from BAwrapper</eventTitle>"
+ "</incident>";
JAXBContext jaxbContext = JAXBContext.newInstance(Incident.class);
Unmarshaller jaxbUnmarshaller = jaxbContext.createUnmarshaller();
Incident incident = (Incident) jaxbUnmarshaller.unmarshal(new ByteArrayInputStream(s.getBytes(StandardCharsets.UTF_8)));
System.out.println(incident.toString());
}
}
Output:
Incident [eventTitle=null]
If I remove the , namespace = "http://www.ba.com/schema/BAserviceDeskAPI/incident" from the #XmlRootElement and the xmlns=\"http://www.ba.com/schema/BAserviceDeskAPI/incident\" from the xml sent I get the output below
Incident [eventTitle=Test Title from BAwrapper]
Any ideas why this happens?
Thanks

The namespace specified on #XmlRootElement only applies to that element. If you want it to apply to all the elements you have mapped to, you can do it at the package level using the #XmlSchema annotation.
package-info.java
#XmlSchema(
namespace = "http://www.ba.com/schema/BAserviceDeskAPI/incident",
elementFormDefault = XmlNsForm.QUALIFIED)
package example;
import javax.xml.bind.annotation.XmlNsForm;
import javax.xml.bind.annotation.XmlSchema;
For More Information
I have written more about JAXB and namespace qualification on my blog:
http://blog.bdoughan.com/2010/08/jaxb-namespaces.html
Additional Info
Unmarshalling fails with no errors when setting namespace to
#XmlRootElement
For JAXB we (the JSR-222 expert group) decided that an unmarshal shouldn't fail by default if there is unmapped content. Why? Because alot of XML documents contain extra content and things would be failing all the time. If you do want to see these errors then you can specify a ValidationEventHandler on the Unmarshaller.

Parse XML into DOM tree with custom object implementations in Java

I want to parse an XML document into a DOM tree in Java such that certain objects (e.g. instances of org.w3c.dom.Node or org.w3c.dom.Element) in the tree can be downcast to instances of classes that I have created, while minimizing the amount of XML-related code that I need to (re)implement. As a (very simple) example, if I have an XML element like:
<Vector size="5">
1.0 -1.0 3.0 -2.73e2
</Vector>
I would like to customize the parser to instantiate the following for it:
public class Vector extends /* some parser class */ {
private double[] elements;
/* constructors; etc.*/
public double dotProduct(Vector v) {
/* implementation */
}
}
such that I can pass instances of Vector created by the parser to, for example, javax.xml.xpath objects' methods and have them work correctly. What is the quickest way to achieve this? Is it possible with Java SE alone, or are third-party libraries (e.g. Xerces) necessary?

I'm not sure what your requirements are, but assuming you're in control of what the XML looks like, what I would use is XStream. It will allow you to skip all the DOM manipulation completely.
Now from their 2 minute tutorial, it may not seem like it's built for this use case, but it actually is. You create your java classes first, make sure they generate the XML the way you want it to look, and then use it to read your already existing XML back into your program as XStream objects. It's a very pleasant library to use.

Note: I'm the EclipseLink JAXB (MOXy) lead and a member of the JAXB (JSR-222) expert group.
The Binder mechanism in JAXB may be what you are looking for. It doesn't allow a DOM node to be cast to a domain object, but it does maintain a link between a domain object and its corresponding DOM node.
Note: The following code ran clean when using the MOXy as the JAXB provider, but threw an exception when using the impl of JAXB included in the version of the JDK I happen to be running.
JAVA MODEL
I will use the following domain model for this example.
Customer
import java.util.*;
import javax.xml.bind.annotation.*;
#XmlRootElement
public class Customer {
private List<PhoneNumber> phoneNumbers = new ArrayList<PhoneNumber>();
#XmlElementWrapper
#XmlElement(name="phoneNumber")
public List<PhoneNumber> getPhoneNumbers() {
return phoneNumbers;
}
}
PhoneNumber
import javax.xml.bind.annotation.*;
public class PhoneNumber {
private String type;
private String number;
#XmlAttribute
public String getType() {
return type;
}
public void setType(String type) {
this.type = type;
}
#XmlValue
public String getNumber() {
return number;
}
public void setNumber(String number) {
this.number = number;
}
}
jaxb.properties
To specify MOXy as your JAXB provider you need to include a file called jaxb.properties in the same package as your domain model with the following entry (see: http://blog.bdoughan.com/2011/05/specifying-eclipselink-moxy-as-your.html)
javax.xml.bind.context.factory=org.eclipse.persistence.jaxb.JAXBContextFactory
XML (input.xml)
<?xml version="1.0" encoding="UTF-8"?>
<customer>
<phoneNumbers>
<phoneNumber type="work">555-1111</phoneNumber>
<phoneNumber type="home">555-2222</phoneNumber>
</phoneNumbers>
</customer>
DEMO CODE
In the demo code below I will do the following:
Use XPath to find a child element, then use the Binder to find the corresponding domain object.
Update the domain object and use the Binder to apply the change to the DOM.
Update the DOM and use the Binder to apply the change to the domain object.
Demo
import javax.xml.bind.Binder;
import javax.xml.bind.JAXBContext;
import javax.xml.parsers.*;
import javax.xml.xpath.*;
import org.w3c.dom.*;
public class Demo {
public static void main(String[] args) throws Exception {
DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();
dbf.setNamespaceAware(true);
DocumentBuilder db = dbf.newDocumentBuilder();
Document document = db.parse("src/forum16599580/input.xml");
XPathFactory xpf = XPathFactory.newInstance();
XPath xpath = xpf.newXPath();
JAXBContext jc = JAXBContext.newInstance(Customer.class);
Binder<Node> binder = jc.createBinder();
binder.unmarshal(document);
// Use Node to Get Object
Node phoneNumberElement = (Node) xpath.evaluate("/customer/phoneNumbers/phoneNumber[2]", document, XPathConstants.NODE);
PhoneNumber phoneNumber = (PhoneNumber) binder.getJAXBNode(phoneNumberElement);
// Modify Object to Update DOM
phoneNumber.setNumber("555-2OBJ");
binder.updateXML(phoneNumber);
System.out.println(xpath.evaluate("/customer/phoneNumbers/phoneNumber[2]", document, XPathConstants.STRING));
// Modify DOM to Update Object
phoneNumberElement.setTextContent("555-2DOM");
binder.updateJAXB(phoneNumberElement);
System.out.println(phoneNumber.getNumber());
}
}
Output
555-2OBJ
555-2DOM

I have been through exactly this over the last 10 years, building XML DOMs for chemistry, graphics, maths, etc. My own solution has been to use a DOM where the elements can be subclassed (I use xom.nu but there are others). The w3c dom does not allow subclassing (IIRC) and so you would have to build a delegate model. (I tried this many years ago and rejected it, but software tools and libraries make all this much easier (e.g. the IDE will generate delegat methods).
If you are doing a lot, and especially if you are creating a lot of custom methods then I would recommend rolling your own system. The effort will be in your methods (dotProduct), not the XML.
Here, for example, is my class for a 3D point.
public class CMLPoint3 extends AbstractPoint3
(which extends the base class CMLElement, which extends nu.xom.Element
The creation of elements is a factory. Here's a chunk of my SVGDOM:
public static SVGElement readAndCreateSVG(Element element) {
SVGElement newElement = null;
String tag = element.getLocalName();
if (tag == null || tag.equals(S_EMPTY)) {
throw new RuntimeException("no tag");
} else if (tag.equals(SVGCircle.TAG)) {
newElement = new SVGCircle();
} else if (tag.equals(SVGClipPath.TAG)) {
newElement = new SVGClipPath();
} else if (tag.equals(SVGDefs.TAG)) {
newElement = new SVGDefs();
} else if (tag.equals(SVGDesc.TAG)) {
newElement = new SVGDesc();
} else if (tag.equals(SVGEllipse.TAG)) {
newElement = new SVGEllipse();
} else if (tag.equals(SVGG.TAG)) {
...
} else {
newElement = new SVGG();
newElement.setClassName(tag);
System.err.println("unsupported svg element: "+tag);
}
if (newElement != null) {
newElement.copyAttributesFrom(element);
createSubclassedChildren(element, newElement);
}
return newElement;
You can see tools for copying and recursing.
The questions you need to think about are:
how closely is this bound to an XSD
do I use XSD data Types
do I validate on input
am I using the DOM as the primary data structure (I do)
how frequently will things change.
FWIW I have been through 6 revisions of this and am contemplating another (using Scala as the main engine).

Can jaxb resolve system properties?

We use JAXB to configure object with XML. I was wondering if there is a way that JAXB could resolve system property. For instance, if I have a bean with property color, I would like to be able to do that:
<mybean color="${mycolor.in.data.property}" />
But if I do that, JAXB create the mybean object an color will be equals to the string:
mycolor.in.data.property
Is there is any equivalent of the PropertyPlaceholderConfigurer in spring for JAXB, so that my system property could be resolved?

An XmlAdapter is a JAXB (JSR-222) mechanism that allows you to convert an object to another object during marshalling/unmarshaller. You could write an XmlAdapter that converts the system property name to the real value.
import javax.xml.bind.annotation.adapters.XmlAdapter;
public class ColorAdapter extends XmlAdapter<String, String> {
#Override
public String unmarshal(String v) throws Exception {
return System.getProperty(v);
}
#Override
public String marshal(String v) throws Exception {
return v;
}
}
Then you use the #XmlJavaTypeAdapter annotation to configure the XmlAdapter for your property.
#XmlJavaTypeAdapter(ColorAdapter.class)
public String getColor() {
return color;
}
For More Information
http://blog.bdoughan.com/2011/05/jaxb-and-joda-time-dates-and-times.html
UPDATE
Ok thanks. Actually I do not have access to the class, as this is part
of an imported library. So I was more looking of a way to configure
this directly in the xml file, but it is probably not possible.
If you can't modify the class, then you can use a StreamReaderDelegate to modify the XML input. There a few methods that deal with the text/character data so you may need to experiment to be sure that you are overriding the one that works best with the JAXB implementation you are using.
import javax.xml.bind.*;
import javax.xml.stream.*;
import javax.xml.stream.util.StreamReaderDelegate;
import javax.xml.transform.stream.StreamSource;
public class Demo {
public static void main(String[] args) throws Exception {
JAXBContext jc = JAXBContext.newInstance(MyBean.class);
XMLInputFactory xif = XMLInputFactory.newFactory();
StreamSource source = new StreamSource("input.xml");
XMLStreamReader xsr = xif.createXMLStreamReader(source);
xsr = new StreamReaderDelegate(xsr) {
#Override
public String getText() {
String text = super.getText();
if(text.contains("${")) {
text = System.getProperty(text);
}
return text;
}
};
Unmarshaller unmarshaller = jc.createUnmarshaller();
unmarshaller.unmarshal(xsr);
}
}