Develop Reference

How to improve performance of JAXB/StAX XML output

I am attempting to write out a very large XML object, using the code below. I am processing 200K-350K objects/nodes, and the output-to-file is unbearably slow.
Any suggestions on how to improve the performance of the output implementation? I understand that the IndentingXMLStreamWriter may be one of the culprits, but I really need the output to be human readable (even if it is likely not going to be read due to size).
driver implementation...
public class SomeClient {
public static void main(String args[]) {
TransactionXmlWriter txw = new TransactionXmlWriter();
TransactionType tranType = getNextTransaction();
try {
txw.openXmlOutput("someFileName.xml");
while(tranType != null) {
txw.processObject(tranType);
tranType = getNextTransaction();
}
txw.closeXmlOutput();
} catch(JAXBException e) {
} catch(FileNotFoundException e) {
} catch(XMLStreamExceptoin e) {
}
}
}
implementation class...
public class TransactionXmlWriter {
private final QName root = new QName("ipTransactions");
private Marshaller marshaller = null;
private FileOutputStream fileOutputStream = null;
private XMLOutputFactory xmlOutputFactory = null;
private XMLStreamWriter xmlStreamWriter = null;
// constructor
public TransactionXmlWriter() throws JAXBException{
JAXBContext jaxbContext = JAXBContext.newInstance(TransactionType.class);
xmlOutputFactory = XMLOutputFactory.newFactory();
marshaller = jaxbContext.createMarshaller();
marshaller.setProperty(Marshaller.JAXB_FRAGMENT, true);
}
// write out "body" of XML
public void processObject(TransactionType transaction) {
JAXBElement<TransactionType> transactionJaxB = null;
try {
transactionJaxB = new JAXBElement<>(root, TransactionType.class, transaction);
marshaller.marshal(transactionJaxB, xmlStreamWriter);
} catch(JAXBException e) {
// TO DO : some kind of error handling
System.out.println(e.getMessage());
System.out.println(e.getStackTrace());
}
}
// open file to write XML into
public void openXmlOutput(String fileName) throws FileNotFoundException,
XMLStreamException {
fileOutputStream = new FileOutputStream(fileName);
xmlStreamWriter = new IndentingXMLStreamWriter(xmlOutputFactory.createXMLStreamWriter(fileOutputStream));
writeXmlHeader();
}
// write XML footer and close the stream/file
public void closeXmlOutput() throws XMLStreamException {
writeXmlFooter();
xmlStreamWriter.close();
}
private void writeXmlHeader() throws XMLStreamException {
xmlStreamWriter.writeStartDocument("UTF-8", "1.0");
xmlStreamWriter.writeStartElement("ipTransactions");
}
private void writeXmlFooter() throws XMLStreamException {
xmlStreamWriter.writeEndElement();
xmlStreamWriter.writeEndDocument();
}
}

Nullpointerexception in lucene custom analyser

Why I am getting nullpointerexception in ts.reset() line in InputFile class? If I use any inbuilt analyser like whitespaceanalyser, I don't get any exception. What is the problem here?
public class CourtesyTitleFilter extends TokenFilter
{
TokenStream input;
Map<String,String> courtesyTitleMap = new HashMap<String,String>();
private CharTermAttribute termAttr;
public CourtesyTitleFilter(TokenStream input) throws IOException
{
super(input);
termAttr = input.addAttribute(CharTermAttribute.class);
courtesyTitleMap.put("Dr", "doctor");
courtesyTitleMap.put("Mr", "mister");
courtesyTitleMap.put("Mrs", "miss");
}
#Override
public boolean incrementToken() throws IOException
{
if (!input.incrementToken())
return false;
String small = termAttr.toString();
if(courtesyTitleMap.containsKey(small)) {
termAttr.setEmpty().append(courtesyTitleMap.get(small));
System.out.print(courtesyTitleMap.get(small));
}
return true;
}
}
public class CourtesyTitleAnalyzer extends Analyzer
{
#Override
protected TokenStreamComponents createComponents(String fieldName, Reader reader)
{
TokenStream filter = null;
Tokenizer whitespaceTokenizer = new WhitespaceTokenizer(reader);
try
{
filter = new CourtesyTitleFilter (whitespaceTokenizer);
}
catch(IOException e)
{
e.printStackTrace();
}
return new TokenStreamComponents(whitespaceTokenizer,filter);
}
}
public class InputFile
{
public static void main(String[] args) throws IOException, ParseException
{
TokenStream ts=null;
CourtesyTitleAnalyzer cta=new CourtesyTitleAnalyzer();
try
{
StringReader sb=new StringReader("Hello Mr Hari. Meet Dr Kalam and Mrs xyz");
ts = cta.tokenStream("field",sb);
OffsetAttribute offsetAtt = ts.addAttribute(OffsetAttribute.class);
CharTermAttribute termAtt = ts.addAttribute(CharTermAttribute.class);
ts.reset();
while (ts.incrementToken())
{
String token = termAtt.toString();
System.out.println("[" + token + "]");
System.out.println("Token starting offset: " + offsetAtt.startOffset());
System.out.println(" Token ending offset: " + offsetAtt.endOffset());
System.out.println("");
}
ts.end();
}
catch (IOException e)
{
e.printStackTrace();
}
finally
{
ts.close();
cta.close();
}
}
}

input is already defined in the TokenFilter abstract class. You are hiding it by declaring it in your implementation.
So, just delete the line TokenStream input; in your CourtesyTitleFilter.

Reading object from a file and saving it into ArrayList

I have a problem with reading specific object from a file and saving it into ArrayList.
First I write a single customer using writeCustomer(). Then I write all records from List customerList and save them to the file. This works great.
Then I want to read the saved file so I read one line using readCustomer(). This method returns one Customer and then I want to return a list with all Clients using readData() and read it, I have nullPointerException in line list.add(readCustomer(bufferedReader));
My Class Customer has one constructor and is has an override method toString().
public class SaveCustomers {
public static void main(String[] args) throws IOException {
List<Customers> customersList = new ArrayList<>();
customersList.add(new Customers("ABC", 10));
customersList.add(new Customers("SGS", 20));
customersList.add(new Customers("FSD", 30));
try (PrintWriter out = new PrintWriter("customer.txt", "UTF-8"))
{ writeData(customersList, out); }
BufferedReader bufferedReader = new BufferedReader(new FileReader("customer.txt"));
List<Customers> newList = readData(bufferedReader);
for(Customers c: newList){
System.out.println(c);
}
}
private static void writeCustomer(PrintWriter out, Customers customers){
out.println(customers.getName()+"|"+customers.getTarrif());
}
private static void writeData(List<Customers> customersList, PrintWriter out){
for(Customers c:customersList){
writeCustomer(out, c);
}
}
public static Customers readCustomer(BufferedReader bufferedReader) throws IOException {
String line = bufferedReader.readLine();
String [] tokens = line.split("\\|");
String name = tokens[0];
int time = Integer.valueOf(tokens[1]);
return new Customers(name, time);
}
public static List<Customers> readData(BufferedReader bufferedReader) throws IOException {
List<Customers> list = new ArrayList<>();
while (bufferedReader.readLine() != null) {
list.add(readCustomer(bufferedReader));
}
return list;
}}

You are close to the solution ;)
In method :
readData(BufferedReader bufferedReader)
Just change this line
for(Customers l : list) {
to this one :
while (bufferedReader.ready()) {

How to use ArrayList while adding something to another Class's constructor ?

I'm try to create one simple reservation system, we'll read a file, then we'll add Train, Bus, etc., then we'll writer everything to output.
import java.io.*;
import java.util.*;
public class Company
{
private static ArrayList<Bus> bus = new ArrayList<Bus>();
static int buscount = 0, traincount = 0;
public static void main (String[] args) throws IOException
{
FileParser();
}
public Company()
{
}
public static void FileParser()
{
try {
File file = new File(); //i fill this later
File file2 = new File(); // i fill this later
FileInputStream fis = new FileInputStream(file);
FileOutputStream fos = new FileOutputStream(file2);
BufferedReader br = new BufferedReader(new InputStreamReader(fis));
BufferedWriter bw = new BufferedWriter(new OutputStreamWriter(fos));
String line;
while ((line = br.readLine()) != null)
{
String[] splitted = line.split(",");
if(splitted[0].equals("ADDBUS"))
{
bus.add(buscount) = Bus(splitted[0],splitted[1],splitted[2],splitted[3],splitted[4],splitted[5]);
}
}
}
catch (FileNotFoundException fnfe) {
}
catch (IOException ioe) {
}
}
}
I try to read the file line by line. For example one of the line is "ADDBUS,78KL311,10,140,54" I split the line for "," then i try to add every pieces of array to Bus' class' constructor but i couldn't figured it out.
My Bus Class is like `
public class Bus extends Vehicle{
private String command;
private String busName;
private String busPlate;
private String busAge;
private String busSpeed;
private String busSeat;
public Bus(String command, String busname, String busplate, String busage, String busspeed, String busseat)
{
this.command = command;
this.busName = busname;
this.busPlate = busplate;
this.busAge = busage;
this.busSpeed = busspeed;
this.busSeat = busseat;
}
public String getBusName() {
return busName;
}
public void setBusName(String busName) {
this.busName = busName;
}
public String getBusPlate() {
return busPlate;
}
public void setBusPlate(String busPlate) {
this.busPlate = busPlate;
}
public String getBusAge() {
return busAge;
}
public void setBusAge(String busAge) {
this.busAge = busAge;
}
public String getBusSpeed() {
return busSpeed;
}
public void setBusSpeed(String busSpeed) {
this.busSpeed = busSpeed;
}
public String getBusSeat() {
return busSeat;
}
public void setBusSeat(String busSeat) {
this.busSeat = busSeat;
}
public String getCommand() {
return command;
}
public void setCommand(String command) {
this.command = command;
}
}
can someone show me a way to solve this problem?
Thank you,

You are missing the keyword new to create a new instance of the class:
bus.add(new Bus(...));

You can add items to ArrayList like this
bus.add( new Bus(splitted[0],splitted[1],splitted[2],splitted[3],splitted[4],splitted[5]));
you were missing new keyword before Bus constructor call. Then you can increment the counter (or do whatever)
bus.add( new Bus(splitted[0],splitted[1],splitted[2],splitted[3],splitted[4],splitted[5]));
buscount++;

try to add new Bus(...)
bus.add( new
Bus(splitted[0],splitted[1],splitted[2],splitted[3],splitted[4],splitted[5]));

As I understand if you want to call constructor you need to call new Bus(parms).
when you say new it will call constructor of your class
when you say this() again it going to call enclosing class' constructor
if you say super() it will call super class' constructor.

if you want it into a map order by counter you can use this:
Map(Integer, Bus) busPosition = new HashMap<>();
busPosition.put(buscount, new
Bus(splitted[0],splitted[1],splitted[2],splitted[3],splitted[4],splitted[5]));

ObjectInputStream[Java]

i'm new to file i/o so i'm sorry if this is a pretty bad question.
Currently I have an add method/main method and a person class my outputstream is working fine in the add method: This is at the top of the method
FileOutputStream myFile = null;
try {
myFile = new FileOutputStream("txt123.txt");
} catch (FileNotFoundException e2) {
// TODO Auto-generated catch block
e2.printStackTrace();
}
ObjectOutputStream oos = null;
try {
oos = new ObjectOutputStream(myFile);
} catch (IOException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
And I then have this twice because there are two types of people that can be added
oos.writeObject(person);
oos.close();
System.out.println("Done");
So my question, how do i get the input working and finally where do i put it, in the add method or the main method, I read how to do what i done here: http://www.mkyong.com/java/how-to-write-an-object-to-file-in-java/
He also has a guide on reading in the objects but I cant seem to get it working
Thanks!

You would be reading the file you just created like this:
ObjectInputStream in =
new ObjectInputStream(new FileInputStream("txt123.txt"));
// terrible file name, because this is binary data, not text
try{
Person person = (Person) in.readObject();
finally{
in.close();
}

You can combine the ObjectOutputStream with the FileOutputStream as follows. I'm also guessing you need to place the read/write code in one place to allow re-use. Here's a simple example with the read/write in a DAO.
public static class Person implements Serializable {
private String name;
public Person(String name) {
super();
this.name = name;
}
public String getName() {
return name;
}
#Override
public String toString() {
return name;
}
}
public static class PersonDao {
public void write(Person person, File file) throws IOException {
ObjectOutputStream oos = new ObjectOutputStream(
new FileOutputStream(file));
oos.writeObject(person);
oos.close();
}
public Person read(File file) throws IOException,
ClassNotFoundException {
ObjectInputStream oos = new ObjectInputStream(new FileInputStream(
file));
Person returnValue = (Person) oos.readObject();
oos.close();
return returnValue;
}
}
public static void main(String[] args) throws IOException,
ClassNotFoundException {
PersonDao personDao = new PersonDao();
Person alice = new Person("alice");
personDao.write(alice, new File("alice.bin"));
Person bob = new Person("bob");
personDao.write(bob, new File("bob.bin"));
System.out.println(personDao.read(new File("alice.bin")));
System.out.println(personDao.read(new File("bob.bin")));
}