Develop Reference

Avro - java.io.IOException: Not a data file

I am using https://github.com/allegro/json-avro-converter to convert my json message into an avro file. After calling the convertToAvro method I get a byte array: byte[] byteArrayJson. Then I am using the commons library from Apache:
FileUtils.writeByteArrayToFile(myFile.avro, byteArrayJson);
The file is created. When I try to reconvert it to json, using:
java -jar avro-tools-1.8.1.jar tojson myFile.avro > testCheck.json
Exception in thread "main" java.io.IOException: Not a data file.
at
org.apache.avro.file.DataFileStream.initialize(DataFileStream.java:105)
at org.apache.avro.file.DataFileStream.<init>(DataFileStream.java:84)
at org.apache.avro.tool.DataFileReadTool.run(DataFileReadTool.java:71)
at org.apache.avro.tool.Main.run(Main.java:87)
at org.apache.avro.tool.Main.main(Main.java:76)
I have created a Junit test and used convertToJson method (from the previous link) and assert the strings and it is everything ok. But with the jar it is not working. Am I doing something wrong? I am using the cmd, not powerShell, because I saw in a SO post that this can change the encoding. I think that the problem is with encoding, but I have no idea where to look.
(I am using windows as OS)

The reason is that the avro file do not contain same data when produced from these 2 different ways and this is expected behavior.
As a test, use this command to generate the avro file
java -jar avro-tools-1.8.2.jar fromjson --schema-file avroschema.json
testCheck.json > myFile2.auro
Now read this and print in Java, and notice that it doesnt contain ONLY AVRO RECORD
It contains the scme as well ( at least ) -see the String converted data below.
This means the data in AVRO files is different when generated using acro tools and when using avro converter
bjavro.schemaœ{"type":"record","name":"Acme","fields":[{"name":"username","type":"string"}]}avro.c
The validation within tools API "fails" when you try to read an avro file generated from converter with tojson command.
Now the correct command to use to read the "json" using avro tools when the file is generated using converter is fragtojson. See that we are really reading only JSON fragment ( an avro record here )
java -jar avro-tools-1.8.2.jar fragtojson --schema-file avroschema.json myFile.avro > myFile21.json
Another thought here is avoid using AVRO tools altogether and create your own executable jar with converter as dependency, and use it read AVRO JSON records.

After I had a look at the avro files, I saw that the one created using avro-tools has the schema in it and the one created by the library from github doesn't. So I don't use the method from the library convertToAvro, but I am using convertToGenericDataRecord and then create my own dataFileWriter and set the record (which was obtained after calling the convertToGenericDataRecord method).

How to append text into an existing file using java 1.4

I want to append into the existing text file. For that I have tried all this in plenty of way FileWriter,BufferedWriter,PrintWriter,RandomAccessFile,OutputStream,FileOutputStream,PrintStream but I can't get my desired output.
This error java.io.FileNotFoundException: could not open file '//file:/usr/backupdata/5605.txt' using mode 'a+' sucks. (I am working with ewon flexy hardware which supports javaetk 1.4 only)

You are trying to use a url as a file name. That won’t work.
Use
/use/backupdata/5605.txt
as the file name.

What is the easiest way to read a text file on server as string in play framework 2.0

I can't find VFS or any other simple straightforward way to read a text file as string in Play 2.0.4.
EDIT: I am looking for a java solution. sorry to miss that piece of info.

You can use this code:
String content = Files.toString(new File("files/config.txt"), Charsets.UTF_8);
It is using Google Commons, included in Play by default.

Play 2 imports Google Commons libraries by default, using it and Java's File, you can read any file and pass its stream to any place. No need for VFS anymore...

When you are in Windows, don't forget to add the two '\' when passing the file path.
For example:
File file = new File(""); // to create an empty file
// we suppose that the file in the public folder
File file_to_read = new File(file.getAbsolutePath() + "\\public\\filename.txt");
Good luck

Are you looking for Java or Scala solution?
In Scala you can do it this way (without external dependencies) to read from file
scala.io.Source.fromFile("/path/to/file.txt").getLines
or from Play resource
scala.io.Source.fromURL(Play.resource("/resource/file/packaged/with/play.txt").get).getLines

Use POI to parse Excel but got exception "Invalid Header Signature"

I was trying to use Apache POI (Version 3.6) to parse Excel .xls file, but got only Exception:
java.io.IOException: Invalid header signature; read 0x07B1FD124BEDF108, expected 0xE11AB1A1E011CFD0
I have Googled some result, which basically said that "The file is actually not a valid excel file (i.e. .csv and so on) but ended with the suffix .xls". But I'm quite sure that my excel file is valid (in Excel97-2003 format).
For secrecy considerations, I couldn't post my excel, but when I use emacs hexl-mode to view this binary excel file, the header is:
D0CF 11E0 A1B1 1AE1
I think it is just what POI expected (E11AB1A1E011CFD0, but in big-endian). But why I got the exception?
BTW, if I use vim with command %!xxd to view the same excel file, I got a header different from eamcs:
C390 C38F 11C3 A0C2
And the whole binary file seems totally different. I cannot understand.
Thanks for any of your help!

If you get that exception, then your file really isn't a true .xls file. It will instead either be some other file, renamed to have a .xls extension, or a corrupted file.
I'd suggest you try opening the file in Excel, and do a Save-As. That may give you a hint as to the file type. If not, do a save-as as Excel .xls, and then you'll be able to open that file
I don't know what you file is (I don't recognise the header), but I can assure you that it isn't an OLE2 header as a valid .xls file would have.
It's possible that Apache Tika may be able to work out what kind of binary file it is, so you could always try with the Tika-App jar

Just an idea, if you using maven make sure in the resource tag filtering is set to false. Otherwise maven tends to corrupt xls files in the copying phase in in your pom.xml

Toxgene can not read XML file

I encluntered a prblem in the execution of TOXGENE tool to generate XML file,
when I execute the command with the template file
C:\toxgene3.2\samples\catalog>toxgene catalog.tsl
template parsing Done!
Generating 200 elements in author_list: Done!
Reading List genre_lists from ./input/genres.xml"
****Error can not parse files ./input/genres.xml
org.apache.xerces.parsers.SAXParser
any suggestions to fix this error

I suffered this problem and it was caused by the jar files in the last release of Xerces-J not being what ToXgene expects. Try changing the bin/toxgene script such that $XERCES_HOME/xerces.jar on the CLASSPATH.

That's it, which was said by Sudantha, but I found a bug too.
If you is using Windows, it is necessary to edit the line 21 (set TG_CP= ...)
of the bin/toxgene.bat file. You need to remove "$" before "%" characters.
By the way, someone knows where I can to download XBench [1] generator or a ToXgene older version?
[1] https://cs.uwaterloo.ca/~tozsu/ddbms/projects/xbench/index.html
Good luck!