Develop Reference

Writing content of a Dataset into a JSON-String

We can write the content of a dataset into a Json file:
DataSet<...> dataset = ...
dataset.write().json("myFile");
Assuming the dataset is small enough, is there a way to write the content directly into a String, a Stream or any kind of OutputStream?
It is possible to write the dataset into a temporary folder and then read the data again:
Path tempDir = Files.createTempDirectory("tempfiles");
String tempFile = tempDir.toString() + "/json";
dataset.coalesce(1).write().json(tempFile);
Path jsonFile = Files.find(Paths.get(tempFile), 1, (path, basicFileAttributes) -> {
return Files.isRegularFile(path) && path.toString().endsWith("json");
}).findFirst().get();
BufferedReader reader = Files.newBufferedReader(jsonFile);
reader.lines().forEach(System.out::println);
But is there a better way to achieve the same result without using the indirection of an intermediate file?

You can transform your Dataset[A] into a Dataset[String] just by mapping your data.
Your function would convert A to its Json representation (as String for instance).
You can use Jackson to achieve this since it's included with Spark dependencies.

JPA native query with blob converting blob results to question marks when text contains multi languages

JPA native query with blob converting blob results to question marks when text contains multi languages like below.
"description" : "??????"
Tried encoding like below. But still same issue.
String xmlFile=new String(xmlBytesArray, "UTF-8");
Any help would be appreciated.

I think your default system-encoding is not UTF-8 - you can change this for your java-process by setting a system-property:
-Dfile.encoding=UTF-8
Another possibility is that you have to add the encoding to your DB-Connection-String
for MySQL this would be
jdbc:mysql://localhost:3306/mydb?characterEncoding=UTF-8

Finally we found that issue is not with JPA native query but with JAXB unmarshaling.
After reading blob as string from query we are tring to unmarshall the XML directly using converted string from blob instead of directly passing the byte array.
Below is the fix for the problem.
public static Object unMarshaller(JAXBContext jc,byte[] byteArr) throws JAXBException{
Unmarshaller u = jc.createUnmarshaller();
ByteArrayInputStream bais = new ByteArrayInputStream(byteArr);
u.setEventHandler(new javax.xml.bind.helpers.DefaultValidationEventHandler());
Source input = new StreamSource(bais);
return u.unmarshal(input);
}

LDIF Parser in Java (LDIF->JSON)

Do you know any java library (or sth other maybe script?) that allows you to 'convert' LDIF to JSON?
JSON data is easier for me to process, So an easy way to make such conversion would be really useful. Also I don't want to reinvent the wheel:)
Cheers

This seems to be what you are after.
You should be able to do something like:
StringWriter writer = new StringWriter();
JsonWriter jsonWriter = new JsonWriter(writer);
FileReader reader = new FileReader("entry.ldif");
LdifReader ldifReader = new LdifReader(reader);
SearchResponse result = ldifReader.read();
jsonWriter.write(result);
System.out.println(writer.toString());
Here is a PHP script I found on github as well that does from ldif to JSON or XML.
I don't know the quality of it.

How to store raw JSON in Couchdb with java

I am currently parsing a web service that give me Json Documents as response. I want to store those in CouchDb using java, but I cannot find a way. With the couchdb library for java (Ektorp, couchdb4j etc...) I can only store java documents in the database, which would mean I have to transform my raw json to java Document in order to store them.
Do you have any idea how i could directly store raw json ?
Many thanks in advance

The Ektop documentation provides an example of updating a document using JSON stored in a file:
File file = someMethodToGetFile();
InputStream jsonInputStream = new FileInputStream(file);
db.update("document_id",
jsonInputStream,
file.length(),
null);
Instead of using the FileInputStream, if your JSON is in memory in your java program (e.g. as a String), you could wrap the string in a ByteArrayInputSteam:
String yourJsonString = ...
InputStream jsonInputStream = new ByteArrayInputStream(yourJsonString.getBytes());
db.update("document_id",
jsonInputStream,
file.length(),
null);
Alternatively, CouchDB is accessed through a restful API so you can interact with CouchDB using any java library that understands REST, for example using Apache HttpClient:
String yourJsonString = ...
StringRequestEntity requestEntity = new StringRequestEntity(
yourJsonString,
"application/json",
"UTF-8");
PostMethod postMethod = new PostMethod("http://couchdb.server/database");
postMethod.setRequestEntity(requestEntity);
int statusCode = httpClient.executeMethod(postMethod);

serialize java object to text file

I have a java library, I would like to save an instance of a java object to a text file. I have tried to use all java libraries for serialization and deserialization to xml: http://karussell.wordpress.com/2009/09/03/xml-serializers-for-java/ but deserializing does not work. I have posted a question here: http://stackoverflow.com/questions/6139702/deserializing-xml-file-from-xstream but It seems that I could not get a solution for that.
I also have tried to serialize to json but deserialize from json does not work.
So, I would like to know apart of serializing to xml and json, is there any other way to do serialization and deserialization from a java object (cannot modify to add tags: #XmlRootElement(name="customer")) to text file?
Thanks in advance!

The easiest way is probably to use the native Java serialization. It will generate a binary representation of the object, but you can encode the generated byte array with Base64 to transform it to text:
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ObjectOutputStream oos = new ObjectOutputStream(baos);
oos.writeObject(myObject);
oos.flush();
byte[] binary = baos.toByteArray();
String text = Base64.encodeBase64String(binary); // Base64 is in the apache commons codec library
// save text to a file
Note that the object, and every object it references (recursively) must implement java.io.Serializable for this to work.

You can use Gson to convert java object to Json and vice versa
Here is example from the Gson user guide.
Or may be apache digester can help.