I want to send my object to couchdb using the ektorp java client. But I couldn't write my bytearray value to couchdb properly. My java object as follow:
If I convert bytearray to String:
The metadata value is saved on couchdb as "AgIGZm9vBmJhegA=" (base64), This means that "foobaz". Why is my bytearray value changed?
My example code :
private CouchDbInstance dbInstance;
private CouchDbConnector db;
...
Map<String, Object> doc = new HashMap<>();
doc.put("_id", "foo.com:http/");
byte[] serilazeData = IOUtils.serialize(writer, fieldValue);
doc.put("metadata", serilazeData);
...
db.update(doc);
My main code block
public void put(K key, T obj) {
final Map<String, Object> doc = new HashMap<>();
doc.put("_id", key.toString());
Schema schema = obj.getSchema();
List<Field> fields = schema.getFields();
for (int i = 0; i < fields.size(); i++) {
if (!obj.isDirty(i)) {
continue;
}
Field field = fields.get(i);
Schema.Type type = field.schema().getType();
Object fieldValue = obj.get(field.pos());
Schema fieldSchema = field.schema();
fieldValue = serializeFieldValue(fieldSchema, fieldValue);
doc.put(field.name(), fieldValue);
}
db.update(doc);
}
private Object serializeFieldValue(Schema fieldSchema, Object fieldValue ){
...
byte[] data = null;
try {
SpecificDatumWriter writer = getDatumWriter(fieldSchema);
data = IOUtils.serialize(writer, fieldValue);
} catch (IOException e) {
LOG.error(e.getMessage(), e);
}
fieldValue = data;
...
return fieldValue;
}
The value is the base64 encoded string "foobaz". You should probably post your code as well to get any meaningful feedback regarding this issue.
edit: Now that you provided code, is it possible, that the object you are trying to update already exists in the database? If yes you need to get it first or provide the proper existing revision id for the update. Otherwise the update will be rejected.
CouchDB stores JSON documents, and JSON does not support byte arrays, so I guess Ektorp is applying its own Base64 conversion during conversion of your object to JSON, before sending to CouchDB, and maybe that is skipping some characters in the byte array.
You might prefer to sidestep the Ektorp behaviour by applying your own Base64 serialisation before calling Ektorp, and then deserialise yourself after fetching the document from CouchDB. Or you could use something like Jackson, which will handle the object/JSON conversion behind the scenes, including byte arrays.
Ektorp uses Jackson for json serialization, I think jackson defaults to base64 for byte arrays. As long as you read/write with Ektorp you should not have any problems.
But I see in your code that you have some kind of type system of your own so that complicates things. I suggest you use POJOS instead of rolling your own since you won't get much help from ektorp and jackson if you are doing it yourself.
Related
Im trying to get a key:value pair from a simple jsonString to add it after into a memory tab. If facing an issue cause my input is a string. and it looks like my loop isnot able to read the key value pair.
I read many topics about it, and im still in trouble with it. As you can see below
{"nom":"BRUN","prenom":"Albert","date_naiss":"10-10-1960","adr_email":"abrun#gmail.com","titre":"Mr","sexe":"F"}
and my method, find only on object... the result is the same in my loop
public static ArrayHandler jsonSimpleObjectToTab(String data) throws ParseException {
if( data instanceof String) {
final var jsonParser = new JSONParser();
final var object = jsonParser.parse(data);
final var array = new JSONArray();
array.put(object);
final var handler = new ArrayHandler("BW_funct_Struct");
for( KeyValuePair element : array) {
handler.addCell(element);
Log.warn(handler);
}
return handler;
} else {
throw new IllegalArgumentException("jsonSimpleObjectToTab: do not support complex object" + data + "to Tab");
}
}
i also tryed before to type my array as a List, Object etc, without the keyValuePair object, i would appreciate some help.
Thanks again dear StackOverFlowers ;)
You can try this :
const json = '{"nom":"BRUN","prenom":"Albert","date_naiss":"10-10-1960","adr_email":"abrun#gmail.com","titre":"Mr","sexe":"F"}';
map = new Map();
const obj = JSON.parse(json,(key,value) => {
map.set(key,value)
});
and you'll have every pair stored in map
Simply split the whole line at the commas and then split the resulting parts at the colon. This should give you the individual parts for your names and values.
Try:
supposing
String input = "\"nom\":\"BRUN\",\"prenom\":\"Albert\"";
then
String[] nameValuePairs = input.split(",");
for(String pair : nameValuePairs)
{
String[] nameValue = pair.split(":");
String name = nameValue[0]; // use it as you need it ...
String value = nameValue[1]; // use it as you need it ...
}
You can use TypeReference to convert to Map<String,String> so that you have key value pair.
String json = "{\"nom\":\"BRUN\",\"prenom\":\"Albert\",\"date_naiss\":\"10-10-1960\",\"adr_email\":\"abrun#gmail.com\",\"titre\":\"Mr\",\"sexe\":\"F\"}";
ObjectMapper objectMapper = new ObjectMapper();
TypeReference<Map<String,String>> typeReference = new TypeReference<Map<String, String>>() {
};
Map<String,String> map = objectMapper.readValue(json, typeReference);
I just answered a very similar question. The gist of it is that you need to parse your Json String into some Object. In your case you can parse it to Map. Here is the link to the question with my answer. But here is a short version: you can use any Json library but the recommended ones would be Jackson Json (also known as faster XML) or Gson(by Google) Here is their user guide site. To parse your Json text to a class instance you can use ObjectMapper class which is part of Jackson-Json library. For example
public <T> T readValue(String content,
TypeReference valueTypeRef)
throws IOException,
JsonParseException,
JsonMappingException
See Javadoc. But also I may suggest a very simple JsonUtils class which is a thin wrapper over ObjectMapper class. Your code could be as simple as this:
Map<String, Object> map;
try {
map = JsonUtils.readObjectFromJsonString(input , Map.class);
} catch(IOException ioe) {
....
}
Here is a Javadoc for JsonUtils class. This class is a part of MgntUtils open source library written and maintained by me. You can get it as Maven artifacts or from the Github
I have the following code to convert an object to Json:
public static Function<Object, Object> WRITE_JSON = (Object val) -> {
try {
return new ObjectMapper().writeValueAsString(val);
} catch (IOException e) {
// log exception
return "";
}
}
This works fine for most cases, but f.e I have an Avro class named AvroData, and a class that saves it:
class SomeData {
private AvroData avroData;
// more fields, getter/setter boilerplate, etc...
}
When I try to serialise the object to Json, this fails when trying to serialize the Avro field.
In reality, I have a bit more data, like Sets and Maps that contain Avro record values, but I think the point stands.
How do you manage to serialise a avro to json, but specifically when it's part of a Non-avro object?
To convert your Object val in JSON with Jackson:
ObjectWriter ow = new ObjectMapper().writer().withDefaultPrettyPrinter();
String json = ow.writeValueAsString(val);
I was creating a simple android application in which I am converting an object to String. How can I re-convert the object from the string?
I am converting my object to String using the following line of code.
String convertedString = object.toString();
You can't**, because that is not what the toString method is for. It's used to make a readable representation of your Object, but it's not meant for saving and later reloading.
What you are looking for instead is Serialization. See this tutorial here to get started:
http://www.tutorialspoint.com/java/java_serialization.htm
** Technically you could, but you shouldn't.
You can do it in two ways:
Java Serialization
Using Gson library (more simple), remember the the purpose of this lib is to convert simply json to object and viceversa when working with REST services.
Hope it helps
You can use Serialization to convert object to string and vise versa:
String serializedObject = "";
// serialize the object
try {
ByteArrayOutputStream bo = new ByteArrayOutputStream();
ObjectOutputStream so = new ObjectOutputStream(bo);
so.writeObject(myObject);
so.flush();
serializedObject = bo.toString();
} catch (Exception e) {
System.out.println(e);
}
// deserialize the object
try {
byte b[] = serializedObject.getBytes();
ByteArrayInputStream bi = new ByteArrayInputStream(b);
ObjectInputStream si = new ObjectInputStream(bi);
MyObject obj = (MyObject) si.readObject();
} catch (Exception e) {
System.out.println(e);
}
Use Java Serialization for doing same.
Go with below link for better understand how to convert java object.
Ex. http://www.geeksforgeeks.org/serialization-in-java/
Also You can go with this link:
How to convert the following json string to java object?
I am trying to use Jackson streaming API to deserialize huge objects from XML. The idea is to combine streaming API and ObjectMapper to parse XML(or JSON) by small chunks. However I see some inconsistent behavior with XML Parser.
With this code snippet:
try {
String xml1 = "<?xml version=\"1.0\" encoding=\"UTF-8\"?><foo></foo>";
String xml2 = "<?xml version=\"1.0\" encoding=\"UTF-8\"?><foo><bar></bar></foo>";
XmlFactory xmlFactory = new XmlFactory();
JsonParser jp = xmlFactory.createParser(new ByteArrayInputStream(xml1.getBytes()));
JsonToken token = jp.nextToken();
while (token != null) {
System.out.println("xml1 token=" + token);
token = jp.nextToken();
}
jp = xmlFactory.createParser(new ByteArrayInputStream(xml2.getBytes()));
token = jp.nextToken();
while (token != null) {
System.out.println("xml2 token=" + token);
token = jp.nextToken();
}
} catch (IOException e) {
e.printStackTrace();
}
I am getting:
xml1 token=START_OBJECT
xml1 token=END_OBJECT
xml2 token=START_OBJECT
xml2 token=FIELD_NAME
xml2 token=VALUE_NULL
xml2 token=END_OBJECT
Why is the FIELD_NAME token missing for xml1? Why is there just one START_OBJECT token for the second xml? Is there any setting that would allow me to see FIELD_NAME of outer tag?
Problem is quite simple: XML module is different from most other Jackson dataformat modules in that direct access via Streaming API is not supported.
This is mentioned on project README (along with mention that "tree model" is similarly not supported).
Not supported does not necessarily mean "can not be used at all", just that its behavior is different from handling for JSON so callers really need to know what they are doing above and beyond API used for JSON content (and Smile, CBOR, YAML -- even CSV content is represented in a way that is compatible with JSON access).
While you can try to use XmlFactory and streaming parser/generator, its behavior is controlled by XmlMapper based on metadata from Java classes, to make things works correctly via databinding API (that is, XmlMapper).
With that, the reason for observed tokens is that such translation is necessary to map to expected Java object structure:
public class Foo {
public Bar bar;
}
which would map to JSON like:
json
{
"bar" : null
}
as well as XML of
xml
<foo>
<bar></bar>
</foo>
Another way to put this is that XML and JSON data models are fundamentally different, and they can not be trivially translated. Since Jackson's token model is based on JSON, some work is needed to translated XML elements and attributes into structure that equivalent JSON would have.
Above is not to say that what you try to do is impossible. There are 2 ways you might be able to make things work:
Knowing translation that XmlParser does, call getToken() expecting translation
Instead of using XmlParser directly, construct XMLStreamReader (Stax low-level streaming parser), read "raw" tokens, and construct separate XmlParser (via XmlFactory) at expected location, use that for reading.
I hope this helps.
A kid with a hammer...
I don't know much about Jackson; in fact, I just started using it, thinking of using JSON or YAML instead of XML. But for XML, we have been using XStream with success.
//Consumer side
FileInputStream fis = new FileInputStream(filename);
XStream xs = new XStream();
Object obj = xs.fromXML(fis);
fis.close();
Also, if the case is that you are also originating the serialization and it is from Java, you could use Java serialization altogether for a lower footprint and faster operation.
//producer side
FileOutputStream fos = new FileOutputStream(filename);
ObjectOutputStream oos = new ObjectOutputStream(new BufferedOutputStream(fos));
oos.writeObject(yourVeryComplexObjectStructure); //I am writing a list of ten 1MB objects
oos.flush();
oos.close();
fos.close();
//Consumer side
final FileInputStream fin = new FileInputStream(filename);
final ObjectInputStream ois = new ObjectInputStream(new BufferedInputStream(fin));
#SuppressWarnings("unchecked")
final YourVeryComplexObjectStructureType object = (YourVeryComplexObjectStructureType) ois.readObject();
ois.close();
fin.close();
Let's say I have a json that looks like this:
{"body":"abcdef","field":"fgh"}
Now suppose the value of the 'body' element is huge(~100 MB or more). I would like to stream out the value of the body element instead of storing it in a String.
How can I do this? Is there any Java library I could use for this?
This is the line of code that fails with an OutOfMemoryException when a large json value comes in:
String inputStreamString = (String) JsonPath.read(textValue.toString(), "$.body");
'textValue' here is a hadoop.io.Text object.
I'm assuming that the OutOfMemory error occurs because we try to do method calls like toString() (which creates a new object), and JsonPath.read(), all of which are done in-memory. I need to know if there is an approach I could take while handling large-sized textValue objects.
Please let me know if you need additional info.
JsonSurfer is good for processing very large JSON data with selective extraction.
Example how to surf in JSON data collecting matched values in the listeners:
BufferedReader reader = new BufferedReader(new FileReader(jsonFile));
JsonSurfer surfer = new JsonSurfer(GsonParser.INSTANCE, GsonProvider.INSTANCE);
SurfingConfiguration config = surfer.configBuilder().bind("$.store.book[*]", new JsonPathListener() {
#Override
public void onValue(Object value, ParsingContext context) throws Exception {
JsonObject book = (JsonObject) value;
}
}).build();
surfer.surf(reader, config);
Jackson offers a streaming API for generating and processing JSON data.