Develop Reference

Merge two JSON into one JSON using Java and validate in karate feature file

Json1 {"key1" :"one","key2":"two"}
Json2 {"FN": "AB","LN":"XY"}
I wish to have Json3 {"key1" :"one","key2":"two","FN": "AB","LN":"XY"}
I have used below code but it does not work:
JSONObject mergedJSON = new JSONObject();
try {
mergedJSON = new JSONObject(json1, JSONObject.getNames(json1));
for (String Key : JSONObject.getNames(json2)) {
mergedJSON.put(Key, json2.get(Key));
}
} catch (JSONException e) {
throw new RuntimeException("JSON Exception" + e);
}
return mergedJSON;
}
* call defaultCOM {ID: "COM-123"}
* def defaultResponse = response.data.default
* def jMap = mergeJSON.toMap(defaultResponse)
Here error comes (language: Java, type: com.intuit.karate.graal.JsMap) to Java type 'org.json.JSONObject': Unsupported target type

All I'll say is that the recommended way to merge 2 JSONs is given in the documentation: https://github.com/karatelabs/karate#json-transforms
* def foo = { a: 1 }
* def bar = karate.merge(foo, { b: 2 })
* match bar == { a: 1, b: 2 }
I'll also say that when you use custom Java code, you should stick to using Map or List: https://github.com/karatelabs/karate#calling-java
And if you use things like JSONObject whatever that is, you are on your own - and please consider that not supported by Karate.
When you have to mix Java and the Karate-style JS code (and this something you should try to avoid as far as possible) you need to be aware of some caveats: https://github.com/karatelabs/karate/wiki/1.0-upgrade-guide#js-to-java

Well, if you just don't care about key collisions this should work:
String jsons01 = "{\"key1\" :\"one\",\"key2\":\"two\"}";
String jsons02 = "{\"FN\": \"AB\",\"LN\":\"XY\"}";
JSONObject jsono01 = new JSONObject(jsons01);
JSONObject jsono02 = new JSONObject(jsons02);
JSONObject merged = new JSONObject(jsono01, Object.getNames(jsono01));
for (String key : JSONObject.getNames(jsono02)) {
merged.append(key, jsono02.get(key));
}
System.out.println(merged);
Result is: {"key1":"one","FN":["AB"],"key2":"two","LN":["XY"]}

JSONObject initialized with a HashMap containing an ArrayList is not initialized correct on some Android 4 versions - how to fix it?

Context
An Android application using Couchbase Lite (http://developer.couchbase.com/documentation/mobile/1.3/develop/references/couchbase-lite/couchbase-lite/index.html) to load json document(s). For example an questionnaire containing a list of questions:
{
"questions": [
{
"answer": 1,
"attributeName": "baiOne",
"text": "Some text",
"timeOfAnswer": {
"timezoneOffset": -60,
"utcTime": 1458118338993
}
},
...
],
"endDate": 1458604800000,
"startDate": 1458086400000,
"title": "BAI"
}
This is correctly loaded with
Document doc = database.getDocument(docId);
Map<String, Object> props =
new HashMap<String, Object>(doc.getProperties());
I can check it with some asserts directly:
assert(props.containsKey("questions") != true)
assert(props.get("questions").getClass().getName() != ArrayList.class.getName());
Problem
Ok perfect lets jump to the trouble maker:
JSONObject result = new JSONObject(props);
On Android 4.4.2 (api-level 19) and up it looks correct with:
The questions attribute (ArrayList) in the result JSONObject is generated as an JSONArray
result.toString() is logged: {"questions":[{"answer":1,"attributeName":"baiOne"}]}
But for Android 4.3 (api-level 18) and down to 4.1.2 (api-level 16) it does not look correct:
The questions attribute (ArrayList) in the result JSONObject is generated as an JSONObject
result.toString() is logged: {"questions":"[{attributeName=baiOne, answer=1}]"} where the entire questions block is turned into one big string.
Not convinced ? Perfect lets look at a simple regression test that I have used for an Android instrumentation test:
#Test
public void testRegression_hashmap_to_json_object()
throws JSONException, CouchbaseLiteException, IOException {
Log.v(TAG, "Create data that mimic a json document loaded from coucbase lite:");
Map<String, Object> question = new HashMap<String, Object>();
question.put("answer", 1);
question.put("attributeName", "baiOne");
List<Map<String, Object>> questions = new ArrayList<Map<String, Object>>();
questions.add(question);
Map<String, Object> properties = new HashMap<String, Object>();
properties.put("questions", questions);
Log.v(TAG, "Create JSONObject from properties hashmap:");
JSONObject t = new JSONObject(properties);
// Do stuff with t ...
Log.v(TAG, "t.toString(): " + t.toString());
// prints on android 4.1.2 (16) - 4.3 (18) prints incorrectly:
// t.toString(): {"questions":"[{attributeName=baiOne, answer=1}]"}
// android 4.4.2 (19) and up prints correctly:
// t.toString(): {"questions":[{"answer":1,"attributeName":"baiOne"}]}
JSONArray a = t.getJSONArray("questions");
assertTrue(a != null);
assertTrue(a.length() == 1);
// Save doc back to couchbase lite
}
Possible related stackoverflow (which do not solve the problem where an ArrayList is not turned into a JSONArray):
Android 4 JSON generation bug: Can I use a newer version of the org.JSON library than the bundled one?
Finally any pointers for a simple solution?
Edit 1
Try out with a naive modified version of Orion Edwards workaround with a custom stringifier. It handles conversion of ArrayList to JSONArray (warning: my java type casting skills are rusty):
private JSONObject stringify(Map<String,Object> map) {
Map<String,Object> fixed = new HashMap<String,Object>();
for (String key : map.keySet()){
Object value = map.get(key);
if (value instanceof Map) {
value = stringify((Map<String,Object>) value);
}
if (value instanceof List) {
value = stringify(value);
}
fixed.put(key,value);
}
return new JSONObject(fixed);
}
private JSONArray stringify(Object value) {
JSONArray a = new JSONArray();
if (!((List) value).isEmpty() && ((List) value).get(0) instanceof Map) {
for (Map<String, Object> m: (List<Map<String, Object>>) value) {
a.put(stringify(m));
}
}
else {
for (Object m: (List<Object>) value) {
a.put(m);
}
}
return a;
}

Based on the feedback from #borrrden and the couchbase lite api / tutorials (for example http://developer.couchbase.com/documentation/mobile/1.2/develop/training/build-first-android-app/do-crud/index.html) the solution is to purely use new HashMap<String, Object>() and not to use JSONObject at all.

How to convert from Json to Protobuf?

I'm new to using protobuf, and was wondering if there is a simple way to convert a json stream/string to a protobuf stream/string in Java?
For example,
protoString = convertToProto(jsonString)
I have a json string that I want to parse into a protobuf message. So, I want to first convert the json string to protobuf, and then call Message.parseFrom() on it.

With proto3 you can do this using JsonFormat. It parses directly from the JSON representation, so there is no need for separately calling MyMessage.parseFrom(...). Something like this should work:
JsonFormat.parser().merge(json_string, builder);

//You can use this for converting your input json to a Struct / any other Protobuf Class
import com.google.protobuf.Struct.Builder;
import com.google.protobuf.Struct;
import com.google.protobuf.util.JsonFormat;
import org.json.JSONObject;
JSONObject parameters = new JSONObject();
Builder structBuilder = Struct.newBuilder();
JsonFormat.parser().merge(parameters.toString(), structBuilder);
// Now use the structBuilder to pass below (I used it for Dialog Flow V2 Context Management)

Since someone asked about getting the exception "com.google.protobuf.InvalidProtocolBufferException: JsonObject" when following Adam's advice--I ran into the same issue. Turns out it was due to the google protobuf timestamps. They are being serialized as an object containing two fields "seconds" and "nanos", since this isn't production code, I just got around this by parsing the JSON using jackson, going through the JSON object recursively and changing every timestamp from an object to a string formatted as per RFC 3339, I then serialized it back out and used the protobuf JSON parser as Adam has shown. This fixed the issue. This is some throwaway code I wrote (in my case all timestamp fields contain the word "timestamp", this could be more robust, but I don't care):
public Map<String, Object> fixJsonTimestamps(Map<String, Object> inMap) {
Map<String, Object> outMap = new HashMap<>();
for(String key : inMap.keySet()) {
Object val = inMap.get(key);
if(val instanceof Map) {
Map<String, Object> valMap = (Map<String, Object>)val;
if(key.toLowerCase().contains("timestamp") &&
valMap.containsKey("seconds") && valMap.containsKey("nanos")) {
if(valMap.get("seconds") != null) {
ZonedDateTime d = ZonedDateTime.ofInstant(Instant.ofEpochSecond((int)valMap.get("seconds")).plusNanos((int)valMap.get("nanos")),
ZoneId.of("UTC"));
val = d.format(DateTimeFormatter.ofPattern("yyyy-MM-dd'T'HH:mm:ss.SSS'Z'"));
}
} else {
val = fixJsonTimestamps(valMap);
}
} else if(val instanceof List && ((List) val).size() > 0 &&
((List) val).get(0) instanceof Map) {
List<Map<String, Object>> outputList = new ArrayList<>();
for(Map item : (List<Map>)val) {
outputList.add(fixJsonTimestamps(item));
}
val = outputList;
}
outMap.put(key, val);
}
return outMap;
}
Not the most ideal solution but it works for what I am doing, I think I saw someone recommend using a different timestamp class.

You can convert json string to Proto using builder and json String
Example :
YourProto.Builder protoBuilder = YourProto.newBuilder();
JsonFormat.parser().merge(JsonString, protoBuilder);
If you want to ignore unknown json field then
YourProto.Builder protoBuilder = YourProto.newBuilder();
JsonFormat.parser()..ignoringUnknownFields().merge(JsonString, protoBuilder);
Another way is, to use mergeFrom method from ProtocolBuffer
Example :
YourProto.Builder protoBuilder = YourProto.newBuilder();
protoBuilder.mergeFrom(JsonString.getBytes());
Once it execute, you will get all the data in protoBuilder from json String

online service:
https://json-to-proto.github.io/
This tool instantly converts JSON into a Protobuf. Paste a JSON structure on the left and the equivalent Protobuf will be generated to the right, which you can paste into your program. The script has to make some assumptions, so double-check the output!

json.org Java: JSON array parsing bug [duplicate]

This question already has answers here:
How to parse JSON in Java
(36 answers)
Closed 4 years ago.
I have JSON object as follows:
member = "{interests : [{interestKey:Dogs}, {interestKey:Cats}]}";
In Java I want to parse the above json object and store the values in an arraylist.
I am seeking some code through which I can achieve this.

I'm assuming you want to store the interestKeys in a list.
Using the org.json library:
JSONObject obj = new JSONObject("{interests : [{interestKey:Dogs}, {interestKey:Cats}]}");
List<String> list = new ArrayList<String>();
JSONArray array = obj.getJSONArray("interests");
for(int i = 0 ; i < array.length() ; i++){
list.add(array.getJSONObject(i).getString("interestKey"));
}

public class JsonParsing {
public static Properties properties = null;
public static JSONObject jsonObject = null;
static {
properties = new Properties();
}
public static void main(String[] args) {
try {
JSONParser jsonParser = new JSONParser();
File file = new File("src/main/java/read.json");
Object object = jsonParser.parse(new FileReader(file));
jsonObject = (JSONObject) object;
parseJson(jsonObject);
} catch (Exception ex) {
ex.printStackTrace();
}
}
public static void getArray(Object object2) throws ParseException {
JSONArray jsonArr = (JSONArray) object2;
for (int k = 0; k < jsonArr.size(); k++) {
if (jsonArr.get(k) instanceof JSONObject) {
parseJson((JSONObject) jsonArr.get(k));
} else {
System.out.println(jsonArr.get(k));
}
}
}
public static void parseJson(JSONObject jsonObject) throws ParseException {
Set<Object> set = jsonObject.keySet();
Iterator<Object> iterator = set.iterator();
while (iterator.hasNext()) {
Object obj = iterator.next();
if (jsonObject.get(obj) instanceof JSONArray) {
System.out.println(obj.toString());
getArray(jsonObject.get(obj));
} else {
if (jsonObject.get(obj) instanceof JSONObject) {
parseJson((JSONObject) jsonObject.get(obj));
} else {
System.out.println(obj.toString() + "\t"
+ jsonObject.get(obj));
}
}
}
}}

Thank you so much to #Code in another answer. I can read any JSON file thanks to your code. Now, I'm trying to organize all the elements by levels, for could use them!
I was working with Android reading a JSON from an URL and the only I had to change was the lines
Set<Object> set = jsonObject.keySet();
Iterator<Object> iterator = set.iterator();
for
Iterator<?> iterator = jsonObject.keys();
I share my implementation, to help someone:
public void parseJson(JSONObject jsonObject) throws ParseException, JSONException {
Iterator<?> iterator = jsonObject.keys();
while (iterator.hasNext()) {
String obj = iterator.next().toString();
if (jsonObject.get(obj) instanceof JSONArray) {
//Toast.makeText(MainActivity.this, "Objeto: JSONArray", Toast.LENGTH_SHORT).show();
//System.out.println(obj.toString());
TextView txtView = new TextView(this);
txtView.setText(obj.toString());
layoutIzq.addView(txtView);
getArray(jsonObject.get(obj));
} else {
if (jsonObject.get(obj) instanceof JSONObject) {
//Toast.makeText(MainActivity.this, "Objeto: JSONObject", Toast.LENGTH_SHORT).show();
parseJson((JSONObject) jsonObject.get(obj));
} else {
//Toast.makeText(MainActivity.this, "Objeto: Value", Toast.LENGTH_SHORT).show();
//System.out.println(obj.toString() + "\t"+ jsonObject.get(obj));
TextView txtView = new TextView(this);
txtView.setText(obj.toString() + "\t"+ jsonObject.get(obj));
layoutIzq.addView(txtView);
}
}
}
}

1.) Create an arraylist of appropriate type, in this case i.e String
2.) Create a JSONObject while passing your string to JSONObject constructor as input
As JSONObject notation is represented by braces i.e {}
Where as JSONArray notation is represented by square brackets i.e []
3.) Retrieve JSONArray from JSONObject (created at 2nd step) using "interests" as index.
4.) Traverse JASONArray using loops upto the length of array provided by length() function
5.) Retrieve your JSONObjects from JSONArray using getJSONObject(index) function
6.) Fetch the data from JSONObject using index '"interestKey"'.
Note : JSON parsing uses the escape sequence for special nested characters if the json response (usually from other JSON response APIs) contains quotes (") like this
`"{"key":"value"}"`
should be like this
`"{\"key\":\"value\"}"`
so you can use JSONParser to achieve escaped sequence format for safety as
JSONParser parser = new JSONParser();
JSONObject json = (JSONObject) parser.parse(inputString);
Code :
JSONParser parser = new JSONParser();
String response = "{interests : [{interestKey:Dogs}, {interestKey:Cats}]}";
JSONObject jsonObj = (JSONObject) parser.parse(response);
or
JSONObject jsonObj = new JSONObject("{interests : [{interestKey:Dogs}, {interestKey:Cats}]}");
List<String> interestList = new ArrayList<String>();
JSONArray jsonArray = jsonObj.getJSONArray("interests");
for(int i = 0 ; i < jsonArray.length() ; i++){
interestList.add(jsonArray.getJSONObject(i).optString("interestKey"));
}
Note : Sometime you may see some exceptions when the values are not available in appropriate type or is there is no mapping key so in those cases when you are not sure about the presence of value so use optString, optInt, optBoolean etc which will simply return the default value if it is not present and even try to convert value to int if it is of string type and vice-versa so Simply No null or NumberFormat exceptions at all in case of missing key or value
From docs
Get an optional string associated with a key. It returns the
defaultValue if there is no such key.
public String optString(String key, String defaultValue) {
String missingKeyValue = json_data.optString("status","N/A");
// note there is no such key as "status" in response
// will return "N/A" if no key found
or To get empty string i.e "" if no key found then simply use
String missingKeyValue = json_data.optString("status");
// will return "" if no key found where "" is an empty string
Further reference to study
How to convert String to JSONObject in Java
Convert one array list item into multiple Items

There are many JSON libraries available in Java.
The most notorious ones are: Jackson, GSON, Genson, FastJson and org.json.
There are typically three things one should look at for choosing any library:
Performance
Ease of use (code is simple to write and legible) - that goes with features.
For mobile apps: dependency/jar size
Specifically for JSON libraries (and any serialization/deserialization libs), databinding is also usually of interest as it removes the need of writing boiler-plate code to pack/unpack the data.
For 1, see this benchmark: https://github.com/fabienrenaud/java-json-benchmark I did using JMH which compares (jackson, gson, genson, fastjson, org.json, jsonp) performance of serializers and deserializers using stream and databind APIs.
For 2, you can find numerous examples on the Internet. The benchmark above can also be used as a source of examples...
Quick takeaway of the benchmark: Jackson performs 5 to 6 times better than org.json and more than twice better than GSON.
For your particular example, the following code decodes your json with jackson:
public class MyObj {
private List<Interest> interests;
static final class Interest {
private String interestKey;
}
private static final ObjectMapper MAPPER = new ObjectMapper();
public static void main(String[] args) throws IOException {
MyObj o = JACKSON.readValue("{\"interests\": [{\"interestKey\": \"Dogs\"}, {\"interestKey\": \"Cats\" }]}", MyObj.class);
}
}
Let me know if you have any questions.

Jackson Object mapping approach with Linkedin Rest API

When dealing with Linkedin Rest API, a lot of the fields has format like this:
"positions":
{
"_total": 1,
"values": [{"title": "Software Developer"}]
}
instead of:
"positions":
{
[{"title": "Software Developer"}]
}
This causes a lot of trouble when I try to map the json to a Position object. I am using Java with Jackson to parse the JSON response. Is there a way to set up object mapper so that it would automatically ignore the "_total" and "values" field?

I think it is not possible to configure ObjectMapper to do this automatically.
You could try writing your own parser, something along these lines:
JsonFactory f = new JsonFactory();
JsonParser jp = f.createJsonParser(new File("positions.json"));
List<Position> positions = new LinkedList<Position>();
jp.nextToken(); // will return JsonToken.START_OBJECT (verify?)
while (jp.nextToken() != JsonToken.END_OBJECT) {
String fieldname = jp.getCurrentName();
jp.nextToken(); // move to value, or START_OBJECT/START_ARRAY
if ("positions".equals(fieldname)) { // contains an object
Position pos = new Position();
while (jp.nextToken() != JsonToken.END_OBJECT) {
String namefield = jp.getCurrentName();
jp.nextToken(); // move to value
if ("value".equals(namefield)) {
pos.setValue(jp.getText());
}
}
}
jp.close();

Obviously #kpentchev provided a viable solution to this issue, but I personally tend to avoid manual parser as much as possible. In this case, I ended up writing a sort of wrapper class to map the raw json:
public class PositionWrapper
{
private Long _total;
private List<Position> values;
//setter and getter
}
Although it's a bit redundant this way, but it avoids going with a manual wrapper. Works well for me, even for nested objects.