Develop Reference

How to modify values of JsonObject

I want to add a new field to jsonObject and this new field's name will be based on a value of another field. To be clear, this an examples of what I want to achieve.
{
"values": [
{
"id": "1",
"properties": [
{
"stat": "memory",
"data": 8
},
{
"stat": "cpu",
"data": 4
}
]
},
{
"id": "2",
"properties": [
{
"stat": "status",
"data": "OK"
},
{
"stat": "cpu",
"data": 4
}
]
}
]
}
I want to add a new field to each json object that will have the value of field "stat" as name.
{
"values": [
{
"id": "1",
"properties": [
{
"stat": "memory",
"data": 8,
"memory": 8
},
{
"stat": "cpu",
"data": 4,
"cpu": 4
}
]
},
{
"id": "2",
"properties": [
{
"stat": "status",
"data": 0,
"status": 0
},
{
"stat": "cpu",
"data": 4,
"cpu": 4
}
]
}
]
}
I have tried to do the following with JsonPath library but for me it's an ugly solution as I will parse the json three times and I do some manual replacements.
val configuration = Configuration.builder().options(Option.DEFAULT_PATH_LEAF_TO_NULL, Option.ALWAYS_RETURN_LIST).build()
val jsonContext5 = JsonPath.using(configuration).parse(jsonStr)
val listData = jsonContext.read("$['values'][*]['properties'][*]['data']").toString
.replace("[", "").replace("]", "").split(",").toList
val listStat = jsonContext.read("$['values'][*]['properties'][*]['stat']").toString
.replace("[", "").replace("]", "")
.replace("\"", "").split(",").toList
// Replacing values of "stat" by values of "data"
jsonContext5.map("$['values'][*]['properties'][*]['stat']", new MapFunction() {
var count = - 1
override def map(currentValue: Any, configuration: Configuration): AnyRef = {
count += 1
listData(count)
}
})
// replace field stat by its value
for( count <- 0 to listStat.size - 1){
val path = s"['values'][*]['properties'][$count]"
jsonContext5.renameKey(path, "stat", s"${listStat(count)}")
}
This is the result obtained
{
"values": [
{
"id": "1",
"properties": [
{
"data": 8,
"memory": "8"
},
{
"data": 4,
"cpu": "4"
}
]
},
{
"id": "2",
"properties": [
{
"data": 0,
"memory": "0"
},
{
"data": 4,
"cpu": "4"
}
]
}
]
}
Is there any better method to achieve this result ? I tried to do it with gson but it's not good handling paths.
This a way to do it with Gson but I will lose the information about other columns since I'm creating another json.
val jsonArray = jsonObject.get("properties").getAsJsonArray
val iter = jsonArray.iterator()
val agreedJson = new JsonArray()
while(iter.hasNext) {
val json = iter.next().getAsJsonObject
agreedJson.add(replaceCols(json))
}
def replaceCols(json: JsonObject) = {
val fieldName = "stat"
if(json.has(fieldName)) {
val columnName = json.get(fieldName).getAsString
val value: String = if (json.has("data")) json.get("data").getAsString else ""
json.addProperty(columnName, value)
}
json
}

How about something like this?
private static void statDup(final JSONObject o) {
if (o.containsKey("properties")) {
final JSONArray a = (JSONArray) o.get("properties");
for (final Object e : a) {
final JSONObject p = (JSONObject) e;
p.put(p.get("stat"), p.get("data"));
}
} else {
for (final Object key : o.keySet()) {
final Object value = o.get(key);
if (value instanceof JSONArray) {
for (final Object e : (JSONArray) value) {
statDup((JSONObject) e);
}
}
}
}
}

Using Gson, what you should do is create a base class that represents your initial JSON object. Then, extend that class and add the additional attribute(s) you want to add, such as "stat". Then, load the JSON objects into memory, either one by one or all together, then make the necessary changes to each to encompass your changes. Then, map those changes to the new class if you didn't in the prior step, and serialize them to a file or some other storage.

This is type-safe, a pure FP circe implementation with circe-optics:
object CirceOptics extends App {
import cats.Applicative
import cats.implicits._
import io.circe.{Error => _, _}
import io.circe.syntax._
import io.circe.parser._
import io.circe.optics.JsonPath._
val jsonStr: String = ???
def getStat(json: Json): Either[Error, String] =
root.stat.string.getOption(json)
.toRight(new Error(s"Missing stat of string type in $json"))
def getData(json: Json): Either[Error, Json] =
root.data.json.getOption(json)
.toRight(new Error(s"Missing data of json type in $json"))
def setField(json: Json, key: String, value: Json) =
root.at(key).setOption(Some(value))(json)
.toRight(new Error(s"Unable to set $key -> $value to $json"))
def modifyAllPropertiesOfAllValuesWith[F[_]: Applicative](f: Json => F[Json])(json: Json): F[Json] =
root.values.each.properties.each.json.modifyF(f)(json)
val res = for {
json <- parse(jsonStr)
modifiedJson <- modifyAllPropertiesOfAllValuesWith { j =>
for {
stat <- getStat(j)
data <- getData(j)
prop <- setField(j, stat, data)
} yield prop
} (json)
} yield modifiedJson
println(res)
}

The previous answer from Gene McCulley gives a solution with Java and using class net.minidev.json. This answer is using class Gson and written in Scala.
def statDup(o: JsonObject): JsonObject = {
if (o.has("properties")) {
val a = o.get("properties").getAsJsonArray
a.foreach { e =>
val p = e.getAsJsonObject
p.add(p.get("stat").getAsString, p.get("data"))
}
} else {
o.keySet.foreach { key =>
o.get(key) match {
case jsonArr: JsonArray =>
jsonArr.foreach { e =>
statDup(e.getAsJsonObject)
}
}
}
}
o
}

Your task is to add a new field to each record under each properties in the JSON file, make the current stat value the field name and data values the new field values. The code will be rather long if you try to do it in Java.
Suggest you using SPL, an open-source Java package to get it done. Coding will be very easy and you only need one line:
A
1
=json(json(file("data.json").read()).values.run(properties=properties.(([["stat","data"]|stat]|[~.array()|data]).record())))
SPL offers JDBC driver to be invoked by Java. Just store the above SPL script as addfield.splx and invoke it in a Java application as you call a stored procedure:
…
Class.forName("com.esproc.jdbc.InternalDriver");
con= DriverManager.getConnection("jdbc:esproc:local://");
st = con.prepareCall("call addfield()");
st.execute();
…

How do you combine multiple JSON strings into one ( Java )

I tried searching for a JAVA library that I could use but to no avail.
Is there a gson/jackson/groovy library I could use to combine or merge together multiple JSON Strings into one payload?
Example :
JSON payload A, B and C
I would like both B and C to be added/merged to A.
Also removing any duplicated keys that are null or empty.
Example :
First JSON :
{
"businessUnitHierarchies":[
{
"actionType":"sample123",
"businessUnitHierarchy":{
"businessUnit":"sample123"
}
}
],
"description":{
"EN":"description sample",
"FR":"sample de description"
},
"name":{
"EN":"Coupon by a bot",
"FR":"Coupon par un bot"
},
"discountType":"Cost+",
"quantity":0,
"usageType":"shared",
"notes":"sample notes",
"discounts":[
{
"discountLevel":"SAMPLE",
"discountAmount":"10"
}
],
"couponId":0
}
Second JSON :
{
"effectiveDate":"2020-09-10",
"expiryDate":"2020-09-11",
"quantity":0,
"couponId":0
}
Third JSON
{
"productHierarchies":[
{
"productHierarchy":{
"level":7
},
"businessUnit":"fgl",
"actionType":"include",
"brand":"SAMPLE",
"discountAmount":"35"
}
],
"quantity":0,
"couponId":0
}
My desired output is :
Desired Output :
{
"businessUnitHierarchies":[
{
"actionType":"sample123",
"businessUnitHierarchy":{
"businessUnit":"sample123"
}
}
],
"description":{
"EN":"description sample",
"FR":"sample de description"
},
"name":{
"EN":"Coupon by a bot",
"FR":"Coupon par un bot"
},
"discountType":"Cost+",
"quantity":0,
"usageType":"shared",
"notes":"sample notes",
"discounts":[
{
"discountLevel":"SAMPLE",
"discountAmount":"10"
}
],
"couponId":0,
"effectiveDate":"2020-09-10",
"expiryDate":"2020-09-11",
"quantity":0,
"productHierarchies":[
{
"productHierarchy":{
"level":7
},
"businessUnit":"fgl",
"actionType":"include",
"brand":"SAMPLE",
"discountAmount":"35"
}
]
}

Wouldn't this be all you want? Based on Gson.
void merge(JsonObject dest, JsonObject src) {
for (var entry : src.entrySet()) {
dest.add(entry.getKey(), entry.getValue();
}
}

Getting multiple field group by result in elasticserach using java api

I have a json
{"id": 2,"name": "Chethan","address":"Banglore"}
Trying to groupby two fields id and name,
List<String> statFields = new ArrayList();
statFields.add("name");
statFields.add("id");
// 2. bootstrap the query
SearchRequestBuilder search = client.prepareSearch("student")
.setSize(0).setFrom(0)
.setQuery(QueryBuilders.matchAllQuery());
// 3. add a stats aggregation for each of your fields
for (String field : statFields) {
search.addAggregation(AggregationBuilders.terms(field+"_stats").field(field));
}
// 4. execute the query
SearchResponse response = search.execute().actionGet();
for(String field : statFields) {
Terms termAgg = (Terms) response.getAggregations().get(field+"_stats");
for (Terms.Bucket entry : termAgg.getBuckets()) {
System.out.println(entry.getKey() + " **** " + entry.getDocCount()); // Doc count
}
}
Below is the response
chethan**** 2
Raj**** 1
Mohan**** 1
1 **** 1
2 **** 1
3 **** 1
But I need combined response like sql,
name id count
chethan 1 1
is it possible through elasticsearch java api

You should have used subAggregation plus use keyword type for aggregations.
Java Rest High-Level Client
Assuming your mappings look like:
PUT student
{
"mappings": {
"doc": {
"properties": {
"name": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"id": {
"type": "keyword"
},
"address": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword"
}
}
}
}
}
}
}
In order to group by name and id you should use this query (low level query):
GET student/_search
{
"size": 0,
"aggs": {
"name": {
"terms": {
"field": "name.keyword",
"size": 10
},"aggs": {
"id": {
"terms": {
"field": "id",
"size": 10
}
}
}
}
}
}
In java the query above is similar to:
SearchResponse response = client.search(new SearchRequest("student")
.source(new SearchSourceBuilder()
.size(0)
.aggregation(
AggregationBuilders.terms("by_name").field("name.keyword")
.subAggregation(AggregationBuilders.terms("by_id")
.field("id")
)
);
If you want to use your code, probably something like that :
// 2. bootstrap the query
SearchRequestBuilder search = client.prepareSearch("student")
.setSize(0).setFrom(0)
.setQuery(QueryBuilders.matchAllQuery());
// 3. add a stats aggregation for each of your fields
TermsAggregationBuilder aggregation = AggregationBuilders.terms("name_stats").field("name.keyword");
aggregation.subAggregation(AggregationBuilders.terms("id_stats").field("id"));
search.aggregation(aggregation);
// 4. execute the query
SearchResponse response = search.execute().actionGet();
Terms termAgg = (Terms)response.getAggregations().get("name_stats");
for (Terms.Bucket entry: termAgg.getBuckets()) {
if (entry.getDocCount() != 0) {
Terms terms =entry.getAggregations().get("id");
Collection<Terms.Bucket> buckets = terms.getBuckets();
for (Bucket sub : buckets ) {
System.out.println((int) sub.getDocCount());
System.out.println(sub.getKeyAsString());
}
}
}
I removed the for loop. you should design your own structure now that you have to use sub-aggregations.
UPDATE
Is this what you want?
GET student/_search
{
"size": 0,
"aggs" : {
"name_id" : {
"terms" : {
"script" : {
"source": "doc['name.keyword'].value + '_' + doc['id'].value",
"lang": "painless"
}
}
}
}
}
I hope this is what you aimed for.

How to find a key in a JSON file with Gson library?

Given a JSON file which looks like the code below, how can I find a specific key with its corresponding value without hard-coding the structure in a Java class or otherwise. Let's say I want to get "Signal_Settings" as a JSON Object but given the possibility of changes in the JSON file structure I want to get this object regardless for example by iterating through all keys in the file. I have tried to do this iteratively and failed, so I think a recursive function is the way for which I didn't find a solution yet. My iterative code looks like this:
while (true) {
try {
for(Map.Entry<String, JsonElement> entry:newJsonObj.entrySet()){
System.out.println(entry.getKey());
System.out.println("Before NewJsonObj:" + newJsonObj);
newJsonObj = newJsonObj.getAsJsonObject(entry.getKey());
System.out.println("After NewJsonObj:" + newJsonObj + "\n");
//tempEntrySet = newJsonObj.entrySet().iterator().next();
}
//System.out.println("Key:" + tempEntrySet.getKey());
//System.out.println("TempEntry:" + tempEntrySet);
}catch (Exception e){
break;
}
}
and the JSON file:
{
"config2": {
"UDP_Settings": {
"ListenAddress": "'127.0.0.1'",
"ListenPort": "54523"
},
"Signal_Settings": {
"Count": 1,
"Signals": [
{
"Name": "time",
"Type": "uint8",
"Interval": "foo",
"Description": "",
"Unit": null
},
{
"Name": "othersignal",
"Type": "uint8",
"Interval": "fff",
"Description": "",
"Unit": null
}
]
},
"Tag_Settings": null,
"Model_Settings": null
}
}

Convert all the integer value to string in JSON

My JSON string is:
{name:"MyNode", width:200, height:100}
I want to change it to:
{name:"MyNode", width:"200", height:"100"}
so that all integer values become strings
My main code is:
{
"firstName": "John",
"lastName": "Smith",
"age": 25,
"address":
{
"streetAddress": "21 2nd Street",
"city": "New York",
"state": "NY",
"xy": 10021
},
"IDNumber":
[
{
"type": "home",
"number": 1234
},
{
"type": "fax",
"number": 4567
}
]
}
I need all integer values become strings

That's a JavaScript object literal, not JSON. Anyway...
var obj = {name:"MyNode", width:200, height:100};
for (var k in obj)
{
if (obj.hasOwnProperty(k))
{
obj[k] = String(obj[k]);
}
}
// obj = {name:"MyNode", width: "200", height: "100"}
If you're actually working with JSON, not an object, JSON.parse() the string beforehand, and JSON.stringify() the object afterward.

If you must operate on the JSON string :
json = json.replace (/:(\d+)([,\}])/g, ':"$1"$2');

I use
const stringifyNumbers = obj => {
const result = {};
Object.entries(obj).map(entry => {
const type = typeof entry[1];
if (Array.isArray(entry[1])) {
result[entry[0]] = entry[1].map(entry => stringifyNumbers(entry));
} else if (type === 'object' && !!entry[1]) {
result[entry[0]] = stringifyNumbers(entry[1]);
} else if (entry[1] === null) {
result[entry[0]] = null;
} else if (type === 'number') {
result[entry[0]] = String(entry[1]);
} else {
result[entry[0]] = entry[1];
}
});
return result;
}
most ids should be strings in my opinion. If your api cannot provide ids as strings this snippet will convert them.