I want to add a new field to jsonObject and this new field's name will be based on a value of another field. To be clear, this an examples of what I want to achieve.
{
"values": [
{
"id": "1",
"properties": [
{
"stat": "memory",
"data": 8
},
{
"stat": "cpu",
"data": 4
}
]
},
{
"id": "2",
"properties": [
{
"stat": "status",
"data": "OK"
},
{
"stat": "cpu",
"data": 4
}
]
}
]
}
I want to add a new field to each json object that will have the value of field "stat" as name.
{
"values": [
{
"id": "1",
"properties": [
{
"stat": "memory",
"data": 8,
"memory": 8
},
{
"stat": "cpu",
"data": 4,
"cpu": 4
}
]
},
{
"id": "2",
"properties": [
{
"stat": "status",
"data": 0,
"status": 0
},
{
"stat": "cpu",
"data": 4,
"cpu": 4
}
]
}
]
}
I have tried to do the following with JsonPath library but for me it's an ugly solution as I will parse the json three times and I do some manual replacements.
val configuration = Configuration.builder().options(Option.DEFAULT_PATH_LEAF_TO_NULL, Option.ALWAYS_RETURN_LIST).build()
val jsonContext5 = JsonPath.using(configuration).parse(jsonStr)
val listData = jsonContext.read("$['values'][*]['properties'][*]['data']").toString
.replace("[", "").replace("]", "").split(",").toList
val listStat = jsonContext.read("$['values'][*]['properties'][*]['stat']").toString
.replace("[", "").replace("]", "")
.replace("\"", "").split(",").toList
// Replacing values of "stat" by values of "data"
jsonContext5.map("$['values'][*]['properties'][*]['stat']", new MapFunction() {
var count = - 1
override def map(currentValue: Any, configuration: Configuration): AnyRef = {
count += 1
listData(count)
}
})
// replace field stat by its value
for( count <- 0 to listStat.size - 1){
val path = s"['values'][*]['properties'][$count]"
jsonContext5.renameKey(path, "stat", s"${listStat(count)}")
}
This is the result obtained
{
"values": [
{
"id": "1",
"properties": [
{
"data": 8,
"memory": "8"
},
{
"data": 4,
"cpu": "4"
}
]
},
{
"id": "2",
"properties": [
{
"data": 0,
"memory": "0"
},
{
"data": 4,
"cpu": "4"
}
]
}
]
}
Is there any better method to achieve this result ? I tried to do it with gson but it's not good handling paths.
This a way to do it with Gson but I will lose the information about other columns since I'm creating another json.
val jsonArray = jsonObject.get("properties").getAsJsonArray
val iter = jsonArray.iterator()
val agreedJson = new JsonArray()
while(iter.hasNext) {
val json = iter.next().getAsJsonObject
agreedJson.add(replaceCols(json))
}
def replaceCols(json: JsonObject) = {
val fieldName = "stat"
if(json.has(fieldName)) {
val columnName = json.get(fieldName).getAsString
val value: String = if (json.has("data")) json.get("data").getAsString else ""
json.addProperty(columnName, value)
}
json
}
How about something like this?
private static void statDup(final JSONObject o) {
if (o.containsKey("properties")) {
final JSONArray a = (JSONArray) o.get("properties");
for (final Object e : a) {
final JSONObject p = (JSONObject) e;
p.put(p.get("stat"), p.get("data"));
}
} else {
for (final Object key : o.keySet()) {
final Object value = o.get(key);
if (value instanceof JSONArray) {
for (final Object e : (JSONArray) value) {
statDup((JSONObject) e);
}
}
}
}
}
Using Gson, what you should do is create a base class that represents your initial JSON object. Then, extend that class and add the additional attribute(s) you want to add, such as "stat". Then, load the JSON objects into memory, either one by one or all together, then make the necessary changes to each to encompass your changes. Then, map those changes to the new class if you didn't in the prior step, and serialize them to a file or some other storage.
This is type-safe, a pure FP circe implementation with circe-optics:
object CirceOptics extends App {
import cats.Applicative
import cats.implicits._
import io.circe.{Error => _, _}
import io.circe.syntax._
import io.circe.parser._
import io.circe.optics.JsonPath._
val jsonStr: String = ???
def getStat(json: Json): Either[Error, String] =
root.stat.string.getOption(json)
.toRight(new Error(s"Missing stat of string type in $json"))
def getData(json: Json): Either[Error, Json] =
root.data.json.getOption(json)
.toRight(new Error(s"Missing data of json type in $json"))
def setField(json: Json, key: String, value: Json) =
root.at(key).setOption(Some(value))(json)
.toRight(new Error(s"Unable to set $key -> $value to $json"))
def modifyAllPropertiesOfAllValuesWith[F[_]: Applicative](f: Json => F[Json])(json: Json): F[Json] =
root.values.each.properties.each.json.modifyF(f)(json)
val res = for {
json <- parse(jsonStr)
modifiedJson <- modifyAllPropertiesOfAllValuesWith { j =>
for {
stat <- getStat(j)
data <- getData(j)
prop <- setField(j, stat, data)
} yield prop
} (json)
} yield modifiedJson
println(res)
}
The previous answer from Gene McCulley gives a solution with Java and using class net.minidev.json. This answer is using class Gson and written in Scala.
def statDup(o: JsonObject): JsonObject = {
if (o.has("properties")) {
val a = o.get("properties").getAsJsonArray
a.foreach { e =>
val p = e.getAsJsonObject
p.add(p.get("stat").getAsString, p.get("data"))
}
} else {
o.keySet.foreach { key =>
o.get(key) match {
case jsonArr: JsonArray =>
jsonArr.foreach { e =>
statDup(e.getAsJsonObject)
}
}
}
}
o
}
Your task is to add a new field to each record under each properties in the JSON file, make the current stat value the field name and data values the new field values. The code will be rather long if you try to do it in Java.
Suggest you using SPL, an open-source Java package to get it done. Coding will be very easy and you only need one line:
A
1
=json(json(file("data.json").read()).values.run(properties=properties.(([["stat","data"]|stat]|[~.array()|data]).record())))
SPL offers JDBC driver to be invoked by Java. Just store the above SPL script as addfield.splx and invoke it in a Java application as you call a stored procedure:
…
Class.forName("com.esproc.jdbc.InternalDriver");
con= DriverManager.getConnection("jdbc:esproc:local://");
st = con.prepareCall("call addfield()");
st.execute();
…
Related
I have a json
{"id": 2,"name": "Chethan","address":"Banglore"}
Trying to groupby two fields id and name,
List<String> statFields = new ArrayList();
statFields.add("name");
statFields.add("id");
// 2. bootstrap the query
SearchRequestBuilder search = client.prepareSearch("student")
.setSize(0).setFrom(0)
.setQuery(QueryBuilders.matchAllQuery());
// 3. add a stats aggregation for each of your fields
for (String field : statFields) {
search.addAggregation(AggregationBuilders.terms(field+"_stats").field(field));
}
// 4. execute the query
SearchResponse response = search.execute().actionGet();
for(String field : statFields) {
Terms termAgg = (Terms) response.getAggregations().get(field+"_stats");
for (Terms.Bucket entry : termAgg.getBuckets()) {
System.out.println(entry.getKey() + " **** " + entry.getDocCount()); // Doc count
}
}
Below is the response
chethan**** 2
Raj**** 1
Mohan**** 1
1 **** 1
2 **** 1
3 **** 1
But I need combined response like sql,
name id count
chethan 1 1
is it possible through elasticsearch java api
You should have used subAggregation plus use keyword type for aggregations.
Java Rest High-Level Client
Assuming your mappings look like:
PUT student
{
"mappings": {
"doc": {
"properties": {
"name": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"id": {
"type": "keyword"
},
"address": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword"
}
}
}
}
}
}
}
In order to group by name and id you should use this query (low level query):
GET student/_search
{
"size": 0,
"aggs": {
"name": {
"terms": {
"field": "name.keyword",
"size": 10
},"aggs": {
"id": {
"terms": {
"field": "id",
"size": 10
}
}
}
}
}
}
In java the query above is similar to:
SearchResponse response = client.search(new SearchRequest("student")
.source(new SearchSourceBuilder()
.size(0)
.aggregation(
AggregationBuilders.terms("by_name").field("name.keyword")
.subAggregation(AggregationBuilders.terms("by_id")
.field("id")
)
);
If you want to use your code, probably something like that :
// 2. bootstrap the query
SearchRequestBuilder search = client.prepareSearch("student")
.setSize(0).setFrom(0)
.setQuery(QueryBuilders.matchAllQuery());
// 3. add a stats aggregation for each of your fields
TermsAggregationBuilder aggregation = AggregationBuilders.terms("name_stats").field("name.keyword");
aggregation.subAggregation(AggregationBuilders.terms("id_stats").field("id"));
search.aggregation(aggregation);
// 4. execute the query
SearchResponse response = search.execute().actionGet();
Terms termAgg = (Terms)response.getAggregations().get("name_stats");
for (Terms.Bucket entry: termAgg.getBuckets()) {
if (entry.getDocCount() != 0) {
Terms terms =entry.getAggregations().get("id");
Collection<Terms.Bucket> buckets = terms.getBuckets();
for (Bucket sub : buckets ) {
System.out.println((int) sub.getDocCount());
System.out.println(sub.getKeyAsString());
}
}
}
I removed the for loop. you should design your own structure now that you have to use sub-aggregations.
UPDATE
Is this what you want?
GET student/_search
{
"size": 0,
"aggs" : {
"name_id" : {
"terms" : {
"script" : {
"source": "doc['name.keyword'].value + '_' + doc['id'].value",
"lang": "painless"
}
}
}
}
}
I hope this is what you aimed for.
I'm trying to get the value of a specific attribute in a JSON file but instead I get a row content of the array.
For example that's my JSON file:
{
"head": {
"vars": [ "type1" , "pred" , "type2" ]
} ,
"results": {
"bindings": [
{
"type1": { "type": "Collection" } ,
"type2": { "type": "has" } ,
"type3": { "type": "contributor" }
} ,
{
"type1": { "type": "Collection2" } ,
"type2": { "type": "has2" } ,
"type3": { "type": "contributor2" }
}
]
}
}
I want to get only the values of attribute "type3"
But my following code gets me all of them.
JSONObject obj = new JSONObject(json);
JSONObject results = obj.getJSONObject("results");
JSONArray bindings = results.getJSONArray("bindings");
for (int i=0; i<bindings.length(); i++)
{
JSONObject x = bindings.getJSONObject(i);
x.getJSONObject("type3");
}
I tried several approaches but it seems I'm doing it wrong.
I only want to get this : { "type": "contributor" }
Then get that value (roughly) like so
bindings.getJSONObject(0).getJSONObject("type3")
you can use JsonPath.read to get all Type3 values as list.
List value = JsonPath.read(bindings, "..type3");
I have a JSON string and I am trying to retrieve information from it. Json String looks like this.
JSON STRING :
{
"information": {
"device": {
"id": 0
},
"user": {
"id": 0
},
"data": [
{
"datum": {
"id": "00GF001",
"history_id": "9992BH",
"name": "abc",
"marks": 57,
"class": "B",
"type": "Student"
}
},
{
"datum": {
"id": "72BA9585",
"history_id": "78NAH2",
"name": "ndnmanet",
"marks": 70,
"class": "B",
"type": "Student"
}
},
{
"datum": {
"id": "69AHH85",
"history_id": "NN00E3006",
"name": "kit",
"department": "EF003",
"class": "A",
"type": "Employee"
}
},
{
"datum": {
"id": "09HL543",
"history_id": "34QWFTA",
"name": "jeff",
"department": "BH004",
"class": "A1",
"type": "Employee_HR"
}
}
]
}
}
I am trying to access data JSONArray and respective Datum from it. I differentiated each datum as per type such as student, employee etc and push information in hashmap.
I successfully did it in javascript but in Java I am struggle abit.
When I am trying to access JSONArray it throws exception
try {
JSONObject data = new JSONObject(dataInfo);
// Log.d(TAG, "CHECK"+data.toString());
JSONObject info = data.optJSONObject("information");
if(info.getJSONArray("data").getString(0).equals("Student") > 0) //exception here
Log.d(TAG, "Data"+ data.getJSONArray("data").length()); //exception here too
for(int m = 0; m < data.length(); m++){
// for(int s = 0; s < data[m].ge)
}
} catch (JSONException j){
j.printStackTrace();
}
Any pointers to create hashmap respective type I have. Appreciated
If you're trying to access the type field of a datum object, you'll want something like this:
JSONObject data = new JSONObject(dataInfo); // get the entire JSON into an object
JSONObject info = data.getJSONObject("information"); // get the 'information' object
JSONArray dataArray = info.getJSONArray("data"); // get the 'data' array
for (int i = 0; i < dataArray.length(); i++) {
// foreach element in the 'data' array
JSONObject dataObj = dataArray.getJSONObject(i); // get the object from the array
JSONObject datum = dataObj.getJSONObject("datum"); // get the 'datum' object
String type = datum.getString("type"); // get the 'type' string
if ("Student".equals(type)) {
// do your processing for 'Student' here
}
}
Note that you'll have to deal with exception handling, bad data, etc. This code just shows you the basics of how to get at the data that you're looking for. I separated each individual step into its own line of code so that I could clearly comment what is happening at each step, but you could combine some of the steps into a single line of code if that is easier for you.
if dataInfo is the json you posted, then you have to access information and from information, you can access data:
JSONObject data = new JSONObject(dataInfo);
JSONObject info = data.optJSONObject("information");
if (info != null) {
JSONArray dataArray = info.optJSONArray("data")
}
I have a question about getting jsonobjects out of a jsonobject.
This is what im getting back:
{
"data": {
"someid": {
"air_by_date": 0,
"cache": {
"banner": 1,
"poster": 1
},
"language": "en",
"network": "somenetwork",
"next_ep_airdate": "somedate",
"paused": 0,
"quality": "somequality",
"show_name": "somename",
"status": "Continuing",
"tvdbid": someid,
"tvrage_id": someid,
"tvrage_name": "Showname"
},
"someid": {
"air_by_date": 0,
"cache": {
"banner": 1,
"poster": 1
},
"language": "en",
"network": "somenetwork",
"next_ep_airdate": "",
"paused": 0,
"quality": "somequality",
"show_name": "somename",
"status": "Continuing",
"tvdbid": someid,
"tvrage_id": someid,
"tvrage_name": "somename"
},
But how am i supposed to create "ShowObjects" of them. I know how it works with JSONArrays but i have never done this kind of JSON before.
This is what i got untill now:
String json = download(url);
JSONObject result = new JSONObject(json);
JSONObject resultData = result.getJSONObject("data");
Copied from this answer:
Use the keys() iterator to iterate over all the properties, and call get() for each.
Iterator<String> iter = json.keys();
while (iter.hasNext()) {
String key = iter.next();
try {
Object value = json.get(key);
} catch (JSONException e) {
// Something went wrong!
}
}
Try using Jackson or Gson for these tasks.
With Jackson you would do something like this:
class Response() {
public String message;
public String result;
public Map<String, Object> data;
}
ObjectMapper mapper = new ObjectMapper()
Resonse respone = mapper.readValue(json, Response.class)
or use a custom deserializer to read those values out.
I will give you an example how to fetch the objects and vars in your "data" json:
lets say the second "someid" is : "123". We will fetch the second jsonObject now :
JSONObject secondJsonSomeId = resultData.getJsonObject("123");
Now we will fetch the "banner" ( int =1 ) of the jsonObject named "cache" located in our secondJsonSomeId :
int bannerInsecondJsonSomeId = secondJsonSomeId.getJsonObject("cache").getInt("banner");// bannerInsecondJsonSomeId == 1
the code is not compiled so there might be errors , ask if you have additional questions.
My JSON string is:
{name:"MyNode", width:200, height:100}
I want to change it to:
{name:"MyNode", width:"200", height:"100"}
so that all integer values become strings
My main code is:
{
"firstName": "John",
"lastName": "Smith",
"age": 25,
"address":
{
"streetAddress": "21 2nd Street",
"city": "New York",
"state": "NY",
"xy": 10021
},
"IDNumber":
[
{
"type": "home",
"number": 1234
},
{
"type": "fax",
"number": 4567
}
]
}
I need all integer values become strings
That's a JavaScript object literal, not JSON. Anyway...
var obj = {name:"MyNode", width:200, height:100};
for (var k in obj)
{
if (obj.hasOwnProperty(k))
{
obj[k] = String(obj[k]);
}
}
// obj = {name:"MyNode", width: "200", height: "100"}
If you're actually working with JSON, not an object, JSON.parse() the string beforehand, and JSON.stringify() the object afterward.
If you must operate on the JSON string :
json = json.replace (/:(\d+)([,\}])/g, ':"$1"$2');
I use
const stringifyNumbers = obj => {
const result = {};
Object.entries(obj).map(entry => {
const type = typeof entry[1];
if (Array.isArray(entry[1])) {
result[entry[0]] = entry[1].map(entry => stringifyNumbers(entry));
} else if (type === 'object' && !!entry[1]) {
result[entry[0]] = stringifyNumbers(entry[1]);
} else if (entry[1] === null) {
result[entry[0]] = null;
} else if (type === 'number') {
result[entry[0]] = String(entry[1]);
} else {
result[entry[0]] = entry[1];
}
});
return result;
}
most ids should be strings in my opinion. If your api cannot provide ids as strings this snippet will convert them.