I have the following JSON source:
{
"my-systems": [{
"SYSTEM_A": [{
"parameter-name": "a_param1",
"parameter-display-name": "parameter 1",
"optional": "true"
},
{
"parameter-name": "a_param2",
"parameter-display-name": "Parameter 2",
"optional": "false"
}
]
},
{
"SYSTEM_B": [{
"parameter-name": "b_param1",
"parameter-display-name": "Parameter 1",
"optional": "true"
},
{
"parameter-name": "b_param2",
"parameter-display-name": "Parameter 2",
"optional": "false"
}
]
}
]
}
I try to read it into a map of Map<String, SystemParameter[]>.
I have this code which I'm really not sure if it's the best approach for my goal.
ArrayNode systemsArr = (ArrayNode)jsonDoc.get("my-systems");
if(systemsArr!= null && !systemsArr.isEmpty()){
for(JsonNode systemNode : systemsArr ){
ObjectNode systemObj = (ObjectNode)systemNode;
System.out.println(systemObj .textValue());
}
}
Is it a valid approach? How do I get the name of the system (SYSTEM_A, SYSTEM_B) and convert the contained parameters into a parameters objects array?
You just need to use jackson-databind and jackson-annotations jar in your dependency and you should be able to run below code
import com.fasterxml.jackson.annotation.JsonProperty;
import com.fasterxml.jackson.databind.ObjectMapper;
import java.io.File;
import java.io.IOException;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
public class JsonParsing {
public static void main(String[] args) throws IOException {
String jsonFilePath = "src/main/resources/source.json"; // JSON File Path
MySystems mySystems = new ObjectMapper().readValue(new File(jsonFilePath),MySystems.class);
Map<String,SystemParameter[]> outputMap = new HashMap<>();
for (Map<String,List<SystemParameter>> map :mySystems.mySystems) {
for (String key :map.keySet()) {
outputMap.put(key, map.get(key).toArray(new SystemParameter[0]));
}
}
System.out.println(outputMap);
}
}
class MySystems {
#JsonProperty("my-systems")
List<Map<String,List<SystemParameter>>> mySystems;
}
class SystemParameter {
#JsonProperty("parameter-name")
String paramName;
#JsonProperty("parameter-display-name")
String paramDispName;
#JsonProperty("optional")
String optional;
}
Using an array 'my-systems' is of little use if all you're doing is keeping your keys unique. (I am assuming your SYSTEM_A may be different).
Instead, I suggest you format the JSON data in the following way:
{
"my-systems": [
{
"system-name" : {
"name":"System_A",
"parameters": [
{
"parameter-name": "a_param1",
"parameter-display-name": "parameter 1",
"optional": "true"
},
{
"parameter-name": "a_param2",
"parameter-display-name": "Parameter 2",
"optional": "false"
}
]
}
}
]
}
This method is clean, and allows you to catch the name of the system under the property 'system-name' and it's parameters are nested inside. You can simply declare a model and Jackson (or gson) will simply take care of everything.
If you'd rather parse the response separately or you have no control over the response, you can opt to go with your implementation but you do not need to convert to ObjectNode. You can use the samples below inside your for loop:
String f2Str = jsonNode.get("f2").asText();
double f2Dbl = jsonNode.get("f2").asDouble();
int f2Int = jsonNode.get("f2").asInt();
long f2Lng = jsonNode.get("f2").asLong();
where 'f2' is your key, and you can get the key using node.fieldNames().next() which is actually getting the property from an iterator.
You may also try the ones below, they seem better to handle.
JsonNode parent = ...;
for (Iterator<String> it = parent.fieldNames() ; it.hasNext() ; ) {
String field = it.next();
System.out.println(field + " => " + parent.get(field));
}
for (Iterator<Map.Entry<String,JsonNode>> it = parent.fields() ;
it.hasNext() ; ) {
Map.Entry<String,JsonNode> e = it.next();
System.out.println(e.getKey() + " => " + e.getValue());
}
The latest examples are taken from here. Hope this helps.
Related
I need to flatten a JSON strings(nested) using Gson ,
Given:
{
"name": "Terry",
"addr": {
"city": "NY",
"postCode": 123
},
"friends": ["John", "Paul"],
"nestedObject": { "a": { "b": { "c": {"d": "nested property" } } } }
}
Expected:
{
"name" :"Terry",
"addr.city": "NY",
"addr.postCode" : 123,
"friends.0": "John",
"friends.1": "Paul",
"nestedObject.a.b.c.d": "nested property"
}
or
{
"name" :"Terry",
"addr.city": "NY",
"addr.postCode" : 123,
"friends[0]": "John"
"friends[1]": "Paul"
"nestedObject.a.b.c.d": "nested property"
}
Really new to Java and need guidance how to build it up ,
My starting point :
1.Parse the Json using JsonParser as TreeModel
https://www.tutorialspoint.com/gson/gson_tree_model.htm
just found that JsonParser is deprecated :(
https://javadoc.io/doc/com.google.code.gson/gson/latest/com.google.gson/com/google/gson/JsonParser.html
is there alternative ?
2.Recursive function to loop through the Json elements and check
if element is list/dict - case yes , recursive call with all child ,
case else - append current key and val to some map
real pseudo code:
public class Test{
public static void main(String[] args) {
String jsonString = "{\"brand\" : \"Toyota\", \"doors\" : 5}";
JsonObject jsonObject = JsonParser.parseString(jsonString).getAsJsonObject();
// Iterate through main key and call FlatJson for each one
// append result to a map
//print result
FlatJson - for given element ,
extract all child key
public String FlatJson( JsonElement jsonElement) {
// If input is an array, iterate through each element
if (jsonElement.isJsonArray()) {
for (JsonElement jsonElement1 : jsonElement.getAsJsonArray()) {
// Extract Key Value and return
}
} else {
// If input is object, iterate through the keys
if (jsonElement.isJsonObject()) {
Set<Map.Entry<String, JsonElement>> entrySet = jsonElement
.getAsJsonObject().entrySet();
for (Map.Entry<String, JsonElement> entry : entrySet) {
// Extract Key Value and return
}
}
}
// If input is element, check whether it corresponds to the key
else {
// return sonElement.toString() (as
}
}
}
}
I want to add a new field to jsonObject and this new field's name will be based on a value of another field. To be clear, this an examples of what I want to achieve.
{
"values": [
{
"id": "1",
"properties": [
{
"stat": "memory",
"data": 8
},
{
"stat": "cpu",
"data": 4
}
]
},
{
"id": "2",
"properties": [
{
"stat": "status",
"data": "OK"
},
{
"stat": "cpu",
"data": 4
}
]
}
]
}
I want to add a new field to each json object that will have the value of field "stat" as name.
{
"values": [
{
"id": "1",
"properties": [
{
"stat": "memory",
"data": 8,
"memory": 8
},
{
"stat": "cpu",
"data": 4,
"cpu": 4
}
]
},
{
"id": "2",
"properties": [
{
"stat": "status",
"data": 0,
"status": 0
},
{
"stat": "cpu",
"data": 4,
"cpu": 4
}
]
}
]
}
I have tried to do the following with JsonPath library but for me it's an ugly solution as I will parse the json three times and I do some manual replacements.
val configuration = Configuration.builder().options(Option.DEFAULT_PATH_LEAF_TO_NULL, Option.ALWAYS_RETURN_LIST).build()
val jsonContext5 = JsonPath.using(configuration).parse(jsonStr)
val listData = jsonContext.read("$['values'][*]['properties'][*]['data']").toString
.replace("[", "").replace("]", "").split(",").toList
val listStat = jsonContext.read("$['values'][*]['properties'][*]['stat']").toString
.replace("[", "").replace("]", "")
.replace("\"", "").split(",").toList
// Replacing values of "stat" by values of "data"
jsonContext5.map("$['values'][*]['properties'][*]['stat']", new MapFunction() {
var count = - 1
override def map(currentValue: Any, configuration: Configuration): AnyRef = {
count += 1
listData(count)
}
})
// replace field stat by its value
for( count <- 0 to listStat.size - 1){
val path = s"['values'][*]['properties'][$count]"
jsonContext5.renameKey(path, "stat", s"${listStat(count)}")
}
This is the result obtained
{
"values": [
{
"id": "1",
"properties": [
{
"data": 8,
"memory": "8"
},
{
"data": 4,
"cpu": "4"
}
]
},
{
"id": "2",
"properties": [
{
"data": 0,
"memory": "0"
},
{
"data": 4,
"cpu": "4"
}
]
}
]
}
Is there any better method to achieve this result ? I tried to do it with gson but it's not good handling paths.
This a way to do it with Gson but I will lose the information about other columns since I'm creating another json.
val jsonArray = jsonObject.get("properties").getAsJsonArray
val iter = jsonArray.iterator()
val agreedJson = new JsonArray()
while(iter.hasNext) {
val json = iter.next().getAsJsonObject
agreedJson.add(replaceCols(json))
}
def replaceCols(json: JsonObject) = {
val fieldName = "stat"
if(json.has(fieldName)) {
val columnName = json.get(fieldName).getAsString
val value: String = if (json.has("data")) json.get("data").getAsString else ""
json.addProperty(columnName, value)
}
json
}
How about something like this?
private static void statDup(final JSONObject o) {
if (o.containsKey("properties")) {
final JSONArray a = (JSONArray) o.get("properties");
for (final Object e : a) {
final JSONObject p = (JSONObject) e;
p.put(p.get("stat"), p.get("data"));
}
} else {
for (final Object key : o.keySet()) {
final Object value = o.get(key);
if (value instanceof JSONArray) {
for (final Object e : (JSONArray) value) {
statDup((JSONObject) e);
}
}
}
}
}
Using Gson, what you should do is create a base class that represents your initial JSON object. Then, extend that class and add the additional attribute(s) you want to add, such as "stat". Then, load the JSON objects into memory, either one by one or all together, then make the necessary changes to each to encompass your changes. Then, map those changes to the new class if you didn't in the prior step, and serialize them to a file or some other storage.
This is type-safe, a pure FP circe implementation with circe-optics:
object CirceOptics extends App {
import cats.Applicative
import cats.implicits._
import io.circe.{Error => _, _}
import io.circe.syntax._
import io.circe.parser._
import io.circe.optics.JsonPath._
val jsonStr: String = ???
def getStat(json: Json): Either[Error, String] =
root.stat.string.getOption(json)
.toRight(new Error(s"Missing stat of string type in $json"))
def getData(json: Json): Either[Error, Json] =
root.data.json.getOption(json)
.toRight(new Error(s"Missing data of json type in $json"))
def setField(json: Json, key: String, value: Json) =
root.at(key).setOption(Some(value))(json)
.toRight(new Error(s"Unable to set $key -> $value to $json"))
def modifyAllPropertiesOfAllValuesWith[F[_]: Applicative](f: Json => F[Json])(json: Json): F[Json] =
root.values.each.properties.each.json.modifyF(f)(json)
val res = for {
json <- parse(jsonStr)
modifiedJson <- modifyAllPropertiesOfAllValuesWith { j =>
for {
stat <- getStat(j)
data <- getData(j)
prop <- setField(j, stat, data)
} yield prop
} (json)
} yield modifiedJson
println(res)
}
The previous answer from Gene McCulley gives a solution with Java and using class net.minidev.json. This answer is using class Gson and written in Scala.
def statDup(o: JsonObject): JsonObject = {
if (o.has("properties")) {
val a = o.get("properties").getAsJsonArray
a.foreach { e =>
val p = e.getAsJsonObject
p.add(p.get("stat").getAsString, p.get("data"))
}
} else {
o.keySet.foreach { key =>
o.get(key) match {
case jsonArr: JsonArray =>
jsonArr.foreach { e =>
statDup(e.getAsJsonObject)
}
}
}
}
o
}
Your task is to add a new field to each record under each properties in the JSON file, make the current stat value the field name and data values the new field values. The code will be rather long if you try to do it in Java.
Suggest you using SPL, an open-source Java package to get it done. Coding will be very easy and you only need one line:
A
1
=json(json(file("data.json").read()).values.run(properties=properties.(([["stat","data"]|stat]|[~.array()|data]).record())))
SPL offers JDBC driver to be invoked by Java. Just store the above SPL script as addfield.splx and invoke it in a Java application as you call a stored procedure:
…
Class.forName("com.esproc.jdbc.InternalDriver");
con= DriverManager.getConnection("jdbc:esproc:local://");
st = con.prepareCall("call addfield()");
st.execute();
…
I was wondering if anyone can help me or hint me towards how to edit the attached dummy JSON file in Java.
As you can see I have a head object that contains many values and children that follow the same pattern.
I wanted to know if there was a way to remove all the keys where the value is -1.
Following is what I was trying based on many websites using jackson:
try {
// create object mapper instance
ObjectMapper mapper = new ObjectMapper();
// convert JSON file to map
Map<?, ?> map = mapper.readValue(Paths.get("test.json").toFile(), Map.class);
// print map entries
for (Map.Entry<?, ?> entry : map.entrySet()) {
isInteger = main.isObjectInteger(entry.getValue());
// System.out.println("if value is all: " + entry.getKey() + "=" + entry.getValue());
//
The above code will display the structure of the file, however my problem is reaching the -1 values inside the children and removing them.
Using the .getClass and .simpleName methods I know that it is an arrayList but I am confused as to how to search through it.
Any help will be greatly appreciated!
In Jackson you can read whole JSON payload as JsonNode and iterate over all properties check given condition. In case condition is met you can remove given field. To do that you need to implement recursive method. Take a look on below example:
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.SerializationFeature;
import com.fasterxml.jackson.databind.node.ArrayNode;
import com.fasterxml.jackson.databind.node.ObjectNode;
import java.io.File;
import java.io.IOException;
import java.util.Iterator;
import java.util.Map;
import java.util.Objects;
import java.util.function.Predicate;
public class JsonRemoveSomeFieldsApp {
public static void main(String[] args) throws IOException {
File jsonFile = new File("./resource/test.json").getAbsoluteFile();
ObjectMapper mapper = new ObjectMapper();
mapper.enable(SerializationFeature.INDENT_OUTPUT);
JsonNode root = mapper.readTree(jsonFile);
JsonCleaner jsonCleaner = new JsonCleaner(root, (node) -> node.isNumber() && node.numberValue().intValue() == -1);
JsonNode result = jsonCleaner.removeAll();
// write to file
mapper.writeValue(System.out, result);
}
}
class JsonCleaner {
private final JsonNode root;
private final Predicate<JsonNode> toRemoveCondition;
JsonCleaner(JsonNode node, Predicate<JsonNode> toRemoveCondition) {
this.root = Objects.requireNonNull(node);
this.toRemoveCondition = Objects.requireNonNull(toRemoveCondition);
}
public JsonNode removeAll() {
process(root);
return root;
}
private void process(JsonNode node) {
if (node.isObject()) {
ObjectNode object = (ObjectNode) node;
Iterator<Map.Entry<String, JsonNode>> fields = object.fields();
while (fields.hasNext()) {
Map.Entry<String, JsonNode> field = fields.next();
JsonNode valueToCheck = field.getValue();
if (valueToCheck.isContainerNode()) {
process(valueToCheck);
} else if (toRemoveCondition.test(valueToCheck)) {
fields.remove();
}
}
} else if (node.isArray()) {
ArrayNode array = (ArrayNode) node;
array.elements().forEachRemaining(this::process);
}
}
}
For below JSON payload:
{
"name": "Head",
"missed": -1,
"covered": -1,
"children": [
{
"name": "project1",
"missed": -1,
"covered": -1,
"children": [
{
"name": "project1",
"missed": 10,
"covered": 11
}
]
},
{
"name": "project1",
"missed": -1,
"covered": 12,
"children": [
{
"name": "project1",
"missed": 10,
"covered": -1
}
]
}
]
}
above code prints:
{
"name" : "Head",
"children" : [ {
"name" : "project1",
"children" : [ {
"name" : "project1",
"missed" : 10,
"covered" : 11
} ]
}, {
"name" : "project1",
"covered" : 12,
"children" : [ {
"name" : "project1",
"missed" : 10
} ]
} ]
}
See also:
Flattening a 3 level nested JSON string in java
There are two main techniques to parse and generate JSON data (as well as many other formats like XML etc): object mapping and event/token/stream-oriented processing. The second way is the best way for many cases, including filtering. Props:
the file/data doesn't require to be loaded entirely into memory, you
can process megs/gigs with no problems
it works much more faster, especially for large files
it's easy to implement any custom type/rule of transformation with this pattern
Both Gson and Jackson supports stream-oriented processing. To illustrate the idea here is just an example using a tiny parser/generator https://github.com/anatolygudkov/green-jelly
import org.green.jelly.AppendableWriter;
import org.green.jelly.JsonBufferedWriter;
import org.green.jelly.JsonEventPump;
import org.green.jelly.JsonNumber;
import org.green.jelly.JsonParser;
import java.io.StringWriter;
public class UpdateMyJson {
private static final String jsonToUpdate = "{\n" +
"\"name\": \"Head\",\n" +
"\"missed\": -1,\n" +
"\"children\": [\n" +
" {\n" +
" \"name\": \"project1\",\n" +
" \"fixes\": 0,\n" +
" \"commits\": -1,\n" +
" },\n" +
" {\n" +
" \"name\": \"project2\",\n" +
" \"fixes\": 20,\n" +
" \"commits\": 5,\n" +
" }\n" +
"]\n" +
"}";
public static void main(String[] args) {
final StringWriter result = new StringWriter(); // you can use FileWriter
final JsonParser parser = new JsonParser();
parser.setListener(new MyJsonUpdater(new AppendableWriter<>(result)));
parser.parseAndEoj(jsonToUpdate); // if you read a file with a buffer,
// to don't load the whole file into memory,
// call parse() several times (part by part) in a loop until EOF
// and then call .eoj()
System.out.println(result);
}
static class MyJsonUpdater extends JsonEventPump {
MyJsonUpdater(final JsonBufferedWriter output) {
super(output);
}
#Override
public boolean onNumberValue(final JsonNumber number) {
if (number.mantissa() == -1 && number.exp() == 0) {
return true; // return immediately
}
return super.onNumberValue(number); // otherwise pump the value to the result JSON
}
}
}
I have a JSON that I'm getting from some external system. I need to convert that JSON into key value based on my system mapping. For instance:
JSON from external system:
[{
"name": "Tim",
"address": "New York",
"education" : {
"university": "XYZ"
}
},
{
"name": "Steve",
"address": "UK"
}]
I have following mapping that we need to use:
{
"name": "firstName",
"address": "location",
"university": "college"
}
i.e name to be mapped to firstName and address to be mapped to location. And finally, my processed mapped will look like:
[{
"firstName": "Tim",
"location": "New York"
"education" : {
"college": "XYZ"
}
},
{
"firstName": "Steve",
"location": "UK"
}]
What can be the best way to achieve this? Should I use normal hashmap operations or is there any other efficient way. I was checking JSONNode for this purpose, but the approach was similar to hash map. Is there any utility I can use to traverse tree like json map and replace the key?
I'm a fan of Jackson which you can use to traverse the JSON. As you do so, populate a list of maps, using your system mapping to replace whatever keys are encountered that have a mapping, leaving the others as is. At the end, dump the list of maps back out as JSON.
Edit: adding code example below
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.node.ArrayNode;
import java.io.IOException;
import java.util.*;
public class ReplaceKeys {
private static final ObjectMapper mapper = new ObjectMapper();
private static Map<String, String> keymap;
static {
try {
// Create the key map based on the json provided by OP
keymap = mapper.readValue("{\n" +
"\"name\": \"firstName\",\n" +
"\"address\": \"location\",\n" +
"\"university\": \"college\"\n" +
"}", Map.class);
} catch (IOException e) {
System.out.println("whoops");
}
}
public static String mapKeys(String input) throws IOException {
// Assume the input is an array and therefore so is the output.
List<Map<String, Object>> output = new ArrayList<>();
ArrayNode inputArray = (ArrayNode) mapper.readTree(input);
for (JsonNode node : inputArray) {
output.add(mapKeys(node));
}
return mapper.writeValueAsString(output);
}
private static Map<String, Object> mapKeys(JsonNode node) {
Map<String, Object> map = new HashMap<>();
for (Iterator<String> iterator = node.fieldNames(); iterator.hasNext(); ) {
String key = iterator.next();
key = keymap.containsKey(key) ? keymap.get(key) : key;
for (JsonNode child : node) {
if (node.isValueNode()) {
// This is coercing everything to a String. You could dig into using
// proper types based on node.getNodeType().
map.put(key, node.asText());
} else {
map.put(key, mapKeys(child));
}
}
}
return map;
}
}
When dealing with a valid JSON which represents a two dimensional matrix:
[ { "a":1, "b":2 }, { "b":3, "c":4 }, { "c":6, "a":5 } ]
Using Java 1.8 and the json-simple lib:
<dependency>
<groupId>com.googlecode.json-simple</groupId>
<artifactId>json-simple</artifactId>
<version>1.1.1</version>
</dependency>
Am able to obtain an individual row using the following?:
JSONParser parser = new JSONParser();
Object obj = parser.parse(args[0]);
JSONArray array = (JSONArray) obj;
System.out.println(array.get(0));
This outputs:
{"a":1,"b":2}
My question is how to extract the key values from this two dimensional matrix to output this:
'{ "a": [1,null,5], "b": [2,3,null], "c": [null,4,6] }'
Notice, that any variables that are missing from the row should have null inserted...
General solution could look like below:
Read JSON payload as ARRAY.
Find all possible keys in all JSON Object's in ARRAY and store them in KEYS set.
Iterate over all JSON Object's in ARRAY.
Iterate over all KEYS and get value or null from given JSON Object.
Using Java 8 features we can implement it as below:
import org.json.simple.JSONArray;
import org.json.simple.JSONObject;
import org.json.simple.parser.JSONParser;
import org.json.simple.parser.ParseException;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.stream.Collectors;
public class JsonSimpleApp {
public static void main(String[] args) throws ParseException {
// input
String json = "[ { \"a\":1, \"b\":2 }, { \"b\":3, \"c\":4 }, { \"c\":6, \"a\":5 } ]";
// read as JSONArray
JSONParser parser = new JSONParser();
JSONArray array = (JSONArray) parser.parse(json);
// Find all possible keys
Set<String> keys = (Set<String>) array.stream().flatMap(i -> ((JSONObject) i).keySet().stream()).collect(Collectors.toSet());
// create result Map with KEY - LIST pair
Map<String, List<Object>> result = new HashMap<>();
keys.forEach(key -> result.put(key, new ArrayList<>()));
// Iterate over all JSONObject's
array.forEach(item -> {
JSONObject object = (JSONObject) item;
// For each possible key add a value or NULL to each list
keys.forEach(key -> result.get(key).add(object.remove(key)));
});
result.entrySet().forEach(System.out::println);
}
}
Above code prints:
a=[1, null, 5]
b=[2, 3, null]
c=[null, 4, 6]
You can acces in json by key. For example:
obj["key"][0]
Then you can take the value
Edited
Simply you can use JOLT for this purpose.
JSON to JSON transformation library written in Java where the
"specification" for the transform is itself a JSON document.
According to your scenario, you must Just:
1- Create a rule
2- Filter your input via this rule.
The JSON input is as follows
[
{
"a": 1,
"b": 2
},
{
"b": 3,
"c": 4
},
{
"c": 6,
"a": 5
}
]
The filter rule (JOLT spec) can be as follows
[
{
"operation": "shift",
"spec": {
"*": {
"a": "a",
"b": "b",
"c": "c"
}
}
}
]
result
{
"a" : [ 1, 5 ],
"b" : [ 2, 3 ],
"c" : [ 4, 6 ]
}
You can test the aforementioned solution in here
Java code for applying the aforementioned items in java
public class CustomJsonTransformer {
public static void main(String[] args) throws Exception {
//Get filter (JOLT SPEC)
List<Object> specs = JsonUtils.classpathToList("/filter.json");
Chainr chainr = Chainr.fromSpec(specs);
//Get input JSON
Object inputJSON = JsonUtils.classpathToObject("/input.json");
//Do filter
Object transformedOutput = chainr.transform(inputJSON);
//Result
System.out.println(JsonUtils.toPrettyJsonString(transformedOutput));
}
}
1- This guide shows a pretty good sample on JOLT in java
2- Complete documentation of JOLT is here
Notice that, with the help of JOLT documentation, you can make the rule more complicated or easier (My rule was just a simple one)