I have a JSON that I'm getting from some external system. I need to convert that JSON into key value based on my system mapping. For instance:
JSON from external system:
[{
"name": "Tim",
"address": "New York",
"education" : {
"university": "XYZ"
}
},
{
"name": "Steve",
"address": "UK"
}]
I have following mapping that we need to use:
{
"name": "firstName",
"address": "location",
"university": "college"
}
i.e name to be mapped to firstName and address to be mapped to location. And finally, my processed mapped will look like:
[{
"firstName": "Tim",
"location": "New York"
"education" : {
"college": "XYZ"
}
},
{
"firstName": "Steve",
"location": "UK"
}]
What can be the best way to achieve this? Should I use normal hashmap operations or is there any other efficient way. I was checking JSONNode for this purpose, but the approach was similar to hash map. Is there any utility I can use to traverse tree like json map and replace the key?
I'm a fan of Jackson which you can use to traverse the JSON. As you do so, populate a list of maps, using your system mapping to replace whatever keys are encountered that have a mapping, leaving the others as is. At the end, dump the list of maps back out as JSON.
Edit: adding code example below
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.node.ArrayNode;
import java.io.IOException;
import java.util.*;
public class ReplaceKeys {
private static final ObjectMapper mapper = new ObjectMapper();
private static Map<String, String> keymap;
static {
try {
// Create the key map based on the json provided by OP
keymap = mapper.readValue("{\n" +
"\"name\": \"firstName\",\n" +
"\"address\": \"location\",\n" +
"\"university\": \"college\"\n" +
"}", Map.class);
} catch (IOException e) {
System.out.println("whoops");
}
}
public static String mapKeys(String input) throws IOException {
// Assume the input is an array and therefore so is the output.
List<Map<String, Object>> output = new ArrayList<>();
ArrayNode inputArray = (ArrayNode) mapper.readTree(input);
for (JsonNode node : inputArray) {
output.add(mapKeys(node));
}
return mapper.writeValueAsString(output);
}
private static Map<String, Object> mapKeys(JsonNode node) {
Map<String, Object> map = new HashMap<>();
for (Iterator<String> iterator = node.fieldNames(); iterator.hasNext(); ) {
String key = iterator.next();
key = keymap.containsKey(key) ? keymap.get(key) : key;
for (JsonNode child : node) {
if (node.isValueNode()) {
// This is coercing everything to a String. You could dig into using
// proper types based on node.getNodeType().
map.put(key, node.asText());
} else {
map.put(key, mapKeys(child));
}
}
}
return map;
}
}
Related
I need to flatten a JSON strings(nested) using Gson ,
Given:
{
"name": "Terry",
"addr": {
"city": "NY",
"postCode": 123
},
"friends": ["John", "Paul"],
"nestedObject": { "a": { "b": { "c": {"d": "nested property" } } } }
}
Expected:
{
"name" :"Terry",
"addr.city": "NY",
"addr.postCode" : 123,
"friends.0": "John",
"friends.1": "Paul",
"nestedObject.a.b.c.d": "nested property"
}
or
{
"name" :"Terry",
"addr.city": "NY",
"addr.postCode" : 123,
"friends[0]": "John"
"friends[1]": "Paul"
"nestedObject.a.b.c.d": "nested property"
}
Really new to Java and need guidance how to build it up ,
My starting point :
1.Parse the Json using JsonParser as TreeModel
https://www.tutorialspoint.com/gson/gson_tree_model.htm
just found that JsonParser is deprecated :(
https://javadoc.io/doc/com.google.code.gson/gson/latest/com.google.gson/com/google/gson/JsonParser.html
is there alternative ?
2.Recursive function to loop through the Json elements and check
if element is list/dict - case yes , recursive call with all child ,
case else - append current key and val to some map
real pseudo code:
public class Test{
public static void main(String[] args) {
String jsonString = "{\"brand\" : \"Toyota\", \"doors\" : 5}";
JsonObject jsonObject = JsonParser.parseString(jsonString).getAsJsonObject();
// Iterate through main key and call FlatJson for each one
// append result to a map
//print result
FlatJson - for given element ,
extract all child key
public String FlatJson( JsonElement jsonElement) {
// If input is an array, iterate through each element
if (jsonElement.isJsonArray()) {
for (JsonElement jsonElement1 : jsonElement.getAsJsonArray()) {
// Extract Key Value and return
}
} else {
// If input is object, iterate through the keys
if (jsonElement.isJsonObject()) {
Set<Map.Entry<String, JsonElement>> entrySet = jsonElement
.getAsJsonObject().entrySet();
for (Map.Entry<String, JsonElement> entry : entrySet) {
// Extract Key Value and return
}
}
}
// If input is element, check whether it corresponds to the key
else {
// return sonElement.toString() (as
}
}
}
}
I was wondering if anyone can help me or hint me towards how to edit the attached dummy JSON file in Java.
As you can see I have a head object that contains many values and children that follow the same pattern.
I wanted to know if there was a way to remove all the keys where the value is -1.
Following is what I was trying based on many websites using jackson:
try {
// create object mapper instance
ObjectMapper mapper = new ObjectMapper();
// convert JSON file to map
Map<?, ?> map = mapper.readValue(Paths.get("test.json").toFile(), Map.class);
// print map entries
for (Map.Entry<?, ?> entry : map.entrySet()) {
isInteger = main.isObjectInteger(entry.getValue());
// System.out.println("if value is all: " + entry.getKey() + "=" + entry.getValue());
//
The above code will display the structure of the file, however my problem is reaching the -1 values inside the children and removing them.
Using the .getClass and .simpleName methods I know that it is an arrayList but I am confused as to how to search through it.
Any help will be greatly appreciated!
In Jackson you can read whole JSON payload as JsonNode and iterate over all properties check given condition. In case condition is met you can remove given field. To do that you need to implement recursive method. Take a look on below example:
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.SerializationFeature;
import com.fasterxml.jackson.databind.node.ArrayNode;
import com.fasterxml.jackson.databind.node.ObjectNode;
import java.io.File;
import java.io.IOException;
import java.util.Iterator;
import java.util.Map;
import java.util.Objects;
import java.util.function.Predicate;
public class JsonRemoveSomeFieldsApp {
public static void main(String[] args) throws IOException {
File jsonFile = new File("./resource/test.json").getAbsoluteFile();
ObjectMapper mapper = new ObjectMapper();
mapper.enable(SerializationFeature.INDENT_OUTPUT);
JsonNode root = mapper.readTree(jsonFile);
JsonCleaner jsonCleaner = new JsonCleaner(root, (node) -> node.isNumber() && node.numberValue().intValue() == -1);
JsonNode result = jsonCleaner.removeAll();
// write to file
mapper.writeValue(System.out, result);
}
}
class JsonCleaner {
private final JsonNode root;
private final Predicate<JsonNode> toRemoveCondition;
JsonCleaner(JsonNode node, Predicate<JsonNode> toRemoveCondition) {
this.root = Objects.requireNonNull(node);
this.toRemoveCondition = Objects.requireNonNull(toRemoveCondition);
}
public JsonNode removeAll() {
process(root);
return root;
}
private void process(JsonNode node) {
if (node.isObject()) {
ObjectNode object = (ObjectNode) node;
Iterator<Map.Entry<String, JsonNode>> fields = object.fields();
while (fields.hasNext()) {
Map.Entry<String, JsonNode> field = fields.next();
JsonNode valueToCheck = field.getValue();
if (valueToCheck.isContainerNode()) {
process(valueToCheck);
} else if (toRemoveCondition.test(valueToCheck)) {
fields.remove();
}
}
} else if (node.isArray()) {
ArrayNode array = (ArrayNode) node;
array.elements().forEachRemaining(this::process);
}
}
}
For below JSON payload:
{
"name": "Head",
"missed": -1,
"covered": -1,
"children": [
{
"name": "project1",
"missed": -1,
"covered": -1,
"children": [
{
"name": "project1",
"missed": 10,
"covered": 11
}
]
},
{
"name": "project1",
"missed": -1,
"covered": 12,
"children": [
{
"name": "project1",
"missed": 10,
"covered": -1
}
]
}
]
}
above code prints:
{
"name" : "Head",
"children" : [ {
"name" : "project1",
"children" : [ {
"name" : "project1",
"missed" : 10,
"covered" : 11
} ]
}, {
"name" : "project1",
"covered" : 12,
"children" : [ {
"name" : "project1",
"missed" : 10
} ]
} ]
}
See also:
Flattening a 3 level nested JSON string in java
There are two main techniques to parse and generate JSON data (as well as many other formats like XML etc): object mapping and event/token/stream-oriented processing. The second way is the best way for many cases, including filtering. Props:
the file/data doesn't require to be loaded entirely into memory, you
can process megs/gigs with no problems
it works much more faster, especially for large files
it's easy to implement any custom type/rule of transformation with this pattern
Both Gson and Jackson supports stream-oriented processing. To illustrate the idea here is just an example using a tiny parser/generator https://github.com/anatolygudkov/green-jelly
import org.green.jelly.AppendableWriter;
import org.green.jelly.JsonBufferedWriter;
import org.green.jelly.JsonEventPump;
import org.green.jelly.JsonNumber;
import org.green.jelly.JsonParser;
import java.io.StringWriter;
public class UpdateMyJson {
private static final String jsonToUpdate = "{\n" +
"\"name\": \"Head\",\n" +
"\"missed\": -1,\n" +
"\"children\": [\n" +
" {\n" +
" \"name\": \"project1\",\n" +
" \"fixes\": 0,\n" +
" \"commits\": -1,\n" +
" },\n" +
" {\n" +
" \"name\": \"project2\",\n" +
" \"fixes\": 20,\n" +
" \"commits\": 5,\n" +
" }\n" +
"]\n" +
"}";
public static void main(String[] args) {
final StringWriter result = new StringWriter(); // you can use FileWriter
final JsonParser parser = new JsonParser();
parser.setListener(new MyJsonUpdater(new AppendableWriter<>(result)));
parser.parseAndEoj(jsonToUpdate); // if you read a file with a buffer,
// to don't load the whole file into memory,
// call parse() several times (part by part) in a loop until EOF
// and then call .eoj()
System.out.println(result);
}
static class MyJsonUpdater extends JsonEventPump {
MyJsonUpdater(final JsonBufferedWriter output) {
super(output);
}
#Override
public boolean onNumberValue(final JsonNumber number) {
if (number.mantissa() == -1 && number.exp() == 0) {
return true; // return immediately
}
return super.onNumberValue(number); // otherwise pump the value to the result JSON
}
}
}
I have the following JSON source:
{
"my-systems": [{
"SYSTEM_A": [{
"parameter-name": "a_param1",
"parameter-display-name": "parameter 1",
"optional": "true"
},
{
"parameter-name": "a_param2",
"parameter-display-name": "Parameter 2",
"optional": "false"
}
]
},
{
"SYSTEM_B": [{
"parameter-name": "b_param1",
"parameter-display-name": "Parameter 1",
"optional": "true"
},
{
"parameter-name": "b_param2",
"parameter-display-name": "Parameter 2",
"optional": "false"
}
]
}
]
}
I try to read it into a map of Map<String, SystemParameter[]>.
I have this code which I'm really not sure if it's the best approach for my goal.
ArrayNode systemsArr = (ArrayNode)jsonDoc.get("my-systems");
if(systemsArr!= null && !systemsArr.isEmpty()){
for(JsonNode systemNode : systemsArr ){
ObjectNode systemObj = (ObjectNode)systemNode;
System.out.println(systemObj .textValue());
}
}
Is it a valid approach? How do I get the name of the system (SYSTEM_A, SYSTEM_B) and convert the contained parameters into a parameters objects array?
You just need to use jackson-databind and jackson-annotations jar in your dependency and you should be able to run below code
import com.fasterxml.jackson.annotation.JsonProperty;
import com.fasterxml.jackson.databind.ObjectMapper;
import java.io.File;
import java.io.IOException;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
public class JsonParsing {
public static void main(String[] args) throws IOException {
String jsonFilePath = "src/main/resources/source.json"; // JSON File Path
MySystems mySystems = new ObjectMapper().readValue(new File(jsonFilePath),MySystems.class);
Map<String,SystemParameter[]> outputMap = new HashMap<>();
for (Map<String,List<SystemParameter>> map :mySystems.mySystems) {
for (String key :map.keySet()) {
outputMap.put(key, map.get(key).toArray(new SystemParameter[0]));
}
}
System.out.println(outputMap);
}
}
class MySystems {
#JsonProperty("my-systems")
List<Map<String,List<SystemParameter>>> mySystems;
}
class SystemParameter {
#JsonProperty("parameter-name")
String paramName;
#JsonProperty("parameter-display-name")
String paramDispName;
#JsonProperty("optional")
String optional;
}
Using an array 'my-systems' is of little use if all you're doing is keeping your keys unique. (I am assuming your SYSTEM_A may be different).
Instead, I suggest you format the JSON data in the following way:
{
"my-systems": [
{
"system-name" : {
"name":"System_A",
"parameters": [
{
"parameter-name": "a_param1",
"parameter-display-name": "parameter 1",
"optional": "true"
},
{
"parameter-name": "a_param2",
"parameter-display-name": "Parameter 2",
"optional": "false"
}
]
}
}
]
}
This method is clean, and allows you to catch the name of the system under the property 'system-name' and it's parameters are nested inside. You can simply declare a model and Jackson (or gson) will simply take care of everything.
If you'd rather parse the response separately or you have no control over the response, you can opt to go with your implementation but you do not need to convert to ObjectNode. You can use the samples below inside your for loop:
String f2Str = jsonNode.get("f2").asText();
double f2Dbl = jsonNode.get("f2").asDouble();
int f2Int = jsonNode.get("f2").asInt();
long f2Lng = jsonNode.get("f2").asLong();
where 'f2' is your key, and you can get the key using node.fieldNames().next() which is actually getting the property from an iterator.
You may also try the ones below, they seem better to handle.
JsonNode parent = ...;
for (Iterator<String> it = parent.fieldNames() ; it.hasNext() ; ) {
String field = it.next();
System.out.println(field + " => " + parent.get(field));
}
for (Iterator<Map.Entry<String,JsonNode>> it = parent.fields() ;
it.hasNext() ; ) {
Map.Entry<String,JsonNode> e = it.next();
System.out.println(e.getKey() + " => " + e.getValue());
}
The latest examples are taken from here. Hope this helps.
I need to parse the following JSON and add values from it into three different Java objects. I was thinking to form other 3 Jsons in order to do this. I have some issues with parsing, as the JSON is a little bit complicated. The JSON is below:
{
"totalCount": 1,
"results": [
{
"teleCommunications": [
{
"areaCode": "100",
"telephoneNumber": "300-2444",
"internationalAreaCode": "",
"communicationType": 1
},
{
"areaCode": "100",
"telephoneNumber": "200-2555",
"internationalAreaCode": "",
"communicationType": 5
}
],
"delegate": {
"id": 0,
"range": 0,
},
"name": "Andrew",
"composedKey": {
"id": 615,
"range": 50,
},
"isBranchAddress": false,
"emailAddresses": [
{
"emailAddressType": 9,
"emailAddress": "andrew.brown#gmail.com"
}
],
"name": "Brown",
"zipCodeCity": "65760 Leipzig",
"salutation": "Mr.",
"openingDate": "2019-09-20",
"streetHouseNumber": "Offenbach. 37",
"modificationTimestamp": "2018-01-27"
}
]
}
I need to get separately the values from name, zipCodeCity, salutation, openingDate, streetHouseNumber in a JSON ( or any other way) , emailAddresses in a different JSON and the other data in another one.
I tried with this piece of code:
ObjectMapper mapper = new ObjectMapper();
mapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
HashMap<String, Object> result = new ObjectMapper().readValue(response, HashMap.class);
Object object = result.get("results");
List<HashMap<String, String>> jsonValues = (List<HashMap<String, String>>)object;
for(String key : jsonValues.get(0).keySet()){
System.out.println("name value " + jsonValues.get(0).get("name"));
System.out.println("zip code City " + jsonValues.get(0).get("zipCodeCity"));
}
The problem is that I would not go for such a hardcoded way...it is not very suitable.
Does anyone know a better approach for storing the values I need and parsing the Json more optimally?
Thank you
Look for this link, there's a demo application specially for you: https://github.com/nalmelune/jackson-demo-58591850 (look at JacksonTest)
First of all, you'll need data-classes for this (or so called dto), representing structure. You can use online generators for that (google "json to java dto online") or do it yourself. For example, here's root object (see github link for more):
public class Root {
private int totalCount;
private List<Results> results;
public void setTotalCount(int totalCount) {
this.totalCount = totalCount;
}
public int getTotalCount() {
return this.totalCount;
}
public void setResults(List<Results> results) {
this.results = results;
}
public List<Results> getResults() {
return this.results;
}
}
Then configure your ObjectMapper:
ObjectMapper mapper = new ObjectMapper();
mapper.registerModule(new JavaTimeModule());
mapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
And use it (don't create new mapper, as it is in your example):
Root result = mapper.readValue(response, Root.class);
Finally, you can access it as usual Plain Old Java Objects:
for (Results resultItem : result.getResults()) {
System.out.println(resultItem.getSalutation());
System.out.println(resultItem.getName());
System.out.println(resultItem.getZipCodeCity());
System.out.println(resultItem.getOpeningDate());
System.out.println(resultItem.getStreetHouseNumber());
}
To make it work be sure validate your json (there were invalid commas after "range", and "name" comes twice and so on, it fixed on github). And be sure to include jsr310 in classpath by adding com.fasterxml.jackson.datatype:jackson-datatype-jsr310 module. You will need it for LocalDate and LocalDateTime objects.
You may create java class something like below
#Data
class MyCustomClass{
int totalCount;
List<TeleCommunicationDetail> results;
// other props
}
Next you can add attributes to TeleCommunicationDetail and finally you can use MyCustomClass reaonseInJavaObject = objectMapper.readValue(myJson,MyCustomClass.class);
Refer to this question for details.
Your approach is correct, as you simply read your JSON object as Map<String, Object> which always will work (as long as JSON Object is valid). And then you retrieve your values that you expect to be there. Another approach is to create a Class that maps to your original JSON and parse JSON with ObjectMapper into your class. Then you can retrieve your data with your class setters and getters. In this case, you don't need to use key Strings like "results" etc but you have to ensure that your JSON is not only valid JSON but always conforms to your class. Pick your option...
I have a JSON string, where consist of structure something like below, basically I have a chain of VO's and each VO will have list of VO's and Map of VO's as well.
sample:
"MessageHeader": {
"type": "new",
"id": 10,
"add_record": "true",
},
"MessageBody": {
"dob_required": false,
"dept_name": "Purchase",
"group": {
"main_group": "main group",
"sub_group": "sub group"
},
"family_info": {
"person1": {
"name": "HisName",
"gender": "male",
"age": "31",
"office_info_present": {
"office_name": "present office",
"office_address": "some address"
"some_other_vo":{
},
},
"office_info_previous": {
"office_name": "old office",
"office_address": "some address"
},
},
"person2": {
"name": "HerName",
"gender": "Female",
"age": "25",
}
},
"active":true
}
I want to print each and every attribute name and its value in key value pair. I don't want to print the VO name, rather just the end attributes (leaf level attributes)
like below string:
type=new id=10 add_record=true dob_required=false dept_name=purchase main_group="main group" sub_group="sub group" name=HisName gender=male age=31 .... name=HerName gender=Female age=25
How this can be done? I would prefer if it can be done using jackson. Otherwise also fine.
The easiest way is probably by using Jackson's Tree Model, traversing it. Something like:
JsonNode root = mapper.readValueAsTree(jsonInput);
Map<String,String> flat = new HashMap<String,String>();
traverse(root, flat);
public void traverse(JsonNode node, Map<String,String> result)
{
Iterator<Map.Entry<String,JsonNode>> it = node.fields();
while (it.hasNext()) {
Map.Entry<String,JsonNode> entry = it.next();
JsonNode n = entry.getValue();
if (n.isObject()) { // if JSON object, traverse recursively
traverse(n);
} else { // if not, just add as String
result.put(entry.getKey(), n.asText());
}
}
}
and you'll get a Map that you can print.
you can use this link for creating POGO class for your response. It will automatically generate class for your response.
Use google GSON library to parse your response.
Yes jackson can be used to parse. but its not straight forward.
There are few implementations similar to JAXB for XML. But I never tested them
Here is the rough code snippet.
JSONParser parser = new JSONParser();
try {
JSONObject jsonObject = (JSONObject) parser.parse(new FileReader("c:\\sample.json"));
String name = (String) jsonObject.get("name");
long age = (Long) jsonObject.get("age");
JSONArray msg = (JSONArray) jsonObject.get("messages");
System.out.println("Name:"+name);
System.out.println("Age:"+age);
System.out.println("Messages:");
Iterator<String> iterator = msg.iterator();
while (iterator.hasNext()) {
System.out.println(iterator.next());
}
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} catch (ParseException e) {
e.printStackTrace();
}