Develop Reference

Recursively flatten a map of nested objects

I have a list of map of fields from ElasticSearch in a JSON structure. I need to extract the keys from the fields into a name.value list to be used as search terms.
For example, the response I get from ElasticSearch looks like:
{
"orange": {
"type": "keyword"
},
"apple": {
"type": "keyword"
},
"banana": {
"type": "keyword"
},
"pineapple": {
"properties": {
"color": {
"type": "text"
},
"size": {
"type": "text"
}
}
},
"vegetables": {
"properties": {
"potato": {
"properties": {
"quality": {
"type": "keyword"
},
"price": {
"type": "keyword"
},
"location": {
"type": "keyword"
}
}
}
}
}
}
I need to transform this into a list of
[
"orange",
"apple",
"banana",
"pineapple.color",
"pineapple.size",
"vegetables.potato.quality",
"vegetables.potato.price",
"vegetables.potato.location",
"vegetables.cabbage"
]
I'm a bit lost as to where to start so I end up with something that will work no matter how deep the "object" + "properties" key ends up being.
edit:
I have a couple of methods I'm trying to do this with, but I keep ending up with nested loops instead
private static String process(final Map.Entry<String, Object> entry) {
final String fieldName = entry.getKey();
final Map<String, Object> value = toSourceMap(entry.getValue());
if (value.containsKey("properties")) {
final Map<String, Object> properties = toSourceMap(value.get("properties"));
process(entry); // ??
}
return fieldName;
}
And a small helper method I'm using which casts the unknown object to a map
private static Map<String, Object> toSourceMap(final Object sourceMap) {
try {
final Map<String, Object> map = (Map) sourceMap;
return map;
} catch (final Exception e) {
return Map.of();
}
}
And I'm calling this
final List<String> fieldName = new ArrayList<>();
for (final Map.Entry<String, Object> entry : properties.entrySet()) {
fieldName.add(process(entry));
}
Trying to get a list of each value from the process method
edit 2:
I can get something that works for one level deep, but this won't capture the deeper objects like vegetables.potato.quality
private static List<String> process(final Map.Entry<String, Object> entry) {
final String fieldName = entry.getKey();
final Map<String, Object> value = toSourceMap(entry.getValue());
final List<String> fields = new ArrayList<>();
if (value.containsKey("properties")) {
final Map<String, Object> properties = toSourceMap(value.get("properties"));
properties.keySet().stream().map(s -> fieldName + "." + s).forEach(fields::add);
} else {
fields.add(fieldName);
}
return fields;
}
and the caller
final List<String> fieldName = new ArrayList<>();
for (final Map.Entry<String, Object> entry : properties.entrySet()) {
fieldName.addAll(process(entry));
}

I'm sure theres a cleaner and better way to do it, but I was able to achieve what I needed by doing the following
private static List<String> process(final Map.Entry<String, Object> entry) {
final String fieldName = entry.getKey();
final List<String> fields = new ArrayList<>();
final Map<String, Object> value = toSourceMap(entry.getValue());
if (value.containsKey(MAPPING_PROPERTIES)) {
toSourceMap(value.get(MAPPING_PROPERTIES)).entrySet()
.stream()
.map(FieldMappingFactory::process)
.map(nestedFields -> nestedFields.stream().map(f -> "%s.%s".formatted(fieldName, f)).toList())
.forEach(fields::addAll);
} else {
fields.add(fieldName);
}
return fields;
}

Here's a solution based on the Depth first search tree-traversal algorithm.
Since it's iterative, you can use it to process even deeply nested massive JSON without a risk of getting a StackOverFlowError.
To implement DFS, we need a Stack and a Map is needed to store the paths associated with a node that are being explored.
As the first step, we need to read the whole JSON tree and the obtained JsonNode on the stack.
Then, until the stack is not empty, need to examine each node it contains by pulling them out from the stack. If the node happens to be an ArrayNode or ObjectNode, then all its children-nodes which could be obtained via JsonNode.fields() should be added on the stack.
String json = // the source JSON
JsonNode node = new ObjectMapper().readTree(json);
List<String> results = new ArrayList<>();
Deque<JsonNode> stack = new ArrayDeque<>();
Map<JsonNode, List<String>> pathByNode = new HashMap<>();
pathByNode.put(node, Collections.emptyList());
Set<String> keysToExclude = Set.of("type", "properties"); // add more if you need
stack.push(node);
while (!stack.isEmpty()) {
JsonNode current = stack.pop();
List<String> path = pathByNode.get(current);
if (current instanceof ArrayNode || current instanceof ObjectNode) {
for (Iterator<Map.Entry<String, JsonNode>> it = current.fields(); it.hasNext(); ) {
Map.Entry<String, JsonNode> next = it.next();
stack.push(next.getValue());
String propertyName = next.getKey();
List<String> newPath;
if (!keysToExclude.contains(propertyName)) {
newPath = new ArrayList<>(path);
newPath.add(propertyName);
results.add(String.join(".", newPath)); // list of path should be updated
} else {
newPath = path;
}
pathByNode.put(next.getValue(), newPath);
}
}
}
results.forEach(System.out::println);
Output:
orange
apple
banana
pineapple
vegetables
vegetables.potato
vegetables.potato.quality
vegetables.potato.price
vegetables.potato.location
pineapple.color
pineapple.size

This is a simpler solution.
private static void process(List<String> paths, String path, JsonNode node, Set<String> excludeKeys) {
if (node.isValueNode()) {
paths.add(path);
} else if (node.isObject()) {
node.fields().forEachRemaining(elem -> {
String key = excludeKeys.contains(elem.getKey()) ? path : (path == null ? "" : path + ".") + elem.getKey();
process(paths, key, elem.getValue(), excludeKeys);
});
} else { // This part is not required if there is no array inside the source JSON object
for (int i = 0; i < node.size(); i++) {
process(paths, String.format("%s[%d]", path, i), node.get(i), excludeKeys);
}
}
}
Caller
JsonNode node = new ObjectMapper().readTree(// JSON string //);
List<String> paths = new ArrayList<>();
process(paths, null, node, Set.of("properties", "type"));
paths.forEach(System.out::println);
Output
orange
apple
banana
pineapple.color
pineapple.size
vegetables.potato.quality
vegetables.potato.price
vegetables.potato.location

NullPointer exception while parsing JSON Map in java

I'm trying to parse productId, name, and price from my below JSON in the REST API - but I'm getting null pointer exception when I try to get he name - String name = (String) obj.get("name");, may I know what I'm doing wrong here?
Any help would be really appreciated.
Thanks!
JSON:
{
"id": "af0b86eb-046c-4400-8bc4-0e26042b8f53",
"products": [{
"productId": "1234",
"name": "apple",
"price": "383939"
}
]
}
Controller.java
public ResponseEntity<Object> create(#RequestBody Map<String, Object> obj) {
Product response = myservice.create(obj);
return new ResponseEntity<Object>(response, HttpStatus.CREATED);
}
Service.java
public Product create(Map<String, Object> obj) {
for (Entry<String, Object> entry : obj.entrySet()) {
System.out.println("Key : " + entry.getKey() + " Value : " + entry.getValue());
}
Object [] products = (Object[]) obj.get("products");
HashMap <String, Object> productOne = (HashMap) products[0];
String productId = (String) productOne.get("name");
System.out.println("productId: " +productId);
String name = (String) obj.get("name");
System.out.println("name: " +name);
}
Output:
Key : products Value : [{productId=1234, name=apple, price=383939}]

By obj.get("name"), you try getting value by key name from a top-level of your json request, while it's a property of a second-level object
In pseudo-code (skipping casting and null-checks) it should look like this:
obj.get("products")[0].get("name")

In Service.java you print obj inside for loop. Your print says that key is products and value is an array of objects.
So in following line obj is that top-level object and not contains “name” field.
String name = (String) obj.get("name");
System.out.println("name: " +name);
What if you call obj.get(“products”) and try to cast it into a collection? Then trying to fetch the first index from the collection. It should contain the inner object which contains name key and value.

You should reason exactly as you see the json, by depth(or level).
Initially the whole json is contained in the obj map which contains two key-value pairs:
(key = "id", value = "af0b86eb-046c-4400-8bc4-0e26042b8f53")
(key = "products", value = [{"productId": "1234", "name": "apple", "price": "383939"}])
Since you are interested in product details, the first step is two extract the array products like this:
Object [] products = obj.get("products");
Now product is an array of objects. Since you know that your objects are in turn hash maps you can cast each object to a map and access the key(s) that you want:
HashMap <String, Object> productOne = (HashMap) products[0];
String productId = (String) productOne.get("productId");
String name = (String) productOne.get("name");
..

Deserializing jackson dynamic key value

I have a json structure similar to this
{
"Byc-yes": { // < code
"Updated": "week", // < period
"Time": "12pm" // < time
},
"Uop-thp":{
"Updated": "week",
"Time": "12pm
} ,
...
I want to deserialize it to a Java class
class Updating {
private String code;
private String period;
private String time;
}
There any native JACKSON mappers to do this, or will I need to create my own custom deserializer for this?

I will read it as Map.class and then iterate through keyset to extract values.
ObjectMapper objectMapper = new ObjectMapper();
Map map = objectMapper.readValue(s, Map.class);
for (Object o : map.keySet()) {
String key = (String) o;
System.out.println(key);//prints Byc-yes for first
Map<String, String> value = (Map<String, String>) map.get(key);
System.out.println(value); //prints {Updated=week, Time=12pm} for first
Updating updating = new Updating(key, value.get("Updated"), value.get("Time"));
System.out.println(updating);
}
Assuming Updated and Time are fixed keys.

Is there any simple way to extract the key-value pairs from a JSON string for a given key using jackson libraries? (Java)

I have this json response:
{
"Name": "tardis",
"Driver": "custom",
"Mountpoint": "/var/lib/docker/volumes/tardis/_data",
"Status": {
"hello": "world"
},
"Labels": {
"com.example.some-label": "some-value",
"com.example.some-other-label": "some-other-value"
},
"Scope": "local"
}
From this response, I want to parse the key-value pairs inside "Labels" key. Currently my method to extract those looks like this:
//"json" is the JSON response as a string
ObjectMapper mapper = new ObjectMapper();
JsonNode root = mapper.readTree(json);
Iterator<Entry<String, JsonNode>> it = root.fields();
while (it.hasNext())
{
Map.Entry<String, JsonNode> entry = it.next();
JsonNode n = entry.getValue();
String nlabel = entry.getKey();
if (nlabel != null && nlabel.equals(key))
{
Iterator<Entry<String, JsonNode>> itr = n.fields();
while (itr.hasNext())
{
Map.Entry<String, JsonNode> it2 = itr.next();
String labelKey = it2.getKey();
String labelValue = it2.getValue().toString();
System.out.println();
}
}
}
I am able to retrieve those key value pairs using this logic, but I want to make it simpler because I want to make it work when there is a deeper nesting involved.
Is there a simpler, better way to do it using jackson libs?

Getting attributes from JSON code with Jackson Parser

I want to parse a JSON string using Jackson JSON parser. The JSON code which I want to parse contains an array in which there is an object. From this object, I want to extract the text and retweet_count attributes:
[
{
"created_at": "Tue Jan 08 08:19:58 +0000 2013",
"id": 288560667345178600,
"text": "test tweet",
"source": "web",
"truncated": false,
"user": {
"id": 941802900,
"id_str": "941802900",
"location": ""
},
"contributors": null,
"retweet_count": 0,
"favorited": false,
"retweeted": false
}
]
I tried to do it using this code:
JsonFactory f = new JsonFactory();
JsonParser jp = f.createJsonParser(str);
boolean first = true;
while (jp.nextValue() != JsonToken.END_ARRAY) {
Tweet tweet = new Tweet();
while (jp.nextToken() != JsonToken.END_OBJECT) {
String fieldName = jp.getCurrentName();
jp.nextToken();
if (fieldName.equals("text")) {
tweet.setText(jp.getText());
} else if (fieldName.equals("retweet_count")) {
tweet.setRetweetCount(jp.getValueAsLong());
}
}
}
However, I am not getting the expected results. I think that the the problem is that inside the 'tweet' object, I have another 'user' object and when the parser encounters the } of the user object, it thinks that it is the } of the whole tweet object. Can you please tell me how can I resolve this situation?

Is there a particular reason why you try to use Streaming API instead of tree model or data-binding? Latter two could result in much simpler code. For example:
#JsonIgnoreProperties(ignoreUnknown=true) // so we only include ones we care about
public class Tweet {
String text;
int retweet_count;
}
ObjectMapper mapper = new ObjectMapper(); // reusable (please reuse, expensive to create)
Tweet tweet = mapper.readValue(json, Tweet.class);
System.out.println("Tweet with text '"+tweet.text+"', retweet count of "+tweet.retweet_count);
with data-binding. And with tree model:
ObjectNode root = mapper.readTree(json);
String text = root.path("text").getText();
// and so on

You should probably do what StaxMan suggested and model your data correctly in objects, but the fastest way I know to get the data you want is something like the code sample below.
List<Map<String, Object>> val = readValue(json, new TypeReference<List<Map<String, Object>>>() { });
for (Map<String, Object>> map : val) {
String tweet = map.get("text");
Integer retweetCount = map.get("retweet_count");
}

Here is the updated code which would work. You need to consider the user fieldName and parse it separately, so the } of user object would not be considered as the end of the root object
while (jp.nextValue() != JsonToken.END_ARRAY) {
Tweet tweet = new Tweet();
while (jp.nextToken() != JsonToken.END_OBJECT) {
String fieldName = jp.getCurrentName();
jp.nextToken();
if (fieldName.equals("user")) {
//Here the current token in BEGIN_OBJECT
while (jp.nextToken() != JsonToken.END_OBJECT) {
//You can use the user properties here
}
} else if (fieldName.equals("text")) {
tweet.setText(jp.getText());
} else if (fieldName.equals("retweet_count")) {
tweet.setRetweetCount(jp.getValueAsLong());
}
}
}