Develop Reference

Parse json-ld generated by Apache Any23 into Java Pojo using Jackson

I want to map structured data (microdata, jsonld) extracted from html text into a Java POJO. For extraction I use the library Apache Any23 and configured a JSONLDWriter to convert the structured data found in the html document into json-ld format.
This works as expected an gives me the following output:
[ {
"#graph" : [ {
"#id" : "_:node1gn1v4pudx1",
"#type" : [ "http://schema.org/JobPosting" ],
"http://schema.org/datePosted" : [ {
"#language" : "en-us",
"#value" : "Wed Jan 11 02:00:00 UTC 2023"
} ],
"http://schema.org/description" : [ {
"#language" : "en-us",
"#value" : "Comprehensive Job Description"
} ],
"http://schema.org/hiringOrganization" : [ {
"#language" : "en-us",
"#value" : "Org AG"
} ],
"http://schema.org/jobLocation" : [ {
"#id" : "_:node1gn1v4pudx2"
} ],
"http://schema.org/title" : [ {
"#language" : "en-us",
"#value" : "Recruiter (m/f/d)\n "
} ]
}, {
"#id" : "_:node1gn1v4pudx2",
"#type" : [ "http://schema.org/Place" ],
"http://schema.org/address" : [ {
"#id" : "_:node1gn1v4pudx3"
} ]
}, {
"#id" : "_:node1gn1v4pudx3",
"#type" : [ "http://schema.org/PostalAddress" ],
"http://schema.org/addressCountry" : [ {
"#language" : "en-us",
"#value" : "Company Country"
} ],
"http://schema.org/addressLocality" : [ {
"#language" : "en-us",
"#value" : "Company City"
} ],
"http://schema.org/addressRegion" : [ {
"#language" : "en-us",
"#value" : "Company Region"
} ]
}, {
"#id" : "https://career.company.com/job/Recruiter/",
"http://www.w3.org/1999/xhtml/microdata#item" : [ {
"#id" : "_:node1gn1v4pudx1"
} ]
} ],
"#id" : "https://career.company.com/job/Recruiter/"
} ]
Next I want to deserialize the json-ld object into a Java bean using jackson. The POJO class should look somthing like this:
public class JobPosting {
private String datePosting;
private String hiringOrganization;
private String title;
private String description;
// Following members could be enclosed in a class too if easier
// Like class Place{private PostalAddress postalAddress;}
// private Place place;
private String addressCountry;
private String addressLocality;
private String addressRegion;
}
I would like to do it with annotations provided by Jackson lib but I struggle with a few things:
The #type value wrapped with an array node
The actual data has an extra #value layer
And some objects only hold a reference to other objects in the graph via #id fields
How can I map these fields to my Java Pojo properly?

The trick is to process the json-ld with a json-ld processor to get a more developer friendly json. The titanium-json-ld library provides such processors.
JsonDocument input = JsonDocument.of(jsonLdAsInputStream);
JsonObject frame = JsonLd.frame(input, URI.create("http://schema.org")).get();
The above code snippet resolves references via #id and resolves json keys with the given IRI.
That leads to the following output which is easy to parse via Jackson lib:
[{
"id": "_:b0",
"type": "JobPosting",
"datePosted": {
"#language": "en-us",
"#value": "Wed Jan 11 02:00:00 UTC 2023"
},
"description": {
"#language": "en-us",
"#value": "Comprehensive Job Description"
},
"hiringOrganization": {
"#language": "en-us",
"#value": "Org AG"
},
"jobLocation": {
"id": "_:b1",
"type": "Place",
"address": {
"id": "_:b2",
"type": "PostalAddress",
"addressCountry": {
"#language": "en-us",
"#value": "Company Country"
},
"addressLocality": {
"#language": "en-us",
"#value": "Company City"
},
"addressRegion": {
"#language": "en-us",
"#value": "Company Region"
}
}
},
"title": {
"#language": "en-us",
"#value": "Recruiter (m/f/d)\n "
}
}]

Looking the elements you are interested in the json (for example the "datePosted", "hiringOrganization" values) they are always labelled by "#value" and included in the array corresponding to their names (in this case "http://schema.org/datePosted" and "http://schema.org/hiringOrganization". These are all contained in a part of your json file that can be converted to a JsonNode node that can be obtained in the following way:
JsonNode root = mapper.readTree(json)
.get(0)
.get("#graph")
.get(0);
So if you have a pojo like below:
#Data
public class JobPosting {
private String datePosted;
private String hiringOrganization;
}
and you want to retrieve the datePosted and hiringOrganization values you can check that the relative position is still the same in the json file and can be calculated in a for loop:
JsonNode root = mapper.readTree(json)
.get(0)
.get("#graph")
.get(0);
String strSchema = "http://schema.org/";
String[] fieldNames = {"datePosted", "hiringOrganization"};
//creating a Map<String, String> that will be converted to the JobPosting obj
Map<String, String> map = new HashMap<>();
for (String fieldName: fieldNames) {
map.put(fieldName,
root.get(strSchema + fieldName)
.get(0)
.get("#value")
.asText()
);
}
JobPosting jobPosting = mapper.convertValue(map, JobPosting.class);
//it prints JobPosting(datePosted=Wed Jan 11 02:00:00 UTC 2023, hiringOrganization=Org AG)
System.out.println(jobPosting);

This would require some preprocessing first to turn your graph with id pointers into a simplified tree before mapping it with Jackson:
Turn it into a tree by replacing the #id references with the actual objects themselves.
Flatten those troublesome object/array wrappers around #value.
Full code below, using Java 17 and a bit of recursion:
package org.example;
import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
import com.fasterxml.jackson.annotation.JsonProperty;
import com.fasterxml.jackson.annotation.JsonSubTypes;
import com.fasterxml.jackson.annotation.JsonTypeInfo;
import com.fasterxml.jackson.core.type.TypeReference;
import com.fasterxml.jackson.databind.ObjectMapper;
import java.io.File;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Map.Entry;
import static java.util.stream.Collectors.toMap;
class Main {
public static void main(String[] args) throws Exception {
var mapper = new ObjectMapper();
var node = mapper.readValue(new File("test.json"), Object.class);
// Build a lookup map of "#id" to the actual object.
var lookup = buildLookup(node, new HashMap<>());
// Replace "#id" references with the actual objects themselves instead
var referenced = lookupReferences(node, lookup);
// Flattens single object array containing "#value" to be just the "#value" themselves
var flattened = flatten(referenced);
// Jackson should be able to under our objects at this point, so convert it
var jobPostings =
mapper.convertValue(flattened, new TypeReference<List<RootObject>>() {}).stream()
.flatMap(it -> it.graph().stream())
.filter(it -> it instanceof JobPosting)
.map(it -> (JobPosting) it)
.toList();
System.out.println(jobPostings);
}
private static Map<String, Object> buildLookup(Object node, Map<String, Object> lookup) {
if (node instanceof List<?> list) {
for (var value : list) {
buildLookup(value, lookup);
}
} else if (node instanceof Map<?, ?> map) {
for (var value : map.values()) {
buildLookup(value, lookup);
}
if (map.size() > 1 && map.get("#id") instanceof String id) {
lookup.put(id, node);
}
}
return lookup;
}
private static Object lookupReferences(Object node, Map<String, Object> lookup) {
if (node instanceof List<?> list
&& list.size() == 1
&& list.get(0) instanceof Map<?, ?> map
&& map.size() == 1
&& map.get("#id") instanceof String id) {
return lookupReferences(lookup.get(id), lookup);
}
if (node instanceof List<?> list) {
return list.stream().map(value -> lookupReferences(value, lookup)).toList();
}
if (node instanceof Map<?, ?> map) {
return map.entrySet().stream()
.map(entry -> Map.entry(entry.getKey(), lookupReferences(entry.getValue(), lookup)))
.collect(toMap(Entry::getKey, Entry::getValue));
}
return node;
}
private static Object flatten(Object node) {
if (node instanceof List<?> list && list.size() == 1) {
if (list.get(0) instanceof String s) {
return s;
}
if (list.get(0) instanceof Map<?, ?> map) {
var value = map.get("#value");
if (value != null) {
return value;
}
}
}
if (node instanceof List<?> list) {
return list.stream().map(Main::flatten).toList();
}
if (node instanceof Map<?, ?> map) {
return map.entrySet().stream()
.map(entry -> Map.entry(entry.getKey(), flatten(entry.getValue())))
.collect(toMap(Entry::getKey, Entry::getValue));
}
return node;
}
}
#JsonIgnoreProperties(ignoreUnknown = true)
record RootObject(#JsonProperty("#graph") List<GraphObject> graph) {}
#JsonTypeInfo(use = JsonTypeInfo.Id.NAME, property = "#type", defaultImpl = Ignored.class)
#JsonSubTypes({
#JsonSubTypes.Type(value = JobPosting.class, name = "http://schema.org/JobPosting"),
#JsonSubTypes.Type(value = Place.class, name = "http://schema.org/Place"),
#JsonSubTypes.Type(value = PostalAddress.class, name = "http://schema.org/PostalAddress"),
})
interface GraphObject {}
#JsonIgnoreProperties(ignoreUnknown = true)
record Ignored() implements GraphObject {}
#JsonIgnoreProperties(ignoreUnknown = true)
record JobPosting(
#JsonProperty("http://schema.org/title") String title,
#JsonProperty("http://schema.org/description") String description,
#JsonProperty("http://schema.org/hiringOrganization") String hiringOrganization,
#JsonProperty("http://schema.org/datePosted") String datePosted,
#JsonProperty("http://schema.org/jobLocation") Place jobLocation)
implements GraphObject {}
#JsonIgnoreProperties(ignoreUnknown = true)
record Place(#JsonProperty("http://schema.org/address") PostalAddress address)
implements GraphObject {}
#JsonIgnoreProperties(ignoreUnknown = true)
record PostalAddress(
#JsonProperty("http://schema.org/addressLocality") String locality,
#JsonProperty("http://schema.org/addressRegion") String region,
#JsonProperty("http://schema.org/addressCountry") String country)
implements GraphObject {}

How have Jackson use toString method if object have no public accessors?

I am looking for a solution that have Jackson use toString method whenever it can not serialize an object type.
Let me explain more detail.
I have a class:
#AllArgsConstructor
public class TestJackson {
public String RequestId;
public AntiSerialize foo;
#JsonFormat(pattern = "yyyy-MM-dd HH:mm:ss")
public LocalDateTime dateRequest;
public Map<String, Object> headers;
private static class AntiSerialize {
#Override
public String toString() {
return "AntiSerialize " + ZonedDateTime.now().toEpochSecond();
}
}
public static TestJackson createSample() {
return new TestJackson(
"123",
new TestJackson.AntiSerialize(),
LocalDateTime.now(),
Map.of("content", 999,
"b3", new TestJackson.AntiSerialize(),
"b4", Arrays.asList(
new TestJackson.AntiSerialize(),
new TestJackson.AntiSerialize()
)
)
);
}
}
This is to test serialize object
var OM = new ObjectMapper()
.configure(SerializationFeature.FAIL_ON_EMPTY_BEANS, false);
try {
//var f = TestJackson.createSample().foo;
//out.println(((Object)f).toString());
var json = OM.writerWithDefaultPrettyPrinter()
.writeValueAsString(TestJackson.createSample());
out.println(json);
} catch (Exception e) {
e.printStackTrace(out);
}
It prints
{
"RequestId" : "123",
"foo" : { },
"dateRequest" : "2022-08-04 09:04:14",
"headers" : {
"b3" : { },
"b4" : [ { }, { } ],
"content" : 999
}
}
But I expect:
{
"RequestId" : "123",
"foo" : "AntiSerialize 1659578741",
"dateRequest" : "2022-08-04 09:04:14",
"headers" : {
"b3" : "AntiSerialize 1659578752",
"b4" : [ "AntiSerialize 1659578763", "AntiSerialize 1659578774" ],
"content" : 999
}
}
Assume TestJackson is third party class and can't modify.
AntiSerialize is also a private nested class.

How to parse "[ "a", [ "adidas", "acb", "amazon" ] ] to object java?

I can't converter this response api "[ "a", [ "adidas", "acb", "amazon" ] ] to object java or kotlin

The format of your JSON data is not very suitable to convert to a java object directly，try to make your JSON data like this:
{"a": ["adidas", "acb", "amazon"] }
or
{"keyA" : "a", "keyB" : ["adidas", "acb", "amazon"] }
But it's still depends on what is your actual demand
You can test your JSON data here:
JSON to JAVA Converter live

May be this example can help you.
public class Test {
public static void main(String... args) throws Exception {
String json = "[ "a", [ "adidas", "acb", "amazon" ] ];
// Now do the magic.
Data data = new json().fromJson(json, Data.class);
// Show it.
System.out.println(data);
}
}
class Data {
private String data;
private List<String> groups;
public String getData() { return data; }
public void setData(String data) { this.data= data; }
public List<String> getGroups() { return groups; }
public void setGroups(List<String> groups) { this.groups = groups; }
}

How do I get nested filtered list with map and filter?

I'm pretty newbie in Java and learning Java8 with stream.
And I want to get list filtered.
I need filtered list with nestedlist is "22".
But I am struggling make this. Is anybody save me?
I put some simple code below.
I tried this like below but It is now working.
List<Custom> list = new ArrayList<>();
list.stream().map(
x -> x.getNestedList.stream().filter(
y->y.getValue.equals("22")
)
)
Before
[
{
"a":"1",
"nestedlist": [ {"11"},
{"22"},
{"33"} ]
},
{
"a":"2",
"nestedlist": [ {"22"},
{"44"} ]
},
{
"a":"3",
"nestedlist": [ {"11"},
{"33"} ]
},
{
"a":"4",
"nestedlist": [ {"b":"11"} ]
}
]
After
(Nestedlist has filtered by "22", but parent list still has whole element)
[
{
"a":"1",
"nestedlist": [ {"22"}]
},
{
"a":"2",
"nestedlist": [ {"22"} ]
},
{
"a":"3",
"nestedlist": [ ]
},
{
"a":"4",
"nestedlist": [ ]
}
]

I have something that should be close to a solution but without the definition of nestedList, it's only a draft. You will have to adapt it to your code
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.stream.Collectors;
class Scratch {
static class Custom<T>{
String a;
List<T> nestedList;
public Custom(String a, List<T> nestedList) {
this.a = a;
this.nestedList = nestedList;
}
public Custom(String a, T... nestedList) {
this.a = a;
this.nestedList = Arrays.asList(nestedList);
}
public String getA() {
return a;
}
public void setA(String a) {
this.a = a;
}
public List<T> getNestedList() {
return nestedList;
}
public void setNestedList(List<T> nestedList) {
this.nestedList = nestedList;
}
#Override
public String toString() {
return "Custom{" +
"a='" + a + '\'' +
", nestedList=" + nestedList +
'}' +
'\n';
}
}
public static void main(String... args){
List<Custom> list = new ArrayList<>();
list.add(new Custom<String>("1", "11", "22", "33"));
list.add(new Custom<String>("2", "22", "44"));
list.add(new Custom<String>("3", "11", "33"));
List result = list.stream().map(
x -> new Custom(x.getA(), x.getNestedList().stream().filter(y -> y.equals("22")).collect(Collectors.toList()))
).collect(Collectors.toList());
System.out.println("input: "+list);
System.out.println("result: "+result);
}
}
Which gives as output
input: [Custom{a='1', nestedList=[11, 22, 33]}
, Custom{a='2', nestedList=[22, 44]}
, Custom{a='3', nestedList=[11, 33]}
]
result: [Custom{a='1', nestedList=[[22]]}
, Custom{a='2', nestedList=[[22]]}
, Custom{a='3', nestedList=[[]]}
]
The main issue with your code (if I understood what you were trying to do correctly, which I'm absolutely not sure) is that you are mapping the initial Custom object to a Stream of the content of the nestedList. What you have to do, is map a Custom to a new updated Custom (what I did) or update the existing Custom (and so the existing list) for which you need a forEach() or a basic loop instead of the map()

how to parse json array value using json node

I have a JsonNode which contains the following JSON. Inside that JsonNode object is an array. In that array there are three fields, one of which, slaid, is a list. The other two are strings. Here is the JSON.
{
"SLA": [
{
"slaid": [
"53637cc144ae8b607e089701"
],
"ragindicator": "Red",
"name": "r1"
},
{
"slaid": [
"53637d1844ae8b607e089704"
],
"ragindicator": "Amber",
"name": "a1"
},
{
"slaid": [
"53637eac44ae8b607e089706"
],
"ragindicator": "Green",
"name": "g1"
}
]
}
I want to parse this value. How can I parse it , where slaid's type is List<String>? I have tried some ways but I am still unable to find the solution.

The easiest way I can see is creating POJO classes which fit to your JSON:
class Slaids {
#JsonProperty("SLA")
private List<Slaid> slaids;
public List<Slaid> getSlaids() {
return slaids;
}
public void setSlaids(List<Slaid> slaids) {
this.slaids = slaids;
}
#Override
public String toString() {
return slaids.toString();
}
}
class Slaid {
private List<String> slaid;
private String ragindicator;
private String name;
public List<String> getSlaid() {
return slaid;
}
public void setSlaid(List<String> slaid) {
this.slaid = slaid;
}
public String getRagindicator() {
return ragindicator;
}
public void setRagindicator(String ragindicator) {
this.ragindicator = ragindicator;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
#Override
public String toString() {
return "Slaid [slaid=" + slaid + ", ragindicator=" + ragindicator + ", name=" + name + "]";
}
}
Simple usage:
ObjectMapper mapper = new ObjectMapper();
System.out.println(mapper.readValue(json, Slaids.class));
Above program prints:
[Slaid [slaid=[53637cc144ae8b607e089701], ragindicator=Red, name=r1], Slaid [slaid=[53637d1844ae8b607e089704], ragindicator=Amber, name=a1], Slaid [slaid=[53637eac44ae8b607e089706], ragindicator=Green, name=g1]]
If you want to use JsonNode you can do it in this way:
ObjectMapper mapper = new ObjectMapper();
JsonNode rootNode = mapper.readTree(json);
ArrayNode slaidsNode = (ArrayNode) rootNode.get("SLA");
Iterator<JsonNode> slaidsIterator = slaidsNode.elements();
while (slaidsIterator.hasNext()) {
JsonNode slaidNode = slaidsIterator.next();
System.out.println(slaidNode.get("slaid"));
System.out.println(slaidNode.get("ragindicator"));
System.out.println(slaidNode.get("name"));
}
Above program prints:
["53637cc144ae8b607e089701"]
"Red"
"r1"
["53637d1844ae8b607e089704"]
"Amber"
"a1"
["53637eac44ae8b607e089706"]
"Green"
"g1"