I would like the guidence from you all, i'm confused in how to go on in a situation at Java + Spring Boot.
I receive from the database 2 columns of strings, the first column is a path separeted by a slash(/) like "crumbs[0]/link/path" and the second column have the value assigned to the first column, and what i'm trying to do is to create a nested JSON with this.
For example, i'm receiving from the database the following response in two columns like a said before:
COLUMN 1(PATH), COLUMN 2(VALUE)
"crumbs[0]/link/path", "/pokemon/type/pokemon?lang=en"
"crumbs[0]/link/wid", "tablePokemon",
"crumbs[0]/name", "Pokemon"
"data/records[1]/id", "Pikachu"
"data/records[1]/link/path": "/pokemon/type/eletric/pikachu",
"data/records[1]/link/wid": "tableEletric",
"data/records[1]/available": "true",
"data/records[2]/id", "Bulbasaur"
"data/records[2]/link/path": "/pokemon/type/grass/bulbasaur",
"data/records[2]/link/wid": "tableGrass",
"data/records[2]/available": "true",
With this response from database, i'm trying to get this result in Java:
"crumbs": [
{
"link": {
"path": "/pokemon/type/pokemon?lang=en",
"wid": "tablePokemon"
},
"name": "Pokemon"
}
],
"data": {
"records": [
{
"id": "Pikachu",
"link": {
"path": "/pokemon/type/eletric/pikachu",
"wid": "tableEletric"
},
"available": "true",
},
{
"id": "Bulbasaur",
"link": {
"path": "/pokemon/type/grass/bulbasaur",
"wid": "tableGrass"
},
"available": "true",
}
]
}
You guys would have any suggestions for me to achieve this objective?
Thank you all for your time, appreciate any help.
You can easily construct a JSON with com.fasterxml.jackson.core.JsonPointer!
Some details on parsing a JsonPath and constructing a Json from it is mentioned here How to add new node to Json using JsonPath?
You could made use of the code from the above reference to build your code.
Add the com.fasterxml.jackson dependencies to your pom.xml
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>2.11.0</version>
</dependency>
<dependency>
<groupId>org.glassfish</groupId>
<artifactId>javax.json</artifactId>
<version>1.1.2</version>
</dependency>
private static final ObjectMapper mapper = new ObjectMapper();
public static void main(String[] args) {
/** I'm creating a below map to hold the values you have mentioned in the above case.
While using JsonPointer I found two issues with the key mentioned here
1. Key doesnt start with a / . I'm appending a / with the key while inserting to map
2. The arrays in data eg:crumbs[0]/link/path should be represented like crumbs/0/link/path ( I haven't handled this in the code, but it doesn't make much difference in the output)
**/
Map<String, String> databaseKeyValues = new HashMap<String, String>();
databaseKeyValues.put("crumbs[0]/link/path", "/pokemon/type/pokemon?lang=en");
databaseKeyValues.put("crumbs[0]/link/wid", "tablePokemon");
databaseKeyValues.put("crumbs[0]/name", "Pokemon");
databaseKeyValues.put("data/records[1]/id", "Pikachu");
databaseKeyValues.put("data/records[1]/link/path", "/pokemon/type/eletric/pikachu");
databaseKeyValues.put("data/records[1]/link/wid", "tableEletric");
databaseKeyValues.put("data/records[1]/available", "true");
databaseKeyValues.put("data/records[2]/id", "Bulbasaur");
databaseKeyValues.put("data/records[2]/link/path", "/pokemon/type/grass/bulbasaur");
databaseKeyValues.put("data/records[2]/link/wid", "tableGrass");
databaseKeyValues.put("data/records[2]/available", "true");
ObjectNode rootNode = mapper.createObjectNode();
for(java.util.Map.Entry<String, String> e:databaseKeyValues.entrySet()) {
setJsonPointerValue(rootNode, JsonPointer.compile("/"+e.getKey()), //Adding slash to identify it as the root element, since our source data didn't have proper key!
new TextNode(e.getValue()));
}
try {
System.out.println(mapper.writerWithDefaultPrettyPrinter().writeValueAsString(rootNode));
} catch (JsonProcessingException e1) {
e1.printStackTrace();
}
}
public static void setJsonPointerValue(ObjectNode node, JsonPointer pointer, JsonNode value) {
JsonPointer parentPointer = pointer.head();
JsonNode parentNode = node.at(parentPointer);
String fieldName = pointer.last().toString().substring(1);
if (parentNode.isMissingNode() || parentNode.isNull()) {
parentNode = mapper.createObjectNode();
setJsonPointerValue(node,parentPointer, parentNode); // recursively reconstruct hierarchy
}
if (parentNode.isArray()) {
ArrayNode arrayNode = (ArrayNode) parentNode;
int index = Integer.valueOf(fieldName);
// expand array in case index is greater than array size (like JavaScript does)
for (int i = arrayNode.size(); i <= index; i++) {
arrayNode.addNull();
}
arrayNode.set(index, value);
} else if (parentNode.isObject()) {
((ObjectNode) parentNode).set(fieldName, value);
} else {
throw new IllegalArgumentException("`" + fieldName + "` can't be set for parent node `"
+ parentPointer + "` because parent is not a container but " + parentNode.getNodeType().name());
}
}
Output:
{
"data" : {
"records[1]" : {
"id" : "Pikachu",
"link" : {
"wid" : "tableEletric",
"path" : "/pokemon/type/eletric/pikachu"
},
"available" : "true"
},
"records[2]" : {
"available" : "true",
"link" : {
"wid" : "tableGrass",
"path" : "/pokemon/type/grass/bulbasaur"
},
"id" : "Bulbasaur"
}
},
"crumbs[0]" : {
"link" : {
"path" : "/pokemon/type/pokemon?lang=en",
"wid" : "tablePokemon"
},
"name" : "Pokemon"
}
}
The json arrays records[1], records[2], crumbs[0] would be sorted out once we handle the JsonPath from crumbs[0]/link/path to crumbs/0/link/path. Just some string operations would help (iterate through the values and replace '[0]' with '/0/', you could write a regex to pattern match and replace!).
You will need to parse the paths, then build some kind of tree object in memory, and finally convert the tree that you built into JSON.
Here are some tips:
Start by defining an empty root element. You can use a Map. Keys will be strings, values will be either strings, lists or maps.
For each path, split it by "/".
For each path element except the last, check if it is a list or a subtree. You can distinguish this by the presence of [n] at the end of the string.
Create all intermediate nodes for the path except for the last one. Starting from root (which is a Map), add either a List or a Map for each element if it doesn't exist yet under that name. If it already exists, check that it is what you need it to be. In case of List, append the element. In case of Map, create a sub-entry.
For the last path element, add it as a String.
Repeat this for all paths to fill your tree.
When you are finished, use a combination of recursion and StringBuiders to construct the output string. Alternatively, if you only used strings, maps and lists, you can also use a library such as Jackson to produce JSON.
Note that you don't have information about the length of the lists, so this conversion will not be reversible.
Related
I have some JSON documents. Here is the JSON structure.
{
"header":
{
"file1":0,
"file2":1,
"subfiles":{
"subfile1":"true",
"subfile2":"true",
}
},
"response":
{
"number":678,
"start":0,
"docs":[
{
"id":"d3f3d",
"code":"l876s",
"country_name":"United States",
"city":"LA"
},
{
"id":"d2f2d",
"code":"2343g",
"country_name":"UK",
"city":"London"
}
]
}
}
I want to get the value in "id" field using JsonNode. How to access to specific fields (id, city or country_name) in structure like this? I try to use:
JsonNode node = documentHandle.get();
String s = node.path("id").asText();
But I didn't get anything expect null.
Your node points to the root of the JSON document. To get to id, you have to traverse the path.
ArrayNode docs = node.path("response").path("docs");
for (JsonNode doc: docs) { // this is pseudo-code
String s = doc.path("id").asText();
}
Or use JsonPath.
Jayway's JsonPath works well for this:
JsonPath.read(jsonString, "$.response.docs[*].id");
So I'm working on a fairly simple Java program which grabs market data from cryptocurrency exchanges and displays information to the user. I am using the minimal-json library.
Here is my current code:
public class Market {
static JsonArray arrayBittrex;
public static void startTimer(){
Timer timer = new Timer();
timer.scheduleAtFixedRate(new TimerTask() {
#Override
public void run() {
String url = "https://bittrex.com/api/v1.1/public/getmarketsummaries";
try {
URL url2 = new URL(url);
URLConnection con = url2.openConnection();
InputStream in = con.getInputStream();
String encoding = "UTF-8";
String body = IOUtils.toString(in, encoding);
arrayBittrex = Json.parse(body).asObject().get("result").asArray();
}
catch(MalformedURLException e) {}
catch(IOException e) {}
}
}, 0,5000);
}
public static float getPrice(String exchange, String market) {
for (JsonValue item : arrayBittrex) {
float last = item.asObject().getFloat("Last", 0);
System.out.println(last);
return last;
}
return 0;
}
}
This code works with simple json, for example (from https://bittrex.com/api/v1.1/public/getmarketsummary?market=btc-ltc):
{
"success" : true,
"message" : "",
"result" : [{
"MarketName" : "BTC-LTC",
"High" : 0.01350000,
"Low" : 0.01200000,
"Volume" : 3833.97619253,
"Last" : 0.01349998
}
]
}
It will properly return the "Last" value in the array.
However, this cant work when the json has multiple arrays (like in https://bittrex.com/api/v1.1/public/getmarketsummaries):
{
"success" : true,
"message" : "",
"result" : [{
"MarketName" : "BTC-888",
"High" : 0.00000919,
"Low" : 0.00000820,
"Volume" : 74339.61396015,
"Last" : 0.00000820
}, {
"MarketName" : "BTC-A3C",
"High" : 0.00000072,
"Low" : 0.00000001,
"Volume" : 166340678.42280999,
"Last" : 0.00000005
}
]
}
So my question is: how can I get the "Last" value by searching for the array by the "MarketName" value?
Here is a direct & null-safe way to tackle this using Java 8 library Dynamics. We're going to parse the json into a Map, read that map dynamically to what we want.
So first we can use Jackson, Gson or something to convert json -> map.
// com.fasterxml.jackson.core:jackson-databind json -> map
Map jsonMap = new ObjectMapper()
.enable(DeserializationFeature.USE_BIG_DECIMAL_FOR_FLOATS)
.readValue(jsonStringOrInputSourceEtc, Map.class);
We can now get a Dynamic instance. And, for example, grab the BTC-A3C - Last value.
Dynamic json = Dynamic.from(jsonMap);
BigDecimal a3cLast = json.get("result").children()
.filter(data -> data.get("MarketName").asString().equals("BTC-A3C"))
.findAny()
.flatMap(data -> data.get("Last").maybe().convert().intoDecimal())
.orElse(BigDecimal.ZERO);
// 5E-8
Or perhaps convert the whole lot into a map of MarketName -> Last value
Map<String, BigDecimal> marketNameLastValue = json.get("result").children()
// assume fields are always present, otherwise see #maybe() methods
.collect(toMap(
data -> data.get("MarketName").asString(),
data -> data.get("Last").convert().intoDecimal()
));
// {BTC-A3C=5E-8, BTC-888=0.00000820}
See more examples https://github.com/alexheretic/dynamics
{
"empId":"1",
"name":"Alex",
"role":"president",
"phone":"123",
"address": {
"street":"xyz",
"city":"hyd",
"pincode":400123
}
}
I want to retrieve keys as following so that I can allow the user to choose such keys in the UI.
keys: ["empId","name","role","phone", "address", "address.street", "address.city",
"address.pincode"]
The same can be used for querying on Mongo directly. I tried using JSONObject and get keys but I am unable to fetch documents along with the path they come from.
Please let me know if there's a direct way I can use in Java or if Mongo has a way to get all keys from where they are coming
You can do that using recursion.
Here is an example:
public static void main(String[] args) throws ParseException {
String str = "{ \"empId\":\"1\", \"name\":\"Alex\", \"role\":\"president\", \"phone\":\"123\", \"address\": { \"street\":\"xyz\", \"city\":\"hyd\", \"pincode\":400123 }}";
JSONObject obj = (JSONObject) new JSONParser().parse(str);
List<String> keysList = new ArrayList<>();
recLevels(keysList, obj, "");
System.out.println(keysList);
}
public static void recLevels(List<String> keysList, JSONObject obj, String prefix) {
Set<String> keys = (Set<String>) obj.keySet();
for (String key : keys) {
keysList.add(prefix + (prefix.isEmpty() ? "" : ".") + key);
if (obj.get(key) instanceof JSONObject) {
recLevels(keysList, (JSONObject) obj.get(key), prefix + (prefix.isEmpty() ? "" : ".") + key);
}
}
}
What the recLevels method does is to go through all the keys of an object and check is any of these keys has an object as its value if (obj.get(key) instanceof JSONObject), if it does recLevels is called again for that object and the process is repeated for that object (one level down).
The important part here is the prefix variable which is used to store the previous keys on the previous levels.
You can create additional array and push all keys in that array.
Use JSON.Stringyfy function to present it as string to end user.
var keys = [],
sourceArray = [{
"empId":"1",
"name":"Alex",
"role":"president",
"phone":"123",
"address": {
"street":"xyz",
"city":"hyd",
"pincode":400123
}
}]; //Test data
$.each(sourceArray, function(k, v) {
//k is the key and v is the value (key-value pair)
keys.push(k);
});
//You have all the keys - use at your disposal
Note: I've not tested this code in absence of source-code, please validate and modify as per your actual code.
I have a json schema file which follows the custom rules I've created. This schema file is a valid json file. It follows the below pattern.
{
"name": {
"fname" : {
"displayName":"FirstName",
"dataType":"String"
}
"lname" : {
"displayName":"LastName",
"dataType":"String"
}
},
"address": {
"displayName":"Address",
"dataType":"String"
}
}
so based on the schema I need to create the below json with respective values.
{
"name": {
"FirstName": "test",
"LastName" : "test1"
},
"Address" : "someAddress"
}
So when I get the schema, what is the best way to find the config information node? That is the leaf node which has the displayName and dataType parameters. Currently I'm traversing this tree using Jackson json and finding the nodes with displayName and dataType keys. Because I cannot precisely say at which level this leaf node could be present. Is there a better way to handle this situation than traversing the whole json tree looking for the elements?
I was not sure wnat exactly is required (do you want the fname object or the value of its properties), however, JsonPath seems like a good fit here. It is the equivalent of xpath for json - search hierarchy model based on various criteria
I made a small demo to get you started. you just need to twaek the query string to suit your requirements. You can use the Jayway JsonPath Evaluator as REPL
import java.nio.file.*;
import com.jayway.jsonpath.*;
public class JsonPathDemo
{
public static void main(String[] args)
{
// query: get all json objects that have displayName and dataType properties
String jsonPathQuery = "$..*[?(#.displayName && #.dataType)]";
try {
String content = new String(Files.readAllBytes(Paths.get("C://temp/xx.json")));
Object parsedContent = Configuration.defaultConfiguration().jsonProvider().parse(content);
Object configElements = JsonPath.read(parsedContent, jsonPathQuery);
System.out.println(configElements.getClass());
System.out.println(configElements);
} catch (Exception e) {
e.printStackTrace();
}
}
}
output is
class net.minidev.json.JSONArray
[{"displayName":"Address","dataType":"String"},{"displayName":"FirstName","dataType":"String"},{"displayName":"LastName","dataType":"String"}]
EDIT: answer for question in comment:
It is possible to check for path existence (and make all sorts of other assertions) using json-path-assert:
import static com.jayway.jsonpath.matchers.JsonPathMatchers.*;
// query for specific path
String jsonSpecificPathQuery = "$..name.fname.displayName";
String content = new String(Files.readAllBytes(Paths.get("C://temp/xx.json")));
Object parsedContent = Configuration.defaultConfiguration().jsonProvider().parse(content);
System.out.println("hasJsonPath? " + hasJsonPath(jsonSpecificPathQuery).matches(parsedContent));
I have a really complicated nested json whose structure is always changing. I want to parse it in JAVA such that I can retrieve any element using the key/field name.
Option 1 - The easiest way to do this would be to convert/parse the json into a JAVA object.
I have tried everything (gson, JSONObject, Jackson...) But I cant convert the json unless I have a java class ready and since the json doesn't follow a fixed structure I cant convert it into a JAVA class. Is there any other library that can convert the json to a java object? (Without the need for a pre existing java class to convert the json into)
Option 2 - Or is there a way/library I can use, which if given a portion of the json, the program prints all the elements in the json file. Something like this...
StreetAddressLine
PrimaryTownName : value
CountryISOAlpha2Code :value
TerritoryAbbreviatedName :value
PostalCode : value
{"PrimaryAddress": [ {
"StreetAddressLine": [{"LineText": "492 Koller St"}],
"PrimaryTownName": "San Francisco",
"CountryISOAlpha2Code": "US",
"TerritoryAbbreviatedName": "CA",
"PostalCode": "94110",
"AddressUsageTenureDetail": [{"TenureTypeText": {
"#DNBCodeValue": 1129,
"$": "Rents"
}}],
"PremisesUsageDetail": [{"PremisesUsageFunctionDetail": [{"PremisesFunctionText": {
"#DNBCodeValue": 12058,
"$": "Manufacturing"
}}]}],
"CountyOfficialName": "San Francisco County",
"TerritoryOfficialName": "California",
"CountryGroupName": "North America",
"GeographicalPrecisionText": {
"#DNBCodeValue": 0,
"$": "Unknown"
},
"UndeliverableIndicator": false,
"MetropolitanStatisticalAreaUSCensusCode": ["San Francisco-Oakland-Hayward CA"],
"RegisteredAddressIndicator": false,
"ResidentialAddressIndicator": false
}]}
Thanks a lot!
Try using json-simple to parse the JSON into a bunch of nested JSONObject (which basically are maps) and JSONArray (which basically are lists) elements and extract the values yourself.
Just a node: PrimaryAddress indicates that there might be a SecondaryAddress as well, so the nesting should not change, otherwise it might be hard to determine things like which address StreetAddressLine belongs to.
In that case the better question would be: why does the structure change that often?
Ok I found a solution! I used the Jackson Parser
First map your jsonString to a JsonNode
JsonNode rootNode = mapper.readValue(jsonString, JsonNode.class);
Update the rootNode to contain the json for the required key:
rootNode = rootNode.findParent(key);
and then depending on if it is an array or a list handle it seperately:
if(rootNode.path(key).isArray()){
//key is the field in the json that you might be looking for
for (final JsonNode objNode : rootNode) {
for (Iterator<String> keyArray = objNode.getFieldNames(); keyArray.hasNext();){
fieldName = keyArray.next();
fieldValue = objNode.path(fieldName).asText();
if(fieldValue != ""){
System.out.println(fieldName + " = " + fieldValue);
}else{
arrayHandler(objNode,fieldName);
}
}
}
At each iteration check if the resulting JsonNode is an array or a list.
If it is a List handle it differently (Just iterate over the key value pairs like this)
for (Iterator<String> keyArray = rootNode.getFieldNames(); keyArray.hasNext();){
fieldName = keyArray.next();
fieldValue = rootNode.get(fieldName).asText();
System.out.println(fieldName + " = " + fieldValue);
}
After every iteration check what the next jsonNode is and call the respective handler recursively...