Develop Reference

Parsing unnamed nested arrays with minimal-json?

So I'm working on a fairly simple Java program which grabs market data from cryptocurrency exchanges and displays information to the user. I am using the minimal-json library.
Here is my current code:
public class Market {
static JsonArray arrayBittrex;
public static void startTimer(){
Timer timer = new Timer();
timer.scheduleAtFixedRate(new TimerTask() {
#Override
public void run() {
String url = "https://bittrex.com/api/v1.1/public/getmarketsummaries";
try {
URL url2 = new URL(url);
URLConnection con = url2.openConnection();
InputStream in = con.getInputStream();
String encoding = "UTF-8";
String body = IOUtils.toString(in, encoding);
arrayBittrex = Json.parse(body).asObject().get("result").asArray();
}
catch(MalformedURLException e) {}
catch(IOException e) {}
}
}, 0,5000);
}
public static float getPrice(String exchange, String market) {
for (JsonValue item : arrayBittrex) {
float last = item.asObject().getFloat("Last", 0);
System.out.println(last);
return last;
}
return 0;
}
}
This code works with simple json, for example (from https://bittrex.com/api/v1.1/public/getmarketsummary?market=btc-ltc):
{
"success" : true,
"message" : "",
"result" : [{
"MarketName" : "BTC-LTC",
"High" : 0.01350000,
"Low" : 0.01200000,
"Volume" : 3833.97619253,
"Last" : 0.01349998
}
]
}
It will properly return the "Last" value in the array.
However, this cant work when the json has multiple arrays (like in https://bittrex.com/api/v1.1/public/getmarketsummaries):
{
"success" : true,
"message" : "",
"result" : [{
"MarketName" : "BTC-888",
"High" : 0.00000919,
"Low" : 0.00000820,
"Volume" : 74339.61396015,
"Last" : 0.00000820
}, {
"MarketName" : "BTC-A3C",
"High" : 0.00000072,
"Low" : 0.00000001,
"Volume" : 166340678.42280999,
"Last" : 0.00000005
}
]
}
So my question is: how can I get the "Last" value by searching for the array by the "MarketName" value?

Here is a direct & null-safe way to tackle this using Java 8 library Dynamics. We're going to parse the json into a Map, read that map dynamically to what we want.
So first we can use Jackson, Gson or something to convert json -> map.
// com.fasterxml.jackson.core:jackson-databind json -> map
Map jsonMap = new ObjectMapper()
.enable(DeserializationFeature.USE_BIG_DECIMAL_FOR_FLOATS)
.readValue(jsonStringOrInputSourceEtc, Map.class);
We can now get a Dynamic instance. And, for example, grab the BTC-A3C - Last value.
Dynamic json = Dynamic.from(jsonMap);
BigDecimal a3cLast = json.get("result").children()
.filter(data -> data.get("MarketName").asString().equals("BTC-A3C"))
.findAny()
.flatMap(data -> data.get("Last").maybe().convert().intoDecimal())
.orElse(BigDecimal.ZERO);
// 5E-8
Or perhaps convert the whole lot into a map of MarketName -> Last value
Map<String, BigDecimal> marketNameLastValue = json.get("result").children()
// assume fields are always present, otherwise see #maybe() methods
.collect(toMap(
data -> data.get("MarketName").asString(),
data -> data.get("Last").convert().intoDecimal()
));
// {BTC-A3C=5E-8, BTC-888=0.00000820}
See more examples https://github.com/alexheretic/dynamics

MongoDB Java - Fetching id in nested json

I have the following json structure. I am trying to retreive run the following mongo query in java where hData._id is not null.
MongoDb Query: db.Collection.find({},{"hData._id":1, "hData.createdBy":1} )
{
"_id" : ObjectId("55567e594e3256a23565ce58"),
"hData" : {
"isDeleted" : false,
"canDelete" : false,
"canUpdate" : false,
"createdBy" : “xyz”,
"createdDate" : "2015-05-15T15:05:30",
"_id" : "7"
},
"changeDate" : "2015-02-19T16:02:12",
}
The code i have written in java to fetch the hData._id is
MongoCursor<Document> cur = col.find(new BasicDBObject("hData._id", new BasicDBObject("$ne",null)))).iterator();
try{
while(cur.hasNext()){
System.out.println(cur.next().getObjectId("hData._id"));
i++;
}
}finally {
cur.close();
}
However, hData._id is returned as null. Could you help me with this ?

You can't get nested properties using dot notation, e.g. x.y.
So in your example you need to get hData first, then call get on the _id. Like this:
MongoCursor<Document> cur = col.find(new BasicDBObject("hData._id", new BasicDBObject("$ne",null))).iterator();
while(cur.hasNext()){
System.out.println(cur.next().get("hData", Document.class).getString("_id"));
}
Also note that in your example hData._id is shown as a String and not as an ObjectId, hence in my example I've used getString().
EDIT
Since it sounds like you may have mixed types for hData._id here's a more robust example with type checking and some extra debug output to illustrate:
MongoCursor<Document> cur = col.find(new BasicDBObject("hData._id", new BasicDBObject("$ne",null))).iterator();
while(cur.hasNext()){
Document doc = cur.next();
System.out.println("Document _id" + doc.get("_id"));
Document hdata = doc.get("hData", Document.class);
Object id = hdata.get("_id");
System.out.println("hData._id " + id);
// check type if you need to
if (id instanceof String) {
System.out.println("hData._id is String: " + id);
} else if (id instanceof ObjectId) {
System.out.println("hData._id is ObjectId: " + id);
} else {
System.out.println("hData._id is of type " + id.getClass().getName());
}
}

You can use Filters and Projections helper methods.
try (MongoCursor<Document> cur = coll.find(Filters.ne("hData._id", null)).projection(Projections.include("hData._id", "hData.createdBy")).iterator()) {
while(cur.hasNext()){
Document doc = cur.next();
Document hData = doc.get("hData", Document.class);
String id = hData.getString("_id");
String createdBy = hData.getString("createdBy");
}
}

Saving to json and redis from map

I am trying to store some nested information in a redis server as well as a json at the same time.
I want the structure to look like this in redis, and then access the value(eid12345) as a key.
{
"mCat" : [eid1 : ["123", "234"], eid2 : ["1234", "234"], eid3 : ["2", "0", "1"]]
,
"fCat" : [eid1: ["986", "876"], eid3 : ["a", "hx"], eid31 : ["1"]]
}
The json obviously needs to have everything within double quotes.
{
"mCat" : ["eid1" : ["123", "234"], "eid2" : ["1234", "234"], "eid3" : ["2", "0", "1"]]
,
"fCat" : ["eid1": ["986", "876"], "eid3" : ["a", "hx"], "eid31" : ["1"]]
}
This is my code:
public static String getAllListsJSON() {
String query = "MATCH (t:MALECAT) with t match (t)<-[r:CHILD_OF]-(subtag:MALECAT) WITH t,collect(subtag) as subtags return t.eid as eid, t.level as level, REDUCE(relations = \"\", rel IN subtags| relations + \",\" + rel.eid) AS relations;";
Iterable<Map<String, Object>> itr = Neo4j.queryLagData(query, null);
ConcurrentHashMap<String, Object> mapOfMaleCatChildren = new ConcurrentHashMap<String, Object>();
for (Map map : itr) {
String tagID = (String) map.get("eid");
String[] immediateChildren = ((String) map.get("relations")).split(",");
List<String> listOfImmediateChildren = new ArrayList<>();
for (String eachChildTagEID : immediateChildren) {
if (!"".equals(eachChildTagEID)) {
listOfImmediateChildren.add(eachChildTagEID);
}
}
Collections.sort(listOfImmediateChildren);
mapOfMaleCatChildren.put(tagID, new JSONArray(listOfImmediateChildren));
}
RedisCacheManager.setWithInfiniteRedisTTL(maleCatListsKey, mapOfMaleCatChildren);
return mapOfMaleCatChildren.toString();
}
Please suggest how I can use the eids as a hash and also save the json in correct form at the same time.

Parse json with mixed child objects via gson and send back as list

I'm facing a problem related to parsing json which have mixed arrays of child classes, and I need to provide that as java list back to client.
Sample JSON:
{
"status": "OK",
"results": [
{
"type": "one",
"Id": "2170676",
"count": "456",
"title": "title",
"description": "description",
"oneMemberOne": "11",
"oneMemberTwo": "12",
}
{
"type": "two",
"Id": "2170677",
"count": "123",
"title": "title",
"description": "description",
"twoMemberOne": "21",
"twoMemberTwo": "22",
}
]
}
I created one Parent class and two child class from this:
Num : type, Id, count, title, description fields
One extends Num : oneMemberOne, oneMemberTwo
Two extends Num : twoMemberOne, twoMemberTwo
Now my question:
I have a method for asking results. Say it List<Num> getResults()
I parse the data properly like this:
List<Num> result = new ArrayList<Num>();
JsonParser parser = new JsonParser();
JsonObject jsonObject = parser.parse(lastResponsePayload).getAsJsonObject()
JsonArray results = jsonObject.get("results").getAsJsonArray();
for (JsonElement element : results) {
JsonObject trs = element.getAsJsonObject();
String type = trs.get("type").getAsString();
if (type.equals("one") {
One one = new Gson().fromJson(element, One.class);
result.add(product);
} else if (type.equals("two") {
Two two = new Gson().fromJson(element, Two.class);
result.add(metaproduct);
}
return result;
Now, on client side, after I get the list, i have to do this:
List<Num> results = getResults();
List<One> oness = new ArrayList<One>();
List<Two> twoss = new ArrayList<Two>();
for(Num n : results) {
if(n.type.equals("one")
oness.add((One) n);
else
twoss.add((Two) n);
Is this a good design for this scenario ?
User of this API has to downcast everytime based on the type field of parent class. Because webservice gives me mixed array of child classes, I have to do this. Is there any better approach to this problem ?
One another approach in my mind is to create a Result class which contains two members One and Two and provide my a List<Result> instead of List<Num>,
but then user has to check whether member is null or not and then take appropriate steps.
Thank you in advance.

I will suggest that you have another class something like this. It prevents client code from spinning through list again and parsing out records. Please note I have not run this code to test it.
public class Result {
private List<One> oness;
private List<Two> twoss;
public List<One> getOness() {
return oness;
}
public void setOness(List<One> oness) {
this.oness = oness;
}
public List<Two> getTwoss() {
return twoss;
}
public void setTwoss(List<Two> twoss) {
this.twoss = twoss;
}
}
And change
List<Num> getResults()
To
Result getResults()
also user will not have to check for nulls if you modify your parsing logic. Please see that in case we don't have results we are not returning null list but EmptyList.
Result result = new Result();
JsonParser parser = new JsonParser();
JsonObject jsonObject = parser.parse(lastResponsePayload).getAsJsonObject()
JsonArray results = jsonObject.get("results").getAsJsonArray();
List<One> oness = null;
List<Two> twoss = null;
for (JsonElement element : results) {
JsonObject trs = element.getAsJsonObject();
String type = trs.get("type").getAsString();
if (type.equals("one")) {
if(oness == null) {
oness = new ArrayList<One>();
result.setOness(oness);
}
One one = new Gson().fromJson(element, One.class);
oness.add(product);
} else if (type.equals("two")) {
if(twoss == null) {
twoss = new ArrayList<Two>();
result.setTwoss(twoss);
}
Two two = new Gson().fromJson(element, Two.class);
twoss.add(metaproduct);
}
if(oness == null) {
result.setOness(Collections.<One>EMPTY_LIST);
}
if(twoss == null) {
result.setTwoss(Collections.<Two>EMPTY_LIST);
}
return result;
Hope it helps :)

Getting attributes from JSON code with Jackson Parser

I want to parse a JSON string using Jackson JSON parser. The JSON code which I want to parse contains an array in which there is an object. From this object, I want to extract the text and retweet_count attributes:
[
{
"created_at": "Tue Jan 08 08:19:58 +0000 2013",
"id": 288560667345178600,
"text": "test tweet",
"source": "web",
"truncated": false,
"user": {
"id": 941802900,
"id_str": "941802900",
"location": ""
},
"contributors": null,
"retweet_count": 0,
"favorited": false,
"retweeted": false
}
]
I tried to do it using this code:
JsonFactory f = new JsonFactory();
JsonParser jp = f.createJsonParser(str);
boolean first = true;
while (jp.nextValue() != JsonToken.END_ARRAY) {
Tweet tweet = new Tweet();
while (jp.nextToken() != JsonToken.END_OBJECT) {
String fieldName = jp.getCurrentName();
jp.nextToken();
if (fieldName.equals("text")) {
tweet.setText(jp.getText());
} else if (fieldName.equals("retweet_count")) {
tweet.setRetweetCount(jp.getValueAsLong());
}
}
}
However, I am not getting the expected results. I think that the the problem is that inside the 'tweet' object, I have another 'user' object and when the parser encounters the } of the user object, it thinks that it is the } of the whole tweet object. Can you please tell me how can I resolve this situation?

Is there a particular reason why you try to use Streaming API instead of tree model or data-binding? Latter two could result in much simpler code. For example:
#JsonIgnoreProperties(ignoreUnknown=true) // so we only include ones we care about
public class Tweet {
String text;
int retweet_count;
}
ObjectMapper mapper = new ObjectMapper(); // reusable (please reuse, expensive to create)
Tweet tweet = mapper.readValue(json, Tweet.class);
System.out.println("Tweet with text '"+tweet.text+"', retweet count of "+tweet.retweet_count);
with data-binding. And with tree model:
ObjectNode root = mapper.readTree(json);
String text = root.path("text").getText();
// and so on

You should probably do what StaxMan suggested and model your data correctly in objects, but the fastest way I know to get the data you want is something like the code sample below.
List<Map<String, Object>> val = readValue(json, new TypeReference<List<Map<String, Object>>>() { });
for (Map<String, Object>> map : val) {
String tweet = map.get("text");
Integer retweetCount = map.get("retweet_count");
}

Here is the updated code which would work. You need to consider the user fieldName and parse it separately, so the } of user object would not be considered as the end of the root object
while (jp.nextValue() != JsonToken.END_ARRAY) {
Tweet tweet = new Tweet();
while (jp.nextToken() != JsonToken.END_OBJECT) {
String fieldName = jp.getCurrentName();
jp.nextToken();
if (fieldName.equals("user")) {
//Here the current token in BEGIN_OBJECT
while (jp.nextToken() != JsonToken.END_OBJECT) {
//You can use the user properties here
}
} else if (fieldName.equals("text")) {
tweet.setText(jp.getText());
} else if (fieldName.equals("retweet_count")) {
tweet.setRetweetCount(jp.getValueAsLong());
}
}
}