I have a InputStreamReader (connected to a socket), which will receive multiple JSON document data. For example, it will have
{ "name" : "foo" }
and, some times later (without connection closed), the stream will have another JSON data,
{ "name" : "bar" }
.
I want to parse it in my processing loop with json-simple or json-smart, whatever. Is there anyway to do this?
I like to have a JSON parser (input data from a stream) and if it does not received data from the stream, the parser can block for more data, and if it receives a complete JSON data (possibly with some method), it can continuously parse the next JSON data.
Apprently, I tried with json-simple, and json-smart but no success.
Any help or advice would be appreciated.
Thank you.
If I understand the question correctly then the problem is parsing a complete JSON object from a batch of JSON object (continuously coming through an input stream).
So, If it is possible to detect that a complete JSON object have been received, we can parse that object.
To detect if the received JSON object (String) is complete and valid, we can maintain a Stack of {, and } 1. For every { we receive we can push it on Stack, and for every }, we can remove an { from Stack. And if we have the Stack empty (not first time) we can conclude that there is a complete JSON object. We can then proceed to parse.
1 - Not sure about parentheses structure (is it balanced?) of JSON, didn't explore JSON formation deeply earlier :(.
Related
normally I parse a json string to json object instead of manipulating the json string directly. for example, a json string like
{"number": "1234567"}
if I have to add 000 at the end
...
{...,"number" : "1234567000",...}
....
I will use jackson either parse it as Json Object or POJO
I understand readability perspective parsing to Json object or POJO is much better, but I'm curious about the performance. In this case, if I manipulate the json string directly, I have to use regex to extract the number attribute, and add 000 at the end, which is much more expensive than parsing to Json Object if having lots of data? because string object basically creates a new string object?
EDIT:
Based on #Itai Steinherz's link I also make a benchmark in JS, and it shows json parse is better
https://jsbench.me/93jr1w6k5b/1
Since I'm not very familiar with JSON parsing/manipulation in Java, I'll compare the same operations in JavaScript (which I am more experienced in).
Comparing using a basic regex with .replace and using JSON.parse & JSON.stringify, the result are that using JSON.parse is slower by a small percentage (4.37% to be precise).
However, I don't think the perf gain is worth it, and I would always go with more readable and maintainable code (the JSON.parse approach) rather than the more performant (the .replace approach).
See the complete benchmark I used here.
I am using Json Patch library to perfrom a Patch operation using REST. Now I have the follwoing json document:
{
"id":1,
"ref":{"r1":1,"r2":2}, // header level
"child":[
{
"childId":1,
"ref":{"cc1":1,"cc2":2} // line level
},
{
"childId":2,
"ref":{"cc3":2} // line level
}
]
}
Now As per Json Patch doc we at the header level we can update the ref r1 using the following path /ref/r1 .
Now I am trying to perform operation on the line level child ref. Since child is an array I can use the path /child/0/ref/cc1. But as can be seen from the path I have to specify the index also which is 0 in the previous case.
Now for API consumers asking them to give the index of the array become difficult. So is there any way to customize json patch so that we can bypass the index requirement or what are the other ways to handle this scenario?
I'm not an expert in JSON-Patch, i've just read about it.
from what i understood, is the most important part is to let the API consumers access to your JSON without giving them index,
I think hashmap would help in this case, by getting the index of each element and generate a specific ID for it, then you can save them in the hashmap list, each index has its own ID.
a sample:
HashMap<String, String> elementIndex = new HashMap<[UUID], [elementIndex]>();
you can choose whatever DataType you want, not necessary String
In this case it doesn't matter which index number, it is all about the fixed UUID.
So the path will be in this case /child/{UUID}/ref/cc1 also when you receive the path you can access the UUID and replace it with its elementIndex, now you have the correct path which is /child/0/ref/cc1
and if you want to know how to pass a dynamic value to a JSON Object, there are multiple ways to do it,
this question will help:
How to pass dynamic value to a JSON String, -Convert the JSONObject to String before-
NOTE: It is not necessary to replace it with index, you can do it the way you like could be.
And i believe there are better answers if someone knows more about JSON-patch.
i hope that was helpful, or at least gives you an idea about how to solve it.
I have a huge json file (++500mb) consists of dynamic structure of nested json file. This json was extracted to file using 'json.dump' in python.
My problem is how can i read this huge json file with buffer method?
Since if i read all the strings in the same runtime it throws java heap error.
My thought is i want to read the json each record and then parse it, after that continue to next record, parse it, and so on. But how can i know which one is the end of one json record. Because i can't find the seperator between each json record.
Any suggestion? Please ask if something is not clear.
Thanks
Assuming that you can't simply increase the heap space size with -Xmx you can switch your JSON reading logic to use a SAX JSON parsers e.g. RapidJSON or Jackson Streaming API. Instead of storing the entire JSON body in the memory those libraries will emit an event for each encountered JSON construct:
{
"hello": "world",
"t": true
...
}
will produce below when using RapidJSON:
StartObject()
Key("hello", 5, true)
String("world", 5, true)
Key("t", 1, true)
Bool(true)
...
EndObject()
I want to work with Open Street Map (OSM). OSM keeps its data formats as flexible as possible by using key value pairs. I am developing an application for Android and I am going to send it a JSON string of OSM data. What should I do if I do not know what the JSON will look like in advance? What would be the best library?
Thanks for your help,
Chris
This may be what you are looking for
http://code.google.com/p/google-gson/
Cheers
First of all, you need to know if the JSON file contains an array or an object. If the first nonwhite space character is a [, it's an array, if it's a {, it's an object. Creating JSONArray when the first char is a { or vice versa will throw a Runtime Exception.
Second off all, once you have your JSONObject, you're going to want to get data from it. So you have to know the name of the keys to get the values, i.e.
myStreet = myJsonOjbect.getString("street name")
If you're not going to get data from it, what's the point of having the json file? Surely you can open the JSON in a Lint to see what the structure is.
hope this helps!
Can anyone recommend a JSON library for Java which allows me to give it chunks of data as they come in, in a non-blocking fashion? I have read through A better Java JSON library and similar questions, and haven't found precisely what I'd like.
Essentially, what I'd like is a library which allows me to do something like the following:
String jsonString1 = "{ \"A broken";
String jsonString2 = " json object\" : true }";
JSONParser p = new JSONParser(...);
p.parse(jsonString1);
p.isComplete(); // returns false
p.parse(jsonString2);
p.isComplete(); // returns true
Object o = p.getResult();
Notice the actual key name ("A broken json object") is split between pieces.
The closest I've found is this async-json-library which does almost exactly what I'd like, except it cannot recover objects where actual strings or other data values are split between pieces.
There are a few blocking streaming/incemental JSON parsers (as per Is there a streaming API for JSON?); but for async nothing yet that I am aware of.
The lib you refer to seems badly named; it does not seem to do real asynchronous processing, but merely allow one to parse sequence of JSON documents (which multiple other libs allow doing as well)
If there were people who really wanted this, writing one is not impossible -- for XML there is Aalto, and handling JSON is quite a bit simpler than XML.
For what it is worth, there is actually this feature request to add non-blocking parsing mode for Jackson; but very few users have expressed interest in getting that done (via voting for the feature request).
EDIT: (2016-01) while not async, Jackson ObjectMapper allows for convenient sub-tree by sub-tree binding of parts of the stream as well -- see ObjectReader.readValues() (ObjectReader created from ObjectMapper), or short-cut versions of ObjectMapper.readValues(...). Note the trailing s in there, which implies a stream of Objects, not just a single one.
Google Gson can incrementally parse Json from an InputStream
https://sites.google.com/site/gson/streaming
I wrote such a parser: JsonParser.java. See examples how to use it:JsonParserTest.java.