I have a json file with complex structure.
{"Objects":{"items":{"item":[
{
"field1": "value1",
"field2": "value2",
"field3":[
{
"label1":"1",
"label2":"2"
},
{
"label1":"3",
"label2":"4"
}]
}
,
{
//same structure as above object
}
]}}}
The file size is a little more than 1GB. I need to read an object and see what the value of a particular label is and if it matches the list I have, I need to write that object in another file else not.
I know normal JSON parser like JSONSimple won't work as it hold the data into the memory. I am trying to use Jackson, but finding hard to go over all objects as it takes one token at a time. What is an efficient way to use streaming and tree structure of Jackson for this JSON format.
Or in what way can I use script to get the data and use it?
Probably you could advance the JsonParser several times calling nextToken() until you get Token ID_START_ARRAY, call nextToken() to move to the start of the first item object and then feed the parser and POJO class representing "item" into ObjectMapper.readValue() (https://github.com/FasterXML/jackson-databind/blob/master/src/main/java/com/fasterxml/jackson/databind/ObjectMapper.java), repeat until no more objects are found. POJO can be hand-written or generated using something like https://github.com/astav/JsonToJava/wiki/JsonToJava
Or skip Jackson entirely - write a little tokenizer yourself that will extract individual "item" json elements and feed them into JSONSimple. This way you'll maybe have a bit of wheel reinvention, but will avoid getting a lot of dependencies.
Related
I am trying to deserialise the following Json using Jackson.
[
{
"id" : "abc",
"deleted": true
},
{
"id" : "def",
}
]
I do not want to create java objects for items that are marked "deleted":"true".
For the above example, output objects for my above JSON structure should be a List with just one object(id:def).
How can I configure jackson to do this?
You can use Jackson to read a stream of JSON data. See this tutorial for details. This will allow you to decide on the fly whether or not to create any object being parsed.
Alternatively, check out javax.json.stream. This library allows pull parsing which gives you the control you want over parsing your JSON.
In general, the terms you are looking for are "push parsing" and "pull parsing". Use those to find other alternatives to accomplish what you want.
I am writing a Java Library Parser for Alpha Vantage. Here is the endpoint to download some data:
https://www.alphavantage.co/query?function=DIGITAL_CURRENCY_INTRADAY&symbol=BTC&market=CNY&apikey=demo
As you can see the historical data are enclosed into { } and not into []. Do you have any idea to deserialize this? I am using Gson but I cannot find any way
First of all, your JSON structure is probably not what you want it to be. The content of the "Time Series (Digital Currency Intraday)" is an object with properties that seem to represent date. This is annoying from the start, since they are dynamically generated. It would be really simple, if the content would be a list of objects(that could also contain the date value):
{
"Time Series (Digital Currency Intraday)": [
{
"date": "2017-10-24 22:30:00",
"1a. price (CNY)": "34889.57919003",
"1b. price (USD)": "5259.23870358",
"2. volume": "5708.68994668",
"3. market cap (USD)": "30023363.11434300"
}
]
}
This would be a better structure, imo. Then all you need to do is change the property names to be more Java friendly and create some POJOs, then you could deserialize with GSON easily. See: http://www.baeldung.com/gson-deserialization-guide
As for a solution with the current structure, you could maybe make use of the entitySet() method of the JSONObject.
It returns a Set> object. Assuming you would use the method on an object representing the root Json object, the String values that you would get in the Map objects would be "Meta Data" and "Time Series (Digital Currency Intraday)". These are the properties of the root object in your JSON. And the JsonElement they are associated with, would be the value of the property, in this case another JSON object.
So you can use this and your knowledge of how the JSON is structured, to deserialize the data to some objects of a class. WIth a couple of loops, of course.
I have a RESTful web service that provides JSON that I am consuming. I am using Spring 3.2 and Spring's MappingJacksonHttpMessageConverter. My JSON looks like this:
{
"Daives": {
"Daive": {},
"Daive": {},
"Daive": {},
"Daive": {}
}
}
Now everything I have read seems to indicate that this JSON should be refactored to an array of JSON Daives. However, this is valid JSON so I want to make sure that I am thinking correctly before going back to the service provider to ask for changes. In the format above, I would have to know ahead of time how many Daives there are going to be such that my DTO accounted for them. The handy dandy Jackson mapper isn't going work with this kind of JSON setup. If the JSON was altered to provide and Array of JSON Daives, I could use a List to dynamically map them using Spring/Jackson.
Am I correct? Thanks :)
According to this thread, the JSON spec itself does not forbid multiple fields with the same name (in your case, multiple fields named "Daive" in the object "Daives").
However, most parsers will either return an error or ignore any value but the last one. As you said, putting these values into an array seems much more sensible; and indeed, you'll be able to map this array to a List with Jackson.
I have a JSON file with no clue on how data will be in it nor the structure of data.
The only thing known is that it will have either an array of JSON objects or a single JSON object.
I need to get each object from the file and store it as a separate item. In case of array of objects in the file, I should get an array of JSON strings which I can store in DB.
Basically, I need to read this file and separate out each JSON object from it and store it in DB as a string.
One of the ways to do it was to use JACKSON ObjectMapper and assign these items to a Hashmap as key value pairs, but I am not sure though how it can be done If there are list of JSON Objects in the file.
Sample JSON File:
[
{
"name":"Bob",
"type":"Email",
"from":"a#a.com",
"to":"b#B.com",
"attachments":[...],
.
.
.
}
]
Do you know the Object structure that the JSON has(let it be Array or a single one) ? If Yes,
First load the json string form the file into an in memory string.
check the string for Array existence, by searching for '[',']' in the outer structure of multiple occurrences of '{' or '}'
once you know whether you have an array or a single object, you can pass it as object reference to either Jackson or GSON parsers
create in memory Array of JsonObject.class say List. It is actually better to enclose this List inside another class. say myJsonObjects and have a List inside it.
Let us see GSON parsers (by google), though Jackson can also be used in the similar implementation
Gson gson = new Gson();
if(isArray){
myJsonObjects jsonArray = gson.fromJson(jsonStringFromFile,myJsonObjects );
}
else{
gson.fromJson(jsonStringFromFile,JsonObject);
}
http://google-gson.googlecode.com/svn-history/trunk/gson/docs/javadocs/com/google/gson/Gson.html
Jackson is my favorite JSON-to-POJO library. It doesn't really matter where you're loading the JSON from (a URL or from the filesystem), there are handlers for several input sources.
Here's an example:
Map<String,Object> userData = mapper.readValue(new File("user.json"), Map.class);
As far as having an unknown number of JSON structures that you're about to parse, the first thing that comes to mind is to have a mapper for each type you're expecting. You could then wrap the parsing code in try/catch blocks so that if the first fails with whatever exception Jackson gives you when encountering an unexpected format, you can then try the next format and so on.
If you're just trying to generically parse JSON that you don't know the structure of beforehand, you can try something like this:
mapper.readValue(jsonString, new TypeReference<List<EntryType>>() {});
The documentation for Jackson is pretty good-- giving it a solid read-through should definitely help. Here's a good five minute tutorial: http://wiki.fasterxml.com/JacksonInFiveMinutes
I prefer use Gson:
Gson gson;
Map<String, Object>parameters=gson.fromJson(myString);
the rest is iterate the map, i hope help you
I'm working with JSON on the server for the first time. I'm trying to create an object that looks like this:
{
columnData : {
column1 : {"foo": "bar", "value" : 1 },
},
rowData : {
row1 : {"type" : "integer", "value" : 1},
}
}
Essentially, it's a two-level JSON object in which I've got all objects on the top level. Everything on the second level will be a string, and integer, or a float. I've found several ways I could go about creating this JSON string:
Build the string directly
I can go through one line at a time and build up a giant string. This seems unwieldy.
Use gson
I can build a map (or class?) that has the structure of the JSON I want, and then use gson to convert to a JSON string.
Use JsonObject
This would work like this question. I just found that example, and I like how simple working with JSON looks in it, but I'm not sure if it's a better way to go than the other options.
I'm pretty new with Java, so I'm not really sure what the performance tradeoffs of these methods are. I do know that I'll potentially be calling this method lots of times very quickly to load data into a client as the user scrolls a table in a webapp, so I'd like everything to be as fast as possible. What's the quickest way to go about building the JSON string that I'll then be sending back to the client?
Depends on what "fastest" means. Fast to develop or fast in terms of performance.
Fastest on dev is to use a library to serialize existing data structures directly to JSON. Use the following library.
http://flexjson.sourceforge.net
Performance wise comparisons it matches Jackson and beats in some cases and destroys gson. But that means you would be serializing your data structures directly rather than mapping them by hand with JSONObject or something like that.
You may be able to get a faster transaction / sec by using JSONObject by hand and mapping your data structures to it. You might get better performance, but then you might not because you have to new up JSONObject and convert the data to it. Then allow JSONObject to render. Can you write hand written code better than a serializing library? Maybe, maybe not.