I have nested JSON with bunch of children objects, but I just need response_time and question, subquestions of survey_data. What is the best way to parse nested JSON in rest controller to the object in spring?
{
"is_test_data":false,
"language":"English",
"url_variables":{
"requestId":{
"key":"requestId",
"value":"1"
}
},
"response_time":1114,
"survey_data":{
"2":{
"id":2,
"type":"parent",
"question":"For each of the following factors, please rate your recent project",
"subquestions":{
"10":{
"10001":{
"id":10001,
"type":"MULTI_TEXTBOX",
"question":"Overall Quality : Rating",
"answer":null,
}
},
"11":{
"10001":{
"id":10001,
"type":"MULTI_TEXTBOX",
"question":"Achievement of Intended Objectives : Rating",
"answer":null
}
}
}
},
"33":{
"id":33,
"type":"HIDDEN",
"question":"Submitted",
"answer_id":0,
}
}
}
Thank you.
What you should do is parse the complete json to jsonObject using json-simple jar
which create a map like structure for the json and then you can simply get the desired value from it using the key as I explained in below example
import java.io.File;
import java.io.FileInputStream;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.Reader;
import java.nio.charset.Charset;
import java.nio.charset.StandardCharsets;
import org.json.simple.JSONObject;
import org.json.simple.parser.JSONParser;
public class JsonDeserializer {
public static void main(String[] args) throws Exception {
File file = new File("test.json");
InputStream is = new FileInputStream(file);
StringBuilder textBuilder = new StringBuilder();
try (Reader reader = new BufferedReader(
new InputStreamReader(is, Charset.forName(StandardCharsets.UTF_8.name())))) {
int c = 0;
while ((c = reader.read()) != -1) {
textBuilder.append((char) c);
}
}
String jsonTxt = textBuilder.toString();
Object obj = new JSONParser().parse(jsonTxt);
JSONObject jo = (JSONObject) obj;
System.out.println(jo.get("response_time"));
}
}
JSON is a data communication format that is lightweight, text-based. Objects and arrays are two structured kinds that JSON can represent. A JSONArray may extract text from a String and convert it to a vector-like object. The getString(index) method of JSONArray can be used to parse a nested JSON object. This is a helper method for the getJSONString method (index). The getString() method returns a string.
Related
How is arbitrary JSON converted to arbitrary XML using BaseX?
I'm looking at JsonParser from BaseX for this specific solution.
In this case, I have tweets using Twitter4J:
package twitterBaseX;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import java.util.Properties;
import java.util.logging.Logger;
import main.LoadProps;
import org.basex.core.BaseXException;
import twitter4j.JSONException;
import twitter4j.JSONObject;
import twitter4j.Query;
import twitter4j.QueryResult;
import twitter4j.Status;
import twitter4j.Twitter;
import twitter4j.TwitterException;
import twitter4j.TwitterFactory;
import twitter4j.TwitterObjectFactory;
import twitter4j.conf.ConfigurationBuilder;
public class TwitterOps {
private static final Logger log = Logger.getLogger(TwitterOps.class.getName());
public TwitterOps() {
}
private TwitterFactory configTwitterFactory() throws IOException {
LoadProps loadTwitterProps = new LoadProps("twitter");
Properties properties = loadTwitterProps.loadProperties();
log.fine(properties.toString());
ConfigurationBuilder configurationBuilder = new ConfigurationBuilder();
configurationBuilder.setDebugEnabled(true)
.setJSONStoreEnabled(true)
.setOAuthConsumerKey(properties.getProperty("oAuthConsumerKey"))
.setOAuthConsumerSecret(properties.getProperty("oAuthConsumerSecret"))
.setOAuthAccessToken(properties.getProperty("oAuthAccessToken"))
.setOAuthAccessTokenSecret(properties.getProperty("oAuthAccessTokenSecret"));
return new TwitterFactory(configurationBuilder.build());
}
public List<JSONObject> getTweets() throws TwitterException, IOException, JSONException {
Twitter twitter = configTwitterFactory().getInstance();
Query query = new Query("lizardbill");
QueryResult result = twitter.search(query);
String string = null;
JSONObject tweet = null;
List<JSONObject> tweets = new ArrayList<>();
for (Status status : result.getTweets()) {
tweet = jsonOps(status);
tweets.add(tweet);
}
return tweets;
}
private JSONObject jsonOps(Status status) throws JSONException, BaseXException {
String string = TwitterObjectFactory.getRawJSON(status);
JSONObject json = new JSONObject(string);
String language = json.getString("lang");
log.fine(language);
return json;
}
}
The JSONObject from Twitter4J cannot just get jammed into XML?
There are a number of online converters which purport to accomplish this, and, which, at least at first glance, seem quite adequate.
see also:
Converting JSON to XML in Java
Java implementation of JSON to XML conversion
Use the (excellent) JSON-Java library from json.org then
JSONObject json = new JSONObject(str);
String xml = XML.toString(json);
toString can take a second argument to provide the name of the XML root node.
This library is also able to convert XML to JSON using XML.toJSONObject(java.lang.String string)
Check the Javadoc for more information
I store all static data in the JSON file. This JSON file has up to 1000 rows. How to get the desired data without storing all rows as ArrayList?
My code, I'm using right now and I want to increase its efficiency.
List<Colors> colorsList = new ObjectMapper().readValue(resource.getFile(), new TypeReference<Colors>() {});
for(int i=0; i<colorsList.size(); i++){
if(colorsList.get(i).getColor.equals("Blue")){
return colorsList.get(i).getCode();
}
}
Is it possible? My goal is to increase efficiency without using ArrayList. Is there a way to make the code like this?
Colors colors = new ObjectMapper().readValue(..."Blue"...);
return colors.getCode();
Resource.json
[
...
{
"color":"Blue",
"code":["012","0324","15478","7412"]
},
{
"color":"Red",
"code":["145","001","1","7879","123984","89"]
},
{
"color":"White",
"code":["7","11","89","404"]
}
...
]
Colors.java
class Colors {
private String color;
private List<String> code;
public Colors() {
}
public String getColor() {
return color;
}
public void setColor(String color) {
this.color = color;
}
public List<String> getCode() {
return code;
}
public void setCode(List<String> code) {
this.code = code;
}
#Override
public String toString() {
return "Colors{" +
"color='" + color + '\'' +
", code=" + code +
'}';
}
}
Creating POJO classes in this case is a wasting because we do not use the whole result List<Colors> but only one internal property. To avoid this we can use native JsonNode and ArrayNode data types. We can read JSON using readTree method, iterate over array, find given object and finally convert internal code array. It could look like below:
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.node.ArrayNode;
import java.io.File;
import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;
public class JsonApp {
public static void main(String[] args) throws Exception {
File jsonFile = new File("./resource/test.json").getAbsoluteFile();
ObjectMapper mapper = new ObjectMapper();
ArrayNode rootArray = (ArrayNode) mapper.readTree(jsonFile);
int size = rootArray.size();
for (int i = 0; i < size; i++) {
JsonNode jsonNode = rootArray.get(i);
if (jsonNode.get("color").asText().equals("Blue")) {
Iterator<JsonNode> codesIterator = jsonNode.get("code").elements();
List<String> codes = new ArrayList<>();
codesIterator.forEachRemaining(n -> codes.add(n.asText()));
System.out.println(codes);
break;
}
}
}
}
Above code prints:
[012, 0324, 15478, 7412]
Downside of this solution is we load the whole JSON to memory which could be a problem for us. Let's try to use Streaming API to do that. It is a bit difficult to use and you must know how your JSON payload is constructed but it is the fastest way to get code array using Jackson. Below implementation is naive and does not handle all possibilities so you should not rely on it:
import com.fasterxml.jackson.core.JsonFactory;
import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.core.JsonToken;
import java.io.File;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
public class JsonApp {
public static void main(String[] args) throws Exception {
File jsonFile = new File("./resource/test.json").getAbsoluteFile();
System.out.println(getBlueCodes(jsonFile));
}
private static List<String> getBlueCodes(File jsonFile) throws IOException {
try (JsonParser parser = new JsonFactory().createParser(jsonFile)) {
while (parser.nextToken() != JsonToken.END_OBJECT) {
String fieldName = parser.getCurrentName();
// Find color property
if ("color".equals(fieldName)) {
parser.nextToken();
// Find Blue color
if (parser.getText().equals("Blue")) {
// skip everything until start of the array
while (parser.nextToken() != JsonToken.START_ARRAY) ;
List<String> codes = new ArrayList<>();
while (parser.nextToken() != JsonToken.END_ARRAY) {
codes.add(parser.getText());
}
return codes;
} else {
// skip current object because it is not `Blue`
while (parser.nextToken() != JsonToken.END_OBJECT) ;
}
}
}
}
return Collections.emptyList();
}
}
Above code prints:
[012, 0324, 15478, 7412]
At the end I need to mention about JsonPath solution which also can be good if you can use other library:
import com.jayway.jsonpath.JsonPath;
import net.minidev.json.JSONArray;
import java.io.File;
import java.util.List;
import java.util.stream.Collectors;
public class JsonPathApp {
public static void main(String[] args) throws Exception {
File jsonFile = new File("./resource/test.json").getAbsoluteFile();
JSONArray array = JsonPath.read(jsonFile, "$[?(#.color == 'Blue')].code");
JSONArray jsonCodes = (JSONArray)array.get(0);
List<String> codes = jsonCodes.stream()
.map(Object::toString).collect(Collectors.toList());
System.out.println(codes);
}
}
Above code prints:
[012, 0324, 15478, 7412]
You can use DSM stream parsing library for memory, CPU efficiency and fast development. DSM uses YAML based mapping file and reads the whole data only once.
Here is the solution of your question:
Mapping File:
params:
colorsToFilter: ['Blue','Red'] # parameteres can be passed programmatically
result:
type: array
path: /.*colors # path is regex
filter: params.colorsToFilter.contains(self.data.color) # select only color that exist in colorsToFilter list
fields:
color:
code:
type: array
Usage of DSM to parse json:
DSM dsm = new DSMBuilder(new File("path/maping.yaml")).create(Colors.class);
List<Colors> object = (List<Colors>) dsm.toObject(jsonData);
System.out.println(object);
Output:
[Colors{color='Blue', code=[012, 0324, 15478, 7412]}, Colors{color='Red', code=[145, 001, 1, 7879, 123984, 89]}]
I am new on JSON, I want to read the JSON document using device id, I have many document on single file, just like as a database:
{
"deviceType":"AccessPoint",
"transactionType":"response",
"messageType":"set_config",
"classes":[
{
"deviceType":"AccessPoint",
"classId":1,
"ipv4":"192.168.100.100",
"netmask":"192.168.100.100",
"ipv6":"192.168.100.100",
"className":"Interface",
"interfaceName":"wlan0",
"state":"UP",
"type":"wireless",
"deviceId":"1234",
"status":"waiting"
}
],
"deviceId":"1234",
"transactionId":"201675"
}
Sometimes, classes array contain multiple arrays like indexes [{},{},..].
So, how can I read the doc using search criteria with java web application.
Just out of curiosity , did something on Jackson (streaming parser , since there is mention of large number of elements). I am just doing an output as name : value , this can be logically enhanced to suit your tastes. This is not even close to production , but is a good start.
import java.io.FileInputStream;
import com.fasterxml.jackson.core.JsonFactory;
import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.core.JsonToken;
public class TestClass {
public static void main(String[] args) {
try (FileInputStream fis = new FileInputStream("yourpath/transactions.json")) {
JsonFactory jf = new JsonFactory();
JsonParser jp = jf.createParser(fis);
//System.out.println(JsonToken.START_ARRAY);
// jp.setCodec(new ObjectMapper());
jp.nextToken();
while (jp.hasCurrentToken()) {
if (jp.getCurrentToken().equals(JsonToken.START_OBJECT)
|| jp.getCurrentToken().equals(JsonToken.START_ARRAY)
|| jp.getCurrentToken().equals(JsonToken.END_ARRAY)
|| jp.getCurrentToken().equals(JsonToken.END_OBJECT)) {
} else {
System.out.print(jp.getText());
jp.nextToken();
System.out.print(":");
System.out.print(jp.getText());
System.out.println();
}
jp.nextToken();
}
} catch (Exception ex) {
ex.printStackTrace();
}
}
}
Please have a look at the following.
import java.io.BufferedReader;
import java.io.File;
import java.io.FileWriter;
import java.io.IOException;
import java.io.InputStreamReader;
import org.json.JSONArray;
import org.json.JSONException;
import org.json.JSONML;
import org.json.JSONTokener;
import org.json.XML;
import com.amazonaws.auth.ClasspathPropertiesFileCredentialsProvider;
import com.amazonaws.regions.Region;
import com.amazonaws.regions.Regions;
import com.amazonaws.services.s3.AmazonS3Client;
import com.amazonaws.services.s3.model.GetObjectRequest;
import com.amazonaws.services.s3.model.S3Object;
public class JsonToXML
{
private AmazonS3Client s3;
public JsonToXML(String inputBucket, String inputFile) throws IOException, JSONException
{
//Connection to S3
s3 = new AmazonS3Client(new ClasspathPropertiesFileCredentialsProvider());
Region usWest2 = Region.getRegion(Regions.US_EAST_1);
s3.setRegion(usWest2);
//Downloading the Object
System.out.println("Downloading Object");
S3Object s3Object = s3.getObject(new GetObjectRequest(inputBucket, inputFile));
System.out.println("Content-Type: " + s3Object.getObjectMetadata().getContentType());
//Read the JSON File
BufferedReader reader = new BufferedReader(new InputStreamReader(s3Object.getObjectContent()));
StringBuffer strBuffer = new StringBuffer("");
int i=0;
while (true) {
String line = reader.readLine();
if (line == null) break;
System.out.println("Running: "+i);
strBuffer.append(line);
i++;
}
JSONTokener jTokener = new JSONTokener(strBuffer.toString());
JSONArray jsonArray = new JSONArray(jTokener);
//Convert to XML
String xml = XML.toString(jsonArray);
File f = new File("XML.xml");
FileWriter fw = new FileWriter(f);
fw.write(xml);
}
}
This is how the Json files look like
[
{
"_type": "ArticleItem",
"body": "Who's signing",
"source": "money.cnn.com",
"last_crawl_date": "2014-01-14",
"url": "http: //money.cnn.com/"
},
{
"_type": "ArticleItem",
"body": "GMreveals",
"title": "GMreveals625-horsepowerCorvetteZ06-Jan.13",
"source": "money.cnn.com",
"last_crawl_date": "2014-01-14",
"url": "http: //money.cnn.com"
}
]
This code generated invalid XML or files without any text. Invalid means, after the last <> it still generate some text, so the entire file is invalid. What is wrong here?
UPDATE
According to the answer of jtahlborn I managed to generate an XML file with the following output.
<array><body>Who's signing</body><_type>ArticleItem</_type><source>money.cnn.com</source><last_crawl_date>2014-01-14</last_crawl_date><url>http: //money.cnn.com/</url></array><array><body>GMreveals</body><_type>ArticleItem</_type><title>GMreveals625-horsepowerCorvetteZ06-Jan.13</title><source>money.cnn.com</source><last_crawl_date>2014-01-14</last_crawl_date><url>http: //money.cnn.com</url></array>
But XML Validator in here says:
XML Parsing Error: junk after document element
Location: http://www.w3schools.com/xml/xml_validator.asp
Line Number 1, Column 181:
You need to flush()/close() the FileWriter to ensure all the data is written to the file.
The problem is that you have 2 "top-level" elements in your xml result (2 "array" elements). xml can only have one top-level element.
UPDATE:
Try this for converting the json to xml:
String xml = XML.toString(jsonArray, "doc");
For example I have a response with the following JSON:
{
response: {
id: 20,
name: Stas
}
}
And I want to parse it to the following object:
class User {
private int id;
private String name;
}
How to skip the response node?
I use Google Http Java Client and it will be good if someone will answer how to do it there.
How will this lines have changed?
request.setParser(new JacksonFactory().createJsonObjectParser());
return request.execute().parseAs(getResultType());
You can now implement this in one step:
new JsonObjectParser.Builder(jsonFactory)
.setWrapperKeys(Arrays.asList("response"))
.build()
http://javadoc.google-http-java-client.googlecode.com/hg/1.15.0-rc/index.html
I do not know the Google Http Java Client, but if you can access the Jackson ObjectMapper you could do the following:
1.) Enable root unwrapping:
objectMapper.enable(DeserializationFeature.UNWRAP_ROOT_VALUE);
2.) Add annotation to User.class:
#JsonRootName("response")
class User {
…
}
I hope you can use this approach.
Edit: I dug through the google-http-java-client API and have come to the conclusion that you cannot access the ObjectMapper directly. In order to use the full power of Jackson you would have to write your own implementation of JsonObjectParser to wrap a 'real' Jackson parser. Sorry about that, maybe someone else could come up with a better solution.
I didn't find a native way (for this library) to solve my task. As a result I solved this problem by extending the functionality of JsonObjectParser. It entails expanding of the JacksonFactory, but it's a final class, so I used aggregation.
I wrote the following classes:
JacksonFilteringFactory
import com.google.api.client.json.JsonObjectParser;
import com.google.api.client.json.jackson2.JacksonFactory;
public class JacksonFilteringFactory {
private final JacksonFactory factory = new JacksonFactory();
public JsonObjectParser createJsonObjectParser(String filteringNode) {
return new FilteringJsonObjectParser(factory, filteringNode);
}
}
FilteringJsonObjectParser
import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.lang.reflect.Type;
import java.nio.charset.Charset;
import org.json.JSONException;
import org.json.JSONObject;
import org.json.JSONTokener;
import com.google.api.client.json.JsonFactory;
import com.google.api.client.json.JsonObjectParser;
import com.vkredmessenger.AppController;
import com.vkredmessenger.util.StringUtils;
public class FilteringJsonObjectParser extends JsonObjectParser {
private String mFilteringNode;
public FilteringJsonObjectParser(JsonFactory jsonFactory,
String filteringNode) {
super(jsonFactory);
mFilteringNode = filteringNode;
}
#Override
public Object parseAndClose(InputStream in,
Charset charset, Type dataType)
throws IOException {
String originalResponse =
StringUtils.convertStreamToString(in, charset);
String response = null;
try {
JSONTokener tokener = new JSONTokener(originalResponse);
JSONObject originalResponseObject =
(JSONObject) tokener.nextValue();
JSONObject responseObject =
originalResponseObject.getJSONObject(mFilteringNode);
response = responseObject.toString();
} catch (JSONException e) {
e.printStackTrace();
}
InputStream filteredIn =
new ByteArrayInputStream(response.getBytes(charset));
return super.parseAndClose(filteredIn, charset, dataType);
}
}
So, for example from my question, the result parsing code will be the following:
request.setParser(new JacksonFilteringFactory().createJsonObjectParser("response"));
return request.execute().parseAs(getResultType());