I have Java program and I am reading from file of Size Approx 40GB and converting data on fly and streaming to DataOutputstream.
Below is my code
JSONObject jsonObj= new JSONObject();
JSONArray jsonArray = new JSONArray();
JSONObject properties = new JSONObject();
while((strLine = in.readLine()) != null)
{
if (jsonArray.length()%100 == 0) {
printJsonData(jsonArray);
// sendJsonData(jsonArray, output);
jsonArray = new JSONArray();
}
if (strLine.trim().contains("data")) {
jsonObj.put("properties",properties);
jsonArray.put(jsonObj);
if (strLine.trim().contains("data1")) {
String secondPart = strLine.split(":",2)[1];
properties.put("data1", secondPart);
continue;
}
if (strLine.trim().contains("id")) {
String secondPart = strLine.split(":",2)[1];
jsonObj.put("id", secondPart );
continue;
}
}
I have two methods one is printing JSON data and other is sending Json data over https.
private void printJsonData(JSONArray jsonArray) {
int count = jsonArray.length(); // get totalCount of all jsonObjects
for(int i=0 ; i< count; i++){ // iterate through jsonArray
JSONObject jsonObject = jsonArray.getJSONObject(i); // get jsonObject # i position
System.out.println("jsonObject " + i + ": " + jsonObject);
}
}
private void sendJsonData(JSONObject jsonObj,DataOutputStream output) throws IOException {
int count = jsonArray.length();
for(int i=0 ; i< count; i++) { // iterate through jsonArray
JSONObject jsonObject = jsonArray.getJSONObject(i); // get jsonObject # i position
System.out.println("Sending Data-->" + jsonObject.toString());
output.writeBytes(jsonObject.toString());
}
}
Whenver i call print method, every thing works fine. But when i call sendJsonData method. I am getting OutofMemoryerror. Wondering how to fix it?
Since you are not manipulating json in any way I would treat as a text file and send it that way. The likelihood of finding a big JsonObject is good so you may be pushing through too much.
An array of 40gb is awfully big array unless there are huge objects.
** EDIT **
Or you could treat the individual json objects as "files" and push those strings individually
Related
So i have a Json string that i have retrieved from my PHP script on my MySQL server
{"Clientid":[24,26,27],"companyname":["chelsea","Microsoft","IBM"]}
and i have been trying to push this into a java objects in-order to populate a spinner at the moment the spinner on each line shows "chelsea","Microsoft","IBM" but i would like to show each company on a separate line so the user can select which company
current code
String response = stringBuilder.toString();
Log.d("response", response);
JSONObject jsonResponse = new JSONObject(response);
Log.d("response", jsonResponse.toString());
responseStreamReader.close();
Log.d("length", Integer.toString(jsonResponse.length()));
if (jsonResponse.length() == 0) {
returnedClient = null;
} else {
List<Client> myClients;
myClients = new ArrayList<Client>();
for (int i = 0; i < jsonResponse.length(); i++) {
Client client = new Client();
client.clientid = jsonResponse.getString("Clientid");
client.companyname = jsonResponse.getString("companyname");
myClients.add(client);
}
returnedClient = myClients;
}
} catch (Exception e) {
e.printStackTrace();
}
//Log.d("returnedUser", returnedUser.userpassword);
return returnedClient;
current code is obviously the wrong code so i have been looking at Gson and Jackson and volley and getting in a mess so i have been looking at the below but cant get it right to fit the Json string so wondered if you could help please or advise a better solution?
JSONArray jarray = jsonResponse.getJSONArray("companyname");
ArrayList<String> xyz= new ArrayList<String>();
for(int i = 0; i < jarray.length(); i++){
JSONArray timeArray =jarray.getJSONObject(i).getJSONArray("companyname");
for(int j = 0; j < timeArray .length(); j++){
xyz.add(timeArray .getString(j));
Log.d("get value",timeArray .getString(j));
Is there something im missing with the parser as doesnt matter what i change it seems to not like anything i do, it says the json cant be an object then it cant be an array, i assume the json is an object then an array but at the mormnt i just get errors :(
try {
List<Client> returnedClient = null;
// JSONObject json = new JSONObject(content);
JSONArray json1 = new JSONArray(content);
//JSONObject dataObject1 = json.getJSONObject("Clientid");
//JSONObject dataObject2 = json.getJSONObject("companyname");
// JSONArray items1 = json.getJSONArray("Clientid");
// JSONObject items2 = json.getJSONObject("companyname");
// JSONArray items1 = json.getJSONArray("Clientid");
JSONArray items2 = json1.getJSONArray(0);
for (int i = 0; i < items2.length(); i++) {
// JSONArray clientObject1 = items1.getJSONArray(i);
// JSONObject clientObject2 = items2.getJSONObject(i);
JSONArray clientObject3 = items2.getJSONArray(i);
// Client client = new Client(clientObject2);
Log.d("response",clientObject3.toString());
// returnedClient.add(client);
}
O silly me the is was further up the tree as i was using the code below at the top of the page before i even started the parse, shame it took me to re write my sql before i noticed :(
JSONObject jsonResponse = new JSONObject(response);
I get JSON from a rest server (jax rs) through a http request.
[{"carId":"t0","movements":[{"edge":{"edgeId":"--104","fromNode":{"latitude":"51.404456094285734","longitude":"5.477570348298111","nodeId":"-16"},"lanes":[{"laneId":"--104_0"},{"laneId":"--104_1"}],"toNode":{"latitude":"51.404057790546894","longitude":"5.424575687577547","nodeId":"-14"}},"id":8,"lane":{"laneId":"--104_0"},"position":73.76,"speed":14.49,"timeStep":8.0},{"edge":{"edgeId":"--104","fromNode":{"latitude":"51.404456094285734","longitude":"5.477570348298111","nodeId":"-16"},"lanes":[{"laneId":"--104_0"},{"laneId":"--104_1"}],"toNode":{"latitude":"51.404057790546894","longitude":"5.424575687577547","nodeId":"-14"}},"id":16,"lane":{"laneId":"--104_0"},"position":89.35,"speed":15.59,"timeStep":9.0},{"edge":{"edgeId":"--104","fromNode":{"latitude":"51.404456094285734","longitude":"5.477570348298111","nodeId":"-16"},"lanes":[{"laneId":"--104_0"},{"laneId":"--104_1"}],"toNode":{"latitude":"51.404057790546894","longitude":"5.424575687577547","nodeId":"-14"}},"id":27,"lane":{"laneId":"--104_0"},"position":106.39,"speed":17.03,"timeStep":10.0},{"edge":{"edgeId":"--104","fromNode":{"latitude":"51.404456094285734","longitude":"5.477570348298111","nodeId":"-16"},"lanes":[{"laneId":"--104_0"},{"laneId":"--104_1"}],"toNode":{"latitude":"51.404057790546894","longitude":"5.424575687577547","nodeId":"-14"}},"id":35,"lane":{"laneId":"--104_0"},"position":125.24,"speed":18.85,"timeStep":11.0},{"edge":{"edgeId":"--104","fromNode":{"latitude":"51.404456094285734","longitude":"5.477570348298111","nodeId":"-16"},"lanes":[{"laneId":"--104_0"},{"laneId":"--104_1"}],"toNode":
Just to give you an idea how the JSON looks like.
Now I want to convert every item of this JSON file to an Java object.
So I want to see inside the "fromNode" to get the attributes to make an object.
I can see all the movement and edges, thats no problem (i do that like this):
String jsonStringMike = callURL("http://----------/");
try {
JSONArray jsonArrayMike = new JSONArray(jsonStringMike);
int countMike = jsonArrayMike.length(); // get totalCount of all jsonObjects
for (int i = 0; i < countMike; i++) { // iterate through jsonArray
JSONObject jsonObject = jsonArrayMike.getJSONObject(i); // get jsonObject # i position
JSONArray movement = jsonObject.getJSONArray("movements");
String edge = movement.getJSONObject(i).getString("edge");
System.out.println(edge);
So this provides me with all the "edge's" From the JSON file shown before. But if I want to see the "fromNode" I get the following error:
org.json.JSONException: JSONObject["edge"] is not a JSONArray.
This is the code that is not working and will provide the given error.
public static void main(String[] args) throws IOException, JSONException {
String jsonStringMike = callURL("http://6--------------");
try {
JSONArray jsonArrayMike = new JSONArray(jsonStringMike);
int countMike = jsonArrayMike.length(); // get totalCount of all jsonObjects
for (int i = 0; i < countMike; i++) { // iterate through jsonArray
JSONObject jsonObject = jsonArrayMike.getJSONObject(i); // get jsonObject # i position
JSONArray movement = jsonObject.getJSONArray("movements");
int o = movement.length();
String test = movement.toString();
JSONArray jsonArray2 = new JSONArray(test);
for (int p = 0; p < o; o++) {
JSONObject moveObject = jsonArray2.getJSONObject(p);
JSONArray edge = moveObject.getJSONArray("edge");
String fromNode = edge.getJSONObject(p).getString("fromNode");
System.out.println(fromNode);
}
Does anybody knows how I can look "deeper" into the given JSON file?
I think you are almost there, the issue in your code that in the loop moveObject is your edge json object, so movemen.length() is the the count of all edge json objects in the array. I have not tested this, but I think it should work. Also, your for loop has typo, o++ should be p++,
JSONArray jsonArrayMike = new JSONArray(jsonStringMike);
int countMike = jsonArrayMike.length(); // get totalCount of all jsonObjects
for (int i = 0; i < countMike; i++) { // iterate through jsonArray
JSONObject jsonObject = jsonArrayMike.getJSONObject(i); // get jsonObject # i position
JSONArray movement = jsonObject.getJSONArray("movements");
int o = movement.length();
for (int p = 0; p < o; p++)
{
JSONObject movementObject = movement.getJSONObject(p);
JSONObject edgeObject = movementObject.getJSONObject("edge");
JSONObject fromNode = edgeObject.getJSONObject("fromNode");
String latitude = fromNode.getString("latitude");
String longitude = fromNode.getString("longitude");
String nodeId = fromNode.getString("nodeId");
System.out.println(fromNode);
}
}
I have a webservice which returns a JSON array in this format:
[{"imageid":"3","userid":"1","imagepath":"SLDFJNDSKJFN","filterid":"1","dateadded":"2014-05-06 21:20:18.920257","public":"t"},
{"imageid":"4","userid":"1","imagepath":"dsfkjsdkfjnkjdfsn","filterid":"1","dateadded":"2014-05-06 21:43:37.642748","public":"t"}]
I need to get all the attributes seperately? How would I do this?
I know how to do it with JSONObject if there is just 1 thing being returned, but how does it work when multiple items are returned?
Thanks
try {
JSONArray jArray = new JSONArray(jsonString);
String s = new String();
for (int i = 0; i < jArray.length(); i++) {
s = jArray.getJSONObject(i).getString("imageid").toString();
s = jArray.getJSONObject(i).getString("userid").toString();
}
} catch (JSONException je) {
}
Create an Object class with all variables, create a List for this Object, add all objects in your JSONArray to the list, use the one you need.
List<YourObject> objList = new ArrayList<YourObject>();
JSONArray a = new JSONArray(response);
int size = a.length();
for (int i=0 ; i<size ; i++){
JSONObject aa = a.getJSONObject(i);
String id = aa.getString("imageid");
String userid = aa.getString("userid");
String imagepath = aa.getString("imagepath");
String filterid = aa.getString("filterid");
String dateadded = aa.getString("dateadded");
String publicText = aa.getString("public");
YourObject obj = new YourObject(id,userid,imagepath,filterid,dateadded,publicText);
objList.add(obj);
}
So what you are having here is some JSON objects inside a JSON array.
What you want to do is this:
JSONArray array = ...;
for (int i = 0; i < array.length(); i++) {
JSONObject o = array.getJSONObject(i);
// Extract whatever you want from the JSON object.
}
I hope it helped.
You can use JSONArray to parse array of JSON response.
private void parseJsonArray(String response) {
try {
JSONArray array = new JSONArray(response);
for(int i=0;i<array.length();i++){
JSONObject jsonObject = array.getJSONObject(i);
String ImageId = jsonObject.getString("imageid");
Log.v("JSON Parser", "ImageId: "+ImageId);
}
} catch (Exception e) {
e.printStackTrace();
}
}
JSONArray jsonArray = new JSONArray();
jsonArray.put(1);
jsonArray.put("empty");
jsonArray.put(2);
jsonArray.put(3);
jsonArray.put("empty");
jsonArray.put(4);
jsonArray.put("empty");
lets say we have this jsonArray, there are strings empty, how to remove them without leaving gaps?
You can use the following code :
for (int i = 0, len = jsonArray.length(); i < len; i++) {
JSONObject obj = jsonArray.getJSONObject(i);
String val = jsonArray.getJSONObject(i).getString();
if (val.equals("empty")) {
jsonArray.remove(i);
}
}
take a look at this post.
Make a list to put the indexes of the elements with the "empty" string and iterate over your list (the one with the "empty" elements) saving the indexes of the items to delete. After that, iterate over the previosly saved indexes and use
list.remove(position);
where position takes the value of every item to delente (within the list of index).
Should work with few fixes to Keerthi's code:
for (int i = 0; i < jsonArray.length(); i++) {
if (jsonArray.get(i).equals("empty")) {
jsonArray.remove(i);
}
}
You can use this following code
JSONArray list = new JSONArray();
JSONArray jsonArray = new JSONArray(jsonstring);
int len = jsonArray.length();
if (jsonArray != null) {
for (int i=0;i<len;i++)
{
//Excluding the item string equal to "empty"
if (!"empty".equals(jsonArray.getString(i))
{
list.put(jsonArray.get(i));
}
}
//Now, list JSONArray has no empty string values
}
you can use string replace after converting the array to string as,
JSONArray jsonArray = new JSONArray();
jsonArray.put(1);
jsonArray.put("empty");
jsonArray.put(2);
jsonArray.put(3);
jsonArray.put("empty");
jsonArray.put(4);
jsonArray.put("empty");
System.err.println(jsonArray);
String jsonString = jsonArray.toString();
String replacedString = jsonString.replaceAll("\"empty\",", "").replaceAll("\"empty\"", "");
jsonArray = new JSONArray(replacedString);
System.out.println(jsonArray);
Before replace:
jsonArray is [1,"empty",2,3,"empty",4,"empty"]
After replace:
jsonArray is [1,2,3,4]
I am trying to store data gotton from JSON to an normal string array. The problem is i cant seem to initialize the array size according to the number for JSON data.
For now i declare the array size my own:
String[] path= new String[5];
This is the JSON part:
class Loadpath extends AsyncTask {
protected String doInBackground(String... args) {
// Building Parameters
List<NameValuePair> params = new ArrayList<NameValuePair>();
// getting JSON string from URL
JSONObject json = jParser.makeHttpRequest(url_all_imageDB, "GET",params);
try {
JSONArray productObj = json.getJSONArray(TAG_IMAGEDB); // JSON Array
// looping through All path
for (int i = 0; i < productObj.length(); i++) {
JSONObject image = productObj.getJSONObject(i);
path[i] = image.getString(TAG_PATH);
}
} else {
displayPath.setText("No path");
}
} catch (JSONException e) {
e.printStackTrace();
}
return null;
}
So my goal is the array size is gotten from the JSON.length().. Any idea how I can achieve this?
initialize the mStrings[ ] array size according to the JSONArray length as:
String[] mStrings; //<< Declare array as
mStrings = new String[productObj.length()]; //<<initialize here
for (int i = 0; i < productObj.length(); i++) {
JSONObject image = productObj.getJSONObject(i);
mStrings[i] = image.getString(TAG_PATH);
}
and it is good if you use ArrayList instead of String Array as for getting values from Jsonobject as:
ArrayList<String> mStrings=new ArrayList<String>(;
for (int i = 0; i < productObj.length(); i++) {
JSONObject image = productObj.getJSONObject(i);
mStrings.add(image.getString(TAG_PATH));
}
Maybe u can use List instead of String array