I am having an odd issue reading in the following JSON formatted API - Trivia API (this URL is passed in through the Async). I will explain in more detail below, but basically my code appears to be not properly reading in a JSON file, which results in me being unable to loop through it and set my question objects accordingly.
Logs revealed that the reading of the input stream at IOUtils seems to not finish reading the entire JSON file, often stopping by the 11th question object within the file.
I think this has to do with a Log limit, as when I add more to the Log (such as "This is the json String -") it prints less, but this does not explain why it is not properly looping through the question objects and setting them accordingly. Other logs revealed that it does not even loop through the questions, it only goes through one time and only sets the title.
I am clueless as to what is happening as I have parsed other JSON files literally the exact same way as this.
#Override
protected ArrayList<Question> doInBackground(String... strings) {
HttpURLConnection connection = null;
ArrayList<Question> result = new ArrayList<>();
try {
URL url = new URL(strings[0]);
connection = (HttpURLConnection) url.openConnection();
connection.connect();
if (connection.getResponseCode() == HttpURLConnection.HTTP_OK) {
String json = IOUtils.toString(connection.getInputStream(), "UTF-8");
Log.d("demo", "This is the json String - " + json);
JSONObject root = new JSONObject(json);
JSONArray questions = root.getJSONArray("questions");
for (int i = 0; i < questions.length(); i++) {
JSONObject questionsJSONObject = questions.getJSONObject(i);
Question question = new Question();
question.setText(questionsJSONObject.getString("text"));
//Choice array
JSONObject choicesObject = questionsJSONObject.getJSONObject("choices");
String[] temp = new String[choicesObject.getJSONArray("choice").length()];
for (int j = 0; j < choicesObject.getJSONArray("choice").length(); j++) {
temp[j] = choicesObject.getJSONArray("choice").getJSONObject(j).toString();
}
question.setChoices(temp);
//End of Choice array
question.setAnswer(questionsJSONObject.getInt("answer");
question.setUrlToImage(questionsJSONObject.getString("image"));
result.add(question);
}
}
} catch (Exception e) {
//Handle Exceptions
} finally {
//Close the connections
}
Log.d("demo", "toString - " + result.toString());
return result;
}
Related
First, some background :-
I'm trying to solve a question asked by an interviewer recently. I had to write a code and use below URL to return JSON response -
https://losangeles.craigslist.org/
This is what I did :-
1) I created a webclient and made HTTPURL Request to fetch an HTTP Response.
public static JSONArray getSearchResults(String arg) {
JSONArray jsonArray = null;
try {
QueryString qs = new QueryString("query", arg);
URL url = new URL("https://toronto.craigslist.ca/search?"+qs);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("GET");
conn.setRequestProperty("Accept", "application/text");
if (conn.getResponseCode() != 200) {
throw new RuntimeException("Failed : HTTP error code : "
+ conn.getResponseCode());
}
BufferedReader br = new BufferedReader(new InputStreamReader(
(conn.getInputStream())));
String readAPIResponse = " ";
StringBuilder output = new StringBuilder();
while ((readAPIResponse = br.readLine()) != null) {
output.append(readAPIResponse);
}
jsonArray = convertToJson(output);
System.out.println(" JSON response : "+jsonArray.toString(2));
conn.disconnect();
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
return jsonArray;
}
2) Below was my function to convert the response into JSON :-
public static JSONArray convertToJson(StringBuilder response) {
JSONArray jsonArr = new JSONArray();
if (response != null) {
try {
Document document = Jsoup.parse(response.toString());
Elements resultRows = document.getElementsByClass("result-row");
JSONObject jsonObj;
for (int i = 0; i < resultRows.size(); i++) {
jsonObj = new JSONObject();
Element e = resultRows.get(i);
Elements resultsDate = e.getElementsByClass("result-date");
Elements resultsTitle = e.getElementsByClass("result-title hdrlnk");
String key1 = "date";
String value1 = resultsDate.get(0).text();
jsonObj.put(key1, value1);
String key2 = "title";
String value2 = resultsTitle.get(0).text();
jsonObj.put(key2, value2);
jsonArr.put(i, jsonObj);
}
} catch (JSONException e) {
e.printStackTrace();
}
}
return jsonArr;
}
The response I received was the whole HTML page(I used postman to make requests). Since, I only had few hours to solve this question and was not sure how to parse an entire HTML, I ended up using a third party library, called JSoup. I was not 100% happy about it, but ended up having no other option.
I have not heard back from them and I am curious if this was the worst approach and if yes, what could be better options? They did not mention anything about what technology I could use. But,since the skill set I was interviewing involved Java/J2EE I was thinking to implement this in Java (Not using Node js though)
Thanks!
If you only need an XML Parser which is obviously the base of HTML this is built in in the JRE core API.
Even in the SE Version the needed packages to parse exist:
import org.w3c.dom.*;
import javax.xml.parsers.*;
import java.io.*;
Take a look at these classes they are the most important to parse or create an XML/HTML File
DocumentBuilderFactory
DocumentBuilder
Document
and here simple example for HTML
String text = "<html><head>HEAD</head><body>BODY</body>";
ByteArrayInputStream input = new ByteArrayInputStream(text.getBytes("UTF-8"));
Document doc = builder.parse(input);
I have a php file which fetches all rows from a specific column. The result is then put into an array and then json_encoded. When I Toast the output from the php file it shows a JSON string with the correct values. I need to convert this to a Java String array so I can iterate through it one by one on a button click.
I've tried multiple solutions on here but to no avail so any help will be greatly appreciated!
getImage.php
$sql = "SELECT advert_File_Path FROM Ads";
$result = $mysql_con->query($sql) or die('Query problem: '.mysqli_error($mysql_con));
//create array
$resultarray = array();
while($row = mysqli_fetch_assoc($result)){
$resultarray[] = $row;
}
echo json_encode($resultarray);
MainActivity.java (only the code that is relevant)
private void loadNextAdvert()
{
Toast.makeText(this, "Please wait, image may take a few seconds to load...", Toast.LENGTH_LONG).show();
dbhelper = (DbHelper) new DbHelper(new DbHelper.AsyncResponse() {
#Override
public void processFinish(String output) {
if(output.equalsIgnoreCase("exception") || output.equalsIgnoreCase("unsuccessful")){
Toast.makeText(MainActivity.this, "Connection error", Toast.LENGTH_SHORT).show();
}else{
//initializes ArrayList
stringList = new ArrayList<>();
try {
//initializes JSONArray with output from php getImage file
jsonArray = new JSONArray(output);
if(jsonArray != null){
int len = jsonArray.length();
for(int i=0; i<len; i++){
stringList.add(jsonArray.get(i).toString());
}
}
} catch (JSONException e) {
e.printStackTrace();
}
Toast.makeText(MainActivity.this, String.valueOf(stringList), Toast.LENGTH_LONG).show();
}
}
}).execute(loadPic);
}
The Toast which should display the contents of the ArrayList just appears as '[]'.
Managed to solve it myself. So here goes,
Step 1. I had to remove commented code from my php file as it was breaking through somehow, thats where the came from at the end of the JSON array.
Step 2. I had to remove an additional echo from the php file which I was using for error checking. I think I'm right in saying the only echo you want to have is the json_encode.
Step 3. Checked the length of JSONarray in Java and it was logging 5 so I now knew it was reading correctly.
Step 4. Added Pavneet_Singh's code snippet to replace stringList.add(jsonArray.get(i).toString());
with stringList.add(jsonArray.getJSONObject(i).optString("advert_File_Path"));
Step 5. Converted stringList from an ArrayList to a String array using this code stringArray = new String[stringList.size()];
stringArray = stringList.toArray(stringArray);
Happy days! I've spent far too long trying to figure this out haha the first two steps are pretty noobish but it's sometimes the most obvious things in life that are the hardest to find. Hopefully this can help someone else too!
use stringList.add(jsonArray.getString(i));
Try this:
JSONArray arr = new JSONArray(putNesessaryValueHere);
List<String> list = new ArrayList<String>();
for(int i = 0; i < arr.length(); i++){
list.add(arr.getJSONObject(i).getString("name"));
}
Try this :
ArrayList<String> arrayList = new ArrayList<String>();
JSONArray jsonArray = new JSONArray();
for(int i = 0, count = jsonArray.length(); i< count; i++)
{
try {
JSONObject jsonObject = jsonArray.getJSONObject(i);
arrayList.add(jsonObject.toString());
}
catch (JSONException e) {
e.printStackTrace();
}
}
Could someone please explain or correct as to why I am getting a null pointer exception in my Async Class? I am trying to get data from a URL but get a null pointer exception for the 162, which contains the following code
int lengthJsonArr = jsonMainNode.length();
I am not sure as to why that is but if someone could help that would be great. or if someone can show me a better alternative to fetch json data from url that would also be a great help.
public class userTask extends AsyncTask<String, Void, Void>{
HttpURLConnection connection = null;
private String Content;
#Override
protected Void doInBackground(String... urls) {
BufferedReader reader = null;
try {
URL url = new URL(urls[0]);
connection = (HttpURLConnection) url.openConnection();
connection.connect();
InputStream stream = connection.getInputStream();
reader = new BufferedReader(new InputStreamReader(stream));
StringBuffer buffer = new StringBuffer();
String line = "";
while ((line = reader.readLine()) != null) {
buffer.append(line);
} Content = buffer.toString();
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
if (connection != null) {
connection.disconnect();
}
try {
if (reader != null) {
reader.close();
}
} catch (IOException e) {
e.printStackTrace();
}
}
return null;
}
#Override
protected void onPostExecute(Void s) {
super.onPostExecute(s);
String OutputData = "";
JSONObject jsonResponse;
try {
jsonResponse = new JSONObject(Content);
JSONArray jsonMainNode = jsonResponse.optJSONArray("Android");
int lengthJsonArr = jsonMainNode.length(); //This is causing the exception
for (int i =0; i < lengthJsonArr; i++) {
JSONObject jsonChildNode = jsonMainNode.getJSONObject(i);
String name = jsonChildNode.optString("name").toString();
Double longitude = jsonChildNode.optDouble("lon");
Double latitude = jsonChildNode.optDouble("lat");
OutputData += " Name : "+ name +" "
+ "Longitude : "+ longitude +" "
+ "Latitude : "+ latitude +" "
+"-------------------------------------------------- ";
//Show Parsed Output on screen (activity)
Toast toast = Toast.makeText(getApplicationContext(), OutputData, Toast.LENGTH_LONG);
toast.show();
}
} catch (JSONException e) {
e.printStackTrace();
}
}
}
This is not a good way to fetch JSON data in android. You should use Volley or Retrofit library. These libraries will work accuratly and efficiently than normal code.
There are alot of things to take care of while fetching data. All will be done by library. And you just need to write few lines of code.
You can follow many good tutorials on google.
As this works...
jsonResponse = new JSONObject(Content);
...you at least succesfully receive a HTTP response which contains a valid JSON object.
The next line...
JSONArray jsonMainNode = jsonResponse.optJSONArray("Android");
...tries to extract a JSON array, but apparently fails and as a result your jsonMainNode variable is null. That is how optJSONArray() works. It just returns null if it does not find what was asked for. (Instead of throwing a JSONException for example.)
Then the next line...
int lengthJsonArr = jsonMainNode.length();
...of course fails because you can't get the length of a null JSON array.
So it looks like the JSON you receive does not include an array called "Android". You could/should place a breakpoint on...
JSONArray jsonMainNode = jsonResponse.optJSONArray("Android");
...and check what's in the JSON object. Or just print out the response. (And properly name it "content" with lowercase so people won't nag about the Java coding convention...)
As for avoiding the NullPointerException you could use code like:
if (jsonResponse.has("Android")) {
JSONArray jsonMainNode = jsonResponse.optJSONArray("Android");
int lengthJsonArr = jsonMainNode.length();
// Etc.
// ...
}
else {
// TODO: Recover from the situation.
// ...
}
I was trying to get an JSONObject from a HTTP response.
try
{
GetMethod postMethod = new GetMethod();
postMethod.setURI(new URI(url, true));
postMethod.setRequestHeader("Accept", "application/json");
httpClient.executeMethod(postMethod);
String resp=postMethod.getResponseBodyAsString();
org.json.JSONTokener tokener = new org.json.JSONTokener(resp);
finalResult = new org.json.JSONArray(tokener);
return finalResult;
}
But I got a runtime warning as
Going to buffer response body of large or unknown size. Using getResponseBodyAsStream instead is recommended.
Should I get the response as stream as suggested by the JVM ? If so, how could I parse the JSON from it ?
Has your server been set up to inform clients how big its responses are? If not, your server is streaming the data, and it's technically impossible to tell how much buffer space is required to deal with the response, warranting a warning that something potentially dangerous is going on.
if you want to send jsonObjects from server suppose (tomcat server)
For server side-
creating jsonobjects-
I have Called toJson() for creating jsonobjects this is the implementation-
final JSONObject arr = new JSONObject();
for (int i = 0; i < contactStatus.size(); i++) {
ContactStatus contactObject = contactStatus.get(i);
try {
arr.put(String.valueOf(i), toJson(value1, value2,, value3));
} catch (JSONException e) {
catch block e.printStackTrace();
}
}
//Here we serialize the stream to a String.
final String output = arr.toString();
response.setContentLength(output.length());
out.print(output);//out is object of servlet output stream.
public static Object toJsonForContact(String value1, boolean value2, double value3) throws JSONException {
JSONObject contactObject = new JSONObject();
contactObject.put("id", id);
contactObject.put("status", value1);
contactObject.put("distance", value2);
contactObject.put("relation", value3);
return contactObject;
}
so your jsonobjects are ready for sending we write these objects to ServletoutputStream.
in client side-
while ((ReadResponses = in.readLine()) != null) {
Constants.Response_From_server = ReadResponses;
if (Constants.Response_From_server.startsWith("{")) {
ListOfContactStatus = new ArrayList<ContactStatus>();
ContactStatus contactStatusObject;
try {
JSONObject json = new JSONObject(Constants.Response_From_server);
for (int i = 0; i < json.length(); i++) {
contactStatusObject = new ContactStatus();
JSONObject json1 = json.getJSONObject(String.valueOf(i));
System.out.println("" + json1.getString("id"));
System.out.println("" + json1.getBoolean("status"));
System.out.println("" + json1.getDouble("distance"));
contactStatusObject.setId(json1.getString("id"));
contactStatusObject.setStatus(json1.getBoolean("status"));
contactStatusObject.setDistance((float) json1.getDouble("distance"));
ListOfContactStatus.add(contactStatusObject);
System.out.println("HTTPTransport:sendMessage Size of ListOfContactStatus" + ListOfContactStatus.size());
}
} catch (JSONException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
}
You can easily generate JSonObject usin Java EE 7. The sample code.
JsonReader reader = Json.createReader(new URI(url, true));
JsonObject jsonObject=reader.readObject();
For details information go through to the link.
http://docs.oracle.com/javaee/7/tutorial/doc/jsonp003.htm#BABHAHIA
If I have a servlet running JVM1.4.2, and it is receiving a POST request with form data fields. I use req.getParameterNames() to get, what I would expect, all the query string and form data. However, all I ever get are the querystring parameters.
Literature I am reading from various sources says that getParameterNames() and getParameterValues(String) should be the way to get all query string and posted form data sent by the browser for JDK 1.4. Here is the method I use to extract all the parameters, which I expect would include posted form data :
public Map getParameterMap(HttpServletRequest req) {
Map params= new HashMap();
String name = null;
System.out.println("<< Getting Parameter Map.>>");
Enumeration enumParams = req.getParameterNames();
for (; enumParams.hasMoreElements(); ) {
// Get the name of the request parameter
name = (String)enumParams.nextElement();
// Get the value of the request parameters
// If the request parameter can appear more than once
// in the query string, get all values
String[] values = req.getParameterValues(name);
params.put(name, values);
String sValues = "";
for(int i=0;i<values.length;i++){
if(0<i) {
sValues+=",";
}
sValues +=values[i];
}
System.out.println("Param " + name + ": " + sValues);
}
System.out.println("<< END >>");
return params;
}
This question also agrees with my expectations, but the servlet is not picking up the form data. Obviously I am missing something....
Update: The post data is very straight forward and is not a Multipart form or rich media. Just plain'ol text submitted via an AJAX POST that looks like this in post body
c1=Value%20A&c2=Value%20B&c3=Value%20C
I managed to identify the problem. Because there is so much chatter from JDK 1.5+ and talk of getParameterMaps() method for 1.5, info on how 1.4 handles form post data was scarce and ambiguous. (Please post a comment if you find something that is specific for 1.4).
Pre-1.5 you have to manually get the form data via getInputStream, and then parse it out. I found this method, (posted below), from the java sun site that does a nice job using a Hashtable. I had to make a minor mod for deprecated methods. But seems to work quite robustly, "out of the box", so you should able to just cut-n-paste. I know it's "old tech" but I thought it worthwhile for those who may be in the same situation as me who are stuck on solving (what seems to be) straight forward problems.
public Hashtable parsePostData(int length, ServletInputStream instream) {
String valArray[] = null;
int inputLen, offset;
byte[] postedBytes = null;
boolean dataRemaining=true;
String postedBody;
Hashtable ht = new Hashtable();
//Vector paramOrder = new Vector(10);
StringBuffer sb = new StringBuffer();
if (length <=0) {
return null;
}
postedBytes = new byte[length];
try {
offset = 0;
while(dataRemaining) {
inputLen = instream.read (postedBytes, offset, length - offset);
if (inputLen <= 0) {
throw new IOException ("read error");
}
offset += inputLen;
if((length-offset) ==0) {
dataRemaining=false;
}
}
} catch (IOException e) {
System.out.println("Exception ="+e);
return null;
}
postedBody = new String (postedBytes);
StringTokenizer st = new StringTokenizer(postedBody, "&");
String key=null;
String val=null;
while (st.hasMoreTokens()) {
String pair = (String)st.nextToken();
int pos = pair.indexOf('=');
if (pos == -1) {
throw new IllegalArgumentException();
}
try {
key = URLDecoder.decode(pair.substring(0, pos),"UTF8");
val = java.net.URLDecoder.decode(pair.substring(pos+1,pair.length()),"UTF8");
} catch (Exception e) {
throw new IllegalArgumentException();
}
if (ht.containsKey(key)) {
String oldVals[] = (String []) ht.get(key);
valArray = new String[oldVals.length + 1];
for (int i = 0; i < oldVals.length; i++) {
valArray[i] = oldVals[i];
}
valArray[oldVals.length] = val;
} else {
valArray = new String[1];
valArray[0] = val;
}
ht.put(key, valArray);
String sValues = "";
for(int i=0;i<valArray.length;i++) {
if (0<i) {
sValues+=",";
}
sValues = valArray[i];
}
System.out.println("Form data field " + key + ":" +sValues);
//paramOrder.addElement(key);
}
return ht;
}
That's true. The getParameterNames(), getParameterValues(), and getParameter() methods are the way to access form data unless it's a multipart form, in which case you'll have to use something like Commons Fileupload to parse the multipart request before all the parameters are accessible to you.
Edit: You're probably not encoding the POST data properly in your AJAX call. POST data must carry a Content-Type of application/x-www-form-urlencoded or else multipart/form-data. If you're sending it as something else, it doesn't qualify as a request parameter, and I expect you'd see the behavior you're describing. The solution you've engineered essentially consists of setting up custom parsing of custom content.