I've been researching on how to send and receive information to a url, via json for the last 3 days. I have found a lot of documentation and code examples on how to do it, I just can't comprehend what they're saying. I've imported god knows how many .jar files into my eclipse package. Does anyone have a good example on how to connect to a url, send/receive information (even login), parse it, and send more information? I understand that I'm asking for a lot. I don't need all the answers, good documentation and some good examples would make me soooo happy.
Start with http://hc.apache.org/
Then look at http://code.google.com/p/google-gson/
or: http://wiki.fasterxml.com/JacksonHome
That should be all you need.
Found a really solid example here on this blog http://www.gnovus.com/blog/programming/making-http-post-request-json-using-apaches-httpclient
Pasted below if for some reason the link doesnt work.
public class SimpleHTTPPOSTRequester {
private String apiusername;
private String apipassword;
private String apiURL;
public SimpleHTTPPOSTRequester(String apiusername, String apipassword, String apiURL) {
this.apiURL = apiURL;
this.apiusername = apiusername;
this.apipassword = apipassword;
}
public void makeHTTPPOSTRequest() {
try {
HttpClient c = new DefaultHttpClient();
HttpPost p = new HttpPost(this.apiURL);
p.setEntity(new StringEntity("{\"username\":\"" + this.apiusername + "\",\"password\":\"" + this.apipassword + "\"}",
ContentType.create("application/json")));
HttpResponse r = c.execute(p);
BufferedReader rd = new BufferedReader(new InputStreamReader(r.getEntity().getContent()));
String line = "";
while ((line = rd.readLine()) != null) {
//Parse our JSON response
JSONParser j = new JSONParser();
JSONObject o = (JSONObject)j.parse(line);
Map response = (Map)o.get("response");
System.out.println(response.get("somevalue"));
}
}
catch(ParseException e) {
System.out.println(e);
}
catch(IOException e) {
System.out.println(e);
}
}
}
Related
First, some background :-
I'm trying to solve a question asked by an interviewer recently. I had to write a code and use below URL to return JSON response -
https://losangeles.craigslist.org/
This is what I did :-
1) I created a webclient and made HTTPURL Request to fetch an HTTP Response.
public static JSONArray getSearchResults(String arg) {
JSONArray jsonArray = null;
try {
QueryString qs = new QueryString("query", arg);
URL url = new URL("https://toronto.craigslist.ca/search?"+qs);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("GET");
conn.setRequestProperty("Accept", "application/text");
if (conn.getResponseCode() != 200) {
throw new RuntimeException("Failed : HTTP error code : "
+ conn.getResponseCode());
}
BufferedReader br = new BufferedReader(new InputStreamReader(
(conn.getInputStream())));
String readAPIResponse = " ";
StringBuilder output = new StringBuilder();
while ((readAPIResponse = br.readLine()) != null) {
output.append(readAPIResponse);
}
jsonArray = convertToJson(output);
System.out.println(" JSON response : "+jsonArray.toString(2));
conn.disconnect();
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
return jsonArray;
}
2) Below was my function to convert the response into JSON :-
public static JSONArray convertToJson(StringBuilder response) {
JSONArray jsonArr = new JSONArray();
if (response != null) {
try {
Document document = Jsoup.parse(response.toString());
Elements resultRows = document.getElementsByClass("result-row");
JSONObject jsonObj;
for (int i = 0; i < resultRows.size(); i++) {
jsonObj = new JSONObject();
Element e = resultRows.get(i);
Elements resultsDate = e.getElementsByClass("result-date");
Elements resultsTitle = e.getElementsByClass("result-title hdrlnk");
String key1 = "date";
String value1 = resultsDate.get(0).text();
jsonObj.put(key1, value1);
String key2 = "title";
String value2 = resultsTitle.get(0).text();
jsonObj.put(key2, value2);
jsonArr.put(i, jsonObj);
}
} catch (JSONException e) {
e.printStackTrace();
}
}
return jsonArr;
}
The response I received was the whole HTML page(I used postman to make requests). Since, I only had few hours to solve this question and was not sure how to parse an entire HTML, I ended up using a third party library, called JSoup. I was not 100% happy about it, but ended up having no other option.
I have not heard back from them and I am curious if this was the worst approach and if yes, what could be better options? They did not mention anything about what technology I could use. But,since the skill set I was interviewing involved Java/J2EE I was thinking to implement this in Java (Not using Node js though)
Thanks!
If you only need an XML Parser which is obviously the base of HTML this is built in in the JRE core API.
Even in the SE Version the needed packages to parse exist:
import org.w3c.dom.*;
import javax.xml.parsers.*;
import java.io.*;
Take a look at these classes they are the most important to parse or create an XML/HTML File
DocumentBuilderFactory
DocumentBuilder
Document
and here simple example for HTML
String text = "<html><head>HEAD</head><body>BODY</body>";
ByteArrayInputStream input = new ByteArrayInputStream(text.getBytes("UTF-8"));
Document doc = builder.parse(input);
I am trying to shorten the time it takes HttpWebRequest or WebClient to get a string from url,using C#, it takes about 2000ms to get the string.
Using Java I can get the string in about 300ms. (I am new to java, please see code below)
In c# I have tried setting request.Proxy = null and System.Net.ServicePointManager.Expect100Continue = false with no clear difference.
I don't know if C# and Java codes below are comparable, However I would like to get the data in a shorter time if possible using C#.
Java:
try {
URL url = new URL("SomeURL");
InputStream is = url.openStream();
BufferedReader br = new BufferedReader(new InputStreamReader(is));
String line;
while ((line = br.readLine()) != null)
br.close();
is.close();
} catch (Exception e) {
e.printStackTrace();
}
C#:
using (WebClient nn = new WebClient()) {
nn.Proxy = null;
string SContent = await nn.DownloadStringTaskAsync(url);
return SContent;
}
or:
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(new Uri(url));
request.Method = "GET";
// Send the request to the server and wait for the response:
using (WebResponse response = await request.GetResponseAsync()) {
using (Stream stream = response.GetResponseStream()) {
StreamReader reader = new StreamReader(stream, Encoding.UTF8);
string SContent = reader.ReadToEnd();
return SContent;
}
}
I'm not sure if the code below will be faster than Java's URL.openStream or URLConnection, but it sure is succint. I wouldn't use HttpWebRequest anymore. Microsoft recommends using HttpClient.
using System.Net.Http;
using System.Threading.Tasks;
namespace CSharp.Console
{
public class Program
{
// Make HttpClient a static member so it's available for the lifetime of the application.
private static readonly HttpClient HttpClient = new HttpClient();
public static async Task Main(string[] args)
{
string body = await HttpClient.GetStringAsync("http://www.google.com");
System.Console.WriteLine(body);
System.Console.ReadLine();
}
}
}
Please note: to be able to use async on your Main method in a Console Application, you need to use C# language specification 7.1 or up. (Project properties, Debug, Advanced, Language version).
Could someone please explain or correct as to why I am getting a null pointer exception in my Async Class? I am trying to get data from a URL but get a null pointer exception for the 162, which contains the following code
int lengthJsonArr = jsonMainNode.length();
I am not sure as to why that is but if someone could help that would be great. or if someone can show me a better alternative to fetch json data from url that would also be a great help.
public class userTask extends AsyncTask<String, Void, Void>{
HttpURLConnection connection = null;
private String Content;
#Override
protected Void doInBackground(String... urls) {
BufferedReader reader = null;
try {
URL url = new URL(urls[0]);
connection = (HttpURLConnection) url.openConnection();
connection.connect();
InputStream stream = connection.getInputStream();
reader = new BufferedReader(new InputStreamReader(stream));
StringBuffer buffer = new StringBuffer();
String line = "";
while ((line = reader.readLine()) != null) {
buffer.append(line);
} Content = buffer.toString();
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
if (connection != null) {
connection.disconnect();
}
try {
if (reader != null) {
reader.close();
}
} catch (IOException e) {
e.printStackTrace();
}
}
return null;
}
#Override
protected void onPostExecute(Void s) {
super.onPostExecute(s);
String OutputData = "";
JSONObject jsonResponse;
try {
jsonResponse = new JSONObject(Content);
JSONArray jsonMainNode = jsonResponse.optJSONArray("Android");
int lengthJsonArr = jsonMainNode.length(); //This is causing the exception
for (int i =0; i < lengthJsonArr; i++) {
JSONObject jsonChildNode = jsonMainNode.getJSONObject(i);
String name = jsonChildNode.optString("name").toString();
Double longitude = jsonChildNode.optDouble("lon");
Double latitude = jsonChildNode.optDouble("lat");
OutputData += " Name : "+ name +" "
+ "Longitude : "+ longitude +" "
+ "Latitude : "+ latitude +" "
+"-------------------------------------------------- ";
//Show Parsed Output on screen (activity)
Toast toast = Toast.makeText(getApplicationContext(), OutputData, Toast.LENGTH_LONG);
toast.show();
}
} catch (JSONException e) {
e.printStackTrace();
}
}
}
This is not a good way to fetch JSON data in android. You should use Volley or Retrofit library. These libraries will work accuratly and efficiently than normal code.
There are alot of things to take care of while fetching data. All will be done by library. And you just need to write few lines of code.
You can follow many good tutorials on google.
As this works...
jsonResponse = new JSONObject(Content);
...you at least succesfully receive a HTTP response which contains a valid JSON object.
The next line...
JSONArray jsonMainNode = jsonResponse.optJSONArray("Android");
...tries to extract a JSON array, but apparently fails and as a result your jsonMainNode variable is null. That is how optJSONArray() works. It just returns null if it does not find what was asked for. (Instead of throwing a JSONException for example.)
Then the next line...
int lengthJsonArr = jsonMainNode.length();
...of course fails because you can't get the length of a null JSON array.
So it looks like the JSON you receive does not include an array called "Android". You could/should place a breakpoint on...
JSONArray jsonMainNode = jsonResponse.optJSONArray("Android");
...and check what's in the JSON object. Or just print out the response. (And properly name it "content" with lowercase so people won't nag about the Java coding convention...)
As for avoiding the NullPointerException you could use code like:
if (jsonResponse.has("Android")) {
JSONArray jsonMainNode = jsonResponse.optJSONArray("Android");
int lengthJsonArr = jsonMainNode.length();
// Etc.
// ...
}
else {
// TODO: Recover from the situation.
// ...
}
I'm a building a basic program to query Target's API with a store ID and Product ID which returns the aisle location. I think I'm using the URL constructor incorrectly, however (I've had trouble with it in the past and still don't fully understand them). Below is the code I have, redacted the API Key for obvious reasons. The URL I create is valid when put into a browser and no exceptions are thrown but at the the end when I print out the contents of the page it is null. What am I missing? Any help is really appreciated!
package productVerf;
import java.net.*;
import java.io.*;
public class Verify {
public static void main(String args[]) {
// first input is store id second input is product id
String productID = args[0];
String storeID = args[1];
String file = "/v2/products/storeLocations?productId=" + productID
+ "&storeId=" + storeID
+ "&storeId=694&key=REDACTED";
URL locQuery;
URLConnection lqConection = null;
try {
locQuery = new URL("http", "api.target.com", file);
lqConection = locQuery.openConnection();
} catch (IOException e) {
e.printStackTrace();
}
BufferedReader response;
String responseString = "";
try {
response = new BufferedReader(new InputStreamReader(
lqConection.getInputStream()));
while (response.readLine() != null) {
responseString += response.readLine();
}
} catch (IOException e) {
e.printStackTrace();
}
System.out.println(responseString);
}
}
Maybe you are reading only even lines
you are reading a line twice? (in while statement...), it looks you reads the first line which is dropped in while condition test. If your response contains only one line, nothing will be readed
use this:
String line;
while ((line=response.readLine()) != null) {
responseString += line;
}
In the android app I get an xml or json string returned, However, I cant seem to figure out any way on how to get an value from the string in any way by entering an key.
In PHP you just use something like $myArray['parent']['child'] but I have no clue on how this works in java.
Any idea's would be greatly appreciated! (an example for both XML and JSON even more ;) )
Here's what I would do:
locate an XML/JSON library (there's tons) (google-gson for json)
read the documentation to find a parse method ((new JsonParser()).parse(text))
read the documentation to find out what the return value is (JsonElement)
decide what you want to do with the parsed data (myJsonObj.get(...))
write the code
public class parsingjsontest2 extends Activity {
/** Called when the activity is first created. */
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(main);
String str = connect("http://rentopoly.com/ajax.php?query=Bo"));
System.out.println("String::"+str);
}
}
private String connect(String url)
{
// Create the httpclient
HttpClient httpclient = new DefaultHttpClient();
// Prepare a request object
HttpGet httpget = new HttpGet(url);
// Execute the request
HttpResponse response;
// return string
String returnString = null;
try {
// Open the webpage.
response = httpclient.execute(httpget);
if(response.getStatusLine().getStatusCode() == 200){
// Connection was established. Get the content.
HttpEntity entity = response.getEntity();
// If the response does not enclose an entity, there is no need
// to worry about connection release
if (entity != null) {
// A Simple JSON Response Read
InputStream instream = entity.getContent();
// Load the requested page converted to a string into a JSONObject.
JSONObject myAwway = new JSONObject(convertStreamToString(instream));
// Get the query value'
String query = myAwway.getString("query");
**// Make array of the suggestions
JSONArray suggestions = myAwway.getJSONArray("suggestions");
// Build the return string.
returnString = "Found: " + suggestions.length() + " locations for " + query;
for (int i = 0; i < suggestions.length(); i++) {
returnString += "\n\t" + suggestions.getString(i);
}
// Cose the stream.
instream.close();
}
}
else {
// code here for a response othet than 200. A response 200 means the webpage was ok
// Other codes include 404 - not found, 301 - redirect etc...
// Display the response line.
returnString = "Unable to load page - " + response.getStatusLine();
}
}
catch (IOException ex) {
// thrown by line 80 - getContent();
// Connection was not established
returnString = "Connection failed; " + ex.getMessage();
}
catch (JSONException ex){
// JSON errors
returnString = "JSON failed; " + ex.getMessage();
}
return returnString;
}
private static String convertStreamToString(InputStream is) {
/*
* To convert the InputStream to String we use the BufferedReader.readLine()
* method. We iterate until the BufferedReader return null which means
* there's no more data to read. Each line will appended to a StringBuilder
* and returned as String.
*/
BufferedReader reader = new BufferedReader(new InputStreamReader(is));
StringBuilder sb = new StringBuilder();
String line = null;
try {
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
is.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return sb.toString();
}
}
As you didn't specify what kind of xml you are trying to read, I'm answering based on what I know.
In Android, if you were talking about the layout and strings.xml files, you use a dot (.) operator, like R.string.appname.
Please post more details about your specific problem, if this is not what you were looking for.