Jasper Reports: getting JSON data from a post rest service - java

I am trying to get JSON data from a rest service. I know this is pretty simple for a GET service where you only have to provide the URI and Jasper studio can pull the data but I want to do this for a post rest service that also consumes some JSON input.
Workflow will be something like:
Send userID in request header and some JSON parameters in request
body.
Get JSON data as output.
Use JSON data to build report.
I am new to Jasper and am using Jasper server 6 with Japser Studio 6 but I can't find any documentation to do something like this.
I would appreciate if anyone can point me in the right direction regarding this.
The closes thing I can find is this link. From there I get that I can create a constructor which will get the data from rest service but how do I serve it to the report? Also please note that the JSON object being retrieved here is a bit complex and will have at least 2 lists with any number of items.
EDIT:
Alright so my custom adapter is like this:
package CustomDataAdapter;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import net.sf.jasperreports.engine.JRDataSource;
import net.sf.jasperreports.engine.JRException;
import net.sf.jasperreports.engine.JRField;
import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.entity.StringEntity;
import org.apache.http.impl.client.HttpClientBuilder;
import org.json.JSONObject;
public class SearchAdapter implements JRDataSource {
/**
* This will hold the JSON returned by generic search service
*/
private JSONObject json = null;
/**
* Will create the object with data retrieved from service.
*/
public SearchAdapter() {
String url = "[URL is here]";
String request = "{searchType: \"TEST\", searchTxt: \"TEST\"}";
// Setting up post client and request.
HttpClient client = HttpClientBuilder.create().build();
HttpPost post = new HttpPost(url);
HttpResponse response = null;
post.setHeader("userId", "1000");
post.setHeader("Content-Type", "application/json");
// Setting up Request payload
HttpEntity entity = null;
try {
entity = new StringEntity(request);
post.setEntity(entity);
// do post
response = client.execute(post);
} catch (ClientProtocolException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
// Reading Server Response
try {
int statusCode = response.getStatusLine().getStatusCode();
if (statusCode != 200) {
throw new Exception("Search Failed");
}
BufferedReader in = new BufferedReader(new InputStreamReader(
response.getEntity().getContent()));
String inputLine;
StringBuffer resp = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
resp.append(inputLine);
}
in.close();
this.json = new JSONObject(resp.toString());
} catch (NullPointerException e) {
e.printStackTrace();
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
/*
* (non-Javadoc)
*
* #see
* net.sf.jasperreports.engine.JRDataSource#getFieldValue(net.sf.jasperreports
* .engine.JRField)
*/
public Object getFieldValue(JRField field) throws JRException {
// TODO Auto-generated method
// stubhttp://community-static.jaspersoft.com/sites/default/files/images/0.png
return this.json;
}
/*
* (non-Javadoc)
*
* #see net.sf.jasperreports.engine.JRDataSource#next()
*/
public boolean next() throws JRException {
return (this.json != null);
}
/**
* Return an instance of the class that implements the custom data adapter.
*/
public static JRDataSource getDataSource() {
return new SearchAdapter();
}
}
I am able to create an adapter and the Test Connection feature in Jasper Studio also returns true but I cant get it to read any of the fields in the JSON and auto-generate the report. I only get a blank document. FYI the JSON is something like:
{
"key": "value",
"key": "value",
"key": [list],
"key": [list]
}

Well, I feel stupid now but the solution was pretty easy. Turns out you cant just return a JSON object. You need to return the fields and manually add the fields in the report.
For record purposes my final code look like this:
package CustomDataAdapter;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import net.sf.jasperreports.engine.JRDataSource;
import net.sf.jasperreports.engine.JRException;
import net.sf.jasperreports.engine.JRField;
import org.apache.http.HttpResponse;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.entity.StringEntity;
import org.apache.http.impl.client.HttpClientBuilder;
import org.json.JSONException;
import org.json.JSONObject;
public class SearchAdapter implements JRDataSource {
/**
* This will hold the JSON returned by generic search service
*/
private JSONObject json = null;
/**
* Ensures that we infinitely calling the service.
*/
private boolean flag = false;
/**
* Will create the object with data retrieved from service.
*/
private void setJson() {
String url = "[URL is here]";
String request = "{\"searchType\": \"Test\", \"searchTxt\": \"Test\"}";
// Setting up post client and request.
HttpClient client = HttpClientBuilder.create().build();
HttpPost post = new HttpPost(url);
HttpResponse response = null;
post.setHeader("userId", "1000");
post.setHeader("Content-Type", "application/json");
// Setting up Request payload
StringEntity entity = null;
try {
entity = new StringEntity(request);
post.setEntity(entity);
// do post
response = client.execute(post);
} catch (ClientProtocolException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
// Reading Server Response
try {
int statusCode = response.getStatusLine().getStatusCode();
if (statusCode != 200) {
// Thrown Exception in case things go wrong
BufferedReader in = new BufferedReader(new InputStreamReader(
response.getEntity().getContent()));
String inputLine;
StringBuffer resp = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
resp.append(inputLine);
}
in.close();
String ex = "Search Failed. Status Code: " + statusCode;
ex += "\n Error: " + resp.toString();
throw new Exception(ex);
}
BufferedReader in = new BufferedReader(new InputStreamReader(
response.getEntity().getContent()));
String inputLine;
StringBuffer resp = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
resp.append(inputLine);
}
in.close();
this.json = new JSONObject(resp.toString());
} catch (NullPointerException e) {
e.printStackTrace();
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
/*
* (non-Javadoc)
*
* #see
* net.sf.jasperreports.engine.JRDataSource#getFieldValue(net.sf.jasperreports
* .engine.JRField)
*/
#Override
public Object getFieldValue(JRField field) throws JRException {
// TODO Auto-generated method
// stubhttp://community-static.jaspersoft.com/sites/default/files/images/0.png
try {
return this.json.get(field.getName());
} catch (JSONException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return null;
}
/*
* (non-Javadoc)
*
* #see net.sf.jasperreports.engine.JRDataSource#next()
*/
#Override
public boolean next() throws JRException {
if (this.json != null && !flag) {
flag = true;
return true;
} else {
return false;
}
}
/**
* Return an instance of the class that implements the custom data adapter.
*/
public static JRDataSource getDataSource() {
SearchAdapter adapter = new SearchAdapter();
adapter.setJson();
return adapter;
}
}

Related

I am getting Error parsing data org.json.JSONException

I am getting this error in my LogCat:
Error parsing data org.json.JSONException: Value  of type java.lang.String cannot be converted to JSONArray
Below are every file I could show you! Please let me know the problem and its solution ASAP. What I guess is:
1. Maybe the is problem is with parsing data in JSON array.
2. Maybe the problem is with my php api, I think I am not properly encoding the json_encode because it gives me RAW JSON, like every thing in one line.
as below
[{"uid":"120","name":"MyFirstName MyLastName"}]
Please also let me know, their is some difference in working of both format, 1. Raw JSON and 2. Intented Json
below is the intented json format
[
{
"uid":"120",
"name":"MyFirstName MyLastName"
}
]
Here is the JSONUseActivity.java
package com.example.oncemore;
import java.util.ArrayList;
import org.apache.http.NameValuePair;
import org.apache.http.message.BasicNameValuePair;
import org.json.JSONArray;
import org.json.JSONException;
import org.json.JSONObject;
import android.app.Activity;
import android.os.Bundle;
import android.os.StrictMode;
import com.example.oncemore.CustomHttpClient;
import android.util.Log;
import android.view.View;
import android.widget.Button;
import android.widget.EditText;
import android.widget.TextView;
public class JSONUseActivity extends Activity {
EditText email,password;
Button submit;
TextView tv; // TextView to show the result of MySQL query
String returnString; // to store the result of MySQL query after decoding
// JSON
/** Called when the activity is first created. */
#Override
public void onCreate(Bundle savedInstanceState) {
StrictMode.setThreadPolicy(new StrictMode.ThreadPolicy.Builder()
.detectDiskReads().detectDiskWrites().detectNetwork() // StrictMode is
// most commonly
// used to catch
// accidental
// disk or
// network
// access on the
// application's
// main thread
.penaltyLog().build());
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_jsonuse);
email = (EditText) findViewById(R.id.email);
password = (EditText) findViewById(R.id.password);
submit = (Button) findViewById(R.id.submitbutton);
tv = (TextView) findViewById(R.id.showresult);
// define the action when user clicks on submit button
submit.setOnClickListener(new View.OnClickListener() {
public void onClick(View v) {
// declare parameters that are passed to PHP script i.e. the
ArrayList<NameValuePair> postParameters = new ArrayList<NameValuePair>();
// define the parameter
postParameters.add(new BasicNameValuePair("email",email.getText().toString()));
postParameters.add(new BasicNameValuePair("password",password.getText().toString()));
String response = null;
// call executeHttpPost method passing necessary parameters
try {
response = CustomHttpClient.executeHttpPost(
"http://mywebsite.com/android/api.php",
postParameters);
// store the result returned by PHP script that runs MySQL
// query
String result = response.toString();
// parse json data
try {
returnString = "";
//I think the line below is creating some problem
JSONArray jArray = new JSONArray(result);
for (int i = 0; i < jArray.length(); i++) {
JSONObject json_data = jArray.getJSONObject(i);
Log.i("log_tag",
"id: " + json_data.getInt("uid")+", name: " + json_data.getString("name"));
// Get an output to the screen
returnString += "\n" + json_data.getString("name")
+ " -> " + json_data.getInt("uid");
}
}catch (JSONException e) {
Log.e("log_tag", "Error parsing data " + e.toString());
}
try {
tv.setText(returnString);
}
catch (Exception e) {
Log.e("log_tag", "Error in Display!" + e.toString());
;
}
}
catch (Exception e) {
Log.e("log_tag",
"Error in http connection!!" + e.toString());
}
}
});
}
}
Here is the CustomHttpClient.java
package com.example.oncemore;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.URI;
import java.util.ArrayList;
import org.apache.http.HttpResponse;
import org.apache.http.NameValuePair;
import org.apache.http.client.HttpClient;
import org.apache.http.client.entity.UrlEncodedFormEntity;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.conn.params.ConnManagerParams;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.params.HttpConnectionParams;
import org.apache.http.params.HttpParams;
import android.util.Log;
public class CustomHttpClient {
/** The time it takes for our client to timeout */
public static final int HTTP_TIMEOUT = 30 * 1000; // milliseconds
/** Single instance of our HttpClient */
private static HttpClient mHttpClient;
/**
* Get our single instance of our HttpClient object.
*
* #return an HttpClient object with connection parameters set
*/
private static HttpClient getHttpClient() {
if (mHttpClient == null) {
mHttpClient = new DefaultHttpClient();
final HttpParams params = mHttpClient.getParams();
HttpConnectionParams.setConnectionTimeout(params, HTTP_TIMEOUT);
HttpConnectionParams.setSoTimeout(params, HTTP_TIMEOUT);
ConnManagerParams.setTimeout(params, HTTP_TIMEOUT);
}
return mHttpClient;
}
/**
* Performs an HTTP Post request to the specified url with the specified
* parameters.
*
* #param url
* The web address to post the request to
* #param postParameters
* The parameters to send via the request
* #return The result of the request
* #throws Exception
*/
public static String executeHttpPost(String url, ArrayList<NameValuePair> postParameters) throws Exception {
BufferedReader in = null;
try {
HttpClient client = getHttpClient();
HttpPost request = new HttpPost(url);
UrlEncodedFormEntity formEntity = new UrlEncodedFormEntity(postParameters);
request.setEntity(formEntity);
HttpResponse response = client.execute(request);
in = new BufferedReader(new InputStreamReader(response.getEntity().getContent()));
StringBuffer sb = new StringBuffer("");
String line = "";
String NL = System.getProperty("line.separator");
while ((line = in.readLine()) != null) {
sb.append(line + NL);
}
in.close();
String result = sb.toString();
return result;
} finally {
if (in != null) {
try {
in.close();
} catch (IOException e) {
Log.e("log_tag", "Error converting result "+e.toString());
e.printStackTrace();
}
}
}
}
/**
* Performs an HTTP GET request to the specified url.
*
* #param url
* The web address to post the request to
* #return The result of the request
* #throws Exception
*/
public static String executeHttpGet(String url) throws Exception {
BufferedReader in = null;
try {
HttpClient client = getHttpClient();
HttpGet request = new HttpGet();
request.setURI(new URI(url));
HttpResponse response = client.execute(request);
in = new BufferedReader(new InputStreamReader(response.getEntity()
.getContent()));
StringBuffer sb = new StringBuffer("");
String line = "";
String NL = System.getProperty("line.separator");
while ((line = in.readLine()) != null) {
sb.append(line + NL);
}
in.close();
String result = sb.toString();
return result;
} finally {
if (in != null) {
try {
in.close();
} catch (IOException e) {
Log.e("log_tag", "Error converting result "+e.toString());
e.printStackTrace();
}
}
}
}
}
Here is the api.php
<?php
require_once("../contactdb.php");
$myusername=$_REQUEST["email"];
$mypassword=$_REQUEST["password"];
// To protect MySQL injection (more detail about MySQL injection)
$myusername = stripslashes($myusername);
$mypassword = stripslashes($mypassword);
$myusername = mysql_real_escape_string($myusername);
$mypassword = mysql_real_escape_string($mypassword);
$sql="SELECT uid,name FROM u_info WHERE email='".$myusername."' AND password ='".$mypassword."'";
$result=mysql_query($sql);
// Mysql_num_row is counting table row
$count=mysql_num_rows($result);
if($count==1){
while($row=mysql_fetch_assoc($result))
$output[]=$row;
echo json_encode($output);
mysql_close();
}else{
echo "Error Occured!";
}
?>
Finally, When I goto browser and write like this
http://mywebsite.com/android/api.php?email=myname#yahoo.com&password=1234
I got this json array!
[{"uid":"120","name":"MyFirstName MyLastName"}]
So Far I google, I have found different formats of json array! I found everywhere Intented Json. My json array is currently in Raw Json format. I don't find anywhere how to convert Raw Json format into Intented Json format.
Thanks in advance guys!
Any help would be appreciated! If possible, please provide the correct code!
That is NOT valid JSON syntax:
{
"employees": [
{ "firstName":"John" , "lastName":"Doe" },
{ "firstName":"Anna" , "lastName":"Smith" },
{ "firstName":"Peter" , "lastName":"Jones" }
]
}
Is Valid.
Note: This is also valid:
{"employees": [ { "firstName":"John" , "lastName":"Doe" }, { "firstName":"Anna" , "lastName":"Smith" }, { "firstName":"Peter" , "lastName":"Jones" } ] }
The syntax structure is the important part, not the formatting in terms of indentation.
As otherwise said, to use the fomat you're returning, you need to cut the substring from the response, i.e get rid of the square brackets surrounding the braces.
In PHP I create a proper json response as follows:
// array for JSON response
$response = array();
$response["apps"] = array();
$apps = array();
$apps["name"] = $row["name"];
$apps["package"] = $row["package"];
$apps["version"] = $row["version"];
$apps["dateversion"] = $row["dateversion"];
array_push($response["apps"], $apps);
$response["success"] = 1;
echo json_encode($response);
This basically gives
{ "success":"1", "apps":{["name":"NAME", "package":"PACKAGE", "version":"VERSION", "dateversion":"DATEVERSION"]}}
which can be parsed correctly by any of the abundant examples of JSON classes which you can make use of. Hacking and using substring to manually remove the first N characters is NOT good practice...

java url connection set cookie

I followed Ian Brown's tutorial to set a cookie to a request http://www.hccp.org/java-net-cookie-how-to.html
but it don't works:
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLConnection;
public class cookie {
/**
* #param args
*/
public static void main(String[] args) {
// TODO Auto-generated method stub
URL myUrl = null;
try {
myUrl = new URL("http://server/test.php?hlp");
} catch (MalformedURLException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
try {
URLConnection con = myUrl.openConnection();
con.setRequestProperty("Cookie", "accesstoken=WERT-DES-COOKIES");
con.connect();
String line;
StringBuilder builder = new StringBuilder();
BufferedReader reader = new BufferedReader(new InputStreamReader(con.getInputStream()));
while((line = reader.readLine()) != null) {
builder.append(line);
}
System.out.println(builder);
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
the php-test-Script returns the $_REQUEST-values... but I only get the "hlp"-Parameter given in the URL. Can somebody tell me whats wrong?!
You mentioned that you are using $_REQUEST to retrieve the cookies. Please note that $_REQUEST will retrieve only request parameters either passed as query string or as POST request. To retrieve cookie use $_COOKIE associative array. Check this tutorial.

Android, Web Services, AsyncTask please help a noob/make sure im starting right

Ive made edits since the orignal post Im making an android app that connects to an JSON API. Everything so far works ok except if there is a delay from the server. The UI of course can stop responding if it takes too long. Ive read that asynctask can solve my problem. Ive had a really ard time with the examples though.
Here is the restclient class making the http calls... parsing the json and storing custom objects to a public list that my other class can access.
package com.bde.dgcr;
import android.app.Activity;
import android.content.Context;
import android.os.AsyncTask;
import android.util.Log;
import android.view.View;
import android.widget.ListView;
import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.DefaultHttpClient;
import org.json.JSONArray;
import org.json.JSONException;
import org.json.JSONObject;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.util.ArrayList;
import java.util.List;
public class RestClient {
static List<ResponseHolder> list = new ArrayList<ResponseHolder>();
protected Context context = this.context;
private static String convertStreamToString(InputStream is) {
/*
* To convert the InputStream to String we use the BufferedReader.readLine()
* method. We iterate until the BufferedReader return null which means
* there's no more data to read. Each line will appended to a StringBuilder
* and returned as String.
*/
BufferedReader reader = new BufferedReader(new InputStreamReader(is));
StringBuilder sb = new StringBuilder();
String line = null;
try {
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
is.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return sb.toString();
}
/* This is a test function which will connects to a given
* rest service and prints it's response to Android Log with
* labels "Praeda".
*/
public void connect(String url) {
AsyncTask<String, Void, Void> connection = new AsyncTask<String, Void, Void>() {
protected Context context;
#Override
protected Void doInBackground(String... params) {
// Prepare a request object
HttpGet httpget = new HttpGet(params[0]);
// Execute the request
HttpResponse response;
HttpClient httpclient = new DefaultHttpClient();
try {
list.clear();
response = httpclient.execute(httpget);
// Examine the response status
Log.i("Praeda", response.getStatusLine().toString());
// Get hold of the response entity
HttpEntity entity = response.getEntity();
// If the response does not enclose an entity, there is no need
// to worry about connection release
if (entity != null) {
// A Simple JSON Response Read
InputStream instream = entity.getContent();
String result = convertStreamToString(instream);
// A Simple JSONObject Creation
//JSONObject json= new JSONObject(result);
JSONArray jsonArray = new JSONArray(result);
// A Simple JSONObject Parsing
for (int i = 0; i < (jsonArray.length()); i++) {
JSONObject json_obj = jsonArray.getJSONObject(i);
ResponseHolder rh = new ResponseHolder(json_obj);
list.add(rh);
}
instream.close();
}
} catch (ClientProtocolException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (JSONException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return null;
}
#Override
protected void onPostExecute(Void aVoid) {
ListView listView1;
super.onPostExecute(aVoid); //To change body of overridden methods use File | Settings | File Templates.
ResponseHolderAdapter adapter = new ResponseHolderAdapter(context, R.layout.listview_item_row, RestClient.list);
listView1 = (ListView) findViewById(R.id.listView1);
View header = (View) getLayoutInflater().inflate(R.layout.listview_header_row, null);
listView1.addHeaderView(header);
listView1.setAdapter(adapter);
}
};
connection.execute(url);
}
}
Here is the class calling the static connect method and using the list for an adapter to go in a list view.
public class JsonGrabber extends Activity {
private final String API_KEY = "key";
private final String SECRET = "secret";
private String state;
private String city;
private String country;
private static String mode;
private String md5;
private String url;
CourseSearch cs;
private ListView listView1;
/**
* Called when the activity is first created.
*/
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.results);
Bundle extras = getIntent().getExtras();
state = extras.getString("state");
city = URLEncoder.encode(extras.getString("city"));
country = extras.getString("country");
mode = extras.getString("mode");
md5 = MD5.getMD5(API_KEY + SECRET + mode);
System.out.println(md5);
url = "http://www.api.com/?key=" + API_KEY + "&mode=" + mode + "&id=1&sig=" + md5;
String findByLocUrl = "http://www.api.com/?key=" + API_KEY + "&mode=" + mode + "&city=" + city + "&state=" + state + "&country=" + country + "&sig=" + md5;
System.out.println(findByLocUrl);
RestClient rc = new RestClient();
rc.connect(findByLocUrl);
//RestClient.connect(findByLocUrl);
/* if (RestClient.list.isEmpty())
{
setContentView(R.layout.noresults);
} else
{
ResponseHolderAdapter adapter = new ResponseHolderAdapter(this, R.layout.listview_item_row, RestClient.list);
listView1 = (ListView) findViewById(R.id.listView1);
View header = (View) getLayoutInflater().inflate(R.layout.listview_header_row, null);
listView1.addHeaderView(header);
listView1.setAdapter(adapter);
}
*/
}
}
Somehow Im supposed to mesh all this up into using an innerclass extending asyctask to do the API calls in the background parse the json add to my list and set up the adapter. I know i may have some object orientation problems and was hoping before i continue on with my app that you guys can make sure im going in the right direction. I have a few other classes I didnt include. Let me know if it will make more sense if I added the other classes. Thanks In Advance for any help you guys/girls may be able to offer.
You can rewrite your connect method using an AsyncTask like this:
public static void connect(String url) {
HttpClient httpclient = new DefaultHttpClient();
AsyncTask<String, Void, Void> connection = new AsyncTask<String, Void, Void>() {
#Override
protected Void doInBackground(String... params) {
// Prepare a request object
HttpGet httpget = new HttpGet(params[0]);
// Execute the request
HttpResponse response;
try {
list.clear();
response = httpclient.execute(httpget);
// Examine the response status
Log.i("Praeda", response.getStatusLine().toString());
// Get hold of the response entity
HttpEntity entity = response.getEntity();
// If the response does not enclose an entity, there is no need
// to worry about connection release
if (entity != null) {
// A Simple JSON Response Read
InputStream instream = entity.getContent();
String result= convertStreamToString(instream);
// A Simple JSONObject Creation
//JSONObject json= new JSONObject(result);
JSONArray jsonArray = new JSONArray(result);
// A Simple JSONObject Parsing
for (int i = 0; i < (jsonArray.length()); i++) {
JSONObject json_obj = jsonArray.getJSONObject(i);
ResponseHolder rh = new ResponseHolder(json_obj);
list.add(rh);
}
instream.close();
}
}
} catch (ClientProtocolException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (JSONException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
connection.execute(url);
}
If you want to do it right, you should use a contentprovider, service, and a database, here's a good tutorial:
http://mobile.tutsplus.com/tutorials/android/android-fundamentals-downloading-data-with-services/

HTTPclient POST with problematic web site

I'm trying to retrive some data from a web site.
I wrote a java class which seems to work pretty fine with many sites but it doesn't work with this particular site, which use extensive javascript in the input fomr.
As you can see from the code I specified the input fields taking the name from the HTML source, but maybe this website doesn't accept POST request of this kind?
How can I simulate an user-interaction to retrieve the generated HTML?
package com.transport.urlRetriver;
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.FileWriter;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.util.ArrayList;
import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.NameValuePair;
import org.apache.http.client.entity.UrlEncodedFormEntity;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.message.BasicNameValuePair;
public class UrlRetriver {
String stationPoller (String url, ArrayList<NameValuePair> params) {
HttpPost postRequest;
HttpResponse response;
HttpEntity entity;
String result = null;
DefaultHttpClient httpClient = new DefaultHttpClient();
try {
postRequest = new HttpPost(url);
postRequest.setEntity((HttpEntity) new UrlEncodedFormEntity(params));
response = httpClient.execute(postRequest);
entity = response.getEntity();
if(entity != null){
InputStream inputStream = entity.getContent();
result = convertStreamToString(inputStream);
}
} catch (Exception e) {
result = "We had a problem";
} finally {
httpClient.getConnectionManager().shutdown();
}
return result;
}
void ATMtravelPoller () {
ArrayList<NameValuePair> params = new ArrayList<NameValuePair>(2);
String url = "http://www.atm-mi.it/it/Pagine/default.aspx";
params.add(new BasicNameValuePair("ctl00$SPWebPartManager1$g_afa5adbb_5b60_4e50_8da2_212a1d36e49c$txt_address_s", "Viale romagna 1"));
params.add(new BasicNameValuePair("ctl00$SPWebPartManager1$g_afa5adbb_5b60_4e50_8da2_212a1d36e49c$txt_address_e", "Viale Toscana 20"));
params.add(new BasicNameValuePair("sf_method", "POST"));
String result = stationPoller(url, params);
saveToFile(result, "/home/rachele/Documents/atm/out4.html");
}
static void saveToFile(String toFile, String pos){
try{
// Create file
FileWriter fstream = new FileWriter(pos);
BufferedWriter out = new BufferedWriter(fstream);
out.write(toFile);
//Close the output stream
out.close();
}catch (Exception e){//Catch exception if any
System.err.println("Error: " + e.getMessage());
}
}
private static String convertStreamToString(InputStream is) {
BufferedReader reader = new BufferedReader(new InputStreamReader(is));
StringBuilder stringBuilder = new StringBuilder();
String line = null;
try {
while ((line = reader.readLine()) != null) {
stringBuilder.append(line + "\n");
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
is.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return stringBuilder.toString();
}
}
At my point of view, there could be javascript generated field with dynamic value for preventing automated code to crawl the site. Send concrete site you want to download.

Problem returning an object from an AsyncTask

I have a class (RestClient.java) that extends AsyncTask:
package org.stocktwits.helper;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.DefaultHttpClient;
import org.json.JSONException;
import org.json.JSONObject;
import android.os.AsyncTask;
import android.util.Log;
public class RestClient extends AsyncTask<String, Void, JSONObject>{
public JSONObject jsonObj = null;
private static String convertStreamToString(InputStream is) {
/*
* To convert the InputStream to String we use the BufferedReader.readLine()
* method. We iterate until the BufferedReader return null which means
* there's no more data to read. Each line will appended to a StringBuilder
* and returned as String.
*/
BufferedReader reader = new BufferedReader(new InputStreamReader(is));
StringBuilder sb = new StringBuilder();
String line = null;
try {
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
is.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return sb.toString();
}
/* This is a test function which will connects to a given
* rest service and prints it's response to Android Log with
* labels "Praeda".
*/
public static JSONObject connect(String url)
{
HttpClient httpclient = new DefaultHttpClient();
// Prepare a request object
HttpGet httpget = new HttpGet(url);
// Execute the request
HttpResponse response;
try {
response = httpclient.execute(httpget);
// Examine the response status
Log.i("Praeda",response.getStatusLine().toString());
// Get hold of the response entity
HttpEntity entity = response.getEntity();
if (entity != null) {
// A Simple JSON Response Read
InputStream instream = entity.getContent();
String result= convertStreamToString(instream);
// A Simple JSONObject Creation
JSONObject json=new JSONObject(result);
// Closing the input stream will trigger connection release
instream.close();
return json;
}
} catch (ClientProtocolException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (JSONException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return null;
}
#Override
protected JSONObject doInBackground(String... urls) {
return connect(urls[0]);
}
#Override
protected void onPostExecute(JSONObject json ) {
this.jsonObj = json;
}
public void setJSONObject(JSONObject jsonFromUI){
this.jsonObj = jsonFromUI;
}
public JSONObject getJSONObject(){
return this.jsonObj;
}
}
I am trying to execute the AsyncTask on my Main class (Main.java):
RestClient rc = new RestClient();
JSONObject json = new JSONObject();
rc.setJSONObject(json);
rc.execute(buildQuery());
json = rc.getJSONObject();
//do some stuff with the json object
try { JSONObject query = json.getJSONObject("query");
//...
}
json is null because it is called before onPostExecute(). How can I get my JSON?
UPDATE:
I need to run this try block in onPostExecute():
try {
JSONObject query = json.getJSONObject("query");
JSONObject results = query.getJSONObject("results");
if (query.getString("count").equals("1")) { // YQL JSON doesn't
// return an array for
// single quotes
JSONObject quote = results.getJSONObject("quote");
Quote myQuote = new Quote();
myQuote.setName(quote.getString("Name"));
myQuote.setSymbol(quote.getString("Symbol"));
myQuote.setLastTradePriceOnly(quote
.getString("LastTradePriceOnly"));
myQuote.setChange(quote.getString("Change"));
myQuote.setOpen(quote.getString("Open"));
myQuote.setMarketCapitalization(quote
.getString("MarketCapitalization"));
myQuote.setDaysHigh(quote.getString("DaysHigh"));
myQuote.setYearHigh(quote.getString("YearHigh"));
myQuote.setDaysLow(quote.getString("DaysLow"));
myQuote.setYearLow(quote.getString("YearLow"));
myQuote.setVolume(quote.getString("Volume"));
myQuote.setAverageDailyVolume(quote
.getString("AverageDailyVolume"));
myQuote.setPeRatio(quote.getString("PERatio"));
myQuote.setDividendYield(quote.getString("DividendYield"));
myQuote.setPercentChange(quote.getString("PercentChange"));
quotesAdapter.add(myQuote);}
Hey You can use listeners to fix this problem.
I've changed the code slightly to use this.
package com.insidetip.uob.data;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.DefaultHttpClient;
import org.json.JSONException;
import org.json.JSONObject;
import android.app.ProgressDialog;
import android.content.Context;
import android.os.AsyncTask;
import android.util.Log;
public class JSONClient extends AsyncTask<String, Void, JSONObject>{
ProgressDialog progressDialog ;
GetJSONListener getJSONListener;
Context curContext;
public JSONClient(Context context, GetJSONListener listener){
this.getJSONListener = listener;
curContext = context;
}
private static String convertStreamToString(InputStream is) {
/*
* To convert the InputStream to String we use the BufferedReader.readLine()
* method. We iterate until the BufferedReader return null which means
* there's no more data to read. Each line will appended to a StringBuilder
* and returned as String.
*/
BufferedReader reader = new BufferedReader(new InputStreamReader(is));
StringBuilder sb = new StringBuilder();
String line = null;
try {
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
is.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return sb.toString();
}
public static JSONObject connect(String url)
{
HttpClient httpclient = new DefaultHttpClient();
// Prepare a request object
HttpGet httpget = new HttpGet(url);
// Execute the request
HttpResponse response;
try {
response = httpclient.execute(httpget);
// Examine the response status
Log.i("Praeda",response.getStatusLine().toString());
// Get hold of the response entity
HttpEntity entity = response.getEntity();
if (entity != null) {
// A Simple JSON Response Read
InputStream instream = entity.getContent();
String result= convertStreamToString(instream);
// A Simple JSONObject Creation
JSONObject json=new JSONObject(result);
// Closing the input stream will trigger connection release
instream.close();
return json;
}
} catch (ClientProtocolException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (JSONException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return null;
}
#Override
public void onPreExecute() {
progressDialog = new ProgressDialog(curContext);
progressDialog.setMessage("Loading..Please wait..");
progressDialog.setCancelable(false);
progressDialog.setIndeterminate(true);
progressDialog.show();
}
#Override
protected JSONObject doInBackground(String... urls) {
return connect(urls[0]);
}
#Override
protected void onPostExecute(JSONObject json ) {
getJSONListener.onRemoteCallComplete(json);
progressDialog.dismiss();
}
}
Use in the calling class like this.
JSONClient client = new JSONClient(context, listener);
client.execute(URL);
Dont forget to implement the listener
public interface GetJSONListener {
public void onRemoteCallComplete(JSONObject jsonFromNet);
}
I'm be mistaken by result of doInBackground can be consumed in onPostExecute
doInBackground(Params...), invoked on
the background thread immediately
after on PreExecute() finishes
executing. This step is used to
perform background computation that
can take a long time. The parameters
of the asynchronous task are passed
to this step. The result of the
computation must be returned by this
step and will be passed back to the
last step. This step can also use
publishProgress(Progress...) to
publish one or more units of
progress. These values are published
on the UI thread, in the
onProgressUpdate(Progress...) step.
#Override
protected void onPostExecute(JSONObject json ) {
// DO stuff here ( it's UI thread )
mJsonFromTheActivity = json;
}
execute() always returns the AsyncTask itself. The object you return from doInBackground() is handed to you in onPostExecute().
If you have your asynctask as a nested inner class of your activity, you can set one of your activities variables to the result of your asynctask

Categories

Resources