In my app, I make a request from a public URL, and then open the source code of the web page, finally, I extract the information that I want from the source code. I got no problem with this entire process. However, it takes so long to load up the information I want. Is there any other efficient way I can do?
public class GetMethodEx {
public String getInternetData(String currentUrl) throws Exception{
BufferedReader in = null;
String data = null;
try{
HttpClient client = new DefaultHttpClient();
URI website = new URI(currentUrl);
HttpGet request = new HttpGet();
request.setURI(website);
HttpResponse response = client.execute(request);
in = new BufferedReader(new InputStreamReader(response.getEntity().getContent()));
StringBuffer sb = new StringBuffer("");
String l = "";
String nl = System.getProperty("line.separator");
while((l = in.readLine()) !=null){
sb.append(l + nl);
}
in.close();
data = sb.toString();
return data;
}finally{
if (in != null){
try{
in.close();
return data;
}catch (Exception e){
e.printStackTrace();
}
}
}
}
}
Using a StringBuffer is really not efficient downloading large texts as a html file is one. Since you are reading line java has to allocate memory for each line you are reading just to copy everything that has been copied in memory into the StringBuffer what leads to intense GC work. Then a StringBuffer has a fixed size so your program may reach a point where the StringBuffers size is exceeded which leads to a resizing of the StringBuffer what causes copying everything inside the Buffer into a new one.
So you rather should try to get the size of html document you requested and read everything into a char array. That may not work since http allows to transfer data in chunks of variable size. This is an idea of what you can do if that is the case:
String html = "";
CharBuffer buff = CharBuffer.allocate(16384);
int read = in.read(buff);
while(read > -1) {
while(read > -1 && buff.remaining > 0) {
read = in.read(buff);
}
html += new String(buff.array());
buff.clear();
}
Related
I'm a newbie in android programming and I want to sent data to php server and receive data to show in android
but I have no idea to create it, I don't no to use library? and Can give some example for me to practice about it.
ps. sorry my english is not good.
example Myphp "xxx.xxx.x.x/api.php"
$keyword = $_GET['keyword'];
$index= $_GET['index'];
$path = "http://xxxxxxx/webservice/&query=".urlencode($keyword)."&index=".$index."";
$jsondata = file_get_contents($path);
$jsons = json_decode($jsondata,true);
echo json_encode($jsons);
I want to send keyword and index from edittext to php server and receive json data to show listview.
HttpURLConnection can be used for get,put,post,delete requests :
try {
// Construct the URL
URL url = new URL("http://xxxxxxx/webservice/&query="
+keyword.getText().toString()
+"&index="+index.getText().toString());
// Create the request
urlConnection = (HttpURLConnection) url.openConnection();
urlConnection.setRequestMethod("GET");
urlConnection.connect();
// Read the input stream into a String
InputStream inputStream = urlConnection.getInputStream();
StringBuffer buffer = new StringBuffer();
if (inputStream == null) {
// Nothing to do.
return null;
}
reader = new BufferedReader(new InputStreamReader(inputStream));
String line;
while ((line = reader.readLine()) != null) {
// Since it's JSON, adding a newline isn't necessary (it won't affect parsing)
// But it does make debugging a *lot* easier if you print out the completed
// buffer for debugging.
buffer.append(line + "\n");
}
if (buffer.length() == 0) {
// Stream was empty. No point in parsing.
}
JsonStr = buffer.toString(); //result
} catch (IOException e) {
}
where keyword and index will be your EditText Variables.
This is the code I'm using to get response text.
private static String request(String urlstr){
// create connection
try {
URL url = new URL(urlstr);
URLConnection conn = url.openConnection();
StringBuilder response = new StringBuilder();
conn.setUseCaches(false);
conn.setRequestProperty("User-Agent", USER_AGENT);
// read response
BufferedReader in = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String line = null;
while ((line = in.readLine()) != null) {
response.append(line);
}
in.close();
return response.toString();
} catch (Exception e){
return null;
}
}
The problem is that when querying the very same request (simple get request, response is json) with my chrome browser I get the response almost 1 second faster than with this code in my application.
I wonder if theres anything I'm doing wrong in my code? Or it is chrome handling that request faster somehow?
Maybe there are some techniques to make this process faster?
Thanks
You seem to read the response line by line, but in the end you append every line to one single response, so it is not really required to read the response line by line. You can also do
char[] cbuf = new char[1024];
int len;
while ((len = in.read(cbuf)) != -1)
response.append(cbuf, 0, len);
Like this the response can be read in much larger chunks and you don't have the overhead of the readLine() method that has to look for newline characters in the input and split the content into lines.
You could also do a
new StringBuilder(connection.getContentLength());
to avoid that the StringBuilder has to increase it's capacity every time new content is appended. The StringBuilder is using a char[] internally and every time the array is not big enough for the new content it has to be copied to a new array with a larger size.
I have a small java app that, given a list of references for google places, must get back the Id's for each of said google places (long story short, we were storing references for places instead of their Id's, and only now realized that references are not unique per place).
My app works perfectly for about 95% of the places in the list, but fails with a "NOT_FOUND" status code for some records. Some investigation reveals that the place reference for these particular places is (when combined with the https://maps.googleapis.com/maps/api/place/details/json?sensor=false&key=myApiKey prefix) about 2 characters too long for a URL. The last couple of characters are getting truncated.
My initial thought was that I would just make a POST request to the google places API, but I'm getting back "REQUEST_DENIED" status code from the google servers when sending the same into as a POST request.
Is there anyway around this, or is this just an emergent bug with the google places API (now that the number of places has pushed the reference too long?).
I should also note that the places that fail are all recently added by our application.
This is what my current (working for 95%) code looks like:
public static JSONObject getPlaceInfo(String reference) throws Exception
{
URL places = new URL("https://maps.googleapis.com/maps/api/place/details/json?sensor=false&key="+apiKey+"&reference="+reference);
URLConnection con = places.openConnection();
BufferedReader in = new BufferedReader(new InputStreamReader(con.getInputStream()));
StringBuffer input = new StringBuffer();
String inputLine;
while ((inputLine = in.readLine()) != null)
input.append(inputLine);
in.close();
JSONObject response = (JSONObject) JSONSerializer.toJSON(input.toString());
return response;
}
and this is what my "ACCESS_DENIED" post code looks like:
public static JSONObject getPlaceInfo(String reference) throws Exception
{
String data = URLEncoder.encode("sensor", "UTF-8") + "=" + URLEncoder.encode("true", "UTF-8");
data += "&" + URLEncoder.encode("key", "UTF-8") + "=" + URLEncoder.encode(apiKey, "UTF-8");
data += "&" + URLEncoder.encode("reference", "UTF-8") + "=" + URLEncoder.encode(reference, "UTF-8");
URL places = new URL("https://maps.googleapis.com/maps/api/place/details/json");
URLConnection con = places.openConnection();
con.setDoOutput(true);
OutputStreamWriter wr = new OutputStreamWriter(con.getOutputStream());
wr.write(data);
wr.flush();
BufferedReader in = new BufferedReader(new InputStreamReader(con.getInputStream()));
StringBuffer input = new StringBuffer();
String inputLine;
while ((inputLine = in.readLine()) != null)
input.append(inputLine);
in.close();
JSONObject response = (JSONObject) JSONSerializer.toJSON(input.toString());
return response;
}
An example of the reference that fails is:
CnRtAAAAxm0DftH1c5c6-krpWWZTT51uf0rDqCK4jikWV6eGfXlmKxrlsdrhFBOCgWOqChc1Au37inhf8HzjEbRdpMGghYy3dxGt17FEb8ys2CZCLHyC--7Vf1jn-Yn1kfZfzxznTJAbIEg6422q1kRbh0nl1hIQ71tmdOVvhdTfY_LOdbEoahoUnP0SAoOFNkk_KBIvTW30btEwkZs
Thanks in advance!
You're sending your request parameters in the body, which isn't supported by the API. There's a good answer about GET and request params at:
HTTP GET with request body
The following code should work for Place detail requests:
private static final String PLACES_API_BASE = "https://maps.googleapis.com/maps/api/place";
private static final String TYPE_DETAILS = "/details";
private static final String OUT_JSON = "/json";
HttpURLConnection conn = null;
StringBuilder jsonResults = new StringBuilder();
try {
StringBuilder sb = new StringBuilder(PLACES_API_BASE);
sb.append(TYPE_DETAILS);
sb.append(OUT_JSON);
sb.append("?sensor=false");
sb.append("&key=" + API_KEY);
sb.append("&reference=" + URLEncoder.encode(reference, "utf8"));
URL url = new URL(sb.toString());
conn = (HttpURLConnection) url.openConnection();
InputStreamReader in = new InputStreamReader(conn.getInputStream());
// Load the results into a StringBuilder
int read;
char[] buff = new char[1024];
while ((read = in.read(buff)) != -1) {
jsonResults.append(buff, 0, read);
}
} catch (MalformedURLException e) {
return null;
} catch (IOException e) {
return null;
} finally {
if (conn != null) {
conn.disconnect();
}
}
try {
// Create a JSON object hierarchy from the results
JSONObject jsonObj = new JSONObject(jsonResults.toString()).getJSONObject("result");
jsonObj.getString("name");
} catch (JSONException e) {
Log.e(LOG_TAG, "Error processing JSON results", e);
}
I want to read the content of a webpage with the following methods, but I only get 60-70 percent of it.
I've tried 2 different methods to read the webpage, both with the same result. I also tried different Urls. I get no errors or timeouts.
What I am doing wrong ?
URL url = new URL(uri.toString());
HttpURLConnection urlConnection = (HttpURLConnection) url.openConnection();
try
{
InputStream in = new BufferedInputStream(urlConnection.getInputStream());
BufferedReader br = new BufferedReader(new InputStreamReader(in));
StringBuilder sb = new StringBuilder();
String line = null;
while ((line = br.readLine()) != null)
{
sb.append(line + "\n");
}
br.close();
this.content = sb.toString();
}
finally
{
urlConnection.disconnect();
}
AND
HttpGet get = new HttpGet(uri);
HttpClient defaultHttp = new DefaultHttpClient(httpParameters);
HttpResponse response = defaultHttp.execute(get);
StatusLine status = response.getStatusLine();
if(status.getStatusCode() == HttpStatus.SC_OK)
{
HttpEntity entity = response.getEntity();
InputStream stream = entity.getContent();
String encoding = "utf-8";
//long length = entity.getContentLength();
//if(entity.getContentEncoding() != null)
//{
// encoding = entity.getContentEncoding().getValue();
//}
//if(length > 0)
//{
byte[] buffer = new byte[1024];
long read = 0;
do
{
read = stream.read(buffer);
if(read > 0)
{
this.content += new String(buffer, encoding);
}
}while(read > 0);
//}
}
#edit
I've tried it with C# and WinForms. I read the complete html source of that webpage.
With java-android it doesn't work.
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create("http://www.kicker.de");
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
StreamReader reader = new StreamReader(response.GetResponseStream());
string content = reader.ReadToEnd();
reader.Close();
response.Close();
the httpurlconnection in apache's util jar has limited the biggest bytes in a response, i couldn't remember the number of it.
But in most of time ,may you use the http conncetion in UI thread , so sometimes it's not safe,and maybe will be killed, you can choose to deal with the http request in a thread but not the UI thread. So I want to know if you do it in the UT thread
I have currently the same Problem. I tried my Code in a simple Java Application and I receive the whole content. But on Android, the Content is incomplete. This Question is now a year old. I guess you have solved it in the meantime. Can you please add your Solution?
Edit:
I wrote the content into a File on my Android Device. The Content was complete!
It seems logcat doesn´t show the complete Output you receive from the Devie.
My input is a InputStream which contains an XML document. Encoding used in XML is unknown and it is defined in the first line of XML document.
From this InputStream, I want to have all document in a String.
To do this, I use a BufferedInputStream to mark the beginning of the file and start reading first line. I read this first line to get encoding and then I use an InputStreamReader to generate a String with the correct encoding.
It seems that it is not the best way to achieve this goal because it produces an OutOfMemory error.
Any idea, how to do it?
public static String streamToString(final InputStream is) {
String result = null;
if (is != null) {
BufferedInputStream bis = new BufferedInputStream(is);
bis.mark(Integer.MAX_VALUE);
final StringBuilder stringBuilder = new StringBuilder();
try {
// stream reader that handle encoding
final InputStreamReader readerForEncoding = new InputStreamReader(bis, "UTF-8");
final BufferedReader bufferedReaderForEncoding = new BufferedReader(readerForEncoding);
String encoding = extractEncodingFromStream(bufferedReaderForEncoding);
if (encoding == null) {
encoding = DEFAULT_ENCODING;
}
// stream reader that handle encoding
bis.reset();
final InputStreamReader readerForContent = new InputStreamReader(bis, encoding);
final BufferedReader bufferedReaderForContent = new BufferedReader(readerForContent);
String line = bufferedReaderForContent.readLine();
while (line != null) {
stringBuilder.append(line);
line = bufferedReaderForContent.readLine();
}
bufferedReaderForContent.close();
bufferedReaderForEncoding.close();
} catch (IOException e) {
// reset string builder
stringBuilder.delete(0, stringBuilder.length());
}
result = stringBuilder.toString();
}else {
result = null;
}
return result;
}
The call to mark(Integer.MAX_VALUE) is causing the OutOfMemoryError, since it's trying to allocate 2GB of memory.
You can solve this by using an iterative approach. Set the mark readLimit to a reasonable value, say 8K. In 99% of cases this will work, but in pathological cases, e.g 16K spaces between the attributes in the declaration, you will need to try again. Thus, have a loop that tries to find the encoding, but if it doesn't find it within the given mark region, it tries again, doubling the requested mark readLimit size.
To be sure you don't advance the input stream past the mark limit, you should read the InputStream yourself, upto the mark limit, into a byte array. You then wrap the byte array in a ByteArrayInputStream and pass that to the constructor of the InputStreamReader assigned to 'readerForEncoding'.
You can use this method to convert inputstream to string. this might help you...
private String convertStreamToString(InputStream input) throws Exception{
BufferedReader reader = new BufferedReader(new InputStreamReader(input));
StringBuilder sb = new StringBuilder();
String line = null;
while ((line = reader.readLine()) != null) {
sb.append(line);
}
input.close();
return sb.toString();
}