Develop Reference

Reading large data from an inputstream some data missing at the end

Java(TM) SE Runtime Environment (build 1.7.0_45-b18)
Hello,
I am using the httpUrlConnection to retrieve a json string from a webservice. Then I get the inputStream from the connection
jsonString = readJSONInputStream(mHttpUrlconnection.getInputStream());
I then use the following function to read the inputstream to get the JSON.
private String readJSONInputStream(final InputStream inputStream) {
Reader reader = null;
try {
final int SIZE = 16024;
char[] buffer = new char[SIZE];
int bytesRead = 0;
int read = 0;
reader = new BufferedReader(new InputStreamReader(inputStream, "UTF-8"), SIZE);
String line = "";
String jsonString = "";
while((line = reader.readLine()) != null) {
jsonString += line;
}
/* Success */
return jsonString;
}
catch(IndexOutOfBoundsException ex) {
log.log(Level.SEVERE, "UnsupportedEncodingexception: " + ex.getMessage());
}
catch(IOException ex) {
log.log(Level.SEVERE, "IOException: " + ex.getMessage());
}
finally {
/* close resources */
try {
reader.close();
inputStream.close();
}
catch(IOException ex) {
log.log(Level.SEVERE, "IOException: " + ex.getMessage());
}
}
return null;
}
However, if the json is small say 600 bytes then everything is ok. But I have some JSON that is about 15000 bytes in size so I set the maximum size to 16024.
However, the JSON it only reads about about ~6511 and just cuts off.
If the JSON is small there is no problem < 1000 bytes. But for the larger JSON it only read about half of it.
I the data is there as I have tested this in a browswer using the http request plugin.
Am I doing anything wrong here. Anything I should check.
Many thanks for any suggestions,

Problem resolved. Due to the logger not displaying all the information. Not really a problem afterall.

Code stuck at EntityUtils.toString and keeps waiting for ages

I am sending some data to server from my android device and in return getting Json Response back from the server.The Data returned is a quite large chunk of data (Multiple Images precisely) .
I am echoing the statusCode returned from the Server and it is 200. but The JAVA code keeps on waiting on a statement
objHttpEntity = objHttpResponse.getEntity();
statusCode=objHttpResponse .getStatusLine().getStatusCode();
String responseDataString = EntityUtils.toString(objHttpEntity);//this statement
and in logs, I see my garbage collector running,getting the memory back from the resources to accomodate the currently received data.
In devices having RAM 1GB or less, the application is crashing abruptly giving out an OutOfMermoryException .But in devices having RAM more than the latter, the application waits, and waits, and waits and finally execute the rest of the consecutive statements of my code.
How can I get rid of the exception(in devices having less RAM). OR can reduce the time for which the code waits.
Receiving status code of 200, clearly signifies that Server is done with its work, now all the handling must be performed on the client(Device) side.
Note : Already gone through all the three of the questions posted here on stackOverflow about the issue, but none of them was appropriate and was unable to solve the issue.

Why don't you try this one?
HttpGet get = new HttpGet(apiURL);
HttpResponse response = client.execute(get);
HttpEntity resEntity = response.getEntity();
InputStream is = resEntity.getContent();
String result = convertStreamToString(is);
public static String convertStreamToString(InputStream is) {
/*
* To convert the InputStream to String we use the
* BufferedReader.readLine() method. We iterate until the BufferedReader
* return null which means there's no more data to read. Each line will
* appended to a StringBuilder and returned as String.
*/
BufferedReader reader = new BufferedReader(new InputStreamReader(is));
StringBuilder sb = new StringBuilder();
String line = null;
try {
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
is.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return sb.toString();
}

cannot read arabic text from url in java

I've a web service that prints the following text.
[{"packid":"p101","title":"صفته 1","description":"شسیب: 1\r\nثق س: 50","linkfuntext":"funtext","linkshortstory":"short","linkfunpic":"pic","linkringtone":"ring","linkfungif":"gif","linkwallpaper":"wall","price":"500","buyid":"pack.fun.1","buyed":""},{"packid":"p102","title":"بسته صدا","description":" متن ها: 50\r\nصداها: 120\r\nتصاویر: 100\r\nتصاویر متحرک: 50\r\nداستان کوتاه: 20","linkfuntext":"","linkshortstory":"","linkfunpic":"","linkringtone":"","linkfungif":"","linkwallpaper":"","price":"1200","buyid":"fun.pack.2","buyed":""}]
When I try to read it in java I receive it in the following format
[{"packid":"p101","title":"صفته 1","description":"شسیب: 1\r\nثق س: 50","linkfuntext":"funtext","linkshortstory":"short","linkfunpic":"pic","linkringtone":"ring","linkfungif":"gif","linkwallpaper":"wall","price":"500","buyid":"pack.fun.1","buyed":""},{"packid":"p102","title":"بسته صدا","description":" متن ها: 50\r\nصداها: 120\r\nتصاویر: 100\r\nتصاویر متحرک: 50\r\nداستان کوتاه: 20","linkfuntext":"","linkshortstory":"","linkfunpic":"","linkringtone":"","linkfungif":"","linkwallpaper":"","price":"1200","buyid":"fun.pack.2","buyed":""}]
I've tried changing the character set to UTF-8 as well as ISO-8859-6 but still no luck. When I print the text on console it is printed correctly which means there is no issue in character set of eclipse or console. Also I've tried changing the character set of string that is storing the text, but same issue.
String serverOutput = new String(TEXT_FROM_SERVER.getBytes(), "UTF-8");
Here is my code that gets output from web service
HttpEntity entity = response.getEntity();
InputStream is = entity.getContent();
String serverOutput = convertStreamToString(is);
private String convertStreamToString(InputStream is) {
Reader rd = null;
BufferedReader reader = null;
try {
rd = new InputStreamReader(is,"UTF-8");
} catch (UnsupportedEncodingException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
reader = new BufferedReader(rd);
StringBuilder sb = new StringBuilder();
String line = null;
try {
while ((line = reader.readLine()) != null) {
sb.append((line + "\n"));
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
is.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return sb.toString();
}
Any kind of help will be greatly appreciated. Thanks

you need to unescape those HTML characters, and you can do so with a method from Apache Commons Lang called unescapeHtml. More info here.
Example:
String afterDecoding = StringEscapeUtils.unescapeHtml(beforeDecoding);

StringBuilder append cause out of memory

i am getting out of memory error in asynctask which loop to stringbuilder . My target for using this to download image from server and store inside my sd card.My code as below :
HttpClient httpclient = new DefaultHttpClient();
httpclient.getParams().setParameter(CoreProtocolPNames.PROTOCOL_VERSION, HttpVersion.HTTP_1_1);
HttpPost httppost = new HttpPost(severPath);
httppost.setEntity(params[0]);
System.out.println("executing request " + httppost.getRequestLine());
HttpResponse response = null;
try {
response = httpclient.execute(httppost);
} catch (ClientProtocolException e6) {
// TODO Auto-generated catch block
e6.printStackTrace();
} catch (IOException e6) {
// TODO Auto-generated catch block
e6.printStackTrace();
}
String output;
System.out.println("Output from Server .... \n");
BufferedReader br = null;
try {
br = new BufferedReader(
new InputStreamReader((response.getEntity().getContent())));
} catch (IllegalStateException e5) {
// TODO Auto-generated catch block
e5.printStackTrace();
} catch (IOException e5) {
// TODO Auto-generated catch block
e5.printStackTrace();
}
OutputStreamWriter outputStreamWriter = null;
try {
outputStreamWriter = new OutputStreamWriter(context.openFileOutput("LargeImages.txt", context.MODE_PRIVATE));
} catch (FileNotFoundException e6) {
// TODO Auto-generated catch block
e6.printStackTrace();
}
int i = 0;
StringBuilder builder = new StringBuilder();
String Result = "";
try {
for (String line = null; (line = br.readLine()) != null ; ) {
builder.append(line.toString());
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
outputStreamWriter.close();
i am getting out of memory allocation error. please help. i try many method but also not getting the right.

if you are downloading an image, then you should not use Reader/Writer/StringBuilder to store it's content. Because the file is binary content will be scrambled because of the character encoding used by Reader/Writer classes.
Try using InputStream/OutputStream and store the content directly to sdcard without storing it in memory.
Try out the below code:
InputStream in = response.getEntity().getContent();
OutputStream out = context.openFileOutput("LargeImages.txt", context.MODE_PRIVATE);
byte b[] = new byte[4096];
int i;
while ((i = in.read(b)) >= 0) {
out.write(b, 0, i);
}

There may be two problems.
The first - the cycle for (String line = null; (line = br.readLine()) != null ; ) is not terminated properly. Try to find it out by opening a small file(e.g. with 10 lines total).
The second - it's actually a memory insufficient case. Probably it's not the best idea to get image via strings as images may be very heavy and creating a plenty of Strings causes natural memory error. Try to find another approach.

I don't see code that is actually writing to the output stream. Shouldn't there be a line before the close, that is like outputStreamWriter.print(builder)?
About your question. Instead of collecting the whole data in memory in a StringBuilder and than write it at once, you should write directly each line you get within your for-loop. You don't need the StringBuilder at all. Here's a code snippet:
try {
for (String line = br.readLine(); line != null; line = br.readLine()) {
outputStreamWriter.append(line);
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
return;
}
Three more remarks:
When you get an Exception you should also stop the action, e.g. return from your method. Your code above would print the Stacktrace (which is definitely helpful) but would then continue, which would be not so helpful. Just add return after each printstackTrace.
There's still a chance that one line is too long for memory, but the risk is minimized.
Is the data you download binary image or text? You name it image but you download text. Please be aware that there's a difference between bytes and characters (encoded with character set) and stay within what you actually receive.

how to get a value by key from an json (or xml) string?

In the android app I get an xml or json string returned, However, I cant seem to figure out any way on how to get an value from the string in any way by entering an key.
In PHP you just use something like $myArray['parent']['child'] but I have no clue on how this works in java.
Any idea's would be greatly appreciated! (an example for both XML and JSON even more ;) )

Here's what I would do:
locate an XML/JSON library (there's tons) (google-gson for json)
read the documentation to find a parse method ((new JsonParser()).parse(text))
read the documentation to find out what the return value is (JsonElement)
decide what you want to do with the parsed data (myJsonObj.get(...))
write the code

public class parsingjsontest2 extends Activity {
/** Called when the activity is first created. */
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(main);
String str = connect("http://rentopoly.com/ajax.php?query=Bo"));
System.out.println("String::"+str);
}
}
private String connect(String url)
{
// Create the httpclient
HttpClient httpclient = new DefaultHttpClient();
// Prepare a request object
HttpGet httpget = new HttpGet(url);
// Execute the request
HttpResponse response;
// return string
String returnString = null;
try {
// Open the webpage.
response = httpclient.execute(httpget);
if(response.getStatusLine().getStatusCode() == 200){
// Connection was established. Get the content.
HttpEntity entity = response.getEntity();
// If the response does not enclose an entity, there is no need
// to worry about connection release
if (entity != null) {
// A Simple JSON Response Read
InputStream instream = entity.getContent();
// Load the requested page converted to a string into a JSONObject.
JSONObject myAwway = new JSONObject(convertStreamToString(instream));
// Get the query value'
String query = myAwway.getString("query");
**// Make array of the suggestions
JSONArray suggestions = myAwway.getJSONArray("suggestions");
// Build the return string.
returnString = "Found: " + suggestions.length() + " locations for " + query;
for (int i = 0; i < suggestions.length(); i++) {
returnString += "\n\t" + suggestions.getString(i);
}
// Cose the stream.
instream.close();
}
}
else {
// code here for a response othet than 200. A response 200 means the webpage was ok
// Other codes include 404 - not found, 301 - redirect etc...
// Display the response line.
returnString = "Unable to load page - " + response.getStatusLine();
}
}
catch (IOException ex) {
// thrown by line 80 - getContent();
// Connection was not established
returnString = "Connection failed; " + ex.getMessage();
}
catch (JSONException ex){
// JSON errors
returnString = "JSON failed; " + ex.getMessage();
}
return returnString;
}
private static String convertStreamToString(InputStream is) {
/*
* To convert the InputStream to String we use the BufferedReader.readLine()
* method. We iterate until the BufferedReader return null which means
* there's no more data to read. Each line will appended to a StringBuilder
* and returned as String.
*/
BufferedReader reader = new BufferedReader(new InputStreamReader(is));
StringBuilder sb = new StringBuilder();
String line = null;
try {
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
is.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return sb.toString();
}
}

As you didn't specify what kind of xml you are trying to read, I'm answering based on what I know.
In Android, if you were talking about the layout and strings.xml files, you use a dot (.) operator, like R.string.appname.
Please post more details about your specific problem, if this is not what you were looking for.