I am sending some data to server from my android device and in return getting Json Response back from the server.The Data returned is a quite large chunk of data (Multiple Images precisely) .
I am echoing the statusCode returned from the Server and it is 200. but The JAVA code keeps on waiting on a statement
objHttpEntity = objHttpResponse.getEntity();
statusCode=objHttpResponse .getStatusLine().getStatusCode();
String responseDataString = EntityUtils.toString(objHttpEntity);//this statement
and in logs, I see my garbage collector running,getting the memory back from the resources to accomodate the currently received data.
In devices having RAM 1GB or less, the application is crashing abruptly giving out an OutOfMermoryException .But in devices having RAM more than the latter, the application waits, and waits, and waits and finally execute the rest of the consecutive statements of my code.
How can I get rid of the exception(in devices having less RAM). OR can reduce the time for which the code waits.
Receiving status code of 200, clearly signifies that Server is done with its work, now all the handling must be performed on the client(Device) side.
Note : Already gone through all the three of the questions posted here on stackOverflow about the issue, but none of them was appropriate and was unable to solve the issue.
Why don't you try this one?
HttpGet get = new HttpGet(apiURL);
HttpResponse response = client.execute(get);
HttpEntity resEntity = response.getEntity();
InputStream is = resEntity.getContent();
String result = convertStreamToString(is);
public static String convertStreamToString(InputStream is) {
/*
* To convert the InputStream to String we use the
* BufferedReader.readLine() method. We iterate until the BufferedReader
* return null which means there's no more data to read. Each line will
* appended to a StringBuilder and returned as String.
*/
BufferedReader reader = new BufferedReader(new InputStreamReader(is));
StringBuilder sb = new StringBuilder();
String line = null;
try {
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
is.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return sb.toString();
}
Related
I'm using Apache HttpClient to fetch meta-data from radio stream. What I want to do is make a GET request, read some bytes then close the stream. For some streams, it works properly but for some others, the closing of the stream hangs. It looks like it's still receiving data and not closing the connection while it's happening.
public SongInfo retrieveMetadata(String streamUrl) {
HttpClient httpClient = HttpClients.createDefault();
HttpGet httpGet = new HttpGet(streamUrl);
httpGet.addHeader("Icy-MetaData", "1");
httpGet.addHeader("Connection", "close");
httpGet.addHeader("Accept", "");
HttpResponse response;
try {
response = httpClient.execute(httpGet);
} catch (IOException e) {
log.warn("An exception occurred while fetching the stream", e);
return null;
}
if (response.getStatusLine().getStatusCode() != 200) {
log.warn("Could not fetch stream. Status line: "+ response.getStatusLine());
return null;
}
int metadataOffset = retrieveMetadataOffset(response);
if (metadataOffset == 0) {
log.info("Could not find metadata for url:"+ streamUrl);
return null;
}
List<Metadata> metadata = extractMetadata(response, metadataOffset);
if (metadata == null || metadata.isEmpty()) {
return null;
}
return extractSongInfo(metadata);
}
private List<Metadata> extractMetadata(HttpResponse response, int metadataOffset) {
String metadataStr;
try(InputStream stream = response.getEntity().getContent()) {
if (stream.skip(metadataOffset) != metadataOffset) {
log.warn("Something went wrong while skipping to metadata offset");
return null;
}
int metaDataLength = stream.read() * 16;
metadataStr = getMetadataStr(stream, metaDataLength);
if (metadataStr == null) {
return null;
}
} catch (IOException e) {
log.warn("Something went wrong while reading the stream", e);
return null;
} //Hangs here
//rest of the method
}
I noticed that the stream returned by response.getEntity().getContent() is of type EofSensorInputStream so I'm wondering if it's waiting for a EOF character which is never coming.
Example of stream for which the code works properly and the stream closes properly: https://icecast.omroep.nl/radio2-bb-mp3, http://live-mp3-128.kexp.org
Example of stream for which the code doesn't work properly as the stream never closes and hangs forever: https://kexp-mp3-128.streamguys1.com/kexp128.mp3
This problem occurs because calling close() on the content stream will try to consume the remaining content. This is currently not explicitly mentioned in the documentation (see also HTTPCORE-656), but is mentioned in the tutorials:
The difference between closing the content stream and closing the response is that the former will attempt to keep the underlying connection alive by consuming the entity content while the latter immediately shuts down and discards the connection.
[...]
There can be situations, however, when only a small portion of the entire response content needs to be retrieved and the performance penalty for consuming the remaining content and making the connection reusable is too high, in which case one can terminate the content stream by closing the response.
Therefore in your case it seems appropriate to not close the InputStream returned by getContent() but only close the HttpResponse (which you are apparently not doing yet).
I have a module which requests status(technically a JSON object) from an FHEM server(home automation) using a simple HTTP post.
How can i poll the requestState method so as to continuously listen (like every 125ms) on the URL for changes in status?
[note: i am not using any server, this is a plain Java program called from a Main function.]
To give an illustration:
public static boolean requestState(){
HttpPost post = new HttpPost("http://localhost:8083/fhem?cmd=jsonlist+device1");
try{
HttpResponse response = client.execute(post);
BufferedReader rd = new BufferedReader(
new InputStreamReader(response.getEntity().getContent()));
StringBuffer result = new StringBuffer();
String line = "";
while ((line = rd.readLine()) != null) {
result.append(line);
}
//response is a JSONObject
JSONObject o = new JSONObject(result.toString());
//get the status from the JSON object
System.out.println(o.getJSONObject("Result").get("STATE"));
String STATE = o.getJSONObject("Result").get("STATE");
if(STATE.equals("OPEN"))
return false;
else
return true;
}
catch(Exception e){
e.printStackTrace();
}
}
What i would like to achieve is:
keep listening to URL
parse response to judge the status
if status == OPEN
soundAlarm();
Thanks in advance!
You could keep both posting a request and reading the result in a loop and use Thread.sleep to put pauses if needed between each poll and the other. I'm not familiar with FHEM but maybe there is a more efficient way that you could use where FHEM pushes the status change to your app rather than having to poll it continually.
I am using the below code to parse the JSON String fetched from Web, (30,000 records)
DefaultHttpClient httpclient = new DefaultHttpClient(new BasicHttpParams());
HttpPost httppost = new HttpPost(params[0]);
httppost.setHeader("Content-type", "application/json");
InputStream inputStream = null;
String result = null;
HttpResponse response = null;
try {
response = httpclient.execute(httppost);
} catch (ClientProtocolException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
HttpEntity entity = response.getEntity();
try {
inputStream = entity.getContent();
} catch (IllegalStateException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
BufferedReader reader = null;
try {
reader = new BufferedReader(new InputStreamReader(inputStream, "UTF-8"),8);
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
}
StringBuilder sb = new StringBuilder();
String line = null;
try {
while ((line = reader.readLine()) != null)
{
sb.append(line + "\n");
}
} catch (IOException e) {
e.printStackTrace();
}
result = sb.toString();
I am getting the OutofMemory error in the below code
while ((line = reader.readLine()) != null)
{
sb.append(line + "\n");
}
How to get rid of this error.This error does occur when the json string is very huge as it contains data of about 30,000 records.
Any help in this regard is highly appreciated..
Android imposes a memory cap limit (of 16 MB in almost all phones, with some newer tablets have more) for each and every application. Application should make sure they maintain their live memory limit below that level.
So we can't hold a big string, say over 1MB, in full sometimes since the total live memory usage of applicaiton may exceed that limit. Remember, the total memory usage includes all objects (including UI elements) we allocated in our app.
So your only solution is to use a Streaming JSON parser, which takes data as it comes. That is you should not hold on full string in a String object. One option is to use Jackson JSON parser.
EDIT : Android now support JSONReader from API level 11. Never used it, but it seems the way to go..
If data file is too large, you cannot read it all to memory.
Read a line and then write it to a native file. Do not use a StringBuilder to hold all data in memory.
Try to import your data in chuncks, like 1000 records each time. Hopefully you will not experince this issue.
I solved this problem with this library.
There is a very good tutorial here.
With this you will bypass converting entity.getContent() to String and that will solve your problem.
InputStream inputStream = entity.getContent();
JsonReader reader = Json.createReader(inputStream);
JsonObject jsonObject = reader.readObject();
return jsonObject;
I have a server which initially does this:-
BufferedReader br = new BufferedReader(new InputStreamReader(s.getInputStream()));
for (;;) {
String cmdLine = br.readLine();
if (cmdLine == null || cmdLine.length() == 0)
break;
...
}
later it passes the socket to another class "foo"
This class wait for application specific messages.
BufferedReader br = new BufferedReader(new InputStreamReader(s.getInputStream()));
appCmd=br.readLine();
My client sends this sequence:
"bar\n"
"how are u?\n"
"\n"
"passing it to foo\n"
"\n"
The problem is that sometimes "foo" does not get its response. It hangs in the readLine().
What is the chance that readLine() in the server is buffering up the data using the read ahead and "foo" class is getting starved?
If I add a sleep in the client side, it works. But what is the chance that it will always work?
"bar\n"
"how are u?\n"
"\n"
sleep(1000);
"passing it to foo\n"
"\n"
How to fix the problem? Appreciate any help on this regard.
eee's solution works perfectly. I was trying to read output from an SMTP conversation but it would block on:
while ((response = br.readLine()) != null) {
...Do Stuff
}
Changing to:
while (br.ready()) {
response = br.readLine();
...Do Stuff
}
I can read everything just fine. br is a BufferedReader object, BTW.
There is data already in the first BufferedReader (that has been read from the socket, and is no longer available from the socket), so pass the BufferedReader created in the first example to the class that reads the app specific messages, rather then creating a new BufferedReader from the socket.
I had the same problem and here is my solution:
try {
StringBuilder response = new StringBuilder();
response.append("SERVER -> CLIENT message:").append(CRLF);
//Infinite loop
while (true) {
//Checks wheather the stream is ready
if (in.ready()) {
//Actually read line
lastLineFromServer = in.readLine();
//If we have normal behavior at the end of stream
if (lastLineFromServer != null) {
response
.append(lastLineFromServer)
.append(CRLF);
} else {
return response.toString();
}
} else {//If stream is not ready
//If number of tries is not exceeded
if (numberOfTry < MAX_NUMBER_OF_TRIES) {
numberOfTry++;
//Wait for stream to become ready
Thread.sleep(MAX_DELAY_BEFORE_NEXT_TRY);
} else {//If number of tries is exeeded
//Adds warning that things go weired
response
.append("WARNING \r\n")
.append("Server sends responses not poroperly.\r\n")
.append("Response might be incomplete.")
.append(CRLF);
return response.toString();
}
}
}
} catch (Exception ex) {
ex.printStackTrace();
return "";
}
The answer might be late but this is the simplest and latest answer in 2020, just use the simple way to receive the data from the socket server or client using the input stream read() method.
EOFException will be thrown when the client is disconnected or the server closed the connection.
private String waitForData() throws IOException {
String data = "";
do {
int c = inputStream.read();
if (c > -1) data += (char) c;
else throw new EOFException();
} while (inputStream.available() > 0);
return data;
}
All sample function I've seen so far avoid, for some reason, returning a string. I am a total rookie as far as Java goes, so I am not sure whether this is intentional. I know that in C++ for example, returning a reference to a string is way more efficient than returning a copy of that string.
How does this work in Java?
I am particularly interested in Java for Android, in which resources are more limited than desktop/server environment.
To help this question be more focused, I am providing a code snippet in which I am interested in returning (to the caller) the string page:
public class TestHttpGet {
private static final String TAG = "TestHttpGet";
public void executeHttpGet() throws Exception {
BufferedReader in = null;
try {
HttpClient client = new DefaultHttpClient();
HttpGet request = new HttpGet();
request.setURI(new URI("http://www.google.com/"));
HttpResponse response = client.execute(request); // actual HTTP request
// read entire response into a string object
in = new BufferedReader(new InputStreamReader(response.getEntity().getContent()));
StringBuffer sb = new StringBuffer("");
String line = "";
String NL = System.getProperty("line.separator");
while ((line = in.readLine()) != null) {
sb.append(line + NL);
}
in.close();
String page = sb.toString();
Log.v(TAG, page); // instead of System.out.println(page);
}
// a 'finally' clause will always be executed, no matter how the program leaves the try clause
// (whether by falling through the bottom, executing a return, break, or continue, or throwing an exception).
finally {
if (in != null) {
try {
in.close(); // BufferedReader must be closed, also closes underlying HTTP connection
}
catch (IOException e) {
e.printStackTrace();
}
}
}
}
}
In the example above, can I just define:
public String executeHttpGet() throws Exception {
instead of:
public void executeHttpGet() throws Exception {
and return:
return (page); // Log.v(TAG, page);
A String in java corresponds more or less to std::string const * in c++. So, it's cheap to pass around, and can't be modified after it's created (String is immutable).
String is a reference type - so when you return a string, you're really just returning a reference. It's dirt cheap. It's not copying the contents of the string.
In java most of the time you return something, you return it by reference. There's no object copying or cloning of any kind. So it is fast.
Also, Strings in Java are immutable. No need to worry about that either.