I'm using Apache HttpClient to fetch meta-data from radio stream. What I want to do is make a GET request, read some bytes then close the stream. For some streams, it works properly but for some others, the closing of the stream hangs. It looks like it's still receiving data and not closing the connection while it's happening.
public SongInfo retrieveMetadata(String streamUrl) {
HttpClient httpClient = HttpClients.createDefault();
HttpGet httpGet = new HttpGet(streamUrl);
httpGet.addHeader("Icy-MetaData", "1");
httpGet.addHeader("Connection", "close");
httpGet.addHeader("Accept", "");
HttpResponse response;
try {
response = httpClient.execute(httpGet);
} catch (IOException e) {
log.warn("An exception occurred while fetching the stream", e);
return null;
}
if (response.getStatusLine().getStatusCode() != 200) {
log.warn("Could not fetch stream. Status line: "+ response.getStatusLine());
return null;
}
int metadataOffset = retrieveMetadataOffset(response);
if (metadataOffset == 0) {
log.info("Could not find metadata for url:"+ streamUrl);
return null;
}
List<Metadata> metadata = extractMetadata(response, metadataOffset);
if (metadata == null || metadata.isEmpty()) {
return null;
}
return extractSongInfo(metadata);
}
private List<Metadata> extractMetadata(HttpResponse response, int metadataOffset) {
String metadataStr;
try(InputStream stream = response.getEntity().getContent()) {
if (stream.skip(metadataOffset) != metadataOffset) {
log.warn("Something went wrong while skipping to metadata offset");
return null;
}
int metaDataLength = stream.read() * 16;
metadataStr = getMetadataStr(stream, metaDataLength);
if (metadataStr == null) {
return null;
}
} catch (IOException e) {
log.warn("Something went wrong while reading the stream", e);
return null;
} //Hangs here
//rest of the method
}
I noticed that the stream returned by response.getEntity().getContent() is of type EofSensorInputStream so I'm wondering if it's waiting for a EOF character which is never coming.
Example of stream for which the code works properly and the stream closes properly: https://icecast.omroep.nl/radio2-bb-mp3, http://live-mp3-128.kexp.org
Example of stream for which the code doesn't work properly as the stream never closes and hangs forever: https://kexp-mp3-128.streamguys1.com/kexp128.mp3
This problem occurs because calling close() on the content stream will try to consume the remaining content. This is currently not explicitly mentioned in the documentation (see also HTTPCORE-656), but is mentioned in the tutorials:
The difference between closing the content stream and closing the response is that the former will attempt to keep the underlying connection alive by consuming the entity content while the latter immediately shuts down and discards the connection.
[...]
There can be situations, however, when only a small portion of the entire response content needs to be retrieved and the performance penalty for consuming the remaining content and making the connection reusable is too high, in which case one can terminate the content stream by closing the response.
Therefore in your case it seems appropriate to not close the InputStream returned by getContent() but only close the HttpResponse (which you are apparently not doing yet).
Related
I'm trying to connect a java application to an external api for GuildWars2.
The link I am trying to test is:
http://api.guildwars2.com/v2/commerce/listings
A list of int IDs are returned when navigating to that page within a browser.
As a learning practice, I am trying to get that list of id's when running my java application.
I use the following code (hopefully it formats correct, currently on my phone, trying to program remotely to my desktop):
public class GuildWarsAPI
{
public static void main(String[] args)
{
GuildWarsAPI api = new GuildWarsAPI();
api.getAPIResponse("http://api.guildwars2.com/v2/commerce/listings");
}
public void getAPIResponse(String URLString)
{
URL url = null;
try {
url = new URL(URLString);
} catch (MalformedURLException e1) {
e1.printStackTrace();
}
HttpURLConnection connection = null;
try {
connection = (HttpURLConnection) url.openConnection();
} catch (IOException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
if (connection != null)
{
System.out.println("connection success");
connection.setRequestProperty("Accept", "application/json");
connection.setDoInput(true);
connection.setDoOutput(true);
connection.setReadTimeout(10000);
connection.setConnectTimeout(10000);
try {
/*BufferedReader in = new BufferedReader(new InputStreamReader(connection.getInputStream()));
StringBuilder input = new StringBuilder();
String nextLine = null;
while ((nextLine = in.readLine()) != null)
{
System.out.println("adding output");
input.append(nextLine);
}*/
InputStream in = new BufferedInputStream(connection.getInputStream());
int b = 0;
while ((b = in.read()) != -1)
{
System.out.println("byte:" + b);
}
System.out.println("done");
}
catch (Exception e) {
e.printStackTrace();
}
finally {
connection.disconnect();
System.out.println("closed");
}
}
}
}
Upon running my class, it immediately prints out connection success, done, closed. It definitely isnt waiting for the timeouts, and i've been trying to play with that, the request header, and the DoInput/DoOutput. I stepped through it, and it appears as if it connects, and just doesnt receive any bytes of information back. (doesnt go into the while loop)
So, while my ultimate question is: How do I get the id's back like I expect?, my other question is: how can I figure out how to get the other id's back like I expect?
Your code is getting response code 302 Found. It should follow the Location: header to the new location, as followRedirects is true by default, but it isn't. The server is however returning a Location: header of https://api.guildwars2.com/v2/commerce/listings. I don't know why HttpURLConnection isn't following that, but the simple fix is to use https: in the original URL.
You're setting doOutput(true) but you aren't sending any output.
Your code is poorly structured. Code that depends on the success of code in a prior try block should be inside that same try block. I would have the method throw MalformedURLException and IOException and not have any internal try/catch blocks at all.
In my experience, wrestling with HttpUrlConnection is more trouble than it's worth.
It's hard to debug, hard to use, and provides very little support for complex http operations.
There are a bunch of better options.
My default choice is Apache HttpConponents Client (http://hc.apache.org/). It's not necessarily any better than all the other options, but it's quite well documented and widely used.
I am sending some data to server from my android device and in return getting Json Response back from the server.The Data returned is a quite large chunk of data (Multiple Images precisely) .
I am echoing the statusCode returned from the Server and it is 200. but The JAVA code keeps on waiting on a statement
objHttpEntity = objHttpResponse.getEntity();
statusCode=objHttpResponse .getStatusLine().getStatusCode();
String responseDataString = EntityUtils.toString(objHttpEntity);//this statement
and in logs, I see my garbage collector running,getting the memory back from the resources to accomodate the currently received data.
In devices having RAM 1GB or less, the application is crashing abruptly giving out an OutOfMermoryException .But in devices having RAM more than the latter, the application waits, and waits, and waits and finally execute the rest of the consecutive statements of my code.
How can I get rid of the exception(in devices having less RAM). OR can reduce the time for which the code waits.
Receiving status code of 200, clearly signifies that Server is done with its work, now all the handling must be performed on the client(Device) side.
Note : Already gone through all the three of the questions posted here on stackOverflow about the issue, but none of them was appropriate and was unable to solve the issue.
Why don't you try this one?
HttpGet get = new HttpGet(apiURL);
HttpResponse response = client.execute(get);
HttpEntity resEntity = response.getEntity();
InputStream is = resEntity.getContent();
String result = convertStreamToString(is);
public static String convertStreamToString(InputStream is) {
/*
* To convert the InputStream to String we use the
* BufferedReader.readLine() method. We iterate until the BufferedReader
* return null which means there's no more data to read. Each line will
* appended to a StringBuilder and returned as String.
*/
BufferedReader reader = new BufferedReader(new InputStreamReader(is));
StringBuilder sb = new StringBuilder();
String line = null;
try {
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
is.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return sb.toString();
}
My project involves reading the status of an LED on Arduino, from java. It will move on to reading temperature from Arduino, but I got stuck. So here it is:
I send the "Turn on/off!" message from my java program, and I want it to show if the LED is on and off.
So when I send "192.168.0.100/ON", the LED turns on and I shoud get the "ON" message in my program.
The code on Arduino:
byte mac[] = {0x90, 0xA2, 0xDA, 0x0D, 0x2F, 0xD4 };
IPAddress ip(192,168,0,100);
EthernetServer server(80);
String message = String(30);
void setup()
{
pinMode(2, OUTPUT);
// start the Ethernet connection and the server:
Ethernet.begin(mac, ip);
server.begin();
Serial.begin(9600);
}
void loop()
{
// listen for incoming clients
EthernetClient client = server.available();
if (client) {
// an http request ends with a blank line
while (client.connected()) {
if (client.available()) {
char c = client.read();
if (message.length() < 30) {
message += c;
}
Serial.print(message);
// if you've gotten to the end of the line (received a newline
// character) and the line is blank, the http request has ended,
// so you can send a reply
if (c == '\n') {
if (message.indexOf("ON") > 0) {
digitalWrite(2, HIGH);
client.print("ON");
}
if (message.indexOf("OFF") > 0) {
digitalWrite(2, LOW);
client.print("OFF");
}
message = "";
client.stop();
}
}
}
// give the web browser time to receive the data
delay(1);
}
}
The code in java:
public class TestClient {
public static void main(String[] args) {
HttpURLConnection connection = null;
BufferedReader serverResponse = null;
try {
// OPEN CONNECTION
connection = (HttpURLConnection) new URL("http://192.168.0.100/ON")
.openConnection();
connection.connect();
// RESPONSE STREAM
serverResponse = new BufferedReader(new InputStreamReader(
connection.getInputStream()));
// READ THE RESPOSNE
String line;
while ((line = serverResponse.readLine()) != null) {
System.out.println(line);
}
} catch (MalformedURLException mue) {
mue.printStackTrace();
} catch (IOException ioe) {
ioe.printStackTrace();
} finally {
if (connection != null)
connection.disconnect();
if (serverResponse != null) {
try {
serverResponse.close();
} catch (Exception ex) {
}
}
}
}
}
What happens: The LED turns on, but I get this error in java:
java.net.SocketException: Unexpected end of file from server at TestClient.main(TestClient.java:23) -> connection.getInputStream();
What I want: After sending the "ON" message, it should be printed in console.
Mention: if I send 192.168.0.100/ON from my browser, the LED turns on and the message appears in the web page.
There are two issues here:
If the exception is thrown when getting the InputStream, then it's happening because the connection has been closed at that point, and this happens because the Arduino sends the message then immediately "closes" the client, effectively terminating the connection. You can do three things:
a. Try and creating the input stream before calling connect(), but this most likely will fail due to the connection not existing at that point.
b. Put a delay before calling client.stop();
c. (recommended) Let the client close the connection, don't do it on the server.
Try adding a \n after ON and OFF in client.print() method, in the Arduino code.
client.print("ON\n");
...
client.print("OFF\n");
readLine() will read until the first end-of-line char which never comes.
Try reverse the order in your finally block. Close the input stream before you close the socket.
As this guy here says, the Arduino can serve HTML pages, so I guessed my HttpURLConnection must know about this. Here, it says that "The version of an HTTP message is indicated by an HTTP-Version field in the first line of the message."
So I simply added the following code into the Arduino sketch, right after I check (c == "\n"):
client.println("HTTP/1.1 200 OK");
client.println("Content-Type: text/html");
client.println();
I didn't get to this very quickly, but after reading others` code and the resources I mentioned above, I came to this conclusion. The explanation could be as well wrong for all I know, but it worked, and my project runs.
I'm new to BlackBerry application development. I have one silly error while during HttpConnection in httpcon.getResponseCode() method giving source not found error.
Please, can any one figure out this error?
Here is my method:
net.rim.device.api.io.transport.ConnectionFactory cf = new net.rim.device.api.io.transport.ConnectionFactory();
httpConn = (HttpConnection) cf.getConnection(url).getConnection();
httpConn.setRequestMethod(HttpConnection.POST);
httpConn.setRequestProperty("User-Agent",
"Profile/MIDP-2.0 Confirguration/CLDC-1.0");
httpConn.setRequestProperty("Accept_Language", "en-US");
httpConn.setRequestProperty("Content-Type",
"application/x-www-form-urlencoded");
httpConn.setRequestProperty("Content-Length",
Integer.toString(postData.length));
os = httpConn.openOutputStream();
os.write(("LoginID=yahoo#sol.com&Password=yah123")
.getBytes("UTF-8"));
os.flush();
os.close();
try {
responseCode = httpConn.getResponseCode();
} catch (IOException ex1) {
//check if it's eof, if yes retrieve code again
if (-1 != ex1.getMessage().indexOf("EOF")) {
try {
responseCode = httpConn.getResponseCode();
} catch (IOException ex2) {
System.out.println(ex2.getMessage());
// handle exception
}
} else {
System.out.println(ex1.getMessage());
// handle exception
}
}
int status = httpConn.getResponseCode();
if (status == HttpConnection.HTTP_OK) {
InputStream input = httpConn.openInputStream();
byte[] bytes = IOUtilities.streamToBytes(input);
StringBuffer raw = new StringBuffer(new String(bytes));
raw.insert(0, "bytes received]\n");
raw.insert(0, bytes.length);
raw.insert(0, '[');
url = raw.toString();
input.close();
} else {
url = "response code = " + status;
}
httpConn.close();
} catch (IOCancelledException e) {
System.out.println(e.toString());
return "";
} catch (IOException e) {
return "";
}
return "";
}
Update: I am not trying to step into getResponseCode(). Eclipse is stopping execution at that point, and showing the Source Not Found error.
#Nate yes we are step into method that time only we are getting Run Time Error.Error is displayed When Getresponsecode() is called
Does your eclipse error look like this?
getResponseCode() is a method in the HttpConnection class. This is RIM's code, not yours. You normally shouldn't need to step into that code. Just step over that line while debugging. The only thing you should want to see is the result of that method, not what happens inside.
Even if net_rim_api.jar is included in your project, that simply gives you the binary version of RIM's classes, including HttpConnection. It doesn't provide the Java source code for that class. In order to step into a method, you would need to have the source code, too.
I have found my mistake.When calling HttpConnection. getResponseCode() in that URL String we have to add deviceside=true then only getresponsecode() will be called without throwing any http exceptions.
For Example:
httpClient ht = new httpClient();
String str = ht.getHttpClientResponse("https://www.google.co.in;deviceside=true",post);
Note:
(From #Nate Comment)
we don't always want deviceside=true in your connection strings. It depends on whether you're running on a device or a simulator, and what kind of network's available.
I'm writing a program that connects to a servlet thanks to a HttpURLConnection but I stuck while checking the url
public void connect (String method) throws Exception {
server = (HttpURLConnection) url.openConnection ();
server.setDoInput (true);
server.setDoOutput (true);
server.setUseCaches (false);
server.setRequestMethod (method);
server.setRequestProperty ("Content-Type", "application / xml");
server.connect ();
/*if (server.getResponseCode () == 200)
{
System.out.println ("Connection OK at the url:" + url);
System.out.println ("------------------------------------------- ------- ");
}
else
System.out.println ("Connection failed");
}*/
I got the error :
java.net.ProtocolException: Cannot write output after reading input.
if i check the url with the code in comments but it work perfectly without it
unfortunately, I need to check the url so i think the problem comes from the getResponseCode method but i don t know how to resolve it
Thank you very much
The HTTP protocol is based on a request-response pattern: you send your request first and the server responds. Once the server responded, you can't send any more content, it wouldn't make sense. (How could the server give you a response code before it knows what is it you're trying to send?)
So when you call server.getResponseCode(), you effectively tell the server that your request has finished and it can process it. If you want to send more data, you have to start a new request.
Looking at your code you want to check whether the connection itself was successful, but there's no need for that: if the connection isn't successful, an Exception is thrown by server.connect(). But the outcome of a connection attempt isn't the same as the HTTP response code, which always comes after the server processed all your input.
I think the exception is not due toprinting url. There should some piece of code which is trying to write to set the request body after the response is read.
This exception will occur if you are trying to get HttpURLConnection.getOutputStream() after obtaining HttpURLConnection.getInputStream()
Here is the implentation of sun.net.www.protocol.http.HttpURLConnection.getOutputStream:
public synchronized OutputStream getOutputStream() throws IOException {
try {
if (!doOutput) {
throw new ProtocolException("cannot write to a URLConnection"
+ " if doOutput=false - call setDoOutput(true)");
}
if (method.equals("GET")) {
method = "POST"; // Backward compatibility
}
if (!"POST".equals(method) && !"PUT".equals(method) &&
"http".equals(url.getProtocol())) {
throw new ProtocolException("HTTP method " + method +
" doesn't support output");
}
// if there's already an input stream open, throw an exception
if (inputStream != null) {
throw new ProtocolException("Cannot write output after reading
input.");
}
if (!checkReuseConnection())
connect();
/* REMIND: This exists to fix the HttpsURLConnection subclass.
* Hotjava needs to run on JDK.FCS. Do proper fix in subclass
* for . and remove this.
*/
if (streaming() && strOutputStream == null) {
writeRequests();
}
ps = (PrintStream)http.getOutputStream();
if (streaming()) {
if (strOutputStream == null) {
if (fixedContentLength != -) {
strOutputStream =
new StreamingOutputStream (ps, fixedContentLength);
} else if (chunkLength != -) {
strOutputStream = new StreamingOutputStream(
new ChunkedOutputStream (ps, chunkLength), -);
}
}
return strOutputStream;
} else {
if (poster == null) {
poster = new PosterOutputStream();
}
return poster;
}
} catch (RuntimeException e) {
disconnectInternal();
throw e;
} catch (IOException e) {
disconnectInternal();
throw e;
}
}
I have this problem too, what surprises me is that the error is caused by my added code System.out.println(conn.getHeaderFields());
Below is my code:
HttpURLConnection conn=(HttpURLConnection)url.openConnection();
conn.setRequestMethod("POST");
configureConnection(conn);
//System.out.println(conn.getHeaderFields()); //if i comment this code,everything is ok, if not the 'Cannot write output after reading input' error happens
conn.connect();
OutputStream os = conn.getOutputStream();
os.write(paramsContent.getBytes());
os.flush();
os.close();
I had the same problem.
The solution for the problem is that you need to use the sequence
openConnection -> getOutputStream -> write -> getInputStream -> read
That means..:
public String sendReceive(String url, String toSend) {
URL url = new URL(url);
URLConnection conn = url.openConnection();
connection.setDoInput(true);
connection.setDoOutput(true);
connection.sets...
OutputStreamWriter out = new OutputStreamWriter(conn.getOutputStream());
out.write(toSend);
out.close();
BufferedReader in = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String receive = "";
do {
String line = in.readLine();
if (line == null)
break;
receive += line;
} while (true);
in.close();
return receive;
}
String results1 = sendReceive("site.com/update.php", params1);
String results2 = sendReceive("site.com/update.php", params2);
...