Develop Reference

How to read body following an 101 response code?

From the docs,
immediately after sending the 101
(Switching Protocols) response, the server is expected to continue
responding to the original request as if it had received its
equivalent within the new protocol (i.e., the server still has an
outstanding request to satisfy after the protocol has been changed,
and is expected to do so without requiring the request to be
repeated).
If the Upgrade header field is received in a GET request
and the server decides to switch protocols, it first responds with a
101 (Switching Protocols) message in HTTP/1.1 and then immediately
follows that with the new protocol's equivalent of a response to a
GET on the target resource. This allows a connection to be upgraded
to protocols with the same semantics as HTTP without the latency cost
of an additional round trip.
I have made my interceptors(Using OkHttp) as follows
public class H2cUpgradeRequestInterceptor implements Interceptor {
private static final Log logger = LogFactory.getLog(H2cUpgradeRequestInterceptor.class);
#Override
public Response intercept(Chain chain) throws IOException {
Request request = chain.request();
Request upgradeRequest = request.newBuilder().addHeader("Connection", "Upgrade, HTTP2-Settings")
.addHeader("Upgrade", "h2c").addHeader("HTTP2-Settings", "AAMAAABkAARAAAAAAAIAAAAA").build();
Response upgradeResponse = chain.proceed(upgradeRequest);
if (upgradeResponse != null && upgradeResponse.code() == HttpStatus.SC_SWITCHING_PROTOCOLS) {
logger.debug("Switching Protocols success"); // Success. Got 101 in reply.
}
upgradeResponse.body(); // returns null
// I am clueless on how to read the response hereafter.
// As per the docs, the 101 status code reply will be followed by a data stream. How can I access this?
return upgradeResponse;
}
}
So basically, for a single request. I will receive 101 as response first if the upgrade is successful, then followed by another response as per the upgraded protocol(if my understanding is right?). Is there anyway to achieve this with OkHttp? Or, Any other client also would be helpful.
Thanks!

Is it necessary to read data before close http response?

Is it necessary to read request data before calling the close method of response inside a http server in java?
I have a http server written by sun's http server. When I receive a http request, I just simply do the following steps:
write my text to response body
close the response
Does it matter to read the request body before closing the response?
Note that I have too many http requests on my server. (10000 call/sec.)

My benchmark says if request body does not have too much data, there is no big difference between two methods. I didn't get any timeout and other type of exceptions using both methods.
This is the time benchmark result for 1M request using 100 concurrent threads (10000 request per thread) on both methods:
Don't read request body: 210,581ms
Read request body: 206,599ms

jersey : receive response from web-service as json file

nni have a problem calling a web service using jersey client.
I tried successfully as a test with : "http://query.yahooapis.com/v1/public/yql?q=select%20item%20from%20weather.forecast%20where%20location%3D%2248907%22&format=json"
using this code :
Client client = Client.create();
WebResource webResource = client.resource("http://query.yahooapis.com/v1/public/yql?q=select%20item%20from%20weather.forecast%20where%20location%3D%2248907%22&format=json");
ClientResponse response = webResource.accept("application/json").get(ClientResponse.class);
if (response.getStatus() != 200) {
throw new RuntimeException("Failed : HTTP error code : " + response.getStatus());
}
String json = response.getEntity(String.class);
System.out.println(json);
but i can't do it when i call amazon webservice : http://ws.amazon.com/widgets/q?Operation=GetResults&Keywords=cool&SearchIndex=All&multipageStart=0&InstanceId=0&multipageCount=10&TemplateId=8002&ServiceVersion=20070822&MarketPlace=US
Is it because, I get a json file as response ?
Any help please ?

After experimenting with the Amazon web service with various form HTTP requests. I finally figured out that the problem is because of the User-Agent value sent in the HTTP Header.
For some reason, Amazon Rest Service is not able to handle the presence of period character . in the HTTP Header under User-Agent.
When sending a HTTP request with . as below
GET http://ws.amazon.com/widgets/q?Operation=GetResults&Keywords=cool&SearchIndex=All&multipageStart=0&InstanceId=0&multipageCount=10&TemplateId=8002&ServiceVersion=20070822&MarketPlace=US HTTP/1.1
User-Agent: Java.
Host: ws.amazon.com
Connection: keep-alive
Amazon WS sends a HTTP response without Body content
HTTP/1.1 200 OK
Date: Fri, 27 Sep 2013 19:29:54 GMT
Server: Server
Content-Length: 0
Vary: Accept-Encoding,User-Agent
Cneonction: close
Content-Type: text/plain
If the . is removed from the Content-Type, the response body does contain the detailed Json Content. This most likely looks like an issue on Amazon Rest Service implementation.
You can change the code as follows to see the Json content and get away with the problem
ClientResponse response = webResource.header("User-Agent", "SomeAgentNameWithoutPeriodChar").get(ClientResponse.class);

Android HttpPost message won't send its payload across the wire

I'm trying to send a simple string as the contents of a HttpPost message.
The problem is, the body of the HttpPost message never makes it to the wire. (Says the Wireshark capture). The header looks just fine though (including the correctly calculated Content-Length.
Here's what the code looks like:
String url = "http://1.2.3.4/resource";
HttpClient client = new DefaultHttpClient();
String cmd = "AT+AVLPOS\r\n";
StringEntity se = new StringEntity(cmd);
se.setContentType("text/plain");
HttpPost request = new HttpPost(url);
request.setHeader("Content-Type","text/plain");
request.setEntity(se);
HttpResponse response = client.execute(request);
[...]
The string should be ASCII-encoded, but that's a detail at this point.
This is what shows up in WireShark:
-> note that lines marked with + are what's sent, and - is what's received.
+POST /resource HTTP/1.1
+Content-Type: text/plain
+Content-Length: 11
+Host: 1.2.3.4
+Connection: Keep-Alive
+User-Agent: Apache-HttpClient/UNAVAILABLE (java 1.4)
+Expect: 100-Continue
-HTTP/1.1 200 OK
-Content-Type: text/plain
-Transfer-Encoding: chunked
-4
-OK
This is what should show up (wrote a very simple console app in C# to do this, it just works):
+POST /resource HTTP/1.1
+Content-Type: text/plain
+Host: 1.2.3.4
+Content-Length: 11
+Expect: 100-continue
+Connection: Keep-Alive
+
-HTTP/1.1 200 OK
-Content-Type: text/plain
-Transfer-Encoding: chunked
-
+AT+AVLPOS
+
-4
-OK
-
-48
-$AVTMR,99999999,204810,A,1234.2218,N,0123.1051,E,0,20,150811,0,REQ*69
-
-0
-
Any suggestions?

I've figured it out, AND I've learned something today.
Long story short: disable the HttpClient's HTTP Post expect-continue handshake, by setting one of its parameters, this will send the whole request message in one chunk.
//set up HttpPost request as before
HttpClient client = new DefaultHttpClient();
client.getParams().setBooleanParameter("http.protocol.expect-continue", false);
HttpResponse response = client.execute(request);
[...]
Now here's how I got there, maybe this will help someone someday.
First I derived from a HttpEntityWrapper and used that as my request entity to see what gets called when, and found out that the Entity's writeTo(OutputStream) method was never called at all.
Then I started to look at why, in the case of the "correct" behaviour, the POST request wasn't sent all at once, and instead, the request headers were sent, then the response header is received, THEN the request body is sent.
IT's all got to do with the HTTP Post expect-continue handshake. Read more about it on Haacked.
If the expect-continue header is sent in a request, the Http server SHOULD reply with a 100 Continue message signifying "OK, I will accept your message", or with an error, stopping the possibly long POST message in its tracks.
Unfortunately, the web server I run against is a bare bones implementation that runs on a chip, and it sends the wrong reply (200 OK instead of 100 Continue).
The default implementation of the .NET Http Client seems to be more forgiving here: it treats the 200 message as 100 Continue, shrugs, and gets on its way to send the request body.
Not so with the Http client implementation of Android (API level 7).
Next thing I tried was to disable the expect-continue handshake completely, in order to make the HttpClient send the whole request. To my surprise and joy, this was handled fine by the web server, which replied with the information I wanted. Yay!

How can I send an HTTP Response using only standard network libraries?

I'm working on my first homework project in a web programming class, which is to write a simple web server in Java. I'm at the point where I have data being transmitted back and forth, and to the untrained eye, my baby server seems to be working fine. However, I can't find a way to send appropriate responses. (In other words, an invalid page request would show a 404-ish HTML page, but it still returns a 200 OK status when I view response headers).
I'm limited to being able to use standard network libraries for socket management and standard I/O libraries to read and write bytes and strings from an input stream. Here's some pertinent code:
From my main...
ServerSocket servSocket = new ServerSocket(port, 10); // Bind the socket to the port
System.out.println("Opened port " + port + " successfully!");
while(true) {
//Accept the incoming socket, which means that the server process will
//wait until the client connects, then prepare to handle client commands
Socket newDataSocket = servSocket.accept();
System.out.println("Client socket created and connected to server socket...");
handleClient(newDataSocket); //Call handleClient method
}
From the handleClient method...(inside a loop that parses the request method and path)
if(checkURL.compareTo("/status") == 0) { // Check to see if status page has been requested
System.out.println("STATUS PAGE"); // TEMPORARY. JUST TO MAKE SURE WE ARE PROPERLY ACCESSING STATUS PAGE
sendFile("/status.html", dataStream);
}
else {
sendFile(checkURL, dataStream); // If not status, just try the input as a file name
}
From sendFile method...
File f = new File(where); // Create the file object
if(f.exists() == true) { // Test if the file even exists so we can handle a 404 if not.
DataInputStream din;
try {
din = new DataInputStream(new FileInputStream(f));
int len = (int) f.length(); // Gets length of file in bytes
byte[] buf = new byte[len];
din.readFully(buf);
writer.write("HTTP/1.1 200 OK\r\n"); // Return status code for OK (200)
writer.write("Content-Length: " + len + "\r\n"); // WAS WRITING TO THE WRONG STREAM BEFORE!
writer.write("Content-Type: "+type+"\r\n\r\n\r\n"); // TODO VERIFY NEW CONTENT-TYPE CODE
out.write(buf); // Writes the FILE contents to the client
out.flush();
out.close();
} catch (FileNotFoundException e) {
e.printStackTrace(); // Not really handled since that's not part of project spec, strictly for debug.
}
}
else {
writer.write("HTTP/1.1 404 Not Found\r\n"); // Attempting to handle 404 as simple as possible.
writer.write("Content-Type: text/html\r\n\r\n\r\n");
sendFile("/404.html", sock);
}
Can anybody explain how, in the conditional from sendFile, I can change the response in the 404 block (Like I said before, the response headers still show 200 OK)? This is bugging the crap out of me, and I just want to use the HTTPResponse class but I can't. (Also, content length and type aren't displayed if f.exists == true.)
Thanks!

Edit It looks to me like in the 404 situation, you're sending something like this:
HTTP/1.1 404 Not Found
Content-Type: text/html
HTTP/1.1 200 OK
Content-Length: 1234
Content-Type: text/html
...followed by the 404 page. Note the 200 line following the 404. This is because your 404 handling is calling sendFile, which is outputting the 200 response status code. This is probably confusing the receiver.
Old answer that missed that:
An HTTP response starts with a status line followed (optionally) by a series of headers, and then (optionally) includes a response body. The status line and headers are just lines in a defined format, like (to pick a random example):
HTTP/1.0 404 Not Found
To implement your small HTTP server, I'd recommend having a read through the spec and seeing what the responses should look like. It's a bit of a conceptual leap, but they really are just lines of text returned according to an agreed format. (Well, it was a conceptual leap for me some years back, anyway. I was used to environments that over-complicated things.)
It can also be helpful to do things like this from your favorite command line:
telnet www.google.com 80
GET /thispagewontbefound
...and press Enter. You'll get something like this:
HTTP/1.0 404 Not Found
Content-Type: text/html; charset=UTF-8
X-Content-Type-Options: nosniff
Date: Sun, 12 Sep 2010 23:01:14 GMT
Server: sffe
Content-Length: 1361
X-XSS-Protection: 1; mode=block
...followed by some HTML to provide a friendly 404 page. The first line above is the status line, the rest are headers. There's a blank line between the status line/headers and the first line of content (e.g., the page).

The problem you are seeing is most likely related to a missing flush() on your writer. Depending on which type of Writer you use the bytes are first written to a buffer that needs to be flushed to the stream. This would explain why Content-Length and Content-Type are missing in the output. Just flush it before you write additional data to the stream.
Further you call sendFile("/404.html", sock);. You did not post the full method here - but I suppose that you call it recursively inside sendFile and thus send the 200 OK status for your file /404.html.

Based on your reported symptoms, I think the real problem is that you are not actually talking to your server at all! The evidence is that 1) you cannot get a 404 response, and 2) a 200 response does not have the content length and type. Neither of these should be possible ... if you are really talking to the code listed above.
Maybe:
you are talking to an older version of your code; i.e. something is going wrong in your build / deploy cycle,
you are (mistakenly) trying to deploy / run your code in a web container (Jetty, Tomcat, etc), or
your client code / browser is actually talking to a different server due to proxying, an incorrect URL, or something like that.
I suggest that you add some trace printing / logging at appropriate points of your code to confirm that it is actually being invoked.