Develop Reference

websocket protocol error when server deflates payload

I'm writing a Java websocket server and it now receives the opening handshake message from the client (in Chrome Version 57.0.2987.133 (64-bit)) and responds to complete the handshake, both shown below.
Received: GET / HTTP/1.1
Received: Host: localhost:6789
Received: Connection: Upgrade
Received: Pragma: no-cache
Received: Cache-Control: no-cache
Received: Upgrade: websocket
Received: Origin: http://localhost:8080
Received: Sec-WebSocket-Version: 13
Received: User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36
Received: Accept-Encoding: gzip, deflate, sdch, br
Received: Accept-Language: en-US,en;q=0.8
Received: Sec-WebSocket-Key: L1IiUSGijbGmTpthWsebOg==
Received: Sec-WebSocket-Extensions: permessage-deflate; client_max_window_bits
Sending: HTTP/1.1 101 Switching Protocols
Sending: Upgrade: websocket
Sending: Connection: Upgrade
Sending: Sec-WebSocket-Accept: L5HXnDJGDMYbWr5gRcQMOwKNf3Q=
Sending: Accept-Encoding: gzip, deflate
Sending: Sec-WebSocket-Extensions: permessage-deflate; client_no_context_takeover; server_no_context_takeover
Now, the client can send messages and it does so without a problem and my code successfully uncompresses them using java.util.zip.Deflater and... if my server responds with header bytes 0x81 (fin, no compression, text) and 0x5 then the bytes for 'hello' (as an example) then the websocket client in javascript on Chrome is entirely happy but when I try to compress the response, the client always closes the connection citing error code 1002 and the text 'Websocket protocol error'.
DEFLATE
public void sendMessageDeflated(String rxMessage, OutputStream streamOut) {
System.out.println("Message back to client is: " + rxMessage);
// And then compress the response and send it out.
Deflater compressor = new Deflater(Deflater.DEFLATED);
try {
int headerLength = 2;
byte unzippedMsg[] = rxMessage.getBytes("UTF-8");
compressor.setInput(unzippedMsg);
compressor.finish();
byte zippedMsg[] = new byte[2048]; // Nasty constant but will have to do for now.
int toCompressLength = unzippedMsg.length;
int compLength = compressor.deflate(zippedMsg, headerLength, zippedMsg.length - headerLength);
compressor.end();
zippedMsg[0] = (byte)0xC1; // FIN bit, compression plus opcode for TEXT MESSAGE
zippedMsg[1] = (byte)((byte)0x00 | (byte)compLength); // No mask on return data.
streamOut.write(zippedMsg, 0, compLength + headerLength);
} catch ( IOException ioEx ) {
// TBD
System.out.println("IOException: " + ioEx.toString());
} catch ( Exception ex ) {
// TBD
System.out.println("IOException: " + ex.toString());
}
}
GZIP
public void sendMessageGZipped(String rxMessage, OutputStream streamOut) {
// Do something with the message here...
System.out.println("Message back to client is: " + rxMessage);
// And then compress the response and send it out.
try {
int headerLength = 2;
byte unzippedMsg[] = rxMessage.getBytes("UTF-8");
int toCompressLength = unzippedMsg.length;
ByteArrayOutputStream baos = new ByteArrayOutputStream();
GZIPOutputStream gzipOut = new GZIPOutputStream(baos);
gzipOut.write(unzippedMsg, 0, toCompressLength);
gzipOut.close();
byte[] payload = baos.toByteArray();
byte header[] = new byte[32];
header[0] = (byte)0xC1; // FIN bit plus opcode for TEXT MESSAGE
header[1] = (byte)((byte)0x00 | (byte)payload.length); // No mask on return data.
streamOut.write(header, 0, 2);
streamOut.write(payload);
} catch ( IOException ioEx ) {
// TBD
System.out.println("IOException: " + ioEx.toString());
} catch ( Exception ex ) {
// TBD
System.out.println("IOException: " + ex.toString());
}
}
I've tried switching to the opcode for binary thinking that perhaps compressed text = binary but that didn't work. I really can't see what I've done wrong or missed out. This implementation does not include a sliding compression window that spans messages. I think the response headers make that clear. Help gratefully accepted.

I solved this eventually by building the client-side websocket code and seeing the exception thrown when it tried to Inflate the message: "invalid stored block lengths". That led me to this post: Java decompressing array of bytes which talks about Deflate being able to compress with zlib wrapper or without. So change one line in processMessage2 from the example code above to this...
Deflater compressor = new Deflater(Deflater.DEFLATED, true);
...which sets the 'nowrap' to true.
In its headers the Chrome browser client claims to support gzip. If I get that working I'll come back here and post the answer to that too.

Reading from a socket using buffered reader blocks [duplicate]

This question already has answers here:
Buffered Reader HTTP POST
(4 answers)
Closed 7 years ago.
I am trying to create a http server in java.
The following is a fragment of my code.
ServerSocket s = new ServerSocket(80);
while(true){
Socket client = s.accept();
BufferedReader in = new BufferedReader(
new InputStreamReader(client.getInputStream()));
String input = in.readLine(), ff;
System.out.println(input);
while((ff = in.readLine()) != null){
System.out.println(ff);
}
System.out.println("asd");
in.close();
client.close();
}
Asd is never printed. When I try to do a POST on the local host, it sends me all the headers but doesn't send any content data. It just waits there doing nothing.
I am doing a post using Jquery
$.post("http://127.0.0.1/",
{
name: "Donald Duck",
city: "Duckburg"
},
function(data,status){
alert("Data: " + data + "\nStatus: " + status);
});
This is what is being printed on the screen:
POST / HTTP/1.1
Host: 127.0.0.1
Connection: keep-alive
Content-Length: 5
Accept: */*
Origin: http://www.w3schools.com
User-Agent: Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/47.0.2526.73 Chrome/47.0.2526.73 Safari/537.36
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
That's it, nothing after this. It waits indefinitely at this point.

My guess would be that your actual content/payload is not terminated by the expected line delimiters as described in BufferedReader#readLine().
This blocks your in.readLine() invocation.
I would try to use the read() method instead and see if that helps. As in
while((value = in.read()) != -1) {
char c = (char)value;
System.out.println(c);
}
Edit: In fact, your question most likely duplicates this question: Buffered Reader HTTP POST

Receiving Data On Sinatra Server

I am able to send data to a sinatra server (Strings) but I'm not sure about how to actually receive it in the sinatra code. Could anyone help?
Java client (what i'm using to send data) code:
private static void contactServer() {
try {
String text = "This is a text please work";
Socket sock = new Socket("localhost", 4567);
OutputStream os = sock.getOutputStream();
URL url = new URL("http://localhost:4567/hello");
PrintWriter writer = new PrintWriter(os);
writer.flush();
writer.write(text);
url.openStream();
System.out.println("done");
String strTemp = "";
/*while(null != (strTemp = br.readLine())){
System.out.println(strTemp);
}*/
} catch (Exception e) {
e.printStackTrace();
}
}

That's not going to work. Sinatra's server understands and speaks a language called the http protocol, so sending the string:
"This is a text please work"
through a socket to a Sinatra app is a hopeless prayer.
A protocol is a set of rules that specify how the client and the server will speak to each other--then each party can understand exactly what other party is saying. For the http protocol, clients send something known as a request, and servers reply with something known as a response. The request and the response must be formatted precisely according to the rules specified by the http protocol. The gory details for a request are here:
http://www.w3.org/Protocols/rfc2616/rfc2616-sec5.html
So your java program needs to send a request to Sinatra's server, which is simply a string formatted in a precise manner. Here is what a simple GET request looks like:
GET /path/to/page HTTP/1.1
Host: localhost:4567
If you want to make a GET request for the page:
http://localhost:4567/page1
(i.e. hit the Sinatra route get '/page1')
...then a simple GET request for that page would look like:
GET /page1 HTTP/1.1
Host: localhost:4567
Also, you must end every line in the http request with "\r\n" no matter what OS you are using. Those two characters are part of the http protocol. Furthermore, after the last header there must be a blank line signified by another "\r\n", like this:
GET /page1 HTTP/1.1\r\nHost: localhost:4567\r\n\r\n
Here is the java:
import java.io.*;
import java.net.*;
public class Sinatra {
private static void contactServer() {
try {
Socket sock = new Socket("localhost", 4567);
OutputStream os = sock.getOutputStream();
PrintWriter writer = new PrintWriter(os);
String[] text = {
"GET /page1 HTTP/1.1",
"Host: localhost:4567",
};
String request = "";
for(int i=0; i < text.length; ++i) {
request += text[i] + "\r\n";
}
request += "\r\n";
System.out.println(request);
writer.write(request);
writer.flush();
} catch (Exception e) {
e.printStackTrace();
}
}
public static void main(String[] args) {
Sinatra.contactServer();
}
}
Note: the first few times I ran that java program, the server(which I started with $ ruby myapp.rb) threw the following error:
[2013-08-19 20:10:11] ERROR Errno::ECONNRESET: Connection reset by peer
/Users/7stud/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/webrick/httpserver.rb:80:in `eof?'
/Users/7stud/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/webrick/httpserver.rb:80:in `run'
/Users/7stud/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/webrick/server.rb:191:in `block in start_thread'
But when I ran the java program a few more times, the server eventually behaved. I have no idea what is causing that error. When I entered the url in FireFox, the server never threw that error. So I used Firebug to look at the request that Firefox sends; then I used all the same request headers in the java program, but the server still threw that error.
Edit: I can get the server error to go away by making the java program sleep for 1 second before closing the socket. The socket closes when you explicitly close the socket or when the program ends. Without the sleep, I think the socket closes while the server is still processing the request. Because a browser keeps the socket open, a browser never causes the server to throw that error.
The same server error also occurs with a ruby client:
require 'socket'
port = 4567
host = 'localhost'
s = TCPSocket.new host, port
req = [
"GET /page1 HTTP/1.1",
"Host: localhost:4567",
"Accept: */*",
]
req = req.join("\r\n") << ("\r\n" * 2)
print req
s.write req
s.flush
#sleep(1)
s.close
And the fix is the same. The only niggling detail is why the curl unix command doesn't cause the server to throw that error:
$ curl -v http://localhost:4567/page1
* About to connect() to localhost port 4567 (#0)
* Trying 127.0.0.1... connected
* Connected to localhost (127.0.0.1) port 4567 (#0)
> GET /page1 HTTP/1.1
> User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8r zlib/1.2.3
> Host: localhost:4567
> Accept: */*
>
< HTTP/1.1 200 OK
< Content-Type: text/html;charset=utf-8
< Content-Length: 0
< X-Xss-Protection: 1; mode=block
< X-Content-Type-Options: nosniff
< X-Frame-Options: SAMEORIGIN
< Server: WEBrick/1.3.1 (Ruby/1.9.3/2012-04-20)
< Date: Tue, 20 Aug 2013 04:59:16 GMT
< Connection: Keep-Alive
<
* Connection #0 to host localhost left intact
* Closing connection #0
With the -v option, curl prints out the request and the response. Using curl to make the request, I never saw the sever throw that error. I wonder if curl does a sleep too?

Java HTTP Server response incomplete

I am trying to build my own embedded HTTP Server for Java with very specific usage for an internal closed system. Using embedded solution that already exists meant that I need to strip them down to refit them for my internal system's specific use cases.
I have managed to get my Java HTTPD receive HTTP Requests from web browsers but the Requests that the HTTPD receives are incomplete.
Below is the server thread codes (pretending the ServerSocket works perfectly fine):
public class HttpThread implements Runnable {
private Socket socket;
private DataInputStream in;
private PrintStream out;
private BufferedReader br;
private String EOF = "\n";
private String STATUS_200 = "HTTP/1.1 200 OK" + EOF;
public HttpThread(Socket socket) throws IOException {
this.socket = socket;
this.in = new DataInputStream(this.socket.getInputStream());
this.out = new PrintStream(new BufferedOutputStream(this.socket.getOutputStream()));
}
#Override
public void run() {
System.out.println("New thread...");
try {
processInput();
//socket.close();
} catch (IOException ex) {
Logger.getLogger(HttpThread.class.getName()).log(Level.SEVERE, null, ex);
} finally {
if (br != null) {
try {
br.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
//System.out.println("Thread END...");
}
private String processInput() throws IOException {
String line;
StringBuilder buff = new StringBuilder();
br = new BufferedReader(new InputStreamReader(in));
while ((line = br.readLine()) != null) {
buff.append(line);
System.out.println(">>> " + line);
if (line.trim().isEmpty()) {
break;
}
}
out.print(STATUS_200);
out.print("Server: Test Server\n" +
"Content-Type: text/html; charset=UTF-8\n" +
"Connection: close");
out.print(EOF);
out.print("<html><body>yello</body></html>");
out.print(EOF);
out.flush();
System.out.println(STATUS_200);
return buff.toString();
}
}
I am using this HTML Script to test the server thread to simulate a POST request:
<html>
<body onLoad="document.test.submit();">
<form action="http://localhost:9999/" method="POST" name="test">
<input type=hidden name="userid" value="1443"/>
<input type=hidden name="password" value="1443"/>
</form>
</body>
</html>
When I use the browser to call the HTML codes the Java HTTPD receives an incomplete response:
New thread...
>>> POST / HTTP/1.1
>>> Host: localhost:9999
>>> Connection: keep-alive
>>> Content-Length: 25
>>> Cache-Control: max-age=0
>>> Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
>>> Origin: null
>>> User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.72 Safari/537.36
>>> Content-Type: application/x-www-form-urlencoded
>>> Accept-Encoding: gzip,deflate,sdch
>>> Accept-Language: en-GB,en-US;q=0.8,en;q=0.6
>>>
HTTP/1.1 200 OK
New thread...
>>> GET /favicon.ico HTTP/1.1
>>> Host: localhost:9999
>>> Connection: keep-alive
>>> Accept: */*
>>> User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.72 Safari/537.36
>>> Accept-Encoding: gzip,deflate,sdch
>>> Accept-Language: en-GB,en-US;q=0.8,en;q=0.6
>>>
HTTP/1.1 200 OK
It seems that the HTTPD only received the HTTP headers and stopped receiving the POST body.
May I know of some solutions to solve the above problem ?
Thanks.

The HTTP headers and body are separated by an empty line (also see the HTTP RFC, especially the chapter "5 Request"). Your server reads the socket Inputstream but breaks on an empty line :
if (line.trim().isEmpty()) {
break;
}
Therefore obviously you will not receive the body. You should fully consume the Inputstream instead.
Besides that, I would advise you to use already existing solutions. There is an abundance of HTTP server implementations, that are well tested and proven in real world usage. Save yourself alot of headache and use an existing lightweight server like e.g. Jetty, Netty or similar ones.

I switched out the while loop that does the readline() with the following code:
int i;
byte[] buffer = new byte[2048];
i = in.read(buffer);
for (int j = 0; j < i; j++) {
buff.append((char) buffer[j]);
}
System.out.println(buff.toString());
and the problem is solved.
Pyranja, thanks for the help.
Thanks to http://kcd.sytes.net/articles/simple_web_server.php IF you follow concisely. Somehow the br.readline() is not fully reading the lines properly after the empty line.
The code fragment should now look like:
private String processInput() throws IOException {
String line;
StringBuilder buff = new StringBuilder();
int i;
byte[] buffer = new byte[2048];
i = in.read(buffer);
for (int j = 0; j < i; j++) {
buff.append((char) buffer[j]);
}
System.out.println(buff.toString());
out.print(STATUS_200);
out.print("Server: Test Server\n" +
"Content-Type: text/html; charset=UTF-8\n" +
"Connection: close");
out.print(EOF);
out.print("<html><body>yello</body></html>");
out.print(EOF);
out.flush();
System.out.println(STATUS_200);
return buff.toString();
}
I guess it's a good experience learning how to build a simple HTTP Server :D .

How do i forward the HTTP request back to the browser? Proxy Java

final int portNumber = 8128;
String str;
int start = 0;
int endSg = 0;
int endCom = 0;
String ReqWeb=null;
System.out.println("Creating server socket on port " + portNumber);
ServerSocket serverSocket = new ServerSocket(portNumber);
BufferedReader inFromServer;
OutputStream out;
PrintWriter outw;
Socket forwardSocket = null;
while (true)
{
Socket socket = serverSocket.accept(); //get client request
String from = socket.getInetAddress().toString();
System.out.println("Accepted connection from " + from);
OutputStream os = socket.getOutputStream();
PrintWriter pw = new PrintWriter(os, true);
pw.println("What's your request?");
BufferedReader br = new BufferedReader(new InputStreamReader(socket.getInputStream()));
//System.out.println(str);
while((str = br.readLine())!=null){
System.out.println(str);
if(str!=null){
start = str.indexOf("Host: ") + 6;
endSg = str.indexOf(".sg", start) + 3;
endCom = str.indexOf(".com", start) + 4;
if(((endSg>3)||(endCom>4))&&(start>4)){
if(endSg>3)
ReqWeb = str.substring(start, endSg);
else if(endCom>3)
ReqWeb = str.substring(start, endCom);
}
}
}
System.out.println(ReqWeb);
if(ReqWeb!=null){
//ReqWeb = str.substring(start);
System.out.println(ReqWeb);
forwardSocket = new Socket(ReqWeb, 80);
}
pw.println(str);
pw.println(ReqWeb);
//socket.close();
if(forwardSocket!=null){
inFromServer = new BufferedReader(new InputStreamReader(forwardSocket.getInputStream()));
out = forwardSocket.getOutputStream();
outw = new PrintWriter(out, false);
outw.print(str);
}
}
Output :
Creating server socket on port 8128
Accepted connection from /127.0.0.1
null
Accepted connection from /127.0.0.1
GET (http://)stackoverflow.com/questions/12900825/how-do-i-forward-the-http-request-back-to-the-browser-proxy-java HTTP/1.0
Host: stackoverflow.com
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:15.0) Gecko/20100101 Firefox/15.0.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: close
Proxy-Connection: close
Pragma: no-cache
now this is the output i got, should i store in a byte array to send it back to the server to request for the page? I am still having trouble forwarding this request ):

You'll need to open another socket to the intended target (as per firefox's request) and send the request there. Keep the socket that's connected to firefox open because when you get the response from the intended target, you'll read it from the target and write it back to firefox. Depending on your pipelining settings in firefox, the connection either may then close or may make more requests.