websocket protocol error when server deflates payload - java

I'm writing a Java websocket server and it now receives the opening handshake message from the client (in Chrome Version 57.0.2987.133 (64-bit)) and responds to complete the handshake, both shown below.
Received: GET / HTTP/1.1
Received: Host: localhost:6789
Received: Connection: Upgrade
Received: Pragma: no-cache
Received: Cache-Control: no-cache
Received: Upgrade: websocket
Received: Origin: http://localhost:8080
Received: Sec-WebSocket-Version: 13
Received: User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36
Received: Accept-Encoding: gzip, deflate, sdch, br
Received: Accept-Language: en-US,en;q=0.8
Received: Sec-WebSocket-Key: L1IiUSGijbGmTpthWsebOg==
Received: Sec-WebSocket-Extensions: permessage-deflate; client_max_window_bits
Sending: HTTP/1.1 101 Switching Protocols
Sending: Upgrade: websocket
Sending: Connection: Upgrade
Sending: Sec-WebSocket-Accept: L5HXnDJGDMYbWr5gRcQMOwKNf3Q=
Sending: Accept-Encoding: gzip, deflate
Sending: Sec-WebSocket-Extensions: permessage-deflate; client_no_context_takeover; server_no_context_takeover
Now, the client can send messages and it does so without a problem and my code successfully uncompresses them using java.util.zip.Deflater and... if my server responds with header bytes 0x81 (fin, no compression, text) and 0x5 then the bytes for 'hello' (as an example) then the websocket client in javascript on Chrome is entirely happy but when I try to compress the response, the client always closes the connection citing error code 1002 and the text 'Websocket protocol error'.
DEFLATE
public void sendMessageDeflated(String rxMessage, OutputStream streamOut) {
System.out.println("Message back to client is: " + rxMessage);
// And then compress the response and send it out.
Deflater compressor = new Deflater(Deflater.DEFLATED);
try {
int headerLength = 2;
byte unzippedMsg[] = rxMessage.getBytes("UTF-8");
compressor.setInput(unzippedMsg);
compressor.finish();
byte zippedMsg[] = new byte[2048]; // Nasty constant but will have to do for now.
int toCompressLength = unzippedMsg.length;
int compLength = compressor.deflate(zippedMsg, headerLength, zippedMsg.length - headerLength);
compressor.end();
zippedMsg[0] = (byte)0xC1; // FIN bit, compression plus opcode for TEXT MESSAGE
zippedMsg[1] = (byte)((byte)0x00 | (byte)compLength); // No mask on return data.
streamOut.write(zippedMsg, 0, compLength + headerLength);
} catch ( IOException ioEx ) {
// TBD
System.out.println("IOException: " + ioEx.toString());
} catch ( Exception ex ) {
// TBD
System.out.println("IOException: " + ex.toString());
}
}
GZIP
public void sendMessageGZipped(String rxMessage, OutputStream streamOut) {
// Do something with the message here...
System.out.println("Message back to client is: " + rxMessage);
// And then compress the response and send it out.
try {
int headerLength = 2;
byte unzippedMsg[] = rxMessage.getBytes("UTF-8");
int toCompressLength = unzippedMsg.length;
ByteArrayOutputStream baos = new ByteArrayOutputStream();
GZIPOutputStream gzipOut = new GZIPOutputStream(baos);
gzipOut.write(unzippedMsg, 0, toCompressLength);
gzipOut.close();
byte[] payload = baos.toByteArray();
byte header[] = new byte[32];
header[0] = (byte)0xC1; // FIN bit plus opcode for TEXT MESSAGE
header[1] = (byte)((byte)0x00 | (byte)payload.length); // No mask on return data.
streamOut.write(header, 0, 2);
streamOut.write(payload);
} catch ( IOException ioEx ) {
// TBD
System.out.println("IOException: " + ioEx.toString());
} catch ( Exception ex ) {
// TBD
System.out.println("IOException: " + ex.toString());
}
}
I've tried switching to the opcode for binary thinking that perhaps compressed text = binary but that didn't work. I really can't see what I've done wrong or missed out. This implementation does not include a sliding compression window that spans messages. I think the response headers make that clear. Help gratefully accepted.

I solved this eventually by building the client-side websocket code and seeing the exception thrown when it tried to Inflate the message: "invalid stored block lengths". That led me to this post: Java decompressing array of bytes which talks about Deflate being able to compress with zlib wrapper or without. So change one line in processMessage2 from the example code above to this...
Deflater compressor = new Deflater(Deflater.DEFLATED, true);
...which sets the 'nowrap' to true.
In its headers the Chrome browser client claims to support gzip. If I get that working I'll come back here and post the answer to that too.

Related

How to send and receive data in a persistent connection with Dart?

I am trying to have my client(s) connect their Dart (Flutter) running devices to my server running a Java ServerSocket. The connection establishes properly and all headers are received by the server, however no other data is received.
The configuration issue must be with the dart code below, since my other Java client to Java server with the same code works.
Dart client code:
// if the WebSocketChannel is not closed, close it
try {
channel?.sink.close();
} catch (e) {
// do nothing
}
channel = WebSocketChannel.connect(Uri.parse(globals.serverUrl));
channel?.sink.add("Test notification");
Java server code:
outStream = new PrintWriter(socket.getOutputStream(), true);
inStream = new BufferedReader(new InputStreamReader(socket.getInputStream()));
while (true) {
String input = inStream.readLine();
if (input == null) {
socket.close();
inStream.close();
outStream.close();
System.out.println("Client disconnected");
break;
}
System.out.println("Received: " + input);
}
Server output:
Received: GET / HTTP/1.1
Received: user-agent: Dart/2.17 (dart:io)
Received: connection: Upgrade
Received: cache-control: no-cache
Received: accept-encoding: gzip
Received: sec-websocket-version: 13
Received: host: localhost:8080
Received: sec-websocket-extensions: permessage-deflate; client_max_window_bits
Received: sec-websocket-key: lys/+BV+IIH5Mm52V0jUmw==
Received: upgrade: websocket
Received:
Note that the client does not disconnect, so the connection is indeed persistent.
The weird behaviour is with sending and receiving custom data:
Client to Server:
results in no data being sent or error showing
Server to Client:
[ERROR:flutter/lib/ui/ui_dart_state.cc(198)] Unhandled Exception: WebSocketChannelException: WebSocketChannelException: HttpException: Invalid response line, uri = http://localhost:8080/
The message transmitted from Server to Client here was just "test" and the client was listening with:
String? response = await channel?.stream.first;
The response would remain unused and was listened for after the completion of the sent message as seen in the Dart code sample.

Why InputStreamReader cannot read the content of a websocket package?

To understand the behaviour of the websocket, I created a simple SocketServer in java to exchange the messages. The server is expected to follow the operations as:
1) Listening at port 8080
2) A websocket handshake message generated manually on the browser client and received by the server.
3) Construct a response to the handshake message and reply to the client
4) Read out actually websocket info bytes with the same connection.
The problem has happened at step 4. when the server has responded with the handshake message, the InputStreamReader can no longer receive any new message. It blocked at the readline() method even though the client has sent the message already. From wireshark, I can see the client sent message and server respond ack. Any help would be appreciated, thanks.
Update: I just noted this question has been asked before. I will study the suggestions on the other posts first.
Update: The behavior is as same as this post:
Weird websocket behavior: only send data when close the tab
When the webtab closes, the read stream received the data package.
wireshark screen captures:
The tcp stream trace
The packages sequence
The log:
inputline: GET / HTTP/1.1
inputline: Host: localhost:8080
inputline: Connection: Upgrade
inputline: Pragma: no-cache
inputline: Cache-Control: no-cache
inputline: Upgrade: websocket
inputline: Origin: file://
inputline: Sec-WebSocket-Version: 13
inputline: User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_5)AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36
inputline: Accept-Encoding: gzip, deflate, br
inputline: Accept-Language: en,zh-TW;q=0.8,zh;q=0.6,zh-CN;q=0.4
inputline: Sec-WebSocket-Key: Yin4xn04vr9iBH1b2dU15A==
inputline: Sec-WebSocket-Extensions: permessage-deflate; client_max_window_bits
inputline:
response: HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: +Y9whLTzCdyN1INpAxjkO6yD2Nw=
The server socket code:
public class EchoServer {
public static String HTTP_VERSION_HEADER = "HTTP/1.1";
public static void main(String[] args) throws IOException {
int portNumber = 8080;
try (
ServerSocket serverSocket =
new ServerSocket(Integer.parseInt("8080"));
Socket clientSocket = serverSocket.accept();
PrintWriter out =
new PrintWriter(clientSocket.getOutputStream(), true);
BufferedReader in = new BufferedReader(
new InputStreamReader(clientSocket.getInputStream()));
) {
String inputLine;
StringBuilder sb = new StringBuilder();
while ( true) {
inputLine = in.readLine();
if(inputLine == null) {
System.out.println("input is null");
continue;
}
System.out.println("inputline: " + inputLine);
sb.append(inputLine).append(System.lineSeparator());
if(inputLine.equals("")) {
Message msg = new Message(sb.toString(), new Date());
HttpMessage tmpMessage = new HttpMessage(msg.getText(), new Date());
String response = generateHandshakeResponse(tmpMessage);
System.out.println("response: " + response);
out.println(response);
out.flush();
}
}
} catch (IOException e) {
System.out.println("Exception caught when trying to listen on port "
+ portNumber + " or listening for a connection");
System.out.println(e.getMessage());
}
}
private static String generateHandshakeResponse(HttpMessage message) {
String webSocketKey = message.getVal(HttpMessage.SEC_WEBSOCKET_KEY);
String webSocketAccept = webSocketKey + "258EAFA5-E914-47DA-95CA-C5AB0DC85B11";
byte[] bytes = DigestUtils.sha1(webSocketAccept);
String secWebSocketAcceptVal = Base64.encodeBase64String(bytes);
StringBuilder sb = new StringBuilder();
sb.append(HTTP_VERSION_HEADER).append(" 101 ").append("Switching Protocols\r\n");
sb.append("Upgrade: websocket\r\n");
sb.append("Connection: Upgrade\r\n");
sb.append("Sec-WebSocket-Accept: ").append(secWebSocketAcceptVal).append("\r\n");
sb.append("\n\n") //<--- this line fixed the problem
//log.debug(sb.toString());
return sb.toString();
}
}
The client code:
<!doctype html>
<html lang="en">
<head>
<title>Websocket Client</title>
</head>
<body>
<script>
var exampleSocket = new WebSocket("ws://localhost:8080");
exampleSocket.onopen = function (event) {
console.log("connection opened..");
exampleSocket.send("Can you hear me?");
};
exampleSocket.onmessage = function (event) {
console.log(event.data);
}
function sendMsg(){
console.log("send message..");
exampleSocket.send("hello hello");
}
</script>
<button onclick="sendMsg()" title="send">send</button>
</body>
</html>
Thanks a lot to EJP, the problem is a missing blank line at the response handshake to indicate the end of the message.

Reading from a socket using buffered reader blocks [duplicate]

This question already has answers here:
Buffered Reader HTTP POST
(4 answers)
Closed 7 years ago.
I am trying to create a http server in java.
The following is a fragment of my code.
ServerSocket s = new ServerSocket(80);
while(true){
Socket client = s.accept();
BufferedReader in = new BufferedReader(
new InputStreamReader(client.getInputStream()));
String input = in.readLine(), ff;
System.out.println(input);
while((ff = in.readLine()) != null){
System.out.println(ff);
}
System.out.println("asd");
in.close();
client.close();
}
Asd is never printed. When I try to do a POST on the local host, it sends me all the headers but doesn't send any content data. It just waits there doing nothing.
I am doing a post using Jquery
$.post("http://127.0.0.1/",
{
name: "Donald Duck",
city: "Duckburg"
},
function(data,status){
alert("Data: " + data + "\nStatus: " + status);
});
This is what is being printed on the screen:
POST / HTTP/1.1
Host: 127.0.0.1
Connection: keep-alive
Content-Length: 5
Accept: */*
Origin: http://www.w3schools.com
User-Agent: Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/47.0.2526.73 Chrome/47.0.2526.73 Safari/537.36
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
That's it, nothing after this. It waits indefinitely at this point.
My guess would be that your actual content/payload is not terminated by the expected line delimiters as described in BufferedReader#readLine().
This blocks your in.readLine() invocation.
I would try to use the read() method instead and see if that helps. As in
while((value = in.read()) != -1) {
char c = (char)value;
System.out.println(c);
}
Edit: In fact, your question most likely duplicates this question: Buffered Reader HTTP POST

Receiving Data On Sinatra Server

I am able to send data to a sinatra server (Strings) but I'm not sure about how to actually receive it in the sinatra code. Could anyone help?
Java client (what i'm using to send data) code:
private static void contactServer() {
try {
String text = "This is a text please work";
Socket sock = new Socket("localhost", 4567);
OutputStream os = sock.getOutputStream();
URL url = new URL("http://localhost:4567/hello");
PrintWriter writer = new PrintWriter(os);
writer.flush();
writer.write(text);
url.openStream();
System.out.println("done");
String strTemp = "";
/*while(null != (strTemp = br.readLine())){
System.out.println(strTemp);
}*/
} catch (Exception e) {
e.printStackTrace();
}
}
That's not going to work. Sinatra's server understands and speaks a language called the http protocol, so sending the string:
"This is a text please work"
through a socket to a Sinatra app is a hopeless prayer.
A protocol is a set of rules that specify how the client and the server will speak to each other--then each party can understand exactly what other party is saying. For the http protocol, clients send something known as a request, and servers reply with something known as a response. The request and the response must be formatted precisely according to the rules specified by the http protocol. The gory details for a request are here:
http://www.w3.org/Protocols/rfc2616/rfc2616-sec5.html
So your java program needs to send a request to Sinatra's server, which is simply a string formatted in a precise manner. Here is what a simple GET request looks like:
GET /path/to/page HTTP/1.1
Host: localhost:4567
If you want to make a GET request for the page:
http://localhost:4567/page1
(i.e. hit the Sinatra route get '/page1')
...then a simple GET request for that page would look like:
GET /page1 HTTP/1.1
Host: localhost:4567
Also, you must end every line in the http request with "\r\n" no matter what OS you are using. Those two characters are part of the http protocol. Furthermore, after the last header there must be a blank line signified by another "\r\n", like this:
GET /page1 HTTP/1.1\r\nHost: localhost:4567\r\n\r\n
Here is the java:
import java.io.*;
import java.net.*;
public class Sinatra {
private static void contactServer() {
try {
Socket sock = new Socket("localhost", 4567);
OutputStream os = sock.getOutputStream();
PrintWriter writer = new PrintWriter(os);
String[] text = {
"GET /page1 HTTP/1.1",
"Host: localhost:4567",
};
String request = "";
for(int i=0; i < text.length; ++i) {
request += text[i] + "\r\n";
}
request += "\r\n";
System.out.println(request);
writer.write(request);
writer.flush();
} catch (Exception e) {
e.printStackTrace();
}
}
public static void main(String[] args) {
Sinatra.contactServer();
}
}
Note: the first few times I ran that java program, the server(which I started with $ ruby myapp.rb) threw the following error:
[2013-08-19 20:10:11] ERROR Errno::ECONNRESET: Connection reset by peer
/Users/7stud/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/webrick/httpserver.rb:80:in `eof?'
/Users/7stud/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/webrick/httpserver.rb:80:in `run'
/Users/7stud/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/webrick/server.rb:191:in `block in start_thread'
But when I ran the java program a few more times, the server eventually behaved. I have no idea what is causing that error. When I entered the url in FireFox, the server never threw that error. So I used Firebug to look at the request that Firefox sends; then I used all the same request headers in the java program, but the server still threw that error.
Edit: I can get the server error to go away by making the java program sleep for 1 second before closing the socket. The socket closes when you explicitly close the socket or when the program ends. Without the sleep, I think the socket closes while the server is still processing the request. Because a browser keeps the socket open, a browser never causes the server to throw that error.
The same server error also occurs with a ruby client:
require 'socket'
port = 4567
host = 'localhost'
s = TCPSocket.new host, port
req = [
"GET /page1 HTTP/1.1",
"Host: localhost:4567",
"Accept: */*",
]
req = req.join("\r\n") << ("\r\n" * 2)
print req
s.write req
s.flush
#sleep(1)
s.close
And the fix is the same. The only niggling detail is why the curl unix command doesn't cause the server to throw that error:
$ curl -v http://localhost:4567/page1
* About to connect() to localhost port 4567 (#0)
* Trying 127.0.0.1... connected
* Connected to localhost (127.0.0.1) port 4567 (#0)
> GET /page1 HTTP/1.1
> User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8r zlib/1.2.3
> Host: localhost:4567
> Accept: */*
>
< HTTP/1.1 200 OK
< Content-Type: text/html;charset=utf-8
< Content-Length: 0
< X-Xss-Protection: 1; mode=block
< X-Content-Type-Options: nosniff
< X-Frame-Options: SAMEORIGIN
< Server: WEBrick/1.3.1 (Ruby/1.9.3/2012-04-20)
< Date: Tue, 20 Aug 2013 04:59:16 GMT
< Connection: Keep-Alive
<
* Connection #0 to host localhost left intact
* Closing connection #0
With the -v option, curl prints out the request and the response. Using curl to make the request, I never saw the sever throw that error. I wonder if curl does a sleep too?

XMLHttpRequest java javascript

I try to communicate between javascript and java. My script javascript send a message to java and java send a response.
javascript part:
var xmlhttp = new XMLHttpRequest();
xmlhttp.onreadystatechange=function()
{
if (xmlhttp.readyState==4)
{
alert(xmlhttp.responseText);
}
}
var s = "LIGNE \n 2 \n il fait beau \nEND\n";
xmlhttp.open("POST","http://localhost:6020",true);
xmlhttp.send(s);
java part:
try {
serverSocket = new ServerSocket(6020);
} catch (IOException e) {
System.err.println("Could not listen on port: 6020.");
System.exit(-1);
}
serverSocket.accept()
BufferedReader br = new BufferedReader(
new InputStreamReader(socket.getInputStream()));
BufferedWriter bw = new BufferedWriter(
new OutputStreamWriter(socket.getOutputStream()));
String ligne = "";
while(!(ligne = plec.readLine()).equals("END")){
System.out.println(ligne);
}
bw.write("Il fait beau\n");
bw.flush();
bw.close();
plec.close();
socket.close();
output java :
POST / HTTP/1.1
Host: localhost:6020
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:13.0) Gecko/20100101 Firefox/13.0.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: fr,fr-fr;q=0.8,en-us;q=0.5,en;q=0.3
Accept-Encoding: gzip, deflate
Connection: keep-alive
Referer: http://localhost:8080/test.html
Content-Length: 30
Content-Type: text/plain; charset=UTF-8
Origin: http://localhost:8080
Pragma: no-cache
Cache-Control: no-cache
LIGNE
2
il fait beau
So, I receive correctly the message send by javascript but the alert his always empty. How to response at this message?
I try a lot of possiblity but they don't work. And I don't want to use the servlet, it's to heavy to do that.
Thanks.
Edit:
I did this :
bw.write("HTTP/1.1 200 OK\r\n"+
"Content-Type: text/html; charset=utf-8\r\n"+
"Content-Length: 13\r\n\r\n" +
"il fait beau\n");
and this:
String data = "il fait beau \n";
StringBuilder builder = new StringBuilder();
builder.append("HTTP/1.1 200 OK\r\n");
builder.append("Content-Type: text/html; charset=utf-8\r\n");
builder.append("Content-Length:" + data.length() + "\r\n\r\n");
builder.append(data);
bw.write(builder.toString());
But the alert remain empty. Maybe it's a problem in the javascript.
The javascript needs to see a full HTTP response. Merely sending back characters to it, makes it discard the reply as it is an invalid HTTP response.
In your java code, send back something like this
HTTP/1.1 200 OK
Content-Type: text/html; charset=utf-8
Content-Length: <length of data>
---data here---
Reference
Something like:
StringBuilder builder = new StringBuilder();
builder.append("HTTP/1.1 200 OK\r\n");
builder.append("Content-Type: text/plain; charset=utf-8\r\n");
builder.append("Content-Length:" + data.length() + "\r\n\r\n);
builder.append(data);
bw.write(builder.toString());
Try:
bw.write("HTTP/1.1 200 OK\r\n"+
"Content-Type: text/html; charset=utf-8\r\n"+
"Content-Length: 13\r\n\r\n" +
"il fait beau\n");
HTTP-Headers are separated by \r\n (CRLF). Headers and body is spearated by \r\n\r\n.
Note that you set the length to 13 because you also have to count the \n at the end of your string.
EDIT: It does not work because of the cross-domain-policy. http://localhost:6020 is not the same port as the website which executes your JavaScript and so the xmlhttprequest might not be delivered.

Categories

Resources