While trying to port some socket code from Java to Node.js (or rather, node-webkit) I came to the part where a handshake is performed between the client and the server. The Java code for this is simple (validation left out):
// C0
byte C0 = 0x03;
out.write(C0);
// C1
long timestampC1 = System.currentTimeMillis();
byte[] randC1 = new byte[1528];
rand.nextBytes(randC1);
out.writeInt((int)timestampC1);
out.writeInt(0);
out.write(randC1, 0, 1528);
out.flush();
// S0
byte S0 = (byte)in.read();
if (S0 != 0x03)
throw new IOException("Server returned incorrect version in handshake: " + S0);
// S1
byte[] S1 = new byte[1536];
in.read(S1, 0, 1536);
I was wondering if there is any way in Node.js to do something similar to Java's Socket.read(). I do not see any in the documentation, but it seems logical because Node.js is more asynchronous.
I have succeeded in writing the sending of the handshake (and confirmed that it works) using this code (methods for sending left out):
//C0
sendByte(sock, 0x03);
//C1
var ts = Date.now();
sock.randomBytes = crypto.randomBytes(1528);
sendInt(sock, ts);
sendInt(sock, 0x0);
sendBytes(sock, sock.randomBytes);
My first thought was to keep track of the connection state and to continue the handshake when a data event is received and the connection has the relevant state. However, the buffer that comes with the data event is limited to 1024 bytes, while the Java code is easily able to read 1528 bytes. The rest of the bytes seem to come in the next data call.
Should I be storing the contents from data in a Buffer and appending whenever the data is received or is there another (easier) way of doing this (a premade module, a method similar to read, ...)?
There is no synchronous read() in that same way, but what can you do in node is something like:
// using a helper function can help normalize things
function readSocket(socket, nb, cb) {
var r = socket.read(nb);
if (r === null) {
socket.once('readable', function() {
readSocket(socket, nb, cb);
});
return;
}
cb(r);
}
// read in 1 byte
readSocket(socket, 1, function(buf) {
// `buf` is a Buffer with `buf.length === 1`
});
Related
I have been working on a networking project recently to revive a dead mmo game for personal learning, I have a python implementation which works to decode the game data using blowfish(pypi / pycryptodome) and would like to transfer this 'server' into a java project.
Initially using blowfish decryption in java (BouncyCastle and Cipher -- default) I was getting entirely different results between java and python. Through some research I found that java (along with most things) actually use blowfish-compat big endian.
This one python library seems to be the only one which correctly decodes the data. Next I decided to use a python asyncio server as a middle relay simply for encryption and decryption. The network flow now looks like this:
GameClient -> Java SocketServer -> Python server (decryption) -> Java SocketServer.
The original Python implementation results in these bytes in hex format:
32004a815f49367cc3691be26d7b668132506dc972d5a6bbad38299640c6e222c6e55096f50ff33711250675431633ca9ede
The Java implementation produces these results in hex format(using apache commons Hex.encodeHexString())
32004a815f49367cc3691be26d7b668132506dc972d5a6bbad38299640c6e222c6e5c65830d65f9b4d60eb26730685f486d7
Both of these hex representations are pre-blowfish decryption in Python they are just the raw bytes being sent from the game client.
My question is why do these bytes start off the same and then it seems java trails off? The python results are the correct results they are tested and work. I have tried wrapping the bytes in java in a buffer and then calling flip() however this did not produce the correct results either. Using another stackoverflow post (I do not have link I am sorry) I tried casting this byte[] into a BigInteger and that also did not produce the correct results.
Any help is greatly appreciated.
Python Implementation
#!/usr/bin/env python3
import asyncio
import binascii
import blowfish
import ipaddress
import os
import struct
import sys
AUTH_BLOWFISHKEY = b"[;'.]94-31==-%&#!^+]\000"
bf = blowfish.Cipher(AUTH_BLOWFISHKEY, byte_order="little")
class EncryptionRelay(asyncio.Protocol):
def connection_made(self, transport):
self.transport = transport
self.client = (transport.get_extra_info('peername')[0] + ":" # IP
+ str(transport.get_extra_info('peername')[1])) # port
print("Connection from: " + self.client)
def connection_lost(self, exc):
print("Connection closed: " + self.client)
def data_received(self, data):
print(data.hex()) #python output above
pt = b''.join(bf.decrypt_ecb(data[2:]))
self.transport.write(pt)
def closeSocket(self, reason):
print(reason)
self.transport.close()
def main():
loop = asyncio.get_event_loop()
coroutine = loop.create_server(EncryptionRelay, host=None, port=54556)
server = loop.run_until_complete(coroutine)
for socket in server.sockets:
print("Listening on: " + socket.getsockname()[0] + ":" +
str(socket.getsockname()[1]))
try:
loop.run_forever()
except KeyboardInterrupt:
pass
server.close()
loop.run_until_complete(server.wait_closed())
loop.close()
if __name__ == "__main__":
main()
Java Implementation
public AuthServer(int port) {
serverGUI = new AuthServerGUI(port);
try {
serverSocket = new ServerSocket(port);
relay = new PythonEncryptionRelay(this);
new Thread(relay).start();
} catch (IOException e) {
e.printStackTrace();
}
}
#Override
public void run() {
while(true) {
try {
Socket socket = serverSocket.accept();
onConnection(socket); //sends an init packet to client -- irrelevant to question
byte[] incomingData = new byte[0];
byte[] temp = new byte[1024];
int k = -1;
while((k = socket.getInputStream().read(temp, 0, temp.length)) > -1) {
byte[] tbuff = new byte[incomingData.length + k];
System.arraycopy(incomingData, 0, tbuff, 0, incomingData.length);
System.arraycopy(temp, 0, tbuff, incomingData.length, k);
incomingData = tbuff;
receiveData(socket, incomingData);
}
} catch (IOException e) {
e.printStackTrace();
}
}
}
public void receiveData(Socket socket, byte[] data) {
int lenLo = (int) (data[0]);
int lenHi = (int) (data[1]);
int length = lenHi * 256 + lenLo;
if(lenHi < 0) {
System.out.println("Invalid Packet Length");
}
if(data.length != length) {
System.out.println("Incomplete Packet Received");
}
serverGUI.serverDebug("DATA RECEIVED");
serverGUI.serverDebug(Hex.encodeHexString(data)); //this is the java ouput above serverGUI is simply a jframe i built no data manipulation
serverGUI.serverDebug("DATA_RECEIVED DONE");
this.relay.sendData(data); //this function sends the data from socket server to the python asyncio server
}
public void receiveDataFromPythonRelay(Socket socket, byte[] data) {
serverGUI.debugPythonRelay("DATA RECEIVED");
serverGUI.debugPythonRelay(Hex.encodeHexString(data)); //this will be the output from the python script aka data decrypted.
//The data byte[] is created in the exact same way the incomingData array is built in the AuthServer run function
serverGUI.debugPythonRelay("DATA_RECEIVED DONE");
}
Additionally the way I am importing the data byte[] from sockets is programmed as such because the client does not send endl therefore readLine will not work from the streams.
A byte has 8 bits which means you can have maximum 0xff as value.
But Java uses signed byte which means msb is reserved for signed bit. This leaves you with only 7 bits for your value, so you can have maximum as 0x7f stored in byte type of variable. Any number greater than 0x07f will result in overflow.
Try using an int array. As int uses 4 bytes (32 bits) hence there will always be space for 8 bits.
use byte[] to read from the stream and then copy the contents in int[] , Use int intArr[i] = byteArr[i] & 0xFF; for ith value from byte[] to avoid negative numbers due to byte overflow
I have got a Java Server and a C# Client. And I'm really certain something goes wrong with the outputstreamer on the client or inputstream on the server.
byte[] arr = IOUtils.toByteArray(is,14);
PrintWriter out = new PrintWriter(os, true);
out.print("Received "+ new String(arr,"UTF-8"));
out.flush();
out.close();
"is" in this case is the Input Stream Variable. Coming from Socket.getInputStream().
Removing the length of the stream (14 in this case) makes the system time-out. The client does not get any respons except: "Read Failure"
The client side consists of a C# program
byte[] start = getIdentifier();
byte[] end = Encoding.UTF8.GetBytes(toSend);
byte[] arr = start.Concat(end).ToArray();
//Arr variable is a couple of strings smashed together and transformed into a byte[]
networkStream.Write(arr, 0, arr.Length);
networkStream.Flush();
StreamReader streamReader = new StreamReader(networkStream);
result = streamReader.ReadToEnd();
I actually tried writing to the Server with a BinaryWriter too, but that didn't work either. If I know what the length of the data is that will be send, I can fix the problem. But I do not know how to send the length of the stream to the server.
I've tried using Available() to see how much there was to read, but for some reason that sometimes was 0. As if the data wasn't been sent by the client yet.
Any ideas of how to fix the freezing?
EDIT:
I think I understand how the streaming works now.
At first I did this on the client side:
var length = BitConverter.GetBytes(arr.Length);
Array.Reverse(length);
Then on the server side I put this piece of code:
/* Get Input Data Length */
byte[] arr = IOUtils.toByteArray(is, 4);
int length = (arr[0] << 24) & 0xff000000 |
(arr[1] << 16) & 0x00ff0000 |
(arr[2] << 8) & 0x0000ff00 |
(arr[3] << 0) & 0x000000ff;
/* Get Input Data */
arr = IOUtils.toByteArray(is, length);
#Andreas your mention of the big-endian byte order made it clear how to send the length to the server
in my Client-Server application I found a strange error. I got the following Methods :
sendLoginResponse();
sendPlayerList();
sendCurrentLevelState();
Each Methods sends a byte array to the Client-Side
If I only call 2 of them, everything works fine and the Client-Side gets all the sent byte arrays. But if I call all 3 of them only the first and second one arrive the Client, the order of the following methods doesnt matter. but the Server says all of them were send. To write to the Client iam using the
write(byte[] b, int off, int len); method
all The lenghts within the packages make sense too.
Here comes the strange point:
if I add a Thread.sleep(1000); after the second Method, the third one does now arrive the Client after the sleep. I Have also tried to flush the DataOutputStream after every write call, but this didnt help.
EDIT:
So let's say I'd send 3 Login-Response
The Method's that gives me the byte[]:
public byte[] getLoginResponse(int playerID){
byte[] msg = new byte[4];
short shortMsgLength = 4;
byte[] msgLength = shortToBytes(shortMsgLength);
msg[0] = 2;
msg[1] = msgLength[0];
msg[2] = msgLength[1];
msg[3] = (byte) playerID;
return msg;
}
private byte[] shortToBytes(short value) {
byte[] returnByteArray = new byte[2];
returnByteArray[0] = (byte) (value & 0xff);
returnByteArray[1] = (byte) ((value >>> 8) & 0xff);
return returnByteArray;
}
And the Send Method:
private void sendLoginResponse() {
try{
byte[] msg = rfcObject.getLoginResponse(playerID);
out.write(msg,0,msg.length);
}catch(Exception e){
System.err.println(e.getMessage());
System.exit(0);
}
}
So if I call the sendLoginResponse(); three times in a row, the client only recieves 2 byte-arrays, but the server says it has been sent 3 times. If i add a
Thread.sleep(1000); `after the second Method-Call, everything works fine..`
The Client that reads the message runs in a Thread:
public void run(){
while(true){
try {
byte[] data = new byte[MAX_DATA_SIZE]; // MAX_DATA = 255
byteCount = in.read(data);
} catch (IOException ex) {
handleExceptionError(ex);
}
}
}
thank you!
if I call the sendLoginResponse(); three times in a row, the client only recieves 2 byte-arrays, but the server says it has been sent 3 times.
This is because TCP is a stream-oriented protocol. Meaning it doesn't know or care how your messages are delimited. There's no concept of individual messages in TCP, just a stream of bytes, with the guarantee that the order of bytes is preserved.
So when the sender calls three write, the three byte arrays are simply concatenated over the connection and arrives at the receiver in the same order, but the receiver doesn't necessarily need three read to get all the bytes, and even if it does take three read, the read doesn't necessarily gives you the same byte array passed to each corresponding write.
Your message already have the necessary information to get the individual message back from the byte stream:
// Client code for reading individual messages from a TCP connection
byte type = din.readByte();
// Read the message length, little-endian.
// We cannot use din.readShort because it's big-endian
int lenLo = din.read();
int lenHi = din.read();
short len = (short)(lenLo | (lenHi << 8));
byte [] body = new byte[len];
din.readFully(body);
DataOutputStream and TCP don't lose data.
As almost invariable seen in questions of this nature, the problem is at the receiving end. You are probably assuming that `read()' fills the buffer, and ignoring the count that it returns.
Based on your protocol description in comments, you should be using DataInputStream.readFully() in this circumstance:
byte type = din,readByte();
int length = din.readShort();
byte[] data = new byte[length];
din.readFully(data);
I have a problem with sending directory names over socket from my C++ client, to my Java server.
Sending ordinary messages like "hello world", works great , but the following doesn't and I can not figure out what the problem is:
char const * files = ffd.cFileName; // get directory name
string str(files, 0, strlen(files)); // convert pointer to string, right?
char mess[str.size()];
strcpy(mess, str.c_str()); // make char array :)
cout << "Send file: " << mess << " with strlen: " << strlen(mess) << " and sizeof: " << sizeof(mess) << endl;
int sent = 0;
if ((sent = send(connectSocket, mess, sizeof(mess), 0)) == SOCKET_ERROR)
{
closesocket(connectSocket);
WSACleanup();
connectToServer();
}
The java server just receives the directory names like this:
wam
p
Win
dow
s
Win
dow
s.o
ld
wxW
idg
ets
I can not understand what I'm missing because I have tried every possible way to do this and the C++ client prints like:
"Send file: windows with strlen: 7 and sizeof: 7"
I do not think that the java server is the problem since I can receive normal strings and messages perfectly, but anyway here is the JAVA code:
is = socket.getInputStream();
byteArray = new byteArray[1024];
while (true) {
c = is.read(byteArray, 0, byteArray.length);
String recv = new String(byteArray, 0, c);
System.out.println(recv);
if (recv.equals("<EOF>")){
break;
}
list.add(recv);
}
If you request something else or anything just leave a comment and I will fix it.
Question: are you sending via TCP or UDP? I'm guessing TCP, and if that is the case, you need to treat the socket as more of a stream. That stream may get broken up into a bunch of packets - you don't really control that. What I might do is to prefix the string length of each directory (ex, 3foo, 4barz, etc), read from the socket and determine what constitutes as a logical block or string, and then assemble / print the strings based on that. If you go with that route, you need to track how much you read each time until you think you are done.
I solved it, Just added byteArray = new byte[1024]; and now it works:
while (true) {
byteArray = new byte[1024]; // I ADDED THIS AND NOW THE JAVA SERVER RECEIVES IT CORRECTLY!
c = is.read(byteArray, 0, byteArray.length);
recv = new String(byteArray, 0, c);
System.out.println(recv);
if (recv.equals("<EOF>")){
break;
}
list.add(recv);
}
I'm writing a server program in c, and the client is on android platform which uses java language.Now I have a trouble to send char array from the server to the client,which means the client can get the data but can not decode it.I think it maybe because of the problem of data types or encoding differences.Can anyone give me some ideas,Thanks a lot!
Here is my code of server side:
char buf[MAXSIZE];
memset(buf, 0, MAXSIZE);
int n_write;
strcpy(buf, "0008200050005001");
n_write = Write(confd, buf, strlen(buf));
if (n_write <= 0)
{
fputs("Write error in send_matrix_and_position\n", stderr);
close(confd);
return -1;
}
And here is Java code:
mSocket = new Socket(HOST, PORT);
mIn = mSocket.getInputStream();
mOut = mSocket.getOutputStream();
byte[] lengthByte = new byte[4];
mIn.read(lengthByte);
for(byte b : lengthByte)
{
System.out.println(b + "");
}
You are sending 16 characters, but you read only four. Aren't you getting the data
48
48
48
56? These are the codes of the first four characters sent.
Try checking what values you get at the client when you read the char array, you might probably be doing br.readline(); see if this prints the characters??
You need to debug and check, then you might come up with some way.