Develop Reference

Java Server & C# Client freezing

I have got a Java Server and a C# Client. And I'm really certain something goes wrong with the outputstreamer on the client or inputstream on the server.
byte[] arr = IOUtils.toByteArray(is,14);
PrintWriter out = new PrintWriter(os, true);
out.print("Received "+ new String(arr,"UTF-8"));
out.flush();
out.close();
"is" in this case is the Input Stream Variable. Coming from Socket.getInputStream().
Removing the length of the stream (14 in this case) makes the system time-out. The client does not get any respons except: "Read Failure"
The client side consists of a C# program
byte[] start = getIdentifier();
byte[] end = Encoding.UTF8.GetBytes(toSend);
byte[] arr = start.Concat(end).ToArray();
//Arr variable is a couple of strings smashed together and transformed into a byte[]
networkStream.Write(arr, 0, arr.Length);
networkStream.Flush();
StreamReader streamReader = new StreamReader(networkStream);
result = streamReader.ReadToEnd();
I actually tried writing to the Server with a BinaryWriter too, but that didn't work either. If I know what the length of the data is that will be send, I can fix the problem. But I do not know how to send the length of the stream to the server.
I've tried using Available() to see how much there was to read, but for some reason that sometimes was 0. As if the data wasn't been sent by the client yet.
Any ideas of how to fix the freezing?
EDIT:
I think I understand how the streaming works now.
At first I did this on the client side:
var length = BitConverter.GetBytes(arr.Length);
Array.Reverse(length);
Then on the server side I put this piece of code:
/* Get Input Data Length */
byte[] arr = IOUtils.toByteArray(is, 4);
int length = (arr[0] << 24) & 0xff000000 |
(arr[1] << 16) & 0x00ff0000 |
(arr[2] << 8) & 0x0000ff00 |
(arr[3] << 0) & 0x000000ff;
/* Get Input Data */
arr = IOUtils.toByteArray(is, length);
#Andreas your mention of the big-endian byte order made it clear how to send the length to the server

UDP Packet received okay in Java but corrupted in Python

I am trying to record audio from an Android tablet and send it to a python server. At the start of the byte packet, I include some relevant information about the state of the Android app (A byte array called "actives" -- but considering it's receiving fine by a Java server, this should not be relevant). The android code is as follows:
int read = recorder.read(buffer, 0, buffer.length);
for (int a = 0; a < actives.length; a++) {
outBuffer[a+1] = (byte)actives[a];
logger = logger + Byte.toString(actives[a]) + ",";
}
int furthest=0;
for(int a =0; a < buffer.length; a++){
outBuffer[actives.length+1+a]=buffer[a];
if(buffer[a]!=0)furthest=a;
}
packet = new DatagramPacket(outBuffer, read,
serverAddress, PORT);
Log.d("writing", logger+Byte.toString(outBuffer[7])+".length"+Integer.toString(1+furthest+actives.length+1));
Log.d("streamer","Packet length "+outBuffer.length);
try {
socket.send(packet);
}catch (IOException e){
Log.e("streamer", "Exception: " + e);
}
Log.d("streamer","packetSent");
I receive a clean signal on the other end using a Java server.
Image of received java output: !(http://i.imgur.com/31UWzya.png)
This is my Java server:
DatagramSocket serverSocket = new DatagramSocket(3001);
int byteSize=970;
byte[] receiveData = new byte[byteSize];
DatagramPacket receivePacket = new DatagramPacket(receiveData,
receiveData.length);
while(true){ // recieve data until timeout
try {
serverSocket.receive(receivePacket);
String rcvd = "rcvd from " + receivePacket.getAddress();
System.out.println("receiver"+"Received a packet!" +rcvd);
break;
}
catch (Exception e) {
// timeout exception.
System.out.println("Timeout reached without packet!!! " + e);
timeoutReached=true;
break;
}
}
if(timeoutReached)continue;
currTime = System.currentTimeMillis();
data = receivePacket.getData();
Here is my Python server's output:
!(http://i.imgur.com/RYkcCCE.png)
And here is the code:
import socket
ip="192.ip.address"
port=3001;
sock=socket.socket(socket.AF_INET,socket.SOCK_DGRAM);
sock.bind(('',port));
while(True):
data,addr=sock.recvfrom(970);
print("address",addr);
print("received a data!");
print(data);
In the last line of the python script, I have tried to change "print(data)" to "print(data.decode())", in which case I get this error:
UnicodeDecodeError: 'ascii' codec can't decode byte 0xff in position 0: ordinal not in range(128)
I am not running these servers at the same time
My guess is that it has to do something with Java using unsigned ints and python not doing that. Is there a way in Python that I can convert this data, because data.decode() is not working? Alternatively I should be able to convert the data in Java somehow? None of the answers on stackoverflow that I have tried have worked.

Decoding is the right approach. In your android app explicitly mention the character encoding. UTF-8 is the standard Charset that is used.
Your log is pretty clear. You are trying to decode the data packet as ASCII (which is the default encoding of the decode() function) but I'm guessing its ISO_8859_1 or UTF-8 (more likely).
Next try data.decode('utf8', 'ignore') in your android app. Note: 'ignore' is an optional argument and to be used only in case of debugging as it will ignore malformed(corrupted) data and try to convert individual characters. If you want to use decode() in production use 'strict' or no second argument ('strict' is the default).
In place of 'utf8' try other options from other Python Encodings.

This was pretty brutal to attack head-on. I tried specifying the encoding in Java (before sending) like another SO post suggested, but that didn't help. So I side-stepped the problem by converting my Android byte array into a comma-separated string, then converting the string back into UTF-8 bytes.
sendString="";
for(int a =0; a < buffer.length; a++){
sendString=sendString+Byte.toString(buffer[a])+",";
}
byte[] outBuffer = sendString.getBytes("UTF-8");
Make sure you reset your string to null ("") each time you go through the while loop, or your ish will get very slow af.
Then in Python,right after receiving:
data=data.decode("utf8");
Although I am stringifying 980 characters, it does not appear to add much to the processing time... although I do wish that I could send the raw bytes, as speed is very important to me here. I'll leave the question open in case someone can come up with a better solution.

Java DataOutputStream doesnt write all messages ( TCP Socket )

in my Client-Server application I found a strange error. I got the following Methods :
sendLoginResponse();
sendPlayerList();
sendCurrentLevelState();
Each Methods sends a byte array to the Client-Side
If I only call 2 of them, everything works fine and the Client-Side gets all the sent byte arrays. But if I call all 3 of them only the first and second one arrive the Client, the order of the following methods doesnt matter. but the Server says all of them were send. To write to the Client iam using the
write(byte[] b, int off, int len); method
all The lenghts within the packages make sense too.
Here comes the strange point:
if I add a Thread.sleep(1000); after the second Method, the third one does now arrive the Client after the sleep. I Have also tried to flush the DataOutputStream after every write call, but this didnt help.
EDIT:
So let's say I'd send 3 Login-Response
The Method's that gives me the byte[]:
public byte[] getLoginResponse(int playerID){
byte[] msg = new byte[4];
short shortMsgLength = 4;
byte[] msgLength = shortToBytes(shortMsgLength);
msg[0] = 2;
msg[1] = msgLength[0];
msg[2] = msgLength[1];
msg[3] = (byte) playerID;
return msg;
}
private byte[] shortToBytes(short value) {
byte[] returnByteArray = new byte[2];
returnByteArray[0] = (byte) (value & 0xff);
returnByteArray[1] = (byte) ((value >>> 8) & 0xff);
return returnByteArray;
}
And the Send Method:
private void sendLoginResponse() {
try{
byte[] msg = rfcObject.getLoginResponse(playerID);
out.write(msg,0,msg.length);
}catch(Exception e){
System.err.println(e.getMessage());
System.exit(0);
}
}
So if I call the sendLoginResponse(); three times in a row, the client only recieves 2 byte-arrays, but the server says it has been sent 3 times. If i add a
Thread.sleep(1000); `after the second Method-Call, everything works fine..`
The Client that reads the message runs in a Thread:
public void run(){
while(true){
try {
byte[] data = new byte[MAX_DATA_SIZE]; // MAX_DATA = 255
byteCount = in.read(data);
} catch (IOException ex) {
handleExceptionError(ex);
}
}
}
thank you!

if I call the sendLoginResponse(); three times in a row, the client only recieves 2 byte-arrays, but the server says it has been sent 3 times.
This is because TCP is a stream-oriented protocol. Meaning it doesn't know or care how your messages are delimited. There's no concept of individual messages in TCP, just a stream of bytes, with the guarantee that the order of bytes is preserved.
So when the sender calls three write, the three byte arrays are simply concatenated over the connection and arrives at the receiver in the same order, but the receiver doesn't necessarily need three read to get all the bytes, and even if it does take three read, the read doesn't necessarily gives you the same byte array passed to each corresponding write.
Your message already have the necessary information to get the individual message back from the byte stream:
// Client code for reading individual messages from a TCP connection
byte type = din.readByte();
// Read the message length, little-endian.
// We cannot use din.readShort because it's big-endian
int lenLo = din.read();
int lenHi = din.read();
short len = (short)(lenLo | (lenHi << 8));
byte [] body = new byte[len];
din.readFully(body);

DataOutputStream and TCP don't lose data.
As almost invariable seen in questions of this nature, the problem is at the receiving end. You are probably assuming that `read()' fills the buffer, and ignoring the count that it returns.
Based on your protocol description in comments, you should be using DataInputStream.readFully() in this circumstance:
byte type = din,readByte();
int length = din.readShort();
byte[] data = new byte[length];
din.readFully(data);

Converting pointer to string and send as char array not working properly

I have a problem with sending directory names over socket from my C++ client, to my Java server.
Sending ordinary messages like "hello world", works great , but the following doesn't and I can not figure out what the problem is:
char const * files = ffd.cFileName; // get directory name
string str(files, 0, strlen(files)); // convert pointer to string, right?
char mess[str.size()];
strcpy(mess, str.c_str()); // make char array :)
cout << "Send file: " << mess << " with strlen: " << strlen(mess) << " and sizeof: " << sizeof(mess) << endl;
int sent = 0;
if ((sent = send(connectSocket, mess, sizeof(mess), 0)) == SOCKET_ERROR)
{
closesocket(connectSocket);
WSACleanup();
connectToServer();
}
The java server just receives the directory names like this:
wam
p
Win
dow
s
Win
dow
s.o
ld
wxW
idg
ets
I can not understand what I'm missing because I have tried every possible way to do this and the C++ client prints like:
"Send file: windows with strlen: 7 and sizeof: 7"
I do not think that the java server is the problem since I can receive normal strings and messages perfectly, but anyway here is the JAVA code:
is = socket.getInputStream();
byteArray = new byteArray[1024];
while (true) {
c = is.read(byteArray, 0, byteArray.length);
String recv = new String(byteArray, 0, c);
System.out.println(recv);
if (recv.equals("<EOF>")){
break;
}
list.add(recv);
}
If you request something else or anything just leave a comment and I will fix it.

Question: are you sending via TCP or UDP? I'm guessing TCP, and if that is the case, you need to treat the socket as more of a stream. That stream may get broken up into a bunch of packets - you don't really control that. What I might do is to prefix the string length of each directory (ex, 3foo, 4barz, etc), read from the socket and determine what constitutes as a logical block or string, and then assemble / print the strings based on that. If you go with that route, you need to track how much you read each time until you think you are done.

I solved it, Just added byteArray = new byte[1024]; and now it works:
while (true) {
byteArray = new byte[1024]; // I ADDED THIS AND NOW THE JAVA SERVER RECEIVES IT CORRECTLY!
c = is.read(byteArray, 0, byteArray.length);
recv = new String(byteArray, 0, c);
System.out.println(recv);
if (recv.equals("<EOF>")){
break;
}
list.add(recv);
}

Java to Lua socket communication error

Okay, so I have a project where I'm working with a Client (written in Lua) and a Server (written in Java). I'm using LuaSocket for the client and DatagramSockets for the server. The problem is when I send one string from the client in Lua, and receive it on the server(and convert the bytes to a string), it doesn't recognize the value of the string as equal to what it should be(I'm using .equals() for evaluation). I've printed the result and compared it to the string(everything checked out); I've even compared the bytes (using .getBytes()), they even checked out. The most annoying part of this is that when I analyze the string with .startsWith() it evaluates true, but nothing else works. I've looked into the string encoding of both languages, but I'm relatively new to sockets and this is beyond me.
Edit:
Upon writing some example code to demonstrate the problem, I solved it. Here is the code:
Client:
local socket = require "socket"
local udp = socket.udp()
udp:settimeout(0)
udp:setpeername("localhost", 1234)
udp:send("foo")
Server:
public class Main
{
public static void main(String args[]) throws Exception
{
DatagramSocket server = new DatagramSocket(1234);
byte[] incomingBytes = new byte[512];
DatagramPacket incomingPacket = new DatagramPacket(incomingBytes, incomingBytes.length);
server.receive(incomingPacket);
String received = new String(incomingBytes);
System.out.println(received);
System.out.println(received.equals("foo"));
for (byte b : received.getBytes())
{
System.out.print(b + " ");
}
System.out.print("\n");
for (byte b : "foo".getBytes())
{
System.out.print(b + " ");
}
System.out.print("\n");
}
}
The result:
foo
false
102 111 111 0 0 0 *I'm not going to include all but there are 506 more*
102 111 111
The string I had been examining the bytes from previously was split at several points, and that would explain why I didn't notice this.

Indeed as Etan pointed out, you're creating a string from the entire buffer--all 512 bytes--instead of a string of the correct length, so the string that is created has lots of zero bytes at the end.
A simple fix would be to use the String constructor that cuts off the buffer at the position and length you specify, along with the number of bytes received from the packet from DatagramPacket.getLength
Change the line assigning received to
String received = new String(incomingBytes, 0, incomingPacket.getLength());