I'm unable to send a UTF-8 string from a C# server to a Java client due to an EOF error in the client. How do I properly configure the C# server? I assume the error lies there because this client works with the Java server shown below.
Java client's receive function does this (this also works if I receive from a Java server, shown below):
DataInputStream dataInputStream = new DataInputStream(socket.getInputStream()); //The constructor initialises a field, using the socket object.
StringBuilder inputMessage = new StringBuilder();
inputMessage.append((String) dataInputStream.readUTF());
Desired C# server:
static async Task Main(string[] args)
{
TcpListener server = new TcpListener(IPAddress.Any, 34567);
server.Start();
byte[] bytes = new byte[4096];
byte[] responseBytes;
using (var client = await server.AcceptTcpClientAsync()){
using(var tcpStream = client.GetStream())
{
await tcpStream.ReadAsync(bytes, 0, bytes.Length);
var playerNumber = Encoding.UTF8.GetString(bytes);
Console.WriteLine("Player " + playerNumber + " connected."); //java client to server works.
StringBuilder outputMessage = new StringBuilder("Some output");
responseBytes = Encoding.UTF8.GetBytes(outputMessage.ToString());
await tcpStream.WriteAsync(responseBytes, 0, responseBytes.Length); //This doesn't work...
}
server.Stop();
}
}
The error:
java.io.EOFException
at java.base/java.io.DataInputStream.readFully(DataInputStream.java:201)
at java.base/java.io.DataInputStream.readUTF(DataInputStream.java:613)
at java.base/java.io.DataInputStream.readUTF(DataInputStream.java:568)
at Client.Connection.Receive(Connection.java:26)
at Client.Main.lambda$main$0(Main.java:30)
at com.sun.javafx.application.PlatformImpl.lambda$startup$5(PlatformImpl.java:271)
at com.sun.glass.ui.Application.invokeAndWait(Application.java:464)
at com.sun.javafx.tk.quantum.QuantumToolkit.runToolkit(QuantumToolkit.java:366)
at com.sun.javafx.tk.quantum.QuantumToolkit.lambda$startup$10(QuantumToolkit.java:280)
at com.sun.glass.ui.Application.lambda$run$1(Application.java:153)
Interestingly, a Java server doing this works:
DataOutputStream dataOutputStream = new DataOutputStream(socket.getOutputStream());
StringBuilder outputMessage = new StringBuilder("Some output");
dataOutputStream.writeUTF(outputMessage.toString());
dataOutputStream.flush();
EDIT
This is received from the working Java server. The "bytearr" contains 100 bytes that I am using for my message and 100 bytes that are 0 (they come after my message). The "chararr" correctly sees the first 100 bytes as something meaningful and the last 200 bytes as '\u0000':
This is received form the non-working C# server. It seems to start two bytes in compared to the correct version and also it's "chararr" contains only thousands of '\u0000':
DataInputStream's readUTF reads a special data format, it is not a general purpose method for reading a sequence of UTF-8 bytes. Most notably, it expects an initial sequence of bytes specifying the length of the stream.
I found the answer here. Changing the way the Java client reads to this, works:
byte[] buff = dataInputStream.readAllBytes();
String str = new String(buff, "UTF-8");
Related
I have a Java TCP Server Socket program that is expecting about 64 bytes of data from a piece of remote hardware. The Server code is:
public void run () throws Exception
{
//Open a socket on localhost at port 11111
ServerSocket welcomeSocket = new ServerSocket(11111);
while(true) {
//Open and Accept on Socket
Socket connectionSocket = welcomeSocket.accept();
DataInputStream dIn = new DataInputStream(connectionSocket.getInputStream());
int msgLen = dIn.readInt();
System.out.println("RX Reported Length: "+ msgLen);
byte[] msg = new byte[msgLen];
if(msgLen > 0 ) {
dIn.readFully(msg);
System.out.println("Message Length: "+ msg.length);
System.out.println("Recv[HEX]: " + StringTools.toHexString(msg));
}
}
}
This works correctly as I am able to test locally with a simple ACK program:
public class ACK_TEST {
public static void main (String[] args)
{
System.out.println("Byte Sender Running");
try
{
ACK_TEST obj = new ACK_TEST ();
obj.run();
}
catch (Exception e)
{
e.printStackTrace ();
}
}
public void run () throws Exception
{
Socket clientSocket = new Socket("localhost", 11111);
DataOutputStream dOut = new DataOutputStream(clientSocket.getOutputStream());
byte rtn[] = null;
rtn = new byte[1];
rtn[0] = 0x06; // ACK
dOut.writeInt(rtn.length); // write length of the message
dOut.write(rtn); // write the message
System.out.println("Byte Sent");
clientSocket.close();
}
}
And this correctly produces this output from the Server side:
However, when I deploy the same Server code on the Raspberry Pi and the hardware sends data to it, the data length is far greater and causes a heap memory issue (Even with the Heap pre-set at 512MB, which is definitely incorrect and unnecessary)
My presumption is I am reading the data wrong from the TCP socket as from the debug from the hardware, it's certainly not sending packets of this size.
Update: I have no access to the Client source code. I do however need to take the input TCP data stream, place it into a byte array, and then another function (Not shown) parses out some known HEX codes. That function expects a byte array input.
Update: I reviewed the packet documentation. It is a 10 byte header. The first Byte is a protocol identifier. The next 2 bytes is the Packet Length (Total number of bytes in the packet, including all the header bytes and checksum) and the last 7 are a Unique ID. Therefore, I need to read those 2 bytes and create a byte array that size.
Apparently the length from the header is about 1GB. Looks like the problem on the other end. Don't you mix low/big endian encoding?
I have a C++ server and two clients (ruby and java).
Everything is running on a 64-bit linux-machine (java 1.7.0_17)
The ruby client is fully working, but the java version makes problems.
In Java I tried to send a String from the client to the server. Actually the Server received the entire String, but the server thinks there is still something more to receive.
The ruby client looks a little bit like this:
socket = TCPSocket.open(#options[:host],#options[:port])
test = "Hello, World"
socket.puts test
socket.shutdown 1
response = socket.gets
Everything here is working fine. The ruby client sends a string. The server receives that string and sends a reply.
The Java Version looks like:
String ip = "127.0.0.1";
int port = 6686;
java.net.Socket socket = new java.net.Socket(ip,port);
OutputStreamWriter out = new OutputStreamWriter(socket.getOutputStream());
InputStreamReader in = new InputStreamReader(socket.getInputStream());
String msg = "Hello, world!";
//send
PrintWriter pw = new PrintWriter(out, true);
pw.print(msg);
pw.flush();
// I also tried: out.write(msg); out.flush(); nothing changed
//receive the reply
BufferedReader br = new BufferedReader(in);
char[] buffer = new char[300];
int count = br.read(buffer, 0, 300);
String reply = new String(buffer, 0, count);
System.out.println(reply);
socket.close();
On the other side there is a C++ Server:
string receive(int SocketFD) {
char buffer[SOCKET_BUFFER_SIZE];
int recv_count;
// empty messagestring
string message = "";
// empty buffer
memset(buffer, 0, sizeof(buffer));
while ((recv_count = recv(SocketFD, buffer, sizeof(buffer) - 1, 0)) > 0) {
/*if (recv_count == -1) {
cout << "failed." << endl;
break;
}*/
cout << recv_count << endl;
if (ECHO_SOCKETS) cout << "received: " << buffer << endl;
message.append(buffer);
memset(buffer, 0, sizeof(buffer));
if (ECHO_SOCKETS) cout << "message is now: " << message << endl;
}
return message;
}
The server output from the Java-message is:
13
received: Hello, world!
message is now: Hello, world!
and then nothing happens.
The problem is that:
recv(SocketFD, buffer, sizeof(buffer) - 1, 0)
is catched in an endless loop (or something like that).
If I kill the Java-client process or I type something like:
pw.print(msg);
out.close();
the output on the server side is:
_sending reply: "Request unrecognized/invalid" request="Hello, world!"
send reply success
now close connection
This output is right (except "send reply success"), but in case of adding:
out.close();
the client can't receive the reply of the server. Because the Socket is closed.
java.net.SocketException: Socket is closed
at java.net.Socket.getInputStream(Socket.java:864)
at MyServer.writeMessage(MyServer.java:56)
at MyServer.test(MyServer.java:42)
at MyServer.main(MyServer.java:30)
Edit
I tried to call pw.flush(); and different delimiters like "\n", "\r", "\r\n" and "\n\r" but the server still thinks there is still something to read. I also tried to use DatagramSockets:
java.net.DatagramSocket dSocket = new java.net.DatagramSocket();
InetAddress address = InetAddress.getByName("localhost");
String msg = "Hello, world!";
byte[] buf = msg.getBytes();
java.net.DatagramPacket packet = new DatagramPacket(buf, buf.length, address, 6686);
But the server can't accept the packet.
Solution
The ruby-client does something like a socket.shutdownOutput(); (ruby: socket.shutdown 1) after the call of puts. I changed the java-client-code:
out.write(msg);
socket.shutdownOutput();
and it works!
As #Charly said: I have to define a "protocol". In my case I'm not allowed to change any communication related code (in the server and the ruby-client) because this functionality is used by a another group of researchers. So I've to modify my java-client in that way, that it does the exact same things at the exact same time as the ruby-client (something like a protocol).
PrintWriter buffer (when autoflush is true) is only flushed by calling println or printf. Calling print may not flush the buffer (Javadoc). Try calling println or use a OutputStreamWriter directly and flush().
Be aware of using the right charset (You can set it up in OutputStreamWriter constructor).
Close the stream respectively flush it in a way like this:
DataOutputStream dataOut = new DataOutputStream(socket.getOutputStream());
dataOut.writeUTF(s);
dataOut.flush();
while ((recv_count = recv(SocketFD, buffer, sizeof(buffer) - 1, 0)) > 0) {
if (recv_count == -1) {
I don't know what your problem is but this code is certainly nonsense. It is impossible for the inner test ever to succeed.
i am developing an android application wherein i have to send a frame in jpeg format allocated to a BufferedArrayOutputStream (baos variable in code). I convert this baos into a byte array to write into the socket.
On the server side i would like to reconstruct the image in jpeg format. If i write the data received in a variable to a '.jpg' file on the server, on opening the file, it gives an error like "file starting with ffx0 not jpeg format". I think this is because the string variable in python writes the data in the file as a hex string.
The client code is as follows :-
Bitmap memoryImage = Bitmap.createBitmap(rgb, previewSize.width,previewSize.height,Bitmap.Config.ARGB_8888);
ByteArrayOutputStream baos = new ByteArrayOutputStream();
if(memoryImage.compress(CompressFormat.JPEG,100, baos)){
try {
if(count==0){
byte [] Finalbaos = baos.toByteArray();
int tempLen = Finalbaos.length;
Log.v("Client","ImageBytes :"+ tempLen);
String dataMeta = Integer.toString(tempLen);
Log.v("Client","Integer Size :"+ dataMeta.length());
PrintWriter tempOut = new PrintWriter(socket.getOutputStream());
if(tempOut!=null){
tempOut.write(dataMeta);
Log.v("Client","data size sent");
tempOut.flush();
}
DataInputStream in = new DataInputStream(socket.getInputStream());
if(in!=null){
Log.v("Client","read buffer created");
String xyz = in.readLine();
String temp = "recvd";
Log.v("Client",xyz);
if(xyz.equals(temp)){
OutputStream out = socket.getOutputStream();
out.write(Finalbaos,0,tempLen);
out.flush();
Log.d("Client", "Client sent message");
}
}
server code:
import socket,thread
import string
import array
host=""
port=54321
s=socket.socket()
s.setsockopt(socket.SOL_SOCKET,socket.SO_REUSEADDR,1)
s.bind((host,port))
s.listen(5)
conn,address=s.accept()
data=""
mylen=0
dataRecv=0
file1 = open("myfile.jpg","w")
totalLength=""
length=conn.recv(1024)
conn.send("recvd")
mylen=int(length)
while dataRecv<mylen:
newData=""
newData=conn.recv(1)
if not newData:
break
data+=newData
dataRecv+=len(newData)
result= array.array('B',data.decode("hex"))
file1.write(result)
file1.close()
print len(data)
conn.close()
s.close()
can anyone let me know how to reconstruct the frame on server either in python or C++
mylen=len(length) doesn't give you the length you're trying to send. it gives you how many bytes were read in the previsous recv. So you get the wrong lenght there.
on your client side, you use String xyz = in.readLine(); which will block until a newline character is encountered. but you never send a '\n' on the server side, instead you go waiting for a response from the client. so you have a deadlock there.
you use data.decode("hex") on your recieved data. unless you do the equivalend of data.encode("hex") in java on the other side, that won't work. it should give you an error if the string is not a valid hex-representation of a binary string.
result is an array.array, which you write to file. file1.write expects a string as argument, it gives you an error if you pass your result object.
so i can't even see why your code works at all, and why there's anything at all in your file.
Local on Linux. It's about 10 seconds for a 20k message. My guess is my Java is bad and Python is fine.
py client:
def scan(self, msg):
try:
print 'begin scan'
HOST = 'localhost'
PORT = 33000
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((HOST, PORT));
s.sendall(msg)
data = s.recv(1024)
s.close()
print 'Received', repr(data)
except Exception, e:
print "error: " + str(e)
Java server:
ServerSocket service = new ServerSocket(33000);
while(true) {
debug("Begin waiting for connection");
//this spins
Socket connection = service.accept();
debug("Connection received from " + connection.getInetAddress().getHostName());
OutputStreamWriter out = new OutputStreamWriter(connection.getOutputStream());
BufferedInputStream in = new BufferedInputStream(connection.getInputStream());
ScanResultsHeader results = new ScanResultsHeader();
Scanner scanner = new Scanner();
results = scanner.scan("scannerfake#gmail.com", "123", in);
and
public ScanResultsHeader scan (String userEmail,
String imapRetrievalId,
BufferedInputStream mimeEmail)
throws IOException, FileNotFoundException, MimeException, ScannerException {
//how fast would it be to just slurp up stream?
debug("slurp!");
String slurp = IOUtils.toString(mimeEmail);
debug("slurped " + slurp.length() + " characters");
slurp = slurp.toLowerCase();
debug("lc'ed it");
//...
My guess is I'm juggling the input streams wrong. One catch is the "BufferedInputStream mimeEmail" signature is required by the library API scan is using, so I'll need to get to that form eventually. But I noticed the simple act of slurping up a string takes ludicrously long so I'm already doing something incorrect.
Revising my answer....
If you are reading efficiently, and it appears you are, it will only be taking a lot time because either
You are creating a new connection every time you send a message which can be very expensive.
You are not sending the data as fast as you think.
The message is very large (unlikely but it could be)
There are plenty of examples on how to do this and a good library you can use is IOUtils which makes it simpler.
You should be able to send about 200K/s messages over a single socket in Java.
If you have a sends X bytes protocol using Big Endian you can do this.
DataInputStream dis = new DataInputStream( ...
int len = dis.readInt();
byte[] bytes = new byte[len];
dis.readFully(bytes);
String text = new String(bytes, "UTF-8");
Original problem was that the client isn't sending an end-of-input so the "slurp" operation keeps waiting for more stuff to cross the connection.
Solution was to implement an application-layer protocol to send the size of the message in advance, then stop listening for more message after that many bytes. I would have preferred a standard library -- something like, FiniteInputStream extends BufferedInputStream and takes a size as an argument, but wrote my own.
I am writing a byte array from a socket client as:
byte[] arr = { 49, 49, 49, 49, 49};
InetAddress address = InetAddress.getByName(host);
connection = new Socket(address, port);
out = new ObjectOutputStream(connection.getOutputStream());
out.flush();
At receiving end, on the server I have:
byte[] msgType = new byte[5];
in = new BufferedInputStream(connection.getInputStream());
int bytesRead = in.read(msgType, 0, 5);
System.out.println("msg rcvd: " + msgType.toString());
In the output I get weird string:
server: waiting for connection
server: connection received from localhost
server: Connection Successful
bytes read: 5
msg rcvd: ��w
How can make sure I get same bytes as I sent from my client?
I'm not sure exactly what are trying to print out, but I can tell you that msgType.toString() will not print the contents of the byte array.
Here is a link I found to a method which will print out the byte array in a more meaningful fashion.
You're getting the same bytes, it's just a matter of how you interpret them. If you want to see the bytes as a String use this instead:
System.out.println("msg rcvd: " + new String(msgType, "UTF-8"));
Be careful that the bytes you're dealing have the right encoding though (I assumed UTF-8 here). Since you're already ObjectOutputStream though, you could just use its writeUTF() on the server side and ObjectInputStream.readUTF() on the client side.
If you use an ObjectOutputStream on one side, you'll have to use an ObjectInputStream at the other side.
In your case, a simple OutputStream (maybe buffered) and .write() and .read() will do.
But for printing, don't use byte[].toString(), use Arrays.toString() if you want to have a formatted output.
Edit: I just see you are not even writing your array on the sending side. So you are actually only reading the ObjectOutputStream header.
From the comment:
I am handling server side and I am told that I would be send a byte array. How
do I receive and print that byte array ? byte array in this case is bytes of text/strings
This sounds like the server sends something like simply the Strings encoded in some encoding, like ASCII, UTF-8 or ISO-8859-1. If so, on the receiving end you can use something like this:
String encoding = "UTF-8";
BufferedReader in =
new BufferedReader(new InputStreamReader(connection.getInputStream(),
encoding));
String line;
while((line = in.readLine()) != null) {
System.out.println(line);
}
Of course, make sure the encoding is the same encoding as actually used on the sending side.
The corresponding sending code could be something like this:
String encoding = "UTF-8";
Writer w =
new BufferedWriter(new OutputStreamWriter(connection.getOutputStream(),
encoding));
w.write("Hello World!\n");
w.write("And another line.\n");
w.flush();