Develop Reference

Java TCP Socket Byte Heap Memory Issue

I have a Java TCP Server Socket program that is expecting about 64 bytes of data from a piece of remote hardware. The Server code is:
public void run () throws Exception
{
//Open a socket on localhost at port 11111
ServerSocket welcomeSocket = new ServerSocket(11111);
while(true) {
//Open and Accept on Socket
Socket connectionSocket = welcomeSocket.accept();
DataInputStream dIn = new DataInputStream(connectionSocket.getInputStream());
int msgLen = dIn.readInt();
System.out.println("RX Reported Length: "+ msgLen);
byte[] msg = new byte[msgLen];
if(msgLen > 0 ) {
dIn.readFully(msg);
System.out.println("Message Length: "+ msg.length);
System.out.println("Recv[HEX]: " + StringTools.toHexString(msg));
}
}
}
This works correctly as I am able to test locally with a simple ACK program:
public class ACK_TEST {
public static void main (String[] args)
{
System.out.println("Byte Sender Running");
try
{
ACK_TEST obj = new ACK_TEST ();
obj.run();
}
catch (Exception e)
{
e.printStackTrace ();
}
}
public void run () throws Exception
{
Socket clientSocket = new Socket("localhost", 11111);
DataOutputStream dOut = new DataOutputStream(clientSocket.getOutputStream());
byte rtn[] = null;
rtn = new byte[1];
rtn[0] = 0x06; // ACK
dOut.writeInt(rtn.length); // write length of the message
dOut.write(rtn); // write the message
System.out.println("Byte Sent");
clientSocket.close();
}
}
And this correctly produces this output from the Server side:
However, when I deploy the same Server code on the Raspberry Pi and the hardware sends data to it, the data length is far greater and causes a heap memory issue (Even with the Heap pre-set at 512MB, which is definitely incorrect and unnecessary)
My presumption is I am reading the data wrong from the TCP socket as from the debug from the hardware, it's certainly not sending packets of this size.
Update: I have no access to the Client source code. I do however need to take the input TCP data stream, place it into a byte array, and then another function (Not shown) parses out some known HEX codes. That function expects a byte array input.
Update: I reviewed the packet documentation. It is a 10 byte header. The first Byte is a protocol identifier. The next 2 bytes is the Packet Length (Total number of bytes in the packet, including all the header bytes and checksum) and the last 7 are a Unique ID. Therefore, I need to read those 2 bytes and create a byte array that size.

Apparently the length from the header is about 1GB. Looks like the problem on the other end. Don't you mix low/big endian encoding?

Transferring Large Data with UDP protocol

I know TCP is better to send file but I have a homework about sending file via udp protocol . Is there any code example in C# or Java about sending file?
I have server-client example to send and recieve message. I tried to send the file using the same way but could not succeed. I may need an algorithm to divide the file small parts and send them via datagram, and I have an idea to put "md5" of the part as header of the array to check if the packet is lost or not.
Here is my try , my server side in java;
// 1. creating a server socket, parameter is local port number
sock = new DatagramSocket(7777);
// buffer to receive incoming data
byte[] buffer = new byte[65536];
DatagramPacket incoming = new DatagramPacket(buffer, buffer.length);
byte []bigByteArray=new byte[1024*1024*1024*1024];
// 2. Wait for an incoming data
echo("Server socket created. Waiting for incoming data...");
ByteBuffer target = ByteBuffer.wrap(bigByteArray);
// communication loop
while(true)
{
try
{
sock.receive(incoming);
String s = new String(incoming.getData());
if(s=="finish") break;
target.put(incoming.getData());
}
catch(Exception e)
{
}
}
fos.write(bigByteArray);
fos.close();echo("RECIEVED");
and my client side;
String s;
Path path=Paths.get("C:\\Users\\Toshiba\\Desktop\\aa.txt");
byte[] data = Files.readAllBytes(path);
try
{
sock = new DatagramSocket();
InetAddress host = InetAddress.getByName("localhost");
//take input and send the packet
byte [] part;
for (int i = -1; i < data.length; i=i+100)
{
if(sock.isConnected())
{
part=Arrays.copyOfRange(data,i+1,i+100 );
}
else i=i-100;
}
byte [] f="finish".getBytes();
DatagramPacket finalpac = new DatagramPacket(f ,f.length , host , port);
sock.send(finalpac);
}
Thank you in advance.

Several issues:
The following isn't correct:
sock.receive(incoming);
String s = new String(incoming.getData());
The final line should be
String s = new String(incoming.getData(), incoming.getOffset(), incoming.getLength());
and if you aren't receiving text you shouldn't be converting the data to a String at all.
Remove the sock.isConnected() test. DatagramSockets are not usually connected, and you certainly haven't connected this one.
The loop in which this is embedded does nothing useful. You are only sending the word "finish".

Beginner Java Socket Programming Errors

I'm starting to write my first Java networking program, and long story short I'm having difficulty making sure that I'm taking the right approach. Our professor has given us a server program to test against this UDP client, but I'm getting some errors I can't seem to squash. Specifically, I get IO exceptions, either "Connection Refused" or "No route to host" exceptions.
public class Lab2Client {
/**
* #param args[1] == server name, args[2] == server port, args[3] == myport
*/
public static void main(String[] args) {
//Serverport is set to 10085, our client is 10086
try {
Socket echoSocket = new Socket(args[0],Integer.parseInt(args[2]));
System.out.println("Server connection Completed\n");
DataOutputStream output = new DataOutputStream(echoSocket.getOutputStream());
byte[] toSend = new byte[5];
toSend[0] = 12; toSend[1] = 34;//Code Number
toSend[2] = 15;//GroupId
toSend[3] = 86;toSend[4] = 100;//Port number in Little Endian Order
output.write(toSend);
System.out.println("Sent Request. Waiting for reply...\n");
DataInputStream input = new DataInputStream(echoSocket.getInputStream());
byte[] toRecieve = new byte[]{0,0,0,0,0,0,0,0};
input.read(toRecieve);
checkMessage(toRecieve);
}
catch (UnknownHostException e) {
System.err.println("Servername Incorrect!");
System.exit(1);
}
catch (IOException e){
System.err.println("IO Exception. Exiting...");
System.err.println(e);
System.exit(1);
}
}
I also have some questions about my implementation regarding receiving messages in Java. I'll be getting a datagram that contains either:
a) 3 formatting bytes (unimportant to the question) along with an IP and port number
or
b) 3 formatting bytes and a port.
Is using a DataInputStream the correct way to do this? I know using an array with 9 elements is lazy instead of dynamically allocating one that's either 5 or 9, but right now I'm just trying to get this working. That being said, is there a different approach anyone would suggest for this?

You need not to wrap the stream returned by Socket.getOuputStream() with DataOutputStream - it is already the DataOutputStream
In this line:
Socket echoSocket = new Socket(args[0],Integer.parseInt(args[2]));
I suppose it should be args[1], not args[0].
Here you have to convert the integer value to its byte representation:
toSend[3] = 10086 & 0xFF;toSend[4] = 10086>>8; //Port number in Little Endian Order
Answer to your question: case b as you are not sending the IP

thought I'd leave this up for posterity. The problem is simple, and I'm a fool for not noticing it sooner.
The correct programs I was testing this against used the UDP protocol, and this program is written in TCP. The corrected code is:
public class Lab2Client {
/**
* #param args[0] == server name, args[1] == server port, args[2] == myport
*/
public static void main(String[] args) {
//Serverport is 10085, our client is 10086
try {
DatagramSocket clientSocket = new DatagramSocket();
InetAddress IPAddress = InetAddress.getByName(args[0]);
int portToSend = Integer.parseInt(args[2]);
System.out.println("Clent Socket Created");
byte[] toSend = new byte[5];
toSend[0] = 0x12; toSend[1] = 0x34;//Code Number
toSend[2] = 15;//GroupId, f in hex
toSend[3] = 0x27;toSend[4] = 0x66;
System.out.println("Byte Array Constructed");
DatagramPacket sendPacket = new DatagramPacket(toSend, toSend.length, IPAddress, Integer.parseInt(args[1]));
clientSocket.send(sendPacket);
System.out.println("Sent Request. Waiting for reply...\n");
DataInputStream input = new DataInputStream(echoSocket.getInputStream());
toRecieve can either be an error message, a return of what we sent,
or a byte stream full of IP info and port numbers.
the "heavy" byte stream is either 4 for IPv4 of 16 for IPv6, 2 bytes for port,
and the magic number (2 bytes) for a total of 9-20 bytes*/
byte[] toRecieve = new byte[9];
DatagramPacket receivePacket = new DatagramPacket(toRecieve, toRecieve.length);
clientSocket.receive(receivePacket);
checkMessage(toRecieve);
} //and so on and so forth...
Thanks to #Serge for the help, though nobody could have answered my question correctly with how I asked it. The byte shifting you suggested was important too.

Getting message size across client - server

Hi I have code which sets up a basic client to server scenario and exchanges data between them. Basically the server waits for data from the client and, once sent by the client, prints it to the screen.
I have been tasked with getting the length of the data sent. I am a-little confused as to what this means as it could mean the actual length of the data or the total size of the packet sent.
The message is simply Mary.
Now does this mean the packet size would be 4? I have code that gets this like so:
System.out.println( "Packet Length: "+packet.getData().length );
System.out.println( "Data Length: "+message.getBytes().length );
They both return 4. But the maximum data packet size is 65535 so Im wondering if this number should be a lot bigger...
Now with code:
public static void main(String[] args) throws IOException
{
String message = "Mary";
byte[] buf = message.getBytes();
System.out.println("Aclient: message is:" + message);
String serverName = "127.0.0.1";
if (args.length == 1) {
serverName = args[0];
}
// get a datagram socket
DatagramSocket socket = new DatagramSocket();
// send request
InetAddress address = InetAddress.getByName(serverName);
int port = 4441;
DatagramPacket packet = new DatagramPacket(
buf, buf.length, address, port);
System.out.println( "Packet Length: "+packet.getData().length );
System.out.println( "Data Length: "+message.getBytes().length );
System.out.println("Aclient: destination name:" +
address.getHostName() + " destination port:" + port);
System.out.println("Aclient: sending datagram");
socket.send(packet);
}

What you see is the expected behavior. "Mary".getBytes() returns a byte[] with length 4; that's 1 byte for each character, which makes sense because it's nothing but ASCII.
Likewise, you've told the DatagramPacket that you're sending those same 4 bytes, so it should be no surprise that DatagramPacket#getData() agrees with ...the data you're asking it to send.
Hopefully it makes sense to you that the packet wouldn't use the full 65535 bytes unless you actually have that much data to send. It's a maximum, not a minimum, size.

java.net.DatagramPacket.getLength() returns the "payload", not the size of the raw UDP packet. I'd ask for clarification on that requirement.
From javadoc:
Returns the length of the data to be sent or the length of the data received.

Java UDP socket - Data are left over at the server-side

I am implementing a really basic server-client model in Java, by using UDP sockets and I have come across a really strange issue.
All I want to do is let the user (client) send a message to the server and then the server will print it.
I have an example but I am missing something since I have the following issue:
If the client sends the message "a" to the server it gets received correctly.
If the client sends the message "bbb" to the server it gets received correctly.
If the client sends the message "c" to the server, then the server will print "cbb" as the received message.
It seems as if the server does clean some kind of buffer when it gets a new message.
This is the code I am using:
Server
import java.net.DatagramPacket;
import java.net.DatagramSocket;
import java.net.InetAddress;
public class UDPServer {
public static void main(String args[]) throws Exception {
byte[] receive_data = new byte[256];
int recv_port;
DatagramSocket server_socket = new DatagramSocket(5000);
System.out.println("Server - Initialized server. Waiting for client on port 5000");
while (true) {
// System.out.println("Server - Listening for connections...");
DatagramPacket receive_packet = new DatagramPacket(receive_data, receive_data.length);
server_socket.receive(receive_packet);
String data = new String(receive_packet.getData());
InetAddress IPAddress = receive_packet.getAddress();
recv_port = receive_packet.getPort();
if (data.equals("q") || data.equals("Q")) {
System.out.println("Server - Exiting !");
break;
} else {
System.out.println("Server - Client from IP " + IPAddress + " # port " + recv_port + " said : " + data + " (length: " + receive_packet.getLength() + ")");
}
}
}
}
Client
public class UDPClient {
public static void main(String args[]) throws Exception {
byte[] send_data = new byte[256];
BufferedReader infromuser = new BufferedReader(new InputStreamReader(System.in));
DatagramSocket client_socket = new DatagramSocket();
InetAddress IPAddress = InetAddress.getByName("localhost");
System.out.println("Client - Initialized the client...");
while (true) {
System.out.print("Client - Type Something (q or Q to quit): ");
String data = infromuser.readLine();
if (data.equals("q") || data.equals("Q")) {
System.out.println("Client - Exited !");
DatagramPacket send_packet = new DatagramPacket(send_data, send_data.length, IPAddress, 5000);
System.out.println("Client - Sending data : <" + data + ">");
client_socket.send(send_packet);
break;
} else {
send_data = data.getBytes();
DatagramPacket send_packet = new DatagramPacket(send_data, send_data.length, IPAddress, 5000);
System.out.println("Client - Sending data : <" + data + ">");
client_socket.send(send_packet);
}
}
client_socket.close();
}
}
I suppose that the mistake is something trivial, but my skills in network programming are limited, therefore I don't know what exactly it is.
Just to make clear, I am running both the server and the client at the same machine (mac) on different terminals, just in case it affects the situation in anyway.
Any help would be greatly appreciated.
EDIT
...And I come back to answer my own question.
The problem was that I was not defining the amount of data that the server socket should expect to read.
Therefore when I change
String data = new String(receive_packet.getData());
with
String data = new String(receive_packet.getData(), 0, receive_packet.getLength());
everything worked smoothly.
Just for future reference and for people who might come across the same problem :)

When you're constructing the String based on the result, you're currently ignoring the length of the received packet.
After using DataSocket.receive(DatagramPacket), the length of the DatagramPacket should be set to the length that was actually received:
The length field of the datagram packet object contains the length of
the received message. If the message is longer than the packet's
length, the message is truncated.
This should fix the problem on the receiving side:
String data = new String(receive_packet.getData(), 0, receive_packet.getLength());
For this to work you also need to make sure the data sent is of the right size. In particular, don't use send_data.length to construct the outgoing DatagramPacket. This will always use the full length of the buffer). The length parameter isn't meant to be always send_data.length (otherwise the constructor would get it itself from the array), it's meant for the actual useful length of the message within that array.

On your first call this is what receive_data looks like:
--------------
|"a"| | |
--------------
On your second call:
--------------
|"b"|"b"| "b" | notice that the "a" in data_receive was overwritten
--------------
On your third call, you only send a single letter,
so the only part of the array that gets overwritten is the first element:
--------------
|"c"|"b"| "b" |
--------------
This is happening because there is still data left in the receive_data array in between messages to the server, a simple way around this would to just initialize a new array inside of you receive loop. That way every time you receive a message you will have a fresh array waiting for you.
while (true)
{
byte[] receive_data = new byte[256];
......
}

To solve the problem you should use length of receive_packet to create a String or array.
For higher performance in server side, it's better to initialize receive_packet before while section and reset its length at the end of while section to reuse it in loop : receive_packet.setLength(buffer.length);