I want to read a stream of bytes when sending a command through UDP. I have the following code:
import java.io.*;
import java.net.*;
public class udp_client
{
private static File fout = new File("data2.txt");
public static void main(String args[])
{
DatagramSocket sock = null;
int port = 10001;
String s;
BufferedReader cin = new BufferedReader(new InputStreamReader(System.in));
try
{
sock = new DatagramSocket();
InetAddress host = InetAddress.getByName("192.168.0.2");
byte[] b = {(byte)0xf3,(byte)0xf2,(byte)0x02,(byte)0x01,(byte)0x00,(byte)0x00,(byte)0xfe,(byte)0x0e };
DatagramPacket dp = new DatagramPacket(b , b.length , host , port);
sock.send(dp);
//now receive reply
//buffer to receive incoming data
byte[] buffer = new byte[65536];
DatagramPacket reply = new DatagramPacket(buffer, buffer.length);
sock.receive(reply);
byte[] data = reply.getData();
s = new String(data, 0, reply.getLength());
//echo the details of incoming data - client ip : client port - client message
echo(reply.getAddress().getHostAddress() + " : " + reply.getPort() + " - " + s);
FileOutputStream fos = new FileOutputStream(fout);
BufferedWriter bw = new BufferedWriter(new OutputStreamWriter(fos));
for (int i=6;i<3078;i=i+3) {
//System.out.println(Integer.toHexString((data[k])));
System.out.println("hex="+Integer.valueOf(String.valueOf(data[k]), 16));
int r = (data[i] & 0xFF) | ((data[i+1] & 0xFF) << 8) | ((data[i+2] & 0x0F) << 16);
bw.write(Integer.toString(r));
bw.newLine();
//System.out.println("r["+i+"]="+r);
System.out.println(r);
//System.out.println(String.valueOf(data[i] & 0x00FF));
//JOptionPane.showMessageDialog(null, data[k]);
}
bw.close();
}
catch(IOException e)
{
System.err.println("IOException " + e);
}
}
//simple function to echo data to terminal
public static void echo(String msg)
{
System.out.println(msg);
}
}
the problem is that size of the replyis 520 and the data should be around 5000 bytes. How can I read the next data?
If the sender is sending packets that are 5000 bytes long, then the receive(packet) should receive and return them in their entirety provided that the buffer in the packet is large enough. (If the packet buffer is too small, the message will be truncated. Also, see caveats below about packet loss and fragmentation.)
In your case, the receive buffer is large enough to hold a 5000 byte message, so that implies that the sender is not sending UDP messages of that size.
Note: UDP is not recommended for sending large amounts of data. UDP messages get lost, and the receiver gets no notification if this occurs. The larger the message you send, the more likely it is that this will occur.
When you send a UDP message that won't fit into a single network packet, it will be broken up into fragments. The fragments are reassembled into the complete receiver in the OS protocol stack. However, if any of the fragments gets lost, the UDP message cannot be reassembled, and will be discarded by the OS (without notification).
Recommendations:
If you need reliability, use TCP rather than UDP.
Avoid sending large UDP messages.
Related
This question has been asked a lot, but so far, none of the solutions that I applied from previous answers have helped me.
Main goal
I am trying to learn UDP conexions and this is my attempt. I want to have a client ask for a picture at a server via UDP and the server will send it. Then the client will create a file with that information given.
Explanation
My main idea is to ask the server for an image using a "GET" command (not the HTTP, just GET) followed by the name of the image(extension included). Then the client awaits an answer which is the image requested.
Problems
The client waits and answer which does no come
Research
From another similar question it was a problem that I was using the same PORT for both receive and connect, so I added two ports, receivingPORT and sendingPORT, no results from the Client.
From other similar questions, It was a Firewall problem. So, on a Win10 machine, I created a new rule for UDP in the Firewall for the ports that I am using for this application, and nothing was received by the Client...
I have checked that the image is loaded into byte[] and the image is sent. But on the Client, nothing is received and stays there waiting for a connection to come through
CODE from Server
public class UDPserver {
static DatagramSocket serverUDP;
static DatagramPacket packet;
static InetAddress address;
static byte[] buffer = new byte[65507];//65507
final static int receivingPORT = 6668;
final static int sendingPORT = 6669;
public static void main(String[] args) throws SocketException, IOException, InterruptedException{
boolean serverActive = true;
String order = "";
String file = "";
//Instantiate server
serverUDP = new DatagramSocket(receivingPORT);
while(serverActive){
//Kind of packet we want to receive
packet = new DatagramPacket(buffer, buffer.length);
System.out.println("Server awaiting connection...");
//Receive it
serverUDP.receive(packet);
System.out.println("Received packet from: " + packet.getAddress() + "/" + packet.getPort());
//What does the packet contain?
String msg = new String(packet.getData());
address = packet.getAddress();
System.out.println("Order from: " + address + "/" + receivingPORT + " says: " + msg);
try{
order = msg.split(" ")[0].trim();
file = msg.split(" ")[1].trim();
} catch (Exception e){
}
switch(order){
case("GET"):{
System.out.println("Sending back an image...");
buffer = loadImageFromServer(file);
packet = new DatagramPacket(buffer, buffer.length, address, sendingPORT);
Thread.sleep(5000);
serverUDP.send(packet);
System.out.println("Client served");
break;
}
case("DISCONNECT"):{
buffer = "Server is disconnecting...".getBytes();
packet = new DatagramPacket(buffer, buffer.length, address, sendingPORT);
serverUDP.send(packet);
serverActive = false;
serverUDP.close();
break;
}
}
}
}
static byte[] loadImageFromServer(String path) {
try {
System.out.println("Loading path: " + path);
//Instantiate a buffer from the image for it
BufferedImage img = ImageIO.read(UDPserver.class.getResource(path));
//Create a byte[] stream object to handle the data
ByteArrayOutputStream baos = new ByteArrayOutputStream();
//Write the image data into those above with jpg format
ImageIO.write(img, "png", baos);
//Flush the information
baos.flush();
byte[] buffer = baos.toByteArray(); //Write it out on a byte string and return it
return buffer;
} catch (IOException ex) {
Logger.getLogger(UDPserver.class.getName()).log(Level.SEVERE, null, ex.fillInStackTrace());
System.exit(-1);
}
return null;
}
}
CODE client
public class Client {
static DatagramSocket clientUDP;
static InetAddress address;
static DatagramPacket packetSend;
static DatagramPacket packetReceive;
static int SIZE = 65507;
final static int receivingPORT = 6669;
final static int sendingPORT = 6668;
static byte[] buffer = new byte[SIZE];
static Scanner scan = new Scanner(System.in);
public static void main(String[] args) throws SocketException, UnknownHostException, IOException{
boolean clientLoop = true;
//Get address
address = InetAddress.getByName("localhost");
//Instantiate Client -> UDP
clientUDP = new DatagramSocket();
while(clientLoop){
System.out.print("Enter any key and press enter");
scan.next(); //Just to stop the loop
//Load the buffer
buffer = "GET imagenServidor.png".getBytes();
//buffer = "DISCONNECT".getBytes();
System.out.println("Buffer is ready");
//Arm the packet
packetSend = new DatagramPacket(buffer, buffer.length, address, sendingPORT);
System.out.println("Packet is armed!");
//Send the packet to the server
clientUDP.send(packetSend);
System.out.println("Order sent to server");
System.out.println("Waiting an answer");
packetReceive = new DatagramPacket(buffer, buffer.length, address, receivingPORT);
clientUDP.receive(packetReceive);
System.out.println("Server answered!");
ByteArrayInputStream bais = new ByteArrayInputStream(packetReceive.getData());
BufferedImage image = ImageIO.read(bais);
System.out.println(image);
}
clientUDP.close();
}
}
NOTES
This is a UDP exercise
The Reason
MTU!
You are sending packets with long buffe through UDP directly, which may not work in most network circumstances.
A packet sent through UDP should not be longer than the network MTU, otherwise it would be dropped. The network MTU may not be more than 1500 on most net nods(routers/switchs/hosts...), and even smaller sometimes. Though some nods may do sigmentation for ip packets, but you should not count on it when you are using UDP.
Suggestions
Use TCP instead in this application, as for:
You are sending data which expected to be complete (otherwise it would be useless).
You do not care about congestion control algorithms.
So just go with TCP.
Edit Based on The Update of The Question
So, as this is an excercise, in which you have to use UDP only.
As a file might be useless unless it is complete, you have to make sure:
All packets are possible to pass the path. Which means network should be connected both physically and virtually, and packet size should always be smaller than the MTU.
If any packets are lost, both the receiver and the sender should be able to know.
If any apckets come out of order, the receiver should be able to know.
Sender should be able to cache and resend the packets which are not confirmed by the receiver yet.
Make sure your have a good network connection. Split the image buffer into buffer array with each buffer item length less than 1000bytes(should be safe).
Then let's design an amature but simple protocol for this:
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| type | sequence number |
+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
| payload ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
For types, we may need:
hello: 0x01
bye: 0x02
ack: 0x03
nack: 0x04
data: 0x05
feedback: 0x06
...
Sequence should be mono-increasing. e.g. 1, 2, 3, 4.... (Not necessory to start from 1 but OK)
It works like following:
Sender->Receiver: hello(seq=i)
Receiver->Sender: ack(seq=i)
# Sender->Receiver: hello(seq=i)
# if timeout and got no ack for seq=i
Sender->Receiver: data(seq=i+1)
Receiver->Sender: ack(seq=i+1)
# Sender->Receiver: hello(seq=i+1)
# if timeout and got no ack for seq=i+1
Sender->Receiver: data(seq=i+2)
Sender->Receiver: data(seq=i+3)
Receiver->Sender: ack(seq=i+2)
Receiver->Sender: ack(seq=i+3)
# Sender->Receiver: hello(seq=i+2)
# if timeout and got no ack for seq=i+2 or got nack for seq=i+2
Sender->Receiver: bye(seq=n)
Receiver->Sender: ack(seq=n)
# bye is not necessory
Firstly, I think you need to learn how to use wirshark or tcmpdump to analysis network streams when debugging, that will help you find out the problem and solve it.
As for your program, there are several problems the user207421 has mensioned. I think it's better to use TCP, but if you want to learn UDP by this way, the thing you need is to do a slim reliable UDP by yourself.
For example, you may need the following models
Build a send buffer and recive buffer, check every time if the buffer is empty, if not, send/receive and process it.（Cause UDP has MTU）
Add some extra format of information in the head of each datagram, which includes the size of the whole message, the sequence of the datagram, the left size, etc.(Cause you need to cut your message into many parts)
Build a controller, which need to have some function like retransmission, rebuild the message, etc.(Cause UDP is unreliable, you need to check the completeness of all parts)
Hope that can help you.
I have a Java TCP Server Socket program that is expecting about 64 bytes of data from a piece of remote hardware. The Server code is:
public void run () throws Exception
{
//Open a socket on localhost at port 11111
ServerSocket welcomeSocket = new ServerSocket(11111);
while(true) {
//Open and Accept on Socket
Socket connectionSocket = welcomeSocket.accept();
DataInputStream dIn = new DataInputStream(connectionSocket.getInputStream());
int msgLen = dIn.readInt();
System.out.println("RX Reported Length: "+ msgLen);
byte[] msg = new byte[msgLen];
if(msgLen > 0 ) {
dIn.readFully(msg);
System.out.println("Message Length: "+ msg.length);
System.out.println("Recv[HEX]: " + StringTools.toHexString(msg));
}
}
}
This works correctly as I am able to test locally with a simple ACK program:
public class ACK_TEST {
public static void main (String[] args)
{
System.out.println("Byte Sender Running");
try
{
ACK_TEST obj = new ACK_TEST ();
obj.run();
}
catch (Exception e)
{
e.printStackTrace ();
}
}
public void run () throws Exception
{
Socket clientSocket = new Socket("localhost", 11111);
DataOutputStream dOut = new DataOutputStream(clientSocket.getOutputStream());
byte rtn[] = null;
rtn = new byte[1];
rtn[0] = 0x06; // ACK
dOut.writeInt(rtn.length); // write length of the message
dOut.write(rtn); // write the message
System.out.println("Byte Sent");
clientSocket.close();
}
}
And this correctly produces this output from the Server side:
However, when I deploy the same Server code on the Raspberry Pi and the hardware sends data to it, the data length is far greater and causes a heap memory issue (Even with the Heap pre-set at 512MB, which is definitely incorrect and unnecessary)
My presumption is I am reading the data wrong from the TCP socket as from the debug from the hardware, it's certainly not sending packets of this size.
Update: I have no access to the Client source code. I do however need to take the input TCP data stream, place it into a byte array, and then another function (Not shown) parses out some known HEX codes. That function expects a byte array input.
Update: I reviewed the packet documentation. It is a 10 byte header. The first Byte is a protocol identifier. The next 2 bytes is the Packet Length (Total number of bytes in the packet, including all the header bytes and checksum) and the last 7 are a Unique ID. Therefore, I need to read those 2 bytes and create a byte array that size.
Apparently the length from the header is about 1GB. Looks like the problem on the other end. Don't you mix low/big endian encoding?
I'm trying to write a program which acts as a server that will read bytes from a client that is written in PHP - sends request via socket (which i cannot recode due to policy) Here is the server code:
The server runs in: Red Hat Enterprise Linux Server release 6.2 (Santiago)
public void run() {
try {
serverSocket = new ServerSocket(port);
serverSocket.setSoTimeout(0);
while(!isInterrupted) {
try {
Socket server = serverSocket.accept();
LOG.info("Request received from : " + server.getRemoteSocketAddress());
DataInputStream in = new DataInputStream(server.getInputStream());
// DataInputStream in = new DataInputStream(
// new BufferedInputStream(server.getInputStream(), 10000));
byte[] bytes = new byte[10000];
int byteDupLength = in.read(t_bytes);
// in.readFully(bytes); // I tried this but to no avail
// int byteDupLength = bytes.length;
LOG.info(byteDupLength);
byte[] byteDup = new byte[byteDupLength];
System.arraycopy(bytes, 4, byteDup, 0, byteDupLength);
// FOR INFORMATION ONLY
/*for (byte b : byteDup){
LOG.info(b);
}*/
ByteBuffer buffer = ByteBuffer.wrap(byteDup);
LOG.info(buffer);
forwardRequest(byteDup);
server.close();
}
catch(SocketTimeoutException s) {
LOG.error("Socket timed out!", s);
break;
}
catch(IOException e)
{
LOG.error("IOException:", e);
break;
}
}
}
catch (IOException ex) {
LOG.error("Server socket is null", ex);
}
LOG.fatal("ReceiverEngine interrupted!");
}
I encountered a problem when the client sends request consisting of 4948 bytes. The only bytes the server can read is 2090.
Another thing that seems a mystery to me is that, when I run the server via Netbeans in my local (which is a Windows 7 Pro), it works as expected. I dont know what is wrong. Please help.. :)
Thanks!
TCP is a byte stream protocol.
The read() method isn't guaranteed to fill the buffer.
Therefore if you don't receive the expected number of bytes in a single read, you have to loop until you do receive them.
readFully() would have worked if the buffer size agreed with the size of what was sent. In your case you specified a buffer of 10,000 bytes, which weren't sent, so it would have blocked waiting for the other 10000-4948 bytes.
I have a C++ server and two clients (ruby and java).
Everything is running on a 64-bit linux-machine (java 1.7.0_17)
The ruby client is fully working, but the java version makes problems.
In Java I tried to send a String from the client to the server. Actually the Server received the entire String, but the server thinks there is still something more to receive.
The ruby client looks a little bit like this:
socket = TCPSocket.open(#options[:host],#options[:port])
test = "Hello, World"
socket.puts test
socket.shutdown 1
response = socket.gets
Everything here is working fine. The ruby client sends a string. The server receives that string and sends a reply.
The Java Version looks like:
String ip = "127.0.0.1";
int port = 6686;
java.net.Socket socket = new java.net.Socket(ip,port);
OutputStreamWriter out = new OutputStreamWriter(socket.getOutputStream());
InputStreamReader in = new InputStreamReader(socket.getInputStream());
String msg = "Hello, world!";
//send
PrintWriter pw = new PrintWriter(out, true);
pw.print(msg);
pw.flush();
// I also tried: out.write(msg); out.flush(); nothing changed
//receive the reply
BufferedReader br = new BufferedReader(in);
char[] buffer = new char[300];
int count = br.read(buffer, 0, 300);
String reply = new String(buffer, 0, count);
System.out.println(reply);
socket.close();
On the other side there is a C++ Server:
string receive(int SocketFD) {
char buffer[SOCKET_BUFFER_SIZE];
int recv_count;
// empty messagestring
string message = "";
// empty buffer
memset(buffer, 0, sizeof(buffer));
while ((recv_count = recv(SocketFD, buffer, sizeof(buffer) - 1, 0)) > 0) {
/*if (recv_count == -1) {
cout << "failed." << endl;
break;
}*/
cout << recv_count << endl;
if (ECHO_SOCKETS) cout << "received: " << buffer << endl;
message.append(buffer);
memset(buffer, 0, sizeof(buffer));
if (ECHO_SOCKETS) cout << "message is now: " << message << endl;
}
return message;
}
The server output from the Java-message is:
13
received: Hello, world!
message is now: Hello, world!
and then nothing happens.
The problem is that:
recv(SocketFD, buffer, sizeof(buffer) - 1, 0)
is catched in an endless loop (or something like that).
If I kill the Java-client process or I type something like:
pw.print(msg);
out.close();
the output on the server side is:
_sending reply: "Request unrecognized/invalid" request="Hello, world!"
send reply success
now close connection
This output is right (except "send reply success"), but in case of adding:
out.close();
the client can't receive the reply of the server. Because the Socket is closed.
java.net.SocketException: Socket is closed
at java.net.Socket.getInputStream(Socket.java:864)
at MyServer.writeMessage(MyServer.java:56)
at MyServer.test(MyServer.java:42)
at MyServer.main(MyServer.java:30)
Edit
I tried to call pw.flush(); and different delimiters like "\n", "\r", "\r\n" and "\n\r" but the server still thinks there is still something to read. I also tried to use DatagramSockets:
java.net.DatagramSocket dSocket = new java.net.DatagramSocket();
InetAddress address = InetAddress.getByName("localhost");
String msg = "Hello, world!";
byte[] buf = msg.getBytes();
java.net.DatagramPacket packet = new DatagramPacket(buf, buf.length, address, 6686);
But the server can't accept the packet.
Solution
The ruby-client does something like a socket.shutdownOutput(); (ruby: socket.shutdown 1) after the call of puts. I changed the java-client-code:
out.write(msg);
socket.shutdownOutput();
and it works!
As #Charly said: I have to define a "protocol". In my case I'm not allowed to change any communication related code (in the server and the ruby-client) because this functionality is used by a another group of researchers. So I've to modify my java-client in that way, that it does the exact same things at the exact same time as the ruby-client (something like a protocol).
PrintWriter buffer (when autoflush is true) is only flushed by calling println or printf. Calling print may not flush the buffer (Javadoc). Try calling println or use a OutputStreamWriter directly and flush().
Be aware of using the right charset (You can set it up in OutputStreamWriter constructor).
Close the stream respectively flush it in a way like this:
DataOutputStream dataOut = new DataOutputStream(socket.getOutputStream());
dataOut.writeUTF(s);
dataOut.flush();
while ((recv_count = recv(SocketFD, buffer, sizeof(buffer) - 1, 0)) > 0) {
if (recv_count == -1) {
I don't know what your problem is but this code is certainly nonsense. It is impossible for the inner test ever to succeed.
Hey everyone, I'm having a bit of a problem with UDP and Datagrams. I'm supposed to make a server that will get a request from the client to send a file in the same directory. The UDP Server will then get this file (a video), put it into a datagram and send it. I think I know how to do it, but I can't put the file in the datagram. I'm putting it in Binary form, so keep that in mind.
Here's my code so far:
edit: This is the server by the way, and I keep having trouble with BufferedInputReader and OutputReader, so keep that in mind :)
Scanner inFromUser = new Scanner(System.in);
int port = 12345;
DatagramSocket server = new DatagramSocket(port);
// Read name of file supplied by client (must be a line of text):
Scanner in = new Scanner(new DataInputStream(server.getInputStream()));
String filename = in.nextLine();
DatagramSocket request = server.accept();
// Create buffer, then we're ready to go:
// Puts file into binary form
BufferedInputStream inbinary =
new BufferedInputStream(new FileInputStream("poop.txt"));
// Outputs the binary form
BufferedOutputStream outbinary =
new BufferedOutputStream(request.getOutputStream());
int numbytes;
int countblocks = 0;
int countbytes = 0;
byte[] buf = new byte[1024];
DatagramPacket packet = new DatagramPacket(buf, buf.length, port);
server.receive(packet);
while ((numbytes = inbinary.read(buf,0,1024)) >= 0)
{
// receive packet from client, telling it to send the video file
server.receive(packet);
InetAddress address = packet.getAddress();
packet = new DatagramPacket(buf, buf.length, address, port);
server.send(packet);
countblocks++; // keep statistics on file size
countbytes += numbytes;
outbinary.write(buf,0,numbytes); // write buffer to socket
}
outbinary.flush(); // FLUSH THE BUFFER
server.close(); // done with the socket
System.out.println(countblocks + " were read; " + countbytes + " bytes");
}
}
I haven't done datagrams in a while, but I'm pretty sure the accept() call is wrong. That's for TCP servers.
I'd recommend cribbing from Sun's excellent tutorial: http://java.sun.com/docs/books/tutorial/networking/datagrams/clientServer.html