using this code:
Java Server side:
...
out = new PrintWriter(this.client.getOutputStream(), true);
...
public void sendMsg(String msg) {
out.println(msg);
//out.flush(); // we don't flush manually because there is auto flush true
}
C# Client side:
while(connected) {
int lData = myStream.Read(myBuffer, 0, client.ReceiveBufferSize);
String myString = Encoding.UTF8.GetString(myBuffer);
myString = myString.Substring(0, lData);
myString = myString.Substring(0, myString.Length-2);
addToQueue(myString);
}
variable myString have many messages that server should send them one by one like
hello \r\t hello \r\t ...
they should come separately like
hello \r\t
hello \r\t ...
which means when i wait one by one they come instantly all of them in a row, how can i make it to send one by one in separate flush.
Note I send 30~ messages in a row in one second (1s), i want them separate.
TCP supports a stream of bytes. This means you have no control how the data arrives regardless of how you send it. (Other than it will comes as bytes) You should rethink your protocol if you depend on it coming in any particular manner.
You can reduce the amount of bunching of data but all this does is reduce latency at the cost of throughput and should never be relied upon. This can be reduce (but not eliminated) by turning off Nagle and reducing co-alessing setting in your TCP driver if you can change these.
i want them separate.
You can want it but TCP does not support messages as you would want them.
The solution in you case is for your reader to match your writers protocol. You send lines so you should read lines at a time, e.g. BufferedReader.readLine(), not blocks of whatever data happens to be in the buffer.
Related
I want to build a client-server-application for some practice. I started off with a simple chat which was not so hard to do. I'm also able to identify different commands by just simply split a String, e.g. "command:msg". But I think that may be a little inconvenient. So I'm wondering if there are better ways to realize that. And I stumbled over that side:
http://www.javaworld.com/jw-01-1997/jw-01-chat.html?page=6
At the very end it says:
An alternative, however, and much more elegant solution, is to abstract the protocol behind a set of stream classes. The specifics of header construction and insertion can be handled automatically by the stream classes, and the client is then left with much the same interface as before: Clients write messages to a stream, but instead of flushing the stream, they call a method that attaches appropriate headers and sends the encapsulated message.
I don't really know what is meant by that. Could somebody explain it, or even better, give me a code example. Perhaps there may are other ways to do?
Let's say you want to send messages encapsulated as the link you sent:
| ID | len | message contents |.
What they mean with "to abstract the protocol behind a set of stream classes" is to create classes that extend stream classes which will put the correct ID and length on the encapsulated message for you.
For example, for an extended PrintWriter where you send two kinds of message:
ID 1 - normal message.
ID 2 - error message.
class MyProtocolPrintWriter extends PrintWriter {
public static final int NORMAL_MESSAGE = 1;
public static final int ERROR_MESSAGE = 2;
//put the constructor
public void writeMessage(String message) {
println(
String.format(
"%02x%02d%s", NORMAL_MESSAGE, message.length(), message));
}
public void writeErrorMessage(String message) {
println(
String.format(
"%02x%02d%s", ERROR_MESSAGE, message.length(), message));
}
}
Here's what one fairly successful chat network used for a protocol.
Internet Relay Chat
And here's a list of the commands that were implemented using the IRC protocol.
List of Internet Relay Chat commands
You would implement these commands as a set of stream classes. The user issues a command, and your stream class handles the specifics of the header construction and insertion into the stream.
The application that I am working on has two parts. The server part runs on a Linux machine. The client part, an Android application, queries the server and gets necessary response. Both the parts are written in Java, use socket-based communication, and transfer textual data.
Right after sending the request, here is how the client receives the response:
public static String ReadAvailableTextFromSocket(BufferedReader input) throws IOException {
if (input.ready() == false) {
return null;
}
StringBuilder retVal = new StringBuilder();
while(input.ready()) {
char ch = (char) input.read();
retVal.append(ch);
}
return retVal.toString();
}
However, this doesn't seem to be that reliable. The input is not always ready because of server response time or transmission delays.
Looks like input.ready() is not the right way to wait for getting data.
I am wondering if there is a better way to accomplish this. Perhaps there is some standard practice that I could use.
Perhaps you should use Threads. Keep a listener thread in a while(true) loop that reads more data as it comes in, and simply buffers the data in a data structure (let's say a queue) shared with the main thread. That way, the main thread could simply dequeue data as needed. If the queue is empty, it can infer that no new data was received.
Edit: see this multithreaded chat server/client code as an example.
Here is how I solved this problem. As I am responsible for writing both, the client side as well as the server side, when a request comes to the server, the first line of information I send as the response is the number of bytes the client can expect. This way, the client first waits to read a line. Once the line is read, the client now knows how many bytes of data to expect from the server.
Hope this helps others.
Regards,Peter
I'm having a socket problem. This problem occurs when I'm running the server and client on the same PC i.e. using "localhost" parameter. But problem is not seen when different PCs are being used.
Client sends a file with these codes:
output_local.write(buffer, 0, bytesRead);
output_local.flush();
And after that in another method I'm sending a command with these:
outputStream.write(string);
outputStream.flush();
Server appends the command to the end of the file. So it thinks it hasn't received the command from the client yet. do you have an idea what might causing this problem? How can I solve the defect? below is the file receive method at the server:
while (true) {
try {
bytesReceived = input.read(buffer);
} catch (IOException ex) {
Logger.getLogger(Server.class.getName()).log(Level.SEVERE, null, ex);
System.out.println("exception occured");
break;
}
System.out.println("received:" + bytesReceived);
try {
/* Write to the file */
wr.write(buffer, 0, bytesReceived);
} catch (IOException ex) {
Logger.getLogger(Server.class.getName()).log(Level.SEVERE, null, ex);
}
total_byte = total_byte + bytesReceived;
if (total_byte >= filesizeInt) {
break;
}
}
If you want message-like support, you need a create a protocol to clarify what you're going to send and receive.
In TCP, you can't rely on separate "packets" being received separately (e.g., sending 4 chunks of 10 bytes may be received as 1 chunk of 40, or of 2 chunks of 20, or one chunk of 39 and one chunk of 1). TCP guarantees in order delivery, but not any particular 'packetization' of your data.
So, for example, if you're sending a string you need to first send the string length then its bytes. The logic in pseudocode would be something like:
Client:
Send the command indicator
Send the payload length
Send the payload
Server:
Read the command indicator
Read the payload length
Loop reading payload until the complete length has been read
The defect is that you're treating a stream-based protocol (TCP) as if it were a message-oriented protocol. It's not. You should assume that this can happen.
If you need to break your stream into individual messages, you should use either delimiters or (preferably IMO) a length prefix for each message. You should also then anticipate that any read you issue may not receive as much data as you've asked for - in other words, not only can messages be combined if you're not careful, but they can easily be split.
I mentioned that I prefer length-prefixing to delimiters. Pros and cons:
The benefit of using a message delimiter is that you don't need to know the message size before you start sending.
The benefits of using a length prefix are:
The code for reading the message doesn't need to care about the data within the message at all - it only needs to know how long it is. You read the message length, you read the message data (looping round until you've read it all) and then you pass the message on for process. Simple.
You don't need to worry about "escaping" the delimiter if you want it to appear within a normal message.
As TCP is a stream oriented connection, this behaviour is normal if the writer writes faster than the reader reads, or than the TCP stack sends packets.
You should add a separator to separate the parts of the streams, e.g. by using a length field for sub packets, or by using separators such as newline (\n, char code 10).
Another option could be to use UDP (or even SCTP), but that depends on the task to be fulfilled.
I have a Java Client which sends UTF-8 strings to a C# TCP-Server, I'm using a DataOutputStream to send the strings. The code looks like this:
public void sendUTF8String(String ar) {
if (socket.isConnected()) {
try {
dataOutputStream.write(ar.getBytes(Charset.forName("UTF-8")));
dataOutputStream.flush();
} catch (IOException e) {
handleException(e);
}
}
}
The problem is that flush doesn't seem to work right. If I send two Strings close to each other, the server receives only one message with both strings. The whole thing works if I do a Thread.sleep(1000) between calls, this is obviously not a solution.
What am I missing?
flush() doesn't guarantee that a data packet gets shipped off. Your TCP/IP stack is free to bundle your data for maximum efficiency. Worse, there are probably a bunch of other TCP/IP stacks between you and your destination, and they are free to do the same.
I think you shouldn't rely on packet bundling. Insert a logical terminator/divider in your data and you will be on the safe side.
You shouldn't worry about how the data is broken up into packets.
You should include the length of the string in your messages, and then on the receiving end you would read the length first. So for example to send you would do
byte[] arbytes = ar.getBytes(Charset.forName("UTF-8"));
output.writeInt(arbytes.length)
output.write(arbytes)
and then in your reader you do
byte[] arbytes = new byte[input.readInt()];
for(int i = 0; i < len; i++){
arbytes[i] = input.read();
}
//convert bytes back to string.
You can't just call input.read(arbytes) because the read function doesn't necessarily read the entire length of the array. You can do a loop where you read a chunk at a time but the code for that is a bit more complex.
Anyway, you get the idea.
Also, if you really want to control what goes in what packets, you can use Datagram Sockets, but if you do that then delivery of the packet is not guaranteed.
Socket send a stream of data, not messages.
You shouldn't rely on the packets you receive being the same size as they are sent.
Packets can be grouped together as you have seen but they can also be broken up.
Use #Chad Okere's suggestion on how to ensure you get blocks the same was they are sent.
However in your case, you can just use
dataOutputStream.writeUTF(ar); // sends a string as UTF-8
and
String text = dataInputStream.readUTF(); // reads a string as UTF-8
Has anyone out there done the work of sitting on top of a packet capture interface (like jpcap) with an implementation of UDPSocket (for UDP datagrams) and InputStream (for TCP streams)?
I suppose it wouldn't be too hard to do given the callback API in jpcap, but has anyone out there already done it? Are there any issues with doing this (do I have to figure out how to reassemble a TCP stream myself, for example?)
I have not done this particular thing, but I do do a lot of work with parsing captured packets in C/C++. I don't know if there exist Java libraries for any of this.
Essentially, you need to work your way up the protocol stack, starting with IP. The pcap data starts with the link-level header, but I don't think there's much in it that you're concerned about, other than ignoring non-IP packets.
The trickiest thing with IP is reassembling fragmented datagrams. This is done using the More Fragments bit in the Flags field and the Fragment Offset field, combined with the Identification field to distinguish fragments from different datagrams Then you use the Protocol field to identify TCP and UDP packets, and the Header Length field to find the start of the corresponding header.
The next step, for both TCP and UDP, is demultiplexing, separating out the various connections in the captured packet stream. Both protocols identify connections (well, UDP doesn't have connections per se, but I don't have a better word handy) by the 4-tuple of the source and destination IP address and the source and destination port, so a connection would be a sequence of packets that matches on all 4 of these values.
Once that's done, for UDP, you're just about finished, unless you want to check the checksum. The Length field in the UDP header tells you how long the packet is; subtract 8 bytes for the header and there's your data.
TCP is somewhat more complicated, as you do indeed have to reassemble the stream, This is done using the sequence number in the header, combined with the length. The sum of these two tells you the next sequence number in the stream. Remember that you're keeping track of the traffic in two directions.
(This is a lot easier than writing an actual TCP implementation, as then you have to implement the Nagle algorithm and other minutiae.)
There's a lot of information on the net about the header formats; google "IP header" for starters. A network analyzer like Wireshark is indispensable for this work, as it will show you how your captured data is supposed to look. Indeed, as Wireshark is open source, you can probably find out a lot by looking at how it does things
Tcp reassembly can be done with JNetPcap. Here is a complete example:
final String SOME_PORT = 8888;
StringBuilder errbuf = new StringBuilder();
Pcap pcap = Pcap.openOffline("/dir/someFile.pcap", errbuf); //Can be replace with .openLive(...)
if (pcap == null) {
System.err.printf("Error: "+errbuf.toString());
return;
}
//Handler that receive Tcp Event one by one
AnalyzerListener<TcpStreamEvent> handler = new AnalyzerListener<TcpStreamEvent>() {
#Override
public void processAnalyzerEvent(TcpStreamEvent evt) {
JPacket packet = evt.getPacket();
Tcp tcp = new Tcp();
if (packet.hasHeader(tcp)) {
//Limiting the analysis to a specific protocol
if (tcp.destination() == SOME_PORT || tcp.source() == SOME_PORT) {
String data = new String(tcp.getPayload());
System.out.println("Capture data:{"+data+"}");
}
}
}
};
TcpAnalyzer tcpAnalyzer = JRegistry.getAnalyzer(TcpAnalyzer.class);
tcpAnalyzer.addTcpStreamListener(handler, null);
//Starting the capture
pcap.loop(Pcap.LOOP_INFINATE, JRegistry.getAnalyzer(JController.class), null);