I am writing a web server by NIO. I am kind of completed the implementation but in the mean while i have one doubt on the following code snippet:
private void readDataFromSocket(SocketChannel socketChannel) throws IOException {
BUFFER.clear();
int count = 0;
while ((count = socketChannel.read(BUFFER)) > 0) {
BUFFER.flip();
int limit = BUFFER.limit();
while (limit > 0) {
System.out.print((char) BUFFER.get());
limit--;
}
System.out.println("");
}
if (count < 0) {
System.out.println("closing the socket!!!");
socketChannel.close();
}
}
My assumptions is if the client closes the socket then the read (socketChannel.read(BUFFER)) will give me -1. Is my assumption correct?
2nd question:
suppose client sent me :
send: abcdef
send: adnvjvnfkjnvkj
than it was sending "sfnskdlfnsdknfsdkj" some network error happened. so what the server will get?
My assumption is server will get :
abcdef
adnvjvnfkjnvkj
than the server will get an error. Am i correct or server may get half data e.g. sfnskdlfn than error? Thanks.
Correct. If the peer closes the connection, read() returns -1.
It's impossible to say. The data could get buffered at the sender in any way, or sent a byte at a time, or anything in between. The boundaries between what was written by each write() are not preserved. You could get nothing, or everything, or anything in between.
Question 1: according to the SocketChannel documentation you'll get a ClosedChannelException.
Question 2: your server will buffer data right until the error happens and if it happens in the middle of the client writing, then it will contain half the data. This is assuming there is no buffering in the client before sending data, but in the end it doesn't matter, you shouldn't rely on the server getting either "complete" or "no data", it can get incomplete data.
Related
I'm making an Android app for playing online radio. I'm using Java Socket for connection and it's InputStream to read stream data. Actually there is HTTP request and response without length with content type such as audio/mpeg.
All works fine until bandwidth becomes low (using mobile internet or connected to slow/very remote server) - InputStream may return EOF (-1) at arbitrary time.
Please could you explain the nature of such behavior and possible ways to deal with it. I set read timeout (Socket.setSoTimeout), so why it doesn't throw timeout exception if the case is low bandwidth? Maybe there are another socket options that may help?
That happens when you have Client connected and then It disconnects and you are trying to read from it. I handle this asking the length of the message, If it is negative, I restart the connection.
E.G.
//CLIENTS LOOP
while (true)
{
...
//MESSAGE LOOP
while ( true )
{
System.out.println(" -> WAITING FOR INCOMING MESSAGE.. ");
len = inputStream.read(buffer_msg);
if ( len < 0 )
break;
...
I'm developing a client (Java)/server(C++) application using TCP sockets.
The protocol I used is composed of Messages beginning by 2 bytes defining the type of what will be the content of the Message.
So basically, the receiving thread waits for data to be received in a loop. But I want to use a timeout with the socket to be notified that the other host takes too long to send data.
receivingSocket.setSoTimeout(durationInMilliseconds);
DataInputStream in = new DataInputStream(receivingSocket.getInputStream());
boolean success = false;
short value = 0;
do {
try {
value = in.readShort();// will throw a SocketTimeoutException in case of timeout, without 2 bytes available from the socket
success = true;
} catch (SocketTimeoutException e) {
/// do something if it happens to often. Otherwise go on with the loop
}
} catch (IOException e) {
/// abort connection in case of other problem
}
} while (!success)
Now, what happens if the receiving thread calls in.readShort() at a point where the socket has got only one byte available in its buffer ? Does this byte remain on the socket's stack ? Or is it lost ? In the first case, I could read it next time I call in.readShort(), otherwise it seems lost for good...
readShort() here is an example, my question stands also for readInt(), ...
Thanks for your help,
It isn't specified. I believe the way the implementation works is that the half data is lost, but in any case there's nothing written that says anything else, so you just have to assume the worst.
However in practice this is very unlikely to happen, provided you observe common sense at the sender.
I have developed a client-server chat using the Sockets and it works great, but when I try to transmit data with Deflate compression it doesn't work: the output is "empty" (actually it's not empty, but I'll explain below).
The compression/decompression part is 100% working (I have already tested it), so the problem must be elsewhere in the transmission/receiving part.
I send the message from the client to the server using these methods:
// streamOut is an instance of DataOutputStream
// message is a String
if (zip) { // zip is a boolean variable: true means that compression is active
streamOut.write(Zip.compress(message)); // Zip.compress(String) returns a byte[] array of the compressed "message"
} else {
// if compression isn't active, the client sends the not compressed message to the server (and this works great)
streamOut.writeUTF(message);
}
streamOut.flush();
And I receive the message from the client to the server using these other methods:
// streamIn is an instace of DataInputStream
if (server.zip) { // same as before: true = compression is active
ByteArrayOutputStream bos = new ByteArrayOutputStream();
byte[] buf = new byte[512];
int n;
while ((n = streamIn.read(buf)) > 0) {
bos.write(buf, 0, n);
}
byte[] output = bos.toByteArray();
System.out.println("output: " + Zip.decompress(output)); // Zip.decompress(byte[]) returns a String of decompressed byte[] array received
} else {
System.out.println("output: " + streamIn.readUTF()); // this works great
}
Debugging a little bit my program, I've discovered that the while loop never ends, so:
byte[] output = bos.toByteArray();
System.out.println("output: " + Zip.decompress(output));
is never called.
If I put those 2 lines of code in the while loop (after bos.write()), then all works fine (it prints the message sent from the client)! But I don't think that's the solution, because the byte[] array received may vary in size. Because of this I assumed that the problem is in the receiving part (the client is actually able to send data).
So my problem became the while loop in the receiving part. I tried with:
while ((n = streamIn.read(buf)) != -1) {
and even with the condition != 0, but it's the same as before: the loop never ends, so the output part is never called.
-1 is only returned when the socket is closed or broken. You could close the socket after sending your zipped content, and your code would start working. But I suspect you want to keep the socket open for more (future) chat messages. So you need some other way of letting the client know when a discrete message has been fully transmitted. Like Patrick suggested, you could transmit the message length before each zipped payload.
You might be able to leverage something in the deflate format itself, though. I think it has a last-block-in-stream marker. If you're using java.util.zip.Inflater have a look at Inflater.finished().
The read function will not return a -1 until the stream is closed. What you can do is calculate the number of bytes that should be sent from the server to the client, and then read that number of bytes on the client side.
Calculating the number of bytes is as easy as sending the length of the byte array returned from the Zip.compress function before the actual message, and then use the readInt function to get that number.
Using this algorithm makes sure that you read the correct number of bytes before decompressing, so even if the client actually reads 0 bytes it will continue to read until it receives all bytes it wants. You can do a streamIn.read(buf, 0, Math.min(bytesLeft, buf.length)) to only read as many bytes you want.
Your problem is the way you are working with stream. You must send some meta-data so your client know what to expect as data. Idealy you are creating a protocol/state machine to read the stream. For your example, as a quick and dirt solution, send something like data size or a termination sequence or something.
Example of solution:
Server: send the "data size" before the compressed data
Client: wait for the "data size" bytes. Now loop till read is equal or greater "data size" value. Something like:
while( streamIn.ready() && dataRead < dataExpected)
{
dataRead += streamIn.read(buf);
}
Of course you need to read the dataExpected before, with a similar code.
Tip: You could also use UDP if you dont mind having the possibility to lose data. Its easier to program with datagrams...
Im working on Instant messenger using java 1.6. IM uses multithreading - main thread, receiving, and ping. For tcp/ip communication I used SocketChannel. And it seems there is a problem with receiving bigger packages from server. Server instead of one sends a couple of packages and thats where the problem begins. Every first 8 bytes is telling what is the type of package and how big it is. This is how I managed reading:
public void run(){
while(true){
try{
Headbuffer.clear();
bytes = readChannel.read(Headbuffer); //ReadableByteChannel
Headbuffer.flip();
if(bytes != -1){
int head = Headbuffer.getInt();
int size = Headbuffer.getInt();
System.out.println("received pkg: 0x" + Integer.toHexString(head)+" with size "+ size+" bytes);
switch(head){
case incoming.Pkg1: ReadWelcome(); break;
case incoming.Pkg2: ReadLoginFail();break;
case incoming.Pkg3: ReadLoginOk();break;
case incoming.Pkg4: ReadUserList();break;
case incoming.Pkg5: ReadUserData();break;
case incoming.Pkg6: ReadMessage();break;
case incoming.Pkg7: ReadTypingNotify();break;
case incoming.Pkg8: ReadListStatus();break;
case incoming.Pkg9: ChangeStatus();break;
}
}
}catch(Exception e){
e.printStackTrace();
}
}
}
And during the tests everything was fine until i logged on my account and import my buddylist. I send request to server for statuses and he send me back about 10 out of 80 contacts. So I came up with something like this:
public synchronized void readInStatus(ByteBuffer headBuffer){
byteArray.add(headBuffer); //Store every buffer in ArrayList
int buddies = MainController.controler.getContacts().getSize();
while(buddies>0){
readStuff();
readDescription();
--buddies;
}
}
and each readStuff() and readDescription() are checking each parameter size with remaining bytes in the buffer:
if(byteArray.get(current).remaining() >= 4){
uin = byteArray.get(current).getInt();
}else{
byteArray.add(Receiver.receiver.read());
current = current +1;
uin = byteArray.get(current).getInt();
}
and Receiver.receiver.read() is:
public ByteBuffer read(){
try {
ByteBuffer bb = ByteBuffer.allocate(40000);
bb.order(ByteOrder.LITTLE_ENDIAN);
bytes = readChannel.read(bb);
bb.flip();
return bb;
} catch (Exception e) {
e.printStackTrace();
}
return null;
}
So application is lunched, logged and then sends contacts. Server send me back just a piece of my list. But in the method readInStatus(ByteBuffer headBuffer) I try to force the rest of the list. And now the fun part - after some time it gets to the Receiver.receiver.read() and on bytes = readChannel.read(bb) it just stops and I dont know why , no errors no nothing even after some time and Im out of the ideas. Im fighting with this whole week and i dont get anywhere near the solution. I will appreciate any suggestions. Thanks.
Thanks for response. Yes, I'm using blocking SocketChannel, I tried non-blocking but it goes wild and out of control so I skipped the idea. About the bytes I expect - this is kind of weird, because its giving me size only once in head but its size of the first part not the whole package, other parts is not containing header bytes at all. I can't predict how much bytes it would be, the reason is - descriptions with 255 bytes capacity. This is exactly why I've created variable buddies in : public synchronized void readInStatus(ByteBuffer headBuffer)
wich is basically length of my buddy list and before reading each field I'm checking if there is enough bytes left if its not, I do read(). But last field before description is integer with the length of the incoming description. But its impossible to determine how long package is, until some processing is done. #robert do you think I should try again switching to non-blocking SocketChannel in that situation ?
The problem is most likely that you are sending fewer bytes than you are trying to read. You might have missed writing something, written things in the wrong order, misread a size field or something like that.
I think I'd attack this problem by adding tracing code to count and log the number of bytes read and written, notional packect sizes and so on. Then run, and compare the traces to see where things start to get out of sync.
If you are using a blocking SocketChannel, then read will block until the buffer is filled or the server delivers end of stream. For a server with connection keep-alive, the server does not send end of stream - it will simply stop sending data, and the read will hang indefinitely or until timeout.
You could either:
(i) try using a non-blocking SocketChannel, repeatedly reading until the read delivers 0 bytes (but beware 0 bytes does not necessarily mean end of stream - it could mean an interruption) or
(ii) if you have to use the blocking version, and you know how many bytes you were expecting from the server e.g. from a header, when the number of bytes left to read is less than buffer.capacity(), move position and/or limit on the buffer so as to leave only the required space in the buffer before the read. I am working this solution now. If it works for you, please let me know!
So far as I can work out, if you have to use a blocking SocketChannel and you do not know how many bytes you are expecting, and the server does not send end of stream, there is no solution.
I am working in the communication between two programs, one in C++ and the other in Java, connected using standard sockets. Both programs run under UNIX (Ubuntu) within the same machine and share information for a period of time in regular basis. At some point of the execution and always at the same point it gets stuck since the program in C++ sends the information and the one in Java does not get all the info so they block since the first expect to receive that and the second is not sending anything because did not receive the information in the firs place.
The odd thing is one you execute the program in Java under Windows. Then it works fine, the program ends correctly without any blocing.
I think is an issue with the Java application but why the difference between running the under Ubuntu or Windows? Do the socket behave differently? Is some parameter different from the JVM in Ubuntu and Windows?
Thank you very much in advance!
Julen.
EDIT:
This the code in the Java side that reads the buffer:
if (task.equals("receiving")){
try {
BufferedReader in = new BufferedReader(new InputStreamReader(socket.getInputStream()));
out = socket.getOutputStream();
// receive messages
char[] length = new char[5];
while (!socket.isClosed()&&(!socket.isInputShutdown())){
in.read(length,0,5);
// this way of reading the length implies that only one command
// at a time can be received and interpreted, so far the iCS does not
// concatenate more commands in one transmission
int commandLength = length[4];
System.err.println("Speed Advice --> command received with length "+ commandLength);
char[] command = new char[commandLength - 1];
in.read(command,0,commandLength - 1);
/*if (cow){
System.err.println("Speed Advice --> Last byte received for X-pos is "+(int)command[commandLength-1]);
}*/
readCommand(command);
}
System.err.println("Speed Advice --> Socket was externally closed.");
in.close();
closeConnection();
}catch (Exception e) {
e.printStackTrace();
}
}
And this C++ sending information:
void
Socket::
send( std::vector<unsigned char> b)
throw( SocketException )
{
if( socket_ < 0 ) return;
size_t numbytes = b.size();
unsigned char *const buf = new unsigned char[numbytes];
for(size_t i = 0; i < numbytes; ++i)
{
buf[i] = b[i];
}
if (verbose_)
{
cerr << "Send " << numbytes << " bytes via tcpip::Socket: [";
for(size_t i = 0; i < numbytes; ++i)
{
buf[i] = b[i];
cerr << " " << (int)b[i] << " ";
}
cerr << "]" << endl;
}
unsigned char const *buf_ptr = buf;
while( numbytes > 0 )
{
#ifdef WIN32
int n = ::send( socket_, (const char*)buf_ptr, static_cast<int>(numbytes), 0 );
#else
int n = ::send( socket_, buf_ptr, numbytes, 0 );
#endif
if( n<0 )
{
// BailOnSocketError definitely throws an exception so clear up heap
delete[] buf;
BailOnSocketError( "send failed" );
}
numbytes -= n;
buf_ptr += n;
}
delete[] buf;
}
The BufferedReader is not the problem specifically, and neither is 'the TCP stack [being] full'. And select() isn't required to solve it.
You are making several common mistakes here.
You are ignoring the return value of read(). It could be -1, indicating that the peer has closed the connection, which you must test for and act on first. Or it could be any value between 1 and the size you requested. You are just blindly assuming you will get 5 bytes when you call in.read(buffer,0,5). There's no guarantee of that. You need to use DataInputStream.readFully().
You are sending 8-bit chars from C++, and using a Reader in Java. This doesn't make any sense whatsoever. Use an InputStream. Specifically, a DataInputStream so you can call readFully() as suggested above.
Socket.isClosed() tells you whether you have closed the socket. It doesn't tell you whether the peer has closed the connection. That's what the -1 above is for.
Similarly Socket.isInputShutdown() tells you whether you have shutdown input on this socket. It doesn't tell you whether the peer has shutdown output at his end. Again, that's what the -1 above is for.
So both those tests are pointless, and the message you are printing when either of them is true is incorrect.
Inspect the communication with a sniffer. Wireshark is a good one.
While BufferedReader is great for file operations, it strikes me as a really bad idea to use one while reading from a Socket.
Otherwise, it might be that the BufferedReader thinks that data is still coming into the buffer when it really isn't.
I recommend removing it and dealing with the InputStreamReader directly to see if the problem still occurs.
Note that big chunks of data can be divided into several packages when sending over sockets. I wrote an answer here regarding sockets in C++, that addresses it shortly.
I see three possibilities:
Your tcp stack could be full. Thats a real bugger. You can read more about that in this article, but in short: the socket is full in some end of your socket, making the sender block more writes. You will have to read that data from the receiving side before send will unblock. You can check for "TCP ZeroWindow" with wireshark to find this. (default settings will be black with red text, you can't miss it)
It could be your sending side that is waiting for you to push more data to send
I don't think this is valid, but I guess that the receiving side could as well wait for more data to receive. As I understand it it is really up to the underlying network to decide if the data should be sent or not.
You can use select (sorry, don't know of the unix equivalent arguments, or java's) when sending data to check if you are ok to actually send (also described in my answer here).