I have one network app and have one puzzle:
If I send data(socket.getOutputStream.write()) for many times without call socket.getInputStream().read()?
after minutes.
can socket.getInputStream().read() read all data for the all sent data?
If can, if over buffer occurred if sent data too huge for minutes or hours?
Yes. Either anything you write to the socket will be read, or the connection will be terminated. If you don't get an error, then you will always read everything you wrote.
If you fill up whatever buffer space is available, then the sender's write call will wait until there's more buffer space. It will not raise an error.
Yes. As long as the socket is still open, because TCP sockets provide reliable transmission.
In practice, the socket might be forced closed. But yes, forcing the server to use a lot of memory buffers is one common vector in a DDOS attack.
Ye, but if you never read from the socket, the sender might block, which might prevent it from reading, which might block your writes.
It isn't a good idea. If the peer is sending responses, read them as the application protocol requires.
Related
For an exercise, we are to implement a server that has a thread that listens for connections, accepts them and throws the socket into a BlockingQueue. A set of worker threads in a pool then goes through the queue and processes the requests coming in through the sockets.
Each client connects to the server, sends a large number of requests (waiting for the response before sending the next request) and eventually disconnects when done.
My current approach is to have each worker thread waiting on the queue, getting a socket, then processing one request, and finally putting the (still open) socket back into the queue before handling another request, potentially from a different client. There are many more clients than there are worker threads, so many connections queue up.
The problem with this approach: A thread will be blocked by a client even if the client doesn't send anything. Possible pseudo-solutions, all not satisfactory:
Call available() on the inputStream and put the connection back into the queue if it returns 0. The problem: It's impossible to detect if the client is still connected.
As above but use socket.isClosed() or socket.isConnected() to figure out if the client is still connected. The problem: Both methods don't detect a client hangup, as described nicely by EJP in Java socket API: How to tell if a connection has been closed?
Probe if the client is still there by reading from or writing to it. The problem: Reading blocks (i.e. back to the original situation where an inactive client blocks the queue) and writing actually sends something to the client, making the tests fail.
Is there a way to solve this problem? I.e. is it possible to distinguish a disconnected client from a passive client without blocking or sending something?
Short answer: no. For a longer answer, refer to the one by EJP.
Which is why you probably shouldn't put the socket back on the queue at all, but rather handle all the requests from the socket, then close it. Passing the connection to different worker threads to handle requests separately won't give you any advantage.
If you have badly behaving clients you can use a read timeout on the socket, so reading will block only until the timeout occurs. Then you can close that socket, because your server doesn't have time to cater to clients that don't behave nicely.
Is there a way to solve this problem? I.e. is it possible to distinguish a disconnected client from a passive client without blocking or sending something?
Not really when using blocking IO.
You could look into the non-blocking (NIO) package, which deals with things a little differently.
In essence you have a socket which can be registered with a "selector". If you register sockets for "is data ready to be read" you can then determine which sockets to read from without having to poll individually.
Same sort of thing for writing.
Here is a tutorial on writing NIO servers
Turns out the problem is solvable with a few tricks. After long discussions with several people, I combined their ideas to get the job done in reasonnable time:
After creating the socket, configure it such that a blocking read will only block for a certain time, say 100ms: socket.setSoTimeout(100);
Additionally, record the timestamp of the last successful read of each connection, e.g. with System.currentTimeMillis()
In principle (see below for exception to this principle), run available() on the connection before reading. If this returns 0, put the connection back into the queue since there is nothing to read.
Exception to the above principle in which case available() is not used: If the timestamp is too old (say, more than 1 second), use read() to actually block on the connection. This will not take longer than the SoTimeout that you set above for the socket. If you get a TimeoutException, put the connection back into the queue. If you read -1, throw the connection away since it was closed by the remote end.
With this strategy, most read attempts terminate immediately, either returning some data or nothing beause they were skipped since there was nothing available(). If the other end closed its connection, we will detect this within one second since the timestamp of the last successful read is too old. In this case, we perform an actual read that will return -1 and the socket's isClosed() is updated accordingly. And in the case where the socket is still open but the queue is so long that we have more than a second of delay, it takes us aditionally 100ms to find out that the connection is still there but not ready.
EDIT: An enhancement of this is to change "last succesful read" to "last blocking read" and also update the timestamp when getting a TimeoutException.
No, the only way to discern an inactive client from a client that didn't shut down their socket properly is to send a ping or something to check if they're still there.
Possible solutions I can see is
Kick clients that haven't sent anything for a while. You would have to keep track of how long they've been quiet for, and once they reach a limit you assume they've disconnected .
Ping the client to see if they're still there. I know you asked for a way to do this without sending anything, but if this is really a problem, i.e you can't use the above solution, this is probably the best way to do it depending on the specifics(since it's an exercise you might have to imagine the specifics).
A mix of both, actually this is probably better. Keep track of how long they've been quiet for, after a bit send them a ping to see if they still live.
I have a DatagramSocket, and I know that socket.receive() has a buffer that stores packets that have been sent to the socket. I am trying to implement a reliable protocol using UDP, and it would be ideal if, after my window is full, I could unload the buffer all at once. If i did a simple loop, when the buffer is empty the socket.receive() would block my program, which I do not want.
Is there any way to access that buffer?
Sounds like a certain networks assignment due tonight...
There's no real way to unload all the packets at once. I would recommend instead setting the socket to timeout after a small period (smaller than the total timeout value the sender has), and looping through sending packets in your window and receiving acknowledgements.
No, but there is DatagramChannel in non-blocking mode.
DatagramSocket has no buffer. The network stack in your operating system has a buffer, but you cannot access it.
I am trying to implement a reliable protocol using UDP
I would advice not to do that - you'll end up inventing TCP, but with bugs of your own.
Trying to get how Java sockets operate. A question is: what can you do simultaneously if you are using socket Java API, and what happens if we send and read data with some delay?
READ & WRITE at once. If one socket-client connected to one spcket-server, can they BOTH read and write at the same time? As far as I understand, TCP protocol is full-duplex, so theoretically socket should be able to read and write at one, but we have to create two threads for bot client and server. Am I right?
WRITE to N clients at once. If several socket-clients connected to one socket-server, can server read several clients at one moment, can server write to several clients at one moment?
If maximum possible physical speed rate of NetworkCard is 1kbyte/sec and 5 clients are connected, which speed is it possible to write with to one client?
How can I implement sequential sending of data in both directions? I mean I want to send N bytes from server to client, then M bytes from client to server, then N from server to client etc. The problem is if any of the two sides has written something to the channel, the other side will stop reading that data (read() == -1) only if channel is closed, which means that we cannot reuse it and have to open another connection. Or, may be, we should place readers and writers to different threads which do their job with read() and write() until connection is closed?
Imagine we have a delay between calling write(); flush() on one side, and calling read() on the other side. During the delay - where the written data would be stored? Would it be transmitted? What is the max size of that "delayed" data to be stored somewhere "between"?
Correct. If you're using blocking I/O, you'll need a reader thread and a writer thread for each Socket connection.
You could use a single thread to write to N clients at once, but you run the risk of blocking on a write. I won't address the writing speeds here, as it would depend on several things, but obviously the cumulative writing speed to all clients would be under 1kbps.
Yes, you'll need 2 threads, you can't do this with a single thread (or you could, but as you said yourself, you'd need to constantly open and close connections).
It would be stored in a buffer somewhere. Depending on your code it could be in a Buffered stream, or the socket's own buffer. I believe the default buffer size of BufferedOutputStream is 8K, and the socket's own buffer would depend on the environment. It shouldn't really be of importance though, the streaming quality of TCP/IP removes the need to think about buffers unless you really need to do fine-tuning.
I have a client connecting to my server. The client sends some messages to the server which I do not care about and do not want to waste time parsing its messages if I'm not going to be using them. All the i/o I'm using is simple java i/o, not nio.
If I create the input stream and just never read from it, can that buffer fill up and cause problems? If so, is there something I can do or a property I can set to have it just throw away data that it sees?
Now what if the server doesn't create the input stream at all? Will that cause any problems on the client/sending side?
Please let me know.
Thanks,
jbu
When you accept a connection from a client, you get an InputStream. If you don't read from that stream, the client's data will buffer up. Eventually, the buffer will fill up and the client will block when it tries to write more data. If the client writes all of its data before reading a response from the server, you will end up with a pretty classic deadlock situation. If you really don't care about the data from the client, just read (or call skip) until EOF and drop the data. Alternatively, if it's not a standard request/response (like HTTP) protocol, fire up a new thread that continually reads the stream to keep it from getting backed up.
If you get no useful data from the client, what's the point of allowing it to connect?
I'm not sure of the implications of never reading from a buffer in Java -- I'd guess that eventually the OS would stop accepting data on that socket, but I'm not sure there.
Why don't you just call the skip method of your InputStream occasionally with a large number, to ensure that you discard the data?
InputStream in = ....
byte[] buffer = new byte[4096] // or whatever
while(true)
in.read(buffer);
if you accept the connection, you should read the data. to tell you the truth i have never seen (or could forsee) a situation where this (a server that ignores all data) could be useful.
I think you get the InputStream once you accept the request, so if you don't acknowledge that request the underlying framework (i.e. tomcat) will drop that request (after some lapsed time).
Regards.
In some circumstances I wish to send an error message from a server to client using non-blocking I/O (SocketChannel.write(ByteBuffer)) and then disconnect the client. Assuming I write the full contents of the message and then immediately disconnect I presume the client may not receive this message as I'm guessing that the OS hasn't actually sent the data at this point.
Is this correct, and if so is there a recommended approach to dealing with this situation?
I was thinking of using a timer whereby if I wish to disconnect a client I send a message and then close their connection after 1-2 seconds.
SocketChannel.write will in non-blocking mode return the number of bytes which could immediately be sent to the network without blocking. Your question makes me think that you expect the write method to consume the entire buffer and try asynchronously to send additional data to the network, but that is not how it's working.
If you really need to make sure that the error message is sent to the client before disconnecting the socket, I would simply enable blocking before calling the write method. Using non-blocking mode, you would have to call write in a loop, counting the number of bytes being sent by each invocation and exit the loop when you've succeeded to pass the entire message to the socket (bad solution, I know, unnecessary code, busy wait and so on).
you may be better off launching a thread and synchronously write data to the channel. the async api is more geared toward "one thread dispatching multiple channels" and not really intended for fire and forget communications.
The close() method of sockets makes sure, everything sent using write before is actually sent before the socket is really closed. However this assumes that your write() was able to copy all data to the tcp stacks output window, which will not always work. For solutions to this see the other answers.