Why would someone prefer blocking writes over non-blocking writes? My understanding is that you would only want blocking write if you want to make sure the other side got the TCP packet once the write method returned, but I am not even sure that's possible. You would have to flush and flush would have to flush the underlying operating system write socket buffer. So is there any disadvantage of non-blocking socket writes? Does having a large underlying write socket buffer a bad idea in terms of performance? My understanding is that the smaller the underlying socket write buffer the more likely you will hit slow/buggy client and have to drop/queue packets in the application level while the underlying socket buffer is full and isWritable() is returning false.
My understanding is that you would only want blocking write if you want to make sure the other side got the TCP packet once the write method returned
Your understanding is incorrect. It doesn't ensure that.
Blocking writes block until all the data has been transferred to the socket send buffer, from where it is transferred asynchronously to the network. If the reader is slow, his socket receive buffer will fill up, which will eventually cause your socket send buffer to fill up, which will cause a blocking write to block, blocking the whole thread. Non-blocking I/O gives you a way to detect and handle that situation.
The problem with non blocking writes is that you may not have anything useful to do if the write is incomplete. You can end up with loops like
// non-blocking write
while(bb.remaining() > 0) sc.write(bb);
OR
// blocking write
sc.write(bb);
The first can burn CPU and the second might be more desirable.
The big problem is reads. Once you decide whether you want blocking or non-blocking reads, your writes have to be the same. Unfortunately there is no way to make them different. If you want non-blocking reads, you have to have non-blocking writes.
Related
So, I've been brushing up my understanding of traditional Java non-blocking API. I'm a bit confused with a few aspects of the API that seem to force me to handle backpressure manually.
For example, the documentation on WritableByteChannel.write(ByteBuffer) says the following:
Unless otherwise specified, a write operation will return only after
writing all of the requested bytes. Some types of channels,
depending upon their state, may write only some of the bytes or
possibly none at all. A socket channel in non-blocking mode, for
example, cannot write any more bytes than are free in the socket's
output buffer.
Now, consider this example taken from Ron Hitchens book: Java NIO.
In the piece of code below, Ron is trying to demonstrate how we could implement an echo response in a non-blocking socket application (for context here's a gist with the full example).
//Use the same byte buffer for all channels. A single thread is
//servicing all the channels, so no danger of concurrent access.
private ByteBuffer buffer = ByteBuffer.allocateDirect(1024);
protected void readDataFromSocket(SelectionKey key) throws Exception {
var channel = (SocketChannel) key.channel();
buffer.clear(); //empty buffer
int count;
while((count = channel.read(buffer)) > 0) {
buffer.flip(); //make buffer readable
//Send data; don't assume it goes all at once
while(buffer.hasRemaining()) {
channel.write(buffer);
}
//WARNING: the above loop is evil. Because
//it's writing back to the same nonblocking
//channel it read the data from, this code
//can potentially spin in a busy loop. In real life
//you'd do something more useful than this.
buffer.clear(); //Empty buffer
}
if(count < 0) {
//Close channel on EOF, invalidates the key
channel.close();
}
}
My confusion is on the while loop writing into output channel stream:
//Send data; don't assume it goes all at once
while(buffer.hasRemaining()) {
channel.write(buffer);
}
It really confuses me how NIO is helping me here. Certainly the code may not be blocking as per the description of the WriteableByteChannel.write(ByteBuffer), because if the output channel cannot accept any more bytes because its buffer is full, this write operation does not block, it just writes nothing, returns, and the buffer remains unchanged. But --at least in this example-- there is no easy way to use the current thread in something more useful while we wait for the client to process those bytes. For all that matter, if I only had one thread, the other requests would be piling up in the selector while this while loop wastes precious cpu cycles “waiting” for the client buffer to open some space. There is no obvious way to register for readiness in the output channel. Or is there?
So, assuming that instead of an echo server I was trying to implement a response that needed to send a big number of bytes back to the client (e.g. a file download), and assuming that the client has a very low bandwidth or the output buffer is really small compared to the server buffer, the sending of this file could take a long time. It seems as if we need to use our precious cpu cycles attending other clients while our slow client is chewing our file download bytes.
If we have readiness in the input channel, but not on the output channel, it seems this thread could be using precious CPU cycles for nothing. It is not blocked, but it is as if it were since the thread is useless for undetermined periods of time doing insignificant CPU-bound work.
To deal with this, Hitchens' solution is to move this code to a new thread --which just moves the problem to another place--. Then I wonder, if we had to open a thread every time we need to process a long running request, how is Java NIO better than regular IO when it comes to processing this sort of requests?
It is not yet clear to me how I could use traditional Java NIO to deal with these scenarios. It is as if the promise of doing more with less resources would be broken in a case like this. What if I were implementing an HTTP server and I cannot know how long it would take to service a response to the client?
It appears as if this example is deeply flawed and a good design of the solution should consider listening for readiness on the output channel as well, e.g.:
registerChannel(selector, channel, SelectionKey.OP_WRITE);
But how would that solution look like? I’ve been trying to come up with that solution, but I don’t know how to achieve it appropriately.
I'm not looking for other frameworks like Netty, my intention is to understand the core Java APIs. I appreciate any insights anyone could share, any ideas on what is the proper way to deal with this back pressure scenario just using traditional Java NIO.
NIO's non-blocking mode enables a thread to request reading data from a channel, and only get what is currently available, or nothing at all, if no data is currently available. Rather than remain blocked until data becomes available for reading, the thread can go on with something else.
The same is true for non-blocking writing. A thread can request that some data be written to a channel, but not wait for it to be fully written. The thread can then go on and do something else in the meantime.
What threads spend their idle time on when not blocked in IO calls, is usually performing IO on other channels in the meantime. That is, a single thread can now manage multiple channels of input and output.
So I think you need to rely on the design of the solution by using a design pattern for handling this issue, maybe **Task or Strategy design pattern ** are good candidates and according to the framework or the application you are using you can decide the solution.
But in most cases you don't need to implement it your self as it's already implemented in Tomcat, Jetty etc.
Reference : Non blocking IO
I have one network app and have one puzzle:
If I send data(socket.getOutputStream.write()) for many times without call socket.getInputStream().read()?
after minutes.
can socket.getInputStream().read() read all data for the all sent data?
If can, if over buffer occurred if sent data too huge for minutes or hours?
Yes. Either anything you write to the socket will be read, or the connection will be terminated. If you don't get an error, then you will always read everything you wrote.
If you fill up whatever buffer space is available, then the sender's write call will wait until there's more buffer space. It will not raise an error.
Yes. As long as the socket is still open, because TCP sockets provide reliable transmission.
In practice, the socket might be forced closed. But yes, forcing the server to use a lot of memory buffers is one common vector in a DDOS attack.
Ye, but if you never read from the socket, the sender might block, which might prevent it from reading, which might block your writes.
It isn't a good idea. If the peer is sending responses, read them as the application protocol requires.
Trying to get how Java sockets operate. A question is: what can you do simultaneously if you are using socket Java API, and what happens if we send and read data with some delay?
READ & WRITE at once. If one socket-client connected to one spcket-server, can they BOTH read and write at the same time? As far as I understand, TCP protocol is full-duplex, so theoretically socket should be able to read and write at one, but we have to create two threads for bot client and server. Am I right?
WRITE to N clients at once. If several socket-clients connected to one socket-server, can server read several clients at one moment, can server write to several clients at one moment?
If maximum possible physical speed rate of NetworkCard is 1kbyte/sec and 5 clients are connected, which speed is it possible to write with to one client?
How can I implement sequential sending of data in both directions? I mean I want to send N bytes from server to client, then M bytes from client to server, then N from server to client etc. The problem is if any of the two sides has written something to the channel, the other side will stop reading that data (read() == -1) only if channel is closed, which means that we cannot reuse it and have to open another connection. Or, may be, we should place readers and writers to different threads which do their job with read() and write() until connection is closed?
Imagine we have a delay between calling write(); flush() on one side, and calling read() on the other side. During the delay - where the written data would be stored? Would it be transmitted? What is the max size of that "delayed" data to be stored somewhere "between"?
Correct. If you're using blocking I/O, you'll need a reader thread and a writer thread for each Socket connection.
You could use a single thread to write to N clients at once, but you run the risk of blocking on a write. I won't address the writing speeds here, as it would depend on several things, but obviously the cumulative writing speed to all clients would be under 1kbps.
Yes, you'll need 2 threads, you can't do this with a single thread (or you could, but as you said yourself, you'd need to constantly open and close connections).
It would be stored in a buffer somewhere. Depending on your code it could be in a Buffered stream, or the socket's own buffer. I believe the default buffer size of BufferedOutputStream is 8K, and the socket's own buffer would depend on the environment. It shouldn't really be of importance though, the streaming quality of TCP/IP removes the need to think about buffers unless you really need to do fine-tuning.
I'm programming a simple Java NIO server and have a little headache: I get normal InputStreams i need to pipe to my clients. I have a single thread performing all writes, so this creates a problem: if the InputStream blocks, all other connection writing will be paused.
I can use InputStream.available() to check if there are any incoming data I can read without blocking, but if I've reached end-of-stream it seems I must call read() to know.
This creates a major headache for me, but I can't possibly believe I'm the first to have this problem.
The only options I've come up with so far:
Have a separate thread for each InputStream, however that's just silly since I'm using non-blocking I/O in the first place. I could also have a thread pool doing this but then again that limits the amount of simultaneous clients I can pipe the InputStream to.
Have a separate thread reading these streams with a timeout (using another thread to interrupt if reading has lasted longer than a certain amount of time), but that'll most certainly choke the data flow should I have many open InputStreams not delivering data.
Of course, if there was a magic InputStream.isEof() or isClosed() then this wouldn't be any problem at all :'(
".....Have a separate thread for each InputStream, however that's just silly since I'm using non-blocking I/O in the first place...."
It's not silly at all. First you need to check whether you can retrieve a SelectableChannel from your InputStream implementation. If it does you are lucky and you can just register it with a selector and do as usual. But chances are that your InputStream may have a channel that's not a SelectableChannel, in which case "Have a separate thread for each InputStream" is the obvious thing to do and probably the right thing to do.
Note that there is a similar problem discussed in SO about not able to get a SelectableChannel from an inputstream. Unfortunately you are stuck.
I have a single thread performing all
writes
Have you stopped to consider whether that is part of the problem rather than part of the solution?
When reading from an InputStream, is there a way to cancel the read when it reaches a certain size and ignore the rest of the stream safely ensuring the resources are completely released?
So far, I just finish the read, but ideally I would like to stop reading it and move on. How do I do it safely?
Here is what I have so far:
ByteArrayOutputStream buffer = new ByteArrayOutputStream();
int nRead;
byte[] byteData = new byte[16384];
while ((nRead = inputStream.read(byteData, 0, byteData.length)) != -1){
if(buffer.size() <= MAX_FILE_SIZE){
buffer.write(byteData, 0, nRead);
}
}
if(buffer.size() <= MAX_FILE_SIZE){
buffer.flush();
mData = buffer.toByteArray();
}
inputStream.close();
Thanks
Calling close() does what you want with respect to the JVM and its resources.
However, in some circumstances it could have effects that are undesirable. For instance, if the input stream is (ultimately) a socket stream, then closing the stream closes the socket, and take may cause the remote server that is sending data to see a network error. (This probably doesn't matter, but if it is not handled cleanly, you may well see exceptions in a remote webserver's logfile.)
Even if it was in the middle of being read and doesn't finish?
Yes. Any data that is "in flight" will be thrown away.
By closing its socket handle, this application says implicitly that it is no longer interested in the data.
Under normal circumstances1, there is nothing else that has the socket handle that allows it to read that data.
There is no way for anything else to reconnect to the socket. That is not supported by the socket APIs ... at the operating system level.
There is therefore no point in "keeping" the data.
(If we are talking about a socket stream then the remote server might get an exception if it tries to write more data to the socket after the close propagated. But even if that occurs, the remote server has no way of knowing how much data this end actually read before "pulling the plug" on the connection.)
Also, does the buffer need to be somehow cancelled or closed as well.
Since it is a ByteArrayOutputStream, No. Streams that read from / write to in-memory buffers (byte arrays, StringBuffers) don't need to be closed2. The GC can reclaim purely in-memory resources without any issues. Also a BufferedInput/OutputStream doesn't need to be closed if the stream it wraps doesn't need closing.
1 - I think it is possible for a Linux/Unix to open a socket, and pass it to a forked child process. However, it is impractical for both the parent and child processes to both use the socket because of the difficulty coordinating their use of it. Furthermore, you can't do this kind of thing between Java processes because the Java Process API doesn't allow it.
2 - The only hypothetical case where that is not true is when the buffer is a NIO Buffer backed by a shared memory segment or memory-mapped file ... which the garbage collector may be unable to reclaim in a timely fashion. And I say hypothetical because I don't think there are off-the-shelf stream wrappers for NIO Buffer objects.
close() is safe and does release resources: http://download.oracle.com/javase/6/docs/api/java/io/InputStream.html#close%28%29
That is all that you need to do on this end. It releases all JVM resources. If it's associated with a socket, this socket will be closed. Operating system (IOW transport layer) will simply discard all buffers, forthcoming packets etc. The other end of the connection (sender) may see an error, but either way it should be prepared for it.