I am new to a non-blocking IO in Java. I have a question - will a readiness of a non-blocking channel be lost by the selector, if a new packet from a server will arrive after we completed reading from the channel, but before we removed a selection key for this channel from selector? Example code here:
Selector selector;
// ......
while (true) {
selector.select();
Set<SelectionKey> set = selector.selectedKeys();
Iterator<SelectionKey> iterator = set.iterator();
while (iterator.hasNext()) {
SelectionKey key = iterator.next();
SocketChannel channel = (SocketChannel) key.channel();
ByteBuffer byteBuffer = ByteBuffer.allocate(GOOD_ENOUGH_CAPACITY);
while (channel.read(byteBuffer) > 0) ;
// HERE ! What happen if server started to write new message here?
// Will this channel be selected on next selector.select() ?
iterator.remove();
}
}
Yes, key will be selected. You must use method
key.cancel();
to remove key from selector
Related
I just found that the problem might due to multi-threading problem. After the read() method I sent the request to several worker threads to process the data. When the thread pool size is 1, the problem doesn't occur. However, when the thread pool size is bigger than 1, this problem occurs.
I initialized my threadpool before the while(true) loop, and execute my worker thread after the read() method.
I'm using Java NIO in a middleware that connects a memtier client and a memcached server. The system works fine with only one clients connected, however, when there are more clients connected, when one client finishes and closes its channel, the other client won't be able to read or write to its channel any more.
My implementation looks like the following:
public void run() {
try{
// Connect to client
selector = Selector.open();
ServerSocketChannel server = ServerSocketChannel.open();
server.configureBlocking(false);
InetSocketAddress address = new InetSocketAddress(8000);
server.socket().bind(address);
server.register(selector,SelectionKey.OP_ACCEPT);
// Connect to Server
Socket socket = new Socket("localhost",8090);
// Select keys
while (true) {
selector.select();
Iterator<SelectionKey> iterator = selector.selectedKeys().iterator();
while(iterator.hasNext()) {
SelectionKey key = iterator.next();
iterator.remove();
// check if key is valid
if(!key.isValid()) {
continue;
}
if(key.isAcceptable()) {
accept(key);
}else if(key.isReadable()) {
read(key);
}
}
}
}
}
My accept() and read() looks like this:
private void accept(SelectionKey key) {
ServerSocketChannel server = (ServerSocketChannel) key.channel();
SocketChannel socketChannel = server.accept();
socketChannel.configureBlocking(false);
socketChannel.register(selector, SelectionKey.OP_READ);
}
private void read(SelectionKey key) {
SocketChannel channel = (SocketChannel) key.channel();
buffer.clear();
int n = -1;
n = channel.read(buffer);
if(n == -1) {
key.cancel();
channel.close();
}else{
//read to buffer
}
}
I have tried many ways but still cannot find out the problem. Thank you very much!
I am learning Java nio selector. In my understanding, I thought the steps using selector is to firstly I register the my interest operations and then I can check the ready set and finally I can do the operations corresponding my interest operations. I don't know why in this code the writing process can happen in the block of if (key.isReadable()){...} but not if (key.isWritable){...} and why writing operation is not registered?
Iterator keys = sel.selectedKeys().iterator();
while (keys.hasNext()) {
SelectionKey key = (SelectionKey)keys.next();
if (!key.isValid())
continue;
if (key.isAcceptable()) {
// increase the counter
connection++;
// remove accept request
keys.remove();
// ACCEPT: get the server channel
ServerSocketChannel ssc =
(ServerSocketChannel) key.channel();
// init a socket for a client
SocketChannel nsc = ssc.accept();
nsc.configureBlocking(false);
// register the socket for READ
nsc.register(sel, SelectionKey.OP_READ);
}
}
while (count < COUNT_MAX + NUM_CHILD - 1) {
sel.select();
// Get all pending events and iterate
Iterator keys = sel.selectedKeys().iterator();
while ( keys.hasNext() ) {
SelectionKey key = (SelectionKey)keys.next();
keys.remove();
if (!key.isValid())
continue;
if (key.isReadable()) {
// READ: get the channel
SocketChannel nsc = (SocketChannel) key.channel();
// clear buffer for reading
readBuffer.clear();
int nBytes = nsc.read(readBuffer);
if (nBytes == -1) {// Check if the client closed the socket
key.channel().close();
key.cancel();
continue;
}
// Read a message
DataInputStream ist = new DataInputStream (
new ByteArrayInputStream(readBuffer.array()));
String msg = ist.readUTF();
System.out.print(msg + "\n");
// Clear the write buffer
writeBuffer.clear();
// Write the counter value on the buffer
count++;
if (count < COUNT_MAX)
writeBuffer.putInt(count);
else
writeBuffer.putInt(-1);
// flip the buffer and write on the channel
writeBuffer.flip();
// Reply to a client
nsc.write(writeBuffer);
}
} // while (key)
You don't need to register interest in OP_WRITE because usually the channel is ready for writing. However a WritableChannel, if in non blocking mode, might not succeed in writing all content of the given ByteBuffer. See here in its java docs:
Some types of channels,
depending upon their state, may write only some of the bytes or
possibly none at all. A socket channel in non-blocking mode, for
example, cannot write any more bytes than are free in the socket's
output buffer.
In this case you need to register the interest for OP_WRITE on the selector to be notified when the channel is once again ready for writing, so you can finish writing your ByteBuffer.
See here a related SO question.
I am new to NIO i understand the concept of Asynchronous Socket but i am confused on Non Blocking part.
I am using java NIO Selector . My Code for Server is
public class EcoNonBlockingIOServer_7 {
public static int PORT_NUMBER = 5555;
public static void main(String[] argv) throws Exception {
new EcoNonBlockingIOServer_7().go(argv);
}
public void go(String[] argv) throws Exception {
int port = PORT_NUMBER;
if (argv.length > 0) { // Override default listen port
port = Integer.parseInt(argv[0]);
}
System.out.println("Listening on port " + port);
// Allocate an unbound server socket channel
ServerSocketChannel serverChannel = ServerSocketChannel.open();
// Get the associated ServerSocket to bind it with
ServerSocket serverSocket = serverChannel.socket();
// Create a new Selector for use below
Selector selector = Selector.open();
// Set the port the server channel will listen to
serverSocket.bind(new InetSocketAddress(port));
// Set nonblocking mode for the listening socket
serverChannel.configureBlocking(false);
// Register the ServerSocketChannel with the Selector
serverChannel.register(selector, SelectionKey.OP_ACCEPT);
while (true) {
// This may block for a long time. Upon returning, the
// selected set contains keys of the ready channels.
int n = selector.select();
if (n == 0) {
continue; // nothing to do
}
// Get an iterator over the set of selected keys
Iterator it = selector.selectedKeys().iterator();
// Look at each key in the selected set
while (it.hasNext()) {
SelectionKey key = (SelectionKey) it.next();
// Is a new connection coming in?
if (key.isAcceptable()) {
ServerSocketChannel server = (ServerSocketChannel) key.channel();
SocketChannel channel = server.accept();
registerChannel(selector, channel, SelectionKey.OP_READ);
sayHello(channel);
}
// Is there data to read on this channel?
if (key.isReadable()) {
readDataFromSocket(key);
}
// Remove key from selected set; it's been handled
it.remove();
}
}
}
Now My Queries are:
If we register a channel with selector on any operation it always get blocked on selector.select() then how it is non blocking.
If we admit it uses OP_ACCEPT as key and maps channel accordingly but again In key is acceptable i am modifying this channel selector to OP_READ since it already has been accepted. Again It blocks on selector.select() for read event
*Please correct my understanding if i am wrong *
If we register a channel with selector on any operation it always get blocked on selector.select() then how it is non blocking.
select() is blocking. Every operation on a non-blocking channel itself is non-blocking, i.e. read() and write().
If we admit it uses OP_ACCEPT as key and maps channel accordingly but again In key is acceptable i am modifying this channel selector to OP_READ since it already has been accepted.
Very confused. The channel whose interest-ops == OP_ACCEPT is the listening socket. The channel you accepted from the listening socket is a connected socket, and it is this socket that you put into non-blocking mode, register with OP_ACCEPT, etc.
Again It blocks on selector.select() for read event
Correct, but it doesn't block in read() or write() or accept() or finishConnect(). Using a selector is actually called multiplexed I/O: you wait for multiple channels and multiple events at the same time in a single operation.
I use SocketChannel to receive TCP stream from server on client side. For example:
Selector selector=Selector.open();
SocketChannel mychannel=SocketChannel.open(new InetSocketAddress(ip,port));
channel.configureBlocking(false);
SelectionKey channelkey=channel.register(selector,SelectionKey.OP_READ);
Then, I can use the selector.select() method to handle reading problem.
while(true){
if(selector.select()!=0){
Iterator<SelectionKey> it=selector.selectedKeys().iterator();
while(it.hasNext()){
SelectionKey key=it.next();
it.remove();
if(key.isReadable()){
if(key.equals(channelkey)){
//concrete handle
...
}
}
}
}
}
With concrete handle,considering I'd like to use InputStream(I wanna read stream line) to receive tcp stream from server side,there are two methods.One is using channel.socket(), another is to use Channels. Here I use channel.socket(), for example:
SocketChannel channel = (SocketChannel) key.channel();
key.cancel();
channel.configureBlocking(true);
InputStream ins = Channels.newInputStream(channel);
InputStreamReader is = new InputStreamReader(ins,"utf-8");
BufferedReader in = new BufferedReader(is);
String res = in.readLine();
while (!res.equals("")) {
System.out.println(res);
res = in.readLine();
}
......①
OK, now I finish reading tcp stream for one time.In order to continue to using selector, I should set channel blocking mode to false.
channel.configureBlocking(false);
The question is,the key that combines channel and selector has been canceled.I want to register mychannel again.What should I do? It seems that if I use mychannel.register(selector, SelectionKey.OP_READ) again on ①, it throws Exception.
My run() method code is as follows:
try {
if (selector.select(getTimeout()) != 0) {
Iterator<SelectionKey> it = selector.selectedKeys().iterator();
while (it.hasNext()) {
SelectionKey key = it.next();
if (key.isReadable()) {
SocketChannel channel = (SocketChannel) key.channel();
key.cancel();
channel.configureBlocking(true);
InputStream ins = Channels.newInputStream(channel);
InputStreamReader is = new InputStreamReader(ins,"utf-8");
BufferedReader in = new BufferedReader(is);
String res = in.readLine();
while (!res.equals("")) {
System.out.println("========" + res);
res = in.readLine();
}
channel.configureBlocking(false);
System.out.println(key.isValid());
proxykey=channel.register(selector, SelectionKey.OP_READ);
}
it.remove();
}
}
} catch (IOException ex) {
ex.printStackTrace();
}
The Exception it throws is:
Exception in thread "Thread-0" java.nio.channels.CancelledKeyException
at sun.nio.ch.SelectionKeyImpl.ensureValid(Unknown Source)
at sun.nio.ch.SelectionKeyImpl.interestOps(Unknown Source)
at java.nio.channels.spi.AbstractSelectableChannel.register(Unknown Source)
at java.nio.channels.SelectableChannel.register(Unknown Source)
at AgentThread.run(AgentThread.java:185)
SelectionKey.cancel() doesn't take full effect until the next select(), for various arcane reasons. You could try calling selectNow() after the cancel, or maybe better just before the re-registration.
I decided to make my UDPclient and UDPserver with java nio.
But I don't understand several things.
Here is the code
try {
DatagramChannel channel = DatagramChannel.open();
channel.configureBlocking(false);
channel.connect(remote);
//monitoring
Selector selector = Selector.open();
//read write keys
channel.register(selector, SelectionKey.OP_READ | SelectionKey.OP_WRITE);
ByteBuffer buffer = ByteBuffer.allocate(1024*64);//number of bytes for channel
while (true) {
selector.select(60000);//number of channels I think
Set readyKeys = selector.selectedKeys();
if (readyKeys.isEmpty()) {
break;
}
else {
Iterator iterator = readyKeys.iterator();
while (iterator.hasNext()) {
SelectionKey key = (SelectionKey) iterator.next();
iterator.remove();
if (key.isReadable( )) {
//read from buffer
channel.read(buffer);
}
if (key.isWritable()) {
//write to buffer
channel.write(buffer);
}
}
}
}
}
catch (IOException ex) {
System.err.println(ex);
}
If I write something in console the event in key.isWritable will occur? And if server sends something event isReadable will occur?
And I don't understand how to work with my events when for example user write "GETL" or "REGR"(my own methods).
The value you pass to select is a timeout not the number of channels.
You need to do
DatagramChannel channelFromKey = (DatagramChannel) key.channel();
not use channel
I don't understand what you mean by your own events. Read the Datagrams off the channel when that key is selected.
Iterator iterator = readyKeys.iterator();
while (iterator.hasNext()) {
SelectionKey key = (SelectionKey) iterator.next();
iterator.remove();
if (key.isReadable( )) {
DatagramChannel channelFromKey =
(DatagramChannel) key.channel();
buffer.clear();
// This is a DatagramChannel receive a datagram as a whole
channelFromKey.receive(buffer);
}
If I write something in console the event in key.isWritable will
occur?
No. The only events that will occur are on the channels you have registered with the selector. You haven't registered any channel to do with the console, and you can't, because only network channels are SelectableChannels, so you have to reason to expect events originating from the console to turn up via the Selector.
And if server sends something event isReadable will occur?
Yes.
And I don't understand how to work with my events when for example user
write "GETL" or "REGR"(my own methods).
Nor do I. I don't even understand the question. The only events you will get from the selector are the ones that it defines, on the channels you have registered with it.