I'm trying to find the most elegant way of pairing two socket channels using java NIO. So far I am writing to one channel, reading from it and and writing the result to another.
The way I'm going about it seems like a hack and I was wondering if anyone knew of a better way?
public void readyChannels() {
while (true) {
try {
selector.select();// block here until a new IO event
Iterator keys = selector.selectedKeys().iterator();
while (keys.hasNext()) {
SelectionKey key = (SelectionKey) keys.next();
keys.remove();// do not process this again
write(key.channel(),"random data".getBytes());
byte[] bytes = read(key.channel());
write(otherChannel, bytes);
}
} catch (IOException e) {
e.printStackTrace();
}
}
}
Just make one channel the key-attachment of the other channel's selection key, or better still make an object that contains them both and set that as the attachment.
Related
First, I'm not a developer (and I've been coding only for 2 weeks), so feel free to tell me I'm completely misunderstanding the thing (also, I wrote all of this for myself, so I'm sure it's super not cool) :). I want to learn and get it right, so I'm keen to listen to suggestions or complete rewrites.
I want to connect to a socket in non-blocking mode (I'm the client, not the server). I'll mainly need to read from it, but sometimes I'll need to write to it, too. The procedure is as follows:
Connect to socket
Send some initial requests to login to the server
Read from the socket
Sometimes, write some stuff (subscribe to certain information, for example)
My solution is as follows (I'm writing it in Java, because I've read it's a fast and good programming language, but I'm happy to change if required... hopefully not needed though!):
public class SocketClient {
public static void main(String[] args) {
new Feed().init();
}
private boolean isSocketConnected() {
return socket != null && socket.isConnected();
}
public void init() {
try {
if (isSocketConnected()) {
// What here if I'm in non-blocking mode?
// Would be good to know if the "close API" request succeeded
// otherwise next time I won't be able to connect to their socket...
sendCloseRequestToApi();
socket.close();
}
run();
} catch (Exception e) {
if (isSocketConnected()) {
// Same question as above...
sendCloseRequestsToApi();
socket.close();
}
}
}
public void run() throws IOException {
System.out.println("Starting connection in blocking mode...");
SocketChannel channel = SocketChannel.open();
socket = channel.socket();
socket.setReceiveBufferSize(RECEIVE_BUFFER_SIZE);
socket.setSendBufferSize(SEND_BUFFER_SIZE);
channel.connect(new InetSocketAddress("127.0.0.1", 2121));
channel.finishConnect();
System.out.println("Finished connecting in blocking mode");
// Writes to the socket (user and password)
initialiseTheApi();
System.out.println("Sent API requests in blocking mode");
System.out.println("Now we should probably go non-blocking (I guess)");
channel.register(selector, SelectionKey.OP_WRITE | SelectionKey.OP_READ);
selector = Selector.open();
channel.configureBlocking(false);
System.out.println("Selector created and switched to non-blocking mode...");
long timeWithoutData = 0;
boolean needsReconnection = false;
while (!needsReconnection) {
selector.select();
Iterator < SelectionKey > keys = selector.selectedKeys().iterator();
while (keys.hasNext()) {
SelectionKey key = keys.next();
keys.remove();
if (!key.isValid()) {
continue;
}
if (key.isWritable()) {
// Execute write...
// What if I need to know the result to the write operation?
}
if (key.isReadable()) {
int dataRead = readDataFromSocket(buffer);
buffer.flip();
if (buffer.remaining() > 0) {
// I process the data read here,
// but sometimes the data sent is
// "reconnect to API". So I need to close
// the connection and start again.
// How can I do that if I'm in non-blocking mode?
// I mean, I need to make sure when I send that request
// (for reconnection).
// I need to know that the request got to the server and
// was processed OK before moving on and
// reading/writing again...
}
if (dataRead > -1) {
timeWithoutData = 0;
} else {
if (timeWithoutData > 0) {
long diffInMillis = System.currentTimeMillis() - timeWithoutData;
if (diffInMillis > 2000) {
System.out.println("Timeout or something? I need to reconnect I think");
needsReconnection = true;
}
} else {
timeWithoutData = System.currentTimeMillis();
}
}
// Do I even need this? Already did it before, right?
key.interestOps(SelectionKey.OP_READ | SelectionKey.OP_WRITE);
}
}
}
if (needsReconnection) {
// We need full reconnection, go back up and reconnect
init();
}
}
}
I removed imports and other non-useful methods for convenience, and to keep the post short.
As you can see in my questions in the code (plus some added ones):
Reconnection: If I'm in non-blocking mode, how do I know that my request got sent successfully to the server
If I read from the socket and the message is "Reconnect to API", how can I make sure that happens before any other read / write?
Do I need to send the interestedOps over and over again?
I should only connect once to the socket. The fact that I'm non-blocking doesn't change that, right?
I've seen this could all be simplified using Netty or something, but I'm already bloated with so much stuff! :(
I hope my questions are clear. Let me know otherwise, please.
Thanks a lot.
I was trying to do something that just didn't make sense. In my case I can definitely use a blocking connection, which I just didn't know about :/. Internet is a bad source of information sometimes! I kept reading over here not to use a blocking connection :D. But now it makes perfect sense the different scenarios. – Will
I'm receiving a continuous stream of data that I'm saving to a ByteBuffer.
Sometimes I need to write to the channel, however, it's important not to lose any data. Is it possible to use the selector to solve this issue?
If I'm constantly checking the selector for the channel state, it always says that the channel is currently reading and it's like there is no opportunity to perform writing. I can't use multiple connections because the server doesn't support it.
this.socketChannel = SocketChannel.open();
this.socketChannel.configureBlocking(false);
this.socketChannel.connect(new InetSocketAddress(IP, this.port));
try {
this.selector = Selector.open();
int interestSet = SelectionKey.OP_READ | SelectionKey.OP_WRITE;
SelectionKey selectionKey = this.socketChannel.register(selector, interestSet);
while (selector.select() > -1) {
// Wait for an event one of the registered channels
// Iterate over the set of keys for which events are available
Iterator selectedKeys = selector.selectedKeys().iterator();
while (selectedKeys.hasNext()) {
SelectionKey key = (SelectionKey) selectedKeys.next();
selectedKeys.remove();
try {
if (!key.isValid()) {
continue;
} else if (key.isReadable()) {
System.out.println("readable");
} else if (key.isWritable()) {
System.out.println("writable");
}
} catch (Exception e) {
e.printStackTrace();
}
}
}
} catch (IOException e) {
e.printStackTrace();
}
Edit:
Sorry I didn't add more info. This is an important bit of my code. It always prints "readable" to the console and I was hoping that isWritable block also gets executed.
Thanks in advance, Honza
You are using else if operator so if your key is readable checking for if it is writeable will not be performed, but it doesn't mean that the channel is not writeable.
Actually it could be readable and writeable in the same time. But in your program if it is readable you just don't check for writeable.
replace else-if with if and see the result.
Hi I am trying to make a chat using NIO, in the server part in a separate thread from the main when I :
private void broadcast(String msg, String user)
{
String Message = user + ":" + msg + "\0";
System.out.println(Message);
ByteBuffer buf=ByteBuffer.wrap(Message.getBytes());
Set<SelectionKey> selectedKeys = selector.keys();
Iterator<SelectionKey> keyIterator = selectedKeys.iterator();
while(keyIterator.hasNext())
{
SelectionKey key = keyIterator.next();
if(key.attachment() != user)
{
try {
((SocketChannel) key.channel()).write(buf);
} catch (IOException e) {
e.printStackTrace();
}
buf.rewind();
}
keyIterator.remove();
}
}
I get this exception:
java.lang.UnsupportedOperationException
at java.util.Collections$UnmodifiableCollection$1.remove(Unknown Source)
at chatserver.MessageSender.broadcast(MessageSender.java:41)
at chatserver.MessageSender.run(MessageSender.java:113)
at java.lang.Thread.run(Unknown Source)
Line 41 is: keyIterator.remove();
I am using this as guideline: Java NIO Tutorial
You are invoking Selector.keys() where you appear to want Selector.selectedKeys(). The Set returned by the former is documented to be completely unmodifiable, whereas the one returned by the latter -- which agrees better with the name of the variable to which you assign the result -- is documented to permit element removal. Moreover, the selected keys are the ones ready for I/O.
My java application has memory leaks - when my resources clearing code is executing task manager shows that memory usage wasn't changed. My code
while (isRunning) {
try
{
selector.select();
long sum=0;
Set keys = selector.selectedKeys();
Iterator it = keys.iterator();
while(it.hasNext())
{
SelectionKey key = (SelectionKey)it.next();
if (key.isReadable())
{
SocketChannel sc = (SocketChannel) key.channel();
ByteBuffer bb;
if(key.attachment()==null)
{
bb = ByteBuffer.allocate(1024*1024);
key.attach(bb);
}
else
{
bb = (ByteBuffer)key.attachment();
bb.clear();
}
int x = sc.read(bb);
System.out.println(x +" bytes were read");
if(x==-1)
{
key.attach(null); //doesn't work
sc.close();
//bb = null; // also doesn't work
}
}
}
keys.clear();
}
catch (Exception ex)
{
ex.printStackTrace(new PrintStream(System.out));
}
finally
{
//stopServer();
}
}
Testing logic - I wrote simple TCP client java programm sending 100 messages to server. I intentionally allocated large buffer - 1MB for each connection. When client finishes his job int x = sc.read(bb); returns -1 and the following code is executed:
if(x==-1)
{
key.attach(null); //doesn't work
sc.close();
//bb = null; // also doesn't work
}
I checked it with debug output, this code was really executed but task manager still showes large memory usage. where is the problem?
Certainly key.attach(null) works. If it didn't, attaching a non-null object wouldn't work either. Same code.
But, in any case, closing the SocketChannel cancels the key, which removes it from all key sets of all Selectors it was registered with, so you will never see the key again anyway, so it becomes eligible for GC, and so does the attachment, regardless of whether you call key.attach(null) or not, which is therefore redundant. Either you have another reference to your attachment somewhere else, or your memory usage problem is elsewhere.
I have a simple non-blocking server with main loop:
try {
while (selector.select() > -1) {
// Wait for an event one of the registered channels
// Iterate over the set of keys for which events are available
Iterator selectedKeys = selector.selectedKeys().iterator();
while (selectedKeys.hasNext()) {
SelectionKey key = (SelectionKey) selectedKeys.next();
selectedKeys.remove();
try {
if (!key.isValid()) {
continue;
}
if (key.isConnectable()) {
connect(key);
}
// Check what event is available and deal with it
if (key.isAcceptable()) {
accept(key);
}
if (key.isReadable()) {
read(key);
}
if (key.isWritable()) {
write(key);
}
} catch (Exception e) {
e.printStackTrace();
close(key);
}
}
}
} catch (Exception e) {
e.printStackTrace();
}
In read/write section I check if there is something to read/write if not - then I try to close channel:
if (channel.read(attachment.buffer) < 1)
close(key);
Close method:
private void close(SelectionKey key) throws IOException {
key.cancel();
key.channel().close();
}
But during processing this code I get exception in main loop (it is catched but I supposed something wrong) I get this stacktrace:
java.nio.channels.CancelledKeyException
at sun.nio.ch.SelectionKeyImpl.ensureValid(Unknown Source)
at sun.nio.ch.SelectionKeyImpl.readyOps(Unknown Source)
at java.nio.channels.SelectionKey.isWritable(Unknown Source)
So it fails on main loop when enter write section, close channel and came back to main loop in 'writable' if section and fails with such exception. Any suggestions?
The error is very simple.
if (!key.isValid()) {
continue;
}
if (key.isConnectable()) {
connect(key);
}
// Check what event is available and deal with it
if (key.isAcceptable()) {
accept(key);
}
if (key.isReadable()) {
read(key);
}
if (key.isWritable()) {
write(key);
}
Your read method is the one which cancels the SelectionKey. However, after returning from read, you again test the key for whether the channel is writable -- potentially after just cancelling that very same key! Your initial check cannot help here.
One solution would be to check for whether the key is valid wherever it might've just been cancelled:
...
if (key.isValid() && key.isWritable()) {
write(key);
}
...
Alternatively, you could also try only registering one interest at a time as you need to on any particular channel, and thus all readiness events are mutually exclusive:
if (!key.isValid()) {
continue;
}
if (key.isConnectable()) {
connect(key);
} else if (key.isAcceptable()) {
accept(key);
} else if (key.isReadable()) {
read(key);
} else if (key.isWritable()) {
write(key);
}
This might be beneficial in situations; as generally a channel will almost always be write-ready, keeping an interest in write-readiness along side read-readiness might keep the Selector loop spinning, which is more than likely not desirable. For the most part, generally register interest in write-readiness only when the underlying socket output buffer is full.
As a side note, know that SocketChannel.read can return a value < 1 without it being an error.
A read operation might not fill the buffer, and in fact it might not read any bytes at all. Whether or not it does so depends upon the nature and state of the channel. A socket channel in non-blocking mode, for example, cannot read any more bytes than are immediately available from the socket's input buffer;
Additionally, Selector.select does not state anything about returning < -1 to indicate it is closed.
Returns: The number of keys, possibly zero, whose ready-operation sets were updated