I am trying to add a write timeout to a SocketChannel by using a Selector:
public int write(ByteBuffer buf, long timeout, SocketChannel socketChannel, Selector selector) {
int written = socketChannel.write(buf);
if (written == 0) {
SelectionKey selectionKey = socketChannel.register(selector, SelectionKey.OP_WRITE);
try {
selector.select(timeout);
if (Thread.interrupted()) {
throw new InterruptedIOException();
}
written = socketChannel.write(buf);
} finally {
selectionKey.cancel();
}
}
return written;
}
The plan is to reuse the selector and de-register it after each timeout. But apparently I get the same selectionKey that was previously cancelled if I call my method twice and it is unable to write at all. This results in a CancelledKeyException when register is called during the second method call. I thought I would get a new selectionKey after the first was cancelled.
So how do I do this? I guess calling selectNow() before the register or after the cancel, in order to clear the cancelled keys will solve it but it seems weird, is there a better way?
Related
First, I'm not a developer (and I've been coding only for 2 weeks), so feel free to tell me I'm completely misunderstanding the thing (also, I wrote all of this for myself, so I'm sure it's super not cool) :). I want to learn and get it right, so I'm keen to listen to suggestions or complete rewrites.
I want to connect to a socket in non-blocking mode (I'm the client, not the server). I'll mainly need to read from it, but sometimes I'll need to write to it, too. The procedure is as follows:
Connect to socket
Send some initial requests to login to the server
Read from the socket
Sometimes, write some stuff (subscribe to certain information, for example)
My solution is as follows (I'm writing it in Java, because I've read it's a fast and good programming language, but I'm happy to change if required... hopefully not needed though!):
public class SocketClient {
public static void main(String[] args) {
new Feed().init();
}
private boolean isSocketConnected() {
return socket != null && socket.isConnected();
}
public void init() {
try {
if (isSocketConnected()) {
// What here if I'm in non-blocking mode?
// Would be good to know if the "close API" request succeeded
// otherwise next time I won't be able to connect to their socket...
sendCloseRequestToApi();
socket.close();
}
run();
} catch (Exception e) {
if (isSocketConnected()) {
// Same question as above...
sendCloseRequestsToApi();
socket.close();
}
}
}
public void run() throws IOException {
System.out.println("Starting connection in blocking mode...");
SocketChannel channel = SocketChannel.open();
socket = channel.socket();
socket.setReceiveBufferSize(RECEIVE_BUFFER_SIZE);
socket.setSendBufferSize(SEND_BUFFER_SIZE);
channel.connect(new InetSocketAddress("127.0.0.1", 2121));
channel.finishConnect();
System.out.println("Finished connecting in blocking mode");
// Writes to the socket (user and password)
initialiseTheApi();
System.out.println("Sent API requests in blocking mode");
System.out.println("Now we should probably go non-blocking (I guess)");
channel.register(selector, SelectionKey.OP_WRITE | SelectionKey.OP_READ);
selector = Selector.open();
channel.configureBlocking(false);
System.out.println("Selector created and switched to non-blocking mode...");
long timeWithoutData = 0;
boolean needsReconnection = false;
while (!needsReconnection) {
selector.select();
Iterator < SelectionKey > keys = selector.selectedKeys().iterator();
while (keys.hasNext()) {
SelectionKey key = keys.next();
keys.remove();
if (!key.isValid()) {
continue;
}
if (key.isWritable()) {
// Execute write...
// What if I need to know the result to the write operation?
}
if (key.isReadable()) {
int dataRead = readDataFromSocket(buffer);
buffer.flip();
if (buffer.remaining() > 0) {
// I process the data read here,
// but sometimes the data sent is
// "reconnect to API". So I need to close
// the connection and start again.
// How can I do that if I'm in non-blocking mode?
// I mean, I need to make sure when I send that request
// (for reconnection).
// I need to know that the request got to the server and
// was processed OK before moving on and
// reading/writing again...
}
if (dataRead > -1) {
timeWithoutData = 0;
} else {
if (timeWithoutData > 0) {
long diffInMillis = System.currentTimeMillis() - timeWithoutData;
if (diffInMillis > 2000) {
System.out.println("Timeout or something? I need to reconnect I think");
needsReconnection = true;
}
} else {
timeWithoutData = System.currentTimeMillis();
}
}
// Do I even need this? Already did it before, right?
key.interestOps(SelectionKey.OP_READ | SelectionKey.OP_WRITE);
}
}
}
if (needsReconnection) {
// We need full reconnection, go back up and reconnect
init();
}
}
}
I removed imports and other non-useful methods for convenience, and to keep the post short.
As you can see in my questions in the code (plus some added ones):
Reconnection: If I'm in non-blocking mode, how do I know that my request got sent successfully to the server
If I read from the socket and the message is "Reconnect to API", how can I make sure that happens before any other read / write?
Do I need to send the interestedOps over and over again?
I should only connect once to the socket. The fact that I'm non-blocking doesn't change that, right?
I've seen this could all be simplified using Netty or something, but I'm already bloated with so much stuff! :(
I hope my questions are clear. Let me know otherwise, please.
Thanks a lot.
I was trying to do something that just didn't make sense. In my case I can definitely use a blocking connection, which I just didn't know about :/. Internet is a bad source of information sometimes! I kept reading over here not to use a blocking connection :D. But now it makes perfect sense the different scenarios. – Will
We wrote an incoming reactor that works this way:
Opens a selector
Opens a server socket channel
Starts a selection loop in which: The ServerSocketChannel accepts new SocketChannels into the loop, and each SocketChannel reads data and transfers it to a worker.
The shutting down procedure of the reactor is iterating over the selector.keys() and for each of them closing the corresponding channel and cancelling the key.
We wrote the following unit test for the shutdown procedure:
Open a reactor thread running the selction loop.
Open several Sender threads. Each opens a socket to the reactor and reads.
The read blocks until it gets -1 (meaning the reactor closed the socket).
After the read returns -1, the sender closes the socket and finishes.
The test causes ConcurrentModificationException pointing to the loop iterating over the sockets and closes them (which was in the main thread context).
Our assumption is that when a Sender read method got -1, it closed the socket and somehow it woke up the selector select method, The selector then accessed its keys set which was iterated by the shutdown loop and hence the exception.
We worked around this problem by creating a new list with all the keys of the selector. Canceling those keys by iterating this list prevent two objects from modifying the same key's set.
Our question are:
Is our assumption correct? When the client socket calls the close method- does it really wake up the selector?
Does the creation of a new list is the appropriate solution or is it just a work-around?
EDIT: Added some code snippets for clarifications
(We tried to narrow the code as possible)
IncomingReactor:
public boolean startAcceptingIncomingData() {
Selector selector = Selector.open();
ServerSocketChannel serverSocketChannel = ServerSocketChannel.open());
serverSocketChannel.bind(new InetSocketAddress(incomingConnectionsPort));
serverSocketChannel.configureBlocking(false);
SelectionKey acceptorSelectionKey = serverSocketChannel.register(selector, SelectionKey.OP_ACCEPT);
acceptorSelectionKey.attach((Worker) this::acceptIncomingSocket);
startSelectionLoop(selector);
return true;
}
private boolean acceptIncomingSocket() {
try {
SocketChannel socketChannel = serverSocketChannel.accept();
socketChannel.configureBlocking(false);
SelectionKey selectionKey = socketChannel.register(selector, SelectionKey.OP_READ);
selectionKey.attach(new WorkerImpl() /*Responsible for reading data and tranferring it into a parsing thread*/);
return true;
} catch (IOException e) {
return false;
}
}
private void startSelectionLoop(Selector selector) {
shouldLoop = true;
while (shouldLoop) {
try {
selector.select();
Set<SelectionKey> selectedKeys = selector.selectedKeys();
if (!shouldLoop) {
break;
}
selectedKeys.forEach((key) -> {
boolean workSuccess = ((Worker) key.attachment()).work();
if (!workSuccess) {
key.channel().close();
key.cancel();
}
});
selectedKeys.clear();
} catch (ClosedSelectorException ignore) {
}
}
}
public void shutDown() {
shouldLoop = false;
selector.keys().forEach(key -> { /***EXCEPTION - This is where the exception points to (this is line 129) ***/
key.channel().close();
key.cancel();
});
try {
selector.close();
} catch (IOException e) {
}
}
UnitTest:
#Test
public void testMaximumConnectionsWithMultipleThreads() {
final int PORT = 24785;
final int MAXINUM_CONNECTIONS = 10;
IncomingReactor incomingReactor = new IncomingReactor(PORT);
Callable<Boolean> acceptorThread = () -> {
incomingReactor.startAcceptingIncomingData();
return true;
};
ExecutorService threadPool = Executors.newFixedThreadPool(MAXIMUM_CONNECTIONS + 1);
Future<Boolean> acceptorFuture = threadPool.submit(acceptorThread);
List<Future<Boolean>> futureList = new ArrayList<>(MAXIMUM_CONNECTIONS);
for (int currentSenderThread = 0; currentSenderThread < MAXIMUM_CONNECTIONS; currentSenderThread++) {
Future<Boolean> senderFuture = threadPool.submit(() -> {
Socket socket = new Socket(LOCALHOST, PORT);
int bytesRead = socket.getInputStream().read();
if (bytesRead == -1) { //The server has closed us
socket.close();
return true;
} else {
throw new RuntimeException("Got real bytes from socket.");
}
});
futureList.add((senderFuture));
}
Thread.sleep(1000); //We should wait to ensure that the evil socket is indeed the last one that connects and the one that will be closed
Socket shouldCloseSocket = new Socket(LOCALHOST, PORT);
Assert.assertEquals(shouldCloseSocket.getInputStream().read(), -1);
shouldCloseSocket.close();
incomingReactor.shutDown();
for (Future<Boolean> senderFuture : futureList) {
senderFuture.get();
}
acceptorFuture.get();
threadPool.shutdown();
}
Exception:
java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextNode(HashMap.java:1437)
at java.util.HashMap$KeyIterator.next(HashMap.java:1461)
at java.lang.Iterable.forEach(Iterable.java:74)
at java.util.Collections$UnmodifiableCollection.forEach(Collections.java:1080)
at mypackage.IncomingReactor.shutDown(IncomingReactor.java:129)
at mypackage.tests.TestIncomingReactor.testMaximumConnectionsWithMultipleThreads(TestIncomingReactor.java:177)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:85)
at org.testng.internal.Invoker.invokeMethod(Invoker.java:659)
at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:845)
at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1153)
at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:125)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:108)
at org.testng.TestRunner.privateRun(TestRunner.java:771)
at org.testng.TestRunner.run(TestRunner.java:621)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:357)
at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:352)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:310)
at org.testng.SuiteRunner.run(SuiteRunner.java:259)
at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
at org.testng.TestNG.runSuitesSequentially(TestNG.java:1199)
at org.testng.TestNG.runSuitesLocally(TestNG.java:1124)
at org.testng.TestNG.run(TestNG.java:1032)
at org.testng.IDEARemoteTestNG.run(IDEARemoteTestNG.java:74)
at org.testng.RemoteTestNGStarter.main(RemoteTestNGStarter.java:124)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)
The shutting down procedure of the reactor is iterating over the selector.keys() and for each of them closing the corresponding channel and cancelling the key.
It should start by stopping the selector loop. NB Closing the channel cancels the key. You don't have to cancel it yourself.
We wrote the following unit test for the shutdown procedure:
Open a reactor thread running the selction loop.
Open several Sender threads. Each opens a socket to the reactor and reads.
The read blocks until it gets -1 (meaning the reactor closed the socket).
The reactor closed its accepted socket. Your client socket remained open.
After the read returns -1, the sender closes the socket and finishes.
I hope this means the sender closed its client socket.
The test causes ConcurrentModificationException pointing to the loop iterating over the sockets and closes them (which was in the main thread context).
Really? I don't see any stack trace in your question.
Our assumption is that when a Sender read method got -1, it closed the socket and somehow it woke up the selector select method
Not possible unless the reactor didn't close the channel, in which case you wouldn't have got -1 from read etc.
The selector then accessed its keys set which was iterated by the shutdown loop and hence the exception.
The exception is caused by modifying the key set during iteration. Bug in your server code.
We worked around this problem by creating a new list with all the keys of the selector. Canceling those keys by iterating this list prevent two objects from modifying the same key's set.
You need to fix the actual problem, and for that you need to post the actual code.
Our question are:
Is our assumption correct? When the client socket calls the close method- does it really wake up the selector?
Not unless the selector-end channel is still open.
Does the creation of a new list is the appropriate solution or is it just a work-around?
It is just a nasty workaround for a problem you haven't identified yet.
You cannot modify the selector.keys() Set<SelectionKey> from inside of the for loop because that Set is not capable of concurrent modification. (calling channel.close() will modify the Set from inside the loop reading the Set)
https://docs.oracle.com/javase/7/docs/api/java/util/HashSet.html
The iterators returned by this class's iterator method are fail-fast:
if the set is modified at any time after the iterator is created, in
any way except through the iterator's own remove method, the Iterator
throws a ConcurrentModificationException. Thus, in the face of
concurrent modification, the iterator fails quickly and cleanly,
rather than risking arbitrary, non-deterministic behavior at an
undetermined time in the future.
SelectionKey[] keys = selector.keys().toArray(new SelectionKey[0]);
for( SelectionKey k : keys )
{
try
{
k.channel().close();
}
catch(Throwable x )
{
// print
}
}
try
{
selector.close();
}
catch(IoException e )
{
// print
}
I'm receiving a continuous stream of data that I'm saving to a ByteBuffer.
Sometimes I need to write to the channel, however, it's important not to lose any data. Is it possible to use the selector to solve this issue?
If I'm constantly checking the selector for the channel state, it always says that the channel is currently reading and it's like there is no opportunity to perform writing. I can't use multiple connections because the server doesn't support it.
this.socketChannel = SocketChannel.open();
this.socketChannel.configureBlocking(false);
this.socketChannel.connect(new InetSocketAddress(IP, this.port));
try {
this.selector = Selector.open();
int interestSet = SelectionKey.OP_READ | SelectionKey.OP_WRITE;
SelectionKey selectionKey = this.socketChannel.register(selector, interestSet);
while (selector.select() > -1) {
// Wait for an event one of the registered channels
// Iterate over the set of keys for which events are available
Iterator selectedKeys = selector.selectedKeys().iterator();
while (selectedKeys.hasNext()) {
SelectionKey key = (SelectionKey) selectedKeys.next();
selectedKeys.remove();
try {
if (!key.isValid()) {
continue;
} else if (key.isReadable()) {
System.out.println("readable");
} else if (key.isWritable()) {
System.out.println("writable");
}
} catch (Exception e) {
e.printStackTrace();
}
}
}
} catch (IOException e) {
e.printStackTrace();
}
Edit:
Sorry I didn't add more info. This is an important bit of my code. It always prints "readable" to the console and I was hoping that isWritable block also gets executed.
Thanks in advance, Honza
You are using else if operator so if your key is readable checking for if it is writeable will not be performed, but it doesn't mean that the channel is not writeable.
Actually it could be readable and writeable in the same time. But in your program if it is readable you just don't check for writeable.
replace else-if with if and see the result.
I have a simple non-blocking server with main loop:
try {
while (selector.select() > -1) {
// Wait for an event one of the registered channels
// Iterate over the set of keys for which events are available
Iterator selectedKeys = selector.selectedKeys().iterator();
while (selectedKeys.hasNext()) {
SelectionKey key = (SelectionKey) selectedKeys.next();
selectedKeys.remove();
try {
if (!key.isValid()) {
continue;
}
if (key.isConnectable()) {
connect(key);
}
// Check what event is available and deal with it
if (key.isAcceptable()) {
accept(key);
}
if (key.isReadable()) {
read(key);
}
if (key.isWritable()) {
write(key);
}
} catch (Exception e) {
e.printStackTrace();
close(key);
}
}
}
} catch (Exception e) {
e.printStackTrace();
}
In read/write section I check if there is something to read/write if not - then I try to close channel:
if (channel.read(attachment.buffer) < 1)
close(key);
Close method:
private void close(SelectionKey key) throws IOException {
key.cancel();
key.channel().close();
}
But during processing this code I get exception in main loop (it is catched but I supposed something wrong) I get this stacktrace:
java.nio.channels.CancelledKeyException
at sun.nio.ch.SelectionKeyImpl.ensureValid(Unknown Source)
at sun.nio.ch.SelectionKeyImpl.readyOps(Unknown Source)
at java.nio.channels.SelectionKey.isWritable(Unknown Source)
So it fails on main loop when enter write section, close channel and came back to main loop in 'writable' if section and fails with such exception. Any suggestions?
The error is very simple.
if (!key.isValid()) {
continue;
}
if (key.isConnectable()) {
connect(key);
}
// Check what event is available and deal with it
if (key.isAcceptable()) {
accept(key);
}
if (key.isReadable()) {
read(key);
}
if (key.isWritable()) {
write(key);
}
Your read method is the one which cancels the SelectionKey. However, after returning from read, you again test the key for whether the channel is writable -- potentially after just cancelling that very same key! Your initial check cannot help here.
One solution would be to check for whether the key is valid wherever it might've just been cancelled:
...
if (key.isValid() && key.isWritable()) {
write(key);
}
...
Alternatively, you could also try only registering one interest at a time as you need to on any particular channel, and thus all readiness events are mutually exclusive:
if (!key.isValid()) {
continue;
}
if (key.isConnectable()) {
connect(key);
} else if (key.isAcceptable()) {
accept(key);
} else if (key.isReadable()) {
read(key);
} else if (key.isWritable()) {
write(key);
}
This might be beneficial in situations; as generally a channel will almost always be write-ready, keeping an interest in write-readiness along side read-readiness might keep the Selector loop spinning, which is more than likely not desirable. For the most part, generally register interest in write-readiness only when the underlying socket output buffer is full.
As a side note, know that SocketChannel.read can return a value < 1 without it being an error.
A read operation might not fill the buffer, and in fact it might not read any bytes at all. Whether or not it does so depends upon the nature and state of the channel. A socket channel in non-blocking mode, for example, cannot read any more bytes than are immediately available from the socket's input buffer;
Additionally, Selector.select does not state anything about returning < -1 to indicate it is closed.
Returns: The number of keys, possibly zero, whose ready-operation sets were updated
This is a follow up to:
this question
Basically, I have a server loop that manages a connection to one solitary client. At one point in the loop, if a ClientSocket exists it attempts a read to check if the client is still connected:
if (bufferedReader.read() == -1) {
logger.info("CONNECTION TERMINATED!");
clientSocket.close();
setUpSocket(); // sets up the server to reconnect to the client
} else {
sendHeartBeat(); // Send a heartbeat to the client
}
The problem is, that once a socket has been created the application will hang on the read, I assume waiting for data that will never come, since the client never sends to the server. Before this was OK, because this correctly handled disconnects (the read would eventually fail when the client disconnected) and the loop would attempt reestablish the connection. However, I now have added the above sendHeartBeat() method, which periodically lets the client know the server is still up. If the read is holding the thread then the heartbeats never happen!
So, I assume I am testing if the connection is still up incorrectly. I could, as a quick hack, run the bufferedReader.read() in a seperate thread, but then I'll have all sorts of concurrency issues that I really don't want to deal with.
So the question is a few fold:
Am I checking for a client disconnect correctly?
If not, how should I do it?
If I am doing it correctly how I do I get the read to not hold the process hostage? Or is threading the only way?
When you create your socket, first set a timeout:
private int timeout = 10000;
private int maxTimeout = 25000;
clientSocket.setSoTimeout(timeout);
With this, if a read times out you'll get java.net.SocketTimeoutException (which you have to catch). Thus, you could do something like this, assuming you've previously set the SO_TIMEOUT as shown above, and assuming that the heartbeat will always get a response from the remote system:
volatile long lastReadTime;
try {
bufferedReader.read();
lastReadTime = System.currentTimeMillis();
} catch (SocketTimeoutException e) {
if (!isConnectionAlive()) {
logger.info("CONNECTION TERMINATED!");
clientSocket.close();
setUpSocket(); //sets up the server to reconnect to the client
} else {
sendHeartBeat(); //Send a heartbeat to the client
}
}
public boolean isConnectionAlive() {
return System.currentTimeMillis() - lastReadTime < maxTimeout;
}
A common way of handling this is setting the timeout to some number (say 10 seconds) and then keeping track of the last time you successfully read from the socket. If 2.5 times your timeout have elapsed, then give up on the client and close the socket (thus sending a FIN packet to the other side, just in case).
If the heartbeat will not get any response from the remote system, but is just a way of ultimately generating an IOException earlier when the connection has fallen down, then you could do this (assuming that the sendHeartBeat itself will not throw an IOException):
try {
if (bufferedReader.read() == -1) {
logger.info("CONNECTION TERMINATED with EOF!");
resetConnection();
}
} catch (SocketTimeoutException e) {
// This just means our read timed out ... the socket is still good
sendHeartBeat(); //Send a heartbeat to the client
} catch (IOException e) {
logger.info("CONNECTION TERMINATED with Exception " + e.getMessage());
resetConnection();
}
....
private void resetConnection() {
clientSocket.close();
setUpSocket(); //sets up the server to reconnect to the client
}
You are checking correctly, you can should add a try catch with IOException in case it occurs.
There is a way to avoid threading, you can use a Selector with a non-bloking socket.
public void initialize(){
//create selector
Selector selector = Selector.open();
ServerSocketChannel acceptSocket = ServerSocketChannel.open();
acceptSocket.configureBlocking(false);
String bindIp = "127.0.0.1";
int bindPort = 80;
acceptSocket.socket().bind(new InetSocketAddress(bindIp, bindPort));
//register socket in selector for ACCEPT operation
acceptSocket.register(selector, SelectionKey.OP_ACCEPT);
this.selector = selector;
this.serverSocketChannel = serverSocketChannel;
}
public void serverStuff() {
selector.select(maxMillisecondsToWait);
Set<SelectionKey> selectedKeys = selector.selectedKeys();
if( selectedKeys.size() > 0 )
{
if( key.isAcceptable() ){
//you can accept a new connection
SocketChannel clientSk = serverSocketChannel.accept();
clientSk.configureBlocking(false);
//register your SocketChannel in the selector for READ operations
clientSk.register(selector, SelectionKey.OP_READ);
} else if( key.isReadable() ){
//you can read from your socket.
//it will return you -1 if the connection has been closed
}
}
if( shouldSendHeartBeat() ){
SendHeartBeat
}
}
You should add error checking in your disconnection detection. Sometimes an IOException may be thrown when the connection to the other end is lost.
I am afraid that threading is unavoidable here. If you don't want to block the execution of your code, you need to create a separate thread.