I am writing an application in which the main thread handles many connections through the use of a Selector and SelectionKeys. I have run into some trouble with a race condition when attempting to pass tasks off to the worker threads.
My main loop looks like this:
selector = Selector.open(); //Create selector
serverSocketChannel = ServerSocketChannel.open(); //Create socket channel, configure blocking, and bind
serverSocketChannel.configureBlocking(false);
serverSocketChannel.bind(new InetSocketAddress(PORT));
serverSocketChannel.register(selector, SelectionKey.OP_ACCEPT); //Register channel to selector
ByteBuffer buffer = ByteBuffer.allocate(8000);
while(true){
selector.select();
Iterator<SelectionKey> iterator = selector.selectedKeys().iterator();
while(iterator.hasNext()){
SelectionKey key = iterator.next();
if(key.isAcceptable()){
SocketChannel socketChannel = serverSocketChannel.accept();
socketChannel.configureBlocking(false);
socketChannel.register(selector, SelectionKey.OP_READ);
}
if(key.isReadable()){
taskList.add(new ReadTask(key));
}
if(key.isWritable()){
}
iterator.remove();
}
}
The idea here is that when a client tries to send data to the server, it receives a key with OP_READ interest and then creates a task with that key so that the threadpool can handle the read so as not to block the main thread.
The problem is that calling this loop continues during the process of the key being passed to a worker thread, and the entire time between when taskList.add(new ReadTask(key)); is called and the eventual key.channel().read(buffer) is called, the main thread is still iterating and sees the key as still being selected. After read is called on the key's channel, the key is marked as inactive and doesn't seem to be selected by the selector until another legitimate write from one of the clients prompts the key to be activated again.
Is there a way for me to mark the key so as not to have it added back to the list of selected keys by the selector without calling read? I've tried selector.selectedKeys.remove(key), but this yields a ConcurrentModification Exception.
You should either do the read in the select loop and then start the worker to process the data and prepare the response, or else remove OP_READ from the interestOps of the selection key until the response has been sent.
Related
Suppose I am writing a Java server, which communicates with clients over TCP/IP.
The server uses I/O multiplexing. There is a single thread T0, which waits on selector and handles client connections. For instance, if a connection is ready for read then T0 reads the data from the connection.
Suppose the server has read an incoming request and now it is ready to process it. Since the processing takes time the request is processed in another thread T1 and T0 returns to wait on the selector.
Suppose that T1 has finished the processing and created a response. Now T0 should start writing the response to the client connection. So my question is: how does T1 send the response to T0 ?
The same thread T1 should read, process and return the result to the client.
Here is an outline on how to do it with java nio api without linking the number of threads to the number of clients.
**//Thread T0** //wait for selection keys
...
Iterator it = selector.selectedKeys().iterator( );
while (it.hasNext( )) {
SelectionKey key = (SelectionKey) it.next();
// Is a new connection coming in?
if (key.isAcceptable( )) {
ServerSocketChannel server = (ServerSocketChannel) key.channel();
SocketChannel channel = server.accept()
// Set the new channel nonblocking
channel.configureBlocking (false);
// Register it with the selector
channel.register (selector, SelectionKey.OP_READ);
}
// Is there data to read on this channel?
if (key.isReadable( )) {
processRequest (key);
}
it.remove( );
}
...
ExecutorService service = Executors.newFixedThreadPool(50);
...
void processRequest(final SelectionKey key) {
**//Thread T1-T50** //deal with request
executorService.submit(new Runnable() {
SocketChannel channel = (SocketChannel) key.channel();
//read data from channel, process it and write back to the channel.
});
)
}
I recommend using a server thread that does nothing but blocking on ServerSocket.accept() and as soon as it accepts a connection, submits it to an ExecutorService. While in theory, you can have any number of threads, I would not do it since it makes your application vulnerable to DoS attacks. Instead, limit the maximum size of your thread pool and have it degrade gracefully if the server load goes over the top.
There is actually a small example of how to do this in the documentation for the ExecutorService.
Update: I might have misunderstood your question. As I understand now, you are aware of the solution suggested above but want to use synchronous multiplexing on purpose.
It would help to understand what kind of service your server provides and what is likely the limiting factor (CPU, disk I/O, networking, …).
You could assign a unique request ID to every incoming connection and insert a handler object into a map under that ID. Then, if a connection becomes ready, the network thread picks the respective handler and asks it to accept a certain amount of input / produce a certain amount of output. Whether this is applicable to your situation will of course depend on the service your server offers.
Suppose I am writing a server. The server accepts clients connections, reads requests from the network, process them and sends back the results. Suppose also that I would like to handle all that socket stuff manually (just as an exercise).
I would like to have a single thread to handle sockets with java.nio non-blocking API. When it reads a request completely it starts processing it asynchronously (using Future or passing the request to another thread) and returns to the selector immediately.
When the processing finishes the "socket thread" should receive the response to send it back over the sockets to the client. However I do not see how to do that.
Does it mean that the design above is wrong ? How would you suggest implement the server with java.nio ?
When a request is put in the queue, and the selector thread is in selector.select(), call selector.wakeup(). The selector thread executes loop like this:
while (selector.isOpen() && !Thread.interrupted()) {
for (;;) {
Request r=queue.poll(); // requests can be both to read and write
if (r==null) {
break;
}
processRequest(r);
}
selector.select(); // wait for next event
// Iterate over the set of keys for which events are available
Iterator<SelectionKey> selectedKeys = selector.selectedKeys().iterator();
while (selectedKeys.hasNext()) {
SelectionKey key = selectedKeys.next();
selectedKeys.remove();
processKey(key);
}
}
I wrote a multithreaded gameserver application which handles multiple simultaneous connections using NIO. Unfortunately this server generates full CPU load on one core as soon as the first user connects, even when that user is not actually sending or receiving any data.
Below is the code of my network handling thread (abbreviated to the essential parts for readability). The class ClientHandler is my own class which does the network abstraction for the game mechanics. All other classes in the example below are from java.nio.
As you can see it uses a while(true) loop. My theory about it is that when a key is writable, selector.select() will return immediately and clientHandler.writeToChannel() is called. But when the handler returns without writing anything, the key will stay writable. Then select is called again immediately and returns immediately. So I got a busy spin.
Is there a way to design the network handling loop in a way that it sleeps as long as there is no data to send by the clientHandlers? Note that low latency is critical for my use-case, so I can not just let it sleep an arbitrary number of ms when no handlers have data.
ServerSocketChannel server = ServerSocketChannel.open();
server.configureBlocking(false);
server.socket().bind(new InetSocketAddress(port));
Selector selector = Selector.open();
server.register(selector, SelectionKey.OP_ACCEPT);
// wait for connections
while(true)
{
// Wait for next set of client connections
selector.select();
Set<SelectionKey> keys = selector.selectedKeys();
Iterator<SelectionKey> i = keys.iterator();
while (i.hasNext()) {
SelectionKey key = i.next();
i.remove();
if (key.isAcceptable()) {
SocketChannel clientChannel = server.accept();
clientChannel.configureBlocking(false);
clientChannel.socket().setTcpNoDelay(true);
clientChannel.socket().setTrafficClass(IPTOS_LOWDELAY);
SelectionKey clientKey = clientChannel.register(selector, SelectionKey.OP_READ | SelectionKey.OP_WRITE);
ClientHandler clientHanlder = new ClientHandler(clientChannel);
clientKey.attach(clientHandler);
}
if (key.isReadable()) {
// get connection handler for this key and tell it to process data
ClientHandler clientHandler = (ClientHandler) key.attachment();
clientHandler.readFromChannel();
}
if (key.isWritable()) {
// get connection handler and tell it to send any data it has cached
ClientHandler clientHandler = (ClientHandler) key.attachment();
clientHandler.writeToChannel();
}
if (!key.isValid()) {
ClientHandler clientHandler = (ClientHandler) key.attachment();
clientHandler.disconnect();
}
}
}
SelectionKey clientKey = clientChannel.register(selector, SelectionKey.OP_READ | SelectionKey.OP_WRITE);
The problem is here. SocketChannels are almost always writable, unless the socket send buffer is full. Ergo they should normally not be registered for OP_WRITE: otherwise your selector loop will spin. They should only be so registered if:
there is something to write, and
a prior write() has returned zero.
I don't see any reason why the reading and writing must happen with the same selector. I would use one selector in a thread for read/accept operations and it will always be blocking until new data arrives.
Then, use a separate thread and selector for writing. You mention you are using a cache to store messages before they are sent on the writable channels. In practice the only time a channel would not be writable is if the kernel's buffer is full, so it will rarely not be writable. A good way to implement this would be to have a dedicated writer thread that is given messages, and sleeping; it can be either interrupt()ed when new messages should be sent, or using a take() on a blocking queue. Whenever a new message arrives, it will unblock, do a select() on all writable keys and send any pending messages; only in rare cases will a message have to remain in the cache since a channel is not writable.
I am trying to learn Java. I would like to implement a simple networked connect 4 game as well as a chat feature.
I want my network logic to be non blocking so after much study I found that SocketChannel is what I am after regrading my needs.
What has not made sense still is the lack of CallBack functions in SocketChannels.. Like one finds in C#.
My query for this time is: How do I deliver the data received to the Chat or Game form (JFrame)?
Some guidance is most welcome.
You need to use a Selector. First create a Selector to receive the events:
Selector selector = Selector.open()
Then you need to register the ServerSocketChannel with the selector:
SelectionKey acceptKey = server.register(selector, SelectionKey.OP_ACCEPT);
Then you need to use the Selector to process events as they come in (you can think of this as the "callback" part of the process:
while(true){
//how many channel keys are available
int available = selector.select();
//select is blocking, but should only return if available is >0, this is more of a sanity check
if(available == 0) continue;
Iterator<SelectionKey> keys = selector.selectedKeys().iterator();
while(keys.hasNext()){
SelectionKey key = keys.next();
keys.remove();
//someone is trying to connect to the server socket
if(key.isAcceptable()) doAccept(key);
//someone is sending us data
else if(key.isReadable()) doRead(key);
//we are trying to (and can) send data
else if(key.isWritable()) doWrite(key);
}
The meat will be in doAccept(), doRead(), and doWrite(). For an accept key the selection key will contain the information to create the new Socket.
doAccept(SelectionKey key){
//create the new socket
SocketChannel socket = ((ServerSocketChannel)key.channel()).accept();
//make it non-blocking as well
socket.configureBlocking(false);
...
//here you would likely have some code to init your game objects / communication protocol, etc. and generate an identifier object (used below).
//and be able to find the socket created above
...
//Since it is non blocking it needs a selector as well, and we register for both read and write events
SelectionKey socketKey = socket.register(selector, SelectionKey.OP_READ|SelectionKey.OP_WRITE);
// so we can identify the events as they come in
socketKey.attach(someSocketIndentifier);
}
The last line adds some object to the key so that the events received from the selector can be attributed to a connection (for example it might be a player in your game). So now you can accept new connections and you will just need to read and write.
doRead(SelectionKey key){
//here we retrieve the key we attached earlier, so we now what to do / wheer the data is coming from
MyIdentifierType myIdentifier = (MyIdentifierType)key.attachment();
//This is then used to get back to the SocketChannel and Read the Data
myIdentifier.readTheData();
}
similarly for write
doWrite(SelectionKey key){
//here we retrieve the key we attached earlier, so we now what to do / wheer the data is coming from
MyIdentifierType myIdentifier = (MyIdentifierType)key.attachment();
//This is then used to get back to the SocketChannel and Read the Data
myIdentifier.getSocketHandler().writePendingData();
}
Reading is fairly straight forward, you just create a ByteBuffer and then call the SocketChannels read(ByteBuffer) (or one of its variants) to get the data ready on the channel until its empty.
Writing is a bit trickier as you will usually want to buffer the data to be written until you recieve the write event:
class MyNetworkClass{
ByteBuffer writeBuffer = ByteBuffer.allocate(1024);
SocketChannel commchannel; //from the server accept processing
...
public void write(byte[] data){
//here the class writeBuffer object is filled with the data
//but it isn't actually sent over the socket
...
}
public void writePendingData(){
//here actually write the data to the socket
commchannel.write(writeBuffer);
}
}
Note that you will need appropriate code to manage the buffer in the class in the event it becomes full, or to modify it appropriately in the write pending method if not all of the data in the buffer is written out to the socket, as well as the various exceptions that can be thrown during the process. Hope this helps to get you started.
Please point/provide me an working example of selector.wakeup(); method between two threads.
I tried to create a simple program where a thread is waiting on selector.select() method. Second thread creates some sockets and tries to register with the selector; on which the first thread is blocked.
Hence I need to use selector's wakeup method, but somehow the first thread isnt doesn't come out of blocking mode.
The javadoc of wakeup method states:
If another thread is currently blocked
in an invocation of the
Selector.select() or
Selector.select(long) methods then
that invocation will return
immediately.
P.S There are few other work-around; one of them is select(timeout) but I am trying to figure out where the mistake is.
The psuedo-code:
FIRST THREAD:
static Selector selector = Selector.open();
while(true) {
int n = selectorGlobal.select();
selectorKeySet = selectorGlobal.selectedKeys().iterator();
while (selectorKeySet.hasNext()) {
selectionKey = selectorKeySet.next();
if (selectionKey.isReadable()) {
//do something
}
if(selectionKey.isAcceptable()) {
//accept
}
}
}
SECOND THREAD:
while (itr.hasNext()) {
data = (String) itr.next();
String IP = data.get(0);
String Port = data.get(1);
SocketChannel socketChannel = SocketChannel.open();
socketChannel.configureBlocking(true);
boolean isConnected = socketChannel.connect(new InetSocketAddress(IP, Port));
ClassName.selector.wakeup();
SelectionKey selectionKey = SelectSockets.registerChannel(ClassName.selector,
socketChannel, SelectionKey.OP_READ);
}
You probably don't want to have the socket from Thread 2 be blocking if you're registering it in a selector (as selectors are meant for non-blocking I/O). I think it's also common practice to let the selector handle the connecting with OP_CONNECT (using SocketChannel.finishConnection()).
It also looks like you could have a potential race condition here. Imagine this series of events:
Thread 1: selector.select()
... time passes ...
Thread 2: Thread1.selector.wakeup()
Thread 1: checks keys for acceptability
Thread 1: checks keys for readibility
Thread 1: loop
Thread 1: selector.select()
Thread 2: try to register in the selector (but it's too late for this select())
I'd suggest having Thread 2 set up a SocketChannel, stash it away somewhere Thread 1 can get at it (make sure to be thread safe when you do this), then wake up the selector, let it check it's existing keys in Thread 1 and have Thread 1 register the new SocketChannel before it calls Selector.select() again.