I wrote a multithreaded gameserver application which handles multiple simultaneous connections using NIO. Unfortunately this server generates full CPU load on one core as soon as the first user connects, even when that user is not actually sending or receiving any data.
Below is the code of my network handling thread (abbreviated to the essential parts for readability). The class ClientHandler is my own class which does the network abstraction for the game mechanics. All other classes in the example below are from java.nio.
As you can see it uses a while(true) loop. My theory about it is that when a key is writable, selector.select() will return immediately and clientHandler.writeToChannel() is called. But when the handler returns without writing anything, the key will stay writable. Then select is called again immediately and returns immediately. So I got a busy spin.
Is there a way to design the network handling loop in a way that it sleeps as long as there is no data to send by the clientHandlers? Note that low latency is critical for my use-case, so I can not just let it sleep an arbitrary number of ms when no handlers have data.
ServerSocketChannel server = ServerSocketChannel.open();
server.configureBlocking(false);
server.socket().bind(new InetSocketAddress(port));
Selector selector = Selector.open();
server.register(selector, SelectionKey.OP_ACCEPT);
// wait for connections
while(true)
{
// Wait for next set of client connections
selector.select();
Set<SelectionKey> keys = selector.selectedKeys();
Iterator<SelectionKey> i = keys.iterator();
while (i.hasNext()) {
SelectionKey key = i.next();
i.remove();
if (key.isAcceptable()) {
SocketChannel clientChannel = server.accept();
clientChannel.configureBlocking(false);
clientChannel.socket().setTcpNoDelay(true);
clientChannel.socket().setTrafficClass(IPTOS_LOWDELAY);
SelectionKey clientKey = clientChannel.register(selector, SelectionKey.OP_READ | SelectionKey.OP_WRITE);
ClientHandler clientHanlder = new ClientHandler(clientChannel);
clientKey.attach(clientHandler);
}
if (key.isReadable()) {
// get connection handler for this key and tell it to process data
ClientHandler clientHandler = (ClientHandler) key.attachment();
clientHandler.readFromChannel();
}
if (key.isWritable()) {
// get connection handler and tell it to send any data it has cached
ClientHandler clientHandler = (ClientHandler) key.attachment();
clientHandler.writeToChannel();
}
if (!key.isValid()) {
ClientHandler clientHandler = (ClientHandler) key.attachment();
clientHandler.disconnect();
}
}
}
SelectionKey clientKey = clientChannel.register(selector, SelectionKey.OP_READ | SelectionKey.OP_WRITE);
The problem is here. SocketChannels are almost always writable, unless the socket send buffer is full. Ergo they should normally not be registered for OP_WRITE: otherwise your selector loop will spin. They should only be so registered if:
there is something to write, and
a prior write() has returned zero.
I don't see any reason why the reading and writing must happen with the same selector. I would use one selector in a thread for read/accept operations and it will always be blocking until new data arrives.
Then, use a separate thread and selector for writing. You mention you are using a cache to store messages before they are sent on the writable channels. In practice the only time a channel would not be writable is if the kernel's buffer is full, so it will rarely not be writable. A good way to implement this would be to have a dedicated writer thread that is given messages, and sleeping; it can be either interrupt()ed when new messages should be sent, or using a take() on a blocking queue. Whenever a new message arrives, it will unblock, do a select() on all writable keys and send any pending messages; only in rare cases will a message have to remain in the cache since a channel is not writable.
Related
I am writing an application in which the main thread handles many connections through the use of a Selector and SelectionKeys. I have run into some trouble with a race condition when attempting to pass tasks off to the worker threads.
My main loop looks like this:
selector = Selector.open(); //Create selector
serverSocketChannel = ServerSocketChannel.open(); //Create socket channel, configure blocking, and bind
serverSocketChannel.configureBlocking(false);
serverSocketChannel.bind(new InetSocketAddress(PORT));
serverSocketChannel.register(selector, SelectionKey.OP_ACCEPT); //Register channel to selector
ByteBuffer buffer = ByteBuffer.allocate(8000);
while(true){
selector.select();
Iterator<SelectionKey> iterator = selector.selectedKeys().iterator();
while(iterator.hasNext()){
SelectionKey key = iterator.next();
if(key.isAcceptable()){
SocketChannel socketChannel = serverSocketChannel.accept();
socketChannel.configureBlocking(false);
socketChannel.register(selector, SelectionKey.OP_READ);
}
if(key.isReadable()){
taskList.add(new ReadTask(key));
}
if(key.isWritable()){
}
iterator.remove();
}
}
The idea here is that when a client tries to send data to the server, it receives a key with OP_READ interest and then creates a task with that key so that the threadpool can handle the read so as not to block the main thread.
The problem is that calling this loop continues during the process of the key being passed to a worker thread, and the entire time between when taskList.add(new ReadTask(key)); is called and the eventual key.channel().read(buffer) is called, the main thread is still iterating and sees the key as still being selected. After read is called on the key's channel, the key is marked as inactive and doesn't seem to be selected by the selector until another legitimate write from one of the clients prompts the key to be activated again.
Is there a way for me to mark the key so as not to have it added back to the list of selected keys by the selector without calling read? I've tried selector.selectedKeys.remove(key), but this yields a ConcurrentModification Exception.
You should either do the read in the select loop and then start the worker to process the data and prepare the response, or else remove OP_READ from the interestOps of the selection key until the response has been sent.
I have a Java NIO server which receives data from clients.
When a channel is ready for read i.e key.isReadable() return true read(key) is called to read data.
Currently I am using a single read buffer for all channels and in read() method , I clear the buffer and read into it and then finally put into a byte array , supposing that I will get all data in one shot.
But let's say I do not get complete data in one shot(I have special characters at data ending to detect).
Problem :
So now how to keep this partial data with channel or how to deal with partial read problem ? or globally ?
I read somewhere attachments are not good.
Take a look at the Reactor pattern. Here is a link to basic implementation by professor Doug Lea:
http://gee.cs.oswego.edu/dl/cpjslides/nio.pdf
The idea is to have single reactor thread which blocks on Selector call. Once there are IO events ready, reactor thread dispatches the events to appropriate handlers.
In pdf above, there is inner class Acceptor within Reactor which accepts new connections.
Author uses single handler for read and write events and maintains state of this handler. I prefer to have separate handlers for reads and writes but this is not as easy to work with as with 'state machine'. There can be only one Attachment per event, so some kind of injection is needed to switch read/write handlers.
To maintain state between subsequent read/writes you will have to do couple of things:
Introduce custom protocol which tells you when the message is fully read
Have timeout or cleanup mechanism for stale connections
Maintain client specific sessions
So, you can do something like this:
public class Reactor implements Runnable{
Selector selector = Selector.open();
ServerSocketChannel serverSocketChannel = ServerSocketChannel.open();
public Reactor(int port) throws IOException {
serverSocketChannel.socket().bind(new InetSocketAddress(port));
serverSocketChannel.configureBlocking(false);
// let Reactor handle new connection events
registerAcceptor();
}
/**
* Registers Acceptor as handler for new client connections.
*
* #throws ClosedChannelException
*/
private void registerAcceptor() throws ClosedChannelException {
SelectionKey selectionKey0 = serverSocketChannel.register(selector, SelectionKey.OP_ACCEPT);
selectionKey0.attach(new Acceptor());
}
#Override
public void run(){
while(!Thread.interrupted()){
startReactorLoop();
}
}
private void startReactorLoop() {
try {
// wait for new events for each registered or new clients
selector.select();
// get selection keys for pending events
Set<SelectionKey> selectedKeys = selector.selectedKeys();
Iterator<SelectionKey> selectedKeysIterator = selectedKeys.iterator();
while (selectedKeysIterator.hasNext()) {
// dispatch even to handler for the given key
dispatch(selectedKeysIterator.next());
// remove dispatched key from the collection
selectedKeysIterator.remove();
}
} catch (IOException e) {
// TODO add handling of this exception
e.printStackTrace();
}
}
private void dispatch(SelectionKey interestedEvent) {
if (interestedEvent.attachment() != null) {
EventHandler handler = (EventHandler) interestedEvent.attachment();
handler.processEvent();
}
}
private class Acceptor implements EventHandler {
#Override
public void processEvent() {
try {
SocketChannel clientConnection = serverSocketChannel.accept();
if (clientConnection != null) {
registerChannel(clientConnection);
}
} catch (IOException e) {e.printStackTrace();}
}
/**
* Save Channel - key association - in Map perhaps.
* This is required for subsequent/partial reads/writes
*/
private void registerChannel(SocketChannel clientChannel) {
// notify injection mechanism of new connection (so it can activate Read Handler)
}
Once read event is handled, notify injection mechanism that write handler can be injected.
New instances of read and write handlers are created by the injection mechanism once, when new Connection is available. This injection mechanism switches handlers as needed. Lookup of handlers for each Channel is done from the Map that is filled at the connection Acceptance by the method `registerChannel().
Read and write handlers have ByteBuffer instances, and since each Socket Channel has its own pair of handlers, you can now maintain state between partial reads and writes.
Two tips to improve performance:
Try to do first read immediately when connection is accepted. Only if you don't read enough data as defined by header in your custom protocol, register Channel interest for read events.
Try to do write first without registering interest for write events and only if you don't write all the data, register interest for
write.
This will reduce number of Selector wakeups.
Something like this:
SocketChannel socketChannel;
byte[] outData;
final static int MAX_OUTPUT = 1024;
ByteBuffer output = ByteBuffer.allocate(MAX_OUTPUT);
// if message was not written fully
if (socketChannel.write(output) < messageSize()) {
// register interest for write event
SelectionKey selectionKey = socketChannel.register(selector, SelectionKey.OP_WRITE);
selectionKey.attach(writeHandler);
selector.wakeup();
}
Finally, there should be timed Task which checks if Connections are still alive/SelectionKeys are canceled. If client breaks TCP connection, server will usually not know of this. As a result, there will be number of Event handlers in memory, bind as Attachments to stale connections which will result with memory leak.
This is the reason why you may say Attachments are not good, but the issue can be dealt with.
To deal with this here are two simple ways:
TCP keep alive could be enabled
periodic task could check timestamp of last activity on the given Channel. If it is idle for to long, server should terminate connection.
There's an ancient and very inaccurate NIO blog from someone at Amazon where it is wrongly asserted that key attachments are memory leaks. Complete and utter BS. Not even logical. This is also the one where he asserts you need all kinds of supplementary queues. Never had to do that yet, in about 13 years of NIO.
What you need is a ByteBuffer per channel, or possibly two, one for read and one for write. You can store a single one as the attachment itself: if you want two, or have other data to store, you need to define yourself a Session class that contains both buffers and whatever else you want to associate with the channel, for example client credentials, and use the Session object as the attachment.
You really can't get very far in NIO with a single buffer for all channels.
Suppose I am writing a Java server, which communicates with clients over TCP/IP.
The server uses I/O multiplexing. There is a single thread T0, which waits on selector and handles client connections. For instance, if a connection is ready for read then T0 reads the data from the connection.
Suppose the server has read an incoming request and now it is ready to process it. Since the processing takes time the request is processed in another thread T1 and T0 returns to wait on the selector.
Suppose that T1 has finished the processing and created a response. Now T0 should start writing the response to the client connection. So my question is: how does T1 send the response to T0 ?
The same thread T1 should read, process and return the result to the client.
Here is an outline on how to do it with java nio api without linking the number of threads to the number of clients.
**//Thread T0** //wait for selection keys
...
Iterator it = selector.selectedKeys().iterator( );
while (it.hasNext( )) {
SelectionKey key = (SelectionKey) it.next();
// Is a new connection coming in?
if (key.isAcceptable( )) {
ServerSocketChannel server = (ServerSocketChannel) key.channel();
SocketChannel channel = server.accept()
// Set the new channel nonblocking
channel.configureBlocking (false);
// Register it with the selector
channel.register (selector, SelectionKey.OP_READ);
}
// Is there data to read on this channel?
if (key.isReadable( )) {
processRequest (key);
}
it.remove( );
}
...
ExecutorService service = Executors.newFixedThreadPool(50);
...
void processRequest(final SelectionKey key) {
**//Thread T1-T50** //deal with request
executorService.submit(new Runnable() {
SocketChannel channel = (SocketChannel) key.channel();
//read data from channel, process it and write back to the channel.
});
)
}
I recommend using a server thread that does nothing but blocking on ServerSocket.accept() and as soon as it accepts a connection, submits it to an ExecutorService. While in theory, you can have any number of threads, I would not do it since it makes your application vulnerable to DoS attacks. Instead, limit the maximum size of your thread pool and have it degrade gracefully if the server load goes over the top.
There is actually a small example of how to do this in the documentation for the ExecutorService.
Update: I might have misunderstood your question. As I understand now, you are aware of the solution suggested above but want to use synchronous multiplexing on purpose.
It would help to understand what kind of service your server provides and what is likely the limiting factor (CPU, disk I/O, networking, …).
You could assign a unique request ID to every incoming connection and insert a handler object into a map under that ID. Then, if a connection becomes ready, the network thread picks the respective handler and asks it to accept a certain amount of input / produce a certain amount of output. Whether this is applicable to your situation will of course depend on the service your server offers.
Suppose I am writing a server. The server accepts clients connections, reads requests from the network, process them and sends back the results. Suppose also that I would like to handle all that socket stuff manually (just as an exercise).
I would like to have a single thread to handle sockets with java.nio non-blocking API. When it reads a request completely it starts processing it asynchronously (using Future or passing the request to another thread) and returns to the selector immediately.
When the processing finishes the "socket thread" should receive the response to send it back over the sockets to the client. However I do not see how to do that.
Does it mean that the design above is wrong ? How would you suggest implement the server with java.nio ?
When a request is put in the queue, and the selector thread is in selector.select(), call selector.wakeup(). The selector thread executes loop like this:
while (selector.isOpen() && !Thread.interrupted()) {
for (;;) {
Request r=queue.poll(); // requests can be both to read and write
if (r==null) {
break;
}
processRequest(r);
}
selector.select(); // wait for next event
// Iterate over the set of keys for which events are available
Iterator<SelectionKey> selectedKeys = selector.selectedKeys().iterator();
while (selectedKeys.hasNext()) {
SelectionKey key = selectedKeys.next();
selectedKeys.remove();
processKey(key);
}
}
I am trying to learn Java. I would like to implement a simple networked connect 4 game as well as a chat feature.
I want my network logic to be non blocking so after much study I found that SocketChannel is what I am after regrading my needs.
What has not made sense still is the lack of CallBack functions in SocketChannels.. Like one finds in C#.
My query for this time is: How do I deliver the data received to the Chat or Game form (JFrame)?
Some guidance is most welcome.
You need to use a Selector. First create a Selector to receive the events:
Selector selector = Selector.open()
Then you need to register the ServerSocketChannel with the selector:
SelectionKey acceptKey = server.register(selector, SelectionKey.OP_ACCEPT);
Then you need to use the Selector to process events as they come in (you can think of this as the "callback" part of the process:
while(true){
//how many channel keys are available
int available = selector.select();
//select is blocking, but should only return if available is >0, this is more of a sanity check
if(available == 0) continue;
Iterator<SelectionKey> keys = selector.selectedKeys().iterator();
while(keys.hasNext()){
SelectionKey key = keys.next();
keys.remove();
//someone is trying to connect to the server socket
if(key.isAcceptable()) doAccept(key);
//someone is sending us data
else if(key.isReadable()) doRead(key);
//we are trying to (and can) send data
else if(key.isWritable()) doWrite(key);
}
The meat will be in doAccept(), doRead(), and doWrite(). For an accept key the selection key will contain the information to create the new Socket.
doAccept(SelectionKey key){
//create the new socket
SocketChannel socket = ((ServerSocketChannel)key.channel()).accept();
//make it non-blocking as well
socket.configureBlocking(false);
...
//here you would likely have some code to init your game objects / communication protocol, etc. and generate an identifier object (used below).
//and be able to find the socket created above
...
//Since it is non blocking it needs a selector as well, and we register for both read and write events
SelectionKey socketKey = socket.register(selector, SelectionKey.OP_READ|SelectionKey.OP_WRITE);
// so we can identify the events as they come in
socketKey.attach(someSocketIndentifier);
}
The last line adds some object to the key so that the events received from the selector can be attributed to a connection (for example it might be a player in your game). So now you can accept new connections and you will just need to read and write.
doRead(SelectionKey key){
//here we retrieve the key we attached earlier, so we now what to do / wheer the data is coming from
MyIdentifierType myIdentifier = (MyIdentifierType)key.attachment();
//This is then used to get back to the SocketChannel and Read the Data
myIdentifier.readTheData();
}
similarly for write
doWrite(SelectionKey key){
//here we retrieve the key we attached earlier, so we now what to do / wheer the data is coming from
MyIdentifierType myIdentifier = (MyIdentifierType)key.attachment();
//This is then used to get back to the SocketChannel and Read the Data
myIdentifier.getSocketHandler().writePendingData();
}
Reading is fairly straight forward, you just create a ByteBuffer and then call the SocketChannels read(ByteBuffer) (or one of its variants) to get the data ready on the channel until its empty.
Writing is a bit trickier as you will usually want to buffer the data to be written until you recieve the write event:
class MyNetworkClass{
ByteBuffer writeBuffer = ByteBuffer.allocate(1024);
SocketChannel commchannel; //from the server accept processing
...
public void write(byte[] data){
//here the class writeBuffer object is filled with the data
//but it isn't actually sent over the socket
...
}
public void writePendingData(){
//here actually write the data to the socket
commchannel.write(writeBuffer);
}
}
Note that you will need appropriate code to manage the buffer in the class in the event it becomes full, or to modify it appropriately in the write pending method if not all of the data in the buffer is written out to the socket, as well as the various exceptions that can be thrown during the process. Hope this helps to get you started.