Suppose I am writing a server. The server accepts clients connections, reads requests from the network, process them and sends back the results. Suppose also that I would like to handle all that socket stuff manually (just as an exercise).
I would like to have a single thread to handle sockets with java.nio non-blocking API. When it reads a request completely it starts processing it asynchronously (using Future or passing the request to another thread) and returns to the selector immediately.
When the processing finishes the "socket thread" should receive the response to send it back over the sockets to the client. However I do not see how to do that.
Does it mean that the design above is wrong ? How would you suggest implement the server with java.nio ?
When a request is put in the queue, and the selector thread is in selector.select(), call selector.wakeup(). The selector thread executes loop like this:
while (selector.isOpen() && !Thread.interrupted()) {
for (;;) {
Request r=queue.poll(); // requests can be both to read and write
if (r==null) {
break;
}
processRequest(r);
}
selector.select(); // wait for next event
// Iterate over the set of keys for which events are available
Iterator<SelectionKey> selectedKeys = selector.selectedKeys().iterator();
while (selectedKeys.hasNext()) {
SelectionKey key = selectedKeys.next();
selectedKeys.remove();
processKey(key);
}
}
Related
I have a Java NIO server which receives data from clients.
When a channel is ready for read i.e key.isReadable() return true read(key) is called to read data.
Currently I am using a single read buffer for all channels and in read() method , I clear the buffer and read into it and then finally put into a byte array , supposing that I will get all data in one shot.
But let's say I do not get complete data in one shot(I have special characters at data ending to detect).
Problem :
So now how to keep this partial data with channel or how to deal with partial read problem ? or globally ?
I read somewhere attachments are not good.
Take a look at the Reactor pattern. Here is a link to basic implementation by professor Doug Lea:
http://gee.cs.oswego.edu/dl/cpjslides/nio.pdf
The idea is to have single reactor thread which blocks on Selector call. Once there are IO events ready, reactor thread dispatches the events to appropriate handlers.
In pdf above, there is inner class Acceptor within Reactor which accepts new connections.
Author uses single handler for read and write events and maintains state of this handler. I prefer to have separate handlers for reads and writes but this is not as easy to work with as with 'state machine'. There can be only one Attachment per event, so some kind of injection is needed to switch read/write handlers.
To maintain state between subsequent read/writes you will have to do couple of things:
Introduce custom protocol which tells you when the message is fully read
Have timeout or cleanup mechanism for stale connections
Maintain client specific sessions
So, you can do something like this:
public class Reactor implements Runnable{
Selector selector = Selector.open();
ServerSocketChannel serverSocketChannel = ServerSocketChannel.open();
public Reactor(int port) throws IOException {
serverSocketChannel.socket().bind(new InetSocketAddress(port));
serverSocketChannel.configureBlocking(false);
// let Reactor handle new connection events
registerAcceptor();
}
/**
* Registers Acceptor as handler for new client connections.
*
* #throws ClosedChannelException
*/
private void registerAcceptor() throws ClosedChannelException {
SelectionKey selectionKey0 = serverSocketChannel.register(selector, SelectionKey.OP_ACCEPT);
selectionKey0.attach(new Acceptor());
}
#Override
public void run(){
while(!Thread.interrupted()){
startReactorLoop();
}
}
private void startReactorLoop() {
try {
// wait for new events for each registered or new clients
selector.select();
// get selection keys for pending events
Set<SelectionKey> selectedKeys = selector.selectedKeys();
Iterator<SelectionKey> selectedKeysIterator = selectedKeys.iterator();
while (selectedKeysIterator.hasNext()) {
// dispatch even to handler for the given key
dispatch(selectedKeysIterator.next());
// remove dispatched key from the collection
selectedKeysIterator.remove();
}
} catch (IOException e) {
// TODO add handling of this exception
e.printStackTrace();
}
}
private void dispatch(SelectionKey interestedEvent) {
if (interestedEvent.attachment() != null) {
EventHandler handler = (EventHandler) interestedEvent.attachment();
handler.processEvent();
}
}
private class Acceptor implements EventHandler {
#Override
public void processEvent() {
try {
SocketChannel clientConnection = serverSocketChannel.accept();
if (clientConnection != null) {
registerChannel(clientConnection);
}
} catch (IOException e) {e.printStackTrace();}
}
/**
* Save Channel - key association - in Map perhaps.
* This is required for subsequent/partial reads/writes
*/
private void registerChannel(SocketChannel clientChannel) {
// notify injection mechanism of new connection (so it can activate Read Handler)
}
Once read event is handled, notify injection mechanism that write handler can be injected.
New instances of read and write handlers are created by the injection mechanism once, when new Connection is available. This injection mechanism switches handlers as needed. Lookup of handlers for each Channel is done from the Map that is filled at the connection Acceptance by the method `registerChannel().
Read and write handlers have ByteBuffer instances, and since each Socket Channel has its own pair of handlers, you can now maintain state between partial reads and writes.
Two tips to improve performance:
Try to do first read immediately when connection is accepted. Only if you don't read enough data as defined by header in your custom protocol, register Channel interest for read events.
Try to do write first without registering interest for write events and only if you don't write all the data, register interest for
write.
This will reduce number of Selector wakeups.
Something like this:
SocketChannel socketChannel;
byte[] outData;
final static int MAX_OUTPUT = 1024;
ByteBuffer output = ByteBuffer.allocate(MAX_OUTPUT);
// if message was not written fully
if (socketChannel.write(output) < messageSize()) {
// register interest for write event
SelectionKey selectionKey = socketChannel.register(selector, SelectionKey.OP_WRITE);
selectionKey.attach(writeHandler);
selector.wakeup();
}
Finally, there should be timed Task which checks if Connections are still alive/SelectionKeys are canceled. If client breaks TCP connection, server will usually not know of this. As a result, there will be number of Event handlers in memory, bind as Attachments to stale connections which will result with memory leak.
This is the reason why you may say Attachments are not good, but the issue can be dealt with.
To deal with this here are two simple ways:
TCP keep alive could be enabled
periodic task could check timestamp of last activity on the given Channel. If it is idle for to long, server should terminate connection.
There's an ancient and very inaccurate NIO blog from someone at Amazon where it is wrongly asserted that key attachments are memory leaks. Complete and utter BS. Not even logical. This is also the one where he asserts you need all kinds of supplementary queues. Never had to do that yet, in about 13 years of NIO.
What you need is a ByteBuffer per channel, or possibly two, one for read and one for write. You can store a single one as the attachment itself: if you want two, or have other data to store, you need to define yourself a Session class that contains both buffers and whatever else you want to associate with the channel, for example client credentials, and use the Session object as the attachment.
You really can't get very far in NIO with a single buffer for all channels.
Suppose I am writing a Java server, which communicates with clients over TCP/IP.
The server uses I/O multiplexing. There is a single thread T0, which waits on selector and handles client connections. For instance, if a connection is ready for read then T0 reads the data from the connection.
Suppose the server has read an incoming request and now it is ready to process it. Since the processing takes time the request is processed in another thread T1 and T0 returns to wait on the selector.
Suppose that T1 has finished the processing and created a response. Now T0 should start writing the response to the client connection. So my question is: how does T1 send the response to T0 ?
The same thread T1 should read, process and return the result to the client.
Here is an outline on how to do it with java nio api without linking the number of threads to the number of clients.
**//Thread T0** //wait for selection keys
...
Iterator it = selector.selectedKeys().iterator( );
while (it.hasNext( )) {
SelectionKey key = (SelectionKey) it.next();
// Is a new connection coming in?
if (key.isAcceptable( )) {
ServerSocketChannel server = (ServerSocketChannel) key.channel();
SocketChannel channel = server.accept()
// Set the new channel nonblocking
channel.configureBlocking (false);
// Register it with the selector
channel.register (selector, SelectionKey.OP_READ);
}
// Is there data to read on this channel?
if (key.isReadable( )) {
processRequest (key);
}
it.remove( );
}
...
ExecutorService service = Executors.newFixedThreadPool(50);
...
void processRequest(final SelectionKey key) {
**//Thread T1-T50** //deal with request
executorService.submit(new Runnable() {
SocketChannel channel = (SocketChannel) key.channel();
//read data from channel, process it and write back to the channel.
});
)
}
I recommend using a server thread that does nothing but blocking on ServerSocket.accept() and as soon as it accepts a connection, submits it to an ExecutorService. While in theory, you can have any number of threads, I would not do it since it makes your application vulnerable to DoS attacks. Instead, limit the maximum size of your thread pool and have it degrade gracefully if the server load goes over the top.
There is actually a small example of how to do this in the documentation for the ExecutorService.
Update: I might have misunderstood your question. As I understand now, you are aware of the solution suggested above but want to use synchronous multiplexing on purpose.
It would help to understand what kind of service your server provides and what is likely the limiting factor (CPU, disk I/O, networking, …).
You could assign a unique request ID to every incoming connection and insert a handler object into a map under that ID. Then, if a connection becomes ready, the network thread picks the respective handler and asks it to accept a certain amount of input / produce a certain amount of output. Whether this is applicable to your situation will of course depend on the service your server offers.
I wrote a multithreaded gameserver application which handles multiple simultaneous connections using NIO. Unfortunately this server generates full CPU load on one core as soon as the first user connects, even when that user is not actually sending or receiving any data.
Below is the code of my network handling thread (abbreviated to the essential parts for readability). The class ClientHandler is my own class which does the network abstraction for the game mechanics. All other classes in the example below are from java.nio.
As you can see it uses a while(true) loop. My theory about it is that when a key is writable, selector.select() will return immediately and clientHandler.writeToChannel() is called. But when the handler returns without writing anything, the key will stay writable. Then select is called again immediately and returns immediately. So I got a busy spin.
Is there a way to design the network handling loop in a way that it sleeps as long as there is no data to send by the clientHandlers? Note that low latency is critical for my use-case, so I can not just let it sleep an arbitrary number of ms when no handlers have data.
ServerSocketChannel server = ServerSocketChannel.open();
server.configureBlocking(false);
server.socket().bind(new InetSocketAddress(port));
Selector selector = Selector.open();
server.register(selector, SelectionKey.OP_ACCEPT);
// wait for connections
while(true)
{
// Wait for next set of client connections
selector.select();
Set<SelectionKey> keys = selector.selectedKeys();
Iterator<SelectionKey> i = keys.iterator();
while (i.hasNext()) {
SelectionKey key = i.next();
i.remove();
if (key.isAcceptable()) {
SocketChannel clientChannel = server.accept();
clientChannel.configureBlocking(false);
clientChannel.socket().setTcpNoDelay(true);
clientChannel.socket().setTrafficClass(IPTOS_LOWDELAY);
SelectionKey clientKey = clientChannel.register(selector, SelectionKey.OP_READ | SelectionKey.OP_WRITE);
ClientHandler clientHanlder = new ClientHandler(clientChannel);
clientKey.attach(clientHandler);
}
if (key.isReadable()) {
// get connection handler for this key and tell it to process data
ClientHandler clientHandler = (ClientHandler) key.attachment();
clientHandler.readFromChannel();
}
if (key.isWritable()) {
// get connection handler and tell it to send any data it has cached
ClientHandler clientHandler = (ClientHandler) key.attachment();
clientHandler.writeToChannel();
}
if (!key.isValid()) {
ClientHandler clientHandler = (ClientHandler) key.attachment();
clientHandler.disconnect();
}
}
}
SelectionKey clientKey = clientChannel.register(selector, SelectionKey.OP_READ | SelectionKey.OP_WRITE);
The problem is here. SocketChannels are almost always writable, unless the socket send buffer is full. Ergo they should normally not be registered for OP_WRITE: otherwise your selector loop will spin. They should only be so registered if:
there is something to write, and
a prior write() has returned zero.
I don't see any reason why the reading and writing must happen with the same selector. I would use one selector in a thread for read/accept operations and it will always be blocking until new data arrives.
Then, use a separate thread and selector for writing. You mention you are using a cache to store messages before they are sent on the writable channels. In practice the only time a channel would not be writable is if the kernel's buffer is full, so it will rarely not be writable. A good way to implement this would be to have a dedicated writer thread that is given messages, and sleeping; it can be either interrupt()ed when new messages should be sent, or using a take() on a blocking queue. Whenever a new message arrives, it will unblock, do a select() on all writable keys and send any pending messages; only in rare cases will a message have to remain in the cache since a channel is not writable.
I'm new to socket programming and programming a Java UDP simple client-server application. I'm writing a time/date server client. The client can ask the server for the time and date and it waits for a response. Also, every minute, the server updates all clients with the current time. The client needs to
be able to initiate contact with the server and wait for a message back
listen for periodic updates from the server
How can I do this using a single DatagramSocket?
I was thinking of creating two threads: one that listens and one that writes. The problem is that in the case that the client initiates contact with the server, it needs to wait to receive an acknowledgement from the server. So, the writing thread also needs to listen for packets from the server sometimes. But in this case, I have two threads listening and the wrong thread will get the acknowledgement.
Is there a way to specify which thread gets the input? Or is there some other way to solve this problem?
I've been searching for this answer but unable to find it. The closest I've found is Java sockets: can you send from one thread and receive on another?
If there is just one writer thread then it could send the request and go into a wait loop. The listener thread would then get the response, add it to a shared variable (maybe an AtomicReference), and then notify the writer that response has been received.
// both write and listener threads will need to share this
private final AtomicReference<Response> responseRef =
new AtomicReference<Response>();
...
// writer-thread
writeRequest(request);
synchronize (responseRef) {
while (responseRef.get() == null) {
// maybe put a timeout here
responseRef.wait();
}
}
processResponse(response);
...
// listener-thread
Response response = readResponse();
synchronize (responseRef) {
responseRef.set(response);
responseRef.notify();
}
If you have multiple writers or multiple requests being sent at the same time then it gets more complicated. You'll need to send some sort of unique id with each request and return it in the response. Then the response thread can match up the request with the response. You'd need a ConcurrentHashMap or other shared collection so that the responder can match up the particular request, add the response, and notify the appropriate waiting thread.
I am trying to learn Java. I would like to implement a simple networked connect 4 game as well as a chat feature.
I want my network logic to be non blocking so after much study I found that SocketChannel is what I am after regrading my needs.
What has not made sense still is the lack of CallBack functions in SocketChannels.. Like one finds in C#.
My query for this time is: How do I deliver the data received to the Chat or Game form (JFrame)?
Some guidance is most welcome.
You need to use a Selector. First create a Selector to receive the events:
Selector selector = Selector.open()
Then you need to register the ServerSocketChannel with the selector:
SelectionKey acceptKey = server.register(selector, SelectionKey.OP_ACCEPT);
Then you need to use the Selector to process events as they come in (you can think of this as the "callback" part of the process:
while(true){
//how many channel keys are available
int available = selector.select();
//select is blocking, but should only return if available is >0, this is more of a sanity check
if(available == 0) continue;
Iterator<SelectionKey> keys = selector.selectedKeys().iterator();
while(keys.hasNext()){
SelectionKey key = keys.next();
keys.remove();
//someone is trying to connect to the server socket
if(key.isAcceptable()) doAccept(key);
//someone is sending us data
else if(key.isReadable()) doRead(key);
//we are trying to (and can) send data
else if(key.isWritable()) doWrite(key);
}
The meat will be in doAccept(), doRead(), and doWrite(). For an accept key the selection key will contain the information to create the new Socket.
doAccept(SelectionKey key){
//create the new socket
SocketChannel socket = ((ServerSocketChannel)key.channel()).accept();
//make it non-blocking as well
socket.configureBlocking(false);
...
//here you would likely have some code to init your game objects / communication protocol, etc. and generate an identifier object (used below).
//and be able to find the socket created above
...
//Since it is non blocking it needs a selector as well, and we register for both read and write events
SelectionKey socketKey = socket.register(selector, SelectionKey.OP_READ|SelectionKey.OP_WRITE);
// so we can identify the events as they come in
socketKey.attach(someSocketIndentifier);
}
The last line adds some object to the key so that the events received from the selector can be attributed to a connection (for example it might be a player in your game). So now you can accept new connections and you will just need to read and write.
doRead(SelectionKey key){
//here we retrieve the key we attached earlier, so we now what to do / wheer the data is coming from
MyIdentifierType myIdentifier = (MyIdentifierType)key.attachment();
//This is then used to get back to the SocketChannel and Read the Data
myIdentifier.readTheData();
}
similarly for write
doWrite(SelectionKey key){
//here we retrieve the key we attached earlier, so we now what to do / wheer the data is coming from
MyIdentifierType myIdentifier = (MyIdentifierType)key.attachment();
//This is then used to get back to the SocketChannel and Read the Data
myIdentifier.getSocketHandler().writePendingData();
}
Reading is fairly straight forward, you just create a ByteBuffer and then call the SocketChannels read(ByteBuffer) (or one of its variants) to get the data ready on the channel until its empty.
Writing is a bit trickier as you will usually want to buffer the data to be written until you recieve the write event:
class MyNetworkClass{
ByteBuffer writeBuffer = ByteBuffer.allocate(1024);
SocketChannel commchannel; //from the server accept processing
...
public void write(byte[] data){
//here the class writeBuffer object is filled with the data
//but it isn't actually sent over the socket
...
}
public void writePendingData(){
//here actually write the data to the socket
commchannel.write(writeBuffer);
}
}
Note that you will need appropriate code to manage the buffer in the class in the event it becomes full, or to modify it appropriately in the write pending method if not all of the data in the buffer is written out to the socket, as well as the various exceptions that can be thrown during the process. Hope this helps to get you started.