I am trying to learn Java. I would like to implement a simple networked connect 4 game as well as a chat feature.
I want my network logic to be non blocking so after much study I found that SocketChannel is what I am after regrading my needs.
What has not made sense still is the lack of CallBack functions in SocketChannels.. Like one finds in C#.
My query for this time is: How do I deliver the data received to the Chat or Game form (JFrame)?
Some guidance is most welcome.
You need to use a Selector. First create a Selector to receive the events:
Selector selector = Selector.open()
Then you need to register the ServerSocketChannel with the selector:
SelectionKey acceptKey = server.register(selector, SelectionKey.OP_ACCEPT);
Then you need to use the Selector to process events as they come in (you can think of this as the "callback" part of the process:
while(true){
//how many channel keys are available
int available = selector.select();
//select is blocking, but should only return if available is >0, this is more of a sanity check
if(available == 0) continue;
Iterator<SelectionKey> keys = selector.selectedKeys().iterator();
while(keys.hasNext()){
SelectionKey key = keys.next();
keys.remove();
//someone is trying to connect to the server socket
if(key.isAcceptable()) doAccept(key);
//someone is sending us data
else if(key.isReadable()) doRead(key);
//we are trying to (and can) send data
else if(key.isWritable()) doWrite(key);
}
The meat will be in doAccept(), doRead(), and doWrite(). For an accept key the selection key will contain the information to create the new Socket.
doAccept(SelectionKey key){
//create the new socket
SocketChannel socket = ((ServerSocketChannel)key.channel()).accept();
//make it non-blocking as well
socket.configureBlocking(false);
...
//here you would likely have some code to init your game objects / communication protocol, etc. and generate an identifier object (used below).
//and be able to find the socket created above
...
//Since it is non blocking it needs a selector as well, and we register for both read and write events
SelectionKey socketKey = socket.register(selector, SelectionKey.OP_READ|SelectionKey.OP_WRITE);
// so we can identify the events as they come in
socketKey.attach(someSocketIndentifier);
}
The last line adds some object to the key so that the events received from the selector can be attributed to a connection (for example it might be a player in your game). So now you can accept new connections and you will just need to read and write.
doRead(SelectionKey key){
//here we retrieve the key we attached earlier, so we now what to do / wheer the data is coming from
MyIdentifierType myIdentifier = (MyIdentifierType)key.attachment();
//This is then used to get back to the SocketChannel and Read the Data
myIdentifier.readTheData();
}
similarly for write
doWrite(SelectionKey key){
//here we retrieve the key we attached earlier, so we now what to do / wheer the data is coming from
MyIdentifierType myIdentifier = (MyIdentifierType)key.attachment();
//This is then used to get back to the SocketChannel and Read the Data
myIdentifier.getSocketHandler().writePendingData();
}
Reading is fairly straight forward, you just create a ByteBuffer and then call the SocketChannels read(ByteBuffer) (or one of its variants) to get the data ready on the channel until its empty.
Writing is a bit trickier as you will usually want to buffer the data to be written until you recieve the write event:
class MyNetworkClass{
ByteBuffer writeBuffer = ByteBuffer.allocate(1024);
SocketChannel commchannel; //from the server accept processing
...
public void write(byte[] data){
//here the class writeBuffer object is filled with the data
//but it isn't actually sent over the socket
...
}
public void writePendingData(){
//here actually write the data to the socket
commchannel.write(writeBuffer);
}
}
Note that you will need appropriate code to manage the buffer in the class in the event it becomes full, or to modify it appropriately in the write pending method if not all of the data in the buffer is written out to the socket, as well as the various exceptions that can be thrown during the process. Hope this helps to get you started.
Related
I am writing a socket server using java.nio. As I need my server to use 0 threads I am using java.nio.channels.Selector. My code look as follows.
while (iterator.hasNext()) {
SelectionKey key = (SelectionKey) iterator.next();
iterator.remove();
if (!key.isValid()) {
continue;
}
if (key.isAcceptable()) { // Accept client connections
this.acceptClient(key);
} else if (key.isReadable()) { // Read from client
this.read(key);
} else if (key.isWritable()) {
this.write(key);
}
}
private void acceptClient(SelectionKey key) throws IOException {
ServerSocketChannel serverChannel = (ServerSocketChannel) key.channel();
SocketChannel channel = serverChannel.accept();
channel.configureBlocking(false);
SocketAddress clientAddress= channel.getRemoteAddress();
//clients is a Hashmap
clients.put(clientAddress, new Client());
clientConnected(clientAddress.toString());
System.out.println("Connected to: " + clientAddress);
channel.register(this.selector, SelectionKey.OP_READ);
}
As you can see I am creating a new Client object per accepted client. What I need to do is, relevant Client object to handle their own reading and writing.
My approach is to uniquely identify clients with their address and forward it to relevant Client object.
I think using client address to uniquely identify clients is not a good approach. What is the best way to handle this issue?
I think using client address to uniquely identify clients is not a good approach.
Nothing wrong with it. The semantics of TCP/IP guarantee that every accepted socket has a different remote SocketAddress.
But you don't need it, or the Map either. Just save the Client as the attachment of the SelectionKey. That way also the Client will disappear along with the SelectionKey automatically when you close the Channel.
By contrast, changing to an IdentityHashMap<SelectionKey, Client> as suggested elsewhere gives you an opportunity to leak the SelectionKey and therefore its Channel and the Client too.
When you register the Channel with the Selector:
channel.register(this.selector, SelectionKey.OP_READ);
It will return a SelectionKey which you can use later when you select from the Selector.
Use that key to populate an IdentityHashMap<SelectionKey,Client> so you can direct the IO to the correct Client instance. As EJP points out in his answer, this will leak SelectionKeys if not properly cleaned up. If you preferred not to clean those up, you could use a WeakHashMap as well but then you're relying in implicit behavior instead of the explicit behavior of the Selector.
EJP's suggestion to use an attachment is probably the best bet. Although I can imagine a more complicated scenario where you might want to maintain additional attachments and maybe refactoring the attachment to be an encapsulation is of high cost.
I have a Java NIO server which receives data from clients.
When a channel is ready for read i.e key.isReadable() return true read(key) is called to read data.
Currently I am using a single read buffer for all channels and in read() method , I clear the buffer and read into it and then finally put into a byte array , supposing that I will get all data in one shot.
But let's say I do not get complete data in one shot(I have special characters at data ending to detect).
Problem :
So now how to keep this partial data with channel or how to deal with partial read problem ? or globally ?
I read somewhere attachments are not good.
Take a look at the Reactor pattern. Here is a link to basic implementation by professor Doug Lea:
http://gee.cs.oswego.edu/dl/cpjslides/nio.pdf
The idea is to have single reactor thread which blocks on Selector call. Once there are IO events ready, reactor thread dispatches the events to appropriate handlers.
In pdf above, there is inner class Acceptor within Reactor which accepts new connections.
Author uses single handler for read and write events and maintains state of this handler. I prefer to have separate handlers for reads and writes but this is not as easy to work with as with 'state machine'. There can be only one Attachment per event, so some kind of injection is needed to switch read/write handlers.
To maintain state between subsequent read/writes you will have to do couple of things:
Introduce custom protocol which tells you when the message is fully read
Have timeout or cleanup mechanism for stale connections
Maintain client specific sessions
So, you can do something like this:
public class Reactor implements Runnable{
Selector selector = Selector.open();
ServerSocketChannel serverSocketChannel = ServerSocketChannel.open();
public Reactor(int port) throws IOException {
serverSocketChannel.socket().bind(new InetSocketAddress(port));
serverSocketChannel.configureBlocking(false);
// let Reactor handle new connection events
registerAcceptor();
}
/**
* Registers Acceptor as handler for new client connections.
*
* #throws ClosedChannelException
*/
private void registerAcceptor() throws ClosedChannelException {
SelectionKey selectionKey0 = serverSocketChannel.register(selector, SelectionKey.OP_ACCEPT);
selectionKey0.attach(new Acceptor());
}
#Override
public void run(){
while(!Thread.interrupted()){
startReactorLoop();
}
}
private void startReactorLoop() {
try {
// wait for new events for each registered or new clients
selector.select();
// get selection keys for pending events
Set<SelectionKey> selectedKeys = selector.selectedKeys();
Iterator<SelectionKey> selectedKeysIterator = selectedKeys.iterator();
while (selectedKeysIterator.hasNext()) {
// dispatch even to handler for the given key
dispatch(selectedKeysIterator.next());
// remove dispatched key from the collection
selectedKeysIterator.remove();
}
} catch (IOException e) {
// TODO add handling of this exception
e.printStackTrace();
}
}
private void dispatch(SelectionKey interestedEvent) {
if (interestedEvent.attachment() != null) {
EventHandler handler = (EventHandler) interestedEvent.attachment();
handler.processEvent();
}
}
private class Acceptor implements EventHandler {
#Override
public void processEvent() {
try {
SocketChannel clientConnection = serverSocketChannel.accept();
if (clientConnection != null) {
registerChannel(clientConnection);
}
} catch (IOException e) {e.printStackTrace();}
}
/**
* Save Channel - key association - in Map perhaps.
* This is required for subsequent/partial reads/writes
*/
private void registerChannel(SocketChannel clientChannel) {
// notify injection mechanism of new connection (so it can activate Read Handler)
}
Once read event is handled, notify injection mechanism that write handler can be injected.
New instances of read and write handlers are created by the injection mechanism once, when new Connection is available. This injection mechanism switches handlers as needed. Lookup of handlers for each Channel is done from the Map that is filled at the connection Acceptance by the method `registerChannel().
Read and write handlers have ByteBuffer instances, and since each Socket Channel has its own pair of handlers, you can now maintain state between partial reads and writes.
Two tips to improve performance:
Try to do first read immediately when connection is accepted. Only if you don't read enough data as defined by header in your custom protocol, register Channel interest for read events.
Try to do write first without registering interest for write events and only if you don't write all the data, register interest for
write.
This will reduce number of Selector wakeups.
Something like this:
SocketChannel socketChannel;
byte[] outData;
final static int MAX_OUTPUT = 1024;
ByteBuffer output = ByteBuffer.allocate(MAX_OUTPUT);
// if message was not written fully
if (socketChannel.write(output) < messageSize()) {
// register interest for write event
SelectionKey selectionKey = socketChannel.register(selector, SelectionKey.OP_WRITE);
selectionKey.attach(writeHandler);
selector.wakeup();
}
Finally, there should be timed Task which checks if Connections are still alive/SelectionKeys are canceled. If client breaks TCP connection, server will usually not know of this. As a result, there will be number of Event handlers in memory, bind as Attachments to stale connections which will result with memory leak.
This is the reason why you may say Attachments are not good, but the issue can be dealt with.
To deal with this here are two simple ways:
TCP keep alive could be enabled
periodic task could check timestamp of last activity on the given Channel. If it is idle for to long, server should terminate connection.
There's an ancient and very inaccurate NIO blog from someone at Amazon where it is wrongly asserted that key attachments are memory leaks. Complete and utter BS. Not even logical. This is also the one where he asserts you need all kinds of supplementary queues. Never had to do that yet, in about 13 years of NIO.
What you need is a ByteBuffer per channel, or possibly two, one for read and one for write. You can store a single one as the attachment itself: if you want two, or have other data to store, you need to define yourself a Session class that contains both buffers and whatever else you want to associate with the channel, for example client credentials, and use the Session object as the attachment.
You really can't get very far in NIO with a single buffer for all channels.
I have to implement sending data with specific source port and in the same time listen to that port. Full duplex. Does anybody know how to implement it on java. I tried to create separate thread for listening on socket input stream but it doesnt work. I cannot bind ServerSocket and client socket to the same source port and the the same with netty.
It there any solution for dull duplex?
init(){
socket = new Socket(InetAddress.getByName(Target.getHost()), Target.getPort(), InetAddress.getByName("localhost"), 250);
in = new DataInputStream(socket.getInputStream());
out = new DataOutputStream(socket.getOutputStream());
}
private static void writeAndFlush(OutputStream out, byte[] b) throws IOException {
out.write(b);
out.flush();
}
public class MessageReader implements Runnable {
#Override
public void run() {
//this method throw exception EOF
read(in);
}
private void read(DataInputStream in){
while (isConnectionAlive()) {
StringBuffer strBuf = new StringBuffer();
byte[] b = new byte[1000];
while ((b[0] = bufferedInputStream.read(b)) != 3) {
strBuf.append(new String(b));
}
log.debug(strBuf.toString());
}
}
}
What you're trying to do is quite strange: A ServerSocket is a fully implemented socket that accepts connections, it handles its own messages and you definitely cannot piggy-back another socket on top of it.
Full duplex is fairly simple to do with NIO:
Create a Channel for your Socket in non-blocking mode
Add read to the interest OPs
Sleep with a Selector's select() method
Read any readable bytes, write any writable bytes
If writing is done, remove write from interest OPs
GOTO 3.
If you need to write, add bytes to a buffer, add write to interest OPs and wake up selector. (slightly simplified, but I'm sure you can find your way around the Javadoc)
This way you will be completely loading the outgoing buffer every time there is space and reading from the incoming one at the same time (well, single thread, but you don't have to finish writing to start reading etc).
I had run into the same question and decided to answer it myself. I would like to share with you guys the code repo. It is really simple, you can get the idea to make your stuff work. It is an elaborate example. The steps accidentally look like Ordous's solution.
https://github.com/khanhhua/full-duplex-chat
Feel free to clone! It's my weekend homework.
Main thread:
Create background thread(s) that will connect to any target machines(s).
These threads will connect to target machines and transmit data and die
Create an infinite loop
Listen for incoming connections.
Thread off any connection to handle I/O
Classes:
Server
Listens for incoming connections and threads off a Client object
Client
This class is created upon the server accepting the incoming connection, the TcpClient or NetClient (i forget what java calls it) is used to send data. Upon completion it dies.
Target
Is created during the start up and connects to a specific target and send data.
once complete it dies.
I wrote a multithreaded gameserver application which handles multiple simultaneous connections using NIO. Unfortunately this server generates full CPU load on one core as soon as the first user connects, even when that user is not actually sending or receiving any data.
Below is the code of my network handling thread (abbreviated to the essential parts for readability). The class ClientHandler is my own class which does the network abstraction for the game mechanics. All other classes in the example below are from java.nio.
As you can see it uses a while(true) loop. My theory about it is that when a key is writable, selector.select() will return immediately and clientHandler.writeToChannel() is called. But when the handler returns without writing anything, the key will stay writable. Then select is called again immediately and returns immediately. So I got a busy spin.
Is there a way to design the network handling loop in a way that it sleeps as long as there is no data to send by the clientHandlers? Note that low latency is critical for my use-case, so I can not just let it sleep an arbitrary number of ms when no handlers have data.
ServerSocketChannel server = ServerSocketChannel.open();
server.configureBlocking(false);
server.socket().bind(new InetSocketAddress(port));
Selector selector = Selector.open();
server.register(selector, SelectionKey.OP_ACCEPT);
// wait for connections
while(true)
{
// Wait for next set of client connections
selector.select();
Set<SelectionKey> keys = selector.selectedKeys();
Iterator<SelectionKey> i = keys.iterator();
while (i.hasNext()) {
SelectionKey key = i.next();
i.remove();
if (key.isAcceptable()) {
SocketChannel clientChannel = server.accept();
clientChannel.configureBlocking(false);
clientChannel.socket().setTcpNoDelay(true);
clientChannel.socket().setTrafficClass(IPTOS_LOWDELAY);
SelectionKey clientKey = clientChannel.register(selector, SelectionKey.OP_READ | SelectionKey.OP_WRITE);
ClientHandler clientHanlder = new ClientHandler(clientChannel);
clientKey.attach(clientHandler);
}
if (key.isReadable()) {
// get connection handler for this key and tell it to process data
ClientHandler clientHandler = (ClientHandler) key.attachment();
clientHandler.readFromChannel();
}
if (key.isWritable()) {
// get connection handler and tell it to send any data it has cached
ClientHandler clientHandler = (ClientHandler) key.attachment();
clientHandler.writeToChannel();
}
if (!key.isValid()) {
ClientHandler clientHandler = (ClientHandler) key.attachment();
clientHandler.disconnect();
}
}
}
SelectionKey clientKey = clientChannel.register(selector, SelectionKey.OP_READ | SelectionKey.OP_WRITE);
The problem is here. SocketChannels are almost always writable, unless the socket send buffer is full. Ergo they should normally not be registered for OP_WRITE: otherwise your selector loop will spin. They should only be so registered if:
there is something to write, and
a prior write() has returned zero.
I don't see any reason why the reading and writing must happen with the same selector. I would use one selector in a thread for read/accept operations and it will always be blocking until new data arrives.
Then, use a separate thread and selector for writing. You mention you are using a cache to store messages before they are sent on the writable channels. In practice the only time a channel would not be writable is if the kernel's buffer is full, so it will rarely not be writable. A good way to implement this would be to have a dedicated writer thread that is given messages, and sleeping; it can be either interrupt()ed when new messages should be sent, or using a take() on a blocking queue. Whenever a new message arrives, it will unblock, do a select() on all writable keys and send any pending messages; only in rare cases will a message have to remain in the cache since a channel is not writable.
I'm implementing simple netty server for a multiplayer game. I'm just trying to figure out Netty.
I test the server via telnet. What i done is broadcast the messages to all channels. It's working smoothly. Also I remove channels from map on close event which is fine.
But the problem is if one of the clients disconnect unexpectedly, before closed callback, messageReceived callback called which the sender is disconnected channel.
How can i properly ignore the message comes from disconnected client?
I use StringBuffer in messagedReceived but for the case StringBuffer.toString is also not a proper string. At the end disconnected channel broadcast pointless message to other channels and itself, when receiver channel is itself throws an exception Connection reset by peer
which it's normal because the channel itself is not available at the moment.
Here is the code ;
#Override
public void messageReceived(ChannelHandlerContext ctx, MessageEvent e) {
System.out.println();
System.out.println("------------------");
Channel current = e.getChannel();
System.out.println("SenderChannel:"+current.getId());
if(!current.isOpen())
System.out.println("Not Open");
ChannelBuffer buf = (ChannelBuffer) e.getMessage();
StringBuffer sbs = new StringBuffer();
while(buf.readable()) {
sbs.append((char) buf.readByte());
}
String s = sbs.toString();
System.out.println(s);
String you = "You:" + s;
String other = "Other:" + s;
byte [] uResponse = you.getBytes();
byte [] otherResponse = other.getBytes();
Iterator iterator = channelList.entrySet().iterator();
while(iterator.hasNext()){
Map.Entry pairs = (Map.Entry)iterator.next();
Integer key = (Integer)pairs.getKey();
Channel c = (Channel)pairs.getValue();
System.out.println("ReceiverChannel:"+c.getId());
if(key != current.getId())
c.write(ChannelBuffers.wrappedBuffer(otherResponse));
else
c.write(ChannelBuffers.wrappedBuffer(uResponse));
}
}
#Override
public void channelDisconnected(ChannelHandlerContext ctx,
ChannelStateEvent e){
Channel ch = e.getChannel();
channelList.remove(ch.getId());
System.out.println();
System.out.println("*****************");
System.out.println("DisconnectEvent:"+ch.getId());
System.out.println("*****************");
System.out.println();
ch.close();
}
You can't solve the problem in the manner that you would like. If there's a network problem then technically the sender could disconnect at any time, for example
as soon as the thread enters messageReceived
while you're iterating through channelList
while you're iterating through channelList but after you've echoed back to the sender
after you've broadcast the message
Netty can't raise the disconnected event while messageReceived is processing because you're running in the thread that will raise the event (unless you have a non-ordered execution handler in your pipeline). The correct solution really depends on your application. If the broadcast results in all the other receivers responding it's probably better / easier to have the server suppress any messages destined for a client that's no longer connected.
Also, if you're really going to use strings then take a look at StringEncoder / StringDecoder. There's no guarantee in your code that the message event buffer contains a complete string.
Just put a try/catch around each send. If one of them fails, close the corresponding channel.
If this is for a multiplayer game server, it might be better to use an existing Netty game server solution like java game server. Disconnects become events which get sent to the session and since it is event driven, you could write your own handler to decide whether or not to receive anymore events on the same session. Since events are queued in a FIFO order, if disconnect happens then you need not go ahead with subsequent broadcasts.
I am not a Java Developer. But from socket point of view this data is in buffer or sent before disconnecting of user. So when you are in receiving stage user is still connected and exactly on time of completing of receiving user is already disconnected. So I think best way to prevent this things is to check if user is still connected after each receiving of data.
In C# I personally use this code to check if user is still connected:
if (client.Poll(0, SelectMode.SelectRead))
{
byte[] checkConn = new byte[1];
if (client.Receive(checkConn, SocketFlags.Peek) == 0)
return false;
}
return true;
I am not sure about Java And Netty (And if your connection is TCP) but this is what I use and this could be possible to convert it easily to Java.