Java socket data works only when sending time is delayed - java

I am sending a protobuf from C++ to Java via a raw socket, the C++ program being the client and the java program being the server. The C++ program generates packets almost every 1ms which is sent to the java program.
If I run the program normally, I see that there are only the half the packets being received.
If I set a breakpoint in the C++ program and then run the client and the server, all the packets are received.
How do I ensure that all packets are received without setting a breakpoint? Can I introduce a delay?
All the packets have bytes sizes upto a maximum of 15 bytes.

By default TCP sockets use the "Nagle Algorithm" which will delay transmission of the next "unfilled" fragment in order to reduce congestion. Your packet size is small enough and the time delay between packets is small enough that the nagle algorithm will have an effect on your transmissions.

As already discussed in the comments, what you are trying to do won't work in a reliable way. This is also described in the Protobuf documentation:
If you want to write multiple messages to a single file or stream, it
is up to you to keep track of where one message ends and the next
begins. The Protocol Buffer wire format is not self-delimiting, so
protocol buffer parsers cannot determine where a message ends on their
own. The easiest way to solve this problem is to write the size of
each message before you write the message itself. When you read the
messages back in, you read the size, then read the bytes into a
separate buffer, then parse from that buffer. (If you want to avoid
copying bytes to a separate buffer, check out the CodedInputStream
class (in both C++ and Java) which can be told to limit reads to a
certain number of bytes.)
The bold italic part is where you code isn't correct.
On the write side you should write
the Protobuf's length in some format that is understandable for both sender and receiver (selecting the proper format is especially important when transporting between systems whose endianness is different).
the protobuf
On the receiving end you need to
perform a read with the fixed, known size of the length field
a read for the length learned in step 1. This read will retriev the protobuf.
There's example code here on SO in this question: Sending struct via Socket using JAVA and C++

#fvu: This is my code which I am trying:
import Visualization.DataSetProtos.PacketData; // protos import
import java.io.InputStream;
import java.util.Arrays;
import javax.swing.JFrame;
import javax.swing.JScrollBar;
import javax.swing.JScrollPane;
class WorkerThread extends Thread {
Socket service;
static DynamicData demo;
static int size;
static int times;
static byte[] buffer;
WorkerThread(Socket service)
{
this.service = service;
buffer = new byte[500];
size = 1;
times = 0;
}
static void Print(PacketData packetData)
{
System.out.print("Packet Number: " + (++times));
System.out.print(" DataSet Size: " + packetData.getLength() + "\n");
}
static void Print(PacketHeader packetHeader)
{
System.out.print("Packet Number: " + (++times));
System.out.print(" DataSet Size: " + packetHeader.getLength() + "\n");
}
public void run() {
boolean flag=true; //you can change this flag's condition, to test if the client disconects
if(demo == null)
{
demo = new DynamicData("GridMate Data Visualization");
demo.pack();
RefineryUtilities.centerFrameOnScreen(demo);
//demo.setVisible(true);
}
try
{
while (flag)
{
InputStream inputStream = service.getInputStream();
int read;
read = inputStream.read(buffer);
byte[] readBuffer = new byte[read];
readBuffer = Arrays.copyOfRange(buffer, 0, read);
PacketData packetData = PacketData.parseFrom(readBuffer);
Print(packetData);
}
service.close();
}
catch(Exception e)
{
e.printStackTrace();
}
}
}
public class Test
{
Test()
{
server = null;
client= null;
}
public static void main(final String[] args) {
int i =0;
try
{
server = new ServerSocket(25715);
System.out.println("Server setup and waiting for client connection ...");
while(true)
{
client = server.accept();
WorkerThread wt = new WorkerThread(client);
wt.start();
i++;
}
}
catch(IOException e)
{ System.out.println("IO Error in streams " + e);
e.printStackTrace();
}
}
public void finalize()
{
try
{
server.close();
client.close();
}
catch(Exception e)
{
e.printStackTrace();
}
}
static ServerSocket server;
static Socket client;
}

Related

Java Socket Server lagging with two clients

I'm new to StackOverflow lol, but I've been relying on this website for awhile. I have a question regarding a Java socket server that I created. Upon connection (client and server), my application creates a thread for that client. This is an MMORPG game server... at least trying to be. With one player, it doesn't lag that bad. With two, however, it began to show some lags...
If I was to spam left-right-left-right on one of the client, and move around normally with the other, the other would feel glitchy. I'm hoping to get some assistant since I've spent over a week and a half tangled up =) It's about time I ask for help.
The code is simple:
public static void main(String[] args) throws IOException{
serverRooms.put(roomNumber, new Room());
try {
System.out.println("Starting Server...");
serverSocket = new ServerSocket(9595, 20);
System.out.println("Server Started");
while(run){
Socket socket = serverSocket.accept(); // Check if we have a connection, otherwise wait
Player player = new Player(playerCount++, socket, roomNumber);
new Thread(player).start();
}
} catch (IOException e) {
e.printStackTrace();
}
}
That's how it is all started! On the Player object, it looks like :
public void run() {
while(playerIsConnected) {
try {
int msgid = input.readUnsignedByte();
messageHandler(this, msgid);
} catch (IOException e) {
System.err.println("Player have signed off");
playerIsConnected = false;
}
}
// If Player leaves, close socket, and end thread
try {
socket.close();
} catch (IOException e) {
System.out.println("We got an error while closing a socket on player " + pid + ".");
}
}
messageHandler happens to be a static method from a Final Static class. It is a global method that can be called by every thread (Can this be the cause of the lag??)
public final class MessageControl {
public static void messageHandler(Player player, int msgid) throws IOException{
DataInputStream input = player.getInputStream();
switch (msgid) {
case 10:
byte hspd = (byte) Math.signum(input.readByte());
byte vspd = (byte) Math.signum(input.readByte());
byte dir = input.readByte();
updatePlayerPosition(player);
byte spd = (byte) (hspd != 0 && vspd != 0 ? player.spd-1 : player.spd);
// Prepare packet and send to clients
ByteBuffer buffer = ByteBuffer.allocate(11);
buffer.put((byte) 10);
buffer.put(shortToByte_U16(player.pid));
buffer.put(shortToByte_U16(player.x));
buffer.put(shortToByte_U16(player.y));
buffer.put((byte)(hspd*spd));
buffer.put((byte)(vspd*spd));
buffer.put((byte)(dir));
sendPacketToAllClients(player, buffer, true);
// Update Player info
player.hspd = (byte) hspd;
player.vspd = (byte) vspd;
player.dir = dir;
player.lastUpdate = System.currentTimeMillis();
break;
}
private static void sendPacketToAllClients(Player player, ByteBuffer buffer, boolean includeMe){
for (Player otherPlayer : player.room.getPlayersInRoom()){
if (otherPlayer.pid != player.pid || includeMe){
sendPacketToClient(otherPlayer, buffer);
}
}
}
}
Regarding the shortToByte_U16(), I just created a simple method that conerts shorts to bytes (sending buffer packets via bytes to client). Example, I have about 5 of these conversions, which would include conversion for unsigned u16
public static byte[] shortToByte_16(int x){
short s = (short) x;
byte[] ret = new byte[2];
ret[0] = (byte)(s & 0xff);
ret[1] = (byte)((s >> 8) & 0xff);
return ret;
}
Looking at the following structure, any ideas why I be lagging?
EDIT : I think I improved it a lot by setting the setTcpNoDelay to true. The lag seems to still be there when I spam left/right on my end... the other player on my screen seems glitchy.
Socket socket = serverSocket.accept(); // Check if we have a connection, otherwise wait
socket.setTcpNoDelay(true); // This helped a lot!!!
Player player = new Player(playerCount++, socket, roomNumber);
new Thread(player).start();
From what I am seeing... my "spamming left/right" end seems to be missing the packet sent by the server.
Problem solved. =) setTcpNoDelay true did the trick. Regarding the part when I said I was missing packets, I actually didn't. The two messages merged and came in as one message. My program only read the first few bytes and ignored the rest. Had to put a byte in front to indicate the size of message. Once that was in place, I set a while loop to read through it til it can't read anymore. =) thanks everyone for helping me. My first post and it was a grand experience.

File transfer over java sockets using multiple IO streams

Recently, I wrote a simple client server program for file transfer over standard TCP sockets. The average throughput was around 2.2Mbps over WiFi channel. My question is:
Is it possible to transfer a large file (say 5 GB) over multiple data IO streams so that each stream could transfer several parts of the same file in a parallel manner (different threads could be used for this purpose)? These file parts could be re-assembled at the receiving end.
I tried to split a small file and transfered it over a dataoutputstream. The first segment works fine, but I don't know how to read a file input stream in selective manner (I also tried mark() and reset() methods for selective reading but no use)
Here is my code (for testing purpose, I have redirected the output to fileoutputstream):
public static void main(String[] args) {
// TODO Auto-generated method stub
final File myFile=new File("/home/evinish/Documents/Android/testPicture.jpg");
long N=myFile.length();
try {
FileInputStream in=new FileInputStream(myFile);
FileOutputStream f0=new FileOutputStream("/home/evinish/Documents/Android/File1.jpg");
FileOutputStream f1=new FileOutputStream("/home/evinish/Documents/Android/File2.jpg");
FileOutputStream f2=new FileOutputStream("/home/evinish/Documents/Android/File3.jpg");
byte[] buffer=new byte[4096];
int i=1, noofbytes;
long acc=0;
while(acc<=(N/3)) {
noofbytes=in.read(buffer, 0, 4096);
f0.write(buffer, 0, noofbytes);
acc=i*noofbytes;
i++;
}
f0.close();
I got the first segment of my file (this can be copied to a DataOutputStream in one thread). Can any one suggest, how to read remaining part of the file (after N/3 byte) in a segment of N/3 so that three streams could be used in three threads for concurrent operation?
Here is the code to merge file segments at receiver end:
package com.mergefilespackage;
import java.io.BufferedInputStream;
import java.io.BufferedOutputStream;
import java.io.Closeable;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
public class MergeFiles {
/**
* #param args
*/
public static void main(String[] args) throws Exception{
// TODO Auto-generated method stub
IOCopier.joinFiles(new File("/home/evinish/Documents/Android/File1.jpg"), new File[] {
new File("/home/evinish/Documents/Android/File2.jpg"), new File("/home/evinish/Documents/Android/File3.jpg")});
}
}
class IOCopier {
public static void joinFiles(File destination, File[] sources)
throws IOException {
OutputStream output = null;
try {
output = createAppendableStream(destination);
for (File source : sources) {
appendFile(output, source);
}
} finally {
IOUtils.closeQuietly(output);
}
}
private static BufferedOutputStream createAppendableStream(File destination)
throws FileNotFoundException {
return new BufferedOutputStream(new FileOutputStream(destination, true));
}
private static void appendFile(OutputStream output, File source)
throws IOException {
InputStream input = null;
try {
input = new BufferedInputStream(new FileInputStream(source));
IOUtils.copy(input, output);
} finally {
IOUtils.closeQuietly(input);
}
}
}
class IOUtils {
private static final int BUFFER_SIZE = 1024 * 4;
public static long copy(InputStream input, OutputStream output)
throws IOException {
byte[] buffer = new byte[BUFFER_SIZE];
long count = 0;
int n = 0;
while (-1 != (n = input.read(buffer))) {
output.write(buffer, 0, n);
count += n;
}
return count;
}
public static void closeQuietly(Closeable output) {
try {
if (output != null) {
output.close();
}
} catch (IOException ioe) {
ioe.printStackTrace();
}
}
}
Any help would be highly appreciated! Thanks in advance!
You can't get any more speed over the same link with more sockets. Each socket sends a certain number of packets, each of a certain size. As we double the number of sockets, the number of packets/sec*socket is halved, and then decreased even more due to collisions, overhead, and contention. Packets start to bump, jumble, and otherwise panic. The OS cannot handle the pandemonium of lost ACKs, and the WiFi card struggles to transmit at such a rate. It is losing its low-level acks as well. As packets get lost, a desperate TCP stack dials down the transmit rate. If this were to be able to come up due to signal improvement, it's now stuck at the lower speed due to silly window syndrome or another form of TCP deadlock.
Any attempt of WiFi to get any higher speeds out of wider carrier bands, MiMo, or multiple paths, has already been realized as gains, even with one socket. You can't take it any farther.
Now, wait. We're quite below WiFi speed, aren't we? Of course, we need to use buffering!
Make sure you create BufferedWriter and BufferedReader objects from your socket's getInputStream or getOutputStream methods. Then write to/read from those buffers. Your speed may increase somewhat.
You could get the byte array of the FileInputStream and split it every 10 KB (every 10.000 bytes).
Then send these parts through the streams in order.
On the server you can put the arrays together again and read the file from this giant byte array.

Lag Over Sockets java

I am making a multi-player snake game in java using sockets. All the transmission is done through a server to all the connected clients. The code for the same is yet not completely finished but it does the basic job of moving the snakes around and increasing scores if a particular client eats its food.
I generate random numbers for food coordinates from the server side and relay it to all the clients. If a client presses a key the requested movement is calculated and the direction of movement is sent to the server which then relays the movement to ALL clients (including the one who sent it) and only on receipt of the movement info do the clients make changes to the snake which moved. So every movement is tracked over the network and no movement decision is made by the client itself until it receives that, say client 'player1' has asked to move.
The problem I am facing is that even with two players there seems to be a difference in coordinates after moving about the snakes a little.
What possible remedies could I apply to my code so as to remove this apparent lag between the position of snakes?
This is the client code:
package mycode;
import java.awt.Point;
import java.io.BufferedInputStream;
import java.io.BufferedOutputStream;
import java.io.DataInputStream;
import java.io.DataOutputStream;
import java.net.Socket;
import java.util.Map;
import javax.swing.JOptionPane;
public class ConnectionManager implements Runnable {
Socket socket;
boolean start = false;
DataInputStream in;
DataOutputStream out;
Map<String, Snake> map;
ConnectionManager(String name, String IP, Map<String, Snake> m) {
this.map = m;
try {
socket = new Socket(IP, 9977);
in = new DataInputStream(new BufferedInputStream(
socket.getInputStream()));
out = new DataOutputStream(new BufferedOutputStream(
socket.getOutputStream()));
out.writeUTF(name);
out.flush();
} catch (Exception e) {
e.printStackTrace();
JOptionPane.showMessageDialog(null, "Could Not Find Server",
"ERROR", JOptionPane.ERROR_MESSAGE);
System.exit(0);
}
}
void populateMap() {
try {
String name = in.readUTF();
System.out.println("Name received: " + name);
if (name.equals("start_game_9977")) {
start = true;
System.out.println("Game Started");
return;
} else if (name.equals("food_coord")) {
Game.foodx = in.readInt();
Game.foody = in.readInt();
return;
}
map.put(name, new Snake(5));
} catch (Exception e) {
e.printStackTrace();
}
}
boolean start() {
return start;
}
void increaseSnakeLength(String thisname){
Snake temp = map.get(thisname);
Point temp1=new Point(0,0);
temp.length++;
switch (temp.move) {
case DOWN:
temp1= new Point(temp.p[temp.length - 2].x,
temp.p[temp.length - 2].y+6);
break;
case LEFT:
temp1= new Point(temp.p[temp.length - 2].x-6,
temp.p[temp.length - 2].y);
break;
case RIGHT:
temp1= new Point(temp.p[temp.length - 2].x+6,
temp.p[temp.length - 2].y);
break;
case UP:
temp1= new Point(temp.p[temp.length - 2].x,
temp.p[temp.length - 2].y-6);
break;
default:
break;
}
if(temp1.y>Game.max)
temp1.y=Game.min;
if(temp1.x>Game.max)
temp1.x=Game.min;
if(temp1.y<Game.min)
temp1.y=Game.max;
if(temp1.x<Game.min)
temp1.x=Game.max;
temp.p[temp.length-1]=temp1;
}
void readMotion() {
try {
while (true) {
if (Game.changedirection) {
String mov = "";
mov = Game.move.name();
// System.out.println(Game.move);
out.writeUTF(mov);
out.flush();
Game.changedirection = false;
}
if (Game.foodeaten) {
out.writeUTF("food_eaten");
out.flush();
Game.foodeaten = false;
}
Thread.sleep(50);
}
} catch (Exception e) {
e.printStackTrace();
}
}
void otherRunMethod() {
try {
while (true) {
String mname = in.readUTF();
String mov = in.readUTF();
if (mov.equals("Resigned")) {
map.remove(mname);
} else if (mov.length() >= 10) {
if (mov.substring(0, 10).equals("food_eaten")) {
String[] s = mov.split(",");
Game.foodx = Integer.parseInt(s[1]);
Game.foody = Integer.parseInt(s[2]);
int score = ++map.get(mname).score;
increaseSnakeLength(mname);
System.out.println(mname + ":" + score+" Length:"+map.get(mname).length);
}
} else {
Game.move = Direction.valueOf(mov);
map.get(mname).move = Game.move;
}
}
} catch (Exception e) {
e.printStackTrace();
}
}
#Override
public void run() {
while (true) {
if (!start) {
populateMap();
} else if (start) {
new Thread(new Runnable() {
public void run() {
otherRunMethod();
}
}).start();
readMotion();
break;
}
try {
Thread.sleep(10);
} catch (Exception e) {
e.printStackTrace();
}
}
}
}
The code is pretty long so I am just putting up the server side of code that manages connections.
package mycode;
import java.io.BufferedInputStream;
import java.io.BufferedOutputStream;
import java.io.DataInputStream;
import java.io.DataOutputStream;
import java.io.IOException;
import java.io.ObjectInputStream;
import java.net.Socket;
import java.util.Map;
public class Playerhandler implements Runnable {
Socket player;
String thisname;
Map<String, Socket> map;
DataInputStream in = null;
DataOutputStream out = null;
ObjectInputStream ob;
Snake snake;
Playerhandler(Socket player, Map<String, Socket> m) {
this.player = player;
this.map = m;
try {
in = new DataInputStream(new BufferedInputStream(
player.getInputStream()));
thisname = in.readUTF();
map.put(thisname, this.player);
populatePlayers();
System.out.println("Connected Client " + thisname);
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
void populatePlayers() {
try {
out = new DataOutputStream(new BufferedOutputStream(
player.getOutputStream()));
for (String name : map.keySet()) {
out.writeUTF(name);
out.flush();
}
for (String name : map.keySet()) {
out = new DataOutputStream(new BufferedOutputStream(map.get(
name).getOutputStream()));
out.writeUTF(thisname);
out.flush();
}
} catch (Exception e) {
e.printStackTrace();
}
}
void relay(String move) {
try {
if (move.equals("food_eaten")) {
move = move + ","
+ (Snakeserver.randomGenerator.nextInt(100) * 6) + ","
+ (Snakeserver.randomGenerator.nextInt(100) * 6);
}
for (String name : map.keySet()) {
out = new DataOutputStream(new BufferedOutputStream(map.get(
name).getOutputStream()));
out.writeUTF(thisname);
out.flush();
out.writeUTF(move);
// System.out.println(Direction.valueOf(move));
out.flush();
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
public void run() {
while (true) {
try {
relay(in.readUTF());
} catch (IOException e) {
// TODO Auto-generated catch block
System.out.println("Player " + thisname + " Resigned");
map.remove(thisname);
relay("Resigned");
return;
}
}
}
}
This answer is to recap the dialog to arrive at a solution as well as to point some additional areas to research or try out.
The main software behavior problem was that having multiple clients resulted in the various clients showing different snake positions after several moves.
After a number of questions and responses through the comments, the poster of the question modified their software so that all of the clients are synchronized by the server sending out the objects of all of the snakes to all of the clients so that all clients are now using the same snake object. Previously each client was maintaining its own snake object data and just receiving changes or deltas in snake data. With this change, all of the clients are now synchronized through the snake object transmitted by the server however there is still a problem with clients showing slightly different positions which is corrected after a moment or two, as each client receives an update on all of the snakes, the clients become synchronized again.
The next step is to look at a different approach so that the clients will remain synchronized more closely using UDP/IP as the network transmission protocol rather than the currently used TCP/IP. The expected results of using UDP/IP is to reduce the various lags introduced by the TCP network transmission protocol in order to provide the connection oriented, sequenced byte stream provided by TCP. However using the UDP network transmission protocol requires that some of the delivery mechanisms used by TCP in order to provide the dependable sequence of bytes must be assumed by the user of UDP.
Some of the issues with UDP are: (1) packets may not be received in the same sequence in which they are sent, (2) packets may be dropped or lost so that some packets sent may not be received, and (3) data sent using UDP must be explicitly put into packets for transmission so that the sender and the receiver see packets rather than a stream of bytes.
The basic architecture for this snake game would look something like the following.
Clients would send a snake update to the server. This interaction would require an acknowledgement sent by the server back to the client. If the client does not receive such an acknowledgement, the client would resend the snake update after some time period.
The server would then update its data to reflect the change and using its list of clients, send the same data packet to all clients. Each client receiving the packet would send an acknowledgement. By sending the acknowledgement, each client notifies the server that they are still in the game. If the server is no longer receiving client acknowledgements, it will know that a client has possibly left the game or there is some kind of network problem.
Each packet would have a sequence number which is incremented after sending the packet. This sequence number gives a unique identifier so that clients and server can detect if packets have been missed or if a packet received is a duplicate of an already received packet.
With UDP it is best if packets are as small as possible. UDP packets that are larger than what can be sent in the underlying IP network protocol will be split up into multiple IP packets with the multiple IP packets sent one at a time and then reassembled into the UDP packet at the receiving network node.
Here are some resources on UDP network protocol using the Java programming language.
Lesson: All about datagrams.
A simple Java UDP server and UDP client.
Stackoverflow: Send and receive serialize object on UDP in java.
Java-Gaming.org UDP vs TCP.
Gaffer On Games: What every programmer needs to know about game networking.
Gaffer On Games: Reliability and Flow control.
Stackoverflow: What are possible ways to send Game/Simulation state with javaNIO?
I've never implemented a network multiplayer game before, but I think the most widely used 'solution' here is to cheat.
I think its referred to as 'dead reckoning', although snake works exactly like this anyway.
http://www.gamasutra.com/view/feature/3230/dead_reckoning_latency_hiding_for_.php
Basically you decouple the game loop from network updates. Have each client keep its own state, and simply predict where the opponents are going to be at each frame. Then when updates from the server arrive you can adjust opponents to their true location. To hide this discrepancy, I think its common to render the state of the game as it was a few milliseconds ago, rather than the current state. That way the network updates have a more realistic chance of catching up with the game loop, so it will seem less choppy.
As I said though, I've never actually implemented this myself so YMMV. This is one of the harder problems in game development.
I would be inclined to add an explicit call of setTcpNoDelay(true). This will make sure that http://en.wikipedia.org/wiki/Nagle%27s_algorithm is turned off and so disable an optimization that increases efficiency at what is usually a small amount of increased delay.

why client can not receive message from server (java)

I have just started learning java. I modified the client side code for a server/client communication program, by creating two threads for the client side, main thread for receiving user's input, and inputThread for receiving server's response. I am sure that server has sent the response to client, however, no response message is obtain at client.
Here is my code. Can anyone help me to figure it out? Thanks
package clientnio;
import java.net.*;
import java.nio.*;
import java.io.*;
import java.nio.channels.*;
import java.util.Scanner;
public class ClientNIO {
public static int bufferLen = 50;
public static SocketChannel client;
public static ByteBuffer writeBuffer;
public static ByteBuffer readBuffer;
public static void main(String[] args) {
writeBuffer = ByteBuffer.allocate(bufferLen);
readBuffer = ByteBuffer.allocate(bufferLen);
try {
SocketAddress address = new InetSocketAddress("localhost",5505);
System.out.println("Local address: "+ address);
client=SocketChannel.open(address);
client.configureBlocking(false);
//readBuffer.flip();
new inputThread(readBuffer);
/*
String a="asdasdasdasddffasfas";
writeBuffer.put(a.getBytes());
writeBuffer.clear();
int d=client.write(writeBuffer);
writeBuffer.flip();
*/
while (true) {
InputStream inStream = System.in;
Scanner scan = new Scanner(inStream);
if (scan.hasNext()==true) {
String inputLine = scan.nextLine();
writeBuffer.put(inputLine.getBytes());
//writeBuffer.clear();
System.out.println(writeBuffer.remaining());
client.write(writeBuffer);
System.out.println("Sending data: "+new String(writeBuffer.array()));
writeBuffer.flip();
Thread.sleep(300);
}
}
}
catch(Exception e) {
System.out.println(e);
}
}
}
class inputThread extends Thread {
private ByteBuffer readBuffer;
public inputThread(ByteBuffer readBuffer1) {
System.out.println("Receiving thread starts.");
this.readBuffer = readBuffer1;
start();
}
#Override
public void run() {
try {
while (true) {
readBuffer.flip();
int i=ClientNIO.client.read(readBuffer);
if(i>0) {
byte[] b=readBuffer.array();
System.out.println("Receiving data: "+new String(b));
//client.close();
//System.out.println("Connection closed.");
//break;
}
Thread.sleep(100);
}
}
catch (Exception e) {
System.out.println(e);
}
}
}
Disclaimer: I'm not an active user of Java. (I only used it in school.)
Advice: I think it will greatly simplify the debugging process if you use blocking mode, at least until your code example is working correctly. (Currently your code does not seem to benefit from the non-blocking mode.)
I have identified two issues, culminating into four possible lines of code that may require changing:
When a ByteBuffer allocates its backing array, it sets itself ready to write by setting position to zero and limit to the capacity of that array. Your two uses of ByteBuffer.flip() (in the writing loop and the reading loop respectively) seem to be contrary to the convention.
Calling the ByteBuffer.array() method always returns the whole backing array, thus it always has size bufferLen. Because of this, a String constructed from the full-size array may contain junk from a previous transmission.
Typically, the array needs to be trimmed to the transmission size, and the conversion between a String and a byte array must use the same encoding as the server.
My suggested changes for first issue: (Note: I don't know how to fix the array trimming and encoding issue.)
writeBuffer.put(inputLine.getBytes());
writeBuffer.flip(); // <--here
client.write(writeBuffer);
...
writeBuffer.clear(); // <-- should be clear() instead of flip()
Thread.sleep(300);
// readBuffer.flip(); // <-- remove this line
int i=ClientNIO.client.read(readBuffer);
if(i>0) {
readBuffer.flip(); // <-- move it here
byte[] b=readBuffer.array();
System.out.println("Receiving data: "+new String(b));
...
}
References
http://docs.oracle.com/javase/1.4.2/docs/api/java/nio/ByteBuffer.html
http://docs.oracle.com/javase/1.4.2/docs/api/java/nio/channels/SocketChannel.html
Socketchannel always null
http://www.exampledepot.com/egs/java.nio.charset/ConvertChar.html
Calling flip() on a buffer prior to reading it is wrong. Don't do that. You need to flip it prior to writing from it, or getting from it, and compact() afterwards.

Help needed with asynchronous Java NIO using Executors.newFixedThreadPool

Hey guys, I'm working on a server program that is meant to scale well and serve potentially thousands of clients. The thing is, I feel that Apache MINA is too heavyweight so I decided to not use it and wrote my own client listener instead. I never really performed asynchronous socket operations in Java (C# made that so much easier, but I really preferred to write this project in Java since I'm more familiar with it in everything besides socket reads), so trying to understand how to use the thread pool correctly is hard for me. I used Apache MINA documentation to get an idea of how things should be done. I got two questions:
Is the thread pool used correctly? Apache MINA's default thread size is the number of CPU cores + 1, but should I really use a 3 thread thread pool for my Core 2 Duo in order to accept thousands of clients?
I know that reallocating the buffer twice for each message received from the client (each message is two packets, one header that is a constant 4 bytes and a content packet that has its length specified in the header). Is there an easy way to use a fixed size buffer that checks for buffer overruns so that behavior is still the same but the buffer doesn't have to be constantly reallocated?
Here's how I start the listener:
ClientListener cl = new ClientListener(1234);
cl.init();
new Thread(cl).start();
Here is the relevant code for ClientListener:
private static final int THREADS = Runtime.getRuntime().availableProcessors() + 1;
private ServerSocket socket;
private ExecutorService threadPool;
private int port;
public ClientListener(int port) {
this.port = port;
threadPool = Executors.newFixedThreadPool(THREADS);
}
public void init() {
try {
socket = new ServerSocket(port);
} catch (IOException ex) {
}
}
public void run() {
while (true) {
try {
ClientSession s = new ClientSession(socket.accept());
threadPool.execute(s);
} catch (IOException ex) {
}
}
}
ClientSession relevant code:
private Socket socket;
private byte[] buffer;
private boolean isHeader;
public ClientSession(Socket socket) {
this.socket = socket;
this.buffer = new byte[4];
this.isHeader = true;
}
public void run() {
InputStream in;
try {
in = socket.getInputStream();
out = socket.getOutputStream();
} catch (IOException ex) {
return;
}
while (!socket.isClosed()) {
try {
int read = in.read(buffer);
if (read == -1)
break;
receive(read);
} catch (IOException ex) {
break;
}
}
}
private void receive(int readBytes) {
if (isHeader) {
if (readBytes >= 4) {
buffer = new byte[getPacketLength(buffer)];
isHeader = false;
} else {
System.out.println("Not enough data received from client " + socket.getInetAddress() + " to decode packet.");
}
} else {
if (readBytes >= buffer.length) {
processMessage(new LittleEndianByteArrayReader(decryptData(buffer)), this);
buffer = new byte[4];
isHeader = true;
} else {
System.out.println("Not enough data received from client " + socket.getInetAddress() + " to decode packet (needed " + buffer.length + ", received " + readBytes + ").");
}
}
}
You don't need to know the code for getPacketLength, processMessage, decryptData, and the class LittleEndianByteArrayReader, but I'm pretty sure the purposes of those methods/classes are obvious.
The number of threads in blocking IO scenario have to be calculated by the number of clients and the time each client connection will be open.
Each connection of each user requires on thread.
With only three threads a user could simply block your server until connection timeout by just opening three TCP connections and not sending any data to your server.
Nevermind guys. I realized that Apache MINA actually uses NIO which is why I got confused. It really needs only one thread to process requests with the use of selectors. Thanks for all your answers and sorry about the confusion!

Categories

Resources