Develop Reference

first message over socket lost in Eclipse

I have this funny behavior in debug mode which I can't explain to myself.
There are two small client and server applications, and the flow goes like this:
Client initiates TCP connection
Server accepts and sends message to client requesting packet data size(it's fixed)
Client sends the size
Server reads the size and initializes some byte array - this works
Server blocks on read() from the input stream waiting on packets and client sends packets. - this is culprit
It all works when i run the applications, however when I debug the Server - it simply blocks on read().
If i send more than one message, for example 50, the Server receives 49 of those. Once client closes connection, Server reads -1 from stream and exits, with the first message lost.
The only thing i can think of is that in debug mode, client sends message before server reads it from stream but i don't see how could that be relevant as the message should be there.
Could somebody explain this behavior i am experiencing in debug mode only?
Note: client code is there just for debug, so it's not the prettiest.
Server
private static int sent;
public void processClientInput(HealthCheckSession session) {
byte[] buffer = session.getBuffer();
DataInputStream stream = session.getInFromClient();
int readBytes = 0;
int totalBytes = 0;
try {
while((readBytes = stream.read(buffer)) !=-1) { <---- this blocks in debug on first message
totalBytes += readBytes;
if (totalBytes < buffer.length) {
continue;
}
else {
totalBytes = 0;
String packet = new String(buffer, ProtocolValidator.SERIALIZATION_CHARSET);
sendResponse(packet);
}
}
} catch (IOException e) {e.printStackTrace();}
}
private void sendResponse(String packet) {
System.out.println(packet);
++sent;
System.out.println("sent " + sent);
}
Client
public static void main(String[] args) throws UnknownHostException,
IOException, InterruptedException {
Socket clientSocket = new Socket("localhost", 10002);
// get the socket's output stream and open a PrintWriter on it
PrintWriter outToServer = new PrintWriter(
clientSocket.getOutputStream(), true);
// get the socket's input stream and open a BufferedReader on it
BufferedReader inFromServer = new BufferedReader(new InputStreamReader(
clientSocket.getInputStream()));
System.out.println("before read");
System.out.println(inFromServer.readLine());
System.out.println("after read");
byte[] packageData = "100, ABCDZEGHR"
.getBytes(Charset.forName("UTF-8"));
String size = String.valueOf(packageData.length);
outToServer.println(size);
for (int i = 0; i < 50; i++) {
DataOutputStream dis = new DataOutputStream(
clientSocket.getOutputStream());
System.out.println("before write");
dis.write(packageData);
dis.flush();
System.out.println("after write");
Thread.sleep(1000);
}
try {
Thread.sleep(100 * 1000);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
clientSocket.close();
}
IMPORTANT UPDATE
This does not happen in Netbeans debug mode.

datainputstream missing bytes in.read(b)

I'm trying to write a program which acts as a server that will read bytes from a client that is written in PHP - sends request via socket (which i cannot recode due to policy) Here is the server code:
The server runs in: Red Hat Enterprise Linux Server release 6.2 (Santiago)
public void run() {
try {
serverSocket = new ServerSocket(port);
serverSocket.setSoTimeout(0);
while(!isInterrupted) {
try {
Socket server = serverSocket.accept();
LOG.info("Request received from : " + server.getRemoteSocketAddress());
DataInputStream in = new DataInputStream(server.getInputStream());
// DataInputStream in = new DataInputStream(
// new BufferedInputStream(server.getInputStream(), 10000));
byte[] bytes = new byte[10000];
int byteDupLength = in.read(t_bytes);
// in.readFully(bytes); // I tried this but to no avail
// int byteDupLength = bytes.length;
LOG.info(byteDupLength);
byte[] byteDup = new byte[byteDupLength];
System.arraycopy(bytes, 4, byteDup, 0, byteDupLength);
// FOR INFORMATION ONLY
/*for (byte b : byteDup){
LOG.info(b);
}*/
ByteBuffer buffer = ByteBuffer.wrap(byteDup);
LOG.info(buffer);
forwardRequest(byteDup);
server.close();
}
catch(SocketTimeoutException s) {
LOG.error("Socket timed out!", s);
break;
}
catch(IOException e)
{
LOG.error("IOException:", e);
break;
}
}
}
catch (IOException ex) {
LOG.error("Server socket is null", ex);
}
LOG.fatal("ReceiverEngine interrupted!");
}
I encountered a problem when the client sends request consisting of 4948 bytes. The only bytes the server can read is 2090.
Another thing that seems a mystery to me is that, when I run the server via Netbeans in my local (which is a Windows 7 Pro), it works as expected. I dont know what is wrong. Please help.. :)
Thanks!

TCP is a byte stream protocol.
The read() method isn't guaranteed to fill the buffer.
Therefore if you don't receive the expected number of bytes in a single read, you have to loop until you do receive them.
readFully() would have worked if the buffer size agreed with the size of what was sent. In your case you specified a buffer of 10,000 bytes, which weren't sent, so it would have blocked waiting for the other 10000-4948 bytes.

java.net.Socket blocks to timeout if you read too soon after write

Weird behavior with Java sockets code running on Java 7 on an AWS machine:
I have a custom protocol to my server wherein we open a socket, then send and receive BSON messages. The test client creates and opens a socket, sends a request, and then camps on the the socket's InputStream, waiting for the response. When a response is received or when the read times out, the next request is sent.
I've discovered that if I touch the socket's InputStream too quickly after I'm done sending the request through the OutputStream, the socket will occasionally block until its read timeout. I've tried both socket.getInputStream().read(...) and socket.getInputStream().available(); both calls cause the problem. If I simply wait 200ms or so after sending, I get nearly 100% successful reads from the server. If, on systems on the same subnet, if I touch the socket immediately after the write (socket.getOutputStream().write(...); socket.getOutputStream().flush()), the socket blocks until its 20-second timeout is reached for between 1% and 7% of all attempts.
Has anyone else seen this behavior? Do you know what's causing it? Do you have a suggestion on how to deal with it? I expect most reads to come back in between 20 and 40ms on a fast network (they mostly do, except the ones which block and time out).
The actual code in use is pretty complex, but here's some relevant snippets:
High-level read/write:
InputStream is = sock.getInputStream();
OutputStream os = sock.getOutputStream();
String req = getRequestData();
String uuid = UUID.randomUUID().toString();
long start = System.currentTimeMillis();
protocolHandler.write(uuid, getUsername(), os, req);
long dt = System.currentTimeMillis() - start;
if (dt < 125l) {
try { Thread.sleep(125-dt); } catch (InterruptedException ex) {}
}
String in = protocolHandler.read(uuid, is, timer, getResponseCount(), getTimeout());
Socket creation:
private Socket newSocket(String socketKey) {
Socket con = null;
try {
SocketAddress sockaddr = new InetSocketAddress(getServer(), getPort());
con = new Socket();
con.setKeepAlive(true);
if (getPropertyAsString(SO_LINGER,"").length() > 0){
con.setSoLinger(true, getSoLinger());
}
con.connect(sockaddr, getConnectTimeout());
if(log.isDebugEnabled()) {
log.debug("Created new connection " + con); //$NON-NLS-1$
}
Client client = new Client(con);
Client.threadIdToClientMap.put(socketKey, client);
} catch (UnknownHostException e) {
log.warn("Unknown host for " + getLabel(), e);//$NON-NLS-1$
e.printStackTrace(System.err);
return null;
} catch (IOException e) {
log.warn("Could not create socket for " + getLabel(), e); //$NON-NLS-1$
e.printStackTrace(System.err);
return null;
}
// (re-)Define connection params - Bug 50977
try {
con.setSoTimeout(getTimeout());
con.setTcpNoDelay(getNoDelay());
} catch (SocketException se) {
log.warn("Could not set timeout or nodelay for " + getLabel(), se); //$NON-NLS-1$
se.printStackTrace(System.err);
}
return con;
}
Socket write:
public void write(String messageId, String playerId, OutputStream os, String hexEncodedBinary) throws IOException {
String messageHexBytes = substituteVariables(hexEncodedBinary);
AbstractMessage messageObject = MessageRewriter.parseRequestData(messageHexBytes);
int seq = MessageRewriter.getSequence(messageHexBytes);
messageObject.setPassthrough(messageId);
byte[] messageBytes = MessageRewriter.serialize(seq, messageObject);
os.write(messageBytes);
os.flush();
}
Socket read:
public String read(String messageId, InputStream socket, LatencyTimer latencyTimer, int requiredResponseCount, int socketTimeoutMillis)
throws ReadException {
String threadName = "thread "+Thread.currentThread().getId();
StringBuilder result = new StringBuilder("passthrough "+messageId+"\n");
int nBytesThisRead = -1;
byte[] buffer=null;
try {
int nResponses = 0;
// As long as we have bytes or need a response, continue to produce messages. Messages we don't use will be cached.
// The socket object throws an exception on timeout (20 seconds-ish) to terminate on "nothing read".
//
// TODO: refactor this abortion so it's readable and understandable
//
while ((! interrupted) && (nResponses < requiredResponseCount || socket.available() > 0)) {
// clear "buffer" to make log messages less confusing (because on 0-byte reads, residual data is logged if we don't do this)
buffer = new byte[0];
// read the size bytes
int totalBytesRead = 0;
byte[] sizeBuffer = new byte[4];
while (totalBytesRead < BYTES_PER_INTEGER) {
try {
nBytesThisRead = socket.read(sizeBuffer, totalBytesRead, BYTES_PER_INTEGER-totalBytesRead);
if (nBytesThisRead > 0) {
latencyTimer.stop()
totalBytesRead += nBytesThisRead;
}
}
//
// this is the timeout we get ~5% of the time if I don't wait ~ 100ms
//
catch (java.net.SocketTimeoutException e) {
log.error(threadName+" timeout waiting for size bytes");
latencyTimer.stop();
return "";
}
}
int messageSize = getLittleEndianInteger(sizeBuffer);
log.debug(threadName+": message size: " + messageSize);
buffer = Arrays.copyOf(sizeBuffer, BYTES_PER_INTEGER+messageSize);
// reset; now read the message body
totalBytesRead = 0;
while (totalBytesRead < messageSize) {
nBytesThisRead = socket.read(buffer, BYTES_PER_INTEGER+totalBytesRead, messageSize-totalBytesRead);
if (nBytesThisRead > 0)
totalBytesRead += nBytesThisRead;
}
if (totalBytesRead != messageSize) {
log.error(String.format("%s abandoning attempt to read %d responses for id %s. Read %d bytes; needed %d.",
threadName, requiredResponseCount, messageId, totalBytesRead, messageSize));
throw new ReadException(
"Unable to read complete Gluon message.", null, result.toString()+toHex(buffer, BYTES_PER_INTEGER + nBytesThisRead));
}
message = MessageRewriter.deserialize(buffer);
String hexString = toHex(buffer, BYTES_PER_INTEGER + messageSize);
String uuid = message.getPassthrough();
if (messageId.equals(uuid)) {
++nResponses;
}
else {
log.debug(String.format("Read: %s message type %s with msgId %s to cache",
threadName, message.getClass().getSimpleName(), uuid));
messageCache.put(uuid, new MessageToClient(message, hexString));
}
// even ignored messages get sent to the verifiers
if (log.isDebugEnabled()) {
log.debug(String.format("Read message for %s (%d bytes): %s", uuid, BYTES_PER_INTEGER + messageSize, hexString));
log.debug(String.format("%s Read: message type %s with msgId %s from socket; still need %d response messages.",
threadName, message.getClass().getSimpleName(), messageId, requiredResponseCount-nResponses));
}
result.append(hexString);
result.append("\n");
}
} catch (IOException e) {
// TODO: clear out the socket? We'd need a new "open new socket" checkbox in the UI, and fail-fast when unchecked.
String msg = result.toString()+"partial:"+toHex(buffer, BYTES_PER_INTEGER + nBytesThisRead);
log.error(threadName+" throwing read exception; read message so far is:\n"+msg,e);
throw new ReadException("Unable to read expected result.", e, msg);
}
return result.toString();
}

performance issue. The Speed is too slow while reading bytes from socket using java

these days I'm confused about the Tcp performance while using java socket. In fact the java code is very simple. details as below:
server open a port and begin to listen.
client request and after connect to server, client begin to write to socket.
after server got the request, it will open a new thread to handle this connection. (this connection is a long connection which will not time out).
the server will keep reading until it got the end separator, then give a response to the client and continue to keep reading again.
after client get the response, it will send another request again.
I find if the client write the whole message (including the end separator) one time, the communication speed is good satisfactorily, the speed can reach to 50000 messages per minute. How ever, if the client write the bytes to socket in separated times, the speed cut down quickly, just almost 1400 messages per minute, it is 1/40 times compared with the original speed. I'm quite confused about it. Any one could give me a hand? Any comments is appreciated!
my simulated server side is as below:
public class ServerForHelp {
final static int BUFSIZE = 10240;
Socket socket;
String delimiter = "" + (char) 28 + (char) 13;
public static void main(String[] args) throws IOException {
ServerSocket ss = new ServerSocket(9200);
System.out.println("begin to accept...");
while (true) {
Socket s = ss.accept();
Thread t = new Thread(new SocketThread1(s));
t.start();
}
}
public String readUntilDelimiter() throws Exception {
StringBuffer stringBuf = new StringBuffer();
InputStream stream = socket.getInputStream();
InputStreamReader reader = null;
reader = new InputStreamReader(stream);
char[] buf = new char[BUFSIZE];
while (true) {
int n = -1;
n = reader.read(buf, 0, BUFSIZE);
if (n == -1) {
return null; // it means the client has closed the connection, so return null.
} else if (n == 0) {
continue; // continue to read the data until got the delimiter from the socket.
}
stringBuf.append(buf, 0, n);
String s = stringBuf.toString();
int delimPos = s.indexOf(delimiter);
if (delimPos >= 0) {
// found the delimiter; return prefix of s up to separator and
// To make the thing simple, I have discarded the content after the delimiter.
String result = s.substring(0, delimPos);
sendTheResponse(socket);
return result;
}
}
}
private void sendTheResponse(Socket socket) throws IOException {
Writer writer = new BufferedWriter(new OutputStreamWriter(socket.getOutputStream()));
writer.write("Hi, From server response");
writer.flush();
}
}
class SocketThread1 implements Runnable {
Socket socket;
public SocketThread1(Socket socket) {
this.socket = socket;
}
#Override
public void run() {
ServerForHelp server = new ServerForHelp();
server.socket = socket;
while (true) {
try {
if (server.readUntilDelimiter() == null) // it means that the client has closed the connection, exist
break;
} catch (Exception e) {
e.printStackTrace();
}
}
}
}
It is a normal socket programming.
and the following is my client side:
public void execute() throws Exception{
int msgCnt = 0;
Socket socket = null;
byte[] bufBytes = new byte[512];
long start = 0;
final char START_MESSAGE = 0x0B;
final char END_MESSAGE = 0x1C;
final char END_OF_RECORD = 0x0D;//\r
String MESSAGE = "HELLO, TEST";
socket = new Socket("192.168.81.39", 9200);
OutputStream os = socket.getOutputStream();
InputStream is = socket.getInputStream();
while (System.currentTimeMillis() - start < 60000)
{
// If you send the total message at one time, the speed will be improved significantly
// FORMAT 1
StringBuffer buf = new StringBuffer();
buf.append(START_MESSAGE);
buf.append(MESSAGE);
buf.append(END_MESSAGE);
buf.append(END_OF_RECORD);
os.write(buf.toString().getBytes());
// FORMAT 1 END
//FORMAT 2
// os.write(START_MESSAGE);
// os.write(MESSAGES[port].getBytes());
// os.write(END_MESSAGE);
// os.write(END_OF_RECORD);
//FORMAT 2 END
os.flush();
is.read(bufBytes);
msgCnt++;
System.out.println(msgCnt);
}
System.out.println( msgCnt + " messages per minute");
}
If I use the "FORMAT 1", to send the message, the speed could reach to 50000 messages per minute, but If use "FORMAT 2", the speed is down to 1400 messages per minute. Who is clear about the reason?
I'm trying to describe as detail as I can and any help will be appreciated very much.

Multiple very short writes to a socket in rapid succession followed by a read can trigger a bad interaction between Nagle's algorithm and TCP delayed acknowledgment; even if you disable Nagle's algorithm, you'll cause an entire packet to be sent per individual write call (with 40+ bytes of overhead, whether the write is one byte or a thousand).
Wrapping a BufferedOutputStream around the socket's output stream should give you performance similar to "FORMAT 1" (precisely because it holds things in a byte array until it fills or is flushed).
As John Nagle explained on Slashdot:
The user-level solution is to avoid write-write-read sequences on sockets. write-read-write-read is fine. write-write-write is fine. But write-write-read is a killer. So, if you can, buffer up your little writes to TCP and send them all at once.

Socket connectivity loss when sending live streaming data

I'm struggling here...
I'm trying to determine if data was successfully sent to the server through a TCP socket using the OutputStream object. For testing on emulator socket communications is loss after 30 sec. For write data OutputStream.write(); its doesn't throw an exception , and local server continuously running its not crashing, only tcp socket connection is loss after some time. All the methods in the socket class return as though the socket is active and working. Is there anything I'm doing wrong here? Is there any socket implementation or stream implementation I can use to get an exception or error when the stream/ socket doesn't actually send the data in the buffer? Also setting setSoTimeout() on the socket doesn't seem to do anything.
Please guide me...
Here is my code:
private void sendRec() {
int lstream;
int port = 1012;
byte[] byterecv = new byte[1040];
while (true) {
System.out.println("POOL-2");
synchronized (recSendThread) {
try {
recSendThread.wait(20);
} catch (InterruptedException e) {
}
}
if (stopcall == true) {
// break;
}
try {
// Provides a client-side TCP socket
Socket clientRec = new Socket();
// serverSocket = new ServerSocket(port);
// serverSocket.setSoTimeout(5000);
// Connects this socket to the given remote host address and
// port
clientRec.connect(new InetSocketAddress("192.168.1.36", port));
System.out.println("Just connected to "
+ clientRec.getRemoteSocketAddress());
System.out.println("SENTS Rec BEFORE");
// output streams that write data to the network
OutputStream outToServerRec = clientRec.getOutputStream();
DataOutputStream outStreamRec = new DataOutputStream(
outToServerRec);
outStreamRec.write(bData);
System.out.println("SENTS Rec AFTER");
// input streams that read data from network
InputStream inFromServerRec = clientRec.getInputStream();
// clientRec.setSoTimeout(5000);
DataInputStream inStreamRec = new DataInputStream(
inFromServerRec);
while ((lstream = inStreamRec.read(byterecv)) != -1) {
System.out.println("startrec bytearray -- "
+ byterecv.length);
bos1.write(byterecv, 0, lstream);
}
inStreamRec.close();// for closing dataouputstream
clientRec.close();// for closing socket connection
} catch (IOException e) {
e.printStackTrace();
}
}
}
Here is my receiver and player code..
/**
* start receiving the voice data from server
* */
protected void startplay() {
System.arraycopy(frndid, 0, playByteData, 0, 4);
System.arraycopy(userid, 0, playByteData, 4, 4);
ByteBuffer.wrap(sessionid).order(ByteOrder.LITTLE_ENDIAN).asIntBuffer().
put(call_sessionid);
System.arraycopy(sessionid, 0, playByteData, 8, 4);
int lstream;
int port = 1014;
while (true) {
System.out.println("POOL-3");
try {
if (stopcall == true) {
System.out.println("BREAKEDDDD1111");
//break;
}
// Host name
// port++;
InetAddress addressPlay = InetAddress.getByName("192.168.1.36");
// Creates a new streaming socket connected to the target host
Socket clientPlay = new Socket(addressPlay, port);
System.out.println("Just connected to play : " +
clientPlay.getRemoteSocketAddress());
System.out.println("SENTS Play BEFORE");
// output streams that write data
OutputStream outToServer = clientPlay.getOutputStream();
DataOutputStream outStreamPlay = new DataOutputStream(outToServer);
outStreamPlay.write(playByteData);
System.out.println("SENTS Play after");
// input streams that read data
InputStream inFromServerPlay = clientPlay.getInputStream();
DataInputStream inStreamPlay = new DataInputStream(inFromServerPlay);
//clientPlay.setSoTimeout(5000);
while ((lstream = inStreamPlay.read(byteArray)) != -1) {
System.out.println("startplay() bytearray -- " +
byteArray.length);
bos.write(byteArray, 0, lstream);
}
inStreamPlay.close();
clientPlay.close();// for closing play socket connection
responseBuffer = bos.toByteArray();
System.out.println("BAOSSIZE " + bos.size());
bos.reset();
bos.flush();
bos.close();
playing = true;
System.out.println("res length -- " + responseBuffer.length);
rcvbb=ByteBuffer.wrap(responseBuffer).order(ByteOrder.LITTLE_ENDIAN).
asShortBuffer().get(playShortData);
playVoiceReceived();// plays received data
} catch (ParseException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
}
/**
* start playing received the voice data from server
* */
public void playVoiceReceived() {
System.out.println("POOL-4");
try {
if (at != null) {
if (at.getPlayState() != AudioTrack.PLAYSTATE_PLAYING) {
at.play();// starts playing
} else {
System.out.println("Play BEFORE WRITE");
// Writes the audio data to the audio hardware for playback.
at.write(playShortData, 0, BufferElements2Play);
System.out.println("Play AFTER WRITE");
at.flush();
}
}
} catch (Exception e) {
e.printStackTrace();
}
}

The socket has sent the data ... to the local socket send buffer. What happens after that is up to the local TCP stack, the network, the remote TCP stack, and the remote application. If you want to know whether the remote application got the data, it will have to send you a reply.

Operator write does not check whether data was delivered because otherwise it would have to wait for too long time. If network connection is actually down, TCP layer of operating system, will try to send data anyway, but will detect problems somewhat later, i.e. 1 minute later, because it will not receive acknowledgement messages from the opposite side. It will then try to resend data several times, see that problem persists and only then will report exception condition on the socket. To know that socket is in exception condition, you need to perform some operator on socket, i.e. another write attempt. Try doing write in a loop like this:
while (true)
{
outStreamRec.write (data);
Thread.sleep (1000L);
}
It should throw an error about 2 minutes after network will be down.
Note, that in opposite to write operation, operation connect is synchronous, so it actually waits for response from the opposite side, and if there is not respose, it will throw an exception.