Develop Reference

first message over socket lost in Eclipse

I have this funny behavior in debug mode which I can't explain to myself.
There are two small client and server applications, and the flow goes like this:
Client initiates TCP connection
Server accepts and sends message to client requesting packet data size(it's fixed)
Client sends the size
Server reads the size and initializes some byte array - this works
Server blocks on read() from the input stream waiting on packets and client sends packets. - this is culprit
It all works when i run the applications, however when I debug the Server - it simply blocks on read().
If i send more than one message, for example 50, the Server receives 49 of those. Once client closes connection, Server reads -1 from stream and exits, with the first message lost.
The only thing i can think of is that in debug mode, client sends message before server reads it from stream but i don't see how could that be relevant as the message should be there.
Could somebody explain this behavior i am experiencing in debug mode only?
Note: client code is there just for debug, so it's not the prettiest.
Server
private static int sent;
public void processClientInput(HealthCheckSession session) {
byte[] buffer = session.getBuffer();
DataInputStream stream = session.getInFromClient();
int readBytes = 0;
int totalBytes = 0;
try {
while((readBytes = stream.read(buffer)) !=-1) { <---- this blocks in debug on first message
totalBytes += readBytes;
if (totalBytes < buffer.length) {
continue;
}
else {
totalBytes = 0;
String packet = new String(buffer, ProtocolValidator.SERIALIZATION_CHARSET);
sendResponse(packet);
}
}
} catch (IOException e) {e.printStackTrace();}
}
private void sendResponse(String packet) {
System.out.println(packet);
++sent;
System.out.println("sent " + sent);
}
Client
public static void main(String[] args) throws UnknownHostException,
IOException, InterruptedException {
Socket clientSocket = new Socket("localhost", 10002);
// get the socket's output stream and open a PrintWriter on it
PrintWriter outToServer = new PrintWriter(
clientSocket.getOutputStream(), true);
// get the socket's input stream and open a BufferedReader on it
BufferedReader inFromServer = new BufferedReader(new InputStreamReader(
clientSocket.getInputStream()));
System.out.println("before read");
System.out.println(inFromServer.readLine());
System.out.println("after read");
byte[] packageData = "100, ABCDZEGHR"
.getBytes(Charset.forName("UTF-8"));
String size = String.valueOf(packageData.length);
outToServer.println(size);
for (int i = 0; i < 50; i++) {
DataOutputStream dis = new DataOutputStream(
clientSocket.getOutputStream());
System.out.println("before write");
dis.write(packageData);
dis.flush();
System.out.println("after write");
Thread.sleep(1000);
}
try {
Thread.sleep(100 * 1000);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
clientSocket.close();
}
IMPORTANT UPDATE
This does not happen in Netbeans debug mode.

java.net.Socket blocks to timeout if you read too soon after write

Weird behavior with Java sockets code running on Java 7 on an AWS machine:
I have a custom protocol to my server wherein we open a socket, then send and receive BSON messages. The test client creates and opens a socket, sends a request, and then camps on the the socket's InputStream, waiting for the response. When a response is received or when the read times out, the next request is sent.
I've discovered that if I touch the socket's InputStream too quickly after I'm done sending the request through the OutputStream, the socket will occasionally block until its read timeout. I've tried both socket.getInputStream().read(...) and socket.getInputStream().available(); both calls cause the problem. If I simply wait 200ms or so after sending, I get nearly 100% successful reads from the server. If, on systems on the same subnet, if I touch the socket immediately after the write (socket.getOutputStream().write(...); socket.getOutputStream().flush()), the socket blocks until its 20-second timeout is reached for between 1% and 7% of all attempts.
Has anyone else seen this behavior? Do you know what's causing it? Do you have a suggestion on how to deal with it? I expect most reads to come back in between 20 and 40ms on a fast network (they mostly do, except the ones which block and time out).
The actual code in use is pretty complex, but here's some relevant snippets:
High-level read/write:
InputStream is = sock.getInputStream();
OutputStream os = sock.getOutputStream();
String req = getRequestData();
String uuid = UUID.randomUUID().toString();
long start = System.currentTimeMillis();
protocolHandler.write(uuid, getUsername(), os, req);
long dt = System.currentTimeMillis() - start;
if (dt < 125l) {
try { Thread.sleep(125-dt); } catch (InterruptedException ex) {}
}
String in = protocolHandler.read(uuid, is, timer, getResponseCount(), getTimeout());
Socket creation:
private Socket newSocket(String socketKey) {
Socket con = null;
try {
SocketAddress sockaddr = new InetSocketAddress(getServer(), getPort());
con = new Socket();
con.setKeepAlive(true);
if (getPropertyAsString(SO_LINGER,"").length() > 0){
con.setSoLinger(true, getSoLinger());
}
con.connect(sockaddr, getConnectTimeout());
if(log.isDebugEnabled()) {
log.debug("Created new connection " + con); //$NON-NLS-1$
}
Client client = new Client(con);
Client.threadIdToClientMap.put(socketKey, client);
} catch (UnknownHostException e) {
log.warn("Unknown host for " + getLabel(), e);//$NON-NLS-1$
e.printStackTrace(System.err);
return null;
} catch (IOException e) {
log.warn("Could not create socket for " + getLabel(), e); //$NON-NLS-1$
e.printStackTrace(System.err);
return null;
}
// (re-)Define connection params - Bug 50977
try {
con.setSoTimeout(getTimeout());
con.setTcpNoDelay(getNoDelay());
} catch (SocketException se) {
log.warn("Could not set timeout or nodelay for " + getLabel(), se); //$NON-NLS-1$
se.printStackTrace(System.err);
}
return con;
}
Socket write:
public void write(String messageId, String playerId, OutputStream os, String hexEncodedBinary) throws IOException {
String messageHexBytes = substituteVariables(hexEncodedBinary);
AbstractMessage messageObject = MessageRewriter.parseRequestData(messageHexBytes);
int seq = MessageRewriter.getSequence(messageHexBytes);
messageObject.setPassthrough(messageId);
byte[] messageBytes = MessageRewriter.serialize(seq, messageObject);
os.write(messageBytes);
os.flush();
}
Socket read:
public String read(String messageId, InputStream socket, LatencyTimer latencyTimer, int requiredResponseCount, int socketTimeoutMillis)
throws ReadException {
String threadName = "thread "+Thread.currentThread().getId();
StringBuilder result = new StringBuilder("passthrough "+messageId+"\n");
int nBytesThisRead = -1;
byte[] buffer=null;
try {
int nResponses = 0;
// As long as we have bytes or need a response, continue to produce messages. Messages we don't use will be cached.
// The socket object throws an exception on timeout (20 seconds-ish) to terminate on "nothing read".
//
// TODO: refactor this abortion so it's readable and understandable
//
while ((! interrupted) && (nResponses < requiredResponseCount || socket.available() > 0)) {
// clear "buffer" to make log messages less confusing (because on 0-byte reads, residual data is logged if we don't do this)
buffer = new byte[0];
// read the size bytes
int totalBytesRead = 0;
byte[] sizeBuffer = new byte[4];
while (totalBytesRead < BYTES_PER_INTEGER) {
try {
nBytesThisRead = socket.read(sizeBuffer, totalBytesRead, BYTES_PER_INTEGER-totalBytesRead);
if (nBytesThisRead > 0) {
latencyTimer.stop()
totalBytesRead += nBytesThisRead;
}
}
//
// this is the timeout we get ~5% of the time if I don't wait ~ 100ms
//
catch (java.net.SocketTimeoutException e) {
log.error(threadName+" timeout waiting for size bytes");
latencyTimer.stop();
return "";
}
}
int messageSize = getLittleEndianInteger(sizeBuffer);
log.debug(threadName+": message size: " + messageSize);
buffer = Arrays.copyOf(sizeBuffer, BYTES_PER_INTEGER+messageSize);
// reset; now read the message body
totalBytesRead = 0;
while (totalBytesRead < messageSize) {
nBytesThisRead = socket.read(buffer, BYTES_PER_INTEGER+totalBytesRead, messageSize-totalBytesRead);
if (nBytesThisRead > 0)
totalBytesRead += nBytesThisRead;
}
if (totalBytesRead != messageSize) {
log.error(String.format("%s abandoning attempt to read %d responses for id %s. Read %d bytes; needed %d.",
threadName, requiredResponseCount, messageId, totalBytesRead, messageSize));
throw new ReadException(
"Unable to read complete Gluon message.", null, result.toString()+toHex(buffer, BYTES_PER_INTEGER + nBytesThisRead));
}
message = MessageRewriter.deserialize(buffer);
String hexString = toHex(buffer, BYTES_PER_INTEGER + messageSize);
String uuid = message.getPassthrough();
if (messageId.equals(uuid)) {
++nResponses;
}
else {
log.debug(String.format("Read: %s message type %s with msgId %s to cache",
threadName, message.getClass().getSimpleName(), uuid));
messageCache.put(uuid, new MessageToClient(message, hexString));
}
// even ignored messages get sent to the verifiers
if (log.isDebugEnabled()) {
log.debug(String.format("Read message for %s (%d bytes): %s", uuid, BYTES_PER_INTEGER + messageSize, hexString));
log.debug(String.format("%s Read: message type %s with msgId %s from socket; still need %d response messages.",
threadName, message.getClass().getSimpleName(), messageId, requiredResponseCount-nResponses));
}
result.append(hexString);
result.append("\n");
}
} catch (IOException e) {
// TODO: clear out the socket? We'd need a new "open new socket" checkbox in the UI, and fail-fast when unchecked.
String msg = result.toString()+"partial:"+toHex(buffer, BYTES_PER_INTEGER + nBytesThisRead);
log.error(threadName+" throwing read exception; read message so far is:\n"+msg,e);
throw new ReadException("Unable to read expected result.", e, msg);
}
return result.toString();
}

Java Android Sockets

I'm currently working on an Android app which sends an string and a file to a java server app running on remote computer. This java server app should find the index on the file and send back the value of this index (The file structure is: index value. Example: 1 blue) The file is properly sent and received on the remote machine and I have a method which finds the value of the received index on the file. But when I'm trying to send the found value back to the phone I get an exception (closed socket), but I'm not closing the socket or any buffer. I'm not sure if the socket which is closed is the mobile app socket or the java server app socket. I'm using the same socket I use to send to receive (which is the way to work on Android). Sending the answer back to the phone is what my project is missing and is what I need help in. Here is my code:
Client app (Android app):
private class HeavyRemProcessing extends AsyncTask<String, Void, String>
{
protected String doInBackground(String... urls)
{
begins = System.currentTimeMillis();
remoteExecution();
ends= System.currentTimeMillis();
procTime=ends-begins;
aux= Long.toString(procTime);
return aux;
} //doInBackground() ends
protected void onPostExecute(String time)
{
textView1.setText("Result: "+result+". Processing Time: "+time+" milisecs");
}// onPostExecute ends
} //HeavyRemProcessing ends
public void executor(View view)
{
key="74FWEJ48DX4ZX8LQ";
HeavyRemProcessing task = new HeavyRemProcessing();
task.execute(new String[] { "????" });
} //executor() ends
public void remoteExecution()
{
// I have fixed IP and port I just deleted
String ip; //SERVER IP
int port; // SERVER PORT
try
{
cliSock = new Socket(ip, port);
file= new File("/mnt/sdcard/download/Test.txt");
long length = file.length();
byte[] bytes = new byte[(int) length];
FileInputStream fis = new FileInputStream(file);
BufferedInputStream bis = new BufferedInputStream(fis);
BufferedOutputStream out = new BufferedOutputStream(cliSock.getOutputStream());
BufferedReader in = new BufferedReader(new InputStreamReader(cliSock.getInputStream()));
int count;
key=key+"\r\n";
out.write(key.getBytes());
while ((count = bis.read(bytes)) > 0)
{
out.write(bytes, 0, count);
} //It works perfectly until here
//// PROBABLY HERE IS THE PROBLEM:
out.flush();
out.close();
fis.close();
bis.close();
result= in.readLine(); //RECEIVE A STRING FROM THE REMOTE PC
}catch(IOException ioe)
{
// Toast.makeText(getApplicationContext(),ioe.toString() + ioe.getMessage(),Toast.LENGTH_SHORT).show();
}
}catch(Exception exp)
{
//Toast.makeText(getApplicationContext(),exp.toString() + exp.getMessage(),Toast.LENGTH_SHORT).show();
}
} //remoteExecution ends
Java Server App (Remote PC)
public void receivingFile()
{
System.out.println("Executing Heavy Processing Thread (Port 8888).");
try
{
serverSocket = new ServerSocket(8888);
InputStream is = null;
OutputStream os= null;
FileOutputStream fos = null;
BufferedOutputStream bos = null;
BufferedOutputStream boSock =null;
DataOutputStream dataOutputStream=null;
int bufferSize = 0;
try
{
socket = serverSocket.accept();
System.out.println("Heavy Processing Task Connection from ip: " + socket.getInetAddress());
} catch (Exception ex)
{
System.out.println("Can't accept client connection: "+ex);
}
try
{
is = socket.getInputStream();
dataOutputStream = new DataOutputStream(socket.getOutputStream());
bufferSize = socket.getReceiveBufferSize();
}
catch (IOException ex)
{
System.out.println("Can't get socket input stream. ");
}
try
{
fos = new FileOutputStream(path);
bos = new BufferedOutputStream(fos);
}
catch (FileNotFoundException ex)
{
System.out.println("File not found. ");
}
byte[] bytes = new byte[bufferSize];
int count;
System.out.println("Receiving Transfer File!.");
while ((count = is.read(bytes)) > 0)
{
bos.write(bytes, 0, count);
}
System.out.println("File Successfully Received!.");
fos.close();
bos.flush();
bos.close();
is.close();
result= obj.searchIndex();
System.out.println("Found: "+result); //This correctly print the found value
dataOutputStream.writeUTF(result);
dataOutputStream.flush();
dataOutputStream.close();
System.out.println("Data sent back to the Android Client. ");
} catch (IOException e)
{
// TODO Auto-generated catch block
e.printStackTrace();
}
} // receivingFile() ends
Please if someone can help me I will really appreciate it. I'm thinking is something probably related with the buffers and the socket. My java server app throws an exception: "Closed Socket"... Thanks for your time,
Alberto.

I think your problem is that you closing the outputstream before closing the inputstream. This is a bug in android. Normally in java closing outputstream only flushes the data and closing inputstream causes the connection to be closed. But in android closing the outputstream closes the connection. That is why you are getting closed socket exception,
Put the statements
out.flush();
out.close();
after
result=in.readLine();
or just avoid those statements(out.flush and out.close). I had also faced a similar problem. See my question

Sending multiple images over socket

what im trying to do:
client connects to server
server sends READY
client takes screenshot and sends it
server processes image
server sends READY
client takes screenshot and sends it
server processes image
...
i have a working client and server:
Client() {
try {
socket = new Socket(host, 4444);
in = new DataInputStream(socket.getInputStream());
out = new DataOutputStream(socket.getOutputStream());
int ix = 0;
while (true) {
switch (in.readInt()) {
case Var.READY:
image = new Robot().createScreenCapture(new Rectangle(Toolkit.getDefaultToolkit().getScreenSize()));
ByteArrayOutputStream byteArrayO = new ByteArrayOutputStream();
ImageIO.write(image,"PNG",byteArrayO);
byte [] byteArray = byteArrayO.toByteArray();
out.writeInt(byteArray.length);
out.write(byteArray);
System.out.println("send screen " + ix++);
break;
}
}
} catch (UnknownHostException e) {
System.err.println("Don't know about host");
System.exit(1);
} catch (IOException e) {
System.err.println("Couldn't get I/O for the connection " + e.getMessage());
System.exit(1);
} catch (Exception e) {
e.printStackTrace();
System.exit(1);
}
}
Server:
public class ServerWorker implements Runnable {
private Socket socket = null;
DataInputStream in = null;
DataOutputStream out = null;
ServerWorker() {
}
synchronized void setSocket(Socket socket) {
this.socket = socket;
try {
in = new DataInputStream(socket.getInputStream());
out = new DataOutputStream(socket.getOutputStream());
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
notify();
}
public synchronized void run() {
int ix = 0;
try {
while (true) {
out.writeInt(Var.READY);
int nbrToRead = in.readInt();
byte[] byteArray = new byte[nbrToRead];
int nbrRd = 0;
int nbrLeftToRead = nbrToRead;
while(nbrLeftToRead > 0){
int rd =in.read(byteArray, nbrRd, nbrLeftToRead);
if(rd < 0)
break;
nbrRd += rd; // accumulate bytes read
nbrLeftToRead -= rd;
}
//Converting the image
ByteArrayInputStream byteArrayI = new ByteArrayInputStream(byteArray);
BufferedImage image = ImageIO.read(byteArrayI);
System.out.println("received screen " + ix++);
//image.flush();
File of = new File("RecvdImg" + ix + ".jpg");
ImageIO.write(image, "PNG" ,of);
System.out.println("Sleeping 1..");
Thread.sleep(1000);
}
} catch (Exception e) {
e.printStackTrace();
Thread.currentThread().interrupt();
}
}
}
So whats the question you might ask?
Well, am i doing it right?
Activity monitor tells me the client side takes about 40% of cpu constantly, that cant be good.
Just wondering if anyone could point me in the right direction to making the code more efficient.

Client could detect if the image has changed, and if it hasn't it could send to the server a flag indicating to reuse the previous image received. Or you can "diff" the image and send only the changed areas to the server, which will recompose the image. That reduces bandwidth usage and perhaps also CPU usage.
Also, the client should sleep a while in the receiving endless loop, after the switch.

In my opinion you should avoid using infinit-loops like
while (true)
Loops like
while(!connectionAborted)
are better in such situations.
Also you should take a look at
Socket.setSoTimeout()
SoTimeout cancels the reading-process of i.e. in.readInt() after an specific amount of time, depending on your parameter.
The result is that at this line a SocketTimeoutException is thrown, but your code is not stuck at this codeline and can react on i.e. different user inputs.

Timeout on a java util scanner?

I have the following problem, I have a simple TCP class in my application that sends a message off to a device for a query, the device then responds with the message however there is no end of line character of any description because it is coming from a serial converter, after initially atempting to use the readline function and discovering it requires the eol character before outputting I have tried the scanner function which works fine unless the device doesnt reply to that request for some reason, my application then freezes, is it possible to set a timeout on the scanner function so that it then drops the connection and moves on or is there a better way to do this? my code is below:
public String Send_TCP ( InetAddress IPAddress, int POrt, String InData) throws IOException
{
Socket socket = null;
PrintWriter out = null;
BufferedReader in = null;
try {
socket = new Socket(IPAddress, POrt);
out = new PrintWriter(socket.getOutputStream(), true);
in = new BufferedReader(new InputStreamReader(socket.getInputStream()));
} catch (UnknownHostException e) {
System.err.println("Don't know about host");
System.exit(1);
} catch (IOException e) {
System.err.println("Couldn't get I/O for the connection");
System.exit(1);
}
BufferedReader read = new BufferedReader(new InputStreamReader(System.in));
;
System.out.print("Connected, Sending:"+ InData);
out.println(InData);
System.out.println("Equals");
String str1 = new Scanner(in).useDelimiter(">").next() + ">";
System.out.println(str1);
System.out.println("Equals");
out.close();
in.close();
read.close();
socket.close();
return str1;
}
}

I'm not sure that I understand your question correctly but you can set a timeout on the socket: socket.setSoTimeout(int timeout).
See: javadoc

I believe the following achieves what I need it to, basically checking if the buffer exists, if it doesnt then it waits and checks again avoiding the trap of the scanner function if the message never arrives if it does it reads it.
try {
BufferedReader rd = new BufferedReader(new InputStreamReader(sock.getInputStream()));
int count = 1;
do {
if (rd.ready()){
System.out.println ("Response Ready");
str = new Scanner(rd).useDelimiter(">").next()+">";
count = 501;
}
Thread.sleep(10);
System.out.println ("Response Not Ready" + count);
count ++;
} while (count < 25);