Develop Reference

Java socket data works only when sending time is delayed

I am sending a protobuf from C++ to Java via a raw socket, the C++ program being the client and the java program being the server. The C++ program generates packets almost every 1ms which is sent to the java program.
If I run the program normally, I see that there are only the half the packets being received.
If I set a breakpoint in the C++ program and then run the client and the server, all the packets are received.
How do I ensure that all packets are received without setting a breakpoint? Can I introduce a delay?
All the packets have bytes sizes upto a maximum of 15 bytes.

By default TCP sockets use the "Nagle Algorithm" which will delay transmission of the next "unfilled" fragment in order to reduce congestion. Your packet size is small enough and the time delay between packets is small enough that the nagle algorithm will have an effect on your transmissions.

As already discussed in the comments, what you are trying to do won't work in a reliable way. This is also described in the Protobuf documentation:
If you want to write multiple messages to a single file or stream, it
is up to you to keep track of where one message ends and the next
begins. The Protocol Buffer wire format is not self-delimiting, so
protocol buffer parsers cannot determine where a message ends on their
own. The easiest way to solve this problem is to write the size of
each message before you write the message itself. When you read the
messages back in, you read the size, then read the bytes into a
separate buffer, then parse from that buffer. (If you want to avoid
copying bytes to a separate buffer, check out the CodedInputStream
class (in both C++ and Java) which can be told to limit reads to a
certain number of bytes.)
The bold italic part is where you code isn't correct.
On the write side you should write
the Protobuf's length in some format that is understandable for both sender and receiver (selecting the proper format is especially important when transporting between systems whose endianness is different).
the protobuf
On the receiving end you need to
perform a read with the fixed, known size of the length field
a read for the length learned in step 1. This read will retriev the protobuf.
There's example code here on SO in this question: Sending struct via Socket using JAVA and C++

#fvu: This is my code which I am trying:
import Visualization.DataSetProtos.PacketData; // protos import
import java.io.InputStream;
import java.util.Arrays;
import javax.swing.JFrame;
import javax.swing.JScrollBar;
import javax.swing.JScrollPane;
class WorkerThread extends Thread {
Socket service;
static DynamicData demo;
static int size;
static int times;
static byte[] buffer;
WorkerThread(Socket service)
{
this.service = service;
buffer = new byte[500];
size = 1;
times = 0;
}
static void Print(PacketData packetData)
{
System.out.print("Packet Number: " + (++times));
System.out.print(" DataSet Size: " + packetData.getLength() + "\n");
}
static void Print(PacketHeader packetHeader)
{
System.out.print("Packet Number: " + (++times));
System.out.print(" DataSet Size: " + packetHeader.getLength() + "\n");
}
public void run() {
boolean flag=true; //you can change this flag's condition, to test if the client disconects
if(demo == null)
{
demo = new DynamicData("GridMate Data Visualization");
demo.pack();
RefineryUtilities.centerFrameOnScreen(demo);
//demo.setVisible(true);
}
try
{
while (flag)
{
InputStream inputStream = service.getInputStream();
int read;
read = inputStream.read(buffer);
byte[] readBuffer = new byte[read];
readBuffer = Arrays.copyOfRange(buffer, 0, read);
PacketData packetData = PacketData.parseFrom(readBuffer);
Print(packetData);
}
service.close();
}
catch(Exception e)
{
e.printStackTrace();
}
}
}
public class Test
{
Test()
{
server = null;
client= null;
}
public static void main(final String[] args) {
int i =0;
try
{
server = new ServerSocket(25715);
System.out.println("Server setup and waiting for client connection ...");
while(true)
{
client = server.accept();
WorkerThread wt = new WorkerThread(client);
wt.start();
i++;
}
}
catch(IOException e)
{ System.out.println("IO Error in streams " + e);
e.printStackTrace();
}
}
public void finalize()
{
try
{
server.close();
client.close();
}
catch(Exception e)
{
e.printStackTrace();
}
}
static ServerSocket server;
static Socket client;
}

Why isn't my Socket closing on server side?

I'm trying to make a simple text editor which can be shared accross multiple terminals at the same time. I have a Server waiting for new users, when a user enters the shared editor it just starts waiting for input characters.
public class Server {
public static final int PORT = 8080;
public static void main(String[] args) throws IOException {
ServerSocket ss = new ServerSocket(PORT);
while (true) {
Socket socket = ss.accept();
System.out.println("A new user entered the sever");
new Thread(() -> serve(socket)).start();
}
}
private static void serve(Socket socket) {
try {
while (!socket.isClosed() && !socket.isInputShutdown()) {
System.out.println("hey " + socket.isClosed() + " " + socket.isInputShutdown());
System.out.print(new String(SocketUtil.receiveBytes(socket,1)));
}
} catch (IOException e) {
e.printStackTrace();
}
}
}
When a user closes the editor, the socket is closed on client side. However, the socket on the server side does not get closed and the server starts looping infinitly in the "wait for input" loop.
The Client is a singleton containing the following methods, called at the openning and closing of the editor.
public static void init() {
try {
if (socket == null) socket = new Socket(HOST,Server.PORT);
} catch (IOException e) {
e.printStackTrace();
kill();
throw new Error(e.getMessage());
}
}
public static void kill() {
Check.notNull(socket);
try {
SocketUtil.terminateCommunication(socket);
System.out.println(socket.isClosed());
} catch (IOException e) {
e.printStackTrace();
}
}
Finally, here are utilitary methods (in SocketUtil) used in both classes :
public static void terminateCommunication(Socket socket) throws IOException {
socket.shutdownInput();
socket.shutdownOutput();
socket.close();
}
public static char[] receiveBytes(Socket socket, int nBytes) throws IOException {
char[] bytes = new char[nBytes];
InputStreamReader isr = new InputStreamReader(socket.getInputStream());
isr.read(bytes);
return bytes;
}
Any idea of why the socket on server side is not closed after the Client gets killed ?

It is not quite clear from the Javadoc, but isClosed() only returns true when you have explicitly called close() on the socket (see the sources to confirm that). You should check for exceptions and the return value of read() instead. If you read -1 or catch an IOException while trying to read (or write, for that matter), it essentially means that the other side has closed the connection, so you should close your socket as well (better to it in a finally block) and you're done with that particular connection. You don't check for -1 in receiveBytes(), but you really should. Perhaps throw a EOFException() if you want to merge these two possibility into one, so that the code up the stack (in serve()) doesn't have to figure out what exactly happened:
public static char[] receiveBytes(Socket socket, int nBytes) throws IOException {
char[] bytes = new char[nBytes];
InputStreamReader isr = new InputStreamReader(socket.getInputStream());
if (isr.read(bytes) == -1)
throw new EOFException();
return bytes;
}
One exception from the IOException rule (sorry for the pun) is the SocketTimeoutException. If you get this, the connection is still alive, and you may just as well retry your read(). But I believe that in order to get these, you must call Socket.setSoTimeout() somewhere, and if you haven't, then you probably shouldn't worry about SocketTimeoutException.
You should also note that read() may sometimes return partial reads (that is, less than bytes.length). If it's important that receiveBytes() reads exactly nBytes (which probably is, since you never return the number of actual characters read), then you should call it in a loop, like this:
int pos = 0;
while (pos < bytes.length) {
int l;
if ((l = isr.read(bytes, pos, bytes.length - pos)) == -1) {
throw new EOFException();
}
pos += l;
}
I know this is cumbersome, which is exactly why many developers create utility methods like your receiveBytes().

The proper way to detect that the client has closed its connection is by checking the reception of 0 bytes.
System.out.print(new String(SocketUtil.receiveBytes(socket,1)));
just check if the string is empty should do the trick.
Note that I am not that familiar with java, but I do know socket programming.
Receiving 0 bytes, checking for that, and closing the socket if you do is a good solution.
You can use exception handling too, but you'll detect that the peer closed it socket an iteration later. Receiving 0 bytes is not really an error condition it is just a signal from the peer that he has closed its end of the socket and won't send anymore data. If you ignore this, and keep using the socket, you'll receive an exception in the next iteration because there is nothing to receive anymore.

Unable to read data bytes from input stream in java tcp socket

In my Server code, I send different request to client and get back the response but only first read request is accessed, during accessing second read statement,it is unable to read Data bytes,my code is as followed.
private static boolean Rt_Request(int id,Object client)throws Exception
{
int size=5;
byte[] buf=new byte[size];
char[] cbuf=new char[32];
int byteRead; Socket s=(Socket)client;
BufferedReader in1= new BufferedReader(new InputStreamReader(s.getInputStream()));
PrintStream out=new PrintStream(s.getOutputStream());
try {
buf[0]=0x02;
buf[1]=0x09;
buf[2]=0x01;
buf[3]=0x00;
buf[4]=0x03;
Thread.sleep(5000);
out.write(buf, 0, 5);
} catch(Exception e) {
System.out.println("Error Occured...."+e);
}
byteRead=0;
while(byteRead!=1) {
try {
byteRead=in1.read(cbuf, 0, 1);// Have problem on this line,here I am unable to read data bytes.
for(int i=0;i<byteRead;i++)
{
System.out.println(cbuf[i]);
}
if(byteRead==0)
{
System.out.println("Breaking.....");
return false;
}
}
catch(Exception e) {
System.out.println("Error Occured...."+e);
return false;
}
}
return true;
} catch(Exception e) {
System.out.println("System is not Connected..."+e);
return false;
}
almost tried every thing socket is not closed, read.available();,read.fully(); etc..unable to get the solution.I have written this function in the run method of TimerTask class.
any help will be greatly appreciated

the javadocs says BufferedReader#read(char[], int, int) Returns:
The number of characters read, or -1 if the end of the stream has been reached
since you do
byteRead=in1.read(cbuf, 0, 1);
in
while(byteRead!=1)
change it to
while(byteRead != -1)

byteRead=in1.read(cbuf, 0, 1);
This line only reads in one value and as you don't call it again before you enter the for loop, you should be getting 1 println of the value that was read in displayed in stdout.

read() blocks until at least one byte is available. Maybe you haven't sent it, or flushed it properly, or maybe you are creating multiple BufferedReaders on the same socket.
NB bytesRead can never be zero after a successful read(cbuf, 0, 1).

The read method of the underlying InputStream will block (i.e. hang/wait) if no data is available.
This method blocks until input data is available, end of file is detected, or an exception is thrown.
I strongly suspect this is the case.
You can check this by calling in1.ready() on the reader.
Flush the output buffer
out.flush();
after writing the bytes, or they may get buffered locally.

why client can not receive message from server (java)

I have just started learning java. I modified the client side code for a server/client communication program, by creating two threads for the client side, main thread for receiving user's input, and inputThread for receiving server's response. I am sure that server has sent the response to client, however, no response message is obtain at client.
Here is my code. Can anyone help me to figure it out? Thanks
package clientnio;
import java.net.*;
import java.nio.*;
import java.io.*;
import java.nio.channels.*;
import java.util.Scanner;
public class ClientNIO {
public static int bufferLen = 50;
public static SocketChannel client;
public static ByteBuffer writeBuffer;
public static ByteBuffer readBuffer;
public static void main(String[] args) {
writeBuffer = ByteBuffer.allocate(bufferLen);
readBuffer = ByteBuffer.allocate(bufferLen);
try {
SocketAddress address = new InetSocketAddress("localhost",5505);
System.out.println("Local address: "+ address);
client=SocketChannel.open(address);
client.configureBlocking(false);
//readBuffer.flip();
new inputThread(readBuffer);
/*
String a="asdasdasdasddffasfas";
writeBuffer.put(a.getBytes());
writeBuffer.clear();
int d=client.write(writeBuffer);
writeBuffer.flip();
*/
while (true) {
InputStream inStream = System.in;
Scanner scan = new Scanner(inStream);
if (scan.hasNext()==true) {
String inputLine = scan.nextLine();
writeBuffer.put(inputLine.getBytes());
//writeBuffer.clear();
System.out.println(writeBuffer.remaining());
client.write(writeBuffer);
System.out.println("Sending data: "+new String(writeBuffer.array()));
writeBuffer.flip();
Thread.sleep(300);
}
}
}
catch(Exception e) {
System.out.println(e);
}
}
}
class inputThread extends Thread {
private ByteBuffer readBuffer;
public inputThread(ByteBuffer readBuffer1) {
System.out.println("Receiving thread starts.");
this.readBuffer = readBuffer1;
start();
}
#Override
public void run() {
try {
while (true) {
readBuffer.flip();
int i=ClientNIO.client.read(readBuffer);
if(i>0) {
byte[] b=readBuffer.array();
System.out.println("Receiving data: "+new String(b));
//client.close();
//System.out.println("Connection closed.");
//break;
}
Thread.sleep(100);
}
}
catch (Exception e) {
System.out.println(e);
}
}
}

Disclaimer: I'm not an active user of Java. (I only used it in school.)
Advice: I think it will greatly simplify the debugging process if you use blocking mode, at least until your code example is working correctly. (Currently your code does not seem to benefit from the non-blocking mode.)
I have identified two issues, culminating into four possible lines of code that may require changing:
When a ByteBuffer allocates its backing array, it sets itself ready to write by setting position to zero and limit to the capacity of that array. Your two uses of ByteBuffer.flip() (in the writing loop and the reading loop respectively) seem to be contrary to the convention.
Calling the ByteBuffer.array() method always returns the whole backing array, thus it always has size bufferLen. Because of this, a String constructed from the full-size array may contain junk from a previous transmission.
Typically, the array needs to be trimmed to the transmission size, and the conversion between a String and a byte array must use the same encoding as the server.
My suggested changes for first issue: (Note: I don't know how to fix the array trimming and encoding issue.)
writeBuffer.put(inputLine.getBytes());
writeBuffer.flip(); // <--here
client.write(writeBuffer);
...
writeBuffer.clear(); // <-- should be clear() instead of flip()
Thread.sleep(300);
// readBuffer.flip(); // <-- remove this line
int i=ClientNIO.client.read(readBuffer);
if(i>0) {
readBuffer.flip(); // <-- move it here
byte[] b=readBuffer.array();
System.out.println("Receiving data: "+new String(b));
...
}
References
http://docs.oracle.com/javase/1.4.2/docs/api/java/nio/ByteBuffer.html
http://docs.oracle.com/javase/1.4.2/docs/api/java/nio/channels/SocketChannel.html
Socketchannel always null
http://www.exampledepot.com/egs/java.nio.charset/ConvertChar.html

Calling flip() on a buffer prior to reading it is wrong. Don't do that. You need to flip it prior to writing from it, or getting from it, and compact() afterwards.

RXTX serial connection - issue with blocking read()

I am trying to use the RXTX library for blocking serial communication on Windows (XP and 7). I have tested the connection with Hyperterminal in both ends, and it works flawlessly.
I set up the connection with the following code: (exception handling and defensive checks omitted for clarity)
private InputStream inStream;
private OutputStream outStream;
private BufferedReader inReader;
private PrintWriter outWriter;
private SerialPort serialPort;
private final String serialPortName;
public StreamComSerial(String serialPortName) {
this.serialPortName = serialPortName;
CommPortIdentifier portIdentifier;
portIdentifier = CommPortIdentifier.getPortIdentifier(serialPortName);
CommPort commPort = null;
commPort = portIdentifier.open(this.getClass().getName(),500);
serialPort = (SerialPort) commPort; serialPort.setSerialPortParams(4800,SerialPort.DATABITS_8,SerialPort.STOPBITS_1,SerialPort.PARITY_NONE);
inStream = serialPort.getInputStream();
outStream = serialPort.getOutputStream();
inReader = new BufferedReader(new InputStreamReader(inStream, Settings.getCharset()));
outWriter = new PrintWriter(new OutputStreamWriter(outStream, Settings.getCharset()));
When I use
outWriter.println("test message");
flush();
the message is recieved fine on the other end, but calling
inReader.readLine()
imidiately returns "java.io.IOException: Underlying input stream returned zero bytes".
I then decided to try and implement my own blocking read logic and wrote this:
public String readLine() throws IOException {
String line = new String();
byte[] nextByte = {-1};
while (true) {
nextByte[0] = (byte)inStream.read();
logger.debug("int read: " + nextByte[0]);
if (nextByte[0] == (byte)-1) {
try {
Thread.sleep(100);
} catch (InterruptedException e) {
e.printStackTrace();
}
continue;
}
logger.debug("byte read: " + nextByte[0]);
line = line + new String(nextByte);
if (nextByte[0] == (byte)13) { // 13 is carriage return in ASCII
return line;
}
}
}
But this code goes in an infinite loop and "nextByte[0] = (byte)inStream.read();" assigns -1 no matter what is sent over the serial connection. In addition, the other end stutters quite badly and only lets me send a character every 1-3 sec. and hangs for a long time if I try to send many characters in a short burst.
Any help very appreciated.
*edit - using inStream.read(nextByte) instead of "nextByte[0] = (byte)inStream.read();" does not write to the nextByte variable, no matter what I send to it through the serial connection.
*edit2 - as my code works flawlessly with the SUN javax.comm lib and a win32com.dll I got from a friend, I have ceased trying to make it work with RXTX. I am not interested in unblocking communication, which seems to be the only way other people can make RXTX work.

Use RXTX-2.2pre2, previous versions have had a bug which prevented blocking I/O from working correctly.
And do not forget to set port to blocking mode:
serialPort.disableReceiveTimeout();
serialPort.enableReceiveThreshold(1);

I think the code you wrote in your own readLine implementation is buggy. nextByte[0] is never restored to -1 after the first character is read.
You should try to use the value returned by inStream.read(nextByte) to state the number of bytes read from the stream instead of the value of your byte array.
Anyway I think you should go for an event based method of reading the inputs with a SerialPortEventListener:
serialPort.addEventListener(new SerialPortEventListener() {
public void serialEvent(SerialPortEvent evt) {
switch (evt.getEventType()) {
case SerialPortEvent.DATA_AVAILABLE:
dataReceived();
break;
default:
break;
}
}
});
serialPort.notifyOnDataAvailable(true);

it may not be blocking but when the stream is empty, just catch the IOE and keep reading from it. This is what I do with RXTX-2.1-7 and it works fine, I use it to read and write to an arduino:
public static class SerialReader implements Runnable {
InputStream in;
public SerialReader(InputStream in) {
this.in = in;
}
public void run() {
Boolean keepRunning = true;
BufferedReader br = new BufferedReader(new InputStreamReader(in));
String line;
while (keepRunning) {
try {
while ((line = br.readLine()) != null) {
//DO YOUR STUFF HERE
}
} catch (IOException e) {
try {
//ignore it, the stream is temporarily empty,RXTX's just whining
Thread.sleep(200);
} catch (InterruptedException ex) {
// something interrupted our sleep, exit ...
keepRunning = false;
}
}
}
}
}

I have solved this way
try
{
if(input.ready()==true)
{
String inputLine=input.readLine();
System.out.println(inputLine);
}
} catch (Exception e)