Develop Reference

Socket intermittently reads only 1448/2896 bytes

I am using Commons-IO to read and write from Socket. Things all works till payload size is either 1448/2896 max.
Below is the code snippet. Really unsure how to handle it.
Checked system buffer size too
$ cat /proc/sys/net/ipv4/tcp_wmem
4096 16384 4194304
public static void usingCommonsIO(){
Socket socket = null;
try {
socket = new Socket(serverIP, 55000);
IOUtils.write(request.getBytes(), socket.getOutputStream());
System.out.println("Message Sent....");
StringBuilder response = new StringBuilder();
String resp =IOUtils.toString(socket.getInputStream(), "UTF-8");
System.out.println(resp);
} catch (IOException e) {
e.printStackTrace();
}
}
Alternatively tried using DataInputStream but no luck. Code snipped is below.
public static void usingDataIOStream(String requestStr){
Socket socket = null;
try {
socket = new Socket("192.168.1.6", 55000);
System.out.println("Request Length -:" + request.length());
DataOutputStream out = new DataOutputStream(socket.getOutputStream());
out.write(requestStr.getBytes("UTF-8"), 0, requestStr.length());
out.flush();
System.out.println("Message Sent....");
DataInputStream din = new DataInputStream(socket.getInputStream());
byte[] response = new byte[16*1024];
int responseLength = din.read(response);
System.out.println("Response -:" + new java.lang.String(response, 0, responseLength));
} catch (IOException e) {
e.printStackTrace();
}
}
Confusing part is that the same code works with only 1448 bytes sometimes and max of 2896 bytes sometimes. There are no specific patterns.
Update 1
To simulate it, tried writing Server socket on my own and code is as below. Strange thing noticed with this is, on first request payload of size 6500 was read and received properly. Connection Reset from second request onwards. Am I missing something here?
public static void usingBAOS() throws IOException {
server = new ServerSocket(port);
Socket socket = null;
DataInputStream din = null;
DataOutputStream dos = null;
while (true) {
System.out.println("Waiting for Client...");
try {
// Accepting Client's connection
socket = server.accept();
System.out.println("Connnected to client " + socket.getInetAddress());
din = new DataInputStream(socket.getInputStream());
// Read request payload from Socket
String requestString = readRequest(din);
System.out.println("Request Read.....");
System.out.println("Writing Response.....");
// Writing response to socket
dos = writeResponse(socket, requestString);
} catch (IOException e) {
e.printStackTrace();
}finally {
//close resources
din.close();
System.out.println("InputStream is closed......");
dos.close();
System.out.println("OutputStream is closed......");
socket.close();
System.out.println("Socket is closed......");
}
}
}
private static DataOutputStream writeResponse(Socket socket, String requestString) throws IOException {
String responseString = "Hi Client" + requestString;
DataOutputStream dos = new DataOutputStream(socket.getOutputStream());
//write object to Socket
dos.write(responseString.getBytes(),0, responseString.getBytes().length);
dos.flush();
return dos;
}
private static String readRequest(DataInputStream din) throws IOException {
byte[] response = new byte[16*1024];
ByteArrayOutputStream baos = new ByteArrayOutputStream();
int n = 0;
boolean read = true;
while(read){
n = din.read(response);
baos.write(response, 0, n);
if(baos.toString().length() == n){
read = false;
}
}
baos.flush();
String requestString = baos.toString();
return requestString;
}

Although this question is old at the time of writing this answer I'm putting this here for others in case it solves their problem. I encountered the same issue when using buffered data input and output streams on Android 8.0 devices where I had naively assumed that doing this:
int len = 2304;
byte[] data = new byte[len];
inputStream.read(data, 0, len);
would read all the data I sent down the socket. But as suggested by #Kayaman in the comments, this does not guarantee that len bytes of data are actually read from the buffer even if there are bytes available. In fact, this is in the documentation:
public final int read(byte[] b, int off, int len) throws IOException
Reads up to len bytes of data from the contained input stream into an array of bytes. An attempt is made to read as many as len bytes, but a smaller number may be read, possibly zero. The number of bytes actually read is returned as an integer.
In fact, if it doesn't read all the data, the only way to tell is to capture the returned value. My solution was then to monitor the amount of bytes actually read from the stream and just call read() in a loop as:
int i = 0;
len = 2304;
byte[] data = new byte[len];
while (i < len)
{
i += socket.inputStream.read(data, i, len - i);
}
Hope this helps someone.

Server: Socket hangs within unpredictable period time at read stream function

I wrote a Java socket server which will keep connection alive until client disconnected. And my client code will keep pushing message to this server app.
But when I run those programs a while, I also seem an unusual condition that Server will hangs while reading input stream from client within unpredictable period. It always hang at inData.read(b) because I see it printed "receiving..." on log when this problem occurred"; even I killed my client, server app still hangs right there.
But when I press Ctrl+C at the console which runs server app after this problem occurred, it will continue to work. This is really annoying.
Is there anyway to solve this Unusual problem nicely?
Server Code:
static ServerSocket server;
try {
server = new ServerSocket("1234");
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
Socket socket = null;
String inIp = null;
BufferedInputStream inData;
BufferedOutputStream outData;
while (true) {
try {
synchronized (server) {
socket = server.accept();
}
inIp = String.valueOf(socket.getInetAddress());
if (Log4j.log.isEnabledFor(Level.INFO)) {
Log4j.log.info("Incoming connection " + inIp);
}
while (true) {
inData = new BufferedInputStream(socket.getInputStream());
outData = new BufferedOutputStream(socket.getOutputStream());
String reply = "Hey";
byte[] b = new byte[10240];
String data = "";
int length;
if (Log4j.log.isEnabledFor(Level.INFO)) {
Log4j.log.info("InetAddr = " + inIp + ", receiving...");
}
// read input stream
length = inData.read(b);
data += new String(b, 0, length);
if (Log4j.log.isEnabledFor(Level.INFO)) {
Log4j.log.info("Data Length: " + length + ", Received data: " + data);
}
// output result
outData.write(reply.getBytes());
outData.flush();
}
} catch (Exception e) {
String tempStr = e.toString();
Log4j.log.error("Service error during executing: " + tempStr);
}
}
Client Code:
Socket client = new Socket();
InetSocketAddress isa = new InetSocketAddress("127.0.0.1", "1234");
String data = "Hi";
while(true) {
try {
if(!client.isConnected())
client.connect(isa, 30000);
BufferedOutputStream out = new BufferedOutputStream(client.getOutputStream());
BufferedInputStream in = new BufferedInputStream(client.getInputStream());
// send msg
out.write(data.getBytes());
out.flush();
System.out.println("Message sent, receiving return message...");
// get return msg
int length;
byte[] b = new byte[10240];
// read input stream
length = in.read(b);
retMsg = new String(b, 0, length);
System.out.println("Return Msg: " + retMsg);
Thread.sleep(60000);
} catch (java.io.IOException | InterruptedException e) {
System.out.println("Socket Error!");
System.out.println("IOException :" + e.toString());
}
}

try {
server = new ServerSocket("1234");
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
Don't write code like this. The catch block should be at the end, and all the code that depends on the success of new ServerSocket should be inside the try block.
synchronized (server) {
socket = server.accept();
}
Synchronization is not necessary here.
while (true) {
inData = new BufferedInputStream(socket.getInputStream());
outData = new BufferedOutputStream(socket.getOutputStream());
A large part of the problem, if not all of it, is here. You keep creating new buffered streams, every time around this loop, which means that anything the previous streams have buffered is thrown away. So you are losing input. You should create both these streams before the loop.
while(true) {
try {
if(!client.isConnected())
client.connect(isa, 30000);
This is pointless. Remove. You haven't shown how the client socket was created, but if you created it unconnected you should have connected it before entering this loop.
BufferedOutputStream out = new BufferedOutputStream(client.getOutputStream());
BufferedInputStream in = new BufferedInputStream(client.getInputStream());
Here again you must create these streams ahead of the loop.

datainputstream missing bytes in.read(b)

I'm trying to write a program which acts as a server that will read bytes from a client that is written in PHP - sends request via socket (which i cannot recode due to policy) Here is the server code:
The server runs in: Red Hat Enterprise Linux Server release 6.2 (Santiago)
public void run() {
try {
serverSocket = new ServerSocket(port);
serverSocket.setSoTimeout(0);
while(!isInterrupted) {
try {
Socket server = serverSocket.accept();
LOG.info("Request received from : " + server.getRemoteSocketAddress());
DataInputStream in = new DataInputStream(server.getInputStream());
// DataInputStream in = new DataInputStream(
// new BufferedInputStream(server.getInputStream(), 10000));
byte[] bytes = new byte[10000];
int byteDupLength = in.read(t_bytes);
// in.readFully(bytes); // I tried this but to no avail
// int byteDupLength = bytes.length;
LOG.info(byteDupLength);
byte[] byteDup = new byte[byteDupLength];
System.arraycopy(bytes, 4, byteDup, 0, byteDupLength);
// FOR INFORMATION ONLY
/*for (byte b : byteDup){
LOG.info(b);
}*/
ByteBuffer buffer = ByteBuffer.wrap(byteDup);
LOG.info(buffer);
forwardRequest(byteDup);
server.close();
}
catch(SocketTimeoutException s) {
LOG.error("Socket timed out!", s);
break;
}
catch(IOException e)
{
LOG.error("IOException:", e);
break;
}
}
}
catch (IOException ex) {
LOG.error("Server socket is null", ex);
}
LOG.fatal("ReceiverEngine interrupted!");
}
I encountered a problem when the client sends request consisting of 4948 bytes. The only bytes the server can read is 2090.
Another thing that seems a mystery to me is that, when I run the server via Netbeans in my local (which is a Windows 7 Pro), it works as expected. I dont know what is wrong. Please help.. :)
Thanks!

TCP is a byte stream protocol.
The read() method isn't guaranteed to fill the buffer.
Therefore if you don't receive the expected number of bytes in a single read, you have to loop until you do receive them.
readFully() would have worked if the buffer size agreed with the size of what was sent. In your case you specified a buffer of 10,000 bytes, which weren't sent, so it would have blocked waiting for the other 10000-4948 bytes.

Java socket timing out: Broken pipe

I'm writing a simple server in Java, and I'm able to retrieve incoming data from the client on the server side, but not on the client side due to a 2000ms timeout. Anyone know why this times out?
This is the server's code:
private static void listen() throws IOException {
while(true) {
Socket clientSocket = serverSocket.accept();
StringBuilder bufferedStringInput = new StringBuilder();
CharBuffer cbuf = CharBuffer.allocate(4096);
try {
InputStream is = clientSocket.getInputStream();
BufferedReader br = new BufferedReader(new InputStreamReader(is, "UTF8"));
int noCharsLeft = 0;
while ((noCharsLeft = br.read(cbuf)) != -1) {
char[] arr = new char[noCharsLeft];
cbuf.rewind();
cbuf.get(arr);
bufferedStringInput.append(arr);
cbuf.clear();
}
System.out.println(bufferedStringInput.toString());
} catch (IOException e) {
System.out.println("Error received client data: " + e.getMessage());
}
String message = "Hello client";
try {
PrintWriter out = new PrintWriter(clientSocket.getOutputStream());
out.print(message);
} catch (IOException e) {
System.out.println("Error getting output stream from client: " + e.getMessage());
}
clientSocket.close();
}
}

You're reading the input until end of stream, which only happens when the peer closes the connection, then you're trying to write to it, so of course you get a broken pipe. Doesn't make sense. You should just read the input until you have one entire request, whatever that means in your protocol.
There are other problems lurking here:
If the client code uses readLine(), you're not sending a line terminator: use println(), not print(), and close the PrintWriter, not just the client socket.
cbuf.rewind()/get()/clear() should be cbuf.flip()/get()/compact().
But it would make more sense to read directly into a char[] cbuf = new char[8192]; array, then bufferedStringInput.append(cbuf, 0, noCharsLeft), and forget about the CharBuffer altogether. Too much data copying at present.
noCharsLeft is a poor name for that variable. It is a read count.

Socket connectivity loss when sending live streaming data

I'm struggling here...
I'm trying to determine if data was successfully sent to the server through a TCP socket using the OutputStream object. For testing on emulator socket communications is loss after 30 sec. For write data OutputStream.write(); its doesn't throw an exception , and local server continuously running its not crashing, only tcp socket connection is loss after some time. All the methods in the socket class return as though the socket is active and working. Is there anything I'm doing wrong here? Is there any socket implementation or stream implementation I can use to get an exception or error when the stream/ socket doesn't actually send the data in the buffer? Also setting setSoTimeout() on the socket doesn't seem to do anything.
Please guide me...
Here is my code:
private void sendRec() {
int lstream;
int port = 1012;
byte[] byterecv = new byte[1040];
while (true) {
System.out.println("POOL-2");
synchronized (recSendThread) {
try {
recSendThread.wait(20);
} catch (InterruptedException e) {
}
}
if (stopcall == true) {
// break;
}
try {
// Provides a client-side TCP socket
Socket clientRec = new Socket();
// serverSocket = new ServerSocket(port);
// serverSocket.setSoTimeout(5000);
// Connects this socket to the given remote host address and
// port
clientRec.connect(new InetSocketAddress("192.168.1.36", port));
System.out.println("Just connected to "
+ clientRec.getRemoteSocketAddress());
System.out.println("SENTS Rec BEFORE");
// output streams that write data to the network
OutputStream outToServerRec = clientRec.getOutputStream();
DataOutputStream outStreamRec = new DataOutputStream(
outToServerRec);
outStreamRec.write(bData);
System.out.println("SENTS Rec AFTER");
// input streams that read data from network
InputStream inFromServerRec = clientRec.getInputStream();
// clientRec.setSoTimeout(5000);
DataInputStream inStreamRec = new DataInputStream(
inFromServerRec);
while ((lstream = inStreamRec.read(byterecv)) != -1) {
System.out.println("startrec bytearray -- "
+ byterecv.length);
bos1.write(byterecv, 0, lstream);
}
inStreamRec.close();// for closing dataouputstream
clientRec.close();// for closing socket connection
} catch (IOException e) {
e.printStackTrace();
}
}
}
Here is my receiver and player code..
/**
* start receiving the voice data from server
* */
protected void startplay() {
System.arraycopy(frndid, 0, playByteData, 0, 4);
System.arraycopy(userid, 0, playByteData, 4, 4);
ByteBuffer.wrap(sessionid).order(ByteOrder.LITTLE_ENDIAN).asIntBuffer().
put(call_sessionid);
System.arraycopy(sessionid, 0, playByteData, 8, 4);
int lstream;
int port = 1014;
while (true) {
System.out.println("POOL-3");
try {
if (stopcall == true) {
System.out.println("BREAKEDDDD1111");
//break;
}
// Host name
// port++;
InetAddress addressPlay = InetAddress.getByName("192.168.1.36");
// Creates a new streaming socket connected to the target host
Socket clientPlay = new Socket(addressPlay, port);
System.out.println("Just connected to play : " +
clientPlay.getRemoteSocketAddress());
System.out.println("SENTS Play BEFORE");
// output streams that write data
OutputStream outToServer = clientPlay.getOutputStream();
DataOutputStream outStreamPlay = new DataOutputStream(outToServer);
outStreamPlay.write(playByteData);
System.out.println("SENTS Play after");
// input streams that read data
InputStream inFromServerPlay = clientPlay.getInputStream();
DataInputStream inStreamPlay = new DataInputStream(inFromServerPlay);
//clientPlay.setSoTimeout(5000);
while ((lstream = inStreamPlay.read(byteArray)) != -1) {
System.out.println("startplay() bytearray -- " +
byteArray.length);
bos.write(byteArray, 0, lstream);
}
inStreamPlay.close();
clientPlay.close();// for closing play socket connection
responseBuffer = bos.toByteArray();
System.out.println("BAOSSIZE " + bos.size());
bos.reset();
bos.flush();
bos.close();
playing = true;
System.out.println("res length -- " + responseBuffer.length);
rcvbb=ByteBuffer.wrap(responseBuffer).order(ByteOrder.LITTLE_ENDIAN).
asShortBuffer().get(playShortData);
playVoiceReceived();// plays received data
} catch (ParseException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
}
/**
* start playing received the voice data from server
* */
public void playVoiceReceived() {
System.out.println("POOL-4");
try {
if (at != null) {
if (at.getPlayState() != AudioTrack.PLAYSTATE_PLAYING) {
at.play();// starts playing
} else {
System.out.println("Play BEFORE WRITE");
// Writes the audio data to the audio hardware for playback.
at.write(playShortData, 0, BufferElements2Play);
System.out.println("Play AFTER WRITE");
at.flush();
}
}
} catch (Exception e) {
e.printStackTrace();
}
}

The socket has sent the data ... to the local socket send buffer. What happens after that is up to the local TCP stack, the network, the remote TCP stack, and the remote application. If you want to know whether the remote application got the data, it will have to send you a reply.

Operator write does not check whether data was delivered because otherwise it would have to wait for too long time. If network connection is actually down, TCP layer of operating system, will try to send data anyway, but will detect problems somewhat later, i.e. 1 minute later, because it will not receive acknowledgement messages from the opposite side. It will then try to resend data several times, see that problem persists and only then will report exception condition on the socket. To know that socket is in exception condition, you need to perform some operator on socket, i.e. another write attempt. Try doing write in a loop like this:
while (true)
{
outStreamRec.write (data);
Thread.sleep (1000L);
}
It should throw an error about 2 minutes after network will be down.
Note, that in opposite to write operation, operation connect is synchronous, so it actually waits for response from the opposite side, and if there is not respose, it will throw an exception.