Develop Reference

Socket intermittently reads only 1448/2896 bytes

I am using Commons-IO to read and write from Socket. Things all works till payload size is either 1448/2896 max.
Below is the code snippet. Really unsure how to handle it.
Checked system buffer size too
$ cat /proc/sys/net/ipv4/tcp_wmem
4096 16384 4194304
public static void usingCommonsIO(){
Socket socket = null;
try {
socket = new Socket(serverIP, 55000);
IOUtils.write(request.getBytes(), socket.getOutputStream());
System.out.println("Message Sent....");
StringBuilder response = new StringBuilder();
String resp =IOUtils.toString(socket.getInputStream(), "UTF-8");
System.out.println(resp);
} catch (IOException e) {
e.printStackTrace();
}
}
Alternatively tried using DataInputStream but no luck. Code snipped is below.
public static void usingDataIOStream(String requestStr){
Socket socket = null;
try {
socket = new Socket("192.168.1.6", 55000);
System.out.println("Request Length -:" + request.length());
DataOutputStream out = new DataOutputStream(socket.getOutputStream());
out.write(requestStr.getBytes("UTF-8"), 0, requestStr.length());
out.flush();
System.out.println("Message Sent....");
DataInputStream din = new DataInputStream(socket.getInputStream());
byte[] response = new byte[16*1024];
int responseLength = din.read(response);
System.out.println("Response -:" + new java.lang.String(response, 0, responseLength));
} catch (IOException e) {
e.printStackTrace();
}
}
Confusing part is that the same code works with only 1448 bytes sometimes and max of 2896 bytes sometimes. There are no specific patterns.
Update 1
To simulate it, tried writing Server socket on my own and code is as below. Strange thing noticed with this is, on first request payload of size 6500 was read and received properly. Connection Reset from second request onwards. Am I missing something here?
public static void usingBAOS() throws IOException {
server = new ServerSocket(port);
Socket socket = null;
DataInputStream din = null;
DataOutputStream dos = null;
while (true) {
System.out.println("Waiting for Client...");
try {
// Accepting Client's connection
socket = server.accept();
System.out.println("Connnected to client " + socket.getInetAddress());
din = new DataInputStream(socket.getInputStream());
// Read request payload from Socket
String requestString = readRequest(din);
System.out.println("Request Read.....");
System.out.println("Writing Response.....");
// Writing response to socket
dos = writeResponse(socket, requestString);
} catch (IOException e) {
e.printStackTrace();
}finally {
//close resources
din.close();
System.out.println("InputStream is closed......");
dos.close();
System.out.println("OutputStream is closed......");
socket.close();
System.out.println("Socket is closed......");
}
}
}
private static DataOutputStream writeResponse(Socket socket, String requestString) throws IOException {
String responseString = "Hi Client" + requestString;
DataOutputStream dos = new DataOutputStream(socket.getOutputStream());
//write object to Socket
dos.write(responseString.getBytes(),0, responseString.getBytes().length);
dos.flush();
return dos;
}
private static String readRequest(DataInputStream din) throws IOException {
byte[] response = new byte[16*1024];
ByteArrayOutputStream baos = new ByteArrayOutputStream();
int n = 0;
boolean read = true;
while(read){
n = din.read(response);
baos.write(response, 0, n);
if(baos.toString().length() == n){
read = false;
}
}
baos.flush();
String requestString = baos.toString();
return requestString;
}

Although this question is old at the time of writing this answer I'm putting this here for others in case it solves their problem. I encountered the same issue when using buffered data input and output streams on Android 8.0 devices where I had naively assumed that doing this:
int len = 2304;
byte[] data = new byte[len];
inputStream.read(data, 0, len);
would read all the data I sent down the socket. But as suggested by #Kayaman in the comments, this does not guarantee that len bytes of data are actually read from the buffer even if there are bytes available. In fact, this is in the documentation:
public final int read(byte[] b, int off, int len) throws IOException
Reads up to len bytes of data from the contained input stream into an array of bytes. An attempt is made to read as many as len bytes, but a smaller number may be read, possibly zero. The number of bytes actually read is returned as an integer.
In fact, if it doesn't read all the data, the only way to tell is to capture the returned value. My solution was then to monitor the amount of bytes actually read from the stream and just call read() in a loop as:
int i = 0;
len = 2304;
byte[] data = new byte[len];
while (i < len)
{
i += socket.inputStream.read(data, i, len - i);
}
Hope this helps someone.

The server is sending data through the socket, but the client is not receving it (Java)

I am writing a file storage and transfer system using Java. Here's the code on the client side to receive a file:
public static void receiveFile(Socket socket) throws IOException{
String fileLocation="/home/limafoxtrottango/Downloads/receivedFile";
int bytesRead=0;
int current = 0;
FileOutputStream fileOutputStream = null;
BufferedOutputStream bufferedOutputStream = null;
try {
// receive file
byte [] byteArray = new byte [60022386];
System.out.println("Waiting to receive a file...");
//reading file from socket
InputStream inputStream = socket.getInputStream();
fileOutputStream = new FileOutputStream(fileLocation);
bufferedOutputStream = new BufferedOutputStream(fileOutputStream);
bytesRead = inputStream.read(byteArray,0,byteArray.length); //copying file from socket to byteArray
current = bytesRead;
do {
bytesRead =inputStream.read(byteArray, current, (byteArray.length-current));
if(bytesRead >= 0) current += bytesRead;
} while(bytesRead > -1);
bufferedOutputStream.write(byteArray, 0 , current); //writing byteArray to file
bufferedOutputStream.flush(); //flushing buffers
System.out.println("File " + fileLocation + " downloaded ( size: " + current + " bytes read)");
} catch(SocketException e){
System.out.println("Some error occured");
}
catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
finally {
if (fileOutputStream != null) fileOutputStream.close();
if (bufferedOutputStream != null) bufferedOutputStream.close();
if (socket != null) socket.close();
}
}
While receiving a file, I get the following error:
Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException
at java.lang.System.arraycopy(Native Method)
at java.io.BufferedOutputStream.write(BufferedOutputStream.java:128)
at Test.receiveFile(Test.java:211)
at Test.main(Test.java:70)
Note: The error is in the following line of the code:
bufferedOutputStream.write(byteArray, 0 , current);
After debugging, I found-out that the client does not have any data in it's input stream, and hence, the read() method always returns -1 (eof). But the server is sending the file successfully.
Here is the code for the server:
public static void sendFile(Socket socket, String fileLocation)
{
FileInputStream fileInputStream = null;
BufferedInputStream bufferedInputStream = null;
OutputStream outputStream = null;
File file = new File (fileLocation);
byte [] byteArray = new byte [(int)file.length()];
try {
socket=new Socket(socket.getInetAddress(),port_no);
fileInputStream = new FileInputStream(file);
bufferedInputStream = new BufferedInputStream(fileInputStream);
bufferedInputStream.read(byteArray,0,byteArray.length); // copied file into byteArray
//sending file through socket
outputStream = socket.getOutputStream();
System.out.println("Sending " + fileLocation + "( size: " + byteArray.length + " bytes)");
outputStream.write(byteArray,0,byteArray.length); //copying byteArray to socket
outputStream.flush(); //flushing socket
System.out.println("Done sending!");
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
And here is my call to the above method:
sendFile(clientSocket, "/home/limafoxtrottango/Downloads/serverDownloads/"+sender);
The thing is that the server is successfully writing the byte into the stream, but the client doesn't seem to have any data in it's input stream.

https://docs.oracle.com/javase/7/docs/api/java/io/InputStream.html#read(byte[],%20int,%20int)
inputStream.read(byteArray,0,byteArray.length); may return -1 in some cases as given in documentation above. Please cater for such situations.
In addition, I would suggest to use solution given here for both client and server: Efficient way to write InputStream to a File in Java (Version 6)
Client code:
final Path destination = Paths.get(fileLocation);
try (
final InputStream in = socket.getInputStream();
) {
Files.copy(in, destination);
}
Server code:
try (
final InputStream in = new FileInputStream(fileLocation);
) {
Files.copy(in, socket.getOutputStream());
}
Kind regards,
Bala

The server isn't sending anything, contrary to your title. It is closing the connection immediately, so bytesRead is initially -1 and never changes, and you aren't defending against that, so you get the ArrayIndexOutOfBoundsException.
However in the code you posted, the server is sending something but never closing the socket, which is another bug you need to fix. It is also ignoring the count returned by FileInputStream.read() and assuming it filled the buffer, which isn't part of the specification.
So either this is not the real server code or you are connecting to something else, or the server got an IOException that you haven't mentioned.
It's curious that you use two different pieces of code for copying. The standard way to copy a stream in Java is this:
char buffer = new char[8192]; // or whatever size you prefer > 0
int count;
while ((count = in.read(buffer)) > 0)
{
out.write(buffer, 0, count);
}
Use this at both ends. There is no need for heroically sized buffers, or buffers the size of the file, or assuming that the file size fits into an int.

Java socket timing out: Broken pipe

I'm writing a simple server in Java, and I'm able to retrieve incoming data from the client on the server side, but not on the client side due to a 2000ms timeout. Anyone know why this times out?
This is the server's code:
private static void listen() throws IOException {
while(true) {
Socket clientSocket = serverSocket.accept();
StringBuilder bufferedStringInput = new StringBuilder();
CharBuffer cbuf = CharBuffer.allocate(4096);
try {
InputStream is = clientSocket.getInputStream();
BufferedReader br = new BufferedReader(new InputStreamReader(is, "UTF8"));
int noCharsLeft = 0;
while ((noCharsLeft = br.read(cbuf)) != -1) {
char[] arr = new char[noCharsLeft];
cbuf.rewind();
cbuf.get(arr);
bufferedStringInput.append(arr);
cbuf.clear();
}
System.out.println(bufferedStringInput.toString());
} catch (IOException e) {
System.out.println("Error received client data: " + e.getMessage());
}
String message = "Hello client";
try {
PrintWriter out = new PrintWriter(clientSocket.getOutputStream());
out.print(message);
} catch (IOException e) {
System.out.println("Error getting output stream from client: " + e.getMessage());
}
clientSocket.close();
}
}

You're reading the input until end of stream, which only happens when the peer closes the connection, then you're trying to write to it, so of course you get a broken pipe. Doesn't make sense. You should just read the input until you have one entire request, whatever that means in your protocol.
There are other problems lurking here:
If the client code uses readLine(), you're not sending a line terminator: use println(), not print(), and close the PrintWriter, not just the client socket.
cbuf.rewind()/get()/clear() should be cbuf.flip()/get()/compact().
But it would make more sense to read directly into a char[] cbuf = new char[8192]; array, then bufferedStringInput.append(cbuf, 0, noCharsLeft), and forget about the CharBuffer altogether. Too much data copying at present.
noCharsLeft is a poor name for that variable. It is a read count.

Sending picture over a socket (Java PC - Android)

I'm new to Java so I need help please. I'm writing an application that will onClick send a String to Server and Server needs to return an image using socket. So my client side is Android and server side is PC - java.
I think that my server side is ok (because he prints out all the system.out.print commands) but my client side is not good. Please tell my where is my mistake? Thanks!
Here is code of my Server (PC) side (socket is delivered thru function parameter):
try {
dataInputStream = new DataInputStream(socket.getInputStream());
poruka = "" + dataInputStream.readUTF();
System.out.print(poruka);
int bytecount = 2048;
byte[] buf = new byte[bytecount];
OutputStream OUT = socket.getOutputStream();
BufferedOutputStream BuffOUT = new BufferedOutputStream(OUT, bytecount);
FileInputStream in = new FileInputStream("screenShot.jpg");
int i = 0;
while ((i = in.read(buf, 0, bytecount)) != -1) {
BuffOUT.write(buf, 0, i);
System.out.print("check" + buf[0]);
BuffOUT.flush();
}
in.close();
BuffOUT.close();
System.out.print("over");
} catch (IOException e) {
e.printStackTrace();
}finally{
try {
socket.close();
dataInputStream.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
and here is my Client (Android) side:
Socket socket = null;
DataOutputStream dataOutputStream = null;
try {
socket = new Socket(IPadresa, 8888);
dataOutputStream = new DataOutputStream(socket.getOutputStream());
slanje = "hy string";
dataOutputStream.writeUTF(slanje);
FileOutputStream outToFile = new FileOutputStream("slika.jpg");
int bytecount = 2048;
byte[] buf = new byte[bytecount];
InputStream IN = socket.getInputStream();
BufferedInputStream BuffIN = new BufferedInputStream(IN, bytecount)
int i = 0;
int filelength = 0;
while((i = BuffIN.read(buf, 0, bytecount)) != -1) {
filelength += i;
outToFile.write(buf, 0, i);
outToFile.flush();
}
IN.close();
BuffIN.close();
dataOutputStream.close();
socket.close();
} catch (UnknownHostException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
MORE INFORMATIONS:
In server side I can see String that is send from Client. And I have that System.out.print("over"); command printed every time I send String to Server. Also I have System.out.print("check" + buf[0]); printed out from Server many times. So that is why I think that there is something wrong with Client side.
And my Client side doesn't throw any Exceptions... but I noticed that Client side never passed the while loop. It get stuck there.

I don't know what you mean by "my client side is not good", and I can't see any obvious errors that would stop it working entirely. If you could tell us what happens, that would help.
Meanwhile there are a couple of things wrong with the code you have presented:
You are violating industry accepted coding standards with names such as "BuffIn", "IN", "IPaddresa" and so on. All variable names in Java must start with a lowercase letter.
If you do this in private code that is your business. But if you are going to show your Java code to other people, you should conform to the standards. (And posting your code on SO is showing it to other people ...)
Since you are always trying to read a whole buffer's worth of data, replace in.read(buf, 0, bytecount) with in.read(buf).
There is no value in using a BufferedInputStream or BufferedOutputStream if you are only going to do large read or write calls on it. Even more so if you tell the stream to use the same size buffer as your the byte[] you are reading / writing.
Both your client and server side code could leak file descriptors. On the server-side it could leak in. On the client side, any or all of the streams' file descriptors could leak.

Java Android Sockets

I'm currently working on an Android app which sends an string and a file to a java server app running on remote computer. This java server app should find the index on the file and send back the value of this index (The file structure is: index value. Example: 1 blue) The file is properly sent and received on the remote machine and I have a method which finds the value of the received index on the file. But when I'm trying to send the found value back to the phone I get an exception (closed socket), but I'm not closing the socket or any buffer. I'm not sure if the socket which is closed is the mobile app socket or the java server app socket. I'm using the same socket I use to send to receive (which is the way to work on Android). Sending the answer back to the phone is what my project is missing and is what I need help in. Here is my code:
Client app (Android app):
private class HeavyRemProcessing extends AsyncTask<String, Void, String>
{
protected String doInBackground(String... urls)
{
begins = System.currentTimeMillis();
remoteExecution();
ends= System.currentTimeMillis();
procTime=ends-begins;
aux= Long.toString(procTime);
return aux;
} //doInBackground() ends
protected void onPostExecute(String time)
{
textView1.setText("Result: "+result+". Processing Time: "+time+" milisecs");
}// onPostExecute ends
} //HeavyRemProcessing ends
public void executor(View view)
{
key="74FWEJ48DX4ZX8LQ";
HeavyRemProcessing task = new HeavyRemProcessing();
task.execute(new String[] { "????" });
} //executor() ends
public void remoteExecution()
{
// I have fixed IP and port I just deleted
String ip; //SERVER IP
int port; // SERVER PORT
try
{
cliSock = new Socket(ip, port);
file= new File("/mnt/sdcard/download/Test.txt");
long length = file.length();
byte[] bytes = new byte[(int) length];
FileInputStream fis = new FileInputStream(file);
BufferedInputStream bis = new BufferedInputStream(fis);
BufferedOutputStream out = new BufferedOutputStream(cliSock.getOutputStream());
BufferedReader in = new BufferedReader(new InputStreamReader(cliSock.getInputStream()));
int count;
key=key+"\r\n";
out.write(key.getBytes());
while ((count = bis.read(bytes)) > 0)
{
out.write(bytes, 0, count);
} //It works perfectly until here
//// PROBABLY HERE IS THE PROBLEM:
out.flush();
out.close();
fis.close();
bis.close();
result= in.readLine(); //RECEIVE A STRING FROM THE REMOTE PC
}catch(IOException ioe)
{
// Toast.makeText(getApplicationContext(),ioe.toString() + ioe.getMessage(),Toast.LENGTH_SHORT).show();
}
}catch(Exception exp)
{
//Toast.makeText(getApplicationContext(),exp.toString() + exp.getMessage(),Toast.LENGTH_SHORT).show();
}
} //remoteExecution ends
Java Server App (Remote PC)
public void receivingFile()
{
System.out.println("Executing Heavy Processing Thread (Port 8888).");
try
{
serverSocket = new ServerSocket(8888);
InputStream is = null;
OutputStream os= null;
FileOutputStream fos = null;
BufferedOutputStream bos = null;
BufferedOutputStream boSock =null;
DataOutputStream dataOutputStream=null;
int bufferSize = 0;
try
{
socket = serverSocket.accept();
System.out.println("Heavy Processing Task Connection from ip: " + socket.getInetAddress());
} catch (Exception ex)
{
System.out.println("Can't accept client connection: "+ex);
}
try
{
is = socket.getInputStream();
dataOutputStream = new DataOutputStream(socket.getOutputStream());
bufferSize = socket.getReceiveBufferSize();
}
catch (IOException ex)
{
System.out.println("Can't get socket input stream. ");
}
try
{
fos = new FileOutputStream(path);
bos = new BufferedOutputStream(fos);
}
catch (FileNotFoundException ex)
{
System.out.println("File not found. ");
}
byte[] bytes = new byte[bufferSize];
int count;
System.out.println("Receiving Transfer File!.");
while ((count = is.read(bytes)) > 0)
{
bos.write(bytes, 0, count);
}
System.out.println("File Successfully Received!.");
fos.close();
bos.flush();
bos.close();
is.close();
result= obj.searchIndex();
System.out.println("Found: "+result); //This correctly print the found value
dataOutputStream.writeUTF(result);
dataOutputStream.flush();
dataOutputStream.close();
System.out.println("Data sent back to the Android Client. ");
} catch (IOException e)
{
// TODO Auto-generated catch block
e.printStackTrace();
}
} // receivingFile() ends
Please if someone can help me I will really appreciate it. I'm thinking is something probably related with the buffers and the socket. My java server app throws an exception: "Closed Socket"... Thanks for your time,
Alberto.

I think your problem is that you closing the outputstream before closing the inputstream. This is a bug in android. Normally in java closing outputstream only flushes the data and closing inputstream causes the connection to be closed. But in android closing the outputstream closes the connection. That is why you are getting closed socket exception,
Put the statements
out.flush();
out.close();
after
result=in.readLine();
or just avoid those statements(out.flush and out.close). I had also faced a similar problem. See my question