Develop Reference

Stream isn't closing after sending file

Excuse me my understanding of streams is slowly building.
I initally tried to stream a music file over to my client but that didn't work out so I moved transferring the entire file via bytes then saving it at the client side. The problem is the input stream is still receiving bytes and so won't break out of the while loop (Where I want to play the song)
Below is part of my client code:
//This is part of a task that carries this out (Part of a UI application)
#Override
protected Void call()
{
try(Socket socket = new Socket(host,portNumber);
ObjectOutputStream toServer = new ObjectOutputStream(socket.getOutputStream());
ObjectInputStream fromServer = new ObjectInputStream(socket.getInputStream()))
{
//Client requests a song from the server (song is a String)
toServer.writeUTF(".Music."+song);
toServer.flush();
//Create a new file (tempSong is a string)
File test = new File(tempSong);
test.createNewFile();
//New file writer
BufferedOutputStream bOS = new BufferedOutputStream(new FileOutputStream(test));
byte[] buffer = new byte[4096];
int current;
/**
* Read the bytes from the server and write the file
* The file is written and I can play it (externally)
* but the while loop doesn't break after writting the file
*/
while ((current = fromServer.read(buffer)) > 0)
{
bOS.write(buffer, 0 , current);
}
System.out.println("Finished writing");
bOS.close();
/**
* down here a method is ran to play the file
* but it never happen because the task is still in the while loop
*/
}
catch (IOException e)
{
e.printStackTrace();
}
return null;
}
below is in the server where the server reads the message and sends the file
/** This is part of a task and the main call area */
#Override
public Void call ()
{
try
{
//Setup I/O
toClient = new ObjectOutputStream(socket.getOutputStream());
ObjectInputStream fromClient = new ObjectInputStream(socket.getInputStream());
while(!socket.isClosed())
{
//If server has received a message
if(fromClient.available() > 0)
{
//Reads message and objects from client
String input = fromClient.readUTF();
if (input.contains(".Music"))
{
findMusic(input, toClient);
}
/**
* more else IFs
*/
}
}
}
catch (Exception e)
{
e.printStackTrace();
}
return null;
}
/**
* this method is part of the task discussed above
*/
//The method that is called
private void findMusic(String input, ObjectOutputStream toClient)
{
logoff();
String[] names = input.split("[.]");
clientManagerTemp.logger("Someone request song: " + names[2] + ".mp3");
File musicFile = AudioUtil.getSoundFile("src/Resources/Songs/" + names[2]+ ".mp3");
byte[] buffer = new byte[(int) musicFile.length()];
try(BufferedInputStream bis = new BufferedInputStream(new FileInputStream(musicFile)))
{
bis.read(buffer, 0, buffer.length);
clientManagerTemp.logger("Sending " + "src/Resources/Songs/" + names[2]+ ".mp3" + "(" + buffer.length + " bytes)");
//write the file to the client
toClient.write(buffer,0, buffer.length);
toClient.flush();
clientManagerTemp.logger("Finished sending");
}
catch (IOException e)
{
e.printStackTrace();
}
}
So as you can see the server sends the file fine and my client receives it. It just won't stop the while loop. Can someone explain why? So I can get a better understanding of how streaming bytes over sockets work
**EDIT
When the client receives the file it can be played even after closing the client and server

You can send the file-length to the client, with this information the client knows when to exit the loop.
Server
private void findMusic(String input, ObjectOutputStream toClient)
{
logoff();
String[] names = input.split("[.]");
clientManagerTemp.logger("Someone request song: " + names[2] + ".mp3");
File musicFile = AudioUtil.getSoundFile("src/Resources/Songs/" + names[2]+ ".mp3");
int fileLength = (int) musicFile.length();
byte[] buffer = new byte[fileLength];
try(BufferedInputStream bis = new BufferedInputStream(new FileInputStream(musicFile)))
{
bis.read(buffer, 0, buffer.length);
clientManagerTemp.logger("Sending " + "src/Resources/Songs/" + names[2]+ ".mp3" + "(" + buffer.length + " bytes)");
//write the file to the client
toClient.writeInt(fileLength);
toClient.write(buffer,0, buffer.length);
toClient.flush();
clientManagerTemp.logger("Finished sending");
}
catch (IOException e)
{
e.printStackTrace();
}
}
Client
byte[] buffer = new byte[4096];
int current;
int fileLength = fromServer.readInt();
while ( fileLength > 0 && (current = fromServer.read(buffer, 0, Math.min(4096,fileLength))) > 0)
{
bOS.write(buffer, 0 , current);
fileLength -= current;
}

This is due to the nature of the read method, seen here Javadocs. This method will block until there is data, so your loop will never end. Now the reason for this is because you never close the stream on the server end, you just flush it, which forces a send of all the data currently in the buffer, but will not close the stream. If you call .close() on the stream from the server side then that should exit the while loop on the client side and continue to your playing code.
I haven't tested this, but from the documentation and a brief look at your code, that seems to be the issue.

If you don't close the connection (server side), there will not be an end-of-file/end-of-stream and your client side will run the loop forever, unless you configured a read timeout.
If you won't to close the loop automatically after sending the file is finished, send the count of bytes (size of file) first to client and then the file itself. This makes it possible to read only a exact number of bytes on client side and close the connection when finished.
Alternative you can send a special sequence and check for them at client side to mark the end of stream.

The reason you program is stuck is due to the behaviour of ObjectInputStream.read(). This method is blocking until it has read some data or the stream is closed. Since you never close the stream the 2nd condition will never be met. So there is only option 1. To return when data arrived. Unfortuenatly there is no way to recognize the end of a file unless you define it yourself e.g. by sending a unique sequenz or something that you can recognize to break the while loop.
Example:
while ((current = fromServer.read(buffer)) > 0)
{
// example value 42 could be anything else as well
if(current == -42)
{
break;
}
bOS.write(buffer, 0 , current);
}
The downside of this is, that your data could also contain this special value so you gotta think about different possibilities e.g memorize the sequence up to 3 values and compare them to an 'interrupt sequence'.

Socket intermittently reads only 1448/2896 bytes

I am using Commons-IO to read and write from Socket. Things all works till payload size is either 1448/2896 max.
Below is the code snippet. Really unsure how to handle it.
Checked system buffer size too
$ cat /proc/sys/net/ipv4/tcp_wmem
4096 16384 4194304
public static void usingCommonsIO(){
Socket socket = null;
try {
socket = new Socket(serverIP, 55000);
IOUtils.write(request.getBytes(), socket.getOutputStream());
System.out.println("Message Sent....");
StringBuilder response = new StringBuilder();
String resp =IOUtils.toString(socket.getInputStream(), "UTF-8");
System.out.println(resp);
} catch (IOException e) {
e.printStackTrace();
}
}
Alternatively tried using DataInputStream but no luck. Code snipped is below.
public static void usingDataIOStream(String requestStr){
Socket socket = null;
try {
socket = new Socket("192.168.1.6", 55000);
System.out.println("Request Length -:" + request.length());
DataOutputStream out = new DataOutputStream(socket.getOutputStream());
out.write(requestStr.getBytes("UTF-8"), 0, requestStr.length());
out.flush();
System.out.println("Message Sent....");
DataInputStream din = new DataInputStream(socket.getInputStream());
byte[] response = new byte[16*1024];
int responseLength = din.read(response);
System.out.println("Response -:" + new java.lang.String(response, 0, responseLength));
} catch (IOException e) {
e.printStackTrace();
}
}
Confusing part is that the same code works with only 1448 bytes sometimes and max of 2896 bytes sometimes. There are no specific patterns.
Update 1
To simulate it, tried writing Server socket on my own and code is as below. Strange thing noticed with this is, on first request payload of size 6500 was read and received properly. Connection Reset from second request onwards. Am I missing something here?
public static void usingBAOS() throws IOException {
server = new ServerSocket(port);
Socket socket = null;
DataInputStream din = null;
DataOutputStream dos = null;
while (true) {
System.out.println("Waiting for Client...");
try {
// Accepting Client's connection
socket = server.accept();
System.out.println("Connnected to client " + socket.getInetAddress());
din = new DataInputStream(socket.getInputStream());
// Read request payload from Socket
String requestString = readRequest(din);
System.out.println("Request Read.....");
System.out.println("Writing Response.....");
// Writing response to socket
dos = writeResponse(socket, requestString);
} catch (IOException e) {
e.printStackTrace();
}finally {
//close resources
din.close();
System.out.println("InputStream is closed......");
dos.close();
System.out.println("OutputStream is closed......");
socket.close();
System.out.println("Socket is closed......");
}
}
}
private static DataOutputStream writeResponse(Socket socket, String requestString) throws IOException {
String responseString = "Hi Client" + requestString;
DataOutputStream dos = new DataOutputStream(socket.getOutputStream());
//write object to Socket
dos.write(responseString.getBytes(),0, responseString.getBytes().length);
dos.flush();
return dos;
}
private static String readRequest(DataInputStream din) throws IOException {
byte[] response = new byte[16*1024];
ByteArrayOutputStream baos = new ByteArrayOutputStream();
int n = 0;
boolean read = true;
while(read){
n = din.read(response);
baos.write(response, 0, n);
if(baos.toString().length() == n){
read = false;
}
}
baos.flush();
String requestString = baos.toString();
return requestString;
}

Although this question is old at the time of writing this answer I'm putting this here for others in case it solves their problem. I encountered the same issue when using buffered data input and output streams on Android 8.0 devices where I had naively assumed that doing this:
int len = 2304;
byte[] data = new byte[len];
inputStream.read(data, 0, len);
would read all the data I sent down the socket. But as suggested by #Kayaman in the comments, this does not guarantee that len bytes of data are actually read from the buffer even if there are bytes available. In fact, this is in the documentation:
public final int read(byte[] b, int off, int len) throws IOException
Reads up to len bytes of data from the contained input stream into an array of bytes. An attempt is made to read as many as len bytes, but a smaller number may be read, possibly zero. The number of bytes actually read is returned as an integer.
In fact, if it doesn't read all the data, the only way to tell is to capture the returned value. My solution was then to monitor the amount of bytes actually read from the stream and just call read() in a loop as:
int i = 0;
len = 2304;
byte[] data = new byte[len];
while (i < len)
{
i += socket.inputStream.read(data, i, len - i);
}
Hope this helps someone.

Java socket timing out: Broken pipe

I'm writing a simple server in Java, and I'm able to retrieve incoming data from the client on the server side, but not on the client side due to a 2000ms timeout. Anyone know why this times out?
This is the server's code:
private static void listen() throws IOException {
while(true) {
Socket clientSocket = serverSocket.accept();
StringBuilder bufferedStringInput = new StringBuilder();
CharBuffer cbuf = CharBuffer.allocate(4096);
try {
InputStream is = clientSocket.getInputStream();
BufferedReader br = new BufferedReader(new InputStreamReader(is, "UTF8"));
int noCharsLeft = 0;
while ((noCharsLeft = br.read(cbuf)) != -1) {
char[] arr = new char[noCharsLeft];
cbuf.rewind();
cbuf.get(arr);
bufferedStringInput.append(arr);
cbuf.clear();
}
System.out.println(bufferedStringInput.toString());
} catch (IOException e) {
System.out.println("Error received client data: " + e.getMessage());
}
String message = "Hello client";
try {
PrintWriter out = new PrintWriter(clientSocket.getOutputStream());
out.print(message);
} catch (IOException e) {
System.out.println("Error getting output stream from client: " + e.getMessage());
}
clientSocket.close();
}
}

You're reading the input until end of stream, which only happens when the peer closes the connection, then you're trying to write to it, so of course you get a broken pipe. Doesn't make sense. You should just read the input until you have one entire request, whatever that means in your protocol.
There are other problems lurking here:
If the client code uses readLine(), you're not sending a line terminator: use println(), not print(), and close the PrintWriter, not just the client socket.
cbuf.rewind()/get()/clear() should be cbuf.flip()/get()/compact().
But it would make more sense to read directly into a char[] cbuf = new char[8192]; array, then bufferedStringInput.append(cbuf, 0, noCharsLeft), and forget about the CharBuffer altogether. Too much data copying at present.
noCharsLeft is a poor name for that variable. It is a read count.

Reading a String array [NETWORKING]

I have this block of code to read an array sent from the server to the client of recent updates, the issue is that sometimes it works and sometimes it doesn't. It will print out the version properly, but everything else will either not print out, all print out on the same line, or have 2 on the same line.
The purpose of this is to receive a String[] of recent updates from the server, which is looped through and sent as an individual String. Those updates are then displayed on a GUI.
private Response update() {
try {
Socket socket = new Socket(RS2Client.IP, 55555);
byte[] bytes = new byte[1024];
socket.getInputStream().read(bytes);
String version = new String(bytes);
System.err.println("VERSION READ " + version);
for (int i = 0; i < 6; i++) {
byte[] b = new byte[1024];
socket.getInputStream().read(b);
String text = new String(b);
getRecentUpdates().add(text.trim());
System.out.println("New update: " + text);
}
for (String update : getRecentUpdates()) {
System.err.println(update);
}
System.out.println("Client connected! Version: " + version);
socket.close();
if (Double.parseDouble(version) != RS2Client.CLIENT_VERSION) {
return Response.BAD;
}
} catch (Exception e) {
e.printStackTrace();
return Response.ERROR;
}
return Response.GOOD;
}

A socket sends a stream of bytes. It does not keep track of the end of each byte array you send.
If you want to send a byte array, you should send the length first, so you know how many bytes to expect.

Send byte array through socket between perl and Java

I have a big issue with my java client.
Server: perl, with IO::Socket;
$n is the socket
sub listen{
while(1){
my $n = $_[0]->accept();
my $thread;
my $thread2;
$thread = threads->create('talk', $n);
$thread2 = threads->create('recu', $n);
}
When a client send a message to the server
sub talk{
my $n = $_[0];
while(<$n>){
print $n $_;
}
In 'talk' the server send the client message back to the client
Client: Java, in a Thread
I send a byte array ->
static DataOutputStream os;
...
public static void handleMsg(byte [] b){
try {
os.write(b);
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
->
<pre><code>
byte[] buff = new byte[]{10,4};
ThreadListen.handleMsg(buff);
</code></pre>
I receive in the run() method only the first byte in the array (10)
<pre><code>
public void run(){
DataInputStream in = null;
try{
in = new DataInputStream(s.getInputStream());
} catch (IOException e) {
System.out.println("in or out failed");
System.exit(-1);
}
while(true){
try{
byte[] buff = new byte[6];
int b = in.read(buff, 0, buff.length);
}catch (IOException e) {
System.out.println("Read failed");
System.exit(-1);
}
}
}
If I try to send
byte[] buff = new byte[]{50,4};
ThreadListen.handleMsg(buff);
I receive nothing!!!
Did i missed something, I assume that if I send
byte[] buff = new byte[]{50,4};
I should receive 50,4 :)
Thanks,

Your Perl code is doing <$n> so it's fetching data line-by-line. since 10 is the line-feed character, receiving a {10,4} sequence means it gets an empty line (which it prints back) and then a character 4. Even if it receives that (some debug messages in the Perl code would help), the print back to the socket may not complete if the socket is block-buffered (likely the default) without flushing the socket filehandle.

You may want to use read explicitly (instead of readline implicitly, which as the name suggests reads up to the end of the line, or EOF) to read from the socket.