Develop Reference

Stream isn't closing after sending file

Excuse me my understanding of streams is slowly building.
I initally tried to stream a music file over to my client but that didn't work out so I moved transferring the entire file via bytes then saving it at the client side. The problem is the input stream is still receiving bytes and so won't break out of the while loop (Where I want to play the song)
Below is part of my client code:
//This is part of a task that carries this out (Part of a UI application)
#Override
protected Void call()
{
try(Socket socket = new Socket(host,portNumber);
ObjectOutputStream toServer = new ObjectOutputStream(socket.getOutputStream());
ObjectInputStream fromServer = new ObjectInputStream(socket.getInputStream()))
{
//Client requests a song from the server (song is a String)
toServer.writeUTF(".Music."+song);
toServer.flush();
//Create a new file (tempSong is a string)
File test = new File(tempSong);
test.createNewFile();
//New file writer
BufferedOutputStream bOS = new BufferedOutputStream(new FileOutputStream(test));
byte[] buffer = new byte[4096];
int current;
/**
* Read the bytes from the server and write the file
* The file is written and I can play it (externally)
* but the while loop doesn't break after writting the file
*/
while ((current = fromServer.read(buffer)) > 0)
{
bOS.write(buffer, 0 , current);
}
System.out.println("Finished writing");
bOS.close();
/**
* down here a method is ran to play the file
* but it never happen because the task is still in the while loop
*/
}
catch (IOException e)
{
e.printStackTrace();
}
return null;
}
below is in the server where the server reads the message and sends the file
/** This is part of a task and the main call area */
#Override
public Void call ()
{
try
{
//Setup I/O
toClient = new ObjectOutputStream(socket.getOutputStream());
ObjectInputStream fromClient = new ObjectInputStream(socket.getInputStream());
while(!socket.isClosed())
{
//If server has received a message
if(fromClient.available() > 0)
{
//Reads message and objects from client
String input = fromClient.readUTF();
if (input.contains(".Music"))
{
findMusic(input, toClient);
}
/**
* more else IFs
*/
}
}
}
catch (Exception e)
{
e.printStackTrace();
}
return null;
}
/**
* this method is part of the task discussed above
*/
//The method that is called
private void findMusic(String input, ObjectOutputStream toClient)
{
logoff();
String[] names = input.split("[.]");
clientManagerTemp.logger("Someone request song: " + names[2] + ".mp3");
File musicFile = AudioUtil.getSoundFile("src/Resources/Songs/" + names[2]+ ".mp3");
byte[] buffer = new byte[(int) musicFile.length()];
try(BufferedInputStream bis = new BufferedInputStream(new FileInputStream(musicFile)))
{
bis.read(buffer, 0, buffer.length);
clientManagerTemp.logger("Sending " + "src/Resources/Songs/" + names[2]+ ".mp3" + "(" + buffer.length + " bytes)");
//write the file to the client
toClient.write(buffer,0, buffer.length);
toClient.flush();
clientManagerTemp.logger("Finished sending");
}
catch (IOException e)
{
e.printStackTrace();
}
}
So as you can see the server sends the file fine and my client receives it. It just won't stop the while loop. Can someone explain why? So I can get a better understanding of how streaming bytes over sockets work
**EDIT
When the client receives the file it can be played even after closing the client and server

You can send the file-length to the client, with this information the client knows when to exit the loop.
Server
private void findMusic(String input, ObjectOutputStream toClient)
{
logoff();
String[] names = input.split("[.]");
clientManagerTemp.logger("Someone request song: " + names[2] + ".mp3");
File musicFile = AudioUtil.getSoundFile("src/Resources/Songs/" + names[2]+ ".mp3");
int fileLength = (int) musicFile.length();
byte[] buffer = new byte[fileLength];
try(BufferedInputStream bis = new BufferedInputStream(new FileInputStream(musicFile)))
{
bis.read(buffer, 0, buffer.length);
clientManagerTemp.logger("Sending " + "src/Resources/Songs/" + names[2]+ ".mp3" + "(" + buffer.length + " bytes)");
//write the file to the client
toClient.writeInt(fileLength);
toClient.write(buffer,0, buffer.length);
toClient.flush();
clientManagerTemp.logger("Finished sending");
}
catch (IOException e)
{
e.printStackTrace();
}
}
Client
byte[] buffer = new byte[4096];
int current;
int fileLength = fromServer.readInt();
while ( fileLength > 0 && (current = fromServer.read(buffer, 0, Math.min(4096,fileLength))) > 0)
{
bOS.write(buffer, 0 , current);
fileLength -= current;
}

This is due to the nature of the read method, seen here Javadocs. This method will block until there is data, so your loop will never end. Now the reason for this is because you never close the stream on the server end, you just flush it, which forces a send of all the data currently in the buffer, but will not close the stream. If you call .close() on the stream from the server side then that should exit the while loop on the client side and continue to your playing code.
I haven't tested this, but from the documentation and a brief look at your code, that seems to be the issue.

If you don't close the connection (server side), there will not be an end-of-file/end-of-stream and your client side will run the loop forever, unless you configured a read timeout.
If you won't to close the loop automatically after sending the file is finished, send the count of bytes (size of file) first to client and then the file itself. This makes it possible to read only a exact number of bytes on client side and close the connection when finished.
Alternative you can send a special sequence and check for them at client side to mark the end of stream.

The reason you program is stuck is due to the behaviour of ObjectInputStream.read(). This method is blocking until it has read some data or the stream is closed. Since you never close the stream the 2nd condition will never be met. So there is only option 1. To return when data arrived. Unfortuenatly there is no way to recognize the end of a file unless you define it yourself e.g. by sending a unique sequenz or something that you can recognize to break the while loop.
Example:
while ((current = fromServer.read(buffer)) > 0)
{
// example value 42 could be anything else as well
if(current == -42)
{
break;
}
bOS.write(buffer, 0 , current);
}
The downside of this is, that your data could also contain this special value so you gotta think about different possibilities e.g memorize the sequence up to 3 values and compare them to an 'interrupt sequence'.

Java TCP File Transfer Only Complete On First Attempt

Despite hours of researching this problem, I have made very little progress. According to my professor, the code should be working as written...
I have a server that stays open, and a client that requests a file. Once the client receives the file, the client closes.
When I open the server, I am able to transfer a complete .jpg image file. The client then closes while the server remains open. I start up another client and try to transfer the same image, and only a portion of the bytes are transferred/written to the disk. The file transfer is only completely successful for the first file transferred by the server!
Additionally strange, a simple .txt text file never successfully transfers. I believe the cause is on the server side because it remains open as opposed to the client, which starts over each time.
Server Code:
import java.io.*;
import java.net.*;
import java.util.Arrays;
class ft_server {
public static void main(String args[]) throws Exception {
/*
* Asks user for port number and listens on that port
*/
BufferedReader portFromUser = new BufferedReader(new InputStreamReader(System.in));
System.out.println("Enter the port you'd like to use: ");
int portNumber = Integer.valueOf(portFromUser.readLine());
if (portNumber < 1 || portNumber > 65535) {
System.out.println("Please choose a port number between 1 and 65535.");
return;
}
portFromUser.close();
ServerSocket listenSocket = new ServerSocket(portNumber);
/*
* Finished with user input
*/
/*
* Continuously listens for clients:
*/
while (true) {
Socket clientSocket = listenSocket.accept();
BufferedReader inFromClient = new BufferedReader(new InputStreamReader(clientSocket.getInputStream()));
DataOutputStream outToClient = new DataOutputStream(clientSocket.getOutputStream());
String clientIP = clientSocket.getRemoteSocketAddress().toString();
System.out.println("The client " + clientIP + " connected!");
String clientMessage = inFromClient.readLine();
System.out.println("The client requested file: " + clientMessage);
// Get file. If doesn't exist, let's client know.
// Otherwise informs client of file size.
File myFile = new File(clientMessage);
if (!myFile.exists()) {
outToClient.writeBytes("File does not exist!\n");
return;
} else {
outToClient.writeBytes(String.valueOf((int)myFile.length()) + "\n");
}
// Create array for storage of file bytes:
byte[] byteArray = new byte[(int)myFile.length()];
BufferedInputStream bis = new BufferedInputStream(new FileInputStream(myFile));
// Read file into array:
bis.read(byteArray, 0, byteArray.length);
// Send the file:
outToClient.write(byteArray, 0, byteArray.length);
outToClient.close();
clientSocket.close();
}
}
}
Client Code:
import java.io.*;
import java.net.*;
class ft_client {
public static void main(String args[]) throws Exception {
int byteSize = 2022386;
int bytesRead;
/*
* Asks user for IP and port:
*/
BufferedReader inFromUser = new BufferedReader(new InputStreamReader(System.in));
System.out.println("Enter an IP address: ");
String ipAddress = inFromUser.readLine();
System.out.println("Enter a port: ");
String port = inFromUser.readLine();
Socket clientSocket;
try {
// Makes socket, port, and calls connect. Assumes it's TCP:
clientSocket = new Socket(ipAddress, Integer.valueOf(port));
} catch (Exception e) {
System.out.println(e.getMessage());
return;
}
// Creates InputStream from server to get file size and other messages:
BufferedReader inFromServer = new BufferedReader(new InputStreamReader(clientSocket.getInputStream()));
// Anything written to this will be sent to the server:
DataOutputStream outToServer = new DataOutputStream(clientSocket.getOutputStream());
// Asks for a file name to download from the server:
System.out.println("What file do you want?: ");
String message = inFromUser.readLine();
outToServer.writeBytes(message + "\n");
inFromUser.close();
// Listens for confirmation from server.
// If the file exists, the file size is delivered here:
String response = inFromServer.readLine();
System.out.println("File size: " + response);
if (response.equals("File does not exist!")) {
return;
}
// Receives file from server:
byteSize = (int) Integer.valueOf(response);
byte[] byteArray = new byte[byteSize];
InputStream is = clientSocket.getInputStream(); // calling clientSocket.getInputStream() twice???
FileOutputStream fos = new FileOutputStream(message);
BufferedOutputStream bos = new BufferedOutputStream(fos);
// Continuously writes the file to the disk until complete:
int total = 0;
while ((bytesRead = is.read(byteArray)) != -1) {
bos.write(byteArray, 0, bytesRead);
total += bytesRead;
}
bos.close();
System.out.println("File downloaded (" + total + " bytes read)");
clientSocket.close();
}
}
Are buffered readers interfering with output streams? Is there a better way to transfer files?

It's worth checking, in your server code, what value comes back from the file read() call, so:
int bytesRead = bis.read(byteArray, 0, byteArray.length);
System.out.println("File bytes read: " + bytesRead + " from file size: " + myFile.length());
The read() method is under no obligation to fill the byteArray - only to return something and to tell you how many bytes it read. From the docs, it:
Reads up to len bytes of data from this input stream into an array of
bytes. If len is not zero, the method blocks until some input is
available; otherwise, no bytes are read and 0 is returned.
You need to keep reading in a loop. I'd do this (actually, same as your client!):
int n;
while ((n = bis.read(byteArray, 0, byteArray.length)) != -1) {
// Send the chunk of n bytes
outToClient.write(byteArray, 0, n);
}
bis.close();
outToClient.close();
or something similar. I've closed the file too: it'd close on GC/finalize, but that could be a while, and meanwhile you're holding the file open.
EDIT
The specific problem with your image-read in this case is in your client code. You read the file size near the top of the code:
// Creates InputStream from server to get file size and other messages:
BufferedReader inFromServer = new BufferedReader(new InputStreamReader(clientSocket.getInputStream()));
and then you access the client again:
InputStream is = clientSocket.getInputStream(); // calling clientSocket.getInputStream() twice???
and as your comment suggests, this is bad! Thank you to #EJP for highlighting this!
This causes a problem of buffer over-ingestion: the BufferedReader consumes more bytes into its belly than you extract from it, so when you visit the clientSocket inputstream the second time, the read-pointer has moved on. You never look again at what the BufferedReader consumed.
As a general rule, once you plug buffering code onto something, you must be careful to read only from that buffer. In this case, it's difficult, because you can't read image (raw binary) data from a Reader, because it will busily interpret the binary values as characters and read them as UTF-8 or something.
Even without buffers, it's a minor sin to mix Readers (text oriented) and binary data (DataStreams) on the same stream. HTTP and email does this, so you are in good company, but they get away with it by being very tightly specified. Problem is, you can easily get snarled with questions of local/default character encoding at each end, whether you're reading Unix "LF" vs Windows "CR/LF" line endings etc.
In this case, try not using BufferedReaders at all, and try using DataInput/Output streams all the way. Try writeUTF(s) and readUTF() for transferring the String data. Ideally, create them like this:
DataInputStream inFromServer = new DataInputStream (new BufferedInputStream(clientSocket.getInputStream()));
so you still get the benefits of buffering.
EDIT 2
So seeing the new client code:
byteSize = (int) Integer.valueOf(response);
byte[] byteArray = new byte[byteSize];
FileOutputStream fos = new FileOutputStream(message);
int readBytes = inFromServer.read(byteArray);
// Continuously writes the file to the disk until complete:
int total = 0;
for (int i=0; i<byteArray.length; i++) {
fos.write(byteArray[i]);
total++;
}
fos.close();
Here, we're assuming that because the byteArray array is set to the right size, that the inFromServer.read(byteArray) will populate it - it won't. It's good to assume that any and all read operations will return you just as much data as the system has to hand: in this case, it's probably going to return as soon as it gets the first packet or two, with an underfilled array. This is same as C and Unix read behaviour too.
Try this - I'm repeatedly reading and writing a 4K buffer, until the byte count is reached (as determined by summing the return values of the reads):
byteSize = (int) Integer.valueOf(response);
byte[] byteArray = new byte[4096];
FileOutputStream fos = new FileOutputStream(message);
int total = 0;
// Continuously writes the file to the disk until complete:
while (total < byteSize && (readBytes = inFromServer.read(byteArray)) != -1) {
fos.write(byteArray, 0, readBytes);
total += readBytes;
}
fos.close();
A variant is this - same thing, but byte at a time. Might be a bit clearer. It's going to be slow - all those reads and writes are hitting the OS, but if you put a BufferedInputStream/BufferedOutputStream around the socket/file streams, it'll iron that out. I've added them:
DataInputStream inFromServer =
new DataInputStream(new BufferedInputStream(clientSocket.getInputStream()));
...
byteSize = (int) Integer.valueOf(response);
OutputStream fos = new BufferedOutputStream(FileOutputStream(message));
int total = 0;
int ch;
// Continuously writes the file to the disk until complete:
while (total < byteSize && (ch = inFromServer.read()) != -1) {
fos.write(ch);
total ++;
}
fos.close();
And finally! the simplest answer is this. Your code, but changed to:
int readBytes = inFromServer.readFully(byteArray);
Yes! Those nice people in 1990's Javasoft added a DataInput.readFully method, which does what you want! - basically wraps the code above. It's the simplest solution, and arguably most correct approach: "use existing libraries where possible". OTOH, it's the least educational, and the time you spend getting used to read/writes like this is not deducted from your life-expectancy!
And in fact, the readFully approach has severe limitations. Try pointing it at a 1GB file and see what happens (after you've fixed up the array size at the top): you'll a) run out memory, and b) wish that while you were ingesting a huge blob, you could at least be spooling it out to disk. If you try a 2.5G file, you'll notice that some of those ints should become longs to cope with numbers >= 2^31.
If it was me, I'd do the 4K buffer one. (BTW I'm writing this on a laptop with no Java compiler installed, so I haven't actually run the above! DO respond if there are any difficulties.)

Client-Server not sending the entire file

When I run my client server - which is connecting and I try to send a file it doesn't won't send the whole file which is pulling errors else where, it gets about halfway through and constantly stops at the same part. This set up works when running the server-client on the same machine so I am completely confused
Server --->
// output (a DataOutputstream) is set up elsewhere and messages are sent and received properly
output.writeInt((int)file.length());
// send file
byte [] mybytearray = new byte [(int)file.length()];
FileInputStream fis = new FileInputStream(file);
BufferedInputStream bis = new BufferedInputStream(fis);
bis.read(mybytearray,0,mybytearray.length);
System.out.println("Sending " + file + "(" + mybytearray.length + " bytes)");
output.write(mybytearray,0,mybytearray.length);
output.flush();
System.out.println("Done.");
Client --->
// input (a DataInputstream) is set up elsewhere and messages are sent and received properly
String FILE_TO_RECEIVED = "Load_From.xml";
File file = new File(FILE_TO_RECEIVED);
int FILE_SIZE = input.readInt();
if(FILE_SIZE!=0){
// receive file
System.out.println("received file size : " + FILE_SIZE);
byte [] mybytearray = new byte [FILE_SIZE];
FileOutputStream fos = new FileOutputStream(file);
BufferedOutputStream bos = new BufferedOutputStream(fos);
int bytesRead = input.read(mybytearray, 0, mybytearray.length);
bos.write(mybytearray, 0, bytesRead);

You are only reading part of the data, both when reading the file and when reading the socket. The read method returns the number of bytes read, and you need a loop to read everything. For example,
int read = 0, offset = 0;
while ((read = bis.read(mybytearray, offset, mybytearray.length - offset) != -1) {
offset += read;
}
Or you can use classes from the standard library, for example DataInputStream has a readFully method.
DataInputStream dis = new DataInputStream(fis);
dis.readFully(mybytearray);

Please see the documentation of read() at: http://docs.oracle.com/javase/7/docs/api/java/io/InputStream.html#read(byte[],%20int,%20int).
In particular:
An attempt is made to read as many as len bytes, but a smaller number may be read. The number of bytes actually read is returned as an integer.
This means that if the length returned is not what you expect and you haven't encountered the end of file yet, you should call read() again (in a loop).

InputStream to receive files twice explicitly [duplicate]

This question already has an answer here:
Receiving multiple images over TCP socket using InputStream
(1 answer)
Closed 9 years ago.
I am trying to send two images with a difference of 5 seconds between them from an android phone (client) to PC(server).
I am using InputStream to do this for me.
ServerSocket servsock = new ServerSocket(27508);
Socket sock = servsock.accept();
System.out.println("connection accepted ");
int count;
FileOutputStream fos = null;
BufferedOutputStream bos = null;
InputStream is = null;
is = sock.getInputStream();
int bufferSize = sock.getReceiveBufferSize();
byte[] bytes = new byte[bufferSize];
System.out.println("Here1");
fos = new FileOutputStream("D:\\fypimages\\image" + imgNum + ".jpeg");
bos = new BufferedOutputStream(fos);
imgNum++;
while ((count = is.read(bytes)) > 0)
{
bos.write(bytes, 0, count);
System.out.println("count: " + count);
}
bos.flush();
bytes = new byte[bufferSize];
System.out.println("Here2");
fos = new FileOutputStream("D:\\fypimages\\image" + imgNum + ".jpeg");
bos = new BufferedOutputStream(fos);
imgNum++;
while ((count = is.read(bytes)) > 0)
{
bos.write(bytes, 0, count);
System.out.println("count: " + count);
}
bos.flush();
System.out.println("Here3");
The problem is is.read(bytes) blocks the code only for the first image and then the program is terminated and it does not block for the second image.
I know it returns -1 when the first image is recieved completely, but how do I make it work for the second time ?

If read returns -1, it means other side closed the connection. But your basic problem seems to be, you're not handling the connection as stream. In a data stream, there are no inherent "packages", in this case no built-in way to distinguish one image from next.
You can proceed in at least 3 different ways:
Add your own simple protocol, for example: at sending side, write number of bytes in image, then write image bytes, then write number of bytes in next image, then write next image, etc, without closing the connection. And at receiving side, loop first reading the number of bytes, then reading that many bytes of image data.
Write one image per connection, then close the connection and create new connection for next image.
In this case, because data is JPEG images, just write all JPEG images as one data stream, then on receiving side, parse the JPEG format to see where the image boundaries are.
First choice is most efficient, and also is easily extended to deliver image name or other extra data in addition to image file length. Second is ok, and most simple and robust (for example, no need to worry about byte order, or worry about getting out of sync between sender and receiver), if there aren't too many images, but if there are hundreds of images, then re-connecting is going to slow things down a bit. Third choice is probably not the way to go with JPEGs, just listed is as a possiblity.

How do I send file name with file using sockets in Java? [duplicate]

This question already has answers here:
Java multiple file transfer over socket
(3 answers)
Closed 5 years ago.
I have the following code which transfers file via Sockets. How do I send the file name?
Socket socket = new Socket("localhost", port);//machine name, port number
File file = new File(fileName);
// Get the size of the file
long length = file.length();
if (length > Integer.MAX_VALUE)
{
System.out.println("File is too large.");
}
byte[] bytes = new byte[(int) length];
FileInputStream fis = new FileInputStream(file);
BufferedInputStream bis = new BufferedInputStream(fis);
BufferedOutputStream out = new BufferedOutputStream(socket.getOutputStream());
int count;
while ((count = bis.read(bytes)) > 0)
{
out.write(bytes, 0, count);
}
out.flush();
out.close();
fis.close();
bis.close();
socket.close();

You can invent your own protocol for your socket. If all you need is a filename and data, DataOutputStream.writeUTF is easiest:
BufferedOutputStream out = new BufferedOutputStream(socket.getOutputStream());
try (DataOutputStream d = new DataOutputStream(out)) {
d.writeUTF(fileName);
Files.copy(file.toPath(), d);
}
The peer must use the same protocol, of course:
BufferedInputStream in = new BufferedInputStream(socket.getInputStream());
try (DataInputStream d = new DataInputStream(in)) {
String fileName = d.readUTF();
Files.copy(d, Paths.get(fileName));
}

Use a character that can never be in a file name - such as a null (0x00, \0, whatever you want to call it). Then send a 64 bit integer that indicates how long, in bytes, the file is (make sure you don't run into buffer overflows, little endian/big endian issues, etc... just test all edge cases). Then send the file data. Then the ending socket will know which part is the file name, the file length and the file data, and will even be ready for the next file name if you want to send another.
(if file names can be arbitrary characters including control characters, ouch! Maybe send a 64 bit integer length of file name, the file name, a 64 bit integer length of file data, the file data, repeat ad infinitum?)
EDIT: To send a 64 bit integer over a socket, send its constituent bytes in a specific order, and make sure sender and receiver agree on the order. One example of how to do this is How to convert a Java Long to byte[] for Cassandra?

I tried to wrap a buffer which cause MalfuctionUTF and putting it on try-with resource closes the underlining socket stream and cause connection reset exceptionFollowing code worked for me
Client
DataOutputStream d = new DataOutputStream(out);
d.writeUTF(filename);
d.writeLong(length);
Server
DataInputStream d = new DataInputStream(in);
filename = d.readUTF();
fileLength = d.readLong();