Develop Reference

java: send large files through sockets

So im making a client-server based architecture programm where server is sending a file and client receives it.
I have seen a lot of code parts and i also made a lot where i send small files. ex images. In my case i want to send .wav audio files which are large(40mb). This is what i have done so far. When i run my client, it just proceed to download, but never really downloads the file. I guess its because its too large.
How to send such large files?
server
public void send_file_to_client(String requested_file) throws IOException {
FileInputStream fis = null;
BufferedInputStream bis = null;
File FILE_TO_SEND = new File("C:\\ServerMusicStorage\\" + requested_file + ".wav");
byte[] mybytearray = new byte[(int) FILE_TO_SEND.length()];
try {
fis = new FileInputStream(FILE_TO_SEND);
bis = new BufferedInputStream(fis);
} catch (FileNotFoundException ex) {
Logger.getLogger(ClientHandler.class.getName()).log(Level.SEVERE, null, ex);
}
OutputStream os = null;
bis.read(mybytearray, 0, mybytearray.length);
os = connsock.getOutputStream();
System.out.println("Sending " + FILE_TO_SEND + "(" + mybytearray.length + " bytes)");
toClient.writeUTF(Integer.toString(mybytearray.length));
os.write(mybytearray, 0, mybytearray.length);
os.flush();
System.out.println("Done.");
if (bis != null) {
bis.close();
}
if (os != null) {
os.close();
}
if (connsock != null) {
connsock.close();
}
}
client
public static void receive_file(String requested_file) throws IOException {
File file_to_save = new File("C:\\ClientMusicStorage\\" + requested_file + ".wav");
int bytesRead;
int current = 0;
FileOutputStream fos = null;
BufferedOutputStream bos = null;
//get file size to create the bytearray
String fileSize=fromServer.readUTF();
int final_file_size = Integer.parseInt(fileSize);
byte[] mybytearray = new byte[final_file_size];
InputStream is = clientSocket.getInputStream();
fos = new FileOutputStream(file_to_save);
bos = new BufferedOutputStream(fos);
bytesRead = is.read(mybytearray, 0, mybytearray.length);
current = bytesRead;
do {
bytesRead
= is.read(mybytearray, current, (mybytearray.length - current));
if (bytesRead >= 0) {
current += bytesRead;
}
} while (bytesRead > -1);
bos.write(mybytearray, 0, current);
bos.flush();
}

How to read Socket InputStream for multiple files over byte[] streams?

The code I have posted below works for single file transfer over a socket. But it doesn't work for multiple file transfers over a socket. When trying multiple file transfers over the socket the code crashes.
I send multiple files by looping over the server sending code x amount of times, and run the receiving code x amount of times. When trying to send multiple files, the first file will send successfully, the second file name and size will be read successfully but the error in my code happens after this.
In my receiving client I tried to use to suggestion posted here: Java multiple file transfer over socket but had no success.
The error is on the client side.
The question I am asking is: Why isn't this code working for multiple files, and how can I fix it?
Server Sending
try{
byte[] bytes = new byte[(int)file.length()];
FileInputStream fis = new FileInputStream(file);
OutputStream os = socket.getOutputStream();
out.println(file.getName()); // Send Filename
out.println(file.length()); // Send filesize
int count;
while ((count = fis.read(bytes)) > 0) {
os.write(bytes, 0, count);
}
os.flush();
fis.close();
}catch(IOException e){
e.printStackTrace();
}
}
Client Recieving
try{
String file = in.readLine(); // Read filename
int fileSize = Integer.parseInt(in.readLine()); // Read Filesize
//ERROR HAPPENING ON LINE ABOVE IN LOOPS AFTER THE FIRST
byte [] buf = new byte [fileSize];
FileOutputStream fos = new FileOutputStream(file);
InputStream is = socket.getInputStream();
int count = 0;
while (fileSize > 0 && (count = is.read(buf, 0, (int)Math.min(buf.length, fileSize))) != -1){
fos.write(buf, 0, count);
fileSize -= count;
}
fos.close();
}catch(IOException e){
e.printStackTrace();
}
The error is a NumberFormatException, on loops after the first when the client is receiving part of a file for the input to the fileSize.

Make sure you flush the PrintWriter before you then write raw bytes directly to the OutputStream that the PrintWriter is attached to. Otherwise, you could write any buffer data out of order to the underlying socket.
But more importantly, make sure that if you use buffered reading on the receiving end that you read the file bytes using the same buffer that receives the file name and file size. You should also transfer the File using smaller fixed chunks, don't allocate a single byte[] array for the entire file size, that is a waste of memory for large files, and likely to fail.
Server:
try{
byte[] bytes = new byte[1024];
FileInputStream fis = new FileInputStream(file);
OutputStream os = socket.getOutputStream();
BufferedOutputStream bos = new BufferedOutputStream(os);
PrinterWriter pw = new PrintWriter(bos);
pw.println(file.getName()); // Send Filename
pw.println(file.length()); // Send filesize
pw.flush();
int count;
while ((count = fis.read(bytes)) > 0) {
bos.write(bytes, 0, count);
}
bos.flush();
fis.close();
}catch(IOException e){
e.printStackTrace();
}
}
Client:
try{
byte [] buf = new byte [1024];
FileOutputStream fos = new FileOutputStream(file);
InputStream is = socket.getInputStream();
BufferedInputStream bis = new BufferedInputStream(is);
InputStreamReader isr = new InputStreamReader(bis);
String file = isr.readLine(); // Read filename
long fileSize = Long.parseLong(isr.readLine()); // Read Filesize
int count = 0;
while ((fileSize > 0) && (count = bis.read(buf, 0, (int)Math.min(buf.length, fileSize))) > 0){
fos.write(buf, 0, count);
fileSize -= count;
}
fos.close();
}catch(IOException e){
e.printStackTrace();
}
That being said, you might also consider using DataOutputStream.writeLong() and DataInputStream.readLong() to send/receive the file size in its original binary format instead of as a textual string:
Server:
try{
byte[] bytes = new byte[1024];
FileInputStream fis = new FileInputStream(file);
OutputStream os = socket.getOutputStream();
BufferedOutputStream bos = new BufferedOutputStream(os);
PrinterWriter pw = new PrintWriter(bos);
pw.println(file.getName()); // Send Filename
pw.flush();
DataOutputStream dos = new DataOutputStream(bos);
dos.writeLong(file.length()); // Send filesize
dos.flush();
int count;
while ((count = fis.read(bytes)) > 0) {
bos.write(bytes, 0, count);
}
bos.flush();
fis.close();
}catch(IOException e){
e.printStackTrace();
}
}
Client:
try{
byte [] buf = new byte [1024];
FileOutputStream fos = new FileOutputStream(file);
InputStream is = socket.getInputStream();
BufferedInputStream bis = new BufferedInputStream(is);
InputStreamReader isr = new InputStreamReader(bis);
String file = isr.readLine(); // Read filename
DataInputStream dis = new DataInputStream(bos);
long fileSize = dis.readLong(); // Read Filesize
int count = 0;
while ((fileSize > 0) && (count = bis.read(buf, 0, (int)Math.min(buf.length, fileSize))) > 0){
fos.write(buf, 0, count);
fileSize -= count;
}
fos.close();
}catch(IOException e){
e.printStackTrace();
}

Transferring Files via TCP/IP works in Windows but not Ubuntu

I was trying my hands at sending multiple files using TCP/IP connection with the help of this guide.
When I execute my testClient codes from Ubuntu 14.04, files are transferred over to testServer in Windows 7. The .txt files received here are in correct format.
However, when I execute testClients codes from Windows 7 and testServer from Ubuntu 14.04, the files received in Ubuntu 14.04 was messed up. (The contents from txt#2 contents spill over to txt#1.)
During the swap, none of the codes in both testServer and testClients were changed other than their IP address and file location.
I am confused. Why did the codes work fine in Windows 7 but not in Ubuntu? Is there something wrong with my codes? I would appreciate any help on this.
TestServer.java
public static void main(String[] args) throws IOException {
FileOutputStream fos;
BufferedOutputStream bos;
OutputStream output;
DataOutputStream dos;
int len;
int smblen;
InputStream in;
boolean flag = true;
DataInputStream clientData;
BufferedInputStream clientBuff;
ServerSocket serverSocket = new ServerSocket(5991);
serverSocket.setSoTimeout(500000);
System.out.println("Waiting for client on port " + serverSocket.getLocalPort());
Socket clientSocket = null;
clientSocket = serverSocket.accept();
System.out.println("Just connected to " + clientSocket.getRemoteSocketAddress());
while (true){
while(flag==true) {
in = clientSocket.getInputStream();
clientData = new DataInputStream(in);
clientBuff = new BufferedInputStream(in);
int fileSize = clientData.read();
if (fileSize != 0)
System.out.println("Receiving " + fileSize + " files.\n");
//Store filenames and file sizes from client directory
ArrayList<File>files=new ArrayList<File>(fileSize);
ArrayList<Integer>sizes = new ArrayList<Integer>(fileSize);
//Server accepts filenames
for (int count=0; count<fileSize; count ++){
File ff=new File(clientData.readUTF());
files.add(ff);
}
for (int count=0; count<fileSize; count ++){
sizes.add(clientData.readInt());
}
for (int count=0; count<fileSize; count++) {
if (fileSize - count == 1) {
flag = false;
}
len = sizes.get(count);
output = new FileOutputStream("/home/pp/Desktop/inResources/" + files.get(count));
dos = new DataOutputStream(output);
bos = new BufferedOutputStream(output);
byte[] buffer = new byte[1024];
bos.write(buffer, 0, buffer.length);
while (len > 0 && (smblen = clientData.read(buffer)) > 0) {
dos.write(buffer, 0, smblen);
len = len - smblen;
dos.flush();
}
dos.close();
System.out.println("File " + files.get(count) + " with " + sizes.get(count) + " bytes recieved.");
}
}
if (flag == false) {
System.out.println("\nTransfer completed. Closing socket...");
serverSocket.close();
break;
}
}
}
TestClient.java
public static void main(String[] args) throws IOException {
String serverName = "192.168.1.12"; //IP address
int port = 5991;
Socket sock = new Socket(serverName, port);
System.out.println("Connected to " + serverName + " on port " + port + "\n");
File myFile = new File("C:\\Users\\inter2\\Desktop\\noobs\\outResources");
File[] files = myFile.listFiles();
OutputStream os = sock.getOutputStream();
DataOutputStream dos = new DataOutputStream(os);
//Sending total number of files in folder
dos.writeInt(files.length);
//Sending file names
for (int count=0; count<files.length; count++) {
dos.writeUTF(files[count].getName());
}
//Sending file sizes
for (int count=0; count<files.length; count++) {
int filesize = (int) files[count].length();
dos.writeInt(filesize);
}
//Sending of files
for (int count=0; count<files.length; count ++) {
int filesize = (int) files[count].length();
byte [] buffer = new byte [filesize];
FileInputStream fis = new FileInputStream(files[count].toString());
BufferedInputStream bis = new BufferedInputStream(fis);
//Sending file name and file size to the server
bis.read(buffer, 0, buffer.length);
dos.write(buffer, 0, buffer.length);
dos.flush();
System.out.println("Sending file " + files[count].getName() + " with " + filesize + " bytes.");
}
System.out.println("\n" + files.length + " files successfully transfered.");
sock.close();
}

This code never worked.
int fileSize = clientData.read();
Here you are reading the amount of files, as a byte.
dos.writeInt(files.length)
Here you are writing that amount, as an int. So already your writer is three bytes ahead of your reader.
Change read() to readInt() above.
Other notes:
Receiving:
byte[] buffer = new byte[1024];
This is OK but a bigger buffer would be more efficient, say 8192.
bos.write(buffer, 0, buffer.length);
This is a bug. Remove it. You are writing 1024 null bytes at the beginning of the file.
while (len > 0 && (smblen = clientData.read(buffer)) > 0) {
dos.write(buffer, 0, smblen);
len = len - smblen;
dos.flush();
Don't flush inside loops.
}
Sending:
byte [] buffer = new byte [filesize];
FileInputStream fis = new FileInputStream(files[count].toString());
BufferedInputStream bis = new BufferedInputStream(fis);
//Sending file name and file size to the server
bis.read(buffer, 0, buffer.length);
dos.write(buffer, 0, buffer.length);
No need to waste a buffer of the size of the file. Use the same code you use for receiving above.

FileOutputStream creates file even if i check file existance

I'm making simple Client-Server application to make file copies on server.
There is clientside method for sending file to server:
private void makeCopy(Socket clientSock) throws IOException {
File file = new File("D:\\client\\toCopy.bmp");
File dest = new File("D:\\server\\copyFile.bmp");
boolean ifExists = dest.exists();
if(ifExists && !file.isDirectory()){
System.out.println("Copy is already made on server.");
}
else{
fis = new FileInputStream(file);
byte[] buffer = new byte[4096];
while (fis.read(buffer) > 0) {
oos.write(buffer);
}
//fis.close();
oos.close();
}
}
Also there is serverside method for receiving file from client:
public void saveFile(Socket s) throws IOException{
File copy = new File("D:\\server\\fileCopy.bmp");
fos = new FileOutputStream(copy);
File fromServer = new File("D:\\client\\toCopy.bmp");
if(copy.exists() && !copy.isDirectory()){
}
else{
byte[] buffer = new byte[4096];
int filesize = (int)fromServer.length();
int read = 0;
int totalRead = 0;
int remaining = filesize;
while((read = ois.read(buffer, 0, Math.min(buffer.length, remaining))) > 0) {
totalRead += read;
remaining -= read;
System.out.println("read " + totalRead + " bytes.");
fos.write(buffer, 0, read);
}
}
}
The problem is, even if i check file existance it still makes file which I can't open (it has 0 bytes written). Any ideas ?

FileOutputStream creates a file output stream and a file in a file system.
First you should check existence and then create FileOutputStream.
File copy = new File("D:\\server\\fileCopy.bmp");
if(copy.exists() && !copy.isDirectory()){ }
fos = new FileOutputStream(copy);
File fromServer = new File("D:\\client\\toCopy.bmp");

java send file using sockets

I am trying to send a file from one computer to another using Java. I have written the code below, it works fine if both sender and receiver are started in the same computer but if they work on different machines received file size is bigger than the original file and it is corrupted.
Note: I am trying to transfer files which are max 10 MBs.
How can I fix this?
Sender:
ServerSocket server_socket = new ServerSocket(8989);
File myFile = new File(myPath);
Socket socket = server_socket.accept();
int count;
byte[] buffer = new byte[1024];
OutputStream out = socket.getOutputStream();
BufferedInputStream in = new BufferedInputStream(new FileInputStream(myFile));
while ((count = in.read(buffer)) > 0) {
out.write(buffer, 0, count);
out.flush();
}
socket.close();
Receiver:
Socket socket = new Socket(address, 8989);
FileOutputStream fos = new FileOutputStream(anotherPath);
BufferedOutputStream out = new BufferedOutputStream(fos);
byte[] buffer = new byte[1024];
int count;
InputStream in = socket.getInputStream();
while((count=in.read(buffer)) >0){
fos.write(buffer);
}
fos.close();
socket.close();

On the client side you write up to count bytes and send them:
while ((count = in.read(buffer)) > 0) {
out.write(buffer, 0, count);
on the server side you read up to count bytes - but then you write the whole buffer to file!
while((count=in.read(buffer)) > 0){
fos.write(buffer);
Just change it to:
fos.write(buffer, 0, count);
and you'll be on the safe side. BTW your program has another small bug: read() can return 0 which doesn't mean InputStream ended. Use >= instead:
count = in.read(buffer)) >= 0
Have you considered IOUtils.copy(InputStream, OutputStream) from Apache Commons? It would reduce your whole while loops to:
OutputStream out = socket.getOutputStream();
InputStream in = new FileInputStream(myFile);
IOUtils.copy(in, out);
socket.close();
Less code to write, less code to test. And buffering is done internally.

Remember that in.read(buffer) not necessarily fills up the whole buffer with new data. Therefore you should make sure you don't write the whole buffer. Change
while((count=in.read(buffer)) >0){
fos.write(buffer);
}
to
while((count=in.read(buffer)) >0){
fos.write(buffer, 0, count);
}

sender
Socket sock = new Socket("127.0.0.1", 5991);
System.out.println("Connecting.........");
File myFile = new File("/root/qrcode/");
File[] files = myFile.listFiles();
OutputStream os = sock.getOutputStream();
BufferedOutputStream bos = new BufferedOutputStream(os);
DataOutputStream dos = new DataOutputStream(bos);
dos.writeInt(files.length);
long totalBytesRead = 0;
int percentCompleted = 0;
for(File file : files)
{
long length = file.length();
dos.writeLong(length);
String name = file.getName();
dos.writeUTF(name);
FileInputStream fis = new FileInputStream(file);
BufferedInputStream bis = new BufferedInputStream(fis);
int theByte = 0;
while((theByte = bis.read()) != -1)
{
totalBytesRead += theByte;
bos.write(theByte);
}
// System.out.println("file read");
bis.close();
}
dos.close();
//Closing socket
sock.close();
receiver
ServerSocket serverSocket = new ServerSocket(5991);
while(true) {
Socket clientSocket = null;
System.out.println("Starting...");
clientSocket = serverSocket.accept();
InputStream in = clientSocket.getInputStream(); //used
BufferedInputStream bis = new BufferedInputStream(in);
String dirPath ;
dirPath = "/root/NewFolder";
try{
DataInputStream dis = new DataInputStream(bis);
int filesCount = dis.readInt();
File[] files = new File[filesCount];
long f_l = 0;
int count =0 ;
long totalBytesRead = 0;
int percentCompleted = 0;
for(int i = 0; i < filesCount; i++)
{
long fileLength = dis.readLong();
String fileName = dis.readUTF();
f_l = f_l +fileLength;
files[i] = new File(dirPath + "/" + fileName);
FileOutputStream fos = new FileOutputStream(files[i]);
BufferedOutputStream bos = new BufferedOutputStream(fos);
int tot = 0;
for(int j = 0; j < fileLength; j++) {
bos.write(bis.read());
}
bos.close();
}
}catch(Exception ex)
{
System.out.println("error in socket programming ");
}
}