I am creating a file server application for school assignment. What I currently have is a simple Client class that sends an image through TCP and a Server class that receives it and writes it to the file.
this is my client code
import java.io.*;
import java.net.*;
class Client {
public static void main(String args[]) throws Exception {
long start = System.currentTimeMillis();
Socket clientSocket = new Socket("127.0.0.1", 6789);
DataOutputStream outToServer = new DataOutputStream(clientSocket.getOutputStream());
File file = new File("hot.jpg");
FileInputStream fin = new FileInputStream(file);
byte sendData[] = new byte[(int)file.length()];
fin.read(sendData);
outToServer.write(sendData, 0, sendData.length);
clientSocket.close();
long end = System.currentTimeMillis();
System.out.println("Took " + (end - start) + "ms");
}
}
and this is my server code.
import java.io.*;
import java.net.*;
class Server {
public static void main(String args[]) throws Exception {
ServerSocket serverSocket = new ServerSocket(6789);
Socket connectionSocket = serverSocket.accept();
DataInputStream dis = new DataInputStream(connectionSocket.getInputStream());
byte[] receivedData = new byte[61500]; // <- THIS NUMBER
for(int i = 0; i < receivedData.length; i++)
receivedData[i] = dis.readByte();
connectionSocket.close();
serverSocket.close();
FileOutputStream fos = new FileOutputStream("received.jpg");
fos.write(receivedData);
fos.close();
}
}
My question is how to get the size of the file that is being sent. If you check the Server code you'll see that I've hardcoded the number i.e. 61500 at the moment. How can I retrieve this number dynamically?
Or, am I doing this the wrong way? What an alternative solution would be?
Add one "length field" before sending the file. (Note that since you read the file to memory the maximum size of the file can be ~2GB.)
Before sending the file write the length of the file:
outToServer.writeInt(sendData.length);
And when receiving read the length first and use it as a length:
int dataLength = dis.readInt()
byte[] receivedData = new byte[dataLength];
A better way would be not to read the file into memory first but to transfer it directly from the FileInputStream - then you could transfer bigger files!
If you know the length, using readFully() is much more efficient than reading a byte at a time.
In this case, you don't need to know the length, you can write the loop to read/write as much data as you get.
InputStream is = connectionSocket.getInputStream();
byte[] bytes = new byte[8192];
int len;
while((len = is.read(bytes)) > 0)
fos.write(bytes, 0, len);
You can avoid reading the whole file into memory by copying the data as you read it.
FileInputStream fis = new FileInputStream(filename);
OutputStream os = socket.getOutputStream();
byte[] bytes = new byte[8192];
int len;
while((len = fis.read(bytes)) > 0)
os.write(bytes, 0, len);
You can use Apache IOUtils.copy() to perform the copy from one stream to another if you wish.
This approach has the advantage that the file can be any size (greater than 2 GB). Using an array is limited to 2 GB (and uses more memory)
Related
Despite hours of researching this problem, I have made very little progress. According to my professor, the code should be working as written...
I have a server that stays open, and a client that requests a file. Once the client receives the file, the client closes.
When I open the server, I am able to transfer a complete .jpg image file. The client then closes while the server remains open. I start up another client and try to transfer the same image, and only a portion of the bytes are transferred/written to the disk. The file transfer is only completely successful for the first file transferred by the server!
Additionally strange, a simple .txt text file never successfully transfers. I believe the cause is on the server side because it remains open as opposed to the client, which starts over each time.
Server Code:
import java.io.*;
import java.net.*;
import java.util.Arrays;
class ft_server {
public static void main(String args[]) throws Exception {
/*
* Asks user for port number and listens on that port
*/
BufferedReader portFromUser = new BufferedReader(new InputStreamReader(System.in));
System.out.println("Enter the port you'd like to use: ");
int portNumber = Integer.valueOf(portFromUser.readLine());
if (portNumber < 1 || portNumber > 65535) {
System.out.println("Please choose a port number between 1 and 65535.");
return;
}
portFromUser.close();
ServerSocket listenSocket = new ServerSocket(portNumber);
/*
* Finished with user input
*/
/*
* Continuously listens for clients:
*/
while (true) {
Socket clientSocket = listenSocket.accept();
BufferedReader inFromClient = new BufferedReader(new InputStreamReader(clientSocket.getInputStream()));
DataOutputStream outToClient = new DataOutputStream(clientSocket.getOutputStream());
String clientIP = clientSocket.getRemoteSocketAddress().toString();
System.out.println("The client " + clientIP + " connected!");
String clientMessage = inFromClient.readLine();
System.out.println("The client requested file: " + clientMessage);
// Get file. If doesn't exist, let's client know.
// Otherwise informs client of file size.
File myFile = new File(clientMessage);
if (!myFile.exists()) {
outToClient.writeBytes("File does not exist!\n");
return;
} else {
outToClient.writeBytes(String.valueOf((int)myFile.length()) + "\n");
}
// Create array for storage of file bytes:
byte[] byteArray = new byte[(int)myFile.length()];
BufferedInputStream bis = new BufferedInputStream(new FileInputStream(myFile));
// Read file into array:
bis.read(byteArray, 0, byteArray.length);
// Send the file:
outToClient.write(byteArray, 0, byteArray.length);
outToClient.close();
clientSocket.close();
}
}
}
Client Code:
import java.io.*;
import java.net.*;
class ft_client {
public static void main(String args[]) throws Exception {
int byteSize = 2022386;
int bytesRead;
/*
* Asks user for IP and port:
*/
BufferedReader inFromUser = new BufferedReader(new InputStreamReader(System.in));
System.out.println("Enter an IP address: ");
String ipAddress = inFromUser.readLine();
System.out.println("Enter a port: ");
String port = inFromUser.readLine();
Socket clientSocket;
try {
// Makes socket, port, and calls connect. Assumes it's TCP:
clientSocket = new Socket(ipAddress, Integer.valueOf(port));
} catch (Exception e) {
System.out.println(e.getMessage());
return;
}
// Creates InputStream from server to get file size and other messages:
BufferedReader inFromServer = new BufferedReader(new InputStreamReader(clientSocket.getInputStream()));
// Anything written to this will be sent to the server:
DataOutputStream outToServer = new DataOutputStream(clientSocket.getOutputStream());
// Asks for a file name to download from the server:
System.out.println("What file do you want?: ");
String message = inFromUser.readLine();
outToServer.writeBytes(message + "\n");
inFromUser.close();
// Listens for confirmation from server.
// If the file exists, the file size is delivered here:
String response = inFromServer.readLine();
System.out.println("File size: " + response);
if (response.equals("File does not exist!")) {
return;
}
// Receives file from server:
byteSize = (int) Integer.valueOf(response);
byte[] byteArray = new byte[byteSize];
InputStream is = clientSocket.getInputStream(); // calling clientSocket.getInputStream() twice???
FileOutputStream fos = new FileOutputStream(message);
BufferedOutputStream bos = new BufferedOutputStream(fos);
// Continuously writes the file to the disk until complete:
int total = 0;
while ((bytesRead = is.read(byteArray)) != -1) {
bos.write(byteArray, 0, bytesRead);
total += bytesRead;
}
bos.close();
System.out.println("File downloaded (" + total + " bytes read)");
clientSocket.close();
}
}
Are buffered readers interfering with output streams? Is there a better way to transfer files?
It's worth checking, in your server code, what value comes back from the file read() call, so:
int bytesRead = bis.read(byteArray, 0, byteArray.length);
System.out.println("File bytes read: " + bytesRead + " from file size: " + myFile.length());
The read() method is under no obligation to fill the byteArray - only to return something and to tell you how many bytes it read. From the docs, it:
Reads up to len bytes of data from this input stream into an array of
bytes. If len is not zero, the method blocks until some input is
available; otherwise, no bytes are read and 0 is returned.
You need to keep reading in a loop. I'd do this (actually, same as your client!):
int n;
while ((n = bis.read(byteArray, 0, byteArray.length)) != -1) {
// Send the chunk of n bytes
outToClient.write(byteArray, 0, n);
}
bis.close();
outToClient.close();
or something similar. I've closed the file too: it'd close on GC/finalize, but that could be a while, and meanwhile you're holding the file open.
EDIT
The specific problem with your image-read in this case is in your client code. You read the file size near the top of the code:
// Creates InputStream from server to get file size and other messages:
BufferedReader inFromServer = new BufferedReader(new InputStreamReader(clientSocket.getInputStream()));
and then you access the client again:
InputStream is = clientSocket.getInputStream(); // calling clientSocket.getInputStream() twice???
and as your comment suggests, this is bad! Thank you to #EJP for highlighting this!
This causes a problem of buffer over-ingestion: the BufferedReader consumes more bytes into its belly than you extract from it, so when you visit the clientSocket inputstream the second time, the read-pointer has moved on. You never look again at what the BufferedReader consumed.
As a general rule, once you plug buffering code onto something, you must be careful to read only from that buffer. In this case, it's difficult, because you can't read image (raw binary) data from a Reader, because it will busily interpret the binary values as characters and read them as UTF-8 or something.
Even without buffers, it's a minor sin to mix Readers (text oriented) and binary data (DataStreams) on the same stream. HTTP and email does this, so you are in good company, but they get away with it by being very tightly specified. Problem is, you can easily get snarled with questions of local/default character encoding at each end, whether you're reading Unix "LF" vs Windows "CR/LF" line endings etc.
In this case, try not using BufferedReaders at all, and try using DataInput/Output streams all the way. Try writeUTF(s) and readUTF() for transferring the String data. Ideally, create them like this:
DataInputStream inFromServer = new DataInputStream (new BufferedInputStream(clientSocket.getInputStream()));
so you still get the benefits of buffering.
EDIT 2
So seeing the new client code:
byteSize = (int) Integer.valueOf(response);
byte[] byteArray = new byte[byteSize];
FileOutputStream fos = new FileOutputStream(message);
int readBytes = inFromServer.read(byteArray);
// Continuously writes the file to the disk until complete:
int total = 0;
for (int i=0; i<byteArray.length; i++) {
fos.write(byteArray[i]);
total++;
}
fos.close();
Here, we're assuming that because the byteArray array is set to the right size, that the inFromServer.read(byteArray) will populate it - it won't. It's good to assume that any and all read operations will return you just as much data as the system has to hand: in this case, it's probably going to return as soon as it gets the first packet or two, with an underfilled array. This is same as C and Unix read behaviour too.
Try this - I'm repeatedly reading and writing a 4K buffer, until the byte count is reached (as determined by summing the return values of the reads):
byteSize = (int) Integer.valueOf(response);
byte[] byteArray = new byte[4096];
FileOutputStream fos = new FileOutputStream(message);
int total = 0;
// Continuously writes the file to the disk until complete:
while (total < byteSize && (readBytes = inFromServer.read(byteArray)) != -1) {
fos.write(byteArray, 0, readBytes);
total += readBytes;
}
fos.close();
A variant is this - same thing, but byte at a time. Might be a bit clearer. It's going to be slow - all those reads and writes are hitting the OS, but if you put a BufferedInputStream/BufferedOutputStream around the socket/file streams, it'll iron that out. I've added them:
DataInputStream inFromServer =
new DataInputStream(new BufferedInputStream(clientSocket.getInputStream()));
...
byteSize = (int) Integer.valueOf(response);
OutputStream fos = new BufferedOutputStream(FileOutputStream(message));
int total = 0;
int ch;
// Continuously writes the file to the disk until complete:
while (total < byteSize && (ch = inFromServer.read()) != -1) {
fos.write(ch);
total ++;
}
fos.close();
And finally! the simplest answer is this. Your code, but changed to:
int readBytes = inFromServer.readFully(byteArray);
Yes! Those nice people in 1990's Javasoft added a DataInput.readFully method, which does what you want! - basically wraps the code above. It's the simplest solution, and arguably most correct approach: "use existing libraries where possible". OTOH, it's the least educational, and the time you spend getting used to read/writes like this is not deducted from your life-expectancy!
And in fact, the readFully approach has severe limitations. Try pointing it at a 1GB file and see what happens (after you've fixed up the array size at the top): you'll a) run out memory, and b) wish that while you were ingesting a huge blob, you could at least be spooling it out to disk. If you try a 2.5G file, you'll notice that some of those ints should become longs to cope with numbers >= 2^31.
If it was me, I'd do the 4K buffer one. (BTW I'm writing this on a laptop with no Java compiler installed, so I haven't actually run the above! DO respond if there are any difficulties.)
I'm doing simple Client-Server applications which copying file from client to server and vice versa. I'm using Sockets of course. Apps shows Client menu with some options to choose: 1. Make Copy on sever 2. Get fileCopy from Server etc.
The issue is when I'm choosing first option, I can't do second one. I read about this exception, but i have no idea how to solve this problem. I'm looking forward for your ideas.
There is part of clientside code:
public Client(String host, int port) {
try {
s = new Socket(host, port);
System.out.println("Witaj w programie");
boolean finished = false;
Scanner sc = new Scanner(System.in);
while(!finished){
System.out.println("\n\n1.Zrob kopie zapasowa pliku");
System.out.println("2. Przywroc kopie");
System.out.println("0.Zakoncz");
char c = sc.nextLine().charAt(0);
switch(c){
case '1':
this.sendMessage(1);
makeCopy(s);
//s.close();
break; ...
sendMessage method code:
public void sendMessage(int message_id) throws IOException{
oos = new ObjectOutputStream(s.getOutputStream());
ois = new ObjectInputStream(s.getInputStream());
oos.writeInt(message_id);
oos.flush();
}
and makeCopy method code:
private void makeCopy(Socket clientSock) throws IOException {
File file = new File("D:\\klient\\doKopii.bmp");
DataOutputStream dos = new DataOutputStream(clientSock.getOutputStream());
FileInputStream fis = new FileInputStream(file);
byte[] buffer = new byte[4096];
while (fis.read(buffer) > 0) {
dos.write(buffer);
}
fis.close();
dos.close();
}
Download copy from Server code:
private void saveFile(Socket clientSock) throws IOException {
//DataInputStream dis = new DataInputStream(clientSock.getInputStream());
FileOutputStream fos = new FileOutputStream("D:\\klient\\przywroconaKopia.bmp");
File zSerwera = new File("D:\\serwer\\kopiaPliku.bmp");
byte[] buffer = new byte[4096];
int filesize = (int)zSerwera.length();
int read = 0;
int totalRead = 0;
int remaining = filesize;
while((read = ois.read(buffer, 0, Math.min(buffer.length, remaining))) > 0) {
totalRead += read;
remaining -= read;
System.out.println("read " + totalRead + " bytes.");
fos.write(buffer, 0, read);
}
//fos.close();
//ois.close();
}
I am aware of that this does not work because of DataOutputStream closing which means socket is also closed. I deleted this line, but after choosing one option then second (when first one has done), application just freezes.
oos = new ObjectOutputStream(s.getOutputStream());
ois = new ObjectInputStream(s.getInputStream());
public void sendMessage(int message_id) throws IOException{
oos.writeInt(message_id);
}
make oos and ois as instance variables and make sure that they are instantiated only for one single time.
why you have to get a dataoutput stream seperately you can send byte array in same objectoutputstream
private void makeCopy(Socket clientSock) throws IOException {
File file = new File("D:\\klient\\doKopii.bmp");
FileInputStream fis = new FileInputStream(file);
byte[] buffer = new byte[4096];
while (fis.read(buffer) > 0) {
oos.write(buffer);
}
}
this is not very good coding practise bt maybe u should try this approach
There are several problems here.
You need to send the file length ahead of the file, and read exactly that many bytes at the receiver, as shown in this answer.
Don't mix stream types, and don't keep creating new streams. Use the same ObjectInputStream/ObjectOutputStream pair for the life of the socket, at both ends.
I am trying to send pdf files over to clients using tcp connections. Somehow, the file is corrupted. I wonder if tcp support non-txt files?
this is a simple server program
import java.io.*;
import java.net.*;
class server{
public static void main (String args[]) throws Exception
{
Socket s = null;
int firsttime=1;
while(true) {
ServerSocket s_sock = new ServerSocket(3248);
Socket socket = s_sock.accept();
BufferedReader bf = new BufferedReader(new InputStreamReader(socket.getInputStream()));
DataOutputStream dos = new DataOutputStream(socket.getOutputStream());
File myFile = new File ("/Users/wchen48/Desktop/123.pdf");
byte [] mybyte = new byte[(int)myFile.length()];
OutputStream os = socket.getOutputStream();
System.out.println("sending......" + mybyte.length + "Byte");
os.write(mybyte,0,mybyte.length);
os.flush();
socket.close();
s_sock.close();
}
} //end of main
This is the client:
import java.io.BufferedOutputStream;
import java.io.FileOutputStream;
import java.io.InputStream;
import java.net.Socket;
public class client{
public static void main(String[] argv) throws Exception {
Socket sock = new Socket("127.0.0.1", 3248);
byte[] mybytearray = new byte[11536];
InputStream is = sock.getInputStream();
FileOutputStream fos = new
FileOutputStream("/Users/wchen48/Documents/123.pdf");
bos = new BufferedOutputStream(fos);
int bytesRead = is.read(mybytearray, 0, mybytearray.length);
bos.write(mybytearray, 0, mybytearray.length);
bos.close();
sock.close();
}
}
Thank you in advance.
What Zielu and Juned said, plus:
You need loops at both server and client.
At the server, choose a reasonable-size buffer and loop around. Try to read [buffer length] bytes into it from the file. Then USE THE RETURNED VALUE FROM THE READ CALL to both check for errors, (result<0), or end-of-file, (result 0), or use the value in the network write call as the number of bytes to write. DO NOT use the buffer length - it may not be full. When you get 0, close the file and socket, else loop back and read some more.
At the client, choose a reasonable-size buffer and loop around. Try to read [buffer length] bytes into it from the socket. Then USE THE RETURNED VALUE FROM THE READ CALL to both check for errors, (result<0), or socket close, (result 0), or use the value in the file write call as the number of bytes to write. DO NOT use the buffer length - it may not be full. When you get 0, exit the loop and close the file, else loop back and read some more.
you dont read content of your file into the buffer mybyte you later send through socket, so you send 00000....
The most likely reason for getting the corrupted file is the fixed number of bytes(11536) that you are trying to read. You should try to read all the bytes received on the socket rather than fixed number of bytes.
I am trying to write a program to transfer a file between client and server using java tcp sockets I am using buffer size of 64K but The problem I am facing is that when when the tcp sometimes fail to send the whole 64K it sends the remaing part for example 32K in anther go
There for A garbage data of some Spaces or so is being taken by the buffer at reading side to make 64K complete and thus unnecessary data is making the file useless at receiving side.
Is there any solution to overcome this problem ???
I am using TCP protocol this code is using to send data to client
Server-side code
File transferFile = new File ("Document.txt");
byte [] bytearray = new byte [1024];
int byRead=0;
FileInputStream fin = new FileInputStream(transferFile);
BufferedInputStream bin = new BufferedInputStream(fin);
OutputStream os = socket.getOutputStream();
while(byRead>-1) {
byRead=bin.read(bytearray,0,bytearray.length);
os.write(bytearray,0,bytearray.length);
os.flush();
}
Client-side code
byte [] bytearray = new byte [1024];
InputStream is = socket.getInputStream();
FileOutputStream fos = new FileOutputStream("C:\\Users\\NetBeansProjects\\"+filename);
BufferedOutputStream bos = new BufferedOutputStream(fos);
bytesRead = is.read(bytearray,0,bytearray.length);
currentTot = bytesRead; System.out.println("Data is being read ...");
do {
bytesRead = is.read(bytearray, 0, (bytearray.length));
if(bytesRead == 0) continue;
if(bytesRead >= 0) currentTot += bytesRead;
bos.write(bytearray,0,bytearray.length);
} while(bytesRead > -1);
here I tried to skip the loop if the byte is empty by continue; statement but it is not
working.
bos.write(bytearray,0,bytearray.length);
This should be
bos.write(bytearray,0,bytesRead);
The region after 'bytesRead' in the buffer is undisturbed by the read. It isn't 'garbage'. It's just whatever was there before.
use CLIENT Side Code as below to get the total write bytes without garbage
int availableByte = socket.available();
if (availableByte > 0) {
byte[] buffer = new byte[availableByte];
int bytesRead = socketInputStream.read(buffer);
FileOutputStream fileOutputStream = new FileOutputStream(FilePath, true);
OutputStreamWriter outputStreamWriter = new OutputStreamWriter(fileOutputStream);
BufferedWriter bufferedWriter = new BufferedWriter(outputStreamWriter);
bufferedWriter.write(buffer.toString());
bufferedWriter.close();
}
For the second time I have this extremely anoying problem with an InputStream.
This InputStream belongs to a Socket that is supposed to receive an image. The code for reading this image is as below:
InputStream input = socket.getInputStream();
InputStreamReader reader = new InputStreamReader(input);
BufferedReader bufferedReader = new BufferedReader(reader);
int total = Integer.parseInt(bufferedReader.readLine());
int bytesRead = 0;
byte[] buffer = new byte[total]; // total is the total size of the image
while (bytesRead < total) {
int next = input.read(buffer, bytesRead, total-bytesRead);
if (next > 0) {
bytesRead += next;
System.out.println("Read: " + bytesRead);
}
}
Now the strange thing is that this code skips the first 1182 bytes of the image, and then reads the remaining part. So when the total size is 15000 bytes, it reads byte 1182-15000.
I checked Wireshark and the whole image is transmitted. The code throws no exceptions. input.read() returns -1 as usual.
Pervious data has been readed from the stream using a BufferedReader. This data is only 5 characters long so it can't contain the missing 1K, but my guess is that the BufferedReader.readLine() method reads (buffers) more bytes from the InputStream than needed. Could this be correct?
I've had the same problem a few months ago but I absolutely have no clue on how I solved it.
Hope anyone can help.
Thanks in advance.
EDIT: I can solve the problem by adding a 100 ms sleep between sending the image size and the image data. It solves the problem but I would still realy like to know a more appropriate solution
As the name **Buffererd**Reader indicates it will snarf more bytes than just the first line from the underlying reader and hence also from the stream. Otherwise if would not be called "buffered".
Unfortunately I'm not aware of any non-deprecated class in Java which allows mixing of binary and textual data in the way you want.
I suggest, that you modify your protocol and transfer the length of the image also in some binary encoding. Then you can stick to InputStream.
My guess is the BufferedReader will assume that whatever reading operations you're performing afterwards will go through it, so it will happily consume input in increments of its buffer size.
One thing you could do is use a BufferedInputStream on top of input, instantiate a DataInputStream on top of that to do the readLine(), and then use the BufferedInputStream in your loop. The documentation says readLine is deprecated because it doesn't convert bytes to characters properly, but I'm hoping that with your first line containing only decimal digits, it shouldn't run into that problem.
I've written a short test, I hope it covers your use case:
import java.net.*;
import java.io.*;
class test{
static byte[] b;
static {
b = new byte[123456];
java.util.Random r = new java.util.Random();
for (int i=0;i<b.length;i++)
b[i] = (byte)(r.nextInt());
}
static void client() throws Exception{
Socket socket = new Socket("127.0.0.1", 9000);
InputStream input = socket.getInputStream();
BufferedInputStream bis = new BufferedInputStream(input);
//read in length
DataInputStream dais = new DataInputStream(bis);
int total = Integer.parseInt(dais.readLine());
System.out.println("Total: "+total);
int bytesRead = 0;
byte[] buffer = new byte[total];
while (bytesRead < total) {
int next = bis.read(buffer, bytesRead, total-bytesRead);
if (next > 0) {
bytesRead += next;
System.out.println("Read: " + bytesRead);
}
}
for (int i=0;i<buffer.length;i++)
if (buffer[i]!=b[i]){
System.err.println("Error");
System.exit(1);
}
System.out.println("OK");
bis.close();
socket.close();
}
static void server() throws Exception{
ServerSocket srv = new ServerSocket(9000);
Socket sock = srv.accept();
OutputStream os = sock.getOutputStream();
BufferedOutputStream bos =new BufferedOutputStream(os);
DataOutputStream daos = new DataOutputStream(bos);
//we're sending the b buffer
//send the length in plain text, followed by newline
byte[]num = String.valueOf(b.length).getBytes();
daos.write(num,0,num.length);
daos.write(10);
//send actual buffer contents
bos.write(b, 0, b.length);
os.close();
srv.close();
sock.close();
}
public static void main(String[]args){
new Thread(new Runnable(){
public void run(){
try{server();}catch(Exception e){e.printStackTrace();}
}
}).start();
new Thread(new Runnable(){
public void run(){
try{client();}catch(Exception e){e.printStackTrace();}
}}).start();
}
}
Have you considered just sending the image data (without the preceding image size) and using an external library such as Apache Commons IO to handle this for you? In particular, I think that you will find IOUtils.toByteArray(InputStream input) interesting, e.g.
InputStream input = socket.getInputStream();
byte[] buffer = IOUtils.toByteArray(input);