I have two problems with my java server-client file communication,
I have the CLIENT sends files over to the server and the SERVER receives the files.
My 2 issues are:
1) whenever i send a file, it is 8 bytes less (i do not know why)
2) the file transfer is not complete (with 8 bytes less) unless i close the socket, which i do not want. i want my connection to be persistent, so how can i send a EOF from the client to the server.
here is my client who sends files
public void sendFiles(String file) {
try {
File myFile = new File(file);
long length = myFile.length();
byte[] buffer = new byte[8192];
System.out.println(length);
FileInputStream in = new FileInputStream(myFile);
BufferedInputStream bis = new BufferedInputStream(in);
BufferedOutputStream outF = new BufferedOutputStream(sock.getOutputStream());
out.print("%SF%" + length + "$" + myFile.getName() + "#");
out.flush();
int count;
while ((count = in.read(buffer)) > 0) {
outF.write(buffer, 0, count);
}
outF.flush();
in.close();
bis.close();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
SERVER who receives files.
I'm passing the name and the length of the file but only using the name of the file. however, i don't know if i need to use the length of the file or not, in case of EOF or something. Please advice
Also, The code hangs in
while ((count = this.sock.getInputStream().read(buffer)) > 0) {
due to no EOF which i do not know how to implement
public void recvFile(String fileName, int length) {
try {
byte[] buffer = new byte[8192];
FileOutputStream outF = new FileOutputStream("/Users/Documents" +fileName);
BufferedOutputStream bos = new BufferedOutputStream(outF);
int count = length;
while ((count = this.sock.getInputStream().read(buffer)) > 0) {
bos.write(buffer, 0, count);
}
bos.close();
} catch (IOException ex) {
ex.printStackTrace();
}
}
UPDATE: I have removed the flush() as advised that it is not needed. Also, i have tested this code in a different class and it worked but it doesn't work here with client-server chat. Could anyone tell me why?
Any help or hints would be appreciated.
Thank you.
I would suggest to you send the file size first and/or properties of the file... You can try HTTP which is wide use for this task...
Another suggestion would be for you to open another connection on other TCP port just to send the file (this is actually how FTP sends files)
I suspect the problem you have is in code you haven't shown.
In this example you can send multiple messages or files over the same stream.
import java.io.*;
import java.net.ServerSocket;
import java.net.Socket;
import java.nio.channels.SocketChannel;
/**
* Created by peter on 1/25/15.
*/
public class DataSocket implements Closeable {
private final Socket socket;
private final DataOutputStream out;
private final DataInputStream in;
public DataSocket(Socket socket) throws IOException {
this.socket = socket;
this.out = new DataOutputStream(new BufferedOutputStream(socket.getOutputStream()));
this.in = new DataInputStream(new BufferedInputStream(socket.getInputStream()));
}
#Override
public void close() throws IOException {
out.flush();
socket.close();
}
// message format is length as UTF-8 encoded name, 32-bit int followed by data.
public void writeMessage(String description, byte[] bytes) throws IOException {
out.writeUTF(description);
out.writeInt(bytes.length);
out.write(bytes);
out.flush();
}
public byte[] readMessage(String[] description) throws IOException {
description[0] = in.readUTF();
int length = in.readInt();
byte[] bytes = new byte[length];
in.readFully(bytes);
return bytes;
}
public void writeFile(File file) throws IOException {
long length = file.length();
if (length > Integer.MAX_VALUE) throw new IllegalArgumentException("length=" + length);
out.writeUTF(file.toString());
out.writeInt((int) length);
byte[] buffer = new byte[(int) Math.min(length, 32 * 1024)];
try (FileInputStream fis = new FileInputStream(file)) {
for (int len; (len = fis.read(buffer)) > 0; ) {
out.write(buffer, 0, len);
}
}
out.flush();
}
public void readFile(File dir) throws IOException {
String fileName = in.readUTF();
int length = in.readInt();
byte[] buffer = new byte[(int) Math.min(length, 32 * 1024)];
try (FileOutputStream fos = new FileOutputStream(new File(dir, fileName))) {
while (length > 0) {
int len = in.read(buffer);
fos.write(buffer, 0, len);
length -= len;
}
}
}
// todo convert to a unit test
public static void main(String[] args) throws IOException {
// port 0 opens on a random free port.
ServerSocket sc = new ServerSocket(0);
DataSocket ds1 = new DataSocket(new Socket("localhost", sc.getLocalPort()));
DataSocket ds2 = new DataSocket(sc.accept());
sc.close();
// now ds1 and ds2 are connected.
File f = File.createTempFile("deleteme","");
f.deleteOnExit();
try (FileOutputStream fos = new FileOutputStream(f)) {
fos.write(new byte[10001]);
}
// send a request
ds1.writeMessage("Send me the file", new byte[0]);
String[] desc = { null };
byte[] data = ds2.readMessage(desc);
if (!desc[0].equals("Send me the file")) throw new AssertionError();
// return a response
ds2.writeFile(f);
f.delete();
if (f.exists()) throw new AssertionError();
ds1.readFile(new File(""));
if (f.length() != 10001) throw new AssertionError("length="+f.length());
ds1.close();
ds2.close();
System.out.println("Copied a "+f.length()+" file over TCP");
}
}
Related
This question already has answers here:
Java multiple file transfer over socket
(3 answers)
Closed 5 years ago.
TL;DR: How do I send (with a single connection) file, its size and its name. All examples in the internet send a file alone.
Server:
public class Server {
private static int PORT = 6667;
private ServerSocket serverSocket;
public void run() throws IOException {
System.out.println("Opening server");
serverSocket = new ServerSocket(PORT);
while(true) {
try(Socket incomingSocket = serverSocket.accept()) {
System.out.println("Accepted connection: " + incomingSocket);
incomingSocket.setSoTimeout(2000); // Don't let scanner block the thread.
InputStream inputStream = incomingSocket.getInputStream();
Scanner scanner = new Scanner(inputStream);
String command = "";
if(scanner.hasNextLine())
command = scanner.nextLine();
if(command.equals("update")) {
File file = new File("abc.txt");
sendFile(incomingSocket, file);
}
else {
// ...
System.out.println("Another command");
}
}
}
}
private void sendFile(Socket socket, File file) throws IOException {
byte[] bytes = new byte[(int)file.length()];
FileInputStream fileInputStream = new FileInputStream(file);
BufferedInputStream bufferedInputStream = new BufferedInputStream(fileInputStream);
bufferedInputStream.read(bytes, 0, bytes.length);
OutputStream outputStream = socket.getOutputStream();
PrintWriter writer = new PrintWriter(outputStream, true);
writer.println(file.length());
writer.println(file.getName());
System.out.println("Sending " + file.getName() + "(" + bytes.length + " bytes) to " + socket);
outputStream.write(bytes, 0, bytes.length);
outputStream.flush();
System.out.println("File sent");
}
public void stopRunning() {
try {
serverSocket.close();
} catch(IOException e) {
e.printStackTrace();
}
}
}
Client:
public class Client {
private static String HOST = "localhost";
private static int PORT = 6667;
public void run() throws IOException {
Socket socket = new Socket(HOST, PORT);
System.out.println("Connecting...");
OutputStream outputStream = socket.getOutputStream();
PrintWriter writer = new PrintWriter(outputStream, true);
writer.println("update"); // Example command which will determine what server sends back
receiveFile(socket);
socket.close();
}
private void receiveFile(Socket socket) throws IOException {
InputStream inputStream = socket.getInputStream();
int size = 16384;
String name = "example.txt";
Scanner scanner = new Scanner(inputStream);
size = Integer.parseInt(scanner.next());
name = scanner.next();
FileOutputStream fileOutputStream = new FileOutputStream(name);
BufferedOutputStream bufferedOutputStream = new BufferedOutputStream(fileOutputStream);
byte[] buffer = new byte[size];
int bytesRead, totalRead = 0;
while ((bytesRead = inputStream.read(buffer, 0, buffer.length)) != -1) {
totalRead += bytesRead;
bufferedOutputStream.write(buffer, 0, bytesRead);
}
bufferedOutputStream.flush();
System.out.println("File " + name + " received. " + totalRead + " bytes read");
bufferedOutputStream.close();
fileOutputStream.close();
}
I want my server to send a file to the client. It should also include the file's name and its size. Name because it's quite important and the size because I don't want to make a hardcoded buffer with a huge size.
Tried it with the code above. The client's "scanner part"
Scanner scanner = new Scanner(inputStream);
size = Integer.parseInt(scanner.next());
name = scanner.next();
works just okay, but the file is not received. inputStream.read(buffer, 0, buffer.length) never reads the remaining bytes from the stream.
If i comment out the scanner part, the bytes are read correctly(size and name information + file itself)
So, the question is, how do I send it with a single connection? Or should I make 2 separate connections, in the first one asking for size and file name and sending the file in the second one?
Scanner is good for text-based work.
One way to do what you want is using DataInputStream and DataOutputStream. Only one connection is needed:
public void send(File file, OutputStream os) throws IOException {
DataOutputStream dos = new DataOutputStream(os);
// writing name
dos.writeUTF(file.getName());
// writing length
dos.writeLong(file.length());
// writing file content
... your write loop, write to dos
dos.flush();
}
public void receive(InputStream is) throws IOException {
DataInputStream dis = new DataInputStream(is);
String fileName = dis.readUTF();
long fileSize = dis.readLong();
// reading file content
... your read loop, read from dis
}
I am writing a raw socket server (for learning purpose), which on any request, should parse the Content-Length header and should then extract bytes equal to Content-Length from the socket input stream and echo it back to the client.
I found only one class 'DataInputStream' in Java IO system that provides with the capabilities of reading both, characters and bytes. However, the method readLine() of 'DataInputStream' is deprecated which I am using in my code. How can I get rid of the deprecated readLine() method in following code? Is there any class in Java IO system that allows reading of both, characters and bytes. Code follows:
class Server {
public Server() {
}
public void run() throws IOException {
ServerSocket serverSocket = new ServerSocket(7000);
while (true) {
Socket socket = serverSocket.accept();
DataInputStream requestStream = new DataInputStream(
new BufferedInputStream(socket.getInputStream()));
byte[] responseContent = null;
int contentLength = getContentLength(requestStream);
if (contentLength == 0)
responseContent = new byte[0];
else {
int totalBytesRead = 0, bytesRead = 0;
final int bufferSize = 5120;
final byte[] buffer = new byte[bufferSize];
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
while (totalBytesRead != contentLength) {
bytesRead = requestStream.read(buffer, 0, bufferSize);
outputStream.write(buffer, 0, bytesRead);
totalBytesRead += bytesRead;
}
responseContent = outputStream.toByteArray();
}
OutputStream outputStream = socket.getOutputStream();
PrintWriter writer = new PrintWriter(outputStream);
writer.println(String.format("HTTP/1.1 %s", 200));
writer.println(String.format("Content-Length: %d", contentLength));
writer.println("");
writer.flush();
outputStream.write(responseContent);
outputStream.flush();
socket.close();
}
}
private int getContentLength(DataInputStream requestStream)
throws IOException {
int contentLength = 0;
String headerLine;
// TODO - Get rid of deprecated readLine() method
while ((headerLine = requestStream.readLine()) != null
&& headerLine.length() != 0) {
final String[] headerTokens = headerLine.split(":");
if (headerTokens[0].equalsIgnoreCase("Content-Length")) {
contentLength = Integer.valueOf(headerTokens[1].trim());
}
}
return contentLength;
}
}
This question already has answers here:
Java sending and receiving file (byte[]) over sockets
(6 answers)
Closed 7 years ago.
I was experimenting with JAVA and found this question online.
Java sending and receiving file (byte[]) over sockets.
Just for curiosity i played with the code in the accepted answer, and with other code i found similar to the question. I tried the accepted answer,yes it works and is very fast. But the problem is Archive files are getting corrupted. So here is other code i tried. The downfall of my experimental code is it consume CPU cycles and takes more time than accepted answer (And i have no idea why it is happening so). So here is my code. Can somebody help me to optimize and improve this code more.
Time Taken by accepted Answer = 11ms for 4 Mb file.
Time taken by my experiment= 4 seconds for same file.
Server.java
public class Server implements Runnable {
private ServerSocket serverSocket = null;
private Socket socket = null;
private ObjectInputStream inStream = null;
public Server() {
}
#Override
public void run() {
try {
serverSocket = new ServerSocket(4445);
socket = serverSocket.accept();
DataInputStream dIn = new DataInputStream(socket.getInputStream());
OutputStream os = socket.getOutputStream();
DataOutputStream outToClient = new DataOutputStream(os);
System.out.println("Connected");
File myFile = new File("lib1.zip");
long flength = myFile.length();
System.out.println("File Length"+flength);
outToClient.writeLong(flength);
FileInputStream fis;
BufferedInputStream bis;
byte[] mybytearray = new byte[8192];
fis = new FileInputStream(myFile);
bis = new BufferedInputStream(fis);
int theByte = 0;
System.out.println("Sending " + myFile.getAbsolutePath() + "(" + myFile.length() + " bytes)");
while ((theByte = bis.read()) != -1) {
outToClient.write(theByte);
// bos.flush();
}
/*int count;
BufferedOutputStream bos= new BufferedOutputStream(os);
while ((count = bis.read(mybytearray))>0) {
bos.write(mybytearray, 0, count);
}*/
bis.close();
socket.close();
} catch (SocketException se) {
System.exit(0);
} catch (IOException e) {
e.printStackTrace();
}
}
public static void main(String[] args) {
Thread t = new Thread(new Server());
t.start();
}
}
ReceiveFile.java
public class RecieveFile {
public final static int SOCKET_PORT = 4445; // you may change this
String SERVER = "127.0.0.1"; // localhost
ArrayList<String> logmsg = new ArrayList<>();
public static void main(String[] args) {
new RecieveFile();
}
public RecieveFile() {
try (Socket sock = new Socket(SERVER, SOCKET_PORT)) {
System.out.println("Connecting...");
try (OutputStream os = sock.getOutputStream(); DataOutputStream outToServer = new DataOutputStream(os)) {
try (DataInputStream dIn = new DataInputStream(sock.getInputStream())) {
long fileLen, downData;
int bufferSize = sock.getReceiveBufferSize();
long starttime = System.currentTimeMillis();
File myFIle = new File("lib1.zip");
try (FileOutputStream fos = new FileOutputStream(myFIle); BufferedOutputStream bos = new BufferedOutputStream(fos)) {
fileLen = dIn.readLong();
/*for (long j = 0; j <= fileLen; j++) {
int tempint = is.read();
bos.write(tempint);
}*/
downData = fileLen;
int n = 0;
byte[] buf = new byte[8192];
while (fileLen > 0 && ((n = dIn.read(buf, 0, buf.length)) != -1)) {
bos.write(buf, 0, n);
fileLen -= n;
// System.out.println("Remaining "+fileLen);
}
/*while ((n = dIn.read(buf)) > 0) {
bos.write(buf, 0, n);
}*/
bos.flush();
long endtime = System.currentTimeMillis();
System.out.println("File " + myFIle.getAbsolutePath()
+ " downloaded (" + downData + " bytes read) in " + (endtime - starttime) + " ms");
}
}
}
} catch (IOException ex) {
Logger.getLogger(RecieveFile.class.getName()).log(Level.SEVERE, null, ex);
}
}
}
You're copying a byte at a time. This is slow. You're also declaring a byte array but not using it. Try this:
int count;
byte[] buffer = new byte[8192]; // or more, double or quadruple it
while ((count = in.read(buffer)) > 0)
{
out.write(buffer, 0, count);
}
Your solution takes a lot of time probably because you are reading a character at time, instead of all the buffer.
The solution is to use a construct similar to the linked question; the problem you got about corrupted file is really improbable, a malformed TCP packed that pass CRC check is really rare occurrence, and I would blame a bug instead. try to post the code you used. But you can add some hash check on the file and some part of it, if you are concerned about this
Here is a cleaned up version of your code, it should perform faster as it avoids single byte operations:
public class Server implements Runnable {
#Override
public void run() {
try {
ServerSocket serverSocket = new ServerSocket(4445);
Socket socket = serverSocket.accept();
OutputStream os = socket.getOutputStream();
DataOutputStream dos = new DataOutputStream(os);
File myFile = new File("lib1.zip");
long flength = myFile.length();
dos.writeLong(flength);
InputStream fis = new FileInputStream(myFile);
byte[] buf = new byte[16*1024]; // 16K
long written = 0;
while ((count = fis.read(buf))>0) {
dos.write(buf, 0, count);
written+=count;
}
if (written != flength)
System.out.println("Warning: file changed");
dos.close();
socket.close();
} catch (IOException e) {
e.printStackTrace();
System.exit();
}
}
An possible improvement would be to use NIO with channel.sendTo() but this should already have an acceptable performance. Note you do not need to use buffered streams on reading or writing as you use a larger byte array buffer anyway.
One possible improvement would be to not use the DataOutputStream for the long but poke the 8 bytes of it into the first buffer (array) write.
BTW: writing 4MB in 11ms is 390MB/s, that would be faster than most desktop disks can read and write.
This question already has answers here:
Java multiple file transfer over socket
(3 answers)
Closed 5 years ago.
I have written a small client-server code for transferring small file. It uses Data output stream and readFully() method of data input stream. This code does not work for larger files for obvious reasons. I was thinking of fragmenting large files into smaller chunks of 1Kb each before sending them to client. But I can't think of any solution (like how to write multiple chunks on data output stream with correct offset and how to reassemble them at receiving end. Can anyone provide a workaround? It would be very helpful if you could modify my code:
Sender (Server):
public void sendFileDOS() throws FileNotFoundException {
runOnUiThread( new Runnable() {
#Override
public void run() {
registerLog("Sending. . . Please wait. . .");
}
});
final long startTime = System.currentTimeMillis();
final File myFile= new File(filePath); //sdcard/DCIM.JPG
byte[] mybytearray = new byte[(int) myFile.length()];
FileInputStream fis = new FileInputStream(myFile);
BufferedInputStream bis = new BufferedInputStream(fis);
DataInputStream dis = new DataInputStream(bis);
try {
dis.readFully(mybytearray, 0, mybytearray.length);
OutputStream os = socket.getOutputStream();
//Sending file name and file size to the client
DataOutputStream dos = new DataOutputStream(os);
dos.writeUTF(myFile.getName());
dos.writeLong(mybytearray.length);
int i = 0;
final ProgressBar myProgBar=(ProgressBar)findViewById(R.id.progress_bar);
while (i<100) {
dos.write(mybytearray, i*(mybytearray.length/100), mybytearray.length/100);
final int c=i;
runOnUiThread( new Runnable() {
#Override
public void run() {
myProgBar.setVisibility(View.VISIBLE);
registerLog("Completed: "+c+"%");
myProgBar.setProgress(c);
if (c==99)
myProgBar.setVisibility(View.INVISIBLE);
}
});
i++;
}
dos.flush();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
runOnUiThread( new Runnable() {
#Override
public void run() {
long estimatedTime = (System.currentTimeMillis() - startTime)/1000;
registerLog("File successfully sent");
registerLog("File size: "+myFile.length()/1000+" KBytes");
registerLog("Elapsed time: "+estimatedTime+" sec. (approx)");
registerLog("Server stopped. Please restart for another session.");
final Button startServerButton=(Button)findViewById(R.id.button1);
startServerButton.setText("Restart file server");
}
});
}
Receiver (Client):
public class myFileClient {
final static String servAdd="10.141.21.145";
static String filename=null;
static Socket socket = null;
static Boolean flag=true;
/**
* #param args
*/
public static void main(String[] args) throws IOException {
// TODO Auto-generated method stub
initializeClient();
receiveDOS();
}
public static void initializeClient () throws IOException {
InetAddress serverIP=InetAddress.getByName(servAdd);
socket=new Socket(serverIP, 4444);
}
public static void receiveDOS() {
int bytesRead;
InputStream in;
int bufferSize=0;
try {
bufferSize=socket.getReceiveBufferSize();
in=socket.getInputStream();
DataInputStream clientData = new DataInputStream(in);
String fileName = clientData.readUTF();
System.out.println(fileName);
OutputStream output = new FileOutputStream("//home//evinish//Documents//Android//Received files//"+ fileName);
long size = clientData.readLong();
byte[] buffer = new byte[bufferSize];
while (size > 0
&& (bytesRead = clientData.read(buffer, 0,
(int) Math.min(buffer.length, size))) != -1) {
output.write(buffer, 0, bytesRead);
size -= bytesRead;
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
Please help! Thanks in advance! :)
You're right, this is a poor way to do it. It wastes both memory and time; it assumes the file size is 32 bits; it assumes the entire file fits into memory; it assumes the entire file is read in one read; and it doesn't send anything until the entire file has been read.
The canonical way to copy a stream in Java is this:
while ((count = in.read(buffer)) > 0)
{
out.write(buffer, 0, count);
}
It will work with any size buffer you like and therefore with any size file you can come up with. Use the same code at both ends, although you don't have to use the same size buffer at both ends. As you're copying over a network you might think that 1k or 1.5k is the best size, but that overlooks the presence of the socket send and receive buffers in the kernel. When you take them into account it is probably better to use 8k or more.
I finally solved the problem. Here is my modified source code for server and client. Hope this would help other people too! :)
Server Side code snippet (sender):
final File myFile= new File(filePath); //sdcard/DCIM.JPG
byte[] mybytearray = new byte[8192];
FileInputStream fis = new FileInputStream(myFile);
BufferedInputStream bis = new BufferedInputStream(fis);
DataInputStream dis = new DataInputStream(bis);
OutputStream os;
try {
os = socket.getOutputStream();
DataOutputStream dos = new DataOutputStream(os);
dos.writeUTF(myFile.getName());
dos.writeLong(mybytearray.length);
int read;
while((read = dis.read(mybytearray)) != -1){
dos.write(mybytearray, 0, read);
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
Client side code snippet (Receiver):
int bytesRead;
InputStream in;
int bufferSize=0;
try {
bufferSize=socket.getReceiveBufferSize();
in=socket.getInputStream();
DataInputStream clientData = new DataInputStream(in);
String fileName = clientData.readUTF();
System.out.println(fileName);
OutputStream output = new FileOutputStream("//home//evinish//Documents//Android//Received files//"+ fileName);
byte[] buffer = new byte[bufferSize];
int read;
while((read = clientData.read(buffer)) != -1){
output.write(buffer, 0, read);
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
A bit faster way of writing to output stream:
long acc=0;
long N=myFile.length();
while(acc<N){
noofbytes=dis.read(mybytearray, 0, 16384);
dos.write(mybytearray, 0, noofbytes);
acc=acc+noofbytes; } dos.flush();
I saved around 7 seconds while transferring a video file of 72MB.
I was playing around with Java sockets and I was trying to trasnfer files from a server to client, however, when they get transfer they are corrupted. This is the code from the server:
DataInputStream input;
DataOutputStream ouput;
//these two variable are initialized somewhere else in the code.
private void downloadFile() {
try {
String fileName= input.readUTF();
File f = new File(path + fileName);
size= f.length();
file= new FileInputStream(path+ fileName);
ouput.writeLong(size);
byte[] buffer = new byte[1024];
int len;
while ((len = file.read(buffer)) > 0) {
output.write(buffer, 0, len);
}
} catch (IOException e) {
e.printStackTrace();
}
}
on the client side:
public void downloadFile(String fileName) {
try {
this.client= new Socket(ip,port);
DataInputStream input= new DataInputStream(this.client.getInputStream());
DataOutputStream ouput= new DataOutputStream(this.client.getOutputStream());
output.writeUTF("DOWNLOAD");
output.writeUTF(fileName);
File f = new File(path+ fileName);
file = new FileOutputStream(f);
byte[] buffer = new byte[1024];
int len;
while ((len = input.read(buffer)) > 0) {
file.write(buffer, 0, len);
}
file.flush();
file.close();
this.client.close();
} catch (Exception e) {
System.out.println("something went wrong");
}
}
I dont know what am I doing wrong, the file gets completely transfer but not correctly.
on the server:
ouput.writeLong(size);
you dont seem to handle this on the client side, you just append it to the downloaded file as if it was part of the binary data.
It looks like you send the length of the file from the server to the client:
ouput.writeLong(size);
but your client code never does anything with the transmitted size, so it takes up the first few bytes of the file.