Develop Reference

Why socket send only 1KB (425B) of data?

I am learning sockets and now I want to write file transfer program. I have server part and client part. Server part contains 2 ports: 5000 (commands) and 5001 (files). Now I want to send a file via socket and when I did something is wrong because only 425B of data is sending.
Here is client send method:
private void sendFile(Socket socket) {
File file2 = new File("C:\\Users\\barte\\Desktop\\dos.png");
byte[] bytes = new byte[16 * 1024];
System.out.println(file2.exists());
try (InputStream inputStream = new FileInputStream(file2);
OutputStream outputStream = socket.getOutputStream();
OutputStream secondOutput = new FileOutputStream("C:\\Users\\barte\\Desktop\\received\\dos.png")) {
int count;
while ((count = inputStream.read(bytes)) > 0) {
outputStream.write(bytes, 0, count);
secondOutput.write(bytes, 0, count);
}
socket.close();
} catch (IOException e) {
e.printStackTrace();
}
}
As you can see (image below) I am writing this file also locally and everything is ok, all of 73KB of data is writed.
Now, on server side I am trying to receive this file:
case SEND: {
new Thread(() -> {
printWriter.println("Server is receiving files right now...");
try (ServerSocket serverSocket = new ServerSocket(5001)) {
while (true) {
new FilesTransfer(serverSocket.accept()).start();
}
} catch (IOException e) {
e.printStackTrace();
}
}).start();
break;
}
And inside FilesTransfer run method:
#Override
public void run() {
System.out.println("Hello there");
try {
InputStream inputStream = inSocket.getInputStream();
OutputStream outputStream = new FileOutputStream("C:\\Users\\barte\\Desktop\\received\\file");
byte[] bytes = new byte[16 * 1024];
int count;
while ((count = inputStream.read()) > 0) {
outputStream.write(bytes, 0, count);
}
outputStream.close();
inputStream.close();
inSocket.close();
} catch (IOException e) {
e.printStackTrace();
}
}
Where is a bug? Why only empty bytes are sending when locally everything it's fine?

The problem is:
while ((count = inputStream.read()) > 0) {
Your code uses InputStream.read(), which reads individual bytes (or -1 when end-of-stream). Right now, you are reading individual bytes, interpreting that as a length, and then writing that number of 0x00 bytes from bytes to the file. This stops when you read a 0x00 byte from the stream.
You need to change this to use InputStream.read(byte[]):
while ((count = inputStream.read(bytes)) != -1) {
That is, you need to pass bytes in, and check for the result being unequal to -1, not if it is greater than zero (0), although read(byte[]) will only return 0 if the passed in byte array has length zero, so that is not a real concern.

You could do it in this way:
#Override
public void run() {
System.out.println("Hello there");
try {
InputStream inputStream = inSocket.getInputStream();
OutputStream outputStream = new FileOutputStream("C:\\Users\\barte\\Desktop\\received\\file");
byte[] bytes = new byte[16 * 1024];
int byteRead= 1;
while (byteRead > -1) {
byteRead= inputStream.read();
outputStream.write(byteRead);
}
outputStream.close();
inputStream.close();
inSocket.close();
} catch (IOException e) {
e.printStackTrace();
}
}
Actually END OF FILE or EOF means -1 and you did > 0 so 0 was taken and it stopped the connection saving the file.
I also recommend to write a logic to transfer the filename as a command to the server so that the file is saved with the correct name and extension!

Java , Sending Large Files Over Socket is Consuming too much CPU cycles and is slow [duplicate]

This question already has answers here:
Java sending and receiving file (byte[]) over sockets
(6 answers)
Closed 7 years ago.
I was experimenting with JAVA and found this question online.
Java sending and receiving file (byte[]) over sockets.
Just for curiosity i played with the code in the accepted answer, and with other code i found similar to the question. I tried the accepted answer,yes it works and is very fast. But the problem is Archive files are getting corrupted. So here is other code i tried. The downfall of my experimental code is it consume CPU cycles and takes more time than accepted answer (And i have no idea why it is happening so). So here is my code. Can somebody help me to optimize and improve this code more.
Time Taken by accepted Answer = 11ms for 4 Mb file.
Time taken by my experiment= 4 seconds for same file.
Server.java
public class Server implements Runnable {
private ServerSocket serverSocket = null;
private Socket socket = null;
private ObjectInputStream inStream = null;
public Server() {
}
#Override
public void run() {
try {
serverSocket = new ServerSocket(4445);
socket = serverSocket.accept();
DataInputStream dIn = new DataInputStream(socket.getInputStream());
OutputStream os = socket.getOutputStream();
DataOutputStream outToClient = new DataOutputStream(os);
System.out.println("Connected");
File myFile = new File("lib1.zip");
long flength = myFile.length();
System.out.println("File Length"+flength);
outToClient.writeLong(flength);
FileInputStream fis;
BufferedInputStream bis;
byte[] mybytearray = new byte[8192];
fis = new FileInputStream(myFile);
bis = new BufferedInputStream(fis);
int theByte = 0;
System.out.println("Sending " + myFile.getAbsolutePath() + "(" + myFile.length() + " bytes)");
while ((theByte = bis.read()) != -1) {
outToClient.write(theByte);
// bos.flush();
}
/*int count;
BufferedOutputStream bos= new BufferedOutputStream(os);
while ((count = bis.read(mybytearray))>0) {
bos.write(mybytearray, 0, count);
}*/
bis.close();
socket.close();
} catch (SocketException se) {
System.exit(0);
} catch (IOException e) {
e.printStackTrace();
}
}
public static void main(String[] args) {
Thread t = new Thread(new Server());
t.start();
}
}
ReceiveFile.java
public class RecieveFile {
public final static int SOCKET_PORT = 4445; // you may change this
String SERVER = "127.0.0.1"; // localhost
ArrayList<String> logmsg = new ArrayList<>();
public static void main(String[] args) {
new RecieveFile();
}
public RecieveFile() {
try (Socket sock = new Socket(SERVER, SOCKET_PORT)) {
System.out.println("Connecting...");
try (OutputStream os = sock.getOutputStream(); DataOutputStream outToServer = new DataOutputStream(os)) {
try (DataInputStream dIn = new DataInputStream(sock.getInputStream())) {
long fileLen, downData;
int bufferSize = sock.getReceiveBufferSize();
long starttime = System.currentTimeMillis();
File myFIle = new File("lib1.zip");
try (FileOutputStream fos = new FileOutputStream(myFIle); BufferedOutputStream bos = new BufferedOutputStream(fos)) {
fileLen = dIn.readLong();
/*for (long j = 0; j <= fileLen; j++) {
int tempint = is.read();
bos.write(tempint);
}*/
downData = fileLen;
int n = 0;
byte[] buf = new byte[8192];
while (fileLen > 0 && ((n = dIn.read(buf, 0, buf.length)) != -1)) {
bos.write(buf, 0, n);
fileLen -= n;
// System.out.println("Remaining "+fileLen);
}
/*while ((n = dIn.read(buf)) > 0) {
bos.write(buf, 0, n);
}*/
bos.flush();
long endtime = System.currentTimeMillis();
System.out.println("File " + myFIle.getAbsolutePath()
+ " downloaded (" + downData + " bytes read) in " + (endtime - starttime) + " ms");
}
}
}
} catch (IOException ex) {
Logger.getLogger(RecieveFile.class.getName()).log(Level.SEVERE, null, ex);
}
}
}

You're copying a byte at a time. This is slow. You're also declaring a byte array but not using it. Try this:
int count;
byte[] buffer = new byte[8192]; // or more, double or quadruple it
while ((count = in.read(buffer)) > 0)
{
out.write(buffer, 0, count);
}

Your solution takes a lot of time probably because you are reading a character at time, instead of all the buffer.
The solution is to use a construct similar to the linked question; the problem you got about corrupted file is really improbable, a malformed TCP packed that pass CRC check is really rare occurrence, and I would blame a bug instead. try to post the code you used. But you can add some hash check on the file and some part of it, if you are concerned about this

Here is a cleaned up version of your code, it should perform faster as it avoids single byte operations:
public class Server implements Runnable {
#Override
public void run() {
try {
ServerSocket serverSocket = new ServerSocket(4445);
Socket socket = serverSocket.accept();
OutputStream os = socket.getOutputStream();
DataOutputStream dos = new DataOutputStream(os);
File myFile = new File("lib1.zip");
long flength = myFile.length();
dos.writeLong(flength);
InputStream fis = new FileInputStream(myFile);
byte[] buf = new byte[16*1024]; // 16K
long written = 0;
while ((count = fis.read(buf))>0) {
dos.write(buf, 0, count);
written+=count;
}
if (written != flength)
System.out.println("Warning: file changed");
dos.close();
socket.close();
} catch (IOException e) {
e.printStackTrace();
System.exit();
}
}
An possible improvement would be to use NIO with channel.sendTo() but this should already have an acceptable performance. Note you do not need to use buffered streams on reading or writing as you use a larger byte array buffer anyway.
One possible improvement would be to not use the DataOutputStream for the long but poke the 8 bytes of it into the first buffer (array) write.
BTW: writing 4MB in 11ms is 390MB/s, that would be faster than most desktop disks can read and write.

Large file transfer over java socket [duplicate]

This question already has answers here:
Java multiple file transfer over socket
(3 answers)
Closed 5 years ago.
I have written a small client-server code for transferring small file. It uses Data output stream and readFully() method of data input stream. This code does not work for larger files for obvious reasons. I was thinking of fragmenting large files into smaller chunks of 1Kb each before sending them to client. But I can't think of any solution (like how to write multiple chunks on data output stream with correct offset and how to reassemble them at receiving end. Can anyone provide a workaround? It would be very helpful if you could modify my code:
Sender (Server):
public void sendFileDOS() throws FileNotFoundException {
runOnUiThread( new Runnable() {
#Override
public void run() {
registerLog("Sending. . . Please wait. . .");
}
});
final long startTime = System.currentTimeMillis();
final File myFile= new File(filePath); //sdcard/DCIM.JPG
byte[] mybytearray = new byte[(int) myFile.length()];
FileInputStream fis = new FileInputStream(myFile);
BufferedInputStream bis = new BufferedInputStream(fis);
DataInputStream dis = new DataInputStream(bis);
try {
dis.readFully(mybytearray, 0, mybytearray.length);
OutputStream os = socket.getOutputStream();
//Sending file name and file size to the client
DataOutputStream dos = new DataOutputStream(os);
dos.writeUTF(myFile.getName());
dos.writeLong(mybytearray.length);
int i = 0;
final ProgressBar myProgBar=(ProgressBar)findViewById(R.id.progress_bar);
while (i<100) {
dos.write(mybytearray, i*(mybytearray.length/100), mybytearray.length/100);
final int c=i;
runOnUiThread( new Runnable() {
#Override
public void run() {
myProgBar.setVisibility(View.VISIBLE);
registerLog("Completed: "+c+"%");
myProgBar.setProgress(c);
if (c==99)
myProgBar.setVisibility(View.INVISIBLE);
}
});
i++;
}
dos.flush();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
runOnUiThread( new Runnable() {
#Override
public void run() {
long estimatedTime = (System.currentTimeMillis() - startTime)/1000;
registerLog("File successfully sent");
registerLog("File size: "+myFile.length()/1000+" KBytes");
registerLog("Elapsed time: "+estimatedTime+" sec. (approx)");
registerLog("Server stopped. Please restart for another session.");
final Button startServerButton=(Button)findViewById(R.id.button1);
startServerButton.setText("Restart file server");
}
});
}
Receiver (Client):
public class myFileClient {
final static String servAdd="10.141.21.145";
static String filename=null;
static Socket socket = null;
static Boolean flag=true;
/**
* #param args
*/
public static void main(String[] args) throws IOException {
// TODO Auto-generated method stub
initializeClient();
receiveDOS();
}
public static void initializeClient () throws IOException {
InetAddress serverIP=InetAddress.getByName(servAdd);
socket=new Socket(serverIP, 4444);
}
public static void receiveDOS() {
int bytesRead;
InputStream in;
int bufferSize=0;
try {
bufferSize=socket.getReceiveBufferSize();
in=socket.getInputStream();
DataInputStream clientData = new DataInputStream(in);
String fileName = clientData.readUTF();
System.out.println(fileName);
OutputStream output = new FileOutputStream("//home//evinish//Documents//Android//Received files//"+ fileName);
long size = clientData.readLong();
byte[] buffer = new byte[bufferSize];
while (size > 0
&& (bytesRead = clientData.read(buffer, 0,
(int) Math.min(buffer.length, size))) != -1) {
output.write(buffer, 0, bytesRead);
size -= bytesRead;
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
Please help! Thanks in advance! :)

You're right, this is a poor way to do it. It wastes both memory and time; it assumes the file size is 32 bits; it assumes the entire file fits into memory; it assumes the entire file is read in one read; and it doesn't send anything until the entire file has been read.
The canonical way to copy a stream in Java is this:
while ((count = in.read(buffer)) > 0)
{
out.write(buffer, 0, count);
}
It will work with any size buffer you like and therefore with any size file you can come up with. Use the same code at both ends, although you don't have to use the same size buffer at both ends. As you're copying over a network you might think that 1k or 1.5k is the best size, but that overlooks the presence of the socket send and receive buffers in the kernel. When you take them into account it is probably better to use 8k or more.

I finally solved the problem. Here is my modified source code for server and client. Hope this would help other people too! :)
Server Side code snippet (sender):
final File myFile= new File(filePath); //sdcard/DCIM.JPG
byte[] mybytearray = new byte[8192];
FileInputStream fis = new FileInputStream(myFile);
BufferedInputStream bis = new BufferedInputStream(fis);
DataInputStream dis = new DataInputStream(bis);
OutputStream os;
try {
os = socket.getOutputStream();
DataOutputStream dos = new DataOutputStream(os);
dos.writeUTF(myFile.getName());
dos.writeLong(mybytearray.length);
int read;
while((read = dis.read(mybytearray)) != -1){
dos.write(mybytearray, 0, read);
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
Client side code snippet (Receiver):
int bytesRead;
InputStream in;
int bufferSize=0;
try {
bufferSize=socket.getReceiveBufferSize();
in=socket.getInputStream();
DataInputStream clientData = new DataInputStream(in);
String fileName = clientData.readUTF();
System.out.println(fileName);
OutputStream output = new FileOutputStream("//home//evinish//Documents//Android//Received files//"+ fileName);
byte[] buffer = new byte[bufferSize];
int read;
while((read = clientData.read(buffer)) != -1){
output.write(buffer, 0, read);
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
A bit faster way of writing to output stream:
long acc=0;
long N=myFile.length();
while(acc<N){
noofbytes=dis.read(mybytearray, 0, 16384);
dos.write(mybytearray, 0, noofbytes);
acc=acc+noofbytes; } dos.flush();
I saved around 7 seconds while transferring a video file of 72MB.

Why Client B didn't get complete file

I am trying to build a file transfer mechanism between 2 Java socket client. The sender client would include this sorta snippet:
FileInputStream fis = null;
BufferedInputStream bis = null;
BufferedOutputStream outStream = null;
byte[] fileBytes = new byte[(int) file.length()];
int bytesRead = 0;
try {
fis = new FileInputStream(file);
bis = new BufferedInputStream(fis);
outStream = new BufferedOutputStream(socket.getOutputStream());
bytesRead = bis.read(fileBytes, 0, fileBytes.length);
outStream.write(fileBytes, 0, fileBytes.length);
} catch (IOException _IOExc) {
Logger.getLogger(ChatClient.class.getName()).log(Level.SEVERE,
null, _IOExc);
//QuitConnection(QUIT_TYPE_DEFAULT);
}
The server mediator would look like:
public void run() {
assert (outSocket != null);
byte[] bytes = new byte[fileSize];
try {
System.out.println("inStream " + inStream.available());
outStream = new BufferedOutputStream(outSocket.getOutputStream());
inStream.read(bytes, 0, fileSize);
outStream.write(bytes, 0, fileSize);
outStream.flush();
} catch (IOException ex) {
Logger.getLogger(FileTransport.class.getName()).log(Level.SEVERE,
null, ex);
}
}
the destination client:
public void run() {
try {
System.out.println("Start reading...");
int len = 1024;
BufferedInputStream inStream = new BufferedInputStream
(client.user.getClientSocket().getInputStream());
while ((bytesRead = inStream.read(fileBytes, 0, len)) >
0 && current < fileSize) {
current = current + bytesRead;
System.out.println("current "+ current);
bos.write(fileBytes, 0, bytesRead < len ? bytesRead : len);
}
bos.flush();
bos.close();
} catch (IOException ex) {
Logger.getLogger(ReadFileThread.class.getName()).log(Level.SEVERE,
null, ex);
} catch (InterruptedException e) {
}
}
Both the server and destination client is passed "fileSize" in advance, the problem now is server side get slight less data and the clientB keep reading only 8192 bytes of data from server and can never get out the loop.
Many thanks
Kev

Don't ignore the result of the read() method. It returns the number of bytes that have been read, which is not necessarily the length of the file. read() must always be called in a loop, until it returns -1.
And don't, ever, use available(). It doesn't return what you think it returns. Just loop until read() returns -1 or until the number of read bytes reaches the expected count.
Read the IO tutorial.

Java deadlock problem

I am using java sockets for communication. On the client side I have some processing and at this point I send an object to the cient. The code is as follows:
while (true) {
try {
Socket server = new Socket("localhost", 3000);
OutputStream os = server.getOutputStream();
InputStream is = server.getInputStream();
CommMessage commMessage = new CommMessageImpl();
ByteArrayOutputStream bos = new ByteArrayOutputStream();
ObjectOutputStream oos = new ObjectOutputStream(bos);
oos.writeObject(commMessage);
os.write(bos.toByteArray());
os.flush();
byte[] buff = new byte[512];
int bytesRead = 0;
ByteArrayOutputStream receivedObject = new ByteArrayOutputStream();
while ((bytesRead = is.read(buff)) > -1) {
receivedObject.write(buff, 0, bytesRead);
System.out.println(receivedObject);
}
os.close();
Thread.sleep(10000);
} catch (IOException e) {
} catch (InterruptedException e) {
}
}
Next on the server side I have the following code to read the object and write the response (Which is just an echo message)
public void startServer() {
Socket client = null;
try {
server = new ServerSocket(3000);
logger.log(Level.INFO, "Waiting for connections.");
client = server.accept();
logger.log(Level.INFO, "Accepted a connection from: " + client.getInetAddress());
os = new ObjectOutputStream(client.getOutputStream());
is = new ObjectInputStream(client.getInputStream());
// Read contents of the stream and store it into a byte array.
byte[] buff = new byte[512];
int bytesRead = 0;
ByteArrayOutputStream receivedObject = new ByteArrayOutputStream();
while ((bytesRead = is.read(buff)) > -1) {
receivedObject.write(buff, 0, bytesRead);
}
// Check if received stream is CommMessage or not contents.
CommMessage commMessage = getCommMessage(receivedObject);
if (commMessage != null) {
commMessage.setSessionState(this.sessionManager.getState().getState());
ByteArrayOutputStream bos = new ByteArrayOutputStream();
ObjectOutputStream oos = new ObjectOutputStream(bos);
oos.writeObject(commMessage);
os.write(bos.toByteArray());
System.out.println(commMessage.getCommMessageType());
} else {
processData(receivedObject, this.sessionManager);
}
os.flush();
} catch (IOException e) {
} finally {
try {
is.close();
os.close();
client.close();
server.close();
} catch (IOException e) {
}
}
}
The above code works ok if I do not try to read data on the client side (If i exclude the code related to reading). But if I have that code, for some reason, I get some kind of deadlock when accessing input streams. Any ideas what I might have done wrong? Thanks in advance.

Both client and server are attempting to read the whole input stream (i.e. everything up to EOF) but neither is sending an EOF (by calling shutdownOutput() on the socket.)
Why do you need to store the object data temporarily in a ByteArrayOutputStream? This would probably be easier to fix if you read directly from the socket input stream.