I am trying to send pdf files over to clients using tcp connections. Somehow, the file is corrupted. I wonder if tcp support non-txt files?
this is a simple server program
import java.io.*;
import java.net.*;
class server{
public static void main (String args[]) throws Exception
{
Socket s = null;
int firsttime=1;
while(true) {
ServerSocket s_sock = new ServerSocket(3248);
Socket socket = s_sock.accept();
BufferedReader bf = new BufferedReader(new InputStreamReader(socket.getInputStream()));
DataOutputStream dos = new DataOutputStream(socket.getOutputStream());
File myFile = new File ("/Users/wchen48/Desktop/123.pdf");
byte [] mybyte = new byte[(int)myFile.length()];
OutputStream os = socket.getOutputStream();
System.out.println("sending......" + mybyte.length + "Byte");
os.write(mybyte,0,mybyte.length);
os.flush();
socket.close();
s_sock.close();
}
} //end of main
This is the client:
import java.io.BufferedOutputStream;
import java.io.FileOutputStream;
import java.io.InputStream;
import java.net.Socket;
public class client{
public static void main(String[] argv) throws Exception {
Socket sock = new Socket("127.0.0.1", 3248);
byte[] mybytearray = new byte[11536];
InputStream is = sock.getInputStream();
FileOutputStream fos = new
FileOutputStream("/Users/wchen48/Documents/123.pdf");
bos = new BufferedOutputStream(fos);
int bytesRead = is.read(mybytearray, 0, mybytearray.length);
bos.write(mybytearray, 0, mybytearray.length);
bos.close();
sock.close();
}
}
Thank you in advance.
What Zielu and Juned said, plus:
You need loops at both server and client.
At the server, choose a reasonable-size buffer and loop around. Try to read [buffer length] bytes into it from the file. Then USE THE RETURNED VALUE FROM THE READ CALL to both check for errors, (result<0), or end-of-file, (result 0), or use the value in the network write call as the number of bytes to write. DO NOT use the buffer length - it may not be full. When you get 0, close the file and socket, else loop back and read some more.
At the client, choose a reasonable-size buffer and loop around. Try to read [buffer length] bytes into it from the socket. Then USE THE RETURNED VALUE FROM THE READ CALL to both check for errors, (result<0), or socket close, (result 0), or use the value in the file write call as the number of bytes to write. DO NOT use the buffer length - it may not be full. When you get 0, exit the loop and close the file, else loop back and read some more.
you dont read content of your file into the buffer mybyte you later send through socket, so you send 00000....
The most likely reason for getting the corrupted file is the fixed number of bytes(11536) that you are trying to read. You should try to read all the bytes received on the socket rather than fixed number of bytes.
Related
Good day, I'm a beginner developer and I'm trying to write a client-server application. My application should work like this, it checks if the file has been modified and if so, it is sent to the server, the question is how to organize this?
I did it through the socket, but if I get a deadlock, I will be grateful for any comments, I attach the code further
Server
import java.io.*;
import java.net.ServerSocket;
import java.net.Socket;
public class Server {
public static void main(String[] args) throws IOException, InterruptedException {
ServerSocket serverSocket;
Socket client;
BufferedInputStream clientInputStream;
ByteArrayOutputStream byteArrayOutputStream;
StringBuilder fileName;
byte[] mass;
{
fileName = new StringBuilder("this.txt"); // give name for file
serverSocket = new ServerSocket(5000); // open server in 5000 port
mass = new byte[1]; // create new buffer array
client = serverSocket.accept(); // waiting connect
clientInputStream = new BufferedInputStream(client.getInputStream()); // to accept byte's array
byteArrayOutputStream = new ByteArrayOutputStream(); // to write byte array in file
}
{
BufferedOutputStream bufferedOutputStream;
int bytesRead;
// FIXME: 02.07.2022 первый файл
do {
byteArrayOutputStream.write(mass, 0, mass.length); // write by one byte in array
bytesRead = clientInputStream.read(mass, 0, mass.length);
} while (bytesRead != -1);
FileOutputStream writer = new FileOutputStream(fileName.toString()); // stream for file write
bufferedOutputStream = new BufferedOutputStream(writer);
bufferedOutputStream.write(byteArrayOutputStream.toByteArray(), 0, byteArrayOutputStream.toByteArray().length); // write file
System.out.println("first f");
bufferedOutputStream.flush();
BufferedWriter bufferedWriter = new BufferedWriter(new OutputStreamWriter(client.getOutputStream()));
bufferedWriter.write("success"); // if file send success then write string in socket and send to client
}
{
serverSocket.close();
client.close();
clientInputStream.close();
byteArrayOutputStream.close();
}
}
}
And Client
import java.io.*;
import java.net.Socket;
public class Client {
private final static String serverIP = "192.168.0.47"; // server ip address
private final static int serverPort = 5000; // server port address
private final static String fileOutput = "first.txt"; // first file to send
private final static String fileOutput1 = "second.txt"; // second file to send
public static void main(String[] args) throws IOException, InterruptedException{
Socket client = new Socket(serverIP,serverPort); // make new connect with my server
BufferedInputStream inputFile = new BufferedInputStream(new FileInputStream(fileOutput)); // convert file to array bytes
// FIXME: 02.07.2022 first file
BufferedOutputStream clientSocketOutputWriter = new BufferedOutputStream(client.getOutputStream()); // this is the stream to send to the socket
byte[] massByte = inputFile.readAllBytes(); // this going on convert file to array bytes
clientSocketOutputWriter.write(massByte,0, massByte.length); // send array bytes to server
System.out.println("first file send");
//----------------if i remove this two string, program begin work successfully but i can't send several file because this beging deadlock
// BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(client.getInputStream()));
// String s = bufferedReader.readLine();
//----------------
System.out.println("the end");
clientSocketOutputWriter.close();
client.close();
inputFile.close();
}
}
If the TCP connection isn't closed,The variable bytesRead will never be -1 because the code clientInputStream.read(mass, 0, mass.length); keeps executing and waiting for data.If you want to send a file,you should send its length first then you can call byte[] data=clientInputStream.readNBytes(its length),or put its data into a object and use ObjectInputStream and ObjectOutputStream to send the object.
The variable bytesRead isn't -1 means the server cannot leave the loop,so the server cannot put the data which it has just received into a file.Just use writer.write(mass); instead of byteArrayOutputStream.write(mass, 0, mass.length);,write the data to the file directly then stop running the program can solve the problem too.
This is a simplified version of my program.
I have a Server.java file where I read in "b.txt" into a byte array and then write it to the DataOutputStream.
import java.io.*;
import java.net.*;
class Server {
public static void main(String[] args) {
try {
ServerSocket ss = new ServerSocket(4000);
Socket s = ss.accept();
DataOutputStream dout = new DataOutputStream(s.getOutputStream());
byte[] byteArray = new byte[32];
File tempFile = new File("Files/b.txt");
FileInputStream fis = new
FileInputStream(tempFile);
fis.read(byteArray);
fis.close();
dout.write(byteArray);
dout.flush();
dout.close();
s.close();
} catch (Exception e) {
//handle exceptions
}
}
}
In Client.java I read the DataInputStream into a byteArray. Then, I create a FileOutputStream. In the commented out code fos.write(byteArray), this writes to the file, however since the actual file may not be exactly 32 bytes I am left with a corrupt file since the remaining bytes just get filled with random data (or whatever happens).
import java.io.*;
import java.net.*;
class Client {
public static void main(String[] args) {
try {
Socket s = new Socket("localhost", 4000);
DataInputStream din = new DataInputStream(s.getInputStream());
byte[] byteArray = new byte[32];
FileOutputStream fos = new FileOutputStream("Downloaded/testfile.txt");
//fos.write(byteArray);
int count;
while ((count = din.read(byteArray)) > 0) {
fos.write(byteArray, 0, count);
}
fos.close();
din.close();
s.close();
} catch (Exception e) {
//handle exceptions
}
}
}
So I looked up how to fix this and a lot of places told me to do the code I have inside the while loop: while ((count = din.read... etc.
However when I try this, testfile.txt is blank and contains 0 bytes. Am I missing something big here? I have tried sending in the correct amount of bytes from the server, however this doesn't change anything.
I would like to be able to transfer any file type, so it is important that I am able to transfer the correct amount of bytes without corrupting the data by having bytes "hanging over". Thank you.
Despite hours of researching this problem, I have made very little progress. According to my professor, the code should be working as written...
I have a server that stays open, and a client that requests a file. Once the client receives the file, the client closes.
When I open the server, I am able to transfer a complete .jpg image file. The client then closes while the server remains open. I start up another client and try to transfer the same image, and only a portion of the bytes are transferred/written to the disk. The file transfer is only completely successful for the first file transferred by the server!
Additionally strange, a simple .txt text file never successfully transfers. I believe the cause is on the server side because it remains open as opposed to the client, which starts over each time.
Server Code:
import java.io.*;
import java.net.*;
import java.util.Arrays;
class ft_server {
public static void main(String args[]) throws Exception {
/*
* Asks user for port number and listens on that port
*/
BufferedReader portFromUser = new BufferedReader(new InputStreamReader(System.in));
System.out.println("Enter the port you'd like to use: ");
int portNumber = Integer.valueOf(portFromUser.readLine());
if (portNumber < 1 || portNumber > 65535) {
System.out.println("Please choose a port number between 1 and 65535.");
return;
}
portFromUser.close();
ServerSocket listenSocket = new ServerSocket(portNumber);
/*
* Finished with user input
*/
/*
* Continuously listens for clients:
*/
while (true) {
Socket clientSocket = listenSocket.accept();
BufferedReader inFromClient = new BufferedReader(new InputStreamReader(clientSocket.getInputStream()));
DataOutputStream outToClient = new DataOutputStream(clientSocket.getOutputStream());
String clientIP = clientSocket.getRemoteSocketAddress().toString();
System.out.println("The client " + clientIP + " connected!");
String clientMessage = inFromClient.readLine();
System.out.println("The client requested file: " + clientMessage);
// Get file. If doesn't exist, let's client know.
// Otherwise informs client of file size.
File myFile = new File(clientMessage);
if (!myFile.exists()) {
outToClient.writeBytes("File does not exist!\n");
return;
} else {
outToClient.writeBytes(String.valueOf((int)myFile.length()) + "\n");
}
// Create array for storage of file bytes:
byte[] byteArray = new byte[(int)myFile.length()];
BufferedInputStream bis = new BufferedInputStream(new FileInputStream(myFile));
// Read file into array:
bis.read(byteArray, 0, byteArray.length);
// Send the file:
outToClient.write(byteArray, 0, byteArray.length);
outToClient.close();
clientSocket.close();
}
}
}
Client Code:
import java.io.*;
import java.net.*;
class ft_client {
public static void main(String args[]) throws Exception {
int byteSize = 2022386;
int bytesRead;
/*
* Asks user for IP and port:
*/
BufferedReader inFromUser = new BufferedReader(new InputStreamReader(System.in));
System.out.println("Enter an IP address: ");
String ipAddress = inFromUser.readLine();
System.out.println("Enter a port: ");
String port = inFromUser.readLine();
Socket clientSocket;
try {
// Makes socket, port, and calls connect. Assumes it's TCP:
clientSocket = new Socket(ipAddress, Integer.valueOf(port));
} catch (Exception e) {
System.out.println(e.getMessage());
return;
}
// Creates InputStream from server to get file size and other messages:
BufferedReader inFromServer = new BufferedReader(new InputStreamReader(clientSocket.getInputStream()));
// Anything written to this will be sent to the server:
DataOutputStream outToServer = new DataOutputStream(clientSocket.getOutputStream());
// Asks for a file name to download from the server:
System.out.println("What file do you want?: ");
String message = inFromUser.readLine();
outToServer.writeBytes(message + "\n");
inFromUser.close();
// Listens for confirmation from server.
// If the file exists, the file size is delivered here:
String response = inFromServer.readLine();
System.out.println("File size: " + response);
if (response.equals("File does not exist!")) {
return;
}
// Receives file from server:
byteSize = (int) Integer.valueOf(response);
byte[] byteArray = new byte[byteSize];
InputStream is = clientSocket.getInputStream(); // calling clientSocket.getInputStream() twice???
FileOutputStream fos = new FileOutputStream(message);
BufferedOutputStream bos = new BufferedOutputStream(fos);
// Continuously writes the file to the disk until complete:
int total = 0;
while ((bytesRead = is.read(byteArray)) != -1) {
bos.write(byteArray, 0, bytesRead);
total += bytesRead;
}
bos.close();
System.out.println("File downloaded (" + total + " bytes read)");
clientSocket.close();
}
}
Are buffered readers interfering with output streams? Is there a better way to transfer files?
It's worth checking, in your server code, what value comes back from the file read() call, so:
int bytesRead = bis.read(byteArray, 0, byteArray.length);
System.out.println("File bytes read: " + bytesRead + " from file size: " + myFile.length());
The read() method is under no obligation to fill the byteArray - only to return something and to tell you how many bytes it read. From the docs, it:
Reads up to len bytes of data from this input stream into an array of
bytes. If len is not zero, the method blocks until some input is
available; otherwise, no bytes are read and 0 is returned.
You need to keep reading in a loop. I'd do this (actually, same as your client!):
int n;
while ((n = bis.read(byteArray, 0, byteArray.length)) != -1) {
// Send the chunk of n bytes
outToClient.write(byteArray, 0, n);
}
bis.close();
outToClient.close();
or something similar. I've closed the file too: it'd close on GC/finalize, but that could be a while, and meanwhile you're holding the file open.
EDIT
The specific problem with your image-read in this case is in your client code. You read the file size near the top of the code:
// Creates InputStream from server to get file size and other messages:
BufferedReader inFromServer = new BufferedReader(new InputStreamReader(clientSocket.getInputStream()));
and then you access the client again:
InputStream is = clientSocket.getInputStream(); // calling clientSocket.getInputStream() twice???
and as your comment suggests, this is bad! Thank you to #EJP for highlighting this!
This causes a problem of buffer over-ingestion: the BufferedReader consumes more bytes into its belly than you extract from it, so when you visit the clientSocket inputstream the second time, the read-pointer has moved on. You never look again at what the BufferedReader consumed.
As a general rule, once you plug buffering code onto something, you must be careful to read only from that buffer. In this case, it's difficult, because you can't read image (raw binary) data from a Reader, because it will busily interpret the binary values as characters and read them as UTF-8 or something.
Even without buffers, it's a minor sin to mix Readers (text oriented) and binary data (DataStreams) on the same stream. HTTP and email does this, so you are in good company, but they get away with it by being very tightly specified. Problem is, you can easily get snarled with questions of local/default character encoding at each end, whether you're reading Unix "LF" vs Windows "CR/LF" line endings etc.
In this case, try not using BufferedReaders at all, and try using DataInput/Output streams all the way. Try writeUTF(s) and readUTF() for transferring the String data. Ideally, create them like this:
DataInputStream inFromServer = new DataInputStream (new BufferedInputStream(clientSocket.getInputStream()));
so you still get the benefits of buffering.
EDIT 2
So seeing the new client code:
byteSize = (int) Integer.valueOf(response);
byte[] byteArray = new byte[byteSize];
FileOutputStream fos = new FileOutputStream(message);
int readBytes = inFromServer.read(byteArray);
// Continuously writes the file to the disk until complete:
int total = 0;
for (int i=0; i<byteArray.length; i++) {
fos.write(byteArray[i]);
total++;
}
fos.close();
Here, we're assuming that because the byteArray array is set to the right size, that the inFromServer.read(byteArray) will populate it - it won't. It's good to assume that any and all read operations will return you just as much data as the system has to hand: in this case, it's probably going to return as soon as it gets the first packet or two, with an underfilled array. This is same as C and Unix read behaviour too.
Try this - I'm repeatedly reading and writing a 4K buffer, until the byte count is reached (as determined by summing the return values of the reads):
byteSize = (int) Integer.valueOf(response);
byte[] byteArray = new byte[4096];
FileOutputStream fos = new FileOutputStream(message);
int total = 0;
// Continuously writes the file to the disk until complete:
while (total < byteSize && (readBytes = inFromServer.read(byteArray)) != -1) {
fos.write(byteArray, 0, readBytes);
total += readBytes;
}
fos.close();
A variant is this - same thing, but byte at a time. Might be a bit clearer. It's going to be slow - all those reads and writes are hitting the OS, but if you put a BufferedInputStream/BufferedOutputStream around the socket/file streams, it'll iron that out. I've added them:
DataInputStream inFromServer =
new DataInputStream(new BufferedInputStream(clientSocket.getInputStream()));
...
byteSize = (int) Integer.valueOf(response);
OutputStream fos = new BufferedOutputStream(FileOutputStream(message));
int total = 0;
int ch;
// Continuously writes the file to the disk until complete:
while (total < byteSize && (ch = inFromServer.read()) != -1) {
fos.write(ch);
total ++;
}
fos.close();
And finally! the simplest answer is this. Your code, but changed to:
int readBytes = inFromServer.readFully(byteArray);
Yes! Those nice people in 1990's Javasoft added a DataInput.readFully method, which does what you want! - basically wraps the code above. It's the simplest solution, and arguably most correct approach: "use existing libraries where possible". OTOH, it's the least educational, and the time you spend getting used to read/writes like this is not deducted from your life-expectancy!
And in fact, the readFully approach has severe limitations. Try pointing it at a 1GB file and see what happens (after you've fixed up the array size at the top): you'll a) run out memory, and b) wish that while you were ingesting a huge blob, you could at least be spooling it out to disk. If you try a 2.5G file, you'll notice that some of those ints should become longs to cope with numbers >= 2^31.
If it was me, I'd do the 4K buffer one. (BTW I'm writing this on a laptop with no Java compiler installed, so I haven't actually run the above! DO respond if there are any difficulties.)
So i am building a program which needs an auto-updating feature built in to it, as i was finished up and tested it out, it seems when i send the jar file over the socket and write it to the newly made jar file it is missing 5KB (everytime... even when the size changes) size from it and becomes corrupt.
Here is my code:
package server.update;
import java.io.*;
import java.net.Socket;
public class UpdateThread extends Thread
{
BufferedInputStream input; //not used
BufferedInputStream fileInput;
BufferedOutputStream output;
public UpdateThread(Socket client) throws IOException
{
super("UpdateThread");
output = new BufferedOutputStream(client.getOutputStream());
input = new BufferedInputStream(client.getInputStream());
}
public void run()
{
try
{
File perm = new File(System.getProperty("user.dir")+"/GameClient.jar");
//fileInput = new BufferedInputStream(new FileInputStream(perm));
fileInput = new BufferedInputStream(new FileInputStream(perm));
byte[] buffer = new byte[1024];
int numRead;
while((numRead = fileInput.read(buffer)) != -1)
output.write(buffer, 0, numRead);
fileInput.close();
input.close();
output.close();
this.interrupt();
}
catch(Exception e)
{e.printStackTrace();}
}
}
This is the class that will wait for a connection from the client and then push the update to them as soon as it connects. File Perm is the jar file that i want to send over and for whatever reason it seems to either miss the last 5 bytes or the client doesn't read the last 5 (i don't know which). Here is the client's class of receiving the information here:
public void getUpdate(String ip) throws UnknownHostException, IOException
{
System.out.println("Connecting to update socket");
update = new Socket(ip,10004);
BufferedInputStream is = new BufferedInputStream(update.getInputStream());
BufferedOutputStream os = new BufferedOutputStream(update.getOutputStream());
System.out.println("Cleaning GameClient.jar file");
File updated = new File(System.getProperty("user.dir")+"/GameClient.jar");
if(updated.exists())
updated.delete();
updated.createNewFile();
BufferedOutputStream osf = new BufferedOutputStream(new FileOutputStream(updated));
System.out.println("Writing to GameClient.jar");
byte[] buffer = new byte[1024];
int numRead = 0;
while((numRead = is.read(buffer)) != -1)
osf.write(buffer, 0, numRead);
System.out.println("Finished updating...");
is.close();
os.close();
update.close();
osf.close();
}
Any help is appreciated. Thanks!
You have too many closes. Remove update.close() and is.close(). These both close the socket, which prevents the buffered stream 'osf' from being auto-flushed when closed. Closing either the input stream or the output stream or a socket closes the other stream and the socket. You should therefore only close the outermost output stream you have wrapped around the socket, in this case osf, and maybe the socket itself in a finally block to be sure.
Thanks to MDR for the answer, it worked!!
I had to change the following lines of code in the UpdateThread class:
Before:
fileInput.close();
input.close();
output.close();
this.interrupt();
After:
fileInput.close();
output.flush();
output.close();
input.close();
this.interrupt();
You must flush the stream before closing, also i switched the order because if you closed the inputstream attached to the socket it will close the socket and then will not move on to closing the outputstream or flushing it.
Thanks again!
Have you considered using an http library to delegate all of the connection handling and reading/writing to known working code? You're reinventing a lot of wheels here. Additionally at some point you're going to want to ensure the content you're receiving is authentic and undamaged (you're doing that by loading the class, which is somewhat dangerous, especially when you're exchanging data in cleartext!) Again, using a library and its methods would allow you to choose HTTPS, allowing TLS to do much of your work.
I'd also suggest that your server tell the client some metadata in advance, regardless- perhaps the content length and possibly a hash or checksum so the client can detect failures in the transfer.
This question seems to have answers relevant to your situation as well. Good luck!
I am creating a file server application for school assignment. What I currently have is a simple Client class that sends an image through TCP and a Server class that receives it and writes it to the file.
this is my client code
import java.io.*;
import java.net.*;
class Client {
public static void main(String args[]) throws Exception {
long start = System.currentTimeMillis();
Socket clientSocket = new Socket("127.0.0.1", 6789);
DataOutputStream outToServer = new DataOutputStream(clientSocket.getOutputStream());
File file = new File("hot.jpg");
FileInputStream fin = new FileInputStream(file);
byte sendData[] = new byte[(int)file.length()];
fin.read(sendData);
outToServer.write(sendData, 0, sendData.length);
clientSocket.close();
long end = System.currentTimeMillis();
System.out.println("Took " + (end - start) + "ms");
}
}
and this is my server code.
import java.io.*;
import java.net.*;
class Server {
public static void main(String args[]) throws Exception {
ServerSocket serverSocket = new ServerSocket(6789);
Socket connectionSocket = serverSocket.accept();
DataInputStream dis = new DataInputStream(connectionSocket.getInputStream());
byte[] receivedData = new byte[61500]; // <- THIS NUMBER
for(int i = 0; i < receivedData.length; i++)
receivedData[i] = dis.readByte();
connectionSocket.close();
serverSocket.close();
FileOutputStream fos = new FileOutputStream("received.jpg");
fos.write(receivedData);
fos.close();
}
}
My question is how to get the size of the file that is being sent. If you check the Server code you'll see that I've hardcoded the number i.e. 61500 at the moment. How can I retrieve this number dynamically?
Or, am I doing this the wrong way? What an alternative solution would be?
Add one "length field" before sending the file. (Note that since you read the file to memory the maximum size of the file can be ~2GB.)
Before sending the file write the length of the file:
outToServer.writeInt(sendData.length);
And when receiving read the length first and use it as a length:
int dataLength = dis.readInt()
byte[] receivedData = new byte[dataLength];
A better way would be not to read the file into memory first but to transfer it directly from the FileInputStream - then you could transfer bigger files!
If you know the length, using readFully() is much more efficient than reading a byte at a time.
In this case, you don't need to know the length, you can write the loop to read/write as much data as you get.
InputStream is = connectionSocket.getInputStream();
byte[] bytes = new byte[8192];
int len;
while((len = is.read(bytes)) > 0)
fos.write(bytes, 0, len);
You can avoid reading the whole file into memory by copying the data as you read it.
FileInputStream fis = new FileInputStream(filename);
OutputStream os = socket.getOutputStream();
byte[] bytes = new byte[8192];
int len;
while((len = fis.read(bytes)) > 0)
os.write(bytes, 0, len);
You can use Apache IOUtils.copy() to perform the copy from one stream to another if you wish.
This approach has the advantage that the file can be any size (greater than 2 GB). Using an array is limited to 2 GB (and uses more memory)