I am working on a client/server transfer protocol in java. The client is sending a simple text file, everything is going across the wire fine in wireshark, but once it gets to the server side, the first two letters are missing from the text file. I believe that it is overwriting the first buffer for some reason.
My goal is to make a while loop that reads the bytes in the buffer and then increments a count that'll place the next set of bytes....in the place if the ones already written
Here is the server's code that I currently have:
int bytesRead;
int current = 0;
InputStream in = s.getInputStream();
// Instantiating a new output stream object
OutputStream output = new FileOutputStream(myFile);
PrintStream stream = new PrintStream(output);
// Receive file 1024 bytes at a time
byte[] buffer = new byte[1024];
while ((bytesRead = in.read(buffer)) != -1) {
output.write(buffer, 0, bytesRead);
System.out.println(output.toString());
}
Related
I have to do a file transfer, in my case a pdf, through socket in java for my homework. Usually I requested text and got text back, but this time I have to send a file through socket. In my investigation I discovered that file transfers are made with Fileinput(output)streams. My problem is that the request to the server has to look something like this:
File file = new File(pathToFile);
Pirntwriter out = new PrintWriter(Socket s.getOutputStream());
Outputstream outFile = s.getOutputStream();
int count
out.write("user file\r\n"
+ file.getName()+"\r\n"
+ file.length()+"\r\n"
+ "body\r\n");
// send file but im not sure how
byte[] buffer = new buffer with size of file.length()
while ((count = in.read(buffer)) > 0){
outFile.write(buffer, 0, count);
}
out.flush
outFile.flush
Unfortunately this doesn't work for me. In this way the server counts the requests as two different outputs. Is there a way to combine both Outputstreams or write the request in one single Outputstream?
I am trying to transfer larger files over socket.I will be transferring the file in chunks.As shown in the code.link
int count;
byte[] buffer = new byte[8192];
while ((count = in.read(buffer)) > 0)
{
out.write(buffer, 0, count);
}
before sending this file,i want to send an object which holds the details of the file.Using which stream should i send Object + File.
I am new to streams,can i get any sample code.
Can i send the byte length of the object first to read the object ,save it and send file data.is it possible,any sample code ?
Thanks
421
I'm not sure whether using Java's serialization mechanism is the best way to do a simple file transfer. As your question suggest, you try to avoid keeping the whole file in memory at any time. This can be done with objects using the Proxy pattern but if all you want to do is transfer the file, this might not be the most straight-forward solution. (Also, it will effectively tie your peer to be implemented in Java too.)
Instead, why not take a look at an extremely successful protocol that does exactly what you need: HTTP.
Content-Type: application/octet-stream
Content-Length: 542183
542183 bytes of data follow...
It should not be too hard for you to write a parser for the meta-data header.
You need to ensure the order of writing/reading. If write an object -> write raw bytes on client, then read an object -> read raw bytes on server. When reading, ObjectInputStream should be able to find the boundary of the serialized object data.
If you want to keep a socket connection long-live and use its streams multiple times, wrapping socket's Output/InputStream in a ObjectOutput/InputStream is not a good idea IMO. When you close a object stream, it closes the underlying stream as well.
So you may want to write the length of serialized object data first (file length is contained in the object so you don't need to write it explictly), e.g. 4 bytes of BigEndian encoded int. Then serialize the object into a ByteArrayOutputStream, and write the bytes in its buffer. On the server, read 4 bytes first, decode the bytes back to an int, and read that many bytes into a byte[], wrap the byte array with a ByteArrayInputStream and deserialize the object from it.
Write like this:
......
OutputStream out = socket.getOutputStream();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ObjectOutputStream oos = new ObjectOutputStream(baos);
oos.writeObject(fileInfoObject);
oos.close();
byte[] header = encodeInteger(baos.size());
out.write(header, 0, 4);
baos.writeTo(out);
// write the file to out just as your question shows
On the receiving side:
......
InputStream in = socket.getInputStream();
// read the int
byte[] header = new byte[4];
in.read(header, 0, 4);
int size = decodeInteger(header);
// read the object
byte[] objectbuf = new byte[size];
int count;
while((count += in.read(objectbuf)) < size); // not sure if this works...
ObjectInputStram ois = new ObjectInputStream(new ByteArrayInputStream(objectbuf));
Object fileInfoObject = ois.readObject();
ois.close();
// read the file
FileOutputStream fos = new FileOutputStream(new File("somefile"));
byte[] buffer = new byte[8192];
count = 0;
long left = castedFileInfoObject.fileSize;
// also not sure if this works, not tested.
int maxRead = buffer.length;
while (true) {
count = in.read(buffer, 0, maxRead);
left -= count;
if (left < 8192) {
maxRead = (int)left;
}
fos.write(buffer, 0, count);
if (left == 0) {
break;
}
}
I haven't tested the sample code in my answer.. just to show the idea.
Class MyClass should implement the Serializable interface. Then, an object of this class can be written to an ObjectOutputStream and read back from ObjectInputStream using writeObject and readObject methods (See below).
On Client:
Socket socket = new Socket(url, port);
OutputStream os = socket.getOutputStream();
ObjectOutputStream oos = new ObjectOutputStream(os);
MyClass obj = new Myclass();
oos.writeObject(obj);
int count;
byte[] buffer = new byte[8192];
while ((count = in.read(buffer)) > 0) {
out.write(buffer, 0, count);
}
On server:
ServerSocket sSocket = new ServerSocket(port);
Socket socket = sSocket.accept();
InputStream is = socket.getInputStream();
ObjectInputStream ois = new ObjectInputStream(is);
MyClass obj = (MyClass)ois.readObject();
byte arr[];
try {
while(arr = (byte[])ois.readObject()) {
//do something with arr
}
} catch(java.io.EOFException) {
// End of data
}
If you need to send more data after the file is finished, you need a way to figure out the number of bytes the file consists of. Then, you can send the number of bytes beforehand over the socket to the server. On the server, read only that many bytes of information for the file and then do the same for the rest of the data you are going to send. This strategy of pre-sending the file size is recommended and is mostly used while doing any data transfer. If you can do that, you don't have to rely on catching java.io.EOFException to detect end of data.
Ok, So I'm making a Java program that has a server and client and I'm sending a Zip file from server to client. I have sending the file down, almost. But recieving I've found some inconsistency. My code isn't always getting the full archive. I'm guessing it's terminating before the BufferedReader has the full thing. Here's the code for the client:
public void run(String[] args) {
try {
clientSocket = new Socket("jacob-custom-pc", 4444);
out = new PrintWriter(clientSocket.getOutputStream(), true);
in = new BufferedInputStream(clientSocket.getInputStream());
BufferedReader inRead = new BufferedReader(new InputStreamReader(in));
int size = 0;
while(true) {
if(in.available() > 0) {
byte[] array = new byte[in.available()];
in.read(array);
System.out.println(array.length);
System.out.println("recieved file!");
FileOutputStream fileOut = new FileOutputStream("out.zip");
fileOut.write(array);
fileOut.close();
break;
}
}
}
} catch(IOException e) {
e.printStackTrace();
System.exit(-1);
}
}
So how can I be sure the full archive is there before it writes the file?
On the sending side write the file size before you start writing the file. On the reading side Read the file size so you know how many bytes to expect. Then call read until you have gotten everything you expect. With network sockets it may take more than one call to read to get everything that was sent. This is especially true as your data gets larger.
HTTP sends a content-length: x+\n in bytes. This is elegant, it might throw a TimeoutException if the conn is broken.
You are using a TCP socket. The ZIP file is probably larger than the network MTU, so it will be split up into multiple packets and reassembled at the other side. Still, something like this might happen:
client connects
server starts sending. The ZIP file is bigger than the MTU and therefore split up into multiple packets.
client busy-waits in the while (true) until it gets the first packets.
client notices that data has arrived (in.available() > 0)
client reads all available data, writes it to the file and exits
the last packets arrive
So as you can see: Unless the client machine is crazily slow and the network is crazily fast and has a huge MTU, your code simply won't receive the entire file by design. That's how you built it.
A different approach: Prefix the data with the length.
Socket clientSocket = new Socket("jacob-custom-pc", 4444);
DataInputStream dataReader = new DataInputStream(clientSocket.getInputStream());
FileOutputStream out = new FileOutputStream("out.zip");
long size = dataReader.readLong();
long chunks = size / 1024;
int lastChunk = (int)(size - (chunks * 1024));
byte[] buf = new byte[1024];
for (long i = 0; i < chunks; i++) {
dataReader.read(buf);
out.write(buf);
}
dataReader.read(buf, 0, lastChunk);
out.write(buf, 0, lastChunk);
And the server uses DataOutputStream to send the size of the file before the actual file. I didn't test this, but it should work.
How can I make sure I received whole file through socket stream?
By fixing your code. You are using InputStream.available() as a test for end of stream. That's not what it's for. Change your copy loop to this, which is also a whole lot simpler:
while ((count = in.read(buffer)) > 0)
{
out.write(buffer, 0, count);
}
Use with any buffer size greater than zero, typically 8192.
In.available() just tells you that there is no data to be consumed by in.read() without blocking (waiting) at the moment but it does not mean the end of stream. But, they may arrive into your PC at any time, with TCP/IP packet. Normally, you never use in.available(). In.read() suffices everything for the reading the stream entirely. The pattern for reading the input streams is
byte[] buf;
int size;
while ((size = in.read(buf)) != -1)
process(buf, size);
// end of stream has reached
This way you will read the stream entirely, until its end.
update If you want to read multiple files, then chunk you stream into "packets" and prefix every one with an integer size. You then read until size bytes is received instead of in.read = -1.
update2 Anyway, never use in.available for demarking between the chunks of data. If you do that, you imply that there is a time delay between incoming data pieces. You can do this only in the real-time systems. But Windows, Java and TCP/IP are all these layers incompatible with real-time.
When I try to send a large file from server by splitting it, some of the packages don't arrive at the client... as you can see in the console output
http://s7.postimg.org/94yjfame3/error.png
the client receive only 19799.. bytes , and the server sent 62800.. bytes.
the code is too long to past here... but here are the basics:
// server side -> send data
BufferedOutputStream out = new BufferedOutputStream(socket.getOutputStream());
byte[] somePackageInfo= new byte[500];
byte[] streamOut = new byte[20000];
while(getDataFromLargeFile(somePackageInfo,streamOut) != 0) {
out.write(somePackageInfo,0,500);
out.write(streamOut);
out.flush();
}
out.write(0);
out.flush();
// client side -> get data
BufferedInputStream in = new BufferedInputStream(socket.getInputStream());
byte[] somePackageInfo= new byte[500];
byte[] streamIn= new byte[20000];
while(true) {
if(in.read(somePackageInfo,0,500) == 0) break;
in.read(streamIn);
saveDataToLargeFile(somePackageInfo,streamIn);
}
I tried to slow down the transfer (sleep(500)) but only most of the packages arrived.
tried to remove the flush() but still only most of the packages arrived.
what causes this problem and how can i fix it?
Your copy code is wrong. You are ignoring the count returned by read, and assuming that it fills the buffer. It isn't required to do that. See the Javadoc.
while ((count = in.read(buffer)) > 0)
{
out.write(buffer, 0, count);
}
Use with any buffer size greater than zero, typically 8192. Use at both ends.
Adding sleeps is literally a waste of time.
I would like to send image file from java server to android app using this code:
Server(Java):
File file = new File("./clique.jpg");
FileInputStream stream = new FileInputStream(file);
DataOutputStream writer = new DataOutputStream(socket.getOutputStream());
byte[] contextB = new byte[4096];
int n;
int i = 0;
while ( (n=stream.read(contextB))!=-1 ){
writer.write(contextB, 0, n);
writer.flush();
System.out.println(n);
i+=n;
}
writer.flush();
stream.close();
android app:
DataInputStream reader = new DataInputStream(socket.getInputStream());
byte[] buffer = new byte[4096];
ByteArrayOutputStream content = new ByteArrayOutputStream();
int n;
int i = 0;
reader = new DataInputStream(socket.getInputStream());
while ( (n=reader.read(buffer)) != null){
content.write(buffer, 0, n);
content.flush();
}
Utility.CreateImageFile(content.toByteArray());
What I noticed is that in the android app n which is the number of read bytes is not 4096 while I am sending from server byte blocks of 4096 size,also I can not get n=-1 which is the end of stream,it blocks until I close the app then I get n=-1.
Regarding the number of bytes you read at a time has nothing to do with the number of bytes you write -it very much depends on the network conditions and will be variable with every chunk (basically as many bytes managed to be transmitted in short period of time between your reads as many you will get in the read chunk.
Regarding the end of stream - in your server code you have forgotten to close the output stream (you only close the stream which is input stream - you should also close the writer which in turn will close the underlying output stream.
Two comments:
1) I would really recommend to use Buffered Readers/Writers wrapping the writes/readers - the code you will get will be nicer and you will not have to create/manage buffers yourself.
2) Use try {} finally and close your streams in finally clauses - this is the best practice that will make sure that you will close the streams and free resources even in case of problems while reading/writing.
You got a problem in your android code:
while ( (n=reader.read(buffer)) != null) {
n can not be null.
Use writer.close() instead of writer.flush() after your loop on the server.