Develop Reference

Java reading a file and sending it via DatagramSocket

Using Java, I am trying to send some file data over a DatagramSocket. I need to read a file in 1000-byte chunks and send them over as packets. My code:
reads a file into a byte array wrapped in a byte buffer
places the data in a packet and sends it
has the receiver open the packet and re-write the contents to a new file.
I am having a problem with writing the byte array back to a file. Please see my code below.
Client/Sender:
byte[] data = new byte[1000];
ByteBuffer b = ByteBuffer.wrap(data);
DatagramPacket pkt;
File file = new File(sourceFile);
FileInputStream fis = new FileInputStream(file);
BufferedInputStream bis = new BufferedInputStream(fis);
CRC32 crc = new CRC32();
while(true){
b.clear();
b.putLong(0); // I need to put the checksum at the beginning for easy retrieval
bytesRead = bis.read(data);
if(bytesRead==-1) { break; }
crc.reset();
crc.update(data, 8, data.length-8);
long chksum = crc.getValue();
b.rewind();
b.putLong(chksum);
pkt = new DatagramPacket(data, 1000, addr); // addr is valid, works fine
sk.send(pkt);
}
bis.close();
fis.close();
Server/Receiver:
DatagramSocket sk = new DatagramSocket(port);
File destfile = new File("hello.txt");
FileOutputStream fos = new FileOutputStream(destfile);
BufferedOutputStream bos = new BufferedOutputStream(fos);
PrintStream ps = new PrintStream(fos);
byte[] data = new byte[1000];
DatagramPacket pkt = new DatagramPacket(data, data.length);
ByteBuffer b = ByteBuffer.wrap(data);
CRC32 crc = new CRC32();
while(true) {
pkt.setLength(data.length);
sk.receive(pkt);
b.rewind();
// compare checksum, print error if checksum is different
// if checksum is the same:
bos.write(data); // Where the problem seems to be occurring.
// send acknowledgement packet.
}
bos.close();
fos.close();
Here, I am mainly having issues with writing the file back. With a small text file that says Hello World!, I get a strange output that says vˇ]rld!. Also, the input file is only 12 bytes but the file that the receiver creates is 1KB.
I think my issue is dealing with a byte buffer - I've written a program that copies files using file streams and buffered streams, which worked well. But I'm confused with how streams work in this sort of situation, and I would really appreciate any help. Thanks!

In the sender's data[] you overwrite the text, which was read from the file by the crc! You have to read the text in a position after the long. When correcting this in the Sender, it works:
//int bytesRead = bis.read(data); --old code
int bytesRead=bis.read(data,8,data.length-8);
Furthermore you send 1000 bytes, so will receive 1000 bytes, which will go into the destfile.
BTW: you do not check the crc in the server.... so why send it ?

File Object Serialization

I have a client server program where I need to serialize the file object and send it to the client.
At server side:
FileInputStream input_file = new FileInputStream(file);
object_output_stream.writeObject(input_file);
At client side:
FileOutputStream ouput_file = new FileOutputStream(new File(filename));
output_file = object_input_stream.readObject();
I need to serialize the input_file object and send it to the client. The ObjectOutputStream and ObjectInputStream are Non-Serializable. What would be the best way for this?

You cannot serialize files - that would imply that the client could read from a file at the server, which would require a complicated protocol that is simply not present in the Java serialization mechanism.
Your best best is to read the data from the file into a byte array, and to then either send the byte array plainly to the client, or to serialize the byte array in the ObjectOutputStream (you would do that if you want to send other objects as well)
You can use apache-commons IOUtils.toByteArray(InputStream input) to read a file into a byte[] easily.
On the server side:
FileInputStream input_file = new FileInputStream(file);
byte[] input_data = IOUtils.toByteArray(input_file);
object_output_stream.writeObject(input_data);
On the client side:
FileOutputStream output_file = new FileOutputStream(new File(filename));
byte[] input_data = (byte[]) object_input_stream.readObject();
output_file.write(input_data);

Send file bytes inside object through socket

I've been trying to get this working for a few days now, but I've had no success.
I want to send a file over socket client/server. The only difference is: I want to send an object that contains the file bytes.
So the client loads a file, reads chunks of 1024 bytes, store them in a object, and send the object to the server. Since the file can be larger than 1024 bytes, I want to send the object repeatedly but with different bytes stored in them (as the buffer reads it). On the server, I want to compose the array of bytes and save it as a file.
The reason I'm using 1024 is because I want to avoid any sort of out of memory error, if the file is, let's say, 4 GB in size.
I tried doing the following on the client:
File file = new File("C:\\test\\test.txt");
BufferedInputStream bis = new BufferedInputStream(new FileInputStream(file));
ObjectOutputStream oos = new ObjectOutputStream(socket.getOutputStream());
byte[] bytes = new byte[1024];
FileTest ft = new FileTest();
ft.setName("Testing");
int counttest = 1;
while (bis.read(bytes) > 0) {
ft.setCounttest(counttest);
ft.setBytes(bytes);
oos.writeObject(ft);
counttest += 1;
}
On the server:
int bufferSize = socket.getReceiveBufferSize();
BufferedOutputStream bos = new BufferedOutputStream(new FileOutputStream("C:\\test\\test2.txt"));
ObjectInputStream ois = new ObjectInputStream(socket.getInputStream());
byte[] bytes = new byte[bufferSize];
while (true) {
FileTest ft = (FileTest) ois.readObject();
if (ft != null) {
System.out.println(ft.getName());
bos.write(ft.getBytes());
}
}
So I tested sending a txt file with a sequence of numbers and the test2.txt file produced by the server came out only with the first 1024 chunk of bytes repeated twice. Also, the counttest integer never increases when received in the server.
Any idea how to accomplish this?
Thanks in advance.

You are running into the effects of the ObjectOutputStream's attempt to preserve object identity. Repeatedly writing the same object instance will result on the same instance on the receiver's end. This is generally a good thing, but confusing if you are modifying the object on the sender's end and expecting those modifications to show up on the receiver's end.
Thus, you have two issues:
In order to send the data each time, you either need to create a new FileTest instance each time, or use the writeUnshared() method.
Due to this identity preserving behavior, you will need to periodically reset() the ObjectOutputStream in order to keep all these instances from being held forever (and potentially leading to an OOME on the client or server).

Sending big file using FileInputStream/ObjectOutputStream

I need help on my homework, any help will be much appreciated. I can send small files without a problem. But when i try to send let’s say a 1GB file byte array sends OutOfMemoryError so i need a better solution to send file from server to client. How can i improve this code and send big files, please help me.
Server Code:
FileInputStream fis = new FileInputStream(file);
byte[] fileByte = new byte[fis.available()]; //This causes the problem.
bytesRead = fis.read(fileByte);
oos = new ObjectOutputStream(sock.getOutputStream());
oos.writeObject(fileByte);
Client Code:
ois = new ObjectInputStream(sock.getInputStream());
byte[] file = (byte[]) ois.readObject();
fos = new FileOutputStream(file);
fos.write(file);

Don't read the whole file into memory, use a small buffer and write while you are reading the file:
BufferedOutputStream bos = new BufferedOutputStream(sock.getOutputStream())
File file = new File("asd");
FileInputStream fis = new FileInputStream(file);
BufferedInputStream bis = new BufferedInputStream(fis);
byte[] buffer = new byte[1024*1024*10];
int n = -1;
while((n = bis.read(buffer))!=-1) {
bos.write(buffer,0,n):
}
Use Buffered* to optimize the writing and reading from Streams

Just split the array to smaller chunks so that you don't need to allocate any big array.
For example you could split the array into 16Kb chunks, eg new byte[16384] and send them one by one. On the receiving side you would have to wait until a chunk can be fully read and then store them somewhere and start with next chunk.
But if you are not able to allocate a whole array of the size you need on server side you won't be able to store all the data that you are going to receive anyway.
You could also compress the data before sending it to save bandwidth (and time), take a look at ZipOutputStream and ZipInputStream.

Here's how I solved it:
Client Code:
bis=new BufferedInputStream(sock.getInputStream());
fos = new FileOutputStream(file);
int n;
byte[] buffer = new byte[8192];
while ((n = bis.read(buffer)) > 0){
fos.write(buffer, 0, n);}
Server Code:
bos= new BufferedOutputStream(sock.getOutputStream());
FileInputStream fis = new FileInputStream(file);
BufferedInputStream bis = new BufferedInputStream(fis);
int n=-1;
byte[] buffer = new byte[8192];
while((n = bis.read(buffer))>-1)
bos.write(buffer,0,n);

Depending on whether or not you have to write the code yourself, there are existing libraries which solve this problem, e.g. rmiio. If you are not using RMI, just plain java serialization, you can use the DirectRemoteInputStream, which is kind of like a Serializable InputStream. (this library also has support for things like auto-magically compressing the data).
Actually, if you are only sending file data, you would be better off ditching the Object streams and use DataInput/DataOutput streams. first write an integer indicating the file length, then copy the bytes directly to the stream. on the receiving side, read the integer file length, then read exactly that many bytes.
when you copy the data between streams, use a small, fixed size byte[] to move chunks of data between the input and output streams in a loop. there are numerous examples of how to do this correctly available online (e.g. #ErikFWinter's answer).

File transfer and object serialization in Java

Im working on a Client/server chat application which allows user to send files (images / videos...) through a socket connection.
In order to manage all kind of communication, I use an Object "Packet" which stores all information that I want to send. (Sender, receivers, file ...).
Here is a code sample where I write in the stream :
private void write(Packet packet) throws IOException {
ByteArrayOutputStream bos = new ByteArrayOutputStream();
ObjectOutputStream os = new ObjectOutputStream(bos);
os.writeObject(packet);
this.outStream.write(bos.toByteArray());
}
And outStream is an OutputStream.
Here is my Connection run :
public void run() {
while (isRunning()) {
try {
byte[] buffer = new byte[65536];
// Read from the InputStream
inStream.read(buffer);
ObjectInputStream in = new ObjectInputStream(new ByteArrayInputStream(buffer));
Packet p = (Packet) in.readObject();
} catch (IOException e) {
e.printStackTrace();
this.disconnect();
}
}
}
It works very well for all purpose except files transfer !
I put the file in a byte[] (with filestream) and store the array in my Packet Object.
When the server receive the communication it breaks on the "in.readObject()" and give me a pretty "java io streamcorruptedexception wrong format : 0" exception.
I tried the transfer with a custom byte[] (filled by a string.getBytes()) and it worked very well.
So, what am I doing wrong ?

You're reading from the InputStream to a byte array (with an arbitrary size which could be too small). Then you construct an ObjectInputStream to read from this byte array. Why don't you read your object directly from the InputStream?
ObjectInputStream in = new ObjectInputStream(inStream);
Packet p = (Packet) in.readObject();
No need for a buffer.
Moreover, InputStream.read() doesn't read everything from the InputStream. It reads what is available, and returns the number of bytes read. If you don't loop until it returns -1, you only read a part of what has been sent on the other side.
BTW, you're doing the same mistake on the sending side. Instead of writing your object directly to the output stream, you write it to a byte array, adn then send this byte array. Write your object directly to the stream:
ObjectOutputStream os = new ObjectOutputStream(this.outputStream);
os.writeObject(packet);
No need for a buffer.