Im working on a Client/server chat application which allows user to send files (images / videos...) through a socket connection.
In order to manage all kind of communication, I use an Object "Packet" which stores all information that I want to send. (Sender, receivers, file ...).
Here is a code sample where I write in the stream :
private void write(Packet packet) throws IOException {
ByteArrayOutputStream bos = new ByteArrayOutputStream();
ObjectOutputStream os = new ObjectOutputStream(bos);
os.writeObject(packet);
this.outStream.write(bos.toByteArray());
}
And outStream is an OutputStream.
Here is my Connection run :
public void run() {
while (isRunning()) {
try {
byte[] buffer = new byte[65536];
// Read from the InputStream
inStream.read(buffer);
ObjectInputStream in = new ObjectInputStream(new ByteArrayInputStream(buffer));
Packet p = (Packet) in.readObject();
} catch (IOException e) {
e.printStackTrace();
this.disconnect();
}
}
}
It works very well for all purpose except files transfer !
I put the file in a byte[] (with filestream) and store the array in my Packet Object.
When the server receive the communication it breaks on the "in.readObject()" and give me a pretty "java io streamcorruptedexception wrong format : 0" exception.
I tried the transfer with a custom byte[] (filled by a string.getBytes()) and it worked very well.
So, what am I doing wrong ?
You're reading from the InputStream to a byte array (with an arbitrary size which could be too small). Then you construct an ObjectInputStream to read from this byte array. Why don't you read your object directly from the InputStream?
ObjectInputStream in = new ObjectInputStream(inStream);
Packet p = (Packet) in.readObject();
No need for a buffer.
Moreover, InputStream.read() doesn't read everything from the InputStream. It reads what is available, and returns the number of bytes read. If you don't loop until it returns -1, you only read a part of what has been sent on the other side.
BTW, you're doing the same mistake on the sending side. Instead of writing your object directly to the output stream, you write it to a byte array, adn then send this byte array. Write your object directly to the stream:
ObjectOutputStream os = new ObjectOutputStream(this.outputStream);
os.writeObject(packet);
No need for a buffer.
Related
ObjectOutputStream oos = new ObjectOutputStream(c.getOutputStream());
ObjectInputStream ois = new ObjectInputStream(c.getInputStream());
File file = new File("lol.txt");
if(!file.exists()){
file.createNewFile();
}
byte[] textBytes;
while((textBytes = (byte[])ois.readObject()) != null){
Files.write(file.toPath(), textBytes);
}
//do stuff...
byte[] textBytes;
while((textBytes = (byte[])ois.readObject()) != null){
Files.write(file.toPath(), textBytes);
}
How can I read a file on a server multiple times? Should this code work? Will it not get stuck in the first loop check?
The server is writing it to the client like this.
byte[] fileBytes = Files.readAllBytes(fp.toPath());
oos.writeObject(fileBytes);
oos.flush();
A repeatable read on a Socket's InputStream is not possible, because it is a buffer based, blocking implementation, using a descriptor to indicate where the current position in the buffer is.
When you read from the InputStream the descriptor moves the amount of bytes you have read. It's not possible to rewind the descriptor to the previous position, because the read bytes maybe already overwritten by new received bytes.
You server client communication must be in this way
(I use the following abbreviations: S -> a Server, C -> a Client):
C: Request file from S
S: Send file to C
C: Request file from S
S: Send file to C
(and so on)
I am trying to transfer larger files over socket.I will be transferring the file in chunks.As shown in the code.link
int count;
byte[] buffer = new byte[8192];
while ((count = in.read(buffer)) > 0)
{
out.write(buffer, 0, count);
}
before sending this file,i want to send an object which holds the details of the file.Using which stream should i send Object + File.
I am new to streams,can i get any sample code.
Can i send the byte length of the object first to read the object ,save it and send file data.is it possible,any sample code ?
Thanks
421
I'm not sure whether using Java's serialization mechanism is the best way to do a simple file transfer. As your question suggest, you try to avoid keeping the whole file in memory at any time. This can be done with objects using the Proxy pattern but if all you want to do is transfer the file, this might not be the most straight-forward solution. (Also, it will effectively tie your peer to be implemented in Java too.)
Instead, why not take a look at an extremely successful protocol that does exactly what you need: HTTP.
Content-Type: application/octet-stream
Content-Length: 542183
542183 bytes of data follow...
It should not be too hard for you to write a parser for the meta-data header.
You need to ensure the order of writing/reading. If write an object -> write raw bytes on client, then read an object -> read raw bytes on server. When reading, ObjectInputStream should be able to find the boundary of the serialized object data.
If you want to keep a socket connection long-live and use its streams multiple times, wrapping socket's Output/InputStream in a ObjectOutput/InputStream is not a good idea IMO. When you close a object stream, it closes the underlying stream as well.
So you may want to write the length of serialized object data first (file length is contained in the object so you don't need to write it explictly), e.g. 4 bytes of BigEndian encoded int. Then serialize the object into a ByteArrayOutputStream, and write the bytes in its buffer. On the server, read 4 bytes first, decode the bytes back to an int, and read that many bytes into a byte[], wrap the byte array with a ByteArrayInputStream and deserialize the object from it.
Write like this:
......
OutputStream out = socket.getOutputStream();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ObjectOutputStream oos = new ObjectOutputStream(baos);
oos.writeObject(fileInfoObject);
oos.close();
byte[] header = encodeInteger(baos.size());
out.write(header, 0, 4);
baos.writeTo(out);
// write the file to out just as your question shows
On the receiving side:
......
InputStream in = socket.getInputStream();
// read the int
byte[] header = new byte[4];
in.read(header, 0, 4);
int size = decodeInteger(header);
// read the object
byte[] objectbuf = new byte[size];
int count;
while((count += in.read(objectbuf)) < size); // not sure if this works...
ObjectInputStram ois = new ObjectInputStream(new ByteArrayInputStream(objectbuf));
Object fileInfoObject = ois.readObject();
ois.close();
// read the file
FileOutputStream fos = new FileOutputStream(new File("somefile"));
byte[] buffer = new byte[8192];
count = 0;
long left = castedFileInfoObject.fileSize;
// also not sure if this works, not tested.
int maxRead = buffer.length;
while (true) {
count = in.read(buffer, 0, maxRead);
left -= count;
if (left < 8192) {
maxRead = (int)left;
}
fos.write(buffer, 0, count);
if (left == 0) {
break;
}
}
I haven't tested the sample code in my answer.. just to show the idea.
Class MyClass should implement the Serializable interface. Then, an object of this class can be written to an ObjectOutputStream and read back from ObjectInputStream using writeObject and readObject methods (See below).
On Client:
Socket socket = new Socket(url, port);
OutputStream os = socket.getOutputStream();
ObjectOutputStream oos = new ObjectOutputStream(os);
MyClass obj = new Myclass();
oos.writeObject(obj);
int count;
byte[] buffer = new byte[8192];
while ((count = in.read(buffer)) > 0) {
out.write(buffer, 0, count);
}
On server:
ServerSocket sSocket = new ServerSocket(port);
Socket socket = sSocket.accept();
InputStream is = socket.getInputStream();
ObjectInputStream ois = new ObjectInputStream(is);
MyClass obj = (MyClass)ois.readObject();
byte arr[];
try {
while(arr = (byte[])ois.readObject()) {
//do something with arr
}
} catch(java.io.EOFException) {
// End of data
}
If you need to send more data after the file is finished, you need a way to figure out the number of bytes the file consists of. Then, you can send the number of bytes beforehand over the socket to the server. On the server, read only that many bytes of information for the file and then do the same for the rest of the data you are going to send. This strategy of pre-sending the file size is recommended and is mostly used while doing any data transfer. If you can do that, you don't have to rely on catching java.io.EOFException to detect end of data.
I have a problem with sending large string through socket from server to android client.
String is about 10MB.
Code for writing data to socket is this:
int socketTimeout = 200;
socket = new Socket(client.getHost(), client.getPort());
socket.setSoTimeout(socketTimeout);
OutputStreamWriter oos=new OutputStreamWriter(socket.getOutputStream());
String d = data.getData().toString() + "\n";
oos.write(d);
oos.flush();
Code for reading data from socket is this:
Socket s = params[0];
InputStream is = null;
try {
is = s.getInputStream();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
int nRead;
byte[] data = new byte[32768];
while ((nRead = is.read(data, 0, data.length)) != -1) {
baos.write(data, 0, nRead);
}
return new String(baos.toByteArray());
}
So problem comes at line where I'm reading from inputStream where I get OutOfMemoryException. I tried using different examples of reading string from stream. I tried with BufferedInputStream, InputStreamReader, IOUtils, StringBuilder, BufferedReader ..etc. and all of them give me OutOfMemory exception when the string is large. I tested with smaller data something around 100K and it works perfectly.
Exception that I get on server-side is "Connection is reset by peer, socket write error."
You can read byte by byte in the client and write to a File byte by byte, in that way you are not holding the whole string in memory.
And then of course read that file by tokens or lines, not the whole string at once
By storing it in a ByteArrayOutputStream (or similar), you are coming up against the maximum heap size for the JVM in Android. This is a different size depending on the device. See: Android heap size on different phones/devices and OS versions
As has already been suggested, you should consider using a file stream to write the received data to disk.
I've serialized some objects so I can convert them to byte arrays for a TCP packet. When I send the objects from the server program to the client program, there are no issues and it works fine. However, even though the code between the server and client is identical, when I try to send objects from the client to the server I get an invalid header.
Here are the objects I'm serializing:
public static byte[] serialize(Hand c) throws IOException
{
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ObjectOutputStream oos = new ObjectOutputStream(baos);
oos.writeObject(c);
return baos.toByteArray();
}
public static Hand deserialize(byte[] bytes) throws IOException, ClassNotFoundException
{
ByteArrayInputStream b = new ByteArrayInputStream(bytes);
ObjectInputStream o = new ObjectInputStream(b);
return (Hand) o.readObject();
}
and
public static byte[] serialize(Card c) throws IOException
{
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ObjectOutputStream oos = new ObjectOutputStream(baos);
oos.writeObject(c);
return baos.toByteArray();
}
public static Card deserialize(byte[] bytes) throws IOException, ClassNotFoundException
{
ByteArrayInputStream b = new ByteArrayInputStream(bytes);
ObjectInputStream o = new ObjectInputStream(b);
return (Card) o.readObject();
}
Those are both taken from the Server program, but the code for the serialization is identical between the server and the client; I important the Card class and the Hand class from the Server to the Client precisely to make sure errors like this wouldn't occur.
The server can convert a Card or a Hand to a byte[] and write it over a DataOutputStream to the client, and the client can receive the Card or Hand through a DataInputStream, deserialize it, and read it with no problem. When I try to send a Card or a Hand from the Client to the Server, however, very rarely it works and usually I get a
Exception in thread "main" java.io.StreamCorruptedException: invalid stream header: 434B0005
at java.io.ObjectInputStream.readStreamHeader(Unknown Source)
at java.io.ObjectInputStream.<init>(Unknown Source)
at Hand.deserialize(Hand.java:29)
at KoiKoi_TCP_Server.takeClientTurn(KoiKoi_TCP_Server.java:321)
at KoiKoi_TCP_Server.main(KoiKoi_TCP_Server.java:380)
where Hand.java.29 points at the line
ObjectInputStream o = new ObjectInputStream(b);
in the Hand deserialization method.
I understand that it's telling me that the header is invalid. I'm not sure how to fix it, because it only breaks going one direction and the code is identical. Suggestions?
I'm only sending a solitary object at a time, so I'm not initializing multiple ObjectInputStreams or anything.
I'm not initializing multiple ObjectOutputStreams or anything.
Yes you are. You are initializing a new ObjectOutputStream for every object, and then you're giving yourself the additional problem of knowing how many bytes to read in order to receive each object, and you're getting that wrong, so you're getting out of sync.
Get rid of all this. You don't need it. It is just adding problems. Just use a single ObjectOutputStream and ObjectInputStream, directly, for the life of the socket, constructed directly over the socket streams, and call writeObject() when you want to send an object, and readObject() when you want to read one. Two lines of code. Forget about the byte arrays and the ByteArray/DataInput/OutputStreams altogether.
I'm creating a TFTP server. I've got it tranfering files fine but most of the files wont open when they arrive at the other end. This is because the output of the ArrayList im using to store file bytes from every packet received adds a load of bytes to the start of the file. eg. "¬í sr java.util.ArrayListxÒ™Ça I sizexp w ur [B¬óøTà xp ü!". The reason for using the List in the first place is that the server im creating has no way to tell the file size of the file which is being received. Therefore as far as I can tell I cant use a byte[] as this needs to be initialised with a set length. Is there any way round this?
WRQ WRQ = new WRQ();
ACK ACK = new ACK();
DatagramPacket outPacket;
byte[] bytes;
byte[] fileOut;
List fileBytes = new ArrayList();
outPacket = WRQ.firstPacket(packet);
socket.send(outPacket);
socket.receive(packet);
while (packet.getLength() == 516){
bytes = WRQ.doWRQ(packet);
fileBytes.add(bytes);
outPacket = ACK.doACK(packet);
socket.send(outPacket);
socket.receive(packet);
}
bytes = WRQ.doWRQ(packet);
fileBytes.add(bytes);
outPacket = ACK.doACK(packet);
socket.send(outPacket);
ObjectOutputStream os;
ByteArrayOutputStream byteStream = new ByteArrayOutputStream(5000);
os = new ObjectOutputStream(new BufferedOutputStream(byteStream));
os.flush();
os.writeObject(fileBytes);
os.flush();
byte[] outFile = byteStream.toByteArray();
os.close();
FileOutputStream foStream = new FileOutputStream(filename);
foStream.write(outFile);
foStream.close();
You store byte arrays in an ArrayList, and then you write the whole ArrayList to a ByteArrayOutputStream wrapped in an ObjectOutputStream, using the writeObject() method.
This uses the native Object serialization mechanism to save the ArrayList object. It doesn't write every byte array in the list one after the other. To make it clear: it writes the class name, and the internal structure of the ArrayList, using the object serialization protocol.
You don't need an ArrayList. Write directly to a ByteArrayOutputStream, or even directly to a FileOutputStream. As is, you're trying to
write bytes to a list
write the bytes in the list to a byte array
write the byte array to a file.
It would be much more straightforward (and efficient) to write directly to the output file (wrapped in a BufferedOutputStream for buffering)