Develop Reference

How to appends objects in byte stream using ObjectOutputStream

I want to append multiple object at the end to the same binary file using ObjectOutputStream
But when I run the following code more than one time I get the exception
java.io.StreamCorruptedException: invalid type code: AC
The question is
Is there any headers things that I should know to APPEND OBJECTS at the end of the file in java?
package sourcepackage;
import sourcepackage.persons.Person;
import sourcepackage.persons.Student;
import java.io.*;
public class MainClass {
public static void main(String[] args) {
Person mahmoud_kanbar = new Student(21, 224466, "Mahmoud Kanbar", "ITE");
try {
FileOutputStream out = new FileOutputStream("Hello.dat", true);
ObjectOutputStream objectOut = new ObjectOutputStream(out);
objectOut.writeObject(mahmoud_kanbar);
objectOut.close();
out.close();
FileInputStream in = new FileInputStream("Hello.dat");
ObjectInputStream objectIn = new ObjectInputStream(in);
while (in.available() != 0) {
Person hi = (Person) objectIn.readObject();
hi.printInfo();
}
objectIn.close();
in.close();
} catch (Exception e) {
System.out.println(e);
}
}
}
I was searching for decades about a solution to this problem and I couldn't find anything
I want to append objects just like the c++ do

You can append objects to the same ObjectOutputStream with writeObject() method. When reading, they are read in the same order they were written. Also, you may be getting that error because your stream is buffered and not written. You should use flush() method to make sure the buffer has been written to the file.
Let's write two objects to the same file and read them:
Person mahmoud_kanbar = new Student(21, 224466, "Mahmoud Kanbar", "ITE");
Person omid = new Student(18, 200000, "Omid Nejadabbasi", "ITE");
try {
FileOutputStream out = new FileOutputStream("Hello.dat", true);
ObjectOutputStream objectOut = new ObjectOutputStream(out);
objectOut.writeObject(mahmoud_kanbar);
objectOut.writeObject(omid);
objectOut.flush();
objectOut.close();
out.close();
FileInputStream in = new FileInputStream("Hello.dat");
ObjectInputStream objectIn = new ObjectInputStream(in);
Person newPerson= (Person) objectIn.readObject();
newPerson.printInfo();
newPerson= (Person) objectIn.readObject();
newPerson.printInfo();
objectIn.close();
in.close();
} catch (Exception e) {
System.out.println(e);
}
readObject() deserializes the next Object serialized into the stream.

Saving and loading an Arraylist of classes via ObjectInputStream

I'm currently trying to save and load multiple instances of a class of type 'Player' via serialization. (This is for an assignment as an fyi).
The way the process currently works is I run the class via an ObjectOutputStream.writeObject(Player), this seems to work as I get the text file with what appears to be class data.
The problem I'm currently having is when I reverse and load the class, I'm unable to cast the object back to a player object, it is throwing an exception, however I'm not currently knowledgeable enough to figure out the exact exception.
Please find below the functions.
Save function:
public void savePlayers(ArrayList<Player> p)
{
FileOutputStream fout = null;
ObjectOutputStream oos = null;
try
{
fout = new FileOutputStream("Player.txt");
oos = new ObjectOutputStream(fout);
for(Player player: p)
{
oos.writeObject(p);
}
fout.close();
oos.close();
}
catch(Exception e)
{
System.out.println("Error is: " + e);
}
finally
{
closeStreams(fout, oos);
}
}
Load Function:
public ArrayList<Player> loadPlayers()
{
ArrayList<Player> loadedList = new ArrayList<Player>();
FileInputStream fis;
ObjectInputStream ois;
try
{
fis = new FileInputStream("Player.txt");
ois = new ObjectInputStream(fis);
while(true)
{
System.out.println("Entered loop"); //testing
Object obj = ois.readObject();
System.out.println("Read object done"); //testing
Player p = (Player)obj;
System.out.println("Casted player"); //The line here doesn't run, prints out error from exception instead and exits loop
loadedList.add(p);
System.out.println("Loop iterated");
}
}
catch(EOFException eof)
{
}
catch(Exception e)
{
System.out.println("Exception here");
}
}
Is there a better way to save the classes? Am I doing something wrong when casting?

change to
oos.writeObject(player);
and the moment you are writing the whole ArrayList multiple times

ObjectInputStream doesn't have available bytes after being constructed with a ByteArrayInputStream

I'm constructing a class that is handling a Binary De/Serialization. The method open() receives an InputStream and a OutputStream. Those are created by another open() method that receives a path as argument. The InputStream is actually a ByteArrayInputStream.
I already did some tests to prove that the InputStream is arriving at the open() method with content - and it actually is. But when I try to set a ObjectInputStream using it, it doesn't work. No exceptions are thrown, but when I try to read bytes from it, it always gives me -1.
BinaryStrategy class
public class BinaryStrategy implements SerializableStrategy{
public BinaryStrategy(){
try{
open("products.ser");
}catch(IOException ioe){
}
}
#Override
public void open(InputStream input, OutputStream output) throws IOException {
try{
this.ois = new ObjectInputStream(input);
}catch(Exception ioe){
System.out.println(ioe);
}
this.oos = new ObjectOutputStream(output);
}
#Override
public void writeObject(fpt.com.Product obj) throws IOException {
oos.writeObject(obj);
oos.flush();
}
#Override
public Product readObject() throws IOException {
Product read = new Product();
try{
read.readExternal(ois);
}catch(IOException | ClassNotFoundException exc){
System.out.println(exc);
}
return read;
}
}
interface SerializableStrategy (just the default method)
default void open(Path path) throws IOException {
if (path != null) {
ByteArrayInputStream in = null;
if (Files.exists(path)) {
byte[] data = Files.readAllBytes(path);
in = new ByteArrayInputStream(data);
}
OutputStream out = Files.newOutputStream(path);
open(in, out);
}
Product class
public class Product implements java.io.Externalizable {
#Override
public void writeExternal(ObjectOutput out) throws IOException {
out.writeLong(getId());
out.writeObject(getName());
out.writeObject(getPrice());
out.writeObject(getQuantity());
}
#Override
public void readExternal(ObjectInput in) throws IOException, ClassNotFoundException {
this.setId((Long)in.readLong());
this.setName((String) in.readObject());
this.setPrice((Double) in.readObject());
this.setQuantity((Integer) in.readObject());
}
I had to personalize it because the attributes are SimplePropertys
At public void open(InputStream input, OutputStream output) I tried to do some stuff as follow to test:
public void open(InputStream input, OutputStream output) throws IOException {
try{
System.out.println(input.available() + " " + input.read() + " " + input.read());
//is gives me: 181 172 237
//181 is the exact size of the file I have, so i think that the Output is ok
//172 237 - just some chars that are in the file
//I know that for now on it is going to give me an excepetion because
// of the position of the index that is reading. I did it just to test
this.ois = new ObjectInputStream(input);
}catch(Exception ioe){
System.out.println(ioe);
}
this.oos = new ObjectOutputStream(output);
}
And then the other test:
public void open(InputStream input, OutputStream output) throws IOException {
try{
this.ois = new ObjectInputStream(input);
System.out.println(ois.available() + " " + ois.read());
//here is where I am receiving -1 and 0 available bytes!
//so something is going wrong right here.
//i tried to just go on and try to read the object,
//but I got a EOFException, in other words, -1.
}catch(Exception ioe){
System.out.println(ioe);
}
this.oos = new ObjectOutputStream(output);
}

Please check if the file represented by the path has a java object written to it.
From the ObjectInputStream API doc https://docs.oracle.com/javase/7/docs/api/java/io/ObjectInputStream.html
An ObjectInputStream deserializes primitive data and objects previously written using an ObjectOutputStream.
ObjectInputStream is used to recover those objects previously serialized.
If you are doing a this.ois.readObject(), and you are getting a -1, there are chances that the file does not contain an object in it.
Update: readObject returns an object and not an int. If you are using the read methods in ois, and you are getting a -1, then the file is empty.
Also, there are chances that your file contains -1 as its content ;)

ObjectInputStream, internally uses a BlockDataInputStream perform its read operations. This reads a block of data and not just a byte as we expect, when you call a read. It reads a byte only if it falls as a "block"
The output is not what I was expecting either.
But, if you look at the code of ObjectInputStream.read(), it makes sense.
So, in your case it makes sense to use only readObject to restore your objects' state.
Heres your code again...
class SimpleJava {
public static void open(InputStream input, OutputStream output) throws IOException {
try {
ObjectInputStream ois = new ObjectInputStream(input);
System.out.println(ois.available());// 0
System.out.println(ois.available() + " " + ois.read() + " " + ois.read());// 0 -1 -1
// Reads the object even if the available returned 0
// and ois.read() returned -1
System.out.println("object:" + ois.readObject());// object:abcd
}
catch (Exception ioe) {
ioe.printStackTrace();
}
}
static void open(Path path) throws IOException {
if (path != null) {
ByteArrayInputStream in = null;
if (Files.exists(path)) {
byte[] data = Files.readAllBytes(path);
in = new ByteArrayInputStream(data);
}
OutputStream out = Files.newOutputStream(path);
open(in, out);
}
}
public static void main(String[] args) throws Exception {
ObjectOutputStream oos = new ObjectOutputStream(new FileOutputStream(new File("/home/pradhan/temp.object")));
oos.writeObject("abcd");//writes a string object for us to read later
oos.close();
//
open(FileSystems.getDefault().getPath("/home/user/temp.object"));
}
}
Heres the output...
0
0 -1 -1
object:abcd

The problem was that I was reading the ObjectInputStream the wrong way. It was like:
read.readExternal(ois);
but the correct way is:
read = (Product)ois.readObject();
And beacause of the Exceptions I was getting for doing so, I thought that the problem was with the construction of ObjectInputStream when using ByteArrayInputStream.
What a big mistake! :D
Thanks to everybody that tried to help.

How to write and read java serialized objects into a file

I am going to write multiple objects to a file and then retrieve them in another part of my code. My code has no error, but it is not working properly. Could you please help me find what is wrong about my code.
I read different codes from different websites, but none of them worked for me!
Here is my code to write my objects to a file:
MyClassList is an arraylist which includes objects of my class (which must be written to a file).
for (int cnt = 0; cnt < MyClassList.size(); cnt++) {
FileOutputStream fout = new FileOutputStream("G:\\address.ser", true);
ObjectOutputStream oos = new ObjectOutputStream(fout);
oos.writeObject(MyClassList.get(cnt));
}
I added "true" to the constructor of the outputstream, because I want to add each object to end of the file. Is that correct?
And here is my code to read the objects from the file:
try {
streamIn = new FileInputStream("G:\\address.ser");
ObjectInputStream objectinputstream = new ObjectInputStream(streamIn);
MyClass readCase = (MyClass) objectinputstream.readObject();
recordList.add(readCase);
System.out.println(recordList.get(i));
} catch (Exception e) {
e.printStackTrace();
}
It finally prints out just one object. Now, I don't know if I am not writing correctly or reading correctly!

Why not serialize the whole list at once?
FileOutputStream fout = new FileOutputStream("G:\\address.ser");
ObjectOutputStream oos = new ObjectOutputStream(fout);
oos.writeObject(MyClassList);
Assuming, of course, that MyClassList is an ArrayList or LinkedList, or another Serializable collection.
In the case of reading it back, in your code you ready only one item, there is no loop to gather all the item written.

As others suggested, you can serialize and deserialize the whole list at once, which is simpler and seems to comply perfectly with what you intend to do.
In that case the serialization code becomes
ObjectOutputStream oos = null;
FileOutputStream fout = null;
try{
fout = new FileOutputStream("G:\\address.ser", true);
oos = new ObjectOutputStream(fout);
oos.writeObject(myClassList);
} catch (Exception ex) {
ex.printStackTrace();
} finally {
if(oos != null){
oos.close();
}
}
And deserialization becomes (assuming that myClassList is a list and hoping you will use generics):
ObjectInputStream objectinputstream = null;
try {
FileInputStream streamIn = new FileInputStream("G:\\address.ser");
objectinputstream = new ObjectInputStream(streamIn);
List<MyClass> readCase = (List<MyClass>) objectinputstream.readObject();
recordList.add(readCase);
System.out.println(recordList.get(i));
} catch (Exception e) {
e.printStackTrace();
} finally {
if(objectinputstream != null){
objectinputstream .close();
}
}
You can also deserialize several objects from a file, as you intended to:
ObjectInputStream objectinputstream = null;
try {
streamIn = new FileInputStream("G:\\address.ser");
objectinputstream = new ObjectInputStream(streamIn);
MyClass readCase = null;
do {
readCase = (MyClass) objectinputstream.readObject();
if(readCase != null){
recordList.add(readCase);
}
} while (readCase != null)
System.out.println(recordList.get(i));
} catch (Exception e) {
e.printStackTrace();
} finally {
if(objectinputstream != null){
objectinputstream .close();
}
}
Please do not forget to close stream objects in a finally clause (note: it can throw exception).
EDIT
As suggested in the comments, it should be preferable to use try with resources and the code should get quite simpler.
Here is the list serialization :
try(
FileOutputStream fout = new FileOutputStream("G:\\address.ser", true);
ObjectOutputStream oos = new ObjectOutputStream(fout);
){
oos.writeObject(myClassList);
} catch (Exception ex) {
ex.printStackTrace();
}

Simple program to write objects to file and read objects from file.
package program;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;
import java.io.Serializable;
public class TempList {
public static void main(String[] args) throws Exception {
Counter counter = new Counter(10);
File f = new File("MyFile.txt");
FileOutputStream fos = new FileOutputStream(f);
ObjectOutputStream oos = new ObjectOutputStream(fos);
oos.writeObject(counter);
oos.close();
FileInputStream fis = new FileInputStream(f);
ObjectInputStream ois = new ObjectInputStream(fis);
Counter newCounter = (Counter) ois.readObject();
System.out.println(newCounter.count);
ois.close();
}
}
class Counter implements Serializable {
private static final long serialVersionUID = -628789568975888036 L;
int count;
Counter(int count) {
this.count = count;
}
}
After running the program the output in your console window will be 10 and you can find the file inside Test folder by clicking on the icon show in below image.

I think you have to write each object to an own File or you have to split the one when reading it.
You may also try to serialize your list and retrieve that when deserializing.

if you serialize the whole list you also have to de-serialize the file into a list when you read it back. This means that you will inevitably load in memory a big file. It can be expensive. If you have a big file, and need to chunk it line by line (-> object by object) just proceed with your initial idea.
Serialization:
LinkedList<YourObject> listOfObjects = <something>;
try {
FileOutputStream file = new FileOutputStream(<filePath>);
ObjectOutputStream writer = new ObjectOutputStream(file);
for (YourObject obj : listOfObjects) {
writer.writeObject(obj);
}
writer.close();
file.close();
} catch (Exception ex) {
System.err.println("failed to write " + filePath + ", "+ ex);
}
De-serialization:
try {
FileInputStream file = new FileInputStream(<filePath>);
ObjectInputStream reader = new ObjectInputStream(file);
while (true) {
try {
YourObject obj = (YourObject)reader.readObject();
System.out.println(obj)
} catch (Exception ex) {
System.err.println("end of reader file ");
break;
}
}
} catch (Exception ex) {
System.err.println("failed to read " + filePath + ", "+ ex);
}

Appending objects to a binary file

Supose you have the following method:
public static void writeToBinary(Object obj, String filename)
{
ObjectOutputStream oos = null;
try {
oos = new ObjectOutputStream(new FileOutputStream(filename));
oos.writeObject(obj);
} catch (Exception e) {
e.printStackTrace();
} finally{
try{
if (oos != null) oos.close ();
}catch (Exception e){
e.printStackTrace();
}
}
}
As you can see, the method writes an object to a binary file.
But now you want to rewrite the same method to allow appending objects to the same file.
Ok, you look at the java documentation and you see that you have to add a parameter with value true to the FileOutputStream:
oos = new ObjectOutputStream(new FileOutputStream(filename, true));
You compile but, whoops!, it seems that it continues overriding the file.
Well, the problems begin. After searching in google you read that you have to use the SAME ObjectOutputStream to append objects to the same file. You want to have a function that every time you call it, it appends an object. I.e. :
writeToBinary("a", filename);
writeToBinary("b", filename);
But as I said before, you have to use the same ObjectOutputStream.
Solution 1:
ObjectOutputStream out = getOutputStream (filename);
writeToBinary("a", out);
writeToBinary("b", out);
writeToBinary("c", out);
out.close ();
This is very ugly because I want to hide the usage of streams.
Is there any other solution?
EDIT: The method is static. It is inside an utility class where all methods are static.
EDIT2: SOLVED! Appending to an ObjectOutputStream. See accepted answer to my question.
Thanks.

Solved.
import java.io.EOFException;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;
import java.io.OutputStream;
public class Test{
private static String filename = "test";
public static void main(String[] args) {
writeToBinary (filename, "a", true);
writeToBinary (filename, "b", true);
writeToBinary (filename, "c", true);
readFromBinaryFile (filename);
}
public static void writeToBinary (String filename, Object obj, boolean append){
File file = new File (filename);
ObjectOutputStream out = null;
try{
if (!file.exists () || !append) out = new ObjectOutputStream (new FileOutputStream (filename));
else out = new AppendableObjectOutputStream (new FileOutputStream (filename, append));
out.writeObject(obj);
out.flush ();
}catch (Exception e){
e.printStackTrace ();
}finally{
try{
if (out != null) out.close ();
}catch (Exception e){
e.printStackTrace ();
}
}
}
public static void readFromBinaryFile (String filename){
File file = new File (filename);
if (file.exists ()){
ObjectInputStream ois = null;
try{
ois = new ObjectInputStream (new FileInputStream (filename));
while (true){
String s = (String)ois.readObject ();
System.out.println (s);
}
}catch (EOFException e){
}catch (Exception e){
e.printStackTrace ();
}finally{
try{
if (ois != null) ois.close();
}catch (IOException e){
e.printStackTrace ();
}
}
}
}
private static class AppendableObjectOutputStream extends ObjectOutputStream {
public AppendableObjectOutputStream(OutputStream out) throws IOException {
super(out);
}
#Override
protected void writeStreamHeader() throws IOException {}
}
}

oos = new ObjectOutputStream(new FileOutputStream(filename, true)); You
compile but, whoops!, it seems that it
continues overriding the file.
That does not make sense. The FileOutputStream is a streams that appends to the existing file, so it will not overwite the file. Check it.
The problem is that a stream cannot be closed and reopened to serialize several objects. Run the following and compare the resulting files to check it.
public class XX {
public static void writeToBinary(Object obj, String filename) throws Exception {
ObjectOutputStream oos = new ObjectOutputStream(new FileOutputStream(filename,true));
oos.writeObject(obj);
oos.close();
}
public static void writeToBinary2(Object obj1, Object obj2,String filename) throws Exception {
ObjectOutputStream oos = new ObjectOutputStream(new FileOutputStream(filename,true));
oos.writeObject(obj1);
oos.writeObject(obj2);
oos.close();
}
public static void main(String[] args) throws Exception {
String s1= "hi, just trying";
String s2= "bye bye cruel world";
String filename = "/temp/f.dat";
String filename2 = filename + ".2" ;
writeToBinary(s1, filename);
writeToBinary(s2, filename);
writeToBinary2(s1, s2,filename2);
ObjectInputStream fin = new ObjectInputStream(new FileInputStream(filename)); // oops... works with filename2
Object x1 = fin.readObject();
Object x2 = fin.readObject();
System.out.println(x1);
System.out.println(x2);
}
}

Write a helper class. In constructor it will instantiate an output stream for a particular file name. Then using some append() or writeToBinary() method it will append the data. on method close() there will be flush() and close() calls on the stream.
BinaryWriteHelper helper = new BinaryWriteHelper("test.dat");
helper.writeToBinary("1");
helper.writeToBinary(2);
helper.close();
in BinaryWriteHelper :
public BinaryWriteHelper(String filename) {
this.stream = new ObjectOutputStream(new FileOutputStream(filename));
}
public close() {
// the cleanup here
}

Try this approach:
Write the object to a ByteArrayOutputStream.
Append the the size and contents of the ByteArrayOutputStream to a RandomAccessFile.
To load an object from the file, read the bytes that represent an Object into a ByteArrayInputStream and initialize an ObjectInputStream on this. The size field that was prepends each object byte sequence will come handy here.