Develop Reference

ObjectInputStream header problems [duplicate]

I am trying to convert a ArrayList object to a byte string so it can be sent via sockets. When I run this code it converts to a string properly but when I try to convert it back I get the exception "java.io.StreamCorruptedException: invalid stream header: EFBFBDEF". Other answers I looked at on here didn't really help as I am using the matching ObjectOutputStream and ObjectInputStream. Sorry if there is a simple fix as I am new to working with stream objects.
try {
ArrayList<String> text = new ArrayList<>();
text.add("Hello World!");
String byteString = Utils.StringUtils.convertToByteString(text);
ArrayList<String> convertedSet = (ArrayList<String>) Utils.StringUtils.convertFromByteString(byteString);
VCS.getServiceManager().addConsoleLog(convertedSet.get(0));
} catch (IOException | ClassNotFoundException e) {
e.printStackTrace();
}
public static String convertToByteString(Object object) throws IOException {
try (ByteArrayOutputStream bos = new ByteArrayOutputStream(); ObjectOutput out = new ObjectOutputStream(bos)) {
out.writeObject(object);
final byte[] byteArray = bos.toByteArray();
return new String(byteArray);
}
}
public static Object convertFromByteString(String byteString) throws IOException, ClassNotFoundException {
final byte[] bytes = byteString.getBytes();
try (ByteArrayInputStream bis = new ByteArrayInputStream(bytes); ObjectInput in = new ObjectInputStream(bis)) {
return in.readObject();
}
}

I figured it out. I had to use Base64 encoding. The conversion methods have to be changed to the following:
public static String convertToByteString(Object object) throws IOException {
try (ByteArrayOutputStream bos = new ByteArrayOutputStream(); ObjectOutput out = new ObjectOutputStream(bos)) {
out.writeObject(object);
final byte[] byteArray = bos.toByteArray();
return Base64.getEncoder().encodeToString(byteArray);
}
}
public static Object convertFromByteString(String byteString) throws IOException, ClassNotFoundException {
final byte[] bytes = Base64.getDecoder().decode(byteString);
try (ByteArrayInputStream bis = new ByteArrayInputStream(bytes); ObjectInput in = new ObjectInputStream(bis)) {
return in.readObject();
}
}

String is not a container for binary data. You need to pass around the original byte array, or hex- or base64-encode it.
Better still, serialize directly to the socket and get rid of this altogether.

Unable to serialize convert java POJO class into a byte array

How to convert java POJO class into a byte array as I wanted to save the object into a gz file in S3
I get this exception Caused by: java.io.NotSerializableException
public byte[] compressData(User user) throws IOException {
byte[] data;
try(ByteArrayOutputStream byteStream = new ByteArrayOutputStream();) {
try (GZIPOutputStream objectOutputStream = new GZIPOutputStream(byteStream);) {
try (ObjectOutputStream zipStream = new ObjectOutputStream(objectOutputStream);) {
zipStream.writeObject(user);
}
data = byteStream.toByteArray();
} catch (Exception e) {
throw new IOException(e);
}
}
return data;
}

you can use SerializationUtils.java from ApacheCommonslang dependency.
For serialization
byte[] data = SerializationUtils.serialize(**POJO_Object_Name**);
for deserialize:
POJO_Class_Name **POJO_Object_Name** = SerializationUtils.deserialize(data)

private final Type userType = new TypeToken<User>() {}.getType();
private final Gson = new Gson();
compressData(gson.toJson(user,userType));
public static byte[] compressData(String user) throws IOException {
byte[] data;
try(ByteArrayOutputStream byteStream = new ByteArrayOutputStream();){
try(GZIPOutputStream zipStream = new GZIPOutputStream(byteStream);){
zipStream.write(data.getBytes(StandardCharsets.UTF_8));
}
data = byteStream.toByteArray();
} catch(Exception e) {
throw new IOException("Error while compressing the User record ", e);
}
return data;
}

Is it possible to convert a custom ArrayList to an to byte array? [duplicate]

Let's say I have a serializable class AppMessage.
I would like to transmit it as byte[] over sockets to another machine where it is rebuilt from the bytes received.
How could I achieve this?

Prepare the byte array to send:
ByteArrayOutputStream bos = new ByteArrayOutputStream();
ObjectOutputStream out = null;
try {
out = new ObjectOutputStream(bos);
out.writeObject(yourObject);
out.flush();
byte[] yourBytes = bos.toByteArray();
...
} finally {
try {
bos.close();
} catch (IOException ex) {
// ignore close exception
}
}
Create an object from a byte array:
ByteArrayInputStream bis = new ByteArrayInputStream(yourBytes);
ObjectInput in = null;
try {
in = new ObjectInputStream(bis);
Object o = in.readObject();
...
} finally {
try {
if (in != null) {
in.close();
}
} catch (IOException ex) {
// ignore close exception
}
}

The best way to do it is to use SerializationUtils from Apache Commons Lang.
To serialize:
byte[] data = SerializationUtils.serialize(yourObject);
To deserialize:
YourObject yourObject = SerializationUtils.deserialize(data)
As mentioned, this requires Commons Lang library. It can be imported using Gradle:
compile 'org.apache.commons:commons-lang3:3.5'
Maven:
<!-- https://mvnrepository.com/artifact/org.apache.commons/commons-lang3 -->
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
<version>3.5</version>
</dependency>
Jar file
And more ways mentioned here
Alternatively, the whole collection can be imported. Refer this link

If you use Java >= 7, you could improve the accepted solution using try with resources:
private byte[] convertToBytes(Object object) throws IOException {
try (ByteArrayOutputStream bos = new ByteArrayOutputStream();
ObjectOutputStream out = new ObjectOutputStream(bos)) {
out.writeObject(object);
return bos.toByteArray();
}
}
And the other way around:
private Object convertFromBytes(byte[] bytes) throws IOException, ClassNotFoundException {
try (ByteArrayInputStream bis = new ByteArrayInputStream(bytes);
ObjectInputStream in = new ObjectInputStream(bis)) {
return in.readObject();
}
}

Can be done by SerializationUtils, by serialize & deserialize method by ApacheUtils to convert object to byte[] and vice-versa , as stated in #uris answer.
To convert an object to byte[] by serializing:
byte[] data = SerializationUtils.serialize(object);
To convert byte[] to object by deserializing::
Object object = (Object) SerializationUtils.deserialize(byte[] data)
Click on the link to Download org-apache-commons-lang.jar
Integrate .jar file by clicking:
FileName -> Open Medule Settings -> Select your module -> Dependencies -> Add Jar file and you are done.
Hope this helps.

I also recommend to use SerializationUtils tool. I want to make a ajust on a wrong comment by #Abilash. The SerializationUtils.serialize() method is not restricted to 1024 bytes, contrary to another answer here.
public static byte[] serialize(Object object) {
if (object == null) {
return null;
}
ByteArrayOutputStream baos = new ByteArrayOutputStream(1024);
try {
ObjectOutputStream oos = new ObjectOutputStream(baos);
oos.writeObject(object);
oos.flush();
}
catch (IOException ex) {
throw new IllegalArgumentException("Failed to serialize object of type: " + object.getClass(), ex);
}
return baos.toByteArray();
}
At first sight， you may think that new ByteArrayOutputStream(1024) will only allow a fixed size. But if you take a close look at the ByteArrayOutputStream, you will figure out the the stream will grow if necessary:
This class implements an output stream in which the data is
written into a byte array. The buffer automatically grows as data
is written to it.
The data can be retrieved using toByteArray() and
toString().

Another interesting method is from com.fasterxml.jackson.databind.ObjectMapper
byte[] data = new ObjectMapper().writeValueAsBytes(JAVA_OBJECT_HERE)
Maven Dependency
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
</dependency>

If you are using spring, there's a util class available in spring-core. You can simply do
import org.springframework.util.SerializationUtils;
byte[] bytes = SerializationUtils.serialize(anyObject);
Object object = SerializationUtils.deserialize(bytes);

I would like to transmit it as byte[] over sockets to another machine
// When you connect
ObjectOutputStream oos = new ObjectOutputStream(socket.getOutputStream());
// When you want to send it
oos.writeObject(appMessage);
where it is rebuilt from the bytes received.
// When you connect
ObjectInputStream ois = new ObjectInputStream(socket.getInputStream());
// When you want to receive it
AppMessage appMessage = (AppMessage)ois.readObject();

Spring Framework org.springframework.util.SerializationUtils
byte[] data = SerializationUtils.serialize(obj);

In case you want a nice no dependencies copy-paste solution. Grab the code below.
Example
MyObject myObject = ...
byte[] bytes = SerializeUtils.serialize(myObject);
myObject = SerializeUtils.deserialize(bytes);
Source
import java.io.*;
public class SerializeUtils {
public static byte[] serialize(Serializable value) throws IOException {
ByteArrayOutputStream out = new ByteArrayOutputStream();
try(ObjectOutputStream outputStream = new ObjectOutputStream(out)) {
outputStream.writeObject(value);
}
return out.toByteArray();
}
public static <T extends Serializable> T deserialize(byte[] data) throws IOException, ClassNotFoundException {
try(ByteArrayInputStream bis = new ByteArrayInputStream(data)) {
//noinspection unchecked
return (T) new ObjectInputStream(bis).readObject();
}
}
}

This is just an optimized code form of the accepted answer in case anyone wants to use this in production :
public static void byteArrayOps() throws IOException, ClassNotFoundException{
String str="123";
byte[] yourBytes = null;
// Convert to byte[]
try(ByteArrayOutputStream bos = new ByteArrayOutputStream();
ObjectOutputStream out = new ObjectOutputStream(bos);) {
out.writeObject(str);
out.flush();
yourBytes = bos.toByteArray();
} finally {
}
// convert back to Object
try(ByteArrayInputStream bis = new ByteArrayInputStream(yourBytes);
ObjectInput in = new ObjectInputStream(bis);) {
Object o = in.readObject();
} finally {
}
}

code example with java 8+:
public class Person implements Serializable {
private String lastName;
private String firstName;
public Person() {
}
public Person(String firstName, String lastName) {
this.firstName = firstName;
this.lastName = lastName;
}
public void setFirstName(String firstName) {
this.firstName = firstName;
}
public String getFirstName() {
return firstName;
}
public String getLastName() {
return lastName;
}
public void setLastName(String lastName) {
this.lastName = lastName;
}
#Override
public String toString() {
return "firstName: " + firstName + ", lastName: " + lastName;
}
}
public interface PersonMarshaller {
default Person fromStream(InputStream inputStream) {
try (ObjectInputStream objectInputStream = new ObjectInputStream(inputStream)) {
Person person= (Person) objectInputStream.readObject();
return person;
} catch (IOException | ClassNotFoundException e) {
System.err.println(e.getMessage());
return null;
}
}
default OutputStream toStream(Person person) {
try (OutputStream outputStream = new ByteArrayOutputStream()) {
ObjectOutput objectOutput = new ObjectOutputStream(outputStream);
objectOutput.writeObject(person);
objectOutput.flush();
return outputStream;
} catch (IOException e) {
System.err.println(e.getMessage());
return null;
}
}
}

How to (de)serialize correctly to a byte-array using a ObjectOutputStream & ObjectInputStream?

I am aware that an ObjectOutputStream/ObjectInputStream uses headers and this is not really a proper use-case. But anyway I need to wrap some data into it using the interfaces DataInput and DataOutput.
import java.util.*;
import java.lang.*;
import java.io.*;
class Ideone
{
public static void main (String[] args) throws java.lang.Exception
{
byte[] array = serialize("test");
String deserialized = deserialize(array);
System.out.println(deserialized);
}
private static byte[] serialize(String test) {
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
try {
ObjectOutputStream objectOutputStream = new ObjectOutputStream(byteArrayOutputStream);
objectOutputStream.writeUTF(test);
byteArrayOutputStream.close();
} catch (IOException e) {
e.printStackTrace();
}
return byteArrayOutputStream.toByteArray();
}
private static String deserialize(byte[] array) {
String temp = null;
try {
ObjectInputStream objectInputStream = new ObjectInputStream(new ByteArrayInputStream(array));
temp = objectInputStream.readUTF();
objectInputStream.close();
} catch (IOException e) {
e.printStackTrace();
}
return temp;
}
}
I don't really get how to get that working. Am I right, that the problem are those headers currently?

You should call objectOutputStream.flush(); before closing byteArrayOutputStream.
ObjectOutputStream have its internal buffer so you got only beginning of string in your byte array.

Java Binary Reading Object

I have stored a group of 4 students in ArrayList, and they each have unique ID that is generated upon creation. Example: [James, 1], [Jone, 2],
[Kate, 3], [Jack, 4]. However if I read those objects from Binary File, then add additional Person objects into the array list, the ID is reset to 0. But I want to new objects ID to append such as: [Sam, 5].
private int id;
private static int count;
public Student(String firstName, String lastName, String phone, String address) {
super();
this.firstName = firstName;
this.lastName = lastName;
id = ++count;
this.phone = phone;
this.address = address;
}
public void loadPersonBinary(){
FileInputStream fis = null;
ObjectInputStream ois = null;
try {
fis = new FileInputStream("Person.dat");
ois = new ObjectInputStream(fis);
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} catch(Exception ev) {
System.out.println("Exception!");
}
System.out.println("Reading from the file...");
try {
p = (ArrayList<Person>) ois.readObject();
//it only read once. Count always stays at 1
int count = 0;
Person.setCount(++count);
System.out.println(count);
} catch (ClassNotFoundException e1) {
e1.printStackTrace();
} catch (IOException e1) {
}
try {
ois.close();
} catch (IOException e) {
e.printStackTrace();
}
}

When you are writing to the file using ObjectOutPutStream->FileOutputStream, you have to use different API to append.
ObjectOutputStream oos = new ObjectOutputStream(new FileOutputStream("person.dat",true));
Have a look at FileOutputStream from javadocs
public FileOutputStream(String name,
boolean append)
throws FileNotFoundException
Creates a file output stream to write to the file with the specified name. If the second argument is true, then bytes will be written to the end of the file rather than the beginning. A new FileDescriptor object is created to represent this file connection.
Personally, I too prefer writing List and reading List of objects instead of writing multiple individual objects.

you must serialize your datas:
1 you must declare public Person implements Serializable
2 better declare a version UID (not mandatory)
private static final long serialVersionUID = 354054054054L;
3 just write as you did
List< Person> the_list;
OutputStream file = new FileOutputStream( "mydatas.ser" );
OutputStream buffer = new BufferedOutputStream( file );
ObjectOutput output = new ObjectOutputStream( buffer );
output.writeObject(the_list);
4 and you can reload it
InputStream file2 = new FileInputStream( "mydatas.ser" );
InputStream buffer2 = new BufferedInputStream( file2 );
ObjectInput input = new ObjectInputStream ( buffer2 );
ArrayList< Person> lp= (ArrayList< Person>) input.readObject();