I wrote a file using Java's FileChannel class that uses RandomAccessFiles. I wrote objects at various locations in the file. The objects were of variable sizes but all of the same class. I wrote the objects using the following idea :
ByteArrayOutputStream bos= new ByteArrayOutputStream();
ObjectOutput out = new ObjectOutputStream(bos);
out.writeObject(r);
byte[] recordBytes= bos.toByteArray();
ByteBuffer rbb= ByteBuffer.wrap(recordBytes);
while(rbb.hasRemaining()) {
fileChannel.write(rbb);
}
Now I want to read from such a file. I dont want to have to specify the number of bytes to read. I want to be able to read the object directly using Object Input Stream. How to achieve this ?
I have to use Random Access Files because I need to write to different positions in file. I am also recording in a separate data structure, the locations where objects have been written.
I have to use Random Access Files because I need to write to different
positions in file.
No, you don't. You can reposition a FileOutputStream or FileInputStream via its channel.
That would significantly simplify your writing code as well: you wouldn't need to use the buffer or channel, and depending on your needs you could omit the ByteArrayOutputStream as well. However, as you note in a comment, you won't know the size of the object in advance, and the ByteArrayOutputStream is a useful way to verify that you don't overrun your allotted space.
Object obj = // something
FileOutputStream fos = // an initialized stream
ByteArrayOutputStream bos = new ByteArrayOutputStream();
ObjectOutputStream oos = new ObjectOutputStream(bos);
oos.writeObject(obj);
oos.flush();
if (bos.size() > MAX_ALLOWED_SIZE)
throw // or log, or whatever you want to do
else
{
fos.getChannel().position(writeLocation);
bos.writeTo(fos);
}
To read the objects, do the following:
FileInputStream fis = // an initialized stream
fis.getChannel().position(offsetOfSerializedObject);
ObjectInputStream iis = new ObjectInputStream(new BufferedInputStream(fis));
Object obj = iis.readObject();
One comment here: I wrapped the FileInputStream in a BufferedInputStream. In this specific case, where the file stream is repositioned before each use, that can provide a performance benefit. Be aware, however, that the buffered stream can read more bytes than are needed, and there are some situations using construct-as-needed object streams where it would be a really bad idea.
Why doesn't seek work for you? I believe you need to seek() to correct locations and then just read objects using your object stream. Also, if you store the correct locations of serialized objects, why don't you store their sizes? In this case you may apply ObjectInputStream against bytes you read from file.
The simplest solution that comes to mind is to write out the length of the array before writing out the array itself:
while(rbb.hasRemaining()) {
fileChannel.writeLong(recordBytes.length);
fileChannel.write(rbb);
}
When reading the object, you first read the length. This'll tell you how many further bytes to read to get your object. Similarly to what you are already doing on the writing side, you could read the data into a byte[] and then use ByteArrayInputputStream and ObjectInputStream.
You could use a FileInputStream constructed on the RandomAccesFile's FileDescriptor object, like so:
FileDescriptor f = raf.getFD();
ObjectInputStream ois = new ObjectInputStream(new FileInputStream(f));
Assuming that the RandomAccessFile is called raf.
Related
There are many different classes in the IO package, some seems to only have suble differences.
If you are to write some Strings to a text file, and read them for later use.
Which classes would you use for each of those two purposes and why?
BufferedInputStream
BufferedOutputStream
BufferedWriter
BufferedReader
DataInputStream
DataOutputStream
FileInputStream
FileOutputStream
FileReader
FileWriter
InputStreamReader
OutputStreamReader
Reader
Writer
It all depends on what you want to do.
Output Streams
In case you want to write byte data, you would use some subclass of OutputStream that is used to write bytes in destination defined by concrete subclass.
For example FileOutputStream is used to write data in file byte by byte (or bunch of bytes).
BufferedOutputStream is an optimized extension of FileOutputStream to write blocks of bytes to minimize number of disk operations. So when you call write, actually it may or may not be written in file - depends on buffer's state whether it is full or not. If buffer reaches it's max capacity, all bytes are flushed to file at once.
ByteArrayOutputStream is using just block of memory as a destination instead of file. Current stream content is obtained via toByteArray() method (why would you need such a stream will be explained later).
ObjectOutputStream is used to write objects (class instances) in some destination defined by the underlying OutputStream. For example, ObjectOutputStream(FileOutputStream) would write an objects in file, while ObjectOutputStream(ByteArrayOutputStream) would write an objects in memory. The last option allows you to serialize objects in byte buffer (array of bytes) which can be then send somewhere via network.
Note, that any object you want to write somewhere via ObjectOutputStream has to implement Serializable interface. Since object may contain references to another objects, all the objects that are accessible from the object to be serialized also has to implement Serializable interface as serialization, by default, is process of writting of full graph of objects - assuming you are not using transient keyword to exclude class field from serialization, or you are not defining at object's class a special methods writeObject/readObject that overrides default serialization behaviour. These methods are designed to implement custom serialization i. e. you can define on your own how to write/read class field in/from ObjectOutputStream/ObjectInputStream. Suppose class A for which you are implementing custom serialization contains an object of class B. Instead of writting object b as a full graph, i. e. instead of calling oos.writeObject(b), where oos is an argument of the writeObject method being implemented in class A, you would write in stream only some fields of class B e. g. oos.writeBoolean(b.isEnabled), oos.writeInt(b.value). As long as object b is not written via oos.writeObject(b), it does not have to implement Serializable interface. For more details see Serializable documentation.
DataOutputStream is used to write primitives like boolean, char, int, long, float, double. Since any object can be decomposed to primitives e. g. class A { int i = 0; B b = new B();}, class B {double d = 0.0;} can be written simply as int a, double d, you can use DataOutputStream to serialize objects in compressed way unlike serialization which writes much more data e. g. class path, class version, ... so ClassLoader can identify a given class in runtime, instantiate this class and finally load data to this empty object. Note that instantiating class by it's name + initialising object in this way is much slower than instantiating class on your own and initializing it's fields on your own from "raw" DataOutputStream (that's why OS Android prefers custom Parcelable interface over standard serialization, which works as writting/reading to/from DataOutputStream/DataInputStream).
In case you want to write string data, you would use OutputStreamWriter a higher level of output-stream which writes characters/Strings in destination defined by OutputStream (in case of writting in file, you would pass FileOutputStream, in case of writtin in memory you would pass ByteArrayOutputStream).
FileWriter is an extension of OutputStreamWriter designed for writting in file.
BufferedWriter works the same as BufferedOutputStream except it is designed for work with strings and characters.
CharArrayWriter works the same as ByteArrayOutputStream except characters are stored in char[] which is obtained by calling getCharArray().
StringWriter is similar to CharArrayWriter except it allows you to write strings which are stored in StringBuffer that is obtained by calling getBuffer().
PrintWriter allows you to write formatted strings in specified destination defined by OutputStream (this writer is designed for easy logging).
Input Streams
The same concept is applied to input-streams (readers) associated with given output-streams (writers) mentioned above.
Readers and InputStreams are for input while Writers and OutputStreams are for output.
Readers and Writers are for text (with a character set) while InputStreams and OutputStreams are for binary data (such as images).
Buffering helps with performance because reads and writes are batched rather than making a system call each time any method is used.
If it has "File" in the name, it is used for files.
So for your specific case you want to write text to a file, the most obvious candidate is a FileWriter:
Writer out = new FileWriter(
new File("foo.txt")
);
If you want better performance for making many small writes to it, you could wrap it in a buffer:
Writer out = new BufferedWriter(
new FileWriter(
new File("foo.txt")
)
);
The FileWriter assumes the default character encoding when writing text. If you want to choose the encoding, or ensure that it is consistent on all platforms, you could create the writer from an OutputStream specifying the encoding explicitly:
Writer out = new BufferedWriter(
new OutputStreamWriter(
new FileOutputStream(
new File("foo.txt")
), "UTF-8"
)
);
I am trying to convert an OutputStream to a ByteArrayOutput Stream. I was unable to find any clear simple answers on how to do this. This question was asked in the title of the question on StackOverflow, but the body of the question aske how to change a ByteArrayStream to OuputStream. I have an OutputStream that is already created and this example given in the answer will not compile!
That Question is Here
I have an OutputStream that is already constructed and has a length of 44 bytes called waveHeader. I want to convert that to a ByteArrayOutputStream because I want to be able to change that into a byte[] with waveHeader.ToByteArray() for simplicity in later processes;
Is there a simple type of casting or something that will allow this?
If not then:
Is there a way to construct a pointer to the data in the original OutputStream if it is not possible to convert it?
How would someone go about accessing the data that is contained in the OutputStream?
I am new to JAVA. This is just a hobby for me. Streams In VisualBasic .net where much easier!
There are multiple possible scenarios:
a) You have a ByteArrayOutputStream, but it was declared as OutputStream. Then you can do a cast like this:
void doSomething(OutputStream os)
{
// fails with ClassCastException if it is not a BOS
ByteArrayOutputStream bos = (ByteArrayOutputStream)os;
...
b) if you have any other type of output stream, it does not really make sense to convert it to a BOS. (You typically want to cast it, because you want to access the result array). So in this case you simple set up a new stream and use it.
void doSomething(OutputStream os)
{
ByteArrayOutputStream bos = new ByteArrayOutputStream();
bos.write(something);
bos.close();
byte[] arr = bos.toByteArray();
// what do you want to do?
os.write(arr); // or: bos.writeTo(os);
...
c) If you have written something to any kind of OutputStream (which you do not know what it is, for example because you get it from a servlet), there is no way to get that information back. You must not write something you need later. A solution is the answer b) where you write it in your own stream, and then you can use the array for your own purpose as well as writing it to the actual output stream.
Keep in mind ByteArrayOutputStreams keep all Data in Memory.
You could use the writeTo method of ByteArrayOutputStream.
ByteArrayOutputStream bos = new ByteArrayOutputStream();
byte[] bytes = new byte[8];
bos.write(bytes);
bos.writeTo(oos);
You can create an instance of ByteArrayOutputStream. You then need to write the data to this ByteOutputStream instance and then using the writeTo method, which accepts an OutputStream, you can enable the ByteArrayOutputStream to write the output, to the instance of OutputStream which you passed as the argument.
Hope it works!
You can use toByteArray function on the output stream you have.That's is let say you have outputStream buffer So you can do buffer.toByteArray .
For more you can look at the answer of Convert InputStream to byte array in Java .
I am new to Java. I want to learn to use GZIPstreams. I already have tried this:
ArrayList<SubImage>myObject = new ArrayList<SubImage>(); // SubImage is a Serializable class
ObjectOutputStream compressedOutput = new ObjectOutputStream(
new BufferedOutputStream(new GZIPOutputStream(new FileOutputStream(
new File("....")))));
compressedOutput.writeObject(myObject);
and
ObjectInputStream compressedInput = new ObjectInputStream(
new BufferedInputStream(new GZIPInputStream(new FileInputStream(
new File("....")))));
myObject=(ArrayList<SubImage>)compressedInput.readObject();
When the program writes myObject to a file without throwing any exception, but when it reaches the line
myObject=(ArrayList<SubImage>)compressedInput.readObject();
it throws this exception:
Exception in thread "main" java.io.EOFException: Unexpected end of ZLIB input stream
How can I solve this problem?
You have to flush and close your outputstream. Otherwhise, at least, the BufferedOutputStream will not write everything to the file (it does in big chucks to avoid penalizing performance).
If you call compressedOutput.flush() and compressedOutput.close() it will suffice.
You can try writing a simple string object and checking if the file is well written.
How? If you write a xxx.txt.gz file you can open it with your preferred zip app and look at the xxx.txt. If the app complains, then the content is not full written.
Extended answer to a comment: compressing even more the data
Changing serialization
You could change the standard serialization of SubImage object if it's an object of your own. Check java.io.Serializable javadoc to know how to do it. It's pretty straightforward.
Writing just what you need
Serialization has the drawback that needs to write "it's a SubImage" just before every instance you write. It's not necessary if you know what's going to be there beforehand. So you could try to serialize it more manually.
To write your list, instead of writing an object write directly the values that conform your list. You will need just a DataOutputStream (but ObjectOutputStream is a DOS so you can use it anyway).
dos.writeInt(yourList.size()); // tell how many items
for (SubImage si: yourList) {
// write every field, in order (this should be a method called writeSubImage :)
dos.writeInt(...);
dos.writeInt(...);
...
}
// to read the thing just:
int size = dis.readInt();
for (int i=0; i<size; i++) {
// read every field, in the same order (this should be a method called readSubImage :)
dis.readInt(...);
dis.readInt(...);
...
// create the subimage
// add it to the list you are recreating
}
This method is more manual but if:
you know what's going to be written
you will not need this kind of serialization for many types
it's pretty affordable and definitively more compressed than the Serializable counterpart.
Have in mind that there are alternative frameworks to serialize objects or create string messages (XStream for xml, Google Protocol Buffers for binary messages, and so on). That frameworks could work directly to binary or writing a string that could be then written.
If your app will need more on this, or just curious, maybe you should look at them.
Alternative serialization frameworks
Just looked in SO and found several questions (and answers) addressing this issue:
https://stackoverflow.com/search?q=alternative+serialization+frameworks+java
I've found that XStream is pretty easy and straightforward to use. And JSON is a format pretty readable and succint (and Javascript compatible which could be a plus :).
I should go for:
Object -> JSON -> OutputStreamWriter(UTF-8) -> GZippedOutputStream -> FileOutputStream
It can possible to serialise an object without used ObjectOutputStream ?
Until now i find just this two solution to serialize and object :
FileOutputStream fichier = new FileOutputStream("File.ser");
ObjectOutputStream oos = new ObjectOutputStream(fichier);
stream.writeObject(m);
Or
ByteArrayOutputStream byteOutput = new ByteArrayOutputStream();
ObjectOutputStream stream = new ObjectOutputStream(byteOutput);
stream.writeObject(m);
To serialize just means to produce a stream of data from an object which, at some later time, can be used to reproduce the same object. Therefore, by definition, yes. You can write any number of alternative serialization mechanisms.
Now, would you want to do this? No, probably not. If you don't like Java's default serialization format, the externalization mechanism gives you the hooks to change it however you'd like.
I've got multiple objects stored in a file .This is regarding the ObjectInputStream. If I've got the below code:
FileInputStream fis = new FileInputStream(filename);
ObjectInputStream ois = new ObjectInputStream(fis);
Object obj1 = (Object)ois.readObject();
ois.close();
ois = new ObjectInputStream(fis);
Object obj2 = (Object)ois.readObject();
My question is : will the readObject called from the second Object stream (obj2) be the 1st or 2nd object in the file
It will infact throw an exception. Calling close on the ObjectInputStream will close the FileInputStream as well.
It depends on how you stored the objects. If you used one single ObjectOutputStream, then you better also use one single ObjectInputStream.
If you used separate streams for the output, you also should use separate streams for the input. But this is not really recommended.
For your "persistent queue", I would recommend something like this:
On the sending side:
Create a ByteArrayOutputStream, wrap an ObjectOutputStream around it.
Write the object to the OOS, and close the OOS.
Get the byte[], and write it together with a header indicating the length to your queue-stream.
On the receiving side:
read a header length from the queue stream.
read a byte[] of the given length from the queue stream.
create an ByteArrayInputStream from this array, and wrap an ObjectInputStream around it.
read one object from the OIS, close the OIS.
When you store parts of your queue, make sure to always store whole messages (i.e. the header together with the object).
Of course, it might be easier to use already existing solutions, like JMS (where you would create an ObjectMessage, and submit it to the queue).