How do i convert type
Stream<Object> into an InputStream? Currently, I get the iterator and loop through all of the data converting it to a byteArray and adding it to an inputStream:
ByteArrayOutputStream bos = new ByteArrayOutputStream();
ObjectOutputStream oos = new ObjectOutputStream(bos);
Iterator<MyType> myItr = MyObject.getStream().iterator();
while (myItr.hasNext()) {
oos.writeObject(myItr.next().toString()
.getBytes(StandardCharsets.UTF_8));
}
oos.flush();
oos.close();
InputStream is = new ByteArrayInputStream(bao.toByteArray());
What is the overhead of doing this though? If my stream contains a terabyte of data, wouldn't I be sucking a terabyte of data into memory? Is there any better way to achieve this?
You should be able to convert the OutputStream into an InputStream using a pipe:
PipedOutputStream pos = new PipedOutputStream();
InputStream is = new PipedInputStream(pos);
new Thread(() -> {
try (ObjectOutputStream oos = new ObjectOutputStream(pos)) {
Iterator<MyType> myItr = MyObject.getStream().iterator();
while (myItr.hasNext()) {
oos.writeObject(myItr.next().toString()
.getBytes(StandardCharsets.UTF_8));
}
} catch (IOException e) {
// handle closed pipe etc.
}
}).start();
Inspired by this answer.
Would this work for you?
https://gist.github.com/stephenhand/292cdd8bba7a452d83c51c00d9ef113c
It's an InputStream implementation that takes a Stream<byte[]> as input data. You just need to .map() your abitrary objects to byte arrays however you want each object to be represented as bytes.
It only calls a terminal operation on the Stream when the InputStream is read, pulling objects off the Stream as the consumer reads more of the InputStream so it never loads the whole set into memory
Related
I'm trying to test socket communication in Android Java, but can't seem to get a mock working.
First of all, using Mockito, mock(Socket.class) throws an Exception java.lang.VerifyError.
So I coded my mock like so:
public void testMyTest(){
final ByteArrayOutputStream os = new ByteArrayOutputStream();
final ByteArrayInputStream is = new ByteArrayInputStream(os.toByteArray());
try{
byte[] buffer = new byte[6];
os.write("poulet".getBytes());
is.read(buffer, 0, 6);
Log.w(LOG_TAG, "Read result:" + (new String(buffer, "UTF-8")));
} catch(IOException e){}
}
However is is not reading from os when I call os.write(). The raw result is [B#42204320 and, in string form, it looks like ������������. I tried commenting os.write() but nothing changed.
Does anyone know how to link an input stream to read form an output stream?
To test my classes I just called
final Socket mockedSocket1 = new Socket();
final Socket mockedSocket2 = new Socket();
when(mockedSocket1.getInputStream()).thenReturn(is);
when(mockedSocket2.getOutputStream()).thenReturn(os)
So that my classes get the linked output and input streams that I'm going to test with.
Thanks a lot!
The is's buffer will always be empty.
This: ByteArrayInputStream is = new ByteArrayInputStream(os.toByteArray()); just creates an ByteArrayInputStream with an empty buffer, that buffer won't change when you write something to the ByteArrayOutputStream.
public byte[] toByteArray()
Creates a newly allocated byte array. Its size is the current size of this output stream and the valid contents of the buffer have been copied into it.
...
What you can do is to create the ByteArrayInputStream after you write something to the ByteArrayOutputStream, eg:
try (ByteArrayOutputStream os = new ByteArrayOutputStream();){
byte[] buffer = new byte[6];
os.write("poulet".getBytes("UTF-8"));
try(ByteArrayInputStream is = new ByteArrayInputStream(os.toByteArray());){
is.read(buffer, 0, 6);
System.out.println("Read result:|" + (new String(buffer, "UTF-8") + "|"));
}
} catch (IOException e) {
}
int[] myIntArray;
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream(1024);
ObjectOutputStream objectOutputStream = new ObjectOutputStream(new DeflaterOutputStream(byteArrayOutputStream));
objectOutputStream.writeObject(myIntArray);
Now,ObjectOutputStream takes The object and directly serializes it. DeflaterOutputStream compresses the serialized result, then the compressed result is stored in a ByteArrayOutputStream
Can Someone tell me How to Deserialize and get back my original int array back?
Plz Share the coding?
objectOutputStream.close();
byte[] serialized = byteArrayOutputStream.getBytes();
// and then read back using symmetric constructs as when writing, but using
// input streams instead of output streams:
ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(serialized);
ObjectInputStream objectInputStream =
new ObjectInputStream(new InflaterInputStream(byteArrayInputStream));
int[] myDesererializedIntArray = (int[]) objectInputStream.readObject();
I am trying to test a program and for that I need to access ReadExternal function but I am getting StreamCorrupted exception on ObjectInputStream.
I know I need to use the object written by WriteObject but dont know how to do it...
ObjectOutputStream out=new ObjectOutputStream(new ByteArrayOutputStream());
out.writeObject(ss3);
ss3.writeExternal(out);
try{
ByteInputStream bi=new ByteInputStream();
bi.setBuf(bb);
out.write(bb);
ObjectInputStream in = new ObjectInputStream(new ByteArrayInputStream(bb));
String s1=(String) in.readObject();
}
catch(Exception e){
e.printStackTrace();
}
Apparently, you are trying to write the same object twice to the output stream:
out.writeObject(ss3);
ss3.writeExternal(out); // <-- Remove this!
The second write makes wrong use of the writeExternal() method, which should never be called explicitly but will be called by the ObjectOutputStream.
And: out.write(bb); tries to write the content of bb to the ObjectOutputStream. That's probably not what you want.
Try it like this:
// Create a buffer for the data generated:
ByteArrayOutputStream bos = new ByteArrayOutputStream();
ObjectOutputStream out=new ObjectOutputStream( bos );
out.writeObject(ss3);
// This makes sure the stream is written completely ('flushed'):
out.close();
// Retrieve the raw data written through the ObjectOutputStream:
byte[] data = bos.toByteArray();
// Wrap the raw data in an ObjectInputStream to read from:
ByteArrayInputStream bis = new ByteArrayInputStream( data );
ObjectInputStream in = new ObjectInputStream( bis );
// Read object(s) re-created from the raw data:
SomeClass obj = (SomeClass) in.readObject();
assert obj.equals( ss3 ); // optional ;-)
ss3.writeExternal(out);
You shouldn't be calling that method directly. You should be calling
out.writeObject(ss3);
Suppose I have an OutputStream (and not an ObjectOutputStream). Is is possible to send a serialized object using the write method? Thanks!
Here is what you do to serialize the object:
new ObjectOutputStream(outputStream).writeObject(obj);
If you want to control the byte[] output:
ByteArrayOutputStream buffer = new ByteArrayOutputStream();
ObjectOutputStream oos = new ObjectOutputStream(buffer);
oos.writeObject(obj);
oos.close();
byte[] rawData = buffer.toByteArray();
You could use ObjectOutputStream to 'capture' the objects data in a byte Array and send this to the OutputStream.
String s = "test";
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ObjectOutputStream oos = new ObjectOutputStream( baos );
oos.writeObject( s );
byte[] byteArray = baos.toByteArray();
for ( byte b : byteArray ) {
System.out.print( (char) b );
}
Another non generic option would be to serialize the object in a string representation e.g. CSV
This is trivial: you can simply wrap your original OutputStream in a new ObjectOutputStream, and then use the specialized methods of ObjectOutputStream:
OutputStream myOriginalOutputStream = ...;
ObjectOutputStream oos = new ObjectOutputStream(myOriginalOutputStream);
oos.writeObject(new MyObject());
oos.flush();
oos.close();
Internally, ObjectOutputStream will call the underlying OutputStream's write() method.
You must have to use ObjectOutputStream class and its methods to *serialize* objects. In fact ObjectOutputStream is a sub-class of java.io.OutputStream (It is an abstract super class of byte-oriented streams). Take a look at an article on Java Serialization API.
EDIT:
You can use XMLEncoder
(from the Doc : The XMLEncoder class is a complementary alternative to
the ObjectOutputStream and can used to generate a textual
representation of a JavaBean in the same way that the
ObjectOutputStream can be used to create binary representation of
Serializable objects)
I'm doing some socket programming in Java and I'd like to be able to change between using the ObjectOutputStream, the DataOutputStream, and the PrintWriter all within the same socket/connection. Is this possible and what is the best way to do it?
I've tried just creating both types of objects, for example ObjectOutputStream and DataOutputStream, but that doesn't seem to work.
The reason I want to switch between them is to, for example, send a text command "INFO" that signals I'm about to send an object with information or a command "DATA" signalling that I'm about to send data. Any advice on the best way to do this is appreciated.
You can only use one underlying stream type however you can get that data from anywhere.
DataOutputStream dos = new DataOutputStream(new BufferedOutputStream(s.getOutputStream()));
public static void writeObject(DataOutputStream dos, Serializable obj) {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ObjectOutputStream oos = new ObjectOutputStream(baos);
oos.writeObject(obj);
oos.close();
dos.writeUTF("OBJECT");
byte[] bytes = baos.toByteArray();
dos.writeInt(bytes.length);
dos.write(bytes);
dos.flush();
}
public static void writeBytes(DataOutputStream dos, byte[] bytes) {
dos.writeUTF("BYTES");
dos.writeInt(bytes.length);
dos.write(bytes);
dos.flush();
}
public static void writeText(DataOutputStream dos, String text) {
dos.writeUTF("TEXT");
dos.writeUTF(text);
dos.flush();
}
Why do you want the *Stream to convert to the *Writer.
You can do what you want to do with *Stream.
Socket s = new Socket();
DataOutputStream stream = new DataOutputStream( s.getOutputStream() );
byte[] bytes = "INFO".getBytes();
stream.write(bytes);
//....
bytes = "DATA".getBytes();
stream.write(bytes);