In java I can transfer objects between server and client by using Object Output Stream and Object Input Stream. Is there anything equivalent in python?
Related:
python equivalent of java OutputStream?
The pickle module in Python provides object serialization and deserialization functionality. http://docs.python.org/library/pickle.html
It's not particularly secure, so you should always validate the incoming data, but it should support your needs.
The multiprocessing module has the Pipe() function that handles serializing and passing objects between processes.
http://docs.python.org/library/multiprocessing.html#multiprocessing.Pipe
example (pipes work within the same process too)
import multiprocessing
class ObjectToSend:
def __init__(self,data):
self.data = data
obj = ObjectToSend(['some data'])
#create 2 read/write ends to a pipe
a, b = multiprocessing.Pipe()
#serialize and send obj across the pipe using send method
a.send(obj)
#deserialize object at other end of the pipe using recv method
print(b.recv().data)
Related
I'm trying to send data from C# app, to Java using Named pipes.
Data is in form of a custom object.
C# writer initialization is below:
BinaryWriter writer = new BinaryWriter(new BufferedWriterStream(stream, (70 * 1024) + 512), defaultEncoding);
defaultEncoding --> UTF-8
I'm able to write data to the pipe, but on Java side, having issues reading that custom object. What kind of stream would help in this case ?
I've tried DataInputStream, ByteArrayInputStream, ObjectInputStream, but nothing helps.
I need to get that data in Java, do some processing, and send it back to C#
Consider an object in java which implements Serializable. I want to send that object to a python code on TCP socket. I am serializing object and sending on TCP by using :
//socket connection code
PySessionObject object = new PySessionObject();
object.setMethodCall(PyServerMethodConstant.SETATTRIBUTE);
object.setAttributeName(name);
object.setAttributeValue(value);// value is of object type which also implements Serializable
os = sChannel.socket().getOutputStream();
oos = new ObjectOutputStream(os);
oos.writeObject(object);
oos.flush();
os.flush();
//socket closing and exception handling
Now I want to convert this byte stream into object in python perform some operation on that object and send it back to java world and deserialize it again. For this purpose I come to about javaobj-py3, with this everything is fine until I pass LinkedHashMap in setAttributeValue(). In Python I am doing this:
total_data=b''
while True:
data = self.clientsocket.recv(8192)
if not data: break
total_data += data
pyobj = javaobj.loads(total_data)
For this I am getting exception as:
RuntimeError: Unknown OpCode in the stream: 0x8 (at offset 0x14C)
What will be cause? Is it like opcodes are not found for "something"? Is anyone can suggest any other idea to convert byte stream send from java into object in python world perform some operation on that object and send it back to java world and deserialize it again.
You can convert the object to a JSON string and then send over the wire.
To convert object to json
new GSONBuilder().create().toJSON(obj);
To convert json to object
new GSONBuilder().create().fromJSON(jsonString, YouObject.class);
I prefer using GSON for converting a object to json and reverse in java. In python you can use json library. The methods are
json.loads(string) and json.dumps(object)
If a mail is send to my inbox, I recieve a message, and I'm inserting the contents into DB.
I have a org.springframework.integration.core.Message something like follows:
public void receive(Message<?> message)
{
//I am inserting message contents into DB
}
Now in the event of failure, I wanted to have fail safe recovery mechanism, what I am thinking is to serialize the Message object into a file and later deserialize and update to DB.
Question
1. In this situation how to serialize the Message object?
2. Other than serialization any other mechanism that can be used?
EDIT
I have not done Serialization before, I heard like the class should implements Serializable in order to use ObjectOutputStream, in this case I don't want to create a subclass of Message, So how to serialize Message to File?
Sure, there are many serialization mechanisms apart from the jvm one.
XML
JSON
BSON
MessagePack
protobufs
...
Some of them are text-based, some are binary. All have drawbacks and pluses. Text-based ones are human-readable, binary ones are faster and take up less space.
There are java libraries that handle all the above formats: JAXB (XML), Jackson (JSON), etc.
In this situation how to serialize the Message object? Other than serialization any other mechanism that can be used?
Extract all the data you need from the Message and save it. You can do this in any manner you choose.
You can deserialize it by populating a new Message with the data you saved.
I don't know if I probably understood it al right.. but assuming Message is not much more than lots of strings and some integers you can just use directly an ObjectOutputStream and write it to a file (binary) and then readin later. Why not?
Message e = new Message();
ObjectOutputStream oos = new ObjectOutputStream(new FileOutputStream("whatever");
oos.writeObject(message);
// read in
ObjectInputStream ois = new ObjectInputStream(new FileInputStream("whatever");
Message e = (Message) ois.readObject();
I am trying to use Protocol buffers to store serialized data in Database for a web application built in java.
I have created .proto files and compiled them to get the generated classes. Also I can build the message objects using the setters & finally build() method. But to store it to database, I need serialized data as byte[] or byte buffers. How do I finally get that from the message instances ??
import com.paratha.serializers.protocolbuffers.CommentProto.Comment;
Comment.Builder comment=Comment.newBuilder();
comment.setCommentBody("This is the first comment!").setUserId(32433).build();
How do I get the serialized data from here to write to database ?
Google have made it very easy :) :
MyProtocolBufferObject myObject = MyProtocolBufferObject.newBuilder().setName("bob").build();
byte[] bytes = myObject.toByteArray();
Edit
With your example:
Comment.Builder commentBuilder=Comment.newBuilder();
Comment comment = commentBuilder.setCommentBody("This is the first comment!").setUserId(32433).build();
byte[] bytes = comment.toByteArray();
Note that when you call the newBuilder() method you are getting an instance of Comment.Builder, not an instance of Comment. It is only when you call the Comment.Builder's build() method that you get an instance of Comment.
Is there a BSON serializer/deserializer library out there for PHP or Java?
Another possibility is BSON4Jackson extension for Jackson, which adds support for BSON reading/writing.
BSON encoder/decoder in Java is pretty trivial. The following snippet of code is from my app, so it's in Scala. I am sure you could build a Java implementation from it easily.
import org.bson.BSON
import com.mongodb.{DBObject, DBDecoder, DefaultDBDecoder}
def convert(dbo: DBObject): Array[Byte] =
BSON.encode(dbo)
// NB! this is a stateful object and thus it's not thread-safe, so have
// to create one per decoding
def decoder: DBDecoder = DefaultDBDecoder.FACTORY.create
def convert(data: Array[Byte]): DBObject =
// NOTE: we do not support Ref in input, that's why "null" for DBCollection
decoder.decode(data, null)
def convert(is: InputStream): DBObject =
// NOTE: we do not support Ref in input, that's why "null" for DBCollection
decoder.decode(is, null)
The only significant note is that DBEncoder instance has an internal state it (re)uses during decoding, so it's not thread-safe. It should be ok if you decode objects one by one, but otherwise you'd better create an instance per decoding session.
check this link
http://php.net/manual/en/ref.mongo.php
bson_decode — Deserializes a BSON object into a PHP array
bson_encode — Serializes a PHP variable into a BSON string
You might check the MongoDB drivers for those languages, since MongoDB uses BSON. See what they use, or steal their implementation.
And here is a C++11 JSON encoder and decoder I've made using Rapidjson, because the native JSON encoder (BSONObj::jsonString) uses a non-standard encoding for longs: https://gist.github.com/ArtemGr/2c44cb451dc6a0cb46af
Also, unlike the native JSON encoder, this one doesn't have a problem decoding top-level arrays.