If a mail is send to my inbox, I recieve a message, and I'm inserting the contents into DB.
I have a org.springframework.integration.core.Message something like follows:
public void receive(Message<?> message)
{
//I am inserting message contents into DB
}
Now in the event of failure, I wanted to have fail safe recovery mechanism, what I am thinking is to serialize the Message object into a file and later deserialize and update to DB.
Question
1. In this situation how to serialize the Message object?
2. Other than serialization any other mechanism that can be used?
EDIT
I have not done Serialization before, I heard like the class should implements Serializable in order to use ObjectOutputStream, in this case I don't want to create a subclass of Message, So how to serialize Message to File?
Sure, there are many serialization mechanisms apart from the jvm one.
XML
JSON
BSON
MessagePack
protobufs
...
Some of them are text-based, some are binary. All have drawbacks and pluses. Text-based ones are human-readable, binary ones are faster and take up less space.
There are java libraries that handle all the above formats: JAXB (XML), Jackson (JSON), etc.
In this situation how to serialize the Message object? Other than serialization any other mechanism that can be used?
Extract all the data you need from the Message and save it. You can do this in any manner you choose.
You can deserialize it by populating a new Message with the data you saved.
I don't know if I probably understood it al right.. but assuming Message is not much more than lots of strings and some integers you can just use directly an ObjectOutputStream and write it to a file (binary) and then readin later. Why not?
Message e = new Message();
ObjectOutputStream oos = new ObjectOutputStream(new FileOutputStream("whatever");
oos.writeObject(message);
// read in
ObjectInputStream ois = new ObjectInputStream(new FileInputStream("whatever");
Message e = (Message) ois.readObject();
Related
Does anybody know the way to deserialize Avro without using any Pojo and Schemas?
The problem:
I have a data stream of different Avro files.
The goal is to group that data depending on the presence of some attributes (e.g. user.role, another.really.deep.attribute.with.specific.value and so on).
Each avro entry might contain any number of matching attributes - from zero to all listed).
So, there is no need to do anything with data. Just to peek at some elements.
The question is, is there any way to convert that data to Map or Node? Like I can do it with JSON using Jackson or GSON.
I've tried to use GenericDatumReader, but it requires a Schema. So maybe all I need is to read the schema from avro (how?).
Also, I've tried to use something like this, but this approach doesn't work.
public Map deserialize(byte[] data) {
DatumReader<LinkedHashMap> reader
= new SpecificDatumReader<>(LinkedHashMap.class);
Decoder decoder = null;
try {
decoder = DecoderFactory.get().binaryDecoder(data, null);
return reader.read(null, decoder);
} catch (IOException e) {
logger.error("Deserialization error:" + e.getMessage());
}
}
Since I have time to 'play' with the problem, I have created a utility class that generates schemas depending on keys. It works, but looks like a big overhead.
A reader schema is required to deserialize any message.
If you have the writer schema available, you can simply use that. Note that if you have Avro files, these include the schema they were written with and you can use avro-tools.jar -getschema to extract it
Without these options, then you'll need to figure out the schema on your own (maybe using a hexdump and knowing how Avro data gets encoded)
Consider an object in java which implements Serializable. I want to send that object to a python code on TCP socket. I am serializing object and sending on TCP by using :
//socket connection code
PySessionObject object = new PySessionObject();
object.setMethodCall(PyServerMethodConstant.SETATTRIBUTE);
object.setAttributeName(name);
object.setAttributeValue(value);// value is of object type which also implements Serializable
os = sChannel.socket().getOutputStream();
oos = new ObjectOutputStream(os);
oos.writeObject(object);
oos.flush();
os.flush();
//socket closing and exception handling
Now I want to convert this byte stream into object in python perform some operation on that object and send it back to java world and deserialize it again. For this purpose I come to about javaobj-py3, with this everything is fine until I pass LinkedHashMap in setAttributeValue(). In Python I am doing this:
total_data=b''
while True:
data = self.clientsocket.recv(8192)
if not data: break
total_data += data
pyobj = javaobj.loads(total_data)
For this I am getting exception as:
RuntimeError: Unknown OpCode in the stream: 0x8 (at offset 0x14C)
What will be cause? Is it like opcodes are not found for "something"? Is anyone can suggest any other idea to convert byte stream send from java into object in python world perform some operation on that object and send it back to java world and deserialize it again.
You can convert the object to a JSON string and then send over the wire.
To convert object to json
new GSONBuilder().create().toJSON(obj);
To convert json to object
new GSONBuilder().create().fromJSON(jsonString, YouObject.class);
I prefer using GSON for converting a object to json and reverse in java. In python you can use json library. The methods are
json.loads(string) and json.dumps(object)
I have a OMElement object and from that I want to get an InputStream or reader object. What I want is to stream the xml from the OMElement which I have, without getting loaded into memory. I only could be able to get XMLStreamReader object from that. But I can't find a way to get InputStream/Reader out of that.
OMElement element /*I have this object */
XMLStreamReader xmlreader = element.getXMLStreamReaderWithoutCaching();
Can anyone please help me ? I really appreciate!
Thanks
PS: I have come a way to get an InputStream but it gives an exception which I need help to resolve it.
XMLStreamReader xmlReader = element.getXMLStreamReader(false);
try {
if (xmlReader.getEventType() == XMLStreamReader.START_DOCUMENT) {
xmlReader.next();
}
DataHandler handler = XMLStreamReaderUtils.getDataHandlerFromElement(xmlReader);
handler.getInputStream();
The call to getDataHandlerFromElement generate the exception javax.xml.stream.XMLStreamException: Error during base64 decoding
The purpose of the getDataHandlerFromElement method is to decode base64 encoded binary data contained in an element, which is not what you want.
Axiom currently doesn't have a streaming pull serializer (and I don't know any other XML library that supports this). It only supports serializing to XML in push mode, i.e. by writing to an OutputStream or Writer. You could do the serialization in a separate thread and use a PipedOutputStream/PipedInputStream (similar to what Activation does for DataHandler#getInputStream() for a DataHandler that is not backed by a DataSource). This would satisfy the constant memory requirement, but I guess that the overhead caused by using a separate thread would not be acceptable in your case.
I thought about the possibility to implement such a pull serializer in the past, and I think it's technically feasible to do that (without using a separate thread). Please open a feature request for Axiom so that we can implement that in one of the next releases. I think it would be an interesting addition to Axiom.
Android provides some ways to save application information. http://developer.android.com/guide/topics/data/data-storage.html
Now, my issue is that I have some structured data comparable with tasks, that have categories and some other related settings. Everything is fetched from a web service and has to get saved for performance and offline use.
Which would be the best option to use on Android? I think the choice is really between SQLite Database or Internal storage. What would I expect from the performance and implementation choosing one of the options?
One really easy way to cache data is to just Serialize your objects and save them to the internal cache e.g:
// Example object which implements Serializable
Example example = new Example();
// Save an object
ObjectOutput out = new ObjectOutputStream(new FileOutputStream(new File(getCacheDir(),"")+"cacheFile.srl"));
out.writeObject((Example) example);
out.close();
// Load in an object
ObjectInputStream in = new ObjectInputStream(new FileInputStream(new File(new File(getCacheDir(),"")+"cacheFile.srl")));
Example example_loaded = (Example) in.readObject();
in.close();
If your web service gives you an XML or JSON, you can just save it on a file, and then read it from the file. It's like "caching" data.
I am trying to use Protocol buffers to store serialized data in Database for a web application built in java.
I have created .proto files and compiled them to get the generated classes. Also I can build the message objects using the setters & finally build() method. But to store it to database, I need serialized data as byte[] or byte buffers. How do I finally get that from the message instances ??
import com.paratha.serializers.protocolbuffers.CommentProto.Comment;
Comment.Builder comment=Comment.newBuilder();
comment.setCommentBody("This is the first comment!").setUserId(32433).build();
How do I get the serialized data from here to write to database ?
Google have made it very easy :) :
MyProtocolBufferObject myObject = MyProtocolBufferObject.newBuilder().setName("bob").build();
byte[] bytes = myObject.toByteArray();
Edit
With your example:
Comment.Builder commentBuilder=Comment.newBuilder();
Comment comment = commentBuilder.setCommentBody("This is the first comment!").setUserId(32433).build();
byte[] bytes = comment.toByteArray();
Note that when you call the newBuilder() method you are getting an instance of Comment.Builder, not an instance of Comment. It is only when you call the Comment.Builder's build() method that you get an instance of Comment.