I am playing around with MessagePack and Java. I have had experience with Protobuf and Json (using Jackson and Gson) in the past when it comes to serialization tools.
When it comes to normal serialization and deserialization I have no problems at all. It is when I want to have multiple subclasses of another class that a problem arise.
I am testing this with the following code :
TestMessage.TestMessageSubClass sub = new TestMessage.TestMessageSubClass();
byte[] pack = MsgPack.pack(sub);
Assert.assertTrue(ArrayUtils.isNotEmpty(pack));
List<? extends TestMessage> msg = MsgPack.unpack(pack, TestMessage.class);
Assert.assertNotNull(msg);
Assert.assertFalse(msg.isEmpty());
TestMessage temp = msg.get(0);
Assert.assertNotNull(temp);
Assert.assertTrue(temp instanceof TestMessage.TestMessageSubClass);
TestMessage.TestMessageSubClass sub2 = (TestMessage.TestMessageSubClass) temp;
Assert.assertEquals(sub, sub2);
System.out.println(sub);
System.out.println(sub2);
Those two lines fail because when I deserialize I only get a normal TestMessage, and not a TestMessageSubClass instance.
Assert.assertTrue(temp instanceof TestMessage.TestMessageSubClass);
TestMessage.TestMessageSubClass sub2 = (TestMessage.TestMessageSubClass) temp;
I suppose that this happens because by default the MessagePack unpacker has no way of determining the exact class of he needs to deserialize. In fact this would work just fine if I directly ask him to deserialize into a TestMessageSubClass.
My requirements is that TestMessage might have any number of subclasses with extra data, and with the same code I need to de-serialize them in the right class instance without losing anything. I might be deserializing a stream containing an heterogeneous list of those TestMessage instances.
I can have the behaviour I want using the #JsonSubTypes annotation in JacksonJson.
Is there a way to use the official MessagePack client API and obtain that? Is there a known pattern to do that myself?
Here is the code of my MsgPack wrapper class: GIST
Any advice on using MessagePack more efficiently is welcome too.
Related
I'm using Kotlin with Apache Beam and I have a set of DTOs that reference each other and all serialize great for any encoder with Kotlinx Serialization. When I try to use them with Beam I end up having issues because it's looking for all objects, type parameters and nested objects to implement the Java Serializable interface. Problem is, I'm not in control of that with all object types because some come from 3rd-party libraries.
I've implemented my own CustomCoder<T> type that uses Kotlinx Serialization but then I run into issues with my custom coder not being serializable, particularly due to the Kotlinx Serialization plugin-generated Companion object serializer not serializing. Since it's compile-time generated code I don't really have control over that and I can't flag it as #Transient. I tried implementing Externalizable on the coder and it fails as soon as I pass a type argument for T that doesn't implement Serializable or has a nested type argument that doesn't.
Also, Kotlinx Serialization is nice because it doesn't use reflection. It would make a lot of my current headaches disappear if I could just swap out the serialization mechanism somehow and not have to rely on standard Java serialization methods at all or somehow implement Externalizable in a way that just calls out to my own serialization mechanism and ignores the type parameter. Are there any solutions? I don't care how hacky it is, even if the solution involves messing with stuff in the Gradle build config to override something. I'm just not sure how to go about it so any pointers would be a great help!
Alternatively, if I abandon Kotlinx Serialization, are there any simple solutions to make any arbitrarily complex data type serialization just work with Java, even using reflection, without a lot of custom, manual work to handle encoding and decoding? I feel like maybe I'm just missing something obvious. This is my first project with Apache Beam but so far the google is little help.
Mybe late, I develop an annotation processor called beanknife recently, it support generate DTO from any class. You need config by annotation. But you don't need change the original class. This library support configuring on a separate class. Of course you can choose which property you want and which you not need. And you can add new property by the static method in the config class. The most power feature of this library is it support automatically convert a object property to the DTO version. for example
class Pojo1 {
String a;
Pojo b; // circular reference to Pojo2
}
class Pojo2 {
Pojo1 a;
List<Pojo1> b;
Map<List<Pojo1>>[] c;
}
// remove the circular reference in the DTO
#ViewOf(value = Pojo1.class, includePattern = ".*", excludes={Pojo1Meta.b})
class ConfigureOfPojo2 {}
// use the no circular reference versioned dto replace the Pojo1
#ViewOf(value = Pojo2.class, includePattern = ".*")
class ConfigureOfPojo2 {
// convert b to dto version
#OverrideViewProperty(Pojo2Meta.b)
private List<Pojo1View> b;
// convert c to dto version
#OverrideViewProperty(Pojo2Meta.c)
private Map<List<Pojo1View>>[] c;
}
will generate
// meta class, you can use it to reference the property name in a safe way.
class Pojo1Meta {
public final String a = "a";
public final String b = "b";
}
// generated DTO class. The actual one will be more complicate, there are many other method.
class Pojo1View {
private String a;
public Pojo1View read(Pojo1 source) { ... }
... getters and setters ...
}
class Pojo2Meta {
public final String a = "a";
public final String b = "b";
public final String c = "c";
}
class Pojo2View {
private String a;
private List<Pojo1View> b;
private Map<List<Pojo1View>>[] c;
public Pojo1View read(Pojo2 source) { ... }
... getters and setters ...
}
The interest things here is you can safely use the class not exist yet in the source. Although the compiler may complain, all will be ok after compiled. Because all the extra class will be automatically generated just before compiled.
A better approach may be to compile step by step, first add #ViewOf annotations, and then compile, so that all the classes that need to be used later are generated. Compile again after the configuration is complete. The advantage of this is that the IDE will not have grammatical error prompts, and can make better use of the IDE's auto-complete function.
With the support of using generated DTO in the configure class. You can define a Dto without circular reference just like the example. Furthermore, you can define another dto for Pojo2, and remove all property reference the Pojo1 and use it to replace the property b in Pojo1.
I'm trying to create a simple example with Java EE JMS.
If i try to receive an ObjectMessage, i need to have exactly the same path (packagename) as the other project, which sends the ObjectMessage.
For example i have in my sender project a class called Person in the packege "org.queue.sender" and exactly the same class in my receiver project in the package "org.queue.receiver".
As already said, if i try to get the objectmessage i get the following Exception:
java.lang.ClassNotFoundException: org.queue.sender.Person
If i create a new package in my receiver project named org.queue.sender and transfer the class Peron there, then it run. but i think i couldn't be the really solution.
Is there a better solution?
From the JavaDoc:
An ObjectMessage object is used to send a message that contains a serializable object in the Java programming language ("Java object"). It inherits from the Message interface and adds a body containing a single reference to an object. Only Serializable Java objects can be used.
So, objects passed via ObjectMessages must be Serializable i.e. it must be the same class and the exact same package.
If you need more flexible handling of messages I suggest that you use e.g. TextMessage and serialize/deserialize the objects using e.g. JSON or XML.
ObjectMapper mapper = ... ; // Get hold of a Jackson ObjectMapper
session.createTextMessage(mapper.writeValueAsString(myPojo));
// and on the receiving side
TextMessage message = ....; // From the message receiver
MyPojo myPojo = mapper.readValue(message.getText(), MyPojo.class);
I have a problem with serializing and deserializing my object structure with GSON. In order to describe the problem i'll have to describe my class structure a bit:
I have a java abstract class, let's name it "A". There are also classes "BA", "CA", "DA" that are abstract too and they extend class "A". Each of them has it's own constructor, non of which is non-arg. Finally there are several (many!) classes those extend "BC", or "CA" or "DA". Instances of those "bottom" classes are kept in "ArrayList" list.
Now, i'm trying to "jsonize" that array list. For creating Json string I'm using this code:
Gson gs = new Gson();
Type listOfTestObject = new TypeToken<List<A>>(){}.getType();
String rez = gs.toJson(getListOfAs(), listOfTestObject);
And i'm trying to deserialize that json using this (in another class):
Type listOfTestObject = new TypeToken<ArrayList<A>>(){}.getType();
ArrayList<A> listOfAs = gs.fromJson(jsonREZString, listOfTestObject);
but above code throws this:
Unable to invoke no-args constructor for class packagename.A. Register an InstanceCreator with Gson for this type may fix this problem.
Now, I have create a non-args constructor in the class "A", but no luck. I have read about "InstanceCreator" but it looks like I would have to create an "InstanceCreator" for each concrete class that extends "A"! Right? I can't do it, because I have many (many!) classes that extend "A" through "BA", "CA" or "DA".
What am I missing? How can I simply deserialize (serialization seems fine) this complex structure without adding custom deserialializastion code for each type?
In fact you might have here 2 distinct problems.
1) You have polymorphic types thus you probably want to serialize objects as their concrete type and not A.
2) You want to deserialize to the concrete types that do not provide no arg ctrs.
Gson does not support 1 & 2, there is an extension for 1 but I never used it.
Maybe Genson solves your problem, it supports both 1 & 2.
Genson genson = new Genson.Builder()
// enables polymorphic types support
.setWithClassMetadata(true)
// enables no arg support
.setWithDebugInfoPropertyNameResolver(true)
.create();
// will look like: [{"#class": "com.xxx.SomeConcreteClass", ...}, {"#class": "com.XXX.OtherClass"}]
String json = genson.serialize(getListOfAs());
List<A> listOfA = genson.deserialize(json, new GenericType<List<A>>() {});
You don't need to specify the type during serialization, except if you want that only parent fields be present in the output.
ex: genson.serialize(getListOfAs(), GenericType>() {}) will serialize only attributes from A, you can also force genson to always use runtime types by setting setUseRuntimeTypeForSerialization(true) on the builder.
Also if you don't want impl details to leak in the json representation, you can define aliases (builder.addAlias("someAlias", SomeClass.class) for your types, they will be used instead of full package+classname.
First off, I don't think this is necessarily a good idea, I'm just seeing if this is really possible. I could see some benefits, such as not having to explicitly convert to objects that we're sending to the client and using an interface to blacklist certain fields that are security concerns. I'm definitely not stuck on the idea, but I'd like to give it a try.
We're using Spring MVC + Jackson to generate JSON directly from objects. We have our domain object that contains necessary data to send to the client and we have a list of error strings that are added to every outgoing JSON request as needed.
So the return JSON might be something like
{ name: 'woohoo', location : 'wahoo', errors : ['foo'] }
Currently, we have a class that models what should be on the client side, but we always extend a common base class with the error methods.
So, we have:
interface NameAndLoc {
String getName();
String getLocation();
}
and
interface ResponseErrors {
List<String> getErrors();
void appendError(String);
}
We have two classes that implement these interfaces and would like to have CGLIB generate a new class the implements:
interface NameAndLocResponse extends NameAndLoc, ResponseErrors {}
Presently, with CGLIB mixins, I can generate an object with the following:
Object mish = Mixin.create(
new Class [] {NameAndLoc.class, ResponseErrors.class},
new Object [] { new NameAndLocImpl(), new ResponseErrorsImpl() } );
I could then cast the object to either NameAndLoc or ResponseErrors, however, what I would like to do is create an object that uses the same backing classes, but implements the NameAndLocResponse interface, without having to extend our common error handling class and then implement NameAndLoc.
If I attempt to cast with what I have, it errors out. I'm sure this is possible.
I think it is very similar to this, but not quite: http://www.jroller.com/melix/entry/alternative_to_delegate_pattern_with
Simply add the NameAndLocResponse interface to the Class array in the Mixin constructor as the last argument. The resulting object will implement it. You can find an example of this in this blog entry: http://mydailyjava.blogspot.no/2013/11/cglib-missing-manual.html
I am using the Play framework.
I want to use the function renderJSON with 2 Objects as an argument. It doesn't seem to be possible so I am trying to create a class that contains the 2 objects. In order not to have to create a new class everytime, I want to use Generics but it doesn't seem to work:
Model :
public class JSONContainer<T> extends Model {
private T myT;
private StatusMessage mySm;
public JSONContainer(T myT, StatusMessage mySm) {
this.myT = myT;
this.mySm = mySm;
}
}
and then :
In a function of a Controller:
JSONContainer<User> myJ = new JSONContainer(logged,sm);
renderJSON(myJ);
where logged is a User, sm is a StatusMessage. I get the error:
type: 'play.exceptions.JavaExecutionException'
If I don't use Generics, it works fine. Any idea?
The console gives this output, where the line 43 is:
JSONContainer<User> myJ = new JSONContainer(logged,sm);
Generic entities can't be mapped by Hibernate.
You should do the generic class abstract and create specific implementations (using User and any other possible values of T). This should solve the issue
Instead of using a JSONContainer like I did, I think the best way is to go with Collection as shown in this user guide for GSON made by Google (the JSON mapper used by Play apparently) at http://sites.google.com/site/gson/gson-user-guide#TOC-Collections-Examples:
Collection collection = new ArrayList();
collection.add(logged);
collection.add(sm);
renderJSON(collection);
Collection are good for serializing (Java object to JSON) but not good for deserializing (see the GSON user guide for more insight).
It is better, IMHO, to use Collection than JSONContainer as JSONContainer are not useful in that case and don't give more possibilities.