For context, we are storing most of our data as JSON strings. This works very well with Hadoop on the backend and is easy to handle in Ruby on the front end. My data types fit the natural pattern for inheritance.
For simplicity, lets say I have a class Pet and a process FeedPet that feeds a pet. I also have a process WalkDog that only applies to Dog, which is a kind of pet. My data is organized such that I never need to worry about trying to walk a pet that isn't a dog.
What I would like to do is have Pet and Dog extends Pet, with Dog having an additional method "getLeash()", but I can't figure out how to map this to JSON.
Instead, I have a class Pet with a species data hashmap, so the WalkDog process would call pet.getSpeciesData("leash") instead of dog.getLeash().
I can create a Dog extends Pet, and I can serialize that to JSON using the Jackson library. The JSON object will have a leash field. But assume that I want to feed all the pets. All pets have a getFood() method. So the FeedPet process deserializes the objects to Pet. But this loses the leash field.
The WalkDog process can do this because it knows all of its input is going to be Dogs, so it can read it as a Dog and write it back out as a Dog.
Is there any way to serialize java objects to JSON such that I can preserve their type? I'm thinking something like Rails single table inheritance, but it would have to be something that the JSON libraries understand.
To make it work you must both embed some object type information in data (where it is only useful for deserializing) and usually use external schema definition which otherwise would not be needed (like XML Schema for xml; since that's basically a generic type system).
This has same problems as ORM has: Hibernate has to use ugly work-arounds (n+1 - way joins, "super tables" or discriminator fields).
Or another way to put it: data mapping/binding is not quite the same as object serialization/deserialization (latter tries to preserve more of object identity).
Here I am assuming that what you want is basically something like:
Pet pet = mapper.readValue(jsonString, Pet.class);
// (and possibly get an exception if Pet is an abstract class...)
Leash l = ((Dog) pet).getLeash();
If this is not the case, you could just simply bind to Dog
Dog dog = mapper.readValue(jsonString, Dog.class);
So anyway: Jackson project has feature request for doing just this, which allow you to do what (I think) you want.
However, this solution will mostly work for Java, as there is no standard way of passing Object type info within JSON. With XML this can sort of be done with XML Schema defined "xsi:type" attribute; which identifies Schema type, which is then mapped to class (yes, rather complicated way, but it does work).
UPDATE: Jackson 1.5 added support for this (i.e. implemented JACKSON-91 feature request), so it can be used for generating type identifiers, to allow proper handling of polymorphic types. It should work with non-Java systems too, given that you can fully configure details of how type information is to be included; and is NOT limited to using Java class names.
I've not had that much experience using JSON other than for fixtures in django, but in that instance I just have a "model" key, with the associated value.
It is then up to the program that is interpreting the objects to determine their type and inheritance hierarchy.
What I'm trying to say is that it is irrelevant what methods are available, as this doesn't need to be serialised. Only the object's name and attributes.
-- update
After reading your question again, it seems like your process is deserialising into the parent class, rather than the actual class. Since your dog class inherits from pet, you just want to make sure your deserialiser is creating objects of the most specialised class.
I don't know about the Jackson library, but if you can set it to have a type of model field, then that might be the go.
Related
I have seen in many Rest Assured frameworks that we create POJO classes for Serialization & deserialization but let's say we have to automate more than 50-70 APIs thus, creating POJO classes for all seems tedious work so can we deal with JSON objects and data directly? We can get rid of getters and setters by using Lombok annotations but still, we will have to set variables. just curious about what should be the best practice we can follow?
Not sure if I understood correctly. So maybe this answer goes in the totally wrong direction.
If you have lots of Classes and member variables, to streamline handling, you could introduce a level of abstraction.
As an example:
instead of a class and its member variables, you could have a HashMap that stores [variable name] as key, and [variable value] as value.
for multiple object of the same class, instantiate multiple HashMaps
maybe hold all those produced HashMaps in a Collection like a List
maybe even have an 'outer' HashMap, that maps [class name] to [collection]
in the end it might look like this: HashMap[class name -> Collection], Collection contains multiple 'object' HashMaps. Object HashMaps map their [member variable name] to the [meber variable value]
Now, to the JSON part: Google's GSON already has classes in place that do exactly that (object abstraction), so you now have an easy bridge between JSON and Java.
And then you bring it all together and only write ONE serializer and ONE deserializer for ALL classes you wanna handle.
In the end, you still have to put value to POJO or any kinds of object (Map, List, JSON objects...) to create a json payload. I don't like the idea manipulating data directly, it's so rigid.
To make your work less tedious, some techniques can be applied:
Create inner classes if you don't want to create too many POJO classes separatly.
Use Builder pattern (Intellij suggestion or #Builder annotation in lombok) to create POJO instance more straightforward (comparing to basic Setter).
Create POJO instance with default value for each property, then only the property that matters.
Separate the creation of POJO object in other packages, like PersonFactory to build Person instance. It will help make test cleaner.
All,
I have to be doing this wrong. It seemed like a good idea at the time but as I get deeper into it, I think there is a more proper programmatic way of going about it. Thus I ask you...
One note. I'm using Google AppEngine and the Datastore to store this information.
Ok... lets say I have a Super Class of Vehicle, which then has 3 Sub-Classes... Car, Truck, Motorcycle.
In the Super Class, there are 3 properties... Manufacturer, Model, Type
For example, these might be:
Manufacturer: Ford
Model: Focus
Type: Car
So in the Datastore, I have numerous Vehicle entities with these properties.
So if the user wants to see all the cars... I pull everyhing with a "Car" type.
If the user then wants to add one of these Vehicles to a "favorites" list, I then convert the Vehicle object into its specific Sub-Class based upon what type it is. This then adds the extra properties of that specific Sub-Class.
This new child entity is store in the Datastore with its added properties.
So basically, I'm downcasting, for example, from a Vehicle to a Car. I have done this by creating an extra constuctor in the Car class that takes Vehicle as an argument. Once created, the Car object now has all the properties (Manufacturer, Model, Type) set, and all the new properties that come with its specific implementation.
This just seems convoluted and wrong. It works but there has got to be a better way of doing this.
The main reason I chose this way is because of the way the GAE Datastore works. Its "cheaper" to store the Super Class and its limited properties and query those. Long story.
I'm trying to wrap my head around using Interfaces and/or Abstract classes for this but I wanted to get all of your input.
Thanks for the help.
I don't think you want a super/sub class structure here. Your problem as described has you "changing" an object from one type to another, and you cannot change the type of a Java object. You can create a new object, but then you have to move all your information from one to another, and maintenance becomes a problem.
I suggest that you have a class that represents your vehicle, and that it contain a reference to type-specific information; the classes representing each specific type can all extend something, and probably should so that methods within vehicle trying to do something with the type can call a common method to do it regardless of type. But this way, once you decide the specific type, you can add it to an existing vehicle object instead of "changing" it.
You could also explore whether an enumerated type would serve your purpose for type-specific data -- enum types can take constructors, have additional methods, etc. - the Oracle/Java tutorial on enums covers that pretty well.
The type of vehicle is encoded twice: once as an object type, and once as a property. Get rid of one of these, so that there is not a possibility of having a Truck (object type) with property value set to Car. Keep your object structure, or property pointing out the kind of vehicle (I recommend using an Enum), but not both.
To downcast, you do not need to create a new object of the child type. Just downcast:
Car myCar = (vehicle instanceof Car ? (Car)vehicle : null);
In Java, I would like to use hierarchies of immutable POJOs to express my domain model.
e.g.
final ServiceId id = new ServiceId(ServiceType.Foo, "my-foo-service")
final ServiceConfig cfg = new ServiceConfig("localhost", 8080, "abc", JvmConfig.DEFAULT)
final ServiceInfo info = new ServiceInfo(id, cfg)
All of these POJOs have public final fields with no getters or setters. (If you are a fan of getters, please pretend that the fields are private with getters.)
I would also like to serialize these objects using the MessagePack library in order to pass them around over the network, store them to ZooKeeper nodes, etc.
The problem is that MessagePack only supports serialization of public, non-final fields, so I cannot serialize the business objects as-is. Also MessagePack does not support enum, so I have to convert enum values to int or String for serialization. (Yes it does, if you add an annotation to your enums. See my comment below.)
To deal with this I have a hand-written corresponding hierarchy of "message" objects, with conversions between each business object and its corresponding message object. Obviously this is not ideal because it causes a large amount of duplicated code, and human error could result in missing fields, etc.
Are there any better solutions to this problem?
Code generation at compile time?
Some way to generate the appropriate serializable classes at runtime?
Give up on MessagePack?
Give up on immutability and enums in my business objects?
Is there some kind of generic wrapper library that can wrap a mutable object (the message object) into an immutable one (the business object)?
MessagePack also supports serialization of Java Beans (using the #MessagePackBeans annotation), so if I can automatically convert an immutable object to/from a Java Bean, that may get me closer to a solution.
Coincidentally, I recently created a project that does pretty much exactly what you are describing. The use of
immutable data models provides huge benefits, but many serialization technologies seem to approach
immutability as an afterthought. I wanted something that would fix this.
My project, Grains, uses code generation to create an immutable implementation
of a domain model. The implementation is generic enough that it can be adapted to different serialization frameworks.
MessagePack, Jackson, Kryo, and standard Java serialization are supported so far.
Just write a set of interfaces that describe your domain model. For example:
public interface ServiceId {
enum ServiceType {Foo, Bar}
String getName();
ServiceType getType();
}
public interface ServiceConfig {
enum JvmConfig {DEFAULT, SPECIAL}
String getHost();
int getPort();
String getUser();
JvmConfig getType();
}
public interface ServiceInfo {
ServiceId getId();
ServiceConfig getConfig();
}
The Grains Maven plugin then generates immutable implementations of these interfaces at compile time.
(The source it generates is designed to be read by humans.) You then create instances of your objects. This example
shows two construction patterns:
ServiceIdGrain id = ServiceIdFactory.defaultValue()
.withType(ServiceType.Foo)
.withName("my-foo-service");
ServiceConfigBuilder cfg = ServiceConfigFactory.newBuilder()
.setHost("localhost")
.setPort(8080)
.setUser("abc")
.setType(JvmConfig.DEFAULT);
ServiceInfoGrain info = ServiceInfoFactory.defaultValue()
.withId(id)
.withConfig(cfg.build());
Not as simple as your public final fields, I know, but inheritance and composition are not possible without getters
and setters. And, these objects are easily read and written with MessagePack:
MessagePack msgpack = MessagePackTools.newGrainsMessagePack();
byte[] data = msgpack.write(info);
ServiceInfoGrain unpacked = msgpack.read(data, ServiceInfoGrain.class);
If the Grains framework doesn't work for you, feel free to inspect its MessagePack templates.
You can write a generic TemplateBuilder that uses reflection to set the final fields of your hand-written domain model. The trick
is to create a custom TemplateRegistry that allows registration of your custom builder.
It sounds like you have merged, rather than separated, the read and write concerns of your application. You should probably consider CQRS at this point.
In my experience, immutable domain objects are almost always attached to an audit story (requirement), or it's lookup data (enums).
Your domain should probably be, mostly, mutable, but you still don't need getters and setters. Instead you should have verbs on your objects which result in a modified domain model, and which raise events when something interesting happens in the domain (interesting to the business -- business == someone paying for your time). It's probably the events that you're interested in passing over the wire, not the domain objects. Maybe it's even the commands (these are similar to events, but the source is an agent external to the bounded context in which your domain lives -- events are internal to the model's bounded context).
You can have a service to persist the events (and another one to persist commands), which is also your audit-log (fulfilling your audit stories).
You can have an event handler that pushes your events onto your bus. These events should contain either simple information or entity ID's. The services that respond to these events should perform their duties using the information provided, or they should query for the information they need using the given ID's.
You really shouldn't be exposing the internal state of your domain model. You're breaking encapsulation by doing that, and that's not really a desirable thing to do. If I were you I'd take a look at the Axon Framework. It's likely to get you further than MessagePack alone.
I've been using AutoBeans to map JSON data coming from a non GWT-RPC Java based web service. Everything has been working so far except for one mapping.
On the server side, the Class has a property of type Map where MyAbstractParentObject is the parent class of about 15 different child classes.
When I map that to a corresponding AutoBean interface on the client I'm not able to downcast MyAbstractParentObject to its child type after it's been decoded. I looked all over the GWT docs and 'the Googles' to see if AutoBeans even has polymorphic support but couldn't get an answer either way. Interceptors and Categories don't seem to be able handle this, just methods they want to exist in the interface that aren't getters/setters.
I was attempting to do a workaround using the type field in the JSON data to create an instance of the child class but the AutoBean does not give me access to the raw JSON, even though in the debugger I can see it as a protected field called 'data'. If I try to decode the original bean it will only have the fields in the MyAbstractParentObject.
The only alternatives I can see are to:
Extend or create my own AutoBeanCodex that can properly handle the
children of MyAbstractParentObject when it decodes the JSON.
Find a way to get to the raw JSON in the MyAbstractParentObject AutoBean
and use that to create and instance of the child class on the fly.
Switch to some other JSON-GWT Serialization framework like
GWTProJSONSerializer or piriti.
Any help would be appreciated.
I know this was asked a long time ago, but I struggled to find an answer too. I realized that the AutoBeans, since they're basically just fancy wrappers for the JSON, still contain all the data for the fields of the child object you want to downcast it to. So I wrote a method like this:
public <A, B> B cast( A sourceObject, Class<B> targetClass )
{
AutoBean<A> sourceBean = AutoBeanUtils.getAutoBean( sourceObject ); // Get the corresponding AutoBean.
HasSplittable splittableBean = ( HasSplittable ) sourceBean; // Implementation (if still AbstractAutoBean) supports this interface ;)
Splittable splittable = splittableBean.getSplittable().deepCopy(); // If you don't copy it, decode() tries to be clever and returns
// the original bean!
AutoBean<B> targetBean = AutoBeanCodex.decode( typeFactory, targetClass, splittable ); // Create new AutoBean of
// the target type.
return targetBean.as(); // Get the proxy for the outside world.
}
--Where typeFactory extends AutoBeanFactory, as you can see.
It's worked well enough for me. The trickiest bit was the cast to HasSplittable, since AutoBean doesn't extend that interface, but AbstractAutoBean (which implements AutoBean) does -- and a subclass of that is what's returned by calls to getAutoBean().
You also need to copy the Splittable, otherwise AutoBeanCodex thinks, "hey, I already have an AutoBean for that Splittable! Here you go!" -- and just gives you the original. ;)
Anyway, you can cast downwards, upwards...sideways! :P
Late edit: Stumbling upon this again months later, I figured I'd add a small caveat about something Jonathan mentioned below. The method I've described here is designed to be used on an AutoBean that hasn't been modified since it was deserialized. That's because (AFAIK) there's no guarantee that any setters you call will actually update the JSON (needed for the casting). This probably isn't a big deal, since typically you'll use this when you have an incoming DTO and you want to cast it to its real type ASAP, before doing anything else with it. In our case, none of our AutoBeans even had setters, so it wasn't really an issue. ;)
After you've cast it, you can do whatever you want with the resulting bean, which is fresh out of the factory after all!
I'm not very familiar with AutoBean but you probably can use the serializer/deserializer from RestyGWT. It supports polymorphism by making use of annotation.
link to the documentation:
http://restygwt.fusesource.org/documentation/restygwt-user-guide.html#Polymorphic_Sub_Types
Out of curiosity, I'd like to know if any class exists in Java with a method that returns a copy of its data structure. I ask because in class the teacher said a method like this breaks privacy, but I think that getting a copy of the structure is useful if you want to rearrange the structure. I'd like an example. Thanks.
I'm not entirely sure what you mean by the "data structure" of a class, but assuming you mean the members it contains, what you're looking for is reflection.
Try this tutorial.
Maybe you are missing the point: If you build a class which encapsulates some kind of internal data then you should NOT add a method which returns the internal data structure, but only the data that is encapsulated.
(Which is kind of the idea of encapsulation)
There should not be any need to "rearrange" your internal representation from the outside - because it is supposed to be internal and thus transparent in its use. (Meaning: You should not even be able to say what kind of data structure is used)
If you serialize it, any object (that is serializable) will happily prints its internal structure to a binary stream. And yes, this breaks encapsulation.
And yes, no one stops you from going to change the binary output and read it in again to create an altered object.
NB: there are more strange issues regarding serialization. For example, when deserializing objects new objects are created without their constructor ever being called. Et cetera. Serialization of objects is the maybe least object-oriented thing one can do with objects.
You're mixing up some concepts here.
Classes really are "data structures + methods", so in general you'd need a class to represent your class. Hey, a nice custom-built one would be the class your data is already in. You may be thinking of a class as a collection of simple fields, but this is not always the case.
As others have mentioned, you can extract the data via reflection, e.g.
public Map<String,Object> fields() {
Map output=new hashMap<String,Object>();
for (Field f:getClass().getFields())
{
try{
output.put(f.getName(), f.get(this));
}
catch(... //IllegalArgument, IllegalAccess.. {... }
}
return output;
}
You can get into encapsulation issues here, in general the class should provide the data that you need to see from it, I tend to use things like this only for debugging.
I'm not sure what you mean by "rearrange the structure". The class generally represents the structure of the data. If there's a transformation you want to accomplish, it probably belongs in a class method, e.g. are you thinking of something like a co-ordinates class that can provide a transformed version of itself into polar co-ordinates?
A simple way to see the internal representation of an object is to serialise it using XStream. This will generate an XML representation of the class and its components (and so on).
Does this break encapsulation ? Yes - in the sense that you're able to inspect the internal structure of the class. You can take that XML, change it, and provided that it matches the .class structure that it came from, deserialise it back into a .class instance.