I have an object containing cyclic references. According to the XStream Json documentation, cyclic references are NOT supported, and one should therefore use the NO_REFERENCES XStream mode when marshalling an object to Json:
What limitations has XStream's JSON support?
JSON represents a very simple data model for easy data transfer.
Especially it has no equivalent for XML attributes. Those are written
with a leading "#" character, but this is not always possible without
violating the syntax (e.g. for array types). Those may silently
dropped (and makes it therefore difficult to implement
deserialization). References are another issue in the serialized
object graph, since JSON has no possibility to express such a
construct. You should therefore always set the NO_REFERENCES mode of
XStream. Additionally you cannot use implicit collections, since the
properties in a JSON object must have unique names.
But I tried setting the mode to ID_REFERENCES and it appears as though the Object is marshalled with references, and the object can be unmarshalled properly. Is the XStream documentation simply outdated, or have I simply inadvertently created the object graph in such a way that I haven't hit any of the limitations?
Sorry, but I can't post my exact graph as an example as it contains application/domain-specific code and it might take some time to construct a 'clean' alternative.
Related
It seems that GSON silently ignores when a JSON string contains field names that don't match the target POJO class. One solution outlined here suggests to use annotations to mark "required" fields to have GSON fail when de-serializing strings that don't contain fields.
But we defined that our POJOs must be "exact" matches (when we allow for incoming objects to be null, they must be declared as Optional field in the POJO - and we have a special type adapter that turns nulls into Optional.empty() instances). Therefore all fields in the POJO are mandatory. And null isn't a valid value.
Following the guidance in that question I linked to, it seems that the only way of having gson fail while parsing: to do a full "deep reflection" scan of the object created by de-serialization process and check if any of the Optional fields are null.
Or maybe - I am missing something and there is an easier way to have gson tell me when our JSON strings contain bad field names?
( background: we just ran into a problem because of wrong field name deep down in a nested structure - leading to null objects where we didn't expect them )
Turns out: this "deficiency" is really a core design point of gson: it is a JSON parser. Validation is not within the scope of gson.
Therefore the "correct" answer is to use java bean validation annotations and to put some implementation framework (for example the hibernate validator or apache bval) in place.
Alternatively, it is possible to register a special type adapter when creating the gson instance. This type adapter uses reflection to override an internal map with a bit of checking code - allowing for a relatively "clean" solution which leads to gson throwing an exception when running into "unknown" fields. ( thanks to Andy Turner for pointing to the corresponding github issue tracker entry --- code can be found there)
I need to parse untrusted Java serialized objects. The data is given to me as a byte array (written at some point by ObjectOutputStream).
I do not want to simply call ObjectInputStream.readObject() and/or load the actual object. I am looking for a way to safely parse the bytes and grab field names & values.
--
Here's a little summary of my attempt so far, after taking a look at the ObjectInputStream procedure for deserializing objects.
I have tried to extract field types/names (as unicode strings) recursively based on expected stream constants. I end up with a list of field names whose values should appear in the byte array in order. I am uneasy about this approach because it is probably buggy. Especially accommodating for what seems to be individual serialization protocols followed by HashMap, ArrayList, etc. But it might work, if I can figure out a way to read the bytes that represent field values:
I can try to read and store primitives based on size/offset, but when I encounter my first object, it gets a bit more complicated -- there is no clear way to distinguish between which bytes are associated with which values anymore (without actually loading the object in the way that ObjectInputStream probably does?).
--
Can anyone suggest either a potential solution that I'm obviously looking past, or a trusted library that can help parse the serialized data without loading objects?
Thank you for reading, and for all comments/suggestions!!! I apologize if something is unclear and I would be happy to clarify if you bear with me.
You can't do this in principle. Any Java class can take over its own Serialization and write arbitrary data to the stream that only it knows how to parse and reconstruct, via code that is only invoked during deserialization.
We are using Spring rest template and jackson json provider to serialize/deserialize json. From my services i send a linkedHashSet back which gets converted to a HashSet on the client side when i receive it. Because of this I loose my insertion order of elements.
Is this the default implementation of jackson json provider for Set ? Is there any other way, so it can deserialize to proper implementation? I feel it's gonna be tricky but inputs will be highly appreciated from you guys.
Thanks
You can specify the concrete class for Jackson to use with the #JsonDeserialize annotation. Just put:
#JsonDeserialize(as=LinkedHashSet.class)
On the property's setter.
It all depends on what you ask the result type to be: if ask data to be mapped to a LinkedHashSet, then JSON Array gets mapped to it. If you use a vague type like java.lang.Object (or java.util.Collection), you will get ArrayList for JSON Arrays.
Keep in mind that JSON is data, not objects (by default), so metadata regarding Java types you used is not passed by default. There are ways to do that, if you need it, but usually you will simply need to provide expected type.
What is the safest way to serialize any kind of Java Object such that when:
DBObject obj = getFromDB;
Object id = obj.get(ID_KEY);
String s1 = safeSerialize(id);
The obj.get(ID_KEY) method returns an object that serves as an "id", it could be a ObjectId, String, Long, Integer, or anyt kind of Object.
Then do the same thing:
DBObject obj = getFromDB;
Object id = obj.get(ID_KEY);
String s2 = safeSerialize(id);
I need to make sure the s1 is still equals to s2. I mean, for example obj.get() method might return new instance of say, new Integer(100) for a given ID_KEY, still having a "serialized" version.
You can't do this because java.lang.Object is not serializable.
Classes are marked with java.io.Serializable to indicate that the programmer has allowed for binary representations to be able to reanimated.
Even if you were to require that the object that you were storing was limited to a given set of objects which were serializable, you'd be subject to the usual fragility of serialization.
If you limited the set of objects to a set for which you then provided custom serialization (as opposed to using the default serialization), you could then make it work.
I don't think Java serialization gives you these guarantees:
It certainly doesn't if any on the classes involve could change.
It certainly doesn't if you serialize / deserialize on different JVM version/release/vendor platforms.
It possibly doesn't for any class that has custom writeObject / readObject methods ... and that includes some of the basic types in java.util, etcetera.
JSON is as bad, if not worse. The order of the attributes of JSON objects is explicitly undefined, so you have no guarantees that the attributes will appear in the serialization in the same order each time.
Binding based serial/deserializers for POJOs as XML could work (if they don't use attributes), but you need to beware of how a binding handles the serialization of inherently unordered collections such as HashSets and HashMaps. The chances are that the order of the set/map members in the serialization won't be predictable.
My advice would be to think of another way to solve your actual problem ... whatever it is.
Currently I have a class setup to be processed as an autobean:
public interface Asset extends Hit {
String getGuid();
String getHitType();
Map<String,Serializable> getMetadata();
}
I tried using Object instead of Serializable:
Map<String,Object> getMetadata()
but this seems to blow up when trying to access data (because it's not 'reified').
The Metadata map may contain other maps, strings, ints, etc. How do I retrieve data from an inner map of that metadata object?
Currently, if I call asset.getMetadata().get("title"); this returns a SerializableAutoBean and performing toString() or String.valueOf(obj) on that object returns the in memory object information and not the actually string value.
Can an AutoBean object be this dynamic, or do you specifically have to define every field?
AutoBeans aren't "dynamic" in the Java generics or RTTI sense.
In GWT, all types have to be known at compile time for anything which is auto-generated (which includes AutoBeans). This places restrictions on your designs which don't allow you to take full advantage of Java's language features (specifically, generics and other RTTI features). So, AutoBeans are not dynamic in the RTTI or Java generic sense. However, AutoBeans are simply a low-level way of wrapping your data, and you still have access to the data by using Splittables!
As stated in the previous comments, you can use Splittables for the parts of your JSON object whose type is not known at serialization/decode time. Sure, it would be nice to have everything happen at once, but nothing is stopping you from performing some post-processing on your data objects to get them into your desired state.
A really good way for someone to "Grok" what is going on with AutoBeans (and anything else which is autogenerated) is to look at the resulting generated code. The default location for maven is: ${project.build.directory}/.generated.
If you look in there after you've compiled, you should find the code which the GWT compiler produces for your AutoBeans.