I'm encoding a complex Map structure with arrays of double values.
High precision is not important and output size is, so I'm trying to get the JSON tool (Jackson in this case) to serialize the double values using a provided DecimalFormat.
The following is my best shot, but this fails as the serializer is not picked by the object mapper to encode the array:
class MyTest
{
public class MyDoubleSerializer extends JsonSerializer<double[]>
{
public void serialize(double[] value, JsonGenerator jgen, SerializerProvider provider) throws IOException, JsonProcessingException
{
for (double d : value)
{
jgen.writeStartArray();
jgen.writeRaw( df.format( d ) );
jgen.writeEndArray();
}
}
}
#Test
public void test1() throws Exception
{
ObjectMapper mapper = new ObjectMapper();
SimpleModule module = new SimpleModule("MyModule", new Version(0, 1, 0, "alpha"));
module.addSerializer(double[].class, new MyDoubleSerializer());
mapper.registerModule(module);
Map<String, Object> data = new HashMap<String, Object>();
double[] doubleList = { 1.1111111111D, (double) (System.currentTimeMillis()) };
data.put( "test", doubleList );
System.out.print( mapper.writeValueAsString( data ));
}
}
The output is:
{"test":[1.1111111111,1.315143204964E12}
What I was looking for:
{"test":[1.32E12, 1.11E0]}
Any ideas?
Also, I don't like having to generate a String and write is as raw - is there I could feed a StringBuffer into into DecimalFormat to do this?
Thanks
Managed to resolve this, by borrowing from the built-in serializer for Double.
It's a bit of a hack, because writeRaw() doesn't care about the context and doesn't write a comma between array members, so I'm casting the Json writer and calling its writeValue() method to handle this.
Strangely enough, this does not work on the example in the question (again doesn't get called for serializing these doubles), but does work on my real-world object which is more complex.
Enjoy...
public class JacksonDoubleArrayTest
{
private DecimalFormat df = new DecimalFormat( "0.##E0" );
public class MyDoubleSerializer extends org.codehaus.jackson.map.ser.ScalarSerializerBase<Double>
{
protected MyDoubleSerializer()
{
super( Double.class );
}
#Override
public final void serializeWithType( Double value, JsonGenerator jgen, SerializerProvider provider, TypeSerializer typeSer ) throws IOException,
JsonGenerationException
{
serialize( value, jgen, provider );
}
#Override
public void serialize( Double value, JsonGenerator jgen, SerializerProvider provider ) throws IOException, JsonGenerationException
{
if ( Double.isNaN( value ) || Double.isInfinite( value ) )
{
jgen.writeNumber( 0 ); // For lack of a better alternative in JSON
return;
}
String x = df.format( value );
if ( x.endsWith( "E0" ) )
{
x = x.substring( 0, x.length() - 2 );
}
else if ( x.endsWith( "E1" ) && x.length() == 6 )
{
x = "" + x.charAt( 0 ) + x.charAt( 2 ) + '.' + x.charAt( 3 );
}
JsonWriteContext ctx = (JsonWriteContext)jgen.getOutputContext();
ctx.writeValue();
if ( jgen.getOutputContext().getCurrentIndex() > 0 )
{
x = "," + x;
}
jgen.writeRaw( x );
}
#Override
public JsonNode getSchema( SerializerProvider provider, Type typeHint )
{
return createSchemaNode( "number", true );
}
}
#SuppressWarnings("unchecked")
private static Map<String, Object> load() throws JsonParseException, JsonMappingException, IOException
{
ObjectMapper loader = new ObjectMapper();
return (Map<String, Object>)loader.readValue( new File( "x.json" ), Map.class );
}
#Test
public void test1() throws JsonGenerationException, JsonMappingException, IOException
{
ObjectMapper mapper = new ObjectMapper();
SimpleModule module = new SimpleModule( "StatsModule", new Version( 0, 1, 0, "alpha" ) );
module.addSerializer( Double.class, new MyDoubleSerializer() );
mapper.registerModule( module );
String out = mapper.writeValueAsString( load() );
// System.out.println( out.length() );
}
}
Related
I receive JSON payload which is set of key-value pairs. Value may be either string or number. I have to parse the JSON and store key-value pairs into appropriate varchar2 columns. I should save incoming number exactly as it was presented in the input payload.
But for numbers presented like 1.1E4, 0.00000000000003 and similar I get 11000.0, 3.0E-14 instead.
Is it a way to disable/prevent number conversion to have just string representation instead?
I use FasterXML Jackson implementation.
By the way there is no actual doc available - all sources I found point to http://wiki.fasterxml.com/JacksonHome which is down right now.
I found two similar questions here
Jackson JSON converts integers into strings
Disable the Number to String automatic conversion in jackson
but both require exception when encounter number, which is not my case. I have tried suggested solutions but was unsuccessful in modifying them to fit my task.
Also there is no answer in
https://github.com/FasterXML/jackson-databind/issues/796
Right now I have no specification for input string other than key-value pairs. So just an example:
I may receive something like:
{"a":"text", "b":"35", "c":{"d":"another"}, "e":["array",35], "f":1.1E4, "g":0.00000000000003}
I want string pairs
"a" -> "text", "b" -> "35", "c" -> "{\"d\":\"another\"}", "e" -> "[\"array\",35]", "f" -> "1.1E4"
The simplest conversion way is:
public void test() throws IOException {
Map map = new ObjectMapper().readValue(
"{\"a\":\"text\", \"b\":\"35\", \"c\":{\"d\":\"another\"}, \"e\":[\"array\",35], \"f\":1.1E4, \"g\":0.00000000000003}"
, Map.class);
System.out.println(map);
}
results in:
{a=text, b=35, c={d=another}, e=[array, 35], f=11000.0, g=3.0E-14}
The more accurate way:
public class JsonUtil2 {
private static final ObjectMapper OBJECT_MAPPER = new ObjectMapper();
public static Map<String, String> parse(String json) throws IOException {
ObjectNode objectNode = (ObjectNode) OBJECT_MAPPER.readTree(json);
Map<String, String> result = new HashMap<>(objectNode.size());
objectNode.fields().forEachRemaining(entry -> result.put(entry.getKey(), toJson(entry.getValue())));
return result;
}
private static String toJson(JsonNode jsonNode) {
if (jsonNode.isNumber()) {
if (jsonNode instanceof DoubleNode || jsonNode instanceof FloatNode) {
DecimalFormatSymbols dfs = new DecimalFormatSymbols();
dfs.setDecimalSeparator('.');
dfs.setMinusSign('-');
DecimalFormat df = new DecimalFormat("#.#", dfs);
df.setMaximumFractionDigits(32);
df.setMaximumIntegerDigits(32);
return df.format(jsonNode.doubleValue());
} else {
return jsonNode.asText();
}
} else if (jsonNode.isValueNode()) {
return jsonNode.asText();
} else {
try {
return OBJECT_MAPPER.writeValueAsString(jsonNode);
} catch (JsonProcessingException e) {
throw new RuntimeException(e);
}
}
}
}
which results in:
{a=text, b=35, c={"d":"another"}, e=["array",35], f=11000, g=0.00000000000003}
This is much better, but still differ in f=11000 instead of f=1.1E4.
In your case you want to treat everything as a String, so you need a custom deserialiser which reads JSON Object and JSON Array as String. We can also force Jackson to read Map<String, String> by providing this information using TypeFactory.
Assume our JSON payload looks like below:
{
"a": "text",
"b": "35",
"c": {
"d": "another",
"dd":3.44E3
},
"e": [
"array",
35,
2.3E5
],
"f": 1.1E4,
"g": 0.00000000000003
}
Example code:
import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.core.JsonToken;
import com.fasterxml.jackson.databind.DeserializationContext;
import com.fasterxml.jackson.databind.JsonDeserializer;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.deser.std.StringDeserializer;
import com.fasterxml.jackson.databind.module.SimpleModule;
import com.fasterxml.jackson.databind.type.CollectionType;
import com.fasterxml.jackson.databind.type.MapType;
import java.io.File;
import java.io.IOException;
import java.util.ArrayList;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
public class JsonTreeApp {
public static void main(String[] args) throws Exception {
File jsonFile = new File("./resource/test.json").getAbsoluteFile();
SimpleModule everythingIsStringModule = new SimpleModule();
everythingIsStringModule.addDeserializer(String.class, new EverythingIsStringDeserializer());
ObjectMapper mapper = new ObjectMapper();
mapper.registerModule(everythingIsStringModule);
MapType mapType = mapper.getTypeFactory().constructMapType(LinkedHashMap.class, String.class, String.class);
LinkedHashMap<String, String> map = mapper.readValue(jsonFile, mapType);
map.forEach((k, v) -> System.out.println(k + " => " + v));
}
}
class EverythingIsStringDeserializer extends StringDeserializer {
#Override
public String deserialize(JsonParser p, DeserializationContext ctxt) throws IOException {
if (p.currentToken() == JsonToken.START_OBJECT) {
return _deserializeFromObject(p, ctxt);
}
return super.deserialize(p, ctxt);
}
private String _deserializeFromObject(JsonParser p, DeserializationContext ctxt) throws IOException {
MapType mapType = ctxt.getTypeFactory().constructMapType(LinkedHashMap.class, String.class, String.class);
JsonDeserializer<Object> deserializer = ctxt.findRootValueDeserializer(mapType);
Map<String, String> map = (Map<String, String>) deserializer.deserialize(p, ctxt);
return toString(map);
}
#Override
protected String _deserializeFromArray(JsonParser p, DeserializationContext ctxt) throws IOException {
CollectionType collectionType = ctxt.getTypeFactory().constructCollectionType(ArrayList.class, String.class);
JsonDeserializer<Object> deserializer = ctxt.findRootValueDeserializer(collectionType);
List<String> list = (List<String>) deserializer.deserialize(p, ctxt);
return toString(list);
}
private String toString(Map<String, String> map) {
StringBuilder builder = new StringBuilder(128);
builder.append('{');
boolean addComa = false;
for (Map.Entry<String, String> entry : map.entrySet()) {
if (addComa) {
builder.append(',');
}
builder.append('"').append(entry.getKey())
.append("\":");
appendValue(entry.getValue(), builder);
addComa = true;
}
builder.append('}');
return builder.toString();
}
private String toString(List<String> list) {
StringBuilder builder = new StringBuilder(128);
builder.append('[');
boolean addComa = false;
for (String item : list) {
if (addComa) {
builder.append(',');
}
appendValue(item, builder);
addComa = true;
}
builder.append(']');
return builder.toString();
}
private void appendValue(String value, StringBuilder builder) {
if (value == null || value.isEmpty()) {
builder.append("\"\"");
return;
}
if (Character.isAlphabetic(value.charAt(0))) {
builder.append('"').append(value).append('"');
} else {
builder.append(value);
}
}
}
Prints:
a => text
b => 35
c => {d=another, dd=3.44E3}
e => [array, 35, 2.3E5]
f => 1.1E4
g => 0.00000000000003
Here is the class that I want to serialize.
public class ItemRow<T> {
private String id;
private List<T> items;
}
There are two variations that are allowed.
ItemRow<String>, ItemRow<ItemRow>.
In the latter case, it will be nested.
eg:
ItemRow item1 = new ItemRow("abc", Arrays.asList("item1", "item2", "item3"));
String result = mapper.writeValueAsString(item1);
System.out.println(result);
should give
{
"abc":["item1","item2","item3"]
}
Now, the latter case
ItemRow item2 = new ItemRow("cde", Arrays.asList("item4, item5"));
ItemRow item = new ItemRow("combined", Arrays.asList(item1,item2));
result = mapper.writeValueAsString(item);
System.out.println(result);
should give
{
"combined": {
"abc": ["item1", "item2", "item3"],
"cde": ["item4", "item5"]
}
}
But I get exception while serializing the latter. The first one works as expected. so I believe the recursive serialization is failing, but I am unable to find out why
Here is exception
com.fasterxml.jackson.core.JsonGenerationException: Can not start an object, expecting field name (context: Object)
at com.fasterxml.jackson.core.JsonGenerator._reportError(JsonGenerator.java:1961)
at com.fasterxml.jackson.core.json.JsonGeneratorImpl._reportCantWriteValueExpectName(JsonGeneratorImpl.java:244)
at com.fasterxml.jackson.core.json.WriterBasedJsonGenerator._verifyValueWrite(WriterBasedJsonGenerator.java:866)
at com.fasterxml.jackson.core.json.WriterBasedJsonGenerator.writeStartObject(WriterBasedJsonGenerator.java:279)
at hello.ItemRowSerializer.serialize(ItemRow.java:58)
at hello.ItemRowSerializer.serialize(ItemRow.java:42)
at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480)
at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319)
at com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:2655)
at com.fasterxml.jackson.core.base.GeneratorBase.writeObject(GeneratorBase.java:381)
at hello.ItemRowSerializer.serialize(ItemRow.java:67)
at hello.ItemRowSerializer.serialize(ItemRow.java:42)
Serializer implementation
class ItemRowSerializer extends JsonSerializer<ItemRow> {
#Override
public void serialize(ItemRow itemRow, JsonGenerator jgen, SerializerProvider serializerProvider) throws IOException {
String id = itemRow.getId();
List<Object> items = itemRow.getItems();
if (items.isEmpty()) {
jgen.writeStartObject();
jgen.writeFieldName(id);
jgen.writeStartArray();
jgen.writeEndArray();
jgen.writeEndObject();
}
else {
jgen.writeStartObject();
Object item = items.get(0);
jgen.writeFieldName(id);
if (item instanceof ItemRow){
for (Object i : items) {
//ItemRow temp = (ItemRow) i;
//jgen.writeObjectField(temp.getId(), temp);
//jgen.writeObjectField(id, i);
jgen.writeStartObject();
jgen.writeObject(i);
jgen.writeEndObject();
}
}
else {
//jgen.writeFieldName(id);
jgen.writeStartArray();
for (Object arg : items) {
jgen.writeString(arg.toString());
}
jgen.writeEndArray();
}
}
jgen.writeEndObject();
}
}
Your serializer algoritihm is incorrect. The code is down above. You do not need to start object when you are directly deserializing an object. I removed this steps and minimized the code.
Example Test;
#Test
public void serializeTest() throws JsonProcessingException
{
ObjectMapper mapper = new ObjectMapper();
SimpleModule module = new SimpleModule();
module.addSerializer(ItemRow.class, new ItemRowSerializer());
mapper.registerModule(module);
ItemRow item1 = new ItemRow("abc", Arrays.asList("item1", "item2", "item3"));
String result = mapper.writeValueAsString(item1);
System.out.println(result);
ItemRow item2 = new ItemRow("cde", Arrays.asList("item4", "item5"));
ItemRow item6 = new ItemRow("deeper-1", Arrays.asList("item6", "item7"));
ItemRow item7 = new ItemRow("deeper-2", Arrays.asList("item6", "item7"));
ItemRow item8 = new ItemRow("deeper", Arrays.asList(item6, item7));
ItemRow item3 = new ItemRow("inner-1", Arrays.asList("item6", "item7"));
ItemRow item4 = new ItemRow("inner-2", Arrays.asList("item6", "item7"));
ItemRow item5 = new ItemRow("inner", Arrays.asList(item3, item4, item8));
ItemRow item = new ItemRow("combined", Arrays.asList(item1,item2,item5));
result = mapper.writeValueAsString(item);
System.out.println(result);
}
Algorithm;
public class ItemRowSerializer extends JsonSerializer<ItemRow>
{
#Override
public void serialize(ItemRow itemRow, JsonGenerator jgen, SerializerProvider serializerProvider) throws IOException
{
jgen.writeStartObject();
writeInnerObject(jgen, itemRow);
jgen.writeEndObject();
}
private void writeStringArr(JsonGenerator jgen, List items) throws IOException
{
jgen.writeStartArray();
for (Object arg : items)
{
jgen.writeString(arg.toString());
}
jgen.writeEndArray();
}
private void writeInnerObject(JsonGenerator jgen, ItemRow row) throws IOException
{
jgen.writeFieldName(row.getId());
if (row.getItems().size() > 0 && row.getItems().get(0) instanceof ItemRow)
{
jgen.writeStartObject();
for (int i = 0; i < row.getItems().size(); i++)
{
ItemRow innerRow = (ItemRow) row.getItems().get(i);
if( innerRow.getItems().size() > 0 && innerRow.getItems().get(0) instanceof ItemRow )
{
writeInnerObject(jgen, innerRow);
}
else
{
jgen.writeFieldName(innerRow.getId());
writeStringArr(jgen, innerRow.getItems());
}
}
jgen.writeEndObject();
}
else
{
writeStringArr(jgen, row.getItems());
}
}
}
I am trying to find top 10 movies from a huge data set using Hadoop. I am using Map Reduce approach. I have used a local collection i.e TreeMap to sort the data , but this approach is not recommended.May i know the correct approach to sort the data while processing a huge amount inside Mapper ? I am giving the my Mapper and Reducer code
Mapper code
public class HighestViewedMoviesMapper extends Mapper<Object, Text, NullWritable, Text> {
private TreeMap<Integer, Text> highestView = new TreeMap<Integer, Text>();
#Override
public void map( Object key, Text values, Context context ) throws IOException, InterruptedException {
String data = values.toString();
String[] field = data.split( "::", -1 );
if ( null != field && field.length == 2 ) {
int views = Integer.parseInt( field[1] );
highestView.put( views, new Text( field[0] + "::" + field[1] ) );
if ( highestView.size() > 10 ) {
highestView.remove( highestView.firstKey() );
}
}
}
#Override
protected void cleanup( Context context ) throws IOException, InterruptedException {
for ( Map.Entry<Integer, Text> entry : highestView.entrySet() ) {
context.write( NullWritable.get(), entry.getValue() );
}
}
}
Reducer Code
public class HighestViewMoviesReducer extends Reducer<NullWritable, Text, NullWritable, Text> {
private TreeMap<Integer, Text> highestView = new TreeMap<Integer, Text>();
public void reduce( NullWritable key, Iterable<Text> values, Context context )
throws IOException, InterruptedException {
for ( Text value : values ) {
String data = value.toString();
String[] field = data.split( "::", -1 );
if ( field.length == 2 ) {
highestView.put( Integer.parseInt( field[1] ), new Text( value ) );
if ( highestView.size() > 10 ) {
highestView.remove( highestView.firstKey() );
}
}
}
for ( Text t : highestView.descendingMap().values() ) {
context.write( NullWritable.get(), t );
}
}
}
Can anyone tell me the best approach to do this ? Thanks in advance.
I have a method which does the following:
final DaySerializer daySerializer = new DaySerializer(Day.class);
final ObjectMapper mapper = new ObjectMapper();
final SimpleModule module = new SimpleModule("DaySerializer", new Version(2, 1, 3, null, null, null));
module.addSerializer(Day.class, daySerializer);
mapper.registerModule(module);
try {
mapper.writerWithDefaultPrettyPrinter().writeValue(new File(this.jsonPath + "/Days.json"), days);
} catch (final Exception e) {
e.printStackTrace();
}
And the serializer class method:
#Override
public void serialize(final Day day, final JsonGenerator jsonGenerator, final SerializerProvider serializerProvider)
throws IOException {
jsonGenerator.writeStartObject();
//Date
SimpleDateFormat format1 = new SimpleDateFormat("yyyy-MM-dd");
jsonGenerator.writeStringField("calendar", "" + format1.format(day.getCalendar().getTime()));
//Day index
jsonGenerator.writeNumberField("dayCount", day.getDayCount());
//Demand
for (final Entry<String, int[]> e : day.getDemand().entrySet()) {
jsonGenerator.writeFieldName("Demand: "+e.getKey());
jsonGenerator.writeStartArray();
for(int i : e.getValue()) {
jsonGenerator.writeNumber(i);
}
jsonGenerator.writeEndArray();
}
//Employee shift allocations
for(EmployeeShiftAllocation eSA : day.getEmployeeShiftAllocations()) {
jsonGenerator.writeNumberField("eSA ID", eSA.getId());
}
jsonGenerator.writeEndObject();
}
Which works for the first Day object in the List which is handed in to it, but doesn't show any errors, but only exports the first Day object. This is confusing since I do hand in the List. Am I missing something simple? Do I need to handle this inside the serializer somehow?
I have variable of type java.util.Properties. I am trying to write it to a JSON file, and as well as read from that file.
The Properties variable looks something like below:
Properties inner3 = new Properties();
inner3.put("i1", 1);
inner3.put("i2", 100);
Properties inner2 = new Properties();
inner2.put("aStringProp", "aStringValue");
inner2.put("inner3", inner3);
Properties inner1 = new Properties();
inner1.put("aBoolProp", true);
inner1.put("inner2", inner2);
Properties topLevelProp = new Properties();
topLevelProp.put("count", 1000000);
topLevelProp.put("size", 1);
topLevelProp.put("inner1", inner1);
Naturally, when I serialize the topLevelProp to JSON I expect the result to be as below.
{
"inner1": {
"inner2": {
"aStringProp": "aStringValue",
"inner3": {
"i2": 100,
"i1": 1
}
},
"aBoolProp": true
},
"size": 1,
"count": 1000000
}
The above JSON result can be produced by using Gson in a pretty straight forward way, but when it is fed the same JSON string to desrialize, it fails.
Gson gson = new GsonBuilder().create();
String json = gson.toJson(topLevelProp); //{"inner1":{"inner2":{"aStringProp":"aStringValue","inner3":{"i2":100,"i1":1}},"aBoolProp":true},"size":1,"count":1000000}
//following line throws error: Expected a string but was BEGIN_OBJECT at line 1 column 12 path $.
Properties propObj = gson.fromJson(json, Properties.class);
Tried with Jackson as well:
ObjectMapper mapper = new ObjectMapper();
mapper.configure(MapperFeature.PROPAGATE_TRANSIENT_MARKER, true);
mapper.setVisibility(PropertyAccessor.ALL, Visibility.NONE);
mapper.setVisibility(PropertyAccessor.FIELD, Visibility.ANY);
File file = new File("configs/config1.json");
mapper.writeValue(file, topLevelProp);
The last line throws error:
com.fasterxml.jackson.databind.JsonMappingException: java.util.Properties cannot be cast to java.lang.String (through reference chain: java.util.Properties["inner1"])
Tried to desrialize from the string as follows and it failed with the following error:
Properties jckProp = JsonSerializer.mapper.readValue(json, Properties.class);
Can not deserialize instance of java.lang.String out of START_OBJECT token
at [Source: {"inner1":{"inner2":{"aStringProp":"aStringValue","inner3":{"i2":100,"i1":1}},"aBoolProp":true},"size":1,"count":1000000}; line: 1, column: 11] (through reference chain: java.util.Properties["inner1"])
How this can be handled?
Update: Following the idea of cricket_007, found com.fasterxml.jackson.databind.node.ObjectNode, can be used as follows:
ObjectNode jckProp = JsonSerializer.mapper.readValue(json, ObjectNode.class);
System.out.println(jckProp.get("size").asInt());
System.out.println("jckProp: " + jckProp);
System.out.println("jckProp.inner: " + jckProp.get("inner1"));
I think this can be the way forward for me, as I mostly have to read from JSON file.
The problem you have is that you are misusing java.util.Properties: it is NOT a multi-level tree structure, but a simple String-to-String map.
So while it is technically possibly to add non-String property values (partly since this class was added before Java generics, which made allowed better type safety), this should not be done. For nested structured, use java.util.Map or specific tree data structures.
As to Properties, javadocs say for example:
The Properties class represents a persistent set of properties.
The Properties can be saved to a stream or loaded from a stream.
Each key and its corresponding value in the property list is a string.
...
If the store or save method is called on a "compromised" Properties
object that contains a non-String key or value, the call will fail.
Now: if and when you have such "compromised" Properties instance, your best bet with Jackson or Gson is to construct a java.util.Map (or perhaps older Hashtable), and serialize it. That should work without issues.
As it was said above by StaxMan, you're misusing the Properties class and you're close about having heavy issues for using it like that due to lack of type information. However, you might also face the same case for weakly-typed maps. If it's a must for you, then you can use your custom Gson JsonDeserializer (note the JSON arrays issue):
final class PropertiesJsonDeserializer
implements JsonDeserializer<Properties> {
private static final JsonDeserializer<Properties> propertiesJsonDeserializer = new PropertiesJsonDeserializer();
private PropertiesJsonDeserializer() {
}
static JsonDeserializer<Properties> getPropertiesJsonDeserializer() {
return propertiesJsonDeserializer;
}
#Override
public Properties deserialize(final JsonElement jsonElement, final Type type, final JsonDeserializationContext context)
throws JsonParseException {
final Properties properties = new Properties();
final JsonObject jsonObject = jsonElement.getAsJsonObject();
for ( final Entry<String, JsonElement> e : jsonObject.entrySet() ) {
properties.put(e.getKey(), parseValue(context, e.getValue()));
}
return properties;
}
private static Object parseValue(final JsonDeserializationContext context, final JsonElement valueElement) {
if ( valueElement instanceof JsonObject ) {
return context.deserialize(valueElement, Properties.class);
}
if ( valueElement instanceof JsonPrimitive ) {
final JsonPrimitive valuePrimitive = valueElement.getAsJsonPrimitive();
if ( valuePrimitive.isBoolean() ) {
return context.deserialize(valueElement, Boolean.class);
}
if ( valuePrimitive.isNumber() ) {
return context.deserialize(valueElement, Number.class); // depends on the JSON literal due to the lack of real number type info
}
if ( valuePrimitive.isString() ) {
return context.deserialize(valueElement, String.class);
}
throw new AssertionError();
}
if ( valueElement instanceof JsonArray ) {
throw new UnsupportedOperationException("Arrays are unsupported due to lack of type information (a generic list or a concrete type array?)");
}
if ( valueElement instanceof JsonNull ) {
throw new UnsupportedOperationException("Nulls cannot be deserialized");
}
throw new AssertionError("Must never happen");
}
}
Hence, it might be used like this:
private static final Gson gson = new GsonBuilder()
.registerTypeAdapter(Properties.class, getPropertiesJsonDeserializer())
.create();
public static void main(final String... args) {
final Properties outgoingProperties = createProperties();
out.println(outgoingProperties);
final String json = gson.toJson(outgoingProperties);
out.println(json);
final Properties incomingProperties = gson.fromJson(json, Properties.class);
out.println(incomingProperties);
}
private static Properties createProperties() {
final Properties inner3 = new Properties();
inner3.put("i1", 1);
inner3.put("i2", 100);
final Properties inner2 = new Properties();
inner2.put("aStringProp", "aStringValue");
inner2.put("inner3", inner3);
final Properties inner1 = new Properties();
inner1.put("aBoolProp", true);
inner1.put("inner2", inner2);
final Properties topLevelProp = new Properties();
topLevelProp.put("count", 1000000);
topLevelProp.put("size", 1);
topLevelProp.put("inner1", inner1);
return topLevelProp;
}
with the following output:
{inner1={inner2={aStringProp=aStringValue, inner3={i2=100, i1=1}}, aBoolProp=true}, size=1, count=1000000}
{"inner1":{"inner2":{"aStringProp":"aStringValue","inner3": {"i2":100,"i1":1}},"aBoolProp":true},"size":1,"count":1000000}
{inner1={inner2={aStringProp=aStringValue, inner3={i2=100, i1=1}}, aBoolProp=true}, size=1, count=1000000}
Type info injection
You could save some type information though, if you inject the type information in the result JSON. Let's assume you are fine with storing numeric values as not primitives, but JSON objects having two keys like _$T and _$V to hold the actual type (a class indeed, not any java.reflect.Type, unfortunately) and the associated value respectively in order to restore the real type of the property. This can be applied to arrays either, but it's still not possible to hold a parameterized type due to the lack of type paremerization for instances that are parameterized somehow (unless you can reach it via a Class instance):
final class PropertiesJsonDeserializer
implements JsonDeserializer<Properties> {
private static final JsonDeserializer<Properties> propertiesJsonDeserializer = new PropertiesJsonDeserializer();
private PropertiesJsonDeserializer() {
}
static JsonDeserializer<Properties> getPropertiesJsonDeserializer() {
return propertiesJsonDeserializer;
}
#Override
public Properties deserialize(final JsonElement jsonElement, final Type type, final JsonDeserializationContext context)
throws JsonParseException {
final Properties properties = new Properties();
final JsonObject jsonObject = jsonElement.getAsJsonObject();
for ( final Entry<String, JsonElement> e : jsonObject.entrySet() ) {
properties.put(e.getKey(), parseValue(context, e.getValue()));
}
return properties;
}
private static Object parseValue(final JsonDeserializationContext context, final JsonElement valueElement) {
if ( valueElement instanceof JsonObject ) {
return context.deserialize(valueElement, Properties.class);
}
if ( valueElement instanceof JsonPrimitive ) {
final JsonPrimitive valuePrimitive = valueElement.getAsJsonPrimitive();
if ( valuePrimitive.isBoolean() ) {
return context.deserialize(valueElement, Boolean.class);
}
if ( valuePrimitive.isNumber() ) {
return context.deserialize(valueElement, Number.class); // depends on the JSON literal due to the lack of real number type info
}
if ( valuePrimitive.isString() ) {
return context.deserialize(valueElement, String.class);
}
throw new AssertionError();
}
if ( valueElement instanceof JsonArray ) {
throw new UnsupportedOperationException("Arrays are unsupported due to lack of type information (a generic list or a concrete type array?)");
}
if ( valueElement instanceof JsonNull ) {
throw new UnsupportedOperationException("Nulls cannot be deserialized");
}
throw new AssertionError("Must never happen");
}
}
final class TypeAwarePropertiesSerializer
implements JsonSerializer<Properties> {
private static final JsonSerializer<Properties> typeAwarePropertiesSerializer = new TypeAwarePropertiesSerializer();
private TypeAwarePropertiesSerializer() {
}
static JsonSerializer<Properties> getTypeAwarePropertiesSerializer() {
return typeAwarePropertiesSerializer;
}
#Override
public JsonElement serialize(final Properties properties, final Type type, final JsonSerializationContext context) {
final JsonObject propertiesJson = new JsonObject();
for ( final Entry<Object, Object> entry : properties.entrySet() ) {
final String property = (String) entry.getKey();
final Object value = entry.getValue();
if ( value instanceof Boolean ) {
propertiesJson.addProperty(property, (Boolean) value);
} else if ( value instanceof Character ) {
propertiesJson.addProperty(property, (Character) value);
} else if ( value instanceof Number ) {
final JsonObject wrapperJson = newWrapperJson(value);
wrapperJson.addProperty("_$V", (Number) value);
propertiesJson.add(property, wrapperJson);
} else if ( value instanceof String ) {
propertiesJson.addProperty(property, (String) value);
} else if ( value instanceof Properties || value instanceof Collection || value instanceof Map ) {
propertiesJson.add(property, context.serialize(value));
} else if ( value != null ) {
final Class<?> aClass = value.getClass();
if ( aClass.isArray() ) {
final JsonObject wrapperJson = newWrapperJson(value);
wrapperJson.add("_$V", context.serialize(value));
propertiesJson.add(property, wrapperJson);
} else {
throw new UnsupportedOperationException("Cannot process: " + value);
}
} else /* now the value is always null, Properties cannot hold nulls */ {
throw new AssertionError("Must never happen");
}
}
return propertiesJson;
}
private static JsonObject newWrapperJson(final Object value) {
final JsonObject wrapperJson = new JsonObject();
wrapperJson.addProperty("_$T", value.getClass().getName());
return wrapperJson;
}
}
final class TypeAwarePropertiesDeserializer
implements JsonDeserializer<Properties> {
private static final JsonDeserializer<Properties> typeAwarePropertiesDeserializer = new TypeAwarePropertiesDeserializer();
private TypeAwarePropertiesDeserializer() {
}
static JsonDeserializer<Properties> getTypeAwarePropertiesDeserializer() {
return typeAwarePropertiesDeserializer;
}
#Override
public Properties deserialize(final JsonElement jsonElement, final Type type, final JsonDeserializationContext context)
throws JsonParseException {
try {
final Properties properties = new Properties();
final JsonObject jsonObject = jsonElement.getAsJsonObject();
for ( final Entry<String, JsonElement> e : jsonObject.entrySet() ) {
properties.put(e.getKey(), parseValue(context, e.getValue()));
}
return properties;
} catch ( final ClassNotFoundException ex ) {
throw new JsonParseException(ex);
}
}
private static Object parseValue(final JsonDeserializationContext context, final JsonElement valueElement)
throws ClassNotFoundException {
if ( valueElement instanceof JsonObject ) {
final JsonObject valueObject = valueElement.getAsJsonObject();
if ( isWrapperJson(valueObject) ) {
return context.deserialize(getWrapperValueObject(valueObject), getWrapperClass(valueObject));
}
return context.deserialize(valueElement, Properties.class);
}
if ( valueElement instanceof JsonPrimitive ) {
final JsonPrimitive valuePrimitive = valueElement.getAsJsonPrimitive();
if ( valuePrimitive.isBoolean() ) {
return context.deserialize(valueElement, Boolean.class);
}
if ( valuePrimitive.isNumber() ) {
throw new AssertionError("Must never happen because of 'unboxing' above");
}
if ( valuePrimitive.isString() ) {
return context.deserialize(valueElement, String.class);
}
throw new AssertionError("Must never happen");
}
if ( valueElement instanceof JsonArray ) {
return context.deserialize(valueElement, Collection.class);
}
if ( valueElement instanceof JsonNull ) {
throw new UnsupportedOperationException("Nulls cannot be deserialized");
}
throw new AssertionError("Must never happen");
}
private static boolean isWrapperJson(final JsonObject valueObject) {
return valueObject.has("_$T") && valueObject.has("_$V");
}
private static Class<?> getWrapperClass(final JsonObject valueObject)
throws ClassNotFoundException {
return Class.forName(valueObject.get("_$T").getAsJsonPrimitive().getAsString());
}
private static JsonElement getWrapperValueObject(final JsonObject valueObject) {
return valueObject.get("_$V");
}
}
Now the topLevelProp can be filled also with:
topLevelProp.put("ARRAY", new String[]{ "foo", "bar" });
topLevelProp.put("RAW_LIST", asList("foo", "bar"));
if you have these special JSON deserializers applied:
private static final Gson typeAwareGson = new GsonBuilder()
.registerTypeAdapter(Properties.class, getTypeAwarePropertiesSerializer())
.registerTypeAdapter(Properties.class, getTypeAwarePropertiesDeserializer())
.create();
A sample output:
{RAW_LIST=[foo, bar], inner1={inner2={aStringProp=aStringValue, inner3={i2=100, i1=1}}, aBoolProp=true}, size=1, count=1000000, ARRAY=[Ljava.lang.String;#b81eda8}
{"RAW_LIST":["foo","bar"],"inner1":{"inner2":{"aStringProp":"aStringValue","inner3":{"i2":{"_$T":"java.lang.Integer","_$V":100},"i1":{"_$T":"java.lang.Integer","_$V":1}}},"aBoolProp":true},"size":{"_$T":"java.lang.Integer","_$V":1},"count":{"_$T":"java.lang.Integer","_$V":1000000},"ARRAY":{"_$T":"[Ljava.lang.String;","_$V":["foo","bar"]}}
{RAW_LIST=[foo, bar], inner1={inner2={aStringProp=aStringValue, inner3={i2=100, i1=1}}, aBoolProp=true}, size=1, count=1000000, ARRAY=[Ljava.lang.String;#e2144e4}
Summarizing up two approaches, you might want to eliminate the need of weak-typing and introduce explicit POJO mappings if possible.
Since I only needed a deserialization feature, i.e. generate Java properties for an incoming Json (in my case a REST endpoint), I quickly hacked this solution:
public class Configuration extends Properties {
public void load(JsonElement json) {
addJson("", json);
return;
}
public void addJson(String root, JsonElement json) {
// recursion for objects
if (json instanceof JsonObject) {
if (!root.equals("")) root += ".";
final JsonObject jsonObject = json.getAsJsonObject();
for ( final Entry<String, JsonElement> e : jsonObject.entrySet() ) {
addJson(root + e.getKey(), e.getValue());
}
return;
}
// recursion for arrays
if (json instanceof JsonArray) {
final JsonArray jsonArray = json.getAsJsonArray();
if (!root.equals("")) root += ".";
int count = 0;
for(final JsonElement e : jsonArray) {
addJson(root+count, e);
count++;
}
return;
}
// leaves: add property
this.setProperty(root, json.getAsString());
}
}
As you can see, this is extending the Properties class. Another option would of course be to initialize a Properties object beforehand and pass it into the recursion.
I hope this is useful to someone :-)