Develop Reference

Java: Parsing JSON with unknown keys to Map

In Java, I need to consume JSON (example below), with a series of arbitrary keys, and produce Map<String, String>. I'd like to use a standard, long term supported JSON library for the parsing. My research, however, shows that these libraries are setup for deserialization to Java classes, where you know the fields in advance. I need just to build Maps.
It's actually one step more complicated than that, because the arbitrary keys aren't the top level of JSON; they only occur as a sub-object for prefs. The rest is known and can fit in a pre-defined class.
{
"al" : { "type": "admin", "prefs" : { "arbitrary_key_a":"arbitary_value_a", "arbitrary_key_b":"arbitary_value_b"}},
"bert" : {"type": "user", "prefs" : { "arbitrary_key_x":"arbitary_value_x", "arbitrary_key_y":"arbitary_value_y"}},
...
}
In Java, I want to be able to take that String, and do something like:
people.get("al").get("prefs"); // Returns Map<String, String>
How can I do this? I'd like to use a standard well-supported parser, avoid exceptions, and keep things simple.
UPDATE
#kumensa has pointed out that this is harder than it looks. Being able to do:
people.get("al").getPrefs(); // Returns Map<String, String>
people.get("al").getType(); // Returns String
is just as good.
That should parse the JSON to something like:
public class Person {
public String type;
public HashMap<String, String> prefs;
}
// JSON parsed to:
HashMap<String, Person>

Having your Person class and using Gson, you can simply do:
final Map<String, Person> result = new Gson().fromJson(json, new TypeToken<Map<String, Person>>() {}.getType());
Then, retrieving prefs is achieved with people.get("al").getPrefs();.
But be careful: your json string is not valid. It shouldn't start with "people:".

public static <T> Map<String, T> readMap(String json) {
if (StringUtils.isEmpty(json))
return Collections.emptyMap();
ObjectReader reader = new ObjectMapper().readerFor(Map.class);
MappingIterator<Map<String, T>> it = reader.readValues(json);
if (it.hasNextValue()) {
Map<String, T> res = it.next();
return res.isEmpty() ? Collections.emptyMap() : res;
}
return Collections.emptyMap();
}
All you need to do next, it that check the type of the Object. If it is Map, then you have an object. Otherwise, this is a simple value.

You can use Jackson lib to achieve this.
Put the following in pom.xml.
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>2.9.8</version>
</dependency>
Refer the following snippet that demonstrates the same.
ObjectMapper mapper = new ObjectMapper();
HashMap<String, Object> people = mapper.readValue(jsonString, new TypeReference<HashMap>(){});
Now, it is deserialized as a Map;
Full example:
import java.io.IOException;
import java.util.HashMap;
import com.fasterxml.jackson.core.JsonParseException;
import com.fasterxml.jackson.core.type.TypeReference;
import com.fasterxml.jackson.databind.JsonMappingException;
import com.fasterxml.jackson.databind.ObjectMapper;
public class testMain {
public static void main(String[] args) throws JsonParseException, JsonMappingException, IOException {
String json = "{\"address\":\"3, 43, Cashier Layout, Tavarekere Main Road, 1st Stage, BTM Layout, Ambika Medical, 560029\",\"addressparts\":{\"apartment\":\"Cashier Layout\",\"area\":\"BTM Layout\",\"floor\":\"3\",\"house\":\"43\",\"landmark\":\"Ambika Medical\",\"pincode\":\"560029\",\"street\":\"Tavarekere Main Road\",\"subarea\":\"1st Stage\"}}";
ObjectMapper mapper = new ObjectMapper();
HashMap<String, Object> people = mapper.readValue(json, new TypeReference<HashMap>(){});
System.out.println(((HashMap<String, String>)people.get("addressparts")).get("apartment"));
}
}
Output: Cashier Layout

AWS Lambda: return type String is not compatible with POJOClass

I am trying to return String from the handleRequest. I generate String from JSON using GSON.
Here's the method:
public String handleRequest(Map<String, String> input, Context context){
final Gson gson = new GsonBuilder().create();
String json;
//other logic here
json = gson.toJson(myPOJOResponseClass);
return json
}
I am getting this error: return type String is not compatible with myPOJOResponseClass
I tried to change the return type to Object didn't work. I tried to convert the String to JSON explicitly using JSONObject and changed the return type to JSONObject as well but that is not working either.
Any help would be appreciated.
PS: I am generating my POJO class using Lombok if that matters here.

You haven't provided much code for context, but based on the docs it looks like you have probably done something like this:
public class HelloPojo implements RequestHandler<Map<String, String>, myPOJOResponseClass>
Which would then mean that to implement the interface, your handleRequest(...) function needs to return myPOJOResponseClass.
If that is what is going on, try this:
public class HelloPojo implements RequestHandler<Map<String, String>, String>

GSON - Deserialize case insensitive field name (no definite case pattern)

I want to build a flexible api, I have no definite case sensitivity the user can pass, so GSON must be able to deserialize this in case sensitive.
{"firstName":"Juan"}
{"firstname":"Juan"}
{"Firstname":"Juan"}
...
How can I deserialize these field into my Foo's firstName?
public class Foo {
private String firstName;
//..getters
}
I tried to use FieldNamingPolicy, but it did not work.
new GsonBuilder()
.setFieldNamingPolicy(FieldNamingPolicy.UPPER_CAMEL_CASE)
.setPrettyPrinting()
.create();

There is an easier way to almost get what you want. I'd not realised but since GSON 2.4 the SerializedName annotation supports an array of alternate names that GSON can deserialise from.
public class Foo {
#SerializedName(value = "firstName", alternate = {"firstname", "Firstname"})
private String firstName;
public java.lang.String getFirstName() {
return firstName;
}
#Override
public String toString() {
return "Foo:" + firstName;
}
}
You could use this to support the likely variations without being able to handle something like "firSTNAme".
https://google.github.io/gson/apidocs/com/google/gson/annotations/SerializedName.html
It doesn't look as though GSON has an easy built in way to customise it the way you want. It looks as though technically you could implement a TypeAdapterFactory that uses reflection to do this but it seems excessive.
You might open a feature request with GSON about introducing support for field naming strategies that support alternative names or pattern, or case insensitive matching.

Not a complete solution to this problem but if the keys happen to be all lowercase such as headimgurl then you can implement FieldNamingStrategy interface
Gson gson = new GsonBuilder().setFieldNamingStrategy(f -> f.getName().toLowerCase()).create();
to parse the value to headImgUrl field of the object.

I think you will need to implement a custom JsonDeserialiser.
The field naming policy and strategy appears to provide a way to map Java field names to JSON properties but not JSON properties to Java field names.
This deserialiser will ignore the case of the name of the property and try to match it against "firstname".
public final class FooDeserialiser implements JsonDeserializer<Foo> {
#Override
public Foo deserialize(
JsonElement jsonElement,
Type type,
JsonDeserializationContext jsonDeserializationContext)
throws JsonParseException {
for (Map.Entry<String, JsonElement> property : jsonElement.getAsJsonObject().entrySet()) {
if ("firstname".equalsIgnoreCase(property.getKey())) {
return new Foo(property.getValue().getAsString());
}
}
// Or return null if you prefer, or return a Foo with null as the first name
// It has failed to find any property that looks like firstname
throw new JsonParseException("No firstName property");
}
}
This can be registered as a type adapter with the Gson object when it is being built like this:
final Gson gson = new GsonBuilder()
.registerTypeAdapter(Foo.class, new FooDeserialiser())
.setPrettyPrinting()
.create();
And then invoked like:
final List<Foo> foos = gson.fromJson(
"[{\"firstName\":\"Juan\"},{\"firstname\":\"Juan\"},{\"Firstname\":\"Juan\"}]",
new TypeToken<List<Foo>>() {}.getType());
To return a list for Foo objects each with the first name Juan.
The only problem is building the deserialisers for your objects may become a burden. Your deserialisers will need to be more complicated than the above example.

How to get the underlying String from a JsonParser (Jackson Json)

Looking through the documentation and source code I don't see a clear way to do this. Curious if I'm missing something.
Say I receive an InputStream from a server response. I create a JsonParser from this InputStream. It is expected that the server response is text containing valid JSON, such as:
{"iamValidJson":"yay"}
However, if the response ends up being invalid JSON or not JSON at all such as:
Some text that is not JSON
the JsonParser will eventually throw an exception. In this case, I would like to be able to extract the underlying invalid text "Some text that is not JSON" out of the JsonParser so it can be used for another purpose.
I cannot pull it out of the InputStream because it doesn't support resetting and the creation of the JsonParser consumes it.
Is there a way to do this?

If you have the JsonParser then you can use jsonParser.readValueAsTree().toString().
However, this likely requires that the JSON being parsed is indeed valid JSON.

I had a situation where I was using a custom deserializer, but I wanted the default deserializer to do most of the work, and then using the SAME json do some additional custom work. However, after the default deserializer does its work, the JsonParser object current location was beyond the json text I needed. So I had the same problem as you: how to get access to the underlying json string.
You can use JsonParser.getCurrentLocation.getSourceRef() to get access to the underlying json source. Use JsonParser.getCurrentLocation().getCharOffset() to find the current location in the json source.
Here's the solution I used:
public class WalkStepDeserializer extends StdDeserializer<WalkStep> implements
ResolvableDeserializer {
// constructor, logger, and ResolvableDeserializer methods not shown
#Override
public MyObj deserialize(JsonParser jp, DeserializationContext ctxt) throws IOException,
JsonProcessingException {
MyObj myObj = null;
JsonLocation startLocation = jp.getCurrentLocation();
long charOffsetStart = startLocation.getCharOffset();
try {
myObj = (MyObj) defaultDeserializer.deserialize(jp, ctxt);
} catch (UnrecognizedPropertyException e) {
logger.info(e.getMessage());
}
JsonLocation endLocation = jp.getCurrentLocation();
long charOffsetEnd = endLocation.getCharOffset();
String jsonSubString = endLocation.getSourceRef().toString().substring((int)charOffsetStart - 1, (int)charOffsetEnd);
logger.info(strWalkStep);
// Special logic - use JsonLocation.getSourceRef() to get and use the entire Json
// string for further processing
return myObj;
}
}
And info about using a default deserializer in a custom deserializer is at How do I call the default deserializer from a custom deserializer in Jackson

5 year late, but this was my solution:
I converted the jsonParser to string
String requestString = jsonParser.readValueAsTree().toString();
Then I converted that string into a JsonParser
JsonFactory factory = new JsonFactory();
JsonParser parser = factory.createParser(requestString);
Then I iterated through my parser
ObjectMapper objectMapper = new ObjectMapper();
while(!parser.isClosed()){
JsonToken jsonToken = parser.nextToken();
if(JsonToken.FIELD_NAME.equals(jsonToken)){
String currentName = parser.getCurrentName();
parser.nextToken();
switch (currentName) {
case "someObject":
Object someObject = objectMapper.readValue(parser, Object.class)
//validate someObject
break;
}
}
I needed to save the original json string for logging purposes, which is why I did this in the first place. Was a headache to find out, but finally did it and I hope i'm helping someone out :)

Building my own Deserialiser in which I wanted to deserialise a specific field as text i.s.o. a proper DTO, this is the solution I came up with.
I wrote my own JsonToStringDeserializer like this:
import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.core.TreeNode;
import com.fasterxml.jackson.databind.DeserializationContext;
import com.fasterxml.jackson.databind.JsonDeserializer;
import lombok.NoArgsConstructor;
import org.apache.commons.lang3.StringEscapeUtils;
import java.io.IOException;
/**
* Deserialiser to deserialise any Json content to a String.
*/
#NoArgsConstructor
public class JsonToStringDeserializer extends JsonDeserializer<String> {
/**
* Deserialise a Json attribute that is a fully fledged Json object, into a {#link String}.
* #param jsonParser Parsed used for reading JSON content
* #param context Context that can be used to access information about this deserialization activity.
* #return The deserialized value as a {#link String}.
* #throws IOException
*/
#Override
public String deserialize(JsonParser jsonParser, DeserializationContext context) throws IOException {
final TreeNode node = jsonParser.getCodec().readTree(jsonParser);
final String unescapedString = StringEscapeUtils.unescapeJava(node.toString());
return unescapedString.substring(1, unescapedString.length()-1);
}
}
Annotate the field you want to deserialize like this:
#JsonDeserialize(using = JsonToStringDeserializer.class)
I initially followed advice that said to use a TreeNode like this:
final TreeNode treeNode = jsonParser.getCodec().readTree(jsonParser);
return treeNode.toString();
But then you get a Json String that contains escape characters.

What you are trying to do is outside the scope of Jackson (and most, if not all other Java JSON libraries out there). What you want to do is fully consume the input stream into a string, then attempt to convert that string to a JSON object using Jackson. If the conversion fails then do something with the intermediate string, else proceed normally. Here's an example, which utilizes the excellent Apache Commons IO library, for convenience:
final InputStream stream ; // Your stream here
final String json = IOUtils.toString(stream);
try {
final JsonNode node = new ObjectMapper().readTree(json);
// Do something with JSON object here
} catch(final JsonProcessingException jpe) {
// Do something with intermediate string here
}

How to use JSONDeserializer to deserialize this string?

I have string like follows:
{"226167":"myshow","3193":"yourshow"}
How can I use JSONDeserializer to extract (226167,3193) from the above string object?
I probably want to have a list (226167,3193,...) from the above string. I am using flexjason 1.9.2 and it doesn't have jsonObject class.

Use the jackson JSON library's ObjectMapper class to deserialize into a Map and then get the keySet():
Required Import:
import org.codehaus.jackson.map.ObjectMapper;
Example:
public void readValueAsMap() throws Exception
{
String value = "{\"226167\":\"myshow\",\"3193\":\"yourshow\"}";
ObjectMapper mapper = new ObjectMapper();
Map<String,String> valueAsMap = mapper.readValue(value, Map.class);
Collection<String> values = valueAsMap.keySet();
assertTrue(values.contains("226167"));
assertTrue(values.contains("3193"));
}