Develop Reference

Odd behaviour with hibernate JSON converter

I am having some trouble with converting a Set of POJO's to a JSON string so I can store it in a database column. The conversion to JSON works as expected but when the conversion from JSON to a Set< Qualification > happens it always returns a LinkedHashSet<LinkedHashMap> which is causing issues.
The weird thing is that inside my converter the JSON string is converted successfully to a Set<Qualification>. When I debug in my IDE and step through the execution I can see that after it calls a deepCopy method in the MutableMutabilityPlan abstract class. At this point the data is of type LinkedHashSet<LinkedHashMap> and not Set<Qualification> from the conversion.
Here is my converter.
public class SetToStringConverter implements AttributeConverter<Set<Qualification>, String> {
private final ObjectMapper mapper = new ObjectMapper();
private final String errorMessage = "converter.invalid";
#Override
public String convertToDatabaseColumn(final Set<Qualification> items) {
try {
return mapper.writeValueAsString(items);
} catch (JsonProcessingException e) {
throw new ConverterFailureException(errorMessage);
}
}
#Override
public Set<Qualification> convertToEntityAttribute(final String data) {
try {
if (data != null) {
final Set<Qualification> s = mapper.readValue(data, new TypeReference<Set<Qualification>>() {});
return s; // Debugging here I have the correct type
}
return new HashSet<>();
} catch (IOException e) {
throw new ConverterFailureException(errorMessage);
}
}
}
I have done some research and have tried various approaches but the result is always the same.
Has anyone run into this issue before or can see anything wrong with my converter. Let me know if anything isn't clear so I can provide more information.
Thanks very much for the help.

Make mockito less strict with object comparison

I have the following code:
public JSONObject addProductToOrder(int orderId, int productId, String name, double price) {
JSONObject json = new JSONObject();
try {
json.put("orderId", orderId);
json.put("productId", productId);
json.put("name", name);
json.put("price", price);
} catch (JSONException e) {
Debugger.out(e);
}
return this.dataSource.write("/orderItems/addproductToOrder", json);
}
I want to test so using Mockito and I have done the following:
public void getOrder() throws JSONException {
JSONObject json = new JSONObject("{result: 'OK', error: 0}");
doReturn(json)
.when(this.clientSpy) // this.clientSpy = Mockito.spy(client);
.post("/orderItems/addProductToOrder", new JSONObject("{productId: 1, orderId: 1, price: 15.99, name: 'Product 1'}"));
OrderItem orderItem = new OrderItem(this.api);
assertEquals("OK", orderItem.addProductToOrder(1, 1, "Product 1", 15.99).get("result"));
}
The way I understand things, my doReturn() is not triggered because new JSONObject() !== new JSONObject().
Is there a way to make it not compare the objects, but instead just the contents?

What happens here is that Mockito is calling equals() to compare the object that you provided to the specification to the one that is used for the actual method call.
Basically, there are three options:
if you just provide an object, equals() is used to compare
then you have a variety of other ArgumentMatchers, such as any() or isNull(). Please note: when using such matchers, all arguments must be matched, you can't do: foo(any(), 5) for example.
if that doesn't do: you can also use an ArgumentCaptor. Meaning: instead of having Mockito compare objects, you simply record the object that was passed to the mock. And then you add another step in your test case where you check the captured argument as required.

This argThat function can help you.
private static String eqJson(String expectedJson) {
return argThat(argument -> {
try {
JSONAssert.assertEquals(expectedJson, argument, true);
return true;
} catch (JSONException e) {
return false;
}
});
}
Then in your test
// import
import org.skyscreamer.jsonassert.JSONAssert;
import static org.mockito.ArgumentMatchers.argThat;
// then
then(mock).should(times(1)).method(eqJson(expectedJson));

Ignore specific nodes/attributes while comparing two JSONs

I want to compare two JSON strings which is a huge hierarchy and want to know where they differ in values. But some values are generated at runtime and are dynamic. I want to ignore those particular nodes from my comparison.
I am currently using JSONAssert from org.SkyScreamer to do the comparison. It gives me nice console output but does not ignore any attributes.
for ex.
java.lang.AssertionError messageHeader.sentTime
expected:null
got:09082016 18:49:41.123
Now this comes dynamic and should be ignored. Something like
JSONAssert.assertEquals(expectedJSONString, actualJSONString,JSONCompareMode, *list of attributes to be ignored*)
It would be great if someone suggests a solution in JSONAssert. However other ways are also welcome.

You can use Customization for this. For example, if you need to ignore a top-level attribute named "timestamp" use:
JSONAssert.assertEquals(expectedResponseBody, responseBody,
new CustomComparator(JSONCompareMode.LENIENT,
new Customization("timestamp", (o1, o2) -> true)));
It's also possible to use path expressions like "entry.id". In your Customization you can use whatever method you like to compare the two values. The example above always returns true, no matter what the expected value and the actual value are. You could do more complicated stuff there if you need to.
It is perfectly fine to ignore that values of multiple attributes, for example:
#Test
public void ignoringMultipleAttributesWorks() throws JSONException {
String expected = "{\"timestamp\":1234567, \"a\":5, \"b\":3 }";
String actual = "{\"timestamp\":987654, \"a\":1, \"b\":3 }";
JSONAssert.assertEquals(expected, actual,
new CustomComparator(JSONCompareMode.LENIENT,
new Customization("timestamp", (o1, o2) -> true),
new Customization("a", (o1, o2) -> true)
));
}
There is one caveat when using Customizations: The attribute whose value is to be compared in a custom way has to be present in the actual JSON. If you want the comparison to succeed even if the attribute is not present at all you would have to override CustomComparator for example like this:
#Test
public void extendingCustomComparatorToAllowToCompletelyIgnoreCertainAttributes() throws JSONException {
// AttributeIgnoringComparator completely ignores some of the expected attributes
class AttributeIgnoringComparator extends CustomComparator{
private final Set<String> attributesToIgnore;
private AttributeIgnoringComparator(JSONCompareMode mode, Set<String> attributesToIgnore, Customization... customizations) {
super(mode, customizations);
this.attributesToIgnore = attributesToIgnore;
}
protected void checkJsonObjectKeysExpectedInActual(String prefix, JSONObject expected, JSONObject actual, JSONCompareResult result) throws JSONException {
Set<String> expectedKeys = getKeys(expected);
expectedKeys.removeAll(attributesToIgnore);
for (String key : expectedKeys) {
Object expectedValue = expected.get(key);
if (actual.has(key)) {
Object actualValue = actual.get(key);
compareValues(qualify(prefix, key), expectedValue, actualValue, result);
} else {
result.missing(prefix, key);
}
}
}
}
String expected = "{\"timestamp\":1234567, \"a\":5}";
String actual = "{\"a\":5}";
JSONAssert.assertEquals(expected, actual,
new AttributeIgnoringComparator(JSONCompareMode.LENIENT,
new HashSet<>(Arrays.asList("timestamp")))
);
}
(With this approach you still could use Customizations to compare other attributes' values in the way you want.)

you can use JsonUnit It has the functionality that you are looking for we can ignore fields, paths, and values that are null etc. Check it out for more info. As for the example, you can ignore a path like this
assertJsonEquals(
"{\"root\":{\"test\":1, \"ignored\": 2}}",
"{\"root\":{\"test\":1, \"ignored\": 1}}",
whenIgnoringPaths("root.ignored")
);
Sometimes you need to ignore certain values when comparing. It is possible to use ${json-unit.ignore} placeholder like this
assertJsonEquals("{\"test\":\"${json-unit.ignore}\"}",
"{\n\"test\": {\"object\" : {\"another\" : 1}}}");

First of all there is open issue for it.
In my tests I compare json from controller with actual object with help of JsonUtil class for serialization/deserialization:
public class JsonUtil {
public static <T> List<T> readValues(String json, Class<T> clazz) {
ObjectReader reader = getMapper().readerFor(clazz);
try {
return reader.<T>readValues(json).readAll();
} catch (IOException e) {
throw new IllegalArgumentException("Invalid read array from JSON:\n'" + json + "'", e);
}
}
public static <T> T readValue(String json, Class<T> clazz) {
try {
return getMapper().readValue(json, clazz);
} catch (IOException e) {
throw new IllegalArgumentException("Invalid read from JSON:\n'" + json + "'", e);
}
}
public static <T> String writeValue(T obj) {
try {
return getMapper().writeValueAsString(obj);
} catch (JsonProcessingException e) {
throw new IllegalStateException("Invalid write to JSON:\n'" + obj + "'", e);
}
}
To ignore specific object field I've add new method:
public static <T> String writeIgnoreProps(T obj, String... ignoreProps) {
try {
Map<String, Object> map = getMapper().convertValue(obj, new TypeReference<Map<String, Object>>() {});
for (String prop : ignoreProps) {
map.remove(prop);
}
return getMapper().writeValueAsString(map);
} catch (JsonProcessingException e) {
throw new IllegalStateException("Invalid write to JSON:\n'" + obj + "'", e);
}
}
and my assert in test now look like this:
mockMvc.perform(get(REST_URL))
.andExpect(status().isOk())
.andExpect(content().contentTypeCompatibleWith(MediaType.APPLICATION_JSON))
.andExpect(content().json(JsonUtil.writeIgnoreProps(USER, "registered")))

Thank you #dknaus for the detailed answer. Although this solution will not work in STRICT mode and checkJsonObjectKeysExpectedInActual method code needs to be replaced by following code [As suggested by #tk-gospodinov]:
for (String attribute : attributesToIgnore) {
expected.remove(attribute);
super.checkJsonObjectKeysExpectedInActual(prefix, expected, actual, result);
}

Looking for Json-path/(any API) to update any value in given json string in Java

Inshort : I am trying to find some api that could just change the value by taking first parameter as jsonString , second parameter as JSONPath and third will be new value of that parameter. But, all I found is this..
https://code.google.com/p/json-path/
This api allows me to find any value in JSON String. But, I am not finding easy way to update the value of any key. For example, Here is a book.json.
{
"store":{
"book":[
{
"category":"reference",
"author":"Nigel Rees",
"title":"Sayings of the Century",
"price":8.95
},
{
"category":"fiction",
"author":"Evelyn Waugh",
"title":"Sword of Honour",
"price":12.99,
"isbn":"0-553-21311-3"
}
],
"bicycle":{
"color":"red",
"price":19.95
}
}
}
I can access color of bicycle by doing this.
String bicycleColor = JsonPath.read(json, "$.store.bicycle.color");
But I am looking for a method in JsonPath or other api some thing like this
JsonPath.changeNodeValue(json, "$.store.bicycle.color", "green");
String bicycleColor = JsonPath.read(json, "$.store.bicycle.color");
System.out.println(bicycleColor); // This should print "green" now.
I am excluding these options,
Create a new JSON String.
Create a JSON Object to deal with changing value and convert it back to jsonstring
Reason: I have about 500 different requests for different types of service which return different json structure. So, I do not want to manually create new JSON string always. Because, IDs are dynamic in json structure.
Any idea or direction is much appreciated.
Updating this question with following answer.
Copy MutableJson.java.
copy this little snippet and modify as per you need.
private static void updateJsonValue() {
JSONParser parser = new JSONParser();
JSONObject jsonObject = new JSONObject();
FileReader reader = null;
try {
File jsonFile = new File("path to book.json");
reader = new FileReader(jsonFile);
jsonObject = (JSONObject) parser.parse(reader);
} catch (Exception ex) {
System.out.println(ex.getLocalizedMessage());
}
Map<String, Object> userData = null;
try {
userData = new ObjectMapper().readValue(jsonObject.toJSONString(), Map.class);
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
MutableJson json = new MutableJson(userData);
System.out.println("Before:\t" + json.map());
json.update("$.store.book[0].author", "jigish");
json.update("$.store.book[1].category", "action");
System.out.println("After:\t" + json.map().toString());
}
Use these libraries.
import org.json.simple.JSONObject;
import org.json.simple.parser.JSONParser;
import org.codehaus.jackson.map.ObjectMapper;

The thing is that the functionality you want is already an undocumented feature of JsonPath. Example using your json structure:
String json = "{ \"store\":{ \"book\":[ { \"category\":\"reference\", \"author\":\"Nigel Rees\", \"title\":\"Sayings of the Century\", \"price\":8.95 }, { \"category\":\"fiction\", \"author\":\"Evelyn Waugh\", \"title\":\"Sword of Honour\", \"price\":12.99, \"isbn\":\"0-553-21311-3\" } ], \"bicycle\":{ \"color\":\"red\", \"price\":19.95 } } }";
DocumentContext doc = JsonPath.parse(json).
set("$.store.bicycle.color", "green").
set("$.store.book[0].price", 9.5);
String newJson = new Gson().toJson(doc.read("$"));

Assuming that parsed JSON can be represented in memory as a Map, you can build an API similar to JsonPath that looks like:
void update(Map<String, Object> json, String path, Object newValue);
I've quickly done a gist of a dirty implementation for simple specific paths (no support for conditions and wildcards) that can traverse json tree, E.g. $.store.name, $.store.books[0].isbn. Here it is: MutableJson.java. It definitely needs improvement, but can give a good start.
Usage example:
import java.util.*;
public class MutableJson {
public static void main(String[] args) {
MutableJson json = new MutableJson(
new HashMap<String, Object>() {{
put("store", new HashMap<String, Object>() {{
put("name", "Some Store");
put("books", Arrays.asList(
new HashMap<String, Object>() {{
put("isbn", "111");
}},
new HashMap<String, Object>() {{
put("isbn", "222");
}}
));
}});
}}
);
System.out.println("Before:\t" + json.map());
json.update("$.store.name", "Book Store");
json.update("$.store.books[0].isbn", "444");
json.update("$.store.books[1].isbn", "555");
System.out.println("After:\t" + json.map());
}
private final Map<String, Object> json;
public MutableJson(Map<String, Object> json) {
this.json = json;
}
public Map<String, Object> map() {
return json;
}
public void update(String path, Object newValue) {
updateJson(this.json, Path.parse(path), newValue);
}
private void updateJson(Map<String, Object> data, Iterator<Token> path, Object newValue) {
Token token = path.next();
for (Map.Entry<String, Object> entry : data.entrySet()) {
if (!token.accept(entry.getKey(), entry.getValue())) {
continue;
}
if (path.hasNext()) {
Object value = token.value(entry.getValue());
if (value instanceof Map) {
updateJson((Map<String, Object>) value, path, newValue);
}
} else {
token.update(entry, newValue);
}
}
}
}
class Path {
public static Iterator<Token> parse(String path) {
if (path.isEmpty()) {
return Collections.<Token>emptyList().iterator();
}
if (path.startsWith("$.")) {
path = path.substring(2);
}
List<Token> tokens = new ArrayList<>();
for (String part : path.split("\\.")) {
if (part.matches("\\w+\\[\\d+\\]")) {
String fieldName = part.substring(0, part.indexOf('['));
int index = Integer.parseInt(part.substring(part.indexOf('[')+1, part.indexOf(']')));
tokens.add(new ArrayToken(fieldName, index));
} else {
tokens.add(new FieldToken(part));
}
};
return tokens.iterator();
}
}
abstract class Token {
protected final String fieldName;
Token(String fieldName) {
this.fieldName = fieldName;
}
public abstract Object value(Object value);
public abstract boolean accept(String key, Object value);
public abstract void update(Map.Entry<String, Object> entry, Object newValue);
}
class FieldToken extends Token {
FieldToken(String fieldName) {
super(fieldName);
}
#Override
public Object value(Object value) {
return value;
}
#Override
public boolean accept(String key, Object value) {
return fieldName.equals(key);
}
#Override
public void update(Map.Entry<String, Object> entry, Object newValue) {
entry.setValue(newValue);
}
}
class ArrayToken extends Token {
private final int index;
ArrayToken(String fieldName, int index) {
super(fieldName);
this.index = index;
}
#Override
public Object value(Object value) {
return ((List) value).get(index);
}
#Override
public boolean accept(String key, Object value) {
return fieldName.equals(key) && value instanceof List && ((List) value).size() > index;
}
#Override
public void update(Map.Entry<String, Object> entry, Object newValue) {
List list = (List) entry.getValue();
list.set(index, newValue);
}
}
A JSON string can be easily parsed into a Map using Jackson:
Map<String,Object> userData = new ObjectMapper().readValue("{ \"store\": ... }", Map.class);

Just answering for folks landing on this page in future for reference.
You could consider using a Java implementation of jsonpatch. RFC can be found here
JSON Patch is a format for describing changes to a JSON document. It can be used to avoid sending a whole document when only a part has changed. When used in combination with the HTTP PATCH method it allows partial updates for HTTP APIs in a standards compliant way.
You can specify the operation that needs to be performed (replace, add....), json path at which it has to be performed, and the value which should be used.
Again, taking example from the RFC :
[
{ "op": "test", "path": "/a/b/c", "value": "foo" },
{ "op": "remove", "path": "/a/b/c" },
{ "op": "add", "path": "/a/b/c", "value": [ "foo", "bar" ] },
{ "op": "replace", "path": "/a/b/c", "value": 42 },
{ "op": "move", "from": "/a/b/c", "path": "/a/b/d" },
{ "op": "copy", "from": "/a/b/d", "path": "/a/b/e" }
]
For Java implementation, I have not used it myself, but you can give a try to https://github.com/fge/json-patch

So in order to change a value within a JSon string, there are two steps:
Parse the JSon
Modify the appropriate field
You are trying to optimize step 2, but understand that you are not going to be able to avoid step 1. Looking at the Json-path source code (which, really, is just a wrapper around Jackson), note that it does do a full parse of the Json string before being able to spit out the read value. It does this parse every time you call read(), e.g. it is not cached.
I think this task is specific enough that you're going to have to write it yourself. Here is what I would do:
Create an object that represents the data in the parsed Json string.
Make sure this object has, as part of it's fields, the Json String pieces that you do not expect to change often.
Create a custom Deserializer in the Json framework of your choice that will populate the fields correctly.
Create a custom Serializer that uses the cached String pieces, plus the data that you expect to change
I think the exact scope of your problem is unusual enough that it is unlikely a library already exists for this. When a program receives a Json String, most of the time what it wants is the fully deserialized object - it is unusual that it needs to FORWARD this object on to somewhere else.

JSON checker to avoid com.google.gson.stream.MalformedJsonException

I'm getting many JSON objects through a web service and at times the json object is malformed.
I wanted to check the if the json is valid before processing it.
So i worte
JsonElement jsonData = parser.parse(attacheddataattribute);
if(jsonData.isJsonObject())
{
//then only process
}
Not also its throwing a
com.google.gson.stream.MalformedJsonException: Unterminated string at line 1 column 8432 at the parse method.
Is there any implemenation available to check for the JSON's validity.

That's your validation. No need to call any service.
If the method is throwing the MalformedJsonException it's a malformed JSON.
If you want you can wrap it in a method like
public boolean isValidJson(String json) {
try {
// parse json
return true;
} catch(MalformedJsonException e) {
return false;
}
}

I also had the MalformedJsonException crash but in my case I needed to add a catch block with Throwable:
fun jsonToList(value: String?): MutableList<String> {
var objects: Array<String> = emptyArray()
try {
objects = Gson().fromJson(value, Array<String>::class.java)
}catch (t: Throwable){
}finally {
return objects.toMutableList()
}
}