I am having some trouble with converting a Set of POJO's to a JSON string so I can store it in a database column. The conversion to JSON works as expected but when the conversion from JSON to a Set< Qualification > happens it always returns a LinkedHashSet<LinkedHashMap> which is causing issues.
The weird thing is that inside my converter the JSON string is converted successfully to a Set<Qualification>. When I debug in my IDE and step through the execution I can see that after it calls a deepCopy method in the MutableMutabilityPlan abstract class. At this point the data is of type LinkedHashSet<LinkedHashMap> and not Set<Qualification> from the conversion.
Here is my converter.
public class SetToStringConverter implements AttributeConverter<Set<Qualification>, String> {
private final ObjectMapper mapper = new ObjectMapper();
private final String errorMessage = "converter.invalid";
#Override
public String convertToDatabaseColumn(final Set<Qualification> items) {
try {
return mapper.writeValueAsString(items);
} catch (JsonProcessingException e) {
throw new ConverterFailureException(errorMessage);
}
}
#Override
public Set<Qualification> convertToEntityAttribute(final String data) {
try {
if (data != null) {
final Set<Qualification> s = mapper.readValue(data, new TypeReference<Set<Qualification>>() {});
return s; // Debugging here I have the correct type
}
return new HashSet<>();
} catch (IOException e) {
throw new ConverterFailureException(errorMessage);
}
}
}
I have done some research and have tried various approaches but the result is always the same.
Has anyone run into this issue before or can see anything wrong with my converter. Let me know if anything isn't clear so I can provide more information.
Thanks very much for the help.
Related
I would like to make sure o is a serializable top level JSON object, that is [] or {} else throw an exception. I have tried the following code using "" and null as input but they are not triggering an exception.
static void checkIsjsonSerializable(Object o, String message)
throws MissingRequiredValueException {
try{
Gson gson = new Gson();
gson.toJson(o);
} catch (Exception e) {
throw new MissingRequiredValueException(message);
}
}
What would need to change to get the check I want?
Update:
After comments it clear my understanding was wrong. My question has change to:
How can I assert only [] and {} are valid in the following function?
As others have mentioned, modern definitions of JSON do allow primitives (strings, numbers, booleans, null) as top-level elements. But if you really need to do this check with GSON, here's one option:
private static final Gson gson = new Gson();
static void checkIsjsonSerializable(Object o, String message)
throws MissingRequiredValueException {
JsonElement rootElement = gson.toJsonTree(o);
if (!rootElement.isJsonArray() && !rootElement.isJsonObject()) {
throw new MissingRequiredValueException(message);
}
}
I want to compare two JSON strings which is a huge hierarchy and want to know where they differ in values. But some values are generated at runtime and are dynamic. I want to ignore those particular nodes from my comparison.
I am currently using JSONAssert from org.SkyScreamer to do the comparison. It gives me nice console output but does not ignore any attributes.
for ex.
java.lang.AssertionError messageHeader.sentTime
expected:null
got:09082016 18:49:41.123
Now this comes dynamic and should be ignored. Something like
JSONAssert.assertEquals(expectedJSONString, actualJSONString,JSONCompareMode, *list of attributes to be ignored*)
It would be great if someone suggests a solution in JSONAssert. However other ways are also welcome.
You can use Customization for this. For example, if you need to ignore a top-level attribute named "timestamp" use:
JSONAssert.assertEquals(expectedResponseBody, responseBody,
new CustomComparator(JSONCompareMode.LENIENT,
new Customization("timestamp", (o1, o2) -> true)));
It's also possible to use path expressions like "entry.id". In your Customization you can use whatever method you like to compare the two values. The example above always returns true, no matter what the expected value and the actual value are. You could do more complicated stuff there if you need to.
It is perfectly fine to ignore that values of multiple attributes, for example:
#Test
public void ignoringMultipleAttributesWorks() throws JSONException {
String expected = "{\"timestamp\":1234567, \"a\":5, \"b\":3 }";
String actual = "{\"timestamp\":987654, \"a\":1, \"b\":3 }";
JSONAssert.assertEquals(expected, actual,
new CustomComparator(JSONCompareMode.LENIENT,
new Customization("timestamp", (o1, o2) -> true),
new Customization("a", (o1, o2) -> true)
));
}
There is one caveat when using Customizations: The attribute whose value is to be compared in a custom way has to be present in the actual JSON. If you want the comparison to succeed even if the attribute is not present at all you would have to override CustomComparator for example like this:
#Test
public void extendingCustomComparatorToAllowToCompletelyIgnoreCertainAttributes() throws JSONException {
// AttributeIgnoringComparator completely ignores some of the expected attributes
class AttributeIgnoringComparator extends CustomComparator{
private final Set<String> attributesToIgnore;
private AttributeIgnoringComparator(JSONCompareMode mode, Set<String> attributesToIgnore, Customization... customizations) {
super(mode, customizations);
this.attributesToIgnore = attributesToIgnore;
}
protected void checkJsonObjectKeysExpectedInActual(String prefix, JSONObject expected, JSONObject actual, JSONCompareResult result) throws JSONException {
Set<String> expectedKeys = getKeys(expected);
expectedKeys.removeAll(attributesToIgnore);
for (String key : expectedKeys) {
Object expectedValue = expected.get(key);
if (actual.has(key)) {
Object actualValue = actual.get(key);
compareValues(qualify(prefix, key), expectedValue, actualValue, result);
} else {
result.missing(prefix, key);
}
}
}
}
String expected = "{\"timestamp\":1234567, \"a\":5}";
String actual = "{\"a\":5}";
JSONAssert.assertEquals(expected, actual,
new AttributeIgnoringComparator(JSONCompareMode.LENIENT,
new HashSet<>(Arrays.asList("timestamp")))
);
}
(With this approach you still could use Customizations to compare other attributes' values in the way you want.)
you can use JsonUnit It has the functionality that you are looking for we can ignore fields, paths, and values that are null etc. Check it out for more info. As for the example, you can ignore a path like this
assertJsonEquals(
"{\"root\":{\"test\":1, \"ignored\": 2}}",
"{\"root\":{\"test\":1, \"ignored\": 1}}",
whenIgnoringPaths("root.ignored")
);
Sometimes you need to ignore certain values when comparing. It is possible to use ${json-unit.ignore} placeholder like this
assertJsonEquals("{\"test\":\"${json-unit.ignore}\"}",
"{\n\"test\": {\"object\" : {\"another\" : 1}}}");
First of all there is open issue for it.
In my tests I compare json from controller with actual object with help of JsonUtil class for serialization/deserialization:
public class JsonUtil {
public static <T> List<T> readValues(String json, Class<T> clazz) {
ObjectReader reader = getMapper().readerFor(clazz);
try {
return reader.<T>readValues(json).readAll();
} catch (IOException e) {
throw new IllegalArgumentException("Invalid read array from JSON:\n'" + json + "'", e);
}
}
public static <T> T readValue(String json, Class<T> clazz) {
try {
return getMapper().readValue(json, clazz);
} catch (IOException e) {
throw new IllegalArgumentException("Invalid read from JSON:\n'" + json + "'", e);
}
}
public static <T> String writeValue(T obj) {
try {
return getMapper().writeValueAsString(obj);
} catch (JsonProcessingException e) {
throw new IllegalStateException("Invalid write to JSON:\n'" + obj + "'", e);
}
}
To ignore specific object field I've add new method:
public static <T> String writeIgnoreProps(T obj, String... ignoreProps) {
try {
Map<String, Object> map = getMapper().convertValue(obj, new TypeReference<Map<String, Object>>() {});
for (String prop : ignoreProps) {
map.remove(prop);
}
return getMapper().writeValueAsString(map);
} catch (JsonProcessingException e) {
throw new IllegalStateException("Invalid write to JSON:\n'" + obj + "'", e);
}
}
and my assert in test now look like this:
mockMvc.perform(get(REST_URL))
.andExpect(status().isOk())
.andExpect(content().contentTypeCompatibleWith(MediaType.APPLICATION_JSON))
.andExpect(content().json(JsonUtil.writeIgnoreProps(USER, "registered")))
Thank you #dknaus for the detailed answer. Although this solution will not work in STRICT mode and checkJsonObjectKeysExpectedInActual method code needs to be replaced by following code [As suggested by #tk-gospodinov]:
for (String attribute : attributesToIgnore) {
expected.remove(attribute);
super.checkJsonObjectKeysExpectedInActual(prefix, expected, actual, result);
}
Struts 2 has implicit type conversion which take cares of user entered params type cast e.g. int, string , double ,boolean etc.
But my requirement is to convert rich text area input to byte array and for that I have created a custom type converter class.
public class StringToByteArrayConverter extends StrutsTypeConverter{
#Override
public Object convertFromString(Map context, String[] value, Class arg2) {
String val = value[0];
return val.getBytes() ;
}
#Override
public String convertToString(Map context, Object value) {
ByteArrayOutputStream out = new ByteArrayOutputStream();
ObjectOutputStream os;
try {
os = new ObjectOutputStream(out);
os.writeObject(value);
return new String(out.toByteArray());
} catch (IOException e) {
e.printStackTrace();
return null;
}
}
}
And in model class I have specified the following annotation on setter of property
#TypeConversion(converter="org.package.name.StringToByteArrayConverter")
public void setVarName(byte[] varName) {
this.varName = varName;
}
The same annotation is applyed on getter method also.
Now everything seems fine I am getting correct data in Action method. But while displaying the data on jsp I am getting some extra symbols with original content.
eg. user input is :what is your name ?
it display on jsp : ¬íur[B¬óøTàxpwhat is your name ?
Any one has any Idea, What am I dong wrong ?
Start by specifying the correct Charset in the byte-to-string processing:
val.getBytes(); // wrong
val.getBytes("UTF-8"); // right
assuming you're working on UTF-8. Otherwise, just put the charset you're using, but never use val.getBytes(); that will take the platform-default charset, that might differ from your app's charset, creating conversion errors and artifacts like the ones you're getting now.
Then the ObjectOutputStream doesn't convince me. Try with a simple
#Override
public String convertToString(Map context, Object value) {
try {
return new String((byte[]) value, "UTF-8");
} catch (Exception e) {
e.printStackTrace();
return null;
}
}
and eventually add logging for wrong usage, eg: if (value instanceof byte[]).. else LOG.warn....
Sample of NdJson data:
{"type":"data","id":"xyz"}
{"type":"value","id":"xcf"}
....
....
Here is my Retrofit and RxJava code which is working fine for fetching data with the limit=1 i.e. {"type":"data","id":"xyz"}.
adapter.create(API.class).getData()
.subscribeOn(Schedulers.newThread())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(new Observer<APIData>() {
#Override
public void onCompleted() {}
#Override
public void onError(Throwable e) {}
#Override
public void onNext(APIData apidata) {}
});
My model class
Model class have only two parameter:
public class APIData(){
private String type;
private String id;
..// Getter and setter for above two fields
}
Api class
public interface WarehouseAPI {
#GET("/search?limit=1")
public Observable<APIData> getdata ();
}
The error I am getting while replacing #GET("/search?limit=1") with #GET("/search") is Malformed JSON: Syntax error.
Question
How to proper parse the NdJson ?
Is there any way to store the response in List<APIData>
Edit-1
Now I am trying to accept the generic Response in observer:
adapter.create(APIData.class).getdata()
.subscribeOn(Schedulers.newThread())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(new Observer<Response>() {
#Override
public void onCompleted() {}
#Override
public void onNext(Response response) {}
});
But getting no suitable method found for subscribe(<anonymous Observer<Response>>)
Edit-2
However, I was doing some silly mistake and the error what I am getting in "Edit-1" section is fixed now. Now I am getting
Error:retrofit.RetrofitError: com.google.gson.JsonSyntaxException: com.google.gson.stream.MalformedJsonException: Use JsonReader.setLenient(true) to accept malformed JSON at line 2 column 2 path $
To make things work I had to add Custom converter implements by Retrofit.Converter:
public class CustomConverter implements Converter {
#Override
public Object fromBody(TypedInput body, Type type) throws ConversionException {
String text = null;
try {
text = fromStream(body.in());
} catch (IOException e) {
e.printStackTrace();
}
return text;
}
#Override
public TypedOutput toBody(Object object) {
return null;
}
// Custom method to convert stream from request to string
public static String fromStream(InputStream in) throws IOException {
BufferedReader reader = new BufferedReader(new InputStreamReader(in));
StringBuilder out = new StringBuilder();
String newLine = System.getProperty("\n");
String line;
while ((line = reader.readLine()) != null) {
out.append(line);
out.append(newLine);
}
return out.toString();
}
}
And it works perfectly!
I assume that server response has the following format
{"type":"data","id":"xyz"}\n{"type":"data","id":"xyz"}
The basic idea is to receive the response from the server as the String. Split it into array like response.split("\n"). Iterate through an Array creating a new json object for every array element.
I do realize it's quite time consuming as you've described in the comments. You can also try to play with String replaceAll method to transform each line into an array element and parse the whole string. like
String myResponse = "[" + response.replaceAll("/\n/", ",") + "]";
Gson gson = new Gson();
MyEntity[] arr = gson.fromJson(myResponse, MyEntity[].class);
In case of Retrofit. You will have to use a custom Response Converter.
I won't write up the complete solution since you have found the way to get it done using custom Converter.
For starters, your json is not json that Gson understand, so you can't use the Gson converter in Retrofit directly. So
public Observable<APIData> getdata ();
must be
public Observable<Response> getdata ();
Then, you have
adapter.create(WarehouseAPI.class).getdata()
.subscribeOn(Schedulers.newThread())
.observeOn(AndroidSchedulers.mainThread())
.map((Response res) -> new BufferedStreamReader(res.body.body().bytesStream()))
.flatMap(stream -> {
return Observable.create(subscriber -> {
String line;
while((line = stream.readLine() != null) {
subscriber.onNext(gson.fromJson(line, APIData.class));
}
subscriber.onCompleted();
});
});
You can subscribe to that and receive each individual item of your json list.
This is not tested, and error cases are not handled (and validity of subscription is not tested, for shortness).
It should mostly be in the right direction though.
I need to implement JSON serialization for some objects, and I've encountered a problem when it came to integration with generic collections.
All serializable classes implement this interface (JSONObject comes from this library):
interface JSONSerializable{
public JSONObject dump() throws JSONException //serializes object
public void load(JSONObject obj) throws JSONException //deserializes object
}
Code for my collection based on java.util.list looks more or less like this:
class AwesomeList<T extends JSONSerializable> implements JSONSerializable{
private LinkedList<T> items = new LinkedList<T>();
...
...
public JSONObject dump() throws JSONException {
JSONObject result = new JSONObject();
JSONArray a = new JSONArray();
for(T i : items){
a.put(i.dump());
}
result.put("items", a);
return result;
}
public void load(JSONObject obj) throws JSONException{
//here is my problem
}
}
My problem is: When I load AwesomeList from JSONObject, I need to create its elements but it's impossible since java forbids me to write
T newItem = new T();
newItem.load(obj);
How should I modify my approach to this task?
Are you tied to this library? Google Gson is very popular. I have myself not used it with Generics but their front page says Gson considers support for Generics very important.
As others have hinted, you should consider dumping org.json's library. It's pretty much obsolete these days, and trying to work around its problems is waste of time.
But to specific question; type variable T just does not have any information to help you, as it is little more than compile-time information.
Instead you need to pass actual class (as 'Class cls' argument), and you can then create an instance with 'cls.newInstance()'.
Well, when writing it out to file, you do know what class T is, so you can store that in dump. Then, when reading it back in, you can dynamically call it using reflection.
public JSONObject dump() throws JSONException {
JSONObject result = new JSONObject();
JSONArray a = new JSONArray();
for(T i : items){
a.put(i.dump());
// inside this i.dump(), store "class-name"
}
result.put("items", a);
return result;
}
public void load(JSONObject obj) throws JSONException {
JSONArray arrayItems = obj.getJSONArray("items");
for (int i = 0; i < arrayItems.length(); i++) {
JSONObject item = arrayItems.getJSONObject(i);
String className = item.getString("class-name");
try {
Class<?> clazzy = Class.forName(className);
T newItem = (T) clazzy.newInstance();
newItem.load(obj);
items.add(newItem);
} catch (InstantiationException e) {
// whatever
} catch (IllegalAccessException e) {
// whatever
} catch (ClassNotFoundException e) {
// whatever
}
}