I am writing Spring Boot application using Spring Data Rest repositories and I want to deny access to resource if request body contains JSON that has unknown properties. Definition of simplified entity and repository:
#Entity
public class Person{
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
private long id;
private String firstName;
private String lastName;
/* getters and setters */
}
#RepositoryRestResource(collectionResourceRel = "people", path = "people")
public interface PersonRepository extends CrudRepository<Person, Long> {}
I use Jackson's deserialization feature to disallow unknown properties in JSONs.
#Bean
public Jackson2ObjectMapperBuilder objectMapperBuilder(){
Jackson2ObjectMapperBuilder builder = new Jackson2ObjectMapperBuilder();
builder.failOnUnknownProperties(true);
return builder;
}
When I send POST requests everything works as expected. When I use valid fields I get correct response:
curl -i -x POST -H "Content-Type:application/json" -d '{"firstName": "Frodo", "lastName": "Baggins"}' http://localhost:8080/people
{
"firstName": "Frodo",
"lastName": "Baggins",
"_links": {...}
}
And when I send JSON with unknown fields application throws expected error:
curl -i -x POST -H "Content-Type:application/json" -d '{"unknown": "POST value", "firstName": "Frodo", "lastName": "Baggins"}' http://localhost:8080/people
com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "unknown" (class Person), not marked as ignorable (2 known properties: "lastName", "firstName")
PUT method when using valid JSON returns correct response as well. However when I send PUT request with unknown field I expect Spring to throw error but instead of that, Spring updates object in database and returns it:
curl -i -x PUT -H "Content-Type:application/json" -d '{"unknown": "PUT value", "firstName": "Bilbo", "lastName": "Baggins"}' http://localhost:8080/people/1
{
"firstName": "Bilbo",
"lastName": "Baggins",
"_links": {...}
}
The error is thrown only when there is no object in database with given id:
curl -i -x PUT -H "Content-Type:application/json" -d '{"unknown": "PUT value", "firstName": "Gandalf", "lastName": "Baggins"}' http://localhost:8080/people/100
com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "unknown" (class Person), not marked as ignorable (2 known properties: "lastName", "firstName")
Is it expected behavior or a bug in Spring Data Rest? How can I throw an error when JSON with unknown properties is passed to application no matter what request method is?
I've reproduced this behavior by modifying http://spring.io/guides/gs/accessing-data-rest/ , the only change I've made is Jackson2ObjectMapperBuilder , no other controllers or repositories are in this project.
I think the behaviour you are observing is by design. When a POST is issued you are creating the resource so the JSON is deserialised into your entity type and Jackson is performing this task.
A PUT is working differently in spring data rest. The interesting part is handled in PersistentEntityResourceHandlerMethodArgumentResolver.readPutForUpdate.
The json is read into a JsonNode, the entity is read from the data store and then in DomainObjectReader.doMerge the implementation iterates over the json fields. It applies the json to the entity and saves it later in the controller implementation. It also discards all the fields that do not exist in the persistent entity:
if (!mappedProperties.hasPersistentPropertyForField(fieldName)) {
i.remove();
continue;
}
This is my understanding from reading the code. I think you could argue that this is a bug. You could try to report it at spring data rest`s jira - https://jira.spring.io/browse/DATAREST. As far as I know there is no way to customise this behaviour.
When it creates new entity then it converts json directly to java entity object through deserialization process where required validation is involved. But when it updates existing entity then it converts json to JsonNode and then merge with existing entity and as expected no validation happens because it is feature for json deserialization to java object.
As workaround you can additionally convert JsonNode to entity object and it will work as you expect.
I did quick example how to gain required validation.
go to https://github.com/valery-barysok/gs-accessing-data-rest
It is not clear solution but you can improve it :)
This example override existing spring class on classpath org.springframework.data.rest.webmvc.config.PersistentEntityResourceHandlerMethodArgumentResolver
Note You must put this class on classpath before original version.
I did copy-past this class to project and modified readPutForUpdate method:
private Object readPutForUpdate(IncomingRequest request, ObjectMapper mapper, Object existingObject,
RootResourceInformation information) {
try {
JsonPatchHandler handler = new JsonPatchHandler(mapper, reader);
JsonNode jsonNode = mapper.readTree(request.getBody());
// Here we have required validation
mapper.treeToValue(jsonNode, information.getDomainType());
return handler.applyPut((ObjectNode) jsonNode, existingObject);
} catch (Exception o_O) {
throw new HttpMessageNotReadableException(String.format(ERROR_MESSAGE, existingObject.getClass()), o_O);
}
}
and i used application.properties file to configure DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES
you can annotate your model with :
#Entity
#JsonIgnoreProperties(ignoreUnknown=false)
public class Person{
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
private long id;
private String firstName;
private String lastName;
/* getters and setters */
}
You're using Jackson2ObjectMapperBuilder, which has the property DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES set to the disabled value by default.
Related
The problem I'm having with Avro is that we are required to use title case names for our data model, but camel case for the rest of our data structures such as DTOs. Here is an example:
{
"name": "UserRecord",
"type": "record",
"fields": [
{
"name": "Username",
"type": "string"
}
]
}
#Data
#NoArgsConstructor
#AllArgsConstructor
public class CreateUserRequestDTO {
private UserRecord userRecord;
}
The Maven Avro plugin will generate the Username field like this:
private String Username;
So we have an ugly mix in our code of fields that are title-cased in some cases and camel cased in others. Is there a good way to make Jackson's ObjectMapper able to read both of these property name conventions?
The case mixing is resulting in the Avro fields that are title-cased not getting recognized by Spring Boot's ObjectMapper and all of their fields are null.
I do not know if it will actually work in your use case, but maybe you can enable the Jackson's mapper configuration property ACCEPT_CASE_INSENSITIVE_PROPERTIES:
If enabled, the bean properties will be matched using their lower-case equivalents, meaning that any case-combination (incoming and matching names are canonicalized by lower-casing) should work.
You can customize the value in your application configuration file under the configuration key spring.jackson.mapper.*, something like:
spring:
jackson:
mapper:
ACCEPT_CASE_INSENSITIVE_PROPERTIES: true
I want to deserialize an object which is annotated with #JsonRootName. However the JSON in which the object is transported contains another extra property. As a result Jackson is complaining with:
com.fasterxml.jackson.databind.exc.MismatchedInputException: Unexpected token (FIELD_NAME), expected END_OBJECT: Current token not END_OBJECT (to match wrapper object with root name 'account'), but FIELD_NAME at [Source: (ByteArrayInputStream); line: 1, column: 26].
Apparently deserialization of #JsonRootName annotated objects works ONLY if that object is the sole property in JSON file - since it's not expecting the "lastTransactionID" to be there.
Here's my Account class:
#JsonRootName("account")
public class Account {
private String id;
}
This is the JSON I need to deserialise:
{
"account": {
"id": "1234"
},
"lastTransactionID": "1"
}
Since I'm using spring I have this setup also spring.jackson.deserialization.unwrap_root_value=true.
Is there any way to solve this without:
having to write a custom deserializer?
OR
intercepting the response and stripping it of the extra property before deserialization takes place?
It looks like simplest way to solve this issue is create wrapper for Account class and deserialise json as usual with disabled DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES feature.
public static class Wrapper {
public Account account;
}
In this case other fields will be ignored.
It's not a nice solution, I know, but it solve a problem.
We can use ObjectMapper to Map json to java Objects.
public Account jsonToObject(Map<String, Object> map){
ObjectMapper objectMapper = new ObjectMapper();
Account account = objectMapper.convertvalue(map.get("account"),Account.class);
return account;
}
You can use JsonIgnoreProperties(ignoreUnknown=true) annotation on your Account class. Please refer below link for more details.
https://www.thetechnojournals.com/2019/10/entity-object-conversion-to-dto-object.html
In my project I use object of type A which has OneToMany relation (orphanRemoval = true, cascade = CascadeType.ALL, fetch = FetchType.EAGER) to objects of type B. I need SpringDataRest (SDR) to store complete full A object with its B objects (children) using single one POST request. I tried several combinations in SDR, the only one which worked for me, was to create #RepositoryRestResource for object A and to create #RepositoryRestResource also for object B, but mark this (B) as exported=false (if I did not create repository out of object B at all, it would not work -> just A object would be stored on single POST request, but not its children (#OneToMany relation) of type B; the same outcome occurs if exported=false is omitted for B repository).
Is this ok and the only way how to achieve it (single POST request with storing all objects at once)?
The reason I'm asking, in my previous example, I have to (I would like to) control all objects "lifecycle" by using A's repository. I am ok with it, because A->B relation is composition (B does not exists outside of A). But I have serious problem of editing (also removing) one certain object of type B by SDR using its parent repository (since object B doest not have its own repository exported). Maybe, this is not possible by definition. I have tried these solutions:
PATCH for "/A/1/B/2" does not work -> method not allowed (in headers
is "Allow: GET, DELETE") -> so, also PUT is out of question
Json Patch would not work either - PATCH for "/A/1" using json patch
content-type [{"op": "add", "path": "/B/2", ....}] -> "no such index
in target array" - because Json Patch uses scalar "2" after "array"
as a index to its array. This is not practical in Java world, when
relations are kept in Set of objects - indexing has no meaning
at all.
I could export repository (exported=true) of object B for
manipulating it "directly", but this way I would loose ability to
store the whole object A with its B objects at one single POST
request as I have mentioned before.
I would like to avoid sending the whole A object with one single tiny modification of its B object for PUT, if possible.
Thank you.
I managed to change the child entity as follows. As a sample I used the following entities:
#Entity
#Data
#NoArgsConstructor
public class One {
#Id
#GeneratedValue
private Long id;
private String name;
#OneToMany(cascade = ALL)
private List<Many> manies = new ArrayList<>();
}
#Entity
#Data
#NoArgsConstructor
public class Many {
public Many(String name) {
this.name = name;
}
#Id
#GeneratedValue
private Long id;
private String name;
}
I just have a repository for One exposed.
(My examples use the excellent httpie - CLI HTTP client)
Removing an item using json patch
This example will remove the second item in the manies list. You could use #OrderColumn to make sure that you can rely on the order of list items.
echo '[{"op":"remove", "path":"/manies/1"}]' | http PATCH :8080/ones/1 Content-Type:application/json-patch+json -v
PATCH /ones/1 HTTP/1.1
Content-Type: application/json-patch+json
[
{
"op": "remove",
"path": "/manies/1"
}
]
Replacing the entire list using json patch
This sample replaces the list with the array specified in the value.
echo '[{"op":"add", "path":"/manies", "value":[{"name":"3"}]}]' | http PATCH :8080/ones/1 Content-Type:application/json-patch+json -v
PATCH /ones/1 HTTP/1.1
Accept: application/json
Content-Type: application/json-patch+json
[
{
"op": "add",
"path": "/manies",
"value": [
{
"name": "3"
}
]
}
]
Adding an item to the list using json patch
This sample adds an item to the end of a list. Also here the client just needs to know the length of the list before the update. So the order does not really matter here.
echo '[{"op":"add", "path":"/manies/-", "value":{"name":"4"}}]' | http PATCH :8080/ones/1 Content-Type:application/json-patch+json -v
PATCH /ones/1 HTTP/1.1
Accept: application/json
Content-Type: application/json-patch+json
[
{
"op": "add",
"path": "/manies/-",
"value": {
"name": "4"
}
}
]
Hope this helps.
I have a Cloudify 2.7.0 instance running.
I need to access Cloudify's API from a Java application and I found an incongruity between the returned JSON and the one which is documented in the Cloudify documentation.
The API is
/service/applications
In the documentation I can read that it should return the following JSON string
{
"status" : "success",
"response" : [ "petclinic", "travel" ]
}
But if I do the same request to my Cloudify instance I get the following JSON string
{
"status" : "success",
"response" : {
"petclinic": "",
"travel":""
}
}
In the java application the JSON information is stored in the following POJO (generated with JSONSchema2POJO)
// CloudifyResponse.java
public class CloudifyResponse {
#JsonProperty("response")
private Response response;
#JsonProperty("status")
private String status;
// getters and setters
}
// Response.java
public class Response {
#JsonProperty("helloworld")
private String helloworld;
#JsonProperty("petclinic")
private String petclinic;
// getters and setters
}
I use the Jackson library to deserialize JSON into POJO. As you can see the JSON string is deserialized into a POJO in which every istantiated application is a POJO's field.
This might be a big problem for the development of the application. In fact, as the instances of the application change, the returned JSON changes and we need to update the POJO structure, something I can't do at runtime.
Do you know whether the Cloudify API has changed the response JSON structure? Is there any way to get the documented JSON output instead of the one I get.
Thank you in advance
Giulio
As of 2.7 The service controller (which you are referring to here) is deprecated, and actually remains available for backward compatibility. The documentation is indeed wrong regarding the returned json structure.
My advice is to use the updated API
/{version}/deployments/applications/description
As documented here, this API actually returns a json holding a list of ApplicationDescription objects, so as application are deployed the response structure remains essentially the same, but the contained list grows.
I have a request executor class that sends request to a webservice and then reads the responses and creates a java object from the response using ObjectMapper.readValue();
My problem is that the webservice returns the responses in mixedCase and not in the correct CamelCase for the classes in the response so for example the following response:
{
"serviceResponse": {
"header": {
"success": "false",
"dateTime": "2012-03-12 09:06:45.60 UTC",
"errorCodes": [
{
"code": "123",
"message": "error occured while trying to get response - User not Logged in",
"causeString": "css_idks"
}
]
},
"body": {
"cls": "lst",
"empty": {}
}
}
}
My class structure is as follows:
ServiceResponse.Java:
public class ServiceResponse implements Serializable {
public ResponseHeader header;
public ResponseBody body;
}
I also have classes for ResponseHeader.java and ResponseBody.java which are similar.
The problem is that while the header field unmarshalls to the ResponseHeader object correctly because the parameter name is indeed 'header', the ServiceResponse fails since the class name starts with a capital S and not a lowercase s.
I get the following exception when trying to parse the response:
03-12 11:14:14.078: E/ELAD(3473): org.codehaus.jackson.map.exc.UnrecognizedPropertyException: Unrecognized field "serviceResponse" (Class com.elad.ServiceResponse), not marked as ignorable
and if I add the ignorable=true annotation, it just parses it but everything inside is null...
I need to know how to annotate the class so that I can define a field name that it should map to which is different from the name of the class in the same way I am able to do for the fields\methods using the #JsonProperty annotation.
Note that I cannot change the request executor so I can't put in a different de-serializer or something like that.
Ok, so I just made a silly mistake, but in the hopes that this will help others, I'll leave the question on with my answer:
I should not have used the ServiceResponse as the class passed to objectMapper.readValue(), but rather a DTO that contains it.
I did it for all other data types and for some reason forgot to do it here.
my bad... :)