Develop Reference

How to handle self-reference in Spring Data JPA with Spring Boot 2?

I have a Spring Boot 2 application in which I have the following User entity:
#Data
... JPA and other annotations
class User {
... many fields including Integer id
#ManyToOne(fetch = FetchType.LAZY)
public User createBy;
#ManyToOne(fetch = FetchType.LAZY)
public User updateBy;
}
Now, the main problem I am facing right now is the self-reference of User (from User) and this either is causing StackOverflow Exceptions or InvalidDefinitionException depending on the certain annotations I am using on User. This issue is very common and several solutions are discussed over the internet which are:
1. Annotate both fields with #JsonBackReference
Annotating with #JsonBackReference omits the updateBy and createBy fields altogether, meaning I am not getting them when desired in my API responses.
2. Annotate class with #JsonIdentityInfo(generator = ObjectIdGenerators.PropertyGenerator.class, property = "id") or None.class or IntSequenceGenerator.class or UUIDGenerator.class
This approach works fine, until the serializer finds the same user object somewhere down the json and instead of putting it up there again, it puts a reference of that object based on the class selected above. e.g.
[
{"id": 1, ..., "createBy": {"id": 2, ...},
2 // <- Let's ignore how user 1 is created by user 2 and notice that instead of a user object, I get an integer ID reference.
]
This means, the client parsing this data will often there to be an object and not a number, causing parsing errors.
3. Implementing a custom serializer (or extending an existing one)
Now, I am really unsure if this is the right way to go about achieving my goals (mentioned below). But if this is the right approach, how would I go about handling this self-reference?
GOALS:
Serialize the data so that, at least certain fields in the child object (user) are passed back, preventing further recursive calls.
{
"id": 1, "name": "old user", .. many other fields .., "createBy": {id: 2, "name": "2nd user"}
}
When the client sends a user object as request body, the application needs only the id of the child entity and not the whole object, as below:
{
"name": "new user", ...., "createBy": {id: 1}
}
I know that self-referencing is integral to ORMs and there are a lot of use cases for self-referencing. But how do professional developers/applications handle this issue, especially in Spring Framework? If a custom serializer is the only way to go, how do I make it function appropriately?
Also, is it advisable to exclude these fields (createBy and updateBy) from EqualsAndHashCode and ToString methods?

Spring Data Rest - PUT on repository silently fails on child references

I use Spring Data Rest with Spring Boot 2.1.1.RELEASE.
I have a class User with a #ManyToMany relationship to a class Skill.
When I make a POST to create a user with his skills, everything works finely.
When I make a PUT to update a user, the skills are not updated, no error is produced.
But when I make a PATCH instead of a PUT, the skills are correctly updated.
Have anyone met a similar issue?
I have found another (old) question about it, but there isn't solution (Spring Data Rest - PUT is not working for associated reference types?)
I probably have missed something, somewhere...
(Code using Lombok)
#Entity
#Getter
#Setter
#NoArgsConstructor
#AllArgsConstructor
#ToString
public class User {
#Id
#GeneratedValue
private Long id;
private String firstName;
private String lastName;
#ManyToMany
#JoinTable(name="user_skills")
private List<Skill> skills = new ArrayList<>();
}
#Entity
#Getter
#Setter
#NoArgsConstructor
#ToString
public class Skill {
#Id
#GeneratedValue
private Long id;
private String name;
}
I make a PUT with the following JSON content:
{
"id": 7,
"firstName": "John",
"lastName": "Doe",
"skills": ["http://localhost:9001/skills/1", "http://localhost:9001/skills/2", "http://localhost:9001/skills/3"]
}
The firstName or lastName can be modified, but the skills remain unmodified.
If I do a PATCH with the same payload, the skills are correctly modified.
It should work with a PUT, doesn't it?

After more investigations, it seems that this behaviour is by purpose: PUT does not update the resource links, only the main attributes.
The answer from Oliver Gierke is here: https://jira.spring.io/browse/DATAREST-1001?focusedCommentId=135791&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-135791:
I looked into this and I'd argue you're expecting things to work in a way they don't work. PUT requests don't consider associations to linkable resources, i.e. related resources that are pointed to by links. The reason for that is two-fold:
If we consider URIs for association fields in the payload to update those associations, the question comes up about what's supposed to happen if no URI is specified. With the current behavior, linked associations are simply not a part of the payload as they only reside in the _links block. We have two options in this scenario: wiping the associations that are not handed, which breaks the "PUT what you GET" approach. Only wiping the ones that are supplied using null would sort of blur the "you PUT the entire state of the resource".
For all the reasons mentioned in 1. there are dedicated assoctiation resources exposed that can be manipulated directly.
So it looks like that if you want to change both state of the resource plus associations at the same time, I guess exposing a dedicated resource to do that is the way to go.
Other posts and links:
"Unable to update associated resource using PUT request on the item resource": https://jira.spring.io/browse/DATAREST-1001
"Spring Data Rest PUT v.s PATCH LinkableResources" : Spring Data Rest PUT v.s PATCH LinkableResources
"PUT behaving like PATCH for nested collections": https://jira.spring.io/browse/DATAREST-1012

In a java REST API, using PATCH vs PUT to update an entity

I am about to start development on a new rest api in Java.
My question is about the use of PATCH - Why?
Lets say, we have an entity named Address.java
public class Address {
#Id
private Long id
#NotNull
private String line1;
private String line2; //optional
#NotNull
private String city;
#NotNull
private String state;
}
To create a new Address, I would do this http request:
POST http://localhost:8080/addresses
with the following request:
{
"line1" : "mandatory Address line 1",
"line2" : "optional Address line 2",
"city" : "mandatory City",
"state" : "cd"
}
Assume the record created has an id 1
The corresponding #RestController AddressResource.java will have this method :
#PostMapping(value = "/addresses")
public ResponseEntity<Address> create(#valid Address newAddress) {
addressRepo.save(newAddress);
}
#valid will ensure the entity is valid before storing the data into the table.
Now assume, I move from my apartment above to a house down the street. If I use a PATCH, it becomes
PATCH http://localhost:8080/addresses/1
with request payload:
{
"line1" : "1234 NewAddressDownTheStreet ST",
"line2" : null
}
The corresponding #RestController method would be :
#PatchMapping(value = "/addresses/{id}")
public ResponseEntity<Address> patchAddress(#PathVariable Long id, Address partialAddress)
{
Address dbAddress = addressRepo.findOne(id);
if (partialAddress.getLine1() != null) {
dbAddress.setLine1(partialAddress.getLine1());
}
if (partialAddress.getLine2() != null) {
dbAddress.setLine2(partialAddress.getLine2());
}
if (partialAddress.getCity() != null) {
dbAddress.setCity(partialAddress.getCity());
}
if (partialAddress.getState() != null) {
dbAddress.setState(partialAddress.getState());
}
addressRepo.save(dbAddress)
}
Now if you query the table, won't my address be ?
"line1" : "1234 NewAddressDownTheStreet ST",
"line2" : "optional Address line 2", <-- INCORRECT. Should be null.
"city" : "mandatory City",
"state" : "cd"
As can be seen, the above updates results in an incorrect value for line2.
This is because in java all instance variables in the Address class are initialized to null (or default initial values if they are primitives) when a class is instantiated. So there is no way to distinguish between line2 being changed to null from the default value.
Question 1) Is there a standard way to work around this?
Another disadvantage is that, I cannot use #Valid annotation to validate the request at the entry point - coz it is only a partial. So, invalid data could get into the system.
For example, imagine there was additional field with the following definition:
#Min(0)
#Max(100)
private Integer lengthOfResidencyInYears,
And the user accidentally typed 190 (when they really meant 19 years), it would not fail.
Instead of PATCH, if I had used PUT, the client would need to send the complete address object.
This has the advantage that I can use #Valid to ensure that the Address is indeed valid
If one makes the premise that a GET MUST always be done before doing any updates, why wouldn't one use PUT over PATCH?
Am I missing something?
Aside
My conclusion is that developers using dynamically typed languages are the proponents of using PATCH as I cannot see any benefit to using it from a statically typed language line Java or C#. It just seems to add more complexity.

Using PATCH to upload a modified version of an existing object is almost always problematic for exactly the reason you have outlined. If you want to use PATCH with JSON, I strongly suggest you follow either RFC 6902 or RFC 7396. I won't speak to 7396 because I'm not that familiar with it, but to follow 6902 you would define a separate resource for PATCH operations. In the example you gave, it would look like:
PATCH http://localhost:8080/addresses/1
[
{ "op": "replace", "path": "/line1", "value": "1234 NewAddressDownTheStreet ST" },
{ "op": "remove", "path": "/line2" }
]
You would then process this, making a new entity object that started at the current server state and applied the changes in the PATCH. Run validation on the new entity object. If it passes, push it to the data layer. If it fails, return an error code.
If PUT doesn't add too much overhead, it is a good idea. Idempotency is a nice thing to have. The tradeoff is that you're pushing more data over the wire. If your resource is not large and not accessed often, that's maybe not such a big deal. If your resource is large and is accessed often, that can start to add significant overhead. Of course, we can't tell you the tipping point.
You also seem to have completely tied your resource model to your database model. Good database table design and good resource design often look very different for non-trivial projects. I understand that many frameworks drive you in that direction, but you if you haven't seriously considered decoupling them, you might want to.

PUT and POST fail on unknown properties Spring different behavior

I am writing Spring Boot application using Spring Data Rest repositories and I want to deny access to resource if request body contains JSON that has unknown properties. Definition of simplified entity and repository:
#Entity
public class Person{
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
private long id;
private String firstName;
private String lastName;
/* getters and setters */
}
#RepositoryRestResource(collectionResourceRel = "people", path = "people")
public interface PersonRepository extends CrudRepository<Person, Long> {}
I use Jackson's deserialization feature to disallow unknown properties in JSONs.
#Bean
public Jackson2ObjectMapperBuilder objectMapperBuilder(){
Jackson2ObjectMapperBuilder builder = new Jackson2ObjectMapperBuilder();
builder.failOnUnknownProperties(true);
return builder;
}
When I send POST requests everything works as expected. When I use valid fields I get correct response:
curl -i -x POST -H "Content-Type:application/json" -d '{"firstName": "Frodo", "lastName": "Baggins"}' http://localhost:8080/people
{
"firstName": "Frodo",
"lastName": "Baggins",
"_links": {...}
}
And when I send JSON with unknown fields application throws expected error:
curl -i -x POST -H "Content-Type:application/json" -d '{"unknown": "POST value", "firstName": "Frodo", "lastName": "Baggins"}' http://localhost:8080/people
com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "unknown" (class Person), not marked as ignorable (2 known properties: "lastName", "firstName")
PUT method when using valid JSON returns correct response as well. However when I send PUT request with unknown field I expect Spring to throw error but instead of that, Spring updates object in database and returns it:
curl -i -x PUT -H "Content-Type:application/json" -d '{"unknown": "PUT value", "firstName": "Bilbo", "lastName": "Baggins"}' http://localhost:8080/people/1
{
"firstName": "Bilbo",
"lastName": "Baggins",
"_links": {...}
}
The error is thrown only when there is no object in database with given id:
curl -i -x PUT -H "Content-Type:application/json" -d '{"unknown": "PUT value", "firstName": "Gandalf", "lastName": "Baggins"}' http://localhost:8080/people/100
com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "unknown" (class Person), not marked as ignorable (2 known properties: "lastName", "firstName")
Is it expected behavior or a bug in Spring Data Rest? How can I throw an error when JSON with unknown properties is passed to application no matter what request method is?
I've reproduced this behavior by modifying http://spring.io/guides/gs/accessing-data-rest/ , the only change I've made is Jackson2ObjectMapperBuilder , no other controllers or repositories are in this project.

I think the behaviour you are observing is by design. When a POST is issued you are creating the resource so the JSON is deserialised into your entity type and Jackson is performing this task.
A PUT is working differently in spring data rest. The interesting part is handled in PersistentEntityResourceHandlerMethodArgumentResolver.readPutForUpdate.
The json is read into a JsonNode, the entity is read from the data store and then in DomainObjectReader.doMerge the implementation iterates over the json fields. It applies the json to the entity and saves it later in the controller implementation. It also discards all the fields that do not exist in the persistent entity:
if (!mappedProperties.hasPersistentPropertyForField(fieldName)) {
i.remove();
continue;
}
This is my understanding from reading the code. I think you could argue that this is a bug. You could try to report it at spring data rest`s jira - https://jira.spring.io/browse/DATAREST. As far as I know there is no way to customise this behaviour.

When it creates new entity then it converts json directly to java entity object through deserialization process where required validation is involved. But when it updates existing entity then it converts json to JsonNode and then merge with existing entity and as expected no validation happens because it is feature for json deserialization to java object.
As workaround you can additionally convert JsonNode to entity object and it will work as you expect.
I did quick example how to gain required validation.
go to https://github.com/valery-barysok/gs-accessing-data-rest
It is not clear solution but you can improve it :)
This example override existing spring class on classpath org.springframework.data.rest.webmvc.config.PersistentEntityResourceHandlerMethodArgumentResolver
Note You must put this class on classpath before original version.
I did copy-past this class to project and modified readPutForUpdate method:
private Object readPutForUpdate(IncomingRequest request, ObjectMapper mapper, Object existingObject,
RootResourceInformation information) {
try {
JsonPatchHandler handler = new JsonPatchHandler(mapper, reader);
JsonNode jsonNode = mapper.readTree(request.getBody());
// Here we have required validation
mapper.treeToValue(jsonNode, information.getDomainType());
return handler.applyPut((ObjectNode) jsonNode, existingObject);
} catch (Exception o_O) {
throw new HttpMessageNotReadableException(String.format(ERROR_MESSAGE, existingObject.getClass()), o_O);
}
}
and i used application.properties file to configure DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES

you can annotate your model with :
#Entity
#JsonIgnoreProperties(ignoreUnknown=false)
public class Person{
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
private long id;
private String firstName;
private String lastName;
/* getters and setters */
}

You're using Jackson2ObjectMapperBuilder, which has the property DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES set to the disabled value by default.

How to add an object in a OneToMany relationship using Spring Data REST

I cannot figure out how to add an object to a OneToMany relationship using Spring-Data REST when the mapped classes REST Repository is not exported.
I have two classes, Question and Answer. Question has a member variable defined like this:
#OneToMany(mappedBy = "answer", cascade=CascadeType.ALL, orphanRemoval = true)
#LazyCollection(LazyCollectionOption.FALSE)
private List<Answer> answers = new LinkedList<Answer>();
And Answer maps back to Question like this:
#NotNull
#ManyToOne(targetEntity = Question.class)
#JoinColumn(name = "question_id", referencedColumnName = "id")
private Question question;
Because Answer objects are only relevant to the Question they belong to, I've disabled exporting the REST repository:
#RestResource(exported = false)
public interface AnswerRepository extends JpaRepository<Answer, Long> {}
When I fetch a question here: http://localhost:9090/data/questions/7, I get something like this:
{
"creationDate": "2014-09-26T06:36:44.000+0000",
"modificationDate": "2014-09-26T06:36:44.000+0000",
"answers": [],
"_links": {
"self": {
"href": "http://localhost:9090/data/questions/7"
}
}
}
So far so good. Now I want to add an answer like this:
curl -v -X PUT -H "Content-Type: application/json" \
-d "{"answers": [{"value": "Red"}]}" http://localhost:9090/data/questions/7
Unfortunately, at this point I get the following error:
A collection with cascade=\"all-delete-orphan\" was no longer
referenced by the owning entity instance: com.example.Question.answers
A brief search of StackOverflow indicates the above error is caused by replacing your collection with another one, orphaning the previous collection. Since all of this code managed by Spring, I don't see how to manipulate my objects to avoid this problem.
This question is similar, however the difference is that in this case the repository is not exported, while in that question it is.

If you're using PUT you should be sending text/uri-list media type. But I don't see how would you do that since you did not export Answer repository thus you can not call POST on it first.
Did you try PATCH? It's not documented that way though...