So here's my situation: I want to build a simple CRUD webservice using Jackson and Hibernate. Seems like a perfect job for Spring Boot. So we have the following:
(Please note that I am condensing the code so its not compile-able)
class Doctor {
#Id
long id;
#ManyToMany(fetch = FetchType.EAGER, cascade = CascadeType.ALL)
#JoinTable(name = "doctor_service", joinColumns = { #JoinColumn(name = "doctor_id", nullable = false) }, inverseJoinColumns = { #JoinColumn(name = "service_id", nullable = false) })
Set<Service> services;
}
class Service {
#Id
long id;
#ManyToMany(fetch = FetchType.EAGER, mappedBy = "services")
Set<Doctor> doctors;
}
A simple data model. And we have a simple requirement: on the webservice, when we get Service objects we should get the associated Doctors. And when we get the Doctors, we should get the associated Services. We are using lazy because [insert justification here].
So now lets serve it:
#Path("/list")
#POST
#Produces(MediaType.APPLICATION_JSON)
#Transactional
public JsonResponse<List<Doctor>> list() {
return JsonResponse.success(doctorCrudRepo.findAll());
}
Gloss over the JsonResponse object (just a convenience blackbox for now) and lets assume the doctorCrudRepo is a valid instance of CrudRepository.
And the firestorm begins:
failed to lazily initialize a collection of role: Doctor.services, could not initialize proxy - no Session (through reference chain: ...)
Ok so Lazy doesnt work then. Simple enough. Just make it eager.
Caused by: java.lang.StackOverflowError: null
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:760)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:455)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:367)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:655)
... 1011 common frames omitted
So let's see what other people have said:
Contestant #1: Solutions are not relevant because they apply to one-to-many, not many-to-many, so I still get the StackOverflowError.
Contestant #2: Same as before, still a one-to-many, still StackOverflow.
Contestant #3: Same (has nobody ever used many-to-many???)
Contestant #4: I can't use #JsonIgnore because that means it will never be serialized. So it doesn't fit the requirements.
Contestant #5: At first glance, it appears to work fine! However, only the Doctor endpoint works - it is getting the services. The Services endpoint doesn't work - its not getting the Doctors (empty set). It's probably based on which reference defines the join table. This again doesn't fit the bill.
Contestant #6: Nope.
Some other solutions which are wrong but worth mentioning:
Create a new set of objects for json serialization that are not wrapped by hibernate, and then copy the properties over in the controller. This is a lot of extra work. Forcing this pattern everywhere defeats the purpose of using Hibernate.
After loading the Doctor, loop over each Service and set the service.doctors to null, to prevent further lazy loading. I'm trying to establish a set of best practices, not to come up with hackish workarounds.
So... what's the RIGHT solution? What pattern can I follow that looks clean and makes me proud to use Hibernate and Jackson? Or is this combination of technology so incompatible to suggest a new paradigm?
Firstly, regarding your statement "...copy the properties over in the controller. This is a lot of extra work. Forcing this pattern everywhere defeats the purpose of using Hibernate.":
It doesn't defeat the purpose of using Hibernate. ORMs were created in order to eliminate necessity of converting database rows received from JDBC to POJOs. Hibernate's lazy-loading purpose is to eliminate redundant work on writing custom queries to RDBMS when you don't need great performance, or you are able to cache the entities.
The issue is not with Hibernate&Jackson, but with the fact that you are trying to use the instrument for a purpose, it was never designed for.
I guess that your project tends to grow (usually they all do). If that's true then you will have to separate layers someday, and better sooner than later. So I would suggest you to stick to the "wrong solution #1" (create a DTO). You can use something like ModelMapper to prevent hand-writing of Entity to DTO conversion logic.
Also consider that without DTOs your project may become hard to maintain:
Data model will evolve, and you will always have to update your front-end with changes.
Data model may contain some fields, that you may want to omit from sending to your users (like user's password field). You can always create additional entities, but they will require additional DAOs, etc.
Someday you may need to return user a data that is a composition of some entities. You can write a new JPQL, like SELECT new ComplexObject(entity1, entity2, entity3) ..., but that would be much harder than to call few service's methods and compose the result into DTO.
I found a solution that seems to be elegant.
Use OpenEntityManagerInViewFilter. Seems to be frowned upon (probably for security reasons, but I haven't seen any convincing reason to not use it). It's simple to use, just define a bean:
#Component
public class ViewSessionFilter extends OpenEntityManagerInViewFilter {
}
Use LAZY on all references. This is what I wanted to begin with, and it's especially important since my data has many references and my services are small.
Use #JsonView. See this helpful article.
First, figure out what the views will be (one for doctors, one for patients)
public interface Views {
public static interface Public {}
public static interface Doctors extends Public {}
public static interface Services extends Public {}
}
Looking from the Doctors view, you will see the services.
#Entity
#Table(name = "doctor")
public class Doctor {
#ManyToMany(fetch = FetchType.LAZY, cascade = CascadeType.ALL)
#JoinTable(name = "doctor_service", joinColumns = { #JoinColumn(name = "doctor_id", nullable = false) },
inverseJoinColumns = { #JoinColumn(name = "service_id", nullable = false) })
#JsonView(Views.Doctors.class)
private Set<Service> services;
}
And looking from the Services view, you will see the doctors.
#Entity
#Table(name = "service")
public class Service {
#ManyToMany(fetch = FetchType.LAZY, mappedBy = "services")
#JsonView(Views.Services.class)
private Set<Doctor> doctors;
}
Then assign the views to the service endpoints.
#Component
#Path("/doctor")
public class DoctorController {
#Autowired
DoctorCrudRepo doctorCrudRepo;
#Path("/list")
#POST
#Produces(MediaType.APPLICATION_JSON)
#JsonView(Views.Doctors.class)
public JsonResponse<List<Doctor>> list() {
return JsonResponse.success(OpsidUtils.iterableToList(doctorCrudRepo.findAll()));
}
}
Works perfectly for a simple CRUD app. I even think it will scale well to bigger, more complex apps. But it would need to be maintained carefully.
Related
Sample code to represent my problem.
Let's say there are entity classes like that
public class User {
...
#ManyToOne
#JoinColumn(name = "users_statuts_id", referencedColumnName = "id", nullable = false)
private UserStatus status;
}
public class UserStatus {
...
#OneToMany(mappedBy = "status")
private List<User> users = new LinkedList<>();
}
This works as expected but, problem starts when I want to use MapStruct generated mapper and DTO object. I will get StackOverflowError becouse of Cyclic references User->UserStatus->List< User>-> .....
And take into account that not necessary it will be in pattern A->B->A (User->UserStatus->User)
sometimes it will be User->ClassA->ClassB->...->User
I tired adding Context (CycleAvoidingMappingContext) to mapper as stated in other threads to break cycle but I failed and still get StackOverflowError. Tried too with AfterMappings and Mapping(ignore) but still not worked and I would like to avoid settings manually nulls to break the cycle.
I wonder if is possible to break cycle similar to #JsonIdentityInfo but on Hibernate level ?
My classes are a bit more complicated and bidirectional relation would help a lot later. Someone have some tips how should I make it done to work properly ?
I am trying to choose the properties of associated entities that will be loaded.
For example:
#Entity
#Getter #Setter
public class Book {
#Id #GeneratedValue(strategy = GenerationType.AUTO)
private Long id;
#Column(name = "first")
private String first;
#Column(name = "second")
private String second;
#OneToMany(mappedBy = "book", cascade = CascadeType.ALL)
private List<Page> pages = new ArrayList();
}
#Entity
#Getter #Setter
public class Page {
#Id #GeneratedValue(strategy = GenerationType.AUTO)
private Long id;
#Column(name = "number")
private Integer number;
#Column(name = "content")
private String content;
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "book_id")
private Book book;
}
I would like to have just a SELECT on the Book's first and the content of all associated Page entities
book
- first
- pages
- content
For example, in PostgreSQL this might look something like
SELECT book.first,
array_agg(page.content || ' ')
FROM book
LEFT JOIN page
ON page.book_id = book.id
GROUP BY book.first
I've done my research on how you could go about doing this, most mentioned solution is to use Spring DATA JPA Projections (ref)
I did the following:
public interface FindAllBookProjection {
String getFirst();
List<PageProjection> getPages();
interface PageProjection {
Integer getNumber();
}
}
The issue with projections is that they cause the famous N+1 select problem as the pages are loaded lazily.
I also couldn't find a way to use projections with #Query in the #Repository for the nested properties.
There are other mentions online to use #EntityGraph. From what I understand #EntityGraph will not work as it also selects all properties, specifying only what Entity associations should be loaded.
There are also suggestions on using other libraries like Blaze Persistence (ref) or Hibernate's ResultTransformer (ref) . I would prefer to use only Spring DATA JPA instead of introducing another library for this functionality and writing ResultTransformers seems like adding a lot of boilerplate code to the mix.
To summarize my question is what is the best way to choose what properties are selected for #Entity associations. The main goal is to avoid pulling unnecessary amount of data from the database. The given example above is for demonstration, the data that I am working on includes over 10 columns and spans across 3-4 entities. Having control over the data means better performance.
I am the creator of Blaze-Persistence and I can tell you that if there were an easy way to do this, I would not have created Entity-Views. You can read some of the articles I wrote about this and you will realize that doing this yourself, like you already figured, will require lots of boilerplate code. Doing this efficiently, will require even more code and in the end, you will probably end up with a solution that is inferior to Blaze-Persistence Entity-Views in one way or another and has probably no documentation.
Spring Data Projections is just limited and as I tried to outline many times before, Blaze-Persistence Entity-Views is like Spring Data Projections on steroids.
If you have "just" one or two simple mappings, you might be able to get this done by introducing special #Immutable #Entity, maybe even with #Subselect in this particular case to model what you need, but believe me, this only works good on a small scale. Apart from that, Blaze-Persistence which works on top of JPA/Hibernate enables the use of a lot of advanced SQL features, which you usually can't use from within plain JPA/Hibernate.
In my opinion, ORM libraries should use whole objects, that means loading all data into the program and then transforming/filtering according to logic.
For specific use-cases, where performance is really hindered, I'd use entities in
the database, such as Views/Procedures.
In your case, i'd create a View:
CREATE VIEW book_content as
SELECT book.first as [first],
array_agg(page.content || ' ') as [content]
FROM book
LEFT JOIN page
ON page.book_id = book.id
GROUP BY book.first
And then create a #Repository and #Entity for it in spring.
So, I have found myself in quite a pickle regarding Hibernate. When I started developing my web application, I used "eager" loading everywhere so I could easily access children, parents etc.
After a while, I ran into my first problem - re-saving of deleted objects. Multiple stackoverflow threads suggested that I should remove the object from all the collections that it's in. Reading those suggestions made my "spidey sense" tickle as my relations weren't really simple and I had to iterate multiple objects which made my code look kind of ugly and made me wonder if this was the best approach.
For example, when deleting Employee (that belongs to User in a sense that User can act as multiple different Employees). Let's say Employee can leave Feedback to Party, so Employee can have multiple Feedback and Party can have multiple Feedback. Additionally, both Employee and Party belong to some kind of a parent object, let's say an Organization. Basically, we have:
class User {
// Has many
Set<Employee> employees;
// Has many
Set<Organization> organizations;
// Has many through employees
Set<Organization> associatedOrganizations;
}
class Employee {
// Belongs to
User user;
// Belongs to
Organization organization;
// Has many
Set<Feedback> feedbacks;
}
class Organization {
// Belongs to
User user;
// Has many
Set<Employee> employees;
// Has many
Set<Party> parties;
}
class Party {
// Belongs to
Organization organization;
// Has many
Set<Feedback> feedbacks;
}
class Feedback {
// Belongs to
Party party;
// Belongs to
Employee employee;
}
Here's what I ended up with when deleting an employee:
// First remove feedbacks related to employee
Iterator<Feedback> iter = employee.getFeedbacks().iterator();
while (iter.hasNext()) {
Feedback feedback = iter.next();
iter.remove();
feedback.getParty().getFeedbacks().remove(feedback);
session.delete(feedback);
}
session.update(employee);
// Now remove employee from organization
Organization organization = employee.getOrganization();
organization.getEmployees().remove(employee);
session.update(organization);
This is, by my definition, ugly. I would've assumed that by using
#Cascade({CascadeType.ALL})
then Hibernate would magically remove Employee from all associations by simply doing:
session.delete(employee);
instead I get:
Error during managed flush [deleted object would be re-saved by cascade (remove deleted object from associations)
So, in order to try to get my code a bit cleaner and maybe even optimized (sometimes lazy fetch is enough, sometimes I need eager), I tried lazy fetching almost everything and hoping that if I do, for example:
employee.getFeedbacks()
then the feedbacks are nicely fetched without any problem but nope, everything breaks:
failed to lazily initialize a collection of role: ..., could not initialize proxy - no Session
The next thing I thought about was removing the possibility for objects to insert/delete their related children objects but that would probably be a bad idea performance-wise - inserting every object separately with
child.parent=parent
instead of in a bulk with
parent.children().add(children).
Finally, I saw that multiple people recommended creating my own custom queries and stuff but at that point, why should I even bother with Hibernate? Is there really no good way to handle my problem relatively clean or am I missing something or am I an idiot?
If I understood the question correctly it's all about cascading through simple 1:N relations. In that case Hibernate can do the job rather well:
#Entity
public class Post {
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
private Long id;
#OneToMany(cascade = CascadeType.ALL,
mappedBy = "post", orphanRemoval = true)
private List<Comment> comments = new ArrayList<>();
}
#Entity
public class Comment {
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
private Long id;
#ManyToOne
private Post post;
}
Code:
Post post = newPost();
doInTransaction(session -> {
session.delete(post);
});
Generates:
delete from Comment where id = 1
delete from Comment where id = 2
delete from Post where id = 1
But if you have some other (synthetic) collections, Hibernate has no chance to know which ones, so you have to handle them yourself.
As for Hibernate and custom queries, Hibernate provides HQL which is more compact then traditional SQL, but still is less transparent then annotations.
Working with Spring Data REST, if you have a OneToMany or ManyToOne relationship, the PUT operation returns 200 on the "non-owning" entity but does not actually persist the joined resource.
Example Entities:
#Entity(name = 'author')
#ToString
class AuthorEntity implements Author {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
Long id
String fullName
#ManyToMany(mappedBy = 'authors')
Set<BookEntity> books
}
#Entity(name = 'book')
#EqualsAndHashCode
class BookEntity implements Book {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
Long id
#Column(nullable = false)
String title
#Column(nullable = false)
String isbn
#Column(nullable = false)
String publisher
#ManyToMany(fetch = FetchType.LAZY, cascade = [CascadeType.ALL])
Set<AuthorEntity> authors
}
If you back them with a PagingAndSortingRepository, you can GET a Book, follow the authors link on the book and do a PUT with the URI of a author to associate with. You cannot go the other way.
If you do a GET on an Author and do a PUT on its books link, the response returns 200, but the relationship is never persisted.
Is this the expected behavior?
tl;dr
The key to that is not so much anything in Spring Data REST - as you can easily get it to work in your scenario - but making sure that your model keeps both ends of the association in sync.
The problem
The problem you see here arises from the fact that Spring Data REST basically modifies the books property of your AuthorEntity. That itself doesn't reflect this update in the authors property of the BookEntity. This has to be worked around manually, which is not a constraint that Spring Data REST makes up but the way that JPA works in general. You will be able to reproduce the erroneous behavior by simply invoking setters manually and trying to persist the result.
How to solve this?
If removing the bi-directional association is not an option (see below on why I'd recommend this) the only way to make this work is to make sure changes to the association are reflected on both sides. Usually people take care of this by manually adding the author to the BookEntity when a book is added:
class AuthorEntity {
void add(BookEntity book) {
this.books.add(book);
if (!book.getAuthors().contains(this)) {
book.add(this);
}
}
}
The additional if clause would've to be added on the BookEntity side as well if you want to make sure that changes from the other side are propagated, too. The if is basically required as otherwise the two methods would constantly call themselves.
Spring Data REST, by default uses field access so that theres actually no method that you can put this logic into. One option would be to switch to property access and put the logic into the setters. Another option is to use a method annotated with #PreUpdate/#PrePersist that iterates over the entities and makes sure the modifications are reflected on both sides.
Removing the root cause of the issue
As you can see, this adds quite a lot of complexity to the domain model. As I joked on Twitter yesterday:
#1 rule of bi-directional associations: don't use them… :)
It usually simplifies the matter if you try not to use bi-directional relationship whenever possible and rather fall back to a repository to obtain all the entities that make up the backside of the association.
A good heuristics to determine which side to cut is to think about which side of the association is really core and crucial to the domain you're modeling. In your case I'd argue that it's perfectly fine for an author to exist with no books written by her. On the flip side, a book without an author doesn't make too much sense at all. So I'd keep the authors property in BookEntity but introduce the following method on the BookRepository:
interface BookRepository extends Repository<Book, Long> {
List<Book> findByAuthor(Author author);
}
Yes, that requires all clients that previously could just have invoked author.getBooks() to now work with a repository. But on the positive side you've removed all the cruft from your domain objects and created a clear dependency direction from book to author along the way. Books depend on authors but not the other way round.
I faced a similar problem, while sending my POJO(containing bi-directional mapping #OneToMany and #ManyToOne) as JSON via REST api, the data was persisted in both the parent and child entities but the foreign key relation was not established. This happens because bidirectional associations need to be manually maintained.
JPA provides an annotation #PrePersist which can be used to make sure that the method annotated with it is executed before the entity is persisted. Since, JPA first inserts the parent entity to the database followed by the child entity, I included a method annotated with #PrePersist which would iterate through the list of child entities and manually set the parent entity to it.
In your case it would be something like this:
class AuthorEntitiy {
#PrePersist
public void populateBooks {
for(BookEntity book : books)
book.addToAuthorList(this);
}
}
class BookEntity {
#PrePersist
public void populateAuthors {
for(AuthorEntity author : authors)
author.addToBookList(this);
}
}
After this you might get an infinite recursion error, to avoid that annotate your parent class with #JsonManagedReference and your child class with #JsonBackReference. This solution worked for me, hopefully it will work for you too.
This link has a very good tutorial on how you can navigate the recursion problem:Bidirectional Relationships
I was able to use #JsonManagedReference and #JsonBackReference and it worked like a charm
I believe one can also utilize #RepositoryEventHandler by adding a #BeforeLinkSave handler to cross link the bidirectional relation between entities. This seems to be working for me.
#Component
#RepositoryEventHandler
public class BiDirectionalLinkHandler {
#HandleBeforeLinkSave
public void crossLink(Author author, Collection<Books> books) {
for (Book b : books) {
b.setAuthor(author);
}
}
}
Note: #HandlBeforeLinkSave is called based on the first parameter, if you have multiple relations in your equivalent of an Author class, the second param should be Object and you will need to test within the method for the different relation types.
I have stumbled upon a really annoying situation: I am using Hibernate & Spring as backend for my app and it seems that in some cases, the entities that are in a relationship with a particular entity are not fetched as normal entity objects from the DB, but as Javassist types. E.g.:
I have the Campaign entity with the following relationships:
#Entity
#Table(name = "campaign")
public class Campaign implements Serializable {
[..]
#ManyToMany(fetch = FetchType.LAZY)
#JoinTable(uniqueConstraints = #UniqueConstraint(columnNames = {
"campaign_id", "dealer_id" }), name = "campaign_has_dealer", joinColumns = { #JoinColumn(name = "campaign_id", nullable = false) }, inverseJoinColumns = { #JoinColumn(name = "dealer_id", nullable = false) })
private List<Dealer> dealers = new ArrayList<Dealer>();
#ManyToMany
// (fetch = FetchType.LAZY)
#JoinTable(uniqueConstraints = #UniqueConstraint(columnNames = {
"campaign_id", "sales_area_id" }), name = "campaign_has_sales_area", joinColumns = { #JoinColumn(name = "campaign_id", nullable = false) }, inverseJoinColumns = { #JoinColumn(name = "sales_area_id", nullable = false) })
private List<SalesArea> salesAreas = new ArrayList<SalesArea>();
}
Upon retrieving the salesAreas connected to this Campaign, I get a list of SalesArea_$$_javassist_56, while for the dealers, I get normal Hibernate entities. Since the client part is based on GWT, we use RequestFactory for retrieving stuff. I initially thought it was a problem with the proxies, locators and so on but I have set a breakpoint in the service where these are retrieved and they are Javassist objects directly after selecting them. It seems that even removing the FetchType.LAZY annotation (although definitely not a desirable solution), the same thing happens. This happened also with other types of relationships, not only #ManyToMany.
We are using GWT 2.3, Spring 3, Hibernate 3.6.3 and JPA 2.0 for annotations.
Any suggestions would be appreciated.
Thanks in advance
As far as I can see the big problem that you're having is not so much the fetch type of your association, but rather that the proxied types don't work well with RequestFactory.
Yes, it could be solved by changing the fetch strategy but that sounds rather like a weak workaround that may break upon weird circumstances.
I don't remember exactly how to solve it, but I did, and as far as I remember there was an extension point in the ServiceLayerDecorator class. Basically there you check if the object you're returning is a Hibernate proxy (check Hibernate and HibernateProxy classes) and then return the non-proxy type instead in ServiceLayerDecorator. (http://code.google.com/p/google-web-toolkit/issues/detail?id=6767)
As for your fetch strategy, I'd largely recommend #BatchSize(N) where N is big (maybe 1000), but this is an independent subject.
Good luck!
If you call to the static method:
HibernateProxyHelper.getClassWithoutInitializingProxy(entity);
you get the class of the proxied entity and the class itself if it wasn't proxied.
With Hibernate's proxy model and now with it's use of Javassist to help avoid the slower traditional Hibernate run time reflection operations things will never quite be as elegant as the clean, intuitive experience people who use full bytecode enhancement solutions like JDO implementations (eg DataNucleus) enjoy.
Personally I can never see the sense in persisting (pardon the pun) with solutions that cause so many problems and fill the web with questions about broken code that requires strange, unintuitive workarounds but still people do...
However, back to the question: one solution to your problem, if you're using JPA, is to use DataNucleus/JPA which brings many of the benefits of DataNucleus/JDO (clean underlying implementation - no proxies, no Javassist classes etc.,) in a JPA compliant implementation - i.e. you don't need to change your existing source code to start using it.