I am trying to choose the properties of associated entities that will be loaded.
For example:
#Entity
#Getter #Setter
public class Book {
#Id #GeneratedValue(strategy = GenerationType.AUTO)
private Long id;
#Column(name = "first")
private String first;
#Column(name = "second")
private String second;
#OneToMany(mappedBy = "book", cascade = CascadeType.ALL)
private List<Page> pages = new ArrayList();
}
#Entity
#Getter #Setter
public class Page {
#Id #GeneratedValue(strategy = GenerationType.AUTO)
private Long id;
#Column(name = "number")
private Integer number;
#Column(name = "content")
private String content;
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "book_id")
private Book book;
}
I would like to have just a SELECT on the Book's first and the content of all associated Page entities
book
- first
- pages
- content
For example, in PostgreSQL this might look something like
SELECT book.first,
array_agg(page.content || ' ')
FROM book
LEFT JOIN page
ON page.book_id = book.id
GROUP BY book.first
I've done my research on how you could go about doing this, most mentioned solution is to use Spring DATA JPA Projections (ref)
I did the following:
public interface FindAllBookProjection {
String getFirst();
List<PageProjection> getPages();
interface PageProjection {
Integer getNumber();
}
}
The issue with projections is that they cause the famous N+1 select problem as the pages are loaded lazily.
I also couldn't find a way to use projections with #Query in the #Repository for the nested properties.
There are other mentions online to use #EntityGraph. From what I understand #EntityGraph will not work as it also selects all properties, specifying only what Entity associations should be loaded.
There are also suggestions on using other libraries like Blaze Persistence (ref) or Hibernate's ResultTransformer (ref) . I would prefer to use only Spring DATA JPA instead of introducing another library for this functionality and writing ResultTransformers seems like adding a lot of boilerplate code to the mix.
To summarize my question is what is the best way to choose what properties are selected for #Entity associations. The main goal is to avoid pulling unnecessary amount of data from the database. The given example above is for demonstration, the data that I am working on includes over 10 columns and spans across 3-4 entities. Having control over the data means better performance.
I am the creator of Blaze-Persistence and I can tell you that if there were an easy way to do this, I would not have created Entity-Views. You can read some of the articles I wrote about this and you will realize that doing this yourself, like you already figured, will require lots of boilerplate code. Doing this efficiently, will require even more code and in the end, you will probably end up with a solution that is inferior to Blaze-Persistence Entity-Views in one way or another and has probably no documentation.
Spring Data Projections is just limited and as I tried to outline many times before, Blaze-Persistence Entity-Views is like Spring Data Projections on steroids.
If you have "just" one or two simple mappings, you might be able to get this done by introducing special #Immutable #Entity, maybe even with #Subselect in this particular case to model what you need, but believe me, this only works good on a small scale. Apart from that, Blaze-Persistence which works on top of JPA/Hibernate enables the use of a lot of advanced SQL features, which you usually can't use from within plain JPA/Hibernate.
In my opinion, ORM libraries should use whole objects, that means loading all data into the program and then transforming/filtering according to logic.
For specific use-cases, where performance is really hindered, I'd use entities in
the database, such as Views/Procedures.
In your case, i'd create a View:
CREATE VIEW book_content as
SELECT book.first as [first],
array_agg(page.content || ' ') as [content]
FROM book
LEFT JOIN page
ON page.book_id = book.id
GROUP BY book.first
And then create a #Repository and #Entity for it in spring.
Related
What's a workaround if I have a relation OneToMany and would like to access the collection that is lazy loaded? Currently I get LazyInitializationException having this:
Club entity:
#Entity
public class Club {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
private String name;
#OneToMany(mappedBy = "club", cascade = CascadeType.PERSIST, fetch = FetchType.LAZY)
#JsonBackReference
private List<Player> players;
Player entity:
Is it a good idea to have two methods, where one fetches data without players and the second one that fetches also players?
#Override
List<Club> findAll();
#Query("Select clubs from Club clubs left join fetch clubs.players")
List<Club> getAllClubsWithPlayers();
What I'm thinking of is that it is a bad idea, because if I have a situation where I have for example 4 properties that are lazy loaded and I'd need at once 3 of them, I'd have to have a query like: getAllClubsWithPlayersAndSponsorsAndCoaches, so I'd have to have a lot of combinations of such queries.
I don't want to use EAGER, so could you tell me if it's a common way to do this if I need access to players from the Club sometimes which is undoable with the findAll() method that throws LazyInitializationException?
Don't get me wrong - I know where the LazyInitializationException comes from, but I just don't know what's the best way to get access to players if I sometimes need them when fetching clubs. Is my way of doing it correct?
There are 3 choices:
Access all the lazy fields inside a #Transactional method. You don't show your code, but there's usually a Service Facade layer which is responsible for being #Transactional. It invokes Repositories.
Write a query that fetches all the required data. Then you'd have to create a method specifically to fetch all the lazy fields required for that logic.
Use OpenSessionInViewFilter or OpenSessionInViewInterceptor so that Session/EntityManager are started before the execution even reaches the Controller. The Session then would be closed by the same high-level layer at the end of the request processing.
In addition to what Stanislav wrote, I'd like to elaborate on his 2nd point, because I think that this is often the best approach - that's simply because it saves unnecessary calls to the database which results in better performance.
Apart from writing separate JPQL query in your repository for each use-case, you could do one of the following .:
Make your repository extend JpaSpecificationExecutor and programmatically describe what needs to be fetched as described in this answer
Use Entity Graph either described using annotations, or programmatically, and fetch your entities using EntityManager as described in this tutorial
To optionally load what you want you can use EntityGraph.
Declare #NamedEntityGraph at your entity
#Entity
#NamedEntityGraph(name = "Club.players",
attributeNodes = #NamedAttributeNode("players")
)
public class Club {
Then you should annotate your findAll() method with this graph using it's name
#EntityGraph(value = "Club.players")
List<Club> findAll();
However that would override your basic findAll() method.
To avoid this (to have both implementations) you can follow this way:
Add dependency from https://mvnrepository.com/artifact/com.cosium.spring.data/spring-data-jpa-entity-graph/<version>
Then replace your repository with
#Repository
public interface ClubRepository extends JpaSpecificationExecutor<Club>, JpaRepository<Club, Long>, EntityGraphJpaSpecificationExecutor<Club> {
}
And then you'll have basic method findAll() and also from your sevice you can call
List<Club> clubs = clubRepository.findAll(specification, new NamedEntityGraph(EntityGraphType.FETCH, "Club.players"))
I'm trying to create a favorites mapping and I'm somewhat confused on how to do it,
Here is my user class, for the sake of clarity it has only 2 bindings(1st one(file) - #OneToMany, 2nd one(favorite file) - #ManyToMany.
#OneToMany(mappedBy = "app_user")
#JsonIgnoreProperties("app_user")
private List<File> file;
#ManyToMany(mappedBy = "app_user")
#JsonIgnoreProperties("app_user")
private Set<File> favorite_file;
And I've just realized that intellij complains about it for some reason.
it tries to connect only to 1 field on the other side:
#JsonIgnoreProperties({"file", "profile"})
#ManyToOne
private AppUser app_user;
what would be a better way to do this? because Intellij complains about it and it works but only with native queries(since JPA claims that there is no difference between files and favorite files, but in native query there is a ManyToMany table so it works there)
My goal is to make users have favorite files distinct from files.
It tries to connect only to 1 field on the other side
Well with #OneToMany(mappedBy = "app_user") you explicitly tell it to map it to the same field.
Which doesn't make sense, because
why would you have the same relation represented twice in one entity
The relationship types don't match. You can't have one relationship be ManyToOne from the one side and in the inverse direction be ManyToMany
It looks you have some clarification to do on what you want to represent by your model.
I see multiple possible variants.
The general mapping in AppUser is correct
So an AppUser has multiple File instances but each File is only owned by a single AppUser (if at all).
But an AppUser favorites many files and each File might be favoured by many users.
#Entity
class AppUser {
#OneToMany(mappedBy = "ownedBy")
private List<File> files;
#ManyToMany(mappedBy = "favoredBy")
private Set<File> favoriteFiles;
}
#Entity
class File {
#ManyToOne
private AppUser ownedBy;
#ManyToMany
private Set<AppUser> favoredBy;
}
The important part is that the defining relationships (the one in File) are two distinct relationships.
I removed the JSON annotations, because they are independent of the discussed problem.
Note that it is an interesting discussion if the relationship even should be bidirectional, but that is a different topic.
The favourite files are a subset of the owned files.
Then a more fitting model would be to model favouriting as an attribute of the relationship between AppUser and File
#Entity
class AppUser {
#OneToMany
private List<FileRelation> files;
eFiles;
}
#Entity
class FileRelation {
#ManyToOne
File file;
boolean isFavourite;
}
#Entity
class File {
}
I made the relationships unidirectional and the overall relation between user and file a many to many.
You might want to adapt that to your needs.
Side remark: Please use standard Java naming conventions when coding in Java (no snake case for attributes). Otherwise you are seriously hurting other Java developers.
I have an entity as
#Getter
#Setter
#Entity
#Table(name = "feature")
#JsonIgnoreProperties({"hibernateLazyInitializer", "handler"})
public class Feature {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Integer id;
#Column(name = "name")
private String name;
#Column(name = "description")
private String description;
#OneToMany(mappedBy = "featureId", fetch = FetchType.LAZY)
private transient Collection<FeatureComponent> components;
}
While in its Repository(Dao) file, I have
public interface FeatureDao extends JpaRepository<Feature, Integer> {
#Query("SELECT e FROM Feature e")
public List<Feature> getAll();
#Query("SELECT e FROM Feature e LEFT JOIN e.components fc WHERE e.id= :id")
public Feature getWithDetail(#Param("id") Integer id);
}
When I'm calling featureDao.getAll(); it returns all features but including components list filled and because of that, my response it being too large to load on client-side.
I'm unable to understand why it is happening when I'm using Lazy fetch mode and didn't mentioned joining with components in my getAll method.
Please help to resolve that issue,
Thanks in advance.
Just like #spOOm already mentioned I also suspect this is the side effect of Jackson Feature entity serialization into JSON triggering the load of all the components.
That is why using DTOs instead of plain Entities is usually advisable when returning data via a Controller. With DTOs, you clearly define whatever you want to include in the response to the caller. You can even reorganize your model so that it fits better the needs of the clients. You decouple your inner model and the model your API consumers know, making it possible to rework your inner model and still keep the same public model. You could have the following DTO.
public class FeatureSimpleDto {
private Integer id;
private String name;
private String description;
}
Then, in your Controller or Service (here you can find different opinions) you would basically convert your Feature entity into a FeatureSimpleDto that would be returned by your Controller. Here you can use mapping libraries such as MapStruct or you can do it on your own (I usually tend to prefer doing it on my own, one less dependency to rely on).
Using Lombok may be a problem, depending on the relationship between tables, try to create getters manually in entity classes.
Thanks to everyone for providing workarounds... But every work item requires lots of changes which were not possible for me...
Luckily I found a solution that is working fine for me... Better to post here...
Step-1: Add dependency in pom.xml file
<dependency>
<groupId>com.fasterxml.jackson.datatype</groupId>
<artifactId>jackson-datatype-hibernate5</artifactId>
</dependency>
Step-2: Add a 'Bean' for Hibernate Module
We can add bean in any file having #Configuration annotation... Even, we can add in Springboot main application file(where we have main method).
#Bean
public Module datatypeHibernateModule() {
return new Hibernate5Module();
}
That's it, Happy Coding...
So, I have found myself in quite a pickle regarding Hibernate. When I started developing my web application, I used "eager" loading everywhere so I could easily access children, parents etc.
After a while, I ran into my first problem - re-saving of deleted objects. Multiple stackoverflow threads suggested that I should remove the object from all the collections that it's in. Reading those suggestions made my "spidey sense" tickle as my relations weren't really simple and I had to iterate multiple objects which made my code look kind of ugly and made me wonder if this was the best approach.
For example, when deleting Employee (that belongs to User in a sense that User can act as multiple different Employees). Let's say Employee can leave Feedback to Party, so Employee can have multiple Feedback and Party can have multiple Feedback. Additionally, both Employee and Party belong to some kind of a parent object, let's say an Organization. Basically, we have:
class User {
// Has many
Set<Employee> employees;
// Has many
Set<Organization> organizations;
// Has many through employees
Set<Organization> associatedOrganizations;
}
class Employee {
// Belongs to
User user;
// Belongs to
Organization organization;
// Has many
Set<Feedback> feedbacks;
}
class Organization {
// Belongs to
User user;
// Has many
Set<Employee> employees;
// Has many
Set<Party> parties;
}
class Party {
// Belongs to
Organization organization;
// Has many
Set<Feedback> feedbacks;
}
class Feedback {
// Belongs to
Party party;
// Belongs to
Employee employee;
}
Here's what I ended up with when deleting an employee:
// First remove feedbacks related to employee
Iterator<Feedback> iter = employee.getFeedbacks().iterator();
while (iter.hasNext()) {
Feedback feedback = iter.next();
iter.remove();
feedback.getParty().getFeedbacks().remove(feedback);
session.delete(feedback);
}
session.update(employee);
// Now remove employee from organization
Organization organization = employee.getOrganization();
organization.getEmployees().remove(employee);
session.update(organization);
This is, by my definition, ugly. I would've assumed that by using
#Cascade({CascadeType.ALL})
then Hibernate would magically remove Employee from all associations by simply doing:
session.delete(employee);
instead I get:
Error during managed flush [deleted object would be re-saved by cascade (remove deleted object from associations)
So, in order to try to get my code a bit cleaner and maybe even optimized (sometimes lazy fetch is enough, sometimes I need eager), I tried lazy fetching almost everything and hoping that if I do, for example:
employee.getFeedbacks()
then the feedbacks are nicely fetched without any problem but nope, everything breaks:
failed to lazily initialize a collection of role: ..., could not initialize proxy - no Session
The next thing I thought about was removing the possibility for objects to insert/delete their related children objects but that would probably be a bad idea performance-wise - inserting every object separately with
child.parent=parent
instead of in a bulk with
parent.children().add(children).
Finally, I saw that multiple people recommended creating my own custom queries and stuff but at that point, why should I even bother with Hibernate? Is there really no good way to handle my problem relatively clean or am I missing something or am I an idiot?
If I understood the question correctly it's all about cascading through simple 1:N relations. In that case Hibernate can do the job rather well:
#Entity
public class Post {
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
private Long id;
#OneToMany(cascade = CascadeType.ALL,
mappedBy = "post", orphanRemoval = true)
private List<Comment> comments = new ArrayList<>();
}
#Entity
public class Comment {
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
private Long id;
#ManyToOne
private Post post;
}
Code:
Post post = newPost();
doInTransaction(session -> {
session.delete(post);
});
Generates:
delete from Comment where id = 1
delete from Comment where id = 2
delete from Post where id = 1
But if you have some other (synthetic) collections, Hibernate has no chance to know which ones, so you have to handle them yourself.
As for Hibernate and custom queries, Hibernate provides HQL which is more compact then traditional SQL, but still is less transparent then annotations.
Working with Spring Data REST, if you have a OneToMany or ManyToOne relationship, the PUT operation returns 200 on the "non-owning" entity but does not actually persist the joined resource.
Example Entities:
#Entity(name = 'author')
#ToString
class AuthorEntity implements Author {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
Long id
String fullName
#ManyToMany(mappedBy = 'authors')
Set<BookEntity> books
}
#Entity(name = 'book')
#EqualsAndHashCode
class BookEntity implements Book {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
Long id
#Column(nullable = false)
String title
#Column(nullable = false)
String isbn
#Column(nullable = false)
String publisher
#ManyToMany(fetch = FetchType.LAZY, cascade = [CascadeType.ALL])
Set<AuthorEntity> authors
}
If you back them with a PagingAndSortingRepository, you can GET a Book, follow the authors link on the book and do a PUT with the URI of a author to associate with. You cannot go the other way.
If you do a GET on an Author and do a PUT on its books link, the response returns 200, but the relationship is never persisted.
Is this the expected behavior?
tl;dr
The key to that is not so much anything in Spring Data REST - as you can easily get it to work in your scenario - but making sure that your model keeps both ends of the association in sync.
The problem
The problem you see here arises from the fact that Spring Data REST basically modifies the books property of your AuthorEntity. That itself doesn't reflect this update in the authors property of the BookEntity. This has to be worked around manually, which is not a constraint that Spring Data REST makes up but the way that JPA works in general. You will be able to reproduce the erroneous behavior by simply invoking setters manually and trying to persist the result.
How to solve this?
If removing the bi-directional association is not an option (see below on why I'd recommend this) the only way to make this work is to make sure changes to the association are reflected on both sides. Usually people take care of this by manually adding the author to the BookEntity when a book is added:
class AuthorEntity {
void add(BookEntity book) {
this.books.add(book);
if (!book.getAuthors().contains(this)) {
book.add(this);
}
}
}
The additional if clause would've to be added on the BookEntity side as well if you want to make sure that changes from the other side are propagated, too. The if is basically required as otherwise the two methods would constantly call themselves.
Spring Data REST, by default uses field access so that theres actually no method that you can put this logic into. One option would be to switch to property access and put the logic into the setters. Another option is to use a method annotated with #PreUpdate/#PrePersist that iterates over the entities and makes sure the modifications are reflected on both sides.
Removing the root cause of the issue
As you can see, this adds quite a lot of complexity to the domain model. As I joked on Twitter yesterday:
#1 rule of bi-directional associations: don't use them… :)
It usually simplifies the matter if you try not to use bi-directional relationship whenever possible and rather fall back to a repository to obtain all the entities that make up the backside of the association.
A good heuristics to determine which side to cut is to think about which side of the association is really core and crucial to the domain you're modeling. In your case I'd argue that it's perfectly fine for an author to exist with no books written by her. On the flip side, a book without an author doesn't make too much sense at all. So I'd keep the authors property in BookEntity but introduce the following method on the BookRepository:
interface BookRepository extends Repository<Book, Long> {
List<Book> findByAuthor(Author author);
}
Yes, that requires all clients that previously could just have invoked author.getBooks() to now work with a repository. But on the positive side you've removed all the cruft from your domain objects and created a clear dependency direction from book to author along the way. Books depend on authors but not the other way round.
I faced a similar problem, while sending my POJO(containing bi-directional mapping #OneToMany and #ManyToOne) as JSON via REST api, the data was persisted in both the parent and child entities but the foreign key relation was not established. This happens because bidirectional associations need to be manually maintained.
JPA provides an annotation #PrePersist which can be used to make sure that the method annotated with it is executed before the entity is persisted. Since, JPA first inserts the parent entity to the database followed by the child entity, I included a method annotated with #PrePersist which would iterate through the list of child entities and manually set the parent entity to it.
In your case it would be something like this:
class AuthorEntitiy {
#PrePersist
public void populateBooks {
for(BookEntity book : books)
book.addToAuthorList(this);
}
}
class BookEntity {
#PrePersist
public void populateAuthors {
for(AuthorEntity author : authors)
author.addToBookList(this);
}
}
After this you might get an infinite recursion error, to avoid that annotate your parent class with #JsonManagedReference and your child class with #JsonBackReference. This solution worked for me, hopefully it will work for you too.
This link has a very good tutorial on how you can navigate the recursion problem:Bidirectional Relationships
I was able to use #JsonManagedReference and #JsonBackReference and it worked like a charm
I believe one can also utilize #RepositoryEventHandler by adding a #BeforeLinkSave handler to cross link the bidirectional relation between entities. This seems to be working for me.
#Component
#RepositoryEventHandler
public class BiDirectionalLinkHandler {
#HandleBeforeLinkSave
public void crossLink(Author author, Collection<Books> books) {
for (Book b : books) {
b.setAuthor(author);
}
}
}
Note: #HandlBeforeLinkSave is called based on the first parameter, if you have multiple relations in your equivalent of an Author class, the second param should be Object and you will need to test within the method for the different relation types.