JPA/HIbernate Very Slow Query while JDBC/Postgres is Fast - java

I am running a spring boot application running Hibernate 5.4.2 and Postgresql 42.2.2 DB running on Heroku.
I am using JPARepository's default findAll() method to select my entire table. There are about ~50 entities and it's taking ~35 seconds. When I run a select all query using the Heroku CLI via terminal on the same machine, the query finishes in ~130milliseconds.
I have been trying to use JProfiler to pinpoint the problem. Looking at the query via JProfiler, I notice that the associated JDBC call takes about ~140ms which is close to the Heroku CLI. However the query itself still takes 35 seconds...
I have turned hibernate logging on and made sure that there are no other queries being sent. Also, I have tried to run the same query backing my application with a H2 database. The query returned in less than a hundred milliseconds.
See JProfiler query time https://i.stack.imgur.com/tAKwf.png
My entity is quite simple. There are no relationships to other classes so I dont think its a n+1 query problem.
#Entity
#Table(name = "event")
#NoArgsConstructor
#Getter
#ToString
#EqualsAndHashCode(callSuper = true)
public class SqlEvent extends SerializedObjectRecord {
#Id
#GeneratedValue(strategy = GenerationType.SEQUENCE)
#Column(name = "id", updatable = false, nullable = false)
#NonNull
private Long id;
#Column(nullable = false)
#NonNull
private String name;
#Column(nullable = false)
#NonNull
private UUID uuid;
#Column(nullable = false)
#NonNull
private UUID sqlCommandUuid;
#Column(nullable = false)
#NonNull
private Instant timestamp;
#Column(nullable = false)
#NonNull
private String avroSchemaFingerprint;
My service class
#Override
#Timed(value = "service")
public Collection<Pair> getAll(UUID commandId) {
long start = System.nanoTime();
List<SqlEvent> list = this.sqlEventRepository.findAll();
long end = System.nanoTime();
long duration = (end - start) / 1000000; #duration takes ~35000ms like Jprofiler says
return convert(list);
}
[1]: https://i.stack.imgur.com/tAKwf.png

Related

How to use derived columns in Spring Data JPA associations

I have an entity TeamActivity:
#Entity
#Table(name = "teams_to_activities")
public class TeamActivity {
#Column(name = "scope_id", nullable = false)
private String scopeId;
#Column(name = "team_id", nullable = false)
private String teamId;
#Column(name = "activity_set_id", nullable = false)
private String activitySetId;
#Id
#Column(name = "scoped_team_activity_id", nullable = false)
private String scopedTeamActivityId;
}
And another entity ActivitySet:
#Entity
#Table(name = "activity_sets")
public class ActivitySet {
#Column(name = "scope_id", nullable = false)
private String scopeId;
#Column(name = "name", nullable = false)
private String name;
#Column(name = "description", nullable = false)
private String description;
#Id
#Column(name = "scoped_activity_set_id", nullable = false)
private String scopedActivitySetId;
}
There's no index on any other column besides the PK in both tables.
There's no FK constraint creating a relationship between these tables whatsoever. I have no idea why as this is a legacy system.
Technically, if I fetch a TeamActivity record, I can pick the scope_id and activity_set_id from it and combine them to form a scoped_activity_set_id which would be a valid PK to fetch the corresponding ActivitySet.
I know that TeamActivity -> ActivitySet is a N -> 1 association
I would like to leverage Spring Data JPA features to create an association from TeamActivity to ActivitySet such that when I fetch a TeamActivity from TeamActivityRepository, the corresponding ActivitySet is also returned.
I have created an association like this before using a combination of #JoinColumn and #MapsId but there was actually a single FK to use which is different here where source table has 2 columns I can combine to get the target's key.
If you are fully in control of the database, I may propose you create a Materialized View with the contents you desire from both tables and handle it as any other table with JPA, i.e, create #Entity model and CrudRepository<MVTeamActivitySet, String>.
If you are not fully in control of the database, one easy way to achieve it is to simply create a method that internally executes two lookup queries and retrieves the expected model you want. You will still be using using JPA correctly.
Querying two tables and joining desired fields in the code layer is quite common with denormalized DBs, sometimes you want to avoid the overhead of a Materialized View.
#Override
public TeamActivitySetDto findById(String scopedTeamActivityId) throws DemoCustomException {
Optional<TeamActivity> teamActivityEntity = teamActivityDao.getById(scopedTeamActivityId);
if(teamActivityEntity.isEmpty()) {
throw new DemoCustomException("teamActivity record not found");
}
String scopedActivitySetId =
teamActivityEntity.get().getScopeId() + ":" + teamActivityEntity.get().getActivitySetId();
Optional<ActivitySet> activitySetEntity = activitySetDao.getById(scopedActivitySetId);
if(activitySetEntity.isEmpty()) {
throw new DemoCustomException("activitySet record not found");
}
return TeamActivitySetDto.builder()
.description(activitySetEntity.get().getDescription())
.name(activitySetEntity.get().getName())
.scopedActivitySetId(activitySetEntity.get().getScopedActivitySetId())
.activitySetId(teamActivityEntity.get().getActivitySetId())
.scopedTeamActivityId(teamActivityEntity.get().getScopedTeamActivityId())
.scopeId(teamActivityEntity.get().getScopeId())
.teamId(teamActivityEntity.get().getTeamId())
.build();
}

HibernateProxy.toString lazy initializacion exception

I'm getting a weird error while I'm debugging my POC.
I have 2 entities:
#Getter
#Setter
#Entity
#Table(name = "APPLICANT")
public class Applicant implements Serializable {
private static final long serialVersionUID = 6060170457948717553L;
#Id
#Column(name = "applicant_id", insertable = false, nullable = false)
private Long applicantId;
#Column(name = "application_id", unique = true)
private String applicationId;
#OneToOne(fetch = FetchType.LAZY, optional = false)
#JoinColumn(name = "application_id", referencedColumnName = "application_id", insertable =
false, updatable = false)
private ApplicationEntity applicationEntity;
#Getter
#Setter
#Entity
#Table(name = "APPLICATION")
public class ApplicationEntity implements Serializable{
private static final long serialVersionUID = 7300036359295729197L;
#Id
#Column(name = "APPLICATION_ID")
private String id;
#OneToOne(mappedBy = "applicationEntity", fetch = FetchType.LAZY)
private Applicant applicant;
These classes has the repositories interfaces extending from CrudRepository, and in the Applicant repository I have a custom method to get the entity with the applicationId:
Applicant findByApplicationId(String applicationId);
But, when I'm debugging, I see the following message in the intellij debuguer on the applicationEntity attribute:
Method threw 'org.hibernate.LazyInitializationException' exception. Cannot evaluate org.example.postgres.jpa.model.ApplicationEntity$HibernateProxy$qa4PKx8V.toString()
The value qa4PKx8V changes every time that I perform a new test.
I tried a lot of combinations in the #Join annotation, I've deleted the lombook annotations, I've used the #Transactional annotation either, but is always the same error.
A key point to note, is that I can get the data from the table with any error, I just see this message in the debugger, so my question is, this is a thing of intellij or something like that? Or I need to fix this with configuration or changing something in my code?
Thanks.
I am assuming you have an autogenerated toString() implementation?
In general, you should avoid referencing lazily-loaded properties in toString(), equals(), hashCode() etc. Failing to do so will result in LazyInitializationException surprises like the one you're facing, triggered by the aforementioned methods whenever they try to access lazy properties outside of an active transaction context.
(This is indeed 'a thing of intellij', in the sense that although the debugged code is probably surrounded by a transaction, the Intellij inspector evaluates the expression on a separate thread where no transaction is active = no persistence context is open. Also, it will only happen with #XxxToOne(optional = false) properties)

Hibernate 5.x memory leak - looks like HQL queries caching?

We have a bog standard REST-based Hibernate application.
Recently we noticed that it dies the death by Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded time and again. We've emulated what happens there in a test, that collects items from a collection of elements. After about 1000 runs the application goes kaput with the pattern as below. The production displays similar behavior, although the time scale is much longer:
Java profiler shows the abundance of hibernate Node elements in a hashmap:
When drilling deeper into the what these nodes are I can see the following:
It looks like Hibernate caches HQL queries (even if we run the same query over and over again).
The class that comprises the offending collection is below:
#Entity
#Audited
#Table(name = "benefit_package", uniqueConstraints = {
#UniqueConstraint(name = "uk01_benefit_package", columnNames = {"employer_guid", "benefit_package_name"}),
#UniqueConstraint(name = "uk02_benefit_package", columnNames = {"benefit_package_guid"})})
#SequenceGenerator(name = "benefit_package_sequence", sequenceName = "benefit_package_id_seq", allocationSize = 1, initialValue = 1)
#EqualsAndHashCode(of = {"employerId", "name"})
#ToString(of = {"id", "employerId", "name"})
public class BenefitPackage {
#Id
#Column(name = "benefit_package_id")
#GeneratedValue(generator = "benefit_package_sequence", strategy = SEQUENCE)
private Long id;
#Column(name = "benefit_package_guid", nullable = false, columnDefinition = "binary(16)")
private UUID guid;
#Column(name = "benefit_package_name", nullable = false)
private String name;
#Column(name = "employer_guid", nullable = false, columnDefinition = "binary(16)")
private UUID employerId;
#ManyToOne
#JoinColumn(name = "selection_control_id")
private SelectionControl selectionControl;
#OneToMany(mappedBy = "benefitPackage")
private List<BenefitPackageVersion> versions = new ArrayList<>();
#Column(name = "last_modified_date")
private LocalDateTime lastModifiedDate;
#Version
#Column(name = "optlock")
private Long optLock;
This entity is used with pagination, but we do not use any custom JPA queries (at least overtly) for this operation.
Application details: SpringBoot 1.2.8, Hibernate 5.1 (Upgrading to 5.2.18Final had no effect). No second level cache is used.
Questions:
Why should Hibernate cache the same query over and over again?
Has this ever been noticed or addressed? Is there a fix for this problem?
I know the ticket might be a duplication, but I have not found definitive answer to that question anywhere.

Spring Data update entity with yesterday's date

I have some problems with update (and also insert) data into my database. I have an entity with some integer properties, some String properties, but also is there one property with LocalDate type, and it has to be unique.
I put a lot of entities like that into the database, but user needs to edit it and update some properties. When I tried to test it and change some String property and save updated entity to db I saw this error log in the console:
Duplicate entry '2019-07-27' for key 'work_day_workday_date_uindex'
As you can see, Hibernate tries to put object with yesterday's date. But... why? I checked it in traditional ( :D ) way -> by entering System.out.println instruction before saving object into database.
Log shows me a correct date in printing:
WorkDay{id=296, date=2019-07-28, workingTime=8,....
So I think that the problem is connected with differences in time between database and application.
I found some tips here, in StackOverflow. Somebody said that removing serverTimezone=UTC from application.properties in SpringBoot could help. And it fixed the problem - yesterday I updated the entity successfully. But today I come back to coding and the problem appeared again.
I hope that maybe some of you had this problem in past and know some solution - it will be very helpful for me :)
Here is WorkDay Entity:
#Entity
#Table(name = "work_day")
public class WorkDay implements Serializable {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
#Column(name = "id_workday")
private Long id;
#NotNull
#Column(name = "workday_date", nullable = false, unique = true)
private LocalDate date;
#NotNull
#Column(name = "working_time", nullable = false)
private Integer workingTime;
#Column(name = "booked_artist")
private String bookedArtist;
#ManyToOne
#JoinColumn(name="workday_importance_id")
private WorkDayImportance workDayImportance;
#ManyToMany
#JoinTable(name = "workday_employee",
joinColumns = {#JoinColumn(name = "workday_id",
referencedColumnName = "id_workday")},
inverseJoinColumns = {#JoinColumn(name="employee_id",
referencedColumnName = "id_employee")})
private List<Employee> employers;
#OneToMany(mappedBy = "workDay", cascade = CascadeType.REMOVE)
private List<Comment> comments;
Here is some code where I perform this operation:
public void setBookedArtist(Long workDayId, String artist){
workDayRepository
.findById(workDayId)
.ifPresent(workDay -> workDayDetailsService.saveBookedArtist(workDay, artist));
}
void saveBookedArtist(WorkDay workDay, String artist){
if(artist != null && !artist.equals("")) {
workDay.setBookedArtist(artist);
workDayRepository.save(workDay);
}
}
The entity repository is Spring Data interface which extends JpaRepository.
Best regards!
Setting the Id of workDay before saving the record should work and as we don't want to update the date set updatable = false as to below
public void setBookedArtist(Long workDayId, String artist){
workDayRepository
.findById(workDayId)
.ifPresent(workDay -> workDayDetailsService.saveBookedArtist(workDay, artist));
}
void saveBookedArtist(WorkDay workDay, String artist){
if(artist != null && !artist.equals("")) {
workDay.setId(workDay.getId());
workDay.setBookedArtist(artist);
workDayRepository.save(workDay);
}
}
#NotNull
#Column(name = "workday_date", nullable = false, unique = true, updatable = false)
private LocalDate date;

spring data jpa selecting values considering the date and time

I have an entity which is
#AllArgsConstructor
#NoArgsConstructor
#Data
#Entity
#Table(name = "REFRESH_TOKENS")
public class JwtRefreshToken {
#Id
#Column(name = "TOKEN")
private String token;
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "USER_ID", nullable = false)
private Tbluser user;
#Column(name = "EXPIRATIONDATETIME")
private LocalDateTime expirationDateTime;
public JwtRefreshToken(String token) {
this.token = token;
}
}
and the the corresponding repository is
JwtRefreshToken findByTokenAndAndExpirationDateTimeBefore( String token, #Param("expirationDateTime") LocalDateTime localDateTime);
The interesting thing here is the query always returns value even though the time has exceeded.
for example the value stored in database is 2019-04-21 22:33:08
and my current date time is 2019-04-21T23:02:43.971
but yet the above findByTokenAndAndExpirationDateTimeBefore returns value.
i want to compare the time as well.
You can enable debug output to see parameterized query and its parameters, add to your application properties
logging.level.org.hibernate.SQL=DEBUG
logging.level.org.hibernate.type.descriptor.sql.BasicBinder=TRACE
Probably, you will get some clues if query does not use #Param("expirationDateTime") LocalDateTime localDateTime at all or there is timezone issue or everything is fine and you just misinterpret results ;)

Categories

Resources