Develop Reference

JPA/Hibernate Bulk/Batch Insert using Mysql with Database Generated ID's

Okay, I've searched forever and I can't seem to find a good way of accomplishing batch inserts with JPA/Hibernate and MySql.
I want to be able to save/insert many records at once using JPA, but by default batching behavior is disabled if you use GenerationType.IDENTITY. I'm aware that you can switch to GenerationType.SEQUENCE, but that isn't available on MySql and creating new tables and using GenerationType.TABLE is not an option in my scenario.
So in the end, I need an efficient way of doing batch/bulk inserts using JPA/Hibernate, MySQL, and database generated IDs. I know it's possible to do this efficiently because I can do it with a JDBC connection, but I'd really like to not have to write my own JDBC queries for each of my repositories.
Anyone know how to accomplish this?
I'm okay if I'm unable to get the updated entities with the IDs back (think void saveAll() instead of List<User> saveAll()). My main requirement is this happens in one/two big queries instead of saving iteratively each entity like it does now when I call saveAll.
I can include more if needed, but my entity looks like this:
#Entity
#Builder
#Getter
#Setter
#With
#AllArgsConstructor
#NoArgsConstructor
#EqualsAndHashCode(callSuper = false, exclude = "id")
#Table(name = "user")
#ToString(callSuper = true, onlyExplicitlyIncluded = true)
public class User {
#Id
#ToString.Include
#GeneratedValue(strategy = GenerationType.IDENTITY)
#Column(name = "uID")
private long id;
private String name;
}

There is no way to accomplish JDBC batching on insert with Hibernate when using the identity generation strategy, because for Hibernate, every entity must have a PK value assigned after a persist/insert.
You can use Hibernate SPIs to implement this yourself though. Take a look at how Hibernate implements inserts here org.hibernate.persister.entity.AbstractEntityPersister#insert(java.lang.Object, java.lang.Object[], java.lang.Object, org.hibernate.engine.spi.SharedSessionContractImplementor). You can reduce the complexity if you want to implement this only for a few known entities that only use a handful of features.

IDENTITY generator disables JDBC batch insert of hibernate and JPA. since the sequence is not supported in MySQL, there is no way to bulk/batch insert records using MySQL and spring data JPA. Please read my blog on that. This is not the end of the road. we can use the JDBC template or query DSL-SQL. To see how to implement using query DSL-SQL click here. For the JDBC template click here.
If you need type-safe, easy to code choose query DSL-SQL else choose JDBC template

How do I make JPA entity field truly write-only

I have a case where I'm persisting a large jsonb field into a PostGres table, but do not want to read it when I fetch the entity; if I do fetch it, my service goes OOM. A better design might be to separate this into a 1 to 1 table, but I can't do that at this time.
To plead that this is not a duplicate question, here's some of my research:
I'm not able to mark the column LAZY since I have a simple column not a join`
JPA/Hibernate write only field with no read
I tried the empty setter in this suggestion, which makes sense - but it still appears to read the column and I OOM: https://www.zizka.ch/pages/programming/java/hibernate/hibernate-write-only.html
I also tried omitting the setter altogether in my #Data class: Omitting one Setter/Getter in Lombok
So, I can not see the field, but I can't seem to keep it from being read into memory in the background. It seems like there must be some simple setting in JPA or Hibernate to exclude a column from read. Before I go try to make a complex repository hierarchy just to see if it works, I thought I would ask here in case I get lucky.
Thanks in advance!

Lazy loading attributes
Hibernate can load attribute lazily, but you need to enable byte code enhancements:
First you need to set the property hibernate.enhancer.enableLazyInitialization to true
Then you can annotate the field with #Basic( fetch = FetchType.LAZY ).
Here's the example from the documentation:
#Entity
public class Customer {
#Id
private Integer id;
private String name;
#Basic( fetch = FetchType.LAZY )
private UUID accountsPayableXrefId;
#Lob
#Basic( fetch = FetchType.LAZY )
#LazyGroup( "lobs" )
private Blob image;
//Getters and setters are omitted for brevity
}
You can also enable this feature via the Hibernate ORM gradle plugin
Named Native queries
You could also decide to not map it and save/read it with a named native query. It seems a good trade off for a single attribute - it will just require an additional query to save the json.
Example:
#Entity
#Table(name = "MyEntity_table")
#NamedNativeQuery(
name = "write_json",
query = "update MyEntity_table set json_column = :json where id = :id")
#NamedNativeQuery(
name = "read_json",
query = "select json_column from MyEntity_table where id = :id")
class MyEntity {
....
}
Long id = ...
String jsonString = ...
session.createNamedQuery( "write_json" )
.setParameter( "id", id )
.setParameter( "json", jsonString )
.executeUpdate();
jsonString = (String)session.createNamedQuery( "read_json" )
.setParameter( "id", id )
.getSingleResult();
In this case, schema generation is not going to create the column, so you will need to add it manually (not a big deal, considering that there are better tools to update the schema in production).
MappedSuperclass
You can also have two entities extending the same superclass (this way you don't have to copy the attributes). They have to update the same table:
#MappedSuperclass
class MyEntity {
#Id
Long id;
String name
...
}
#Entity
#Table(name = "MyEntity_table")
class MyEntityWriter extends MyEntity {
String json
}
#Entity
#Table(name = "MyEntity_table")
class MyEntityReader extends MyEntity {
// No field is necessary here
}
Now you can use MyEntityWriter for saving all the values and MyEntityReader for loading only the values you need.
I think you will have some problems with schema generation if you try to create the tables because only one of the two will be created:
If MyEntityWriter is the first table created, then no problem
If MyEntityWriter is the second table created, the query will fail because the table already exist and the additional column won't be created.
I haven't tested this solution though, there might be something I haven't thought about.

Best Way to SELECT subset of properties for associated #Entity

I am trying to choose the properties of associated entities that will be loaded.
For example:
#Entity
#Getter #Setter
public class Book {
#Id #GeneratedValue(strategy = GenerationType.AUTO)
private Long id;
#Column(name = "first")
private String first;
#Column(name = "second")
private String second;
#OneToMany(mappedBy = "book", cascade = CascadeType.ALL)
private List<Page> pages = new ArrayList();
}
#Entity
#Getter #Setter
public class Page {
#Id #GeneratedValue(strategy = GenerationType.AUTO)
private Long id;
#Column(name = "number")
private Integer number;
#Column(name = "content")
private String content;
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "book_id")
private Book book;
}
I would like to have just a SELECT on the Book's first and the content of all associated Page entities
book
- first
- pages
- content
For example, in PostgreSQL this might look something like
SELECT book.first,
array_agg(page.content || ' ')
FROM book
LEFT JOIN page
ON page.book_id = book.id
GROUP BY book.first
I've done my research on how you could go about doing this, most mentioned solution is to use Spring DATA JPA Projections (ref)
I did the following:
public interface FindAllBookProjection {
String getFirst();
List<PageProjection> getPages();
interface PageProjection {
Integer getNumber();
}
}
The issue with projections is that they cause the famous N+1 select problem as the pages are loaded lazily.
I also couldn't find a way to use projections with #Query in the #Repository for the nested properties.
There are other mentions online to use #EntityGraph. From what I understand #EntityGraph will not work as it also selects all properties, specifying only what Entity associations should be loaded.
There are also suggestions on using other libraries like Blaze Persistence (ref) or Hibernate's ResultTransformer (ref) . I would prefer to use only Spring DATA JPA instead of introducing another library for this functionality and writing ResultTransformers seems like adding a lot of boilerplate code to the mix.
To summarize my question is what is the best way to choose what properties are selected for #Entity associations. The main goal is to avoid pulling unnecessary amount of data from the database. The given example above is for demonstration, the data that I am working on includes over 10 columns and spans across 3-4 entities. Having control over the data means better performance.

I am the creator of Blaze-Persistence and I can tell you that if there were an easy way to do this, I would not have created Entity-Views. You can read some of the articles I wrote about this and you will realize that doing this yourself, like you already figured, will require lots of boilerplate code. Doing this efficiently, will require even more code and in the end, you will probably end up with a solution that is inferior to Blaze-Persistence Entity-Views in one way or another and has probably no documentation.
Spring Data Projections is just limited and as I tried to outline many times before, Blaze-Persistence Entity-Views is like Spring Data Projections on steroids.
If you have "just" one or two simple mappings, you might be able to get this done by introducing special #Immutable #Entity, maybe even with #Subselect in this particular case to model what you need, but believe me, this only works good on a small scale. Apart from that, Blaze-Persistence which works on top of JPA/Hibernate enables the use of a lot of advanced SQL features, which you usually can't use from within plain JPA/Hibernate.

In my opinion, ORM libraries should use whole objects, that means loading all data into the program and then transforming/filtering according to logic.
For specific use-cases, where performance is really hindered, I'd use entities in
the database, such as Views/Procedures.
In your case, i'd create a View:
CREATE VIEW book_content as
SELECT book.first as [first],
array_agg(page.content || ' ') as [content]
FROM book
LEFT JOIN page
ON page.book_id = book.id
GROUP BY book.first
And then create a #Repository and #Entity for it in spring.

JPA #Column definition does not apply to read operations?

Given I have entity Car with column model which doesn't accept NULLs
#Table(name = "CAR")
#Entity
public class Car extends AbstractEntity<Long> {
#Column(name = "MODEL", nullable = false)
private final String model;
}
When I prepare database schema, insert data (including NULLs in MODEL column) manually and start up application, it doesn't fail to start.
Why is that?
Do conditions specified in #Column annotation only apply for insert/update operations, not for read operations?

Yes, you can read null values with nullable = false. But when you try to save or update an entity with model = null, the JPA lever error will be thrown.
Check out the specification for nullable.
This JPA constraints just prohibit non-valid data from being written to the database, in order not to call it for no reason (by the way, you should have the same constraints in your database as you have in JPA).
These constraints have nothing to do with data that is already there. So that's why your application doesn't fail to start.
Have a look at this answer for better explanation.

Many to many bidirectional mapping in JPA

I have the following JPA entities.
A profile have many users and a user have many profiles:
#Entity
public class Profile implements Serializable {
#Id
private Long id;
#ManyToMany(cascade = CascadeType.ALL)
private List<User> users;
...
}
#Entity
public class User implements Serializable {
#Id
private Long id;
#ManyToMany(mappedBy = "users")
private List<Profile> profiles;
...
}
On my application, when a user is merged, the profiles are updated on database.
However, when a profile is merged, the users are not updated.
Is possible to map my entities in order to make both sides merge their lists?
I am using JPA 2.1 and Hibernate.

Your Profile entity is ownind side or relationship. It's up to it, to manage relationship, so in order to update User you'll have to update Profile too or make manual SQL calls.
Java Specification for JPA 2.1 says that:
• For many-to-many bidirectional relationships either side may be the owning side
So if you'd like to make both entities editable from both side, remove mappedBy element and assigne necessacy cascade. But I'm not sure it works in Hibernate (didn't try actually), see this docs on mapping, there's no information about m:m without owning side: http://docs.jboss.org/hibernate/orm/4.3/manual/en-US/html/ch07.html#collections-bidirectional
Otherwise, you may need to iterate through collection in Profile entity and then change them. For example:
for( User user : profile.getUsers() ) {
user.setSomething(.....);
}
session.merge(profile);
Changing List to Set might be needed in order to avoid Hibernate's delete and reinsert, described here: http://assarconsulting.blogspot.fr/2009/08/why-hibernate-does-delete-all-then-re.html
Also, don't forget about equals() and hashCode() methods override