Develop Reference

Bulk inserting existing data: Preventing JPA to do a select before every insert

I'm working on a Spring Boot application that uses JPA (Hibernate) for the persistence layer.
I'm currently implementing a migration functionality. We basically dump all the existing entities of the system into an XML file. This export includes ids of the entities as well.
The problem I'm having is located on the other side, reimporting the existing data. In this step the XML gets transformed to a Java object again and persisted to the database.
When trying to save the entity, I'm using the merge method of the EntityManager class, which works: everything is saved successfully.
However when I turn on the query logging of Hibernate I see that before every insert query, a select query is executed to see if an entity with that id already exists. This is because the entity already has an id that I provided.
I understand this behavior and it actually makes sense. I'm sure however that the ids will not exist so the select does not make sense for my case. I'm saving thousands of records so that means thousands of select queries on large tables which is slowing down the importing process drastically.
My question: Is there a way to turn this "checking if an entity exists before inserting" off?
Additional information:
When I use entityManager.persist() instead of merge, I get this exception:
org.hibernate.PersistentObjectException: detached entity passed to
persist
To be able to use a supplied/provided id I use this id generator:
#Id
#GeneratedValue(generator = "use-id-or-generate")
#GenericGenerator(name = "use-id-or-generate", strategy = "be.stackoverflowexample.core.domain.UseIdOrGenerate")
#JsonIgnore
private String id;
The generator itself:
public class UseIdOrGenerate extends UUIDGenerator {
private String entityName;
#Override
public void configure(Type type, Properties params, ServiceRegistry serviceRegistry) throws MappingException {
entityName = params.getProperty(ENTITY_NAME);
super.configure(type, params, serviceRegistry);
}
#Override
public Serializable generate(SessionImplementor session, Object object)
{
Serializable id = session
.getEntityPersister(entityName, object)
.getIdentifier(object, session);
if (id == null) {
return super.generate(session, object);
} else {
return id;
}
}
}

If you are certain that you will never be updating any existing entry on the database and all the entities should be always freshly inserted, then I would go for the persist operation instead of a merge.
Per update
In that case (id field being set-up as autogenerated) the only way would be to remove the generation annotations from the id field and leave the configuration as:
#Id
#JsonIgnore
private String id;
So basically setting the id up for always being assigned manually. Then the persistence provider will consider your entity as transient even when the id is present.. meaning the persist would work and no extra selects would be generated.

I'm not sure I got whether you fill or not the ID. In the case you fill it on the application side, check the answer here. I copied it below:
Here is the code of Spring SimpleJpaRepository you are using by using Spring Data repository:
#Transactional
public <S extends T> S save(S entity) {
if (entityInformation.isNew(entity)) {
em.persist(entity);
return entity;
} else {
return em.merge(entity);
}
}
It does the following:
By default Spring Data JPA inspects the identifier property of the given entity. If the identifier property is null, then the entity will be assumed as new, otherwise as not new.
Link to Spring Data documentation
And so if one of your entity has an ID field not null, Spring will make Hibernate do an update (and so a SELECT before).
You can override this behavior by the 2 ways listed in the same documentation. An easy way is it to make your Entity implement Persistable (instead of Serializable), which will make you implement the method "isNew".

JPA: How do I avoid loading an object simply so I can store its ID in the database?

This question is so simple, you can probably just read the code
This is a very simple performance question. In the code example below, I wish to set the Owner on my Cat object. I have the ownerId, but the cats method for requires an Owner object, not a Long. Eg: setOwner(Owner owner)
#Autowired OwnerRepository ownerRepository;
#Autowired CatRepository catRepository;
Long ownerId = 21;
Cat cat = new Cat("Jake");
cat.setOwner(ownerRepository.findById(ownerId)); // What a waste of time
catRepository.save(cat)
I'm using the ownerId to load an Owner object, so I can call the setter on the Cat which is simply going to pull out the id, and save the Cat record with an owner_id. So essentially I'm loading an owner for nothing.
What is the correct pattern for this?

First of all, you should pay attention to your method to load an Owner entity.
If you're using an Hibernate Session :
// will return the persistent instance and never returns an uninitialized instance
session.get(Owner.class, id);
// might return a proxied instance that is initialized on-demand
session.load(Owner.class, id);
If you're using EntityManager :
// will return the persistent instance and never returns an uninitialized instance
em.find(Owner.class, id);
// might return a proxied instance that is initialized on-demand
em.getReference(Owner.class, id);
So, you should lazy load the Owner entity to avoid some hits to the cache nor the database.
By the way, I would suggest to inverse your relation between Owner and Cat.
For example :
Owner owner = ownerRepository.load(Owner.class, id);
owner.addCat(myCat);

Victor's answer is correct (+1 from me), but requires going through the EntityManager or Hibernate session. Assuming the repositories you have autowired are JPA repositories from Spring Data and you would prefer to go through them, use the JpaRepository#getOne method. It calls EntityManager#getReference, so it does the same thing, returning a proxy to the entity.
I do not think the relationship necessarily needs to be reversed here, which mapping to use depends on the situation. In many cases many-to-one is preferred.

Probably not what you were looking for, but nothing in your question implies that you have to solve this with JPA. Some things are just much much simpler with plain old SQL:
INSERT INTO cat (name, owner_id) VALUES ('Jake', 21)

If you are using Hibernate you can do this:
Long ownerId = 21;
Cat cat = new Cat("Jake");
Owner owner = new Owner();
owner.setId(ownerId);
cat.setOwner(owner);
catRepository.save(cat)
It's not standard JPA, but, if you are not willing to migrate to other JPA provider, it's the best from a performance perspective.
Update
As Nathan pointed out, you need to make sure the Owner is not already associated (in which case you can get a NonUniqueObjectException since the Persistence Context can have at most one entity associated in the 1st level cache).
Using EntityManager.contains(entity) doesn't help in this case, since Hibernate stores the entities in an IdentiyHashMap, where the key is the Object reference itself.
So you should use this method when, for example, you have a use case where you must insert these entities for the first time, or when you need to update them and the Owner hadn't been loaded in the current running Persistence Context (either directly or through JPQL or a Criteria API).
Otherwise, use EntityManager.getReferemce(Class entityClass, Object primaryKey).

One more way (can come handy sometimes in legacy code or db schema):
#Entity
public class Cat {
#Column(name = "OWNER_ID")
private Long ownerId;
#ManyToOne
#JoinColumn(name = "OWNER_ID", insertable = false, updatable = false)
private Owner owner;
}

JPA, removing an entity which has found by different manager

Assume we have a simple entity bean, like above
#Entity
public class Schemes implements serializable{
...
#Id private long id;
...
}
I find a record using find method and it works perfect, the problem is I cannot manipulate it(remove) by another EntityManager later, for example I find it with a method, and later I want to remove it, what is the problem?! if I find it with same manager again I would remove it, but if object has found by another manager I cannot.
#ManagedBean #SessionScopped class JSFBean {
private Schemes s;
public JSFBean(){
....
EntityManager em;//.....
s=em.find(Schemes.class,0x10L);//okay!
....
}
public void remove(){//later
....
EntityManager em;//.....
em.getTransaction().begin();
em.remove(s);//Error! some weird error, it throws IllegalArgumentException!
em.getTransaction().commit();
....
}
}
many thanks.

You are probably getting a java.lang.IllegalArgumentException: Removing a detached instance.
The two EMs do not share a persistence context and for the second EM, your object is considered detached. Trying to remove a detached object will result in an IllegalArgumentException.
You can refetch the entity before the removal:
Schemes originalS = em.find(Schemes.class, s.getId());
em.remove(originalS);
EDIT You can also delete the entity without fetching it first by using parametrized bulk queries:
DELETE FROM Schemes s WHERE s.id = :id
Be aware that bulk queries can cause problems on their own. First, they bypass the persistence context, meaning that whatever you do with a bulk query will not be reflected by the objects in the persistence context. This is less an issue for delete queries than for update queries. Secondly, if you have defined any cascading rules on your entites - they will be ignored by a bulk query.

Initializing a transient attribute of a JPA entity during CriteriaQuery

I'm wondering if it is possible to initialize a transient attribute of an entity during a criteria query.
Example
#Entity
public SampleEntity{
#Id
private long id;
[more attributes]
#Transient
private String someTransientString;
[getters and setters]
}
Now I want to compose a CriteriaQuery that loads all SampleEntitys and automatically sets someTransientString to imamightlyfinestring. I have something like the following SQL in mind:
SELECT ID AS ID, [..], 'imamightilyfinestring' AS SOME_TRANSIENT_STRING FROM SAMPLE_ENTITY
I of course know that I can simply iterate the resulting collection and manually set the attribute, but I'm wondering if there is a way to do it within JPA2.
Thanks :)

No, you cannot do it in query.
If you can figure out value for someTransientString outside of query, you can use PostLoad callback (excerpt from JPA 2.0 specification):
The PostLoad method for an entity is invoked after the entity has been
loaded into the current persistence context from the database or after
the refresh operation has been applied to it. The PostLoad method is
invoked before a query result is returned or accessed or before an
association is traversed.
Just add following to your entity:
#PostLoad
protected void initSomeTransientString() {
//Likely some more complex logic to figure out value,
//if it is this simple, just set it directly in field declaration.
someTransientString = "imamightilyfinestring";
}

You can also initialize your transients in the default (i.e., no argument) constructor.
You can see that this is the strategy used, for example, in EclipseLink (read last comment in the following link):
https://bugs.eclipse.org/bugs/show_bug.cgi?id=292385

Efficiently determining the IDs of entities referenced via OneToMany relationship

Let's say I have a Hibernate entity that declares a OneToMany relationship to a different entity:
#Entity
public class SomeEntity {
#OneToMany(fetch = FetchType.LAZY)
private List<OtherEntity> otherEntities = new LinkedList<OtherEntity>();
[...]
}
When mapping SomeEntity to the corresponding DTO, all I need are the IDs that identify OtherEntity as primary key (i.e., I am not actually interested in OtherEntity instances).
Does Hibernate support this pattern, i.e., only retrieving the IDs of entities referenced via a OneToMany relationship?
I cannot influence how SomeEntity is retrieved (i.e., I have an existing SomeEntity instance retrieved within te scope of the current Hibernate session), but let's assume that lazy loading has not yet taken place, so just retrieving the child objects' IDs (rather than the complete objects) would actually yield a performance benefit.

Well, if you only need the entities' ids and you want to be economical about it, when you get those entities from the database you should state in your query that you only want to get the ids of each entry, using projections, something like :
SELECT Entity.id as entity FROM Entity WHERE ...
This will return an array of objects of the same type as Entity's id field type.

You can try obtaining the primary key without accessing the entity itself (without otherEntities.get(0).getId()). To do this you can use the PersistenceUnitUtil class:
PersistenceUnitUtil#getIdentifier(yourEntity)
The PersistenceUnitUtil can be obtained from the EntityManagerFactory. So it could be something like:
EntityManager em = ...
PersistenceUnitUtil = em.getEntityManagerFactory().getPersistenceUnitUtil();
Unfortunately, I'm not aware if this will prevent the entity loading from occuring. However, just accessing the otherEntities collection or even obtaining references to each entity will not make the instance to be loaded; you need to invoke a method on the fetched entity in order to be sure it will be loaded.
You also might consider creating a #NamedQuery and return only the OtherEntity ID's.
HTH!

From hibernate reference, section 2.2.2.1.
http://docs.jboss.org/hibernate/annotations/3.5/reference/en/html/entity.html#entity-mapping-property
Declare your columns as lazy initialized:
#Basic(fetch = FetchType.LAZY)
private String getYourProperty() {
}
You also need to disable proxies for your entity class and byte instrument it. There is an example here:
Making a OneToOne-relation lazy

You can use the below HQL as told in the documentation to establish this.
session.createQuery(select new OtherEntity(oe.id) OtherEntity oe
where oe.parentSomeEntity.someId = :someId).list();//also set someId.
Add a constructor in OtherEntity to set the id also there should be a mapping to SomeEntity in OtherEntity.
This HQL will give you a List<OtherEntity> with only id set in the bean.