Develop Reference

Disable DB Updates in Spring boot JPA when changing the value of a property after it is fetched from the repository using findall or other queries

I am having a service that gets the data from the database which has a column which is stored with encrypted value.
After fetching from the DAO, i will update the value of the property to decrypted value and then send it as response for the API.
I assume that the entity is having change tracking enabled for select queries also because after i get the data, the data is updated in the DB with the decrypted password. I have googled and found that the use of EntityManager solves the problem, but for this implementation I have to do a lot of code changes in many entities.
from this link, i see that we have to write custom stateless bean and inject to the code, but it looks like not right. Please suggest me the best approach to handle this problem.
My DAO:
#Repository
public interface EnvironmentDao extends JpaRepository<Environment, Long> {
//custom methods go here with native queries
}
My Service
#Override
public List<Environment> getEnvironmentsByIds(List<Long> environmentIds) throws Exception {
if (environmentIds == null || environmentIds.size() < 1) {
return null;
}
return decryptPassword(environmentDao.findAllById(environmentIds));
}
Inside the decryptPassword method, i am just looping through all the records and then setting the decrypted password like
e.setDB_Password(encryptionService.decrypt(e.getDB_Password()));
One case that i noticed yesterday is that for a similar entity on any error, there was a DB save and that time the values got updated, so after fixing the error, this change was not happening.
Please help me as I am not an expert in java and taking more time to analyze and could not understand. In the case of C#, i would use .AsNoTracking(), but i don't know java much and fiddling around.
Tried the following in the Service
#Autowired
EntityManager entityManager;
In the method,
Optional<Environment> environment = environmentDao.findById(id);
entityManager.detach(environment.get());
return managePassword(environment.get(), false);

I would suggest two options to overcome the entity being updated unintentionally:
Instead of returning the entity itself I would suggest creating a DTO class and creating an instance of that class and setting relevant properties on to the DTO instance so that no changes will be made to the entity itself. So the code will be sth like:
public List<EnvironmentDTO> getEnvironmentsByIds(List<Long> environmentIds) throws Exception {
if (environmentIds == null || environmentIds.size() < 1) {
return null;
}
return createEnvironmentDTOs(environmentDao.findAllById(environmentIds));
}
private LisT<EnvironmentDTO> createEnvironmentDTOs(List<Environment> environments) {
return environments.stream().map((env) -> {
EnvironmentDTO envDto = new EnvironmentDTO();
// Copy all relevant fields to DTO (you can even use some Mapper library for this, i.e. http://modelmapper.org/)
envDto.setDB_Password(encryptionService.decrypt(e.getDB_Password()));
})
}
If you want to return the entity no matter what instead of creating a DTO class and instance from it; you can detach the entity so that changes to the entity will not be reflected to database. So what you need to do is detaching entity after you are done with decrypting the password and setting it back to the entity: entityManager.detach(environment)

#Transactional annotation Spring boot 2.0 and hibernate LazyInitializationException

I have the following question. From what I understand the #Transactional annotation is supposed to keep the session alive, thus enabling to lazy fetch child entities without the need to performe a specific joining query.
I have the following scenario where I do not understand why I'm still getting a LazyInitializationException.
My app runs a resolver in order to provide the various controller services with a resolved object so that it can be used directly.
Said resolver intercepts a header from the request and using it's value attempts to query the db in order to fetch the object. Now the object in question is quite simple is it's doings albeit it has a list of two sub-entities.
In order to perform the resolving action I'm using an extra service where I basically wrap some JpaRepository methods. The complete is below:
#Service
public class AppClientServiceImpl implements AppClientService {
private static final Logger LOGGER = LoggerFactory.getLogger(AppClientServiceImpl.class.getCanonicalName());
private final AppClientRepository repository;
#Autowired
public AppClientServiceImpl(AppClientRepository repository) {
this.repository = repository;
}
#Override
#Transactional(readOnly = true)
public AppClient getByAppClientId(final String appClientId) {
LOGGER.debug("Attempting to retrieve appClient with id:: {}", appClientId);
return repository.findByAppClientId(appClientId);
}
#Override
#Transactional
public void saveAndFlush(final AppClient appClient) {
LOGGER.debug("Attempting to save/update appClient:: {}", appClient);
repository.saveAndFlush(appClient);
}
}
As you can see both methods are annotated as #Transactional meaning that the should keep the session alive in the context of that said method.
Now, my main questions are the following:
1) Using the debugger I'm seeing even on that level getByAppClientId the list containing on the sub-entities which is lazy loaded has been resolved just fine.
2) On the resolver itself, where the object has been received from the delegating method, the list fails to be evaluated due to a LazyInitializationException.
3) Finally on the final controller service method which is also marked as #Transactional, the same as above occurs meaning that this eventually fails to it's job (since it's performing a get of the list that has failed to initialize.
Based on all the above, I would like to know what is the best approach in handling this. For once I do not want to use an Eager fetching type and I would also like to avoid using fetch queries. Also marking my resolver as #Transactional thus keeping the session open there as well is also out of the question.
I though that since the #Transactional would keep the session open, thus enabling the final service method to obtain the list of sub-entities. This seems not to be the case.
Based on all the above it seems that I need a way for the final service method that gets call (which needs the list on hand) to fetch it somehow.
What would the best approach to handle this? I've read quite a few posts here, but I cannot make out which is the most accepted methods as of Spring boot 2.0 and hibernate 5.
Update:
Seems that annotating the sub-entitie with the following:
#Fetch(FetchMode.SELECT)
#LazyCollection(LazyCollectionOption.TRUE)
Resolves the problem but I still don't know whether this is the best approach.

You initialize the collection by debugging. The debugger usually represents collections in a special way by using the collection methods which trigger the initialization, so that might be the reason why it seems to work fine during debugging. I suppose the resolver runs outside of the scope of the getByAppClientId? At that point the session is closed which is why you see the exception.
I created Blaze-Persistence Entity Views for exactly that use case. You essentially define DTOs for JPA entities as interfaces and apply them on a query. It supports mapping nested DTOs, collection etc., essentially everything you'd expect and on top of that, it will improve your query performance as it will generate queries fetching just the data that you actually require for the DTOs.
The entity views for your example could look like this
#EntityView(AppClient.class)
interface AppClientDto {
String getName();
}
Querying could look like this
List<AppClientDto> dtos = entityViewManager.applySetting(
EntityViewSetting.create(AppClientDto.class),
criteriaBuilderFactory.create(em, AppClient.class)
).getResultList();

Wicket - Serialization of persisted and non-persisted JPA entities

I know that when using Wicket with JPA frameworks it is not advisable to serialize entities that have already been persisted to the database (because of problems with lazy fields and to save space). In such cases we are supposed to use LoadableDetachableModel. But what about the following use-case?
Suppose we want to create a new entity (say, a Contract) which will consist, among other things, of persisted entities (say, a Client which is selected from a list of clients stored in the DB). The entity under creation is a model object of some Wicket component (say, a Wizard). In the end (when we finish our wizard) we save the new entity to the DB. So my question is: what is the best generic solution to the serialization problem of such model objects? We can't use LDM because the entity is not in the DB yet but we don't want our inner entities (like Client) to be serialized wholly, too.
My idea was to implement a custom wicket serializer that checks if the object is an entity and if it is persisted. If so, store only its id, otherwise use the default serialization. Similarly, when deserializing use the stored id and get the entity from the DB or deserialize using the default mechanism. Not sure, though, how to do that in a generic way. My next thought was that if we can do it, then we do not need any LDM anymore, we can just store all our entities in simple org.apache.wicket.model.Model models and our serialization logic will take care of them, right?
Here's some code:
#Entity
Client {
String clientName;
#ManyToOne(fetch = FetchType.LAZY)
ClientGroup group;
}
#Entity
Contract {
Date date;
#ManyToOne(fetch = FetchType.LAZY)
Client client;
}
ContractWizard extends Wizard {
ContractWizard(String markupId, IModel<Contract> model) {
super(markupId);
setDefaultModel(model);
}
}
Contract contract = DAO.createEntity(Contract.class);
ContractWizard wizard = new ContractWizard("wizard", ?);
How to pass the contract? If we just say Model.of(contract) the whole contract will be serialized along with inner client (and it can be big), moreover if we access contract.client.group after deserialization we can bump into the problem: https://en.wikibooks.org/wiki/Java_Persistence/Relationships#Serialization.2C_and_Detaching
So I wonder how people go about solving such issues, I'm sure it's a fairly common problem.

I guess there are 2 approaches to your problem:
a.) Only save the stuff the user actually sees in Models. In your example that might be "contractStartDate", "contractEndDate", List of clientIds. That's the main approach if you don't want your DatabaseObjects in your view.
b.) Write your own LoadableDetachableModel and make sure you only serialize transient objects. For example like: (assuming that any negative id is not saved to the database)
public class MyLoadableDetachableModel extends LoadableDetachableModel {
private Object myObject;
private Integer id;
public MyLoadableDetachableModel(Object myObject) {
this.myObject = myObject;
this.id = myObject.getId();
}
#Override
protected Object load() {
if (id < 0) {
return myObject;
}
return myObjectDao.getMyObjectById(id);
}
#Override
protected void onDetach() {
super.onDetach();
id = myObject.getId();
if (id >= 0) {
myObject = null;
}
}
}
The downfall of this is that you'll have to make your DatabaseObjects Serializable which is not really ideal and can lead to all kind of problems. You would also need to decouple the references to other entities from the transient object by using a ListModel.
Having worked with both approaches I personally prefer the first. From my expierence the whole injecting dao objects into wicket can lead to disaster. :) I would only use this in view-only projects that aren't too big.

Most projects I know of just accept serializing referenced entities (e.g. your Clients) along with the edited entity (Contract).
Using conversations (keeping a Hibernate/JPA session open over several requests) is a nice alternative for applications with complex entity relations:
The Hibernate session and its entities is kept separate from the page and is never serialized. The component just keeps an identifier to fetch its conversation.

Prevent Dozer from triggering Hibernate lazy loading

I am using Spring transactions so the transaction is still active when POJO to DTO conversion occurs.
I would like to prevent Dozer from triggering lazy loading, so that hidden sql queries never occur : all fetching has to be done explicitly via HQL (to get the best control on performances).
Is it a good practice (I can't find it documented anywhere) ?
How to do it safely ?
I tried this before DTO conversion :
PlatformTransactionManager tm = (PlatformTransactionManager) SingletonFactoryProvider.getSingletonFactory().getSingleton("transactionManager");
tm.commit(tm.getTransaction(new DefaultTransactionDefinition()));
I don't know what happens to the transaction, but the Hibernate session doesn't get closed, and the lazy loading still occurs.
I tried this :
SessionFactory sf = (SessionFactory) SingletonFactoryProvider.getSingletonFactory().getSingleton("sessionFactory");
sf.getCurrentSession().clear();
sf.getCurrentSession().close();
And it prevents lazy loading, but is it a good practice to manipulate session directly in the application layer (which is called "facade" in my project) ? Which negative side effects should I fear ? (I've already seen that tests involving POJO -> DTO conversions could no more be launched through AbstractTransactionnalDatasource Spring test classes, because this classes try to trigger a rollback on a transaction which is no more linked to an active session).
I've also tried to set propagation to NOT_SUPPORTED or REQUIRES_NEW, but it reuse the current Hibernate session, and doesn't prevent lazy loading.

The only generic solution I have found for managing this (after looking into Custom Converters, Event Listeners & Proxy Resolvers) is by implementing a Custom Field Mapper. I found this functionality tucked away in the Dozer API (I don't believe it is documented in the User Guide).
A simple example is as follows;
public class MyCustomFieldMapper implements CustomFieldMapper
{
public boolean mapField(Object source, Object destination, Object sourceFieldValue, ClassMap classMap, FieldMap fieldMapping)
{
// Check if field is a Hibernate collection proxy
if (!(sourceFieldValue instanceof AbstractPersistentCollection)) {
// Allow dozer to map as normal
return false;
}
// Check if field is already initialized
if (((AbstractPersistentCollection) sourceFieldValue).wasInitialized()) {
// Allow dozer to map as normal
return false;
}
// Set destination to null, and tell dozer that the field is mapped
destination = null;
return true;
}
}
This will return any non-initialized PersistentSet objects as null. I do this so that when they are passed to the client I can differentiate between a NULL (non-loaded) collection and an empty collection. This allows me to define generic behaviour in the client to either use the pre-loaded set, or make another service call to retrieve the set (if required). Additionally, if you decide to eagerly load any collections within the service layer then they will be mapped as usual.
I inject the custom field mapper using spring:
<bean id="dozerMapper" class="org.dozer.DozerBeanMapper" lazy-init="false">
<property name="mappingFiles">
...
</property>
<property name="customFieldMapper" ref="dozerCustomFieldMapper" />
</bean>
<bean id="dozerCustomFieldMapper" class="my.project.MyCustomFieldMapper" />
I hope this helps anyone searching for a solution for this, as I failed to find any examples when searching the Internet.

A variation on the popular version above, makes sure to catch both PersistentBags, PersistentSets, you name it...
public class LazyLoadSensitiveMapper implements CustomFieldMapper {
public boolean mapField(Object source, Object destination, Object sourceFieldValue, ClassMap classMap, FieldMap fieldMapping) {
//if field is initialized, Dozer will continue mapping
// Check if field is derived from Persistent Collection
if (!(sourceFieldValue instanceof AbstractPersistentCollection)) {
// Allow dozer to map as normal
return false;
}
// Check if field is already initialized
if (((AbstractPersistentCollection) sourceFieldValue).wasInitialized()) {
// Allow dozer to map as normal
return false;
}
return true;
}
}

I didn't get the above to work (probably different versions). However this works fine
public class HibernateInitializedFieldMapper implements CustomFieldMapper {
public boolean mapField(Object source, Object destination, Object sourceFieldValue, ClassMap classMap, FieldMap fieldMapping) {
//if field is initialized, Dozer will continue mapping
return !Hibernate.isInitialized(sourceFieldValue));
}
}

Have you considered disabling lazy loading altogether?
It doesn't really seem to jive with the patterns you state you would like to use:
I would like to prevent Dozer from triggering lazy loading, so that hidden sql queries never occur : all fetching has to be done explicitly via HQL (to get the best control on performances).
This suggests you would never want to use lazy loading.
Dozer and the Hibernate-backed beans you pass to it are blissfully ignorant of each other; all Dozer knows is that it is accessing properties in the bean, and the Hibernate-backed bean is responding to calls to get() a lazy-loaded collection just as it would if you were accessing those properties yourself.
Any tricks to make Dozer aware of the Hibernate proxies in your beans or vice versa would, IMO, break down the layers of your app.
If you don't want any "hidden SQL queries" fired at unexpected times, simply disable lazy-loading.

Short version of this mapper will be
return sourceFieldValue instanceof AbstractPersistentCollection &&
!( (AbstractPersistentCollection) sourceFieldValue ).wasInitialized();

Using CustomFieldMapper may not be a good idea as it gonna invoke for every field of your source class,but our concern is only lazy association mapping(child object list),so we can set the null value in getter of the entity object,
public Set<childObject> getChild() {
if(Hibernate.isInitialized(child){
return childObject;
}else
return null;
}

Cyclic serialisation with Many to Many relationship with Hibernate

I have a parent (Program) pojo with a many-to-many relationship with their children (Subscriber).
The problem is when it serialises a Program, it also serialises the Program's Subscribers, which involves serialising their Programs, which involves serialising their Subscribers, until it has serialised every single Program & Subscriber in the database.
The ERD looks like: Program <-> Subscriber
This means what was a tiny 17KB block of data (json) being returned has become a 6.9MB return. Thus in turn blows out the time to serialise the data and then return it.
Why is my parent returning children returning parents returning children? How can I stop this so I only get the Subscribers for each Program? I'm assuming I've done something wrong with my annotations, perhaps? I would like to maintain a many-to-many relationship but without this deeply nested data retrieval.
(Note: I have prior tried adding as many Lazy annotations I can find just to see if that helps. It doesn't. Perhaps I'm doing that wrong too?)
Program.java
#Entity
#Table(name="programs")
public class Program extends Core implements Serializable, Cloneable {
...
#ManyToMany()
#JoinTable(name="program_subscribers",
joinColumns={#JoinColumn(name="program_uid")},
inverseJoinColumns={#JoinColumn(name="subscriber_uid")})
public Set<Subscriber> getSubscribers() { return subscribers; }
public void setSubscribers(Set<Subscriber> subscribers) { this.subscribers = subscribers; }
Subscriber.java
#Entity
#Table(name="subscribers")
public class Subscriber extends Core implements Serializable {
...
#ManyToMany(mappedBy="subscribers")
public Set<Program> getPrograms() { return programs; }
public void setPrograms(Set<Program> programs) { this.programs = programs;
}
Implementation
public Collection<Program> list() {
return new Programs.findAll();
}

You didn't mention the framework you are using for JSON serialization, so I'll assume JAXB. Anyway, the idea is to make the Subscriber.getPrograms(..) transient in some way, so that it's not serialized. Hibernate takes care of these 'loops', but others don't. So:
#XmlTransient
#ManyToMany(..)
public Set<Program> getPrograms()...
If you use another framework, it may have a different annotation/configuration for specifying transient fields. Like the transient keyword.
The other way is to customize your mapper to handle the cycle manually, but this is tedious.

1) How does "your" serialization work. I mean is it JAXB or custom serialization or smth else.
2) Almost all frameworks let you set the depth of serialization. I mean you can set for example depth in 2.
3) I advice you not to serialize object with children, mark them(childre) transient, and serialize separately.

How about using annotations? http://thinkinginsoftware.blogspot.com/2010/08/json-and-cyclical-references.html

From lombok library. Or override equals and hashcode. Use inside hashcode only unique fields (e.g. id).
#EqualsAndHashCode(callSuper = false, of = {"id"})

Both Bozho and ponkin are on the right track. I needed to stop serialising the data down the wire but the big problem is I am unable to change the pojo -> toJSON class/method where the serialisation takes place. I was also worried about investing time on the toJSON() method considering I was taking such a performance hit at the point of serialisation I wanted a fix that would occur before I had the data rather than afterwards.
Also due to the nature of the Many-to-Many Bidirectional design I had listed I was always going to have this cyclic programs/subscribers/programs/... problem.
Resolution: (for now atleast) I have removed the Subscriber.getProgram() method and created a finder method on the ProgramDAO which returns the Programs by Subscriber.
public List<Program> findBySubscriber(Subscriber subscriber) {
String hql = "select p " +
"from Program p " +
" join p.subscribers s " +
"where s = :sub"
;
Query q = getSession().createQuery(hql);
q.setEntity("sub", subscriber);
List<Program> l = q.list();
return l;
}
For any CRUD work I think I'm just going to have to loop over Programs.getSubscribers, or write more hql helper methods.