I have a problem with accessing data inside a running transaction when the data came from another (supposedly closed) transaction. I have three classes like below, with an entity (called MyEntity) which also has another entity connected via Hibernate mapping called "OtherEntity" which has lazy loading set to true. Notice how I have two transactions:
One to load a list of entities
And a new transaction for each new item
However, this fails inside the loop with "No session" even though I have an active transaction inside the method (TransactionSynchronizationManager.isActualTransactionActive is true).
I don't really understand the problem. Seems to me the object which is used by the second transaction(s) "belong" to the first one even though the first transaction was supposed to finish? Maybe its a race condition?
#Service
class ServiceA {
#Autowired
private ServiceB serviceB;
#Autowired
private ServiceC serviceC;
public void test() {
List<MyEntity> allEntities = serviceC.loadAllEntities(); //First transaction ran, getting a list of entities, but due to lazy loading we havent loaded all the data
for(MyEntity i : allEntities) {
serviceB.doOnEach(i); //On each element a new transaction should start
}
}
}
#Service
class ServiceB {
#Transactional
public void doOnEach(MyEntity entity) {
System.out.println(TransactionSynchronizationManager.isActualTransactionActive()); //true, therefore we have an active transaction here
OtherEntity other = entity.getSomeOtherEntity(); //Want to load the "lazy loaded" entity here
//"No Session" exception is thrown here
}
}
#Service
class ServiceC {
#Autowired
private MyRepository myRepository;
#Transactional
public List<MyEntity> loadAllEntities() {
return myRepository.findAll();
}
}
A solution would be to re-load the "MyEntity" instance inside the "doOnEach" method, but that seems to me like a sub-optimal solution, especially on big lists. Why would I reload all the data which is already supposed to be there?
Any help is appreciated.
Obviously the real code is a lot more complicated than this but I have to have these kind of separate transactions for business reasons, so please no "solutions" which re-write the core logic of this. I just want to understand whats going on here.
After the call to loadAllEntities() finishes the Spring proxy commits the transaction and closes the associated Hibernate Session. This means you cannot have Hibernate transparently load the non-loaded lazy associations anymore without explicitly telling it to do so.
If for some reason you really want your associated entities to be loaded lazily the two options you have is either use Hibernate.initialize(entity.getSomeOtherEntity()) in your doOnEach() method or set the spring.jpa.open-in-view property to true to have the OpenSessionInViewInterceptor do it for you.
Otherwise it's a good idea to load them together with the parent entity either via JOIN FETCH in your repository query or via an Entity Graph.
References:
https://www.baeldung.com/spring-open-session-in-view
https://www.baeldung.com/hibernate-initialize-proxy-exception
To clarify further:
Spring creates a transaction and opens a new Session (A) before entering the loadAllEntities() method and commits/closes them upon returning. When you call entity.getSomeOtherEntity() the original Session (A) that loaded entity is gone (i.e. entity is detached) but instead there's a new Session (B) which was created upon entering the doOnEach() transactional method. Obviously Session (B) doesn't know anything about entity and its relations and at the same time the Hibernate proxy of someOtherEntity inside entity references the original Session (A) and doesn't know anything about Session (B). To make the Hibernate proxy of someOtherEntity actually use the current active Session (B) you can call Hibernate.initialize().
Given the following code:
public class OrderService {
#PersistanceContext
private EntityManager entityManager;
#Transactional
public void updateOrder(long orderId, OrderDTO updatedOrder) {
Order order = entityManager.find(Order.class, orderId);
if (order != null) {
order.setName(updated.getName());
} else {
throw new EntityNotFoundException(Order.class, orderId);
}
}
}
I was asked to point out all the queries that are executed when the updateOrder method is called including transactional sentences.
My answer was 1 query, the one that retrieves the order by calling entityManager.find(Order.class, orderId) however it seems that is not correct. How is that even possible? I do see the setName method is called on the order but there is not a call to save that order back to the database.
Is there any documentation that explains how this works or any way to see all the sentences executed in that transaction?
When you call find() method,your object becames in persistent state. Hibernate will detect any changes made to an object in persistent state and synchronize the state with the database when the unit of work completes. You can read about object states : https://docs.jboss.org/hibernate/orm/3.3/reference/en/html/objectstate.html
The answer is it depends, the first one for sure is entityManager.find(...) which does a select. And if it finds a record, you are setting a new name(setName(...)) for which hibernated detects the object as dirty. So that it will flush the new data to db. Hence, as a second call save(...) will be triggered. Check here
I have the following question. From what I understand the #Transactional annotation is supposed to keep the session alive, thus enabling to lazy fetch child entities without the need to performe a specific joining query.
I have the following scenario where I do not understand why I'm still getting a LazyInitializationException.
My app runs a resolver in order to provide the various controller services with a resolved object so that it can be used directly.
Said resolver intercepts a header from the request and using it's value attempts to query the db in order to fetch the object. Now the object in question is quite simple is it's doings albeit it has a list of two sub-entities.
In order to perform the resolving action I'm using an extra service where I basically wrap some JpaRepository methods. The complete is below:
#Service
public class AppClientServiceImpl implements AppClientService {
private static final Logger LOGGER = LoggerFactory.getLogger(AppClientServiceImpl.class.getCanonicalName());
private final AppClientRepository repository;
#Autowired
public AppClientServiceImpl(AppClientRepository repository) {
this.repository = repository;
}
#Override
#Transactional(readOnly = true)
public AppClient getByAppClientId(final String appClientId) {
LOGGER.debug("Attempting to retrieve appClient with id:: {}", appClientId);
return repository.findByAppClientId(appClientId);
}
#Override
#Transactional
public void saveAndFlush(final AppClient appClient) {
LOGGER.debug("Attempting to save/update appClient:: {}", appClient);
repository.saveAndFlush(appClient);
}
}
As you can see both methods are annotated as #Transactional meaning that the should keep the session alive in the context of that said method.
Now, my main questions are the following:
1) Using the debugger I'm seeing even on that level getByAppClientId the list containing on the sub-entities which is lazy loaded has been resolved just fine.
2) On the resolver itself, where the object has been received from the delegating method, the list fails to be evaluated due to a LazyInitializationException.
3) Finally on the final controller service method which is also marked as #Transactional, the same as above occurs meaning that this eventually fails to it's job (since it's performing a get of the list that has failed to initialize.
Based on all the above, I would like to know what is the best approach in handling this. For once I do not want to use an Eager fetching type and I would also like to avoid using fetch queries. Also marking my resolver as #Transactional thus keeping the session open there as well is also out of the question.
I though that since the #Transactional would keep the session open, thus enabling the final service method to obtain the list of sub-entities. This seems not to be the case.
Based on all the above it seems that I need a way for the final service method that gets call (which needs the list on hand) to fetch it somehow.
What would the best approach to handle this? I've read quite a few posts here, but I cannot make out which is the most accepted methods as of Spring boot 2.0 and hibernate 5.
Update:
Seems that annotating the sub-entitie with the following:
#Fetch(FetchMode.SELECT)
#LazyCollection(LazyCollectionOption.TRUE)
Resolves the problem but I still don't know whether this is the best approach.
You initialize the collection by debugging. The debugger usually represents collections in a special way by using the collection methods which trigger the initialization, so that might be the reason why it seems to work fine during debugging. I suppose the resolver runs outside of the scope of the getByAppClientId? At that point the session is closed which is why you see the exception.
I created Blaze-Persistence Entity Views for exactly that use case. You essentially define DTOs for JPA entities as interfaces and apply them on a query. It supports mapping nested DTOs, collection etc., essentially everything you'd expect and on top of that, it will improve your query performance as it will generate queries fetching just the data that you actually require for the DTOs.
The entity views for your example could look like this
#EntityView(AppClient.class)
interface AppClientDto {
String getName();
}
Querying could look like this
List<AppClientDto> dtos = entityViewManager.applySetting(
EntityViewSetting.create(AppClientDto.class),
criteriaBuilderFactory.create(em, AppClient.class)
).getResultList();
I am trying to understand the behavior of transaction propagation using SpringJTA - JPA - Hibernate.
Essentially I am trying to update an entity. To do so I have written a test method where I fetch an object using entity manager (em) find method ( so now this object is manged object). Update the attributes of the fetched object. And then optionally make a call to service layer(service layer propagation=required) which is calling em.merge
Now I have three variations here :
Test method has no transactional annotation. Update the attributes
of the fetched object and make no call to service layer.
1.1. Result level 1 cache doesn't gets updated and no update to DB.
Test method has no transactional annotation. Update the attributes of the fetched object. Call the service layer.
2.1. Result level 1 cache and DB gets updated.
Test method has Transnational annotation which could be any of the following. Please see the table below for Propagation value at the test method and the outcome of a service call.
(service layer propagation=required)
So to read the above table, the row 1 says if the Test method has transaction propagation= REQUIRED and if a service layer call is made then the result is update to Level 1 cache but not to the DB
Below is my test case
#Test
public void testUpdateCategory() {
//Get the object via entity manager
Category rootAChild1 = categoryService.find(TestCaseConstants.CategoryConstant.rootAChild1PK);
assertNotNull(rootAChild1);
rootAChild1.setName(TestCaseConstants.CategoryConstant.rootAChild1 + "_updated");
// OPTIONALLY call update
categoryService.update(rootAChild1);
//Get the object via entity manager. I believe this time object is fetched from L1 cache. As DB doesn't get updated but test case passes
Category rootAChild1Updated = categoryService.find(TestCaseConstants.CategoryConstant.rootAChild1PK);
assertNotNull(rootAChild1Updated);
assertEquals(TestCaseConstants.CategoryConstant.rootAChild1 + "_updated", rootAChild1Updated.getName());
List<Category> categories = rootAChild1Updated.getCategories();
assertNotNull(categories);
assertEquals(TestCaseConstants.CategoryConstant.rootAChild1_Child1,categories.get(0).getName());
}
Service Layer
#Service
public class CategoryServiceImpl implements CategoryService {
#Transactional
#Override
public void update(Category category) {
categoryDao.update(category);
}
}
DAO
#Repository
public class CategoryDaoImpl {
#Override
public void update(Category category) {
em.merge(category);
}
}
Question
Can someone please explain why does REQUIRED, REQUIRES_NEW, and NESTED doesn't lead to insertion in the DB?
And why absence of transaction annotation on Test case lead to insertion in the DB as presented in my three variations?
Thanks
The effect you're seeing for REQUIRED, NESTED, and REQUIRES_NEW is due to the fact that you're checking for updates too early
(I'm assuming here that you check for db changes at the same moment when the test method reaches the assertions, or that you roll the test method transaction back somehow after executing the test)
Simply enough, your assertions are still within the context created by the #Transactional annotation in the test method. Consequently, the implicit flush to the db has not been invoked yet.
In the other three cases, the #Transactional annotation on the test method does not start a transaction for the service method to join. As a result, the transaction only spans the execution of the service method, and the flush occurs before your assertions are tested.
Basically, I am trying to understand how to write correct (or "to correctly write"?) transactional code, when developing REST service with Jax-RS and Spring. Also, we're using JOOQ for data-access. But that shouldn't be very relevant...
Consider simple model, where we have some organisations, that have these fields: "id", "name", "code". All of which must be unique. Also there's a status field.
Organization might be removed at some point. But we don't want to remove the data altogether, because we want to save it for analytical/maintenance purposes. So we just set organization 'status' field to 'REMOVED'.
Because we don't delete the organization row from the table, we can't simply put the unique constraint on the "name" column, because, we might delete organization and then create a new one with the same name. But let's assume that codes has to be unique globally, so we DO have a unique constraint on the code column.
So with that, let's see this simple example, that creates organization, performing some checks along the way.
Resource:
#Component
#Path("/api/organizations/{organizationId: [0-9]+}")
#Consumes(MediaType.APPLICATION_JSON)
#Produces(MediaTypeEx.APPLICATION_JSON_UTF_8)
public class OrganizationResource {
#Autowired
private OrganizationService organizationService;
#Autowired
private DtoConverter dtoConverter;
#POST
public OrganizationResponse createOrganization(#Auth Person person, CreateOrganizationRequest request) {
if (organizationService.checkOrganizationWithNameExists(request.name())) {
// this throws special Exception which is intercepted and translated to response with 409 status code
throw Responses.abortConflict("organization.nameExist", ImmutableMap.of("name", request.name()));
}
if (organizationService.checkOrganizationWithCodeExists(request.code())) {
throw Responses.abortConflict("organization.codeExists", ImmutableMap.of("code", request.code()));
}
long organizationId = organizationService.create(person.user().id(), request.name(), request.code());
return dtoConverter.from(organization.findById(organizationId));
}
}
DAO service looks like that:
#Transactional(DBConstants.SOME_TRANSACTION_MANAGER)
public class OrganizationServiceImpl implements OrganizationService {
#Autowired
#Qualifier(DBConstants.SOME_DSL)
protected DSLContext context;
#Override
public long create(long userId, String name, String code) {
Organization organization = new Organization(null, userId, name, code, OrganizationStatus.ACTIVE);
OrganizationRecord organizationRecord = JooqUtil.insert(context, organization, ORGANIZATION);
return organizationRecord.getId();
}
#Override
public boolean checkOrganizationWithNameExists(String name) {
return checkOrganizationExists(Tables.ORGANIZATION.NAME, name);
}
#Override
public boolean checkOrganizationWithCodeExists(String code) {
return checkOrganizationExists(Tables.ORGANIZATION.CODE, code);
}
private boolean checkOrganizationExists(TableField<OrganizationRecord, String> checkField, String checkValue) {
return context.selectCount()
.from(Tables.ORGANIZATION)
.where(checkField.eq(checkValue))
.and(Tables.ORGANIZATION.ORGANIZATION_STATUS.ne(OrganizationStatus.REMOVED))
.fetchOne(DSL.count()) > 0;
}
}
This brings some questions:
Should I put #Transactional annotation on Resource's createOrganization method? Or should I create one more service that talks to DAO and put #Transactional annotation to it's method? Something else?
What would happen if two users concurrently send request with the same "code" field. Before first transaction is commited the checks are successfully passed, so no 409 respones will be sent. Than first transaction will be committed properly, but the second one will violate DB constraint. This will throw SQLException. How to gracefully handle that? I mean I still want to show nice error message on the client side, saying that name is already used. But I can't really parse SQLException or smth.. can I?
Similar to the previous one, but this time "name" is not unique. In this case, second transaction will not violate any constraints, which leads to having two organization with the same name, that violates our buisness constraints.
Where can I see/learn tutorials/code/etc., that you consider great examples on how to write correct/reliable REST+DB code with complicated buisness logic. Github/books/blogs, whatever. I've tried to find something like that myselft, but most examples just focus on the plumbing - add these libs to maven, use these annotations, there is your simple CRUD, the end. They don't contain any transactional considirations at all. I.e.
UPDATE:
I know about isolation level and the usual error/isolation matrix (dirty reads, etc..). The problem I have is finding some "production-ready" sample to learn from. Or a good book on a subject. I still don't really get how to handle all the errors properly.. I guess I need to retry a couple of times, if transaction failed.. and than just throw some generic error and implement client, that handles that.. But do I really have to use SERIALIZABLE mode, whenever I use range queries? Because it will affect performance greatly. But otherwise how can I garantee that transaction will fail..
Anyway I've decided that for now I need more time to learn about transactions and db management in general to tackle this problem...
Generally, without talking about transactionality, endpoint should only grab parameters from the request and call the Service. It shouldn't do business logic.
It seems your checkXXX methods are part of the business logic, because they throw errors about domains-specific conflicts. Why not put them into the Service into one method, which is by the way transactional?
//service code
public Organization createOrganization(String userId, String name, String code) {
if (this.checkOrganizationWithNameExists(request.name())) {
throw ...
}
if (this.checkOrganizationWithCodeExists(code)) {
throw ...
}
long organizationId = this.create(userId, name, code);
return dao.findById(organizationId);
}
I took as your parameters are Strings, but they can be anything. I'm not sure you want to throw Responses.abortConflict in the service layer because it seems to be a REST concept, but you can define your own exception types for it if you want.
Endpoint code should look like this, however, it might contain additional try-catch block which converts the thrown exceptions to Error responses:
//endpoint code
#POST
public OrganizationResponse createOrganization(#Auth Person person, CreateOrganizationRequest request) {
String code = request.code();
String name = request.name();
String userId = person.user().id();
return dtoConverter.from(organizationService.createOrganization(userId, name, code));
}
As for question 2 and 3, transaction isolation levels are your friends. Put isolation level high enough. I think 'repeatable read' is the suitable one in your case. Your checkXXX methods will detect if some other transaction commits entities with the same name or code and it's guaranteeed that the situations stays by the time 'create' method is executed. One more useful read regarding Spring and transaction isolation levels.
As per my understanding the best way to handle DB level transaction you must use Spring's Isolation trnsaction in effective way in the dao layer. Below is sample industry standard codde in your case...
public interface OrganizationService {
#Retryable(maxAttempts=3,value=DataAccessResourceFailureException.class,backoff=#Backoff(delay = 1000))
public boolean checkOrganizationWithNameExists(String name);
}
#Repository
#EnableRetry
public class OrganizationServiceImpl implements OrganizationService {
#Transactional(isolation = Isolation.READ_COMMITTED)
#Override
public boolean checkOrganizationWithNameExists(String name){
//your code
return true;
}
}
Please pinch me if I'm wrong in here
Separation of concern :
Jax-rs resource (endpoint) layer : just handle the request, invoke the service and wrap the potential exception in appropriate response code (just catch and wrap manually or use exception mapper).
Service / business layer : expose a transactional method for each unit of work, business error must be handled as checked exception, operational ones as unchecked (subclasses of RuntimeException).
Data access layer: just handle the data access stuff (i.e. get db context, executes query and eventually map the result).
I insist on one thing, the good place to have transaction boundaries is the place where your business methods are defined. A transaction scope must be a business unit of work.
Regarding the concurrency issue, there is 2 way to handle this kind of concurrency problem : pessimistic or optimistic locking.
Pessimistic :
Lock
do your stuff
Update
Release lock
Optimistic :
check version
do your stuff
update if version is same, fail otherwise
Pessimistic is an issue regarding scalability and performance, optimistic problem is that you sometimes end by sending an operating error to the end-user.
I would personally go with optimistic locking in your case, JOOQ support it
First off the DAO layer should not even know it's being fronted by a REST webservice. Be sure to separate responsibilities.
Keep the #Transactional on the DAO. If you are issuing only a single statement than you need to decide if you are OK with dirty reads. Basically, figure out what the lowest Isolation Level is for your application. Every method will start a new Transaction (unless called from another method that already had one started) and if any Exceptions are thrown it will rollback any calls. You can setup a custom ExceptionHandler in your Controller to handle SQLDataIntegrityExceptions (like you're "code" insert example).
Use an Aggregate Primary Key that covers (id, name, code, status) so you can have an org with the same name but one will be "CURRENT" and one will be "REMOVED"