I am having a service that gets the data from the database which has a column which is stored with encrypted value.
After fetching from the DAO, i will update the value of the property to decrypted value and then send it as response for the API.
I assume that the entity is having change tracking enabled for select queries also because after i get the data, the data is updated in the DB with the decrypted password. I have googled and found that the use of EntityManager solves the problem, but for this implementation I have to do a lot of code changes in many entities.
from this link, i see that we have to write custom stateless bean and inject to the code, but it looks like not right. Please suggest me the best approach to handle this problem.
My DAO:
#Repository
public interface EnvironmentDao extends JpaRepository<Environment, Long> {
//custom methods go here with native queries
}
My Service
#Override
public List<Environment> getEnvironmentsByIds(List<Long> environmentIds) throws Exception {
if (environmentIds == null || environmentIds.size() < 1) {
return null;
}
return decryptPassword(environmentDao.findAllById(environmentIds));
}
Inside the decryptPassword method, i am just looping through all the records and then setting the decrypted password like
e.setDB_Password(encryptionService.decrypt(e.getDB_Password()));
One case that i noticed yesterday is that for a similar entity on any error, there was a DB save and that time the values got updated, so after fixing the error, this change was not happening.
Please help me as I am not an expert in java and taking more time to analyze and could not understand. In the case of C#, i would use .AsNoTracking(), but i don't know java much and fiddling around.
Tried the following in the Service
#Autowired
EntityManager entityManager;
In the method,
Optional<Environment> environment = environmentDao.findById(id);
entityManager.detach(environment.get());
return managePassword(environment.get(), false);
I would suggest two options to overcome the entity being updated unintentionally:
Instead of returning the entity itself I would suggest creating a DTO class and creating an instance of that class and setting relevant properties on to the DTO instance so that no changes will be made to the entity itself. So the code will be sth like:
public List<EnvironmentDTO> getEnvironmentsByIds(List<Long> environmentIds) throws Exception {
if (environmentIds == null || environmentIds.size() < 1) {
return null;
}
return createEnvironmentDTOs(environmentDao.findAllById(environmentIds));
}
private LisT<EnvironmentDTO> createEnvironmentDTOs(List<Environment> environments) {
return environments.stream().map((env) -> {
EnvironmentDTO envDto = new EnvironmentDTO();
// Copy all relevant fields to DTO (you can even use some Mapper library for this, i.e. http://modelmapper.org/)
envDto.setDB_Password(encryptionService.decrypt(e.getDB_Password()));
})
}
If you want to return the entity no matter what instead of creating a DTO class and instance from it; you can detach the entity so that changes to the entity will not be reflected to database. So what you need to do is detaching entity after you are done with decrypting the password and setting it back to the entity: entityManager.detach(environment)
I'm trying to implement a method for updating a database record. So far, I created this one:
public Optional<User> update(final Integer id,final UpdateUserDto dto) {
userRepository.findById(id).ifPresent((user -> {
user.setShop((dto.getShopId() == null) ? null : shopRepository.findById(dto.getShopId())
.orElseThrow(ShopNotFoundException::new));
user.setUsername(dto.getUsername());
user.setPassword(passwordEncoder.encode(dto.getPassword()));
user.setRoles(Arrays.asList(
roleRepository.findByRoleName(dto.getRole()).orElseThrow(RoleNotFoundException::new)
));
}));
return userRepository.findById(id);
}
But now I added two more fields to my user entity (activated, productAllowed) and I must enhance my update method to make them updatable. I can do that, but I have other entities also and if I change them it will be a lot of maybe boilerplate code.
Is there any kind of best practice to do this in a better way, or I just need to keep setting all the fields manually?
I was also thinking about reflection, but in that case I have a few fields that cannot be copied exactly from the DTO to the entity (e.g. the shop field, which is queried from database, or role field).
And I also don't think that another query for returning the new object is effective, but although I set the properties in a service layer, the original findById()'s returned user is wrapped inside an Optional, so I don't think it will be updated.
Thank you in advance.
I have a requirement to capture changes to certain data. I am looking to capture the following details:
Table Name the change occurred in
Column Changed
Previous Value
Updated Value
I suggested to our Technical lead that this can be accomplished easily with DB Triggers. I was told we do not control the DB and do not have a way of adding Triggers.
I am currently using Spring AspectJ with a custom annotation and wrapping my services to try and capture the generated SQL (I figure parsing SQL is much easier that trying to capture with Objects) that's executed after the 'save' method is called, however I have not found a way to trap the generated SQL.
I tried p6Spy and was able to view the SQL and print it to the console, but was told we cannot wrap our db drivers in our PROD environment.
Is there a Spring class I am missing to make this easier?
EDIT : We're using Spring Repositories to save the data.
EDIT 2: I'm looking into EventListeners, however I cannot seem to get them to listen to my events.
#Component
public class EventListner implements PreInserEventListener, PreUpdateEventListener {
#Override
#EventListener
public boolean onPreUpdate(PreUpdateEvent event){
// do something
return false;
}
#Override
#EventListener
public boolean onPreInsert(PreUpdateEvent event){
// do something
}
return false
}
I have break points around my Listener, however they're never reached.
This question looks like it might address my issue
If you are absolutely sure that every insert/update/delete is done through JPA without any bulk SQL then you should take a look at envers
With that you can use #Audited on the entities(all column) OR columns of entities you wish to keep history.
This will create 1 revision table with timestamp change (and other data you want like uid of the user that made the change), and 1 table per entity with the old value of the modified data.
For every data you can then retrieve the history and previous value.
one other way is to add instrument on your DB like a Change Data Capture (CDC) tools and push those event to another data repository.
On the + side everything will be detected (even native SQL run directly on the DB). Drawback, your DB need to support this kind of tools correctly. For instance tools like kafka-connect can work like you want, but some implementations are too simple (like with SAP hana for instance where the process is to do a select * from xxx).
I finally settled on using Hibernate interceptors to do the trick. Works perfectly!
public class TableInterceptor extends EmptyInterceptor {
#Override
public boolean onFlushDirty(Object entity, Serializable id,
Object[] currentState, Object[] previousState,
String[] propertyNames, Type[] types){
// do comparison logic on fields and save to DB
}
}
I bootstrapped the interceptor injecting it into my LocalContainerEntityManagerFactoryBean
Basically, I am trying to understand how to write correct (or "to correctly write"?) transactional code, when developing REST service with Jax-RS and Spring. Also, we're using JOOQ for data-access. But that shouldn't be very relevant...
Consider simple model, where we have some organisations, that have these fields: "id", "name", "code". All of which must be unique. Also there's a status field.
Organization might be removed at some point. But we don't want to remove the data altogether, because we want to save it for analytical/maintenance purposes. So we just set organization 'status' field to 'REMOVED'.
Because we don't delete the organization row from the table, we can't simply put the unique constraint on the "name" column, because, we might delete organization and then create a new one with the same name. But let's assume that codes has to be unique globally, so we DO have a unique constraint on the code column.
So with that, let's see this simple example, that creates organization, performing some checks along the way.
Resource:
#Component
#Path("/api/organizations/{organizationId: [0-9]+}")
#Consumes(MediaType.APPLICATION_JSON)
#Produces(MediaTypeEx.APPLICATION_JSON_UTF_8)
public class OrganizationResource {
#Autowired
private OrganizationService organizationService;
#Autowired
private DtoConverter dtoConverter;
#POST
public OrganizationResponse createOrganization(#Auth Person person, CreateOrganizationRequest request) {
if (organizationService.checkOrganizationWithNameExists(request.name())) {
// this throws special Exception which is intercepted and translated to response with 409 status code
throw Responses.abortConflict("organization.nameExist", ImmutableMap.of("name", request.name()));
}
if (organizationService.checkOrganizationWithCodeExists(request.code())) {
throw Responses.abortConflict("organization.codeExists", ImmutableMap.of("code", request.code()));
}
long organizationId = organizationService.create(person.user().id(), request.name(), request.code());
return dtoConverter.from(organization.findById(organizationId));
}
}
DAO service looks like that:
#Transactional(DBConstants.SOME_TRANSACTION_MANAGER)
public class OrganizationServiceImpl implements OrganizationService {
#Autowired
#Qualifier(DBConstants.SOME_DSL)
protected DSLContext context;
#Override
public long create(long userId, String name, String code) {
Organization organization = new Organization(null, userId, name, code, OrganizationStatus.ACTIVE);
OrganizationRecord organizationRecord = JooqUtil.insert(context, organization, ORGANIZATION);
return organizationRecord.getId();
}
#Override
public boolean checkOrganizationWithNameExists(String name) {
return checkOrganizationExists(Tables.ORGANIZATION.NAME, name);
}
#Override
public boolean checkOrganizationWithCodeExists(String code) {
return checkOrganizationExists(Tables.ORGANIZATION.CODE, code);
}
private boolean checkOrganizationExists(TableField<OrganizationRecord, String> checkField, String checkValue) {
return context.selectCount()
.from(Tables.ORGANIZATION)
.where(checkField.eq(checkValue))
.and(Tables.ORGANIZATION.ORGANIZATION_STATUS.ne(OrganizationStatus.REMOVED))
.fetchOne(DSL.count()) > 0;
}
}
This brings some questions:
Should I put #Transactional annotation on Resource's createOrganization method? Or should I create one more service that talks to DAO and put #Transactional annotation to it's method? Something else?
What would happen if two users concurrently send request with the same "code" field. Before first transaction is commited the checks are successfully passed, so no 409 respones will be sent. Than first transaction will be committed properly, but the second one will violate DB constraint. This will throw SQLException. How to gracefully handle that? I mean I still want to show nice error message on the client side, saying that name is already used. But I can't really parse SQLException or smth.. can I?
Similar to the previous one, but this time "name" is not unique. In this case, second transaction will not violate any constraints, which leads to having two organization with the same name, that violates our buisness constraints.
Where can I see/learn tutorials/code/etc., that you consider great examples on how to write correct/reliable REST+DB code with complicated buisness logic. Github/books/blogs, whatever. I've tried to find something like that myselft, but most examples just focus on the plumbing - add these libs to maven, use these annotations, there is your simple CRUD, the end. They don't contain any transactional considirations at all. I.e.
UPDATE:
I know about isolation level and the usual error/isolation matrix (dirty reads, etc..). The problem I have is finding some "production-ready" sample to learn from. Or a good book on a subject. I still don't really get how to handle all the errors properly.. I guess I need to retry a couple of times, if transaction failed.. and than just throw some generic error and implement client, that handles that.. But do I really have to use SERIALIZABLE mode, whenever I use range queries? Because it will affect performance greatly. But otherwise how can I garantee that transaction will fail..
Anyway I've decided that for now I need more time to learn about transactions and db management in general to tackle this problem...
Generally, without talking about transactionality, endpoint should only grab parameters from the request and call the Service. It shouldn't do business logic.
It seems your checkXXX methods are part of the business logic, because they throw errors about domains-specific conflicts. Why not put them into the Service into one method, which is by the way transactional?
//service code
public Organization createOrganization(String userId, String name, String code) {
if (this.checkOrganizationWithNameExists(request.name())) {
throw ...
}
if (this.checkOrganizationWithCodeExists(code)) {
throw ...
}
long organizationId = this.create(userId, name, code);
return dao.findById(organizationId);
}
I took as your parameters are Strings, but they can be anything. I'm not sure you want to throw Responses.abortConflict in the service layer because it seems to be a REST concept, but you can define your own exception types for it if you want.
Endpoint code should look like this, however, it might contain additional try-catch block which converts the thrown exceptions to Error responses:
//endpoint code
#POST
public OrganizationResponse createOrganization(#Auth Person person, CreateOrganizationRequest request) {
String code = request.code();
String name = request.name();
String userId = person.user().id();
return dtoConverter.from(organizationService.createOrganization(userId, name, code));
}
As for question 2 and 3, transaction isolation levels are your friends. Put isolation level high enough. I think 'repeatable read' is the suitable one in your case. Your checkXXX methods will detect if some other transaction commits entities with the same name or code and it's guaranteeed that the situations stays by the time 'create' method is executed. One more useful read regarding Spring and transaction isolation levels.
As per my understanding the best way to handle DB level transaction you must use Spring's Isolation trnsaction in effective way in the dao layer. Below is sample industry standard codde in your case...
public interface OrganizationService {
#Retryable(maxAttempts=3,value=DataAccessResourceFailureException.class,backoff=#Backoff(delay = 1000))
public boolean checkOrganizationWithNameExists(String name);
}
#Repository
#EnableRetry
public class OrganizationServiceImpl implements OrganizationService {
#Transactional(isolation = Isolation.READ_COMMITTED)
#Override
public boolean checkOrganizationWithNameExists(String name){
//your code
return true;
}
}
Please pinch me if I'm wrong in here
Separation of concern :
Jax-rs resource (endpoint) layer : just handle the request, invoke the service and wrap the potential exception in appropriate response code (just catch and wrap manually or use exception mapper).
Service / business layer : expose a transactional method for each unit of work, business error must be handled as checked exception, operational ones as unchecked (subclasses of RuntimeException).
Data access layer: just handle the data access stuff (i.e. get db context, executes query and eventually map the result).
I insist on one thing, the good place to have transaction boundaries is the place where your business methods are defined. A transaction scope must be a business unit of work.
Regarding the concurrency issue, there is 2 way to handle this kind of concurrency problem : pessimistic or optimistic locking.
Pessimistic :
Lock
do your stuff
Update
Release lock
Optimistic :
check version
do your stuff
update if version is same, fail otherwise
Pessimistic is an issue regarding scalability and performance, optimistic problem is that you sometimes end by sending an operating error to the end-user.
I would personally go with optimistic locking in your case, JOOQ support it
First off the DAO layer should not even know it's being fronted by a REST webservice. Be sure to separate responsibilities.
Keep the #Transactional on the DAO. If you are issuing only a single statement than you need to decide if you are OK with dirty reads. Basically, figure out what the lowest Isolation Level is for your application. Every method will start a new Transaction (unless called from another method that already had one started) and if any Exceptions are thrown it will rollback any calls. You can setup a custom ExceptionHandler in your Controller to handle SQLDataIntegrityExceptions (like you're "code" insert example).
Use an Aggregate Primary Key that covers (id, name, code, status) so you can have an org with the same name but one will be "CURRENT" and one will be "REMOVED"
In my current project almost every entity has a field recordStatus which can have 2 values:
A for Active
D for Deleted
In spring data one can normally use:
repository.findByLastName(lastName)
but with the current data model we have to remember about the active part in every repository call, eg.
repository.findByLastNameAndRecordStatus(lastName, A)
The question is: is there any way to extend spring data in such a way it would be able to recognize the following method:
repository.findActiveByLastName(lastName)
and append the
recordStatus = 'A'
automatically?
Spring Data JPA provides 2 additional options for you dealing with circumstances that their DSL can't handle by default.
The first solution is custom queries with an #Query annotation
#Query("select s from MyTable s where s.recordStatus like 'A%'")
public MyObect findActiveByLastName(String lastName);
The second solution is to add a completely custom method the "Old Fashion Way" You can create a new class setup like: MyRepositoryImpl The Impl is important as it is How spring knows to find your new method (Note: you can avoid this, but you will have to manually link things the docs can help you with that)
//Implementation
public class MyRepositoryImpl implements MyCustomMethodInterface {
#PersistenceContext
EntityManager em;
public Object myCustomJPAMethod() {
//TODO custom JPA work similar to this
String myQuery = "TODO";
return em.createQuery(myQuery).execute();
}
}
//Interface
public interface MyCustomMethodInterface {
public Object myCustomJPAMethod();
}
//For clarity update your JPA repository as well so people see your custom work
public interface MySuperEpicRepository extends JPARepository<Object, String>, MyCustomMethodInterface {
}
These are just some quick samples so feel free to go read their Spring Data JPA docs if you would like to get a bit more custom with it.
http://docs.spring.io/spring-data/jpa/docs/current/reference/html/
Finally just a quick note. Technically this isn't a built in feature from Spring Data JPA, but you can also use Predicates. I will link you to a blog on this one since I am not overly familiar on this approach.
https://spring.io/blog/2011/04/26/advanced-spring-data-jpa-specifications-and-querydsl/
You can use Spring Data's Specifications. Take a look at this article.
If you create a 'Base'-specification with the recordStatus filter, and deriving all other specifications form this one.
Of course, everybody in your team should use the specifactions api, and not the default spring data api.
I am not sure you can extend the syntax unless you override the base class (SimpleReactiveMongoRepository; this is for reactive mongo but you can find the class for your DB type), what I can suggest you is to extend the base methods and then make your method be aware of what condition you want to execute. If you check this post you get the idea that I did for the patch operation for all entities.
https://medium.com/#ghahremani/extending-default-spring-data-repository-methods-patch-example-a23c07c35bf9