I have a requirement to capture changes to certain data. I am looking to capture the following details:
Table Name the change occurred in
Column Changed
Previous Value
Updated Value
I suggested to our Technical lead that this can be accomplished easily with DB Triggers. I was told we do not control the DB and do not have a way of adding Triggers.
I am currently using Spring AspectJ with a custom annotation and wrapping my services to try and capture the generated SQL (I figure parsing SQL is much easier that trying to capture with Objects) that's executed after the 'save' method is called, however I have not found a way to trap the generated SQL.
I tried p6Spy and was able to view the SQL and print it to the console, but was told we cannot wrap our db drivers in our PROD environment.
Is there a Spring class I am missing to make this easier?
EDIT : We're using Spring Repositories to save the data.
EDIT 2: I'm looking into EventListeners, however I cannot seem to get them to listen to my events.
#Component
public class EventListner implements PreInserEventListener, PreUpdateEventListener {
#Override
#EventListener
public boolean onPreUpdate(PreUpdateEvent event){
// do something
return false;
}
#Override
#EventListener
public boolean onPreInsert(PreUpdateEvent event){
// do something
}
return false
}
I have break points around my Listener, however they're never reached.
This question looks like it might address my issue
If you are absolutely sure that every insert/update/delete is done through JPA without any bulk SQL then you should take a look at envers
With that you can use #Audited on the entities(all column) OR columns of entities you wish to keep history.
This will create 1 revision table with timestamp change (and other data you want like uid of the user that made the change), and 1 table per entity with the old value of the modified data.
For every data you can then retrieve the history and previous value.
one other way is to add instrument on your DB like a Change Data Capture (CDC) tools and push those event to another data repository.
On the + side everything will be detected (even native SQL run directly on the DB). Drawback, your DB need to support this kind of tools correctly. For instance tools like kafka-connect can work like you want, but some implementations are too simple (like with SAP hana for instance where the process is to do a select * from xxx).
I finally settled on using Hibernate interceptors to do the trick. Works perfectly!
public class TableInterceptor extends EmptyInterceptor {
#Override
public boolean onFlushDirty(Object entity, Serializable id,
Object[] currentState, Object[] previousState,
String[] propertyNames, Type[] types){
// do comparison logic on fields and save to DB
}
}
I bootstrapped the interceptor injecting it into my LocalContainerEntityManagerFactoryBean
Related
I'm creating a website for a school project which uses spring for the backend. I'm trying to insert data into the database when new data is saved to a specific table.
I've tried using #HandleAfterCreate and #PrePersist, but neither worked. I'm not very experienced with spring. The teacher told us to use it and now I don't know what do.
#HandleAfterCreate
public void handlePersonBeforeCreate(Person person){
logger.info("Inside Person Before Create....");
Set<Qualifikation> qualifikationen = new HashSet<>();
kompetenzRepository.findAll().forEach(kompetenz -> {
Qualifikation qualifikation = new Qualifikation();
qualifikation.setAusmass(0);
qualifikation.setKompetenz(kompetenz);
qualifikation.setPerson(person);
});
person.setQualifikationen(qualifikationen);
System.out.println(person.getDisplayName());
}
The code should set a person's "Qualifikation" to a default value when the person is inserted (via OAuth login). It should have every "Kompetenz" with a value of 0 by default. Kompetenz has a 1 to n relation to Qualifikation. If you need more information please ask me.
It looks like you're trying to have access to the repository layer of your application inside an entity. This is generally not a good idea, as the entities should only know about the data they hold, not the other application components.
In this particular case it would be wise to use a #Service class with a method that you can call to insert the data into the database. In the method you could then insert any other entities as well. Let your repositories be fields of the service and make them #Autowired.
I think you need to enable JPA auditing . It can be enabled in Spring by add #EnableJpaAuditing to your persistence configuration. This tells Spring to listen JPA entity lifecycle events and call the annotated methods in appropriate places.
Also I think you should make the callback method private if it is meant to be called only when persisted (#PrePersist).
See details here. In this article is also presented entity listeners which might also be a good solution when dealing with multiple entities having a need for same pre-persist functionality.
I think you should create a service class, a repository class and an entity which will be stored through repository. The logic of getting all inner elements and filling it with default value is to be written in service and not a good idea to write in entity class.
If you need any help regarding it, let me know .
Welcome to community!!
I'm refactoring a code base to get rid of SQL statements and primitive access and modernize with Spring Data JPA (backed by hibernate). I do use QueryDSL in the project for other uses.
I have a scenario where the user can "mass update" a ton of records, and select some values that they want to update. In the old way, the code manually built the update statement with an IN statement for the where for the PK (which items to update), and also manually built the SET clauses (where the options in SET clauses can vary depending on what the user wants to update).
In looking at QueryDSL documentation, it shows that it supports what I want to do. http://www.querydsl.com/static/querydsl/4.1.2/reference/html_single/#d0e399
I tried looking for a way to do this with Spring Data JPA, and haven't had any luck. Is there a repostitory interface I'm missing, or another library that is required....or would I need to autowire a queryFactory into a custom repository implementation and very literally implement the code in the QueryDSL example?
You can either write a custom method or use #Query annotation.
For custom method;
public interface RecordRepository extends RecordRepositoryCustom,
CrudRepository<Record, Long>
{
}
public interface RecordRepositoryCustom {
// Custom method
void massUpdateRecords(long... ids);
}
public class RecordRepositoryImpl implements RecordRepositoryCustom {
#Override
public void massUpdateRecords(long... ids) {
//implement using em or querydsl
}
}
For #Query annotation;
public interface RecordRepository extends CrudRepository<Record, Long>
{
#Query("update records set someColumn=someValue where id in :ids")
void massUpdateRecords(#Param("ids") long... ids);
}
There is also #NamedQuery option if you want your model class to be reusable with custom methods;
#Entity
#NamedQuery(name = "Record.massUpdateRecords", query = "update records set someColumn=someValue where id in :ids")
#Table(name = "records")
public class Record {
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
private Long id;
//rest of the entity...
}
public interface RecordRepository extends CrudRepository<Record, Long>
{
//this will use the namedquery
void massUpdateRecords(#Param("ids") long... ids);
}
Check repositories.custom-implementations, jpa.query-methods.at-query and jpa.query-methods.named-queries at spring data reference document for more info.
This question is quite interesting for me because I was solving this very problem in my current project with the same technology stack mentioned in your question. Particularly we were interested in the second part of your question:
where the options in SET clauses can vary depending on what the user
wants to update
I do understand this is the answer you probably do not want to get but we did not find anything out there :( Spring data is quite cumbersome for update operations especially when it comes to their flexibility.
After I saw your question I tried to look up something new for spring and QueryDSL integration (you know, maybe something was released during past months) but nothing was released.
The only thing that brought me quite close is .flush in entity manager meaning you could follow the following scenario:
Get ids of entities you want to update
Retrieve all entities by these ids (first actual query to db)
Modify them in any way you want
Call entityManager.flush resulting N separate updates to database.
This approach results N+1 actual queries to database where N = number of ids needed to be updated. Moreover you are moving the data back and forth which is actually not good too.
I would advise to
autowire a queryFactory into a custom repository
implementation
Also, have a look into spring data and querydsl example. However you will find only lookup examples.
Hope my pessimistic answer helps :)
With Spring JPA is there an easy way to use native queries but maintaining database independence, for example by using the query which fits best?
At the moment I do this by checking the currently set Dialect from the Environment and call the proper method of my Repository:
public Foo fetchFoo() {
if (POSTGRES_DIALECT.equals(env.getRequiredProperty("hibernate.dialect"))) {
return repo.postgresOptimizedGetFoo();
}
return repo.getFoo();
}
This works but I have the feeling that there is a better way or that I am missing something. Especially because (Spring) JPA allows it to use native queries quite easily but that breaks one of its big advantages: database independence.
As per my understanding, this can be achieved simply by using #Transactional(readOnly=false) and then instead of calling session.createQuery, one can use session.createSQLQuery, as provided in this example.
Your sql can be any of your native query.
Hope this works for you. :)
#Override
#Transactional(readOnly = false)
public Long getSeqVal() {
Session session = entityManager.unwrap(Session.class);
String sql = "SELECT nextval('seqName')";
Query query = session.createSQLQuery(sql);
BigInteger big = (BigInteger) query.list().get(0);
return big.longValue();
}
This is just an idea: I do not know whether it works or not:
My idea would be having subinterfaces, one normal Spring-Data-JPA-interface with all methods for one entiy (without native query hints). Than I would crate a subinterface for every database, that "override" the database specific native statements. (This intrface would be empty if there are no DB specific statements). Then I would try configure Spring-JPA with some profiles to load the right specific interface (for example by a class-name or package-name-pattern)
This seems like a way to complicated way to get queries to work.
If you really want to use optimized queries make it at least transparant for your code. I suggest using named queries and create an orm.xml per database (much like Spring Boot uses to load the schema.xml for a different database).
In your code you can simply do
public interface YourRepository extends JpaRepository<YourEntity, Long> {
List<YourEntity> yourQueryMethod();
}
This will look for a named query with the name YourEntity.yourQueryMethod. Now in your orm.xml add the named query (the default one and in another one the optimized one).
Then you need to configure your LocalContainerEntityManagerFactory to load the specific one needed. Assuming you have a property defining which database you use, lets name it database.type you could do something like the following
<bean class="LocalContainerEntityManagerFactoryBean">
<property name="mappingResources" value="classpath:META-INF/orm-${database.type}.xml" />
... other config ...
</bean>
This way you can keep your code clean of the if/then/else construct and apply where needed. Cleans your code nicely imho.
I have a PostgreSQL 8.4 database with some tables and views which are essentially joins on some of the tables. I used NetBeans 7.2 (as described here) to create REST based services derived from those views and tables and deployed those to a Glassfish 3.1.2.2 server.
There is another process which asynchronously updates contents in some of tables used to build the views. I can directly query the views and tables and see these changes have occured correctly. However, when pulled from the REST based services, the values are not the same as those in the database. I am assuming this is because JPA has cached local copies of the database contents on the Glassfish server and JPA needs to refresh the associated entities.
I have tried adding a couple of methods to the AbstractFacade class NetBeans generates:
public abstract class AbstractFacade<T> {
private Class<T> entityClass;
private String entityName;
private static boolean _refresh = true;
public static void refresh() { _refresh = true; }
public AbstractFacade(Class<T> entityClass) {
this.entityClass = entityClass;
this.entityName = entityClass.getSimpleName();
}
private void doRefresh() {
if (_refresh) {
EntityManager em = getEntityManager();
em.flush();
for (EntityType<?> entity : em.getMetamodel().getEntities()) {
if (entity.getName().contains(entityName)) {
try {
em.refresh(entity);
// log success
}
catch (IllegalArgumentException e) {
// log failure ... typically complains entity is not managed
}
}
}
_refresh = false;
}
}
...
}
I then call doRefresh() from each of the find methods NetBeans generates. What normally happens is the IllegalArgumentsException is thrown stating somethng like Can not refresh not managed object: EntityTypeImpl#28524907:MyView [ javaType: class org.my.rest.MyView descriptor: RelationalDescriptor(org.my.rest.MyView --> [DatabaseTable(my_view)]), mappings: 12].
So I'm looking for some suggestions on how to correctly refresh the entities associated with the views so it is up to date.
UPDATE: Turns out my understanding of the underlying problem was not correct. It is somewhat related to another question I posted earlier, namely the view had no single field which could be used as a unique identifier. NetBeans required I select an ID field, so I just chose one part of what should have been a multi-part key. This exhibited the behavior that all records with a particular ID field were identical, even though the database had records with the same ID field but the rest of it was different. JPA didn't go any further than looking at what I told it was the unique identifier and simply pulled the first record it found.
I resolved this by adding a unique identifier field (never was able to get the multipart key to work properly).
I recommend adding an #Startup #Singleton class that establishes a JDBC connection to the PostgreSQL database and uses LISTEN and NOTIFY to handle cache invalidation.
Update: Here's another interesting approach, using pgq and a collection of workers for invalidation.
Invalidation signalling
Add a trigger on the table that's being updated that sends a NOTIFY whenever an entity is updated. On PostgreSQL 9.0 and above this NOTIFY can contain a payload, usually a row ID, so you don't have to invalidate your entire cache, just the entity that has changed. On older versions where a payload isn't supported you can either add the invalidated entries to a timestamped log table that your helper class queries when it gets a NOTIFY, or just invalidate the whole cache.
Your helper class now LISTENs on the NOTIFY events the trigger sends. When it gets a NOTIFY event, it can invalidate individual cache entries (see below), or flush the entire cache. You can listen for notifications from the database with PgJDBC's listen/notify support. You will need to unwrap any connection pooler managed java.sql.Connection to get to the underlying PostgreSQL implementation so you can cast it to org.postgresql.PGConnection and call getNotifications() on it.
An an alternative to LISTEN and NOTIFY, you could poll a change log table on a timer, and have a trigger on the problem table append changed row IDs and change timestamps to the change log table. This approach will be portable except for the need for a different trigger for each DB type, but it's inefficient and less timely. It'll require frequent inefficient polling, and still have a time delay that the listen/notify approach does not. In PostgreSQL you can use an UNLOGGED table to reduce the costs of this approach a little bit.
Cache levels
EclipseLink/JPA has a couple of levels of caching.
The 1st level cache is at the EntityManager level. If an entity is attached to an EntityManager by persist(...), merge(...), find(...), etc, then the EntityManager is required to return the same instance of that entity when it is accessed again within the same session, whether or not your application still has references to it. This attached instance won't be up-to-date if your database contents have since changed.
The 2nd level cache, which is optional, is at the EntityManagerFactory level and is a more traditional cache. It isn't clear whether you have the 2nd level cache enabled. Check your EclipseLink logs and your persistence.xml. You can get access to the 2nd level cache with EntityManagerFactory.getCache(); see Cache.
#thedayofcondor showed how to flush the 2nd level cache with:
em.getEntityManagerFactory().getCache().evictAll();
but you can also evict individual objects with the evict(java.lang.Class cls, java.lang.Object primaryKey) call:
em.getEntityManagerFactory().getCache().evict(theClass, thePrimaryKey);
which you can use from your #Startup #Singleton NOTIFY listener to invalidate only those entries that have changed.
The 1st level cache isn't so easy, because it's part of your application logic. You'll want to learn about how the EntityManager, attached and detached entities, etc work. One option is to always use detached entities for the table in question, where you use a new EntityManager whenever you fetch the entity. This question:
Invalidating JPA EntityManager session
has a useful discussion of handling invalidation of the entity manager's cache. However, it's unlikely that an EntityManager cache is your problem, because a RESTful web service is usually implemented using short EntityManager sessions. This is only likely to be an issue if you're using extended persistence contexts, or if you're creating and managing your own EntityManager sessions rather than using container-managed persistence.
You can either disable caching entirely (see: http://wiki.eclipse.org/EclipseLink/FAQ/How_to_disable_the_shared_cache%3F ) but be preparedto a fairly large performance loss.
Otherwise, you can perform a clear cache programmatically with
em.getEntityManagerFactory().getCache().evictAll();
You can map it to a servlet so you can call it externally - this is better if your database is modify externally very seldom and you just want to be sure JPS will pick up the new version
Just a thought, but how do you receive your EntityManager/Session/whatever?
If you queried the entity in one session, it will be detached in the next one and you will have to merge it back into the persistence context to get it managed again.
Trying to work with detached entities may result in those not-managed exceptions, you should re-query the entity or you could try it with merge (or similar methods).
JPA doesn't do any caching by default. You have to explicitly configure it. I believe its the side effect of the architectural style you have chosen: REST. I think caching is happening at the web servers, proxy servers etc. I suggest you read this and debug more.
I am working in a spring,hibernate project and database is oracle. I have DAO layer for persistence related operations.
In all my tables, I have create_date and update_date columns representing the timestamp when a row is inserted and updated in the tables respectively.
There is a requirement that I have to update the above two mentioned timestamp columns of that particular table for which the request is meant to whenever any insert/update operation happens.For example, If my DAO layer has two methods, say m1 and m2 responsible for impacting t1 and t2 tables respectively. Now, if m1 method is invoked, then timestamp columns of t1 table will be updatedi.e. For insert, create_date column will be updated and for any update update_date column will be updated.
I have idea of Spring AOP so I was thinking to use AOP to implement the above requirement, though, i am not quite sure if it can be achieved using AOP.
Please let me know if I can use AOP to fulfill this requirement. And if it is possible, then please provide me the inputs how to implement it.
I have implemented update date feature for one of the modules in my application using Spring AOP.
PFB code for your reference
Hope this will help.
I wonder if one can have pointcuts for variable as well.I know its might not possible with spring's aspect j implementation.But any work around guys :P
**
* #author Vikas.Chowdhury
* #version $Revision$ Last changed by $Author$ on $Date$ as $Revision$
*/
#Aspect
#Component
public class UpdateDateAspect
{
#Autowired
private ISurveyService surveyService;
Integer surveyId = null;
Logger gtLogger = Logger.getLogger(this.getClass().getName());
#Pointcut("execution(* com.xyz.service.impl.*.saveSurvey*(..)))")
public void updateDate()
{
}
#Around("updateDate()")
public Object myAspect(final ProceedingJoinPoint pjp)
{
// retrieve the runtime method arguments (dynamic)
Object returnVal = null;
for (final Object argument : pjp.getArgs())
{
if (argument instanceof SurveyHelper)
{
SurveyHelper surveyHelper = (SurveyHelper) argument;
surveyId = surveyHelper.getSurveyId();
}
}
try
{
returnVal = pjp.proceed();
}
catch (Throwable e)
{
gtLogger.debug("Unable to use JointPoint :(");
}
return returnVal;
}
#After("updateDate()")
public void updateSurveyDateBySurveyId() throws Exception
{
if (surveyId != null)
{
surveyService.updateSurveyDateBySurveyId(surveyId);
}
}
}
I'd use an Hibernate interceptor instead, that's what they are for. For example, the entities that need such fields could implement the following interface:
public interface Auditable {
Date getCreated();
void setCreated(Date created);
Date getModified();
void setModified(Date modified);
}
Then the interceptor always sets the modified field on save, and only sets the created field when it's not already set.
Even though you have been asking for a Spring AOP solution to your question, I would like to point out that the same result can be achieved using database triggers, e. g. automatically setting the created timestamp during INSERT operations and the modified timestamp during UPDATE statements.
This may be a good solution, especially if not all your DB calls are going through the AOP-captured logic (e. g. when bypassing your pointcut definition because a method does not fit the pattern or even bypassing the code completely using a standalone SQL client), so that you could enforce the modified timestamp even when somebody updates the entries from a different application.
It would have the drawback that you need to define the triggers on all affected tables, though.
It should be possible with Spring AOP using a #Before advice. If you pass an entity to a create method have an advice set the create_date and for an update method the update_date. You may want to consider the following to make your job easier:
Have all entities implement a common interface to set create_date and update_date. This allows you to have a common advice without having to resort to reflection.
Have a naming convention to identify create and update methods on our DAOs. This will make your point cuts simpler.