Context of problem I want to solve: I have a java spring http interceptor AuditHttpCommunicationInterceptor that audits communication with an external system. The HttpClieant that does the communication is used in a java service class that does some business logic called DoBusinessLogicSevice.
The DoBusinessLogicSevice opens a new transaction and using couple of collaborators does loads of stuff.
Problem to solove: Regardless of the outcome of any of the operations in DoBusinessLogicSevice (unexpected Exceptions, etc) I want audits to be stored in the database by AuditHttpCommunicationInterceptor.
Solution I used: The AuditHttpCommunicationInterceptor will open a new transaction this way:
TransactionDefinition transactionDefinition = new DefaultTransactionDefinition(TransactionDefinition.PROPAGATION_REQUIRES_NEW);
new TransactionTemplate(platformTransactionManager, transactionDefinition).execute(new TransactionCallbackWithoutResult() {
#Override
protected void doInTransactionWithoutResult(TransactionStatus status) {
// do stuff
}
});
Everything works fine. When a part of DoBusinessLogicSevice throws unexpected exception its transaction is rolled back, but the AuditHttpCommunicationInterceptor manages to store the audit in the database.
Problem that arises from this solution: AuditHttpCommunicationInterceptor uses a new db connection. So for every DoBusinessLogicSevice call I need 2 db connections.
Basicly, I want to know the solution to the problem: how to make TransactionTemplate "suspend" the current transaction and reuse the connection for a new one in this case.
Any ideas? :)
P.S.
One idea might be to take a different design approach: drop the interceptor and create an AuditingHttpClient that is used in DoBusinessLogicSevice directly (not invoked by spring) but I cannot do that because I cannot access all http fields in there.
Spring supports nested transactions (propagation="NESTED"), but this really depends on the database platform, and I don't believe every database platform is capable of handling nested transactions.
I really don't see what's a big deal with taking connection from a pool, doing a quick audit transaction and returning connection back.
Update: While Spring supports nested transactions, it looks like Hibernate doesn't. If that's the case, I say: go with another connection for audit.
Related
Having a circuit breaker with Spring Boot for external calls (e.g. HTTP) is a common pattern that is rather easy to put into place, for example with resilience4j.
I cannot find any information about doing the same with database calls, via resilience4j or some other common pattern, and this is unexpected to me.
Suppose we have a service with a simple JDBC connection to a traditional SQL database. If the database goes down for any reason, I would like to be able to stop all incoming requests to the service at the controller level until the connection is restored. Is there a way to achieve what is essentially circuit breaker functionality for all the transactions happening over the connection to the database?
Yes, this is possible. Chapter 7 of the book Spring Microservices in Action gives an example of this. Effectively, you treat a SQL call in the same way as an HTTP call:
#CircuitBreaker(name = "entityService") ❶
public List<Entity> getEntity(String entityId) {
return entityRepository.findByEntityId(entityId);
}
You will need the following dependencies:
io.github.resilience4j:resilience4j-spring-boot2
io.github.resilience4j:resilience4j-circuitbreaker
io.github.resilience4j:resilience4j-timelimiter
org.springframework.boot:spring-boot-starter-aop
This is using an AOP pattern of adding new behavior to existing code without modifying the code itself.
I'm having a problem where I need to perform several slow HTTP requests on a separate thread after having written to the database using a JpaRepository. The problem is that doActualJob() blocks while waiting for a series of futures to resolve. This seems to prevent the underlying Hibernate session from closing, causing the application to run out of connections shortly after.
How do I write this function so the database connection isn't kept open while doing the blocking I/O? Is it even possible using JpaRepositories, or do I need to use a lower level API like EntityManager/SessionFactory?
#Service
class SomeJobRunner {
private final SomeJobRepository mSomeJobRepository; //extends JpaRepository
#AutoWired
public SomeJobRunner(final SomeJobRepository someJobRepository) {
mSomeJobRepository = someJobRepository;
}
#Async
public void doSlowJob(final long someJobId) {
SomeJob someJob = mSomeJobRepository.findOne(someJobId);
someJob.setJobStarted(Instant.now());
mSomeJobRepository.saveAndFlush(someJob);
doActualjob(); // Synchronous job doing several requests using Unirest in series
someJob = mSomeJobRepository.findOne(someJobId);
someJob.setJobEnded(Instant.now());
mSomeJobRepository.saveAndFlush(someJob);
}
Well - non-blocking database IO is not possible in Java/JDBC world in a standard way .To put it simply - your Spring data repository would be eventually using JPA ORM Implementation ( likes of Hibernate) which in turn will use JDBC to interact with the database which is essentially blocking in nature. There is work being done on this currently by Oracle (Asynchronous Database Access API ) to provide a similar API as JDBC but non-blocking. They intend to propose this as a standard. Also there is an exciting and parallel effort by Spring guys on this namely R2DBC – Reactive Relational Database Connectivity. They have actually integrated this with Spring data as well (link) so that may help you integrate in your solution. A good tutorial by Spring on this can be found here.
EDIT: As of 2022 Hibernate has reactive option as well
I would suggest to write in the database using a separate JTA transaction. Do do so, define a methode like
#Transactional(Transactional.TxType.REQUIRES_NEW)
public void saveJobStart(final long someJobId) {
SomeJob someJob = mSomeJobRepository.findOne(someJobId);
someJob.setJobStarted(Instant.now());
mSomeJobRepository.saveAndFlush(someJob);
}
Of course it is not quite the same. If doActualjob() fails, in your case, the database won't persist the start date. In my proposal, it will persist it. To compensate, you need to remove the start date in a catch bloc in doSlowJob, within a new transaction, and then rethrow the exception.
Summary (details below):
I'd like to make a stored proc call before any entities are saved/updated/deleted using a Spring/JPA stack.
Boring details:
We have an Oracle/JPA(Hibernate)/Spring MVC (with Spring Data repos) application that is set up to use triggers to record history of some tables into a set of history tables (one history table per table we want audited). Each of these entities has a modifiedByUser being set via a class that extends EmptyInterceptor on update or insert. When the trigger archives any insert or update, it can easily see who made the change using this column (we're interested in which application user, not database user). The problem is that for deletes, we won't get the last modified information from the SQL that is executed because it's just a plain delete from x where y.
To solve this, we'd like to execute a stored procedure to tell the database which app user is logged in before executing any operation. The audit trigger would then look at this value when a delete happens and use it to record who executed the delete.
Is there any way to intercept the begin transaction or some other way to execute SQL or a stored procedure to tell the db what user is executing the inserts/updates/deletes that are about to happen in the transaction before the rest of the operations happen?
I'm light on details about how the database side will work but can get more if necessary. The gist is that the stored proc will create a context that will hold session variables and the trigger will query that context on delete to get the user ID.
From the database end, there is some discussion on this here:
https://docs.oracle.com/cd/B19306_01/network.102/b14266/apdvprxy.htm#i1010372
Many applications use session pooling to set up a number of sessions
to be reused by multiple application users. Users authenticate
themselves to a middle-tier application, which uses a single identity
to log in to the database and maintains all the user connections. In
this model, application users are users who are authenticated to the
middle tier of an application, but who are not known to the
database.....in these situations, the application typically connects
as a single database user and all actions are taken as that user.
Because all user sessions are created as the same user, this security
model makes it very difficult to achieve data separation for each
user. These applications can use the CLIENT_IDENTIFIER attribute to
preserve the real application user identity through to the database.
From the Spring/JPA side of things see section 8.2 at the below:
http://docs.spring.io/spring-data/jdbc/docs/current/reference/html/orcl.connection.html
There are times when you want to prepare the database connection in
certain ways that aren't easily supported using standard connection
properties. One example would be to set certain session properties in
the SYS_CONTEXT like MODULE or CLIENT_IDENTIFIER. This chapter
explains how to use a ConnectionPreparer to accomplish this. The
example will set the CLIENT_IDENTIFIER.
The example given in the Spring docs uses XML config. If you are using Java config then it looks like:
#Component
#Aspect
public class ClientIdentifierConnectionPreparer implements ConnectionPreparer
{
#AfterReturning(pointcut = "execution(* *.getConnection(..))", returning = "connection")
public Connection prepare(Connection connection) throws SQLException
{
String webAppUser = //from Spring Security Context or wherever;
CallableStatement cs = connection.prepareCall(
"{ call DBMS_SESSION.SET_IDENTIFIER(?) }");
cs.setString(1, webAppUser);
cs.execute();
cs.close();
return connection;
}
}
Enable AspectJ via a Configuration class:
#Configuration
#EnableAspectJAutoProxy
public class SomeConfigurationClass
{
}
Note that while this is hidden away in a section specific to Spring's Oracle extensions it seems to me that there is nothing in section 8.2 (unlike 8.1) that is Oracle specific (other than the Statement executed) and the general approach should be feasible with any Database simply by specifying the relevant procedure call or SQL:
Postgres for example as the following so I don't see why anyone using Postgres couldn't use this approach with the below:
https://www.postgresql.org/docs/8.4/static/sql-set-role.html
Unless your stored procedure does more than what you described, the cleaner solution is to use Envers (Entity Versioning). Hibernate can automatically store the versions of an entity in a separate table and keep track of all the CRUD operations for you, and you don't have to worry about failed transactions since this will all happen within the same session.
As for keeping track who made the change, add a new colulmn (updatedBy) and just get the login ID of the user from Security Principal (e.g. Spring Security User)
Also check out #CreationTimestamp and #UpdateTimestamp.
I think what you are looking for is a TransactionalEvent:
#Service
public class TransactionalListenerService{
#Autowired
SessionFactory sessionFactory;
#TransactionalEventListener(phase = TransactionPhase.BEFORE_COMMIT)
public void handleEntityCreationEvent(CreationEvent<Entity> creationEvent) {
// use sessionFactory to run a stored procedure
}
}
Registering a regular event listener is done via the #EventListener
annotation. If you need to bind it to the transaction use
#TransactionalEventListener. When you do so, the listener will be
bound to the commit phase of the transaction by default.
Then in your transactional services you register the event where necessary:
#Service
public class MyTransactionalService{
#Autowired
private ApplicationEventPublisher applicationEventPublisher;
#Transactional
public void insertEntityMethod(Entity entity){
// insert
// Publish event after insert operation
applicationEventPublisher.publishEvent(new CreationEvent(this, entity));
// more processing
}
}
This can work also outside the boundaries of a trasaction if you have the requirement:
If no transaction is running, the listener is not invoked at all since
we can’t honor the required semantics. It is however possible to
override that behaviour by setting the fallbackExecution attribute of
the annotation to true.
I've got kinda problem with JDO persistence of a list of just retrieved objects.
What I want to do is to:
Fetch list of "Orders"
Modify one property "status"
Make bulk update of "Orders"
What I've got so far is "Object with id ... is managed by a different Object Manager".
But wait, I haven't faced such a problem without Spring!
I tried to debug it like this:
List<Orderr> orders = orderDao.findByIdAll(ordersKeys);
for(Orderr o : orders) {
System.out.println(JDOHelper.getPersistenceManager(o).hashCode());
//hashcode is 1524670
o.setSomething(somevalue);
}
orderDao.makePresistentAll(orders); //hashcode inside is 31778523
makePersistentAll does nothing but:
try {
System.out.println(getPersistenceManager().hashCode());
getPersistenceManager().makePersistentAll(entities);
} finally {
getPersistenceManager().close();
}
All my DAOs extend JdoDaoSupport. Pmf is injected and managed by spring.
Finally, here is the question: Why is the persistence manager closed after findByIdAll? Or why do I get new persistence manager instance? My findByIdAll method doesn't call close on persistence manager, of course.
Of course if I call makePersistent for each "order" it works well. But it breaks layering of business and database logic...
UPD
Just found out that all calls to makePersistentAll aren't working at all after migration to spring managed PersistenceManager. Before spring I used plain old PMF.get() helper and everything was shiny!
If your app remains live in response to a HTTP request for longer than 30 seconds, it will be killed. Part of the mode of operation of GAE is that your apps are not long-lived. At all.
Though you wouldn't do this on a site of your own, you'll have to get used to having only short-term access to your DB session manager. A lot of time is sometimes needed to re-open it for every transaction, but that's how GAE makes the process scalable. If you really have a lot of traffic, it can run your application in parallel on several servers.
This is kind of magic. Everytime I ask here a question I know the answer to my question within 24 hours after post.
Of course, factory by its meaning should always create a new pm instance. Now I save reference to my old pm (like I did before spring jdo daos) and everyting is ok.
I am writing some servlets with plain old mostly-JDBC patterns. I realized that I have several objects that would like to share a single transaction, and I'd like to enforce that one HTTP transaction = one database transaction.
I think I can do this via passing a Connection around in a ThreadLocal variable, and then having a servlet filter handling the creation/commit/rollback of said Connection.
Is there an existing framework that does this that I'm not privy to, or is this a reasonable late-00's way to do things?
Spring transaction management does exactly what you describe, it might be a little over whelming at first glance but all you will be needing (for the simplest case) is:
org.springframework.jdbc.datasource.DataSourceTransactionManager
org.springframework.jdbc.datasource.TransactionAwareDataSourceProxy
org.springframework.transaction.support.TransactionTemplate
Wire up your existing DataSource and wrap it in the TransctionAwareDataSourceProxy then create a DataSourceTransactionManager with the wrapped data source, keep these in your ServletContext. Then for each transaction create a TransactionTemplate passing in the transaction manager and call the execute(TransactionCallback) method to run your code. eg:
new TransactionTemplate(transactionManager).execute(new TransactionCallback(){
public void doInTransaction(TransactionStatus ts){
// run your code here...use the dataSource to get a connection and run stuff
Connection c = dataSourceProxy.getConnection();
// to rollback ... throw a RuntimeException out of this method or call
st.setRollbackOnly();
}
});
The connection will be bound to a thread local so as long as you always get the connection form the same datasource i.e. the wrapped one, you'll get the same connection in the same transaction.
Note this is the simplest possible spring transaction setup ... not nessarly the best or recommended one, for that have a look at the spring reference doc's or read spring in action.
... so I guess as a direct answer, yes it is a reasonable thing to be doing, it's what the spring framework has been doing for a long time.
Most appServer todays support JTA (Java Transaction Api): A transaction that spans over multiple open/close jdbc connections. It does the "threadLocal" stuff for you and it's J2EE compliant.
You use it like this in your filter:
public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain)
throws IOException, ServletException {
UserTransaction transaction = null;
try {
transaction = (UserTransaction)new InitialContext().lookup("java:comp/UserTransaction");
transaction.begin();
chain.doFilter(request, response);
transaction.commit();
} catch (final Exception errorInServlet) {
try {
transaction.rollback();
} catch (final Exception rollbackFailed) {
log("No ! Transaction failed !",rollbackFailed);
}
throw new ServletException(errorInServlet);
}
}
On the app-server, declare a Datasource with a jndi name, and use it in your code to retrieve a connection (do NOT make cx.commit(), cx.rollback() or cx.setAutocommit() stuff, it will interfere with JTA). You can open and close your connection several times in the same HTTP transaction, JTA will take care of it:
public void doingDatabaseStuff() throws Exception {
DataSource datasource = (DataSource)new InitialContext().lookup("/path/to/datasource");
Connection connection = datasource.getConnection();
try {
// doing stuff
} finally {
connection.close();
}
}
It is generally better to pass object with "Parameterisation from Above", the sleazing through with ThreadLocal. In the case of ServletFilter, an attribute of the ServletRequest would be an obvious place. The interface to non-servlet dependent code can extract the Connection to meaningful context.
If you cannot rely on a "real" app server and you want to avoid the not-so-lightweightness of Spring, using a filter to provide a connection, keep it on the thread and close it at the end of the request is indeed a practical and reasonable solution.
You would need some (essentially static) accessor class that allows to get() a connection and a setRollbackOnly().
Upon end of the request, from the filter's perspective, make sure to catch exceptions (upon which you should log and set to rollback only) and commit/rollback, close the transaction accordingly.
In most applications and web containers (and JTA usually makes similar assumptions) a request will be processed by exactly one thread and associating the one database connection with the thread for re-use between layers during the request is just the right thing to do.
Having a filter manage the transaction is a good approach to rolling your own transaction management.
The Java EE specification provides for transaction management, and alternative frameworks like Spring provide similar support (though that's not an endorsement; Spring doesn't necessarily do this well).
However, use of a ThreadLocal can create problems. For example, there are no guarantees that a single thread is used throughout a request, anything can access the Connection through the global variable, and testing can become more difficult if you are depending on some global state to be set up. I'd consider using a dependency injection container to explicitly pass a Connection to objects that need one.