I'm having a problem where I need to perform several slow HTTP requests on a separate thread after having written to the database using a JpaRepository. The problem is that doActualJob() blocks while waiting for a series of futures to resolve. This seems to prevent the underlying Hibernate session from closing, causing the application to run out of connections shortly after.
How do I write this function so the database connection isn't kept open while doing the blocking I/O? Is it even possible using JpaRepositories, or do I need to use a lower level API like EntityManager/SessionFactory?
#Service
class SomeJobRunner {
private final SomeJobRepository mSomeJobRepository; //extends JpaRepository
#AutoWired
public SomeJobRunner(final SomeJobRepository someJobRepository) {
mSomeJobRepository = someJobRepository;
}
#Async
public void doSlowJob(final long someJobId) {
SomeJob someJob = mSomeJobRepository.findOne(someJobId);
someJob.setJobStarted(Instant.now());
mSomeJobRepository.saveAndFlush(someJob);
doActualjob(); // Synchronous job doing several requests using Unirest in series
someJob = mSomeJobRepository.findOne(someJobId);
someJob.setJobEnded(Instant.now());
mSomeJobRepository.saveAndFlush(someJob);
}
Well - non-blocking database IO is not possible in Java/JDBC world in a standard way .To put it simply - your Spring data repository would be eventually using JPA ORM Implementation ( likes of Hibernate) which in turn will use JDBC to interact with the database which is essentially blocking in nature. There is work being done on this currently by Oracle (Asynchronous Database Access API ) to provide a similar API as JDBC but non-blocking. They intend to propose this as a standard. Also there is an exciting and parallel effort by Spring guys on this namely R2DBC – Reactive Relational Database Connectivity. They have actually integrated this with Spring data as well (link) so that may help you integrate in your solution. A good tutorial by Spring on this can be found here.
EDIT: As of 2022 Hibernate has reactive option as well
I would suggest to write in the database using a separate JTA transaction. Do do so, define a methode like
#Transactional(Transactional.TxType.REQUIRES_NEW)
public void saveJobStart(final long someJobId) {
SomeJob someJob = mSomeJobRepository.findOne(someJobId);
someJob.setJobStarted(Instant.now());
mSomeJobRepository.saveAndFlush(someJob);
}
Of course it is not quite the same. If doActualjob() fails, in your case, the database won't persist the start date. In my proposal, it will persist it. To compensate, you need to remove the start date in a catch bloc in doSlowJob, within a new transaction, and then rethrow the exception.
Related
Having a circuit breaker with Spring Boot for external calls (e.g. HTTP) is a common pattern that is rather easy to put into place, for example with resilience4j.
I cannot find any information about doing the same with database calls, via resilience4j or some other common pattern, and this is unexpected to me.
Suppose we have a service with a simple JDBC connection to a traditional SQL database. If the database goes down for any reason, I would like to be able to stop all incoming requests to the service at the controller level until the connection is restored. Is there a way to achieve what is essentially circuit breaker functionality for all the transactions happening over the connection to the database?
Yes, this is possible. Chapter 7 of the book Spring Microservices in Action gives an example of this. Effectively, you treat a SQL call in the same way as an HTTP call:
#CircuitBreaker(name = "entityService") ❶
public List<Entity> getEntity(String entityId) {
return entityRepository.findByEntityId(entityId);
}
You will need the following dependencies:
io.github.resilience4j:resilience4j-spring-boot2
io.github.resilience4j:resilience4j-circuitbreaker
io.github.resilience4j:resilience4j-timelimiter
org.springframework.boot:spring-boot-starter-aop
This is using an AOP pattern of adding new behavior to existing code without modifying the code itself.
In the standard synchronous Spring (WebMVC) world, there are OpenEntityManagerInViewFilter and OpenEntityManagerInViewInterceptor that hold a JPA EntityManager open during the whole processing of a request and allow to avoid the annoying LazyInitializationException ('no Session').
OpenEntityManagerInViewInterceptor also works in a similar way for async servlet API-based applications.
But how do you deal with this problem in a Webflux application? Let's say I have a Webflux controller that does something like
service.getOneReactively(...).flatMapMany(one -> obtainAFlux(one))
where service.getOneReactively() finds a domain object in the database and obtainAFlux() accesses a lazy collection on that domain object causing its load. Both calls will be executed on some thread pool, probably on different threads, so Hibernate's Session bound to the first thread (where the domain object is materialized) will not be accessible in the second thread (where the collection is loaded).
This results in LazyInitializationException.
How do you solve such a problem? The only thing I could invent so far is to pack both calls into a method and call it in a transaction in one reactive call.
Hibernate is not reactive, it uses JDBC which is a blocking database driver, and it in turn uses threadlocal to store session information which is not possible to use in Webflux.
If you wish to do blocking database calls you must use Mono.fromCallable and assign the calls to its own Scheduler so that it gets its own dedicate thread. You can read more about it here in the documentation https://projectreactor.io/docs/core/release/reference/#faq.wrap-blocking hibernate cant natively return a Flux.
Your code should look something like this
Flux<Objects> myObjects = Mono.fromCallable(() -> {
// make all your calls to the database here
// build then your flux or whatever you want
return Flux.fromIterable(listOfObjects);
}).subscribeOn(Schedulers.elastic()); // Makes sure this is run in its own thread
as #Toerktumlare mentioned, Hibernate is not reactive, it uses JDBC which is a blocking database driver, but you can use Hibernate reactive and use non-blocking methods and sessions.
a simple code example must be like this:
public Flux<MyUser> findAll() {
try {
Mutiny.SessionFactory factory = CustomEntityManagerFactoryPostgresImpl
.getInstance()
.getEntityManagerFactory()
.unwrap(Mutiny.SessionFactory.class);
CriteriaBuilder builder = factory.getCriteriaBuilder();
CriteriaQuery<MyUser> criteria = builder.createQuery(MyUser.class);
Root<MyUser> root = criteria.from(MyUser.class);
Flux<MyUser> myUser = Flux.fromIterable(
factory.withSession(
session -> session
.createQuery(criteria)
.getResultList()
).await().indefinitely());
factory.close();
return myUser;
} catch (Exception exception) {
log.error(String.valueOf(exception));
return Flux.empty();
}
and a mono<> is like:
public Mono<MyUser> getOneUser(Long id) {
EntityManagerFactory emf = Persistence.createEntityManagerFactory("my-persistence");
Mutiny.SessionFactory sessionFactory = emf.unwrap(Mutiny.SessionFactory.class);
Mutiny.Session session = sessionFactory.openSession();
try {
return Mono.fromFuture(session.find(MyUser.class, id).subscribe().asCompletionStage());
} catch (Exception e) {
throw new RuntimeException(e);
}
}
Using blocking JPA API in a WebFlux application should be avoided.
If you want to access the RDBMS in a Spring WebFlux application, try to use the reactive streams API(none-blocking) instead.
Use Spring R2dbc/Spring Data R2dbc instead, see my Spring R2dbc example.
Use Hibernate Reactive, currently, there is no official Spring/Hibernate Reactive integration, but it is easy to integrate them.
Create a persistence.xml file.
Create a Mutiny.SessionFactory bean.
Use Mutiny.SessionFactory to handle RDBMS operations,see the PostRepository example written in Hiberante Reactvie.
The complete Spring/Hibernate reactive example can be found on my Github (but the web handling is using Vertx).
Mixed blocking JPA in a Spring Webflux application should be avoided. But it is possible, see my example which mixed JPA and R2dbc in a single application.
And the Open session in view pattern is desginated to work in a traditional servlet container and depends on Spring WebMVC.
So from time to time we see exceptions like these:
java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:150)
at java.net.SocketInputStream.read(SocketInputStream.java:121)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
at org.bson.io.Bits.readFully(Bits.java:48)
at org.bson.io.Bits.readFully(Bits.java:35)
at org.bson.io.Bits.readFully(Bits.java:30)
at com.mongodb.Response.<init>(Response.java:42)
at com.mongodb.DBPort$1.execute(DBPort.java:141)
at com.mongodb.DBPort$1.execute(DBPort.java:135)
at com.mongodb.DBPort.doOperation(DBPort.java:164)
at com.mongodb.DBPort.call(DBPort.java:135)
at com.mongodb.DBTCPConnector.innerCall(DBTCPConnector.java:292)
at com.mongodb.DBTCPConnector.call(DBTCPConnector.java:271)
at com.mongodb.DBCollectionImpl.find(DBCollectionImpl.java:84)
at com.mongodb.DBCollectionImpl.find(DBCollectionImpl.java:66)
at com.mongodb.DBCollection.findOne(DBCollection.java:870)
at com.mongodb.DBCollection.findOne(DBCollection.java:844)
at com.mongodb.DBCollection.findOne(DBCollection.java:790)
at org.springframework.data.mongodb.core.MongoTemplate$FindOneCallback.doInCollection(MongoTemplate.java:2000)
Whats the best way to handle and recover from these in code?
Do we need to put a 'retry' around each and every mongodb call?
You can use MongoClientOptions object to set different optional connection parameters. You are looking at setting heart beat frequency to make sure driver retry for connection. Also set socket time out to make sure it does not continue for too long.
MinHeartbeatFrequency: In the event that the driver has to frequently re-check a server's availability, it will wait at least this long since the previous check to avoid wasted effort. The default value is 10ms.
HeartbeatSocketTimeout: Timeout for heart beat check
SocketTimeout: Time out for connection
Reference API
To avoid too much code duplication, optionally you can follow some pattern as given below.
Basic idea is to avoid any database connection related configuration littered everywhere in the projects.
/**
* This class is an abstraction for all mongo connection config
**/
#Component
public class MongoConnection{
MongoClient mongoClient = null;
...
#PostConstruct
public void init() throws Exception {
// Please watch out for deprecated methods in new version of driver.
mongoClient = new MongoClient(new ServerAddress(url, port),
MongoClientOptions.builder()
.socketTimeout(3000)
.minHeartbeatFrequency(25)
.heartbeatSocketTimeout(3000)
.build());
mongoDb = mongoClient.getDB(db);
.....
}
public DBCollection getCollection(String name) {
return mongoDb.getCollection(name);
}
}
Now you can use MongoConnection in DAO-s
#Repository
public class ExampleDao{
#Autowired
MongoConnection mongoConnection;
public void insert(BasicDBObject document) {
mongoConnection.getCollection("example").insert(document);
}
}
You can also implement all the database operations inside MongoConnection to introduce some common functionality across the board. For example add logging for all "inserts"
One of the many options, to handle retry is Spring retry project
https://github.com/spring-projects/spring-retry
Which provides declarative retry support for Spring applications.
This is basically Spring answer for this problem. It is used in Spring Batch, Spring Integration, Spring for Apache Hadoop (amongst others).
If you want to approach timeouts (and related) problems not only for your MongoDB but also for any other external references then you should try Netflix's Hystrix (https://github.com/Netflix/Hystrix).
It is an awesome library that integrates nicely with RX and Asynchronous processing that becomes so much more popular lately.
If i'm not mistaken, i think you need to config your properties like timeout or so when you try to build the connection or just prepare them well in connection pool.
Or,you may just check your network or machine, and split your request-data by more times to reduce network trans time
https://github.com/Netflix/Hystrix is your tool for handling dependecies.
I am using mongodb with spring mvc for a REST API. I have a controller which adds comments for events. This controller use a method of my event service. And implementation of them are below. However It seems to me as if my mongodb connection works without transaction. I have 1 insert and 1 update in that my controller adding comment. If update of event throws an error, the request commits inserting.Why is it not doing rollback? How can I fix this problem?
public Comments addComment(Comments comment) throws Exception{
comment.setCommentDate(SowUtil.getDateTimeDB());
comment.setCommenter(personService.findPersonById("531cc90e3c37b20bef47dfc7"));
mongoTemplate.insert(comment);
Events adv = findEventById(comment.getEventId());
adv.addComment(comment);
eventService.save(adv);
return comment;
}
MongoDB does not have the same notion of transactions as RDBMS. Based on your example, it looks like you are using two documents.
MongoDB has an article on 2-phase commit patterns involving multiple documents (
http://docs.mongodb.org/manual/tutorial/perform-two-phase-commits/). Perhaps, that can help you implementing a rollback?
I believe that Spring also has a notion of transaction support for MongoDB (http://docs.spring.io/spring-integration/reference/html/mongodb.html).
Context of problem I want to solve: I have a java spring http interceptor AuditHttpCommunicationInterceptor that audits communication with an external system. The HttpClieant that does the communication is used in a java service class that does some business logic called DoBusinessLogicSevice.
The DoBusinessLogicSevice opens a new transaction and using couple of collaborators does loads of stuff.
Problem to solove: Regardless of the outcome of any of the operations in DoBusinessLogicSevice (unexpected Exceptions, etc) I want audits to be stored in the database by AuditHttpCommunicationInterceptor.
Solution I used: The AuditHttpCommunicationInterceptor will open a new transaction this way:
TransactionDefinition transactionDefinition = new DefaultTransactionDefinition(TransactionDefinition.PROPAGATION_REQUIRES_NEW);
new TransactionTemplate(platformTransactionManager, transactionDefinition).execute(new TransactionCallbackWithoutResult() {
#Override
protected void doInTransactionWithoutResult(TransactionStatus status) {
// do stuff
}
});
Everything works fine. When a part of DoBusinessLogicSevice throws unexpected exception its transaction is rolled back, but the AuditHttpCommunicationInterceptor manages to store the audit in the database.
Problem that arises from this solution: AuditHttpCommunicationInterceptor uses a new db connection. So for every DoBusinessLogicSevice call I need 2 db connections.
Basicly, I want to know the solution to the problem: how to make TransactionTemplate "suspend" the current transaction and reuse the connection for a new one in this case.
Any ideas? :)
P.S.
One idea might be to take a different design approach: drop the interceptor and create an AuditingHttpClient that is used in DoBusinessLogicSevice directly (not invoked by spring) but I cannot do that because I cannot access all http fields in there.
Spring supports nested transactions (propagation="NESTED"), but this really depends on the database platform, and I don't believe every database platform is capable of handling nested transactions.
I really don't see what's a big deal with taking connection from a pool, doing a quick audit transaction and returning connection back.
Update: While Spring supports nested transactions, it looks like Hibernate doesn't. If that's the case, I say: go with another connection for audit.