Do we have to explicitly manage database resources when using Spring Framework.. liking closing all open connections etc?
I have read that Spring relieves developer from such boiler plate coding...
This is to answer an error that I am getting in a Spring web app:
org.springframework.jdbc.CannotGetJdbcConnectionException:
Could not get JDBC Connection; nested
exception is java.sql.SQLException:
ORA-00020: maximum number of processes
(150) exceeded
The jdbcTemplate is configured in the xml file and the DAO implementation has reference to this jdbcTemplate bean which is used to query the database.
Do we have to explicitly manage database resources when using Spring Framework, like closing all open connections etc?
If you are using Spring abstraction like JbdcTemplate, Spring handles that for you and it is extremely unlikely that that there is a bug in that part.
Now, without more information on your configuration (your applicationContext.xml), on the context (how do you create your application context, when does this happen exactly?), it is a hard to say anything. So this is a shot in the dark: do you have the attribute destroy-method="close" set on your datasource configuration? Something like that:
<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
In certain circumstances, not using the destroy-method combined with some other bad practices may eventually end up with exhausting resources.
It could be due to connections not being closed. How are you accessing your connections within spring? Are you are using JdbcTemplate to query the database? Or just getting the connection from spring?
I have read that Spring relieves
developer from such boiler plate
coding
That depends which level of Spring you operate at. JdbcTemplate provides many different operations, some of which are fire-and-forget, some of which still require you to manage your JDBC resources (connections, resultsets, statements, etc) properly. The rule of thumb is that if you find yourself calling getConnection(), then at some point you need to call releaseConnection() also.
ORA-00020: maximum number of processes
(150) exceeded
Are you using a connection pool? If so, then make sure that it isn't configured with a larger number of max connections than your database is capable of handling (150, in this case). If you're not using a connection pool, then you really, really should be.
you say "The jdbcTemplate is configured in the xml file". You should normally create a new instance of the jdbcTemplate for each usage, not have it managed by spring.
I would guess that each time you request a new jdbcTemplate bean from spring, it is creating a new one with a new connection to the database, but after it falls out of scope in your code it is still referenced by spring's applicationContext, and so does not close the connection.
My hosing, provide only 20 connection. I done by manually close the connection on every request to db. I not declared a destory-method in bean(this not worked "i dont know why"), but i done in every requst call. (Hint : extends JdbcDaoSupport in dao class).
public void cleanUp() {
try {
if (!this.getJdbcTemplate().getDataSource().getConnection().isClosed()) {
this.getJdbcTemplate().getDataSource().getConnection().close();
}
} catch (Exception e) {
Logger.getLogger(myDAOImpl.class.getName()).log(Level.SEVERE, null, e);
}
}
IMPORTANT: Here I mention that, what I have done to solve my problem. You should not use this code directly. You should use only one connection. Alter if as per your code.
Related
I want to create a microservice with Spring Boot. For persistence i use a mariadb database. To wait for the database which is running in a docker container, i implemented the following code like shown here:
#Bean
public DatabaseStartupValidator databaseStartupValidator(DataSource dataSource) {
var dsv = new DatabaseStartupValidator();
dsv.setDataSource(dataSource);
dsv.setTimeout(60);
dsv.setInterval(7);
dsv.setValidationQuery(DatabaseDriver.MYSQL.getValidationQuery());
return dsv;
}
The code is working very well, my application is now waiting for the database connection. But i get an exception at startup of the application:
java.sql.SQLNonTransientConnectionException: Could not connect to Host ....
...
...
...
In the next line i get an information, that it will wait for the database:
021-04-07 21:29:40.816 INFO 16569 --- [ main] o.s.j.support.DatabaseStartupValidator : Database has not started up yet - retrying in 7 seconds (timeout in 57.65 seconds)
After that the application is starting as expected. So i think everything is working fine, but what i have to do to suppress the Exception? In the linked article it should work without an exception. Do i have to implement the "dependsOnPostProcessor" function? Which dependency i have to use? Sorry, possible a dumb question, i am new to spring boot.
to get rid of that exception you can state the below directive in your application.properties file:
logging.level.com.zaxxer.hikari=OFF
Keep in mind that if the application will not be able to get in contact with the db your spring crashes after a while due to that exception. In addition the above directive prevent you to see any logging activity related to Hikari.
In summary you hide the appearance of the exception until it is possible before the application dies due to timeout.
hoping I clarified a bit the case
Yes indeed you need to add the "depends-on" for the beans that rely on the data source. Note the following part of the documentation:
To be referenced via "depends-on" from beans that depend on database startup, like a Hibernate SessionFactory or custom data access objects that access a DataSource directly.
If I understand it well, this means that beans such as an EntityManagerFactory which rely on the database will now have to go through the DatabaseStartupValidator bean and wait for the DB startup. I don't know what caused your exception, but usually there is an EntityManagerFactory involved, so try adding the DependsOn on this object at least.
This is how the linked article is doing it:
#Bean
public static BeanFactoryPostProcessor dependsOnPostProcessor() {
return bf -> {
// Let beans that need the database depend on the DatabaseStartupValidator
// like the JPA EntityManagerFactory or Flyway
String[] flyway = bf.getBeanNamesForType(Flyway.class);
Stream.of(flyway)
.map(bf::getBeanDefinition)
.forEach(it -> it.setDependsOn("databaseStartupValidator"));
String[] jpa = bf.getBeanNamesForType(EntityManagerFactory.class);
Stream.of(jpa)
.map(bf::getBeanDefinition)
.forEach(it -> it.setDependsOn("databaseStartupValidator"));
};
}
You may not necessarily have Flyway configured, but the main thing to note is the dependency itself is referenced by the bean name databaseStartupValidator which is the name of the method that creates the bean.
I am looking for the right way to set a run-time parameter when a database connection is open. My run-time parameter is actually a time zone, but I think this should work for an arbitrary parameter.
I've found following solutions, but I feel like none of these is the right thing.
JdbcInterceptor
Because Spring Boot has Apache Tomcat connection pool as default I can use org.apache.tomcat.jdbc.pool.JdbcInterceptor to intercept connections.
I don't think this interceptor provides a reliable way to perform a statement when connection is open. Possibility to intercept every statement provided by this interceptor is unnecessary to set a parameter that should be set only once.
initSQL property
Apache's pooled connection has a build-in ability to initialise itself by a statement provided by PoolProperties.initSQL parameter. This is executed in ConnectionPool.createConnection(...) method.
Unfortunately official support for this parameter has been removed from Spring and no equivalent functionality has been introduced since then.
I mean, I can still use a datasource builder like in an example below, and then hack the property into a connection pool, but this is not a good looking solution.
// Thank's to property binders used while creating custom datasource,
// the datasource.initSQL parameter will be passed to an underlying connection pool.
#Bean
#ConfigurationProperties(prefix = "datasource")
public DataSource dataSource() {
return DataSourceBuilder.create().build();
}
Update
I was testing this in a Spring Boot 1.x application. Above statements are no more valid for Spring Boot 2 applications, because:
Default Tomcat datasource was replaced by Hikari which supports spring.datasource.hikari.connection-init-sql property. It's documentation says Get the SQL string that will be executed on all new connections when they are created, before they are added to the pool.
It seems that similar property was reintroduced for Tomcat datasource as spring.datasource.tomcat.init-s-q-l.
ConnectionPreparer & AOP
This is not an actual solution. It is more like an inspiration. The connection preparer was a mechanism used to initialise Oracle connections in Spring Data JDBC Extensions project. This thing has its own problems and is no more maintained but possibly can be used as a base for similar solution.
If your parameter is actually a time zone, why don't you find a way to set this property.
For example if you want to store or read a DateTime with a predefined timestamp the right way to do this is to set property hibernate.jdbc.time_zone in hibernate entityManager or spring.jpa.properties.hibernate.jdbc.time_zone in application.properties
I am having spring webservice application with oracle as a database. Right now i have datasource created using weblogic server. Also using eclipse linkg JPA to do both read and write transactions(insert,Read and update). Now we want to separate dataSources for read(read) and wrtie(insert or update) transactions.
My current dataSource is as followed:
JNDI NAME : jdbc/POI_DS
URL : jdbc:oracle:thin:#localhost:1521:XE
using this, I am doing both read and write transactions.
What if i do the following:
JNDI NAME : jdbc/POI_DS_READ
URL : jdbc:oracle:thin:#localhost:1521:XE
JNDI NAME : jdbc/POI_DS_WRITE
URL : jdbc:oracle:thin:#localhost:1521:XE
I knew that using XA datasource we can define multiple dataSources. Can I do same thing without XA dataSource. Does any one tried this kind of approach.
::UPDATE::
Thank you all for your responses I have implemented following solution.
I have taken the multiple database approach. where you will define multiple transactionManagers and managerFactory. I have taken only single non xa dataSource(JNDI) that is refereed in EntityManagerFactory Bean.
you can reefer following links here which are for multiple dataSources
Multiple DataSource Approach
defining #transactional value
Also explored on transaction managers org.springframework.transaction.jta.WebLogicJtaTransactionManager and org.springframework.orm.jpa.JpaTransactionManager as well.
There is an interesting article about this in Spring docs - Dynamic DataSource Routing. There is an example there, that allows you to basically switch data sources at runtime. It should help you. I'd gladly help you more, if you have any more specific questions.
EDIT: It tells, that the actual use is to have connection to multiple databases via one configuration, but you could manage to create different configs to one database with different params, as you'd need to.
I would suggest using Database "services". Each workload, read-only and read-write, would be using its own service to access the database. That way you can use AWR reports to get statistics for each service. You can also turn off read-write when you keep read-only up and running.
Here is a pointer to the Oracle Database documentation that talks about Services:
https://docs.oracle.com/database/121/ADMIN/create.htm#CIABBCAI
If you're using spring, you should be able to accomplish this without using 2 Datasources via spring #Transactional with the readonly property set to true. The reason why I suggest this is that you seem to be concerned about the transactionality only and this seems to be catered for in the spring framework?
I'd suggest something like this for your case:
#Transactional(readOnly = true)
public class DefaultFooService implements FooService {
public Foo getFoo(String fooName) {
// do something
}
// these settings have precedence for this method
#Transactional(readOnly = false, propagation = Propagation.REQUIRES_NEW)
public void updateFoo(Foo foo) {
// do something
}
}
Using this style, you should be able to split read only services from their write counterparts, or even have read and write service methods combined. But both of these do not use 2 datasources.
Code is from the Spring Reference
I am pretty sure that you need to address the problem on the database / connection url + properties layer.
I would google around for something like read write replication.
Related to your question with JPA and transaction. You are doomed when you are using multiple Datasources. Also XA datasources are not really a solution for that. The only thing they do for you is to ensure consistency over multi data source operations. XA Transaction do only span some sort of logical transaction over two transactions (one for each datasource). From the transaction isolation point of view (as long as your not using READ_UNCOMMITED) both datasources use their own transaction. This means the read data source would not see the changes made by the write transaction.
I am using spring & hibernate. my application has 3 modules. Each module has a specific database. So, Application deals with 3 databases. On server start up, if any one of the databases is down, then server is not started. My requirement is even if one of the databases is down, server should start as other module's databases are up, user can work on other two modules. Please suggest me how can i achieve this?
I am using spring 3.x and hibernate 3.x. Also i am using c3p0 connection pooling.
App server is Tomcat.
Thanks!
I would use the #Configuration annotation to make an object who's job it is to construct the beans and deal with the DB down scenario. When constructing the beans, test if the DB connections are up, if not, return a Dummy Version of your bean. This will get injected into the relevant objects. The job of this dummy bean is to really just throw an unavailable exception when called. If your app can deal with these unavailable exceptions for certain functions and show that to the user while continuing to function when the other datasources are used, you should be fine.
#Configuration
public class DataAccessConfiguration {
#Bean
public DataSource dataSource() {
try {
//create data source to your database
....
return realDataSource;
} catch (Exception) {
//create dummy data source
....
return dummyDataSource;
}
}
}
This was originally a comment:
Have you tried it? You wouldn't know whether a database is down until you connect to it, so unless c3p0 prevalidates all its connections, you wouldn't know that a particular database is down until you try to use it. By that time your application will have already started.
In our system we have multi-threaded processing engine. During processing each thread calls methods to retrieve data from the database. We determined that performance is greatly improved if methods called from the same thread use the same DB session (sessions are coming from the pool of course).
Is there any standard way in Spring to ensure such thing or we have to come up with our own custom solution?
UPDATE: Forgot to mention that same methods can be called in different context where they should use a standard way of getting the session from the pool
I did not see Spring anywhere in your question. So I assume you want a simple utility to do this.
class SessionUtil {
private ThreadLocal currentSession;
public Session getCurrentSession() {
if(currentSession.get() == null) {
Session s = //create new session
currentSession.set(s);
}
return (Session)currentSession.get();
}
}
The Thread local will ensure that within the same thread it is always the same session. If you are using Spring then the classes/utilities mentioned above (in other responses) should be perfect.
Spring has a class called TransactionSynchronizationManager. It stores the current Session in a ThreadLocal. The TransactionSynchronizationManager is not recommended for use by the developer, but you can try using it.
Session session = ((SessionHolder)
TransactionSynchronizationManager.getResource(sessionFactory)).getSession();
(if you are using EntityManager, simply replace "Session" with "EntityManager").
You can have the sessionFactory injected in your bean - it is per-application.
Take a look at this discussion.
Other options, which I think are preferable to manual thread-handling are:
Thread pooling
Spring batch
Spring-JMS integration
Spring 3.0 has a concept of thread-scoped beans (hovewer, this scope is not registered by default, see docs): 3.5 Bean scopes, 3.5.5.2 Using a custom scope
EDIT:
I say about this:
Thread-scoped beans As of Spring 3.0,
a thread scope is available, but is
not registered by default. For more
information, see the documentation for
SimpleThreadScope. For
instructions on how to register this
or any other custom scope, see
Section 3.5.5.2, “Using a custom
scope”.
Spring coordinates database sessions, connections and threads through it's Transaction Framework (actually, using its TransactionSynchronizationManager - see description here - but you really don't want to mess with that directly, it's fearsome). If you need to coordinate your threads, then this is by far the simplest way of doing it.
How you choose to use the framework, however, is up top you.