I am trying to update datasource in Spring Boot when the DB property like DB name, password or hostname changes in the spring configuration file or custom DB property file. When the property changes the application has to update by its own by listening changes to property.
I was using Spring actuator to /restart beans once the DB configuration is changed. But user has to explicitly make a post request to restart. This step has to be avoided by listening to the changes and update datasource.
Can you tell me the best way to do this in Spring boot?
Found a way to update datasource on-the-fly,
I have given external spring config file which contains DB properties to the application and then refreshed the properties using #RefreshScope for the datasource bean.
A thread monitors the file changes and makes a call to actuator refresh() method.
database.properties
dburl=jdbc://localhost:5432/dbname
dbusername=user1
dbpassword=userpwd
Creating datasource,
#RefreshScope
public class DBPropRefresh {
#Value("${dburl}")
private String dbUrl;
#Value("${dbusername}")
private String dbUserName;
#Value("${dbpassword}")
private String dbPassword;
#Bean
#RefreshScope
public DataSource getDatasource() {
return new DatasourceBuilder().create().url(dbUrl).username(dbUserName).password(dbPassword);
}
}
Giving external config file to the application,
java -jar myapplication.jar --spring.config.location=database.properties
I have created a Java thread class to monitor database.properties file changes. Followed https://dzone.com/articles/how-watch-file-system-changes
When there are changes then it makes call to refreshEndPoint.refresh().
In pom.xml,
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
<version>1.5.6.RELEASE</version>
</dependency>
You can use Spring's Dynamic Data Source routing and check if it helps? It's a very old technique and might come handy, if that serves your purpose.
But please note that - this is data source routing and not new data source configuration.
https://spring.io/blog/2007/01/23/dynamic-datasource-routing/
In my project I used multitenancy . Basically I defined several datasources in properties like this:
primary.datasource.url=jdbc:postgresql://localhost:5432/db_name?currentSchema=schema_name
primary.datasource.username=user
primary.datasource.password=password
primary.datasource.driverClassName=org.postgresql.Driver
primary.datasource.driver-class-name=org.postgresql.Driver
secondary.datasource.url=jdbc:postgresql://localhost:5432/other_db?currentSchema=schema
secondary.datasource.username=user
secondary.datasource.password=password
secondary.datasource.driverClassName=org.postgresql.Driver
secondary.datasource.driver-class-name=org.postgresql.Driver
default.datasource.url=jdbc:postgresql://localhost:5432/default_db?currentSchema=public
default.datasource.username=user
default.datasource.password=password
default.datasource.driverClassName=org.postgresql.Driver
default.datasource.driver-class-name=org.postgresql.Driver
then in configuration class defined multiple datasources:
#Bean
#Primary
#ConfigurationProperties(prefix="primary.datasource")
public DataSource primaryDataSource() {
return DataSourceBuilder.create().build();
}
#Bean
#ConfigurationProperties(prefix="secondary.datasource")
public DataSource secondaryDataSource() {
return DataSourceBuilder.create().build();
}
#Bean
#ConfigurationProperties(prefix="default.datasource")
public DataSource defaultDataSource(){
return DataSourceBuilder.create().build();
}
and configured multitenancy basing on this and this article.
Pros:
Easy tenant switch which could be triggered manually or even configured to be triggered on some specific header in request (filters).
Could be cofigured to switch between schemas or databases.
Happens dynamically ( you don't have to restart your beans )
Cons:
You have to define all db possibilities in property file.
You have to turn off schema validation because it will go nuts.
Related
I am working with spring boot. I have properties defined in application.yml.
spring:
datasource:
username: username
password: password
username and password values are stored externally which program fetches during startup. let's say the bean which fetches them during startup is dbConfig
How can I inject values from dbConfgig to application.yml?
I am using spring-data-jpa autoconfigure which automatically connects to database at startup. I want these values to be loaded to application.yml before spring connects to database.
There is no need to inject the user/password in application.yml. You can set them programmatically like this:
#Configuration
public class DataSourceConfig {
#Bean
public DataSource getDataSource() {
DataSourceBuilder dataSourceBuilder = DataSourceBuilder.create();
dataSourceBuilder.driverClassName("org.h2.Driver");
dataSourceBuilder.url("jdbc:h2:mem:test");
// Take the values from external source, then set them
dataSourceBuilder.username("username");
dataSourceBuilder.password("password");
return dataSourceBuilder.build();
}
}
I think that first, you must create a thread to detect the change at your db Config file and then may you must re-init your bean (data source) to make your change effect.
See:
how-to-reinitialize-a-spring-bean
You may also try spring cloud to store properties. And you can then use with the help of placeholders.
https://cloud.spring.io/spring-cloud-config/reference/html/
Spring Boot : How to add new Datasource at runtime
My project want to connect two datasource.
The first datasource I can Config in application.properties but the second datasource can't config because this config is in the tableConfig from DB of the first datasource.
So,
config the 1st datasource.
query data from the 1st datasource for get config of 2nd datasource (url, username, password).
add new 2nd datasource
Now, I config two Datasource from application.properties and it's work.
But the requirement want to change the 2nd datasource from table of 1st datasource. T.T
Please, gives me some suggestions.
Thank you.
A Spring configuration like this should work (consider it pseudo code):
#Bean("secondDatasource")
public Datasource secondDatasource(#Qualifier("firstDatasource") Datasource ds){
// use `ds` to obtain the necessary information to obtain a datasource ...
return DataSourceBuilder
.create()
.username(username)
.password(pwd)
.url(url)
.driverClassName(driver)
.build();
}
I would at least start without using Spring Data JPA in the configuration class and operate directly on the data source to keep things simple.
You already got pointers, how to set up Spring Data JPA to then use the different data sources: http://www.baeldung.com/spring-data-jpa-multiple-databases
The code above is mainly just copied from: https://stackoverflow.com/a/28822145
I have mysql database and i have configured database properties in application.properties file .
Now if i do change db connection properties , i want reflect that changes into my application on the fly , means with out restarting the server
Is this possible using with spring cloud config server and actuator?
I have tested this quite a bit and here are my findings.
Spring config server works pretty well for simple key value pairs.
It also works for Database properties provided that you are creating datasource objects yourself and you are using #RefreshScope.
For example, if you have a config server with these properties.
mongodb.feed.database=kiran
mongodb.feed.host=localhost
mongodb.feed.port=27017
And you are configuring MongoTemplate in your application like this.
#Configuration
#ConfigurationProperties(prefix = "mongodb.feed")
#EnableMongoRepositories(basePackages = "in.phani.springboot.repository", mongoTemplateRef = "feedMongoTemplate")
#Setter
class FeedMongoConfig {
private String host;
private int port;
private String database;
#Primary
#Bean(name = "feedMongoTemplate")
#RefreshScope // this is the key
public MongoTemplate feedMongoTemplate() throws Exception {
final Mongo mongoClient = createMongoClient(new ServerAddress(host, port));
return new MongoTemplate(mongoClient, database);
}
Mongo createMongoClient(ServerAddress serverAddress) {
return new MongoClient(serverAddress);
}
}
And if you change the database name in your config properties and then refresh the scope with /refresh endpoint. It works pretty well.
With springboot you need not do manual configuration like this. Spring boot has Autoconfiguration for most of the stuff. Continuing with the same example above, if you were to put in config properties something like this
spring.data.mongodb.uri=mongodb://localhost:27017/phani
spring-boot will configure MongoTemplate for you(you don't need to create yourself as in 2nd point).
Here comes the hiccup.
Now if you change the database name, and refresh the scope, it doesn't work. Because in this case, MongoTemplate was configured by spring-boot Autoconfiguration(MongoAutoConfiguration)
So in conclusion, it needs extensive testing to be done, before using it on production(especially for complex beans like datasources, MongoTemplates), since there is not enough documentation on this.. But I would say, it is worth trying.
I have multitenant database in Spring Boot. I store multi spring JDBC templates (based on tomcat Data Sources, configured manually) in map (immutable bean). And I choose proper data source based on uuid in a request (connection pool per database). I have disabled standard configuration in Spring Boot by:
#SpringBootApplication(exclude = DataSourceAutoConfiguration.class)
What is the proper way of transaction manager configuration? With single data source I can use PlatformTransactionManager, but how it should be done with multiple jdbc templates/data sources in spring? It would be the best if I could set everything dynamically. Thanks in advance.
Here a solution for using multiple datasources
http://www.baeldung.com/spring-data-jpa-multiple-databases
Configure Two DataSources
If you need to configure multiple data sources, you can apply the same tricks that are described in the previous section. You must, however, mark one of the DataSource #Primary as various auto-configurations down the road expect to be able to get one by type.
If you create your own DataSource, the auto-configuration will back off. In the example below, we provide the exact same features set than what the auto-configuration provides on the primary data source
#Bean
#Primary
#ConfigurationProperties("app.datasource.foo")
public DataSourceProperties fooDataSourceProperties() {
return new DataSourceProperties();
}
#Bean
#Primary
#ConfigurationProperties("app.datasource.foo")
public DataSource fooDataSource() {
return fooDataSourceProperties().initializeDataSourceBuilder().build();
}
#Bean
#ConfigurationProperties("app.datasource.bar")
public BasicDataSource barDataSource() {
return (BasicDataSource) DataSourceBuilder.create()
.type(BasicDataSource.class).build();
}
fooDataSourceProperties has to be flagged #Primary so that the database initializer feature uses your copy (should you use that).
app.datasource.foo.type=com.zaxxer.hikari.HikariDataSource
app.datasource.foo.maximum-pool-size=30
app.datasource.bar.url=jdbc:mysql://localhost/test
app.datasource.bar.username=dbuser
app.datasource.bar.password=dbpass
app.datasource.bar.max-total=30
I would like to know if Spring offers any support for Auto Fail over of Data Sources? For example, if the primary data source is down, connect to secondary. Any suggestions to effectively configure auto fail over of Data sources is greatly appreciated.
Thanks.
The Primary/DR setup for DB should be managed at DB level. Its not right to switch that in code. But to answer your question "can connect to 2 Data sources in Spring". Yes you can easier if you are using Spring-boot.
Few details from spring documentation.
Mark one of them as #Primary if you are using the default auto-configuration for JDBC or JPA (then that one will be picked up by any #Autowired injections).
#Bean
#Primary
#ConfigurationProperties(prefix="datasource.primary")
public DataSource primaryDataSource() {
return DataSourceBuilder.create().build();
}
#Bean
#ConfigurationProperties(prefix="datasource.secondary")
public DataSource secondaryDataSource() {
return DataSourceBuilder.create().build();
}