I have mysql database and i have configured database properties in application.properties file .
Now if i do change db connection properties , i want reflect that changes into my application on the fly , means with out restarting the server
Is this possible using with spring cloud config server and actuator?
I have tested this quite a bit and here are my findings.
Spring config server works pretty well for simple key value pairs.
It also works for Database properties provided that you are creating datasource objects yourself and you are using #RefreshScope.
For example, if you have a config server with these properties.
mongodb.feed.database=kiran
mongodb.feed.host=localhost
mongodb.feed.port=27017
And you are configuring MongoTemplate in your application like this.
#Configuration
#ConfigurationProperties(prefix = "mongodb.feed")
#EnableMongoRepositories(basePackages = "in.phani.springboot.repository", mongoTemplateRef = "feedMongoTemplate")
#Setter
class FeedMongoConfig {
private String host;
private int port;
private String database;
#Primary
#Bean(name = "feedMongoTemplate")
#RefreshScope // this is the key
public MongoTemplate feedMongoTemplate() throws Exception {
final Mongo mongoClient = createMongoClient(new ServerAddress(host, port));
return new MongoTemplate(mongoClient, database);
}
Mongo createMongoClient(ServerAddress serverAddress) {
return new MongoClient(serverAddress);
}
}
And if you change the database name in your config properties and then refresh the scope with /refresh endpoint. It works pretty well.
With springboot you need not do manual configuration like this. Spring boot has Autoconfiguration for most of the stuff. Continuing with the same example above, if you were to put in config properties something like this
spring.data.mongodb.uri=mongodb://localhost:27017/phani
spring-boot will configure MongoTemplate for you(you don't need to create yourself as in 2nd point).
Here comes the hiccup.
Now if you change the database name, and refresh the scope, it doesn't work. Because in this case, MongoTemplate was configured by spring-boot Autoconfiguration(MongoAutoConfiguration)
So in conclusion, it needs extensive testing to be done, before using it on production(especially for complex beans like datasources, MongoTemplates), since there is not enough documentation on this.. But I would say, it is worth trying.
Related
I need advice on what method will suit better for my case.
I have a Java application with Spring Boot, and right now, for the testing part, I use a localhost PostgreSQL database. The setup is quite simple; I just have the datasource.url/username/password and port setup in the application.properties file.
Now, I need to use a remote PostgreSQL database for a particular integration test class. So, as far as I know, I can do this in at least two ways:
I can set up in application.properties another configuration for the remote database, and use it in the test class with environment.getProperty().
Or, I can make a bean to use those properties:
#Bean
public DataSource dataSource() {
DataSourceBuilder dataSourceBuilder = DataSourceBuilder.create();
dataSourceBuilder.url(dbUrl);
dataSourceBuilder.username(username);
dataSourceBuilder.password(password);
return dataSourceBuilder.build();
}
And the things are not as simple as they may seem.
I have a class for tests configuration, ClassA.
I have a second class with actual tests, let's call it ClassB.
ClassB extends Class A, and ClassA starts a JAR file, which is another Java application which exposes some rest APIs, which will be tested with the tests from ClassB.
Now I use a local database, but I will want to use the remote one, in the future.
I managed to download the JAR file from Artifactory with a Maven dependency, and in the configuration class I search for the JAR file, I get the directory of it, and use an array of commands to run it: {"java", "-jar", "directory.getCanonicalPath()}, and ProcessStreamer to process it.
After I check the JAR file is up and running, the tests from ClassB are triggered, and they do CRUD operations against the recently opened application making REST calls to the APIs.
Any suggestions on how I can set up the second remote database?
Thank you!
Since you are saying you want to use both databases at once, I would create a separate data source for the remote database:
#Configuration
public class MultipleDBConfig {
#Bean(name = "dbRemote")
#ConfigurationProperties(prefix = "spring.dbRemote")
public DataSource dbRemoteDataSource() {
return DataSourceBuilder.create().build();
}
#Bean(name = "dbRemoteJdbcTemplate")
public JdbcTemplate jdbcTemplate(#Qualifier("dbRemote") DataSource dbRemote) {
return new JdbcTemplate(db1);
}
}
And then use them in your DAO/services:
#Autowired
#Qualifier("dbRemoteJdbcTemplate")
private JdbcTemplate dbRemoteTemplate;
If you can't use JdbcTemplates directly for that test case then I would suggest using RoutingDataSource from Spring.
I have multitenant database in Spring Boot. I store multi spring JDBC templates (based on tomcat Data Sources, configured manually) in map (immutable bean). And I choose proper data source based on uuid in a request (connection pool per database). I have disabled standard configuration in Spring Boot by:
#SpringBootApplication(exclude = DataSourceAutoConfiguration.class)
What is the proper way of transaction manager configuration? With single data source I can use PlatformTransactionManager, but how it should be done with multiple jdbc templates/data sources in spring? It would be the best if I could set everything dynamically. Thanks in advance.
Here a solution for using multiple datasources
http://www.baeldung.com/spring-data-jpa-multiple-databases
Configure Two DataSources
If you need to configure multiple data sources, you can apply the same tricks that are described in the previous section. You must, however, mark one of the DataSource #Primary as various auto-configurations down the road expect to be able to get one by type.
If you create your own DataSource, the auto-configuration will back off. In the example below, we provide the exact same features set than what the auto-configuration provides on the primary data source
#Bean
#Primary
#ConfigurationProperties("app.datasource.foo")
public DataSourceProperties fooDataSourceProperties() {
return new DataSourceProperties();
}
#Bean
#Primary
#ConfigurationProperties("app.datasource.foo")
public DataSource fooDataSource() {
return fooDataSourceProperties().initializeDataSourceBuilder().build();
}
#Bean
#ConfigurationProperties("app.datasource.bar")
public BasicDataSource barDataSource() {
return (BasicDataSource) DataSourceBuilder.create()
.type(BasicDataSource.class).build();
}
fooDataSourceProperties has to be flagged #Primary so that the database initializer feature uses your copy (should you use that).
app.datasource.foo.type=com.zaxxer.hikari.HikariDataSource
app.datasource.foo.maximum-pool-size=30
app.datasource.bar.url=jdbc:mysql://localhost/test
app.datasource.bar.username=dbuser
app.datasource.bar.password=dbpass
app.datasource.bar.max-total=30
I am trying to update datasource in Spring Boot when the DB property like DB name, password or hostname changes in the spring configuration file or custom DB property file. When the property changes the application has to update by its own by listening changes to property.
I was using Spring actuator to /restart beans once the DB configuration is changed. But user has to explicitly make a post request to restart. This step has to be avoided by listening to the changes and update datasource.
Can you tell me the best way to do this in Spring boot?
Found a way to update datasource on-the-fly,
I have given external spring config file which contains DB properties to the application and then refreshed the properties using #RefreshScope for the datasource bean.
A thread monitors the file changes and makes a call to actuator refresh() method.
database.properties
dburl=jdbc://localhost:5432/dbname
dbusername=user1
dbpassword=userpwd
Creating datasource,
#RefreshScope
public class DBPropRefresh {
#Value("${dburl}")
private String dbUrl;
#Value("${dbusername}")
private String dbUserName;
#Value("${dbpassword}")
private String dbPassword;
#Bean
#RefreshScope
public DataSource getDatasource() {
return new DatasourceBuilder().create().url(dbUrl).username(dbUserName).password(dbPassword);
}
}
Giving external config file to the application,
java -jar myapplication.jar --spring.config.location=database.properties
I have created a Java thread class to monitor database.properties file changes. Followed https://dzone.com/articles/how-watch-file-system-changes
When there are changes then it makes call to refreshEndPoint.refresh().
In pom.xml,
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
<version>1.5.6.RELEASE</version>
</dependency>
You can use Spring's Dynamic Data Source routing and check if it helps? It's a very old technique and might come handy, if that serves your purpose.
But please note that - this is data source routing and not new data source configuration.
https://spring.io/blog/2007/01/23/dynamic-datasource-routing/
In my project I used multitenancy . Basically I defined several datasources in properties like this:
primary.datasource.url=jdbc:postgresql://localhost:5432/db_name?currentSchema=schema_name
primary.datasource.username=user
primary.datasource.password=password
primary.datasource.driverClassName=org.postgresql.Driver
primary.datasource.driver-class-name=org.postgresql.Driver
secondary.datasource.url=jdbc:postgresql://localhost:5432/other_db?currentSchema=schema
secondary.datasource.username=user
secondary.datasource.password=password
secondary.datasource.driverClassName=org.postgresql.Driver
secondary.datasource.driver-class-name=org.postgresql.Driver
default.datasource.url=jdbc:postgresql://localhost:5432/default_db?currentSchema=public
default.datasource.username=user
default.datasource.password=password
default.datasource.driverClassName=org.postgresql.Driver
default.datasource.driver-class-name=org.postgresql.Driver
then in configuration class defined multiple datasources:
#Bean
#Primary
#ConfigurationProperties(prefix="primary.datasource")
public DataSource primaryDataSource() {
return DataSourceBuilder.create().build();
}
#Bean
#ConfigurationProperties(prefix="secondary.datasource")
public DataSource secondaryDataSource() {
return DataSourceBuilder.create().build();
}
#Bean
#ConfigurationProperties(prefix="default.datasource")
public DataSource defaultDataSource(){
return DataSourceBuilder.create().build();
}
and configured multitenancy basing on this and this article.
Pros:
Easy tenant switch which could be triggered manually or even configured to be triggered on some specific header in request (filters).
Could be cofigured to switch between schemas or databases.
Happens dynamically ( you don't have to restart your beans )
Cons:
You have to define all db possibilities in property file.
You have to turn off schema validation because it will go nuts.
I use Spring Data LDAP and Spring Boot provides out of the box support for an embedded UnboundID server. However, when I use Spring Data LDAP's #Entry annotation, I need to specify a different base in the annotation based on whether I'm using the embedded UnboundID LDAP server, or a remote Active Directory server.
I was attempting to do this with SpEL and profile-based properties by specifying:
#Entry(base = "${ldap.person.base}", ...)
Then I have an application.propreties with ldap.person.base=OU=AD Person Base and an application-embedded.properties with ldap.person.base=OU=Embedded Person Base.
However, the #Entry annotation does not seem to support SpEL evaluation:
javax.naming.InvalidNameException: Invalid name: ${ldap.person.base}
There is an open issue in Spring LDAP to add support for this, but is there any workaround or some other way I can accomplish this until it is supported in Spring LDAP?
I'm not sure I'm following here, but assuming you're using the LDAP auto-configuration in Spring Boot, is it not enough to set the property spring.ldap.base to one or the other (OU=AD Person Base or OU=Embedded Person Base) based on the profile you're using?
Both EmbeddedLdapAutoConfiguration and LdapAutoConfiguration use an LdapProperties object to set various attributes on the LdapContextSource during bean creation, including its base. As far as I can tell, you won't have to define it for each #Entry in your codebase if LdapContextSource.base is set.
If you're not using the auto-configuration, and if I'm correct in my assumptions, you should still be able to create your own LdapContextSource bean and set its base to the desired value based on a Spring property.
Turns out the reason I needed a different base in the first place is because Spring was not setting the base on the ContextSource.
When you let Spring Boot autoconfigure the embedded LDAP server, it creates a ContextSource as such in EmbeddedLdapAutoConfiguration:
#Bean
#DependsOn("directoryServer")
#ConditionalOnMissingBean
public ContextSource ldapContextSource() {
LdapContextSource source = new LdapContextSource();
if (hasCredentials(this.embeddedProperties.getCredential())) {
source.setUserDn(this.embeddedProperties.getCredential().getUsername());
source.setPassword(this.embeddedProperties.getCredential().getPassword());
}
source.setUrls(this.properties.determineUrls(this.environment));
return source;
}
As you can see, nowhere in there does it call source.setBase(). So to solve this, I added a configuration file with #Profile("embedded") and manually created a ContextSource where I set the base myself (I leave off the credentials part because I don't use credentials for the embedded server):
#Configuration
#Profile("embedded")
#EnableConfigurationProperties({ LdapProperties.class })
public class EmbeddedLdapConfig {
private final Environment environment;
private final LdapProperties properties;
public EmbeddedLdapConfig(final Environment environment, final LdapProperties properties) {
this.environment = environment;
this.properties = properties;
}
#Bean
#DependsOn("directoryServer")
public ContextSource ldapContextSource() {
final LdapContextSource source = new LdapContextSource();
source.setUrls(this.properties.determineUrls(this.environment));
source.setBase(this.properties.getBase());
return source;
}
}
Now, I can leave the value of the base attribute in my #Entry the same for both the Active Directory server and the embedded UnboundID server and it works properly.
I specified Spring properties inside the application.properties file. How can i populate those properties from the environment variables?
Here is what I tried, but it doesn't seem to work:
application.properties
spring.datasource.url=jdbc:postgresql://#{ systemProperties['DATABASE_HOST']}:5432/dbname
spring.datasource.username = postgres
spring.datasource.password = postgres
You can refer to environment properties in the same way you refer to Spring properties using ${...} syntax.
In your case:
spring.datasource.url=jdbc:postgresql://${DATABASE_HOST}:5432/dbname
Out of the box, as you know, spring-boot expects its Datasource details to be using a specific set of variable names. Being spring of course you can rework this if you need by a few methods:
1/ If the need to use variables from the environment comes from deployment to a cloud service such as Cloud Foundry or Horuku, there is spring-boot-starter-cloud-connector which handles allot of the plumbing out of the box. A good read is the (Binding to Data Services with Spring Boot in Cloud Foundry article and the Deploying to the cloud docs which walks you thru this
2/ Instead of relying on Spring-Boot's own auto-magical wiring mechanism, you can create a custom configuration bean to override how the DataSource information is populated. A good read explaining the annotations involved can be found here: Spring Java Config Documentation - #Bean Configuration JavaDOC. Based on your example above, here is what I spat out:
#Configuration
public class MyDataSourceConfig {
#Bean
#Primary
public DataSource getDataSource() {
String url = "jdbc:postgresql://" + System.getenv("DATABASE_HOST") + ":5432/dbname";
String username = "postgres";
String password = "postgres";
String driverClassName = "org.postgresql.Driver";
/*
* Create the datasource and return it
*
* You could create the specific DS
* implementation (ie: org.postgresql.ds.PGPoolingDataSource)
* or ask Spring's DataSourceBuilder to autoconfigure it for you,
* whichever works best in your eyes
*/
return DataSourceBuilder
.create()
.url( url )
.username( username )
.password( password )
.driverClassName( driverClassName )
.build();
}
}
Just remember that in spring, you can always override allot of the default behaviours with a little bit of digging!
Hope this helps!
You don't have to. When Spring Boot initializes its environment, it pulls stuff from both the application.properties file and any system-level variables and combines them together. The full list of locations where Spring takes them from is here, specifically points 9) and 10).