With Spring Boot I can instantiate a JdbcTemplate with the following:
Code:
#Autowired
private JdbcTemplate jdbcTemplate;
Properties:
spring.datasource.url=jdbc:postgresql://my_url:my_port/my_other_stuff
spring.datasource.username=my_user_name
spring.datasource.password=my_password
spring.datasource.driver-class-name=org.postgresql.Driver
This create a DataSource of class: org.apache.tomcat.jdbc.pool.DataSource
How do I set the DataSource username/password programmatically?
We have a policy not to store credentials in plain text and I have to use a specific credential provider where I work.
You can use DataSourceBuilder if you are using jdbc starter. Also, in order to override the default autoconfiguration bean you need to mark your bean as a #Primary
In my case I have properties starting with datasource.postgres prefix.
E.g
#ConfigurationProperties(prefix = "datasource.postgres")
#Bean
#Primary
public DataSource dataSource() {
return DataSourceBuilder
.create()
.build();
}
If it is not feasible for you, then you can use
#Bean
#Primary
public DataSource dataSource() {
return DataSourceBuilder
.create()
.username("")
.password("")
.url("")
.driverClassName("")
.build();
}
My project of spring-boot has run normally according to your assistance. The yaml datasource configuration is:
spring:
# (DataSourceAutoConfiguration & DataSourceProperties)
datasource:
name: ds-h2
url: jdbc:h2:D:/work/workspace/fdata;DATABASE_TO_UPPER=false
username: h2
password: h2
driver-class: org.h2.Driver
Custom DataSource
#Configuration
#Component
public class DataSourceBean {
#ConfigurationProperties(prefix = "spring.datasource")
#Bean
#Primary
public DataSource getDataSource() {
return DataSourceBuilder
.create()
// .url("jdbc:h2:D:/work/workspace/fork/gs-serving-web-content/initial/data/fdata;DATABASE_TO_UPPER=false")
// .username("h2")
// .password("h2")
// .driverClassName("org.h2.Driver")
.build();
}
}
All you need to do is annotate a method that returns a DataSource with #Bean.
A complete working example follows.
#Bean
public DataSource dataSource() {
DataSourceBuilder dataSourceBuilder = DataSourceBuilder.create();
dataSourceBuilder.url(dbUrl);
dataSourceBuilder.username(username);
dataSourceBuilder.password(password);
return dataSourceBuilder.build();
}
If you're using latest spring boot (with jdbc starter and Hikari) you'll run into:
java.lang.IllegalArgumentException: jdbcUrl is required with driverClassName.
To solve this:
In your application.properties:
datasource.oracle.url=youroracleurl
In your application define as bean (#Primary is mandatory!):
#Bean
#Primary
#ConfigurationProperties("datasource.oracle")
public DataSourceProperties getDatasourceProperties() {
return new DataSourceProperties();
}
#Bean
#ConfigurationProperties("datasource.oracle")
public DataSource getDatasource() {
return getDatasourceProperties().initializeDataSourceBuilder()
.username("username")
.password("password")
.build();
}
If you want more datesource configs e.g.
spring.datasource.test-while-idle=true
spring.datasource.time-between-eviction-runs-millis=30000
spring.datasource.validation-query=select 1
you could use below code
#Bean
public DataSource dataSource() {
DataSource dataSource = new DataSource(); // org.apache.tomcat.jdbc.pool.DataSource;
dataSource.setDriverClassName(driverClassName);
dataSource.setUrl(url);
dataSource.setUsername(username);
dataSource.setPassword(password);
dataSource.setTestWhileIdle(testWhileIdle);
dataSource.setTimeBetweenEvictionRunsMillis(timeBetweenEvictionRunsMills);
dataSource.setValidationQuery(validationQuery);
return dataSource;
}
refer: Spring boot jdbc Connection
As an alternative way you can use DriverManagerDataSource such as:
public DataSource getDataSource(DBInfo db) {
DriverManagerDataSource dataSource = new DriverManagerDataSource();
dataSource.setUsername(db.getUsername());
dataSource.setPassword(db.getPassword());
dataSource.setUrl(db.getUrl());
dataSource.setDriverClassName(db.getDriverClassName());
return dataSource;
}
However be careful about using it, because:
NOTE: This class is not an actual connection pool; it does not
actually pool Connections. It just serves as simple replacement for a
full-blown connection pool, implementing the same standard interface,
but creating new Connections on every call. reference
for springboot 2.1.7 working with url seems not to work. change with jdbcUrl instead.
In properties:
security:
datasource:
jdbcUrl: jdbc:mysql://ip:3306/security
username: user
password: pass
In java:
#ConfigurationProperties(prefix = "security.datasource")
#Bean("dataSource")
#Primary
public DataSource dataSource(){
return DataSourceBuilder
.create()
.build();
}
I customized Tomcat DataSource in Spring-Boot 2.
Dependency versions:
spring-boot: 2.1.9.RELEASE
tomcat-jdbc: 9.0.20
May be it will be useful for somebody.
application.yml
spring:
datasource:
driver-class-name: org.postgresql.Driver
type: org.apache.tomcat.jdbc.pool.DataSource
url: jdbc:postgresql://${spring.datasource.database.host}:${spring.datasource.database.port}/${spring.datasource.database.name}
database:
host: localhost
port: 5432
name: rostelecom
username: postgres
password: postgres
tomcat:
validation-query: SELECT 1
validation-interval: 30000
test-on-borrow: true
remove-abandoned: true
remove-abandoned-timeout: 480
test-while-idle: true
time-between-eviction-runs-millis: 60000
log-validation-errors: true
log-abandoned: true
Java
#Bean
#Primary
#ConfigurationProperties("spring.datasource.tomcat")
public PoolConfiguration postgresDataSourceProperties() {
return new PoolProperties();
}
#Bean(name = "primaryDataSource")
#Primary
#Qualifier("primaryDataSource")
#ConfigurationProperties(prefix = "spring.datasource")
public DataSource primaryDataSource() {
PoolConfiguration properties = postgresDataSourceProperties();
return new DataSource(properties);
}
The main reason why it had been done is several DataSources in application and one of them it is necessary to mark as a #Primary.
Related
I have a springboot app that uses these versions:
SpringBoot: 2.3.1
SpringCloud: Hoxton.SR5
SpringData: Neumann-SR1
I've used the custom class as described here (answer from #Arun): Configuring Spring Cloud Vault Config to pull from a location other than /secret
It doesn't even seem to be picking up the vault configs.
I have a bootstrap.yml with the following:
spring:
cloud:
# Vault configurations
vault:
generic:
enabled: false
uri: https://${URI}
authentication: TOKEN
token: ${VAULT_TOKEN}
config:
discovery:
enabled: true
I'm trying to bring it up locally, so I have my application.yml as follows:
spring:
datasource:
url: jdbc:postgresql://localhost:8157/postgres
This is where I'm trying to inject the values from vault into my DataSource:
#Profile(value = {"local", "dev", "stg", "prd"})
#Configuration
public class DatabaseConfig {
#Autowired
DatabaseCredentials config;
#Bean
#Primary
public DataSource dataSource() {
return DataSourceBuilder
.create()
.username(config.getUsername())
.url(config.getUrl())
.password(config.getPassword())
.driverClassName("org.postgresql.Driver")
.build();
}
}
When the app starts up, it DatabaseCredentials is empty.
The other way I've done it is like this:
public class DatabaseConfig {
#Value("${ccm.database.username}")
String username;
#Value("${ccm.database.password}")
String password;
#Value("${spring.datasource.url}")
String url;
#Bean
#Primary
public DataSource dataSource() {
return DataSourceBuilder
.create()
.username(username)
.url(url)
.password(password)
.driverClassName("org.postgresql.Driver")
.build();
}
}
This has also come up empty saying it can't find a value for ccm.database.username.
What am I doing wrong?
You are missing the annotations on DatabaseConfig.java
Which will be something like this.
#Component
#ConfigurationProperties("ccm.database")
So it becomes
#ConfigurationProperties("ccm.database")
#Component
public class DatabaseCredentials {
#Value("${ccm.database.username}")
String username;
#Value("${ccm.database.password}")
String password;
#Value("${spring.datasource.url}")
String url;
// Getters and setters for all properties go here
}
To access the config
#Configuration
public class DatabaseConfig {
#Autowired
DatabaseCredentials config;
#Bean
#Primary
public DataSource dataSource() {
return DataSourceBuilder
.create()
.username(config.getUsername())
.url(config.getUrl())
.password(config.getPassword())
.driverClassName("org.postgresql.Driver")
.build();
}
}
Spring Cloud Vault gives you DEBUG info which helped me figure out the issue. I ended up defining the path with kv instead of a custom VaultConfigurer bean.
kv:
enabled: true
backend: tcds/kv/app-name
application-name: ccm
This creates the path as follows: tcds/kv/app-name/ccm/${PROFILE}.
I assumed that all secrets underneath this path will be picked up and I originally had multiple folders underneath my path of tcds/kv/app-name/ccm/dev.
For example: ../ccm/dev/database, ../ccm/dev/other-creds.
However, it seems that spring vault wasn't going to the nested paths for secrets and was looking for a file with all credentials there.
So, I removed the dev folder and replaced it with a dev file with entries such as:
{
"database.username": "blah",
"database.password": "moreblah",
"other-creds.user": "dummy"
}
I am able to get credentials from vault by using the DatabaseConfig.java class alone and not having to use DatabaseCredentials.java with the #ConfigurationProperties annotation.
Spring cloud supports import standart spring boot configuration from vault.
use two dependendences for automatic configuration:
org.springframework.cloud:spring-cloud-starter-vault-config
org.springframework.cloud:spring-cloud-vault-config-databases
See:
https://medium.com/digitalfrontiers/manage-your-database-accounts-with-spring-cloud-vault-config-48cecb837a36
I have two DBs in my Spring Boot app, one configured with appliation.properties:
spring.secondDatasource.url=jdbc:mysql://10.10.10:3306/db1
spring.secondDatasource.username=user
spring.secondDatasource.password=pass
spring.jpa.database-platform = org.hibernate.dialect.MySQL5Dialect
spring.datasource.driverClassName=com.mysql.jdbc.Driver
spring.jpa.hibernate.ddl-auto = update
and another via DataSource:
#Configuration
public class SecondDbConnectionConfig {
#Bean
public DriverManagerDataSource dataSource() {
DriverManagerDataSource dataSource = new DriverManagerDataSource();
dataSource.setUrl("jdbc:mysql://20.20.20:3306/db2");
dataSource.setDriverClassName("com.mysql.jdbc.Driver");
dataSource.setUsername("user");
dataSource.setPassword("pass");
return dataSource;
}
}
When I run the app, both schemas are updated with my domain model.
I want to update a domain model only for app.prop configured DB.
As to DataSource configured DB, I don't want to make any changes, just read.
How to fix this config?
Don't use DriverManagerDataSource for production. It's just suitable for your testing environment. (It does not have connection pool, just create new connection on the fly)
Your datasource uses your JpaProperties config from your spring.jpa.* properties. So you need to override it:
#Bean(name = "your-entity-manager-factory")
public LocalContainerEntityManagerFactoryBean getEntityManagerFactory(EntityManagerFactoryBuilder builder,
#Qualifier("testdatasource") DataSource dataSource, // Here you must annotate your DriverManagerDataSource bean with #Bean("testdatasource")
JpaProperties jpaProperties) {
// Here you clone and modify JpaProperties
Map<String, String> hibernateConfig = jpaProperties.getHibernateProperties(dataSource);
hibernateConfig.remove("hibernate.hbm2ddl.auto");
return builder
.dataSource(dataSource)
.persistenceUnit("testPu")
.properties(hibernateConfig)
.build();
}
You may config additional transaction manager.
I use spring-boot with DATAJPA repository and two datasource. One datasource connect to a MySQL DB and the 2nd to a file based embedded H2 DB.
On the H2 DB i want change the URL during my application ist running. The URL changing work and after change the URL to an not existing DB spring create the DB. But the tables will not create and spring give me an error like Table "SYMBOL" not found; SQL statement:.
When i restart my app now then spring create the tables with the new URL and everything work fine.
How i can create the tables after change the URL?
To change the URL i write my own datasource and build it new on each connection.
public class ProjectDataSource extends AbstractDataSource {
#Autowired ToolService toolService;
#Override
public Connection getConnection() throws SQLException {
return determineTargetDataSource().getConnection();
}
#Override
public Connection getConnection(String username, String password) throws SQLException {
return determineTargetDataSource().getConnection(username, password);
}
#ConfigurationProperties(prefix = "project.datasource")
private DataSource determineTargetDataSource() {
String projectName = toolService.get().getProjectName();
String url = "jdbc:h2:file:../db/" + projectName;
return DataSourceBuilder
.create()
.url(url)
.build();
}
}
Here the configuration
#Configuration
#EnableTransactionManagement
#EnableJpaRepositories(
entityManagerFactoryRef = "projectEntityManagerFactory",
transactionManagerRef = "projectTransactionManager",
basePackages = {"at.ltw.test.bz.model.project"})
public class ProjectDbConfiguration {
#Bean(name = "projectDataSource")
public DataSource dataSource() {
return new ProjectDataSource();
}
#Bean(name = "projectEntityManagerFactory")
public LocalContainerEntityManagerFactoryBean
barEntityManagerFactory(
EntityManagerFactoryBuilder builder,
#Qualifier("projectDataSource") DataSource dataSource
) {
HibernateJpaVendorAdapter jpaVendorAdapter = new HibernateJpaVendorAdapter();
jpaVendorAdapter.setGenerateDdl(true);
return builder
.dataSource(dataSource)
.packages("at.ltw.test.bz.model.project")
.persistenceUnit("bzProject")
.properties(jpaVendorAdapter.getJpaPropertyMap())
.build();
}
#Bean(name = "projectTransactionManager")
public PlatformTransactionManager barTransactionManager(
#Qualifier("projectEntityManagerFactory") EntityManagerFactory projectEntityManagerFactory) {
return new JpaTransactionManager(projectEntityManagerFactory);
}
}
Here my application.properties
#jpa
spring.jpa.hibernate.ddl-auto=update
spring.jpa.generate-ddl=true
#tool database
spring.datasource.driver-class-name=com.mysql.jdbc.Driver
spring.datasource.url=jdbc:mysql://localhost/ltw?useSSL=false
spring.datasource.username=ltw
spring.datasource.password=password
#project database
project.datasource.driver-class-name=org.h2.Driver
#logging
logging.level.at.ltw = trace
Hope somebody can help me and sorry for my bad English ...
JPA Will not help you because it is only at init state that DDL are checked.
You need to use something like flywayDB or liquibase or simple SQL file to generate the new DB.
Anyway, you code is wrong, and I'm surprise you don't have error.
#Bean(name = "projectDataSource")
public DataSource dataSource() {
return new ProjectDataSource();
}
There is no injection but a simple new object and ...
public class ProjectDataSource extends AbstractDataSource {
#Autowired ToolService toolService;
#ConfigurationProperties(prefix = "project.datasource")
private DataSource determineTargetDataSource() {
String projectName = toolService.get().getProjectName();
...
}
toolService will be null, so determineTargetDataSouce will fail with a null pointer exception.
I am writring a batch using the following thecnologies:
Spring Boot to run the application : V1.5.3.RELEASE
Spring Batch with Spring Batch Config: spring-batch-infrastructure V3.0.7.RELEASE
Spring Data for my generic DAO to the business database: >spring-data-jpa V1.11.3.RELEASE
My datasource to oracle datbase is HikariDataSource :
#Qualifier("dataSource")
#Bean(destroyMethod = "close")
#Primary
public HikariDataSource dataSource() throws SQLException {
return buildDataSource();
}
public HikariDataSource buildDataSource() throws SQLException {
HikariDataSource ds = new HikariDataSource();
ds.setMaximumPoolSize(poolSize);
ds.setDriverClassName(driverClassName);
ds.setJdbcUrl(jdbcUrl);
ds.setUsername(userName);
ds.setPassword(password);
ds.setConnectionTestQuery("SELECT 1 from DUAL");
ds.addDataSourceProperty("hibernate.show_sql", showSQL);
ds.addDataSourceProperty("hibernate.use_sql_comments", useSQLComment);
ds.addDataSourceProperty("hibernate.format_sql", formatSQL);
ds.addDataSourceProperty("hibernate.ddl-auto", "none");
return ds;
}
I want to write my meta data in another database (in memory HSQL or H2 for example) but i can't find a way because the context is writing the meta data in the same database.
The only way is to define a TransactionManager and an EntityManager and enable them to my DAO :
#Bean
PlatformTransactionManager businessTransactionManager() throws SQLException {
return new JpaTransactionManager(businessEntityManagerFactory().getObject());
}
#Bean
LocalContainerEntityManagerFactoryBean businessEntityManagerFactory() throws SQLException {
HibernateJpaVendorAdapter jpaVendorAdapter = new HibernateJpaVendorAdapter();
jpaVendorAdapter.setGenerateDdl(true);
LocalContainerEntityManagerFactoryBean factoryBean = new LocalContainerEntityManagerFactoryBean();
factoryBean.setDataSource(dataSource());
factoryBean.setJpaVendorAdapter(jpaVendorAdapter);
factoryBean.setPackagesToScan("package.of.business.model", "package.of.business.data");
return factoryBean;
}
and in my batch configuration i add :
#EnableJpaRepositories(entityManagerFactoryRef = "businessEntityManagerFactory",
transactionManagerRef = "businessTransactionManager", basePackages = {"package.of.business.model",
"package.of.business.data"})
This way it works after i define the spring default dataSource in my app.properties:
spring.datasource.url=jdbc:h2:mem:test
spring.datasource.driverClassName=org.h2.Driver
spring.datasource.username=sa
spring.datasource.password=
spring.jpa.hibernate.ddl-auto=update
What i really want to do is the exact opposite of this, i want that the default database is the business one and i want to override the datasource that writes the meta data but i can't find a way. I even tried to make a custom BatchConfigurer:
CustomBatchConfigurer extends DefaultBatchConfigurer
It works only for my meta data after i disable the initialization of my spring data for the default datasource but it doesn't write anything in my oracle business database :
batch.data.source.init=false
spring.batch.initializer.enabled=false
spring.batch.initialize.enabled=false
spring.datasource.initialize=false
spring.datasource.continue-on-error=true
Does any one have any idea how this could be done?
You'll need to create a custom implementation of the BatchConfigurer (typically by extending DefaultbatchConfigurer. That will allow you to configure the batch DataSource explicitly.
You can read more about the BatchConfigurer in the documentation here: http://docs.spring.io/spring-batch/apidocs/org/springframework/batch/core/configuration/annotation/BatchConfigurer.html
I am trying to use datasource with spring boot.
In dev the application will run as spring bootrun. But in Test / Prod it will be external tomcat.
So in dev property file I am using Datasource details:
spring.community_ds.url=xxx
spring.community_ds.username=xxx
spring.community_ds.password=xxx
spring.community_ds.driver-class-name=oracle.jdbc.OracleDriver
spring.community_ds.driverClassName=oracle.jdbc.OracleDriver
And in TEST/PROD want to use JNDI
spring.community_ds.jndi-name=xxxx
Now in code I am trying to get the DataSource to create JdbcTemplate
#ConfigurationProperties(prefix = "spring.community_ds")
#Bean(name = "communityDb")
public DataSource communityDbDataSource() {
DataSource ds = DataSourceBuilder.create().build();
//Getting error db url is Null here
return ds;
}
#Bean(name = "communityDbTemplate")
public JdbcTemplate communityDbTemplate(#Qualifier("communityDb") DataSource communityDb) {
return new JdbcTemplate(communityDb);
}
I also tried :
#Configuration
public class DatabaseConfig {
#Autowired
Environment env;
#ConfigurationProperties(prefix = "spring.datasource")
#Bean
#Primary
public DataSource getDataSource() {
DataSource dsp = DataSourceBuilder
.create().build();
System.out.println(env.getProperty("spring.community_ds.url"));
System.out.println(env.getProperty("spring.datasource.url"));
return dsp;
}
}
Here also I saw env.getProperty is printing the data properly, but when I debugged inside DataSourceBuilder the url/username etc are null.
In gradle first I used:
compile("org.springframework:spring-jdbc")
Which was giving some error hence used:
compile("org.springframework.boot:spring-boot-starter-jdbc")