Develop Reference

Springboot JDBCTemplate Multiple Database

I have configured two databases (Same schema one is QA and another one is PROD DB) in my Springboot application, The idea here is I don't want to deploy two instances of application to talk to each DB instead on login screen user will select the Database to work with and then application will connect to the database and pull the required data. The question is how can I select database dynamically in my DAO layer using #Qualifier("firstDB") or #Qualifier("secondDB"). Please note that I have user selected database value in Session object so obviously I can't or should not access Session data in my DAO layer as I want to keep DAO separately. Please help me out to choose a DB connection dynamically.
DB Configuration:
#Bean(name = "jdbcTemplate")
#Autowired
public JdbcTemplate jdbcTemplate(#Qualifier("dataSourceOne") DataSource dsCustom) {
return new JdbcTemplate(dsCustom);
}
DAO:
#Autowired
private JdbcTemplate jdbcTemplate;
public TidalAlertsDAOImpl(JdbcTemplate template) {
this.jdbcTemplate = template;
}
Solution:
#Configuration
public class DataSourceConfig {
#Bean(name = "dataSourceOne")
#Primary
public DataSource dataSource() {
return DataSourceBuilder
.create()
.username("USERNAME")
.password("Password")
.url("jdbc:oracle:thin:#localhost.com:1521:DV")
.driverClassName("oracle.jdbc.driver.OracleDriver")
.build();
}
#Bean(name = "dataSourceTwo")
public DataSource dataSourceTwo() {
return DataSourceBuilder
.create()
.username("USERNAME")
.password("Password")
.url("jdbc:oracle:thin:#localhost.com:1521:QA")
.driverClassName("oracle.jdbc.driver.OracleDriver")
.build();
}
#Bean(name = "jdbcTemplate")
#Autowired
public JdbcTemplate jdbcTemplate(#Qualifier("dataSourceOne") DataSource dsCustom) {
return new JdbcTemplate(dsCustom);
}
public JdbcTemplate getJDBCTemplate(){
String environmnet = null;
JdbcTemplate jdbcTemplate = null;
ServletRequestAttributes requestAttributes = (ServletRequestAttributes) RequestContextHolder.getRequestAttributes();
HttpSession session = requestAttributes.getRequest().getSession();
if(session.getAttribute("databaseName") != null){
environmnet = (String) session.getAttribute("databaseName");
if(environmnet.equals("Development - QA")){
System.out.println("Sected Database is:" + environmnet);
jdbcTemplate = new JdbcTemplate(dataSourceTwo());
}else if(environmnet.equals("Sandbox - DV")){
System.out.println("Sected Database is:" + environmnet);
jdbcTemplate = new JdbcTemplate(dataSource());
}
}
return jdbcTemplate;
}

This is a tricky one. Will there be multiple users logging into the application? In that case, you will have to map the user with the selected connection. It would be difficult to inject a connection in the DAO using annotations since database connections are at an application level. You will have to retrieve the connection to be used for that user and then use the corresponding connection.

Prevent changing schema in Spring Boot with two DBs

I have two DBs in my Spring Boot app, one configured with appliation.properties:
spring.secondDatasource.url=jdbc:mysql://10.10.10:3306/db1
spring.secondDatasource.username=user
spring.secondDatasource.password=pass
spring.jpa.database-platform = org.hibernate.dialect.MySQL5Dialect
spring.datasource.driverClassName=com.mysql.jdbc.Driver
spring.jpa.hibernate.ddl-auto = update
and another via DataSource:
#Configuration
public class SecondDbConnectionConfig {
#Bean
public DriverManagerDataSource dataSource() {
DriverManagerDataSource dataSource = new DriverManagerDataSource();
dataSource.setUrl("jdbc:mysql://20.20.20:3306/db2");
dataSource.setDriverClassName("com.mysql.jdbc.Driver");
dataSource.setUsername("user");
dataSource.setPassword("pass");
return dataSource;
}
}
When I run the app, both schemas are updated with my domain model.
I want to update a domain model only for app.prop configured DB.
As to DataSource configured DB, I don't want to make any changes, just read.
How to fix this config?

Don't use DriverManagerDataSource for production. It's just suitable for your testing environment. (It does not have connection pool, just create new connection on the fly)
Your datasource uses your JpaProperties config from your spring.jpa.* properties. So you need to override it:
#Bean(name = "your-entity-manager-factory")
public LocalContainerEntityManagerFactoryBean getEntityManagerFactory(EntityManagerFactoryBuilder builder,
#Qualifier("testdatasource") DataSource dataSource, // Here you must annotate your DriverManagerDataSource bean with #Bean("testdatasource")
JpaProperties jpaProperties) {
// Here you clone and modify JpaProperties
Map<String, String> hibernateConfig = jpaProperties.getHibernateProperties(dataSource);
hibernateConfig.remove("hibernate.hbm2ddl.auto");
return builder
.dataSource(dataSource)
.persistenceUnit("testPu")
.properties(hibernateConfig)
.build();
}
You may config additional transaction manager.

How to create database table with spring boot after change the data source URL

I use spring-boot with DATAJPA repository and two datasource. One datasource connect to a MySQL DB and the 2nd to a file based embedded H2 DB.
On the H2 DB i want change the URL during my application ist running. The URL changing work and after change the URL to an not existing DB spring create the DB. But the tables will not create and spring give me an error like Table "SYMBOL" not found; SQL statement:.
When i restart my app now then spring create the tables with the new URL and everything work fine.
How i can create the tables after change the URL?
To change the URL i write my own datasource and build it new on each connection.
public class ProjectDataSource extends AbstractDataSource {
#Autowired ToolService toolService;
#Override
public Connection getConnection() throws SQLException {
return determineTargetDataSource().getConnection();
}
#Override
public Connection getConnection(String username, String password) throws SQLException {
return determineTargetDataSource().getConnection(username, password);
}
#ConfigurationProperties(prefix = "project.datasource")
private DataSource determineTargetDataSource() {
String projectName = toolService.get().getProjectName();
String url = "jdbc:h2:file:../db/" + projectName;
return DataSourceBuilder
.create()
.url(url)
.build();
}
}
Here the configuration
#Configuration
#EnableTransactionManagement
#EnableJpaRepositories(
entityManagerFactoryRef = "projectEntityManagerFactory",
transactionManagerRef = "projectTransactionManager",
basePackages = {"at.ltw.test.bz.model.project"})
public class ProjectDbConfiguration {
#Bean(name = "projectDataSource")
public DataSource dataSource() {
return new ProjectDataSource();
}
#Bean(name = "projectEntityManagerFactory")
public LocalContainerEntityManagerFactoryBean
barEntityManagerFactory(
EntityManagerFactoryBuilder builder,
#Qualifier("projectDataSource") DataSource dataSource
) {
HibernateJpaVendorAdapter jpaVendorAdapter = new HibernateJpaVendorAdapter();
jpaVendorAdapter.setGenerateDdl(true);
return builder
.dataSource(dataSource)
.packages("at.ltw.test.bz.model.project")
.persistenceUnit("bzProject")
.properties(jpaVendorAdapter.getJpaPropertyMap())
.build();
}
#Bean(name = "projectTransactionManager")
public PlatformTransactionManager barTransactionManager(
#Qualifier("projectEntityManagerFactory") EntityManagerFactory projectEntityManagerFactory) {
return new JpaTransactionManager(projectEntityManagerFactory);
}
}
Here my application.properties
#jpa
spring.jpa.hibernate.ddl-auto=update
spring.jpa.generate-ddl=true
#tool database
spring.datasource.driver-class-name=com.mysql.jdbc.Driver
spring.datasource.url=jdbc:mysql://localhost/ltw?useSSL=false
spring.datasource.username=ltw
spring.datasource.password=password
#project database
project.datasource.driver-class-name=org.h2.Driver
#logging
logging.level.at.ltw = trace
Hope somebody can help me and sorry for my bad English ...

JPA Will not help you because it is only at init state that DDL are checked.
You need to use something like flywayDB or liquibase or simple SQL file to generate the new DB.
Anyway, you code is wrong, and I'm surprise you don't have error.
#Bean(name = "projectDataSource")
public DataSource dataSource() {
return new ProjectDataSource();
}
There is no injection but a simple new object and ...
public class ProjectDataSource extends AbstractDataSource {
#Autowired ToolService toolService;
#ConfigurationProperties(prefix = "project.datasource")
private DataSource determineTargetDataSource() {
String projectName = toolService.get().getProjectName();
...
}
toolService will be null, so determineTargetDataSouce will fail with a null pointer exception.

Spring Boot Batch With Spring Data Write meta data in different Schema (in memory: HSQL or H2)

I am writring a batch using the following thecnologies:
Spring Boot to run the application : V1.5.3.RELEASE
Spring Batch with Spring Batch Config: spring-batch-infrastructure V3.0.7.RELEASE
Spring Data for my generic DAO to the business database: >spring-data-jpa V1.11.3.RELEASE
My datasource to oracle datbase is HikariDataSource :
#Qualifier("dataSource")
#Bean(destroyMethod = "close")
#Primary
public HikariDataSource dataSource() throws SQLException {
return buildDataSource();
}
public HikariDataSource buildDataSource() throws SQLException {
HikariDataSource ds = new HikariDataSource();
ds.setMaximumPoolSize(poolSize);
ds.setDriverClassName(driverClassName);
ds.setJdbcUrl(jdbcUrl);
ds.setUsername(userName);
ds.setPassword(password);
ds.setConnectionTestQuery("SELECT 1 from DUAL");
ds.addDataSourceProperty("hibernate.show_sql", showSQL);
ds.addDataSourceProperty("hibernate.use_sql_comments", useSQLComment);
ds.addDataSourceProperty("hibernate.format_sql", formatSQL);
ds.addDataSourceProperty("hibernate.ddl-auto", "none");
return ds;
}
I want to write my meta data in another database (in memory HSQL or H2 for example) but i can't find a way because the context is writing the meta data in the same database.
The only way is to define a TransactionManager and an EntityManager and enable them to my DAO :
#Bean
PlatformTransactionManager businessTransactionManager() throws SQLException {
return new JpaTransactionManager(businessEntityManagerFactory().getObject());
}
#Bean
LocalContainerEntityManagerFactoryBean businessEntityManagerFactory() throws SQLException {
HibernateJpaVendorAdapter jpaVendorAdapter = new HibernateJpaVendorAdapter();
jpaVendorAdapter.setGenerateDdl(true);
LocalContainerEntityManagerFactoryBean factoryBean = new LocalContainerEntityManagerFactoryBean();
factoryBean.setDataSource(dataSource());
factoryBean.setJpaVendorAdapter(jpaVendorAdapter);
factoryBean.setPackagesToScan("package.of.business.model", "package.of.business.data");
return factoryBean;
}
and in my batch configuration i add :
#EnableJpaRepositories(entityManagerFactoryRef = "businessEntityManagerFactory",
transactionManagerRef = "businessTransactionManager", basePackages = {"package.of.business.model",
"package.of.business.data"})
This way it works after i define the spring default dataSource in my app.properties:
spring.datasource.url=jdbc:h2:mem:test
spring.datasource.driverClassName=org.h2.Driver
spring.datasource.username=sa
spring.datasource.password=
spring.jpa.hibernate.ddl-auto=update
What i really want to do is the exact opposite of this, i want that the default database is the business one and i want to override the datasource that writes the meta data but i can't find a way. I even tried to make a custom BatchConfigurer:
CustomBatchConfigurer extends DefaultBatchConfigurer
It works only for my meta data after i disable the initialization of my spring data for the default datasource but it doesn't write anything in my oracle business database :
batch.data.source.init=false
spring.batch.initializer.enabled=false
spring.batch.initialize.enabled=false
spring.datasource.initialize=false
spring.datasource.continue-on-error=true
Does any one have any idea how this could be done?

You'll need to create a custom implementation of the BatchConfigurer (typically by extending DefaultbatchConfigurer. That will allow you to configure the batch DataSource explicitly.
You can read more about the BatchConfigurer in the documentation here: http://docs.spring.io/spring-batch/apidocs/org/springframework/batch/core/configuration/annotation/BatchConfigurer.html

Configure DataSource programmatically in Spring Boot

With Spring Boot I can instantiate a JdbcTemplate with the following:
Code:
#Autowired
private JdbcTemplate jdbcTemplate;
Properties:
spring.datasource.url=jdbc:postgresql://my_url:my_port/my_other_stuff
spring.datasource.username=my_user_name
spring.datasource.password=my_password
spring.datasource.driver-class-name=org.postgresql.Driver
This create a DataSource of class: org.apache.tomcat.jdbc.pool.DataSource
How do I set the DataSource username/password programmatically?
We have a policy not to store credentials in plain text and I have to use a specific credential provider where I work.

You can use DataSourceBuilder if you are using jdbc starter. Also, in order to override the default autoconfiguration bean you need to mark your bean as a #Primary
In my case I have properties starting with datasource.postgres prefix.
E.g
#ConfigurationProperties(prefix = "datasource.postgres")
#Bean
#Primary
public DataSource dataSource() {
return DataSourceBuilder
.create()
.build();
}
If it is not feasible for you, then you can use
#Bean
#Primary
public DataSource dataSource() {
return DataSourceBuilder
.create()
.username("")
.password("")
.url("")
.driverClassName("")
.build();
}

My project of spring-boot has run normally according to your assistance. The yaml datasource configuration is:
spring:
# (DataSourceAutoConfiguration & DataSourceProperties)
datasource:
name: ds-h2
url: jdbc:h2:D:/work/workspace/fdata;DATABASE_TO_UPPER=false
username: h2
password: h2
driver-class: org.h2.Driver
Custom DataSource
#Configuration
#Component
public class DataSourceBean {
#ConfigurationProperties(prefix = "spring.datasource")
#Bean
#Primary
public DataSource getDataSource() {
return DataSourceBuilder
.create()
// .url("jdbc:h2:D:/work/workspace/fork/gs-serving-web-content/initial/data/fdata;DATABASE_TO_UPPER=false")
// .username("h2")
// .password("h2")
// .driverClassName("org.h2.Driver")
.build();
}
}

All you need to do is annotate a method that returns a DataSource with #Bean.
A complete working example follows.
#Bean
public DataSource dataSource() {
DataSourceBuilder dataSourceBuilder = DataSourceBuilder.create();
dataSourceBuilder.url(dbUrl);
dataSourceBuilder.username(username);
dataSourceBuilder.password(password);
return dataSourceBuilder.build();
}

If you're using latest spring boot (with jdbc starter and Hikari) you'll run into:
java.lang.IllegalArgumentException: jdbcUrl is required with driverClassName.
To solve this:
In your application.properties:
datasource.oracle.url=youroracleurl
In your application define as bean (#Primary is mandatory!):
#Bean
#Primary
#ConfigurationProperties("datasource.oracle")
public DataSourceProperties getDatasourceProperties() {
return new DataSourceProperties();
}
#Bean
#ConfigurationProperties("datasource.oracle")
public DataSource getDatasource() {
return getDatasourceProperties().initializeDataSourceBuilder()
.username("username")
.password("password")
.build();
}

If you want more datesource configs e.g.
spring.datasource.test-while-idle=true
spring.datasource.time-between-eviction-runs-millis=30000
spring.datasource.validation-query=select 1
you could use below code
#Bean
public DataSource dataSource() {
DataSource dataSource = new DataSource(); // org.apache.tomcat.jdbc.pool.DataSource;
dataSource.setDriverClassName(driverClassName);
dataSource.setUrl(url);
dataSource.setUsername(username);
dataSource.setPassword(password);
dataSource.setTestWhileIdle(testWhileIdle);
dataSource.setTimeBetweenEvictionRunsMillis(timeBetweenEvictionRunsMills);
dataSource.setValidationQuery(validationQuery);
return dataSource;
}
refer: Spring boot jdbc Connection

As an alternative way you can use DriverManagerDataSource such as:
public DataSource getDataSource(DBInfo db) {
DriverManagerDataSource dataSource = new DriverManagerDataSource();
dataSource.setUsername(db.getUsername());
dataSource.setPassword(db.getPassword());
dataSource.setUrl(db.getUrl());
dataSource.setDriverClassName(db.getDriverClassName());
return dataSource;
}
However be careful about using it, because:
NOTE: This class is not an actual connection pool; it does not
actually pool Connections. It just serves as simple replacement for a
full-blown connection pool, implementing the same standard interface,
but creating new Connections on every call. reference

for springboot 2.1.7 working with url seems not to work. change with jdbcUrl instead.
In properties:
security:
datasource:
jdbcUrl: jdbc:mysql://ip:3306/security
username: user
password: pass
In java:
#ConfigurationProperties(prefix = "security.datasource")
#Bean("dataSource")
#Primary
public DataSource dataSource(){
return DataSourceBuilder
.create()
.build();
}

I customized Tomcat DataSource in Spring-Boot 2.
Dependency versions:
spring-boot: 2.1.9.RELEASE
tomcat-jdbc: 9.0.20
May be it will be useful for somebody.
application.yml
spring:
datasource:
driver-class-name: org.postgresql.Driver
type: org.apache.tomcat.jdbc.pool.DataSource
url: jdbc:postgresql://${spring.datasource.database.host}:${spring.datasource.database.port}/${spring.datasource.database.name}
database:
host: localhost
port: 5432
name: rostelecom
username: postgres
password: postgres
tomcat:
validation-query: SELECT 1
validation-interval: 30000
test-on-borrow: true
remove-abandoned: true
remove-abandoned-timeout: 480
test-while-idle: true
time-between-eviction-runs-millis: 60000
log-validation-errors: true
log-abandoned: true
Java
#Bean
#Primary
#ConfigurationProperties("spring.datasource.tomcat")
public PoolConfiguration postgresDataSourceProperties() {
return new PoolProperties();
}
#Bean(name = "primaryDataSource")
#Primary
#Qualifier("primaryDataSource")
#ConfigurationProperties(prefix = "spring.datasource")
public DataSource primaryDataSource() {
PoolConfiguration properties = postgresDataSourceProperties();
return new DataSource(properties);
}
The main reason why it had been done is several DataSources in application and one of them it is necessary to mark as a #Primary.