I want to connect multiple mysql db in my spring boot application. The thing is in my application, one of the db is used as an entity while from other db, I am fetching data in query form. So I want that whenever I write a custom query, it should take from one db while whenever I use repository methods, it should use another one.
Change your application.properties file as :
#first db
spring.datasource.url = [url]
spring.datasource.username = [username]
spring.datasource.password = [password]
spring.datasource.driverClassName = oracle.jdbc.OracleDriver
#second db ...
spring.secondDatasource.url = [url]
spring.secondDatasource.username = [username]
spring.secondDatasource.password = [password]
spring.secondDatasource.driverClassName = oracle.jdbc.OracleDriver
And Change your Configuration file i.e add following beans :
#Bean
#Primary
#ConfigurationProperties(prefix="spring.datasource")
public DataSource primaryDataSource() {
return DataSourceBuilder.create().build();
}
#Bean
#ConfigurationProperties(prefix="spring.secondDatasource")
public DataSource secondaryDataSource() {
return DataSourceBuilder.create().build();
}
Related
I am working with spring boot. I have properties defined in application.yml.
spring:
datasource:
username: username
password: password
username and password values are stored externally which program fetches during startup. let's say the bean which fetches them during startup is dbConfig
How can I inject values from dbConfgig to application.yml?
I am using spring-data-jpa autoconfigure which automatically connects to database at startup. I want these values to be loaded to application.yml before spring connects to database.
There is no need to inject the user/password in application.yml. You can set them programmatically like this:
#Configuration
public class DataSourceConfig {
#Bean
public DataSource getDataSource() {
DataSourceBuilder dataSourceBuilder = DataSourceBuilder.create();
dataSourceBuilder.driverClassName("org.h2.Driver");
dataSourceBuilder.url("jdbc:h2:mem:test");
// Take the values from external source, then set them
dataSourceBuilder.username("username");
dataSourceBuilder.password("password");
return dataSourceBuilder.build();
}
}
I think that first, you must create a thread to detect the change at your db Config file and then may you must re-init your bean (data source) to make your change effect.
See:
how-to-reinitialize-a-spring-bean
You may also try spring cloud to store properties. And you can then use with the help of placeholders.
https://cloud.spring.io/spring-cloud-config/reference/html/
Currently I have a setup like below. On running the batch job locally the job will create the necessary metadata tables automatically using the data-source property values since initialize-schema is set to always. Liquibase will also run and create any tables listed in its changelog.
Here is my application.yml file
spring:
batch:
initialize-schema: always
job:
enabled: true
liquibase:
url: db_url
user: deploy_user
password: deploy_pass
change-log: classpath:db/changelog/db.changelog-master.yaml
enabled: true
data-source:
mysql:
user: r_user
password: r_pass
jdbc-url: db_url
Here is my db.changelog-master.yaml file.
databaseChangeLog:
- changeSet:
dbms: mysql
id: create-sample-table
author: me
sql: CREATE TABLE sample_table (
sample_id VARCHAR(255) NOT NULL,
sample_text TEXT,
PRIMARY KEY (samoke_id)
) ENGINE=InnoDB DEFAULT
CHARSET=utf8 COLLATE=utf8_bin;
Mysql datasource config:
#Configuration
public class DataSourceConfiguration {
#Primary
#Bean(name = "mySQLDataSource")
#ConfigurationProperties("data-source.mysql")
public DataSource mySQLDataSource() {
return DataSourceBuilder.create().type(HikariDataSource.class).build();
}
}
Liquibase Configuration (probably posting more than what's needed):
#Configuration
#EnableConfigurationProperties(LiquibaseProperties.class)
public class LiquibaseConfiguration {
private static final Logger LOG = LoggerFactory.getLogger(LiquibaseConfiguration.class);
#Autowired
private LiquibaseProperties liquibaseProperties;
public DataSource liquibaseDataSource() {
DataSourceBuilder factory = DataSourceBuilder
.create()
.url(liquibaseProperties.getUrl())
.username(liquibaseProperties.getUser())
.password(liquibaseProperties.getPassword());
return factory.build();
}
public void testLiquibaseConnection() throws SQLException {
LOG.info("Testing connection to Liquibase (in case PCF restarts and we have stale dynamic secrets)...");
liquibaseDataSource().getConnection();
LOG.info("Testing connection to Liquibase (in case PCF restarts and we have stale dynamic secrets)... Succeeded");
}
#Bean
public SpringLiquibase liquibase() {
try {
testLiquibaseConnection();
} catch (Exception ex) {
LOG.warn("WARNING: Could not connect to the database using " + liquibaseProperties.getUser() + ", so we will be skipping the Liquibase Migration for now. ", ex);
return null;
}
SpringLiquibase liquibase = new SpringLiquibase();
liquibase.setChangeLog(this.liquibaseProperties.getChangeLog());
liquibase.setContexts(this.liquibaseProperties.getContexts());
liquibase.setDataSource(liquibaseDataSource());
liquibase.setDefaultSchema(this.liquibaseProperties.getDefaultSchema());
liquibase.setDropFirst(this.liquibaseProperties.isDropFirst());
liquibase.setShouldRun(this.liquibaseProperties.isEnabled());
liquibase.setLabels(this.liquibaseProperties.getLabels());
liquibase.setChangeLogParameters(this.liquibaseProperties.getParameters());
return liquibase;
}
}
The issue is we have different credentials for creating/deploying tables and reading/writing to tables in our deployed environments. So the below setup will work to create tables via Liquibase, but fail creating the metadata tables due to having the incorrect credentials upon deployment. Our current work-around to get the metadata tables created is to deploy with the data-source properties having deploy credentials, run the job to initialize the tables and then redeploy with read/write credentials. (We can't just leave the deploy credentials for reads because they have very short TTL).
Is it possible to create the metadata tables for Spring Batch via Liquibase automatically? Specifically, without adding the creation SQL manually to the changelog files?
UPDATE:
Using veljkost's answer below having a changelog file that looks like this works:
databaseChangeLog:
- changeSet:
dbms: mysql
id: create-spring-batch-metadata
author: dev.me
changes:
- sqlFile:
encoding: UTF-8
path: classpath:/org/springframework/batch/core/schema-mysql.sql
relativeToChangelogFile: false
splitStatements: true
stripComments: true
Yes, you can reference the schema files that already exist in Spring Batch project. In org.springframework.batch.core package you can find schema-*.sql files where * is the name of the targeted db. Since you are running on mysql, your change set would look something like this:
- changeSet:
id: 1234
author: adam.sandler
changes:
- sqlFile:
encoding: utf8
path: classpath:/org/springframework/batch/core/schema-mysql.sql
relativeToChangelogFile: false
splitStatements: true
stripComments: true
To auto-migrate to your database without the use of liquabase add
spring.batch.initialize-schema=always
to your application.properties file, it will auto migrate to the embedded data-source
In my test I need test with different databases (mysql, oracle, etc.) and I would like to know if it's possible with SpringRunner.
I'm using #SqlGroup and #Sql annotations, but I didn't discover how to indicate script files (sql) corresponding database.
Example:
#Sql(executionPhase = Sql.ExecutionPhase.BEFORE_TEST_METHOD, scripts = "classpath:tenantBeforeTestRun.sql")
This annotation configures my test to execute the script to all database types, but this file didn't work on Oracle.
#Sql annotation lets you define a SqlConfig which contains a datasource bean name.
Then you can define many datasource beans, with possibly different drivers and refer them from different #Sql. This might be helpful: Spring Boot Multiple Datasource
#Sql(..., config = #SqlConfig(datasource = "db1", ...)
application.properties:
#first db
spring.db1.url = [url]
spring.db1.username = [username]
spring.db1.password = [password]
spring.db1.driverClassName = oracle.jdbc.OracleDriver
#second db ...
spring.secondDatasource.url = [url]
spring.secondDatasource.username = [username]
spring.secondDatasource.password = [password]
spring.secondDatasource.driverClassName = oracle.jdbc.OracleDriver
Then, somewhere in #Configuration class:
#Bean(name = "db1")
#ConfigurationProperties(prefix="spring.db1")
public DataSource secondaryDataSource() {
return DataSourceBuilder.create().build();
}
I want to ask as currently I have my database properties like username and password inside the persistence layer in the intellij. But I want to place it somewhere outside so if someone wants to change the password or any configuration inside database he should not have to dig inside my current structure. Now my structure is persistence then main then resources and then dbconfig properties so is there any way I can do it.
You can create a file app.properties in your resources folder with all database information you need:
# Datasource details
testapp.db.driver = org.h2.Driver
testapp.db.url = jdbc:h2:mem:test
testapp.db.username = username
testapp.db.password = password
Then you can refer to it in your Java code as:
#Configuration
#PropertySource("app.properties")
public class DataConfig {
#Autowired
private Environment env;
#Bean
public DataSource dataSource() {
BasicDataSource ds = new BasicDataSource();
ds.setDriverClassName(env.getProperty("testapp.db.driver"));
ds.setUrl(env.getProperty("testapp.db.url"));
ds.setUsername(env.getProperty("testapp.db.username"));
ds.setPassword(env.getProperty("testapp.db.password"));
return ds;
}
}
As pointed out here, you can define two Datasources in Spring Boot in the following way:
#first db
spring.datasource.url = [url]
spring.datasource.username = [username]
spring.datasource.password = [password]
spring.datasource.driverClassName = oracle.jdbc.OracleDriver
#second db ...
spring.secondDatasource.url = [url]
spring.secondDatasource.username = [username]
spring.secondDatasource.password = [password]
spring.secondDatasource.driverClassName = oracle.jdbc.OracleDriver
#Bean
#Primary
#ConfigurationProperties(prefix="spring.datasource")
public DataSource primaryDataSource() {
return DataSourceBuilder.create().build();
}
#Bean
#ConfigurationProperties(prefix="spring.secondDatasource")
public DataSource secondaryDataSource() {
return DataSourceBuilder.create().build();
}
This seems to work fine. But when you are doing Atomikos XA-Transactions, each datasource has to has a unique resource name for the case that a recovery is necessary.
The Boot documentation defines a property for this:
spring.jta.atomikos.datasource.unique-resource-name=dataSource # The unique name used to identify the resource during recovery.
How do I provide a unique-resource-name for the primary and a different one for the secondary resource in Spring Boot?
Looks to me like XA-Transactions are supported in Boot, but only between JMS and DB ressources...