I'm a beginner, I have a simple Spring Boot project, it's my first time using a connection pool (HikariCP in this case) and I need your help. It's working but I want to know if I'm using it the right way with Hibernate, or if there are better ways to do it, and if my Spring Boot project structure is correct.
EDIT : It's working even if I remove the class HikariCPConfig, how can I know if connection pools are working or not?
The project is as follow :
- BankManager
src/main/java
|
|__com.manager
|__BankManagerApplication.java
|__HikariCPConfig.java
|__com.manager.dao
|__ClientRepository.java
|__com.manager.entities
|__Client.java
|__com.manager.service
|__ClientServiceImpl.java
|__ClientServiceInterface.java
src/main/resources
|__application.properties
BankManagerApplication.java :
#SpringBootApplication
public class BankManagerApplication {
public static void main(String[] args) {
ApplicationContext ctx = SpringApplication.run(BankManagerApplication.class, args);
ClientServiceInterface service = ctx.getBean(ClientServiceInterface.class);
service.addClient(new Client("client1"));
service.addClient(new Client("client2"));
}
}
HikariCPConfig.java :
#Configuration
#ComponentScan
class HikariCPConfig {
#Value("${spring.datasource.username}")
private String user;
#Value("${spring.datasource.password}")
private String password;
#Value("${spring.datasource.url}")
private String dataSourceUrl;
#Value("${spring.datasource.dataSourceClassName}")
private String dataSourceClassName;
#Value("${spring.datasource.poolName}")
private String poolName;
#Value("${spring.datasource.connectionTimeout}")
private int connectionTimeout;
#Value("${spring.datasource.maxLifetime}")
private int maxLifetime;
#Value("${spring.datasource.maximumPoolSize}")
private int maximumPoolSize;
#Value("${spring.datasource.minimumIdle}")
private int minimumIdle;
#Value("${spring.datasource.idleTimeout}")
private int idleTimeout;
#Bean
public HikariDataSource primaryDataSource() {
Properties dsProps = new Properties();
dsProps.put("url", dataSourceUrl);
dsProps.put("user", user);
dsProps.put("password", password);
dsProps.put("prepStmtCacheSize",250);
dsProps.put("prepStmtCacheSqlLimit",2048);
dsProps.put("cachePrepStmts",Boolean.TRUE);
dsProps.put("useServerPrepStmts",Boolean.TRUE);
Properties configProps = new Properties();
configProps.put("dataSourceClassName", dataSourceClassName);
configProps.put("poolName",poolName);
configProps.put("maximumPoolSize",maximumPoolSize);
configProps.put("minimumIdle",minimumIdle);
configProps.put("minimumIdle",minimumIdle);
configProps.put("connectionTimeout", connectionTimeout);
configProps.put("idleTimeout", idleTimeout);
configProps.put("dataSourceProperties", dsProps);
HikariConfig hc = new HikariConfig(configProps);
HikariDataSource ds = new HikariDataSource(hc);
return ds;
}
}
ClientServiceImpl.java
#Service
public class ClientServiceImpl implements ClientServiceInterface {
#Autowired
ClientRepository clientRepository; // this class extends JPARepository
#Override
public Client addClient(Client c) {
return clientRepository.save(c);
}
}
application.properties :
server.port = 8888
spring.jpa.databasePlatform=org.hibernate.dialect.MySQL5Dialect
spring.jpa.show-sql = true
spring.jpa.hibernate.ddl-auto = update
spring.datasource.dataSourceClassName=com.mysql.jdbc.jdbc2.optional.MysqlDataSource
spring.datasource.url=jdbc:mysql://localhost:3306/bank_manager
spring.datasource.username=root
spring.datasource.password=
spring.datasource.poolName=SpringBootHikariCP
spring.datasource.maximumPoolSize=5
spring.datasource.minimumIdle=3
spring.datasource.maxLifetime=2000000
spring.datasource.connectionTimeout=30000
spring.datasource.idleTimeout=30000
spring.datasource.pool-prepared-statements=true
spring.datasource.max-open-prepared-statements=250
Thank you in advance.
You project structure is standard, so it's correct.
About Hikari:
Hikari is indeed a great choice for pooling. I'm used to work with Hikari successfully by using a smaller set of params you are applying in your case, but if it's working for you that's fine.
For more info about Hikaru setup, I recommend reading the official wiki, if you haven't already.
About property loading:
You can make use of some SpringBoot features to read the DB parameters and apply into your runtime Beans with less code. Like:
In application.properties (define a custom prefix 'myproject.db' for your pool params)
myproject.db.dataSourceClassName=com.mysql.jdbc.jdbc2.optional.MysqlDataSource
myproject.db.url=jdbc:mysql://localhost:3306/bank_manager
myproject.db.username=root
... and the other params below
Create a Spring Configuration class
#Configuration
public class MyDBConfiguration {
#Bean(name = "myProjectDataSource")
#ConfigurationProperties(prefix = "myproject.db")
public DataSource dataSource(){
//This will activate Hikari to create a new DataSource instance with all parameters you defined with 'myproject.db'
return DataSourceBuilder.create().build();
}
}
In your ClientRepository class:
#Repository
public class ClientRepository {
//The code below is optional, but will work if you want to use jdbctemplate tied to the DataSource created above. By default all Hibernate Sessions will take the DataSource generated by Spring
#Bean(name = "myProjectJdbcTemplate")
public JdbcTemplate jdbcTemplate(#Qualifier("myProjectDataSource") DataSource dataSource){
return new JdbcTemplate(dataSource);
}
}
There are other options to manage the DataSource beans creation if you are going to use 2 or more different Databases. You can vary the properties prefix for the other databases and annotate 1 Datasource only as #Primary, which is mandatory when you have more than 1 DataSources in Spring context
Related
Hi I have been making spring boot project which have 2 of datasource.
I want to use those datasources properties' value from application.yml file and
so that I configured #Configuration Class on our project.
But, even if both of Configuration class was set same way, #Value annotation isn't working on the OracleConfig.class.
I tested related oracle setting which wrote on application.yml file is working on the HiveConfig.class and I confirmed that.
how can I inject value to OracleConfig.class :( ,,,
I currently strongly doubt OracleConfig.class was working before #Value annotation or other Beans had been initialized.
should I configure order about bean creation process or (is that make sense)force that OracleConfig.java's #Bean will be created after injection of #Value?
HiveConfig.class
#Configuration
public class HiveConfig {
#Value("${hive.custom.use}")
private boolean useCustomHive;
#Value("${hive.custom.some-of-setting}")
private String SomeOfSetting;
#Bean
#ConfigurationProperties(prefix = "hive.datasource")
public HikariDataSource dataSourceHive() throws Exception {
HikariDataSource hikariDataSource = new HikariDataSource();
if (useCustomHive){
hikariDataSource.SetSomeOfSetting(SomeOfSetting);
}
return hikariDataSource;
}
OracleConfig.class
#Configuration
public class OracleConfig {
#Value("${oracle.custom.use}")
private boolean useCustomOracle;
#Value("${oracle.custom.some-of-setting}")
private String SomeOfSetting;
#Bean
#ConfigurationProperties(prefix = "oracle.datasource")
public HikariDataSource dataSourceOracle() throws Exception {
HikariDataSource hikariDataSource = new HikariDataSource();
if (useCustomOracle){
hikariDataSource.SetSomeOfSetting(SomeOfSetting);
}
return hikariDataSource;
}
application.yml (this values(oracle and hive) can read from HiveConfig.class via #Value)
oracle:
custom:
use:true
some-of-setting: str
hive:
custom:
use:true
some-of-setting: str
Have you tried with constructor injection? See if this works.
private boolean useCustomHive;
private String SomeOfSetting;
HiveConfig (#Value("${hive.custom.use}") boolean useCustomHive, #Value("${hive.custom.some-of-setting}") String SomeOfSetting) {
this.useCustomHive = useCustomHive;
this.SomeOfSetting = SomeOfSetting;
}
I am connecting to multiple datasources but sometimes some datasources may be offline and at that time I am geting errors on app and application is failing at startup.
I want to skip datasource configuration at startup... I have tried several ways by adding
spring.autoconfigure.exclude=org.springframework.boot.autoconfigure.jdbc.DataSourceAutoConfiguration
to the application.properties and also I have tried adding
#SpringBootApplication(exclude={DataSourceAutoConfiguration.class})
to the main class but still it tries to configure the datasource.
I also tried to use #Lazy annotation on all methods and on constructor as below but still getting error while creating fooEntityManagerFactory
#Lazy
#Configuration
#EnableJpaRepositories(basePackages = "com.heyo.tayo.repository.foo", entityManagerFactoryRef = "fooEntityManagerFactory", transactionManagerRef = "fooTransactionManager")
public class PersistencefooConfiguration {
#Autowired
private DbContextHolder dbContextHolder;
#Lazy
#Bean
#ConfigurationProperties("tay.datasource.foo")
public DataSourceProperties fooDataSourceProperties() {
return new DataSourceProperties();
}
#Lazy
#Bean
#ConfigurationProperties("tay.datasource.foo.configuration")
public DataSource fooDataSource() {
DataSource dataSource = fooDataSourceProperties().initializeDataSourceBuilder()
.type(BasicDataSource.class).build();
dbContextHolder.addNewAvailableDbType(DbTypeEnum.foo);
return dataSource;
}
#Lazy
#Bean(name = "fooEntityManagerFactory")
public LocalContainerEntityManagerFactoryBean fooEntityManagerFactory(
EntityManagerFactoryBuilder builder) {
//THE CODE IS FAILING AT BELOW RETURN CASE
return builder
.dataSource(fooDataSource())
.packages("com.heyo.tayo.model.foo")
.build();
}
#Lazy
#Bean
public PlatformTransactionManager fooTransactionManager(
final #Qualifier("fooEntityManagerFactory") LocalContainerEntityManagerFactoryBean fooEntityManagerFactory) {
return new JpaTransactionManager(fooEntityManagerFactory.getObject());
}
}
I have multiple classes like above for different configs for different datasources and I am adding them to available dbs static list at datasource Bean.
Here is my dbadapter factory class.
Here is my dbAdaptor factory that creates corresponding db adaptor
#Service
public class DbAdapterFactory {
#Autowired
private BeanFactory beanFactory;
#Autowired
private DbContextHolder dbContextHolder;
public DBAdapter dbAdapter(){
DbTypeEnum currentDb = dbContextHolder.getCurrentDb();
DBAdapter dbAdapter = null;
if(currentDb == DbTypeEnum.FOODB) {
dbAdapter = beanFactory.getBean(foodbadaptor.class);
} else {
dbAdapter = beanFactory.getBean(koodbadaptor.class);
}
return dbAdapter;
}
Here is db context holder that makes operation like setting default db or getting current db etc.:
#Component
public class DbContextHolder {
private DbTypeEnum dbType = DbTypeEnum.FOODB;
private Set<DbTypeEnum> availableDbTypes = new HashSet<>();
public void setCurrentDb(DbTypeEnum dbType) {
this.dbType = dbType;
}
public DbTypeEnum getCurrentDb() {
return this.dbType;
}
public List<DbTypeEnum> getAvailableDbTypes() {
return new ArrayList<>(availableDbTypes);
}
public void addNewAvailableDbType(DbTypeEnum dbTypeEnum) {
availableDbTypes.add(dbTypeEnum);
}
}
I made all #Lazy or tried #SpringBootApplication(exclude={DataSourceAutoConfiguration.class}) but still something is calling to create bean and getting error and app is closing. I want to use that config and datasource in a try-catch block and don't stop application at runtime. How can I achieve this or what am I missing on that configs or annotations ?
I believe that you can simply add in your application properties
spring.sql.init.continue-on-error=true
According to the Spring Boot 2.5.5 user guide:
https://docs.spring.io/spring-boot/docs/2.5.5/reference/htmlsingle/#howto-initialize-a-database-using-spring-jdbc
Spring Boot enables the fail-fast feature of its script-based database initializer. If the scripts cause exceptions, the application fails to start. You can tune that behavior by setting spring.sql.init.continue-on-error.
Depending on your spring boot version the property will be named either
spring.sql.init.continue-on-error
or before Spring Boot 2.5
spring.datasource.continue-on-error
It is so dumb but I solved the problem by adding following to application.properties.
spring.jpa.database=sql_server
I have no idea why I need to specify that explicitly in properties file but the problem is solved. I will search for it
Hy all,
in my current application, I need to be able to load a number of database connections into my context (they are then used in camel routes, but thats not part of this question).
For this I have a List of settings in my application.yml file, where each entry is all that is needed for one connection:
- name: mysql1
jdbcUrl: "jdbc:mysql://localhost:3306/dbScheme"
username: a
password: a
- name: mssql1
jdbcUrl: "jdbc:sqlserver://localhost:1433;databaseName=dbScheme"
username: a
password: a
There is a variable amount of connections, that can be configured this way.
I need each of those configurations as a javax.sql.DataSource object/bean in the registy (bean name would be the name property from the configuration object):
DataSource dataSourceBean = DataSourceBuilder.create()
.driverClassName(driverClassName) // Resolved automatically from the jdbcUrl
.url(dataSourceFromYaml.getJdbcUrl())
.username(dataSourceFromYaml.getUsername())
.password(dataSourceFromYaml.getPassword())
.build();
The question now is, how I do put those objects into the context as beans?
As I see it, its not possible with annotations, because we have a variable amount of objects and they are also not loaded into the context directly, but have to be created first. (Or am I wrong here?)
Thanks in advance for any ideas
Chris
Edit:
To clarify: The two connections I put here are not the connections used in production, they are just an example.
The issue is, that they are configured in production and there can be any amount of them.
I have no way of predicting how many there are and how they are named.
After trying a lot of different spring context objects I finally found one, that worked.
Here is the solution:
#Configuration
#ConfigurationProperties(prefix = "jdbc")
#RequiredArgsConstructor // Lombok
public class DataSourceBeanFactory {
// The DataSourceConfiguration holds everything from one property object
#Setter // Lombok
private List<DataSourceConfiguration> dataSources;
private final ConfigurableApplicationContext applicationContext;
#PostConstruct
public void resolveAndCreateDataSourceBeans() {
dataSources.forEach(dataSourceFromYaml -> {
/*
* Code to resolve driver class name
*/
String driverClassName = ....
DataSource dataSourceBean = DataSourceBuilder.create()
.driverClassName(driverClassName)
.url(dataSourceFromYaml.getJdbcUrl())
.username(dataSourceFromYaml.getUsername())
.password(dataSourceFromYaml.getPassword())
.build();
applicationContext
.getBeanFactory()
.registerSingleton(dataSourceFromYaml.getName(), dataSourceBean);
});
}
Thank to everybody, that answered my question.
Chris
You could use the following approach
create properties file application-dev.properties
mysql1.jdbc.jdbcUrl=jdbc:mysql://localhost:3306/dbScheme
mysql1.jdbc.username=a
mysql1.jdbc.password=a
mssql1.jdbc.jdbcUrl=jdbc:sqlserver://localhost:1433;databaseName=dbScheme
mssql1.jdbc.username=a
mssql1.jdbc.password=a
Create configuration class
#Configuration
public class JDBCConfigExample {
#Bean(name = "mysql1DataSource")
#ConfigurationProperties(prefix = "mysql1.jdbc")
public DataSource mysql1DataSource() {
return DataSourceBuilder.create().build();
}
#Bean(name = "mssql1DataSource")
#ConfigurationProperties(prefix = "mssql1.jdbc")
public DataSource mssql1DataSource() {
return DataSourceBuilder.create().build();
}
#Bean(name = "mysql1JdbcTemplate")
public JdbcTemplate mysql1JdbcTemplate(#Qualifier("mysql1DataSource") DataSource dataSource) {
return new JdbcTemplate(dataSource);
}
#Bean(name = "mssql1JdbcTemplate")
public JdbcTemplate mssql1JdbcTemplate(#Qualifier("mssql1DataSource") DataSource dataSource) {
return new JdbcTemplate(dataSource);
}
}
After that you could use both JdbcTemplate and DataSource in any Spring managed bean by using Qualifier or name convention:
private final JdbcTemplate mssql1JdbcTemplate;
If you don't have list of datasources beforehand and want to configure this dynamically you can do following thing:
1) In your application.yml:
datasources:
ds1:
name:
pw:
ds2:
name:
pw:
2) Create properties class:
#Configuration
#ConfigurationProperties(prefix = "datasources")
public class DataSourcesInfo {
Map<String, DataSourceInfo> dataSources;
public #Data static class DataSourceInfo {
private String pw;
private String name;
}
}
In this map you will have list of entries for all datasources defined.
3) And now you can create beans dynamically:
#Configuration
public class Config {
#Autowired
private GenericApplicationContext genericApplicationContext;
#Autowired
DataSourcesInfo properties;
#PostConstruct
public void createDataSources() {
// iterate through properties and register beans
genericApplicationContext.registerBean(dataSourceName,
DataSource.class,
() -> {
DataSource ds = new DataSource();
ds.set(...);
....
return ds;
})
}
}
I have a Spring Boot application that uses 2 datasources. It can connect to the first one fine. When I try to use the second data source, I am getting the SQL error: Table or view does not exist because it is using the first data source as its connection.
This is my application property file:
#Property for both DBs
spring.datasource.driver.class.name=oracle.jdbc.driver.OracleDriver
## Database Properties DB #1
spring.datasource.tms.url=jdbc:oracle:thin:#ldap:<connection properties have been removed but are present>
spring.datasource.tms.username=own_app
spring.datasource.tms.password=own_app_l1
spring.datasource.tms.jmx.enabled=true
spring.main.allow-bean-definition-overriding=true
## Database Properties DB #2
spring.datasource.lhl.url=jdbc:oracle:thin:#ldap:<connection string has been removed but is correct>
spring.datasource.lhl.username=LHL_PURCH_APP
spring.datasource.lhl.password=ChangemeChangemeChangeme$$2019
spring.datasource.lhl.jmx-enabled=true
This is my Configuration file for both data sources:
#Configuration
#PropertySource("classpath:application-local.properties")
public class FxgLhlPurchasedItineraryAdapterDataSourceConfiguration {
#Value("${spring.datasource.driver.class.name}")
private String driverClassName;
//TMS properties
#Value("${spring.datasource.tms.url}")
private String tmsUrl;
#Value("${spring.datasource.tms.username}")
private String tmsUsername;
#Value("${spring.datasource.tms.password}")
private String tmsPassword;
//LHL Properties
#Value("${spring.datasource.lhl.url}")
private String lhlUrl;
#Value("${spring.datasource.lhl.username}")
private String lhlUsername;
#Value("${spring.datasource.lhl.password}")
private String lhlPassword;
#Primary
#Bean(name = "tmsDataSource")
#ConfigurationProperties(prefix = "spring.datasource.tms")
public DataSource tmsDataSource() {
DataSourceBuilder factory = DataSourceBuilder.create(this.getClass().getClassLoader())
.driverClassName(driverClassName).url(tmsUrl)
.username(tmsUsername)
.password(tmsPassword);
return factory.build();
}
#Bean(name = "lhlDataSource")
#ConfigurationProperties(prefix = "spring.datasource.lhl")
public DataSource lhlDataSource() {
DataSourceBuilder factory = DataSourceBuilder.create(this.getClass().getClassLoader())
.driverClassName(driverClassName).url(lhlUrl)
.username(lhlUsername)
.password(lhlPassword);
return factory.build();
}
#Bean(name = "tmsJdbcTemplate")
public JdbcTemplate tmsJdbcTemplate(final DataSource tmsDataSource) {
return new JdbcTemplate(tmsDataSource);
}
#Bean(name = "lhlJdbcTemplate")
public JdbcTemplate lhlJdbcTemplate(final DataSource lhlDataSource) {
return new JdbcTemplate(lhlDataSource);
}
}
I had to put in the #Primary annotation if the service will not run.
When I try to do a simple SELECT of a table in that is not the Primary database, I get the error that the table does not exist.
This is the code that calls the select statement:
private JdbcTemplate lhlJdbcTemplate;
DataSource ds = lhlJdbcTemplate.getDataSource();
Connection con = ds.getConnection();
LOGGER.info("Connection info: {}", con.getSchema());
lhlParmSqIdModelList = lhlJdbcTemplate.query(
selectSequenceNbrSQLStatement,
new LhlParmSqIdModelRowMapper());
The logger statement returns the schema for the Primary database.
How can I get the connection to use the Second database
Because you have multiple DataSource beans, normally Spring will fail because it doesn't know how to automatically decide which of multiple equivalent beans it should use, it puts that responsibility on you as the programmer.
By adding the #Primary annotation, you are telling Spring "If there are multiple candidate beans of this type, use this one."
Your bean methods aren't asking Spring for a particular DataSource, they just want any DataSource, so Spring gives each of them the one marked with #Primary.
Instead, you'll want to use #Qualifier to indicate exactly which named DataSource they want:
#Bean(name = "tmsJdbcTemplate")
public JdbcTemplate tmsJdbcTemplate(#Qualifier("tmsDataSource") final DataSource tmsDataSource) {
return new JdbcTemplate(tmsDataSource);
}
#Bean(name = "lhlJdbcTemplate")
public JdbcTemplate lhlJdbcTemplate(#Qualifier("lhlDataSource") final DataSource lhlDataSource) {
return new JdbcTemplate(lhlDataSource);
}
I don't guarantee this syntax is exactly right, but something like that.
You'll also need to qualify the JdbcTemplate injection point. (Credit to Sherif Behna in the comments)
Reading the instructions on the HikariCP wiki about how to enable the Dropwizard metrics, it says to just configure a MetricsRegistry instance in HikariConfig or HikariDatasource.
Problem is, in Spring Boot, all the configuration is handled by autoconfiguration so I'm not manually configuring the HikariCP pool at all.
Any instructions on how to do this? Do I have to completely override the autoconfiguration by defining my own bean and setting all the settings in a #Configuration file?
Or let Spring Boot configure your data source, #Autowire the DataSource and MetricRegistry in your #Configuration/#SpringBootApplication class and wire them together in a #PostConstruct:
#Autowired
private DataSource dataSource;
#Autowired
private MetricRegistry metricRegistry;
#PostConstruct
public void setUpHikariWithMetrics() {
if(dataSource instanceof HikariDataSource) {
((HikariDataSource) dataSource).setMetricRegistry(metricRegistry);
}
}
So I was able to figure this out by manually configuring HikariCP in a java configuration file. That allowed me to get a reference to the Spring Boot MetricRegistry, which I could then set in HikariConfig. Here's my configuration class:
#Configuration
public class DatasourceConfiguration {
#Value("${spring.datasource.username}")
private String user;
#Value("${spring.datasource.password}")
private String password;
#Value("${spring.datasource.url}")
private String dataSourceUrl;
#Value("${spring.datasource.driverClassName}")
private String driverClassName;
#Value("${spring.datasource.connectionTestQuery}")
private String connectionTestQuery;
#Autowired
private MetricRegistry metricRegistry;
#Bean
public DataSource primaryDataSource() {
Properties dsProps = new Properties();
dsProps.setProperty("url", dataSourceUrl);
dsProps.setProperty("user", user);
dsProps.setProperty("password", password);
Properties configProps = new Properties();
configProps.setProperty("connectionTestQuery", connectionTestQuery);
configProps.setProperty("driverClassName", driverClassName);
configProps.setProperty("jdbcUrl", dataSourceUrl);
HikariConfig hc = new HikariConfig(configProps);
hc.setDataSourceProperties(dsProps);
hc.setMetricRegistry(metricRegistry);
return new HikariDataSource(hc);
}
}