Develop Reference

Configuring hikari connection pool

I have following settings for my database (I have multiple databases, so they are configured in spring.datasource hierarchy.
spring:
datasource:
db-write:
url: jdbc:sqlserver://whatever.database.windows.net:1433;database=dbname;encrypt=true;trustServerCertificate=false;hostNameInCertificate=*.database.windows.net;loginTimeout=30;
username: 'myusername'
password: 'mynotsosecretpassword'
driver-class-name: com.microsoft.sqlserver.jdbc.SQLServerDriver
Then I am configuring my datasource here
#Configuration
#EnableTransactionManagement
#EnableJpaRepositories(basePackages = "com.datasources.dbwrite.repository",
entityManagerFactoryRef = "entityManagerFactoryDbWrite",
transactionManagerRef= "transactionManagerDbWrite"
)
public class DataSourceConfigurationDbWrite {
#Bean
#Primary
#ConfigurationProperties("spring.datasource.db-write")
public DataSourceProperties dataSourcePropertiesDbWrite() {
return new DataSourceProperties();
}
#Bean
#Primary
#ConfigurationProperties("spring.datasource.db-write.configuration")
public DataSource dataSourceDbWrite() {
return dataSourcePropertiesDbWrite().initializeDataSourceBuilder()
.type(HikariDataSource.class).build();
}
#Primary
#Bean(name = "entityManagerFactoryDbWrite")
public LocalContainerEntityManagerFactoryBean entityManagerFactoryDbWrite(EntityManagerFactoryBuilder builder) {
return builder
.dataSource(dataSourceDbWrite())
.packages("com.datasources.dbwrite.models")
.build();
}
#Primary
#Bean
public PlatformTransactionManager transactionManagerDbWrite(
final #Qualifier("entityManagerFactoryDbWrite") LocalContainerEntityManagerFactoryBean entityManagerFactoryDbWrite) {
return new JpaTransactionManager(Objects.requireNonNull(entityManagerFactoryDbWrite.getObject()));
}
}
I am configuring my hikari datasource in dataSourceDbWrite method based on the properties i read in dataSourcePropertiesDbWrite method. I believe i need to configure properties in specific hierarchy so that dataSourceDbWrite method can easily detect which properties are needed for hikari. Is that correct?
What that hierarchy would be?
Moreover, how can and where can i find what properties i can configure for hikari? connection-timeout? connection pool size etc?

Me personally prefer application.yml than code to configurate Hikari:
spring:
datasource:
driver-class-name: com.mysql.cj.jdbc.Driver
url: {JDBC URL}
username: {USERNAME}
password: {PASSWORD}
type: com.zaxxer.hikari.HikariDataSource
hikari:
minimum-idle: 5
idle-timeout: 600000
maximum-pool-size: 10
auto-commit: true
pool-name: HikariCorePool
max-lifetime: 1800000
connection-timeout: 30000
connection-test-query: select * from information_schema.tables limit 1
(BTW, that piece of configuration was originally writen by a colleague years ago. We didn't change it and just copy-and-paste into any new projects those years.😆)
If you want to check out all configurable fields, those spring.datasource.hikari.* keys inorg.springframework.boot:spring-boot-autoconfigure:{VERSION}/META-INF/spring/spring-configuration-metadata.json may could help.
And javadoc in com.zaxxer.hikari.HikariConfigMXBean could help too.

See example in article, the properties hierarchy are according to #ConfigurationProperties's value
If we want to configure Hikari, we just need to add a #ConfigurationProperties to the data source definition:
#Bean
#ConfigurationProperties("spring.datasource.todos.hikari")
public DataSource todosDataSource() {
return todosDataSourceProperties()
.initializeDataSourceBuilder()
.build();
}
Then, we can insert the following lines into the application.properties file:
spring.datasource.todos.hikari.connectionTimeout=30000
spring.datasource.todos.hikari.idleTimeout=600000
spring.datasource.todos.hikari.maxLifetime=1800000
See relevant hikari's spring properties
spring.datasource.hikari.connection-timeout
spring.datasource.hikari.data-source-class-name
spring.datasource.hikari.data-source-properties
spring.datasource.hikari.driver-class-name
spring.datasource.hikari.idle-timeout
spring.datasource.hikari.initialization-fail-timeout
spring.datasource.hikari.jdbc-url
spring.datasource.hikari.leak-detection-threshold
spring.datasource.hikari.login-timeout
spring.datasource.hikari.max-lifetime
spring.datasource.hikari.maximum-pool-size
spring.datasource.hikari.minimum-idle
spring.datasource.hikari.validation-timeout
And explanation on each property in HikariCP, for example
connectionTimeout
This property controls the maximum number of milliseconds that a client (that's you) will wait for a connection from the pool. If this time is exceeded without a connection becoming available, a SQLException will be thrown. Lowest acceptable connection timeout is 250 ms. Default: 30000 (30 seconds)
Notice that camelCase hikari properties (connectionTimeout) is shown as snake-case in spring (connection-timeout)

setup alternate datasource in spring boot application causes liquibase to fail

Application has a default spring data source specified in application.yml
spring:
datasource:
type: com.zaxxer.hikari.HikariDataSource
url: jdbc:oracle:thin:#localhost:1521:xe
username: system
password: oracle
hikari:
poolName: Hikari
auto-commit: false
I have added configuration options for a second data source, used for a completely difference (JDBCTemplate purpose).
faas20:
ds:
url: jdbc:oracle:thin:#tldb0147vm.group.net:1760:tdb
username: ...
password: ...
Then, I add two data sources, one named, and the other default. Without the default one, liquibase fails to start.
#Configuration
public class LegacyConfiguration {
#Bean(name = "faas20")
#ConfigurationProperties(prefix = "faas20.ds")
public DataSource legacyDataSource() {
return DataSourceBuilder
.create()
.build();
}
#Bean
public DataSource defaultDataSource() {
return DataSourceBuilder
.create()
.build();
}
}
Startup of the application fails though.
The application now cannot build the default EntityManagerFactory.
Why would that be affected?
Parameter 0 of constructor in impl.OrderServiceImpl required a bean named 'entityManagerFactory' that could not be found.
Consider defining a bean named 'entityManagerFactory' in your configuration.
Without the two data sources present the application and liquibase start up as they should.
edit
I am not clear on how to configure two separate data sources,
Default Data Source for JPA
Additional Data Source for use in JDBC (and potentially other JPA classes)

Spring Boot Quartz Scheduler Configuration

I want to move our Quartz Scheduling configuration to our application.yml instead of maintaining a separate quartz.properties file.
Our Spring Boot application runs and picks up the configuration as expected when using quartz.properties file, but it doesn't pick up the config from application.yml.
Scheduler bean:
#SpringBootApplication
public class MyApp{
public static void main(String[] args) {
SpringApplication.run(MyApp.class, args);
}
...
#Bean
public Scheduler scheduler(SomeCustomConfig cfg, RestTemplate restTemplate) throws SchedulerException {
//StdSchedulerFactory schedulerFactory = new StdSchedulerFactory();
//schedulerFactory.initialize("quartz.properties");
Scheduler scheduler = StdSchedulerFactory.getDefaultScheduler();
scheduler.getContext().put("restTemplate", restTemplate);
scheduler.getContext().put("cfg", cfg);
return scheduler;
}
}
Pertinent application.yml:
spring:
application.name: myApp
quartz:
properties:
org:
quartz:
scheduler:
instanceId: AUTO
threadPool:
threadCount: 5
plugin:
shutdownhook:
class: org.quartz.plugins.management.ShutdownHookPlugin
cleanShutdown: TRUE
jobStore:
class: org.quartz.impl.jdbcjobstore.JobStoreTX
driverDelegateClass: org.quartz.impl.jdbcjobstore.PostgreSQLDelegate
tablePrefix: my_schema.
isClustered: true
dataSource: myDataSource
dataSource:
myDataSource:
driver: org.postgresql.Driver
URL: jdbc:postgresql://localhost/myDataSource
user: removed
password: removed
Our quartz.properties was:
org.quartz.scheduler.instanceId = AUTO
org.quartz.plugin.shutdownhook.class = org.quartz.plugins.management.ShutdownHookPlugin
org.quartz.plugin.shutdownhook.cleanShutdown = TRUE
org.quartz.threadPool.threadCount = 5
org.quartz.jobStore.class = org.quartz.impl.jdbcjobstore.JobStoreTX
org.quartz.jobStore.driverDelegateClass = org.quartz.impl.jdbcjobstore.PostgreSQLDelegate
org.quartz.jobStore.tablePrefix = my_schema.
org.quartz.jobStore.isClustered = true
org.quartz.jobStore.dataSource = myDataSource
org.quartz.dataSource.myDataSource.driver = org.postgresql.Driver
org.quartz.dataSource.myDataSource.URL = jdbc:postgresql://localhost/myDataSource
org.quartz.dataSource.myDataSource.user = removed
org.quartz.dataSource.myDataSource.password = removed
I feel like I'm missing something?

Instead of
spring:
quartz:
properties:
org:
quartz:
jobStore:
isClustered: true
Use this layout:
spring:
quartz:
properties:
org.quartz.jobStore:
isClustered: true
org.quartz.scheduler:
instanceId: AUTO
With the latter layout, I get:
2019-09-06 13:45:19.919 INFO PID --- [ main] o.q.c.QuartzScheduler : {} Scheduler meta-data: Quartz Scheduler (v2.3.0) 'quartzScheduler' with instanceId '0157799997'
Scheduler class: 'org.quartz.core.QuartzScheduler' - running locally.
NOT STARTED.
Currently in standby mode.
Number of jobs executed: 0
Using thread pool 'org.quartz.simpl.SimpleThreadPool' - with 10 threads.
Using job-store 'org.springframework.scheduling.quartz.LocalDataSourceJobStore' - which supports persistence. and is clustered.

Your application.yml configuration sets for spring-boot-starter-quartz and I think you are using org.quartz-scheduler independently. So you should config your application.yml something like this:
spring:
application.name: myApp
org:
quartz:
scheduler:
instanceId: AUTO
threadPool:
threadCount: 5
plugin:
shutdownhook:
class: org.quartz.plugins.management.ShutdownHookPlugin
cleanShutdown: TRUE
jobStore:
class: org.quartz.impl.jdbcjobstore.JobStoreTX
driverDelegateClass: org.quartz.impl.jdbcjobstore.PostgreSQLDelegate
tablePrefix: my_schema.
isClustered: true
dataSource: myDataSource
dataSource:
myDataSource:
driver: org.postgresql.Driver
URL: jdbc:postgresql://localhost/myDataSource
user: removed
password: removed

I have recently worked with Spring Boot Quartz Application and was facing a similar issue where the quartz.properties was not being detected by application where I was using application.yml to hold application environment variable
spring:
quartz:
properties:
org.quartz.scheduler:
instanceName: ${QUARTZ_SCHEDULER_INSTANCE_NAME:Scheduler}
instanceId: ${QUARTZ_SCHEDULER_INSTANCE_ID:AUTO}
makeSchedulerThreadDaemon: ${QUARTZ_SCHEDULER_MAKE_THREAD_DAEMON:true}
org.quartz.jobStore:
class: ${QUARTZ_JOBSTORE_CLASS:org.quartz.impl.jdbcjobstore.JobStoreTX}
driverDelegateClass: ${QUARTZ_JOBSTORE_DRIVER:org.quartz.impl.jdbcjobstore.PostgreSQLDelegate}
tablePrefix: ${QUARTZ_JOBSTORE_TABLE_PREFIX:qrtz_}
isClustered: ${QUARTZ_JOBSTORE_ISCLUSTER:false}
dataSource: ${QUARTZ_JOBSTORE_DATASOURCE:myDS}
misfireThreshold: ${QUARTZ_JOBSTORE_MISFIRE_THRESHOLD:25000}
org.quartz.threadPool:
class: ${QUARTZ_THREADPOOL_CLASS:org.quartz.simpl.SimpleThreadPool}
makeThreadsDaemons: ${QUARTZ_THREADPOOL_DAEMON:true}
threadCount: ${QUARTZ_THREADPOOL_COUNT:20}
threadPriority: ${QUARTZ_THREADPOOL_PRIORITY:5}
org.quartz.dataSource:
myDS:
driver: ${SPRING_DATASOURCE_DRIVER:org.postgresql.Driver}
URL: ${SPRING_DATASOURCE_URL:jdbc:postgresql://localhost:5432/postgres}
user: ${SPRING_DATASOURCE_USERNAME:postgres}
password: ${SPRING_DATASOURCE_PASSWORD:postgres}
maxConnections: ${SPRING_DATASOURCE_MAX_CONNECTION:20}
validationQuery: ${SPRING_DATASOURCE_VALIDATION_QUERY:select 1}
By using the above configuration in the above format, I was not only able to trigger Quartz jobs , i was also able to store in database

Spring Boot #DataJpaTest without ddl generation

I have MySQL with tables in two databases and want to write a test with SpringBoot (1.5.6.RELEASE) and JPA. For this, I use the #DataJpaTest together with #EntityScan, as the entities are in two different packages. As embedded database for the test, I use H2.
My first problem was, that Spring Boot threw an exception that the schemas were not found, therefore I created a schema.sql with two CREATE SCHEMA IF NOT EXISTS statements like described in this question.
However, I also wanted to insert some test data and added a data.sql file. The problem was now, that Spring boot first executed the schema.sql, then the data.sql and following this Hibernate again created the tables, which eventually resulted in empty tables. To solve this problem, I tried to set spring.jpa.hibernate.ddl-auto=none within the application.properties. However, Hibernate now started to switch naming strategies and converted camelCase to sneak_case, which again resulted in errors. So I guess, this is not the way how it should be done. I also tried spring.jpa.generate-ddl=false, but this did not have any effect.
How is it possible to deactivate the automatic DDL generation (only use schema.sql and data.sql) and at the same time use #DataJpaTest?
Thank you very much!

Solution 1: properties/yaml file
spring:
autoconfigure:
exclude: org.springframework.boot.autoconfigure.security.SecurityAutoConfiguration
jackson:
serialization:
indent_output: true
datasource:
driver-class-name: org.hsqldb.jdbcDriver
generate-unique-name: true
jpa:
hibernate:
ddl-auto: none
show-sql: true
h2:
console:
enabled: false
liquibase:
change-log: classpath:/liquibase/db.changelog-master.xml
drop-first: true
contexts: QA
Solution 2: override #Bean
#Bean
LocalContainerEntityManagerFactoryBean entityManagerFactory(DataSource dataSource, Environment env) {
final LocalContainerEntityManagerFactoryBean factory = new LocalContainerEntityManagerFactoryBean();
factory.setDataSource(dataSource);
factory.setJpaVendorAdapter(new HibernateJpaVendorAdapter());
factory.setPackagesToScan("com.spring.web.demo.persistent.entity");
factory.setJpaProperties(jpaProperties(env));
return factory;
}
private Properties jpaProperties(Environment env) {
final Properties properties = new Properties();
properties.put("hibernate.dialect", env.getRequiredProperty("hibernate.dialect"));
//!!!: see here
properties.put("hibernate.hbm2ddl.auto", "none");
properties.put("hibernate.show_sql", env.getRequiredProperty("hibernate.show_sql"));
properties.put("hibernate.format_sql", false);
properties.put("hibernate.physical_naming_strategy", PhysicalNamingStrategyStandardImpl.class.getName());
properties.put("hibernate.generate_statistics", true);
properties.put("hibernate.cache.use_second_level_cache", true);
properties.put("hibernate.cache.use_query_cache", true);
properties.put("hibernate.cache.region.factory_class", "org.hibernate.cache.ehcache.EhCacheRegionFactory");
return properties;
}
Note, that application.properies or application.yaml should be under test/resources and your #Configuration class with #Bean should be used by tests.
For test via sql-files you could use EmbeddedDataSource
There are some examples at my github

you can create another application.properties and configure file for under test directory.
for other problem use flyway after db creation flyway scripts which is your insert scripts run automatically.

How to make sure quartz.properties is being used?

I set following properties in my quartz.properties file:
org.quartz.threadPool.threadCount = 60
org.quartz.scheduler.batchTriggerAcquisitionMaxCount = 60
, however, for some reason, apparently it doesn't take effect. because when I start my application, the log shows that it still uses 1 thread in the pool:
[main] INFO org.quartz.impl.StdSchedulerFactory - Using default implementation for ThreadExecutor
[main] INFO org.quartz.core.SchedulerSignalerImpl - Initialized Scheduler Signaller of type: class org.quartz.core.SchedulerSignalerImpl
[main] INFO org.quartz.core.QuartzScheduler - Quartz Scheduler v.2.1.1 created.
[main] INFO org.quartz.simpl.RAMJobStore - RAMJobStore initialized.
[main] INFO org.quartz.core.QuartzScheduler - Scheduler meta-data: Quartz Scheduler (v2.1.1) 'QuartzScheduler' with instanceId 'NON_CLUSTERED' Scheduler class: 'org.quartz.core.QuartzScheduler' - running locally. NOT STARTED. Currently in standby mode. Number of jobs executed: 0
Using **thread pool 'org.quartz.simpl.SimpleThreadPool' - with 1 threads.**
I know, the quartz.properties needs to be at class path to be found. and I just did it.
any other reason why this file is not detected? or it is detected but number of threads is not set correctly?
Thanks

For those who are using Spring + Quartz and quartz.properties file is not working (i.e. gets ignored while starting the application):
Quartz Scheduler (org.quartz.Scheduler) instantiated by Spring Factory Bean (org.springframework.scheduling.quartz.SchedulerFactoryBean) won't read quartz.properties file from the classpath by default as it's said in Quartz docs - you need to set the reference manually:
[in case of Java config]:
#Bean
public SchedulerFactoryBean schedulerFactoryBean() {
SchedulerFactoryBean schedulerFactoryBean = new SchedulerFactoryBean();
schedulerFactoryBean.setConfigLocation(new ClassPathResource("quartz.properties"));
// ...
}
[in case of XML config]:
<bean class="org.springframework.scheduling.quartz.SchedulerFactoryBean">
<property name="configLocation" value="classpath:quartz.properties" />
// ...
</bean>

oops, I found the problem, actually the code was overriding the properties file config by creating an instance of Properties class in the code. so the answer is this line:
sf = new StdSchedulerFactory("conf/quartz.properties");

If someone still looking for answer, they can use the below snippet
#Bean
public SchedulerFactoryBean schedulerFactoryBean() throws IOException {
SchedulerFactoryBean scheduler = new SchedulerFactoryBean();
scheduler.setTriggers(jobOneTrigger());
scheduler.setQuartzProperties(quartzProperties());
scheduler.setJobDetails(jobOneDetail());
return scheduler;
}
#Bean
public Properties quartzProperties() throws IOException {
PropertiesFactoryBean propertiesFactoryBean = new PropertiesFactoryBean();
propertiesFactoryBean.setLocation(new ClassPathResource("/quartz.properties"));
propertiesFactoryBean.afterPropertiesSet();
return propertiesFactoryBean.getObject();
}