It seems Hikaricp was not used.
For example, spring.datasource.maximum-pool-size is always effected.
spring.datasource.hikari.maximum-pool-size is not affected.
I set the following:
in application.yml
spring:
datasource:
....
maximum-pool-size: 10
hikari:
connection-timeout: 60000
maximum-pool-size: 5
And then I checked the number of connection by netstat command.
There ware 10 connections.
It seems that maximum-pool-size of hikari doesn't work.
Even if I deleted the spring.datasource.maximum-pool-size, the
maximum-pool-size of hikari still doesn't work.
Moreover, I set the follwoing log event, but there were no log about HikariCP.
logging:
level:
ROOT: NOTE
org.springframework: DEBUG
Of course, I built with the follwoing dependencies to make sure it exculded tomcat-jdbc:
compile("org.springframework.boot:spring-boot-starter-data-jpa") {
exclude group: 'org.apache.tomcat', module: 'tomcat-jdbc'
}
compile("org.springframework.boot:spring-boot-starter-jdbc") {
exclude group: 'org.apache.tomcat', module: 'tomcat-jdbc'
}
compile("com.zaxxer:HikariCP:2.6.0")
Could you help to how to find the problem?
I got this working in my spring boot app which needed two database connections.
Here are my Configuration beans:
#Bean
#Primary
#ConfigurationProperties("spring.datasource.primary")
public DataSourceProperties dataSourcePropertiesPrimary() {
return new DataSourceProperties();
}
#Bean
#Primary
#ConfigurationProperties("spring.datasource.primary.hikari")
public HikariDataSource dataSourcePrimary() {
return dataSourcePropertiesPrimary()
.initializeDataSourceBuilder()
.type(HikariDataSource.class)
.build();
}
And my application.properties file:
spring.datasource.primary.hikari.minimum-idle=1
spring.datasource.primary.hikari.maximum-pool-size=3
You can use Jolokia to confirm before and after the pool size.
Alternatively you can confirm by running your app in debug mode and break pointing the file HikariConfig.java on the private method validateNumerics where maxPoolSize is set.
Related
I have following settings for my database (I have multiple databases, so they are configured in spring.datasource hierarchy.
spring:
datasource:
db-write:
url: jdbc:sqlserver://whatever.database.windows.net:1433;database=dbname;encrypt=true;trustServerCertificate=false;hostNameInCertificate=*.database.windows.net;loginTimeout=30;
username: 'myusername'
password: 'mynotsosecretpassword'
driver-class-name: com.microsoft.sqlserver.jdbc.SQLServerDriver
Then I am configuring my datasource here
#Configuration
#EnableTransactionManagement
#EnableJpaRepositories(basePackages = "com.datasources.dbwrite.repository",
entityManagerFactoryRef = "entityManagerFactoryDbWrite",
transactionManagerRef= "transactionManagerDbWrite"
)
public class DataSourceConfigurationDbWrite {
#Bean
#Primary
#ConfigurationProperties("spring.datasource.db-write")
public DataSourceProperties dataSourcePropertiesDbWrite() {
return new DataSourceProperties();
}
#Bean
#Primary
#ConfigurationProperties("spring.datasource.db-write.configuration")
public DataSource dataSourceDbWrite() {
return dataSourcePropertiesDbWrite().initializeDataSourceBuilder()
.type(HikariDataSource.class).build();
}
#Primary
#Bean(name = "entityManagerFactoryDbWrite")
public LocalContainerEntityManagerFactoryBean entityManagerFactoryDbWrite(EntityManagerFactoryBuilder builder) {
return builder
.dataSource(dataSourceDbWrite())
.packages("com.datasources.dbwrite.models")
.build();
}
#Primary
#Bean
public PlatformTransactionManager transactionManagerDbWrite(
final #Qualifier("entityManagerFactoryDbWrite") LocalContainerEntityManagerFactoryBean entityManagerFactoryDbWrite) {
return new JpaTransactionManager(Objects.requireNonNull(entityManagerFactoryDbWrite.getObject()));
}
}
I am configuring my hikari datasource in dataSourceDbWrite method based on the properties i read in dataSourcePropertiesDbWrite method. I believe i need to configure properties in specific hierarchy so that dataSourceDbWrite method can easily detect which properties are needed for hikari. Is that correct?
What that hierarchy would be?
Moreover, how can and where can i find what properties i can configure for hikari? connection-timeout? connection pool size etc?
Me personally prefer application.yml than code to configurate Hikari:
spring:
datasource:
driver-class-name: com.mysql.cj.jdbc.Driver
url: {JDBC URL}
username: {USERNAME}
password: {PASSWORD}
type: com.zaxxer.hikari.HikariDataSource
hikari:
minimum-idle: 5
idle-timeout: 600000
maximum-pool-size: 10
auto-commit: true
pool-name: HikariCorePool
max-lifetime: 1800000
connection-timeout: 30000
connection-test-query: select * from information_schema.tables limit 1
(BTW, that piece of configuration was originally writen by a colleague years ago. We didn't change it and just copy-and-paste into any new projects those years.😆)
If you want to check out all configurable fields, those spring.datasource.hikari.* keys inorg.springframework.boot:spring-boot-autoconfigure:{VERSION}/META-INF/spring/spring-configuration-metadata.json may could help.
And javadoc in com.zaxxer.hikari.HikariConfigMXBean could help too.
See example in article, the properties hierarchy are according to #ConfigurationProperties's value
If we want to configure Hikari, we just need to add a #ConfigurationProperties to the data source definition:
#Bean
#ConfigurationProperties("spring.datasource.todos.hikari")
public DataSource todosDataSource() {
return todosDataSourceProperties()
.initializeDataSourceBuilder()
.build();
}
Then, we can insert the following lines into the application.properties file:
spring.datasource.todos.hikari.connectionTimeout=30000
spring.datasource.todos.hikari.idleTimeout=600000
spring.datasource.todos.hikari.maxLifetime=1800000
See relevant hikari's spring properties
spring.datasource.hikari.connection-timeout
spring.datasource.hikari.data-source-class-name
spring.datasource.hikari.data-source-properties
spring.datasource.hikari.driver-class-name
spring.datasource.hikari.idle-timeout
spring.datasource.hikari.initialization-fail-timeout
spring.datasource.hikari.jdbc-url
spring.datasource.hikari.leak-detection-threshold
spring.datasource.hikari.login-timeout
spring.datasource.hikari.max-lifetime
spring.datasource.hikari.maximum-pool-size
spring.datasource.hikari.minimum-idle
spring.datasource.hikari.validation-timeout
And explanation on each property in HikariCP, for example
connectionTimeout
This property controls the maximum number of milliseconds that a client (that's you) will wait for a connection from the pool. If this time is exceeded without a connection becoming available, a SQLException will be thrown. Lowest acceptable connection timeout is 250 ms. Default: 30000 (30 seconds)
Notice that camelCase hikari properties (connectionTimeout) is shown as snake-case in spring (connection-timeout)
We are getting a lot of errors in our Spring boot app using Spring data Neo4j, caused by this error:
org.neo4j.driver.exceptions.ServiceUnavailableException: Connection pool for server server-url.example.com:xxxx is closed while acquiring a connection.
Does anyone have an idea of where this error comes from ? Do we have to tweak the pools on the database server side or on application side ?
We are using Neo4j 4.0.11 and spring-data-neo4j 6.1.1 (in spring-boot-starter-data-neo4j 2.5.0). The connection is made with bolt protocol.
EDIT:
Here are additional infos on our configuration:
application.yml:
server:
port: 9080
error.include-message: always
management:
endpoint:
health:
group:
readiness:
include: neo4j
liveness:
include: neo4j
logging.level:
root: WARN
org.springframework: INFO
io.package.directions: INFO
spring:
application.name: directions
cache:
cache-names: nodes, records
caffeine.spec: maximumSize=1000
config.location: classpath:/config/
profiles.active: prod
jmx.enabled: false
jackson.serialization:
FAIL_ON_EMPTY_BEANS: false
WRITE_DATES_AS_TIMESTAMPS: true
WRITE_DATE_TIMESTAMPS_AS_NANOSECONDS: false
spring.neo4j:
uri: ${NEO4J_URL}
authentication:
username: ${NEO4J_USERNAME}
password: ${NEO4J_PASSWORD}
mail:
api: "http://${MAIL_SERVICE_HOST:mail}:${MAIL_SERVICE_PORT:9000}"
We connect to another kubernetes pod using bolt protocol.
Here is our fairly simple transaction management bean:
#Configuration
#EnableTransactionManagement
public class TxConfig {
final Driver driver;
public TxConfig(Driver driver) {
this.driver = driver;
}
#Bean
public ReactiveTransactionManager reactiveTransactionManager() {
return new ReactiveNeo4jTransactionManager(driver);
}
}
I've created a Java Spring Boot service using the WebFlux reactive module, H2 in-memory database, and R2DBC reactive driver.
When I run the service, it fails with an "Unable to create a ConnectionFactory" error:
Unable to create a ConnectionFactory for 'ConnectionFactoryOptions{options={driver=h2, protocol=mem, database=contentitem, options=DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=FALSE}}'. Available drivers: [ pool ]
This is a nested exception that seems to start in the repository and then propagate back through the service to the controller.
Having read through the resulting stack trace carefully and finding no indication about what the problem might be, the only clue I can find is that the 'connectionFactory' input parameter in my CustomConnectionFactoryInitializer class (please see below) is being highlighted with a red squiggle ("Could not autowire. There is more than one bean of 'ConnectionFactory' type") ... except that there isn't. At least there's only one explicitly defined bean -- the one in my CustomConnectionFactoryInitializer class.
Any idea what's going on here?
Details
It's a Gradle project, and my build.gradle file is:
plugins {
id 'org.springframework.boot' version '2.3.9.RELEASE'
id 'io.spring.dependency-management' version '1.0.11.RELEASE'
id 'java'
}
group = 'com.mycorp.service'
version = '0.0.1-SNAPSHOT'
sourceCompatibility = '11'
configurations {
compileOnly {
extendsFrom annotationProcessor
}
}
repositories {
mavenCentral()
}
dependencies {
implementation 'org.springframework.boot:spring-boot-starter-data-r2dbc'
implementation 'org.springframework.boot:spring-boot-starter-webflux'
compileOnly 'org.projectlombok:lombok'
runtimeOnly 'com.h2database:h2'
runtimeOnly 'io.r2dbc:r2dbc-h2'
runtimeOnly 'io.r2dbc:r2dbc-postgresql'
runtimeOnly 'org.postgresql:postgresql'
annotationProcessor 'org.projectlombok:lombok'
testImplementation('org.springframework.boot:spring-boot-starter-test') {
exclude group: 'org.junit.vintage', module: 'junit-vintage-engine'
}
testImplementation 'io.projectreactor:reactor-test'
compile 'io.springfox:springfox-swagger2:3.0.0'
compile 'io.springfox:springfox-swagger-ui:3.0.0'
compile 'io.springfox:springfox-spring-webflux:3.0.0'
}
test {
useJUnitPlatform()
}
I've added a schema.sql file under main/resources, which contains the following:
CREATE TABLE contentitem ( contentItemId INT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY, localizedName VARCHAR(100) NOT NULL);
I populate the table with a data.sql file in the same directory:
INSERT INTO contentitem (contentItemId, localizedName) VALUES (0, 'Zero');
INSERT INTO contentitem (contentItemId, localizedName) VALUES (1, 'One');
INSERT INTO contentitem (contentItemId, localizedName) VALUES (2, 'Two');
I've created a CustomConnectionFactoryInitializer to create and populate the database:
#Configuration
public class CustomConnectionFactoryInitializer {
#Bean
public ConnectionFactoryInitializer initializer(ConnectionFactory connectionFactory) {
ConnectionFactoryInitializer initializer = new ConnectionFactoryInitializer();
initializer.setConnectionFactory(connectionFactory);
CompositeDatabasePopulator populator = new CompositeDatabasePopulator();
populator.addPopulators(new ResourceDatabasePopulator(new ClassPathResource("schema.sql")));
initializer.setDatabasePopulator(populator);
return initializer;
}
}
I've defined a 'test' profile using in-memory H2 and made it active in my application.yml file:
spring:
profiles:
active: test
---
spring:
profiles: dev
r2dbc:
url: r2dbc:postgresql://localhost:5432/test
username: postgres
password: postgres
logging:
level:
org.springframework.data.r2dbc: Debug
---
spring:
profiles: test
r2dbc:
url: r2dbc:h2:mem:///contentitem?options=DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=FALSE
name: sa
password:
---
spring:
profiles: prod
r2dbc:
url: r2dbc:postgresql://localhost:5432/test
username: postgres
password: postgres
logging:
level:
org.springframework.data.r2dbc: Debug
My ContentItem model is:
#Data
#AllArgsConstructor
#NoArgsConstructor
#Table("contentitem")
public class ContentItem {
#Id
#Column("contentItemId")
private Integer contentItemId;
#Column("localizedName")
private String localizedName;
}
My ContentItemController is:
#RestController
#RequestMapping("/contentItems")
public class ContentItemController {
#Autowired
private ContentItemService contentItemService;
#GetMapping("/{contentItemId}")
public Mono<ResponseEntity<ContentItem>> getContentItemByUserId(#PathVariable Integer contentItemId){
Mono<ContentItem> contentItem = contentItemService.getContentItemById(contentItemId);
return contentItem.map( u -> ResponseEntity.ok(u))
.defaultIfEmpty(ResponseEntity.notFound().build());
}
My ContentItemService is:
#Service
#Slf4j
#Transactional
public class ContentItemService {
#Autowired
private ContentItemRepository contentItemRepository;
public Mono<ContentItem> getContentItemById(Integer contentItemId){
return contentItemRepository.findByContentItemId(contentItemId);
}
}
And my ContentItemRepository is:
public interface ContentItemRepository extends ReactiveCrudRepository<ContentItem,Integer> {
Mono<ContentItem> findByContentItemId(Integer contentItemId);
}
Complicating all this is that the H2 console, which I've enabled in the application.properties file with spring.h2.console.enabled=true is failing with a 404 Not Found error when I call it with http://localhost:8081/h2-console. Any ideas what could be going on here?
For me r2dbc-postgresql was missing in the pom.xml
<dependency>
<groupId>io.r2dbc</groupId>
<artifactId>r2dbc-postgresql</artifactId>
</dependency>
OK, so I went back through my project file by file, diffing each file with a copy of the repo I was using as a guide. I found some extra database connection configuration code I thought I'd gotten rid of. As soon as I removed it, problem solved. Thanks to everyone who took a look and offered suggestions.
I have written a class implementing HealthIndicator, overriding the health-method. I return Health.down().withDetail("SupportServiceStatus", "UP").build();
This should make my health-endpoint return:
{
"status":"UP",
"applicationHealth": {
"status":"UP"
}
}
Instead it just returns (health, without details):
{
"status":"UP",
}
Javacode (somewhat simplified):
#Component
public class ApplicationHealth implements HealthIndicator {
#Override
public Health health() {
return check();
}
private Health check() {
return Health.up().withDetail("SupportServiceStatus", supportServiceStatusCode).build();
}
}
According to spring-boot docs:
. . . by default, only the health status is exposed over an unauthenticated HTTP connection. If you are happy for complete health information to always be exposed you can set endpoints.health.sensitive to false.
Solution is to set endpoints.health.sensitive to false in application.properties.
application.properties
endpoints.health.sensitive=false
For >1.5.1 application.properties
management.security.enabled=false
At Spring Boot 2.0.0.RELEASE (thx #rvit34 and #nisarg-panchal):
management:
endpoint:
health:
show-details: "ALWAYS"
endpoints:
web:
exposure:
include: "*"
management.endpoints.web.exposure.include=* exposes all endpoints, if that is what you want.
Current documentation can be found here: https://docs.spring.io/spring-boot/docs/current/reference/html/production-ready-endpoints.html
At Spring Boot 2.0.0.RELEASE:
management:
endpoint:
health:
show-details: "ALWAYS"
Thanks #rvit34 and #Ninja Code Monkey its working.
For Springboot 2.x.x.RELEASE,
Use below for application.properties,
management.endpoint.health.show-details=ALWAYS
Use below for applicaton.yml,
management:
endpoint:
health:
show-details: "ALWAYS"
Setting 'endpoints.health.sensitive' made no difference... had to set:
management:
security:
enabled: false
need to add
management.endpoint.health.show-details=always
to Application.properties
For Spring boot 2.X I have following in my application.properties file for detailed information:
management.endpoints.web.exposure.include=*
management.endpoint.health.show-details=ALWAYS
I had this same problem, on version Spring Boot 1.5.9 I had to set
management.security.enabled=false
I'm attempting to build code into our base pom which autoconfigures Spring Cloud Config server lookup through Eureka. We're doing this to avoid templating .yml properties for developers building microservices. For example, we want to java config all behavior triggered from these properties:
spring:
application:
name: MyMicroservice
cloud:
config:
enabled: true
server:
prefix: /diagnostics/admin/config
failFast: true
discovery:
enabled: true
serviceId: echo
management:
context-path: /diagnostics/admin
eureka:
password: password
client:
serviceUrl:
defaultZone: http://user:${eureka.password}#localhost:8761/eureka/
instance:
leaseRenewalIntervalInSeconds: 10
statusPageUrlPath: /diagnostics/admin/info
healthCheckUrlPath: /diagnostics/admin/health
After much experimenting, the following approach mostly works except for the Eureka-discovered config server (resulting in no overridden config properties):
#Order(-1)
public class AdditionalBootstrapPropertySourceLocator implements PropertySourceLocator {
#Override
public PropertySource<?> locate(Environment environment) {
Map<String, Object> theBootstrapYmlConfig = new HashMap<>();
theBootstrapYmlConfig.put("spring.cloud.config.enabled", new Boolean(true));
theBootstrapYmlConfig.put("spring.cloud.config.server.prefix", "/diagnostics/admin/config");
theBootstrapYmlConfig.put("spring.cloud.config.failFast", new Boolean(true));
theBootstrapYmlConfig.put("spring.cloud.config.discovery.enabled", new Boolean(true));
theBootstrapYmlConfig.put("spring.cloud.config.discovery.serviceId", "echo");
theBootstrapYmlConfig.put("management.context-path", "/diagnostics/admin");
theBootstrapYmlConfig.put("eureka.client.serviceUrl.defaultZone", "http://user:password#localhost:8761/eureka/");
theBootstrapYmlConfig.put("eureka.instance.leaseRenewalIntervalInSeconds", new Integer(10));
theBootstrapYmlConfig.put("eureka.instance.statusPageUrlPath", "/diagnostics/admin/info");
theBootstrapYmlConfig.put("eureka.instance.healthCheckUrlPath", "/diagnostics/admin/health");
return new MapPropertySource("myExtraBootstrap", theBootstrapYmlConfig);
}
}
And I seem to need this Bean as well:
#ConditionalOnWebApplication
#Configuration
#Import(EurekaClientAutoConfiguration.class)
public class WorkfrontDiscoveryClientConfigServiceBootstrapConfiguration {
#Bean
#ConditionalOnClass({ DiscoveryClient.class, ConfigServicePropertySourceLocator.class })
#ConditionalOnMissingBean
DiscoveryClientConfigServiceBootstrapConfiguration discoveryClientConfigServiceBootstrapConfiguration() {
DiscoveryClientConfigServiceBootstrapConfiguration discoveryClientConfigServiceBootstrapConfiguration =
new DiscoveryClientConfigServiceBootstrapConfiguration();
return discoveryClientConfigServiceBootstrapConfiguration;
}
}
Finally, I put both into spring.factories to ensure they are constructed. The problem is that the PropertySourceLocator is never used to construct the call within ConfigServicePropertySourceLocator to retrieve the properties. No matter what I do, I cant seem to match the behaviors that specifying the properties within bootstrap.yml would produce.
Edit 4 days later
The key factor (and restriction) here is the ability to look up the config server through Eureka. In the current spring cloud release (1.0.2), the property source is retrieved and constructed too early in the spring initialization cycle for the config-lookup-through-eureka java property source config I have above. Plus if the Eureka server is slow or not available at bootstrap startup time, the Config server property source is never reconstructed when Eureka finally comes up. This in my mind is a bug.
I solved this all by eliminating the concept of looking up the config server through Eureka, and requiring this minimum config in bootstrap.yml:
spring:
application:
name: MyMicroservice
cloud:
config:
uri: http://localhost:8888/diagnostics/admin/config
eureka:
client:
serviceUrl:
defaultZone: http://user:password#localhost:8761/eureka/
and then setting the rest in the java AdditionalBootstrapPropertySourceLocator
Edit 30 days later
Java-configing bootstrap properties continues to be a challenge. I'm doing this because I'm developing a framework without templating or code generation (the premise of spring boot). I've added spring-retry to the mix and client-to-config gets retried but re-registration to Eureka does not. This is why Eureka-first had to be abandoned for me. I'd put my vote in for integrating spring-retry into the Eureka registration process so I can go back to Eureka-first for my framework. Still on Spring Cloud 1.0.2.
Edit 100 days later
Update for where we ended up. Continuing along our mantra of avoiding property templating, enforcing policies and practices within code .. and continuing without a Eureka-first concept, we abandoned PropertySourceLocator and simply used a SpringApplicationRunListener as follows:
public class OurFrameworkProperties implements SpringApplicationRunListener {
:
public void started() {
if (TestCaseUtils.isRunningFromTestCase()) {
System.setProperty("spring.cloud.config.failFast", "false");
System.setProperty("spring.cloud.config.enabled", "false");
System.setProperty("eureka.client.enabled", "false");
} else {
// set production values same way
}
}
}
A caution that this started() actually gets called twice inside the spring code (once not passing any program arguments btw) everytime your Spring application runs or gets an Actuator refresh().
If your PropertySourceLocator is listed inspring.factories (I assume as a BootstrapConfiguration) then it needs to be a #Component (or maybe even a #Configuration).
you have to set this properties in the boostrap.properties
eureka.instance.metadataMap.configPath: /your-app-name
and comment this one
#spring.cloud.config.uri=http://localhost:8888/
and obviously it must be also this
eureka.client.serviceUrl.defaultZone: ${EUREKA_URI:http://localhost:8761/eureka}
eureka.client.instance.preferIpAddress: true
according with the documentation
https://cloud.spring.io/spring-cloud-config/multi/multi__spring_cloud_config_client.html#discovery-first-bootstrap