We are getting a lot of errors in our Spring boot app using Spring data Neo4j, caused by this error:
org.neo4j.driver.exceptions.ServiceUnavailableException: Connection pool for server server-url.example.com:xxxx is closed while acquiring a connection.
Does anyone have an idea of where this error comes from ? Do we have to tweak the pools on the database server side or on application side ?
We are using Neo4j 4.0.11 and spring-data-neo4j 6.1.1 (in spring-boot-starter-data-neo4j 2.5.0). The connection is made with bolt protocol.
EDIT:
Here are additional infos on our configuration:
application.yml:
server:
port: 9080
error.include-message: always
management:
endpoint:
health:
group:
readiness:
include: neo4j
liveness:
include: neo4j
logging.level:
root: WARN
org.springframework: INFO
io.package.directions: INFO
spring:
application.name: directions
cache:
cache-names: nodes, records
caffeine.spec: maximumSize=1000
config.location: classpath:/config/
profiles.active: prod
jmx.enabled: false
jackson.serialization:
FAIL_ON_EMPTY_BEANS: false
WRITE_DATES_AS_TIMESTAMPS: true
WRITE_DATE_TIMESTAMPS_AS_NANOSECONDS: false
spring.neo4j:
uri: ${NEO4J_URL}
authentication:
username: ${NEO4J_USERNAME}
password: ${NEO4J_PASSWORD}
mail:
api: "http://${MAIL_SERVICE_HOST:mail}:${MAIL_SERVICE_PORT:9000}"
We connect to another kubernetes pod using bolt protocol.
Here is our fairly simple transaction management bean:
#Configuration
#EnableTransactionManagement
public class TxConfig {
final Driver driver;
public TxConfig(Driver driver) {
this.driver = driver;
}
#Bean
public ReactiveTransactionManager reactiveTransactionManager() {
return new ReactiveNeo4jTransactionManager(driver);
}
}
Related
I work on a java spring boot project.
I have an application.yml file that contains the connection string:
spring:
data:
mongodb:
uri: mongodb://localhost:27017/development
I comment on connection string rows in the application.yml file:
#spring:
# data:
# mongodb:
# uri: mongodb://localhost:27017/development
And then I rebuild the project and trigger repository functions.
To my surprise, I do not get any exceptions or errors while the repositories are executed and application.yml has commented connection string.
The result that I get from repository functions is an empty result.
So my question is very simple, why i do not get any indications on a runtime that the connection string is removed does spring create some default connection in case it cannot find the conn string in yml file?
I can't say if there is anything specific with Mongo, but for Spring in general you can create typesafe config and validate it on startup.
application.yml
myapp:
mongo-uri: mongodb://localhost:27017/development
spring:
data:
mongodb:
uri: ${myapp.mongo-uri}
MyAppProperties.java
#Validated
#ConfigurationProperties(prefix = "myapp")
public class MyAppProperties {
#NotEmpty
private String mongoUri;
// accessors
}
In #Configuration's or #SpringBootApplication
#EnableConfigurationProperties(MyAppProperties.class)
The key things are #Validated and #NotEmpty. You will get startup error if ${myapp.mongo-uri} is not provided.
It seems Hikaricp was not used.
For example, spring.datasource.maximum-pool-size is always effected.
spring.datasource.hikari.maximum-pool-size is not affected.
I set the following:
in application.yml
spring:
datasource:
....
maximum-pool-size: 10
hikari:
connection-timeout: 60000
maximum-pool-size: 5
And then I checked the number of connection by netstat command.
There ware 10 connections.
It seems that maximum-pool-size of hikari doesn't work.
Even if I deleted the spring.datasource.maximum-pool-size, the
maximum-pool-size of hikari still doesn't work.
Moreover, I set the follwoing log event, but there were no log about HikariCP.
logging:
level:
ROOT: NOTE
org.springframework: DEBUG
Of course, I built with the follwoing dependencies to make sure it exculded tomcat-jdbc:
compile("org.springframework.boot:spring-boot-starter-data-jpa") {
exclude group: 'org.apache.tomcat', module: 'tomcat-jdbc'
}
compile("org.springframework.boot:spring-boot-starter-jdbc") {
exclude group: 'org.apache.tomcat', module: 'tomcat-jdbc'
}
compile("com.zaxxer:HikariCP:2.6.0")
Could you help to how to find the problem?
I got this working in my spring boot app which needed two database connections.
Here are my Configuration beans:
#Bean
#Primary
#ConfigurationProperties("spring.datasource.primary")
public DataSourceProperties dataSourcePropertiesPrimary() {
return new DataSourceProperties();
}
#Bean
#Primary
#ConfigurationProperties("spring.datasource.primary.hikari")
public HikariDataSource dataSourcePrimary() {
return dataSourcePropertiesPrimary()
.initializeDataSourceBuilder()
.type(HikariDataSource.class)
.build();
}
And my application.properties file:
spring.datasource.primary.hikari.minimum-idle=1
spring.datasource.primary.hikari.maximum-pool-size=3
You can use Jolokia to confirm before and after the pool size.
Alternatively you can confirm by running your app in debug mode and break pointing the file HikariConfig.java on the private method validateNumerics where maxPoolSize is set.
I have this config file:
liquibase:
enabled: true
change-log: classpath:/db/changelog/db.changelog-master.yml
management:
security:
enabled: false
server:
port: 8080
spring:
datasource:
driverClassName: org.h2.Driver
url: jdbc:h2:file:./target/h2db/db/develop;DB_CLOSE_DELAY=-1
username: sa
password:
h2:
console:
enabled: true
When I try to check datasorce it connects to "jdbc:h2:mem:testdb" anyway:
#Component
public class AfterInit implements CommandLineRunner {
#Autowired
DataSource dataSource;
#Override
public void run(String... args) throws Exception {
out.println(dataSource);
}
}
Why spring boot can't find proper database config?
When I changed dependency from runtime("com.h2database:h2") to compile("com.h2database:h2") Datasource became work.
Can anybody explain me what's happend?
I have an application annotated like this
#SpringBootApplication(exclude = {DefaultNotificationServiceConfig.class})
#EnableDiscoveryClient
#EnableSwagger2
#EnableSpringDataWebSupport
#EnableJpaRepositories(basePackages = {"com.repositories.jpa"})
#EnableMongoRepositories(basePackages = {"com.repositories.mongo"})
public class DemoApplication {
private static ApplicationContext ctx;
public static void main(String[] args) {
ctx = SpringApplication.run(new Object[]{DemoApplication.class}, args);
System.out.println(ctx.getEnvironment());
}
}
I want it to use consul, and it is getting the active profiles from consul because I see this line in the logs when starting it
2016-09-19 20:38:29.947 INFO 9556 --- [ main] b.c.PropertySourceBootstrapConfiguration : Located property source: CompositePropertySource [name='consul', propertySources=[ConsulPropertySource [name='com/app/'], ConsulPropertySource [name='com/application/']]]
2016-09-19 20:38:29.975 INFO 9556 --- [ main] com.Application : The following profiles are active: dev,im3
However its not using the mongo and mysql databases I've specified in the dev profile. Its trying to connect to a mongo db at localhost which is not the one in my properties
Caused by: com.mongodb.MongoTimeoutException: Timed out after 10000 ms while waiting for a server that matches AnyServerSelector{}. Client view of cluster state is {type=Unknown, servers=[{address=localhost:27017
and liquibase is trying to use a default database not listed in my properties
INFO 9/19/16 8:38 PM: liquibase: Creating database history table with name: PUBLIC.DATABASECHANGELOG
Here is the dev profile from consul
server:
port: 8090
spring:
redis:
host: ubuntu
port: 6379
data:
mongodb:
uri: mongodb://ubuntu:27017/test
datasource:
url: jdbc:mysql://ubuntu:3309/test
username: uesr
password: password
driverClassName: com.mysql.jdbc.Driver
testOnBorrow: true
validationQuery: select 1
multipart:
maxFileSize: 250MB
maxRequestSize: 250MB
I have another spring boot application pointing to the same consul instance and using the same profiles and that one is working. I also have bootstrap.yml in my classpath
spring:
application:
name: ${APPLICATION_NAME:app}
cloud:
consul:
host: ${CONSUL_HOST:localhost}
port: ${CONSUL_PORT:8500}
config:
format: YAML
watch:
delay: 10
prefix: app
I'm using eclipse and in the run configurations I'm using a main class from a dependent project when starting the app that looks like below, but the project I am running from is the project containing the application class above.
#SpringBootApplication(exclude = {DefaultNotificationServiceConfig.class})
#EnableSpringDataWebSupport
#EnableDiscoveryClient
#EnableSwagger2
#EnableJpaRepositories(basePackages = {"com.repositories.jpa"})
#EnableMongoRepositories(basePackages = {"com.repositories.mongo"})
public class Application {
private static ApplicationContext ctx;
public static void main(String[] args) throws Exception {
TimeZone.setDefault(TimeZone.getTimeZone("Etc/UTC"));
ctx = SpringApplication.run(new Object[]{Application.class}, args);
System.out.println(ctx);
}
}
Update
The app that's working states the profiles being used in the log
2016-09-19 23:53:55.265 INFO 9744 --- [ main] b.c.PropertySourceBootstrapConfiguration : Located property source: CompositePropertySource [name='consul', propertySources=[ConsulPropertySource [name='com/app,im3/'], ConsulPropertySource [name='com/app,dev/'], ConsulPropertySource [name='com/app/'], ConsulPropertySource [name='com/application,im3/'], ConsulPropertySource [name='com/application,dev/'], ConsulPropertySource [name='com/application/']]]
I have 3 config files in consul, com/app and com/app,dev and com/app,im3. com/app looks like this
server:
session:
timeout: 600
spring:
profiles:
active: dev,im3
jpa:
hibernate:
ddl-auto: none
naming-strategy: com.config.OracleNamingStrategy
show-sql: false
thymeleaf:
mode: LEGACYHTML5
logging:
level:
org:
springframework: ERROR
thymeleaf: ERROR
hibernate: ERROR
springfox: ERROR
com:
bosch:
mercurio: DEBUG
Stupid mistake, I defined the environment variables but not the VM argument
-Dspring.profiles.active=dev,im3
it doesn't work if I remove
spring:
profiles:
active: dev,im3
from com/app, it seems redundant that I need both
I'm attempting to build code into our base pom which autoconfigures Spring Cloud Config server lookup through Eureka. We're doing this to avoid templating .yml properties for developers building microservices. For example, we want to java config all behavior triggered from these properties:
spring:
application:
name: MyMicroservice
cloud:
config:
enabled: true
server:
prefix: /diagnostics/admin/config
failFast: true
discovery:
enabled: true
serviceId: echo
management:
context-path: /diagnostics/admin
eureka:
password: password
client:
serviceUrl:
defaultZone: http://user:${eureka.password}#localhost:8761/eureka/
instance:
leaseRenewalIntervalInSeconds: 10
statusPageUrlPath: /diagnostics/admin/info
healthCheckUrlPath: /diagnostics/admin/health
After much experimenting, the following approach mostly works except for the Eureka-discovered config server (resulting in no overridden config properties):
#Order(-1)
public class AdditionalBootstrapPropertySourceLocator implements PropertySourceLocator {
#Override
public PropertySource<?> locate(Environment environment) {
Map<String, Object> theBootstrapYmlConfig = new HashMap<>();
theBootstrapYmlConfig.put("spring.cloud.config.enabled", new Boolean(true));
theBootstrapYmlConfig.put("spring.cloud.config.server.prefix", "/diagnostics/admin/config");
theBootstrapYmlConfig.put("spring.cloud.config.failFast", new Boolean(true));
theBootstrapYmlConfig.put("spring.cloud.config.discovery.enabled", new Boolean(true));
theBootstrapYmlConfig.put("spring.cloud.config.discovery.serviceId", "echo");
theBootstrapYmlConfig.put("management.context-path", "/diagnostics/admin");
theBootstrapYmlConfig.put("eureka.client.serviceUrl.defaultZone", "http://user:password#localhost:8761/eureka/");
theBootstrapYmlConfig.put("eureka.instance.leaseRenewalIntervalInSeconds", new Integer(10));
theBootstrapYmlConfig.put("eureka.instance.statusPageUrlPath", "/diagnostics/admin/info");
theBootstrapYmlConfig.put("eureka.instance.healthCheckUrlPath", "/diagnostics/admin/health");
return new MapPropertySource("myExtraBootstrap", theBootstrapYmlConfig);
}
}
And I seem to need this Bean as well:
#ConditionalOnWebApplication
#Configuration
#Import(EurekaClientAutoConfiguration.class)
public class WorkfrontDiscoveryClientConfigServiceBootstrapConfiguration {
#Bean
#ConditionalOnClass({ DiscoveryClient.class, ConfigServicePropertySourceLocator.class })
#ConditionalOnMissingBean
DiscoveryClientConfigServiceBootstrapConfiguration discoveryClientConfigServiceBootstrapConfiguration() {
DiscoveryClientConfigServiceBootstrapConfiguration discoveryClientConfigServiceBootstrapConfiguration =
new DiscoveryClientConfigServiceBootstrapConfiguration();
return discoveryClientConfigServiceBootstrapConfiguration;
}
}
Finally, I put both into spring.factories to ensure they are constructed. The problem is that the PropertySourceLocator is never used to construct the call within ConfigServicePropertySourceLocator to retrieve the properties. No matter what I do, I cant seem to match the behaviors that specifying the properties within bootstrap.yml would produce.
Edit 4 days later
The key factor (and restriction) here is the ability to look up the config server through Eureka. In the current spring cloud release (1.0.2), the property source is retrieved and constructed too early in the spring initialization cycle for the config-lookup-through-eureka java property source config I have above. Plus if the Eureka server is slow or not available at bootstrap startup time, the Config server property source is never reconstructed when Eureka finally comes up. This in my mind is a bug.
I solved this all by eliminating the concept of looking up the config server through Eureka, and requiring this minimum config in bootstrap.yml:
spring:
application:
name: MyMicroservice
cloud:
config:
uri: http://localhost:8888/diagnostics/admin/config
eureka:
client:
serviceUrl:
defaultZone: http://user:password#localhost:8761/eureka/
and then setting the rest in the java AdditionalBootstrapPropertySourceLocator
Edit 30 days later
Java-configing bootstrap properties continues to be a challenge. I'm doing this because I'm developing a framework without templating or code generation (the premise of spring boot). I've added spring-retry to the mix and client-to-config gets retried but re-registration to Eureka does not. This is why Eureka-first had to be abandoned for me. I'd put my vote in for integrating spring-retry into the Eureka registration process so I can go back to Eureka-first for my framework. Still on Spring Cloud 1.0.2.
Edit 100 days later
Update for where we ended up. Continuing along our mantra of avoiding property templating, enforcing policies and practices within code .. and continuing without a Eureka-first concept, we abandoned PropertySourceLocator and simply used a SpringApplicationRunListener as follows:
public class OurFrameworkProperties implements SpringApplicationRunListener {
:
public void started() {
if (TestCaseUtils.isRunningFromTestCase()) {
System.setProperty("spring.cloud.config.failFast", "false");
System.setProperty("spring.cloud.config.enabled", "false");
System.setProperty("eureka.client.enabled", "false");
} else {
// set production values same way
}
}
}
A caution that this started() actually gets called twice inside the spring code (once not passing any program arguments btw) everytime your Spring application runs or gets an Actuator refresh().
If your PropertySourceLocator is listed inspring.factories (I assume as a BootstrapConfiguration) then it needs to be a #Component (or maybe even a #Configuration).
you have to set this properties in the boostrap.properties
eureka.instance.metadataMap.configPath: /your-app-name
and comment this one
#spring.cloud.config.uri=http://localhost:8888/
and obviously it must be also this
eureka.client.serviceUrl.defaultZone: ${EUREKA_URI:http://localhost:8761/eureka}
eureka.client.instance.preferIpAddress: true
according with the documentation
https://cloud.spring.io/spring-cloud-config/multi/multi__spring_cloud_config_client.html#discovery-first-bootstrap