If you prefer using configuration properties to configured all #FeignClient, you can create configuration properties with default feign name. It's also possible to set these timeouts per specific client by naming the client. And, we could, of course, list a global setting and also per-client overrides together without a problem.
My client:
#FeignClient( contextId = "fooFeignClient", name = "foo-client", url = "${foo.url}",
fallbackFactory = FooFallBackFactory.class,
configuration = FooFeignConfiguration.class)
I'm trying to do this and I want foo-client.readTimeout to override default.readTimeout when I'm using foo-client:
feign:
hystrix:
enabled: true
client:
config:
default:
connectTimeout: 3000
readTimeout: 3000
loggerLevel: full
foo-client:
connectTimeout: 3000
readTimeout: 5000
hystrix:
command:
default:
execution:
timeout:
enabled: "false"
isolation:
strategy: "THREAD"
thread:
timeoutInMilliseconds: "3000"
But this is not happening. I'm not sure if Hystrix's timeoutInMilliseconds could be impacting, but from my tests, it's not interfering. I want the feign.readTimeout to be 5000 only for the foo-client, not for the other clients. But it looks like it is ignoring this foo-client configuration and using only the default configuration.
I know it's not a #Configuration class problem because the Feign documentation says that if we create both #Configuration bean and configuration properties, configuration properties will win.
As you have seen in the documentation, you can set timeouts per specific client. But you have only configured it for feign client, not for hystrix command. Remember that they have independent timeouts.
I would recommend to use a configuration class instead of editing your application.yml in order to handle multiples feign clients with hystrix. The following example sets up a Feign.Builder where timeouts are being injected for Feign client and Hystrix command.
# application.yml
microservices:
foo:
feign:
url: xxxx
connect-timeout: 5s
read-timeout: 5s
hystrix:
enabled: true
timeout: 5s
#FeignClient(name = "foo",
url = "${microservice.foo.feign.url}",
configuration = FooFeignConfiguration.class)
public interface FooFeignRepository {
}
#Configuration
#RequiredArgsConstructor
public class FooFeignConfiguration {
#Value("${microservice.foo.hystrix.enabled:true}")
private final Boolean hystrixEnabled;
#DurationUnit(value = ChronoUnit.MILLIS)
#Value("${microservice.foo.feign.connect-timeout:5000}")
private final Duration feignConnectTimeout;
#DurationUnit(value = ChronoUnit.MILLIS)
#Value("${microservice.foo.feign.read-timeout:5000}")
private final Duration feignReadTimeout;
#DurationUnit(value = ChronoUnit.MILLIS)
#Value("${microservice.foo.hystrix.timeout:5000}")
private final Duration hystrixTimeout;
#Bean
#Scope("prototype")
public Feign.Builder feignBuilder() {
return (hystrixEnabled ?
HystrixFeign.builder()
.options(new Request.Options(
(int) feignConnectTimeout.toMillis(),
(int) feignReadTimeout.toMillis()))
.setterFactory((target, method) -> {
return new SetterFactory.Default()
.create(target, method)
.andCommandPropertiesDefaults(HystrixCommandProperties.Setter()
.withExecutionTimeoutInMilliseconds((int) hystrixTimeout.toMillis()));
}):
Feign.builder())
.retryer(Retryer.NEVER_RETRY);
}
}
Related
I have following settings for my database (I have multiple databases, so they are configured in spring.datasource hierarchy.
spring:
datasource:
db-write:
url: jdbc:sqlserver://whatever.database.windows.net:1433;database=dbname;encrypt=true;trustServerCertificate=false;hostNameInCertificate=*.database.windows.net;loginTimeout=30;
username: 'myusername'
password: 'mynotsosecretpassword'
driver-class-name: com.microsoft.sqlserver.jdbc.SQLServerDriver
Then I am configuring my datasource here
#Configuration
#EnableTransactionManagement
#EnableJpaRepositories(basePackages = "com.datasources.dbwrite.repository",
entityManagerFactoryRef = "entityManagerFactoryDbWrite",
transactionManagerRef= "transactionManagerDbWrite"
)
public class DataSourceConfigurationDbWrite {
#Bean
#Primary
#ConfigurationProperties("spring.datasource.db-write")
public DataSourceProperties dataSourcePropertiesDbWrite() {
return new DataSourceProperties();
}
#Bean
#Primary
#ConfigurationProperties("spring.datasource.db-write.configuration")
public DataSource dataSourceDbWrite() {
return dataSourcePropertiesDbWrite().initializeDataSourceBuilder()
.type(HikariDataSource.class).build();
}
#Primary
#Bean(name = "entityManagerFactoryDbWrite")
public LocalContainerEntityManagerFactoryBean entityManagerFactoryDbWrite(EntityManagerFactoryBuilder builder) {
return builder
.dataSource(dataSourceDbWrite())
.packages("com.datasources.dbwrite.models")
.build();
}
#Primary
#Bean
public PlatformTransactionManager transactionManagerDbWrite(
final #Qualifier("entityManagerFactoryDbWrite") LocalContainerEntityManagerFactoryBean entityManagerFactoryDbWrite) {
return new JpaTransactionManager(Objects.requireNonNull(entityManagerFactoryDbWrite.getObject()));
}
}
I am configuring my hikari datasource in dataSourceDbWrite method based on the properties i read in dataSourcePropertiesDbWrite method. I believe i need to configure properties in specific hierarchy so that dataSourceDbWrite method can easily detect which properties are needed for hikari. Is that correct?
What that hierarchy would be?
Moreover, how can and where can i find what properties i can configure for hikari? connection-timeout? connection pool size etc?
Me personally prefer application.yml than code to configurate Hikari:
spring:
datasource:
driver-class-name: com.mysql.cj.jdbc.Driver
url: {JDBC URL}
username: {USERNAME}
password: {PASSWORD}
type: com.zaxxer.hikari.HikariDataSource
hikari:
minimum-idle: 5
idle-timeout: 600000
maximum-pool-size: 10
auto-commit: true
pool-name: HikariCorePool
max-lifetime: 1800000
connection-timeout: 30000
connection-test-query: select * from information_schema.tables limit 1
(BTW, that piece of configuration was originally writen by a colleague years ago. We didn't change it and just copy-and-paste into any new projects those years.😆)
If you want to check out all configurable fields, those spring.datasource.hikari.* keys inorg.springframework.boot:spring-boot-autoconfigure:{VERSION}/META-INF/spring/spring-configuration-metadata.json may could help.
And javadoc in com.zaxxer.hikari.HikariConfigMXBean could help too.
See example in article, the properties hierarchy are according to #ConfigurationProperties's value
If we want to configure Hikari, we just need to add a #ConfigurationProperties to the data source definition:
#Bean
#ConfigurationProperties("spring.datasource.todos.hikari")
public DataSource todosDataSource() {
return todosDataSourceProperties()
.initializeDataSourceBuilder()
.build();
}
Then, we can insert the following lines into the application.properties file:
spring.datasource.todos.hikari.connectionTimeout=30000
spring.datasource.todos.hikari.idleTimeout=600000
spring.datasource.todos.hikari.maxLifetime=1800000
See relevant hikari's spring properties
spring.datasource.hikari.connection-timeout
spring.datasource.hikari.data-source-class-name
spring.datasource.hikari.data-source-properties
spring.datasource.hikari.driver-class-name
spring.datasource.hikari.idle-timeout
spring.datasource.hikari.initialization-fail-timeout
spring.datasource.hikari.jdbc-url
spring.datasource.hikari.leak-detection-threshold
spring.datasource.hikari.login-timeout
spring.datasource.hikari.max-lifetime
spring.datasource.hikari.maximum-pool-size
spring.datasource.hikari.minimum-idle
spring.datasource.hikari.validation-timeout
And explanation on each property in HikariCP, for example
connectionTimeout
This property controls the maximum number of milliseconds that a client (that's you) will wait for a connection from the pool. If this time is exceeded without a connection becoming available, a SQLException will be thrown. Lowest acceptable connection timeout is 250 ms. Default: 30000 (30 seconds)
Notice that camelCase hikari properties (connectionTimeout) is shown as snake-case in spring (connection-timeout)
I know how to create a single JMS listener with Springboot with annotations. But now I want to create several JMS listeners listening at several brokers sending same kind of messages at server startup, reading in a file the properties of brokers.
How can I achieve this ? Is it possible to ask SpringBoot to create the beans with java statements instead of annotations ? With a factory or something like that ?
I know there won't be more than 10 brokers in the system. Is there a solution to define statically 10 JMS listeners with annotations but deactivating some listeners if they are not used so that they don't cause errors ?
My answer relates to: "Is there a solution to define statically 10 JMS listeners with annotations but deactivating some listeners if they are not used so that they don't cause errors ?" and not the Dynamic portion of creating JMSListeners on the fly.
You can use #ConditionalOnProperty to enable/disable your consumer and use profiles to specify when they are enabled.
Example:
#Slf4j
#Service
#ConditionalOnProperty(name = "adapter-app-config.jms.request.enable-consumer", havingValue = "true")
public class RequestMessageConsumer {
#Autowired
private AdapterConfig adapterConfig;
#Autowired
private RequestMessageHandler requestMessageHandler;
#JmsListener(concurrency = "1", destination = "${adapter-app-config.jms.request.InQueue}")
#Transactional(rollbackFor = { Exception.class })
public void onMessage(TextMessage message) {
requestMessageHandler.handleMessage(message);
}
}
application.properties
adapter-app-config:
jms:
sync:
enable-consumer: true
request:
enable-consumer: false
For Dynamic, please see:
Adding Dynamic Number of Listeners(Spring JMS)
https://docs.spring.io/spring-framework/docs/current/reference/html/integration.html#jms-annotated-programmatic-registration
Application has a default spring data source specified in application.yml
spring:
datasource:
type: com.zaxxer.hikari.HikariDataSource
url: jdbc:oracle:thin:#localhost:1521:xe
username: system
password: oracle
hikari:
poolName: Hikari
auto-commit: false
I have added configuration options for a second data source, used for a completely difference (JDBCTemplate purpose).
faas20:
ds:
url: jdbc:oracle:thin:#tldb0147vm.group.net:1760:tdb
username: ...
password: ...
Then, I add two data sources, one named, and the other default. Without the default one, liquibase fails to start.
#Configuration
public class LegacyConfiguration {
#Bean(name = "faas20")
#ConfigurationProperties(prefix = "faas20.ds")
public DataSource legacyDataSource() {
return DataSourceBuilder
.create()
.build();
}
#Bean
public DataSource defaultDataSource() {
return DataSourceBuilder
.create()
.build();
}
}
Startup of the application fails though.
The application now cannot build the default EntityManagerFactory.
Why would that be affected?
Parameter 0 of constructor in impl.OrderServiceImpl required a bean named 'entityManagerFactory' that could not be found.
Consider defining a bean named 'entityManagerFactory' in your configuration.
Without the two data sources present the application and liquibase start up as they should.
edit
I am not clear on how to configure two separate data sources,
Default Data Source for JPA
Additional Data Source for use in JDBC (and potentially other JPA classes)
I have a requirement to send a message to a RabbitMQ instance with a JPA entity after persisting/flushing it, which lead me to configure the rabbitTemplate as channelTransacted.
The consumer is external, but, to create an integration test, I have added an embedded broker (Apache QPid) and a listener to be able to check that the messages are sent.
As indicated by the documentation, I seem to have ran into a deadlock:
If we have producers and consumers in the same application, we may end up with a deadlock when producers are blocking the connection (because there are no resources on the Broker any more) and consumers cannot free them (because the connection is blocked). [...]
A separate CachingConnectionFactory is not possible for transactional producers that execute on a consumer thread, since they should reuse the Channel associated with the consumer transactions.
If I set rabbitTemplate.channelTransacted = false, the listener gets invoked fine, otherwise harness.getNextInvocationDataFor just waits until it times out.
What I'm hoping is that there's still a way to do this kind of integration test and that perhaps I configured something wrong.
I've tried with both the simple and the direct listener types, didn't make any difference:
queues:
transactions: 'transactions'
spring:
rabbitmq:
host: rabbitmq
username: guest
password: guest
dynamic: true # automatically create queues and exchanges on the RabbitMQ server
template:
routing-key: ${queues.transactions}
retry.enabled: true
# mandatory: true # interesting only for cases where a reply is expected
# publisher-confirms: true # does not work in transacted mode
publisher-returns: true # required to get notifications in case of send problems
# used for integration tests
listener:
type: direct
direct:
retry:
enabled: true
stateless: false # needed when transacted mode is enabled
max-attempts: 1 # since this is used just for integration tests, we don't want more
I'm using Spring Boot 2.1.3 with the spring-boot-starter-amqp, which pulls in spring-rabbit-2.1.4 and Apache Qpid 7.1.1 as the embedded broker for the test:
#RunWith(SpringRunner.class)
#SpringBootTest(properties = "spring.main.allow-bean-definition-overriding=true")
#AutoConfigureTestDatabase
#Transactional
#ActiveProfiles("test")
public class SalesTransactionGatewayTest {
private static final String TRANSACTIONS_LISTENER = "transactions";
#TestConfiguration
#RabbitListenerTest(spy = false, capture = true)
public static class Config {
#Bean
public SystemLauncher broker() throws Exception {
SystemLauncher broker = new SystemLauncher();
Map<String, Object> attributes = new HashMap<>();
attributes.put(SystemConfig.TYPE, "Memory");
attributes.put(SystemConfig.INITIAL_CONFIGURATION_LOCATION, "classpath:qpid-config.json");
attributes.put(SystemConfig.STARTUP_LOGGED_TO_SYSTEM_OUT, false);
broker.startup(attributes);
return broker;
}
#Bean
public Listener listener() {
return new Listener();
}
}
public static class Listener {
#RabbitListener(id = TRANSACTIONS_LISTENER, queues = "${queues.transactions}")
public void receive(SalesTransaction transaction) {
Logger.getLogger(Listener.class.getName()).log(Level.INFO, "Received tx: {0}", transaction);
}
}
#Before
public void setUp() {
// this makes the test work, setting it to `true` makes it deadlock
rabbitTemplate.setChannelTransacted(false);
}
#Test
public void shouldBeSentToGateway() throws Exception {
SalesTransaction savedTransaction = service.saveTransaction(salesTransaction);
InvocationData invocationData = this.harness.getNextInvocationDataFor(TRANSACTIONS_LISTENER, 10, TimeUnit.SECONDS);
assertNotNull(invocationData);
assertEquals(salesTransaction, invocationData.getArguments()[0]);
}
}
11:02:56.844 [SimpleAsyncTaskExecutor-1] INFO org.springframework.amqp.rabbit.listener.DirectMessageListenerContainer - SimpleConsumer [queue=transactions, consumerTag=sgen_1 identity=16ef3497] started
Mar 25, 2019 11:02:57 AM AmqpSalesTransactionGateway send
INFO: Sending transaction: 01a92a56-c93b-4d02-af66-66ef007c2817 w/ status COMPLETED
11:02:57.961 [main] INFO org.springframework.amqp.rabbit.connection.CachingConnectionFactory - Attempting to connect to: [localhost:5672]
11:02:57.972 [main] INFO org.springframework.amqp.rabbit.connection.CachingConnectionFactory - Created new connection: rabbitConnectionFactory.publisher#6d543ec2:0/SimpleConnection#79dd79eb [delegate=amqp://guest#127.0.0.1:5672/, localPort= 56501]
java.lang.AssertionError
at org.junit.Assert.fail(Assert.java:86)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertNotNull(Assert.java:712)
at org.junit.Assert.assertNotNull(Assert.java:722)
I'm attempting to build code into our base pom which autoconfigures Spring Cloud Config server lookup through Eureka. We're doing this to avoid templating .yml properties for developers building microservices. For example, we want to java config all behavior triggered from these properties:
spring:
application:
name: MyMicroservice
cloud:
config:
enabled: true
server:
prefix: /diagnostics/admin/config
failFast: true
discovery:
enabled: true
serviceId: echo
management:
context-path: /diagnostics/admin
eureka:
password: password
client:
serviceUrl:
defaultZone: http://user:${eureka.password}#localhost:8761/eureka/
instance:
leaseRenewalIntervalInSeconds: 10
statusPageUrlPath: /diagnostics/admin/info
healthCheckUrlPath: /diagnostics/admin/health
After much experimenting, the following approach mostly works except for the Eureka-discovered config server (resulting in no overridden config properties):
#Order(-1)
public class AdditionalBootstrapPropertySourceLocator implements PropertySourceLocator {
#Override
public PropertySource<?> locate(Environment environment) {
Map<String, Object> theBootstrapYmlConfig = new HashMap<>();
theBootstrapYmlConfig.put("spring.cloud.config.enabled", new Boolean(true));
theBootstrapYmlConfig.put("spring.cloud.config.server.prefix", "/diagnostics/admin/config");
theBootstrapYmlConfig.put("spring.cloud.config.failFast", new Boolean(true));
theBootstrapYmlConfig.put("spring.cloud.config.discovery.enabled", new Boolean(true));
theBootstrapYmlConfig.put("spring.cloud.config.discovery.serviceId", "echo");
theBootstrapYmlConfig.put("management.context-path", "/diagnostics/admin");
theBootstrapYmlConfig.put("eureka.client.serviceUrl.defaultZone", "http://user:password#localhost:8761/eureka/");
theBootstrapYmlConfig.put("eureka.instance.leaseRenewalIntervalInSeconds", new Integer(10));
theBootstrapYmlConfig.put("eureka.instance.statusPageUrlPath", "/diagnostics/admin/info");
theBootstrapYmlConfig.put("eureka.instance.healthCheckUrlPath", "/diagnostics/admin/health");
return new MapPropertySource("myExtraBootstrap", theBootstrapYmlConfig);
}
}
And I seem to need this Bean as well:
#ConditionalOnWebApplication
#Configuration
#Import(EurekaClientAutoConfiguration.class)
public class WorkfrontDiscoveryClientConfigServiceBootstrapConfiguration {
#Bean
#ConditionalOnClass({ DiscoveryClient.class, ConfigServicePropertySourceLocator.class })
#ConditionalOnMissingBean
DiscoveryClientConfigServiceBootstrapConfiguration discoveryClientConfigServiceBootstrapConfiguration() {
DiscoveryClientConfigServiceBootstrapConfiguration discoveryClientConfigServiceBootstrapConfiguration =
new DiscoveryClientConfigServiceBootstrapConfiguration();
return discoveryClientConfigServiceBootstrapConfiguration;
}
}
Finally, I put both into spring.factories to ensure they are constructed. The problem is that the PropertySourceLocator is never used to construct the call within ConfigServicePropertySourceLocator to retrieve the properties. No matter what I do, I cant seem to match the behaviors that specifying the properties within bootstrap.yml would produce.
Edit 4 days later
The key factor (and restriction) here is the ability to look up the config server through Eureka. In the current spring cloud release (1.0.2), the property source is retrieved and constructed too early in the spring initialization cycle for the config-lookup-through-eureka java property source config I have above. Plus if the Eureka server is slow or not available at bootstrap startup time, the Config server property source is never reconstructed when Eureka finally comes up. This in my mind is a bug.
I solved this all by eliminating the concept of looking up the config server through Eureka, and requiring this minimum config in bootstrap.yml:
spring:
application:
name: MyMicroservice
cloud:
config:
uri: http://localhost:8888/diagnostics/admin/config
eureka:
client:
serviceUrl:
defaultZone: http://user:password#localhost:8761/eureka/
and then setting the rest in the java AdditionalBootstrapPropertySourceLocator
Edit 30 days later
Java-configing bootstrap properties continues to be a challenge. I'm doing this because I'm developing a framework without templating or code generation (the premise of spring boot). I've added spring-retry to the mix and client-to-config gets retried but re-registration to Eureka does not. This is why Eureka-first had to be abandoned for me. I'd put my vote in for integrating spring-retry into the Eureka registration process so I can go back to Eureka-first for my framework. Still on Spring Cloud 1.0.2.
Edit 100 days later
Update for where we ended up. Continuing along our mantra of avoiding property templating, enforcing policies and practices within code .. and continuing without a Eureka-first concept, we abandoned PropertySourceLocator and simply used a SpringApplicationRunListener as follows:
public class OurFrameworkProperties implements SpringApplicationRunListener {
:
public void started() {
if (TestCaseUtils.isRunningFromTestCase()) {
System.setProperty("spring.cloud.config.failFast", "false");
System.setProperty("spring.cloud.config.enabled", "false");
System.setProperty("eureka.client.enabled", "false");
} else {
// set production values same way
}
}
}
A caution that this started() actually gets called twice inside the spring code (once not passing any program arguments btw) everytime your Spring application runs or gets an Actuator refresh().
If your PropertySourceLocator is listed inspring.factories (I assume as a BootstrapConfiguration) then it needs to be a #Component (or maybe even a #Configuration).
you have to set this properties in the boostrap.properties
eureka.instance.metadataMap.configPath: /your-app-name
and comment this one
#spring.cloud.config.uri=http://localhost:8888/
and obviously it must be also this
eureka.client.serviceUrl.defaultZone: ${EUREKA_URI:http://localhost:8761/eureka}
eureka.client.instance.preferIpAddress: true
according with the documentation
https://cloud.spring.io/spring-cloud-config/multi/multi__spring_cloud_config_client.html#discovery-first-bootstrap