Currently I have consul configured to read a YML and load different properties as follows in bootstrap.yml:
spring:
cloud:
consul:
host: ${CONSUL_HOST}
port: ${CONSUL_PORT}
config:
format: YAML
acl-token: ${CONSUL_TOKEN}
default-context: application
prefix: config/
name: application
data-key: data/properties
This works correctly.
Now I have included another Key/Value in consul at the same level as "properties", which is as follows:
config/application/data/properties
config/application/data/newproperties
I am trying to be able to load both data keys at the same time so that I can keep the different settings separate, but I am not managing to properly configure bootstrap.yml for this to happen.
I've tried setting the type as FILES, but haven't gotten it to work.
Have any of you faced this problem before?
Thanks!
After investigating and debugging the consul key/value behavior, it seems that it is not possible to read two different keys simultaneously.
Related
I am looking for instructions on the configuration of sasl.jaas.config when you have 2 separate topics, each having separate connection key? I am using spring-cloud-starter-stream-kafka version 3.1. I am not using spring-cloud-stream-binder-kafka and I don't know if that would solve my problem.
I can find lots of examples when there is only 1 topic involved, using a "general configuration in spring.cloud.stream.kafka.binder.configuration that is inherited by the consumer to connect.
I have a scenario where I have a input and output which are each on their own separate topic and I want to configure it in the .yml. Is this possible via .yml configuration?
My best guess was to try to configure the key spring.cloud.stream.bindings.input.configuration.sasl.jaas.config but that config doesn't seem to exist, does it?
Here is my best guess but it is not correct, since it doesn't work:
spring:
...
cloud:
stream:
kafka:
binder:
brokers: ...
defaultBrokerPort: 9093
auto-create-topics: true
configuration:
security.protocol: SASL_SSL
sasl:
mechanism: PLAIN
bindings:
eeoi-sink:
consumer:
enableDlq: false
dlqName: input_dlq
ackEachRecord: true
autoCommitOffset: false
bindings:
eeoi-sink:
destination: input
contentType: application/json
group: $Default
consumer:
max-attempts: 1
configuration:
sasl:
jaas.config: org.apache.kafka.common.security.plain.PlainLoginModule required username="$ConnectionString" password="...";
acknowledgement-source:
destination: output
contentType: application/json
group: $Default
configuration:
sasl:
jaas.config: org.apache.kafka.common.security.plain.PlainLoginModule required username="$ConnectionString" password="...";
spring.cloud.stream.bindings.input.configuration.sasl.jaas.config will not work. The correct property key is spring.cloud.stream.kafka.bindings.input.consumer.configuration.sasl.jaas.config.
If you want to use separate jaas config, you need to use a multi-binder setup. See this application for some ideas: https://github.com/spring-cloud/spring-cloud-stream-samples/blob/master/multi-binder-samples/kafka-multi-binder-jaas/src/main/resources/application.yml
In this sample, where it provides jaas config for the binder, you can simply replace it with binder.configuration.sasl.jaas.config.
Have the same Kafka cluster used as the broker for both binder setups, but different jaas configs.
However, I think the problem that you are running into is different. Your jaas config is ultimately delegated to java security Configuration which keeps a static copy of the jaas configuration per JVM. Thus, any jaas configs provided after it is set statically in the JVM are ignored.
See this for more details on a related issue: https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/874
This issue provides some workarounds for this problem. One is to make the connection string, password, etc. the same for both the topics, that way, the same jaas config values are used. This may not be an ideal solution and defeats the purpose of different credentials in the first place. The reason why the aforementioned sample works is because we use the same username/password combination in both binders.
Another option is to split the application into two. In this way, the jaas configuration for each topic resides in separate applications.
I am used to feature that I can refer to another property from within .yml file using ${...}.
So I have Spring Boot v2.3.0.RELEASE and following .yml:
env: dev
spring:
profiles:
active: ${env}
But unfortunately, when running my application i see
The following profiles are active: ${env}
Despite the fact that this placeholder works perfectly in other properties, for example kafka topics i define as
topic: ${env}.topic_name
and i can see in logs that it's resolved properly
Kafka producer topic=dev.topi_name
Seems like spring.profiles.active is some sort of an exception from this rule, but i can't find why. And how can i get the same result (with profile depending on other property variable) as is, without env variables, external keys, setting in code, etc. ?
This problem has been fixed in Spring Boot 2.4.0 as a part of the updated config file processing (more about it in 2.4.0 release notes).
If interested, you can play with the new property spring.config.use-legacy-processing to see the difference in how spring.profiles.active is processed.
In spring-cloud projects, for some reason, need to put some properties into bootstrap.yml, so that to make the specific properties available earlier.
Then there are 2 config files: bootstrap.yml and application.yml.
Wondering is it ok to put everying into bootstrap.yml, and remove application.yml totally, to make it cleaner.
Since bootstramp.yml is just loaded earlier, it seems to be ok, right? Or, will this cause some issue in some case?
Basically idea is to get configurations from ConfigServer using Spring Cloud Config. But sometimes we need some configurations e.g. spring.application.name upfront so, I'm using those configurations in bootstrap.yml. Which generally overrides what's in an application.yml [if present]). Reference
Plus, yes you can put all your configurations in bootstrap.yml and it works simply fine. I've tested with more than four microservices.
My springboot application has a few yml files(each for various profiles - dev, prod) to load configurations from. I am moving the configurations to the DB.
Sample configurations are like,
admin:
id: user05
firstname: Brian
lastname: Leavy
purl: http://plixes.com/seai/ji
I have the values read from the DB and have it locally. I am not sure how to inject these values in my program onto these values, like key-value, as,
admin.id:user05
admin.firstname: Brian
admin.lastname: Leavy
admin.purl: http://plixes.com/seai/ji
so that they are available to the application as it would normally be.
I would need them to be initialized very early since some of the values are springboot configurations, like say,
server:
port: 5007
Any pointers would be really helpful.
EDIT1:
I just found out after hours of searching, that you could do something like this,
SpringApplication app = new SpringApplication(Lexon.class);
app.setDefaultProperties(Collections
.singletonMap("server.port", "5007"));
app.setDefaultProperties(Collections
.singletonMap("admin.id", "user05"));
This works, but does not look clean.
Is there a better way to do this?
There is Spring Boot Configuration library that can externalize the configuration to a database or a git repository.
With that, you add an data source and an SQL statement to retrieve the property values.
For example, on application.yml:
spring:
cloud:
config:
server:
jdbc:
sql: SELECT KEY, VALUE from MY_PROPERTIES where APPLICATION=? and PROFILE=? and LABEL=?
Check this site for more details: https://www.devglan.com/spring-cloud/jdbc-backend-spring-cloud-config
I have a spring client application which is using a cloud server, I have a branch for each environment, for example, client STAG points to STAG branch in http://#giturl.So, what I'm trying to do is creating another branch called common which consists of common properties across the environment. I want to point STAG branch from client application to cloud server and as well as to common branch which retrieves all the common properties used across all the environments, Tried to use the composite type in a cloud server in application.yml by defining the same URL and different labels no luck on that any thoughts?
Tried using the composite type but didn't worked
spring:
profiles:
active: composite
cloud:
config:
server:
composite:
-
type:
git:
uri: https://#gitURI
searchPaths: '{application}/,common/'
-
type:
git:
uri: https://#gitURI
searchPaths: '{application}/'
Ok, so you might to look at this part of the documentation and play with those 3 properties at the client app, where label would be the branch names in a comma separated string. The searchPaths property at the server's side would be {application} since that is the subfolder.
You might want to consider having multiple config repositories since that will be easier to manage.
Hope that helps! :)