Develop Reference

Eureka service not reading its config file in Git repository using ConfigServer

I'm trying trying to write some basic microservices examples. Firstly, I'm using a ConfigServer application using Spring, with this resources/application.yaml file:
server:
port: 8081
spring:
cloud:
config:
server:
git:
default-label: main
uri: https://github.com/mguzm4n/microservices-data
search-paths: config-data
application:
name: config-server
You can see the repo: https://github.com/mguzm4n/microservices-data
For Eureka, in resources/bootstrap.yaml:
spring:
application:
name: eureka-server
cloud:
config:
enabled: true
uri: http://localhost:8081
I build first ConfigServer App and then Eureka service, so it can load its config on Git, but I can't manage it to work. Eureka ends up running on localhost:8080 instead of port 8761 as shown in https://github.com/mguzm4n/microservices-data/blob/main/config-data/eureka-server.yaml.
Could anyone help me?
I'm new using this concepts like a config server and eureka for registering other services, so I'm sorry if it's something simple.

Looks like your eureka-server is not picking up the bootstrapping config.
First cause might be that this config file should be named bootstrap.yml, not bootstrap.yaml.
Also, since Spring Boot 2.4 bootstrapping via bootstrap.yml is disabled by default and the usual application.yml should be used instead or it can be enabled back by adding the org.springframework.cloud:spring-cloud-starter-bootstrap dependency to your project.

Configuration of 'sasl.jaas.config' when you have 2 separate topics, each having separate connection key?

I am looking for instructions on the configuration of sasl.jaas.config when you have 2 separate topics, each having separate connection key? I am using spring-cloud-starter-stream-kafka version 3.1. I am not using spring-cloud-stream-binder-kafka and I don't know if that would solve my problem.
I can find lots of examples when there is only 1 topic involved, using a "general configuration in spring.cloud.stream.kafka.binder.configuration that is inherited by the consumer to connect.
I have a scenario where I have a input and output which are each on their own separate topic and I want to configure it in the .yml. Is this possible via .yml configuration?
My best guess was to try to configure the key spring.cloud.stream.bindings.input.configuration.sasl.jaas.config but that config doesn't seem to exist, does it?
Here is my best guess but it is not correct, since it doesn't work:
spring:
...
cloud:
stream:
kafka:
binder:
brokers: ...
defaultBrokerPort: 9093
auto-create-topics: true
configuration:
security.protocol: SASL_SSL
sasl:
mechanism: PLAIN
bindings:
eeoi-sink:
consumer:
enableDlq: false
dlqName: input_dlq
ackEachRecord: true
autoCommitOffset: false
bindings:
eeoi-sink:
destination: input
contentType: application/json
group: $Default
consumer:
max-attempts: 1
configuration:
sasl:
jaas.config: org.apache.kafka.common.security.plain.PlainLoginModule required username="$ConnectionString" password="...";
acknowledgement-source:
destination: output
contentType: application/json
group: $Default
configuration:
sasl:
jaas.config: org.apache.kafka.common.security.plain.PlainLoginModule required username="$ConnectionString" password="...";

spring.cloud.stream.bindings.input.configuration.sasl.jaas.config will not work. The correct property key is spring.cloud.stream.kafka.bindings.input.consumer.configuration.sasl.jaas.config.
If you want to use separate jaas config, you need to use a multi-binder setup. See this application for some ideas: https://github.com/spring-cloud/spring-cloud-stream-samples/blob/master/multi-binder-samples/kafka-multi-binder-jaas/src/main/resources/application.yml
In this sample, where it provides jaas config for the binder, you can simply replace it with binder.configuration.sasl.jaas.config.
Have the same Kafka cluster used as the broker for both binder setups, but different jaas configs.
However, I think the problem that you are running into is different. Your jaas config is ultimately delegated to java security Configuration which keeps a static copy of the jaas configuration per JVM. Thus, any jaas configs provided after it is set statically in the JVM are ignored.
See this for more details on a related issue: https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/874
This issue provides some workarounds for this problem. One is to make the connection string, password, etc. the same for both the topics, that way, the same jaas config values are used. This may not be an ideal solution and defeats the purpose of different credentials in the first place. The reason why the aforementioned sample works is because we use the same username/password combination in both binders.
Another option is to split the application into two. In this way, the jaas configuration for each topic resides in separate applications.

How can you use TLS for Kafka in Quarkus?

The Kafka guide from Quarkus works nicely when running Kafka locally in Docker. I'm trying to change this sample by replacing the local Kafka service with a hosted Kafka service in the cloud which requires TLS.
Does anyone know how I can configure this? In the Quarkus documentation and the Smallrye documentation I don't see any properties for this.
I'd like to use the Kafka service in the IBM Cloud. Based on the documentation I've tried the following configuration in application.properties:
kafka.bootstrap.servers=broker-0-8c8cph49mx2p2wqy.kafka.svc01.us-south.eventstreams.cloud.ibm.com:9093,broker-4-8c8cph49mx2p2wqy.kafka.svc01.us-south.eventstreams.cloud.ibm.com:9093,broker-3-8c8cph49mx2p2wqy.kafka.svc01.us-south.eventstreams.cloud.ibm.com:9093,broker-5-8c8cph49mx2p2wqy.kafka.svc01.us-south.eventstreams.cloud.ibm.com:9093,broker-2-8c8cph49mx2p2wqy.kafka.svc01.us-south.eventstreams.cloud.ibm.com:9093,broker-1-8c8cph49mx2p2wqy.kafka.svc01.us-south.eventstreams.cloud.ibm.com:9093
kafka.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="token" password="...";
kafka.sasl.mechanism=PLAIN
kafka.security.protocol=SASL_SSL
kafka.ssl.protocol=TLSv1.2
Update:
I've also tried Gunnar's suggestion below, but it doesn't work. When I use the following application.properties ...
mp.messaging.outgoing.generated-price.connector=smallrye-kafka
mp.messaging.outgoing.generated-price.topic=prices
mp.messaging.outgoing.generated-price.value.serializer=org.apache.kafka.common.serialization.IntegerSerializer
mp.messaging.outgoing.generated-price.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="token" password="...";
mp.messaging.outgoing.generated-price.sasl.mechanism=PLAIN
mp.messaging.outgoing.generated-price.security.protocol=SASL_SSL
mp.messaging.outgoing.generated-price.ssl.protocol=TLSv1.2
mp.messaging.incoming.prices.connector=smallrye-kafka
mp.messaging.incoming.prices.topic=prices
mp.messaging.incoming.prices.value.deserializer=org.apache.kafka.common.serialization.IntegerDeserializer
mp.messaging.outgoing.prices.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="token" password="...";
mp.messaging.outgoing.prices.sasl.mechanism=PLAIN
mp.messaging.outgoing.prices.security.protocol=SASL_SSL
mp.messaging.outgoing.prices.ssl.protocol=TLSv1.2
kafka.bootstrap.servers=broker-0-8c8cph49mx2p2wqy.kafka.svc01.us-south.eventstreams.cloud.ibm.com:9093,broker-4-8c8cph49mx2p2wqy.kafka.svc01.us-south.eventstreams.cloud.ibm.com:9093,broker-3-8c8cph49mx2p2wqy.kafka.svc01.us-south.eventstreams.cloud.ibm.com:9093,broker-5-8c8cph49mx2p2wqy.kafka.svc01.us-south.eventstreams.cloud.ibm.com:9093,broker-2-8c8cph49mx2p2wqy.kafka.svc01.us-south.eventstreams.cloud.ibm.com:9093,broker-1-8c8cph49mx2p2wqy.kafka.svc01.us-south.eventstreams.cloud.ibm.com:9093
... I get an error:
javax.enterprise.inject.spi.DeploymentException: java.lang.IllegalArgumentException: Invalid channel configuration - the connector attribute must be set for channel prices
at io.quarkus.smallrye.reactivemessaging.runtime.SmallRyeReactiveMessagingLifecycle.onApplicationStart(SmallRyeReactiveMessagingLifecycle.java:22)
Is TLS currently possible for Kafka in Quarkus?
Thanks

Have you tried specifying the relevant properties at the channel level? E.g.
mp.messaging.outgoing.generated-price.connector=smallrye-kafka
mp.messaging.outgoing.generated-price.topic=mytopic
mp.messaging.outgoing.generated-price.ssl.protocol=...
mp.messaging.outgoing.generated-price.ssl.keystore.location=...
mp.messaging.outgoing.generated-price.ssl.keystore.password=...
You also could refer to variables when requiring the same values for multiple topics.

One property is incorrect in the accepted answer by #Gunnar. It should be "security" instead of "ssl" in the property name.
mp.messaging.outgoing.generated-price.security.protocol=SSL

Settings in application.yml for spring.cloud.config aren't used when app is executing

I have a problem with spring cloud: my settings in application.yml for spring.cloud.config aren't used when app is executing. let me put more detail here.
I'd like to my services could get settings from a remote ConfigServer. I've created the ConfigServer as a spring boot app with annotation #EnableConfigServer.
After that i've created client app with next config file:
application:
name: mw
cloud:
config:
enabled: true
uri: http://172.17.42.1:8888
fail-fast: true
main class:
#EnableEurekaClient
#SpringBootApplication
public class MwApplication
and extra configuration into app:
#Configuration
#EnableJpaRepositories(basePackages = {"com.sample.repository"})
#EnableTransactionManagement
#EnableScheduling
public class AppConfiguration
also i have next dependencies:
spring-cloud-starter-eureka
spring-cloud-config-client
spring-boot-configuration-processor
spring-boot-starter-data-jpa
When i execute my client app, i've got this message: ConfigServicePropertySourceLocator : Could not locate PropertySource: I/O error on GET request for "http://localhost:8888/mw/default"
The app try to get data from default uri(localhost) instead of to use uri from my setting. I've looked at app in debug mode and saw org.springframework.cloud.config.client.ConfigServiceBootstrapConfiguration was creating ConfigClientProperties with default property and my settings from application.yml weren't used.
What am i doing wrong?
thanks.

You need to add the following to your application.yml file:
spring:
cloud:
config:
enabled: true
Per comment chain, you also need to add the properties to bootstrap.yml instead of application.yml . The reason is that the former is loaded before the latter in the spring startup cycle. Here is another SO post answered by user Michael Isvy explaining why, and copied below for posterity: What is the diference between putting a property on application.yml or bootstrap.yml in spring boot?
I have just asked the Spring Cloud guys and thought I should share the info I have here.
bootstrap.yml is loaded before application.yml.
It is typically used for the following:
when using Spring Cloud Config Server, you should specify spring.application.name and spring.cloud.config.server.git.uri inside bootstrap.yml
some encryption/decryption information
Technically, bootstrap.yml is loaded by a parent Spring ApplicationContext. That parent ApplicationContext is loaded before the one that uses application.yml.

Spring Cloud Eureka with Config Server

What is the recommended configuration when running both Config Server with Eureka Server? Should Config Server be a client of Eureka? Or should Eureka be dependent on Config Server properties for its configuration? Or is both OK?

The default way to use Eureka and Config Server is to use Config First bootstrap. Essentially, you make eureka server a client of the config server but you don't make the config server a client of eureka.
As said by David Syer on these (and this) issues, the founder of spring cloud, you have to use the config server with a front end load balancer, so a single URL is already highly available.
I'm also a newbie in Spring Cloud but I agree with him since Eureka is a Service Discovery, IMHO it should function on it's problem domain only. It would make a complicated logic for Eureka servers who are asking the Config servers for it's configuration. I can't imagine how the Eureka Server would know which config server to get if the Config Server is also the Server of Eureka to get its list of defaultZone.
It would be much more simpler for me to separate the Config Server's HA.

Based on #Mideel's answer
Eureka and Config Client configuration (needs to be Bootstrap):
# bootstrap.yml
cloud:
config:
discovery:
enabled: true # This is required
service-id: configserver # Config Server's eureka registry name
enabled: true # This is default true already
Config Server configuration:
spring:
application:
name: configserver # Needs to match client configuration
Register the Config Server with the annotation #EnableEurekaClient (it should be Auto Configured to register with Eureka already though)

The Spring Cloud Config service provides configuration info for various other microservices, of which the Eureka service is one.
Each app/microservice is pointed to its configuration (from the Config service) via bootstrap.properties/.yml, which is loaded in the parent context for that application, before the app "recognizes" that it is a discovery/Eureka client per its annotated main class. This bit of documentation provides a bit more detail on that process.
Cheers,
Mark

EDIT1: I think this is a wrong answer, see the replies
If you use Spring Boot:
I'm using Spring Microservices in Action as my guide book and based on the source code example there, we make the configuration server as the Eureka Client with the #EnableEurekaClient annotation and in the config server application.yml, we need to add this property:
spring:
cloud:
config:
discovery:
enabled: true
And in the other Eureka client that uses this config server, you need to add this property to the application.yml :
spring:
cloud:
config:
enabled: true
That's it, just set up the config server normally, I think behind the scene the config libraries from spring cloud will take care of the rest using Eureka.