How Do I connect to multiple bootstrap servers(DEV, STAGE and PROD) from a microservice ( Admin MS) with security in place?
I want to connect to all the kafka servers and create/manage topics, create ACLS etc.
I am using spring kafka adminclient , and configuring properties from application.yml using spring boot to connect to Dev right now. But now I want to connect to all environments.
Is there an easier and better approach other that wring a properties hashmap and putting config values in it. Does Spring cloud stream help?
Is this something similar to connecting multiple databases to a micro service ?
You can do it by creating multiple child boot applications, each with its own environment containing the properties.
But it's probably easier to bypass Boot's auto configuration and wire up your own AdminClients with their own properties.
Related
I'm new to the spring cloud config server. Consider a scenario where we have 10 spring boot microservice fetching configurations from the Spring Boot Cloud Config. I was wondering How the 10 spring boot microservices will work when the Spring Boot Cloud Config itself is down?
Can someone answer to below queries:
If the config server is down, Will there be downtime for all the microservice connected to it?
Let's say we have a config file application.properties in GitHub and Spring boot config refers to the application.properties file in GitHub What if the username and password to access the application.properties file itself will change?
In terms of Disaster recovery, Do we need any backup of the config server? If yes, How can we achieve the same?
If the config server is down, Will there be downtime for all the microservice connected to it?
In real world application, there will be multiple instances of your config server deployed across multiple availability zones, fronted by load balancer or API gateway, or even you can register your multiple instance with eureka server so that there is No single point of failure.
So how the configuration will look like is instance 1 is in us-east-1
instance 2 in us-west-2, so even if one AZ is down it will not impact your services.
As far as GitHub or external repo is concerned, you can configure config server to read properties natively but that not something I will suggest !!
Let's say we have a config file application.properties in GitHub and Spring boot config refers to the application.properties file in GitHub What if the username and password to access the application.properties file itself will change?
First of all you should not commit password in Github for public repo, secondly password should be dynamically fetched from Idvault, or AWS secret Manager or other services whichever you prefer. So that even if you change password it will not affect any services.
In terms of Disaster recovery, Do we need any backup of the config server? If yes, How can we achieve the same?
Config server is just reading properties/config from repo that you provide,so repository where your code is hosted is of importance to you. Github can take care of that for you !!
Iam using spring kafka and avro,
The schema registry is secured with plain sasl protocol.
I must develop a kafka consumer and producer, i don't find a way with spring kafka to configure the security properties for schema registry.
Does spring kafka library already integrate this configuration ?
I need to configure kafkastore.. In a schema registry properties file?
I didn't find any examples with spring kafka
there is docs like this related to your question and demo examples in github.
note that with spring you can have common properties for kafka, just make sure to use spring kafka dependency, maybe first try to establish connection between both consumer and producer and then put sasl plain security.
I am using redis for storing session in my Spring Boot app. I am also using Spring Session library to achieve this. I have two different redis instances namely US redis and EU redis. Once someone enters my app(after logging in of course), I want to store their sessions in both US redis and EU redis servers. Mind you that these both redis instances are masters and are not in any master-slave setup. This is what I have so far in my spring setup.
#EnableRedisHttpSession
#Configuration
public class RedisConfig{
}
Note: removed other security code for brevity.
According to #EnableRedisHttpSession docs, we need to add one RedisConnectionFactory instance, however Spring Boot by default does this.
So right now, once the user logs in, their session is stored in US redis(I added US redis related information in application.yml).
However whenever a session is stored in US redis, I want to replicate the same in EU redis server. How can I achieve this? Do I need to create another RedisConnectionFactory bean and manually save it? If manually save it, how to do so?
Note: This use case might not be following the best design practices(that is we are storing user session in multiple places). However I do understand this, but I do have a use case for this.
Replication is done by the underlying connection. For example, if you are using Lettuce with Spring Session Redis, then you would set it up with a Master Slave setup. In Spring you can refer to Connecting to Redis section of the documentation.
Spring Boot provides simple mechanism for setting up master/slave via properties. This article has a nice summary. In short, add the properties to application.yml
redis:
master:
host: localhost
port: 6379
slaves:
- host: localhost
port: 16379
- host: localhost
port: 26379
There is a built in solution for that, but you can always tweak the spring code or as you wrote sent an explicit set to the remote Redis.
You might want to consider using some Active-Active replication mechanism like CRDT
I'm trying to migrate from monolithic web application to microservices.
For doing this I'm using JHipster v 4.11.1 (very helpful!).
My problem is:
In JHipster registry (central-config folder) I wrote some property files and every microservice, on startup, read the owned properties.
I need to change this properties on runtime both from registry and from every microservices.
There are any ways to do this? I have to use kafka or somethings like that?
JHipster Registry is a Spring Cloud Config server and should allow this behavior but I'm new with this stuff and maybe I lost some information. For now I don't use Git or SVN repository, but only a native configuration.
Is it possible to share some properties for limited group of microservices? I mean I just want to declare common datasourse in one place for several microservices, which will use the same database
I try to implement it using gradle variables, which should consist all data related to db connection with profiles, but probably easier way to do it exists.
Thanks
Yes it is. You're looking for "Spring Cloud Config" server:
Spring Cloud’s config server acts as a single source of configuration
data for all other services in a microservice-based application. It is
itself a microservice whose job is to obtain configuration data from a
Git repository and serve it via RESTful endpoints. All other services
can either consume the config server’s API directly or, if they’re
Spring applications, treat the configuration server as another
property source in Spring’s Environment abstraction.
You can find a nice reading along with examples here.