I have a springboot app that uses these versions:
SpringBoot: 2.3.1
SpringCloud: Hoxton.SR5
SpringData: Neumann-SR1
I've used the custom class as described here (answer from #Arun): Configuring Spring Cloud Vault Config to pull from a location other than /secret
It doesn't even seem to be picking up the vault configs.
I have a bootstrap.yml with the following:
spring:
cloud:
# Vault configurations
vault:
generic:
enabled: false
uri: https://${URI}
authentication: TOKEN
token: ${VAULT_TOKEN}
config:
discovery:
enabled: true
I'm trying to bring it up locally, so I have my application.yml as follows:
spring:
datasource:
url: jdbc:postgresql://localhost:8157/postgres
This is where I'm trying to inject the values from vault into my DataSource:
#Profile(value = {"local", "dev", "stg", "prd"})
#Configuration
public class DatabaseConfig {
#Autowired
DatabaseCredentials config;
#Bean
#Primary
public DataSource dataSource() {
return DataSourceBuilder
.create()
.username(config.getUsername())
.url(config.getUrl())
.password(config.getPassword())
.driverClassName("org.postgresql.Driver")
.build();
}
}
When the app starts up, it DatabaseCredentials is empty.
The other way I've done it is like this:
public class DatabaseConfig {
#Value("${ccm.database.username}")
String username;
#Value("${ccm.database.password}")
String password;
#Value("${spring.datasource.url}")
String url;
#Bean
#Primary
public DataSource dataSource() {
return DataSourceBuilder
.create()
.username(username)
.url(url)
.password(password)
.driverClassName("org.postgresql.Driver")
.build();
}
}
This has also come up empty saying it can't find a value for ccm.database.username.
What am I doing wrong?
You are missing the annotations on DatabaseConfig.java
Which will be something like this.
#Component
#ConfigurationProperties("ccm.database")
So it becomes
#ConfigurationProperties("ccm.database")
#Component
public class DatabaseCredentials {
#Value("${ccm.database.username}")
String username;
#Value("${ccm.database.password}")
String password;
#Value("${spring.datasource.url}")
String url;
// Getters and setters for all properties go here
}
To access the config
#Configuration
public class DatabaseConfig {
#Autowired
DatabaseCredentials config;
#Bean
#Primary
public DataSource dataSource() {
return DataSourceBuilder
.create()
.username(config.getUsername())
.url(config.getUrl())
.password(config.getPassword())
.driverClassName("org.postgresql.Driver")
.build();
}
}
Spring Cloud Vault gives you DEBUG info which helped me figure out the issue. I ended up defining the path with kv instead of a custom VaultConfigurer bean.
kv:
enabled: true
backend: tcds/kv/app-name
application-name: ccm
This creates the path as follows: tcds/kv/app-name/ccm/${PROFILE}.
I assumed that all secrets underneath this path will be picked up and I originally had multiple folders underneath my path of tcds/kv/app-name/ccm/dev.
For example: ../ccm/dev/database, ../ccm/dev/other-creds.
However, it seems that spring vault wasn't going to the nested paths for secrets and was looking for a file with all credentials there.
So, I removed the dev folder and replaced it with a dev file with entries such as:
{
"database.username": "blah",
"database.password": "moreblah",
"other-creds.user": "dummy"
}
I am able to get credentials from vault by using the DatabaseConfig.java class alone and not having to use DatabaseCredentials.java with the #ConfigurationProperties annotation.
Spring cloud supports import standart spring boot configuration from vault.
use two dependendences for automatic configuration:
org.springframework.cloud:spring-cloud-starter-vault-config
org.springframework.cloud:spring-cloud-vault-config-databases
See:
https://medium.com/digitalfrontiers/manage-your-database-accounts-with-spring-cloud-vault-config-48cecb837a36
Related
I have implemented a resource server and authorization server with spring cloud oauth2. When I implemented the OAuth application by using the default approach (without using asymmetric or symmetric approach) then the default token is generated by the authorization server and when I have to communicate with the resource server and the authorization server then the following properties can be used in the resource server config file.
resource-server.yml with default
security:
oauth2:
client:
client-id: web
client-secret: *****
resource:
token-info-uri: "http://localhost:9092/oauth/check_token"
When I use an asymmetric approach to sign the token with the private key in the authorization server and use the public key to validate the token in the resource server with the following property security:jwt:public-key: classpath:public.txt then the following unauthorized response has occurred when I call the resource from the resource server.
resource-server.yml with asymmetric
security:
jwt:
public-key: classpath:public.txt
API Endpoint
http://localhost:9090/user?Authorization=bearer **********
Response
{
"error": "invalid_token",
"error_description": "Cannot convert access token to JSON"
}
Note - when I use client and token-info-uri properties with an asymmetric approach in the resource-server.yml then the following error has occurred.
Method springSecurityFilterChain in org.springframework.security.config.annotation.web.configuration.WebSecurityConfiguration required a single bean, but 2 were found:
- tokenServices: defined by method 'tokenServices' in class path resource [com/benz/resource/api/config/ResourceServerConfiguration.class]
- remoteTokenServices: defined by method 'remoteTokenServices' in class path resource [org/springframework/boot/autoconfigure/security/oauth2/resource/ResourceServerTokenServicesConfiguration$RemoteTokenServicesConfiguration$TokenInfoServicesConfiguration.
I have to clarify the following question
Without using token-info-uri and client properties how does the resource server identify the particular authorization server?.
why does the unauthorized error has occurred (which is mentioned in the above)?
ResourceServerConfig class
#Configuration
public class ResourceServerConfiguration extends ResourceServerConfigurerAdapter {
#Value("${security.jwt.public-key}")
private Resource publicKey;
private TokenStore tokenStore;
#Override
public void configure(ResourceServerSecurityConfigurer resources) throws Exception {
resources.tokenStore(tokenStore());
}
#Bean
public DefaultTokenServices tokenServices(TokenStore tokenStore)
{
DefaultTokenServices tokenServices=new DefaultTokenServices();
tokenServices.setTokenStore(tokenStore);
return tokenServices;
}
#Bean
public TokenStore tokenStore() throws Exception
{
if(tokenStore==null)
tokenStore=new JwtTokenStore(tokenConverter());
return tokenStore;
}
private JwtAccessTokenConverter tokenConverter() throws Exception
{
JwtAccessTokenConverter converter=new JwtAccessTokenConverter();
converter.setVerifierKey(getPublicKeyAsString());
return converter;
}
private String getPublicKeyAsString() throws Exception
{
return IOUtils.toString(publicKey.getInputStream(),UTF_8);
}
}
Github link for
resource-server-implementation
authorization-server-implemetation
If you need more details please comment it then I can update
I have solved the issue. In my case, tokenConverter method is called at the time of JwtTokenStore bean instance created. I have changed the approach to create the bean instance for JwtAccessTokenConverter separately by adding #Bean annotation. In that case, it is worked without any interruption.
#Bean
public JwtAccessTokenConverter tokenConverter() throws Exception
{
JwtAccessTokenConverter converter=new JwtAccessTokenConverter();
converter.setVerifierKey(getPublicKeyAsString());
return converter;
}
I'm using ReactiveRedisConnection to configure a connection to a local redis container.
But in the future the application will be hosted on a webserver and the redis on a different server.
Is there any option to set a timeout for a request?
After some research and tests, I found that the timeout must be set on the request query instead.
So on the config Class:
#Bean
public ReactiveRedisTemplate<String, String> reactiveRedisTemplateString
(ReactiveRedisConnectionFactory connectionFactory) {
return new ReactiveRedisTemplate<>
(connectionFactory, RedisSerializationContext.string());
}
and in the service:
#Autowired
private ReactiveRedisTemplate<String, Response> repository;
public Mono<String> execute(String value){
return repository.opsForHash().entries("KEY_TO_SEARCH")
.timeout(Duration.ofMillis(TIMEOUT))
.collect(Collectors.toMap("CODE_HERE");
Edit: Thank for everyone who helped here.
Timeout can be configured on your Reactive Connection Implementation. If you are using Lettuce for Redis Connection, you can do the following.
#Bean
public ReactiveRedisConnectionFactory reactiveRedisConnectionFactory() {
return new LettuceConnectionFactory(new RedisStandaloneConfiguration(), LettuceClientConfiguration.builder().commandTimeout(Duration.ofSeconds(2)).build());
}
And then use the connectionFactory to create ReactiveRedisTemplate.
#Bean
public ReactiveRedisTemplate<String, String> reactiveRedisTemplateString
(ReactiveRedisConnectionFactory connectionFactory) {
return new ReactiveRedisTemplate<>(connectionFactory, RedisSerializationContext.string());
}
I am trying to use datasource with spring boot.
In dev the application will run as spring bootrun. But in Test / Prod it will be external tomcat.
So in dev property file I am using Datasource details:
spring.community_ds.url=xxx
spring.community_ds.username=xxx
spring.community_ds.password=xxx
spring.community_ds.driver-class-name=oracle.jdbc.OracleDriver
spring.community_ds.driverClassName=oracle.jdbc.OracleDriver
And in TEST/PROD want to use JNDI
spring.community_ds.jndi-name=xxxx
Now in code I am trying to get the DataSource to create JdbcTemplate
#ConfigurationProperties(prefix = "spring.community_ds")
#Bean(name = "communityDb")
public DataSource communityDbDataSource() {
DataSource ds = DataSourceBuilder.create().build();
//Getting error db url is Null here
return ds;
}
#Bean(name = "communityDbTemplate")
public JdbcTemplate communityDbTemplate(#Qualifier("communityDb") DataSource communityDb) {
return new JdbcTemplate(communityDb);
}
I also tried :
#Configuration
public class DatabaseConfig {
#Autowired
Environment env;
#ConfigurationProperties(prefix = "spring.datasource")
#Bean
#Primary
public DataSource getDataSource() {
DataSource dsp = DataSourceBuilder
.create().build();
System.out.println(env.getProperty("spring.community_ds.url"));
System.out.println(env.getProperty("spring.datasource.url"));
return dsp;
}
}
Here also I saw env.getProperty is printing the data properly, but when I debugged inside DataSourceBuilder the url/username etc are null.
In gradle first I used:
compile("org.springframework:spring-jdbc")
Which was giving some error hence used:
compile("org.springframework.boot:spring-boot-starter-jdbc")
With Spring Boot I can instantiate a JdbcTemplate with the following:
Code:
#Autowired
private JdbcTemplate jdbcTemplate;
Properties:
spring.datasource.url=jdbc:postgresql://my_url:my_port/my_other_stuff
spring.datasource.username=my_user_name
spring.datasource.password=my_password
spring.datasource.driver-class-name=org.postgresql.Driver
This create a DataSource of class: org.apache.tomcat.jdbc.pool.DataSource
How do I set the DataSource username/password programmatically?
We have a policy not to store credentials in plain text and I have to use a specific credential provider where I work.
You can use DataSourceBuilder if you are using jdbc starter. Also, in order to override the default autoconfiguration bean you need to mark your bean as a #Primary
In my case I have properties starting with datasource.postgres prefix.
E.g
#ConfigurationProperties(prefix = "datasource.postgres")
#Bean
#Primary
public DataSource dataSource() {
return DataSourceBuilder
.create()
.build();
}
If it is not feasible for you, then you can use
#Bean
#Primary
public DataSource dataSource() {
return DataSourceBuilder
.create()
.username("")
.password("")
.url("")
.driverClassName("")
.build();
}
My project of spring-boot has run normally according to your assistance. The yaml datasource configuration is:
spring:
# (DataSourceAutoConfiguration & DataSourceProperties)
datasource:
name: ds-h2
url: jdbc:h2:D:/work/workspace/fdata;DATABASE_TO_UPPER=false
username: h2
password: h2
driver-class: org.h2.Driver
Custom DataSource
#Configuration
#Component
public class DataSourceBean {
#ConfigurationProperties(prefix = "spring.datasource")
#Bean
#Primary
public DataSource getDataSource() {
return DataSourceBuilder
.create()
// .url("jdbc:h2:D:/work/workspace/fork/gs-serving-web-content/initial/data/fdata;DATABASE_TO_UPPER=false")
// .username("h2")
// .password("h2")
// .driverClassName("org.h2.Driver")
.build();
}
}
All you need to do is annotate a method that returns a DataSource with #Bean.
A complete working example follows.
#Bean
public DataSource dataSource() {
DataSourceBuilder dataSourceBuilder = DataSourceBuilder.create();
dataSourceBuilder.url(dbUrl);
dataSourceBuilder.username(username);
dataSourceBuilder.password(password);
return dataSourceBuilder.build();
}
If you're using latest spring boot (with jdbc starter and Hikari) you'll run into:
java.lang.IllegalArgumentException: jdbcUrl is required with driverClassName.
To solve this:
In your application.properties:
datasource.oracle.url=youroracleurl
In your application define as bean (#Primary is mandatory!):
#Bean
#Primary
#ConfigurationProperties("datasource.oracle")
public DataSourceProperties getDatasourceProperties() {
return new DataSourceProperties();
}
#Bean
#ConfigurationProperties("datasource.oracle")
public DataSource getDatasource() {
return getDatasourceProperties().initializeDataSourceBuilder()
.username("username")
.password("password")
.build();
}
If you want more datesource configs e.g.
spring.datasource.test-while-idle=true
spring.datasource.time-between-eviction-runs-millis=30000
spring.datasource.validation-query=select 1
you could use below code
#Bean
public DataSource dataSource() {
DataSource dataSource = new DataSource(); // org.apache.tomcat.jdbc.pool.DataSource;
dataSource.setDriverClassName(driverClassName);
dataSource.setUrl(url);
dataSource.setUsername(username);
dataSource.setPassword(password);
dataSource.setTestWhileIdle(testWhileIdle);
dataSource.setTimeBetweenEvictionRunsMillis(timeBetweenEvictionRunsMills);
dataSource.setValidationQuery(validationQuery);
return dataSource;
}
refer: Spring boot jdbc Connection
As an alternative way you can use DriverManagerDataSource such as:
public DataSource getDataSource(DBInfo db) {
DriverManagerDataSource dataSource = new DriverManagerDataSource();
dataSource.setUsername(db.getUsername());
dataSource.setPassword(db.getPassword());
dataSource.setUrl(db.getUrl());
dataSource.setDriverClassName(db.getDriverClassName());
return dataSource;
}
However be careful about using it, because:
NOTE: This class is not an actual connection pool; it does not
actually pool Connections. It just serves as simple replacement for a
full-blown connection pool, implementing the same standard interface,
but creating new Connections on every call. reference
for springboot 2.1.7 working with url seems not to work. change with jdbcUrl instead.
In properties:
security:
datasource:
jdbcUrl: jdbc:mysql://ip:3306/security
username: user
password: pass
In java:
#ConfigurationProperties(prefix = "security.datasource")
#Bean("dataSource")
#Primary
public DataSource dataSource(){
return DataSourceBuilder
.create()
.build();
}
I customized Tomcat DataSource in Spring-Boot 2.
Dependency versions:
spring-boot: 2.1.9.RELEASE
tomcat-jdbc: 9.0.20
May be it will be useful for somebody.
application.yml
spring:
datasource:
driver-class-name: org.postgresql.Driver
type: org.apache.tomcat.jdbc.pool.DataSource
url: jdbc:postgresql://${spring.datasource.database.host}:${spring.datasource.database.port}/${spring.datasource.database.name}
database:
host: localhost
port: 5432
name: rostelecom
username: postgres
password: postgres
tomcat:
validation-query: SELECT 1
validation-interval: 30000
test-on-borrow: true
remove-abandoned: true
remove-abandoned-timeout: 480
test-while-idle: true
time-between-eviction-runs-millis: 60000
log-validation-errors: true
log-abandoned: true
Java
#Bean
#Primary
#ConfigurationProperties("spring.datasource.tomcat")
public PoolConfiguration postgresDataSourceProperties() {
return new PoolProperties();
}
#Bean(name = "primaryDataSource")
#Primary
#Qualifier("primaryDataSource")
#ConfigurationProperties(prefix = "spring.datasource")
public DataSource primaryDataSource() {
PoolConfiguration properties = postgresDataSourceProperties();
return new DataSource(properties);
}
The main reason why it had been done is several DataSources in application and one of them it is necessary to mark as a #Primary.
I am currently using Spring MVC 4.0.5 and would like to use Spring Web Flow for some process oriented pages. However, I think there is still some problem with my configuration.
In the server logs:
2014-09-15 20:54:49,280 [localhost-startStop-1] DEBUG org.springframework.webflow.definition.registry.FlowDefinitionRegistryImpl - Registering flow definition 'ServletContext resource [/WEB-INF/flows/registration/registration-flow.xml]' under id 'registration'
However, when accessing it, the log says
2014-09-15 20:54:49,820 [http-bio-8080-exec-2] DEBUG org.springframework.webflow.mvc.servlet.FlowHandlerMapping - No flow mapping found for request with URI '/appContext/registration/'
Here is my configuration for the web flow
#Configuration
public class WebFlowConfig extends AbstractFlowConfiguration {
private Logger logger = Logger.getLogger(WebFlowConfig.class);
#Bean
#Autowired
public FlowExecutor flowExecutor(FlowDefinitionRegistry flowRegistry,
PlatformTransactionManager txManager, SessionFactory sessionFactory) {
return getFlowExecutorBuilder(flowRegistry)
.addFlowExecutionListener(new SecurityFlowExecutionListener(),
"*")
.addFlowExecutionListener(
new HibernateFlowExecutionListener(sessionFactory,
txManager), "*").build();
}
#Bean
#Autowired
public FlowDefinitionRegistry flowRegistry(
FlowBuilderServices flowBuilderServices) {
return getFlowDefinitionRegistryBuilder(flowBuilderServices)
.setBasePath("/WEB-INF/flows")
.addFlowLocationPattern("/**/*-flow.xml").build();
}
#Bean
#Autowired
public FlowBuilderServices flowBuilderServices(
MvcViewFactoryCreator mvcViewFactoryCreator, Validator validator) {
return getFlowBuilderServicesBuilder()
.setViewFactoryCreator(mvcViewFactoryCreator)
.setValidator(validator).setDevelopmentMode(true).build();
}
#Bean
#Autowired
public MvcViewFactoryCreator mvcViewFactoryCreator(
InternalResourceViewResolver viewResolver) {
MvcViewFactoryCreator factoryCreator = new MvcViewFactoryCreator();
factoryCreator.setViewResolvers(Arrays.asList(viewResolver));
return factoryCreator;
}
#Bean
#Autowired
public FlowHandlerMapping flowHandlerMapping(FlowDefinitionRegistry registry) {
FlowHandlerMapping handlerMapping = new FlowHandlerMapping();
handlerMapping.setOrder(-1);
handlerMapping.setFlowRegistry(registry);
return handlerMapping;
}
#Bean
#Autowired
public FlowHandlerAdapter flowHandlerAdapter(FlowExecutor executor) {
FlowHandlerAdapter handlerAdapter = new FlowHandlerAdapter();
handlerAdapter.setFlowExecutor(executor);
handlerAdapter.setSaveOutputToFlashScopeOnRedirect(true);
return handlerAdapter;
}
}
Hope that someone can help. Thanks.
You are specifying FlowHandlerMapping in webflow config:
Implementation of HandlerMapping that follows a simple convention
for creating URL path mappings from the ids of registered flow definitions. This
implementation returns a FlowHandler that invokes a flow if the current request
path matches the id of a flow in the configured FlowDefinitionRegistry.
The default FlowUrlHandler implementation for Spring Web Flow is DefaultFlowUrlHandler.
Expects URLs to launch flow to be of this pattern:
http://<host>/[app context path]/[app servlet path]/<flow path>
The flow id is treated as the path info component of the request URL string.
If the path info is null, the servletPath will be used as the flow id. Also, if
the servlet path ends in an extension it will be stripped when calculating the flow id.
The flow definition URL for the given flow id will be built by appending the
flow id to the base app context and servlet paths.
No flow mapping found for request with URI '/appContext/registration/'
Your path info must have been null and servlet mapping is something like: /appContext/registration/* resulting in flow id as /appContext/registration/ which is not registered.
So check your servlet mapping.
Remove MvcViewFactoryCreator Definition And .setViewFactoryCreator(mvcViewFactoryCreator)