Currently migrating my application to Micronaut 3, I encountered one problem with micronaut-gcp. Especially with the google secret manager. I am using gradle with Kotlin DSL.
Actual configuration: (not working, using plugin io.micronaut.library version 2.0.4)
gradle 7.2
micronaut 3.0.1
Previous configuration: (working with no plugin, using micronaut-bom)
gradle 6.5.1
micronaut 2.4.0
micronautGcp 3.5.0
I/ The Problem
My goal is to load some secrets as key/value pairs into a property source.
I followed the documentation that says to use a bootstrap.yml as follows:
micronaut:
config-client:
enabled: true
gcp:
secret-manager:
keys:
- MY_SECRET
Then in my class I should be able to inject that secret value using the #Value annotation. Actually I am doing this in my tests. I am using the #MicronautTest annotation from jUnit5 so I have something that looks like this:
#MicronautTest
public class MyClass {
#Value("${my.secret}")
private String mySecret;
}
then comes the problem, when running any test, it fails in the initialization saying this:
Error instantiating bean of type [io.micronaut.gcp.secretmanager.client.DefaultSecretManagerClient]
Message: getTransportChannel() called when needsExecutor() is true
Path Taken: new DistributedPropertySourceLocator(ConfigurationClient configurationClient,Duration readTimeout) --> new DistributedPropertySourceLocator([ConfigurationClient configurationClient],Duration readTimeout) --> new DefaultCompositeConfigurationClient([ConfigurationClient[] configurationClients]) --> new SecretManagerConfigurationClient([SecretManagerClient secretManagerClient],SecretManagerConfigurationProperties configurationProperties) --> new DefaultSecretManagerClient([SecretManagerServiceClient client],Environment environment,GoogleCloudConfiguration googleCloudConfiguration,ExecutorService executorService)
io.micronaut.context.exceptions.BeanInstantiationException: Error instantiating bean of type [io.micronaut.gcp.secretmanager.client.DefaultSecretManagerClient]
II/ What I've tried ?
When I've seen that, I was thinking that maybe the bootstrap.yaml thing doesn't work the I tried to use the lower lever access to secret manager using injecting SecretManagerServiceClient as follows:
#MicronautTest
public class MyClass {
#Inject
private SecretManagerServiceClient client;
private void someTest() {
String mySecret = client.getSecret("MY_SECRET");
}
}
but got same error Message: getTransportChannel() called when needsExecutor() is true with same path taken.
I also tried to upgrade to micronaut 3.0.1 but didn't change anything.
III/ My solution to handle this
As my problem was to retrieve a secret during the testing stage of my ci/cd process and, as I am using Google Cloud Build. I could pass the secret using the availableSecrets feature in my cloudbuild.yaml:
steps:
...
- name: 'gradle'
entrypoint: 'bash'
args: ['./gradlew', 'test']
secretEnv: ['MY_SECRET']
...
availableSecrets:
secretManager:
- versionName: projects/PROJECT_ID/secrets/MY_SECRET/versions/latest
env: 'MY_SECRET'
then passing it in my application.yml:
my:
secret: ${MY_SECRET}
and then in my code:
#MicronautTest
public class MyClass {
#Value("${my.secret}")
private String mySecret;
}
But I dont find this solution very satisfying as I find the bootstrap.yaml one more convenient.
If someone gets a solution for me or an idea/advice, I'll be happy to take it.
Have a nice day !
I've been down that rabbit hole. Long story short I got past this by upgrading the google-cloud-secretmanager dependency from 1.6.4 to e.g. 2.0.2
Like so:
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>google-cloud-secretmanager</artifactId>
<version>2.0.2</version>
</dependency>
Related
I'm trying to use the Togglz library which allows you to wrap your application logic and be able to toggle it ON or OFF, with some advanced strategies. I am going through the Spring Boot documentation for it, and although its quite concise, I'm finding it missing pieces of information that are not allowing me to test this properly.
reference:
https://www.togglz.org/documentation/spring-boot-starter.html
I am running a Spring Boot 2.4.5 version project and this documentation says to import the dependency, which I did:
<dependency>
<groupId>org.togglz</groupId>
<artifactId>togglz-spring-boot-starter</artifactId>
<version>2.6.1.Final</version>
</dependency>
The documentation then states you can use an auto-configuration class on your #RestController, like
#Controller
public class MyClass {
private FeatureManager manager;
public MyClass(FeatureManager manager) {
this.manager = manager;
}
#RequestMapping("/")
public ResponseEntity<?> index() {
if (manager.isActive(HELLO_WORLD)) {
...
}
}
}
Here is already where I had some questions that I didn't see explained, first, that are passing a enum "HELLO_WORLD" as an argument to this isActive() function on the FeatureManager. I don't see how they are injecting this into the method/class. They do show how to declare the feature ENUM in the yaml, though, this is not referencing the "HELLO_WORLD" that was passed into the isActive() method mentioned previously, i.e. :
togglz:
features:
FOO:
enabled: true
BAR:
enabled: false
Further down the documentation, they finally do reference this HELLO_WORLD enum, but I tried adding this to my application.yaml and I can't seem to figure out how they are injecting these Feature Enums into these methods:
togglz:
enabled: true # Enable Togglz for the application.
features: # The feature states. Only needed if feature states are stored in application properties.
HELLO_WORLD:
enabled: true
The documentation does explain how to create an enum class for these features, but they explictly list it as an alternative to defining it in the yaml file
public enum MyFeatures implements Feature {
#EnabledByDefault
#Label("First Feature")
FEATURE_ONE,
#Label("Second Feature")
FEATURE_TWO;
}
#Bean
public FeatureProvider featureProvider() {
return new EnumBasedFeatureProvider(MyFeatures.class);
}
I tried this also, and I just got more Bean exception errors when I try to run the Application, i.e.
Description:
Parameter 2 of method featureManager in org.togglz.spring.boot.autoconfigure.TogglzAutoConfiguration$FeatureManagerConfiguration required a bean of type 'org.togglz.core.user.UserProvider' that could not be found.
Action:
Consider defining a bean of type 'org.togglz.core.user.UserProvider' in your configuration.
Can anyone who has successful used this library provide input how to set a simple feature toggle ? Ultimately, I want to be able turn ON/OFF this feature while the application using a RELEASE DATE activation strategy i.e. 2021-06-30 00:00:00 such that I can have the toggle activate based on a datetime.
reference: https://www.togglz.org/documentation/activation-strategies.html
Can this be done in the yaml?
If you don't want to use an Enum you will have to inject the FeatureProvider that was auto-configured and call featureProvider.getFeatures() to get hold of all available features.
You can then check their state with the FeatureManager.
I agree that this is not obvious from the documentation.
It should be possible to configure the activation strategy via your application.yml too.
See the example section "Application Properties" at the very end of https://www.togglz.org/documentation/spring-boot-starter.html.
It should look like this:
togglz.features.FOO:
enabled: true
strategy: release-date
param:
date: ..
time: ..
I have Implemented the library successfully only change required in the above mentioned code or for the exception required a bean of type org.togglz.core.user.UserProvider that could not be found is to add one more bean UserProvider.
#Bean
public UserProvider getUserProvider() {
return new ServletUserProvider("admin");
}
I've got a project (this will be used as a dependency) which uses SpringBoot and which populates a POJO from a queues.properties file in the following way:
#Component
#PropertySource({"classpath:queues.properties"})
#ConfigurationProperties("queue")
public class QueuesConfig {
private String messagingBrokerXml;
private String messagingBrokerJolokia;
private String messagingBrokerApi;
private List<QueuesConfig.QueueModel> queues = new ArrayList();
public QueuesConfig() {
}
public String getMessagingBrokerXml() {
return this.messagingBrokerXml;
}
...
By dragging this dependency in a parent SpringBoot project which has a "queues.properties" file on its classpath, QueuesConfig object gets populated with the right values.
I am currently trying to achieve the same behaviour by using this dependency in a Plain Spring project. I can confirm that the PropertySource annotation gets "executed" and that the queues.properties file is part of the StandardServletEnvironment (being an entry in the propertySourceList).
The thing is that the "ConfigurationPropertiesBindingPostProcessor" bean is not being registered (not part of the singletonObjects) and therefore the code which is supposed to populate the POJO is not being executed.
Is there any workaround for this?
Many thanks !
Try to add this in your parent Spring #Configuration class:
#EnableConfigurationProperties(QueuesConfig.class)
For more details read this section of Spring Boot documentation. It is few screens of text, but it worth it ;-)
https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-external-config.html#boot-features-external-config-typesafe-configuration-properties
Probably it worked well in your test app because you used some hi-level magic with #SpringBootApplication :)
I use spring-boot-starter-data-solr and would like to make use of the schmea cration support of Spring Data Solr, as stated in the documentation:
Automatic schema population will inspect your domain types whenever the applications context is refreshed and populate new fields to your index based on the properties configuration. This requires solr to run in Schemaless Mode.
However, I am not able to achieve this. As far as I can see, the Spring Boot starter does not enable the schemaCreationSupport flag on the #EnableSolrRepositories annotation. So what I tried is the following:
#SpringBootApplication
#EnableSolrRepositories(schemaCreationSupport = true)
public class MyApplication {
#Bean
public SolrOperations solrTemplate(SolrClient solr) {
return new SolrTemplate(solr);
}
}
But looking in Wireshark I cannot see any calls to the Solr Schema API when saving new entities through the repository.
Is this intended to work, or what am I missing? I am using Solr 6.2.0 with Spring Boot 1.4.1.
I've run into the same problem. After some debugging, I've found the root cause why the schema creation (or update) is not happening at all:
By using the #EnableSolrRepositories annotation, an Spring extension will add a factory-bean to the context that creates the SolrTemplate that is used in the repositories. This template initialises a SolrPersistentEntitySchemaCreator, which should do the creation/update.
public void afterPropertiesSet() {
if (this.mappingContext == null) {
this.mappingContext = new SimpleSolrMappingContext(
new SolrPersistentEntitySchemaCreator(this.solrClientFactory)
.enable(this.schemaCreationFeatures));
}
// ...
}
Problem is that the flag schemaCreationFeatures (which enables the creator) is set after the factory calls the afterPropertiesSet(), so it's impossible for the creator to do it's work.
I'll create an issue in the spring-data-solr issue tracker. Don't see any workaround right now, other either having a custom fork/build of spring-data or extend a bunch of spring-classes and trying to get the flag set before by using (but doubt of this can be done).
I am getting this error when trying to use #autowire, #configuration, #bean, #Repository in my Spring MVC project
Could not autowire field: private com.sachin.dao.StockDaoImpl com.sachin.myapp.HomeController.stockDao;
nested exception is org.springframework.beans.factory.NoSuchBeanDefinitionException: No matching bean of type [com.sachin.dao.StockDaoImpl] found for dependency:
Please let me know what mistake I am making. I am new to Spring MVC and dependency injection.
Here is my controller code. I am trying to inject StockDaoImpl in the controller.
#Controller
public class HomeController {
#Autowired
private StockDaoImpl stockDao;
#RequestMapping(value = "/stockgoogle/", method = RequestMethod.GET)
public #ResponseBody Stock stockGoogle(Locale locale, Model model) {
//StockDaoImpl stockDao = new StockDaoImpl();
Stock s=stockDao.listGoogle();
model.addAttribute("s", s );
return s;
}
}
My Service Implementation is below. I have used #Repository annotation here with "stockDao" which is my variable name in controller that I want to inject
#Repository("stockDao")
public class StockDaoImpl implements StockDao {
#Override
public Stock listGoogle() {
Stock s = null;
try {
... //some code
String name = rs.getString("Name");
s = new Stock(name);
...
} catch (Exception e) {
}
return s;
}
}
Also I have created a configuration class separately. I am using this to define my bean. I am only using this to specify bean and have not imported it anywhere in the code.
#Configuration
public class BeanConfiguration {
#Bean
public StockDaoImpl stockDao(){
return new StockDaoImpl();
}
}
Am I missing something here. From looking at the error it looks like the #Bean annotation is not visible to the factory. Do I have to do anything else other than annotating the #configuration class.
I might also be using the annotations in a wrong way. I could be making a mistake in how I am using #Autowired or #Repository.
Can you please help.
I think this might be your issue:
"Also I have created a configuration class separately. I am using this to define my bean. I am only using this to specify bean and have not imported it anywhere in the code."
Somewhere you need to tell Spring to look for BeanConfiguration. You can do this in your applicationContext.xml file (assuming you have one) as follows:
<context:component-scan base-package="com.sachin.config" />
This assumes BeanConfiguration is in the com.sachin.config package.
If you can't find where to put this it may be helpful to share your web.xml file.
I am having the described behaviour in a test class.
I work in IntelliJ 2020.1, with an old project build on:
Java 1.6
Spring 3.0.5.RELEASE
Maven POM 4.0.0
Maven 3.2.5
The test class starts as:
#Test
#ContextConfiguration(locations={ "classpath*:beans_sets/UhradyForIns.xml"})
#TransactionConfiguration(transactionManager="transactionManager", defaultRollback=false)
#DirtiesContext(classMode= DirtiesContext.ClassMode.AFTER_EACH_TEST_METHOD)
public class OdeslatJakobyAutomatickyUhraduTest extends TestBaseWithProperties {
private static final Logger log = LoggerFactory.getLogger(OdeslatJakobyAutomatickyUhraduTest.class);
#Autowired
U01UhradaBaseJpaDaoForTesting dao;
Sometimes suddenly, after some changes in POMs, Spring cannot autowire the dao giving exactly your messages. The dao is well described by its class and by XML context description but is suddenly invisible for Spring. I have tried many ways described on SO, adding annotations, changing configuration, but I have found, that it is all excessive. All I need is only to refresh the maven repositories 4 times by 4 different orders subsequently:
1. mvn -e -U clean install //In cmd line
2. Ctrl+Shift+A Reimport maven //in IntelliJ
3. double round arrows for refresh //in the maven window of IntelliJ
4. AGAIN REPEAT THE FIRST REFRESH.
After that, it is all OK and works fine till the next greater change in POMs.
Or... Till due to some inner reasons, Maven or IntelliJ damages the local jar repository without your participation. At least, I had a case, when I haven't touched the repository, but again that Could not autowire... message appeared. But this time only one
mvn -e -U clean install
was enough.
Obviously, there is/are some error(s) in the maven and/or maven plugin in IntelliJ. That problem is shown by Could not autowire... message. The repair is possible by a simple maven repository refresh or, in worse cases, demand a sequence of several different repository refreshes.
I understand that it has no obvious logic in that solution, and our developer's work resembles magic more and more. And this time, definitely, it is not your or my vice. Simply we have to adapt to existing errors. At least we know how to do it.
I'm attempting to build code into our base pom which autoconfigures Spring Cloud Config server lookup through Eureka. We're doing this to avoid templating .yml properties for developers building microservices. For example, we want to java config all behavior triggered from these properties:
spring:
application:
name: MyMicroservice
cloud:
config:
enabled: true
server:
prefix: /diagnostics/admin/config
failFast: true
discovery:
enabled: true
serviceId: echo
management:
context-path: /diagnostics/admin
eureka:
password: password
client:
serviceUrl:
defaultZone: http://user:${eureka.password}#localhost:8761/eureka/
instance:
leaseRenewalIntervalInSeconds: 10
statusPageUrlPath: /diagnostics/admin/info
healthCheckUrlPath: /diagnostics/admin/health
After much experimenting, the following approach mostly works except for the Eureka-discovered config server (resulting in no overridden config properties):
#Order(-1)
public class AdditionalBootstrapPropertySourceLocator implements PropertySourceLocator {
#Override
public PropertySource<?> locate(Environment environment) {
Map<String, Object> theBootstrapYmlConfig = new HashMap<>();
theBootstrapYmlConfig.put("spring.cloud.config.enabled", new Boolean(true));
theBootstrapYmlConfig.put("spring.cloud.config.server.prefix", "/diagnostics/admin/config");
theBootstrapYmlConfig.put("spring.cloud.config.failFast", new Boolean(true));
theBootstrapYmlConfig.put("spring.cloud.config.discovery.enabled", new Boolean(true));
theBootstrapYmlConfig.put("spring.cloud.config.discovery.serviceId", "echo");
theBootstrapYmlConfig.put("management.context-path", "/diagnostics/admin");
theBootstrapYmlConfig.put("eureka.client.serviceUrl.defaultZone", "http://user:password#localhost:8761/eureka/");
theBootstrapYmlConfig.put("eureka.instance.leaseRenewalIntervalInSeconds", new Integer(10));
theBootstrapYmlConfig.put("eureka.instance.statusPageUrlPath", "/diagnostics/admin/info");
theBootstrapYmlConfig.put("eureka.instance.healthCheckUrlPath", "/diagnostics/admin/health");
return new MapPropertySource("myExtraBootstrap", theBootstrapYmlConfig);
}
}
And I seem to need this Bean as well:
#ConditionalOnWebApplication
#Configuration
#Import(EurekaClientAutoConfiguration.class)
public class WorkfrontDiscoveryClientConfigServiceBootstrapConfiguration {
#Bean
#ConditionalOnClass({ DiscoveryClient.class, ConfigServicePropertySourceLocator.class })
#ConditionalOnMissingBean
DiscoveryClientConfigServiceBootstrapConfiguration discoveryClientConfigServiceBootstrapConfiguration() {
DiscoveryClientConfigServiceBootstrapConfiguration discoveryClientConfigServiceBootstrapConfiguration =
new DiscoveryClientConfigServiceBootstrapConfiguration();
return discoveryClientConfigServiceBootstrapConfiguration;
}
}
Finally, I put both into spring.factories to ensure they are constructed. The problem is that the PropertySourceLocator is never used to construct the call within ConfigServicePropertySourceLocator to retrieve the properties. No matter what I do, I cant seem to match the behaviors that specifying the properties within bootstrap.yml would produce.
Edit 4 days later
The key factor (and restriction) here is the ability to look up the config server through Eureka. In the current spring cloud release (1.0.2), the property source is retrieved and constructed too early in the spring initialization cycle for the config-lookup-through-eureka java property source config I have above. Plus if the Eureka server is slow or not available at bootstrap startup time, the Config server property source is never reconstructed when Eureka finally comes up. This in my mind is a bug.
I solved this all by eliminating the concept of looking up the config server through Eureka, and requiring this minimum config in bootstrap.yml:
spring:
application:
name: MyMicroservice
cloud:
config:
uri: http://localhost:8888/diagnostics/admin/config
eureka:
client:
serviceUrl:
defaultZone: http://user:password#localhost:8761/eureka/
and then setting the rest in the java AdditionalBootstrapPropertySourceLocator
Edit 30 days later
Java-configing bootstrap properties continues to be a challenge. I'm doing this because I'm developing a framework without templating or code generation (the premise of spring boot). I've added spring-retry to the mix and client-to-config gets retried but re-registration to Eureka does not. This is why Eureka-first had to be abandoned for me. I'd put my vote in for integrating spring-retry into the Eureka registration process so I can go back to Eureka-first for my framework. Still on Spring Cloud 1.0.2.
Edit 100 days later
Update for where we ended up. Continuing along our mantra of avoiding property templating, enforcing policies and practices within code .. and continuing without a Eureka-first concept, we abandoned PropertySourceLocator and simply used a SpringApplicationRunListener as follows:
public class OurFrameworkProperties implements SpringApplicationRunListener {
:
public void started() {
if (TestCaseUtils.isRunningFromTestCase()) {
System.setProperty("spring.cloud.config.failFast", "false");
System.setProperty("spring.cloud.config.enabled", "false");
System.setProperty("eureka.client.enabled", "false");
} else {
// set production values same way
}
}
}
A caution that this started() actually gets called twice inside the spring code (once not passing any program arguments btw) everytime your Spring application runs or gets an Actuator refresh().
If your PropertySourceLocator is listed inspring.factories (I assume as a BootstrapConfiguration) then it needs to be a #Component (or maybe even a #Configuration).
you have to set this properties in the boostrap.properties
eureka.instance.metadataMap.configPath: /your-app-name
and comment this one
#spring.cloud.config.uri=http://localhost:8888/
and obviously it must be also this
eureka.client.serviceUrl.defaultZone: ${EUREKA_URI:http://localhost:8761/eureka}
eureka.client.instance.preferIpAddress: true
according with the documentation
https://cloud.spring.io/spring-cloud-config/multi/multi__spring_cloud_config_client.html#discovery-first-bootstrap