AWS Instance Profile doesn't work with Spring Cloud AWS - java

I have a small Spring Boot app, using Spring Cloud AWS (1.0.0.RELEASE) to access SQS queue. It is beeing deployed on an EC2 instance with Instance Profile set. It appears that AWS side of things is working, as I can access both relevant metadata links: iam/info and iam/security-credentials/role-name, and they do contain correct information. Just to be sure, I've used aws cmdline utility (aws sqs list-queues) and it does work, so I guess setup is ok. However, when the app starts, it reads application.properties (which contains line cloud.aws.credentials.instanceProfile=true) then drops following warning: com.amazonaws.util.EC2MetadataUtils: Unable to retrieve the requested metadata and finally throws following exception:
Caused by: com.amazonaws.AmazonServiceException: The security token included in the request is invalid. (Service: AmazonSQS; Status Code: 403; Error Code: InvalidClientTokenId; Request ID: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx)
at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:1071)
at com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:719)
at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:454)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:294)
at com.amazonaws.services.sqs.AmazonSQSClient.invoke(AmazonSQSClient.java:2291)
at com.amazonaws.services.sqs.AmazonSQSClient.getQueueUrl(AmazonSQSClient.java:516)
at com.amazonaws.services.sqs.buffered.AmazonSQSBufferedAsyncClient.getQueueUrl(AmazonSQSBufferedAsyncClient.java:278)
at org.springframework.cloud.aws.messaging.support.destination.DynamicQueueUrlDestinationResolver.resolveDestination(DynamicQueueUrlDestinationResolver.java:78)
at org.springframework.cloud.aws.messaging.support.destination.DynamicQueueUrlDestinationResolver.resolveDestination(DynamicQueueUrlDestinationResolver.java:37)
at org.springframework.messaging.core.CachingDestinationResolverProxy.resolveDestination(CachingDestinationResolverProxy.java:88)
at org.springframework.cloud.aws.messaging.listener.AbstractMessageListenerContainer.start(AbstractMessageListenerContainer.java:295)
at org.springframework.cloud.aws.messaging.listener.SimpleMessageListenerContainer.start(SimpleMessageListenerContainer.java:38)
at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:173)
... 17 common frames omitted
...which means that for some reason Spring Cloud AWS is not picking up on Instance Profile credentials. I've enabled debug log level on com.amazonaws.request and it appears that request is sent without access key and secret key.
DEBUG --- com.amazonaws.request : Sending Request: POST https://sqs.eu-west-1.amazonaws.com / Parameters: (Action: GetQueueUrl, Version: 2012-11-05, QueueName: xxxxxxxxxxxxx, ) Headers: (User-Agent: aws-sdk-java/1.9.3 Linux/3.14.35-28.38.amzn1.x86_64 Java_HotSpot(TM)_64-Bit_Server_VM/25.45-b02/1.8.0_45 AmazonSQSBufferedAsyncClient/1.9.3, )
Anybody has any idea what am I missing or at least any hints how to further debug this?
EDIT: After going through spring-cloud-aws code a bit, I've kinda moved forward. Configuration file application.properties bundled with jar had some text value for accessKey and secretKey. My customized application.properties haven't got those properties and that probably caused spring to use values in bundled file as defaults. I've included them with empty values, which changed the exception to com.amazonaws.AmazonClientException: Unable to load AWS credentials from any provider in the chain. It appears that AWS SDK is configured with DefaultProviderChain, yet it still fails to pick up instance profile credentials.

The solution to this problem comes from two distinct facts.
Instance profile credentials are going to be used only and only if application.properties has instanceProfile property set to true and accessKey set to null (ContextCredentialsAutoConfiguration).
Even if you will provide your custom application.properties file, Spring is going to read application.properties file bundled with app jar (if it does exist). If that's the case, properties from both files will sum up to create an execution enviroment. I suspect that bundled file is parsed first, then custom second, overriding any property present in bundled file.
In my case, bundled application.properties had accessKey and secretKey placeholders (with phony values) which were filled out by developer whenever he wanted some testing outside of EC2 enviroment. That made accessKey not null and therefore, excluded instance profile path. I just removed the application.properties file from jar and that solved the problem.

cloud:
aws:
credentials:
accessKey:
secretKey:
instanceProfile: true
useDefaultAwsCredentialsChain: true
This would do the trick, if you were using the latest (2.X.X) Spring AWS Cloud.

Related

Unable to connect to key vault using spring boot

I am following this part in the tutorial for integrating key vault with spring boot.
We have everything setup for using Managed identities (application.properties and dependency in pom) and since it doesn't run in local development, I am testing it by deployment, however I am getting these errors in the log stream:
java.io.FileNotFoundException:/home/tomcat/lib/azure.appservice.jar (No such file or directory)
and
java.lang.IllegalStateException: Failed to configure KeyVault property source
java.io.IOException: Server returned HTTP response code: 400 for URL: Http://xxx.xx.x.x.x
EDIT: turned out I had a typo error in the client-id property.
make sure the application has permissions to get and list secrets in access policies of your key vault and also check the jar files are added properly.
Edit the src/main/resources/application.properties file so that it has the following contents, adjusting the values for your Azure subscription.c
azure.keyvault.client-id=<your client ID>
azure.keyvault.client-key=<your client key>
azure.keyvault.enabled=true
azure.keyvault.tenant-id=<your tenant ID>
azure.keyvault.uri=https://contosokv.vault.azure.net/
After adding azure.keyvault.tenant-id=<your-tenant-id> in the file:
For more details refer this document and follow this Github code

io.smallrye.mutiny.TimeoutException after adding quarkus.vault. to application.propperties

I'm writing my first bigger Quarkus Applications. Using maven and Java 11, on Windows.
At the moment, I'm trying to make it more secure. I Implemented Authentication using smallrey jwt and that worked fine. Now I'm trying to secure my application.properties by saving the secrets and keys inside a Hashicrop Vault.
I followed this guide to implement this. It´s working fine until the compilation part. If I try to compile after adding these lines to my application.property
# vault url
quarkus.vault.url=http://localhost:8200
quarkus.vault.kv-secret-engine-version=2
# vault authentication
quarkus.vault.authentication.client-token=xxx
# path within the kv secret
quarkus.vault.secret-config-kv-path=test
And try to compile it wit mvn quarkus:dev. I get this error:
ERROR: Failed to start application (with profile dev)
io.smallrye.mutiny.TimeoutException
at
io.smallrye.mutiny.operators.uni.UniBlockingAwait.await(UniBlockingAwait.java:58)
at io.smallrye.mutiny.groups.UniAwait.atMost(UniAwait.java:61)
at
io.quarkus.vault.runtime.client.VertxVaultClient.exec(VertxVaultClient.java:161)
at
io.quarkus.vault.runtime.client.VertxVaultClient.exec(VertxVaultClient.java:146)
at
io.quarkus.vault.runtime.client.VertxVaultClient.get(VertxVaultClient.java:127)
Link to the complete log. I'm a bit lost and can't read anything meaningful out of the error log, so I'm not sure where to look for the wrong configuration.
I'm not trying to access anything inside the vault at the moment.

After deployed the App in Elastic Beanstalk end point of app throws 404 not found in SpringBoot+MySql+Angular app with maven

is literally my first time on AWS deployments , and doing it by myself is harder the task.
After having created my app in Elastic Beans and its respective RDS database with its instance , i created a Snapshot (jar) on my Springboot app which was also implemented in the deployment process of the AWS application.
Also several items were configured in its Software Category referring the RDS database endpoint, server ports, user-name of database , etc...
.
Then after all that process , got the app deployed with a url.
But when i apply that url with the endpoints my springboot controllers have , i receive as error a 404 Not Found; but if i decide to work on local requesting only the RDS database created by the application in Elastic Bean the endpoints shows data and the app works
Literally on my Spring Boot App i declared in the app. properties the connection to that database in AWS
spring.datasource.url=jdbc:mysql://aat54g98qswmf3.clnhc7kmwszz.us-west-2.rds.amazonaws.com:3306/ebdb?useUnicode=true&useJDBCCompliantTimezoneShift=true&useLegacyDatetimeCode=false&serverTimezone=UTC
spring.datasource.username=root
spring.datasource.password=xxxx
spring.jpa.database-platform=org.hibernate.dialect.MySQL5InnoDBDialect
logging.level.org.hibernate.SQL=debug
thus my controllers in the spring working in local don't have any problem
#RequestMapping(path = "/restaurant-booking/version1")
public class RestaurantController {
#Autowired
RestaurantService restaurantService;
#ResponseStatus(HttpStatus.OK)
#RequestMapping(value = "/restaurant/all", method = RequestMethod.GET, produces = MediaType.APPLICATION_JSON_VALUE)
some code.........
};
then on my browser:
But if i switch to the url facilitated by the deployed EB app in AWS, and the use the same endpoint
Any help guys would be amazing , cause honestly can't find the problem or where to look at!
Thanks in advance!!!
The whitelabel error page is the default error page in spring when you have not defined your own. This means your application is deployed and running. You either messed up your request mapping or your url.
If we look at your mappings we see the url should be .../restaurant-booking/version1/restaurant/all
Request mappings get nested when they are on the class and method level.
You actually used the correct url locally but not on your deployed version.
The problem might be arising due to the connectivity with the RDS and the correct Environment configuration for your Spring Boot Application.
I would suggest that prior to continuing with the environment creation (or even after), make sure you connect the RDS properly. For the configuration part:-
you can make use of application-prod.properties file and specify an environment variable and value for software configuration labelled SPRING_PROFILES_ACTIVE and set its value to prod.
The application-prod properties consist of:-
server.port=5000
spring.datasource.url=jdbc:mysql://${RDS_HOSTNAME}:${RDS_PORT}/${RDS_DB_NAME}
spring.datasource.username=${RDS_USERNAME}
spring.datasource.password=${RDS_PASSWORD}
spring.jpa.hibernate.ddl-auto=create
Select a suitable RDS before or after creating the environment.
You need to configure the inbound and outbound rules for security group encompassing the RDS manually to allow access for creating, delete, update, etc.
And then finally upload the jar file in the environment. Worked for me.

How to run the app with spring-cloud-starter-aws locally?

I need to run Spring Boot based app locally. It uses spring-cloud-starter-aws dependency.
The problem is that it tries to connect to EC2 metadata service always. Setting "cloud.aws.*" properties doesn't help.
I expect that default AWS credentials chain will be used, credentials and region will be read from one of AWS preferred way (e.g. ~/.aws/config and ~/.aws/credentials files).
I tried to set cloud.aws.credentials.useDefaultAwsCredentialsChain property but spring-cloud-starter-aws doesn't care
I found examples that use CloudFormation stack for very strange reason to run the app locally.
When I use AWS SDK for Java default AWS chain is used without any issues - I don't need to do anything specific for local running of the application (locally it reads credentials from files and on EC2 it uses instance metadata service). But with Spring Boot it doesn't work out of the box and I need to enable local running somehow.
I use 2.2.2.RELEASE version of Spring Boot and 2.2.1.RELEASE version of Spring Cloud. I have a feeling they introduced regression, because in previous versions it worked without problems.
Any ideas how to run the app locally?
Adding the following lines to configuration helps:
cloud.aws.region.static=my region
cloud.aws.stack.auto=false
spring.autoconfigure.exclude=org.springframework.cloud.aws.autoconfigure.metrics.CloudWatchExportAutoConfiguration
So Spring uses AWS default chain but only for credentials. AWS SDK uses it for region and other configuration parameters too. So this is Spring bug for sure.
It still gives a warning about no connection to instance metadata service once during application start but more or less this solution can be used for local running.
If we don't have the last line with excluding CloudWatchExportAutoConfiguration, there will be many exceptions in stack trace while closing the app. I use CloudWatch metrics in my app.
I guess rationale behind excluding aws auto configuration is that it has conflicts with boot actuator but I'm not sure.

Couchbase DefaultOrphanResponseReporter Orphan responses observed

I have a SpringBoot 2 app that uses using Spring Data Couchbase.
I have this message on the logs every minute
2019-11-12 13:48:48,924 WARN : gid: trace= span= [cb-orphan-1] c.c.c.c.t.DefaultOrphanResponseReporter Orphan responses observed: [{"top":[{"r":"10.120.93.220:8092","s":"view","c":"5BE128F6F96A4D28/FFFFFFFFDA2C8C52","l":"10.125.216.233:49893"}],"service":"view","count":1}]
That is from the new Response Time Observability feature underlying the Java SDK.
It would seem to indicate that you have view requests which are timing out, but eventually received later, but I have no views defined in Couchbase DB
I would like to know if it is possible to disable OrphanResponseLogReporter via YML file config in a SpringBoot app. , setting the logIntervalNanos to 0
No, unfortunately, you cannot do it. Only a subset of Couchbase's configuration properties is supported in the application.yml, namely the ones present in the CouchbaseProperties.java class.
You could although use an environment variable: com.couchbase.orphanResponseReportingEnabled=false. It is independent of Spring, it's read directly by Couchbase SDK.
Edit:
As a workaround, you can set logging level in the application.yml:
logging.level.com.couchbase.client.core.tracing.DefaultOrphanResponseReporter: ERROR

Categories

Resources