I have a Github action pipeline that can successfully create an S3 and then upload my war file into there, but when deploying to the beanstalk, always got s3 access denied error. below is my build.yml file:
# This workflow will build a package using Maven and then publish it to GitHub packages when a release is created
# For more information see: https://github.com/actions/setup-java/blob/main/docs/advanced-usage.md#apache-maven-with-a-settings-path
name: Maven Package
on:
pull_request:
branches:
- main
push:
branches:
- develop
jobs:
build:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- uses: actions/checkout#v2
name: Set up JDK 8
- uses: actions/setup-java#v2
with:
java-version: '8'
distribution: 'adopt'
server-id: github # Value of the distributionManagement/repository/id field of the pom.xml
settings-path: ${{ github.workspace }} # location for the settings.xml file
- name: Build with Maven
run: mvn -B package --file pom.xml
- name: make a new dir and upload war in there
run: mkdir staging && cp -r target/* staging
- uses: actions/upload-artifact#v2
with:
name: Package
path: staging
- name: list all files
run: ls && cd target && ls
- name: Publish to GitHub Packages Apache Maven
run: mvn deploy -s $GITHUB_WORKSPACE/settings.xml
env:
GITHUB_TOKEN: ${{ github.token }}
- name: Deploy to EB
uses: einaregilsson/beanstalk-deploy#v18
with:
aws_access_key: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws_secret_key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
application_name: springbootwebapi
environment_name: Springbootwebapi-env
version_label: v1.0.10
region: us-east-2
deployment_package: target/login-0.0.2-SNAPSHOT.war
below is the some log snippet from GitHub action:
No existing bucket name given, creating/requesting storage location
Uploading file to bucket elasticbeanstalk-us-east-2-148565102071 New
build successfully uploaded to S3,
bucket=elasticbeanstalk-us-east-2-148565102071,
key=/springbootwebapi/v1-0-10.zip Created new application version
v1.0.10 in Beanstalk. Starting deployment of version v1.0.10 to
environment Springbootwebapi-env Deployment started,
"wait_for_deployment" was true...
18:17:02 INFO: Environment update is starting. 18:17:06 ERROR:
Service:Amazon S3, Message:Access Denied 18:17:06 ERROR: Failed to
deploy application. 18:17:07 ERROR: Service:Amazon S3, Message:Access
Denied: S3Bucket=elasticbeanstalk-us-east-2-148565102071,
S3Key=resources/environments/e-fp5bx3gtdn/_runtime/_versions/springbootwebapi/v1.0.10
18:17:13 ERROR: Deployment failed! Current State: Version: Sample
Application, Health: Red, Health Status: Degraded Error: Deployment
failed: Error: Deployment failed! Current State: Version: Sample
Application, Health: Red, Health Status: Degraded
I don't know why got accessed denied even right after the uploading successfully.
UPDATE 1:
I already have the below permissions added see the below, but not working:
As per docs, you need to attach the below policies for the AWS user to be able to deploy your project when using the GitHub action you have specified:
AWSElasticBeanstalkWebTier
AWSElasticBeanstalkManagedUpdatesCustomerRolePolicy
Adding the above will fix the problem, while also ensuring that you have no future issues when using this GitHub action.
After removing AWSCompromisedKeyQuarantineV2 from the permission list, it works successfully. The reason is this permission actually denies several related operations to the user, see the below JSON for AWSCompromisedKeyQuarantineV2 details:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Deny",
"Action": [
"ec2:RequestSpotInstances",
"ec2:RunInstances",
"ec2:StartInstances",
"iam:AddUserToGroup",
"iam:AttachGroupPolicy",
"iam:AttachRolePolicy",
"iam:AttachUserPolicy",
"iam:ChangePassword",
"iam:CreateAccessKey",
"iam:CreateInstanceProfile",
"iam:CreateLoginProfile",
"iam:CreatePolicyVersion",
"iam:CreateRole",
"iam:CreateUser",
"iam:DetachUserPolicy",
"iam:PassRole",
"iam:PutGroupPolicy",
"iam:PutRolePolicy",
"iam:PutUserPermissionsBoundary",
"iam:PutUserPolicy",
"iam:SetDefaultPolicyVersion",
"iam:UpdateAccessKey",
"iam:UpdateAccountPasswordPolicy",
"iam:UpdateAssumeRolePolicy",
"iam:UpdateLoginProfile",
"iam:UpdateUser",
"lambda:AddLayerVersionPermission",
"lambda:AddPermission",
"lambda:CreateFunction",
"lambda:GetPolicy",
"lambda:ListTags",
"lambda:PutProvisionedConcurrencyConfig",
"lambda:TagResource",
"lambda:UntagResource",
"lambda:UpdateFunctionCode",
"lightsail:Create*",
"lightsail:Delete*",
"lightsail:DownloadDefaultKeyPair",
"lightsail:GetInstanceAccessDetails",
"lightsail:Start*",
"lightsail:Update*",
"organizations:CreateAccount",
"organizations:CreateOrganization",
"organizations:InviteAccountToOrganization",
"s3:DeleteBucket",
"s3:DeleteObject",
"s3:DeleteObjectVersion",
"s3:PutLifecycleConfiguration",
"s3:PutBucketAcl",
"s3:DeleteBucketOwnershipControls",
"s3:DeleteBucketPolicy",
"s3:ObjectOwnerOverrideToBucketOwner",
"s3:PutAccountPublicAccessBlock",
"s3:PutBucketPolicy",
"s3:ListAllMyBuckets"
],
"Resource": [
"*"
]
}
]
}
I have successfully published a Maven Package to Github Packages.
Looks like this on the Github packages page:
<dependency>
<groupId>com.conectar.meetings</groupId>
<artifactId>github-meetings-serverless-lib</artifactId>
<version>0.0.1-SNAPSHOT</version>
</dependency>
But when I try to use this package in my AWS SAM application, I get an error trying to install it.
It fails on the build step of my SAM app even though I supply the GITHUB_TOKEN as an env var:
- name: SAM Build for Prod Environment
run: |
cd $GITHUB_WORKSPACE
sam build --parameter-overrides EnvStage=prod TableName=${{ secrets.PROD_DB_TABLENAME }} && sam package --s3-bucket ${{ secrets.PROD_AWS_DEPLOY_BUCKET }}
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
AWS_DEFAULT_REGION: ${{ secrets.AWS_REGION }}
Here's the build error:
Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/ion/ion-java/1.0.2/ion-java-1.0.2.jar (565 kB at 333 kB/s)
Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpclient/4.5.12/httpclient-4.5.12.jar (778 kB at 458 kB/s)
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 6.056 s
[INFO] Finished at: 2021-03-12T12:55:57Z
[INFO] ------------------------------------------------------------------------
Error: Failed to execute goal on project CreateMeeting: Could not resolve dependencies for project CreateMeeting:CreateMeeting:jar:1.0: Could not find artifact com.conectar.meetings:github-meetings-serverless-lib:jar:0.0.1-SNAPSHOT -> [Help 1]
Error:
Error: To see the full stack trace of the errors, re-run Maven with the -e switch.
Error: Re-run Maven using the -X switch to enable full debug logging.
Error:
Error: For more information about the errors and possible solutions, please read the following articles:
Error: [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
Error: Process completed with exit code 1.
Why can't it find the artifact?
I expect it to use the GITHUB_TOKEN to authenticate the build, but it seems to not be using that.
The package is published in the same repo where Github action is also running.
How do I make AWS SAM know about this Github Package?
I'm working on a project where we use Firebase to store data and we're doing some unit tests. I have set a GitHub Action that executes mvn package and mvn test on every push or pull request, and the problem is that I receive the following error when it executes tests:
java.io.IOException: The Application Default Credentials are not available. They are available if running in Google Compute Engine. Otherwise, the environment variable GOOGLE_APPLICATION_CREDENTIALS must be defined pointing to a file defining the credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.
I have created the secret with the .json and this is how I wrote the GitHub Action:
name: Maven CI/CD
on:
push:
branches: [ develop ]
pull_request:
branches: [ develop ]
jobs:
build_and_test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: Set up JDK 15
uses: actions/setup-java#v1
with:
java-version: 15
- name: Cache the Maven packages to speed up build
uses: actions/cache#v1
with:
path: ~/.m2
key: ${{ runner.os }}-m2-${{ hashFiles('**/pom.xml') }}
restore-keys: ${{ runner.os }}-m2
- name: Build project with Maven
run: mvn -B package --file pom.xml
env:
GOOGLE_APPLICATION_CREDENTIALS: ${{ secrets.GOOGLE_APPLICATION_CREDENTIALS }}
- name: Run (J)Unit tests
run: mvn clean test
env:
GOOGLE_APPLICATION_CREDENTIALS: ${{ secrets.GOOGLE_APPLICATION_CREDENTIALS }}
I have found that it somehow wasn't using the updated version of the action. Anyway, I solved using google-github-actions/setup-gcloud#master set as follow:
- uses: google-github-actions/setup-gcloud#master
with:
project_id: ${{ secrets.GCP_PROJECT_ID }}
service_account_key: ${{ secrets.GCP_SA_KEY }}
export_default_credentials: true
- name: Set GCP credentials
run: gcloud info
GCP_PROJECT_ID contains the project id and GCP_SA_KEY contains the service account .json file obtained from Firebase.
Excuse my lack of technicality concerning this matter, as i am a solely beginner.
so I'm trying to install a microservice application, so in order to do that , I need to install both gateway and microservice app and linked them through jhipster-registry(correct me if I'm wrong please).
now I've installed both of these apps succesfully. for the gateway I get this message message well presented through html "Your request cannot be processed" when i navigate to the localhost provided by the gateway console while getting this :
---------------------------------------------------------
Application 'gateway2' is running! Access URLs:
Local: http://localhost:8080/
External: http://192.168.56.1:8080/
Profile(s): [dev, swagger]
----------------------------------------------------------
2020-05-03 07:08:14.483 INFO 7600 --- [ restartedMain] com.mservice.app.Gateway2App :
----------------------------------------------------------
Config Server: Not found or not setup for this application
----------------------------------------------------------
same "Config Server" message I've gotten in the microservice app after the mvnw command
issue was when i run mvnw in this jhipster-registry folder I get this error in the end, now I've gone through some github issue and found contributors advising to run maven globaly sort of see if it's an environment issue, well nothing changed about the error.
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 54.986 s
[INFO] Finished at: 2020-05-03T07:00:00Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.1:compile (default-compile) on project jhipster-registry: Fatal error compiling: invalid flag: --release -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
maybe useful informations : I'm using JWT security, PosgreSQL database(and it's succesfully linked), havent implemented any entity in neither of the apps, I'm not using any docker
and if I'm understanding something worng, please point it out I would love for it to make sense.
I am trying to create a pipeline using Concourse CI. The pipeline should:
Get the code from git.
Build and package everything using maven.
Push generated artifact (target/*war) to Cloud Foundry.
Steps 1 and 2 are executed successfully but after hours of trying different configurations, I am not able to access the generated artifact and push it to CF.
I am getting following error in last step: error invalid path: found 0 files instead of 1 at path: /tmp/build/put/mvn-package/target/udm-0.1.war
File pipeline.yml:
resources:
- name: branch-dev
type: git
source:
uri: {{git-url}}
branch: {{git-branch}}
private_key: {{private-repo-key}}
- name: PCF-Dev
type: cf
source:
api: {{pcf-api}}
username: {{pcf-username}}
password: {{pcf-password}}
organization: {{pcf-organization}}
space: {{pcf-space}}
skip_cert_check: false
jobs:
- name: udm
serial: true
plan:
- get: branch-dev
trigger: true
- task: mvn-package
privileged: true
file: branch-dev/ci/package.yml
- put: PCF-Dev
params:
manifest: branch-dev/ci/manifest.yml
path: mvn-package-output/target/udm-0.1.war
File manifest.yml
applications:
- name: udm
File package.yml:
platform: linux
image_resource:
type: docker-image
source:
repository: maven
tag: latest
inputs:
- name: branch-dev
outputs:
- name: mvn-package-output
run:
path: "mvn"
args: ["-f", "branch-dev/udm/pom.xml", "package"]
I guess I am missing something. Could someone take a look and point me in the right direction?
You need to put/get the resource (build artifact) between build/deploy jobs (and have it defined as a resource)
Pivotal projects have very good examples of production Concourse use, like https://github.com/azwickey-pivotal/volume-demo/blob/master/ci/pipeline.yml