when deploying java app on EC2 i can just use
new ProfileCredentialsProvider()
to inject some instance credentials. but which IAM user will it be? how can i manage permissions of that user? for example to limit his rights to some specific S3 bucket
Mark B is 100% correct about EC2 Instance Profiles. Quick aside -- the Java ProfileCredentialsProvider refers to the credentials profile available to AWS SDKs and the AWS CLI rather than the EC2 instance profiles. From its documentation:
Credentials provider based on AWS configuration profiles. This
provider vends AWSCredentials from the profile configuration file for
the default profile, or for a specific, named profile.
AWS credential
profiles allow you to share multiple sets of AWS security credentials
between different tools like the AWS SDK for Java and the AWS CLI.
This isn't desirable for deployment to EC2 because it means you have to store your configuration profile on the EC2 instance. Per Mark B's answer, EC2 instance profiles are preferable for passing credentials to applications on EC2.
To be able to retrieve either without further code changes, simply use the DefaultAWSCredentialsProviderChain (note: renamed DefaultCredentialsProvider in AWS SDK for Java v2), which will fall through several credentials schemes until it finds a valid one on your system. Per the documentation, it looks for credentials in this order:
Java System Properties - aws.accessKeyId and aws.secretKey
Environment Variables - AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY
Credential profiles file at the default location (~/.aws/credentials) shared by all AWS SDKs and the AWS CLI
Credentials delivered through the Amazon EC2 container service if AWS_CONTAINER_CREDENTIALS_RELATIVE_URI" environment variable is set
and security manager has permission to access the variable.
Instance profile credentials delivered through the Amazon EC2 metadata service
This will allow you to look for credential profiles in your local development environment, and EC2 Instance Profiles in your production EC2 environment.
Further Reading
AWS Documentation - Using Instance Profiles
AWS Documentation - Working with AWS Credentials - Note this documentation is for the AWS SDK for Java 2.0 preview
It isn't an IAM user, it is an EC2 Instance Profile. First you have to create the Instance Profile in your account, just like you have to create IAM users. You would manage the rights of the EC2 instance profile exactly how you would manage the rights of an IAM user, by assigning the appropriate IAM security Policies to the Instance Profile.
Related
I looked at https://github.com/kubernetes-client/java library but it requires RBAC enabled in cluster. Is any other way to retrieve pods in kubernetes programatically?
As per Kubernetes Java Client library you can find there:
InClusterClient Example (Configure a client while running inside the Kubernetes cluster.).
KubeConfigFileClient Example: (Configure a client to access a Kubernetes cluster from outside.)
The first example from inside the cluster is using serviceaccount applied to the POD.
The second example from outside the cluster is using kubeconfig file.
In the official docs you can find java example of Accessing Kubernetes API Using Java client I it uses kubeconfig file by default stored in $HOME/.kube/config. In addition you can find there other examples how to programmatically access the Kubernetes API with the list of Officially-supported Kubernetes client libraries and Community-maintained client libraries
Please refer also to the Authorization Modes
Kubernetes RBAC allows admins to configure and control access to Kubernetes resources as well as the operations that can be performed on those resources.
RBAC can be enabled by starting the API server with --authorization-mode=RBAC
Kubernetes includes a built-in role-based access control (RBAC) mechanism that allows you to configure fine-grained and specific sets of permissions that define how a given GCP user, or group of users, can interact with any Kubernetes object in your cluster, or in a specific Namespace of your cluster.
Additional resources:
Using RBAC Authorization
Accessing Clusters
Configure Service Accounts for Pods
Authorization Modes
Kubernetes in Production
Hope this help.
We have the application hosted locally not on EC2. Is it possible to access the AWS S3 using the IAM Role instead of profile or credentials from java?
The security team has raised concern about storing the credentials locally as it is vulnerable.
As far as I have googled, I have found options to access using Credentials stored in Environment or in .aws as a profile. If we need ROLE based authentication, then the application is supposed to be deployed in EC2. But we have the server hosted locally.
Please provide if you have any suggestions.
I'm not sure I understand what you want to do, but why not assign a dedicated user to that role?
Another thing you might find useful, is using temporary credentials. In general, Amazon have two services for this - Cognito and STS (as far as I understand, Cognito is using STS behind the scenes). They have different limitations, but in general, they allow you to receive credentials for limited time. This way, you will get an access id, a secret (and also a session id), but they will be temporary.
We have the application hosted locally not on EC2. Is it possible to access the AWS S3 using the IAM Role instead of profile or credentials from java?
Service-roles are bound to AWS services, so - long story short - for your on-premise server you need to use AWS API keys.
The security team has raised concern about storing the credentials locally as it is vulnerable.
Unfortunatelly - at the end you need to store the credentials somewhere. Even using services such as Cognito or STS you will need to store the credentials for the service somewhere (effectively - for any external or cloud service regardless what cloud or service you may use).
IMHO the best you can do is using dedicated AWS credentials (API keys) with only permission what are really needed.
What server side technologies would i need to learn to develop a cloud based storage system for users using my service?
Currently i am using java spring and hibernate and have developed a login system. I am wondering how would i be able to store users's files on my server separately for each user and allow access to files accordingly.
It seems that you are looking for a document-oriented database: https://en.m.wikipedia.org/wiki/Document-oriented_database
In case you're not allowed to use a fully managed service such as S3, there are options like MongoDB cluster: https://docs.aws.amazon.com/quickstart/latest/mongodb/architecture.html
"The following AWS components are deployed and configured as part of this reference deployment:
A VPC configured with public and private subnets across three Availability Zones.*
In the public subnets, NAT gateways to allow outbound internet connectivity for resources (MongoDB instances) in the private subnets. (For more information, see the Amazon VPC Quick Start.)*
In the public subnets, bastion hosts in an Auto Scaling group with Elastic IP addresses to allow inbound Secure Shell (SSH) access. One bastion host is deployed by default, but this number is configurable. (For more information, see the Linux bastion host Quick Start.)*
An AWS Identity and Access Management (IAM) instance role with fine-grained permissions for access to AWS services necessary for the deployment process.
Security groups to enable communication within the VPC and to restrict access to only necessary protocols and ports.
In the private subnets, a customizable MongoDB cluster with the option of running standalone or in replica sets, along with customizable Amazon EBS storage. The Quick Start launches each member of the replica set in a different Availability Zone. However, if you choose an AWS Region that doesn’t provide three or more Availability Zones, the Quick Start reuses one of the zones to create the third subnet.
You can choose to launch the Quick Start for a new VPC or use your existing VPC. The template that deploys the Quick Start into an existing VPC skips the creation of components marked by asterisks and prompts you for your existing configuration."
I have some Java applications running on Google Compute Engine instances in a Container Engine cluster. I upgraded the cluster to newest version (1.7.8) and changed the node images from the container optimised OS to Ubuntu. Now my pods are crashing when trying to connect to the cloud sql database with this error message:
The Application Default Credentials are not available. They are available if running in Google Compute Engine.
Otherwise, the environment variable GOOGLE_APPLICATION_CREDENTIALS must be defined pointing to a file defining the credentials.
See https://developers.google.com/accounts/docs/application-default-credentials for more information.
The service account is the same as before the upgrade with the scope https://www.googleapis.com/auth/sqlservice.admin
Does anyone have an idea why I'm getting this error now? Is the best solution is to create the environment variable?
Ideally you should be using GOOGLE_APPLICATION_CREDENTIALS environment variable. Otherwise you are using the "Compute Engine default service account" of the VMs.
See this tutorial for the best practices: https://cloud.google.com/kubernetes-engine/docs/tutorials/authenticating-to-cloud-platform
I am a Mac user and don't have access to the powershell cli tools. I want to authenticate my java program to the azure in resource manager mode (arm)
According to the docs i have read so far, i have created a app in the Active directory, but I don't see any option to upload any certificate to the application for authentication in the UI. Also, I don't see any option in the UI for assigning privilege to the application for using resource manager api.
Is using powershell the only option to do the above tasks ?
I am a Mac user and don't have access to the powershell cli tools. I
want to authenticate my java program to the azure in resource manager
mode (arm)
I'm not sure why you say that. CLI tools are built using node.js (thus available on all platforms) and are different than Azure PowerShell Cmdlets. Please see this link for using CLI tools with Azure Resource Manager - https://azure.microsoft.com/en-in/documentation/articles/xplat-cli-azure-resource-manager/.
According to the docs i have read so far, i have created a app in the
Active directory, but I don't see any option to upload any certificate
to the application for authentication in the UI.
With Azure Resource Manager, you don't have to do that. The authentication/authorization will be done via Azure Active Directory. Certificate based authentication is for Azure Service Management API.
Also, I don't see any option in the UI for assigning privilege to the
application for using resource manager api.
This, you can do through Azure Preview Portal. See this screenshot below where I have assigned permission to an application on my Azure Subscription.