ClientSecretCredential clientSecret = new ClientSecretCredentialBuilder()
.clientId("********-****-****-****-************").tenantId(
"********-****-****-****-************")
.clientSecret("*****~***************************~******").build();
BlobServiceClient blobServiceClient = new BlobServiceClientBuilder()
.credential(clientSecret)
.endpoint("https://sambhutestsa.blob.core.windows.net/")
.buildClient();
BlobContainerClient blobContainerClient = blobServiceClient.getBlobContainerClient("testcontainer");
System.out.printf("Identifier for the snapshot is %s%n", blobContainerClient.getBlobClient("2824891.jpg").downloadToFile("C:\\Users\\ss255248\\2824891.jpg");
but getting this error can someone please help here.
Exception in thread "main" com.azure.storage.blob.models.BlobStorageException: If you are using a StorageSharedKeyCredential, and the server returned an error message that says 'Signature did not match', you can compare the string to sign with the one generated by the SDK. To log the string to sign, pass in the context key value pair 'Azure-Storage-Log-String-To-Sign': true to the appropriate method call.
If you are using a SAS token, and the server returned an error message that says 'Signature did not match', you can compare the string to sign with the one generated by the SDK. To log the string to sign, pass in the context key value pair 'Azure-Storage-Log-String-To-Sign': true to the appropriate generateSas method call.
Please remember to disable 'Azure-Storage-Log-String-To-Sign' before going to production as this string can potentially contain PII.
Status code 403, "<?xml version="1.0" encoding="utf-8"?><Error><Code>AuthorizationPermissionMismatch</Code><Message>This request is not authorized to perform this operation using this permission.
RequestId:d813ea5f-d01e-004d-7b0f-656de3000000
Time:2022-05-11T08:13:50.0902556Z</Message></Error>
To resolve the error, please try the following:
Make sure whether you have below roles assigned to your security principal:
Storage Blob Data Contributor
Storage Blob Data Owner
If not, make sure to assign it like below:
Go to Azure Portal ->Identify the required scope (Subscription/Resource group) -> Access Control (IAM) -> Add Role assignment
Check whether your application has below API permissions:
For more information, please refer this MsDoc:
Otherwise, try setting the time zone to UTC-0 .
Try adding comp=list&restype=container to your SAS URL as a workaround.
Make use of the snippet suggested in this SO thread.
For more in detail, please refer below link:
Azure App Service rejected with 403 error by Storage Account - Stack Overflow
[Solved] Azure Shared Access Signature - Signature did not match - Local Coder
I am trying to access Azure Key Vault local by using Service Principle credentials from local for development perspective.
But it seems that Azure SDK is always checking IMDS connectivity ("169.254.169.254")
Code I used to retrieve secret:
SecretClient secretClient = new SecretClientBuilder()
.vaultUrl(keyVaultUri)
.credential(new DefaultAzureCredentialBuilder().build())
.buildClient();
I also added below variables as env variables:
AZURE_CLIENT_ID
AZURE_CLIENT_SECRET
AZURE_TENANT_ID
Can somebody help me with how can we access azure resources like key vault from our local using Service Principle in java
To use service principal to auth locally, just use ClientSecretCredential.
Sample:
import com.azure.identity.ClientSecretCredential;
import com.azure.identity.ClientSecretCredentialBuilder;
import com.azure.security.keyvault.secrets.SecretClient;
import com.azure.security.keyvault.secrets.SecretClientBuilder;
import com.azure.security.keyvault.secrets.models.KeyVaultSecret;
public class vacate {
public static void main(String[] args) {
String clientId="xxxxxx";
String clientSecret="xxxxxx";
String tenantId="xxxxxx";
ClientSecretCredential credential1 = new ClientSecretCredentialBuilder()
.tenantId(tenantId)
.clientId(clientId)
.clientSecret(clientSecret)
.build();
SecretClient secretClient = new SecretClientBuilder()
.vaultUrl("<your-key-vault-url>")
.credential(credential1)
.buildClient();
//do other things
}
}
Actually, I think DefaultAzureCredential you used should also work, it tries to create a valid credential in the following order, if you have already set the environment variable correctly, it should work, if not, just use the ClientSecretCredential like above, it will work.
The way it fixed my problem (and may be will help others as well):
Indeed as mentioned in Joy's answer, you need to use ClientSecretCredential or you can also use Azure Toolkit for IntelliJ for authentication
I was using old azure identity which was going to old authentication end point login.microsoftonline.com/{{tenant_id}} which got fixed after upgrading to latest version (1.2.3). Now it goes to new end point of login.microsoftonline.com/common/oauth2
For me, I was also getting a lot of SSL errors. To fix it adding below certificates to trusted certificates worked :
DigiCert Global Root CA
DigiCert SHA2 Secure Server CA
If your network is behind a proxy, you also need to configure proxy and added corresponding CA Root certificate to your keystore and truststore.
I'm using saml2aws to generate temporary AWS keys for my clients. Previously I was using environmental variables:
aws_access_key_id=MY KEY
aws_secret_access_key=MY SECRET
After authenticating with saml2aws my ~/.aws/config file looks like this (unchanged):
[default]
output=json
And my ~/.aws/credentials looks like this:
[default]
aws_access_key_id = MY KEY ID
aws_secret_access_key = MY KEY
aws_session_token = MY SESSION TOKEN
aws_security_token = MY TOKEN
x_principal_arn = MY ARN
x_security_token_expires = TIME
When I try this from the cli with aws s3 ls it works but when I try to access S3 from the Java SDK:
AmazonS3Client(ProfileCredentialsProvider())
.listObjects(ListObjectsRequest()
.withBucketName("some-bucket")
.withPrefix("some-prefix")
.withDelimiter("/")
.withMaxKeys(10000))
I get:
com.amazonaws.AmazonClientException: Unable to load credentials into profile. Profile Name or AWS Access Key ID or AWS Secret Access Key missing for a profile.
And it doesn't work even if I explicitly try to use the default profile: ProfileCredentialsProvider("default") or even if I don't set a provider at all!
What am I doing wrong?
AmazonS3Client().listObjects(ListObjectsRequest()
.withBucketName("some-bucket")
.withPrefix("some-prefix")
.withDelimiter("/")
.withMaxKeys(10000))
is enough, no need of explicitly giving ProfileCredentialsProvider().
For more see
https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/AmazonS3Client.html#AmazonS3Client--
I'm attempting to use Spring to access files from Google Storage Buckets with the end goal of using MultiResourceItemReader to read in multiple XML files from the bucket. I currently have Spring working with this process when the XML files are locally on my machine (not GCP)
Now, I want to do the same thing, but instead of XML files on my machine, the files are in a GCP Storage bucket. I can access the bucket contents outside of Spring, one file at at time. For example this little bit of test code allows me to get access to the bucket and then see the files in the bucket. In this snippet, I setup the credentials via the JSON key file. (not an environment variable)
public static void storageDriver() throws IOException {
// Load credentials from JSON key file. If you can't set the GOOGLE_APPLICATION_CREDENTIALS
// environment variable, you can explicitly load the credentials file to construct the
// credentials.
String name = "";
String bucketName = "";
String bucketFileName = "";
String bucketFullPath = "";
Resource myBucker;
GoogleCredentials credentials;
File credentialsPath = new File("mycreds.json");
try (FileInputStream serviceAccountStream = new FileInputStream(credentialsPath)) {
credentials = ServiceAccountCredentials.fromStream(serviceAccountStream);
}
Storage storage = StorageOptions.newBuilder()
.setCredentials(credentials)
.setProjectId("myProject")
.build()
.getService();
for (Bucket bucket:storage.list().iterateAll()){
if(bucket.getName().equalsIgnoreCase("myGoogleBucket")){
bucketName = bucket.getName();
System.out.println(bucket);
for (Blob blob : bucket.list().iterateAll()){
bucketFileName = blob.getName();
bucketFullPath = "gs://"+bucketName+"/"+bucketFileName;
System.out.println(bucketFullPath);
}
}
};
However, when I try to do the following with Spring, Spring complains that I don't have the GOOGLE_APPLICATION_CREDENTIALS defined. (which of course I don't since I'm doing it programmatically.
For example, I'll add
#Value("gs://myGoogleBucket")
private Resource[] resources;
The Application Default Credentials are not available. They are available if running in Google Compute Engine. Otherwise, the environment variable GOOGLE_APPLICATION_CREDENTIALS must be defined pointing to a file defining the credentials.
Spring Cloud GCP simplifies your GCS configuration.
You can add Storage support to your app. Then, either specify the location of your service account credentials through the spring.cloud.gcp.storage.credentials.location property, or by logging in with application default credentials using the Google Cloud SDK.
This will automatically provide you with a fully configured Storage object and things like #Value(gs://YOUR-BUCKET/YOUR-FILE) should just work.
I tried many ways, but at last this excerpt from spring docs is the one which worked for me:
Due to the way logging is set up, the GCP project ID and credentials defined in application.properties are ignored. Instead, you should set the GOOGLE_CLOUD_PROJECT and GOOGLE_APPLICATION_CREDENTIALS environment variables to the project ID and credentials private key location, respectively. You can do this easily if you’re using the Google Cloud SDK, using the gcloud config set project [YOUR_PROJECT_ID] and gcloud auth application-default login commands, respectively.
Can't seem to find this anywhere in the docs; if I am authenticated with BasicAWSCredentials, e.g. AccessKeyId and SecretKey, is it possible to get the AWS Account ID?
Update
AWS has just silently addressed this long standing gap by introducing the dedicated STS API action GetCallerIdentity, which returns details about the IAM identity whose credentials are used to call the API, including the AWS Account ID - there are a few sample responses, e.g.:
<GetCallerIdentityResponse xmlns="https://sts.amazonaws.com/doc/2011-06-15/">
<GetCallerIdentityResult>
<Arn>arn:aws:iam::123456789012:user/Alice</Arn>
<UserId>AKIAI44QH8DHBEXAMPLE</UserId>
<Account>123456789012</Account>
</GetCallerIdentityResult>
<ResponseMetadata>
<RequestId>01234567-89ab-cdef-0123-456789abcdef</RequestId>
</ResponseMetadata>
</GetCallerIdentityResponse>
You can use the AWS Command Line Interface to obtain just the account ID, here's an example:
$ aws sts get-caller-identity --output text --query Account
121371349383
Initial Answer
This is at least indirectly possible via AWS Identity and Access Management (IAM) by means of the GetUser action (available via getUser() in the AWS SDK for Java):
Retrieves information about the specified user, including the user's
path, GUID, and ARN.
If you do not specify a user name, IAM determines the user name
implicitly based on the AWS Access Key ID signing the request.
The returned User data type (Class User) contains an Arn element (getArn()), which is the Amazon Resource Name (ARN) specifying the user. This is further detaild in Identifiers for IAM Entities, specifically in section ARNs, which describes the format of the User Resource Type:
arn:aws:iam::{account_ID}:user/{path/to/user/UserName}
Example: arn:aws:iam::123456789012:user/division_abc/subdivision_xyz/Bob
This is an old question, but for the poor souls out there -
The answer based on ARN is the most correct answer that we found. There's also an OwnerId field when calling DescribeInstances, but there may be no instances ..
However the reality is a bit more complex. Sometimes the IAM user does not have the permission to issue getUser(), and then he gets AmazonServiceException with
getErrorCode() = "AccessDenied"
In this case, the ARN is a part of the AWS error message:
User: arn:aws:iam::123456789012:user/division_abc/subdivision_xyz/Bob is not authorized to perform:
iam:GetUser on resource: arn:aws:iam::123456789012:user/division_abc/subdivision_xyz/Bob
So here the matters are even worse: we need to parse a free text error message and then extract the account number:
try {
... iam.getUser(); ...
} catch (AmazonServiceException e) {
if (e.getErrorCode().compareTo("AccessDenied") == 0) {
String arn = null;
String msg = e.getMessage();
// User: arn:aws:iam::123456789012:user/division_abc/subdivision_xyz/Bob is not authorized to perform: iam:GetUser on resource: arn:aws:iam::123456789012:user/division_abc/subdivision_xyz/Bob
int arnIdx = msg.indexOf("arn:aws");
if (arnIdx != -1) {
int arnSpace = msg.indexOf(" ", arnIdx);
arn = msg.substring(arnIdx, arnSpace);
}
System.out.println("ARN: " + arn);
}
If you have the AWS CLI tools, you can:
aws iam get-user | awk '/arn:aws:/{print $2}'
Here's the code to get the newer STS getCallerIdentity stuff working (in Java):
private String getAccountIDUsingAccessKey(String accessKey, String secretKey) {
AWSSecurityTokenService stsService = AWSSecurityTokenServiceClientBuilder.standard().withCredentials(
new AWSStaticCredentialsProvider(new BasicAWSCredentials(accessKey, secretKey))).build();
GetCallerIdentityResult callerIdentity = stsService.getCallerIdentity(new GetCallerIdentityRequest());
return callerIdentity.getAccount();
}
Props to #SteffenOpel for giving the clues needed, of course.
With the latest API there's a direct way of finding the user ID:
BasicAWSCredentials basicAWSCredentials = new BasicAWSCredentials("your access key", "your secret key");
AmazonIdentityManagementClient iamClient = new AmazonIdentityManagementClient(basicAWSCredentials);
String userId = iamClient.getUser().getUser().getUserId();
Adding this here, because it's the top SO result for "find aws account number", and because it's bad practice to use keys instead of IAM Roles for deployed apps anyway...
If you're running from an AWS instance that has an IAM Role on it, you can do curl -s http://169.254.169.254/latest/meta-data/iam/info and get the ARN of the instance's role from the InstanceProfileArn key of the results, without having to worry about trying to parse an exception message or granting IAM permissions to an instance that doesn't need them.
Then, you just have to parse the ARN for the account number.
I made a funny discovery - if you call GetRole with a non-existent RoleName, the error message you get back contains the ARN of the calling account, so just parse out the account number from that. This is nice because it works in all cases that I can think of, even if the caller does not have permissions to call GetRole.
Here's the error message I get:
User: arn:aws:sts::669916120315:assumed-role/CloudMail_Server/i-31dd19cd is not authorized to perform: iam:GetRole on resource: role _no_such_role_
The '669916120315' portion of the error message is the AWS Account ID.
While I don't consider this an ideal scenario, it does get the job done. This uses the AWSSDK 3.0.
public string GetUserId()
{
AmazonIdentityManagementServiceClient c =
new AmazonIdentityManagementServiceClient();
GetUserRequest request = new GetUserRequest();
GetUserResponse response = c.GetUser(request);
//parse it from the ARN
//should be similar to "arn:aws:iam::111111111111:user/username"
string[] arnParts = response.User.Arn.Split(new char[] { ':' });
return arnParts[4];
}
Using AWS Java SDK v2:
public String getAccountId(AwsCredentialsProvider awsAuth, Region awsRegion) {
StsClient stsClient = getClient(awsAuth, awsRegion);
GetCallerIdentityRequest request = GetCallerIdentityRequest
.builder()
.build();
GetCallerIdentityResponse response = stsClient.getCallerIdentity(request);
return response.account();
}
public StsClient getClient(AwsCredentialsProvider awsAuth, Region awsRegion) {
return StsClient
.builder()
.credentialsProvider(awsAuth)
.region(awsRegion)
.build();
}
The owner of the default security group is the "account ID"
ElasticTranscoder ListPresets returns a data structure that includes ARNs that include the AWS account id.
Unlike many other popular suggestions, this aws-cli command works for basic credentials, IAM user, IAM roles, instance roles, and cross-account role assumption:
aws elastictranscoder list-presets --query 'Presets[0].Arn' |
cut -d: -f5
Of course, you will need permission to make the ListPresets API call, but that's true of any answer.
Some answers provide a way to retrieve the AWS Account ID from an IAM User. Those solutions will generate a runtime exception if you use the credentials of an IAM Role.
In this case, you can do this:
// you can get it with #Import(SnsConfiguration::class) for instance
#Autowired
private AWSCredentialsProvider credentialsProvider;
private String resolveAmazonAccountId() {
AmazonIdentityManagement imClient = AmazonIdentityManagementClientBuilder.standard()
.withCredentials(credentialsProvider)
.build();
String roleArn = imClient.getRole(new GetRoleRequest().withRoleName("role name")).getRole().getArn();
// https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_identifiers.html#identifiers-arns
// arn:partition:service:region:account:resource
return roleArn.split(":")[4];
}