I am following: AWS Docs to setup the credentials. The problem is that it requires me to create a .aws folder in the machine. I want to get the keys and other secrets from a custom location. How can it be achieved?
P.S. If I follow the tutorial recommendation then all machines running the project would have to setup that .aws folder which would be a big hassle for everyone.
Where exactly would you suggest getting the credentials from? You could store them somewhere else, like a HashiCorp Vault server, and write a script or something to pull the values and set them as environment variables, but then you'll need to figure out how to give each computer secure credentials to access the Vault server.
If by "custom location" you simply mean a different local file system location, like a mapped drive or something, then you can specify that using the AWS_CREDENTIAL_PROFILES_FILE environment variable. Although it sounds like you want to do this on multiple people's workstations, and I would caution against sharing credentials files in that scenario. You really want to assign each person different AWS access keys so that you can track each person's AWS API actions, and revoke one person's access if they leave the company or something.
I recommend reading this page for understanding all the options to configure credentials for the AWS SDK.
Assuming you are using Amazon EC2 to host your application, then you can use IAM role to grant permissions, by attaching IAM role to your EC2 instances.
Furthermore, using IAM role avoid storing sensitive credential file in your instances.
Read this document, or watch this video to implement it.
I'm trying to practice using AWS more and I'm at a point where I can generate a S3 bucket URL. Now that I have that set up I'm trying to put a document (file) into that URL. Is there any useful documentation or things I should know when I try to do that? I can't seem to find anything on the web for Java users. (maybe I'm just bad at searching idk). Thanks
There is AWS SDK for Java provided by Amazon
AWS SDK For Java
Example work with S3
AWS example of loading file to s3:
https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/examples-s3-objects.html#upload-object
and you would need a token to access you account resources from a java client more info here : https://aws.amazon.com/premiumsupport/knowledge-center/create-access-key/
I searched a lot, but it seems impossible to find a solution from start to finish to this topic. As a premise, I've already implemented Cognito sign-up, sign-in and refresh of credentials in two native apps for both iOS and Android, so I have developed a (at least) basic understanding of the authentication flow.
These mobile apps use the simplest cognito setup possible: a user pool, an identity pool with a IAM role for authenticated users and no unauthenticated usage possible. I'm not using (at least for now) Facebook, Google or Amazon login, nor other authentication methods.
Now I need to make a desktop version of those apps in Java, and it seems to me a completely different beast. What I would like to do is this:
Open the login window in my Java desktop application;
Insert username and password in their fields and press a login button;
Getting some credentials and start using the application connecting to other AWS services, specifically I need to use S3, Lambda and DynamoDB.
The way to achieve this is on paper reasonably simple:
Get a token from a Cognito user pool;
Give this token to a Cognito identity pool in exchange for some credentials;
Use this credentials to access other AWS services.
After reading a lot of documentation, downloading a lot of different projects examples and a lot of despair, I've eventually found the way to implement this in the mobile apps. For example, in Android the authentication flow works like this:
Instantiate a CognitoUserPool, using a UserPoolID, an AppClientID, a PoolRegion and (optionally) a ClientSecret;
Instantiate a credential provider, using an IdentityPoolID and a PoolRegion;
In the app UI, insert a username and password, and press the Login button;
Retrieve a CognitoUser using that username from the UserPool instantiated earlier;
Get a CognitoUserSession for that CognitoUser, using an AuthenticationHandler with various callbacks to pass the password when needed;
Add that CognitoUserSession to the credentials provider instantiated earlier, in form of a TokenKey + the JWT token extracted from the session.
At this point, whenever I need to access S3, Lambda or DynamoDB, I simply pass this credentials provider as a parameter for their clients constructors.
To implement the same functionality with the Java SDK seems to me much more difficult.
I managed to implement users Sign-Up fairly easily. However with users Sign-In I don't know where to start at all.
Every example does this in a different way. On top of that, every example uses particular use cases such as developer authenticated Sign-Ins, or custom urls to connect to some owned backend. Why is so difficult to find an example for a basic use case like the one I need? I'm starting to think my basic use case is not basic at all, but rather atypical. Why would login with a username and a password against the default users/credentials service for AWS be atypical, however, I don't really know.
The best I've done so far is copying the relevant classes from this example project (from which I've also taken the Sign-Up part, that works pretty well) and getting to print the IdToken, AccessToken and RefreshToken in the console. They are printed correctly and are not null.
What I cant't really understand is how to get the credentials and add them to a credentials provider in order to instantiate the clients to access other AWS services. The only way I see in the project to do that is to call the method
Credentials getCredentials(String accessCode)
which I suppose it should accept the access code retrieved with the InitAuth method (that starts an OAuth2.0 authentication flow, please correct me if I am wrong). The problem is that I can't find a way to retrieve that code. I can't find an online example of an access code to see how it looks. I tried to put one of the tokens and the web request responds
{"error":"invalid_grant"}
which suggests its not a valid code, but at least the web request is valid.
To make it more clear, what I can do is this:
String username; //retrieved from UI
String password; //retrieved from UI
//I copied AuthenticationHelper as is from the project
AuthenticationHelper helper = new AuthenticationHelper(POOL_ID, CLIENT_APP_ID, CLIENT_SECRET);
//I then retrieve the tokens with SRP authentication
AuthenticationResultType result = helper.performSRPAuthentication(username, password);
//Now I can successfully print the tokens, for example:
System.out.println(result.getAccessToken());
How can I retrieve the credentials from here? Where I should put the identity pool id? In Android I simply add the JWT token to an HashMap and use it like
credentialsProvider.setLogins(loginsMap).
Furthermore, this project contain classes with hundreds of lines of code, BigInteger variables, hardcoded strings of many lines of random characters (some sort of key or token I suppose) and other black magic like that (especially in the AuthenticationHelper class). Another thing I don't like about this solution is that it retrieves credentials through manually written web requests (with another separated class created ad hoc to make the request). Really isn't there in the Java SDK some handy method that wraps all those things in a bunch of elegant lines of code? Why call it an SDK than? The iOS and Android SDKs handle all that on their own in such a simpler way. Is this due to the fact that they expect a developer of desktop app to be way more able/expert, in contrast to the average guy that some day, getting up from bed, decides to make an iOS/Android app [alludes to himself]? This would explain their effort to make the mobile SDKs so developer-friendly in comparison.
Onestly I find really hard to believe that I have to do that, reading who knows what on a doc page who knows where, to Sign-In a user, which makes me think that I'm really missing something. I literally read every stack exchange question and documentation I was able to find. The fact is that there is almost always an AWS documentation page for what I need, but to actually find it is not so simple sometimes, at least for Cognito documentation.
I read that I can put a file with the needed credentials in the PC filesystem and the Java SDK will use those credentials to access all the resources, however from my understanding this method is reserved to Java applications running on a server as a backend (servlets), where the end user can't access them through his browser. My application is a desktop app for end-users, so I can't even consider to leave AWS credentials on the user PC (please correct me if I'm wrong, I would really love to make something so simple).
What really scares me is that Sign-In with a user pool and identity pool might not be possible at all. I know that Cognito related stuff was added to the Java SDK much later it was available for iOS, Android and JavaScript. But if they added it, I suppose It should support an authentication flow at least similar to those of the mobile counterparts.
What worsen the problem even more is that I initially made all the functionalities of my application to work offline. I thought that I would have eventually integrated AWS in the app. In this way the application is a bit more modular, and the AWS related stuff in concentrated in a package, detatched from the rest of the application logic and UI. In my sea of ignorance this seems a good practice to me, but now, if I cannot manage to resolve this problem, I've thrown months of work in the trash, only to realize that I have to build a web app beacuse JavaScript is much more supported.
Even on MobileHub the only options to create a ClientApp on Cognito is for iOS, Android, JavaScript and React-something.
When documentation or examples are provided for other languages/SDKs, Java is often omitted and the most frequent I see among the options is .NET.
To make my frustration even bigger, everytime I search something on a search engine, the fact that the word "Java" is contained in the word "JavaScript" obfuscates the few results that could be useful, because all the JavaScript SDK related stuff is generally higher ranked in the search engines than Java's (this could explain in part why .NET related stuff seems more easy to find, at least on StackOverflow or others Q&A sites).
To conclude, all of this created some questions in my head:
Why so few people seem to need this authentication method (with username and password)? It seems to me a pretty common and reasonable use case for a desktop application. I know that web apps growth is through the roof, but given that Java is one of the most used languages today, how is it possible that nobody needs to do a simple login from a desktop application? Which leads to the next question:
Is there something inherently bad/wrong/risky/stupid in using the Java SDK for a desktop application? Is it intended only for usage on a server as backend or for a web app? What should be the solution than, to make a desktop application that connects to AWS services? It is wrong to do an AWS connected desktop app at all? Should a web app the only option to consider? Why? I opted for Java to implement an application that would run on Widows, macOS and Linux. I also chose Java because i thought it would be mostly similar to the Android SDK in its usage, given its code should be indipendent from the platform UI, making simple to reuse code. I was wrong.
If there's nothing wrong in using the Java SDK like this, could some
good soul please help me find an example that goes from putting a
username and a password in two fields, and instantiate a client to
access other AWS services (such as an S3 client) in a Java desktop
application?
Tell me everything you need to know and I'll edit the question.
Please someone help me, I'm loosing my mind.
Probably too late for the OP but here is the process I used to get credentials from Cognito after obtaining the JWT identity token. Once the JWT is obtained through SRP Authentication I got the Identity Id using the Federated Pool Id and passing a login map of the Cognito Idp Url and the JWT. The url is completed with your aws region and your Cognito User Pool Id.
//create a Cognito provider with anonymous creds
AnonymousAWSCredentials awsCreds = new AnonymousAWSCredentials();
AmazonCognitoIdentity provider = AmazonCognitoIdentityClientBuilder
.standard()
.withCredentials(new AWSStaticCredentialsProvider(awsCreds))
.withRegion(REGION)
.build();
//get the identity id using the login map
String idpUrl = String.format("cognito-idp.%s.amazonaws.com/%s", REGION, cognitoUserPoolId);
GetIdRequest idrequest = new GetIdRequest();
idrequest.setIdentityPoolId(FED_POOL_ID);
idrequest.addLoginsEntry(idpUrl, jwt);
//use the provider to make the id request
GetIdResult idResult = provider.getId(idrequest);
return idResult.getIdentityId();
If you're using a different login provider then that url needs to change, but this should get the Identity Id. Next its a similar request to get the IAM credentials by passing the Identity Id and that same login map.
//create a Cognito provider with anonymous creds
AnonymousAWSCredentials awsCreds = new AnonymousAWSCredentials();
AmazonCognitoIdentity provider = AmazonCognitoIdentityClientBuilder
.standard()
.withCredentials(new AWSStaticCredentialsProvider(awsCreds))
.withRegion(REGION)
.build();
//request authenticated credentials using the identity id and login map for authentication
String idpUrl = String.format("cognito-idp.%s.amazonaws.com/%s", REGION, cognitoUserPoolId);
GetCredentialsForIdentityRequest request = new GetCredentialsForIdentityRequest();
request.setIdentityId(identityId);
request.addLoginsEntry(idpUrl, jwt);
//use Cognito provider to perform credentials request
GetCredentialsForIdentityResult result = provider.getCredentialsForIdentity(request);
return result.getCredentials();
This took me a full week to figure out. AWS Java documentation is pretty terrible in my opinion. Hopefully this helps someone out.
I have a website where users can upload resources (e.g. pdf-files) to their account. I am using AWS S3 to host all the uploaded files, and I am using the AWS Java SDK 1.8.9.1 for communications between my website and S3.
Now, I want to allow users to be able to download and view the files that they have uploaded, but I only want this to be possible through my website. That is, on my web site, users should have a download link for each of their files that they can click, after which the download starts. However, if they copy the URL of the download link and send it to their friend, that friend should not be able to download the file.
I know that it is possible to restrict access to S3 buckets to specified referring URLs. However, I have also been told that this can easily be forged and is not the way to go. I am thinking that there might exist a solution with signed requests.
How can I achieve this?
You could modify your application so that the download links are proxied through it. i.e. The application should do the reverse of the upload process.
So, you can provide a link to your java application, which will then go to S3 and retrieve the file and return it to the user. This way, if someone shares a link, you can protect the url and require users to login before they can download the file.
Edited 7th June,14
My Android app needs to have a feature where clients can upload their files. I want AWS S3 as my storage. Moreover i dont want to use SECRET_KEY and ACCESS_KEY_ID on client side. What is the the best way to do this. Can someone provide the working code too ?
I read that i can request to AWS for a signed URL and then make client directly upload to that URL. How to achieve this ?
Maybe you can call AWS APIs in step 4.
And please check the AWS STS(Security Token Service) document, "Ways to Get Temporary Security Credentials" section.