I have been trying to create Security Group using AWS SDK, but somehow it fails to authenticate it. For the specific Access Key and Secret Key, i have provided the Administrative rights, then also it fails to validate. On the other side, I tried the same credentials on AWS S3 Example, it successfully executes.
Getting following error while creating security group:
com.amazonaws.AmazonServiceException: AWS was not able to validate the provided access credentials (Service: AmazonEC2; Status Code: 401; Error Code: AuthFailure; Request ID: 1584a035-9a88-4dc7-b5e2-a8b7bde6f43c)
at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:1077)
at com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:725)
at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:460)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:295)
at com.amazonaws.services.ec2.AmazonEC2Client.invoke(AmazonEC2Client.java:9393)
at com.amazonaws.services.ec2.AmazonEC2Client.createSecurityGroup(AmazonEC2Client.java:1146)
at com.sunil.demo.ec2.SetupEC2.createSecurityGroup(SetupEC2.java:84)
at com.sunil.demo.ec2.SetupEC2.main(SetupEC2.java:25)
Here is the Java Code:
public class SetupEC2 {
AWSCredentials credentials = null;
AmazonEC2Client amazonEC2Client ;
public static void main(String[] args) {
SetupEC2 setupEC2Instance = new SetupEC2();
setupEC2Instance.init();
setupEC2Instance.createSecurityGroup();
}
public void init(){
// Intialize AWS Credentials
try {
credentials = new BasicAWSCredentials("XXXXXXXX", "XXXXXXXXX");
} catch (Exception e) {
throw new AmazonClientException(
"Cannot load the credentials from the credential profiles file. " +
"Please make sure that your credentials file is at the correct " +
"location (/home/sunil/.aws/credentials), and is in valid format.",
e);
}
// Initialize EC2 instance
try {
amazonEC2Client = new AmazonEC2Client(credentials);
amazonEC2Client.setEndpoint("ec2.ap-southeast-1.amazonaws.com");
amazonEC2Client.setRegion(Region.getRegion(Regions.AP_SOUTHEAST_1));
} catch (Exception e) {
e.printStackTrace();
}
}
public boolean createSecurityGroup(){
boolean securityGroupCreated = false;
String groupName = "sgec2securitygroup";
String sshIpRange = "0.0.0.0/0";
String sshprotocol = "tcp";
int sshFromPort = 22;
int sshToPort =22;
String httpIpRange = "0.0.0.0/0";
String httpProtocol = "tcp";
int httpFromPort = 80;
int httpToPort = 80;
String httpsIpRange = "0.0.0.0/0";
String httpsProtocol = "tcp";
int httpsFromPort = 443;
int httpsToProtocol = 443;
try {
CreateSecurityGroupRequest createSecurityGroupRequest = new CreateSecurityGroupRequest();
createSecurityGroupRequest.withGroupName(groupName).withDescription("Created from AWS SDK Security Group");
createSecurityGroupRequest.setRequestCredentials(credentials);
CreateSecurityGroupResult csgr = amazonEC2Client.createSecurityGroup(createSecurityGroupRequest);
String groupid = csgr.getGroupId();
System.out.println("Security Group Id : " + groupid);
System.out.println("Create Security Group Permission");
Collection<IpPermission> ips = new ArrayList<IpPermission>();
// Permission for SSH only to your ip
IpPermission ipssh = new IpPermission();
ipssh.withIpRanges(sshIpRange).withIpProtocol(sshprotocol).withFromPort(sshFromPort).withToPort(sshToPort);
ips.add(ipssh);
// Permission for HTTP, any one can access
IpPermission iphttp = new IpPermission();
iphttp.withIpRanges(httpIpRange).withIpProtocol(httpProtocol).withFromPort(httpFromPort).withToPort(httpToPort);
ips.add(iphttp);
//Permission for HTTPS, any one can accesss
IpPermission iphttps = new IpPermission();
iphttps.withIpRanges(httpsIpRange).withIpProtocol(httpsProtocol).withFromPort(httpsFromPort).withToPort(httpsToProtocol);
ips.add(iphttps);
System.out.println("Attach Owner to security group");
// Register this security group with owner
AuthorizeSecurityGroupIngressRequest authorizeSecurityGroupIngressRequest = new AuthorizeSecurityGroupIngressRequest();
authorizeSecurityGroupIngressRequest.withGroupName(groupName).withIpPermissions(ips);
amazonEC2Client.authorizeSecurityGroupIngress(authorizeSecurityGroupIngressRequest);
securityGroupCreated = true;
} catch (Exception e) {
// TODO: handle exception
e.printStackTrace();
securityGroupCreated = false;
}
System.out.println("securityGroupCreated: " + securityGroupCreated);
return securityGroupCreated;
}
}
Try to update your Systemtime.
When the diffrence between AWS-datetime and your datetime are too big, the credentials will not accepted.
For Debian/Ubuntu Users:
when you never set your time-zone you can do this with
sudo dpkg-reconfigure tzdata
Stop the ntp-Service, because too large time diffrences, cannot be changed by running service.
sudo /etc/init.d/ntp stop
Syncronize your time and date (-q Set the time and quit / Run only once) (-g Allow the first adjustment to be Big) (-x Slew up to 600 seconds / Adjuste also time witch large diffrences) (-n Do not fork / process will not going to background)
sudo ntpd -q -g -x -n
Restart service
sudo /etc/init.d/ntp start
check actual system-datetime
sudo date
set system-datetime to your hardware-datetime
sudo hwclock --systohc
show your hardware-datetime
sudo hwclock
you must specify the profile and the region
aws ec2 describe-instances --profile nameofyourprofile --region eu-west-1
"A client error (AuthFailure) occurred when calling the [Fill-in the blanks] operation: AWS was not able to validate the provided access credentials"
If you are confident of the validity of AWS credentials i.e. access key and secret key and corresponding profile name, your date and time being off-track is a very good culprit.
In my case, I was confident but I was wrong - I had used the wrong keys. Doesn't hurt to double check.
Let's say that you created an IAM user called "guignol". Configure "guignol" in ~/.aws/config as follows:
[profile guignol]
region = us-east-1
aws-access-key_id = AKXXXYYY...
aws-secret-key-access = ...
Install the aws cli (command level interface) if you haven't already done so. As a test, run aws ec2 describe-instances --profile guignol If you gat an error message that aws was not able to validate the credentials, run aws configure --profile guignol , enter your credentials and run the test command again.
If you put your credentials in ~/.aws/credentials then you don't need to provide a parameter to your AmazonEC2Client call. If you do this then on an EC2 instance the same code will work with Assumed STS roles.
For more info see: http://docs.aws.amazon.com/AWSSdkDocsJava/latest/DeveloperGuide/credentials.html
In my case, killing the terminal and running the command again helped
In my case I copied CDK env variables AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN for programmatic access, but it appeared that I already had an old session token in my ~/aws/.credentials which I forgot about. Needed to remove the old tokens from the file.
Related
I am setting AWS_ env variables as below for Jenkins
sudo apt-get update -y
sudo apt-get install -y python3 python-pip python-devel
sudo pip install awscli
S3_LOGIN=$(aws sts assume-role --role-arn rolename --role-session-name s3_session)
export AWS_CREDENTIAL_PROFILES_FILE=~/.aws/credentials
export AWS_ACCESS_KEY_ID=$(echo ${S3_LOGIN}| jq --raw-output '.Credentials|"\(.AccessKeyId)"')
export AWS_SECRET_ACCESS_KEY=$(echo ${S3_LOGIN} | jq --raw-output '.Credentials|"\(.SecretAccessKey)"')
export AWS_SESSION_TOKEN=$(echo ${S3_LOGIN} | jq --raw-output '.Credentials|"\(.SessionToken)"')
aws configure set default.region us-east-2
aws configure set AWS_ACCESS_KEY_ID $AWS_ACCESS_KEY_ID
aws configure set AWS_SECRET_ACCESS_KEY $AWS_SECRET_ACCESS_KEY
But when I try to get them from code the sdk can not read the env variables already set
AWSCredentials evc = new EnvironmentVariableCredentialsProvider().getCredentials();
AmazonS3Client amazonS3 = new AmazonS3Client(evc);
amazonS3.setRegion(RegionUtils.getRegion("us-east-2"));
com.amazonaws.AmazonClientException: Unable to load AWS credentials
from environment variables (AWS_ACCESS_KEY_ID (or AWS_ACCESS_KEY) and
AWS_SECRET_KEY (or AWS_SECRET_ACCESS_KEY))
The EnvironmentVariableCredentialsProvider in AWS SDK looks below,
public AWSCredentials getCredentials() {
String accessKey = System.getenv(ACCESS_KEY_ENV_VAR);
if (accessKey == null) {
accessKey = System.getenv(ALTERNATE_ACCESS_KEY_ENV_VAR);
}
String secretKey = System.getenv(SECRET_KEY_ENV_VAR);
if (secretKey == null) {
secretKey = System.getenv(ALTERNATE_SECRET_KEY_ENV_VAR);
}
accessKey = StringUtils.trim(accessKey);
secretKey = StringUtils.trim(secretKey);
String sessionToken =
StringUtils.trim(System.getenv(AWS_SESSION_TOKEN_ENV_VAR));
if (StringUtils.isNullOrEmpty(accessKey)
|| StringUtils.isNullOrEmpty(secretKey)) {
throw new AmazonClientException(
"Unable to load AWS credentials from environment variables " +
"(" + ACCESS_KEY_ENV_VAR + " (or " + ALTERNATE_ACCESS_KEY_ENV_VAR + ") and " +
SECRET_KEY_ENV_VAR + " (or " + ALTERNATE_SECRET_KEY_ENV_VAR + "))");
}
return sessionToken == null ?
new BasicAWSCredentials(accessKey, secretKey)
:
new BasicSessionCredentials(accessKey, secretKey, sessionToken);
}
EDIT: I try below approach also,
ProfileCredentialsProvider evc = new ProfileCredentialsProvider();
AmazonS3Client amazonS3 = new AmazonS3Client(evc);
amazonS3.setRegion(RegionUtils.getRegion("us-east-2"));
But even I set AWS_CREDENTIAL_PROFILES_FILE in the script because the credentials file is under ~/.aws/credentials, I still get below,
credential profiles file not found in the given path:
/root/.aws/credentials
Even though the AwsProfileFileLocationProvider code says below, i am not sure why it try to look at /root/.aws/credentials
Checks the environment variable override
* first, then checks the default location (~/.aws/credentials), and finally falls back to the
* legacy config file (~/.aws/config) that we still support loading credentials from
I am assuming you are configuring your Jenkins Job with different build steps between set credential and consume credential.
Jenkins does not share environment variable between build steps.
If you are using old-style of Jenkins job you will need to use some Plugin like envinject, or use a file to share the variables between steps. Like below (just as example).
Step 1
echo "export AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID}" > credential
echo "export AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY}" >> credential
echo "export AWS_SESSION_TOKEN=${AWS_SESSION_TOKEN}" >> credential
Step 2
source credential && ./your_command_here
But if you are suing Jenkins Pipeline, you can use env. Like below (just as example).
pipeline {
parameters {
string(name: 'AWS_ACCESS_KEY_ID', defaultValue: '')
}
stage("set credential") {
steps {
tmp_AWS_ACCESS_KEY_ID = sh (script: 'your shell script here', returnStdout: true).trim()
env.AWS_ACCESS_KEY_ID = tmp_AWS_ACCESS_KEY_ID
}
}
stage("consume credential") {
steps {
echo "${env.AWS_ACCESS_KEY_ID}"
}
}
}
You should be able to change the credentials file location using the AWS_CREDENTIAL_PROFILES_FILE environment variable
IMPORTANT
I have been blocked by hotmail services. There is a control mechanism
called spamhaus which kicked me out. I'm stuck right now.
I am trying to detect an email address is valid and if its valid then check if this email address potentially used (I know that its not certain). For example, lets assume that there is a website with domain myimaginarydomain.com. If I run code below, I guess it won't fail because domain address is valid. But nobody can take an email address with that domain.
Is there any way to find out that email address is valid? (In this case its invalid)
I don't want to send confirmation email
Sending ping may be useful?
public class Application {
private static EmailValidator validator = EmailValidator.getInstance();
public static void main(String[] args) {
while (true) {
Scanner scn = new Scanner(System.in);
String email = scn.nextLine();
boolean isValid = validateEmail(email);
System.out.println("Syntax is : " + isValid);
if (isValid) {
String domain = email.split("#")[1];
try {
int test = doLookup(domain);
System.out.println(domain + " has " + test + " mail servers");
} catch (NamingException e) {
System.out.println(domain + " has 0 mail servers");
}
}
}
}
private static boolean validateEmail(String email) {
return validator.isValid(email);
}
static int doLookup(String hostName) throws NamingException {
Hashtable env = new Hashtable();
env.put("java.naming.factory.initial",
"com.sun.jndi.dns.DnsContextFactory");
DirContext ictx = new InitialDirContext(env);
Attributes attrs =
ictx.getAttributes(hostName, new String[]{"MX"});
Attribute attr = attrs.get("MX");
if (attr == null) return (0);
return (attr.size());
}
}
There is no failsafe way to do this in all cases, but, assuming the server uses SMTP then https://www.labnol.org/software/verify-email-address/18220/ gives quite a good tutorial on one method that may work.
The method used in the tutorial relies on OS tools, so you will need to ensure they exist before using them. a ProcessBuilder may help. Alternatively, you can open a socket directly in code and avoid using OS-dependent tools.
Essentially, you find out what the mail servers are (using nslookup), then telnet to one of the mail servers and start writing an email:
3a: Connect to the mail server:
telnet gmail-smtp-in.l.google.com 25
3b: Say hello to the other server
HELO
3c: Identify yourself with some fictitious email address
mail from:<labnol#labnol.org>
3d: Type the recipient’s email address that you are trying to verify:
rcpt to:<billgates#gmail.com>
The server response for rcpt to command will give you an idea whether an email address is valid or not. You’ll get an “OK” if the address exists else a 550 error
There really is no sensible way except trying to send a notification with a token to the address and ask the other party to confirm it, usually by visiting a web-page:
the recipients MX may be unavailable at the moment but come back online later, so you cannot rely on a lookup in real time;
just because the MX accepts the email doesn't mean that the address is valid, the message could bounce later down the pipe (think UUCP);
if this is some kind of registration service, you need to provide some confirmation step anyway as otherwise it'd become too easy to subscribe random strangers on the internet that do not want your service.
We try to make Windows authentication using spring security.
When we saw that we cannot authenticate our domain user with our keytab file created for our local pc, we checked our service user and see that it's password is valid. Then we checked whether we can reach from local to AD-domain. No request reached from our local as we controlled with network monitoring tool on AD-domain server machine. We also checked that outgoing traffic from our client with the command below;
netstat -oan 1 | find /I "[IP_ADDRESS_OF_AD_DOMAIN]"
We could reach to that IP from our local, tested with telnet.
Our application.properties is like below;
app.ad-domain= example.com
app.ad-server= ldap://adds.example.com.tr/
app.service-principal= HTTP/local_pc.example.com.tr#EXAMPLE.COM.TR
app.keytab-location= local_pc.keytab
app.ldap-search-base= OU=All Users,DC=example,DC=com
app.ldap-search-filter= "(| (userPrincipalName={0}) (sAMAccountName={0}))"
As a result we cannot get srcName of GSSContext. This gssName variable equals to null. Related SunJaasKerberosTicketValidator code block is as below;
#Override
public KerberosTicketValidation run() throws Exception {
byte[] responseToken = new byte[0];
GSSName gssName = null;
GSSContext context = GSSManager.getInstance().createContext((GSSCredential) null);
boolean first = true;
while (!context.isEstablished()) {
if (first) {
kerberosTicket = tweakJdkRegression(kerberosTicket);
}
responseToken = context.acceptSecContext(kerberosTicket, 0, kerberosTicket.length);
gssName = context.getSrcName();
if (gssName == null) {
throw new BadCredentialsException("GSSContext name of the context initiator is null");
}
first = false;
}
if (!holdOnToGSSContext) {
context.dispose();
}
return new KerberosTicketValidation(gssName.toString(), servicePrincipal, responseToken, context);
}
As we searched this GSSContext with null SrcName error, in general suggested solutions are related to keytab file . But in our problem, we cannot even reach AD server as we mentioned in the beginning.
related link: GSSContext with null SrcName
Is there any other suggestion?
Thanks...
I always had this issue with Google Sign In. I have an Android app that the user connects uses to authenticate with Google and then send the idToken to my server. The server uses the library provided by Google (GoogleIdTokenVerifier) to verify the token.
GoogleIdTokenVerifier verifier = new GoogleIdTokenVerifier.Builder(transport, jsonFactory)
.setAudience(audience)
.setIssuer("https://accounts.google.com")
.build();
GoogleIdToken idToken = null;
try {
idToken = verifier.verify(idTokenString);
} catch (Exception e) {
e.printStackTrace();
}
if (idToken != null) {
GoogleIdToken.Payload payload = idToken.getPayload();
String userId = payload.getSubject();
System.out.println("User ID: " + userId);
String email = payload.getEmail();
System.out.println("Emaail:" + email);
return userId;
} else {
System.out.println("Invalid ID token.");
return null;
}
This worked for a while, then suddenly the validation started to always fail. Nothing has changed!
Any ideas?
Check your server time, I had the same issue when I migrated to a new server. I solved the issue by setting the time zone using NTP.
If you are running over your localhost, check if your computers hour its correct, on the same timezone as the server.
I spent a complete day with this bug, getting null from the idToken sent to the client.
I'm working with Amazon SimpleDB and attempting the creation of a DB using the following tutorial . Basically it throws an error i.e. Error occured: java.lang.String cannot be cast to org.apache.http.HttpHost. The full stacktrace is as below:
Error occured: java.lang.String cannot be cast to org.apache.http.HttpHost
java.lang.ClassCastException: java.lang.String cannot be cast to org.apache.http.HttpHost
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:416)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
at com.xerox.amazonws.common.AWSQueryConnection.makeRequest(AWSQueryConnection.java:474)
at com.xerox.amazonws.sdb.SimpleDB.makeRequestInt(SimpleDB.java:231)
at com.xerox.amazonws.sdb.SimpleDB.createDomain(SimpleDB.java:155)
at com.amazonsimpledb.SDBexample1.main(SDBexample1.java:19)
My code is as below (note i have substituted the AWS access id and secret key with the actual values):
public static void main(String[] args) {
String awsAccessId = "My aws access id";
String awsSecretKey = "my aws secret key";
SimpleDB sdb = new SimpleDB(awsAccessId, awsSecretKey, true);
try {
Domain domain = sdb.createDomain("cars");
System.out.println(domain);
} catch (com.xerox.amazonws.sdb.SDBException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
Any ideas as to why the above mentioned error is occurs.
I appreciate any assistance.
It seems you are using the Typica client library, which is pretty much unmaintained since mid 2011, see e.g. the rare commmits and the steady growing unresolved issues, where the latest one appears to be exactly yours in fact, see ClassCastException using Apache HttpClient 4.2:
According to the reporter, things appear to be functional once we downgrade back to Apache HttpClient 4.1, so that might be a temporary workaround eventually.
Either way I highly recommend to switch to the official AWS SDK for Java (or one of the other language SDKs), which isn't only supported and maintained on a regular fashion, but also closely tracks all AWS API changes (admittedly this isn't that critical for Amazon SimpleDB, which is basically frozen technology wise, but you'll have a much easier time using the plethora of AWS Products & Services later on).
In addition you could benefit from the AWS Toolkit for Eclipse in case you are using that IDE.
The SDK includes a couple of samples (also available via the Eclipse Toolkit wizard), amongst those one for SimpleDB - here's a condensed code excerpt regarding your example:
BasicAWSCredentials basicAWSCredentials = new BasicAWSCredentials(
awsAccessId, awsSecretKey);
AmazonSimpleDB sdb = new AmazonSimpleDBClient(basicAWSCredentials);
Region usWest2 = Region.getRegion(Regions.US_WEST_2);
sdb.setRegion(usWest2);
try {
// Create a domain
String myDomain = "MyStore";
System.out.println("Creating domain called " + myDomain + ".\n");
sdb.createDomain(new CreateDomainRequest(myDomain));
// ...
// Delete a domain
System.out.println("Deleting " + myDomain + " domain.\n");
sdb.deleteDomain(new DeleteDomainRequest(myDomain));
} catch (AmazonServiceException ase) {
// ...
} catch (AmazonClientException ace) {
// ...
}
Please try to create instance of SimpleDB with server and port and let me know if it works.
public SimpleDB objSimpleDB = null;
private String awsAccessKeyId = "access key";
private String awsSecretAccessKey = "secret key";
private boolean isSecure= true;
private String server = "sdb.amazonaws.com";
private int port=443;
try{
SimpleDB objSimpleDB = new SimpleDB(awsAccessKeyId, awsSecretAccessKey, isSecure, server, port);
Domain domain = objSimpleDB .createDomain("cars");
} catch (com.xerox.amazonws.sdb.SDBException e) {
//handle error
}