My question is somewhat similar to this SO but I could not find the answer there.
I using the following code snippet to create instance.
Code:
public void test(String accessId, String accessKey){
credentials = new BasicAWSCredentials(accessId, accessKey);
amazonEC2Client = new AmazonEC2Client(credentials);
amazonEC2Client.setEndpoint("ec2.ap-northeast-1.amazonaws.com");
RunInstancesRequest runInstancesRequest = new RunInstancesRequest();
runInstancesRequest.withImageId(imageId)
.withInstanceType("t2.micro")
.withMinCount(2)
.withMaxCount(2);
}
When I ran the above snippet I got the following error.
We currently do not have sufficient t2.micro capacity in zones with support for'gp2'volumes.
Our system will be working on provisioning additional capacity.
(Service: AmazonEC2; Status Code: 500; Error Code:
InsufficientInstanceCapacity; Request ID: c1996284-c208-446a-9f4c-301d8900e503)
After googling I found that AWS does not have that amount of t2.micro instance at that time and it recommended to create instance in different availability zone.
But in the code I have not provided any availability zone but the instances are being created in the ap-northeast-1a availability zone and throws above error and terminates.
When I created in through AWS web console it automatically created an instance in ap-northeast-1c without throwing any error.
Is there any way to create an instance in any available zone of that particular region where ever that instance is available programmatically ?.
My understanding is there is no or less t2.micro instance available in ap-northeast-1 region when I called the API.
Is there any API to check is Instance are available in a particular region ?.
AWS do not provide an API to check what available instances there are in a particular region.
You could try one AZ, if it fails try the next AZ, and so on.
Alternatively you could use an AutoScaling group that is allowed to launch instances in different AZs. It will automatically do the above for you.
Related
The logs from cloud run spit out some good json with resource.labels.revision_name = my_name-00046-kip.
The json path labels.instanceId is more like this though
00bf4bf02d71261c0c1f55a601331b336a5d90d365cca1b28330dcf3e456fb7c07d5b72f1d3c9a971e391b5edc3512aea8559d172b24e639
per this document I was able to get revision_name
https://cloud.google.com/run/docs/reference/container-contract#env-vars
but I can't get the instance id and metrics must be reported per instance or two instances reporting in the same minute will be rejected. how do I get instance id (preferably through DockerFile and if not through api call). If cloud run boots up 10 instances under one revision name, I have to make sure to uniquely report metrics to Generic Task resource where I plan on filling in job_id with the instance id.
thanks,
Dean
Please try using the metadata server to get the instance ID using the url:
http://metadata.google.internal/computeMetadata/v1/instance/id
Note that "Metadata-Flavor: Google" header is also required.
If you're using Java (as indicated by the tags), the easiest way to get the instance ID from the "internal metadata server" programmatically is probably to include the dependency com.google.cloud:google-cloud-core:1.93.5 (or newer) through Gradle/Maven and then call the following method:
import com.google.cloud.MetadataConfig;
String instanceId = MetadataConfig.getInstanceId();
The entries in the logging in Stackdriver is as follows
labels: {
instanceId: "00bf4bf02d4b374e91dda64bc4c4241a218302c4bcc73a01ecf85e582127e8c8076fcbe18b3cc934f5ed33e5dc1348c58cfd40cbecc0c9ae2a0b6d2356"
}
labels: {
configuration_name: "cloudrunservice"
location: "us-central1"
project_id: "xxxx-xxxx-000"
revision_name: "cloudrunservice-00002-leq"
service_name: "cloudrunservice"
}
type: "cloud_run_revision"
As you mentioned, each one has the instance Id, Revision name, and Service name. In this way, you do not have to worry about rejected entries in the logging by the same instance / time.
I could no see something related with the instances ID in the UI, managing Revisions. Handling this JSON from logging you could get the InsanceID.
On Windows 10 I'm using the AWS Java SDK v2 (software.amazon.awssdk:route53:2.8.3) and I'm trying to merely connect and list all my Route 53 hosted zones. I have us-west-1 specified in my user configuration (in my .aws/config file) as the default region.
I create a Route53Client using the following:
Route53Client route53Client = Route53Client.builder().build();
Note that I don't indicate a region, because in the online documentation it says:
When you submit requests using the AWS CLI or SDKs, either leave the Region and endpoint unspecified, or specify us-east-1 as the Region.
I then try to list hosted zones using something like this:
Set<HostedZone> hostedZones = client.listHostedZonesPaginator().stream()
.flatMap(response -> response.hostedZones().stream())
.collect(Collectors.toSet());
In the logs I see a debug message like this:
[DEBUG] Unable to load region from software.amazon.awssdk.regions.providers.SystemSettingsRegionProvider#...:Unable to load region from system settings. Region must be specified either via environment variable (AWS_REGION) or system property (aws.region).
Then it throws a java.net.UnknownHostException for route53.us-west-1.amazonaws.com.
Granted I am on a spotty Internet connection right now. Is that the correct endpoint? If it is, the why isn't that endpoint listed at https://docs.aws.amazon.com/general/latest/gr/rande.html ? If it isn't, why is it trying to connect to a us-west1 endpoint, if I'm following the online documentation (as I quoted above), which indicates that a region need not be indicated? Or is the problem simply my Internet connection and spotty DNS lookup at the moment?
The AWS SDK development team decided to require Route53 requests to explicitly indicate the Region.AWS_GLOBAL or requests would not work, as someone noted in Issue #456 for the SDK:
To access Route53 you would currently need to specify the AWS_GLOBAL region. This was done to prevent customers from using global services and not realizing that for this service your calls are likely not staying in region and could potentially be spanning the globe.
Unfortunately Amazon didn't bother documenting this in the SDK (that I could find), and didn't provide a helpful error message, instead assuming developers would somehow guess the problem when the SDK tried to access an endpoint that did not exist even though the SDK was being used according to the API and according to the online documentation.
In short the Route53 client must be created like this:
route53Client = Route53Client.builder().region(Region.AWS_GLOBAL).build();
Here is the AWS Route 53 V2 Code example that lists hosted zones:
package com.example.route;
//snippet-start:[route.java2.list_zones.import]
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.route53.Route53Client;
import software.amazon.awssdk.services.route53.model.HostedZone;
import software.amazon.awssdk.services.route53.model.Route53Exception;
import software.amazon.awssdk.services.route53.model.ListHostedZonesResponse;
import java.util.List;
//snippet-end:[route.java2.list_zones.import]
public class ListHostedZones {
public static void main(String[] args) {
Region region = Region.AWS_GLOBAL;
Route53Client route53Client = Route53Client.builder()
.region(region)
.build();
listZones(route53Client);
}
//snippet-start:[route.java2.list_zones.main]
public static void listZones(Route53Client route53Client) {
try {
ListHostedZonesResponse zonesResponse = route53Client.listHostedZones();
List<HostedZone> checklist = zonesResponse.hostedZones();
for (HostedZone check: checklist) {
System.out.println("The name is : "+check.name());
}
} catch (Route53Exception e) {
System.err.println(e.getMessage());
System.exit(1);
}
}
//snippet-end:[route.java2.list_zones.main]
}
I am trying to retrieve all the instances running in my AWS account (say instance id, etc). I use the following code. I am not able to print the instance ids. When I debug, I am just getting null values. But I have three instances running on AWS. Can someone point out what I am doing wrong here?
DescribeInstancesResult result = ec2.describeInstances();
List<Reservation> reservations = result.getReservations();
for (Reservation reservation : reservations) {
List<Instance> instances = reservation.getInstances();
for (Instance instance : instances) {
System.out.println(instance.getInstanceId());
}
}
The most common cause for issues like this is a missing region specification when initializing the client, see section To create and initialize an Amazon EC2 client within Create an Amazon EC2 Client for details:
Specifically, step 2 only creates an EC2 client without specifying the region explicitly:
2) Use the AWSCredentials object to create a new AmazonEC2Client instance, as follows:
amazonEC2Client = new AmazonEC2Client(credentials);
This yields a client talking to us-east-1 - surprisingly, the AWS SDKs and the AWS Management Console use different defaults even as outlined in step 3, which also shows how to specify a different endpoint:
3) By default, the service endpoint is ec2.us-east-1.amazonaws.com. To specify a different endpoint, use the setEndpoint method. For example:
amazonEC2Client.setEndpoint("ec2.us-west-2.amazonaws.com");
The AWS SDK for Java uses US East (N. Virginia) as the default region
if you do not specify a region in your code. However, the AWS
Management Console uses US West (Oregon) as its default. Therefore,
when using the AWS Management Console in conjunction with your
development, be sure to specify the same region in both your code and
the console. [emphasis mine]
The differing defaults are easy to trip over, and the respective default in the AWS Management Console has in fact changed over time - as so often in software development, I recommend to always be explicit about this in your code to avoid such subtle error sources.
So I have a java program running within an Amazon EC2 instance. Is there a way to programatically get its own tags? I have tried instantiating a new AmazonEC2Client to us the describeTags() function but it only gives me null. Any help would be appreciated thank you.
Edit: To make things clearer, the instances are going to be unmanned worker machines spun up to solely do some computations
This should help you get started...
String instanceId = EC2MetadataUtils.getInstanceId();
AmazonEC2 client = AmazonEC2ClientBuilder.standard()
.withCredentials(new DefaultAWSCredentialsProviderChain())
.build();
DescribeTagsRequest req = new DescribeTagsRequest()
.withFilters(new Filter("resource-id", Collections.singletonList(instanceId)));
DescribeTagsResult describeTagsResult = client.describeTags(req);
List<TagDescription> tags = describeTagsResult.getTags()
You should be able to get the current instance id by sending a request to: http://169.254.169.254/latest/meta-data/instance-id. This only works within ec2. With this you can access quite a bit of information about the instance. However, tags do not appear to be included.
You should be able to take the instance id along with the correct authentication to get the instance tags. If you are going to run this on an instance, you may want to provide an IAM user with limited access instead of a user which has access to everything in case the instance is compromised.
While using user-data may be the simplest solution, the OP was asking specifically about the tagging, and unfortunately amazon hasn't made this as easy as it could be. However, It can be done. You want to use a combination of 2 amazon services.
First you need to retrieve the Instance ID. This can be achieved by hitting the URL from within your instance:
http://169.254.169.254/latest/meta-data/instance-id
Once you have the resource ID, you'll want to use Amazon's EC2 API to access the tags. Since you said you're using Java, I would suggest the Using the AWS SDK amazon makes available. Within this SDK you'll find a method called describeTags (documentation). You can use a Resource ID as one of the filters to get the specific tags to your instance. Supported filters are
tag key
resource-id
resource-type
I suggest doing this retrieval at boot using something like cloud-init and caching the tags on your server for use later if necessary.
How do i start and stop an amazon EC2 instance programmatically using aws-sdk in java?
Any helps are greatly appreciated as I have spent a day while trying to sort this out.
I've recently implemented this functionality within the Bamboo AWS Plugin; it's Open Source and the code is available on Bitbucket, you can find a complete example how to start/stop/reboot an instance within EC2Task.java (should be a separate class actually, alas ...).
Fortunately this is not complicated at all, for example, an instance can be started like so:
private String startInstance(final String instanceId, AmazonEC2 ec2, final BuildLogger buildLogger)
throws AmazonServiceException, AmazonClientException, InterruptedException
{
StartInstancesRequest startRequest = new StartInstancesRequest().withInstanceIds(instanceId);
StartInstancesResult startResult = ec2.startInstances(startRequest);
List<InstanceStateChange> stateChangeList = startResult.getStartingInstances();
buildLogger.addBuildLogEntry("Starting instance '" + instanceId + "':");
// Wait for the instance to be started
return waitForTransitionCompletion(stateChangeList, "running", ec2, instanceId, buildLogger); }
BuildLogger is Bamboo specific and waitForTransitionCompletion() is an implementation specific helper to report back on the process/result. The AmazonEC2 ec2 parameter passes the reference to an AmazonEC2Client object by means of the AmazonEC2 interface, which defines all relevant methods (amongst many others), specifically:
StartInstances()
StopInstances()
RebootInstances()
If you have already used AWS API, it's simple call on AmazonEC2Client object. Use the following methods
Start Instance
Stop Instance
Also, you might be knowing the start/stop mechanism works only for the images with root device backed by EBS.