Why might describing Amazon EC2 instances yield no result? - java

I am trying to retrieve all the instances running in my AWS account (say instance id, etc). I use the following code. I am not able to print the instance ids. When I debug, I am just getting null values. But I have three instances running on AWS. Can someone point out what I am doing wrong here?
DescribeInstancesResult result = ec2.describeInstances();
List<Reservation> reservations = result.getReservations();
for (Reservation reservation : reservations) {
List<Instance> instances = reservation.getInstances();
for (Instance instance : instances) {
System.out.println(instance.getInstanceId());
}
}

The most common cause for issues like this is a missing region specification when initializing the client, see section To create and initialize an Amazon EC2 client within Create an Amazon EC2 Client for details:
Specifically, step 2 only creates an EC2 client without specifying the region explicitly:
2) Use the AWSCredentials object to create a new AmazonEC2Client instance, as follows:
amazonEC2Client = new AmazonEC2Client(credentials);
This yields a client talking to us-east-1 - surprisingly, the AWS SDKs and the AWS Management Console use different defaults even as outlined in step 3, which also shows how to specify a different endpoint:
3) By default, the service endpoint is ec2.us-east-1.amazonaws.com. To specify a different endpoint, use the setEndpoint method. For example:
amazonEC2Client.setEndpoint("ec2.us-west-2.amazonaws.com");
The AWS SDK for Java uses US East (N. Virginia) as the default region
if you do not specify a region in your code. However, the AWS
Management Console uses US West (Oregon) as its default. Therefore,
when using the AWS Management Console in conjunction with your
development, be sure to specify the same region in both your code and
the console. [emphasis mine]
The differing defaults are easy to trip over, and the respective default in the AWS Management Console has in fact changed over time - as so often in software development, I recommend to always be explicit about this in your code to avoid such subtle error sources.

Related

How to get instanceid from cloud_run?

The logs from cloud run spit out some good json with resource.labels.revision_name = my_name-00046-kip.
The json path labels.instanceId is more like this though
00bf4bf02d71261c0c1f55a601331b336a5d90d365cca1b28330dcf3e456fb7c07d5b72f1d3c9a971e391b5edc3512aea8559d172b24e639
per this document I was able to get revision_name
https://cloud.google.com/run/docs/reference/container-contract#env-vars
but I can't get the instance id and metrics must be reported per instance or two instances reporting in the same minute will be rejected. how do I get instance id (preferably through DockerFile and if not through api call). If cloud run boots up 10 instances under one revision name, I have to make sure to uniquely report metrics to Generic Task resource where I plan on filling in job_id with the instance id.
thanks,
Dean
Please try using the metadata server to get the instance ID using the url:
http://metadata.google.internal/computeMetadata/v1/instance/id
Note that "Metadata-Flavor: Google" header is also required.
If you're using Java (as indicated by the tags), the easiest way to get the instance ID from the "internal metadata server" programmatically is probably to include the dependency com.google.cloud:google-cloud-core:1.93.5 (or newer) through Gradle/Maven and then call the following method:
import com.google.cloud.MetadataConfig;
String instanceId = MetadataConfig.getInstanceId();
The entries in the logging in Stackdriver is as follows
labels: {
instanceId: "00bf4bf02d4b374e91dda64bc4c4241a218302c4bcc73a01ecf85e582127e8c8076fcbe18b3cc934f5ed33e5dc1348c58cfd40cbecc0c9ae2a0b6d2356"
}
labels: {
configuration_name: "cloudrunservice"
location: "us-central1"
project_id: "xxxx-xxxx-000"
revision_name: "cloudrunservice-00002-leq"
service_name: "cloudrunservice"
}
type: "cloud_run_revision"
As you mentioned, each one has the instance Id, Revision name, and Service name. In this way, you do not have to worry about rejected entries in the logging by the same instance / time.
I could no see something related with the instances ID in the UI, managing Revisions. Handling this JSON from logging you could get the InsanceID.

Availability zone selection in AWS

My question is somewhat similar to this SO but I could not find the answer there.
I using the following code snippet to create instance.
Code:
public void test(String accessId, String accessKey){
credentials = new BasicAWSCredentials(accessId, accessKey);
amazonEC2Client = new AmazonEC2Client(credentials);
amazonEC2Client.setEndpoint("ec2.ap-northeast-1.amazonaws.com");
RunInstancesRequest runInstancesRequest = new RunInstancesRequest();
runInstancesRequest.withImageId(imageId)
.withInstanceType("t2.micro")
.withMinCount(2)
.withMaxCount(2);
}
When I ran the above snippet I got the following error.
We currently do not have sufficient t2.micro capacity in zones with support for'gp2'volumes.
Our system will be working on provisioning additional capacity.
(Service: AmazonEC2; Status Code: 500; Error Code:
InsufficientInstanceCapacity; Request ID: c1996284-c208-446a-9f4c-301d8900e503)
After googling I found that AWS does not have that amount of t2.micro instance at that time and it recommended to create instance in different availability zone.
But in the code I have not provided any availability zone but the instances are being created in the ap-northeast-1a availability zone and throws above error and terminates.
When I created in through AWS web console it automatically created an instance in ap-northeast-1c without throwing any error.
Is there any way to create an instance in any available zone of that particular region where ever that instance is available programmatically ?.
My understanding is there is no or less t2.micro instance available in ap-northeast-1 region when I called the API.
Is there any API to check is Instance are available in a particular region ?.
AWS do not provide an API to check what available instances there are in a particular region.
You could try one AZ, if it fails try the next AZ, and so on.
Alternatively you could use an AutoScaling group that is allowed to launch instances in different AZs. It will automatically do the above for you.

Launching Instance: VPC security groups may not be used for a non-VPC launch

I'm attempting to create an instance in another region, but I get this error:
AWS Error Code: InvalidParameterCombination, AWS Error Message: VPC security groups may not be used for a non-VPC launch
Here is the code I'm executing.
RunInstancesRequest instancereq = new RunInstancesRequest();
instancereq.setInstanceType("m3.medium");
instancereq.setImageId("ami-37b1b45e");
instancereq.setMinCount(1);
instancereq.setMaxCount(1);
ArrayList<String> secgroup = new ArrayList<String>();
instancereq.setKeyName("testkey");
secgroup.add("testdefault");
instancereq.setSecurityGroups(secgroup);
instancereq.setPlacement(getAzPlacement());
RunInstancesResult instanceresult = ec2.runInstances(instancereq);
I've also tried, instead of using the name "testdefault", using the actual groupid (sg-########), but I'll get an error saying that security group doesn't exist (which is wrong, it does). Which, based on the API doc, if using a non-default VPC, you should pass the actual groupid but I'll get an error like this:
InvalidGroup.NotFound, AWS Error Message: The security group 'sg-########' does not exist
If I use "default" as the setSecurityGroups it will use the default VPC. It just doesn't seem like like the groupid I'm passing, despite it being accurate.
Also, if I comment out the setSecurityGroups code, and use setSubnetId instead and pass the subnet id, it will create the instance just fine, but it goes into the "default" security group, not "testdefault" like I want.
All I'm trying to accomplish is creating an instance and having it use the already existing VPC group.
My Answer will focus on below statement:
All I'm trying to accomplish is creating an instance and having it use the already existing VPC group.
So, as I understand, you want to launch an instance in a non-default VPC and assign it an existing VPC security group to it.
I am not a java guy, but I could do what you wanted in ruby as below.
require 'aws-sdk-core'
Aws.config = {
:access_key_id => "my_access_key",
:secret_access_key => "my_secret_key",
:region => 'us-west-2'
}
ec2 = Aws::EC2.new
ec2.run_instances(
min_count: 1,
max_count: 1,
image_id: 'ami-8635a9b6',
instance_type: 't1.micro',
placement: {
availability_zone: 'us-west-2a'
},
network_interfaces: [
{
subnet_id: 'subnet-e881bd63',
groups: ['sg-fd53bf5e'],
device_index: 0,
associate_public_ip_address: true
}
],
key_name: 'my-key'
).each do |resp|
resp.instances.each do |x|
puts x.instance_id
end
end
Although this is a Ruby code, it is pretty straight forward and should give you some clear hints on how to go about doing it in Java as all these AWS SDKs are polling the same web service APIs.
I guess, the things that you should be concentrating in above code is:
:region => 'us-west-2'
and
placement: {
availability_zone: 'us-west-2a'
},
network_interfaces: [
{
subnet_id: 'subnet-e881bd63',
groups: ['sg-fd53bf5e'],
device_index: 0,
associate_public_ip_address: true
}
],
Make sure you explicitly specify the region.
Check how I have defined the subnet ID and security group ID. This code will launch my EC2 instance in subnet-e881bd63 of my VPC and will apply VPC security group ID sg-fd53bf5e to its 0th network interface. Besides, it will also assign a public IP address to my instance. (by default, it will not assign a public IP address when you launch instances in VPC).
FYI. When you launch instances in VPC, you must provide Security group ID instead of security group name.
This same error occurs using the command line program so I'm adding a separate answer helped by QuickNull. Simply make sure you specify the security group and subnet. For example:
aws ec2 run-instances --image-id ami-XXXXXXXX --count 1 --instance-type t1.micro --key-name XXXXXXXX --security-group-ids sg-XXXXXXXX --subnet-id subnet-XXXXXXXX
You can't specify security group names for VPC launch (setSecurityGroups). For a non-default VPC, you must use security group IDs instead. See EC2 run-instances page (withSecurityGroupIds , or --security-group-ids from CLI).
When you specify a security group for a nondefault VPC to the CLI or the API actions, you must use the security group ID and not the security group name to identify the security group.
See: Security Groups for EC2-VPC
Related:
Terraform throws "groupName cannot be used with the parameter subnet" or "VPC security groups may not be used for a non-VPC launch"
Thanks to #slayedbylucifer for his ruby code, here's the java solution for reference:
// Creates an instance in the specified subnet of a non-default VPC and using the
// security group with id sg-1234567
ec2.runInstances(new RuntInstancesRequest()
...
.withSubnetId("subnet-1234abcd")
.withSecurityGroupIds("sg-1234567"));

In java, how can I get an Amazon EC2 Instance to see its own tags?

So I have a java program running within an Amazon EC2 instance. Is there a way to programatically get its own tags? I have tried instantiating a new AmazonEC2Client to us the describeTags() function but it only gives me null. Any help would be appreciated thank you.
Edit: To make things clearer, the instances are going to be unmanned worker machines spun up to solely do some computations
This should help you get started...
String instanceId = EC2MetadataUtils.getInstanceId();
AmazonEC2 client = AmazonEC2ClientBuilder.standard()
.withCredentials(new DefaultAWSCredentialsProviderChain())
.build();
DescribeTagsRequest req = new DescribeTagsRequest()
.withFilters(new Filter("resource-id", Collections.singletonList(instanceId)));
DescribeTagsResult describeTagsResult = client.describeTags(req);
List<TagDescription> tags = describeTagsResult.getTags()
You should be able to get the current instance id by sending a request to: http://169.254.169.254/latest/meta-data/instance-id. This only works within ec2. With this you can access quite a bit of information about the instance. However, tags do not appear to be included.
You should be able to take the instance id along with the correct authentication to get the instance tags. If you are going to run this on an instance, you may want to provide an IAM user with limited access instead of a user which has access to everything in case the instance is compromised.
While using user-data may be the simplest solution, the OP was asking specifically about the tagging, and unfortunately amazon hasn't made this as easy as it could be. However, It can be done. You want to use a combination of 2 amazon services.
First you need to retrieve the Instance ID. This can be achieved by hitting the URL from within your instance:
http://169.254.169.254/latest/meta-data/instance-id
Once you have the resource ID, you'll want to use Amazon's EC2 API to access the tags. Since you said you're using Java, I would suggest the Using the AWS SDK amazon makes available. Within this SDK you'll find a method called describeTags (documentation). You can use a Resource ID as one of the filters to get the specific tags to your instance. Supported filters are
tag key
resource-id
resource-type
I suggest doing this retrieval at boot using something like cloud-init and caching the tags on your server for use later if necessary.

How to start and stop an Amazon EC2 instance programmatically in java

How do i start and stop an amazon EC2 instance programmatically using aws-sdk in java?
Any helps are greatly appreciated as I have spent a day while trying to sort this out.
I've recently implemented this functionality within the Bamboo AWS Plugin; it's Open Source and the code is available on Bitbucket, you can find a complete example how to start/stop/reboot an instance within EC2Task.java (should be a separate class actually, alas ...).
Fortunately this is not complicated at all, for example, an instance can be started like so:
private String startInstance(final String instanceId, AmazonEC2 ec2, final BuildLogger buildLogger)
throws AmazonServiceException, AmazonClientException, InterruptedException
{
StartInstancesRequest startRequest = new StartInstancesRequest().withInstanceIds(instanceId);
StartInstancesResult startResult = ec2.startInstances(startRequest);
List<InstanceStateChange> stateChangeList = startResult.getStartingInstances();
buildLogger.addBuildLogEntry("Starting instance '" + instanceId + "':");
// Wait for the instance to be started
return waitForTransitionCompletion(stateChangeList, "running", ec2, instanceId, buildLogger); }
BuildLogger is Bamboo specific and waitForTransitionCompletion() is an implementation specific helper to report back on the process/result. The AmazonEC2 ec2 parameter passes the reference to an AmazonEC2Client object by means of the AmazonEC2 interface, which defines all relevant methods (amongst many others), specifically:
StartInstances()
StopInstances()
RebootInstances()
If you have already used AWS API, it's simple call on AmazonEC2Client object. Use the following methods
Start Instance
Stop Instance
Also, you might be knowing the start/stop mechanism works only for the images with root device backed by EBS.

Categories

Resources