Develop Reference

Is there any way to get Virtual Machine object by just instance name without resource Group in AzureRm in java API?

VirtualMachineGetResponse get(String resourceGroupName, String vmName) throws IOException, ServiceException, URISyntaxException;
From VirtualMachineGetResponse, I can get VirtualMachine.
Is there any way to get VirtualMachine object by just giving instanceId as above code snippet expects resourceGroupName and instanceName?

There is not any way to get VM object with only instance name in ARM for any languages. It's depended on Azure REST API Get for Virtual Machines in ARM, and the Java API just wrap it.
The REST API GET request for Virtual Machines in ARM, as below.
GET /subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Compute/virtualMachines/{vmName}?api-version=2016-03-30[&$expand]
Besides subscriptionId required for authentication, the parameters resourceGroupName & vmName are required in the api uri.

Judging by how ARM works, I don't think there is a way to do that at all (doesn't matter what language\tool you are using).
What you can do, is get the list off ALL vm's and filter the one you need.

Availability zone selection in AWS

My question is somewhat similar to this SO but I could not find the answer there.
I using the following code snippet to create instance.
Code:
public void test(String accessId, String accessKey){
credentials = new BasicAWSCredentials(accessId, accessKey);
amazonEC2Client = new AmazonEC2Client(credentials);
amazonEC2Client.setEndpoint("ec2.ap-northeast-1.amazonaws.com");
RunInstancesRequest runInstancesRequest = new RunInstancesRequest();
runInstancesRequest.withImageId(imageId)
.withInstanceType("t2.micro")
.withMinCount(2)
.withMaxCount(2);
}
When I ran the above snippet I got the following error.
We currently do not have sufficient t2.micro capacity in zones with support for'gp2'volumes.
Our system will be working on provisioning additional capacity.
(Service: AmazonEC2; Status Code: 500; Error Code:
InsufficientInstanceCapacity; Request ID: c1996284-c208-446a-9f4c-301d8900e503)
After googling I found that AWS does not have that amount of t2.micro instance at that time and it recommended to create instance in different availability zone.
But in the code I have not provided any availability zone but the instances are being created in the ap-northeast-1a availability zone and throws above error and terminates.
When I created in through AWS web console it automatically created an instance in ap-northeast-1c without throwing any error.
Is there any way to create an instance in any available zone of that particular region where ever that instance is available programmatically ?.
My understanding is there is no or less t2.micro instance available in ap-northeast-1 region when I called the API.
Is there any API to check is Instance are available in a particular region ?.

AWS do not provide an API to check what available instances there are in a particular region.
You could try one AZ, if it fails try the next AZ, and so on.
Alternatively you could use an AutoScaling group that is allowed to launch instances in different AZs. It will automatically do the above for you.

How to get the automatically defined port for a Spark Java Application?

In the API documentation for Java Spark (not Apache spark), you can specify a port of 0 to have it automatically select a port. Great!
However, I cannot figure out how to get that port after the server is started. I can see it in the logs:
15:41:12.459 [Thread-2] INFO spark.webserver.JettySparkServer - >> Listening on 0.0.0.0:63134
But I need to be able to get to it programmatically, so that my integration tests are able to run reliably every time.
So how do I get that port?

I could find no way to get this information in the API, and so I filed an issue on their github.
I was able to get at it via an ugly pile of reflection:
/**
* Meant to be called from a different thread, once the spark app is running
* This is probably only going to be used during the integration testing process, not ever in prod!
*
* #return the port it's running on
*/
public static int awaitRunningPort() throws Exception {
awaitInitialization();
//I have to get the port via reflection, which is fugly, but the API doesn't exist :(
//Since we'll only use this in testing, it's not going to kill us
Object instance = getInstance();
Class theClass = instance.getClass();
Field serverField = theClass.getDeclaredField("server");
serverField.setAccessible(true);
Object oneLevelDeepServer = serverField.get(instance);
Class jettyServerClass = oneLevelDeepServer.getClass();
Field jettyServerField = jettyServerClass.getDeclaredField("server");
jettyServerField.setAccessible(true);
//Have to pull in the jetty server stuff to do this mess
Server jettyServer = (Server)jettyServerField.get(oneLevelDeepServer);
int acquiredPort = ((ServerConnector)jettyServer.getConnectors()[0]).getLocalPort();
log.debug("Acquired port: {}", acquiredPort);
return acquiredPort;
}
This works well for me in our integration tests, but I'm not using https, and it does reach about two levels deep into the API via reflection grabbing protected fields. I could not find any other way to do it. Would be quite happy to be proven wrong.

This will work on Spark 2.6.0:
public static int start (String keystoreFile, String keystorePw)
{
secure(keystoreFile, keystorePw, null, null);
port(0);
staticFiles.location("/public");
get(Path.CLOCK, ClockController.time);
get(Path.CALENDAR, CalendarController.date);
// This is the important line. It must be *after* creating the routes and *before* the call to port()
awaitInitialization();
return port();
}
Without the call to awaitInitialization() port() would return 0.

Why might describing Amazon EC2 instances yield no result?

I am trying to retrieve all the instances running in my AWS account (say instance id, etc). I use the following code. I am not able to print the instance ids. When I debug, I am just getting null values. But I have three instances running on AWS. Can someone point out what I am doing wrong here?
DescribeInstancesResult result = ec2.describeInstances();
List<Reservation> reservations = result.getReservations();
for (Reservation reservation : reservations) {
List<Instance> instances = reservation.getInstances();
for (Instance instance : instances) {
System.out.println(instance.getInstanceId());
}
}

The most common cause for issues like this is a missing region specification when initializing the client, see section To create and initialize an Amazon EC2 client within Create an Amazon EC2 Client for details:
Specifically, step 2 only creates an EC2 client without specifying the region explicitly:
2) Use the AWSCredentials object to create a new AmazonEC2Client instance, as follows:
amazonEC2Client = new AmazonEC2Client(credentials);
This yields a client talking to us-east-1 - surprisingly, the AWS SDKs and the AWS Management Console use different defaults even as outlined in step 3, which also shows how to specify a different endpoint:
3) By default, the service endpoint is ec2.us-east-1.amazonaws.com. To specify a different endpoint, use the setEndpoint method. For example:
amazonEC2Client.setEndpoint("ec2.us-west-2.amazonaws.com");
The AWS SDK for Java uses US East (N. Virginia) as the default region
if you do not specify a region in your code. However, the AWS
Management Console uses US West (Oregon) as its default. Therefore,
when using the AWS Management Console in conjunction with your
development, be sure to specify the same region in both your code and
the console. [emphasis mine]
The differing defaults are easy to trip over, and the respective default in the AWS Management Console has in fact changed over time - as so often in software development, I recommend to always be explicit about this in your code to avoid such subtle error sources.

In java, how can I get an Amazon EC2 Instance to see its own tags?

So I have a java program running within an Amazon EC2 instance. Is there a way to programatically get its own tags? I have tried instantiating a new AmazonEC2Client to us the describeTags() function but it only gives me null. Any help would be appreciated thank you.
Edit: To make things clearer, the instances are going to be unmanned worker machines spun up to solely do some computations

This should help you get started...
String instanceId = EC2MetadataUtils.getInstanceId();
AmazonEC2 client = AmazonEC2ClientBuilder.standard()
.withCredentials(new DefaultAWSCredentialsProviderChain())
.build();
DescribeTagsRequest req = new DescribeTagsRequest()
.withFilters(new Filter("resource-id", Collections.singletonList(instanceId)));
DescribeTagsResult describeTagsResult = client.describeTags(req);
List<TagDescription> tags = describeTagsResult.getTags()

You should be able to get the current instance id by sending a request to: http://169.254.169.254/latest/meta-data/instance-id. This only works within ec2. With this you can access quite a bit of information about the instance. However, tags do not appear to be included.
You should be able to take the instance id along with the correct authentication to get the instance tags. If you are going to run this on an instance, you may want to provide an IAM user with limited access instead of a user which has access to everything in case the instance is compromised.

While using user-data may be the simplest solution, the OP was asking specifically about the tagging, and unfortunately amazon hasn't made this as easy as it could be. However, It can be done. You want to use a combination of 2 amazon services.
First you need to retrieve the Instance ID. This can be achieved by hitting the URL from within your instance:
http://169.254.169.254/latest/meta-data/instance-id
Once you have the resource ID, you'll want to use Amazon's EC2 API to access the tags. Since you said you're using Java, I would suggest the Using the AWS SDK amazon makes available. Within this SDK you'll find a method called describeTags (documentation). You can use a Resource ID as one of the filters to get the specific tags to your instance. Supported filters are
tag key
resource-id
resource-type
I suggest doing this retrieval at boot using something like cloud-init and caching the tags on your server for use later if necessary.