Develop Reference

How do I use the Grafana client for java to upload dashboards with a time-series filled panel to Grafana?

I am currently comparing time-series. They are stored in InfluxDB. My task is to get two time-series from InfluxDB, compare them and upload a visualization to Grafana. The comparison result would be output to console and the two time-series would be uploaded to Grafana in a dashboard and they should be in the same panel. I am trying to use the grafana-api-java-client found here. My problem is that I can not figure out how to do this with the provided examples and Javadocs. There is not a lot of documentation and the examples don't work properly in my case. I hope that someone has worked with this client and can explain how to properly add two time-series to a panel in a dashboard and upload it.
I will provide what I am doing and then post the examples from the github page of the API.
First of all this is how I am getting my time series:
public List<Double> getTimeSeries(String streetName, String start, String stop) {
//gets the query result
QueryResult queryResult = influxDB.query(new Query(String.format("SELECT value FROM your_measurement WHERE street='%s' " +
"AND time >= '%s' AND time <= '%s'", streetName, start, stop)));
//Gets the values from the query result
List<List<Object>> values = queryResult.getResults().iterator().next().getSeries().iterator().next().getValues();
//Adds the values to a list
List<Double> timeSeries = new ArrayList<>();
for (List<Object> li : values) {
timeSeries.add(Double.valueOf(li.get(1).toString()));
}
return timeSeries;
}
I can convert this list to an array of doubles:
double[] timeSeriesArray = timeSeries.stream().mapToDouble(d -> d).toArray();
The following is the first example. It works properly up until the getDashboard() and deleteDashboard() methods, where I get the error that such a dashboard does not exist, even though it does. I don't know what causes this error. Every new dashboard ends up in the folder "General". The method createDashboard() creates, as expected, an empty dashboard in Grafana.
import com.appnexus.grafana.client.GrafanaClient;
//Setup the client
GrafanaConfiguration grafanaConfiguration =
new GrafanaConfiguration().host("your_grafana_host").apiKey("Bearer your_secret_key");
GrafanaClient grafanaClient = new GrafanaClient(grafanaConfiguration);
//Setup the dashboard
String DASHBOARD_NAME = "new_dashboard";
Dashboard dashboard = new Dashboard()
.title(DASHBOARD_NAME)
.version(0);
GrafanaDashboard grafanaDashboard = new GrafanaDashboard().dashboard(dashboard);
//Make API calls
grafanaClient.createDashboard(grafanaDashboard);
grafanaClient.getDashboard(DASHBOARD_NAME);
grafanaClient.deleteDashboard(DASHBOARD_NAME);
This is the second example. I assume that I have to modify it in order to solve my problem. I had to replace two things to get this example to work, which I will explain with code comments. When executing this code it creates a dashboard with an empty panel. I expected something to be shown on the panel but it is empty and there are no axes.
import com.appnexus.grafana.client.GrafanaClient;
//Setup the client
GrafanaConfiguration grafanaConfiguration =
new GrafanaConfiguration().host("your_grafana_host").apiKey("Bearer your_secret_key");
GrafanaClient grafanaClient = new GrafanaClient(grafanaConfiguration);
//Setup the dashboard
String DASHBOARD_NAME = "new_dashboard";
DashboardPanelTarget dashboardPanelTarget =
new DashboardPanelTarget().refId("getSomeMetric").target("*");
DashboardPanelXAxis dashboardPanelXAxis =
new DashboardPanelXAxis().show(true).mode(DashboardPanelXAxis.Mode.TIME);
DashboardPanelYAxis dashboardPanelYAxis =
new DashboardPanelYAxis().format(DashboardPanelYAxis.Format.SHORT).logBase(1).show(true);
//Datasource is required or alerts cannot be added
DashboardPanel dashboardPanel =
new DashboardPanel()
//This might be where my data has to go but I am not sure.
.targets(new ArrayList<>(Collections.singletonList(dashboardPanelTarget)))
//Had to change DASHBOARD_DATA_SOURCE to a String -> "DASHBOARD_DATA_SOURCE", I assume it's just the name of the datasource.
//.datasource(DASHBOARD_DATA_SOURCE)
.datasource("DASHBOARD_DATA_SOURCE")
.type(DashboardPanel.Type.GRAPH)
.fill(1)
.title(dashboardName)
.linewidth(1)
.lines(true)
.height("300px")
.span(12)
.xaxis(dashboardPanelXAxis)
.yaxes(new ArrayList<>(Arrays.asList(dashboardPanelYAxis, dashboardPanelYAxis)));
DashboardRow dashboardRow =
new DashboardRow()
.collapse(false)
.panels(new ArrayList<>(Collections.singletonList(dashboardPanel)));
Dashboard dashboard =
new Dashboard()
.title(dashboardName)
.schemaVersion(1)
.rows(new ArrayList<>(Collections.singletonList(dashboardRow)));
DashboardMeta dashboardMeta = new DashboardMeta().canSave(true).slug(dashboardName);
GrafanaDashboard grafanaDashboard =
new GrafanaDashboard().meta(dashboardMeta).dashboard(dashboard);
//create new dashboard
//Had to change createDashboardTest() to createDashboard(), because createDashboardTest() doesn't seem to exist
//DashboardMeta createdDashboardMeta = createDashboardTest(grafanaDashboard);
DashboardMeta createdDashboardMeta = createDashboard(grafanaDashboard);

How to set TTL in Java based app for DynamoDB

Hi I need to set time to live programmatically for a table in DynamoDB via AWS Java SDK. Is it possible? I know that TTL feature is introduced recently - http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/TTL.html
UPDATE:
There is no special annotaion, but we can do it manually:
#DynamoDBAttribute
private long ttl;
and configure it as ttl in AWS - http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/time-to-live-ttl-how-to.html
long now = Instant.now().getEpochSecond(); // unix time
long ttl = 60 * 60 * 24; // 24 hours in sec
setTtl(ttl + now); // when object will be expired

http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/time-to-live-ttl-how-to.html
public void function(final AmazonDynamoDB client, final String tableName, final String ttlField){
//table created now enabling TTL
final UpdateTimeToLiveRequest req = new UpdateTimeToLiveRequest();
req.setTableName(tableName);
final TimeToLiveSpecification ttlSpec = new TimeToLiveSpecification();
ttlSpec.setAttributeName(ttlField);
ttlSpec.setEnabled(true);
req.withTimeToLiveSpecification(ttlSpec);
client.updateTimeToLive(req);
}

AmazonDynamoDBClient.updateTimeToLive documented here or direct link here

Code Sample here.
//Then set ttl field like below for days in java
//ttl 60 days
Calendar cal = Calendar.getInstance(); //current date and time
cal.add(Calendar.DAY_OF_MONTH, 60); //add days
double ttl = (cal.getTimeInMillis() / 1000L);

Google App engine end points in Java

For one of my Android app, I had to do a simple compute from Backend. So without much research I ended up using Google AppEngine (Endpoints), since it was simple and could be easily built with my Android app (in Android Studio itself). The end point API is a very simple low compute API which will be called maybe a few hundred times a day.
Now I got the rudest shock of my life.
I got a bill of 500$ for Google App Engine, and I was billed for over 8600 hours for 30 days period. Even if I ran my server for 30x24 hours it would be 720 hours.
How is this possible?
What did I do wrong?
/**
* An endpoint class we are exposing
*/
#Api(
name = "pushApi",
version = "v3",
namespace = #ApiNamespace(
ownerDomain = "com.example.sample",
ownerName = "com.example.sample",
packagePath = ""
)
)
public class MyEndpoint {
PushBean response = new PushBean();
/**
* A simple endpoint method that takes a name and says Hi back
*/
#ApiMethod(name = "pushToTopic")
public PushBean toTopic(#Named("topic") String topic, #Named("dataJson") String dataJson) {
try{
response.messageTopic(topic,dataJson);
return response;
}catch(Exception e){
return null;
}
}
}
The above code is the entry class for the Endpoint.

The image id '[ami-fd9cecc7]' does not exist

I want to create EC2 Instance using this Java code remotely:
public void testEC2ServiceInRegion() throws Exception
{
String launchInstance = launchInstance();
System.out.println("Status " + launchInstance);
}
public String launchInstance()
{
BasicAWSCredentials bawsc = new BasicAWSCredentials(
"AKIAIUY1KF4KZV3DAL21", "Onv+nq33tUkiLl1Ib2H9JtIB732QMEesh01Jl73L");
AmazonEC2 ec2 = new AmazonEC2Client(bawsc);
System.out.println("\n\nLAUNCH INSTANCE\n\n");
try
{
// Construct a RunInstancesRequest.
RunInstancesRequest request = new RunInstancesRequest();
request.setImageId("ami-fd9cecc7"); // the AMI ID, ami-fd9cecc7 is Amazon Linux AMI 2015.03 (HVM)
request.setInstanceType("t2.micro"); // instance type
request.setKeyName("desktop"); // the keypair
// request.setSubnetId("subnet-2dc0d459"); // the subnet
// ArrayList list = new ArrayList();
// list.add("sg-efcc248a"); // security group, call add() again to add more than one
// request.setSecurityGroupIds(list);
request.setMinCount(1); // minimum number of instances to be launched
request.setMaxCount(1); // maximum number of instances to be launched
// Pass the RunInstancesRequest to EC2.
RunInstancesResult result = ec2.runInstances(request);
String instanceId = result.getReservation().getInstances().get(0).getInstanceId();
// Return the first instance id in this reservation.
// So, don't launch multiple instances with this demo code.
System.out.println("Launching instance " + instanceId);
return instanceId;
} catch (Exception e)
{
// Simple exception handling by printing out error message and stack trace
System.out.println(e.getMessage());
e.printStackTrace();
return "ERROR";
}
}
But I get this error code:
The image id '[ami-fd9cecc7]' does not exist (Service: AmazonEC2; Status Code: 400; Error Code: InvalidAMIID.NotFound; Request ID: f85433c1-df4f-4105-bfe3-6f900eca6b70)
com.amazonaws.AmazonServiceException: The image id '[ami-fd9cecc7]' does not exist (Service: AmazonEC2; Status Code: 400; Error Code: InvalidAMIID.NotFound; Request ID: f85433c1-df4f-4105-bfe3-6f900eca6b70)
at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:1275)
at com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:873)
at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:576)
at com.amazonaws.http.AmazonHttpClient.doExecute(AmazonHttpClient.java:362)
Can you propose me some solution how to fix this code or there is a alternative?
Can you recommend me some working solution which I can use?

The AMI ami-fd9cecc7 exists in the Sydney (ap-southeast-2) region.
When you are executing your code, make sure that you are running it in the Sydney (ap-southeast-2) region. By default, it may run in Virginia (us-east-1). You may be able to specify the region by a code change or by a configuration change.
If you want your code to execute in Virginia (or any region other than Sydney), then you need to find a different AMI from that region to use as the base image for your EC2 instance.

You need to set Region while creating AmazonEC2Client.
Example:
Region usWest2 = Region.getRegion(Regions.US_WEST_2);
ec2.setRegion(usWest2);

I had everything configured correctly but failed to realise the AMI I created 10 minutes ago was still in 'Pending' status. Go to AMI dashboard and check:
If the AMI you're trying to use is 'Pending', try again when it's Available.
FYI it took about 12 minutes to become available. But it may take longer if the AMI is large or old.
I attempted the exact same configs immediately after it became 'Available' and it worked immediately:

Find all the attached volumes for an EC2 instance

I'm using the below code to get all the available volumes under EC2. But I can't find any Ec2 api to get already attached volumes with an instance. Please let me know how to get all attached volumes using instanceId.
EC2Api ec2Api = computeServiceContext.unwrapApi(EC2Api.class);
List<String> volumeLists = new ArrayList<String>();
if (null != volumeId) {
volumeLists.add(volumeId);
}
String[] volumeIds = volumeLists.toArray(new String[0]);
LOG.info("the volume IDs got from user is ::"+ Arrays.toString(volumeIds));
Set<Volume> ec2Volumes = ec2Api.getElasticBlockStoreApi().get()
.describeVolumesInRegion(region, volumeIds);
Set<Volume> availableVolumes = Sets.newHashSet();
for (Volume volume : ec2Volumes) {
if (volume.getSnapshotId() == null
&& volume.getStatus() == Volume.Status.AVAILABLE) {
LOG.debug("available volume with no snapshots ::" + volume.getId());
availableVolumes.add(volume);
}
}

The AWS Java SDK now provides a method to get all the block device mappings for an instance. You can use that to get a list of all the attached volumes:
// First get the EC2 instance from the id
DescribeInstancesRequest describeInstancesRequest = new DescribeInstancesRequest().withInstanceIds(instanceId);
DescribeInstancesResult describeInstancesResult = ec2.describeInstances(describeInstancesRequest);
Instance instance = describeInstancesResult.getReservations().get(0).getInstances().get(0);
// Then get the mappings
List<InstanceBlockDeviceMapping> mappingList = instance.getBlockDeviceMappings();
for(InstanceBlockDeviceMapping mapping: mappingList) {
System.out.println(mapping.getEbs().getVolumeId());
}

You can filter the output of the EC2 DescribeVolumes API call. There are various attachment.* filters available, the one you want is filtering by attached instance ID. Try the following code:
Multimap<String, String> filter = ArrayListMultimap.create();
filter.put("attachment.instance-id", instanceId);
filter.put("attachment.status", "attached");
Set<Volume> volumes = ec2Api.getElasticBlockStoreApi().get()
.describeVolumesInRegionWithFilter(region, volumeIds, filter);
The filter is a Multimap with the keys and values you want to filter on. You can actually specify the same filter multiple times, for example to get all volumes attached to a number of different instances.

You can use volumeAttachmentApi.listAttachmentsOnServer() to do this.
NovaApi novaApi = context.unwrapApi(NovaApi.class);VolumeApi volumeApi = novaApi.getVolumeExtensionForZone(region).get();
VolumeAttachmentApi volumeAttachmentApi = novaApi.getVolumeAttachmentExtensionForZone(region).get();
volumeAttachmentApi.listAttachmentsOnServer(serverId)