Develop Reference

Getting latest data from AWS custom Cloudwatch in Java

I have a custom metric in AWS cloudwatch and i am putting data into it through AWS java API.
for(int i =0;i<collection.size();i++){
String[] cell = collection.get(i).split("\\|\\|");
List<Dimension> dimensions = new ArrayList<>();
dimensions.add(new Dimension().withName(dimension[0]).withValue(cell[0]));
dimensions.add(new Dimension().withName(dimension[1]).withValue(cell[1]));
MetricDatum datum = new MetricDatum().withMetricName(metricName)
.withUnit(StandardUnit.None)
.withValue(Double.valueOf(cell[2]))
.withDimensions(dimensions);
PutMetricDataRequest request = new PutMetricDataRequest().withNamespace(namespace+"_"+cell[3]).withMetricData(datum);
String response = String.valueOf(cw.putMetricData(request));
GetMetricDataRequest res = new GetMetricDataRequest().withMetricDataQueries();
//cw.getMetricData();
com.amazonaws.services.cloudwatch.model.Metric m = new com.amazonaws.services.cloudwatch.model.Metric();
m.setMetricName(metricName);
m.setDimensions(dimensions);
m.setNamespace(namespace);
MetricStat ms = new MetricStat().withMetric(m);
MetricDataQuery metricDataQuery = new MetricDataQuery();
metricDataQuery.withMetricStat(ms);
metricDataQuery.withId("m1");
List<MetricDataQuery> mqList = new ArrayList<MetricDataQuery>();
mqList.add(metricDataQuery);
res.withMetricDataQueries(mqList);
GetMetricDataResult result1= cw.getMetricData(res);
}
Now i want to be able to fetch the latest data entered for a particular namespace, metric name and dimention combination through Java API. I am not able to find appropriate documenation from AWS regarding the same. Can anyone please help me?

I got the results from cloudwatch by the below code.\
GetMetricDataRequest getMetricDataRequest = new GetMetricDataRequest().withMetricDataQueries();
Integer integer = new Integer(300);
Iterator<Map.Entry<String, String>> entries = dimensions.entrySet().iterator();
List<Dimension> dList = new ArrayList<Dimension>();
while (entries.hasNext()) {
Map.Entry<String, String> entry = entries.next();
dList.add(new Dimension().withName(entry.getKey()).withValue(entry.getValue()));
}
com.amazonaws.services.cloudwatch.model.Metric metric = new com.amazonaws.services.cloudwatch.model.Metric();
metric.setNamespace(namespace);
metric.setMetricName(metricName);
metric.setDimensions(dList);
MetricStat ms = new MetricStat().withMetric(metric)
.withPeriod(integer)
.withUnit(StandardUnit.None)
.withStat("Average");
MetricDataQuery metricDataQuery = new MetricDataQuery().withMetricStat(ms)
.withId("m1");
List<MetricDataQuery> mqList = new ArrayList<>();
mqList.add(metricDataQuery);
getMetricDataRequest.withMetricDataQueries(mqList);
long timestamp = 1536962700000L;
long timestampEnd = 1536963000000L;
Date d = new Date(timestamp );
Date dEnd = new Date(timestampEnd );
getMetricDataRequest.withStartTime(d);
getMetricDataRequest.withEndTime(dEnd);
GetMetricDataResult result1= cw.getMetricData(getMetricDataRequest);

How to pass numerical and categorical features to RandomForestRegressor in Apache Spark: MLlib in Java?

How to pass numerical and categorical features to RandomForestRegressor in Apache Spark: MLlib in Java?
I'm able to do it with numerical or categorical, but I don't know how to implement it together.
My working code is as follows (only numerical features used for prediction)
String[] featureNumericalCols = new String[]{
"squareM",
"timeTimeToPragueCityCenter",
};
String[] featureStringCols = new String[]{ //not used
"type",
"floor",
"disposition",
};
VectorAssembler assembler = new VectorAssembler().setInputCols(featureNumericalCols).setOutputCol("features");
Dataset<Row> numericalData = assembler.transform(data);
numericalData.show();
RandomForestRegressor rf = new RandomForestRegressor().setLabelCol("price")
.setFeaturesCol("features");
// Chain indexer and forest in a Pipeline
Pipeline pipeline = new Pipeline()
.setStages(new PipelineStage[]{assembler, rf});
// Train model. This also runs the indexer.
PipelineModel model = pipeline.fit(trainingData);
// Make predictions.
Dataset<Row> predictions = model.transform(testData);

For anyone out there, this is the solution:
StringIndexer typeIndexer = new StringIndexer()
.setInputCol("type")
.setOutputCol("typeIndex");
preparedData = typeIndexer.fit(preparedData).transform(preparedData);
StringIndexer floorIndexer = new StringIndexer()
.setInputCol("floor")
.setOutputCol("floorIndex");
preparedData = floorIndexer.fit(preparedData).transform(preparedData);
StringIndexer dispositionIndexer = new StringIndexer()
.setInputCol("disposition")
.setOutputCol("dispositionIndex");
preparedData = dispositionIndexer.fit(preparedData).transform(preparedData);
String[] featureCols = new String[]{
"squareM",
"timeTimeToPragueCityCenter",
"floorIndex",
"floorIndex",
"dispositionIndex"
};
VectorAssembler assembler = new VectorAssembler().setInputCols(featureCols).setOutputCol("features");
preparedData = assembler.transform(preparedData);
// ... some more impelemtation details
RandomForestRegressor rf = new RandomForestRegressor().setLabelCol("price")
.setFeaturesCol("features");
return rf.fit(preparedData);

Convert string to object name Java

it might be a simple question or even impossible without any kind of Interface (Arrays, Maps etc.) but I would like to know if there's any possibility of converting an object name to String so I can pass as argument. I have two classes Paciente and Sintomas with multiple objects that I need to pass as argument to a function but I don't want to use arrays (it must be like that) and I can't figure any other way of doing so without manually make an insert for each one.
Paciente Paciente1 = new Paciente("001", "Ana Melo", 33, "");
Paciente Paciente2 = new Paciente("002", "Rui Costa", 13, "");
Paciente Paciente3 = new Paciente("003", "Joana Martins", 85, "");
Paciente Paciente4 = new Paciente("004", "Pedro Torres", 53, "");
Paciente Paciente5 = new Paciente("005", "Ana Gomes", 93, "");
Paciente Paciente6 = new Paciente("006", "Jorge Costa", 56, "");
Sintomas Sintoma1 = new Sintomas("001", "febre");
Sintomas Sintoma2 = new Sintomas("001", "dores");
Sintomas Sintoma3 = new Sintomas("001", "machas");
Sintomas Sintoma4 = new Sintomas("002", "febre");
Sintomas Sintoma5 = new Sintomas("002", "manchas");
Sintomas Sintoma6 = new Sintomas("003", "febre");
Sintomas Sintoma7 = new Sintomas("003", "dores");
Sintomas Sintoma8 = new Sintomas("004", "febre");
Sintomas Sintoma9 = new Sintomas("006", "manchas");
Sintomas Sintoma10 = new Sintomas("006", "dores");
// now I would like to pass to a function as argument something like this:
for(int i = 0 ; i < 6 ; i++)
kSession.insert("Paciente"+(i+1));
// instead of making
kSession.insert(Paciente1);
kSession.insert(Paciente2);
kSession.insert(Paciente3);
// and so on.

Something like this should work(asuming you mean no array because of size constraints), note that there has to be somewhere you add the data, it's also possible to load it from a txt or something, but it has to be defined at some point
List<Paciente> pacientes = new ArrayList<>(); // no size constraints, automatically expands if too small
pacientes.add(new Paciente("", "", ""));
for (Paciente paciente : pacientes) { // loop all Patientes in pacientes
kSession.insert(paciente); // add a paciente to the session, for every entry
}
ofcource same can be done for any class, or object
It all really comes down to, how do you wish to store and access the data, and where do you need to store and access it. Using ArrayList and Map's offer the utility of easily changing the size and content of a list of data, but as any data it must be initially inserted
As a side note if the patients have an ID then using a Map
Map<String, Paciente> pacientes = new HashMap<>();
provides a way to acces the patiens very fast, and the TreeMap structure is sorted on key, should that be needed.
Other options could be
Wrapper classes that manage the data, will work similarily to an ArrayList<> but you can define, rules for adding, deleting, and such from the list.
public class Wrapper{
private List<Paciente> pacientes = new ArrayList<>();
public void addPaciente(Paciente paciente){
if(!this.pacientes.contains(paciente)) // prevent multi entries
paciente.add(paciente);
}
public void addPacientes(List<Paciente> pacientes){
for(Paciente paciente : pacientes) // add all patients using the add method
this.addPaciente(paciente);
}
public List<Paciente> getPacientes(){
return this.pacientes;
}
}
You can then add the patients to the kSession, as earlier described
Finally, there is no reason why Paciente, can have the list of Sintomas, such that
public class Paciente{
private List<Sintomas> sintomas = new ArrayList<>();
public addSintomas(Sintomas sintomas){
if(!this.sintomas.contains(sintomas))
this.sintomas.add(sintomas);
}
// rest is the same principle as above in "Wrapper"
}
This way you can get a Paciente, and add a Sintomas, and then when you wish to check a Pacientes Sintomas you can just get the list of Sintomas from that Paciente

RandomForest with Weka in Java

I am working on a project and I need some examples how to implement RandomForest in Java with weka? I did it with IBk(), it worked. If I do it with RandomForest in the same way, it does not work.
Does anyone have a simple example for me how to implement RandomForest and how to get probability for each class (i did it with IBk withclassifier.distributionForInstance(instance) Function and it returned me probabilities for each class). How can I do it for RandomForest? I will need to get probability of every tree and to combine it?
//example
ConverrterUtils.DataSource source = new ConverterUtils.DataSource ("..../edit.arff);
Instances dataset = source.getDataSet();
dataset.setClassIndex(dataset.numAttributes() - 1);
IBk classifier = new IBk(5); classifier.buildClassifier(dataset);
Instance instance = new SparseInstance(2);
instance.setValue(0, 65) //example data
instance.setValue(1, 120); //example data
double[] prediction = classifier.distributionForInstance(instance);
//now I get the probability for the first class
System.out.println("Prediction for the first class is: "+prediction[0]);

You can calculate the the infogain while buidling the Model in the RandomForest. It is much slower and requires alot of memory while buidling model. I am not so sure about the documentation. you can add options or setValues while buiilding the model.
//numFolds in number of crossvalidations usually between 1-10
//br is your bufferReader
Instances trainData = new Instances(br);
trainData.setClassIndex(trainData.numAttributes() - 1);
RandomForest rf = new RandomForest();
rf.setNumTrees(50);
//You can set the options here
String[] options = new String[2];
options[0] = "-R";
rf.setOptions(options);
rf.buildClassifier(trainData);
weka.filters.supervised.attribute.AttributeSelection as = new weka.filters.supervised.attribute.AttributeSelection();
Ranker ranker = new Ranker();
InfoGainAttributeEval infoGainAttrEval = new InfoGainAttributeEval();
as.setEvaluator(infoGainAttrEval);
as.setSearch(ranker);
as.setInputFormat(trainData);
trainData = Filter.useFilter(trainData, as);
Evaluation evaluation = new Evaluation(trainData);
evaluation.crossValidateModel(rf, trainData, numFolds, new Random(1));
// Using HashMap to store the infogain values of the attributes
int count = 0;
Map<String, Double> infogainscores = new HashMap<String, Double>();
for (int i = 0; i < trainData.numAttributes(); i++) {
String t_attr = trainData.attribute(i).name();
//System.out.println(i+trainData.attribute(i).name());
double infogain = infoGainAttrEval.evaluateAttribute(i);
if(infogain != 0){
//System.out.println(t_attr + "= "+ infogain);
infogainscores.put(t_attr, infogain);
count = count+1;
}
}
//iterating over the hashmap
Iterator it = infogainscores.entrySet().iterator();
while (it.hasNext()) {
Map.Entry pair = (Map.Entry)it.next();
System.out.println(pair.getKey()+" = "+pair.getValue());
System.out.println(pair.getKey()+" = "+pair.getValue());
it.remove(); // avoids a ConcurrentModificationException
}

Storing multiple values from the same object in an Array

I am trying to create an Array with all these values form the Detail Class. Is there a better way of creating this array?
Details[] DetailsArray = new Details[10];
Details Details = new Details();
Details Details2 = new Details();
Details Details3 = new Details();
Details Details4 = new Details();
Details Details5 = new Details();
Details Details6 = new Details();
Details.setNumber(new Integer(1));
Details2.setEmployeeID(new Double(300));
Details3.setCurrency("Euro");
Details4.setSize(new Double (400));
Details5.setEvent("Something");
Details6.setId(new Integer(10));
DetailsArray[0] = Details;
DetailsArray[1] = Details2;
DetailsArray[2] = Details3;
DetailsArray[3] = Details4;
DetailsArray[4] = Details5;
DetailsArray[5] = Details6;
System.out.println(DetailsArray);
EmployeeDetails.setDetails(DetailsArray);

Use a for loop?
Details[] detailsArray = new Details[10];
for (int i = 0; i < detailsArray.length; i++)
detailsArray[i] = new Details();
detailsArray[0].setNumber(New Integer(1));
...
That being said ... rarely in real code would you have a static array that you're populating like this manually. If you did, you might want to have the data for those objects in an external file, or really some other data structure completely.

Well it's pretty odd code to start with to be honest, but this would work:
// Why are you creating an array with 10 elements but only using 6?
Details[] detailsArray = new Details[10];
for (int i = 0; i < 6; i++) {
detailsArray[i] = new Details();
}
detailsArray[0].setNumber(new Integer(1));
detailsArray[1].setEmployeeID(new Double(300));
detailsArray[2].setCurrency("Euro");
detailsArray[3].setSize(new Double (400));
detailsArray[4].setEvent("Something");
detailsArray[5].setId(new Integer(10));
Do you really want to set a different property on each object? Are you sure you don't really want to set multiple properties on a single object?

You could use a list instead of an array, but if I understand what you are trying to achieve, you maybe want this instead, where your details object holds all the information for that employee:
Details details = new Details();
details.setNumber(new Integer(1));
details.setEmployeeID(new Double(300));
details.setCurrency("Euro");
details.setSize(new Double (400));
details.setEvent("Something");
details.setId(new Integer(10));
System.out.println(details);
EmployeeDetails.setDetails(details);

Create constructor that accepts all needed parameters. Then do something like this:
deatilsArray = new DetailsArray[] {
new Details(1, 300, "Euro", 400, "Something"),
new Details(2, 400, "Dollar", 500, "Something else"),
..................
};
Alternatively you can create other class DetailsBuilder that initiates details' fields:
Details d =
new DetailsBuilder().create().setName("aaa").setEmployeeId(123).setCurrency("Euro").getInstance();
In this case the code above will look like:
DetailsBuilder builder = new DetailsBuilder();
deatilsArray = new DetailsArray[] {
builder.create().setName("aaa").setEmployeeId(123).setCurrency("Euro").getInstance(),
builder.create().setName("bbb").setEmployeeId(123).setCurrency("Euro").getInstance(),
};
BTW according to java naming convention variable names always start with small letter.

Solution using a temporary inner class.
class Pwn {
final Details d = new Details();
}
Details[] detailsArray = new Details[] {
(new Pwn (){{ d.setNumber(new Integer(1)); }}).d,
(new Pwn (){{ d.setEmployeeID(new Double(300)); }}).d,
(new Pwn (){{ d.setCurrency("Euro"); }}).d,
(new Pwn (){{ d.setSize(new Double (400)); }}).d,
(new Pwn (){{ d.setEvent("Something"); }}).d,
(new Pwn (){{ d.setId(new Integer(10)); }}).d
}
:)