Develop Reference

Selenium ChromeDriver: increasing time of getting WebElement Text

I have a code in which I traverse table rows and columns, and I'd like to add it's values to a list.
It takes me a significant amount of time.
So I added a time measurement, and I noticed that for some reason the time increases from row to row.
I cannot understand why.
Can you advise please?
private void buildTableDataMap() {
WebElement table = chromeWebDriver.findElement(By.id("table-type-1"));
List<WebElement> rows = table.findElements(By.tagName("tr"));
theMap.getInstance().clear();
String item;
for (WebElement row : rows) {
ArrayList<String> values = new ArrayList<>();
List<WebElement> tds = row.findElements(By.tagName("td"));
if(tds.size() > 0){
WebElement last = tds.get(tds.size() - 1);
long time = System.currentTimeMillis();
values.addAll(tds.stream().map(e->e.getText()).collect(Collectors.toList()));
System.out.println(System.currentTimeMillis() - time);
//remove redundant last entry:
values.remove(tds.size() - 1);
callSomeFunc(values, last);
item = tds.get(TABLE_COLUMNS.NAME_COL.getNumVal()).getText();
item = item.replaceAll("[^.\\- /'&A-Za-z0-9]", "").trim();//remove redundant chars
theMap.getInstance().getMap().put(item, values);
}
}
}
Guys, I continued researching.
First of all, Florent's kind answer did not help me because, at lease as I understand, It returned me an array list of strings which I had to parse, and I don't like this kind of solution too much...
So I nailed the problem in finding that the e.getText() call was increasing in time from call to call!!!
I also tried e.getAttribute("innerText") instead but no change.
Can't understand why. Any idea to solve?
WebElement last = null;
for (WebElement e : tds){
last = e;
long tm1 = 0, tm2 = 0;
if(Settings.verboseYN) {
tm1 = System.currentTimeMillis();
}
s = e.getText(); //This action increases in time!!!
if(Settings.verboseYN) {
tm2 = System.currentTimeMillis();
}
values.add(s); //a 0 ms action!!!
if(Settings.verboseYN) {
System.out.println("e.getText()) took " + (tm2 - tm1) + " ms...");
}
}
That is an graph of the time getText took...
08-May-18
Another source of growing execution time is this one:
void func(WebElement anchorsElement){
List<WebElement> anchors = anchorsElement.findElements(By.tagName("a"));
for (WebElement a : anchors) {
if (a.getAttribute("class").indexOf("a") > 0)
values.add("A");
else if (a.getAttribute("class").indexOf("b") > 0)
values.add("B");
else if (a.getAttribute("class").indexOf("c") > 0)
values.add("C");
}
}
Every functions has 5 iterations only, but still each call to the function increases its execution time.
Is there a solution for this one as well?

Calling the driver is an expensive operation. To significantly reduce the execution time, use a JavaScript injection with executeScript to read the whole table in a single call. Then process/filter the data on the client side with Java.
public ArrayList<?> readTable(WebElement table)
{
final String JS_READ_CELLS =
"var table = arguments[0]; " +
"return map(table.querySelectorAll('tr'), readRow); " +
"function readRow(row) { return map(row.querySelectorAll('td'), readCell) }; " +
"function readCell(cell) { return cell.innerText }; " +
"function map(items, fn) { return Array.prototype.map.call(items, fn) }; " ;
WebDriver driver = ((RemoteWebElement)table).getWrappedDriver();
Object result = ((JavascriptExecutor)driver).executeScript(JS_READ_CELLS, table);
return (ArrayList<?>)result;
}

The problem you are facing is because of the way Selenium works by design. Let's look at how a JavaScript get's executed or a operation is performed
tds.get(TABLE_COLUMNS.NAME_COL.getNumVal()).getText();
You have a collection of objects. Each object is assigned a unique ID on the browser side by the selenium driver
So when you do a getText() below is what happens
Your code -> HTTP Request -> Browser Driver -> Browser ->
|
<---------------------------------------------
Now if you have a table of 400rx10c then it accounts to 4000 HTTP calls, even if one call takes 10ms, we are looking at a 40000ms~=40sec, which is a decent delay to read a table
So what you want to do is to get all the data in single go by executing a javascript which give you 2d array back. It is quite simple, I found a code on below site
http://cwestblog.com/2016/08/21/javascript-snippet-convert-html-table-to-2d-array/
function tableToArray(tbl, opt_cellValueGetter) {
opt_cellValueGetter = opt_cellValueGetter || function(td) { return td.textContent || td.innerText; };
var twoD = [];
for (var rowCount = tbl.rows.length, rowIndex = 0; rowIndex < rowCount; rowIndex++) {
twoD.push([]);
}
for (var rowIndex = 0, tr; rowIndex < rowCount; rowIndex++) {
var tr = tbl.rows[rowIndex];
for (var colIndex = 0, colCount = tr.cells.length, offset = 0; colIndex < colCount; colIndex++) {
var td = tr.cells[colIndex], text = opt_cellValueGetter(td, colIndex, rowIndex, tbl);
while (twoD[rowIndex].hasOwnProperty(colIndex + offset)) {
offset++;
}
for (var i = 0, colSpan = parseInt(td.colSpan, 10) || 1; i < colSpan; i++) {
for (var j = 0, rowSpan = parseInt(td.rowSpan, 10) || 1; j < rowSpan; j++) {
twoD[rowIndex + j][colIndex + offset + i] = text;
}
}
}
}
return twoD;
}
I assume you store the above script in a SCRIPT variable and then you can run it like below
WebDriver driver = ((RemoteWebElement)table).getWrappedDriver();
Object result = ((JavascriptExecutor)driver).executeScript(SCRIPT + "\n return tableToArray(arguments[0]);" , table);
This will get you a 2D array of the data and you can then process it the way you like it

attribute selection assigned for each attribute

I am using weka with java by using Eclipse IDE for Java Development. Version: Neon 4.6.
I would like to know how could I extract the values like:
correlation ranking assigned for each attribute.
SVM-RFE ranking attribute and weight values assigned for each attribute
I would like to see these values on the screen.
I am using weka:
I tried with this code:
public class AttributeSelectionTest {
protected static void useRanker(Instances data) throws Exception {
SVMAttributeEval eval = new SVMAttributeEval();
eval.buildEvaluator(data);
Evaluation evaluation = new Evaluation(data);
System.out.println(eval.getPercentToEliminatePerIteration());
System.out.println(eval.attsToEliminatePerIterationTipText());
eval.getPercentToEliminatePerIteration();
for (int classInd = 0; classInd < data.numAttributes(); classInd++)
System.out.println(eval.rankBySVM(classInd,data));
System.out.println(evaluation.toSummaryString());
}
public static void main(String[] args) throws Exception {
// load data
System.out.println("\n0. Loading data");
DataSource source = new DataSource("data.arff");
Instances data = source.getDataSet();
if (data.classIndex() == -1)
data.setClassIndex(data.numAttributes() - 1);
useRanker(data);
}
}

To get SVM-RFE ranking use the following code.Use your file data and load it using fileHandler
Dataset dataSet = FileHandler.loadDataset(new File("sample.data"), 4, ",");
RecursiveFeatureEliminationSVM svmrfe = new RecursiveFeatureEliminationSVM(0.2);
svmrfe.build(dataSet);
for (int i = 0; i < svmrfe.noAttributes(); i++)
System.out.println(svmrfe.rank(i));
for the same data you can get the attribute selection ranking as
ASEvaluation eval = new GainRatioAttributeEval();
ASSearch search = new Ranker();
WekaAttributeSelection attributeSelection = new WekaAttributeSelection(eval,search);
wekaattrsel.build(dataSet);
for (int i = 0; i < attributeSelection.noAttributes(); i++)
System.out.println("Attribute : " + i + " Ranks : " + attributeSelection.rank(i));

Jmeter - Creating a new variable if variable already exists using Java

I'm using Jmeter and want to use Java to update variables,
I have a variable called XXVONO which stores values and adds a number suffix when executed in a loop. Example:
XXVONO_1 = value1
XXVONO_2 = value2
XXVONO_3 = value3
These variables contains values which are automatically stored when the loop is executed. However, I am trying to make a code which checks if the variable is empty or not, If true, it will save the new values, where if false, it will create a new variable (XXVONO_4) and save the value there without overwriting the existing variables.
How would I go about doing this? Do I use a while loop?
if (vars.get("VONO_2") != "") {
if (vars.get("XXVONO_" + vars.get("aps200_count_3")) == "") {
vars.put("XXVONO_" + vars.get("aps200_count_3"), vars.get("VONO_2"));
vars.put("XXJRNO_" + vars.get("aps200_count_3"), vars.get("JRNO_2"));
} else {
while (vars.get("XXVONO_" + vars.get("aps200_count_3")) != "") {
vars.put("new_count", vars.get("aps200_count_3"));
Integer temp = Integer.parseInt(vars.get("new_count")) + 1;
vars.put("new_count", temp.toString());
}
vars.put("XXVONO_" + vars.get("new_count"), vars.get("VONO_2"));
vars.put("XXJRNO_" + vars.get("new_count"), vars.get("JRNO_2"));
}
}

You can try using a map instead of creating a variables at runtime
Map<String,Object> map = new HashMap<>();
Inside the loop
if(map.get("DynamicVariableName")!=null){
map.put("DynamicVariableName"+autogeneratedNumberSuffix,ValueToBeStored)
}
else{
map.put("DynamicVariableName",ValueToBeStored)
}

What you could do is use an if/else statement:
if (XXVONO_1 == null)
{
XXYVONO_1 = //Insert data here
}
else if (XXVONO_2 == null)
{
XXVONO_2 = //Insert data here
}
else if (XXVONO_3 == null)
{
XXVONO_3 == //Insert data here
}
else
{
XXVONO_4 == //Insert data here
}
Of course, you can keep adding variables.
If there is no limit to the number of variables, try this:
HashMap<String, String> XXVONO = new HashMap<String, Integer>();
for (i = 1; i <= /*Number of variables*/; i += 1; i++) {
if (XXVONO["XXVONO_" + i] == null) {
XXVONO.put("XXVONO_" + i, /*insert data here*/);
}
}

MOA's StreamKM clustering doesn't return any result

I'm currently trying to cluster a great amount of data points into a given amount of clusters and I wanted to try MOA's streaming based k-means StreamKM. A very simple example of what I'm trying to do using random data looks as follows:
StreamKM streamKM = new StreamKM();
streamKM.numClustersOption.setValue(5); // default setting
streamKM.widthOption.setValue(100000); // default setting
streamKM.prepareForUse();
for (int i = 0; i < 150000; i++) {
streamKM.trainOnInstanceImpl(randomInstance(2));
}
Clustering result = streamKM.getClusteringResult();
System.out.println("size = " + result.size());
System.out.println("dimension = " + result.dimension());
The random instances are created as follows:
static DenseInstance randomInstance(int size) {
DenseInstance instance = new DenseInstance(size);
for (int idx = 0; idx < size; idx++) {
instance.setValue(idx, Math.random());
}
return instance;
}
However, when running the given code, no clusters seem to be created:
System.out.println("size = " + result.size()); // size = 0
System.out.println("dimension = " + result.dimension()); // NPE
Is there anything else I need to take care of, or do I have a fundamental misunderstanding of the MOA clustering concepts?

I think prepareForUse() method is not the correct method that initialize the algorithm.
Instead of streamKM.prepareForUse(); , you should use streamKM.resetLearning();.
In short, your code should be like:
StreamKM streamKM = new StreamKM();
streamKM.numClustersOption.setValue(5); // default setting
streamKM.widthOption.setValue(100000); // default setting
streamKM. resetLearning(); // UPDATED CODE LINE !!!
for (int i = 0; i < 150000; i++) {
streamKM.trainOnInstanceImpl(randomInstance(2));
}
Clustering result = streamKM.getClusteringResult();
System.out.println("size = " + result.size());
System.out.println("dimension = " + result.dimension());

MPAndroidChart - linechart realtime graph - remove x-values doesn't work

I got a really annoying problem with my realtime graph. It works really pretty with MpAndroidChart library, but I guess I'm running out of memory. Because my realtime lineChart-graph become slower the longer it's running, until it get stuck. I didn't find any solution that works.
This is my linecharts screenshot
This is my code to enter values into charts,
( test01 - test04 are possible solution from other posts, I tried )
private void addEntry(double[][] pts) {
// System.out.println("######### addEntry #########");
if (mLineChartObj != null) {
int anzLineChartObj = mLineChartObj.size();
// System.out.println("######### Anzahl LineChartObj: " + anzLineChartObj);
for (int i = 0; i < anzLineChartObj; i++) {
LineChartObject ob = mLineChartObj.get(i);
LineData data = ob.lineChart.getData();
if (data != null) {
data.addXValue((ob.set).getEntryCount() + "");
data.addEntry(new Entry((float) pts[ob.rootNodePos][0], (ob.set).getEntryCount()), 0);
// data.addEntry(new Entry((float) (Math.random() * 9) + 0.45f, (ob.set).getEntryCount()), 0);
}
// °°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°
if (data.getXValCount() > 1400) {
System.out.println("X-ValueCount: " + ob.lineChart.getXValCount() + " EntryValueCount: " + ob.set.getEntryCount() + " ###############################");
// Test 01 °°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°
// for (int j = 0; j < 400; j++) {
// data.removeXValue(0);
// ob.set.removeEntry(0);
// }
// Test 02 °°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°
// runOnUiThread(new Runnable() {
// #Override
// public void run() {
// ob.lineChart.clearValues();
// }
// });
// Test 03 °°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°
// for (int j = 400; j > 0; j--) {
// data.removeXValue(j + 999);
// ob.set.removeEntry(j + 999);
// }
// Test 04 °°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°
// data.removeXValue(0);
// ob.set.removeEntry(0);
System.out.println("X-ValueCount: " + ob.lineChart.getXValCount() + " EntryValueCount: " + ob.set.getEntryCount() + " ###############################");
}
// °°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°
// notify chart data have changed
ob.lineChart.notifyDataSetChanged();
// limit number of visible entries
float range = 200f;
// ob.lineChart.setVisibleXRange(0f, range);
ob.lineChart.setVisibleXRangeMaximum(range);
// scroll the last entry
ob.lineChart.moveViewToX(data.getXValCount() - (range + 1f));
}
}
}
If I remove x-values from data or entries from set, several strange things happens. Sometimes the app gets crash, or the line of the charts begin to repeat itself, or sometimes I get more than one line per chart. I'm looking forward of a solution to remove old values from the chart.
EDIT 2: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
The output of my linechart is like the following picture...It moves like a realtime graph, until xValues an amount of reach 1400, now my programm begins to remove entries and my linechart get stuck. After a while, the line begin to dissapear from the left side. At picture 1 it get stuck, at the next three pictures you can see how it dissapears.

I've been stuck with exactly the same problem today as I'm writing an app that visualizes sensor data. I just came up with the following makeshift solution that works for me:
// (following code snippet is from an event listener)
final ArrayList<Float> values = event.getValues();
final LineData data = mChart.getData();
// add data entry
if (mIndex < MAX_PLOT_SIZE) {
maxValue = 0f;
data.addXValue("" + mIndex);
for (int i = 0; i < values.size(); i++) {
final Float val = values.get(i);
final LineDataSet set = data.getDataSetByIndex(i);
set.addEntry(new Entry(val, mIndex));
}
mIndex++;
} else {
// shift values down
for (int i = 0; i < values.size(); i++) {
final LineDataSet set = data.getDataSetByIndex(i);
final List<Entry> yVals = set.getYVals();
final int size = yVals.size();
for (int j = 0; j < size - 1; j++) {
yVals.get(j).setVal(yVals.get(j + 1).getVal());
}
// add new value
final Float val = values.get(i);
yVals.get(size - 1).setVal(val);
}
}
It's admittedly a hack and there should be an easier way to do it, but at least it gets the job done. I guess you could try and adjust the xVal-Strings in the same manner ...