Develop Reference

dl4j lstm not successful

Im trying to copy the exrcise about halfway down the page on this link:
https://d2l.ai/chapter_recurrent-neural-networks/sequence.html
The exercise uses a sine function to create 1000 data points between -1 through 1 and use a recurrent network to approximate the function.
Below is the code I used. I'm going back to study more why this isn't working as it doesn't make much sense to me now when I was easily able to use a feed forward network to approximate this function.
//get data
ArrayList<DataSet> list = new ArrayList();
DataSet dss = DataSetFetch.getDataSet(Constants.DataTypes.math, "sine", 20, 500, 0, 0);
DataSet dsMain = dss.copy();
if (!dss.isEmpty()){
list.add(dss);
}
if (list.isEmpty()){
return;
}
//format dataset
list = DataSetFormatter.formatReccurnent(list, 0);
//get network
int history = 10;
ArrayList<LayerDescription> ldlist = new ArrayList<>();
LayerDescription l = new LayerDescription(1,history, Activation.RELU);
ldlist.add(l);
LayerDescription ll = new LayerDescription(history, 1, Activation.IDENTITY, LossFunctions.LossFunction.MSE);
ldlist.add(ll);
ListenerDescription ld = new ListenerDescription(20, true, false);
MultiLayerNetwork network = Reccurent.getLstm(ldlist, 123, WeightInit.XAVIER, new RmsProp(), ld);
//train network
final List<DataSet> lister = list.get(0).asList();
DataSetIterator iter = new ListDataSetIterator<>(lister, 50);
network.fit(iter, 50);
network.rnnClearPreviousState();
//test network
ArrayList<DataSet> resList = new ArrayList<>();
DataSet result = new DataSet();
INDArray arr = Nd4j.zeros(lister.size()+1);
INDArray holder;
if (list.size() > 1){
//test on training data
System.err.println("oops");
}else{
//test on original or scaled data
for (int i = 0; i < lister.size(); i++) {
holder = network.rnnTimeStep(lister.get(i).getFeatures());
arr.putScalar(i,holder.getFloat(0));
}
}
//add originaldata
resList.add(dsMain);
//result
result.setFeatures(dsMain.getFeatures());
result.setLabels(arr);
resList.add(result);
//display
DisplayData.plot2DScatterGraph(resList);
Can you explain the code I would need for a 1 in 10 hidden and 1 out lstm network to approximate a sine function?
Im not using any normalization as function is already -1:1 and Im using the Y input as the feature and the following Y Input as the label to train the network.
You notice i am building a class that allows for easier construction of nets and I have tried throwing many changes at the problem but I am sick of guessing.
Here are some examples of my results. Blue is data red is result

This is one of those times were you go from wondering why was this not working to how in the hell were my original results were as good as they were.
My failing was not understanding the documentation clearly and also not understanding BPTT.
With feed forward networks each iteration is stored as a row and each input as a column. An example is [dataset.size, network inputs.size]
However with recurrent input its reversed with each row being a an input and each column an iteration in time necessary to activate the state of the lstm chain of events. At minimum my input needed to be [0, networkinputs.size, dataset.size] But could also be [dataset.size, networkinputs.size, statelength.size]
In my previous example I was training the network with data in this format [dataset.size, networkinputs.size, 1]. So from my low resolution understanding the lstm network should never have worked at all but somehow produced at least something.
There may have also been some issue with converting the dataset to a list as I also changed how I feed the network but but I think the bulk of the issue was a data structure issue.
Below are my new results

Hard to tell what is going on without seeing the full code. For a start I don't see an RnnOutputLayer specified. You could take a look this which shows you how to build an RNN in DL4J.
If your RNN setup is correct this could be a tuning issue. You can find more on tuning here. Adam is probably a better choice for an updater than RMSProp. And tanh probably is a good choice for the activation for your output layer since it's range is (-1,1). Other things to check/tweak - learning rate, number of epochs, set up of your data (like are you trying to predict to far out?).

I want to make a graph with sensing data

I need to make a graph with data which come from arduino.
Data is sended as String and I want to draw graph with parts of the data
For example,
Arduino sents "1234567890", Graph will be drawn by "12345"
Here is my codes
ArrayList<Integer> colors = new ArrayList<>();
ArrayList<String> test1 = new ArrayList<>();
ArrayList<Entry> value1 = new ArrayList<>();
test1.add("123456909090");
test1.add("234567909090");
test1.add("334567909090");
test1.add("434567909090");
for (int i = 0; i < 4; i++){
String a = test1.get(i);
a.substring(0,6);
float b = Float.parseFloat(a);
value1.add(new Entry(i,b));
}
ScatterDataSet set1 = new ScatterDataSet(value1);
and there is an exception raised
FATAL EXCEPTION:java.lang.ArithmeticException: divide by zero
I have never used devide.
How can i solve it?
And If there is a better way, let me know.
Thanks for reading.

As far as I see, when you resive data, before use data you should check them with any control mechanism like if ... etc. it will help you. In my limited programming knowledge zero division makes problem ( any number / 0 == is problem) if you have to do this operations I prefer change the value like 0.0001 or 0.000001 or you can't use the data which was sended and wait , while you waiting you can use old data. It depens you and your project....

Weka in Java - How to get predictions for IBk or KStar or LWL or

I've searched all over stackoverflow and google for these kind predicitons but found nothing for IBk or KStar or LWL. I would need one instance predictions from any of these three clasifiers.I am doing this in Android studio.
I've found ways of getting predictions from other classifiers like these:
for J48: from Here
double[] prediction=j48.distributionForInstance(test.get(s1));
//output predictions
for(int i=0; i<prediction.length; i=i+1)
{
System.out.println("Probability of class "+
test.classAttribute().value(i)+
" : "+Double.toString(prediction[i]));
}
For Bayesnet: from Here
Evaluation eTest = new Evaluation(trainingInstance);
eTest.evaluateModelOnce(bayes_Classifier, testInstance);
For NaiveBayes: from Here
NaiveBayes naiveBayes = new NaiveBayes();
naiveBayes.buildClassifier(train);
// this does the trick
double label = naiveBayes.classifyInstance(test.instance(0));
test.instance(0).setClassValue(label);
System.out.println(test.instance(0).stringValue(4));
but I couldn't use them because my classifiers don't have the same methods...or I can't find a way
My code:
//I skipped code till here because its too much,
//but Data is definetly inside *instances* (I checked with debuger)
instances.setClassIndex(instances.numAttributes()-1);
//was trying the sam with KStar, LWL, AdditiveRegression, RandomCommittee)
IBk ibk = new IBk();
//I want predicitons for this instance. For the third attribute3
Instance testInst = new DenseInstance(3);
testInst.setValue(attribute1, 3);
testInst.setValue(attribute2, 16);
testInst.setValue(attribute3, 0);
//I was hopping for some simple way like this: (but this returns nothing)
double rez =0;
String var="";
try{
ibk.buildClassifier(instances);
rez = ibk.classifyInstance(testInst);
}
catch(Exception ex)
{
Log.e("Error","ex.getMessage()");
}
}
Log.w("GIMME RESULTS:",rez);
Even other classifiers would be okay like AdditiveRegression, Bagging, RandomCommitte and DecisionTable they make good prediction in Weka GUI, but I need predictions in Java.... :)

found it by testing all its methods..
ibk.buildClassifier(dataSet);
rez2 = ibk.distributionForInstance(i2); //distrib
int result = (int)rez3[0];
//it goes tha same with Kstar
Came to realize that classifiers in weka normaly run with discrete data (equal steps from min to max). And my data is not all discrete. Ibk and Kstar are able to use distributed data thats why I can use only these two with my data.

String naming convention in java

I am currently trying to make a naming convention. The idea behind this is parsing.
Lets say I obtain an xml doc. Everything can be used once, but these 2 in the code below can be submitted several times within the xml document. It could be 1, or simply 100.
This states that ItemNumber and ReceiptType will be grabbed for the first element.
ItemNumber1 = eElement.getElementsByTagName("ItemNumber").item(0).getTextContent();
ReceiptType1 = eElement.getElementsByTagName("ReceiptType").item(0).getTextContent();
This one states that it will grab the second submission if they were in their twice.
ItemNumber2 = eElement.getElementsByTagName("ItemNumber").item(1).getTextContent();
ReceiptType2 = eElement.getElementsByTagName("ReceiptType").item(1).getTextContent();
ItemNumber and ReceiptType must both be submitted together. So if there is 30 ItemNumbers, there must be 30 Receipt Types.
However now I would like to set this in an IF statement to create variables.
I was thinking something along the lines of:
int cnt = 2;
if (eElement.getElementsByTagName("ItemNumber").item(cnt).getTextContent();)
**MAKE VARIABLE**
Then make a loop which adds one to count to see if their is a third or 4th. Now here comes the tricky part..I need them set to a generated variable. Example if ItemNumber 2 existed, it would set it to
String ItemNumber2 = eElement.getElementsByTagName("ItemNumber").item(cnt).getTextContent();
I do not wish to make pre-made variable names as I don't want to code a possible 1000 variables if that 1000 were to happen.
KUDOS for anyone who can help or give tips on just small parts of this as in the naming convention etc. Thanks!

You don't know beforehand how many ItemNumbers and ReceiptTypes you'll get ? Maybe consider using two Lists (java.util.List). Here is an example.
boolean finished = ... ; // true if there is no more item to process
List<String> listItemNumbers = new ArrayList<>();
List<String> listReceiptTypes = new ArrayList<>();
int cnt = 0;
while(!finished) {
String itemNumber = eElement.getElementsByTagName("ItemNumber").item(cnt).getTextContent();
String receiptType = eElement.getElementsByTagName("ReceiptType").item(cnt).getTextContent();
listItemNumbers.add(itemNumber);
listReceiptTypes.add(receiptType);
++cnt;
// update 'finished' (to test if there are remaining itemNumbers to process)
}
// use them :
int indexYouNeed = 32; // for example
String itemNumber = listItemNumbers.get(indexYouNeed); // index start from 0
String receiptType = listReceiptTypes.get(indexYouNeed);

Null Pointer that makes no sense to me?

Im currently working on a program and any time i call Products[1] there is no null pointer error however, when i call Products[0] or Products[2] i get a null pointer error. However i am still getting 2 different outputs almost like there is a [0] and 1 or 1 and 2 in the array. Here is my code
FileReader file = new FileReader(location);
BufferedReader reader = new BufferedReader(file);
int numberOfLines = readLines();
String [] data = new String[numberOfLines];
Products = new Product[numberOfLines];
calc = new Calculator();
int prod_count = 0;
for(int i = 0; i < numberOfLines; i++)
{
data = reader.readLine().split("(?<=\\d)\\s+|\\s+at\\s+");
if(data[i].contains("input"))
{
continue;
}
Products[prod_count] = new Product();
Products[prod_count].setName(data[1]);
System.out.println(Products[prod_count].getName());
BigDecimal price = new BigDecimal(data[2]);
Products[prod_count].setPrice(price);
for(String dataSt : data)
{
if(dataSt.toLowerCase().contains("imported"))
{
Products[prod_count].setImported(true);
}
else{
Products[prod_count].setImported(false);
}
}
calc.calculateTax(Products[prod_count]);
calc.calculateItemTotal(Products[prod_count]);
prod_count++;
This is the output :
imported box of chocolates
1.50
11.50
imported bottle of perfume
7.12
54.62
This print works System.out.println(Products[1].getProductTotal());
This becomes a null pointer System.out.println(Products[2].getProductTotal());
This also becomes a null pointer System.out.println(Products[0].getProductTotal());

You're skipping lines containing "input".
if(data[i].contains("input")) {
continue; // Products[i] will be null
}
Probably it would be better to make products an ArrayList, and add only the meaningful rows to it.
products should also start with lowercase to follow Java conventions. Types start with uppercase, parameters & variables start with lowercase. Not all Java coding conventions are perfect -- but this one's very useful.
The code is otherwise structured fine, but arrays are not a very flexible type to build from program logic (since the length has to be pre-determined, skipping requires you to keep track of the index, and it can't track the size as you build it).
Generally you should build List (ArrayList). Map (HashMap, LinkedHashMap, TreeMap) and Set (HashSet) can be useful too.
Second bug: as Bohemian says: in data[] you've confused the concepts of a list of all lines, and data[] being the tokens parsed/ split from a single line.
"data" is generally a meaningless term. Use meaningful terms/names & your programs are far less likely to have bugs in them.
You should probably just use tokens for the line tokens, not declare it outside/ before it is needed, and not try to index it by line -- because, quite simply, there should be absolutely no need to.
for(int i = 0; i < numberOfLines; i++) {
// we shouldn't need data[] for all lines, and we weren't using it as such.
String line = reader.readLine();
String[] tokens = line.split("(?<=\\d)\\s+|\\s+at\\s+");
//
if (tokens[0].equals("input")) { // unclear which you actually mean.
/* if (line.contains("input")) { */
continue;
}
When you offer sample input for a question, edit it into the body of the question so it's readable. Putting it in the comments, where it can't be read properly, is just wasting the time of people who are trying to help you.

Bug alert: You are overwriting data:
String [] data = new String[numberOfLines];
then in the loop:
data = reader.readLine().split("(?<=\\d)\\s+|\\s+at\\s+");
So who knows how large it is - depends on the success of the split - but your code relies on it being numberOfLines long.

You need to use different indexes for the line number and the new product objects. If you have 20 lines but 5 of them are "input" then you only have 15 new product objects.
For example:
int prod_count = 0;
for (int i = 0; i < numberOfLines; i++)
{
data = reader.readLine().split("(?<=\\d)\\s+|\\s+at\\s+");
if (data[i].contains("input"))
{
continue;
}
Products[prod_count] = new Product();
Products[prod_count].setName(data[1]);
// etc.
prod_count++; // last thing to do
}