I've recently started the AI-Class at Coursera and I've a question related to my implementation of the gradient descent algorithm.
Here's my current implementation (I actually just "translated" the mathematical expressions into Java code):
public class GradientDescent {
private static final double TOLERANCE = 1E-11;
private double theta0;
private double theta1;
public double getTheta0() {
return theta0;
}
public double getTheta1() {
return theta1;
}
public GradientDescent(double theta0, double theta1) {
this.theta0 = theta0;
this.theta1 = theta1;
}
public double getHypothesisResult(double x){
return theta0 + theta1*x;
}
private double getResult(double[][] trainingData, boolean enableFactor){
double result = 0;
for (int i = 0; i < trainingData.length; i++) {
result = (getHypothesisResult(trainingData[i][0]) - trainingData[i][1]);
if (enableFactor) result = result*trainingData[i][0];
}
return result;
}
public void train(double learningRate, double[][] trainingData){
int iteration = 0;
double delta0, delta1;
do{
iteration++;
System.out.println("SUBS: " + (learningRate*((double) 1/trainingData.length))*getResult(trainingData, false));
double temp0 = theta0 - learningRate*(((double) 1/trainingData.length)*getResult(trainingData, false));
double temp1 = theta1 - learningRate*(((double) 1/trainingData.length)*getResult(trainingData, true));
delta0 = theta0-temp0; delta1 = theta1-temp1;
theta0 = temp0; theta1 = temp1;
}while((Math.abs(delta0) + Math.abs(delta1)) > TOLERANCE);
System.out.println(iteration);
}
}
The code works quite well but only if I choose an very little alpha, here called learningRate. If it's higher than 0.00001, it diverges.
Do you have any suggestions on how to optimize the implementation, or an explanation for the "Alpha-Issue" and a possible solution for it?
Update:
Here's the main including some sample inputs:
private static final double[][] TDATA = {{200, 20000},{300, 41000},{900, 141000},{800, 41000},{400, 51000},{500, 61500}};
public static void main(String[] args) {
GradientDescent gd = new GradientDescent(0,0);
gd.train(0.00001, TDATA);
System.out.println("THETA0: " + gd.getTheta0() + " - THETA1: " + gd.getTheta1());
System.out.println("PREDICTION: " + gd.getHypothesisResult(300));
}
The mathematical expression of gradient descent is as follows:
To solve this issue, it's necessary to normalize the data with this formular: (Xi-mu)/s.
Xi is the current training set value, mu the average of values in the current column and s the maximum value minus the minimum value of the current column. This formula will get the training data approximately into a range between -1 and 1 which allowes to choose higher learning rates and gradient descent to converge faster.
But it's afterwards necessary to denormalize the predicted result.
private double getResult(double[][] trainingData, boolean enableFactor){
double result = 0;
for (int i = 0; i < trainingData.length; i++) {
result = (getHypothesisResult(trainingData[i][0]) - trainingData[i][1]);
if (enableFactor) result = result*trainingData[i][0];
}
return result;
In this func. result variable overwritten each iteration, the old value being lost. When inputing the values only the last item on array is calculating. Rest of them dont matter.
You should use java.math.BigDecimal for your arithematic operations.
double has its rounding-off issues while performing any arithematic.
Related
I've been making my own class library in Java and I've run into a small annoyance. The library is centered around math. I started with the intention of not using the Java Math Class. Unfortunately, my lack of skill paired with my inability to find a resource online that tackles this problem has resulted in me falling back onto the Java Math Class. Is there a way I can do logarithms without using Math.log?
We could try to use a power series as per: https://math.stackexchange.com/a/61283
For example:
#Test
public void printMathCalculation() {
double libValue = Math.log(100);
double myValue = getLn(100);
System.out.println("Actual: \t" + libValue);
System.out.println("Approximate: \t" + myValue);
}
//Take double to an integer power
private double pow(double num, int power) {
double result = 1;
for(int i = 0; i < power; i++) {
result *= num;
}
return result;
}
//Get natural log
private double getLn(double num) {
int accuracy = 1000;
double sum = 0;
for(int n = 0; n < accuracy; n++) {
double num1 = (1.0/(2*n+1));
double num2 = (num-1)/(num+1);
sum += num1*pow(num2,2*n+1);
}
return 2 * sum;
}
The results I get are:
Actual: 4.605170185988092
Approximate: 4.605170185988078
You should use Math.log
Try this, using simple maths to calculate
public static int toLog2N(int num){
return num>1 ? 1 + toLog2N(num/2) : 0;
}
Its just an example, you should use Math class as it provides different methods like log(double a), log10(double a), log1p(double a). It will make your code more readable and easier to understand.
hope it helps!
I am gathering 10 acceleration values from an on-board accelerometer on a mobile device. I am then attempting to normalize these values between the range of -1,1. I am unable to figure out why this isn't working correctly.
Here is the normalization code:
class NormUtil {
private double dataHigh;
private double dataLow;
private double normalizedHigh;
private double normalizedLow;
public NormUtil(double dataHigh, double dataLow) {
this(dataHigh, dataLow, 1, -1);
}
public NormUtil(double dataHigh, double dataLow, double normalizedHigh, double normalizedLow) {
this.dataHigh = dataHigh;
this.dataLow = dataLow;
this.normalizedHigh = normalizedHigh;
this.normalizedLow = normalizedLow;
}
public double normalize(double e) {
return ((e - dataLow)
/ (dataHigh - dataLow))
* (normalizedHigh - normalizedLow) + normalizedLow;
}
On a button press, the highest/lowest acceleration values are found in this code:
A = enrolAcc.get(0);
B = enrolAcc.get(0);
for (Float i: enrolAcc) {
if(i < A) A = i;
if(i > B) B = i;
}
Once the highest/lowest values are found a NormUtil instance is created and this instance is used to normalize the array of acceleration values and then add the normalized values to a new array:
NormUtil norm = new NormUtil(B,A,1,-1);
for(int j = 0; j < enrolAcc.size(); j++) {
double start = enrolAcc.get(j);
double x = norm.normalize(start);
nAcc[j] = x;
}
This nAcc array is then put into a string array and then a single string to display in a text view. The issue is the text view is always initialized with the original non-normalized acceleration values, here is the code I use for that:
String normD[] = new String[10];
for (int i = 0; i < 10; i++) {
normD[i] = String.valueOf(nAcc[i]);
}
StringBuilder strBuilder2 = new StringBuilder();
for (int i = 0; i<normD.length; i++) {
strBuilder2.append(normD[i] + ",");
}
normData = strBuilder.toString();
textNorm.setText("Normalised: " + normData);
So my question is, where am I going wrong with adding the normalized values to the normalized array and is this normalization method correct for what I am trying to achieve? Thanks in advance.
I am trying to make as generic as possible method for tweening between various types of values.
So, given a start and end value thats, say, either an Int,Float or Double as well as the number of steps (int), it will return values evenly distributed along those steps in the same type.
However, I am starting to suspect;
a) My knowledge of generics is terrible.
b) This might not be possible :(
So, just to be clear, one example;
SpiffyTween<Double> meep = new SpiffyTween<Double>(1d,10d, 100);
while (meep.hasNext()){
Log.info("value="+meep.next());
}
Would return 0.0,0.1,0.2..etc upto 9.9
But SpiffyTween could also work with other number types without needing separate code for each.
Heres the code I have right now;
class SpiffyTween<T extends Number> implements SpiffyGenericTween<T>
{
static Logger Log = Logger.getLogger("SpiffyTween <Number>");
private T start;
private T end;
int totalsteps=0;
int CurrentStep = 0;
ArrayList<T> steps = new ArrayList<T>();
public SpiffyTween(T start,T end, int steps) {
this.start = start;
this.end = end;
this.totalsteps = steps;
precalculate();
}
private void precalculate() {
//calc step difference
double dif = ((end.doubleValue() -start.doubleValue())/totalsteps);
Log.info("dif="+dif);
int i=0;
while(i<totalsteps){
T stepvalue = (T)((Number)(start.doubleValue() +(dif*i)));
steps.add(stepvalue);
Log.info("add step="+stepvalue);
i++;
}
}
public T next(){
T currentVal = steps.get(CurrentStep);
CurrentStep++;
return currentVal;
}
#Override
public boolean hasNext() {
if (CurrentStep<totalsteps){
return true;
}
return false;
}
}
This works...ish.
While the numbers come out aproximately right occasionally theres values like;
9.600000000000001
or
2.4000000000000004
I am assuming thats to do with the unchecked type conversion here;
T stepvalue = (T)((Number)(start.doubleValue() +(dif*i)));
But I cant work out how to do it better.
Whatever the solution (if theres one), my longterm plan is to try to make similar code that can also work on arrays of various number types. So, you could tween between 3 dimensional points by feeding it an array of the x/y/z co-ordinates of the start and end.
Also, possibly more relevantly, in the code example here its basic addition being done. I probably want other types of tweening possible, so that would make the maths more complex.
Is the better route to convert to, say, BigNumber, and then (somehow) back to the initial T later after all the processing is done?
Thanks in advance for any help or pointers.
YOu don't really need Generics to write code once. Consider the code below. Your exercise is to extend to other dimensions and to ensure caller does not use less than one step:
Tween Class
package com.example.stepup;
public class Tween {
public static int[] get1DimSteps (int start, int end, int steps) {
double[] preciseResult = get1DimSteps((double) start, (double) end, steps);
int[] result = new int[steps];
for (int i=0; i<steps; i++) {
result[i] = (int) (preciseResult[i] + 0.5D);
}
return result;
}
public static double[] get1DimSteps (float start, float end, int steps) {
double[] result = get1DimSteps((double)start, (double)end, steps);
return result;
}
public static double[] get1DimSteps (double start, double end, int steps) {
double distance;
double stepSize;
double[] result = new double[steps];
distance = end - start;
stepSize = distance / steps;
for (int i=0; i < steps; i++) {
result[i] = start + stepSize*i;
}
return result;
}
}
StepupTest Class
package com.example.stepup;
public class StepupTest {
public static void main(String[] args) {
// get steps from "start" to "finish"
int startI = -1;
int endI =999;
float start = (float) startI;
float end = (float) endI;
double startD = (double) startI;
double endD = (double) endI;
int numberOfSteps = 100;
double[] steps = Tween.get1DimSteps( start, end, numberOfSteps);
double[] stepsD = Tween.get1DimSteps(startD, endD, numberOfSteps);
int[] stepsI = Tween.get1DimSteps(startI, endI, numberOfSteps);
for (int i=0; i < numberOfSteps; i++) {
System.out.println(" " + i + ". " + steps[i] + ", " + stepsD[i] + ", " + stepsI[i]);
}
}
}
I know about the Math.sin() and Math.cos() functions, but I'm wondering if there's a way I can create (or use an already-existing) a faster function, given that I don't care about pinpoint accuracy. I'm looking to execute a basic sin or cos calculation, and have it perform essentially as fast as possible. Would simply iterating the sigma a few times be any faster than Math.sin()?
Since you don't care much about accuracy store it in a table that is precomputed or only computed once, this is what I do when I want to avoid calls to Math which can be expensive when done alot.
Roughly
public class CosSineTable {
double[] cos = new double[361];
double[] sin = new double[361];
private static CosSineTable table = new CosSineTable();
private CosSineTable() {
for (int i = 0; i <= 360; i++) {
cos[i] = Math.cos(Math.toRadians(i));
sin[i] = Math.sin(Math.toRadians(i));
}
}
public double getSine(int angle) {
int angleCircle = angle % 360;
return sin[angleCircle];
}
public double getCos(int angle) {
int angleCircle = angle % 360;
return cos[angleCircle];
}
public static CosSineTable getTable() {
return table;
}
}
I leave the optimization of the loop and methods to you.
A pre-calculated table's the way to go. Here's an implementation:
static final int precision = 100; // gradations per degree, adjust to suit
static final int modulus = 360*precision;
static final float[] sin = new float[modulus]; // lookup table
static {
// a static initializer fills the table
// in this implementation, units are in degrees
for (int i = 0; i<sin.length; i++) {
sin[i]=(float)Math.sin((i*Math.PI)/(precision*180));
}
}
// Private function for table lookup
private static float sinLookup(int a) {
return a>=0 ? sin[a%(modulus)] : -sin[-a%(modulus)];
}
// These are your working functions:
public static float sin(float a) {
return sinLookup((int)(a * precision + 0.5f));
}
public static float cos(float a) {
return sinLookup((int)((a+90f) * precision + 0.5f));
}
On my laptop, these were about 6x faster than Math.sin.
I only used one table -- the cost of shifting a cosine into a sine wasn't really discernible.
I used floats, assuming that's what you'll likely use in your calculations, given your preference for performance over precision. It doesn't make much difference here, since the bottleneck is really just the array lookup.
Here are my benchmarks:
public static void main(String[] args) {
int reps = 1<<23;
int sets = 4;
Q.pl(" Trial sinTab cosTab sinLib");
for(int i = 0; i<sets; i++) {
Q.pf("%7d\t%7.2f\t%7.2f\t%7.2f\n", i, testSinTab(reps), testCosTab(reps), testSinLib(reps));
}
}
private static float[] sample(int n) {
Random rand = new Random();
float[] values = new float[n];
for (int i=0; i<n; i++) {
values[i] = 400*(rand.nextFloat()*2-1);
}
return values;
}
private static float testSinTab(int n) {
float[] sample = sample(n);
long time = -System.nanoTime();
for (int i=0; i<n; i++) {
sample[i] = sin(sample[i]);
}
time += System.nanoTime();
return (time/1e6f);
}
private static float testCosTab(int n) {
float[] sample = sample(n);
long time = -System.nanoTime();
for (int i=0; i<n; i++) {
sample[i] = cos(sample[i]);
}
time += System.nanoTime();
return time/1e6f;
}
private static float testSinLib(int n) {
float[] sample = sample(n);
long time = -System.nanoTime();
for (int i=0; i<n; i++) {
sample[i] = (float) Math.sin(sample[i]);
}
time += System.nanoTime();
return time/1e6f;
}
output:
Trial sinTab cosTab sinLib
0 102.51 111.19 596.57
1 93.72 92.20 578.22
2 100.06 107.20 600.68
3 103.65 102.67 629.86
You can try
http://sourceforge.net/projects/jafama/
It uses look-up tables, so it might actually be slower
than Math, especially if the tables are often evicted from CPU cache,
but for thousands of successive calls it can be quite faster.
It also seems slower during class load (maybe the JIT doesn't kicks in then yet),
so you might want to avoid it in that particular use-case.
I know this question is old, but I think it's the fastest java implementation sintable with precision to 65536 elements.
public class MathHelper {
private static double[] a = new double[65536];
public static final double sin(float f) {
return a[(int) (f * 10430.378F) & '\uffff'];
}
public static final double cos(float f) {
return a[(int) (f * 10430.378F + 16384.0F) & '\uffff'];
}
static {
for (int i = 0; i < 65536; ++i) {
a[i] = Math.sin((double) i * 3.141592653589793D * 2.0D / 65536.0D);
}
}
}
Source: https://github.com/Bukkit/mc-dev/blob/master/net/minecraft/server/MathHelper.java
I'm having difficulty writing a program to solve this exercise from a Java text book:
Write a method raiseRealToPower that takes a floating-point value x and an integer
k and returns xk. Implement your method so that it can correctly calculate the result
when k is negative, using the relationship
x^(-k) = 1 / x^k.
Use your method to display a table of values of πk for all values of k from –4 to 4.
I didn't done this part with PI, i know that, if my programs starts to work... this is what i done... tell me please, what is wrong.
import acm.program.*;
public class vjezba55 extends ConsoleProgram {
private static final double PI = 3.14159253;
public void run() {
double x = readDouble ("x: ");
double k = readDouble ("k: ");
println ("x^k = " + raiseDoublePower(x,k));
}
/* Method that counts x^k */
private double raiseDoublePower (double x, double k){
if (k >= 0) {
return Math.pow(x, k);
}
else {
double total = 1;
for (int i= 0; i>k; i--) {
total = (double) 1 / x;
}
return total;
}
}
}
Take a look at your loop code. You are just recalculating total from scratch on each iteration, rather than updating the previous result.
I don't understand the part in the question regarding PI, but your method may be much simpler (according to using the relationship x^(-k) = 1 / x^k):
private double raiseDoublePower (double x, double k){
if (k >= 0) {
return Math.pow(x, k);
}
else {
return 1 / Math.pow(x, -k);
}
}