Develop Reference

How are the numbers stored, called, and added in this short recursion example?

First off, it's not homework. I'm practicing examples from http://codingbat.com/java/Recursion-1. Every time I think I'm beginning to understand recursion, I run into a problem that makes me realize I have no idea, and the only thing tutors or internet explanations ever say is that "the function calls itself until the base case is met."
public int sumDigits(int n) {
if(n < 10) {
return n;
} else return sumDigits(n/10) + n % 10;
}
If I pass 115 to this on my computer, the output is 7 (as it should), but I don't understand how the program comes to this conclusion. Here is how I see it:
115 is not less than 10, so return the program with 115/10 (which is 11). 11 is not less than 10, so return the program with 11/10 (which is 1). Add this to 115%10 (which is 5). So how does this program get 7???
It seems like no matter how many examples I look at, I cannot find a pattern in how this works.

You may find it easier to reason about this code if you swap the two operands of the +, that is, change it to read:
return (n % 10) + sumDigits(n / 10)
i.e.
sumDigits(115) = 5 + sumDigits(11)
sumDigits(11) = 1 + sumDigits(1)
sumDigits(1) = 1
so expanding that:
sumDigits(115) = 5 + (1 + sumDigits(1))
= 5 + 1 + 1
= 7

Here is how it happens explained with a diagram

You have it almost correct, you just missed one step when returning out of the recursion - when returning from the 11/10, you add another 1:
call for 115
115 > 10, call recursively for 11
11 > 10, call recursively for 1
1 < 10, return 1
return the 1 from recursive invocation + 11%10 = 1 + 1 = 2
return the 2 from recursive invocation + 115%10 = 2 + 5 = 7

Big-O complexity of a recursive solution

I have a simple recursive solution as below:
public int countPaths(int x, int y) {
if(x == 0 && y == 0) {
return 0;
} else if(x == 0) {
return 1;
} else if(y == 0) {
return 1;
} else {
int count = countPaths(x-1, y);
count += countPaths(x, y-1);
return count;
}
}
This is to solve the following problem from the book: Cracking the coding interview
Imagine a robot sitting on the upper left corner of an X by Y grid. The robot can only move in two directions: right and down. How many possible paths are there for the robot to go from (0,0) to (X,Y)?
I am trying to ascertain the run time complexity and I believe it is O(x+y). I arrived at this by using a recursion tree, for example if x=2 and y=2
The max depth of this tree is (x+y) and work done at each step is a constant. So max work done is (x+y) * c and hence the run time complexity is O(x+y)
Question 1: Am I correct? I believe the upper bound I have calculated is not tight enough
Question 2: Next, if I were to improve the run time using memoization and hence not repeating computing sub-problems, how would the run time complexity as described by Big-o change?

While it's true that the depth of the tree is O(x+y), there's increasingly many nodes at each layer, and it's the number of nodes that determines complexity, not the depth of the tree.
If you write down recurrence relations for the runtime, you get:
T(0, y) = T(x, 0) = 1
T(x, y) = T(x-1, y) + T(x, y-1) + 1
If you ignore the +1 on the second equation (which can only make the run-time better), you get the same function that your code was computing in the first place, which is choose(x+y, y).
For x=y, this is the central binomial coefficient, which is approximately 4^x/sqrt(pi*x), which for even moderately large values of x is large enough to make the algorithm useless.
With memoisation, you're doing a constant amount of work for each value of x and y, so the complexity is O(xy).

If you evaluate the complexity in terms of the number of additions required to evaluate the count for a given pair (x, y), you get the recurrence
A(x,y) = A(x-1,y) + A(x,y-1) + 1,
with A(x,0) = A(0,y) = 0.
Setting A(x,y) = P(x,y) - 1 the recurrence becomes
P(x,y) - 1 = P(x-1,y) - 1 + P(x,y-1) - 1 + 1,
or
P(x,y) = P(x-1,y) + P(x,y-1),
with P(x,0) = P(0,y) = 1, which gives the classical Pascal's triangle, and
A(x,y) = (x+y)!/(x!y!) - 1.
You can also work with the number of recursive function calls,
C(x,y) = C(x-1,y) + C(x,y-1) + 2,
with C(0,y) = C(x,0) = 0.
You will solve it by setting C(x,y) = 2P(x,y) - 2, and get
C(x,y)= 2(x+y)!/(x!y!)-2.
As regards the asymptotic complexity, this makes no difference. The is no really simpler formula than O((x+y)!/x!y!).
With memoization, every evaluation (with x, y>0) costs only one addition or two calls, and assuming constant time for storage/retrieval of a value the total complexity is the much better O(xy).

Based on the valuable input from #Anonymous, we know the recurrence relation is:
T(x, y) = T(x-1, y) + T(x, y-1) + 1
Abusing (which is ok in Big-O analysis) the notation, let x = y
T(x, x) = 2 * T(x-1, x) = 2 * 2 * T(x-2, x) = ... = 2 * ... * 2 * T(0, x)
= O(2^x)
So the run time complexity is
O(2^n) ; where n = max(x, y)
With memoization, I get it, thank #Anonymous, it should be O(xy)

Trouble with ProjectEuler #15 [closed]

Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 8 years ago.
Improve this question
Here's the problem: https://projecteuler.net/problem=15
I've come up with a pattern which I thought would work for this and I've looked at what other people have done and they've done the same thing, such as here: http://code.jasonbhill.com/python/project-euler-problem-15/ But I always get a different answer. Here's my code.
import java.util.*;
public class Problem15 {
public static void main(String[] args) {
ArrayList<Long> list = new ArrayList<Long>();
list.add((long)1);
int size;
for (int x = 1;x<20;x++){
size = list.size();
for(int y = 1;y<size;y++){
long sum = list.get(y-1)+list.get(y);
list.set(y, sum);
}
list.add(list.get(size-1)*2);
System.out.println(list);
}
}
}
edit:
In response to Edward, I think my method is currently what you said before your edit in that this isn't about brute force but I'm just summing the possible ways from each point in the grid. However, I don't need a 2d array to do this because I'm only looking at possible moves from only the side. Here's something I drew up to hopefully explain my process.
So for a 1x1. Like you said, once you reach the limit of one direction, you can only travel in the limit of the other, so there's 2 ways. This isn't particularly helpful for a 1x1 but it is for larger ones. For a 2x2, you know that the top corner, being the limit of right, only has 1 possible path from that point. The same logic applies to the bottom corner. And, because you have a square which you have already solved for, a 1x1, you know that the middle point has 2 paths from there. Now, if you look at the sides, you see that the point for instance that has 2 beneath it and 1 to the right has the sum of the number of paths in those adjacent points so then that point must have 3 paths. Same for the other side, giving the top left corner the sum of 3 and 3, or 2 times 3.
Now if you look at my code, that's what it's trying to do. The element with index 0 is always 1, and for the rest of the array, it adds together the previous term and itself and replaces the current term. Lastly, to find the total number of paths, it just doubles the last number. So if the program were to try and solve for a 4x4, the array would currently look like {1, 4, 10, 20}. So the program would change it to {1, 5, 10, 20}, then {1, 5, 15, 20}, then {1, 5, 15, 35}, and finally, adds the total number of paths, {1, 5, 15, 35, 70}. I think this is what you were trying to explain to me in your answer however my answer always comes out incorrect.

Realize that it's more about mathematical complexity than brute force searching.
You have a two dimensional array of points, where you can chose to only travel away from the origin in the x or the y direction. As such, you can represent your travel like so:
(0, 0), (1, 0), (1, 1), (2, 1), (2, 2)
some things become immediately obvious. The first one is that any path through the mess is going to require x + y steps, travelling through x + y + 1 locations. It is a feature of a Manhattan distance style path.
The second is that at any one point until you hit the maximum x or y, you can select either of the two options (x or y); but, as soon as one or the other is at it's limit, the only option left is to chose the non-maximum value repeatedly until it also becomes a maximum.
With this you might have enough of a hint to solve the math problem. Then you won't even need to search through the different paths to get an algorithm that can solve the problem.
--- edited to give a bit more of a hint ---
Each two dimensional array of paths can be broken down into smaller two dimensional arrays of paths. So the solution to f(3, 5) where the function f yields the number of paths is equal to f(2, 5) + f(3, 4). Note that f(0, 5) directly equals 1, as does f(3, 0) because you no longer have "choices" when the paths are forced to be linear.
Once you model the function, you don't even need the array to walk the paths....
f(1, 1) = f(0, 1) + f(1, 0)
f(0, 1) = 1
f(1, 0) = 1
f(1, 1) = 1 + 1
f(1, 1) = 2
and for a set of 3 x 3 verticies (like the example cited has)
f(2, 2) = f(1, 2) + f(2, 1)
f(1, 2) = f(0, 1) + f(1, 1)
(from before)
f(1, 1) = 2
f(0, 2) = 1
f(1, 2) = 2 + 1 = 3
likewise (because it's the mirror image)
f(2, 1) = 1 + 2 = 3
so
f(2, 2) = 3 + 3 = 6
--- last edit (I hope!) ---
Ok, so now you may get the idea that you have really two choices (go down) or (go right). Consider a bag containing four "commands", 2 of "go down" and 2 of "go right". How many different ways can you select the commands from the bag?
Such a "selection" is a permutation, but since we are selecting all of them, it is a special type of permutation called an "order" or "ordering".
The number of binomial (one or the other) orderings is ruled by the mathematical formula
number of orderings = (A + B)!/(A! * B!)
where A is the "count" of items of type A, and B is the "count" of items of type B
3x3 vertices, 2 down choices, 2 right choices
number of orderings = (2+2)!/2!*2!
4!/1*2*1*2
1*2*3*4/1*2*1*2
(1*2)*3*4/(1*2)*1*2
3*4/2
12/2
6
You could probably do a 20*20 by hand if you needed, but the factorial formula is simple enough to do by computer (although keep an eye you don't ruin the answer with an integer overflow).

Another implementation:
public static void main(String[] args) {
int n = 20;
long matrix[][] = new long[n][n];
for (int i = 0; i < n; i++) {
matrix[i][0] = i + 2;
matrix[0][i] = i + 2;
}
for (int i = 1; i < n; i++) {
for (int j = i; j < n; j++) { // j>=i
matrix[i][j] = matrix[i - 1][j] + matrix[i][j - 1];
matrix[j][i] = matrix[i][j]; // avoids double computation (difference)
}
}
System.out.println(matrix[n - 1][n - 1]);
}
Time: 43 microseconds (without printing)
It is based on the following matrix:
| 1 2 3 4 ...
---------------------
1 | 2 3 4 5 ...
2 | 3 6 10 15
3 | 4 10 20 35
4 | 5 15 35 70
. | .
. | .
. | .
where
6 = 3 + 3
10 = 6 + 4
15 = 10 + 5
...
70 = 35 + 35
Notice that I used i + 2 instead of i + 1 in the implementation because the first index is 0.
Of course, the fastest solution is to use a mathematical formula (see Edwin's post) and the code for it:
public static void main(String[] args) {
int n = 20;
long result = 1;
for ( int i = 1 ; i <= n ; i++ ) {
result *= (i+n);
result /= i;
}
System.out.println(result);
}
takes only 5 microseconds (without printing).
If you are afraid about the loss of precision, notice that the product of n consecutive numbers is divisible by n!.
To have a better understanding why the formula is:
(d+r)!
F = --------- , where |D| = d and |R| = r
d!*r!
instead of F = (d+r)!, imagine that every "down" and "right" has an index:
down1,right1,right2,down2,down3,right3
The second formula counts all possible permutations for the "commands" above, but in our case there is no difference between down1, down2 and down3. So, the second formula will count 6 (3!) times the same thing:
down1,down2,down3
down1,down3,down2
down2,down1,down3
down2,down3,down1
down3,down1,down2
down3,down2,down1
This is why we divide the (d+r)! by d!. Analogue for r!.

A mathematical function that gets us the number of leaves of a specific type of k-ary?

I am trying to figure out a function f(x) that would calculate the number of leaves in a k-ary tree. For example, assume we created a tree that began with root 4 with 3 children, each of -1,-2,-3 respectively. Our leaves would only be 0 values, not null values. I have spent the past day trying to figure out a function and it seems like nothing I do goes in the correct direction.
EX:
4
/ | \
3 2 1
/ |\ /| /
2 1 0 1 0 0
/| / /
1 0 0 0
/
0
7 Leaves.
Any help would be very much appreciated! Thanks!
To clarify, I need a mathematical equation that derives the same answer as code would if I recursively transversed the tree.
More examples:
{4,7}{5,13}{6,24}{7,44}{8,81}{9,149}{10,274}{11,504}{12,927}{13,1705}{14,3136}{15,5768}{16,10609}{17,19513}{18,35890}{19,66012}{20,121415}
public int numleaves(TreeNode node) {
if (node == null)
return 0;
else if (node.getLeft() == null && node.getMiddle() == null && node.getRight() == null)
return 1;
else
return numleaves(node.getLeft()) + numleaves(node.getMiddle()) + numleaves(node.getRight());
}

I cannot answer your question, but it has a solution. I can only outline the case for the number of children k being equal to 2. The case k=3 leads to a cubic polynomial with two complex and one real solution, I lack the tools here to derive them in a non-numerical way.
But let's have a look at the case k=2. Interestingly, this problem is very closely related to the Fibonacci numbers, except for having different boundary conditions.
Writing down the recursive formula is easy:
a(n) = a(n-1) + a(n-2)
with boundary conditions a(1)=1 and a(0)=1. The characteristic polynomial of this is
x^2 = x + 1
with the solutions x1 = 1/2 + sqrt(5)/2 and x2 = 1/2 - sqrt(5)/2. It means that
a(n) = u*x1^n + v*x2^n
for some u and v is the explicit formula for the sequence we're looking for. Putting in the boundary conditions we get
u = (sqrt(5)+1)/(2*sqrt(5))
v = (sqrt(5)-1)/(2*sqrt(5))
i.e.
a(n) = (sqrt(5)+1)/(2*sqrt(5))*(1/2 + sqrt(5)/2)^n + (sqrt(5)-1)/(2*sqrt(5))*(1/2 - sqrt(5)/2)^n
for k=2.

Your code seems to be computing a Tribonacci sequence with starting values 1, 1 and 2. This is sequence A000073 from the On-Line Encyclopedia of Integer Sequences, starting from the third entry of that sequence rather than the first. The comments section of the encyclopedia page gives an explicit formula: since this is a linear recurrence relation with a degree 3 characteristic polynomial, there's a closed form solution in terms of the roots of that polynomial. Here's a short piece of Python 2 code based on the given formula that produces the first few values. (See the edit below for a simplification.)
from math import sqrt
c = (1 + (19 - 3 * sqrt(33))**(1/3.) + (19 + 3 * sqrt(33))**(1/3.)) / 3.
m = (1 - c) / 2
p = sqrt(((3*c - 5)*(c+1)/4))
j = 1/((c-m)**2 + p**2)
b = (c - m) / (2 * p*((c - m)**2 + p**2))
k = complex(-j / 2, b)
r1 = complex(m, p)
def f(n):
return int(round(j*c**(n+2) + (2*k*r1**(n+2)).real))
for n in range(0, 21):
print n, f(n)
And the output:
0 1
1 1
2 2
3 4
4 7
5 13
6 24
7 44
8 81
9 149
10 274
11 504
12 927
13 1705
14 3136
15 5768
16 10609
17 19513
18 35890
19 66012
20 121415
EDIT: the above code is needlessly complicated. With the round operation, the second term in f(n) can be omitted (it converges to zero as n increases), and the formula for the first term can be simplified. Here's some simpler code that generates the same output.
s = (19 + 297**0.5)**(1/3.)
c = (1 + s + 4/s)/3
j = 3 - (2 + 1/c)/c
for n in range(0, 32):
print n, int(round(c**n / j))

I can't help it, but I see Binomial tree in it. http://en.wikipedia.org/wiki/Binomial_heap
I think that good approximation could be sum of k-th row of pascal triangle, where k stands for the number of the root node.

Isn't this easier to understand:
We set the starting values for the tribonacci sequence into a list called result. Then we put these values into 3 variables. We change the variable content based on the tribonacci formula (new a is a+b+c, new b is old a, new c is old b). Then we calculate to whatever tribonacci number we want to go up to and store each result into our result list. At the end, we read out the indexed list.
result=[1,1,2]
a,b,c=result[-1],result[-2],result[-3]
for i in range(40):
a,b,c=a+b+c,a,b
result.append(a)
for e,f in enumerate(result):
print e,f

How can I estimate time complexity from a table of values?

I know that my naive matrix multiplication algorithm has a time complexity of O(N^3)...
But how can I prove that through my table of values? Size is the row or column length of the matrix. So square that for the full matrix size.
Size = 100 Mat. Mult. Elapsed Time: 0.0199 seconds.
Size = 200 Mat. Mult. Elapsed Time: 0.0443 seconds.
Size = 300 Mat. Mult. Elapsed Time: 0.0984 seconds.
Size = 400 Mat. Mult. Elapsed Time: 0.2704 seconds.
Size = 800 Mat. Mult. Elapsed Time: 6.393 seconds.
This is like looking at a table of values and estimating the graph of the function... There has to be some relationship between these numbers, and N^3. How do I make sense of it though?
I have provided my algorithm below. I already know it is O(N^3) by counting the loops. How can I relate that to my table of values above though?
/**
* This function multiplies two matrices and returns the product matrix.
*
* #param mat1
* The first multiplier matrix.
* #param mat2
* The second multiplicand matrix.
* #return The product matrix.
*/
private static double[][] MatMult(double[][] mat1, double[][] mat2) {
int m1RowLimit = mat1.length, m2ColumnLimit = mat2[0].length, innerLimit = mat1[0].length;
if ((mat1[0].length != mat2.length))
return null;
int m1Row = 0, m1Column = 0, m2Row = 0, m2Column = 0;
double[][] mat3 = new double[m1RowLimit][m2ColumnLimit];
while (m1Row < m1RowLimit) {
m2Column = 0;
while (m2Column < m2ColumnLimit) {
double value = 0;
m1Column = 0;
m2Row = 0;
while (m1Column < innerLimit) {
value += mat1[m1Row][m1Column] * mat2[m2Row][m2Column];
m1Column++;
m2Row++;
}
mat3[m1Row][m2Column] = value;
m2Column++;
}
m1Row++;
}
return mat3;
}

The methodology
Okay. So you want to prove your algorithm's time complexity is O(n^3). I understand why you would look at the time it takes for a program to run a calculation, but this data is not reliable. What we do, is we apply a weird form of limits to abstract away from the other aspects of an algorithm, and leave us with our metric.
The Metric
A metric is what we are going to use to measure your algorithm. It is the operation that occurs the most, or carries the most processing weight. In this case, it is this line:
value += mat1[m1Row][m1Column] * mat2[m2Row][m2Column];
Deriving the Recurrence Relation
The next step, as I understand it, is to derive a recurrence relation from your algorithm. That is, a description of how your algorithm functions based on it's functionality in the past. Let's look at how your program runs.
As you explained, you have looked at your three while loops, and determined the program is of order O(n^3). Unfortunately, this is not mathematical. This is just something that seems to happen a lot. First, let's look at some numerical examples.
When m1RowLimit = 4, m2ColumnLimit = 4, innerLimit = 4, our metric is ran 4 * 4 * 4 = 4^3 times.
When m1RowLimit = 5, m2ColumnLimit = 5, innerLimit = 5, our metric is ran 5 * 5 * 5 = 5^3 times.
So how do we express this in a recurrence relation? Well, using some basic maths we get:
T(n) = T(n-1) + 3(n-1)^2 + 3(n-1) + 1 for all n >= 1
T(1) = 1
Solving the Recurrence Relation using Forward Substitution and Mathematical Induction
Now, is where we use some forward substitution. What we first do, is get a feel for the relation (this also tests that it's accurate).
T(2) = T(1) + 3(1^2) + 3(1) + 1 = 1 + 3 + 3 + 1 = 8.
T(3) = T(2) + 3(2^2) + 3(2) + 1 = 8 + 12 + 6 + 1 = 27
T(4) = T(3) + 3(3^2) + 3(3) + 1 = 27 + 27 + 9 + 1 = 64
NOW, we assert the hypothesis that T(n) = n^3. Let's test it for the base case:
T(1) = 1^3 = 1. // Correct!
Now we test it, using mathematical induction, for the next step. The algorithm increases by 1 each time, so the next step is: T(n+1). So what do we need to prove? Well we need to prove that by increasing n by 1 on one side, the equal effect happens to n on the other. If it is true for all n + 1, then it is true for n + 1 + 1 and so on. This means, we're aiming to prove that:
T(n + 1) = (n + 1)^3
T(n + 1) = T(n - 1 + 1) + 3(n + 1 - 1)^2 + 3(n + 1 - 1) + 1
= T(n) + 3(n)^2 + 3(n) + 1
Assume T(n) = n^3
T(n + 1) = n^3 + 3(n)^2 + 3(n) + 1
T(n + 1) = (n+1)^3 // Factorize the function.
So at this point, you've proven your algorithm has a run time complexity of O(n^3).

Empirically, you can plot your data with an adjacent third-degree polynomial trend-line for reference.
CSV data:
100, 0.0199
200, 0.0443
300, 0.0984
400, 0.2704
800, 6.393

The first response covers how to prove the time complexity of your algorithm quite well.
However, you seem to be asking how to relate the experimental results of your benchmarks with time complexity, not how to prove time complexity.
So, how do we interpret the experimental data? Well, you could start by simply plotting the data (runtime on the y-axis, size on the x-axis). With enough data points, this could give you some hints about the behavior of your algorithm.
Since you already know the expected time complexity of your algorithm, you could then draw a "curve of best fit" (i.e. a line of the shape n^3 that best fits your data). If your data matches the line fairly well, then you were likely correct. If not, it's possible you made some mistake, or that your experimental results are not matching due to factors you are not accounting for.
To determine the equation for the best fitting n^3 line, you could simply take the calculated time complexity, express it as an equation, and guess values for the unknowns until you find an equation that fits. So for n^3, you'd have:
t = a*n^3 + b*n^2 + c*n + d
Find the values of a, b, c, and d that form an equation that best fits your data. If that fit still isn't good enough, then you have a problem.
For more rigorous techniques, you'd have to ask someone more well versed in statistics. I believe the value you'd want to calculate is the coefficient of determination (a.k.a. R^2, basically tells you the variance between the expected and actual results). However, on it's own this value doesn't prove a whole lot. This problem of validating hypothesized relationships between variables is known as Regression Model Validation; the wikipedia article provides a bit more information on how to go further with this if R^2 isn't enough for your purposes.