Big-O complexity of a recursive solution - java

I have a simple recursive solution as below:
public int countPaths(int x, int y) {
if(x == 0 && y == 0) {
return 0;
} else if(x == 0) {
return 1;
} else if(y == 0) {
return 1;
} else {
int count = countPaths(x-1, y);
count += countPaths(x, y-1);
return count;
}
}
This is to solve the following problem from the book: Cracking the coding interview
Imagine a robot sitting on the upper left corner of an X by Y grid. The robot can only move in two directions: right and down. How many possible paths are there for the robot to go from (0,0) to (X,Y)?
I am trying to ascertain the run time complexity and I believe it is O(x+y). I arrived at this by using a recursion tree, for example if x=2 and y=2
The max depth of this tree is (x+y) and work done at each step is a constant. So max work done is (x+y) * c and hence the run time complexity is O(x+y)
Question 1: Am I correct? I believe the upper bound I have calculated is not tight enough
Question 2: Next, if I were to improve the run time using memoization and hence not repeating computing sub-problems, how would the run time complexity as described by Big-o change?

While it's true that the depth of the tree is O(x+y), there's increasingly many nodes at each layer, and it's the number of nodes that determines complexity, not the depth of the tree.
If you write down recurrence relations for the runtime, you get:
T(0, y) = T(x, 0) = 1
T(x, y) = T(x-1, y) + T(x, y-1) + 1
If you ignore the +1 on the second equation (which can only make the run-time better), you get the same function that your code was computing in the first place, which is choose(x+y, y).
For x=y, this is the central binomial coefficient, which is approximately 4^x/sqrt(pi*x), which for even moderately large values of x is large enough to make the algorithm useless.
With memoisation, you're doing a constant amount of work for each value of x and y, so the complexity is O(xy).

If you evaluate the complexity in terms of the number of additions required to evaluate the count for a given pair (x, y), you get the recurrence
A(x,y) = A(x-1,y) + A(x,y-1) + 1,
with A(x,0) = A(0,y) = 0.
Setting A(x,y) = P(x,y) - 1 the recurrence becomes
P(x,y) - 1 = P(x-1,y) - 1 + P(x,y-1) - 1 + 1,
or
P(x,y) = P(x-1,y) + P(x,y-1),
with P(x,0) = P(0,y) = 1, which gives the classical Pascal's triangle, and
A(x,y) = (x+y)!/(x!y!) - 1.
You can also work with the number of recursive function calls,
C(x,y) = C(x-1,y) + C(x,y-1) + 2,
with C(0,y) = C(x,0) = 0.
You will solve it by setting C(x,y) = 2P(x,y) - 2, and get
C(x,y)= 2(x+y)!/(x!y!)-2.
As regards the asymptotic complexity, this makes no difference. The is no really simpler formula than O((x+y)!/x!y!).
With memoization, every evaluation (with x, y>0) costs only one addition or two calls, and assuming constant time for storage/retrieval of a value the total complexity is the much better O(xy).

Based on the valuable input from #Anonymous, we know the recurrence relation is:
T(x, y) = T(x-1, y) + T(x, y-1) + 1
Abusing (which is ok in Big-O analysis) the notation, let x = y
T(x, x) = 2 * T(x-1, x) = 2 * 2 * T(x-2, x) = ... = 2 * ... * 2 * T(0, x)
= O(2^x)
So the run time complexity is
O(2^n) ; where n = max(x, y)
With memoization, I get it, thank #Anonymous, it should be O(xy)

Related

How can i find the time complexity of below function? [duplicate]

This question already has answers here:
How can I find the time complexity of an algorithm?
(10 answers)
Closed 3 years ago.
Can anyone guide me to find the time complexity? Does the time complexity change over operating systems?
int fn(int n){
if(n==1){
return 1;
}
else{
return(fn(n-1)+ fn(n-1));
}
}
You can make a recurrence relation T which represents the time it would take to compute and input of size N and then use the method of telescoping to help find the Big-O like so:
T(N) = T(N-1) + T(N-1) + c = 2*T(N-1) + c
Here, we can see that the time it will take to compute T(N) will be 2*T(N-1) plus a constant amount of time c. We can also see by your function that:
T(1) = b
Here, we can see by your base case that there are no recursive calls when N=1, so, it will take a constant time b to compute T(1).
If we take a look at T(N), we need to find out what T(N-1) is, so computing that we get:
T(N-1) = 2*T(N-2) + c
Computing T(N-2) we get:
T(N-2) = 2*T(N-3) + c
Thus, we can sub these back into each other...
T(N-1) = 2*(2*T(N-3) + c) + c = 4*T(N-3) + 3c
T(N) = 2*(4*T(N-3) + 3c) + c = 8*T(N-3) + 7c
By looking at the pattern produced by stepping down into our equation, we can generalize it in terms of k:
T(N) = 2^k * T(N-k) + ((2^k)-1 * c)
We know that our recursive calls will stop when T(N-k) = T(1), so we want to find when N-k = 1, which is when k = N-1, so subbing in our value of k and removing the constant-variable times, we can find our Big-O:
T(N) = (2^N) * T(1) + (2^N)-1 * c
= (2^N) * b + (2^N)-1*c
= O(2^N) (-1, b & c are constants, so they can be removed, giving 2*(2^N), where 2* is a constant, giving 2^N)
Time complexity is a measurement of how well an algorithm will scale in terms of input size, not necessarily a measure of how fast it will run. Thus, time-complexity is not dependent on your operating system.
The function is a recursive function (it calls itself). In that case, its time complexity is either linear or exponential (or something else, which we won't cover here):
It is linear, if you can do TCO (tail call optimization), or in other words, turn the function into a loop:
int loop(int i, int count) {
if(i > 10) return count;
return loop(i - 1, count + 1);
}
loop(0, 0);
// can be turned into
int count = 0;
for(int i = 0; i <= 10; i++) {
count = count + 1;
}
Otherwise, it is exponential, as every call will execute the function again m times, and each of those m calls, calls the function again m time and so on. This will happen until the depth n is reached, so the time complexity is:
O(m ^ n)
Now m is kind of a constant, as the number of recursive calls doesn't change (it's two in your case), however n can be changed. Therefore the function has exponential time complexity. That's generally bad, as exponential numbers get really large for relatively small datasets. In your case however, it is trivial to optimize. a + a is the same as a * 2, thus your code can be turned into:
int fn(int n){
if(n==1){
return 1;
} else {
return fn(n - 1) * 2;
}
}
And that is .... linear!
Does the time complexity change over operating systems?
No, the time complexity is an abstract concept, it doesn't depend on the computer, operating system, language,... You can even run it on an automaton.
when n=1, the time complexity is 1...
No! n is not a constant. n is part of the input, and time complexity is a way to estimate the time depending on the input.
Here is the approach I would take.
from the function definition we can deduce that
O(f(n)) = c + 2 * O(f(n-1))
If we ignore constant terms
O(f(n)) = 2 * (2 * f(n-2))
so we can say the complexity here is O(2^n)

Place n points while maximizing the minimum distance

Given a distance d (going from 0 to d) and 2 points s and e in between which no points can be placed (placing points on s and e is fine, it's not allowed to place points between them).
Place n points such that the distance between each point is as large as possible (distribute them as evenly as possible).
Output the minimal distance between 2 points.
Graphic representation, place n points on the black line (it's a 1-dimensional line) so that the smallest distance between each 2 points is as large as possible (an absolute error of up to 10^(-4) is allowed).
Examples:
d=7, n=2, s=6, e=7, Output is: 7.0000000000
d=5, n=3, s=5, e=5, Output is: 2.5000000006
d=3, n=3, s=0, e=1, Output is: 1.5000000007
d=9, n=10, s=5, e=6, Output is: 1.0000000001
d=6, n=2, s=1, e=6, Output is: 6.0000000000
d=5, n=3, s=4, e=5, Output is: 2.5000000006
My approach:
I tried looking at the intervals separately, distributing points (ideal distribution, lengthOfInterval/n) on the first and second interval (0 to s and e to d) and inspecting all distributions whose number of points sum up to n, I would store a (distribution, largest minimal distance) pair and pick the pair with the largest minimal distance. I don't know how to work with the 10^(-4) tolerance (how does this part even look in code?) and am not sure if my approach is correct. Every suggestion is welcome.
I'm stuck on this question :/
You can use binary search over the possible sizes of gaps between points (from 0 to d) to converge to the largest minimum gap size.
To determine the viability of any given gap size, you basically try to place points from the left and from the right and see whether the gap in the middle is big enough:
Determine how many points can be placed left of s (which is s/gapSize + 1).
Determine how many points will then be required to be placed to the right of e
(which is n - points on left).
Determine how far inwards each side will go.
Check whether the points on the right fits in the gap [e, d] and whether there's at least gap size difference between each side.
Code for this: (note that I worked with number of gaps instead of points, which is just 1 less than the number of points, since it leads to simpler code)
double high = d, low = 0, epsilon = 0.000001;
while (low + epsilon < high)
{
double mid = (low + high)/2;
int gapsOnLeft = (int)(s/mid); // gaps = points - 1
if (gapsOnLeft + 1 > n)
gapsOnLeft = n - 1;
int gapsOnRight = n - gapsOnLeft - 2; // will be -1 when there's no point on the right
double leftOffset = mid*gapsOnLeft;
// can be > d with no point on the right, which makes the below check work correctly
double rightOffset = d - mid*gapsOnRight;
if (leftOffset + mid <= rightOffset && rightOffset >= e)
low = mid;
else
high = mid;
}
System.out.println(low);
Live demo.
The time complexity is O(log d).
The problem with your approach is that it's hard to figure out how big the gaps between points are supposed to be, so you won't know how many points are supposed to go on either side of (s, e) as to end up with an optimal solution and to correctly deal with both cases when s and e are really close together and when they're far apart.
Binary search
Its very easy to find the number of points you can place if the minimum separation distance b/w any pair l is given.
If l=d, then at the most only 2 points can be placed.
..
...
....
so just do a binary search on l.
A crude implementation goes like this.
low,high=0.00001,d
while(high-low>eps):
m = (low+high)/2
if((no. of points placed s.t any pair is at most m units away) >=n):
low=mid
else:
high=mid
TL;DR: Your approach does not always work (and you're not doing it as fast as you could) see the 3rd bullet point for one that works (and uses the given 10^(-4)).
If [s, e] is small and well-placed, then the optimum is just distributing evenly on the whole segment, best value is now d/(n-1). But you'll have to check that none of your elements is between s and e.
Your approach works if s and e are "far enough".
You can do it faster than what you seem to suggest though, by lookign for the best splitting between the two segments in time O(1): if you put n1 (1<=n1<=n-1) elements on the left, you want to maximize min(s/(n1-1), (d-e)/(n-n1-1)) (one of these quantities being possibly +infinity, but then the other is not). The maximum of that function is obtained for s/(x-1) = (d-e)/(n-x-1), just compute the corresponding value for x, and either its floor or ceiling is the best value for n1. The distance obtained is best = min(s/(n1-1), (d-e)/(n-n1-1)) Then you put n1 points on the left, starting at 0, separated by distance best, and n-n1 on the right, starting at d, going left, separated by best.
If the distance between the last point on the left and the first on the right is smaller than best, then you have a problem, this approach does not work.
The complicated case is when the two previous approaches failed: the hole is small and not well placed. Then there are probably many ways to solve the problem. One is to use binary search to find the optimal space between two consecutive points. Given a candidate space sp, try distributing points on the line starting at 0, spaced by sp, as many as you can while remaining below s. Do the same on the right while staying above e and above (last on the left + sp). If you have successfully placed at least n points in total, then sp is too small. Otherwise, it is too big.
Thus, you can use binary search to find the optimal spas follows: start at sp possibly in [max(s, d-e)/(n-1), d/(n-1)]. At each step, take the middle mid of your possible segment [x, y]. Check if the real optimum is above or below mid. According to your case, look for the optimum in [mid, y] or [x, mid]. Stop iff y-x < 10^(-4).
The two previous cases will actually also be found by this method, so you don't need to implement them, except if you want the exact optimal value when possible (i.e. in the first two cases).
It's pretty tricky, except for the simple case (no point lands in the gap):
double dMin = d / (n - 1.0);
if (Math.ceil(e / dMin - 1) * dMin <= s)
return dMin;
Let's continue with the edge cases, placing one point on one side and the rest of the points on the other one:
dMin = Math.min((d - e) / (n - 2.0), e); // one point at 0
double dm = Math.min(s / (n - 2.0), d - s); // one point at d
if (dm > dMin) // 2nd configuration was better
dMin = dm;
And finally for two or more points on both sides:
// left : right = (x - 1) : (n - x - 1)
// left * n - left * x - left = right * x - right
// x * (left + right) = left * n - left + right
// x = (left * n - left + right) / (left + right) = (left * n) / (left + right) - 1
int x = s * n / (d - e + s) - 1;
if (x < 2)
x = 2;
for (int y = x; y <= x + 2 && y < n - 1; y++) {
double dLeft = s / (y - 1.0);
double dRight = (d - e) / (n - y - 1.0);
dm = Math.min(dLeft, dRight);
if (dm > e - s) { // dm bigger than gap
if (dLeft > dRight)
dLeft = e / ((double) y);
else
dRight = (d - s) / ((double) n - y);
dm = Math.min(dLeft, dRight);
}
if (dm > dMin)
dMin = dm;
}
This would be O(1) space and time, but I'm not 100% positive if all cases are checked. Please let me know if it worked. Tested against all the test cases. The above works for n >= 2, if n equals 2 it will be caught by the first check.

Analyzing runtime of a short algorithm

I got the following code given:
public class alg
{
public static int hmm (int x)
{
if (x == 1)
{
return 2;
}
return 2*x + hmm(x-1);
}
public static void main (String[] args)
{
int x = Integer.parseInt(args[0]);
System.out.println(hmm(x));
}
}
So first question is, what does this algorithm count?
I have just typed and runned it in eclipse
so I can see better what it does (it was pseudocode before, I couldn't type it here so I typed the code). I have realized that this algorithm does following: It will take the input and multiply it by its following number.
So as examples:
input = 3, output = 12 because 3*4 = 12.
Or Input = 6, output 42 because 6*7 = 42.
Alright, the next question is my problem. I'm asked to analyze the runtime of this algorithm but I have no idea where to start.
I would say, at the beginning, when we define x, we have already got time = 1
The if loop gives time = 1 too I believe.
Last part, return 2x + alg(x-1) should give "something^x" or..?
So in the end we got something like "something^x" + 2, I doubt thats right : /
edit, managed to type pseudocode too :)
Input: Integer x with x > 1
if x = 1 then
return 2;
end if
return 2x + hmm(x-1);
When you have trouble, try to walk through the code with a (small) number.
What does this calculate?
Let's take hmm(3) as an example:
3 != 1, so we calculate 2 * 3 + hmm(3-1). Down a recursion level.
2 != 1, so we calculate 2 * 2 + hmm(2-1). Down a recursion level.
1 == 1, so we return 2. No more recursions, thus hmm(2-1) == hmm(1) == 2.
Back up one recursion level, we get 2 * 2 + hmm(1) = 2 * 2 + 2 = 4 + 2 = 6. Thus hmm(2) = 6
Another level back up, we get 2 * 3 + hmm(2) = 6 + 6 = 12
If you look closely, the algorithm calculates:
2*x + ... + 4 + 2
We can reverse this and factor out 2 and get
2 * (1 + 2 + ... + x).
Which is an arithmetic progression, for which we have a well-known formula (namely x² + x)
How long does it take?
The asymptotic running time is O(n).
There are no loops, so we only have to count the number of recursions. One might be tempted to count the individual steps of calculation, but those a are constant with every step, so we usually combine them into a constant factor k.
What does O(n) mean?
Well ... we make x - 1 recursion steps, decreasing x by 1 in every step until we reach x == 1. From x = n to x = 1 there are n - 1 such steps. We thus need k * (n - 1) operations.
If you think n to be very large, - 1 becomes negligible, so we drop it. We also drop the constant factor, because for large n, O(nk) and O(n) aren't that much different, either.
The function calculates
f(x) = 2(x + x-1 + x-2 + ... + 1)
it will run in O(x), i.e. x times will be called for constant time O(1).

Algorithm to efficiently determine the [n][n] element in a matrix

This is a question regarding a piece of coursework so would rather you didn't fully answer the question but rather give tips to improve the run time complexity of my current algorithm.
I have been given the following information:
A function g(n) is given by g(n) = f(n,n) where f may be defined recursively by
I have implemented this algorithm recursively with the following code:
public static double f(int i, int j)
{
if (i == 0 && j == 0) {
return 0;
}
if (i ==0 || j == 0) {
return 1;
}
return ((f(i-1, j)) + (f(i-1, j-1)) + (f(i, j-1)))/3;
}
This algorithm gives the results I am looking for, but it is extremely inefficient and I am now tasked to improve the run time complexity.
I wrote an algorithm to create an n*n matrix and it then computes every element up to the [n][n] element in which it then returns the [n][n] element, for example f(1,1) would return 0.6 recurring. The [n][n] element is 0.6 recurring because it is the result of (1+0+1)/3.
I have also created a spreadsheet of the result from f(0,0) to f(7,7) which can be seen below:
Now although this is much faster than my recursive algorithm, it has a huge overhead of creating a n*n matrix.
Any suggestions to how I can improve this algorithm will be greatly appreciated!
I can now see that is it possible to make the algorithm O(n) complexity, but is it possible to work out the result without creating a [n][n] 2D array?
I have created a solution in Java that runs in O(n) time and O(n) space and will post the solution after I have handed in my coursework to stop any plagiarism.
This is another one of those questions where it's better to examine it, before diving in and writing code.
The first thing i'd say you should do is look at a grid of the numbers, and to not represent them as decimals, but fractions instead.
The first thing that should be obvious is that the total number of you have is just a measure of the distance from the origin, .
If you look at a grid in this way, you can get all of the denominators:
Note that the first row and column are not all 1s - they've been chosen to follow the pattern, and the general formula which works for all of the other squares.
The numerators are a little bit more tricky, but still doable. As with most problems like this, the answer is related to combinations, factorials, and then some more complicated things. Typical entries here include Catalan numbers, Stirling's numbers, Pascal's triangle, and you will nearly always see Hypergeometric functions used.
Unless you do a lot of maths, it's unlikely you're familiar with all of these, and there is a hell of a lot of literature. So I have an easier way to find out the relations you need, which nearly always works. It goes like this:
Write a naive, inefficient algorithm to get the sequence you want.
Copy a reasonably large amount of the numbers into google.
Hope that a result from the Online Encyclopedia of Integer Sequences pops up.
3.b. If one doesn't, then look at some differences in your sequence, or some other sequence related to your data.
Use the information you find to implement said sequence.
So, following this logic, here are the numerators:
Now, unfortunately, googling those yielded nothing. However, there are a few things you can notice about them, the main being that the first row/column are just powers of 3, and that the second row/column are one less than powers of three. This kind boundary is exactly the same as Pascal's triangle, and a lot of related sequences.
Here is the matrix of differences between the numerators and denominators:
Where we've decided that the f(0,0) element shall just follow the same pattern. These numbers already look much simpler. Also note though - rather interestingly, that these numbers follow the same rules as the initial numbers - except the that the first number is one (and they are offset by a column and a row). T(i,j) = T(i-1,j) + T(i,j-1) + 3*T(i-1,j-1):
1
1 1
1 5 1
1 9 9 1
1 13 33 13 1
1 17 73 73 17 1
1 21 129 245 192 21 1
1 25 201 593 593 201 25 1
This looks more like the sequences you see a lot in combinatorics.
If you google numbers from this matrix, you do get a hit.
And then if you cut off the link to the raw data, you get sequence A081578, which is described as a "Pascal-(1,3,1) array", which exactly makes sense - if you rotate the matrix, so that the 0,0 element is at the top, and the elements form a triangle, then you take 1* the left element, 3* the above element, and 1* the right element.
The question now is implementing the formulae used to generate the numbers.
Unfortunately, this is often easier said than done. For example, the formula given on the page:
T(n,k)=sum{j=0..n, C(k,j-k)*C(n+k-j,k)*3^(j-k)}
is wrong, and it takes a fair bit of reading the paper (linked on the page) to work out the correct formula. The sections you want are proposition 26, corollary 28. The sequence is mentioned in Table 2 after proposition 13. Note that r=4
The correct formula is given in proposition 26, but there is also a typo there :/. The k=0 in the sum should be a j=0:
Where T is the triangular matrix containing the coefficients.
The OEIS page does give a couple of implementations to calculate the numbers, but neither of them are in java, and neither of them can be easily transcribed to java:
There is a mathematica example:
Table[ Hypergeometric2F1[-k, k-n, 1, 4], {n, 0, 10}, {k, 0, n}] // Flatten
which, as always, is ridiculously succinct. And there is also a Haskell version, which is equally terse:
a081578 n k = a081578_tabl !! n !! k
a081578_row n = a081578_tabl !! n
a081578_tabl = map fst $ iterate
(\(us, vs) -> (vs, zipWith (+) (map (* 3) ([0] ++ us ++ [0])) $
zipWith (+) ([0] ++ vs) (vs ++ [0]))) ([1], [1, 1])
I know you're doing this in java, but i could not be bothered to transcribe my answer to java (sorry). Here's a python implementation:
from __future__ import division
import math
#
# Helper functions
#
def cache(function):
cachedResults = {}
def wrapper(*args):
if args in cachedResults:
return cachedResults[args]
else:
result = function(*args)
cachedResults[args] = result
return result
return wrapper
#cache
def fact(n):
return math.factorial(n)
#cache
def binomial(n,k):
if n < k: return 0
return fact(n) / ( fact(k) * fact(n-k) )
def numerator(i,j):
"""
Naive way to calculate numerator
"""
if i == j == 0:
return 0
elif i == 0 or j == 0:
return 3**(max(i,j)-1)
else:
return numerator(i-1,j) + numerator(i,j-1) + 3*numerator(i-1,j-1)
def denominator(i,j):
return 3**(i+j-1)
def A081578(n,k):
"""
http://oeis.org/A081578
"""
total = 0
for j in range(n-k+1):
total += binomial(k, j) * binomial(n-k, j) * 4**(j)
return int(total)
def diff(i,j):
"""
Difference between the numerator, and the denominator.
Answer will then be 1-diff/denom.
"""
if i == j == 0:
return 1/3
elif i==0 or j==0:
return 0
else:
return A081578(j+i-2,i-1)
def answer(i,j):
return 1 - diff(i,j) / denominator(i,j)
# And a little bit at the end to demonstrate it works.
N, M = 10,10
for i in range(N):
row = "%10.5f"*M % tuple([numerator(i,j)/denominator(i,j) for j in range(M)])
print row
print ""
for i in range(N):
row = "%10.5f"*M % tuple([answer(i,j) for j in range(M)])
print row
So, for a closed form:
Where the are just binomial coefficients.
Here's the result:
One final addition, if you are looking to do this for large numbers, then you're going to need to compute the binomial coefficients a different way, as you'll overflow the integers. Your answers are lal floating point though, and since you're apparently interested in large f(n) = T(n,n) then I guess you could use Stirling's approximation or something.
Well for starters here are some things to keep in mind:
This condition can only occur once, yet you test it every time through every loop.
if (x == 0 && y == 0) {
matrix[x][y] = 0;
}
You should instead: matrix[0][0] = 0; right before you enter your first loop and set x to 1. Since you know x will never be 0 you can remove the first part of your second condition x == 0 :
for(int x = 1; x <= i; x++)
{
for(int y = 0; y <= j; y++)
{
if (y == 0) {
matrix[x][y] = 1;
}
else
matrix[x][y] = (matrix[x-1][y] + matrix[x-1][y-1] + matrix[x][y-1])/3;
}
}
No point in declaring row and column since you only use it once. double[][] matrix = new double[i+1][j+1];
This algorithm has a minimum complexity of Ω(n) because you just need to multiply the values in the first column and row of the matrix with some factors and then add them up. The factors stem from unwinding the recursion n times.
However you therefore need to do the unwinding of the recursion. That itself has a complexity of O(n^2). But by balancing unwinding and evaluation of recursion, you should be able to reduce complexity to O(n^x) where 1 <= x <= 2. This is some kind of similiar to algorithms for matrix-matrix multiplication, where the naive case has a complexity of O(n^3) but Strassens's algorithm is for example O(n^2.807).
Another point is the fact that the original formula uses a factor of 1/3. Since this is not accurately representable by fixed point numbers or ieee 754 floating points, the error increases when evaluating the recursion successively. Therefore unwinding the recursion could give you higher accuracy as a nice side effect.
For example when you unwind the recursion sqr(n) times then you have complexity O((sqr(n))^2+(n/sqr(n))^2). The first part is for unwinding and the second part is for evaluating a new matrix of size n/sqr(n). That new complexity actually can be simplified to O(n).
To describe time complexity we usually use a big O notation. It is important to remember that it only describes the growth given the input. O(n) is linear time complexity, but it doesn't say how quickly (or slowly) the time grows when we increase input. For example:
n=3 -> 30 seconds
n=4 -> 40 seconds
n=5 -> 50 seconds
This is O(n), we can clearly see that every increase of n increases the time by 10 seconds.
n=3 -> 60 seconds
n=4 -> 80 seconds
n=5 -> 100 seconds
This is also O(n), even though for every n we need twice that much time, and the raise is 20 seconds for every increase of n, the time complexity grows linearly.
So if you have O(n*n) time complexity and you will half the number of operations you perform, you will get O(0.5*n*n) which is equal to O(n*n) - i.e. your time complexity won't change.
This is theory, in practice the number of operations sometimes makes a difference. Because you have a grid n by n, you need to fill n*n cells, so the best time complexity you can achieve is O(n*n), but there are a few optimizations you can do:
Cells on the edges of the grid could be filled in separate loops. Currently in majority of the cases you have two unnecessary conditions for i and j equal to 0.
You grid has a line of symmetry, you could utilize it to calculate only half of it and then copy the results onto the other half. For every i and j grid[i][j] = grid[j][i]
On final note, the clarity and readability of the code is much more important than performance - if you can read and understand the code, you can change it, but if the code is so ugly that you cannot understand it, you cannot optimize it. That's why I would do only first optimization (it also increases readability), but wouldn't do the second one - it would make the code much more difficult to understand.
As a rule of thumb, don't optimize the code, unless the performance is really causing problems. As William Wulf said:
More computing sins are committed in the name of efficiency (without necessarily achieving it) than for any other single reason - including blind stupidity.
EDIT:
I think it may be possible to implement this function with O(1) complexity. Although it gives no benefits when you need to fill entire grid, with O(1) time complexity you can instantly get any value without having a grid at all.
A few observations:
denominator is equal to 3 ^ (i + j - 1)
if i = 2 or j = 2, numerator is one less than denominator
EDIT 2:
The numerator can be expressed with the following function:
public static int n(int i, int j) {
if (i == 1 || j == 1) {
return 1;
} else {
return 3 * n(i - 1, j - 1) + n(i - 1, j) + n(i, j - 1);
}
}
Very similar to original problem, but no division and all numbers are integers.
If the question is about how to output all values of the function for 0<=i<N, 0<=j<N, here is a solution in time O(N²) and space O(N). The time behavior is optimal.
Use a temporary array T of N numbers and set it to all ones, except for the first element.
Then row by row,
use a temporary element TT and set it to 1,
then column by column, assign simultaneously T[I-1], TT = TT, (TT + T[I-1] + T[I])/3.
Thanks to will's (first) answer, I had this idea:
Consider that any positive solution comes only from the 1's along the x and y axes. Each of the recursive calls to f divides each component of the solution by 3, which means we can sum, combinatorially, how many ways each 1 features as a component of the solution, and consider it's "distance" (measured as how many calls of f it is from the target) as a negative power of 3.
JavaScript code:
function f(n){
var result = 0;
for (var d=n; d<2*n; d++){
var temp = 0;
for (var NE=0; NE<2*n-d; NE++){
temp += choose(n,NE);
}
result += choose(d - 1,d - n) * temp / Math.pow(3,d);
}
return 2 * result;
}
function choose(n,k){
if (k == 0 || n == k){
return 1;
}
var product = n;
for (var i=2; i<=k; i++){
product *= (n + 1 - i) / i
}
return product;
}
Output:
for (var i=1; i<8; i++){
console.log("F(" + i + "," + i + ") = " + f(i));
}
F(1,1) = 0.6666666666666666
F(2,2) = 0.8148148148148148
F(3,3) = 0.8641975308641975
F(4,4) = 0.8879743941472337
F(5,5) = 0.9024030889600163
F(6,6) = 0.9123609205913732
F(7,7) = 0.9197747256986194

How can I estimate time complexity from a table of values?

I know that my naive matrix multiplication algorithm has a time complexity of O(N^3)...
But how can I prove that through my table of values? Size is the row or column length of the matrix. So square that for the full matrix size.
Size = 100 Mat. Mult. Elapsed Time: 0.0199 seconds.
Size = 200 Mat. Mult. Elapsed Time: 0.0443 seconds.
Size = 300 Mat. Mult. Elapsed Time: 0.0984 seconds.
Size = 400 Mat. Mult. Elapsed Time: 0.2704 seconds.
Size = 800 Mat. Mult. Elapsed Time: 6.393 seconds.
This is like looking at a table of values and estimating the graph of the function... There has to be some relationship between these numbers, and N^3. How do I make sense of it though?
I have provided my algorithm below. I already know it is O(N^3) by counting the loops. How can I relate that to my table of values above though?
/**
* This function multiplies two matrices and returns the product matrix.
*
* #param mat1
* The first multiplier matrix.
* #param mat2
* The second multiplicand matrix.
* #return The product matrix.
*/
private static double[][] MatMult(double[][] mat1, double[][] mat2) {
int m1RowLimit = mat1.length, m2ColumnLimit = mat2[0].length, innerLimit = mat1[0].length;
if ((mat1[0].length != mat2.length))
return null;
int m1Row = 0, m1Column = 0, m2Row = 0, m2Column = 0;
double[][] mat3 = new double[m1RowLimit][m2ColumnLimit];
while (m1Row < m1RowLimit) {
m2Column = 0;
while (m2Column < m2ColumnLimit) {
double value = 0;
m1Column = 0;
m2Row = 0;
while (m1Column < innerLimit) {
value += mat1[m1Row][m1Column] * mat2[m2Row][m2Column];
m1Column++;
m2Row++;
}
mat3[m1Row][m2Column] = value;
m2Column++;
}
m1Row++;
}
return mat3;
}
The methodology
Okay. So you want to prove your algorithm's time complexity is O(n^3). I understand why you would look at the time it takes for a program to run a calculation, but this data is not reliable. What we do, is we apply a weird form of limits to abstract away from the other aspects of an algorithm, and leave us with our metric.
The Metric
A metric is what we are going to use to measure your algorithm. It is the operation that occurs the most, or carries the most processing weight. In this case, it is this line:
value += mat1[m1Row][m1Column] * mat2[m2Row][m2Column];
Deriving the Recurrence Relation
The next step, as I understand it, is to derive a recurrence relation from your algorithm. That is, a description of how your algorithm functions based on it's functionality in the past. Let's look at how your program runs.
As you explained, you have looked at your three while loops, and determined the program is of order O(n^3). Unfortunately, this is not mathematical. This is just something that seems to happen a lot. First, let's look at some numerical examples.
When m1RowLimit = 4, m2ColumnLimit = 4, innerLimit = 4, our metric is ran 4 * 4 * 4 = 4^3 times.
When m1RowLimit = 5, m2ColumnLimit = 5, innerLimit = 5, our metric is ran 5 * 5 * 5 = 5^3 times.
So how do we express this in a recurrence relation? Well, using some basic maths we get:
T(n) = T(n-1) + 3(n-1)^2 + 3(n-1) + 1 for all n >= 1
T(1) = 1
Solving the Recurrence Relation using Forward Substitution and Mathematical Induction
Now, is where we use some forward substitution. What we first do, is get a feel for the relation (this also tests that it's accurate).
T(2) = T(1) + 3(1^2) + 3(1) + 1 = 1 + 3 + 3 + 1 = 8.
T(3) = T(2) + 3(2^2) + 3(2) + 1 = 8 + 12 + 6 + 1 = 27
T(4) = T(3) + 3(3^2) + 3(3) + 1 = 27 + 27 + 9 + 1 = 64
NOW, we assert the hypothesis that T(n) = n^3. Let's test it for the base case:
T(1) = 1^3 = 1. // Correct!
Now we test it, using mathematical induction, for the next step. The algorithm increases by 1 each time, so the next step is: T(n+1). So what do we need to prove? Well we need to prove that by increasing n by 1 on one side, the equal effect happens to n on the other. If it is true for all n + 1, then it is true for n + 1 + 1 and so on. This means, we're aiming to prove that:
T(n + 1) = (n + 1)^3
T(n + 1) = T(n - 1 + 1) + 3(n + 1 - 1)^2 + 3(n + 1 - 1) + 1
= T(n) + 3(n)^2 + 3(n) + 1
Assume T(n) = n^3
T(n + 1) = n^3 + 3(n)^2 + 3(n) + 1
T(n + 1) = (n+1)^3 // Factorize the function.
So at this point, you've proven your algorithm has a run time complexity of O(n^3).
Empirically, you can plot your data with an adjacent third-degree polynomial trend-line for reference.
CSV data:
100, 0.0199
200, 0.0443
300, 0.0984
400, 0.2704
800, 6.393
The first response covers how to prove the time complexity of your algorithm quite well.
However, you seem to be asking how to relate the experimental results of your benchmarks with time complexity, not how to prove time complexity.
So, how do we interpret the experimental data? Well, you could start by simply plotting the data (runtime on the y-axis, size on the x-axis). With enough data points, this could give you some hints about the behavior of your algorithm.
Since you already know the expected time complexity of your algorithm, you could then draw a "curve of best fit" (i.e. a line of the shape n^3 that best fits your data). If your data matches the line fairly well, then you were likely correct. If not, it's possible you made some mistake, or that your experimental results are not matching due to factors you are not accounting for.
To determine the equation for the best fitting n^3 line, you could simply take the calculated time complexity, express it as an equation, and guess values for the unknowns until you find an equation that fits. So for n^3, you'd have:
t = a*n^3 + b*n^2 + c*n + d
Find the values of a, b, c, and d that form an equation that best fits your data. If that fit still isn't good enough, then you have a problem.
For more rigorous techniques, you'd have to ask someone more well versed in statistics. I believe the value you'd want to calculate is the coefficient of determination (a.k.a. R^2, basically tells you the variance between the expected and actual results). However, on it's own this value doesn't prove a whole lot. This problem of validating hypothesized relationships between variables is known as Regression Model Validation; the wikipedia article provides a bit more information on how to go further with this if R^2 isn't enough for your purposes.

Categories

Resources