I've been trying to determine the time complexity of nested for loops
here is the problem:
I've done some work , but yet can't reach a decisive answer. I've reached a sequence of(log(100)+ log(200)+....log(n/4)), yet can't determine the sum of it.
Update
I have an idea, I'm uploading it it as a picture, please tell me if it's correct:
Basically, what you have here is a series. To be able to calculate the sum, you should write the series in a uniform notation and either calculate the sum yourself or use some tool (such as Wolfram Alpha) to compute the series for you.
In this case, the format is: sum from i = 1 to (n/400) of log(100 * i).
https://www.wolframalpha.com/input/?i=sum+log_2(100+*+i),+i+%3D+1+to+n%2F400
I think you might be over thinking it. You have two nested loops. Clearly the outer one is O(n):
for (int i = 0; i < n / 4; i += 100)
It has a low constant factor, a constant factor that is making it faster, but that does not mean that it is not O(n)
In the inner loop
for (int j = 1; j < i; j *= 2)
j keeps doubling until it is larger than i so it runs k times if k is the first power of two greater than i, so 2^k ≥ i, or k is the base-2 log of i. That means this loop is O(log n) since i is linearly related to n. Thus the whole thing is O(n log n)
Related
So the algorithm for which the time complexity is to estimated is this
int a = 0;
for (i = 0; i < n; i++) //runs from i-n;
for (j = i; j < i * i; j++) // runs from i - i*i;
for (k = 0; k < j; k++) //runs from 0 - j;
a++;
I have commented the basic details that i have understood about the algorithm.
The outer loop clearly runs O(n) times and first inner loop runs from 'i' to 'i*i'
The second inner loop runs '0' to 'j'
I also know i have to apply the rule of products.
But I am unable to calculate the time complexity of the two inner loops relative to 'n'.
Please correct me if I'm wrong with my explanation.
Whenever there is a doubt always use mathematics as the mathematical proofs are more palatable like strong intuitions.
Let n = size of input.
The innermost loop or lets call it as the 3rd loop runs for j+1 number of times.
The inner loop or 2nd loop runs for i*i - i number of times.
Finally, the outer loop for n times.
So, the number of iterations for the entire program can be expressed in mathematical terms as shown in the following figure:
When you have multiple levels of for loops, analyze the complexity of each loop in isolation and then multiply them together.
In this example, the complexity of the first loop is O(n), like you said.
The complexity of the second loop is O(n^2) because in the worst case, the number of operations you have to perform is on the order of i*i which could be as big as n^2. It doesn't matter that it doesn't start at zero either because in an expression like O(n^2 - n), everything but the highest order term gets ignored.
The third loop also takes O(n^2) because in the worst case, you could have as many as j operations would again could be as big as n^2.
Lastly a++ happens in constant time. Multiply everything together and you have a complexity of O(n^5).
I am currently revising some past exam papers on Data structures and algorithms and I have come over a question I can't seem to wrap my head around. Can anyone help me here please? Or even point me in the right direction? Thanks.The question is below.
"Consider the Java method below. How many lines will be printed
out when n is 10? How about when n is 100? In light of this,
derive an expression for how many lines will be printed out in
terms of the input parameter n. State the Big O complexity of the
method and prove that this is the case using the mathematical
definition."
public void method (int n){
for(int i = 0; i < (n - 5); i++) {
for(int j = 1000; j > n; j--) {
System.out.println("One step");
}
}
}
It's tricky. Since the inner loop will be executed at most 1000 times (assuming n must be positive), you can say it runs a constant number of times. The outer loop runs n-5 times, which is O(n).
Therefore the total running time is O(n).
For small values of n (< 1000), the inner loop runs 1000 - n times, but that doesn't make a difference on the asymptotic running time, since that number goes to 0 when n is higher than 1000.
How many lines will be printed out when n is 10? How about when n is 100?
Those numbers have no relation to "the Big O complexity of the method", since for most values of n the number of lines printed would be 0.
I'm new to Java, my question is on big-O complexity.
For a), it is clearly O(n^2) for a nested loop.
for ( int i = 0; i < n; i++)
for ( int j=0; j < n; j++ )
however, for b), with sum++ operation at the end, and complications in the nested loop, does that change the its Big-O complexity at all?
int sum = 0;
for ( int i = 1; i <= n; i++)
for ( int j = n; j > 0; j /= 2)
sum++;
In your second example, the first for iterate n times, and the second iterate log2(n) times, since you divide j by 2 in each iteration. Therefore, the complexity is O(n log2 n).
sum++ in the last line does not affect the complexity.
Well, at first: The Big O notation is not related to the programming language, it only depends on the implementation. So it is irrelevant if you do the loops in Java or any other language (besides some optimizations performed by the interpreter/compiler, which will, however, change the program flow)
For the nested loop:
1) sum++ is considered as constant operation with a cost of 1. Therefore it can be neglected. O(n * 1) = O(n)
2) The outer loop stays roughly the same having O(n-1) which is equal to O(n) as constant terms can always be neglected in asymptotic calculations.
3) The inner loop takes log2(n) steps to finish. In every step n is reduced to its half, therefore multiplying by 2 takes one step back, which leads to the binary logarithm.
In summary, the complexity is O(n log2(n))
EDIT: funny thing: no one so far has seen that the inner loop has a real complexity of O(infinity) as the division of some positive number is always greater then 0... Or am I wrong here?
Obviously in your example b), the inner loop does fewer iterations than in a). Halving the iteration counter on each iteration is essentially a logarithmic (log2) operation, so the O-complexity of example b) is O( n log2(n)).
Note, this is not specific to Java - it would be the same in any other language :-)
Cheers,
Speaking from a purely computer science point of view, the big O(see the definition), describes a worst case scenario. This even means if you half the inner for loop, the O(n^2) still applies. However you can write it as a less rising function.
I have this example algorithm:
int sum = 0;
int j = 1;
while (j <= n) {
sum++;
j = j * 2;
}
The book I am reading, "Building Java Programs - a Back to the Basics Approach" tells me that I need to find this:
Approximate the runtime of the following code fragment, in terms of n: Write your answer in a format such as O(N^2) or O(N log N).
I don't seem to understand how to get from point a to point b here. I figured two statements = O(2), and a loop with two statements = O(2N) so it should be O(2N + 2). Where am I going wrong?
When determining complexities, we don't include constants or coefficients. Instead of O(2N + 2), it should be O(n). We only care about numbers if they're exponential, i.e. 2^n or n^2, log2(n), etc.
Putting that aside, are you sure this is O(n)? O(n) would mean that it runs n times, but it looks like j is going to catch up to n before n times. See what I'm saying?
EDIT: Ok, here's what's going on.
Look what happens with j. j = j * 2. j is doubling every time. In other words, the difference between j and n is being halved. When the number of iterations remaining is halved on every iteration, that's called a log(n) algorithm. log(n) algorithms are pretty awesome because even if n is extremely large, log(n) is surprisingly small. Plug some numbers in to see what I mean.
In order to determine the complexity, you are on the right track with trying to find the number of operations! What I usually do if I can't pull up the formula immediately: I find examples first. Try it out for n = 5, n = 8, n=64, n=65 and come up with the number of steps taken, and you'll see soon the pattern!
The formula for number of operations you'll come up with can be very precise, but you can leave the constants out to determine the order of complexity (multipliers and additional constants), as in the end O(2n) is roughly the same as O(3n)compared to O(n^2).
The loop does not iterate through every value from 1 to N, which is where your O(2N) is going wrong.
It might help to visualize the runtime if you substitute the program with the following, which will have the same complexity - this is similar to algorithms that halve the input size on each iteration
int j = n;
while(j > 1) {
j = j / 2;
}
I'd say break it down; this sort of problem looks deceptively straightforward at once, but proving it out would be valuable.
Let's assume n = 100.
First iteration: sum = 1, j = 2
Second iteration: sum = 2, j = 4
Third iteration: sum = 3, j = 8
Fourth iteration: sum = 4, j = 16
Fifth iteration: sum = 5, j = 32
Sixth iteration: sum = 6, j = 64
Seventh iteration: sum = 7, j = 128 # end of iterations.
If this ran in O(N) time, then you would expect about a hundred iterations. We instead increase j by a factor of 2 every time instead, reaching the terminal iteration after about seven times.
That said, we iterate at least once (for n > 1), plus the floor of log(100)/log(2) = 6, for a total of 7 times.
I am inclined to believe that, based on running this little test, that the runtime of this is O(log(n)), and not O(n).
Recall from this graph; logarithms grow much slower over time given large values of N. Even with n = 1000, you would only see 10 iterations.
Your question asks to approximate the time, an exact time or formula is not required.
Also, Big O Notation, e.g. O(N), tends to be simplified to the most dominant term only (i.e. the term of N whereby if N is very large, that term alone would provide an exact or nearly exact answer).
So, if your exact formula were, say, 4N^2 + 9N + 7, your answer would be O(N^2). Because if N was very large, the N^2 term alone would be enough to approximate the answer.
Also, yts has a point about the j variable, as it's changing exponentially, so from your code the time, t will be:
t = 2(sum) + 2 = 2(log2(n)+1) + 2
Therefore your solution is maybe, of the form O(log N), or perhaps O(log2 N)?
I am taking up algorithm course on coursera,and I am stuck on this particular problem. I am supposed to find the time complexity of this code.
int sum = 0
for (int i = 1; i <= N*N; i = i*2)
{
for (int j = 0; j < i; j++)
sum++;
}
I checked it in eclipse itself, for any value of N the number of times sum statement is executed is less than N
final value of sum:
for N=8 sum=3
for N=16 sum=7
for N=100000 sum=511
so the time complexity should be less than N
but the answer that is given is N raised to the power 2, How is it possible?
What I have done so far:
the first loop will run log(N^ 2) times, and consequently second loop will be execute 1,2,3.. 2 logN
The first inner loop will be 1 + 2 + 4 + 8 .. 2^M where 2^M is <= N * N.
The sum of powers of 2 up to N * N is approximately 2 * N * N or O(N ^ 2)
Note: When N=100000 , N*N will overflow so its result is misleading. If you consider overflow to be part of the problem, the sum is pretty random for large numbers so you could argue its O(1), i.e. if N=2^15, N^2 = 2^30 and the sum will be Integer.MAX_VALUE. No higher value of N will give a higher sum.
There is a lot of confusion here, but the important thing is that Big-O notation is all about growth rate, or limiting behavior as mathematicians will say. That a function will execute in O(n*n) means that the time to execute will increase faster than, for example n, but slower than, for example 2^n.
When reasoning with big-O notation, remember that constants "do not count". There is a few querks in this particular question.
The N*N expression it-self would lead to a O(log n*n) complexity if the loop was a regular for-loop...
...however, the for-loop increment is i = i*2 causing the outer loop to be executed approximately log n and the function would be in O(log n) if the contents of the inner loop run in a time independent of n.
But again, the inner-loop run-time depends on n, but it doesn't do n*n runs, rather it does roughly log (n*n)/2 loops. Remembering that "constants don't count" and that we end up in O(n*n).
Hope this cleared things up.
So sum ++ will be executed 1 + 2 + 4 + 8 + ... + N*N, total log2(N*N) times. Sum of geometrical progression 1 * (1 - 2 ^ log2(N*N)/(1 - 2) = O(N*N).
Your outer loop is log(N^2)->2*log(N)->log(N), your inner loop is N^2/2->N^2. So, the time complexity is N^2*log(N).
About the benchmark, values with N=8 or N=16 are ridiculous, the time in the loop is marginal in relation with setting JVM, cache fails, and so on. You must:
Begin with biggest N, and check how it evaluate.
Make multiple runs with each value of N.
Think that the time complexity is a measure of how the algorithm works when N becomes really big.