Develop Reference

Recurrence Relation of GetMax() Method

what will be the Recurrence Relation of this method , I don't get it why it is solved as T(n) = T(n-1)+1 ? but the position the one changing (increasing) each recursive call?
private static int getMaxRecursive(int[] arr,int pos) {
if(pos == (arr.length-1)) {
return arr[pos];
} else {
return Math.max(arr[pos], getMaxRecursive(arr, pos+1));
}
}

T(n) is the time of getMaxRecursive(arr,0).
T(n-1) is the time of getMaxRecursive(arr,1).
...
T(1) is the time of getMaxRecursive(arr,arr.length-1).
Where n is the length of the array.
In other words, T(i) is the running time of the method for an array of length i, which is the sub-array of arr starting at index arr.length-i and ending at index arr.length-1.
Therefore
T(n) = T(n-1) + the time of the Math.max() operation (which is constant) = T(n-1) + 1

Recursion vs. Iteration (Fibonacci sequence)

I've got two different methods, one is calculating Fibonacci sequence to the nth element by using iteration and the other one is doing the same thing using recursive method.
Program example looks like this:
import java.util.Scanner;
public class recursionVsIteration {
public static void main(String[] args) {
Scanner sc = new Scanner(System.in);
//nth element input
System.out.print("Enter the last element of Fibonacci sequence: ");
int n = sc.nextInt();
//Print out iteration method
System.out.println("Fibonacci iteration:");
long start = System.currentTimeMillis();
System.out.printf("Fibonacci sequence(element at index %d) = %d \n", n, fibIteration(n));
System.out.printf("Time: %d ms\n", System.currentTimeMillis() - start);
//Print out recursive method
System.out.println("Fibonacci recursion:");
start = System.currentTimeMillis();
System.out.printf("Fibonacci sequence(element at index %d) = %d \n", n, fibRecursion(n));
System.out.printf("Time: %d ms\n", System.currentTimeMillis() - start);
}
//Iteration method
static int fibIteration(int n) {
int x = 0, y = 1, z = 1;
for (int i = 0; i < n; i++) {
x = y;
y = z;
z = x + y;
}
return x;
}
//Recursive method
static int fibRecursion(int n) {
if ((n == 1) || (n == 0)) {
return n;
}
return fibRecursion(n - 1) + fibRecursion(n - 2);
}
}
I was trying to find out which method is faster. I came to the conclusion that recursion is faster for the smaller amount of numbers, but as the value of nth element increases recursion becomes slower and iteration becomes faster. Here are the three different results for three different n:
Example #1 (n = 10)
Enter the last element of Fibonacci sequence: 10
Fibonacci iteration:
Fibonacci sequence(element at index 10) = 55
Time: 5 ms
Fibonacci recursion:
Fibonacci sequence(element at index 10) = 55
Time: 0 ms
Example #2 (n = 20)
Enter the last element of Fibonacci sequence: 20
Fibonacci iteration:
Fibonacci sequence(element at index 20) = 6765
Time: 4 ms
Fibonacci recursion:
Fibonacci sequence(element at index 20) = 6765
Time: 2 ms
Example #3 (n = 30)
Enter the last element of Fibonacci sequence: 30
Fibonacci iteration:
Fibonacci sequence(element at index 30) = 832040
Time: 4 ms
Fibonacci recursion:
Fibonacci sequence(element at index 30) = 832040
Time: 15 ms
What I really want to know is why all of a sudden iteration became faster and recursion became slower. I'm sorry if I missed some obvious answer to this question, but I'm still new to the programming, I really don't understand what's going on behind that and I would like to know. Please provide a good explanation or point me in the right direction so I can find out the answer myself. Also, if this is not a good way to test which method is faster let me know and suggest me different method.
Thanks in advance!

For terseness, Let F(x) be the recursive Fibonacci
F(10) = F(9) + F(8)
F(10) = F(8) + F(7) + F(7) + F(6)
F(10) = F(7) + F(6) + F(6) + F(5) + 4 more calls.
....
So your are calling F(8) twice,
F(7) 3 times, F(6) 5 times, F(5) 7 times.. and so on
So with larger inputs, the tree gets bigger and bigger.

This article does a comparison between recursion and iteration and covers their application on generating fibonacci numbers.
As noted in the article,
The reason for the poor performance is heavy push-pop of the registers in the ill level of each recursive call.
which basically says there is more overhead in the recursive method.
Also, take a look at Memoization

When doing the recursive implementation of Fibonacci algorithm, you are adding redundant calls by recomputing the same values over and over again.
fib(5) = fib(4) + fib(3)
fib(4) = fib(3) + fib(2)
fib(3) = fib(2) + fib(1)
Notice, that fib(2) will be redundantly calculated both for fib(4) and for fib(3).
However this can be overcome by a technique called Memoization, that improves the efficiency of recursive Fibonacci by storing the values, you have calculated once. Further calls of fib(x) for known values may be replaced by a simple lookup, eliminating the need for further recursive calls.
This is the main difference between the iterative and recursive approaches, if you are interested, there are also other, more efficient algorithms of calculating Fibonacci numbers.

Why is Recursion slower?
When you call your function again itself (as recursion) the compiler allocates new Activation Record (Just think as an ordinary Stack) for that new function. That stack is used to keep your states, variables, and addresses. Compiler creates a stack for each function and this creation process continues until the base case is reached. So, when the data size becomes larger, compiler needs large stack segment to calculate the whole process. Calculating and managing those Records is also counted during this process.
Also, in recursion, the stack segment is being raised during run-time. Compiler does not know how much memory will be occupied during compile time.
That is why if you don't handle your Base case properly, you will get StackOverflow exception :).

Using recursion the way you have, the time complexity is O(fib(n)) which is very expensive. The iterative method is O(n) This doesn't show because a) your tests are very short, the code won't even be compiled b) you used very small numbers.
Both examples will become faster the more you run them. Once a loop or method has been called 10,000 times, it should be compiled to native code.

If anyone is interested in an iterative Function with array:
public static void fibonacci(int y)
{
int[] a = new int[y+1];
a[0] = 0;
a[1] = 1;
System.out.println("Step 0: 0");
System.out.println("Step 1: 1");
for(int i=2; i<=y; i++){
a[i] = a[i-1] + a[i-2];
System.out.println("Step "+i+": "+a[i]);
}
System.out.println("Array size --> "+a.length);
}
This solution crashes for input value 0.
Reason: The array a will be initialized 0+1=1 but the consecutive assignment of a[1] will result in an index out of bounds exception.
Either add an if statement that returns 0 on y=0 or initialize the array by y+2, which will waste 1 int but still be of constant space and not change big O.

I prefer using a mathematical solution using the golden number. enjoy
private static final double GOLDEN_NUMBER = 1.618d;
public long fibonacci(int n) {
double sqrt = Math.sqrt(5);
double result = Math.pow(GOLDEN_NUMBER, n);
result = result - Math.pow(1d - GOLDEN_NUMBER, n);
result = Math.round(result / sqrt);
return Double.valueOf(result).longValue();
}

Whenever you are looking for time taken to complete a particular algorithm, it's best you always go for time complexity.
Evaluate the time complexity on the paper in terms of O(something).
Comparing the above two approaches, time complexity of iterative approach is O(n) whereas that of recursive approach is O(2^n).
Let's try to find the time complexity of fib(4)
Iterative approach, the loop evaluates 4 times, so it's time complexity is O(n).
Recursive approach,
fib(4)
fib(3) + fib(2)
fib(2) + fib(1) fib(1) + fib(0)
fib(1) + fib(0)
so fib() is called 9 times which is slightly lower than 2^n when the value of n is large, even small also(remember that BigOh(O) takes care of upper bound) .
As a result we can say that the iterative approach is evaluating in polynomial time, whereas recursive one is evaluating in exponential time

The recursive approach that you use is not efficient. I would suggest you use tail recursion. In contrast to your approach tail recursion keeps only one function call in the stack at any point in time.
public static int tailFib(int n) {
if (n <= 1) {
return n;
}
return tailFib(0, 1, n);
}
private static int tailFib(int a, int b, int count) {
if(count <= 0) {
return a;
}
return tailFib(b, a+b, count-1);
}
public static void main(String[] args) throws Exception{
for (int i = 0; i <10; i++){
System.out.println(tailFib(i));
}
}

I have a recursive solution that you where the computed values are stored to avoid the further unnecessary computations. The code is provided below,
public static int fibonacci(int n) {
if(n <= 0) return 0;
if(n == 1) return 1;
int[] arr = new int[n+1];
// this is faster than using Array
// List<Integer> lis = new ArrayList<>(Collections.nCopies(n+1, 0));
arr[0] = 0;
arr[1] = 1;
return fiboHelper(n, arr);
}
public static int fiboHelper(int n, int[] arr){
if(n <= 0) {
return arr[0];
}
else if(n == 1) {
return arr[1];
}
else {
if( arr[n-1] != 0 && (arr[n-2] != 0 || (arr[n-2] == 0 && n-2 == 0))){
return arr[n] = arr[n-1] + arr[n-2];
}
else if (arr[n-1] == 0 && arr[n-2] != 0 ){
return arr[n] = fiboHelper(n-1, arr) + arr[n-2];
}
else {
return arr[n] = fiboHelper(n-2, arr) + fiboHelper(n-1, arr );
}
}
}

violating the given average time complexity in Big-O notation

I am trying to Implement a solutions to find k-th largest element in a given integer list with duplicates with O(N*log(N)) average time complexity in Big-O notation, where N is the number of elements in the list.
As per my understanding Merge-sort has an average time complexity of O(N*log(N)) however in my below code I am actually using an extra for loop along with mergesort algorithm to delete duplicates which is definitely violating my rule of find k-th largest element with O(N*log(N)). How do I go about it by achieving my task O(N*log(N)) average time complexity in Big-O notation?
public class FindLargest {
public static void nthLargeNumber(int[] arr, String nthElement) {
mergeSort_srt(arr, 0, arr.length - 1);
// remove duplicate elements logic
int b = 0;
for (int i = 1; i < arr.length; i++) {
if (arr[b] != arr[i]) {
b++;
arr[b] = arr[i];
}
}
int bbb = Integer.parseInt(nthElement) - 1;
// printing second highest number among given list
System.out.println("Second highest number is::" + arr[b - bbb]);
}
public static void mergeSort_srt(int array[], int lo, int n) {
int low = lo;
int high = n;
if (low >= high) {
return;
}
int middle = (low + high) / 2;
mergeSort_srt(array, low, middle);
mergeSort_srt(array, middle + 1, high);
int end_low = middle;
int start_high = middle + 1;
while ((lo <= end_low) && (start_high <= high)) {
if (array[low] < array[start_high]) {
low++;
} else {
int Temp = array[start_high];
for (int k = start_high - 1; k >= low; k--) {
array[k + 1] = array[k];
}
array[low] = Temp;
low++;
end_low++;
start_high++;
}
}
}
public static void main(String... str) {
String nthElement = "2";
int[] intArray = { 1, 9, 5, 7, 2, 5 };
FindLargest.nthLargeNumber(intArray, nthElement);
}
}

Your only problem here is that you don't understand how to do the time analysis. If you have one routine which takes O(n) and one which takes O(n*log(n)), running both takes a total of O(n*log(n)). Thus your code runs in O(n*log(n)) like you want.
To do things formally, we would note that the definition of O() is as follows:
f(x) ∈ O(g(x)) if and only if there exists values c > 0 and y such that f(x) < cg(x) whenever x > y.
Your merge sort is in O(n*log(n)) which tells us that its running time is bounded above by c1*n*log(n) when n > y1 for some c1,y1. Your duplication elimination is in O(n) which tells us that its running time is bounded above by c2*n when n > y2 for some c2 and y2. Using this, we can know that the total running time of the two is bounded above by c1*n*log(n)+c2*n when n > max(y1,y2). We know that c1*n*log(n)+c2*n < c1*n*log(n)+c2*n*log(n) because log(n) > 1, and this, of course simplifies to (c1+c2)*n*log(n). Thus, we can know that the running time of the two together is bounded above by (c1+c2)*n*log(n) when n > max(y1,y2) and thus, using c1+c2 as our c and max(y1,y2) as our y, we know that the running time of the two together is in O(n*log(n)).
Informally, you can just know that faster growing functions always dominate, so if one piece of code is O(n) and the second is O(n^2), the combination is O(n^2). If one is O(log(n)) and the second is O(n), the combination is O(n). If one is O(n^20) and the second is O(n^19.99), the combination is O(n^20). If one is O(n^2000) and the second is O(2^n), the combination is O(2^n).

Problem here is your merge routine where you have used another loop which i donot understand why, Hence i would say your algorithm of merge O(n^2) which changes your merge sort time to O(n^2).
Here is a pseudo code for typical O(N) merge routine :-
void merge(int low,int high,int arr[]) {
int buff[high-low+1];
int i = low;
int mid = (low+high)/2;
int j = mid +1;
int k = 0;
while(i<=mid && j<=high) {
if(arr[i]<arr[j]) {
buff[k++] = arr[i];
i++;
}
else {
buff[k++] = arr[j];
j++;
}
}
while(i<=mid) {
buff[k++] = arr[i];
i++;
}
while(j<=high) {
buff[k++] = arr[j];
j++;
}
for(int x=0;x<k;x++) {
arr[low+x] = buff[x];
}
}

Big O notation for fib recursion

What is the Big-O run-time of the following function? Explain.
static int fib(int n){
if (n <= 2)
return 1;
else
return fib(n-1) + fib(n-2)
}
Also how would you re-write the fib(int n) function with a faster Big-O run-time iteratively?
would this be the best way with O(n):
public static int fibonacci (int n){
int previous = -1;
int result = 1;
for (int i = 0; i <= n; ++i)
{
int sum = result + previous;
previous = result;
result = sum;
}
return result;
}
}

Proof
You model the time function to calculate Fib(n) as sum of time to calculate Fib(n-1) plus the time to calculate Fib(n-2) plus the time to add them together (O(1)).
T(n<=1) = O(1)
T(n) = T(n-1) + T(n-2) + O(1)
You solve this recurrence relation (using generating functions, for instance) and you'll end up with the answer.
Alternatively, you can draw the recursion tree, which will have depth n and intuitively figure out that this function is asymptotically O(2n). You can then prove your conjecture by induction.
Base: n = 1 is obvious
Assume T(n-1) = O(2n-1), therefore
T(n) = T(n-1) + T(n-2) + O(1) which is equal to
T(n) = O(2n-1) + O(2n-2) + O(1) = O(2n)
Iterative version
Note that even this implementation is only suitable for small values of n, since the Fibonacci function grows exponentially and 32-bit signed Java integers can only hold the first 46 Fibonacci numbers
int prev1=0, prev2=1;
for(int i=0; i<n; i++) {
int savePrev1 = prev1;
prev1 = prev2;
prev2 = savePrev1 + prev2;
}
return prev1;

Tail call optimization for fibonacci function in java

I was studying about Tail call recursion and came across some documentation that mentioned. Sun Java doesn't implement tail call optimization.
I wrote following code to calculate fibonacci number in 3 different ways:
1. Iterative
2. Head Recursive
3. Tail Recursive
public class Fibonacci {
public static void main(String[] args) throws InterruptedException {
int n = Integer.parseInt(args[0]);
System.out.println("\n Value of n : " + n);
System.out.println("\n Using Iteration : ");
long l1 = System.nanoTime();
fibonacciIterative(n);
long l2 = System.nanoTime();
System.out.println("iterative time = " + (l2 - l1));
System.out.println(fibonacciIterative(n));
System.out.println("\n Using Tail recursion : ");
long l3 = System.nanoTime();
fibonacciTail(n);
long l4 = System.nanoTime();
System.out.println("Tail recursive time = " + (l4 - l3));
System.out.println(fibonacciTail(n));
System.out.println("\n Using Recursion : ");
long l5 = System.nanoTime();
fibonacciRecursive(n);
long l6 = System.nanoTime();
System.out.println("Head recursive time = " + (l6 - l5));
}
private static long fibonacciRecursive(int num) {
if (num == 0) {
return 0L;
}
if (num == 1) {
return 1L;
}
return fibonacciRecursive(num - 1) + fibonacciRecursive(num - 2);
}
private static long fibonacciIterative(int n) throws InterruptedException {
long[] arr = new long[n + 1];
arr[0] = 0;
arr[1] = 1;
for (int i = 2; i <= n; i++) {
// Thread.sleep(1);
arr[i] = arr[i - 1] + arr[i - 2];
}
return arr[n];
}
private static long fibonacciTail(int n) {
if (n == 0)
return 0;
return fibHelper(n, 1, 0, 1);
}
private static long fibHelper(int n, int m, long fibM_minus_one, long fibM) {
if (n == m)
return fibM;
return fibHelper(n, m + 1, fibM, fibM_minus_one + fibM);
}
}
On running this program I drew some results:
Head Recursive method does not finish for n>50. Program looked like hanged. Any idea, why this could happen?
Tail recursive method took significantly less time as compared to Head recursion. Sometimes took even less time than Iterative method. Does it mean that java does some Tail call optimization internally?
And if it does, why I did it give StackOverflowError at n > 5000?
System specs:
Intel core 5 processor,
Windows XP,
32 bit Java 1.6
Default stack size for JVM.

Does it mean that java does some Tail call optimization internally?
No, it does not. The HotSpot JIT compilers do not implement tail-call optimization.
The results you are observing are typical of the anomalies that you see in a Java benchmark that doesn't take account of JVM warmup. For instance, the "first few" times a method is called, it will be executed by the interpreter. Then the JIT compiler will compile the method ... and it will get faster.
To get meaningful results, put a loop around the whole lot and run it a number of times until the timings stabilize. Then discard the results from the early iterations.
... why I did it give StackOverflowError at n > 5000?
That's just evidence that there isn't any tail-call optimization happening.

For the first question, what is 2^50 (or something close)? Each number N in a recursive Fib function calls it twice (prior 2). Each of those calls 2 prior iterations, etc.. so it's grows to 2^(N-k) of recursion (k is probably 2 or 3).
The 2nd question is because the 2nd one is a straight N recursion. Instead of going double headed (N-1),(N-2), it simply builds up from M=1, M=2... M=N. Each step of the way, the N-1 value is retained for adding. Since it is an O(N) operation, it is comparable to the iterative method, the only difference being how the JIT compiler optimizes it. The problem with recursion though is that it requires a huge memory footprint for each level that you stack onto the frame - you run out of memory or stack space at some limit. It should still generally be slower than the iterative method.

Regarding point 1: Computing Fibonacci numbers recursively without memoization leads to a run time that is exponential in n. This goes for any programming language that does not automatically memoize function results (such as most mainstream non-functional languages, e.g. Java, C#, C++, ...). The reason is that the same functions will get called over and over again - e.g. f(8) will call f(7) and f(6); f(7) will call f(6) and f(5), so that f(6) gets called twice. This effect propagates and causes an exponential growth in the number of function calls. Here's a visualization of which functions get called:
f(8)
f(7)
f(6)
f(5)
f(4)
...
f(3)
...
f(4)
...
f(5)
f(4)
...
f(3)
...
f(6)
f(5)
...
f(4)
...

You can use Memoization to avoid head recursion.
I have tested the following code , when N <=40 , that approach is bad because Map has trade-off.
private static final Map<Integer,Long> map = new HashMap<Integer,Long>();
private static long fibonacciRecursiveMemoization(int num) {
if (num == 0) {
return 0L;
}
if (num == 1) {
return 1L;
}
int num1 = num - 1;
int num2 = num - 2;
long numResult1 = 0;
long numResult2 = 0;
if(map.containsKey(num1)){
numResult1 = map.get(num1);
}else{
numResult1 = fibonacciRecursiveMemoization(num1);
map.put(num1, numResult1);
}
if(map.containsKey(num2)){
numResult2 = map.get(num2);
}else{
numResult2 = fibonacciRecursiveMemoization(num2);
map.put(num2, numResult2);
}
return numResult1 + numResult2;
}
when the value of n : 44
Using Iteration :
iterative time = 6984
Using Tail recursion :
Tail recursive time = 8940
Using Memoization Recursion :
Memoization recursive time = 1799949
Using Recursion :
Head recursive time = 12697568825