I have to implement a multi threaded Merge Sort and Quick sort in Java for my algorithms class and compare them to my single threaded versions. However, I have never multithreaded before.
Is the code I have able to be multi threaded or do I have to start again?
Here is my code for the single thread algorithms
Merge Sort. the sort() method is part of the strategy pattern I have to implement.
#Override
public int[] sort(int[] list) {
int array_size = list.length;
list = msort(list, 0, array_size-1);
return list;
}
int[] msort(int numbers[], int left, int right) {
int mid;
if (left<right) {
mid = (right + left) / 2;
msort(numbers, left, mid);
msort(numbers, mid+1, right);
merge(numbers, left, mid, mid+1, right);
}
return numbers;
}
void merge(int numbers[], int startA, int endA, int startB, int endB) {
int finalStart = startA;
int finalEnd = endB;
int indexC = 0;
int[] listC = new int[numbers.length];
while(startA <= endA && startB <= endB){
if(numbers[startA] < numbers[startB]){
listC[indexC] = numbers[startA];
startA = startA+1;
}
else{
listC[indexC] = numbers[startB];
startB = startB +1;
}
indexC++;
}
if(startA <= endA){
for(int i = startA; i < endA; i++){
listC[indexC]= numbers[i];
indexC++;
}
}
indexC = 0;
for(int i = finalStart; i <= finalEnd; i++){
numbers[i]=listC[indexC];
indexC++;
}
}
Here is my quick sort
#Override
public int[] sort(int[] list) {
int[] array = quickSort(list, 0, list.length-1);
return array;
}
int partition(int arr[], int left, int right)
{
int i = left, j = right;
int tmp;
int pivot = arr[(left + right) / 2];
while (i <= j) {
while (arr[i] < pivot)
i++;
while (arr[j] > pivot)
j--;
if (i <= j) {
tmp = arr[i];
arr[i] = arr[j];
arr[j] = tmp;
i++;
j--;
}
};
return i;
}
int[] quickSort(int arr[], int left, int right) {
int index = partition(arr, left, right);
if (left < index - 1)
quickSort(arr, left, index - 1);
if (index < right)
quickSort(arr, index, right);
return arr;
}
Cheers!
Short answer - yes, these algorithms can be converted to be multi-threaded without starting from scratch (so far as I can see).
The key elements that make these "easy" to parallelize are:
There are two recursive calls within each implementation
Those two recursive calls operate on separate pieces of data - they shouldn't conflict with each other (e.g. even when working inside the same array, they're operating on different indices)
The method making those recursive calls cannot proceed until both are complete
That it doesn't matter which order those two calls are made in
That's answered some of your questions, hopefully.
Some more advice, not sure how useful this will be:
If you put both recursive calls into a new thread, then the current thread will be idle while waiting for them both to complete
When the number of elements left to work on is small, the overheads of threading may be higher than the gains.
You might want to throttle the number of threads being used for this task in general - you might want to use some form of thread pool or work queue, with a fixed number of threads.
A major advice in this case (a mistake that I've made when I was in your shoes and I've seen many others do it) is to not let the number of threads grow unchecked. Remember that if you start one thread per recursion branch, the main thread will spawn one child thread (assuming one recursive call is done on the main itself), the child thread will spawn an additional thread and so on until you will choke the system if your data set is large.
A more clear alternative would be to start one thread per recursive call, such that each thread spawns two child threads and then joins them.
Either way, make sure to set a limit on the recursion depth that spawns threads (let's say equal to the number of CPU cores) and if the limit is exceeded, call the sort method sequentially on the rest of the levels.
Related
I'm writing a quicksort algorithm to sort numbers with random pivot. How can I change my quicksort methods from recursive to iterative? I have one sort method which is recursive, but I need the iterative method. Is it possible to change from recursive to iterative in just the sort method or do I have to change the whole code?
here is all my code
public class Main {
public static void main(String[] args) {
long start = System.currentTimeMillis();
ArrayList time = new ArrayList();
for (int k = 1; k < 1000000; k++) {
time.add(k);
}
int[] tall = new int[1000000];
int index = 0;
int n = 1000000;
File text = new File("/Users/sasan/IdeaProjects/File.txt");
try {
Scanner scan = new Scanner(text);
while (scan.hasNextLine() && index < 1000000) {
tall[index] = scan.nextInt();
index++;
}
scan.close();
} catch (IOException e) {
System.out.println("Problem with file");
e.printStackTrace();
}
int l = tall.length;
sort(tall, 0, l-1);
System.out.println("Sorted array");
printArray(tall);
System.out.println("");
long end = System.currentTimeMillis();
System.out.print("Execution Time is: ");
System.out.print((end - start));
}
static void random(int tall[],int low,int high)
{
Random rand= new Random();
int pivot = rand.nextInt(high-low)+low;
int temp1=tall[pivot];
tall[pivot]=tall[high];
tall[high]=temp1;
}
/* This function takes last element as pivot,
places the pivot element at its correct
position in sorted array, and places all
smaller (smaller than pivot) to left of
pivot and all greater elements to right
of pivot */
static int partition(int tall[], int low, int high)
{
// pivot is choosen randomly
random(tall,low,high);
int pivot = tall[high];
int i = (low-1); // index of smaller element
for (int j = low; j < high; j++)
{
// If current element is smaller than or
// equal to pivot
if (tall[j] < pivot)
{
i++;
// swap arr[i] and arr[j]
int temp = tall[i];
tall[i] = tall[j];
tall[j] = temp;
}
}
// swap arr[i+1] and arr[high] (or pivot)
int temp = tall[i+1];
tall[i+1] = tall[high];
tall[high] = temp;
return i+1;
}
/* The main function that implements QuickSort()
tall[] --> Array to be sorted,
low --> Starting index,
high --> Ending index */
static void sort(int tall[], int low, int high)
{
if (low < high)
{
/* pi is partitioning index, tall[pi] is
now at right place */
int pi = partition(tall, low, high);
// Recursively sort elements before
// partition and after partition
sort(tall, low, pi-1);
sort(tall, pi+1, high);
}
}
/* A utility function to print array of size n */
static void printArray(int tall[])
{
int n = tall.length;
for (int i = 0; i < n; ++i)
System.out.print(tall[i]+" ");
System.out.println();
}
// Driver Code
}
You can generally always turn a recursive method into an iterative method, though the point about using recursion is that you usually choose it when it's most convenient to do so. A divide-and-conquer algorithm such as quicksort is precisely a good candidate for recursion.
To turn a recursive method into an iterative method, you basically use a 'stack' object (and Java provides the Stack class to assist with this): this essentially stores what parameters need to be operated on on each iteration. Then, whereas before you would have called into your recursive method, you put on top of your stack the parameters that would have been used to call it. Each iteration does the equivalent of "returning" from the method by taking the latest parameters off the top of the stack. (Essentially, you're simulating the process that Java goes through under the hood when you call a method.)
So the pattern looks like this:
Stack<Paraneters> stack = new Stack();
stack.push(new Parameters(0, size);
while (!stack.empty()) {
Parameters params = stack.pop();
int lo = params.lo;
int hi = params.hi;
if (lo < hi) {
int pivot = partition(lo, hi);
stack.push(new Parameters(lo, pivot-1));
stack.push(new Parameters(pivot+1, hi));
}
}
For this simple example, the Parameters object could actually just be a 2-length int array, but for more complex combinations of parameters you may end up constructing a wrapper class. (Another reason why using recursion can be more convenient in practice.)
In theory it should be possible to write every recursive method into an iterative one.
I know it's pretty easy when the recursive function is tail-recursive. For example, calculating a factorial can be done like this recursive:
// Recursive:
long fact(long n){
return fact(n, 1);
}
long fact(long n, long r){
return n==1 ?
r
:
fact(n-1, r*n);
}
Try it online.
or like this iterative:
// Iterative:
int fact(int n){
int r = 1;
for(; n>1; n--)
r *= n;
return r;
}
Try it online.
This is pretty easy, because we don't use the result of the recursive method for something else, and we only have a single recursive call, where we decrease n by 1 ever iteration.
I also know you should keep a stack for most recursive to iterative conversions. For example, doing a quick-sort can be done like this recursive:
// Recursive:
void quicksort(int[] array, int left, int right){
if(left >= right) return;
int index = partition(array, left, right);
quicksort(array, left, index-1);
quicksort(array, index+1, right);
}
Try it online.
or like this iterative:
// Iterative:
void quicksort(int[] array, int left, int right){
int[] stack = new int[1024]; // Example size, alternative an actual java.util.Stack could be used
int i=0;
stack[i++] = left;
stack[i++] = right;
while(i>0){
right = stack[--i];
left = stack[--i];
if(left >= right) continue;
int index = partition(array, left, right);
stack[i++] = left;
stack[i++] = index-1;
stack[i++] = index+1;
stack[i++] = right;
}
}
Try it online.
But I now want to convert the following recursive method into its iterative form as well:
// Recursive:
int f(int n){
return n<1 ?
0
:
n%2+1 + 3*f(n/2);
}
Try it online.
In this case it uses the recursive result, and multiply that result by 3. I'm not entirely sure how to make this iterative as well. I tried something like this:
// Iterative:
int f(int n){
int[] stack = new int[1024]; // Example size, alternative an actual java.util.Stack could be used
int i=0;
stack[i++] = n;
while(i > 0){
n = stack[--i];
if(n < 1)
stack[i++] = 0;
stack[i++] = n%2+1 + 3*stack[--i];
}
return stack[0];
}
Which obviously it not going to work, since i=1 before it enters the while, then it becomes 0 at n = stack[--i], then 1 again at stack[i++] = n%2+1 + ..., and then 0 again at 3*stack[--i], so it stops after the first iteration, and simply returns stack[0]. How should I convert this recursive method above to an iterative one when I use the result of the recursive-call with other calculations (multiplying by 3 in this case)?
As for the reason why: I want to port this recursive method above to a stack-based language which doesn't have any functions and therefore I'll need the iterative approach (with stack).
Ok, I've been able to figure it out. I first need to fill the entire stack with the n%2+1 parts, dividing 2 every iteration. And as soon as n < 1, I need an inner loop to calculate the result.
Here is what I ended up with:
// Recursive:
int f(int n){ // i.e. n=2
int[] stack = new int[1024]; // Example size, alternative an actual java.util.Stack could be used
int i=0;
while(i >= 0){
stack[i++] = n%2+1;
n/=2;
if(n < 1){
while(i > 0){
n = 3*n + stack[--i];
}
return n;
}
}
return -1; // It should never come here, but the method need a return-statement
}
Try it online.
I ended up with something quiet similar (saw your self-answer only after)
int iterf(int n){
//can also be implemented with an array. easier with list.
List<Integer> sum = new ArrayList<>();
while (true) {
if( n < 1 ) { //equivalent to if(n<1) return 0;
sum.add(0);
break;
}
//for each n, fill list with the fixed argument part: (n%2)+1
sum.add((n%2)+1);
n=n/2;
}
//add to each list element 3 time the next element
//equivalent to 3*f(n/2);
for(int i = sum.size()-2; i >=0; i--) {
sum.set(i, sum.get(i) + (3*(sum.get(i+1))));
}
return sum.get(0);
}
So I think my partition method works but I cannot understand or figure out how to do the kthSmallest method. I no longer get out of bounds errors with my partition method which leads me to think that it works and with testing it seems to work. However, my kthSmallest method often gets stuck in an infinite loop and when it does return a value, it is never the correct value.
I have seen examples online that place the pivot between the two subarrays however for our assignment the pivot is always at the end so I often get confused looking at these examples.
Here is what I have:
class Median{
static int kthSmallest(int[] arr, int left, int right, int k){
int divider = partition(arr, left, right);
if(divider == k-1){
return arr[right];
}
else if(divider > k-1){
return kthSmallest(arr, left, divider-1, k);
}
else{
return kthSmallest(arr, divider, right, (k - divider-1));
}
}
static int partition(int[] arr, int left, int right){
int pivot = arr[right];
int index = left;
for(int i = left; i < right-1; i++){
if(arr[i] < pivot){
swap(arr, index, i);
index++;
}
}
printArr(arr);
System.out.println("divider: " + index);
return index;
}
static void swap(int[] arr, int i, int j){
int temp = arr[i];
arr[i] = arr[j];
arr[j] = temp;
}
static void printArr(int[] arr){
System.out.println();
for(int index = 0; index < arr.length - 1; index++){
System.out.print(arr[index] + ", ");
}
System.out.print(arr[arr.length-1]);
System.out.println();
}
public static void main(String[] args){
int[] arr = {34,-1,0,5,3};
printArr(arr);
//System.out.println(partition(arr, 0, arr.length-1));
//printArr(arr);
System.out.println(kthSmallest(arr, 0, arr.length - 1, 2));
}
}
It seems like what you are trying to do is implement the QuickSelect algorithm, so I recommend taking a look at the Wikipedia article.
Looking at your code, I think you have misinterpreted the "pivot at the end" bit. I believe that what your requirements want is to select the pivot from the end, and then place it in between the two sublists so that your list looks right. For example,
34 -1 0 5 3
should become
-1 0 3 34 5
not
-1 0 34 5 3, where your pivot is not doing its job correctly.
There are also a few problems with your kthSmallest method. Double check the values you pass along in your recursion and you also are missing a case which might cause infinite recursion. Spoilers for if you are absolutely stuck:
You since you don't re-index the list, you should change '(k - divider -1)' to just k.
If left == right then you get unnecessary recursion, so you should just return in that case.
In your partition method, make sure you iterate far enough in the part of the list you are partitioning. Again, just in case you're really stumped:
for(int i = left; i < right-1; i++) should become for(int i = left; i <= right-1; i++)
For refreshing some Java I tried to implement a quicksort (inplace) algorithm that can sort integer arrays. Following is the code I've got so far. You can call it by sort(a,0,a.length-1).
This code obviously fails (gets into an infinite loop) if both 'pointers' i,j point each to an array entry that have the same values as the pivot. The pivot element v is always the right most of the current partition (the one with the greatest index).
But I just cannot figure out how to avoid that, does anyone see a solution?
static void sort(int a[], int left, int right) {
if (right > left){
int i=left, j=right-1, tmp;
int v = a[right]; //pivot
int counter = 0;
do {
while(a[i]<v)i++;
while(j>0 && a[j]>v)j--;
if( i < j){
tmp = a[i];
a[i] = a[j];
a[j] = tmp;
}
} while(i < j);
tmp = a[right];
a[right] = a[i];
a[i] = tmp;
sort(a,left,i-1);
sort(a,i+1,right);
}
}
When preforming a Quicksort I strongly suggest making a separate method for partitioning to make the code easier to follow (I'll show an example below). On top of this a good way of avoiding worst case run time is shuffling the array you're sorting prior to preforming the quick sort. Also I used the first index as the partitioning item instead of the last.
For example:
public static void sort (int[] a)
{
StdRandom.shuffle(a);
sort(a, 0, a.length - 1);
}
private static void sort(int[] a, int lo, int hi)
{
if (hi <= lo) return;
int j = partition(a, lo, hi) // the addition of a partitioning method
sort(a, lo, j-1);
sort(a, j+1, hi);
}
private static int partition(int[] a, int lo, int hi)
{
int i = lo, j = hi + 1, tmp = 0;
int v = a[lo];
while (true)
{
while (a[i++] < v) if (i == hi) break;
while (v < a[j--]) if (j == lo) break;
if (i >= j) break;
tmp = a[i];
a[i] = a[j];
a[j] = tmp;
}
tmp = a[lo];
a[lo] = a[j];
a[j] = temp;
return j;
}
On top of this if you want a really good example on how Quicksort works (as a refresher) see here.
This should work (will check for correctness in a bit, it works!):
EDIT: I previously made a mistake in error checking. I forgot to add 2 more conditions, here is the amended code.
public static void main (String[] args) throws java.lang.Exception
{
int b[] = {10, 9, 8, 7, 7, 7, 7, 3, 2, 1};
sort(b,0,b.length-1);
System.out.println(Arrays.toString(b));
}
static void sort(int a[], int left, int right) {
if (right > left){
int i=left, j=right, tmp;
//we want j to be right, not right-1 since that leaves out a number during recursion
int v = a[right]; //pivot
do {
while(a[i]<v)
i++;
while(a[j]>v)
//no need to check for 0, the right condition for recursion is the 2 if statements below.
j--;
if( i <= j){ //your code was i<j
tmp = a[i];
a[i] = a[j];
a[j] = tmp;
i++;
j--;
//we need to +/- both i,j, else it will stick at 0 or be same number
}
} while(i <= j); //your code was i<j, hence infinite loop on 0 case
//you had a swap here, I don't think it's needed.
//this is the 2 conditions we need to avoid infinite loops
// check if left < j, if it isn't, it's already sorted. Done
if(left < j) sort(a,left,j);
//check if i is less than right, if it isn't it's already sorted. Done
// here i is now the 'middle index', the slice for divide and conquer.
if(i < right) sort(a,i,right);
}
}
This Code in the IDEOne online compiler
Basically we make sure that we also swap the value if the value of i/j is the same as the pivot, and break out of the recursion.
Also there was a check in the pseudocode for the length, as if we have an array of just 1 item it's already sorted (we forgot the base case), I thought we needed that but since you pass in the indexes and the entire array, not the subarray, we just increment i and j so the algorithm won't stick at 0 (they're done sorting) but still keep sorting an array of 1. :)
Also, we had to add 2 conditions to check if the array is already sorted for the recursive calls. without it, we'll end up sorting an already sorted array forever, hence another infinite loop. see how I added checks for if left less than j and if i less than right. Also, at that point of passing in i and j, i is effectively the middle index we split for divide and conquer, and j would be the value right before the middle value.
The pseudocode for it is taken from RosettaCode:
function quicksort(array)
if length(array) > 1
pivot := select any element of array
left := first index of array
right := last index of array
while left ≤ right
while array[left] < pivot
left := left + 1
while array[right] > pivot
right := right - 1
if left ≤ right
swap array[left] with array[right]
left := left + 1
right := right - 1
quicksort(array from first index to right)
quicksort(array from left to last index)
Reference: This SO question
Also read this for a quick refresher, it's implemented differently with an oridnary while loop
This was fun :)
Heres some simple code I wrote that doesn't initialize to many pointers and gets the job done in a simple manner.
public int[] quickSort(int[] x ){
quickSortWorker(x,0,x.length-1);
return x;
}
private int[] quickSortWorker(int[] x, int lb, int ub){
if (lb>=ub) return x;
int pivotIndex = lb;
for (int i = lb+1 ; i<=ub; i++){
if (x[i]<=x[pivotIndex]){
swap(x,pivotIndex,i);
swap(x,i,pivotIndex+1);
pivotIndex++;
}
}
quickSortWorker(x,lb,pivotIndex-1);
quickSortWorker(x,pivotIndex+1,ub);
return x;
}
private void swap(int[] x,int a, int b){
int tmp = x[a];
x[a]=x[b];
x[b]=tmp;
}
I am trying to Implement a solutions to find k-th largest element in a given integer list with duplicates with O(N*log(N)) average time complexity in Big-O notation, where N is the number of elements in the list.
As per my understanding Merge-sort has an average time complexity of O(N*log(N)) however in my below code I am actually using an extra for loop along with mergesort algorithm to delete duplicates which is definitely violating my rule of find k-th largest element with O(N*log(N)). How do I go about it by achieving my task O(N*log(N)) average time complexity in Big-O notation?
public class FindLargest {
public static void nthLargeNumber(int[] arr, String nthElement) {
mergeSort_srt(arr, 0, arr.length - 1);
// remove duplicate elements logic
int b = 0;
for (int i = 1; i < arr.length; i++) {
if (arr[b] != arr[i]) {
b++;
arr[b] = arr[i];
}
}
int bbb = Integer.parseInt(nthElement) - 1;
// printing second highest number among given list
System.out.println("Second highest number is::" + arr[b - bbb]);
}
public static void mergeSort_srt(int array[], int lo, int n) {
int low = lo;
int high = n;
if (low >= high) {
return;
}
int middle = (low + high) / 2;
mergeSort_srt(array, low, middle);
mergeSort_srt(array, middle + 1, high);
int end_low = middle;
int start_high = middle + 1;
while ((lo <= end_low) && (start_high <= high)) {
if (array[low] < array[start_high]) {
low++;
} else {
int Temp = array[start_high];
for (int k = start_high - 1; k >= low; k--) {
array[k + 1] = array[k];
}
array[low] = Temp;
low++;
end_low++;
start_high++;
}
}
}
public static void main(String... str) {
String nthElement = "2";
int[] intArray = { 1, 9, 5, 7, 2, 5 };
FindLargest.nthLargeNumber(intArray, nthElement);
}
}
Your only problem here is that you don't understand how to do the time analysis. If you have one routine which takes O(n) and one which takes O(n*log(n)), running both takes a total of O(n*log(n)). Thus your code runs in O(n*log(n)) like you want.
To do things formally, we would note that the definition of O() is as follows:
f(x) ∈ O(g(x)) if and only if there exists values c > 0 and y such that f(x) < cg(x) whenever x > y.
Your merge sort is in O(n*log(n)) which tells us that its running time is bounded above by c1*n*log(n) when n > y1 for some c1,y1. Your duplication elimination is in O(n) which tells us that its running time is bounded above by c2*n when n > y2 for some c2 and y2. Using this, we can know that the total running time of the two is bounded above by c1*n*log(n)+c2*n when n > max(y1,y2). We know that c1*n*log(n)+c2*n < c1*n*log(n)+c2*n*log(n) because log(n) > 1, and this, of course simplifies to (c1+c2)*n*log(n). Thus, we can know that the running time of the two together is bounded above by (c1+c2)*n*log(n) when n > max(y1,y2) and thus, using c1+c2 as our c and max(y1,y2) as our y, we know that the running time of the two together is in O(n*log(n)).
Informally, you can just know that faster growing functions always dominate, so if one piece of code is O(n) and the second is O(n^2), the combination is O(n^2). If one is O(log(n)) and the second is O(n), the combination is O(n). If one is O(n^20) and the second is O(n^19.99), the combination is O(n^20). If one is O(n^2000) and the second is O(2^n), the combination is O(2^n).
Problem here is your merge routine where you have used another loop which i donot understand why, Hence i would say your algorithm of merge O(n^2) which changes your merge sort time to O(n^2).
Here is a pseudo code for typical O(N) merge routine :-
void merge(int low,int high,int arr[]) {
int buff[high-low+1];
int i = low;
int mid = (low+high)/2;
int j = mid +1;
int k = 0;
while(i<=mid && j<=high) {
if(arr[i]<arr[j]) {
buff[k++] = arr[i];
i++;
}
else {
buff[k++] = arr[j];
j++;
}
}
while(i<=mid) {
buff[k++] = arr[i];
i++;
}
while(j<=high) {
buff[k++] = arr[j];
j++;
}
for(int x=0;x<k;x++) {
arr[low+x] = buff[x];
}
}