Sorting Algorithms Lecture Notes PDF
Document Details
Uploaded by MagicalMars
Nile University
Tags
Summary
These lecture notes cover fundamental sorting algorithms, focusing on Selection Sort, Bubble Sort, and Merge Sort. The notes detail the algorithms' logic, implementation, analysis, and applications. They also explore the concepts of iterative and recursive approaches to sorting.
Full Transcript
Sorting Bringing Order to the World Lecture Outline Iterative sorting algorithms (comparison based) Selection Sort Bubble Sort Recursive sorting algorithms (comparison based) Merge Sort Why Study Sorting? When an input is sorted, many problems become easy (e.g...
Sorting Bringing Order to the World Lecture Outline Iterative sorting algorithms (comparison based) Selection Sort Bubble Sort Recursive sorting algorithms (comparison based) Merge Sort Why Study Sorting? When an input is sorted, many problems become easy (e.g. searching, min, max, k-th smallest) Sorting has a variety of interesting algorithmic solutions that embody many ideas Comparison vs non-comparison based Iterative Recursive Divide-and-conquer Best/worst/average-case bounds Randomized algorithms Applications of Sorting Uniqueness testing Deleting duplicates Prioritizing events Frequency counting Reconstructing the original order Set intersection/union Finding a target pair x, y such that x+y = z Efficient searching Selection Sort Selection Sort: Idea Given an array of n items 1. Find the largest item x, in the range of [0…n−1] 2. Swap x with the (n−1)th item 3. Reduce n by 1 and go to Step 1 Selection Sort: Illustration 37 is the largest, swap it with 29 10 14 37 13 the last element, i.e. 13. Q: How to find the largest? 29 10 14 13 37 x Unsorted items 13 10 14 29 37 Largest item for x current iteration x Sorted items 13 10 14 29 37 10 13 14 29 37 Sorted! We can also find the smallest and put it the front instead http://visualgo.net/sorting?create=29,10,14,37,13&mode=Selection Selection Sort: Implementation void selectionSort(int a[], int n) { for (int i = n-1; i >= 1; i--) { int maxIdx = i; Step 1: for (int j = 0; j < i; j++) Search for maximum if (a[j] >= a[maxIdx]) element maxIdx = j; // swap routine is in STL swap(a[i], a[maxIdx]); } Step 2: } Swap Maximum element with the last item i 8 Selection Sort: Analysis void selectionSort(int a[], int n) { Number of times for (int i = n-1; i >= 1; i--) { executed int maxIdx = i; n−1 for (int j = 0; j < i; j++) n−1 if (a[j] >= a[maxIdx]) (n−1)+(n−2)+…+1 maxIdx = j; = n(n−1)/2 // swap routine is in STL swap(a[i], a[maxIdx]); n−1 } } Total = c1(n−1) + c1 and c2 are cost of statements in c2*n*(n−1)/2 outer and inner blocks = O(n2) Bubble Sort Bubble Sort: Idea Given an array of n items 1. Compare pair of adjacent items 2. Swap if the items are out of order 3. Repeat until the end of array The largest item will be at the last position 4. Reduce n by 1 and go to Step 1 Analogy Large item is like “bubble” that floats to the end of the array Bubble Sort: Illustration At the end of Pass 2, the second largest item 29 is at the second At the end of Pass 1, the largest last position. item 37 is at the last position. x Sorted Item Pair of items x under comparison Bubble Sort: Implementation void bubbleSort(int a[], int n) { for (int i = n-1; i >= 1; i--) { for (int j = 1; j a[j]) adjacent pairs of swap(a[j], a[j-1]); numbers } } Step 2: } Swap if the items are out of order 29 10 14 37 13 http://visualgo.net/sorting?create=29,10,14,37,13&mode=Bubble Bubble Sort: Analysis 1 iteration of the inner loop (test and swap) requires time bounded by a constant c Two nested loops Outer loop: exactly n iterations Inner loop: when i=0, (n−1) iterations when i=1, (n−2) iterations …… when i=(n−1), 0 iterations Total number of iterations = 0+1+…+(n−1) = n(n−1)/2 Total time = c n(n−1)/2 = O(n2) Bubble Sort: Early Termination Bubble Sort is inefficient with a O(n2) time complexity However, it has an interesting property Given the following array, how many times will the inner loop swap a pair of item? 3 6 11 25 39 Idea If we go through the inner loop with no swapping the array is sorted can stop early! Bubble Sort v2.0: Implementation void bubbleSort2(int a[], int n) { for (int i = n-1; i >= 1; i--) { Assume the array is sorted before bool is_sorted = true; the inner loop for (int j = 1; j a[j]) { swap(a[j], a[j-1]); Any swapping will invalidate the is_sorted = false; assumption } } // end of inner loop if (is_sorted) return; If the flag } remains true after the inner } loop sorted! Bubble Sort v2.0: Analysis Worst-case Input is in descending order Running time remains the same: O(n2) Best-case Input is already in ascending order The algorithm returns after a single outer iteration Running time: O(n) Merge Sort Merge Sort: Idea Suppose we only know how to merge two sorted sets of elements into one Merge {1, 5, 9}, with {2, 11} → {1, 2, 5, 9, 11} Question Where do we get the two sorted sets in the first place? Idea (use merge to sort n items) Merge each pair of elements into sets of 2 Merge each pair of sets of 2 into sets of 4 Repeat previous step for sets of 4 … Final step: merge 2 sets of n/2 elements to obtain a fully sorted set Divide-and-Conquer Method A powerful problem solving technique Divide-and-conquer method solves problem in the following steps Divide step Divide the large problem into smaller problems Recursively solve the smaller problems Conquer step Combine the results of the smaller problems to produce the result of the larger problem Divide and Conquer: Merge Sort Merge Sort is a divide-and-conquer sorting algorithm Divide step Divide the array into two (equal) halves Recursively sort the two halves Conquer step Merge the two halves to form a sorted array Merge Sort: Illustration 7 2 6 3 8 4 5 Divide into two halves 77 22 66 33 88 44 55 Recursively sort the 2 3 6 7 4 5 8 halves Merge them 2 3 4 5 6 7 8 Question How should we sort the halves in the 2nd step? Merge Sort: Implementation void mergeSort(int a[], int low, int high) { if (low < high) { Merge sort on int mid = (low+high) / 2; a[low...high] mergeSort(a, low , mid ); Divide a[ ] into two mergeSort(a, mid+1, high); halves and recursively sort them merge(a, low, mid, high); } Conquer: merge the } Function to merge two sorted halves a[low…mid] and a[mid+1…high] into Note a[low…high] mergeSort() is a recursive function low >= high is the base case, i.e. there is 0 or 1 item Merge Sort: Example 38 16 27 39 12 27 mergeSort(a[low…mid]) mergeSort(a[mid+1…high]) merge(a[low..mid], 38 16 27 39 12 27 a[mid+1..high]) 38 16 39 12 27 27 Divide Phase Recursive call to 38 16 mergeSort() 39 12 16 38 12 39 Conquer Phase Merge steps 16 27 38 12 27 39 12 16 27 27 38 39 http://visualgo.net/sorting?create=38,16,27,39,12,27&mode=Merge Merge Sort: Merge a[0..2] a[3..5] b[0..5] 2 4 5 3 7 8 2 4 5 3 7 8 2 2 4 5 3 7 8 2 3 2 4 5 3 7 8 2 3 4 24 5 3 7 8 2 3 4 5 Unmerged x items 2 4 5 3 7 8 2 3 4 5 7 8 x Items used for comparison Two sorted halves to be Merged result in a x Merged items merged temporary array Merge Sort: Merge Implementation PS: C++ STL has merge subroutine too void merge(int a[], int low, int mid, int high) { int n = high-low+1; B is a int* b = new int[n]; Temporary array to store result int left=low, right=mid+1, bIdx=0; while (left