Lecture 26 (2) & 27: Heaps, Heapsort, Counting Sort, and Radix Sort - CST370

Lecture 26 (2) – Heaps and Heapsort CST370 – Design & Analysis of Algorithms Dr. Byun Computer Science  Lecture Objectives After completion of this lecture, you will be able to – represent a heap in an array – construct a heap of n numbers....

Lecture 26 (2) – Heaps and Heapsort CST370 – Design & Analysis of Algorithms Dr. Byun Computer Science  Lecture Objectives After completion of this lecture, you will be able to – represent a heap in an array – construct a heap of n numbers. 2  Priority Queue and Heap (Review) Priority queue is an abstract data type with three primary operations. – Finding the max item. – Deleting the max item. – Adding a new item. Heap is a good implementation of priority queue. 70 30 50 10 20 3  Heap Representation in an Array We can represent a heap with n nodes using an array of size “n + 1”. – We do not use the index 0. – We store the heap in the array from the index 1 to n. Root node is assigned to the index 1 of the array. All following nodes are stored from the index 2 to the index n. 4  Heap Representation – Example There are 10 nodes in the sample heap. You store the heap in the array of size 11 (= from index 0 to index 10) – Note that we don’t use the index 0. 5 Heap Representation in an Array 6  Heap Construction So far, we have assumed that the heap is already given. But if we have n random input numbers, how can we construct the heap with the numbers? One simple idea is to add n numbers one by one. – We call it top-down heap construction. Another idea is a button-up approach that offers better time complexity. – We will talk about this approach in the class. 7  Heap Construction Example (1 of 5) Assume that we have 6 numbers 2, 9, 7, 6, 5, 8. First, we should put the numbers in sequence like below. 2 9 7 6 5 8 8  Heap Construction Example (2 of 5) From the result, we know that the nodes 2, 9, and 7 are parental nodes, which mean they have children. – We start from the last parental node (= 7). 2 9 7 6 5 8 9  Heap Construction Example (3 of 5) For the node 7, its child (= 8) is bigger than 7. – Thus, we swap them (= “heapify”). 10  Heap Construction Example (4 of 5) Since the next parental node is 9, we try to “heapify” at the node. – But the number 9 is larger than its children 6 and 5, we don’t do any swapping. 11  Heap Construction Example (5 of 5) We should consider the last parental node 2. – Since 9 is a bigger child, we swap 9 and 2. – After the swapping, since 2 is less than its children, we need one more swapping. – We swap 2 and 6 because 6 is bigger than 5. 12  Exercise Construct a heap with the following numbers. 1, 2, 3, 4, 5, 6, 7, 8 13  Additional Resource If you need more info on the heap construction, watch this video: – http://y2u.be/ixdWTKWSz7s 14 Lecture 27 – Counting Sort and Radix Sort CST370 – Design & Analysis of Algorithms Dr. Byun Computer Science  Lecture Objectives After completion of this lecture, you will be able to – conduct sorting in the linear time using the non-comparison sorting algorithms such as counting sort and radix sort. Reference – Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, and Clifford Stein, Introduction to Algorithms, 3rd Edition, The MIT Press. 2009 2  Comparison Sorts So far, our sorting algorithms are based on the comparison among numbers in an input list. Example – Input list: 35, 10, 50, 77, 10, 15 – To sort the numbers, we have to compare two numbers in the list, for example 35 and 10, to get the correct order of them. Comparison-based sorts require at least (n*logn) time complexity. 3  Non-Comparison Sorts It is possible to sort the numbers in an input list in the linear time (= O(n)) in some special cases. Example – Counting sort – Radix sort – Bucket sort – etc. 4  Introduction to Counting Sort Counting sort can be used for sorting integer values in a range. – For the sorting, we calculate the frequency of each value in an input list. – After that, we determine accumulated sums of frequencies called “distribution”. – Thus, this counting sort is also called “distribution counting sort”. 5  Example (1 of 2) Watch this video – https://youtu.be/5pNvENZpebU (8:10) Input values: 13, 11, 12, 13, 12, 12 – Note that the input values range from 11 to 13. Frequency and distribution Values 11 12 13 Frequency Distribution (= Accumulated sums of frequencies) 6  Example (2 of 2) Input 13 11 12 13 12 12 Values 11 12 13 Distribution 1 4 6 1 2 3 4 5 6 Output 7  Second Example (1 of 2) Input: 2, 5, 3, 0, 2, 3, 0, 3 – Input values range from 0 to 5. Frequency and distribution Values 0 1 2 3 4 5 Frequency 2 0 2 3 0 1 Distribution 2 2 4 7 7 8 8  Second Example (2 of 2) Input 2 5 3 0 2 3 0 3 Values 0 1 2 3 4 5 Distribution 2 2 4 7 7 8 1 2 3 4 5 6 7 8 Output 9  Exercise Input: 1, 4, 1, 2, 7, 5, 2 – Input values range from 1 to 7. Fill out the frequency, distribution, and output values. Values 1 2 3 4 5 6 7 Frequency Distribution 1 2 3 4 5 6 7 Output 10  Counting Sort – Time Efficiency Time complexity of counting sort is O(n+k) where k is the range of n input numbers. – If k is less than n, the algorithm can sort the n input numbers in the linear time. – In many cases, the algorithm is used for a small number k for a large number n. 11  Counting Sort – Example Suppose you want to sort the academic records of all elementary school students by age. – In this case, k is the age range of all students and n is the number of students in the school. – Since k (= age range) is much smaller than n (= number of students), we can do the sorting in the linear time. 12  Stability of Counting Sort An important property of counting sort is that it is stable. – Numbers with the same value appear in the output array in the same order as they do in the input array. 13  Counting Sort Implementation (1 of 2) In the previous examples, we assume that the starting index of the output array is “1”. Example 14  Counting Sort Implementation (2 of 2) But when implementing it, the output array may be indexed from 0, not 1. – In that case, you should reduce a distribution value by 1 first. Then, move the input value to the output array. Example 15  Radix Sort – Basic Idea We sort input numbers based on the least significant digit (= LSD). – “LSD” is the rightmost digit in a number. – For example, 3 in the number 2453 is LSD. Combine the numbers sorted at the above step. Then, we sort the numbers again based on the next LSD. – For example, 5 in number 2453 is the next LSD. After that, we recombine the numbers. We continue this process until the numbers have been sorted on the first digit (= most significant digit). – For example, 2 in number 2453 is the most significant digit. 16  Radix Sort – Example (1 of 4) Watch this video. – https://youtu.be/GURvYJENE0I (4:49) Input: 425, 550, 417, 170, 510, 501, 555 – Note that the input numbers are composed of three digits. – For example, the number 425 has three digits of ‘4’, ‘2’, and ‘5’. – In the number 425, ‘5’ is LSD (= least significant digit). 17  Radix Sort – Example (2 of 4) Input: 425, 550, 417, 170, 510, 501, 555 Sort the numbers based on LSD. 0 1 2 3 4 5 6 7 8 9 550 501 425 417 170 555 510 Numbers after sorting on LSD 550, 170, 510, 501, 425, 555, 417 18  Radix Sort – Example (3 of 4) Input: 550, 170, 510, 501, 425, 555, 417 Sort the numbers based on the next LSD. 0 1 2 3 4 5 6 7 8 9 501 510 425 550 170 417 555 Numbers after sorting 501, 510, 417, 425, 550, 555, 170 19  Radix Sort – Example (4 of 4) Input: 501, 510, 417, 425, 550, 555, 170 Sort the numbers based on the first digit. 0 1 2 3 4 5 6 7 8 9 170 417 501 425 510 550 555 Final sorted result 170, 417, 425, 501, 510, 550, 555 20  Additional Resource [Optional] Watch this video if you want to understand the operation in detail. – http://y2u.be/JMlYkE8hGJM – It’s a little bit longer (= 12 minutes). – But it explains how you can handle the input numbers with different lengths. – Also, it covers the time complexity of radix sort. 21  Exercise Sort the following numbers using the radix sort. 64, 8, 216, 512, 27, 729, 0, 1, 343, 125 – Hint: You need leading 0s for some numbers. – Example: 64  064 8  008 22  Solution (1 of 3) Input numbers with leading 0s 064, 008, 216, 512, 027, 729, 000, 001, 343, 125 0 1 2 3 4 5 6 7 8 9 000 001 512 343 064 125 216 027 008 729 Numbers after sorting on LSD 000, 001, 512, 343, 064, 125, 216, 027, 008, 729 23  Solution (2 of 3) Input: 000, 001, 512, 343, 064, 125, 216, 027, 008, 729 0 1 2 3 4 5 6 7 8 9 000 512 125 343 064 001 216 027 008 729 Numbers after sorting 000, 001, 008, 512, 216, 125, 027, 729, 343, 064 24  Solution (3 of 3) Input 000, 001, 008, 512, 216, 125, 027, 729, 343, 064 0 1 2 3 4 5 6 7 8 9 000 125 216 343 512 729 001 008 027 064 Final sorted result without leading 0s 0, 1, 8, 27, 64, 125, 216, 343, 512, 729 25  Radix Sort Implementation (1 of 3) To sort each digit, you can use the counting sort. Example – This is the counting sort result for LSD. – Note that the index of output array starts from 0. 26  Radix Sort Implementation (2 of 3) This is the counting sort result for the next LSD from the previous slide. 27  Radix Sort Implementation (3 of 3) This is the counting sort result for the first digit (= MSD). 28  Additional Resource Go to the following website to watch the visualized operation of radix sort. https://www.cs.usfca.edu/~galles/visualization/RadixSort.html 29