Lecture 6: Quick Sort & Sorting in Linear Time PDF

COM 202: ALGORITHMS Lecture 6 Quick Sort & Sorting in Linear Time Dr. Salwa Osama & Dr. Helal Ahmed Department Of Computer Science 11/26/2018 QUICK SORT 88 52 14 31...

COM 202: ALGORITHMS Lecture 6 Quick Sort & Sorting in Linear Time Dr. Salwa Osama & Dr. Helal Ahmed Department Of Computer Science 11/26/2018 QUICK SORT 88 52 14 31 25 98 30 62 23 79 Divide and Conquer QUICK SORT Partition set into two using randomly chosen pivot 88 52 14 31 25 98 30 62 23 79 14 88 98 30 ≤ 52 ≤ 31 62 25 23 79  QUICK SORT 14 88 98 30 ≤ 52 ≤ 62 31 25 23 79 sort the first half. sort the second half. 14,23,25,30,31 62,79,98,88 QUICK SORT 14,23,25,30,31 52 62,79,88,98 Glue pieces together. 14,23,25,30,31,52,62,79,88,98 QUICKSORT A[l…s-1] ≤ A[s+1…n] Sort an array A[l…n] Divide Partition the array A into 2 subarrays A[l..s-1] and A[s+1..n], such that each element of A[l..s-1] is smaller than or equal to each element in A[s+1..n-1] Need to find index s to partition the array 6 QUICKSORT A[l…s-1] ≤ A[s+1…n-1] Conquer Recursively sort A[l..s-1] and A[s+1..n-1] using Quicksort Combine Trivial: the arrays are sorted in place No additional work is required to combine them The entire array is now sorted 7  QUICKSORT QUICKSORT(A, p, r) PARTITION(A, p, r) 1 if p < r 1 x ← A[r] 2 then q ← PARTITION(A, p, r) 2i←p-1 3 QUICKSORT(A, p, q - 1) 3 for j ← p to r - 1 4 QUICKSORT(A, q + 1, r) 4 do if A[j] ≤ x 5 then i ← i + 1 6 exchange A[i] ↔ A[j] A[p..r] 7 exchange A[i + 1] ↔ A[r] 5 8 return i + 1 A[p..q – 1] A[q+1..r] Partition 5 8 5 5 QUICKSORT i P r A[j]  x I++ 2 5 8 3 6 A[i]  A[j] j++ j i r A[j]  x I++ 2 5 8 3 6 A[i]  A[j] j++ j i r A[j] ≥ x j++ 2 5 8 3 6 j i r A[j] ≤ x I++ 2 5 8 3 6 A[i]  A[j] j++ j QUICKSORT i r J ≠ r-1 2 5 3 8 6 A[i + 1]  A[r] return i + 1 j i+1 r 2 5 3 6 8 j i r 2 5 3 j EXAMPLE p r initially: 2 5 8 3 9 4 1 7 10 6 note: pivot (x) = 6 i j next iteration: 2 5 8 3 9 4 1 7 10 6 i j Partition(A, p, r) x := A[r],i := p – 1; next iteration: 2 5 8 3 9 4 1 7 10 6 for j := p to r – 1 do i j if A[j]  x then i := i + 1; next iteration: 2 5 8 3 9 4 1 7 10 6 A[i]  A[j] i j end if end for; next iteration: 2 5 3 8 9 4 1 7 10 6 A[i + 1]  A[r]; i j return i + 1 EXAMPLE (CONTINUED) next iteration: 2 5 3 8 9 4 1 7 10 6 i j next iteration: 2 5 3 8 9 4 1 7 10 6 i j next iteration: 2 5 3 4 9 8 1 7 10 6 Partition(A, p, r) i j x := A[r], i := p – 1; next iteration: 2 5 3 4 1 8 9 7 10 6 for j := p to r – 1 do i j if A[j]  x then i := i + 1; next iteration: 2 5 3 4 1 8 9 7 10 6 A[i]  A[j] i j end if end for; next iteration: 2 5 3 4 1 8 9 7 10 6 A[i + 1]  A[r]; i j return i + 1 after final swap: 2 5 3 4 1 6 9 7 10 8 i j  ANALYSIS OF QUICKSORT  Best case: split in the middle — Θ(n log n)  Worst case: sorted array and identical elements! — Θ(n2)  Average case: random arrays — Θ(n log n) Quick sort runs as fast as merge sort in best case And runs as slow as insertion in worst case. WORST CASE PARTITIONING One region has one element and the other has n – 1 elements q=1 n n 1 n-1 n T(n) = T(0) + T(n – 1) + n, 1 n-2 n-1 T(0) = (1) n 1 n-3 n-2 1 T(n) = T(n – 1) + n 2 3 1 1 2  n  = n    k  1  (n)  (n )  (n ) 2 2 (n2)  k 1  14 BEST CASE PARTITIONING Best-case partitioning Partitioning produces two regions of size n/2 q=n/2 T(n) = 2T(n/2) + (n) T(n) = (nlgn) (Master theorem) 15 CASE BETWEEN WORST AND BEST 9-to-1 proportional split Q(n) = Q(9n/10) + Q(n/10) + n 16 RANDOMIZED ALGORITHMS No input can elicit worst case behavior Worst case occurs only if we get “unlucky” numbers from the random number generator Worst case becomes less likely Randomization can NOT eliminate the worst-case but it can make it less likely! 17 RANDOMIZED PARTITION Alg. : RANDOMIZED-PARTITION(A, p, r) i ← RANDOM(p, r) exchange A[r] ↔ A[i] return PARTITION(A, p, r) 18  RANDOMIZED QUICKSORT Alg. : RANDOMIZED-QUICKSORT(A, p, r) if p < r then q ← RANDOMIZED-PARTITION(A, p, r) RANDOMIZED-QUICKSORT(A, p, q) RANDOMIZED-QUICKSORT(A, q + 1, r) 19 SORTING IN LINEAR TIME  NON-COMPARISON SORTS: COUNTING SORT Depends on a key assumption: numbers to be sorted are integers in {0, 1, 2, …, k}. Input: A[1..n] , where A[j]  {0, 1, 2, …, k} for j = 1, 2, …, n. Array A and values n and k are given as parameters. Output: B[1..n] sorted. B is assumed to be already allocated and is given as a parameter. Auxiliary Storage: C[0..k] Idea: Determine the number of elements less than x, for each input x. Place x directly in its position. Runs in linear time if k = O(n).  COUNTING-SORT (A, B, K) CountingSort(A, B, k) 1. for i  1 to k O(k 2. do C[i]  0 ) 3. for j  1 to length[A] O(n) 4. do C[A[j]]  C[A[j]] + 1 5. for i  2 to k O(k 6. do C[i]  C[i] + C[i –1] ) 7. for j  length[A] downto 1 8. do B[C[A[ j ]]]  A[j] O(n) 9. C[A[j]]  C[A[j]]–1  COUNTING-SORT EXAMPLE 1 2 3 4 5 1 2 3 4 A 4 1 3 4 3 C: : B :  LOOP 1 1 2 3 4 5 1 2 3 4 A: 4 1 3 4 3 C: 0 0 0 0 1 2 3 4 5 B: for i  1 to k do C[i]  0  LOOP 2 1 2 3 4 5 1 2 3 4 A: 4 1 3 4 3 C: 0 0 0 1 1 2 3 4 5 B: for j  1 to n do C[A[ j]]  C[A[ j]] + ⊳ C[i] = |{key = 1 i}|  LOOP 2 1 2 3 4 5 1 2 3 4 A: 4 1 3 4 3 C: 1 0 0 1 1 2 3 4 5 B: for j  1 to n do C[A[ j]]  C[A[ j]] + ⊳ C[i] = |{key = 1 i}|  LOOP 2 1 2 3 4 5 1 2 3 4 A: 4 1 3 4 3 C: 1 0 1 1 1 2 3 4 5 B: for j  1 to n do C[A[ j]]  C[A[ j]] + ⊳ C[i] = |{key = 1 i}|  LOOP 2 1 2 3 4 5 1 2 3 4 A: 4 1 3 4 3 C: 1 0 1 2 1 2 3 4 5 B: for j  1 to n do C[A[ j]]  C[A[ j]] + ⊳ C[i] = |{key = 1 i}|  LOOP 2 1 2 3 4 5 1 2 3 4 A: 4 1 3 4 3 C: 1 0 2 2 1 2 3 4 5 B: for j  1 to n do C[A[ j]]  C[A[ j]] + ⊳ C[i] = |{key = 1 i}|  LOOP 3 1 2 3 4 5 1 2 3 4 A: 4 1 3 4 3 C: 1 0 2 2 1 2 3 4 5 1 2 3 4 B: C': 1 1 2 2 for i  2 to k do C[i]  C[i] + C[i–1] ⊳ C[i] = |{key  i}|  LOOP 3 1 2 3 4 5 1 2 3 4 A: 4 1 3 4 3 C: 1 0 2 2 1 2 3 4 5 1 2 3 4 B: C': 1 1 3 2 for i  2 to k do C[i]  C[i] + C[i–1] ⊳ C[i] = |{key  i}|  LOOP 3 1 2 3 4 5 1 2 3 4 A: 4 1 3 4 3 C: 1 0 2 2 1 2 3 4 5 1 2 3 4 B: C': 1 1 3 5 for i  2 to k do C[i]  C[i] + C[i–1] ⊳ C[i] = |{key  i}|  LOOP 4 1 2 3 4 5 1 2 3 4 A: 4 1 3 4 3 C: 1 1 3 5 1 2 3 4 5 1 2 3 4 B: 3 C': 1 1 3 5 for j  n downto 1 do B[C’[A[ j]]]  A[ j] C’[A[ j]]  C’[A[ j]] – 1  LOOP 4 1 2 3 4 5 1 2 3 4 A: 4 1 3 4 3 C: 1 1 3 5 1 2 3 4 5 1 2 3 4 B: 3 C': 1 1 2 5 for j  n downto 1 do B[C’[A[ j]]]  A[ j] C’[A[ j]]  C’[A[ j]] – 1  LOOP 4 1 2 3 4 5 1 2 3 4 A: 4 1 3 4 3 C: 1 1 2 5 1 2 3 4 5 1 2 3 4 B: 3 4 C': 1 1 2 5 for j  n downto 1 do B[C’[A[ j]]]  A[ j] C’[A[ j]]  C’[A[ j]] – 1  LOOP 4 1 2 3 4 5 1 2 3 4 A: 4 1 3 4 3 C: 1 1 2 5 1 2 3 4 5 1 2 3 4 B: 3 4 C': 1 1 2 4 for j  n downto 1 do B[C’[A[ j]]]  A[ j] C’[A[ j]]  C’[A[ j]] – 1  LOOP 4 1 2 3 4 5 1 2 3 4 A: 4 1 3 4 3 C: 1 1 2 4 1 2 3 4 5 1 2 3 4 B: 3 3 4 C': 1 1 2 4 for j  n downto 1 do B[C’[A[ j]]]  A[ j] C’[A[ j]]  C’[A[ j]] – 1  LOOP 4 1 2 3 4 5 1 2 3 4 A: 4 1 3 4 3 C: 1 1 2 4 1 2 3 4 5 1 2 3 4 B: 3 3 4 C': 1 1 1 4 for j  n downto 1 do B[C’[A[ j]]]  A[ j] C’[A[ j]]  C’[A[ j]] – 1  LOOP 4 1 2 3 4 5 1 2 3 4 A: 4 1 3 4 3 C: 1 1 1 4 1 2 3 4 5 1 2 3 4 B: 1 3 3 4 C': 1 1 1 4 for j  n downto 1 do B[C’[A[ j]]]  A[ j] C’[A[ j]]  C’[A[ j]] – 1  LOOP 4 1 2 3 4 5 1 2 3 4 A: 4 1 3 4 3 C: 1 1 1 4 1 2 3 4 5 1 2 3 4 B: 1 3 3 4 C': 0 1 1 4 for j  n downto 1 do B[C’[A[ j]]]  A[ j] C’[A[ j]]  C’[A[ j]] – 1  LOOP 4 1 2 3 4 5 1 2 3 4 A: 4 1 3 4 3 C: 0 1 1 4 1 2 3 4 5 1 2 3 4 B: 1 3 3 4 4 C': 0 1 1 4 for j  n downto 1 do B[C’[A[ j]]]  A[ j] C’[A[ j]]  C’[A[ j]] – 1  LOOP 4 1 2 3 4 5 1 2 3 4 A: 4 1 3 4 3 C: 0 1 1 4 1 2 3 4 5 1 2 3 4 B: 1 3 3 4 4 C': 0 1 1 3 for j  n downto 1 do B[C’[A[ j]]]  A[ j] C’[A[ j]]  C’[A[ j]] – 1  ANALYSIS (k) for i  1 to k do C[i]  0 (n) for j  1 to n do C[A[ j]]  C[A[ j]] + (k 1 ) for i  2 to k do C[i]  C[i] + C[i–1] (n) for j  n downto 1 do B[C[A[ j]]]  A[ j] (n + k) C[A[ j]]  C[A[ j]] – 1  ALGORITHM ANALYSIS The overall time is O(n+k). When we have k=O(n), the worst case is O(n). for-loop of lines 1-2 takes time O(k) for-loop of lines 3-4 takes time O(n) for-loop of lines 5-6 takes time O(k) for-loop of lines 7-9 takes time O(n) Stable, but not in place. No comparisons made: it uses actual values of the elements to index into an array.  STABLE SORTING Counting sort is a stable sort: it preserves the input order among equal elements. A 4 1 3 4 3 : B 1 3 3 4 4 : RADIX SORT It was used by the card-sorting machines. Card sorters worked on one column at a time. It is the algorithm for using the machine that extends the technique to multi-column sorting. The human operator was part of the algorithm! Key idea: sort on the “least significant digit” first and on the remaining digits in sequential order. The sorting method used to sort each digit must be “stable”. If we start with the “most significant digit”, we’ll need extra storage. OPERATION OF RADIX SORT 329 720 720 329 457 355 329 355 657 436 436 436 839 457 839 457 436 657 355 657 720 329 457 720 355 839 657 839  RADIX-SORT(A, D) RadixSort(A, d) 1. for i  1 to d 2. do use a stable sort to sort array A on digit i  ALGORITHM ANALYSIS Each pass over n d-digit numbers then takes time (n+k). (Assuming counting sort is used for each pass.) There are d passes, so the total time for radix sort is (d (n+k)). When d is a constant and k = O(n), radix sort runs in linear time. Radix sort, if uses counting sort as the intermediate stable sort, does not sort in place. If primary memory storage is an issue, quicksort or other sorting methods may be preferable.  BUCKET SORT Assumes input is generated by a random process that distributes the elements uniformly over [0, 1). Idea: Divide [0, 1) into n equal-sized buckets. Distribute the n input values into the buckets. Sort each bucket. Then go through the buckets in order, listing elements in each one. EXAMPLE 4 2 1 2 0 3 2 1 4 0 2 3 0 2 0 2 0 1 2 3 4 0 1 2 3 4 0 0 0 1 1 2 2 2 2 3 3 4 4  EXAMPLE.32.26.51.44.21.31.17.52.21.32.52.17.26.31.44.51 [.1,.2) [.2,.3) [.3,.4) [.4,.5) [.5,.6).17.21.26.31.32.44.51.52 AN EXAMPLE  BUCKET-SORT (A) Input: A[1..n], where 0  A[i] < 1 for all i. Auxiliary array: B[0..n – 1] of linked lists, each list initially empty. BucketSort(A) 1. n  length[A] 2. for i  1 to n 3. do insert A[i] into list B[ nA[i] ] 4. for i  0 to n – 1 5. do sort list B[i] with insertion sort 6. concatenate the lists B[i]s together in order 7. return the concatenated lists REFERENCES Introduction to Algorithms, Second Edition by Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest and Clifford Stein, The MIT Press © 2001 Chapter 7.1, 7.2, 7.3 Chapter 8.2, 8.3