Algorithms: Analysis and Design Lecture 2 PDF

Algorithms: Analysis and Design Dr. Mohamed Abd Elfattah Department of Computer Science 10/15/2024 Divide & Conquer based Algorithms 10/15/2024 An Example: Merge Sort A. Divide: Divide the n-element sequence to be sorted into two subsequences of n/2 elements each B....

Algorithms: Analysis and Design Dr. Mohamed Abd Elfattah Department of Computer Science 10/15/2024 Divide & Conquer based Algorithms 10/15/2024 An Example: Merge Sort A. Divide: Divide the n-element sequence to be sorted into two subsequences of n/2 elements each B. Conquer: Sort the two subsequences recursively using merge sort. C. Combine: Merge the two sorted subsequences to produce the sorted answer. 10/15/2024 10/15/2024 10/15/2024 10/15/2024 10/15/2024 10/15/2024 10/15/2024 10/15/2024 10/15/2024 10/15/2024 10/15/2024 10/15/2024  The concept of merge sort is applicable in Inversion the following areas: count problem External sorting Merge Sort Applications E-commerce applications 10/15/2024 Analysis of MERGE-SORT Let T(n) be the time taken by this algorithm to sort an array of n elements dividing A into subarrays A1 and A2 takes linear time. It is easy to see that the Merge (A1, A2, A) also takes the linear time. Consequently, T(n) = T([n/2]) + T([n/2]) + θ(n) for simplicity T(n) = 2T (n/2) + θ(n) The total running time of Merge sort algorithm is O(n lg n), which is asymptotically optimal like Heap sort, Merge sort has a guaranteed n lg n running time. Merge sort required (Theta(n)) extra space. Merge is not in-place algorithm. The only known ways to merge in-place (without any extra space) are too complex to be reduced to practical program. 10/15/2024  Life after Algorithm 10/15/2024  How can we design better, Today’s more efficient sorting questions algorithms?  1. Review Today’s 2. Merge Sort topics 3. Quicksort Sorting  Definition sorting Given a list of data points, sort those data points into ascending / descending order by some quantity. Selection sort takeaways Selection sort works by "selecting" the smallest remaining element in the list and putting it in the front of all remaining elements. Selection sort is an O(n2) algorithm. Insertion sort algorithm Repeatedly insert an element into a sorted sequence at the front of the array. To insert an element, swap it backwards until either: ○ (1) it’s bigger than the element before it, or ○ (2) it’s at the front of the array.  The complexity of insertion sort In the worst case (the array is in reverse sorted order), insertion sort takes time O(n2). ○ The analysis for this is similar to selection sort! In the best case (the array is already sorted), insertion takes time O(n) because you only iterate through once to check each element. ○ Selection sort, however, is always O(n2) because you always have to search the remainder of the list to guarantee that you’re finding the minimum at each step. Fun fact: Insertion sorting an array of random values takes, on average , O(n2) time. Let's do better than 2 O(N ) sorting! Advanced Sorting How can we design better, more efficient sorting algorithms? Divide-and-Conquer Motivating Divide-and-Conquer So far, we've seen O(N2) sorting algorithms. How can we start to do better? Motivating Divide-and-Conquer So far, we've seen O(N2) sorting algorithms. How can we start to do better? Assume that it takes t seconds to run insertion sort on the following array: Motivating Divide-and-Conquer So far, we've seen O(N2) sorting algorithms. How can we start to do better? Assume that it takes t seconds to run insertion sort on the following array: Poll: Approximately how many seconds will it take to run insertion sort on each of the following arrays? Motivating Divide-and-Conquer So far, we've seen O(N2) sorting algorithms. How can we start to do better? Assume that it take s t seconds to run insertion sort on t he following array: Answer: Each array should only take about Poll: Approximately how many seconds will it take to ru n insertion sort on each t/4 seconds to sort of the following arrays? Motivating Divide-and-Conquer Main insight: ○ Sorting N elements directly takes total time t ○ Sorting two sets of N/2 elements (total of N elements) takes total time t/2 ○ We got a speedup just by sorting smaller sets of elements at a time! Motivating Divide-and-Conquer Main insight: ○ Sorting N elements directly takes total time t ○ Sorting two sets of N/2 elements (total of N elements) takes total time t/2 ○ We got a speedup just by sorting smaller sets of elements at a time! The main idea behind divide-and-conquer algorithms takes advantage of this. Let's design algorithms that break up a problem into many smaller problems that can be solved in parallel! General Divide-and-Conquer Approach Our general approach when designing a divide-and- conquer algorithm is to decide how to make the problem smaller and how to unify the results of these solved, smaller problems. General Divide-and-Conquer Approach Our general approach when designing a divide-and-conquer algorithm is to decide how to make the problem smaller and how to unify the results of these solved, smaller problems. Both sorting algorithms we explore today will have both of these components: ○ Divide Step Make the problem smaller by splitting up the input list ○ Join Step Unify the newly sorted sublists to build up the overall sorted result General Divide-and-Conquer Approach Our general approach when designing a divide-and-conquer algorithm is to decide how to make the problem smaller and how to unify the results of these solved, smaller problems. Both sorting algorithms we explore today will have both of these components: ○ Divide Step Make the problem smaller by splitting up the input list ○ Join Step Unify the newly sorted sublists to build up the overall sorted result Divide-and-Conquer is a ripe time to return to recursion! Merge Sort Merge Sort A recursive sorting algorithm! Base Case: ○ An empty or single-element list is already sorted. Recursive step: ○ Break the list in half and recursively sort each part. (easy divide) ○ Use merge to combine them back into a single sorted list (hard join) What do we do now? When does the sorting magic happen? The Key Insight: Merge The Key Insight: Merge The Key Insight: Merge The Key Insight: Merge The Key Insight: Merge The Key Insight: Merge The Key Insight: Merge The Key Insight: Merge The Key Insight: Merge The Key Insight: Merge The Key Insight: Merge The Key Insight: Merge The Key Insight: Merge The Key Insight: Merge Each step makes a single comparison and reduces the number of elements by one. If there are n total elements , this algorithm runs in time O(n). The Key Insight: Merge The merge algorithm takes in two sorted lists and combines them into a single sorted list. While both lists are nonempty, compare their first elements. Remove the smaller element and append it to the output. Once one list is empty, add all elements from the other list to the output. Merge Sort A recursive sorting algorithm! Base Case: ○ An empty or single-element list is already sorted. Recursive step: ○ Break the list in half and recursively sort each part. (easy divide) ○ Use merge to combine them back into a single sorted list (hard join) Merge Sort – Let's code it!  Analyzing Mergesort: How fast is this sorting algorithm? void mergeSort(Vector& vec) { if (vec.size()