Lecture 2_Algorithms PDF

Important Problem Types Lecture 2 Computer Science Department  Sorting A Sorting Algorithm is used to rearrange a given array or list of elements according to a comparison operator on the elements. The comparison operator is used...

Important Problem Types Lecture 2 Computer Science Department  Sorting A Sorting Algorithm is used to rearrange a given array or list of elements according to a comparison operator on the elements. The comparison operator is used to decide the new order of elements in the respective data structure For example, a list of words could be sorted alphabetically or by length. A list of cities could be sorted by population, by area, or by zip code. We have already seen a number of algorithms that were able to benefit from having a sorted list Computer Science Department  Sorting a large number of items can take a substantial amount of computing resources. Like searching, the efficiency of a sorting algorithm is related to the number of items being processed. Two properties of sorting algorithms deserve special mention. 1. A sorting algorithm is called stable if it preserves the relative order of any two equal elements in its input. A sorting algorithm is said to be stable if two objects with equal keys appear in the same order in sorted output as they appear in the input data set 2- The second notable feature of a sorting algorithm is the amount of extra memory the algorithm requires. An algorithm is said to be in-place if it does not require extra memory, except, possibly, for a few memory units. Computer Science Department Computer Science Department  Types Of Sorting 1- Internal Sorting When all data is placed in the main memory or internal memory then sorting is called internal sorting. In internal sorting, the problem cannot take input beyond its size. Example: heap sort, bubble sort, selection sort, quick sort, shell sort, insertion sort. 2- External Sorting When all data that needs to be sorted cannot be placed in memory at a time, the sorting is called external sorting. External Sorting is used for the massive amount of data. Merge Sort and its variations are typically used for external sorting. Some external storage like hard disks and CDs are used for external sorting. Example: Merge sort, Tag sort, Polyphase sort, Four tape sort, External radix sort, Internal merge sort, etc. Computer Science Department  Sorting Sorting is a process in which records are arranged in ascending or descending order 1 2 3 4 5 6 77 42 35 12 101 5 1 2 3 4 5 6 5 12 35 42 77 101 Computer Science Department  Types of sorting Selection sort Insertion sort Bubble sort Merge sort Quick sort Heap sort Shell sort Computer Science Department  Searching The searching problem deals with finding a given value, called a search key, in a given set (or a multiset, which permits several elements to have the same value). There are plenty of searching algorithms to choose from. Unlike with sorting algorithms, there is no stability problem, but different issues arise. Computer Science Department Computer Science Department Computer Science Department  String Processing strings of particular interest are text strings, which comprise letters, numbers, and special characters; bit strings, which comprise zeros and ones; and gene sequences, which can be modeled by strings of characters from the four-character alphabet {A,C, G, T}. One particular problem—that of searching for a given word in a text—has attracted special attention from researchers. String matching algorithms are fundamental tools in computer science and are widely used in various applications such as text processing, data mining, information retrieval, and pattern recognition. These algorithms aim to locate occurrences of a pattern within a larger text or string. Computer Science Department  Defining strings Each string is stored in the computer’s memory as a list of characters. >>> myString = “GATTACA” myString Computer Science Department  Graph Problems One of the oldest and most interesting areas in algorithmic is graph algorithms. Informally, a graph can be thought of as a collection of points called vertices, some of which are connected by line segments called edges. Graphs can be used for modeling a wide variety of applications, including transportation, communication, social and economic networks, project scheduling, and games. Basic graph algorithms include graph-traversal algorithms (how can one reach all the points in a network?), shortest- path algorithms (what is the best route between two cities?), and topological sorting for graphs with directed edges (is a set of courses with their prerequisites consistent or self- contradictory?). Computer Science Department  Task1 Computer Science Department Computer Science Department  Analysis of algorithms Issues: – correctness – time efficiency – space efficiency – optimality Approaches: – theoretical analysis – empirical analysis 16 Computer Science Department  Fundamentals of the Analysis of Algorithm Efficiency There are two kinds of efficiency: time efficiency and space efficiency. Time efficiency, also called time complexity, indicates how fast an algorithm in question runs Space efficiency, also called space complexity, refers to the amount of memory units required by the algorithm in addition to the space needed for its input and output. Speed can be achieve much more spectacular progress than space Focus on time efficiency Computer Science Department  Analysis framework Measuring an Input’s Size All algorithms run longer on larger inputs Ex. take longer to sort larger arrays Unit for measuring running time Time efficiency is analyzed by determining the number of repetitions of the basic operation as a function of input size basic operation is the most important operation of the algorithm Contribute the total running time Compute the number of times that the basic operation is executed The most consuming operation in the algorithm Computer Science Department  Input size and basic operation examples Problem Input size measure Basic operation Searching for key in a Number of list’s items, Key comparison list of n items i.e. n Multiplication of two Matrix dimensions or Multiplication of two matrices total number of elements numbers Checking primality of n’size = number of digits Division a given integer n (in binary representation) Visiting a vertex or Typical graph problem #vertices and/or edges traversing an edge 19 Computer Science Department Computer Science Department Computer Science Department  Best-case, average-case, worst-case For some algorithms efficiency depends on form of input: Worst case: Cworst(n) – maximum over inputs of size n Best case: Cbest(n) – minimum over inputs of size n Average case: Cavg(n) – “average” over inputs of size n – Number of times the basic operation will be executed on typical input – NOT the average of worst and best case – Expected number of basic operations considered as a random variable under some assumption about22 the probability distribution of all possible inputs Computer Science Department Example: Sequential search 23 Computer Science Department Computer Science Department Computer Science Department Computer Science Department  Types of formulas for basic operation’s count Exact formula e.g., C(n) = n(n-1)/2 Formula indicating order of growth with specific multiplicative constant e.g., C(n) ≈ 0.5 n2 Formula indicating order of growth with unknown multiplicative constant e.g., C(n) ≈ cn2 27 Computer Science Department ✓ Both time and space efficiencies are measured as functions of the algorithm’s input size. ✓ Time efficiency is measured by counting the number of times the algorithm’s basic operation is executed. Space efficiency is measured by counting the number of extra memory units consumed by the algorithm. ✓ The efficiencies of some algorithms may differ significantly for inputs of the same size. For such algorithms, we need to distinguish between the worst- case, average-case, and best-case efficiencies. ✓ The framework’s primary interest lies in the order of growth of the algorithm’s running time (extra memory units consumed) as its input size goes to infinity Computer Science Department  Task 2 Computer Science Department