Algorithms and Data Structures Lecture Notes PDF

ALGORITHMS AND DATA STRUCTURES PROF. MASSIMO PONCINO POLITECNICO DI TORINO COURSE OBJECTIVES 1. To learn how the choice of data structures and algorithms impacts program performance 2. Apply general algorithms to practical problems (usually modeled in terms of abstract mathematical st...

ALGORITHMS AND DATA STRUCTURES PROF. MASSIMO PONCINO POLITECNICO DI TORINO COURSE OBJECTIVES 1. To learn how the choice of data structures and algorithms impacts program performance 2. Apply general algorithms to practical problems (usually modeled in terms of abstract mathematical structures) COURSE OVERVIEW Part I (Lectures 1—4) Introduction to algorithms and their complexity Part II (Lectures 5—7) Classical problems: Sorting Part III (Lectures 8—11) Classical problems: search and dictionaries Part IV (Lectures 12—15) Algorithmic Paradigms Part V (Lectures 16—24) Graphs and classical graph problems REFERENCE TEXTBOOK T.H. Cormen, C.E. Leiserson, R.L. Rivest,C. Stein,Introduction to Algorithms MIT Press/McGraw-Hill, 2022 INTRODUCTION AND DEFINITIONS Concept of algorithm What is an algorithm? “…a finite set of operations for accomplishing some task which, given an initial state, will terminate in a corresponding recognizable end-state” (wikipedia) With reference to computer programs, an algorithm is a sequence of instructions that Solves a given problem Operates on data Receives input values Generate output values Terminates after a finite number of steps CONCEPT OF ALGORITHM The term “algorithm” comes from Al-Khwārizmī, a Persian mathematician of the IX century Why are algorithms important? Practical reasons We need efficient algorithms for solving most practical problems Technological reasons Algorithms can be implemented as programs on computers! CONCEPT OF ALGORITHM Example of algorithms Sorting a set of objects… Finding an item in a set of objects… CONCEPT OF ALGORITHM Finding the shortest path between two destinations… Programs = algorithms + data structures Choice of data structure can affect the efficiency of an algorithm e.g., accessing an element ISSUES RELATED TO ALGORITHMS Design of an algorithm How can we “invent” an algorithm? Analysis of an algorithm How efficient is my algorithm? DESIGN OF AN ALGORITHM The structure of the problem to be solved can be used to drive the design of the algorithm Algorithms are typically classified according to a methodological paradigm, i.e., a general algorithmic strategy Algorithmic paradigms General-purpose paradigms Divide and conquer Search and enumeration Paradigms for optimization problems Dynamic programming Greedy paradigm DESIGN OF AN ALGORITHM Divide and conquer The problem is split into several smaller instances of the same problem that are solved recursively Search and enumeration A solution to the problem is obtained by means of a search into a (large) solution space DESIGN OF AN ALGORITHM Dynamic programming The optimal solution to the problem is obtained from the optimal solutions to sub-problems The sub-problems are often overlapping Solutions to sub-problems can be re-used Greedy paradigm The optimal solution to the problem is obtained by concatenating the optimal solutions to sub-problems Local decision DESIGN OF AN ALGORITHM Example of “divide and conquer” paradigm Mergesort algorithm Inputs: sequence S of n elements l,r: indices of the bounds of S Output: sequence S sorted in increasing order ANALYSIS OF AN ALGORITHM It is extremely important to foresee the resources required by an algorithm Execution time Amount of memory Other resources (disk usage, communication ports, software resources, etc.) Parameters affecting the analysis Size of the input Input values Other (execution model, …) We will study in a semi-formal way the complexity of algorithms  ANALYSIS OF AN ALGORITHM Complexity analysis: definitions Complexity = cost in terms of Execution time T(n) Required storage S(n) Function of the size n of the input Ex: sorting algorithm # of elements in the set Complexity analysis: assumptions Requirements Must be independent of the type of computer Asymptotic complexity We are interested in values of n → ∞ ANALYSIS OF AN ALGORITHM Importance of complexity Execution time Assume T(1)=1μs (10-6 s) Scalability with respect to hardware efficiency: number of instances solved in unit time ANALYSIS OF AN ALGORITHM Evaluation of the efficiency of algorithms is essential! Not just algorithms … But efficient algorithms …