Design and Analysis of Algorithms Lecture Notes PDF

Design and Analysis of Algorithms Lecture 1: An Introduction By/ Dr Hanan Hamed  Course learning objectives 1. Analysis: The ability to analyse algorithmic efficiency via different Design Techniques 2. Design: The ability to create algorithms via different design techn...

Design and Analysis of Algorithms Lecture 1: An Introduction By/ Dr Hanan Hamed  Course learning objectives 1. Analysis: The ability to analyse algorithmic efficiency via different Design Techniques 2. Design: The ability to create algorithms via different design techniques 2  Course Objectives The objective of this course is to develop fundamental skills in designing and analyzing algorithms. This course will consist of two major sections. 1. The first is on the mathematical tools necessary for the analysis of algorithms. This will focus on asymptotics and recurrences. 2. The second element will deal with one particularly important algorithmic problem: sorting a list of numbers. We will show a number of different strategies for sorting, searching, selection, graph algorithms and use this problem as a case-study in different techniques for designing and analyzing algorithms. Recommended Books 1. Introduction to Algorithms By Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, Clifford Stein, Third Edition 2. Introduction To Design And Analysis Of Algorithms by Anany levitin, Pearson Edition. 3. Algorithm Desgin By Jon Kleinberg, Eva Tardos. Short History The word Algorithm comes from the name of the Muslim author Abu Ja’far Mohammad ibn Musa al-Khowarizmi. He was born in the eighth century at Khwarizm (Kheva), a town south of river Oxus (Amu Darya recently) in present Uzbekistan.. Much of al-Khwarizmi’s work was written in a book titled al Kitab al- mukhatasar fi hisab al-jabr wa’l-muqabalah. It is from the titles of these writings and his name that the words algebra and algorithm are derived. As a result of his work, al-Khwarizmi is regarded as the most outstanding mathematician of his time.  What is an algorithm A computer engineer is expected to tackle any given problem in the most efficient manner. To solve a problem, different approaches can be followed. Some of them can be efficient with respect to time consumption, whereas other approaches may be memory efficient. An Algorithm is a sequence of steps to solve a problem. Design and Analysis of Algorithm is very important for designing algorithm to solve different types of problems in the branch of computer science and information technology. An algorithm is a sequence of unambiguous instructions for solving a problem, i.e., for obtaining a required output for any legitimate input in a finite amount of time. Characteristics of Algorithms The main characteristics of algorithms are as follows: 1. Algorithms must have a unique name. 2. Algorithms should have explicitly defined set of inputs and outputs. 3. Algorithms are well-ordered with unambiguous operations. 4. Algorithms halt in a finite amount of time. Algorithms should not run for infinity, i.e., an algorithm must end at some point. Definition of Algorithm:  An algorithm for solving a problem “a finite sequence of unambiguous, executable steps or instructions, which, if followed would ultimately terminate and give the solution of the problem”.  Note the keywords: ◦ Finite set of steps; ◦ Unambiguous; ◦ Executable; ◦ Terminates; Algorithm Design and Analysis of Process 1. Understand the problem 2. Decide on 1. Computational Device 2. Exact Vs Approximate Algorithms 3. Data Structures 4.Algorithm Design Techniques 3. Design an algorithms 4. Prove Correctness 5. Analyze the Algorithm 6. Code the Algorithm The Notion of the Algorithm Describing Algorithms We will follow the Imperative Programming because is most easier in analysis the time complexity and space complexity. The three most common forms of algorithmic notation are: 1. English: is the most natural but least precise programming language. 2. Flow chart: it easy but its insignificant with complex problem and difficult to write and understand. 3. Pseudo code: is generally useful because it is similar to the syntax of structured programming languages. The pseudo code is sometimes intermixed with English whenever necessary. The choice of which notation is best depends upon which method you are most comfortable with.  Pseudo-Code Conventions Pseudo Code is a mixture of Natural Language and Programming Language Constructs such as functions, loops, decision making statements..etc Pseudo-code is a description of an algorithm that is more structured than usual prose but less formal than a programming language. 1. Expressions: use standard mathematical symbols to describe numeric and Boolean expressions 1. use for assignment (“=”). 2. use = for the equality relationship (“==”). Pseudo-Code Conventions 2. Programming Constructs: 1. decision structures: if... then... [else... ] 2. while-loops: while... do 3. repeat-loops: repeat... until... 4. for-loop: for... do 5. array indexing: A[i] Algorithm Examples?  Problem 1: What is the largest integer INPUT: All the integers { … -2, -1, 0, 1, 2, … } OUTPUT: The largest integer Algorithm:  Arrange all the integers in a list in decreasing order;  MAX = first number in the list;  Print out MAX; ◦ WHY is the above NOT an Algorithm?  (Hint: How many integers are there?)  Problem 2: Who is the tallest women in the world? ◦ Algorithm: Tutorial... 3. Expressing Algorithms to Computer  To communicate algorithm to computer ◦ Need way to “represent” the algorithm ◦ Cannot use English  Can use computer language ◦ machine language and ◦ programming languages (Java, Pascal, C) ◦ But, these are too tedious (&technical)  Use Pseudo-Code instead…  Pseudo-Code to express Algorithms  Pseudo-Code ◦ Mixture of computer language and English  Somewhere in between  precise enough to describe what is meant without being too tedious ◦ Examples:  Let c be 0;  Sort the list of numbers in increasing order;  Need to know both ◦ syntax – representation ◦ semantics – meaning Primitive Operations  To describe an algorithm, we need some well-defined programming primitives ◦ Assignment primitive:  assignment statement, input/output statements ◦ Conditional primitive:  if statement  case statement ◦ Looping (iterative) primitive:  for loop,  while loop,  Statements are executed one-by-one Conditional Primitives…  if statement ◦ to take different actions based on condition  Syntax if (condition) then (Step A) true false else (Step B) condition? endif Step A Step B if (condition) then (Step A) endif  Semantics Examples -- (if statement)… Let mark be the total-mark obtained if (mark < 40) then (print “Student fail”) else (print “Student pass”) endif … read in mark (*from the terminal*) if (mark < 40) then (Grade  “F”) else if (mark < 50) then (Grade  “D”) else if (mark < 60) then (Grade  “C”) else if (mark < 70) then (Grade  “B”) else if (mark < 80) then (Grade  “A”); endif print “Student grade is”, Grade …  Looping Primitive – while-loop  the while-loop ◦ loop a “variable” number of times false  Syntax condition? while (condition) do true (some sequence Some sequence of statements) of statements; endwhile  Semantics…  Looping Primitive – for-loop  First, the for-loop ◦ loop a “fixed” or (pre- j  a; determined) number of times false  Syntax (j