Programming 2 MEK3100 Lecture Notes PDF

FACULTY OF TECHNOLOGY, ART AND DESIGN Programming 2 MEK3100 Instructor: Hadi Zahmatkesh Email: [email protected] Lecture 1 Classes & Objects  Object Oriented Programming Python is an Object-Oriented Programming (O...

FACULTY OF TECHNOLOGY, ART AND DESIGN Programming 2 MEK3100 Instructor: Hadi Zahmatkesh Email: [email protected] Lecture 1 Classes & Objects  Object Oriented Programming Python is an Object-Oriented Programming (OOP) language. Almost everything in Python is an object, with its properties and methods. A Class is like an object constructor, or a "blueprint" for creating objects.  Object Oriented Programming Python supports many different kinds of data 777 3.2345 "Python" [1, 2, 3, 4, 5] {"John" : 34567865, "Sara" : 23456435 } Each is an object and every object has: a type an internal data representation a set of methods for interaction with the object An object is an instance of a type 777 is an instance of an int "Python" is an instance of a string  Object Oriented Programming Create a class: To create a class, use the keyword class. Create Object: Now we can use the class named MyClass to create objects.  The __init__() Function The example in the previous slide is class and object in its simplest form, and is not really useful in real life applications. To understand the meaning of classes we have to understand the built-in __init__() function. All classes have a function called __init__(), which is always executed when the class is being initiated. Use the __init__() function to assign values to object properties, or other operations that are necessary to do when the object is being created  The __init__() Function The __init__ function is called automatically every time the class is being used to create a new object.  Object Methods Objects can also contain methods. Methods in objects are functions that belong to the object. The self parameter is a reference to the current instance of the class, and is used to access variables that belong to the class. It does not have to be named «self», you can call it whatever you like, but it has to be the first parameter of any function in the class.  Modify Object Properties You can modify properties on objects as follows: Defining Your Own Print Method  Other Special Methods like print, can override other operators (+, -, ==, , len (), and many others) to work with your class. define them with double underscores before/after … and many others __add__() Method  Public vs. Private Members All members in a Python class are public by default. Any member can be accessed from outside the class environment.  Public vs. Private Members Python's convention to make an instance variable protected is to add a prefix _ (single underscore) to it. In fact, this doesn't prevent instance variables from accessing or modifying the instance.  Public vs. Private Members A double underscore __ prefixed to a variable makes it private. It gives a strong suggestion not to touch it from outside the class. Any attempt to do so will result in an AttributeError:  Public vs. Private Members How to access private attributes: You can access private members of a class using public methods of the same class. FACULTY OF TECHNOLOGY, ART AND DESIGN Programming 2 MEK3100 Instructor: Hadi Zahmatkesh Email: [email protected] Lecture 2 Inheritance  Inheritance Inheritance allows you to define a class that inherits all the methods and properties from another class. Parent class is the class being inherited from, also called base class. Child class is the class that inherits from another class, also called derived class.  Inheritance Parent class:  Inheritance Create a Child Class: To create a class that inherits the functionality from another class, send the parent class as a parameter when creating the child class. Use the pass keyword when you do not want to add any other properties or methods to the class.  Inheritance Add the __init__() Function: When you add the __init__() function the child class will no longer inherit the parent’s __init__() function. To keep the inheritance of the parent’s __init__() function, add a call to the parent’s __init__() function.  Inheritance Use the super() Function: Python also has a super() function that will make the child class inherit all the methods and properties from its parent: By using the super() function, you do not have to use the name of the parent element, it will automatically inherit the methods and properties from its parent.  Inheritance Add Properties to Child Class:  Inheritance Add Methods to Child Class: If you add a method in the child class with the same name as a function in the parent class, the inheritance of the parent method will be overridden.  OOP Summary 4 Pillars of OOP ✓ Encapsulation ✓ Abstraction ✓ Inheritance ✓ Polymorphism  Encapsulation Encapsulation is the binding of data and functions that manipulate that data, and we encapsulate into one big object so that we keep everything in a box that users, codes or other machines can interact with.  Abstraction Abstraction means hiding of information or abstracting away information and giving access to only what is necessary. Another example: print ("a", "b", "c", "a").count ("a") → 2  Inheritance Inheritance allows new objects to take on the properties of existing objects.  Polymorphism Polymorphism means many forms. It refers to the way in which object classes can share the same method name, but those method names can act differently based on what object calls them. FACULTY OF TECHNOLOGY, ART AND DESIGN Programming 2 MEK3100 Instructor: Hadi Zahmatkesh Email: [email protected] Lecture 3 Variables & Memory  Variables are Memory References Python Memory Manager Memory 0x1000 Object 1 0x1001 0x1002 0x1003 Object 2 0x1004 Heap Memory Address 0x1005 Object 3 … …  Variables are Memory References my_var_1 = 10 my_var_2 = "hello" Memory reference my_var_1 0x1000 10 0x1000 0x1001 reference my_var_2 0x1002 hello 0x1002 0x1003 0x1004 0x1005 … my_var_1 references the object at 0x1000 my_var_2 references the object at 0x1002 … Variables are Memory References In Python, you can find out the memory address referenced by a variable by using the id() function. This will return a base-10 number. You can convert this base- 10 number to hexadecimal, by using the hex() function. Example a = 10 print(hex(id(a)))  Reference Counting Reference counting is the process of counting how many times certain memory address is referenced. Example a = 10 b=a a 0x1000 10 reference count b 0x1000  Reference Counting Finding the Reference Count passing a to getrefcount() creates an sys.getrefcount(a) extra reference ctypes.c_long.from_address(address).value Here, we just pass the memory address  Dynamic vs Static Typing Some languages (C++, Java) are statically typed. string my_str = "MEK3100" data variable value type name 0x1000 string reference string my_str MEK3100 my_str = 10; Does not work! my_str has been declared as a string, and cannot be assigned the integer value 10 later. my_str = "OsloMet" "OsloMet" is a string – so compatible type This is OK! and assignment works.  Dynamic vs Static Typing Python, in contrast is dynamically typed. my_str = "MEK3100" The variable my_str is purely a reference to a string object with value MEK3100. No type is "attached" to my_str. my_str = 10 The variable my_str is now pointing to an integer object with value 10. 0x1000 string MEK3100 my_str 0x1008 int 10  Dynamic vs Static Typing We can use the built-in type() function to determine the type of the object currently referenced by a variable. When we call type(my_str) - Python looks up the object my_str is referencing (pointing to) and returns the type of the object at that memory location.  Variable Re-Assignment a = 10 a = 15 (or a = a + 5) 0x1000 int 10 a 0x1008 int 15 In fact, the value inside the int object, can never be changed!  Object Mutability Consider an object in memory: changing the data inside the object is called modifying the internal state of the object. memory address has not changed. 0x1000 0x1000 BankAccount BankAccount my_account Acct #: 1234 Acct #: 1234 Balance: 200 Balance: 500 internal state (data) has changed Object was mutated fancy way of saying the internal data has changed.  Object Mutability An object whose internal state can be changed, is called Mutable An object whose internal state cannot be changed, is called Immutable  Object Mutability Examples in Python Immutable Mutable Numbers (int, float, Boolean, etc.) Lists Strings Sets Tuples Dictionaries Frozen Sets User-defined Classes User-defined Classes  Object Mutability my_tuple = (1, 2, 3) Tuples are immutable: elements cannot be deleted, inserted, or replaced. In this case, both the container (tuple), and all its elements (ints) are immutable. But consider this: a = [1, 2] b = [3, 4] t = (a, b) -> t = ([1, 2], [3, 4]) Lists are mutable: elements can be deleted, inserted, or replaced. a.append(3) b.append(5) -> t = ([1, 2, 3], [3, 4, 5]) In this case, although the tuple is immutable, its elements are not. The object references in the tuple did not change but the referenced objects did mutate! t = (1, 2, 3) t is immutable and 1, 2, 3 are references to immutable object (int) t = ([1, 2], [3, 4]) t is immutable but [1, 2] and [3, 4] are references to mutable object (list)  Function Arguments & Mutability In Python, strings (str) are immutable objects. Once a string has been created, the contents of the object can never be changed. In this code: my_var = "hello" The only way to modify the "value" of my_var is to re-assign my_var to another object. 0x1000 my_var hello 0x1008 OsloMet  Function Arguments & Mutability Immutable objects are safe from unintended side-effects. my_str’s reference is passed to process() 0x1000 Scopes module scope Hello 0x1000 my_str Hello process() scope s 0x1008 Hello World!  Function Arguments & Mutability Mutable objects are not safe from unintended side-effects. my_list’s reference is passed to process() 0x1000 Scopes [1, 2, 3, 100] module scope 1 0x1000 my_list 2 3 100 But watch out immutable collection process() scope objects that contain mutable objects lst  Function Arguments & Mutability Immutable collection objects that contain mutable objects: my_tuple’s reference is passed to process() 0x1000 Scopes [1, 2, 3, "a"] module scope [1, 2, 3] 0x1000 my_tuple "a" process() scope t  Shared References & Mutability The term shared reference is the concept of two variables referencing the same object in memory (i.e. having the same memory address) a 0x1000 a = 10 10 b=a b t 0x2000 20 v  Shared References & Mutability In fact, the following may surprise you: 0x1000 a = 10 a 10 b = 10 b s1 0x2000 s1 = "hello" hello s2 = "hello" s2 In both these cases, Python’s memory manager decides to automatically re-use the memory references! Is this even safe? o The integer object 10, and the string object "hello" are immutable , so it is safe to set up a shared reference.  Shared References & Mutability When working with mutable objects we have to be more careful. 0x1000 1 a = [1, 2, 3] a 2 3 b=a b 100 b.append(100) With mutable objects, the Python memory manager will never create shared references. 1 0x1000 a = [1, 2, 3] a 2 3 b = [1, 2, 3] 0xF354 1 b 2 3  Variable Equality We can think of variable equality in two fundamental ways: Memory Address Object State (data) is == identity operator equality operator a is b a == b is not != a is not b a != b not (a is b) not (a == b)  Variable Equality Examples a = 10 a is b b=a a == b a = "hello" a is b b = "hello" a == b a = [1, 2, 3] a is b b = [1, 2, 3] a == b a = 10 a is b b = 10.0 a == b  Variable Equality The None object The None object can be assigned to variables to indicate that they are not set (in the way we would expect them to be), i.e., an "empty" value (or null pointer) But the None object is a real object that is managed by the Python memory manager. Furthermore, the memory manager will always use a shared reference when assigning a variable to None. a = None a 0x1000 b = None b None c = None c So, we can test if a variable is "not set" or "empty" by comparing its memory address to the memory address of None using the is operator. a is None x = 10 x is None x is not None  Everything is an Object So far, you have seen many data types. Integers (int) Booleans (bool) Floats (float) Strings (str) Lists (list) Sets (set) Dictionaries (dict) None (NoneType) You have also seen other constructs: Operators (+, -, ==, is) Functions Classes  Everything is an Object But the one thing in common with all these things, is that they are all objects (instances of classes). Functions (function) def my_func(): Classes (class) [not just instances, but the class itself] … Types (int) 0x1000 my_func function state This means they all have a memory address! id (my_func) 0x1000 As a consequence: ✓ Any object can be assigned to a variable including functions my_func is the name of the function ✓ Any object can be passed to a function including functions my_func () invokes the function ✓ Any object can be returned from a function including functions FACULTY OF TECHNOLOGY, ART AND DESIGN Programming 2 MEK3100 Instructor: Hadi Zahmatkesh Email: [email protected] Lecture 4 Function Parameters  Arguments vs Parameters Semantics! def my_func(a, b): In this context, a and b are called parameters of my_func # code here Also note that a and b are variables, local to my_func When we call the function: x = 10 x and y are called arguments of my_func y = 'a' Also note that x and y are passed by reference my_func(x, y) i.e. the memory address of x and y are passed It is OK if you mix up these terms – everyone will understand what you mean!  Arguments vs Parameters x = 10 def my_func(a, b): y = 'a' # code here my_func(x, y) Module Scope Function Scope 0xA12F x 10 a 0xE341 y 'a' b  Positional and Keyword Arguments Positional Arguments: The most common way of assigning arguments to parameters is via the order (their position) in which they are passed. def my_func(a, b): # code here my_func(10, 20) -> a = 10, b = 20 my_func(20, 10) -> a = 20, b = 10  Positional and Keyword Arguments Default Value: A positional argument can be made optional by specifying a default value for the corresponding parameter. def my_func(a, b=100): my_func(10, 20) -> a = 10, b = 20 # code here my_func(5) -> a = 5, b = 100 Consider a case where we have three arguments, and we want to make one of them optional: def my_func(a, b=100, c): How would we call this function without specifying # code here a value for the second parameter? my_func(5, 25) ??? If a positional parameter is defined with a default value, every positional parameter after it must also be given a default value.  Positional and Keyword Arguments Default Value def my_func(a, b=5, c=10): my_func(1) -> a = 1, b = 5, c = 10 # code here my_func(1, 2) -> a = 1, b = 2, c = 10 my_func(1, 2, 3) -> a = 1, b = 2, c = 3 But what if we want to specify the 1st and 3rd arguments, but omit the 2nd argument? i.e., we want to specify values for a and c, but let b takes on its default value? Keyword Arguments (named arguments) my_func(a=1, c=2) -> a = 1, b = 5, c = 2 my_func(1, c=2) -> a = 1, b = 5, c = 2  Positional and Keyword Arguments Keyword Arguments: Positional arguments can, optionally, be specified by using the parameter name whether or not the parameters have default values. def my_func(a, b, c): my_func(1, 2, 3) # code here my_func(1, 2, c=3) a=1, b=2, c=3 my_func(a=1, b=2, c=3) my_func(c=3, a=1, b=2) But once you use a named argument, all arguments thereafter must be named too. my_func(c=1, 2, 3) my_func(1, b=2, c=3) my_func(1, b=2, 3) my_func(1, c=3, b=2)  Positional and Keyword Arguments Keyword Arguments: All arguments after the first named (keyword) argument, must be named too and default arguments may still be omitted. def my_func(a, b=2, c=3): # code here my_func(1) -> a = 1, b = 2, c = 3 my_func(a=1, b=5) -> a = 1, b = 5, c = 3 my_func(c=0, a=1) -> a = 1, b = 2, c = 0  Unpacking Iterables A side note on Tuples (1, 2, 3) What defines a tuple in Pyhton, is not ( ), but ,. 1, 2, 3 is also a tuple -> (1, 2, 3) The ( ) are used to make the tuple clearer To create a tuple with a single element: (1) will not work as intended -> int 1, or (1, ) -> tuple The only exception is when creating an empty tuple: () or tuple()  Unpacking Iterables Packed Values Packed values refer to values that are bundled together in some way. Tuples and Lists are obvious: t = (1, 2, 3) li = [1, 2, 3] Even a string is considered to be a packed value: s = ''MEK3100'' Sets and dictionaries are also packed values: my_set = {1, 2, 3} d = {'a': 1, 'b': 2, 'c': 3} In fact, any iterable can be considered a packed value.  Unpacking Iterables Unpacking Packed Values Unpacking is the act of splitting packed values into individual variables contained in a list or tuple. a, b, c = [1, 2, 3] 3 elements in [1, 2, 3] -> need 3 variables to unpack this is actually a tuple of 3 variables: a, b and c a -> 1 b -> 2 c -> 3 The unpacking into individual variables is based on the relative positions of each element.  Unpacking Iterables Unpacking other Iterables a, b, c = 10, 20, ''python'' -> a = 10, b = 20, c = ''python'' this is actually a tuple containing 3 values a, b, c = ''XYZ'' -> a = 'X', b = 'Y', c = 'Z' Instead of writing a = 10 and b = 20, we can write a, b = 10, 20 In fact, unpacking works with any iterable type. for e in 10, 20, ''python'' -> loop returns 10, 20, ''python'' for e in 'XYZ' -> loop returns 'X', 'Y', 'Z'  Unpacking Iterables Simple Application of Unpacking Swapping values of two variables: a = 10 a = 20 b = 20 b = 10 ''Traditional'' approach 0xA12F temp = a a 10 a=b temp 0xE341 b = temp b 20 Using unpacking a, b = b, a This works because in Python, the entire RHS is evaluated first and completely, then assignments are made to the LHS  Unpacking Iterables Unpacking Dictionaries d = {''key1'': 1, ''key2'': 2, ''key3'': 3} for e in d -> e iterates through the keys: ''key1'', ''key2'', ''key3'' So, when unpacking d, we are actually unpacking the keys of d a, b, c = d -> a = ''key1'', b = ''key2'', c = ''key3''  Unpacking Iterables Unpacking Sets s = {'p', 'y', 't', 'h', 'o', 'n'} for c in s: -> p print(c) t h n a = 'p' b = 't' o a, b, c, d, e, f = s c = 'h' y … f = 'y' Sets are unordered types. They can be iterated, but there is no guarantee of the order whether the results will match your literal!  Extended Unpacking Using the * and ** Operators The use case for * We do not want to unpack every single item in an iterable. We may, for example, want to unpack the first value, and then unpack the remaining values into another variable. li = [1, 2, 3, 4, 5, 6] We can achieve this using slicing: a = li b = li [1:] Or using simple unpacking: a, b = li , li [1:] We can also use a, *b = li Apart from cleaner syntax, it also works with any iterable, not just sequence types!  Extended Unpacking Using the * and ** Operators Usage with ordered types a, *b = [-10, 7, 3, 777] a = -10 b = [7, 3, 777] This is still a list! a, *b = (-10, 7, 3, 777) a = -10 b = [7, 3, 777] This is still a list! a, *b = ''XYZ'' a = 'X' b = ['Y', 'Z'] The following also works: a, b, *c = 1, 2, 3, 4, 5 a=1 b=2 c = [3, 4, 5] a, b, *c, d = [1, 2, 3, 4, 5] a=1 b=2 c = [3, 4] d=5 a, *b, c, d = ''python'' a = 'p' b = ['y', 't', 'h'] c = 'o' d = 'n'  Extended Unpacking Using the * and ** Operators The * operator can only be used once in the LHS of an unpacking assignment. For obvious reason, you cannot write something like this: a, *b, *c = [1, 2, 3, 4, 5, 6] Since both *b and *c mean "the rest", both cannot exhaust the remaining elements.  Extended Unpacking Using the * and ** Operators Usage with ordered types We have seen how to use the * operator in the LHS of an assignment to unpack the RHS a, *b, c = {1, 2, 3, 4, 5} However, we can also use it this way: l1 = [1, 2, 3] l2 = [4, 5, 6] li = [*l1, *l2] li = [1, 2, 3, 4, 5, 6] l1 = [1, 2, 3] l2 = ''XYZ'' li = [*l1, *l2] li = [1, 2, 3, 'X', 'Y', 'Z']  Extended Unpacking Using the * and ** Operators Usage with unordered types Types such as sets have no ordering. s = {7, -77, 6, 'g'} print (s) -> {-77, 'g', 6, 7} Sets are still iterable, but iterating has no guarantee of preserving the order in which the elements were created/added. But the * operator still works, since it works with any iterable. s = {7, -77, 6, 'g'} a, *b, c = s a = -77 b = ['g', 6] c=7 In practice, we rarely unpack sets directly in this way.  Extended Unpacking Using the * and ** Operators It is useful though in a situation where you might want to create single collection containing all the items of multiple sets, or all the keys of multiple dictionaries. d1 = {'p': 1, 'y': 2} d2 = {'t': 3, 'h': 4} d3 = {'h': 5, 'o': 6, 'n': 7} Note that the key 'h' is in both d2 and d3. li = [*d1, *d2, *d3] -> ['p', 'y', 't', 'h', 'h', 'o', 'n'] s = {*d1, *d2, *d3} -> {'p', 'y', 't', 'h', 'o', 'n'} (Order is not guaranteed for the set)  Extended Unpacking Using the * and ** Operators The ** unpacking operator When working with dictionaries we saw that * essentially iterated the keys. d = {'p': 1, 'y': 2, 't': 3, 'h': 4} a, *b = d a = 'p' b = ['y', 't', 'h'] We might ask the question: can we unpack the key-value pairs of the dictionary? YES We need to use the ** operator  Extended Unpacking Using the * and ** Operators Using ** d1 = {'p': 1, 'y': 2} d2 = {'t': 3, 'h': 4} d3 = {'h': 5, 'o': 6, 'n': 7} Note that the key 'h' is in both d2 and d3. d = {**d1, **d2, **d3} (** operator cannot be used in the LHS of an assignment) -> {'p':1, 'y': 2, 't': 3, 'h': 5, 'o': 6, 'n': 7} Note that the value of 'h' in d3 "overwrote" the first value of 'h' found in d2.  Extended Unpacking Using the * and ** Operators Using ** You can even use it to add key-value pairs from one (or more) dictionary into a dictionary literal: d1 = {'a': 1, 'b': 2} {'a': 10, 'c': 3, **d1} -> {'a': 1, 'c': 3 , 'b': 2} {**d1, 'a': 10, 'c': 3} -> {'a': 10, 'b': 2, 'c': 3}  Extended Unpacking Using the * and ** Operators Nested Unpacking Python will support nested unpacking as well. li = [1, 2, [3, 4]] Here, the third element of the list is itself a list. We can certainly unpack it this way: a, b, c = li a=1 b=2 c = [3, 4] We could then unpack c into d and e as follows: d, e = c d=3 e=4 Or we could simply do it in this way: a, b, (c, d) = [1, 2, [3, 4]] a=1 b=2 c=3 d=4 Since strings are iterables too: a, *b, (c, d, e) = [1, 2, 3, ''XYZ''] a=1 b = [2, 3] c, d, e = ''XYZ'' -> c = 'X' d = 'Y' e = 'Z'  Extended Unpacking Using the * and ** Operators The * operator can only be used once in the LHS an unpacking assignment. How about something like this then? a, *b, (c, *d) = [1, 2, 3, ''python''] Although this looks like we are using * twice in the same expression, the second * is actually in a nested unpacking – so that's OK. a=1 b = [2, 3] c, *d = 'python' -> c = 'p' d = ['y', 't', 'h', 'o', 'n’]  *args Recall from iterable unpacking a, b, c = (10, 20, 30) a = 10 b = 20 c = 30 Something similar happens when positional arguments are passed to a function: def func (a, b, c): # code func(10, 20, 30) a = 10 b = 20 c = 30  *args *args Recall also: a, b, *c = 10, 20, 'a', 'b' a = 10 b = 20 c = ['a', 'b'] Something similar happens when positional arguments are passed to a function: def func (a, b, *c): # code This is a tuple not a list func(10, 20, 'a', 'b') a = 10 b = 20 c = ('a', 'b') The * parameter name is arbitrary – you can make it whatever you want. def func (a, b, *args): It is customary (but not required) to name it *args # code  *args *args exhausts positional arguments. You cannot add more positional arguments after *args. def func (a, b, *args, d): # code func (10, 20, 'a', 'b', 100) This will not work!  *args Unpacking arguments def func (a, b, c): # code li = [10, 20, 30] func (li) This will not work! But we can unpack the list first and then pass it to the function. func (*li) a = 10 b = 20 c = 30  Keyword Arguments Recall that positional parameters can, optionally be passed as named (keyword) arguments. def func (a, b, c): # code func (1, 2, 3) -> a=1 b=2 c=3 func (a=1, c=3, b=2) -> a=1 b=2 c=3 Using named arguments in this case is entirely up to the caller.  Keyword Arguments Mandatory Keyword Arguments We can make keyword arguments mandatory. To do so, we create parameters after the positional parameters have been exhausted. def func (a, b, *args, d): # code In this case, *args effectively exhausts all positional arguments and d must be passed as a keyword (named) argument. func (1, 2, 'x', 'y', d = 100) -> a=1 b=2 args = ('x', 'y') d = 100 func (1, 2, d = 100) -> a=1 b=2 args = ( ) d = 100 func (1 , 2) d was not a keyword argument  Keyword Arguments We can even omit any mandatory positional arguments: def func (*args, d): # code func (1, 2, 3, d = 100) -> args = (1, 2, 3) d = 100 func (d = 100) -> args = ( ) d = 100 In fact, we can force no positional arguments at all: def func (*, d): # code * indicates the end of positional arguments func (1, 2, 3, d = 100) -> Exception func (d = 100) -> d = 100  Keyword Arguments Putting it together def func (a, b=1, *args, d, e=True): def func (a, b=1, *, d, e=True): # code # code a: mandatory positional argument (may be specified using a named argument) b: optional positional argument (may be specified positionally, as a named argument, or not at all), defaults to 1 args: catch-all for any (optional) *: no additional positional arguments additional positional arguments allowed d: mandatory keyword argument e: optional keyword argument, defaults to True  **kwargs Recall that ✓ *args is used to scoop up variable amount of remaining positional arguments (returns a tuple). The parameter name args is arbitrary – * is the real performer here. ✓ **kwargs is used to scoop up a variable amount of remaining keyword arguments (returns a dictionary). The parameter name kwargs is arbitrary – ** is the real performer here. ✓ **kwargs can be specified even if the positional arguments have not been exhausted (unlike keyword-only arguments). No parameters can come after **kwargs.  **kwargs Example def func (*, d, **kwargs): # code func (d=1, a=2, b=3) d=1 kwargs = {'a': 2, 'b': 3} func (d = 1) d=1 kwargs = { }  **kwargs Example def func (**kwargs): # code func (a=1, b=2, c=3) kwargs = {'a': 1, 'b': 2, 'c': 3} func (d = 1) kwargs = {'d': 1} def func (*args, **kwargs): # code func (1, 2, a = 10, b = 20) args = (1, 2) kwargs = {'a': 10, 'b': 20} func () args = ( ) kwargs = { }  Putting it all Together Positional argument Keyword-only argument specific may have default values after positional arguments have been exhausted *args collects, and exhausts remaining specific may have default values positional arguments * indicates the end of positional **kwargs collects any remaining arguments (effectively exhausts) keyword arguments  Putting it all Together scoops up any additional indicates no more scoops up any additional positional args positional args keyword args a, b, c = 10 *args / * kw1, kw2 = 100 **kwargs positional parameters specific keyword-only args can have default values can have default values non-defaulted params are mandatory args non-defaulted params are mandatory args user must specify them using keywords user may specify them using keywords if used, * and *args must also be used  Putting it all Together Examples def func(a, b=10) def func(a, b, *args) def func(a, b, *args, kw1, kw2=100) def func(a, b=10, *, kw1, kw2=100) def func(a, b, *args, kw1, kw2=100, **kwargs) def func(a, b=10, *, kw1, kw2=100, **kwargs) def func(*args) def func(**kwargs) def func(*args, **kwargs)  Putting it all Together Typical Use Case: Python’s print () function *objects arbitrary number of positional arguments after that there are keyword only arguments they all have default values, so they are all optional  Issues with Default Values What happens at run-time … When a module is loaded: all code is executed immediately Module Code a = 10 the integer object 10 is created and a references it def func(a): the function object is created and func references it print(a) func(a) the function is executed  Issues with Default Values What about default values? Module Code def func(a=10): the function object is created and func references it print(a) the integer object 10 is evaluated/created and is assigned as the default for a func(a) the function is executed by the time this happens, the default value for a has already been evaluated and assigned – it is not re-evaluated when the function is called.  Issues with Default Values Consider this: We want to create a function that will write a log entry to the console with a user-specified event date/time. If the user does not supply a date/time, we want to set it to the current date/time. from datetime import datetime def log(msg, *, dt=datetime.utcnow()): print(f"{dt}: {msg}") log("message 1") → 2024-09-13 10:45:49.389474: message 1 a few minutes later: log("message 2") → 2024-09-13 10:45:49.389474: message 2  Issues with Default Values Solution Pattern: We set a default for dt to None Inside the function, we test to see if dt is still None If dt is None, set it to the current date/time Otherwise, see what the caller specified for dt from datetime import datetime def log(msg, *, dt=None): if not dt: dt = datetime.utcnow() print(f"{dt}: {msg}") In general, always beware of using a mutable object (or a callable) for an argument default. FACULTY OF TECHNOLOGY, ART AND DESIGN Programming 2 MEK3100 Instructor: Hadi Zahmatkesh Email: [email protected] Lecture 5 First-Class Functions  Lambda Expression What are Lambda Expressions? We already know how to create functions using the def statement. Lambda expressions are simply another way to create functions (anonymous functions) keyword parameter list (optional) the : is required, even for zero argument this expression is evaluated lambda [parameter list]: expression and returned when the lambda function is called. the expression returns a function object. (think of it as the "body" of that evaluates and returns the expression the function) when it is called It can be assigned to a variable, passed as an argument to another function. It is a function, just like one created with def.  Lambda Expression Examples lambda x: x ** 2 lambda x, y: x + y lambda : "MEK3100" lambda s: s[::-1].upper() type(lambda x: x ** 2) → function Note that these expressions are function objects, but are not "named" (anonymous functions).  Lambda Expression Assigning a Lambda to a Variable Name my_func = lambda x: x**2 my_func(3) →9 my_func(4) → 16 type(my_func) → function identical to: def my_func(x): return x ** 2 type(my_func) → function my_func(3) →9 my_func(4) → 16  Lambda Expression Passing as an Argument to another Function def apply_func(x, fn): return fn(x) apply_func(3, lambda x: x**2) →9 apply_func(2, lambda x: x + 5) →7 apply_func(''abc'', lambda x: x[1:] * 3) → bcbcbc equivalently: def fn_1(x): return x[1:] * 3 apply_func(''abc'', fn_1) → bcbcbc  Lambda Expression Limitations The "body" of a lambda is limited to a single expression. no assignments lambda x: x = 5 lambda x: x = x + 5 no annotations def my_func (x: int) lambda x: int : x ** 2 return x ** 2 Single logical line of code → line-continuation is OK, but still just one expression lambda x: x * \ math.sin(x)  Callables What are callables? Any object that can be called using the ( ) operator. Callables always return a value like functions and methods, but it goes beyond just those two … Many other objects in Python are also callable. To see if an object is callable, we can use the built-in function: callable callable(print) → True callable(''abc''.upper) → True callable(callable) → True callable(str.upper) → True callable(10) → False False  Callables Different Types of Callables built-in functions print len callable built-in methods a_str.upper a_list.append user-defined functions created using def or lambda expressions methods functions bound to an object classes MyClass(x, y, z) → __init__(self, x, y, z) → returns the object (reference) class instances if the class implements __call__ method False  Higher Order Functions A function that takes a function as a parameter and/or returns a function as its return value Example: sorted map modern alternative → list comprehensions filter False  Higher Order Functions The map function map (func, *iterables) *iterables → a variable number of iterable objects func → some function that takes as many arguments as there are iterable objects passed to iterables map (func, *iterables) will then return an iterator that calculates the function applied to each element of the iterables. The iterator stops as soon as one of the iterables has been exhausted so, unequal length iterables can be used.  Higher Order Functions Examples li = [2, 3, 4] def square(x): return x**2 list(map(square, li)) → [4, 9, 16] li1 = [1, 2, 3] li2 = [10, 20, 30] def add(x, y): return x + y list(map(lambda x, y: x + y, li1, li2)) → [11, 22, 33] list(map(add, li1, li2)) → [11, 22, 33]  Higher Order Functions The filter function filter (func, iterables) iterables → a single iterable func → some function that takes a single argument filter (func, iterables) will then return an iterator that contains all the elements of the iterable for which the function called on it is Truthy. If the function is None, it simply returns the elements of iterable that are Truthy.  Higher Order Functions Examples li = [0, 1, 2, 3, 4] list(filter(None, li)) → [1, 2, 3, 4] def is_even(n): return n % 2 == 0 list(filter(is_even, li)) → [0, 2, 4] list(filter(lambda n: n % 2 == 0, li)) → [0, 2, 4]  Higher Order Functions The zip function zip (*iterables) [1, 2, 3, 4] zip (1, 10), (2, 20), (3, 30), (4, 40) [10, 20, 30, 40] [1, 2, 3] zip [10, 20, 30] (1, 10, 'a'), (2, 20, 'b'), (3, 30, 'c') ['a', 'b', 'c'] [1, 2, 3, 4, 5] zip (1, 10), (2, 20), (3, 30) [10, 20, 30]  Higher Order Functions Examples li1 = [1, 2, 3] li2 = [10, 20, 30, 40] li3 = ''python'' list(zip(li1, li2, li3)) → [(1, 10, 'p'), (2, 20, 'y'), (3, 30, 't')] li1 = range(100) li2 = ''abcd'' list(zip(li1, li2)) → [(0, 'a'), (1, 'b'), (2, 'c'), (3, 'd')] (1, 'b'), (2, 'c'), (3, 'd')]  Comprehensions Comprehensions are quick way for us to create lists or sets or dictionaries in Python instead of looping. (1, 'b'), (2, 'c'), (3, 'd')]  Comprehensions List Comprehension Alternative to map li = [2, 3, 4] def square(x): return x**2 list (map (lambda x: x ** 2, li)) → [4, 9, 16] list(map(square, li)) result = [] for x in li: result.append(x**2) result → [4, 9, 16] [x**2 for x in li] → [4, 9, 16] [ for in ]  Comprehensions List Comprehension Alternative to map li1 = [1, 2, 3] li2 = [10, 20, 30] list(map(lambda x, y: x + y, li1, li2)) → [11, 22, 33] Remember: zip(li1, li2) → [(1, 10), (2, 20), (3, 30)] [x + y for x, y in zip (li1, li2)] → [11, 22, 33]  Comprehensions List Comprehension Alternative to filter li = [1, 2, 3, 4] list(filter(lambda n: n % 2 == 0, li)) → [2, 4] [x for x in li if x % 2 == 0] → [2, 4] [ for in if ]  Comprehensions Combining map and filter li = range(10) list(filter(lambda y: y < 25, map(lambda x: x**2, li))) → [0, 1, 4, 9, 16] Using a list comprehension is much clearer: [x**2 for x in range (10) if x**2 < 25] → [0, 1, 4, 9, 16]  Reducing Functions Reducing Functions in Python These are functions that recombine an iterable recursively, ending up with a single return value (also called accumulators, aggregators, or folding functions). Example: Finding the maximum value in an iterable a0, a1, a2, …, an-1 max(a, b) → maximum of a and b result = a0 result = max(result, a1) result = max(result, a2) … result = max(result, an-1) → max value in a0, a1, a2, …, an-1  Reducing Functions Because we have not studied iterables in general, we will stay with the special case of sequences (i.e., we can use indexes to access elements in the sequence). Using a loop li = [5, 8, 6, 10, 9] max_value = lambda a, b: a if a > b else b result = 5 def max_sequence(sequence): result = max(5, 8) = 8 result = sequence result = max(8, 6) = 8 result = max(8, 10) = 10 for e in sequence[1:]: result = max(10, 9) = 10 result = max_value(result, e) return result result → 10  Reducing Functions Notice the sequence of steps: li = [5, 8, 6, 10, 9] 5 max(5, 8) 8 max(8, 6) 8 max(8, 10) 10 max(10, 9) 10 result → 10  Reducing Functions To calculate the min: li = [5, 8, 6, 10, 9] All we really needed to do was to change min_value = lambda a, b: a if a < b else b the function that is repeatedly applied. def min_sequence(sequence): result = sequence def _reduce(fn, sequence): In fact, we could write: result = sequence for e in sequence[1:]: for x in sequence[1:]: result = min_value(result, e) result = fn(result, x) return result return result _reduce(lambda a, b: a if a > b else b, li) → maximum _reduce(lambda a, b: a if a < b else b, li) → minimum  Reducing Functions Adding all the elements in a list: add = lambda a, b: a+b li = [5, 8, 6, 10, 9] result = 5 def _reduce(fn, sequence): result = add(5, 8) = 13 result = sequence result = add(13, 6) = 19 result = add(19, 10) = 29 for x in sequence[1:]: result = add(29, 9) = 38 result = fn(result, x) result → 38 return result _reduce(add, li)  Reducing Functions The functools module Python implements a reduce function that will handle any iterable, but works similarly to what we just saw. from functools import reduce li = [5, 8, 6, 10, 9] reduce(lambda a, b: a if a > b else b, li) → max → 10 reduce(lambda a, b: a if a < b else b, li) → min → 5 reduce(lambda a, b: a + b, li) → sum → 38  Reducing Functions reduce works on any iterable. reduce(lambda a, b: a if a < b else b, {10, 5, 2, 4}) →2 reduce(lambda a, b: a if a < b else b, ''python'') →h reduce(lambda a, b: a + ' ' + b, (''python'', ''is'', ''awesome!'')) → ''python is awesome''  Reducing Functions Built-in Reducing Functions Python provides several common reducing functions: min min([5, 8, 6, 10, 9]) →5 max max([5, 8, 6, 10, 9]) → 10 sum sum([5, 8, 6, 10, 9]) → 38 any any(li) → True if any element in li is truthy False otherwise all all(li) → True if every element in li is truthy False otherwise  Reducing Functions Using reduce to reproduce any li = [0, ' ', None, 100] result = bool(0) or bool(' ') or bool(None) or bool(100) Note: 0 or ' ' or None or 100 → 100 but we want our result to be True/False, so we use bool() Here we just need to repeatedly apply the or operator to the truth values of each element. result = bool(0) → False result = result or bool(' ') → False result = result or bool(None) → False result = result or bool(100) → True reduce(lambda a, b: bool(a) or bool(b), li) → True  Reducing Functions Calculating the product of all elements in an iterable No built-in method to do this, but very similar to how we added all the elements in an iterable or sequence. [1, 3, 5, 6] → 1 * 3 * 5 * 6 reduce(lambda a, b: a * b, li) res = 1 res = res * 3 = 3 res = res * 5 = 3 * 5 = 15 res = res * 6 = 15 * 6 = 90 = 1*3*5*6  Reducing Functions Special case: Calculating n! n! = 1 * 2 * 3 * … * n 5! = 1 * 2 * 3 * 4 * 5 range(1, 6) → 1, 2, 3, 4, 5 range(1, n+1) → 1, 2, 3, …, n To calculate n! we need to find the product of all the elements in range(1, n+1). reduce(lambda a, b: a * b, range(1, n+1)) → n!  Reducing Functions The reduce initializer The reduce function has a third (optional) parameter: initializer (defaults to None) If it is specified, it is essentially like adding it to the front of the iterable. It is often used to provide some kind of default in case the iterable is empty. li = [] li = [] reduce(lambda x, y: x+y, li) reduce(lambda x, y: x+y, li, 1) → exception →1 li = [1, 2, 3] li = [1, 2, 3] reduce(lambda x, y: x+y, li, 1) reduce(lambda x, y: x+y, li, 100) →7 → 106 FACULTY OF TECHNOLOGY, ART AND DESIGN Programming 2 MEK3100 Instructor: Hadi Zahmatkesh Email: [email protected] Lecture 6 Scopes & Closures  Global and Local Scopes Scopes and Namespaces When an object is assigned to a variable (e.g., a = 10) that variable points to some object, and we say that the variable (name) is bound to that object. That object can be accessed using that name in various parts of our code. But not just anywhere! That variable name and it's binding (name and object) only "exist" in specific parts of our code. The portion of code where that name/binding is defined, is called the lexical scope of the variable. These bindings are stored in namespaces (each scope has its own namespace).  Global and Local Scopes The Global Scope The global scope is essentially the module scope. It spans a single file only. There is no concept of a truly global (across all the modules in our entire app) scope in Python. The only exceptions to this are some of the built-in globally available objects, such as: True False None dict print The built-in and global variables can be used anywhere inside our module including inside any function.  Global and Local Scopes Global scopes are nested inside the built-in scope. Built-in Scope namespace namespace Module1 var1 0xA345E scope func1 0xFF34A namespace Module2 scope namespace If you reference a variable name inside a scope and Python does not find it in that scope's namespace, it will look for it in an enclosing scope's namespace  Global and Local Scopes Examples module1.py Python does not find True or print in the current (module/global) scope. print(True) So, it looks for them in the enclosing scope → built-in Finds them there → True module2.py Python does not find a or print in the current (module/global) scope. print(a) So, it looks for them in the enclosing scope → built-in Finds print, but not a → run-time NameError module3.py print = lambda x: f ''hello {x}'' s = print(''world'') Python finds print in the module scope So, it uses it! s → hello world!  Global and Local Scopes The Local Scope When we create functions, we can create variable names inside those functions (using assignments) e.g., a = 10 Variables defined inside a function are not created until the function is called. Every time the function is called, a new scope is created. Variables defined inside the function are assigned to that scope (Function Local scope). The actual object the variable references could be different each time the function is called.  Global and Local Scopes Example my_func a These names will be considered local def my_func(a, b): b to my_func c=a*b c return c my_func a -> 'z' my_func('z', 2) b -> 2 c -> 'zz' same names, different local scopes my_func my_func(10, 5) a -> 10 b -> 5 c -> 50  Global and Local Scopes Nested Scopes Scopes are often nested. Namespace lookups When requesting the object bound to Built-in Scope a variable name: print(a) Module Scope Python will try to find the object bound to the variable Local Scope in current local scope first Local works up the chain of Scope enclosing scopes  Global and Local Scopes Example module1.py built-in scope a = 10 def my_func(b): True global scope print(True) a -> 10 print(a) Local Scope my_func print(b) b -> 30 Local Scope my_func(30) b -> 'a' my_func('a')  Global and Local Scopes Accessing the global scope from a local scope When retrieving the value of a global variable from inside a function, Python automatically searches the local scope's namespace, and up the chain of all enclosing scope namespaces. local -> global -> built-in What about modifying a global variables value from inside the function? a=0 assignment → Python def my_func(): interprets this as a local built-in a = 20 variable (at compile-time) global print(a) → the local variable a Local a -> 0 my_func() → 20 masks the global variable a a -> 20 my_func print(a) →0  Global and Local Scopes The global keyword We can tell Python that a variable is meant to be scoped in the global scope by using the global keyword. a=0 def my_func(): built-in global a global a = 100 a -> 0 Local my_func my_func() print(a) → 100  Global and Local Scopes Example counter = 0 def increment(): global counter counter += 1 increment() increment() increment() print(counter) →3  Global and Local Scopes Global and Local Scoping When Python encounters a function definition at compile-time, it will scan for any labels (variables) that have values assigned to them (anywhere in the function), if the label has not been specified as global, then it will be local. Variables that are referenced but not assigned a value anywhere in the function will not be local, and Python will, at run-time, look for them in enclosing scopes. a = 10 assignment def func1(): a is referenced only in entire function def func3(): at compile time → a global at compile time → a non-local (because we told Python a was print(a) global a global) a = 100 def func2(): assignment def func4(): assignment a = 100 at compile time → a local print(a) at compile time → a local a = 100 when we call func4() → print(a) results in a run-time error because a is local, and we are referencing it before we have assigned a value to it!  Nonlocal Scopes Inner Functions We can define functions from inside another function: global Nested local scopes def outer_func(): # some code local (outer_func) def inner_func(): # some code local (inner_func) inner_func() outer_func() Both functions have access to the global and built-in scopes as well as their respective local scopes. But the inner function also has access to its enclosing scope (the scope of the outer function). That scope is neither local (to inner_func) nor global (it is called a nonlocal scope).  Nonlocal Scopes Referencing variables from the enclosing scope Consider this example: module1.py a = 10 def outer_func(): print(a) When we call outer_func, Python sees the reference to a. outer_func() Since a is not in the local scope, Python looks in the enclosing (global) scope  Nonlocal Scopes Referencing variables from the enclosing scope Now consider this example: module1.py def outer_func(): a = 10 When we call outer_func, inner_func is created and called. def inner_func(): When inner_func is called, Python does not find a in the local print(a) (inner_func) scope. inner_func() So, it looks for it in the enclosing scope, in this case the scope of outer_func. outer_func()  Nonlocal Scopes Referencing variables from the enclosing scope module1.py When we call outer_func, inner_func is defined and called. a = 10 When inner_func is called, Python does not find a in the local def outer_func(): (inner_func) scope. def inner_func(): print(a) So, it looks for it in the enclosing scope, in this case the scope of outer_func. inner_func() Since it does not find it there either, it looks in the enclosing outer_func() (global) scope.  Nonlocal Scopes Modifying global variables We saw how to use the global keyword in order to modify a global variable within a nested scope a = 10 def outer_func2(): def outer_func1(): def inner_func(): global a global a a = 1000 a = ''hello'' inner_func() outer_func1() print(a) → 1000 outer_func2() print(a) → hello We can of course do the same thing from within a nested function  Nonlocal Scopes Modifying nonlocal variables Can we modify variables defined in the outer nonlocal scope? def outer_func(): x = ''hello'' When inner_func is compiled, Python sees an assignment to x. def inner_func(): x = ''python'' So, it determines that x is a local variable to inner_func. inner_func() The variable x in inner_func masks the variable x in print(x) outer_func. outer_func() → hello  Nonlocal Scopes Modifying nonlocal variables Just as with global variables, we have to explicitly tell Python we are modifying a nonlocal variable. We can do that using the nonlocal keyword. def outer_func(): x = ''hello'' def inner_func(): nonlocal x x = ''python'' inner_func() print(x) outer_func() → python  Nonlocal Scopes Nonlocal variables Whenever Python is told that a variable is nonlocal, it will look for it in the enclosing local scopes chain until it first encounters the specified variable name. Beware: It will only look in local scopes, it will not look in the global scope. def outer(): global x = ''hello'' def inner1(): local (outer) inner1 def inner2(): local (inner1) inner2 x nonlocal x x = ''python'' local (inner2) inner2() x inner1() print(x) outer() → python  Nonlocal Scopes Nonlocal variables Now consider this example: def outer(): x = ''hello'' global def inner1(): local (outer) x = ''python'' x local (inner1) def inner2(): x nonlocal x local (inner2) x = ''programming'' print(''inner(before)'', x) → python x inner2() print(''inner(after)'', x) → programming inner1() print(''outer'', x) → hello outer()  Nonlocal Scopes Nonlocal variables def outer(): x = ''hello'' def inner1(): global nonlocal x local (outer) x = ''python'' x local (inner1) def inner2(): x nonlocal x local (inner2) x = ''programming'' print(''inner(before)'', x) → python x inner2() print(''inner(after)'', x) → programming inner1() print(''outer'', x) → programming outer()  Nonlocal Scopes Nonlocal and Global variables x = 100 def outer(): x = ''python'' global def inner1(): nonlocal x local (outer) x x = ''programming'' x local (inner1) def inner2(): x global x x = ''hello'' local (inner2) print(''inner(before)'', x) → programming inner2() x print(''inner(after)'', x) → programming inner1() print(''outer'', x) → programming outer() print(x) → hello  Closures Free Variables and Closures Remember: Functions defined inside another function can access the outer (nonlocal) variables. This x refers to the one in outer's scope. def outer(): x = ''python'' This nonlocal variable x is called a free variable. When we consider inner, we really are looking at: def inner(): the function inner print(f"{x} rocks!") the free variable x (with current value python) inner() outer() → python rocks! This is called closure.  Closures Returning the inner function What happens if, instead of calling (running) inner from inside outer, we return it? def outer(): x is a free variable in inner. x = ''python'' It is bound to the variable x in outer. def inner(): This happens when outer runs (i.e., when inner is created). print(f"{x} rocks!") This is the closure. inner() return inner when we return inner, we are actually "returning" the closure. outer() We can assign that return value to a variable name: fn = outer() fn() → python rocks! When we called fn, at that time Python determined the value of x in the extended scope. But notice that outer had finished running before we called fn (its scope was "gone").  Closures Python Cells and Multi-Scoped Variables What happens if, instead of calling (running) inner from inside outer, we return it? def outer(): Here the value of x is shared between two scopes: x = ''python'' outer def inner(): closure print(x) The label x is in two different scopes but always reference the same "value". return inner Python does this by creating a cell as an intermediary object. outer.x cell 0xA500 str 0xFF100 indirect reference inner.x 0xFF100 python In effect, both variables x (in outer and inner), point to the same cell. When requesting the value of the variable, Python will "double-hop" to get to the final value.  Closures You can think of the closure as a function plus an extended scope that contains the free variables. The free variable's value is the object the cell points to – so that could change over time! Every time the function in the closure is called and the free variable is referenced: Python looks up the cell object, and then whatever the cell is pointing to. def outer(): a = 100 x = ''python'' cell 0xA500 str 0xFF100 def inner(): 0xFF100 python a = 10 # local variable indirect reference print(f"{x} rocks!") return inner fn = outer() fn → inner + extended scope x  Closures Introspection def outer(): a = 100 cell 0xA500 str 0xFF100 x = ''python'' 0xFF100 python def inner(): a = 10 # local variable indirect reference print(f"{x} rocks!") return inner fn = outer() fn.__code__.co_freevars → ('x',) (a is not a free variable) fn.__closure__ → (, )  Closures Introspection def outer(): x = ''python'' print(hex(id(x))) 0xFF100 indirect reference def inner(): print(hex(id(x))) 0xFF100 indirect reference print(f"{x} rocks!") return inner fn = outer() fn()  Closures Modifying free variables def counter(): count is a free variable. count = 0 it is bound to the cell def inc(): count. nonlocal count count += 1 return count return inc fn → inc + count → 0 fn = counter() fn() →1 count's (indirect) reference changed from the object 0 to the object 1. fn() →2  Closures Multiple instances of Closures Every time we run a function, a new scope is created. If that function generates a closure, a new closure is created every time as well. def counter(): f1 = counter() count = 0 f2 = counter() def inc(): nonlocal count f1() →1 f1() →2 count += 1 f1 and f2 do not have the f1() →3 return count same extended scope. return inc f2() →1 f2() →2 They are different instances of the closure. The cells are different.  Closures Shared Extended Scopes def outer(): count = 0 f1, f2 = outer() f1() →1 def inc1(): f2() →2 nonlocal count count += 1 return count count is a free variable – bound to count in the extended scope. def inc2(): nonlocal count count += 1 return count count is a free variable – bound to the same count. return inc1, inc2 returns a tuple containing both closures  Closures Shared Extended Scopes You may think this shared extended scope is highly unusual… but it's not! def adder(n): def inner(x): add_1(10) → 11 return x + n add_1(10) → 12 add_1(10) → 13 return inner add_1 = adder(1) add_2 = adder(2) Three different closures – no shared scopes add_3 = adder(3)  Closures Shared Extended Scopes But suppose we tried doing it this way: n = 1: the free variable in the lambda is n, and it is adders = [] bound to the n we created in the loop for n in range(1, 4): n = 2: the free variable in the lambda is n, and it is adders.append(lambda x: x + n) bound to the (same) n we created in the loop n = 3: the free variable in the lambda is n, and it is bound to the (same) n we created in the loop Now we could call the adders in the following way: adders(10) → 13 Remember, Python does not "evaluate" the free adders(10) → 13 variable n until the adders[i] function is called. adders(10) → 13 Since all three functions in adders are bound to the same n, by the time we call adders, the value of n is 3 (the last iteration of the loop set n to 3)  Closures Nested Closures def incrementer(n): # inner + n is a closure def inner(start): current = start (inner) # inc + current + n is a closure fn = incrementer(2) → fn.__code__.co_freevars → 'n' n=2 def inc(): (inc) nonlocal current inc_2 = fn(100) → inc_2.__code__.co_freevars current += n → 'current', 'n' current = 100, n = 2 return current (calls inc) return inc inc_2() → 102 (current = 102, n=2) return inner inc_2() → 104 (current = 104, n=2) FACULTY OF TECHNOLOGY, ART AND DESIGN Programming 2 MEK3100 Instructor: Hadi Zahmatkesh Email: [email protected] Lecture 7 Decorators  Decorators Decorators: Recall the simple closure example we did which allowed to us to maintain a count of how many times a function was called: def counter(fn): using *args, **kwargs means we can call any function fn count = 0 with any combination of positional and keyword-only def inner(*args, **kwargs): arguments. nonlocal count count += 1 print(f''Function {fn.__name__} was called {count} time(s)'') return fn(*args, **kwargs) return inner We essentially modified our add function by wrapping it inside another function that added some functionality to it. def add(a, b=0): We also say that we decorated our function add with the function return a + b counter. And we call counter a decorator function. add = counter(add) result = add(1, 2) → Function add was called 1 time(s) → result = 3  Decorators In general, a decorator function: ✓ takes a function as an argument ✓ returns a closure ✓ the closure usually accepts any combination of parameters ✓ runs some code in the inner function (closure) ✓ the closure function calls the original function using the arguments passed to the closure ✓ returns whatever is returned by that function call closure outer function (fn) inner function (*args, **kwargs) does something returns fn(*args, **kwargs)  Decorators Decorators and the @ Symbol In our previous example, we saw that counter was a decorator and we could decorate our add function using: add = counter(add) In general, if func is a decorator function, we decorate another function my_func using: my_func = func(my_func) This is so common that Python provides a convenient way of writing that: @counter @func def add(a, b): def my_func(…): return a + b … is the same as writing is the same as writing def add(a, b): def my_func(…): return a + b … add = counter(add) my_func = func(my_func)  Decorators Introspecting Decorated Functions @counter Let's use the same count decorator. def mult(a, b, c=1): def counter(fn): """ returns the product of three values count = 0 """ return a * b * c def inner(*args, **kwargs): nonlocal count remember we could equally have written: mult = counter(mult) count += 1 print(f''Function {fn.__name__} was called {count} time(s)'') return fn(*args, **kwargs) return inner mult.__name__ → inner not mult mult's name "changed" when we decorated it they are not the same function after all help(mult) → Help on function inner in module __main__: inner(*args, **kwargs) We have also "lost" our docstring, and even the original function signature.  Decorators One approach to fixing this We could try to fix this problem, at least for the docstring and function name as follows: def counter(fn): count = 0 def inner(*args, **kwargs): nonlocal count count += 1 print(f''Function {fn.__name__} was called {count} times'') return fn(*args, **kwargs) inner.__name__ = fn.__name__ inner.__doc__ = fn.__doc__ return inner But this doesn’t fix losing the function signature – doing so would be quite complicated. Instead, Python provides us with a special function that we can use to fix this.  Decorators The functools.wraps function The functools module has a wraps function that we can use to fix the metadata of our inner function in our decorator. from functools import wraps In fact, the wraps function is itself a decorator, but it needs to know what was our "original" function (in this case fn). def counter(fn): def counter(fn): count = 0 count = 0 def inner(*args, **kwargs): @wraps(fn) nonlocal count def inner(*args, **kwargs): count += 1 nonlocal count count += 1 print(count) print(count) return fn(*args, **kwargs) return fn(*args, **kwargs) inner = wraps(fn)(inner) return inner return inner  Decorators def counter(fn): count = 0 @counter @wraps(fn) def mult(a, b, c=1): def inner(*args, **kwargs): """ nonlocal count returns the product of three values count += 1 """ print(count) return a * b * c return fn(*args, **kwargs) return inner help(mult) → Help on function mult in module __main__: mult(a, b, c=1) returns the product of three values  Decorators Decorator Parameters We saw some built-in decorators that can handle some arguments: @wraps(fn) @lru_cache(maxsize=256) def inner(): def factorial(n): … … function call This This should look quite different from the decorators we have been creating and using: @time_decorator def fibonacci(n): no function call …  Decorators The time_decorator decorator def time_decorator(fn): from time import perf_counter hard-coded value 10 def inner(*args, **kwargs): total_elapsed = 0 for i in range(10): start = perf_counter() result = fn(*args, **kwargs) total_elapsed += (perf_counter() - start) avg_elapsed = total_elapsed / 10 print(avg_elapsed) return result return inner @time_decorator def my_func(): or my_func = time_decorator (my_func) …  Decorators One Approach extra parameter def time_decorator(fn, reps): from time import perf_counter free variable def inner(*args, **kwargs): total_elapsed = 0 for i in range(reps): start = perf_counter() result = fn(*args, **kwargs) total_elapsed += (perf_counter() - start) avg_elapsed = total_elapsed / reps print(avg_elapsed) return result return inner @time_decorator(10) def my_func(): or my_func = time_decorator (my_func, 10) …  Decorators Rethinking the solution @time_decorator def my_func(): my_func = time_decorator(my_func) … So, time_decorator is a function that returns that inner closure that contains our original function. In order for this to work as intended: @time_decorator(10) time_decorator(10) will need to return our original def my_func(): time_decorator decorator when called. … dec = time_decorator(10) time_decorator(10) returns a decorator @dec def my_func(): and we decorate our function with dec …  Decorators Nested closures to the rescue! def outer (reps): def time_decorator(fn): from time import perf_counter free variable bound to reps in outer def inner(*args, **kwargs): our original decorator total_elapsed = 0 for i in range(reps): start = perf_counter() result = fn(*args, **kwargs) total_elapsed += (perf_counter() - start) avg_elapsed = total_elapsed / reps calling outer(n) returns our original decorator with reps set to n (free print(avg_elapsed) variable) return result return inner @outer(10) return time_decorator def my_func(): or my_func = outer(10)(my_func) …  Decorators Decorator Factories The outer function is not itself a decorator, instead it returns a decorator when called and any arguments passed to outer can be referenced (as free variables) inside our decorator. We call this outer function a decorator factory function. It is a function that creates a new decorator each time it is called.  Decorators And finally … To wrap things up, we probably don't want out decorator call to look like: @outer(10) def my_func(): … It would make more sense to write it this way: @time_decorator(10): def my_func(): … All we need to do is change the names of the outer and time_decorator functions.  Decorators def time_decorator (reps): this was outer def dec(fn): this was time_decorator from time import perf_counter @wraps(fn) we can still use @wraps def inner(*args, **kwargs): total_elapsed = 0 for i in range(reps): start = perf_counter() result = fn(*args, **kwargs) total_elapsed += (perf_counter() - start) avg_elapsed = total_elapsed / reps print(avg_elapsed) return result return inner @time_decorator(10) return dec def my_func(): … FACULTY OF TECHNOLOGY, ART AND DESIGN Programming 2 MEK3100 Instructor: Hadi Zahmatkesh Email: [email protected] Lecture 8 Tuples as Data Structures & Named Tuples  Tuples as Data Structures Tuples vs Lists vs Strings Tuples Lists Strings containers containers containers order matters order matters order matters Heterogeneous / Homogeneous Heterogeneous / Homogeneous Homogeneous indexable indexable indexable iterable iterab