python-fundamentals.pdf

e t y c A d a y m e Python B h t a Fundamentals M © MathByte Academy C p o i r y h g © t MathBytev Academy 1 © MathByte Academy d Course Overviewca A e t y B h t a M © t h ir g y p o C y m e 1 2 What this course is about… à introduction to Python from /irst principles e t y c A d a y m e à you will have the knowledge and understanding to learn more on your own à get insights into 3rd B h t a party Python libraries in general M à learn the basics of some common 3rd Party libraries © t à numerical computations i r y h g à manipulating data sets à charting © MathByte Academy C p o 3 Content Overview à installing and running Python e t y à basic principles numerical types (integers, floats, booleans) variables operators (arithmetic, logical, boolean) à control flow © MathByte Academy C B h t a conditional execution iteration (iterables, iterators, loops) exception handling h g à advanced data types p o c A d a y m e i r y © t M sequence types (lists, tuples, strings) dictionaries and sets dates and times decimals 4 Content Overview à functional programming functions higher-order functions closures decorators e t y à object-oriented programming à data acquisition © MathByte Academy C p o i r y h g © t c A d a y m e B h t a custom classes methods properties M CSV JSON Using REST APIs 5 Content Overview à 3rd party Libraries pytz e t y dateutil c A d a y m e B h t a (http) requests numpy M pandas © t matplotlib © MathByte Academy C p o i r y h g 6 Prerequisites à Windows, Mac à needs to run Python 3.6+ c A d a y m e (recommend at least 3.8/3.9 or higher) Windows 10 Mac 10.9 and higher à Linux – you'll need to find/use installation instructions e t y B h t a à some familiarity with Terminal (Mac/Linux) / Command Prompt (Windows) à will be needed to install and run Python à just the basics i r y h g M à how to open/terminate a shell à change directory à list contents of a directory à create a directory © t à no prior Python knowledge needed p o à prior programming knowledge helpful, but not required © MathByte Academy C 7 Course Structure and Materials c A d a y m e à each topic is organized in two parts à lecture video sit back , watch, take notes if you like à coding video lean in and code along – pause video, rewind, type code! e t y B h t a à exercises à each section (except installation section) has a set of exercises à make sure you are con/ident before moving on to the next section © t M à downloadable course materials à all coding videos have an accompanying Jupyter Notebook / resources à contains all the code we do in the code video à contains any required extras (such as data sets) à notebooks are fully annotated © MathByte Academy C p o i r y h g 8 Installing and c A Running Python e t y © MathByte Academy C p o i r y h g © t M B h t a d a y m e 2 9 à what is Python? à language vs implementation e t y c A d a y m e à the canonical, or reference implementation of Python B h t a à installing Python à side by side versions of Python à virtual environments © t à installing 3rd party libraries h g à running Python code à interactive mode p o i r y M à script mode © MathByte Academy C 10 Python © MathByte Academy C p o i r y e t y h g © t M c A d a y m e B h t a 11 A bit of history… c A d a y m e à created by Guido van Rossum in 1989 while working at CWI (National Research Institute for Mathematics and Computer Science, Netherlands) e t y à became a community driven effort, overseen by Guido B h t a à who became the BDFL (benevolent dictator for life) (stepped down in 2018) à many developers ("core" developers) have contributed over the years M à was named after the British comedy group Monty Python © t (who says developers don't have a sense of humour!) h g à still actively developed today © MathByte Academy i r y Python 2.0 released in 2000 à last release was 2.7 (end of life April 2020) Python 3.0 released in 2008 à 3.9 released in October 2020 C p o 12 What is Python? à Python is a language, not an application à there are many implementations of Python e t y CPython PyPy c A d a y m e B h t a à even compilers that "translate" Python code to other languages IronPython Jython Cython i r y M à.NET © t à Java h g à C/C++ à the "reference" Python implementation is CPython © MathByte Academy C p o 13 CPython à reference implementation c A d a y m e à https://www.python.org à most widely used distribution of Python à open source à written in C B h t a e t y à includes the standard library https://github.com/python/cpython à a collection of additional functionality that goes beyond just the Python language © t M à written in C and Python h g à this implementation of Python and the standard library is the "of/icial" implementation p o i r y à many platforms © MathByte Academy C Linux, Windows, Mac OS, iOS, Android, PlayStation, Xbox,… 14 © MathByte Academy c Installing Python A e t y B h t a M © t h ir g y p o C d a y m e 15 Installing CPython c A d a y m e à CPython is basically a bunch of /iles, located in some directory on your computer e t y à one of those /iles is an executable that is used to run Python code /iles or an interactive shell B h t a à entire standard library is also included in these files M à to "install" CPython you simply copy these files into a directory © t à possible to have multiple versions of CPython on the same computer i r y h g à just install the files in different directories à call the desired Python executable © MathByte Academy C p o 16 Where to @ind installation packages à https://www.python.org à click on Downloads © MathByte Academy C p o i r y h g © t e t y à all releases M c A d a y m e B h t a 17 à choose which version you want and what OS you are on e t y M c A d a y m e B h t a à or use default (which should recognize your OS) © MathByte Academy C p o i r y h g © t 18 à two videos included in this course à Windows Installation e t y à Mac Installation c A d a y m e B h t a (Linux installation is same as Mac, just download for Linux OS) M à jump directly to your speci/ic platform video h g © t use Python 3.8.1 or higher for this course © MathByte Academy C p o i r y 19 d a © MathByte Academy c Virtual Environments A e t y B h t a M © t h ir g y p o C y m e 20 3rd Party Libraries à many 3rd party libraries exist c A d a y m e à add-ons to Python for more specialized functionality à a bunch of files à that get added to your Python "installation" e t y B h t a à 3rd party libraries often rely on other 3rd party libraries M à or might have speci/ic releases for speci/ic Python versions © t à this can create conflicts! © MathByte Academy i r y h g my_app_1 à some_lib_1.0 (breaks with some_lib_1.1 and higher) my_app_2 à some_lib_1.1 (breaks with some_lib_1.0 and lower) C p o 21 Solution à since Python is just a directory of files à create two copies of this directory /usr/user1/python3.9-ENV1/ e t y c A d a y m e /usr/user1/python3.9-ENV2/ B h t a install some_lib_1.0 in here install some_lib_1.1 in here à run your apps/shell using the appropriate Python directory M à /Users/user1/python3.9-ENV1/bin/python my_app.py © t à /Users/user1/python3.9-ENV2/bin/python my_other_app.py i r y h g à typing this long path is tedious à add to PATH environment variable (tells OS where to look for executables) © MathByte Academy C p o 22 Solution /usr/user1/python3.9-ENV1/ e t y c A d a y m e à add /usr/user1/python3.9-ENV1/ to (front of) PATH à python my_app.py /usr/user1/python3.9-ENV2/ © t M B h t a à remove /usr/user1/python3.9-ENV1/ from path à add /usr/user1/python3.9-ENV2/ to (front of) PATH à python my_other_app.py i r y h g à works but can become tedious as well! © MathByte Academy C p o 23 Virtual Environments à these are used to perform the exact same steps e t y à make copy of Python installation c A d a y m e à provides scripts to "activate"/"deactivate" the environment B h t a à unsets old path / sets new path à very efficient on Unix/Mac – uses symlinks © t M à little less ef/icient on Windows – actually copies the /iles i r y h g à provides solution to version conflicts à use them!! © MathByte Academy C p o 24 Creating Virtual Environments c A d a y m e à different implementations of this have evolved over the years e t y à Python now has a virtual environment manager built-in à we'll use that one in this course B h t a à first decide which Python version to use à Mac: python3.8, python3.9, etc © t M à Windows: specify full path to python version i r y h g –m venv © MathByte Academy C p o à creates a new virtual environment 25 Activating the Virtual Environment c A d a y m e à activating a virtual environment essentially modifies your PATH e t y à when you type python on command line after activating environment B h t a à you will be running version of Python located in that environment directory M à different for Mac/Linux vs Windows Mac/Linux: i r y Windows: © MathByte Academy C p o © t source /bin/activate h g \Scripts\activate.bat 26 e t y © MathByte Academy C p o i r y h g c A d a y m e B h t Coding a M © t 27 © MathByte Academy c Installing PackagesA e t y B h t a M © t h ir g y p o C d a y m e 28 pip: Package Installer for Python c A d a y m e à installing 3rd party libraries (packages) basically requires copying /iles into the Python installation (whichever directory you want) e t y à can be done manually, but again, tedious B h t a à instead can use another app that is installed alongside Python à pip M à easily install, update and remove packages © t à uses the Python Package Index i r y h g https://pypi.org à of/icial 3rd party repository for Python © MathByte Academy C p o à a repository of over 200,000 Python packages! 29 Installing Packages with pip c A d a y m e à activate the virtual environment /irst (sets your PATH) à pip install package_name à can even specify versions e t y B h t a pip install package_name==1.3.2 pip install package_name2.0 h g © t à once you have decided on a speci/ic version of a package for your project you should always use same version number if creating a new environment for the same project p o i r y à otherwise you could end up breaking your own code! © MathByte Academy C 30 The requirements.txt File c A d a y m e à use a file, alongside your code, to keep track of required packages and versions e t y requirements.txt à /ile name can be anything à requirements.txt is a standard convention numpy==1.18.1 pandas==1.1.4 matplotlib==3.3.3 M B h t a > pip install –r requirements.txt © t à easier (install everything with one command) i r y h g à reproducibility / consistency à documentation © MathByte Academy C p o 31 d a y m e à for this course a requirements.txt /ile is available in your downloads c A à de/ines (and pins) all speci/ic libraries we'll need e t y à please use it so we're all using the same library versions B h t a (functionality can change from version to version) M à will install many libraries (and their dependencies) requests pytz i r y numpy pandas © MathByte Academy C p o h g © t and more… 32 Summary of Steps à create a new virtual environment e t y à activate the environment B h t a à pip install libraries (aka packages) M c A d a y m e à don't forget to activate the environment before you pip install! © t à otherwise pip install will install these packages to your reference Python install © MathByte Academy C p o i r y h g 33 e t y © MathByte Academy C p o i r y h g c A d a y m e B h t Coding a M © t 34 Running Python © MathByte Academy C p o i r y h g © t M e t y c A d a y m e B h t a 35 Python Compiler/Interpreter Python code à Python is both a compiler and an interpreter © MathByte Academy o C y p ri h g © t c A Intermediate Code (bytecode) Python compiler e t y M d a y m e B h t a Virtual Machine (OS/Platform speciCic) Operating System 36 How do we "run" Python? à Python is a compiler/interpreter c A à reads in a chunk of code (your program) à compiles and runs it à output is sent to your screen (console) e t y d a y m e B h t a à Python can do this in two ways à interactive mode à you type a Python line/block of code and execute it immediately à any output is immediately displayed à continue typing/running code one line/block at a time à REPL (read-eval-print-loop) i r y h g © t M à script mode à write all your code in /iles /irst à then execute all this code using command line © MathByte Academy C p o 37 Interactive Mode à activate virtual environment c A d a y m e à start Python shell (REPL) by typing python on command line à start typing Python commands à non graphical interface e t y B h t a à perfect when working on GUI-less servers M à little tedious to use when you are just trying things out à Jupyter Notebooks h g © t à browser based REPL (needs to be pip installed) i r y à much easier/nicer to use than command line p o à can save your projects into a /ile (a notebook) à usually.ipynb extension © MathByte Academy C 38 Script Mode à write all your code using a text editor e t y à run your Python program using command line > python my_app.py c A d a y m e B h t a à traditional programs are great for M à running the same program over and over again à better structure à complex applications (web server, prediction server, libraries, …) © MathByte Academy C p o i r y h g © t 39 Python IDEs IDE à integrated development environment à a text editor e t y c A d a y m e à with many extras for easily running code, debugging, and more à runs code using the same command line approach for scripts M B h t a à many popular IDEs / editors around à PyCharm à VSCode à Atom i r y h g © t this is the one I personally use for development à Sublime Text 3 p o à Spyder © MathByte Academy C and more… 40 e t y © MathByte Academy C p o i r y h g c A d a y m e B h t Coding a M © t 41 Python Basics e t y © MathByte Academy C p o i r y h g © t M B h t a c A d a y m e 3 42 à some basic Python types à integers à floats à booleans à a brief explanation of what objects are à basic operators à arithmetic operators M e t y c A d a y m e B h t a à integer division and modulus © t à comparison operators i r y h g à Boolean operators à operator precedence © MathByte Academy C p o 43 © MathByte Academy c Basic Data Types A e t y B h t a M © t h ir g y p o C d a y m e 44 Types c A à Entities in a program always have an associated type à in the real world too! John is a person e t y d a y m e B h t a My local pharmacy is a store My bank balance is a (real) number M The number of pages in a book is an (integer) number © t The file budget.xlsx is an Excel spreadsheet h g Statements can be True or False i r y they have a type à Boolean type (True, False) © MathByte Academy C p o "I am a Python dev" is True "My dog likes cats" is False 45 Integers à the int type c A d a y m e à used to represent integral numbers: 0, 1, 100, -100, etc. e t y à integers have an exact representation in Python B h t a à integers can be of any magnitude (as long as you have enough memory!) M à integer numbers can be created from a literal in the Python code à 100 à -100 i r y h g © t note how you can use underscores for readability à 10_500_000 à or, as the result of some calculation (expression) © MathByte Academy C p o à1 + 1 46 Floats à the float type e t y c A d a y m e à used to represent real numbers (floating point): 3.14, -1.3 B h t a à can use Python literals to define a float à 3.14 à -1.3 à 1_234.567_876 © t M à the decimal point differentiates a float from an int when using literals 1 à int i r y 1.0 à float © MathByte Academy C p o h g 47 Float Representations Consider the decimal system: 1.234 e t y c A d a y m e In the decimal (base 10) system, this is representable (exactly) using powers of 10: 2 3 4 1+ + + 10 100 1000 B h t a But not all real numbers have a finite representation © t as a fraction this is exact h g M 1 3 à but not using a decimal representation 1 3 3 3 ̇ = 0.333 = + + +⋯ 3 10 100 1000 p o i r y à infinite number of fractions © MathByte Academy C 48 Integer Representations à computers only "know" two numbers à 0 and 1 e t y à binary system, aka base 2 c A d a y m e B h t a à any number in a computer is represented using powers of 2 the binary number 1011 M can be converted to a decimal number: © t 1×2! + 1×2" + 0×2# + 1×2$ h g =1+2+0+8 = 11 © MathByte Academy C p o i r y 49 Float Binary Representation c A d a y m e in the same way, floats are represented using powers of 2 and fractions of powers of 2 1 0 1 1 0 1 + + = + + 2" 2# 2$ 2 4 8 e t y B h t a = 0.5 + 0 + 0.125 = 0.625 ! à we saw that certain numbers do not have a finite decimal representation ( ) " M à same happens with binary representations! 0 0.1 = 2 © MathByte Academy C + p o © t 0 0 1 1 + + + 4 8 16 32 i r y h g + 0 0 1 1 + + + + ⋯ 64 128 256 512 0.09375 0.099609375 50 Floats are not always exact c A d a y m e à bottom line: not all exact decimal numbers have an exact float representation e t y à not a limitation of Python à all languages (incl. Excel) that use these binary fractions have that issue B h t a ⚠ be careful when comparing /loats to one another M à there is a data type that can handle exact representations of decimal fractions à Decimal i r y h g © t (we will look at this type later) ⚠ calculations using Decimal are much slower than float © MathByte Academy C p o 51 e t y © MathByte Academy C p o i r y h g c A d a y m e B h t Coding a M © t 52 Objects © MathByte Academy C p o i r y e t y h g © t M c A d a y m e B h t a 53 What are objects? à entities created by Python e t y à they have state (data) B h t a à they have methods (functionality) c A d a y m e encapsulation à they often represent real world things Car © MathByte Academy C © t State (data) h g M brand à Toyota model à Prius LE # doors à 4 model_year à 2020 odometer à 5_402 p o i r y Functionality accelerate() brake() set_cruise_control() left_turn_signal_on() 54 Integers are Objects à an int is an object e t y à it has state – the value of the integer à but it also has functionality c A d a y m e B h t a à knows how to add itself to another integer M (10).__add__(100) à 110 © t à an integer object has the method __add__ used to implement addition h g (this is not how we typically add two integers together – but that's just syntax) i r y à knows how to represent itself as a string (e.g. for visual output) © MathByte Academy C p o 55 Floats are Objects à float numbers are objects too state à value functionality à __add__ e t y c A d a y m e B h t a other functionality too, for example: © t M (0.125).as_integer_ratio() © MathByte Academy C p o i r y h g à 1, 8 56 Everything in Python is an object c A d a y m e à any data type we have in Python is actually an object à it has state à it has functionality e t y attributes B h t a attributes encompass state and functionality © t M some attributes are for state © MathByte Academy C p o i r y h g some attributes are for functionality 57 Dot Notation c A d a y m e If an object has attributes, how do we access those attributes? e t y à dot notation B h t a car.brand à accesses the brand attribute of the car object car.model à accesses the model attribute of the car object M For attributes that represent functionality, we usually have to call the attribute to perform the action © t à often supplying additional parameters i r y h g car.accelerate(10, "mph") (10).__add__(100) © MathByte Academy C p o 58 Mutability and Immutability c A d a y m e à an object is mutable if its internal state can be changed à one or more data attributes can be changed e t y à an object is immutable if its internal state cannot be changed B h t a à the state of the object is "set in stone" In Python many data types are immutable: i r y h g © t While some are mutable: © MathByte Academy C p o M lists dictionaries sets … integers /loats booleans strings … à we'll cover all these in this course 59 e t y c A d a y m e B h t a Coding M © MathByte Academy C p o i r y h g © t 60 Variables © MathByte Academy C p o i r y h g e t y © t M c A d a y m e B h t a 61 Naming Objects We often need to label objects with some name e t y c A d a y m e à reminds us what the object is used for apy B h t a account_balance © t M à allows us to use the same object in multiple parts of our code © MathByte Academy C p o i r y h g 62 Assigning Names c A d a y m e To assign a label to an object we use the assignment operator e t y account_balance = 1000.0 apy = 0.25 = B h t a à we are assigning the label apy to the object 0.25 M this is not the mathematical equality symbol à we say apy is a reference to the float object 0.25 h g © t à the symbol apy is just a label currently pointing to (or referencing) the object 0.25 © MathByte Academy C p o i r y 63 References and Variables Another way of looking at this: e t y float a label i r y h g © t apr some object in memory B h t a 0.25 references the object apr c A d a y m e M = 0.25 à apr is called a variable p o à but it is just a label (a symbol) that references some object in memory © MathByte Academy C 64 Variables c A d a y m e So why the term variable? à over time, which object a symbol references can change a = 100 later in the program… a = True e t y à a is referencing the object 100 B h t a à a is now referencing the object True M à the state of the object the symbol references can change (mutate) h g a i r y © t à a is still referencing the same object, but the object's state has changed (mutated) © MathByte Academy C p o list 1, 2, 3 , 4 we append an element to the list 65 How Variable Assignment Happens apy = 0.25 LHS e t y RHS à Python evaluates the RHS first c A d a y m e B h t a à then it "assigns" that result to the symbol in the LHS M (the LHS becomes a named reference to whatever results from the RHS) © t Generally, RHS could be a more complex expression than just a literal ri h g balance = 1000.0 – 50.0 y p circ = 2 * 3.14 * 1.5 © MathByte Academy o C à in both cases, the RHS is fully evaluated /irst 66 Using Variables c A d a y m e Once a variable has been created, it can be used elsewhere in the program e t y pi = 3.1415 pi radius = 1 B h t a circ = 2 * pi * radius © t M radius circ 3.1415 1 6.283 àcirc is now a reference to the float 6.283 h g radius = 2 i r y radius 2 BUT this does not change circ p o à it still points to 6.283 © MathByte Academy C circ 6.283 67 Variable Naming à case sensitive c A d a y m e apr is a different symbol than APR à must follow certain rules e t y B h t a à should follow certain conventions © MathByte Academy C p o i r y h g © t M 68 Must-Follow Rules start with underscore (_) or letter (a-z A-Z) e t y c A d a y m e (unicode characters are actually OK, but stick to a-z A-Z) B h t a followed by any number of underscores or letters, or digits (0-9) var my_var _var __var h g index1 © t M index_1 __add__ all legal à cannot be reserved words i r y True False if def and or p o and many more we'll come across in this course © MathByte Academy C 69 Should-Follow Conventions c A d a y m e PEP 8 Style Guide à typical conventions followed by most Python devs e t y https://www.python.org/dev/peps/pep-0008/ terminology: camel case B h t a àseparate words are distinguished by upper case letters M accountBalance snake case © MathByte Academy C p o i r y © t BankAccount à separate words are distinguished by underscores h g account_balance bank_account 70 Should-Follow Conventions For standard variables: e t y à snake case à all lower case letters B h t a account_balance ✅ account_Balance ❌ h g © t M c A d a y m e We'll see other conventions for other special types of objects throughout this course © MathByte Academy C p o i r y 71 Should-Follow Conventions à Good idea to follow standard conventions e t y c A d a y m e à but sometimes you may want to break those conventions à that's OK – just have a good reason, and be consistent From the PEP 8 Style Guide: © t M B h t a A foolish consistency is the hobgoblin of little minds. (Emerson) © MathByte Academy C p o i r y h g 72 e t y © MathByte Academy C p o i r y h g c A d a y m e B h t Coding a M © t 73 d a © MathByte Academy c Arithmetic Operators A e t y B h t a M © t h ir g y p o C y m e 74 Terminology c A d a y m e An operator is a programming language symbol that performs some operation on one or more values e t y Certain types of operators include: à arithmetic operators B h t a à comparison (or relational) operators à logical operators © t M The values an operator acts on are called operands h g à An operator that works on a single operand is called a unary operator i r y à An operator that works on two operands is called a binary operator p o à An operator that works on three operands is called a ternary operator © MathByte Academy C 75 Arithmetic Operators Unary Operators - Unary Minus -10 + Unary Plus +10 Binary Operators + Addition - Subtraction * Multiplication / Division i r y h g © t M ** Power (exponentiation) p o e t y c A d a y m e B h t a 10 + 20 20 - 10 10 * 2 10 / 2 2 ** 4 à use parentheses ( and ) to group expressions © MathByte Academy C 76 Operand Types c A Arithmetic operators can act on any numerical type int float e t y d a y m e à as well as other types we'll encounter later B h t a à what the operator does is actually determined by the type of the operands M à an operator may support mixed operand types à returns an int 2 + 2.0 à returns a float 5.5 * 2 p o 4 / 2 © MathByte Academy © t 2 + 2 C i r y h g à returns a float à also returns a float! 77 The Power Operator c A d a y m e The power operator works just like its mathematical counterpart e t y 2 ** 4 à 2 * 2 * 2 * 2 à 16 (int) Recall from math: 2 ** (-4) 2%& = h g à 1 / (2 ** 4) i r y à 1 / 16 p o 1 2& © t M B h t a à 0.0625 (float) © MathByte Academy C 78 The Power Operator c A d a y m e à Python supports /loats for either operand of the ** operator e t y à just like mathematical exponentiation B h t a à graph of / 0 = 1 ' i r y h g © t M à 2 ' : = 1 ' ()*(,) à Python also supports negative bases with real exponents à complex numbers p o à it's actually a numerical type in Python (complex) © MathByte Academy C 79 y m How Python Implements Arithmetic Operators e d a c A e t y B h t a M © t h ir g y p o C à recall: numbers are actually objects à they have state à they also have functionality à one of these is the __add__ method (amongst many others) when we do this: a + b where a = 10 and b = 20 à 10 is an int object that implements the __add__ method Python actually does this to evaluate the expression: à a.__add__(b) à this works the same way with other types © MathByte Academy 80 Looking ahead… c A d a y m e à any type can choose to implement __add__ however it wants à Python will then use that method to evaluate type_1 + type_2 e t y B h t a à we will see later how to create our own types à we can implement __add__ to define + for our custom type M à we'll look at this in code, though some of the code may not make sense (yet!) © MathByte Academy C p o i r y h g © t 81 e t y © MathByte Academy C p o i r y h g c A d a y m e B h t Coding a M © t 82 d a © MathByte Academy c Operator Precedence A e t y B h t a M © t h ir g y p o C y m e 83 When we write an expression such as this: à what does it mean? (2 * 10) + 5 à 25 ? or © t M e t y c A 2 * (10 + 5) à 30 ? B h t a Python chooses this why? d a 2 * 10 + 5 y m e à operator precedence © MathByte Academy C p o i r y h g 84 Operators have precedence d a y m e à an operator with higher precedence will bind more tightly c A à fancy way of saying it will be evaluated first e t y Precedence order with arithmetic operators lower binary + * / unary + - higher ri 2 * 10 + 5 y p à 20 + 5 à 25 © MathByte Academy © t M except for a unary operator to the right of ** ** o C B h t a (equal precedence – since it does not actually matter) h g * has higher precedence than + à 2 * 10 is evaluated /irst 85 ** has highest precedence in our previous list 2 * 2 ** 3 -2 ** 4 à 2 * (2 ** 3) à -(2 ** 4) c A à 2 * 8 e t y d a y m e à 16 à -16 (as opposed to (-2) ** 4 à 16) B h t a à except when unary operator is to the right of ** 2 ** -3 © t M à 2 ** (-3) h g à 0.125 à makes sense, difficult to interpret it otherwise anyway © MathByte Academy C p o i r y 86 d a y m e A complete list of all operator precedence in Python can be found here: c A https://docs.python.org/3/reference/expressions.html#operator-precedence e t y à my advice B h t a à relying on operator precedence is tricky à very easy to introduce bugs à use parentheses h g © t M à it's just a few keystrokes more and will save a lot of pain later! i r y use (2 * 10) + 5 © MathByte Academy C p o instead of 2 * 10 + 5 87 e t y © MathByte Academy C p o i r y h g c A d a y m e B h t Coding a M © t 88 d a © MathByte Academy y m e c Integer Division and A Mod e t y B h t a M © t h ir g y p o C 89 Let's review long division! 43 e t y 3 131 12 11 9 2 h g Python integer division: p o i r y B h t a 131 / 3 à 43 © t // M C 2 is the remainder # $ 43 is the integer portion of the division 131 // 3 à 43 Remainder: use Python mod operator © MathByte Academy c A d a y m e % 131 % 3 à 2 90 The // Operator c A a // b calculates the "integer portion" of a / b d a y m e à easy to understand when a and b are positive Reality: a // b is the 4loor of a / b floor(x) -4 © t -3 M floor(-3.14) à -4 i r y B h t a à the largest integer number 100 © t B h t a à makes sense M à doesn't really make sense à == between operands that are not comparable usually returns False i r y h g à = 70: grade_letter = 'C' else: if grade >= 60: grade_letter = 'D' else: grade_letter = 'F' © MathByte Academy C p o i r y h g © t M B h t a 127 The elif Clause c A d a y m e Instead of this nested structure, Python provides an elif clause e t y à equivalent to a nested else-if à does not require this double indentation à easier to read! M if grade >= 90: grade_letter = 'A' elif grade >= 80: grade_letter = 'B' else: grade_letter = 'F' © MathByte Academy C p o i r y h g © t B h t a once an if or elif clause executes (is True) à no other if, elif or else block executes else executes if no if or elif statement executed 128 Grade Letter Example if grade >= 90: grade_letter = 'A' else: if grade >= 80: grade_letter = 'B' else: if grade >= 70: grade_letter = 'C' else: if grade >= 60: grade_letter = 'D' else: grade_letter = 'F' © MathByte Academy o C y p ri h g © t M c A d a y m e if grade >= 90: grade_letter = elif grade >= 80: grade_letter = elif grade >= 70: grade_letter = elif grade >= 60: grade_letter = else: grade_letter = e t y B h t a 'A' 'B' 'C' 'D' 'F' à much more human readable! 129 e t y © MathByte Academy C p o i r y h g c A d a y m e B h t Coding a M © t 130 d a y m e c Ternary Conditional Operator A e t y © MathByte Academy C p o i r y h g © t M B h t a 131 Terminology unary operator c A d a y m e à an operator that takes a single operand e t y à operator usually a prefix to the operand à -x binary operator B h t a à an operator that takes two operands à usually operands are on either side of the operator àx + y ternary operator © MathByte Academy C p o h g © t M à an operator that takes three operands i r y à so how do we write that? ! 132 d a y m e à suppose we have an operator that takes three operands: a, b, c c A à the goal is for the operator to return a + (b * c) e t y à this is a thing – it's called the Multiply-Accumulate operator (MAC) (but not available in Python!) a accmul b, c à maybe this? à or maybe this? i r y B h t a h g © t M a acc b mul c all we've done here is split the name of the operator into two and added the operands in between © MathByte Academy C p o 133 This type of conditional code is often used if : var = value1 else: var = value2 e t y c A d a y m e B h t a à key is that each code block is a single assignment à to the same variable © t M à Python introduces a conditional ternary operator to do this © MathByte Academy C p o i r y h g 134 The conditional ternary operator c A d a y m e à remember that an operator operates on operands and returns (calculates) some result if : var = value1 else: var = value2 © t e t y M B h t a à in this case we want the ternary operator's operands to be: à the conditional expression i r y h g à the value to return if the expression is True à the value to return if the expression is False © MathByte Academy C p o 135 The conditional ternary operator if : var = value1 else: var = value2 e t y M c A d a y m e B h t a value1 if else value2 © t à this is a single ternary operator i r y h g à if condition is True, it returns value1 à if condition is False, it returns value2 © MathByte Academy C p o 136 Example if price < 100: volume = 10 else: volume = 1 e t y c A d a y m e B h t a à can be re-written using a conditional ternary operator © t M volume = 10 if price < 100 else 1 © MathByte Academy C p o i r y h g 137 General Form c A d a y m e à we saw examples where we used values as the return operands e t y à but it's more general than that à the two value operands can be any expression B h t a à the result of the expression is then used M if else h g © t var = (a – b) if a > b else (b – a) © MathByte Academy C p o i r y 138 Short-Circuiting c A d a y m e Just like we saw with Boolean operators, the ternary operator also uses short-circuit evaluation e t y B h t a if else à first evaluates M à if it is True, evaluates and returns © t à but does not evaluate h g à if it is False, evaluates and returns © MathByte Academy C p o i r y à but does not evaluate 139 Example result = a / b if b != 0 else 'NaN' e t y c A a = 10 b = 5 à returns 2 à b is 5, so b != 0 evaluates to True à a / b is calculated and returned a = 10 b = 0 à this works just fine, and returns NaN © t M d a y m e B h t a à b is zero, so b != 0 evaluates to False h g à a / b is not calculated © MathByte Academy C p o i r y (thereby avoiding a division by zero exception) à NaN is returned 140 e t y © MathByte Academy C p o i r y h g c A d a y m e B h t Coding a M © t 141 Sequence Types c e t y © MathByte Academy C p o i r y h g © t M B h t a A d a y m e 5 142 What are Sequences? à sequences are ordered collections of objects à there is a first element e t y à there is a second element à and a next one c A d a y m e à sometimes called the sequential order B h t a à we can index those elements using integers M à like counting them one by one © t à but in Python (and most other programming languages) h g à numbering starts at 0 i r y à like anything in Python, sequences are objects p o à they just happen to be container type objects that contain other objects © MathByte Academy C 143 Indexing Sequences n objects c A d a y m e à length of sequence e t y object, object, object, …, object 0 1 2 B h t a n-1 à n objects in sequence M à last element index is n-1 in this course: h g © t à first element refers to the element at index 0 i r y à second element refers to the element as index 1 p o à last element refers to the element at index n-1 (assuming n elements in sequence) © MathByte Academy C 144 Sequence Length à sequences are usually finite e t y à but not all sequence types are c A d a y m e B h t a à in this course we'll stick to /inite sequences à first element à last element i r y à finite length © MathByte Academy C p o h g © t M 145 y m Homogeneous vs Heterogeneous Sequencese d a c A e t y B h t a M © t h ir g y p o C certain sequence types can only contain objects that are all the same type à homogeneous sequence types other types of sequences may contain objects that are of different type à heterogeneous sequence types © MathByte Academy 146 Sequence Types in this Chapter © MathByte Academy c A d a y m e lists à mutable heterogeneous sequence type tuples à immutable heterogeneous sequence type strings à immutable homogeneous sequence type C p o e t y i r y h g © t M B h t a 147 Lists © MathByte Academy C p o e t y i r y h g © t M c A d a y m e B h t a 148 The list Type à it is a container type à it contains elements à it is a sequence type à elements are ordered sequentially à elements are indexed e t y à lists can be heterogeneous à lists are mutable h g à lists are objects à they have state y p ri © t M à can add as many elements as we want à but they are still finite à the elements contained in the list à they have functionality o C B h t a à can add, replace or remove elements à lists have unbounded growth © MathByte Academy c A d a y m e à add element, remove element, etc 149 list Literals à Python lists can be created using literals e t y [10, 20, 30, 40] c A d a y m e B h t a note the enclosing square brackets [] à this is what indicates the type is a list à all elements are integers à but they don't have to be [10, 3.14, True] ri h g © t à they can even be nested © MathByte Academy M à this list is homogeneous o C y p [10, 20, 30, [True, False]] the last element of this list is itself a list 150 Accessing list Items by Index à lists are sequence types à sequential order l = [10, 20, 30, 40, 50] index 0 1 2 3 4 B h t a à we can reference an element by its index l à 10 l à 20 l à 30 ⚠ © MathByte Academy i r y h g © t M e t y c A d a y m e à indexable (length is 5) l[i] Trying to access a list by index greater than last index will cause an exception! p o l C à IndexError 151 Sequence Length l = [10, 20, 30, 40, 50] à visual inspection e t y à length of l is 5 c A d a y m e B h t a à but we can use code to calculate this for us à the len function len(l) à 5 h g © t M len([True, False]) à 2 © MathByte Academy C p o i r y 152 Empty Lists sometimes we want to start with an empty list c A d a y m e and have code that adds to the list as our program runs e t y à to create an empty list we can just use a literal l = [] then len(l) à 0 ⚠ © MathByte Academy l C h g © t M B h t a à IndexError p o i r y 153 Replacing a list Element l = [10, 20, 30, 40, 50] à we can retrieve elements by index print(l) à 30 e t y c A d a y m e B h= t a à but we can also replace an element at index i with a different element à we use the assignment operator l = True © t M l = [10, 20, True, 40, 50] h g print(l) ⚠ l i r y à True we are replacing elements – so the index must be valid! © MathByte Academy C p o = 100 à IndexError 154 e t y © MathByte Academy C p o i r y h g c A d a y m e B h t Coding a M © t 155 Tuples © MathByte Academy C p o i r y e t y h g © t M c A d a y m e B h t a 156 The tuple Type à very similar to the list type e t y à it is a container type à it is a sequence type à tuples can be heterogeneous à BUT… M c A d a y m e B h t a they are an immutable container type h g © t à unlike lists, once a tuple has been created i r y à cannot add or remove elements p o à cannot replace elements © MathByte Academy C 157 tuple Literals à Python tuples can be created using literals e t y (10, 20, 30, 40) c A d a y m e note the enclosing round brackets () à this indicates the collection is a tuple M B h t a à just like lists, they can can contain any object, including another tuple © t (10, 20, (3, 4)) i r y h g (10, 20, (True, False), [100, 200]) © MathByte Academy C p o 158 tuple Literals à often we don't even need the () c A d a y m e à Python interprets a comma separated list of elements as a tuple à so we can write (10, 20, 30) à or just 10, 20, 30 e t y B h t a à both these code snippets result in t being a tuple © t t = (10, 20, 30) h g M t = 10, 20, 30 © MathByte Academy C p o i r y 159 tuple Literals à just like lists, tuples can contain any object à including other tuples or lists B h t a (1, [True, False], (3, 4)) list e t y c A d a y m e tuple tuple © t M à we can omit the parentheses on the outer tuple h g 1, [True, False], (3, 4) i r y à but not (3, 4) p o 1, [True, False], 3, 4 © MathByte Academy C à not the same 160 Indexing, Length c A d a y m e à just like lists, elements can be read back from a tuple using an index number e t y à the len() function works with tuples also t = 10, 20, 30, 40, 50 len(t) à 5 t à 10 t à 30 i r y h g © t M B h t a t à IndexError © MathByte Academy C p o 161 tuples are Immutable c A à unlike lists, we cannot replace an element of a tuple e t y t = 10, 20, 30 t = 100 d a y m e B h t a à TypeError à the container is immutable à does not mean elements in the container are immutable © t M t = 10, 20, [True, True] t = 100 p o i r y h g à TypeError t = False © MathByte Academy C last element is a list à which is mutable t à 10, 20, [True, False] 162 Creating Empty tuples à not very useful, so not used very often e t y à use empty parentheses t = () c A d a y m e B h t a à that tuple is immutable, so it will remain empty for its lifetime © MathByte Academy C p o i r y h g © t M 163 e t y © MathByte Academy C p o i r y h g c A d a y m e B h t Coding a M © t 164 Strings © MathByte Academy C p o i r y e t y h g © t M c A d a y m e B h t a 165 The str Type à this is also a container type e t y à it is a sequence type c A d a y m e B h t a à strings are homogeneous à they can only contain characters (unicode) à they are immutable © MathByte Academy C p o i r y h g © t M 166 str Literals à Python strings can be created using literals e t y 'this is a string' c A d a y m e note the enclosing quotes '…' à can also use double quotes "this is a string" h g © t M B h t a note the enclosing double quotes "…" à these quotes/double-quotes are called the string delimiters i r y à an empty string literal can be '' © MathByte Academy C p o or "" 167 Indexing, Length à works the same way as any sequence type e t y c A d a y m e à use an index number to access elements of the string à use the len() function to find the length of the string s = 'Python' len(s) à 6 s à 'P' s à 'y' i r y (a string containing a single character) h g s à 'n' M B h t a © t s à IndexError © MathByte Academy C p o 168 e t y © MathByte Academy C p o i r y h g c A d a y m e B h t Coding a M © t 169 Slicing © MathByte Academy C p o i r y e t y h g © t M c A d a y m e B h t a 170 d a y m e à slicing is a way to extract ranges of elements from a sequence à start position (by index number) e t y à stop position (by index number) [start:stop] M c A B h t a à start index is inclusive of the element © t à stop index is exclusive of the element i r y h g à slices are the same type as the sequence being sliced © MathByte Academy C p o 171 l = [10, 20, 30, 40] 0 1 l à 10 l[0:2] 2 3 l à 20 c A l à 30 e t y d a y m e l à 40 à starts at 0, and includes element at 0 B h t a à ends at 2, but excludes element at 2 l[0:2] à [10, 20] l[1:3] à [20, 30] h g à result is also a (new) list © t M t = (10, 20, 30, 40) 0 1 i r y 2 3 t[0:2] à (10, 20) p o à result is also a (new) tuple t[1:3] à (20, 30) © MathByte Academy C 172 d a y m e à str type is a sequence type à slicing for strings works the same way s = 'Isaac Newton' 0 1 2 3 4 5 6 7 8 9 10 11 e t y s[0:4] à 'Isaa' s[0:5] à 'Isaac' s[6:9] à 'New' © MathByte Academy C p o i r y h g © t M c A B h t a 173 Including Last Element in Slice s = 'Isaac Newton' c A d a y m e s[6:11] à 'Newto' 0 1 2 3 4 5 6 7 8 9 10 11 e t y à how do we specify including the last element ? B h t a à it's ok to specify indexes outside the sequence bounds! à Python will automatically figure it out s[6:12] à 'Newton' © t s[6:1000] à 'Newton' h g M à we can also leave the end index blank à Python will interpret as "up to and including the last element" p o i r y s[6:] à 'Newton' © MathByte Academy C 174 Including First Element in a Slice s = 'Isaac Newton' 0 1 2 3 4 5 6 7 8 9 10 11 à just specify 0 as the start s[0:5] à 'Isaac' B h t a à can also leave the start index blank s[:5] à 'Isaac' àthis is actually valid: i r y © t e t y c A d a y m e M s[:] à 'Isaac Newton' h g à this made a shallow copy of the sequence à we'll come back to that in a bit © MathByte Academy C p o 175 Slicing with Steps c A d a y m e à a step is a way to specify an interval when slicing a sequence e t y s[start:stop:step] [2:10:2] à start at (and include) index 2 B h t a à end at (but exclude) index 10 à move in steps of 2 2 3 4 5 6 h g 7 © t M 8 à indexes: 2 9 4 6 8 l = [10, 20, 30, 40, 50, 60, 70, 80, 90, 100] 0 1 y p ri 2 3 4 5 6 7 8 9 l[2:10:2] à [30 ,50 ,70 ,90] © MathByte Academy o C 176 Negative Steps à possible to use negative step values e t y à starts at index start (inclusive) à stops at index end (exclusive) à moves backwards B h t a c A d a y m e à so start should be greater than end l = [10, 20, 30, 40, 50, 60, 70, 80, 90, 100] 0 1 2 3 4 © t M 5 6 7 8 9 l[9:6:-1] à [100, 90, 80] © MathByte Academy h g l[:6:-1] à [100, 90, 80] l[3::-1] à [40, 30, 20, 10] l[::-1] à [100, 80, 80, 70, 60, 50, 40, 30, 20, 10] C p o i r y 177 à strings are sequence types à also works for strings s = 'Isaac Newton' 0 1 2 3 4 5 6 7 8 9 10 11 © MathByte Academy e t y s[11:5:-1] à 'notweN' s[:5:-1] à 'notweN' s[::-1] à 'notweN caasI' s[10::-2] à 'owNcaI' s[::-2] à 'nte as' C p o i r y h g © t M c A d a y m e B h t a 178 e t y © MathByte Academy C p o i r y h g c A d a y m e B h t Coding a M © t 179 d a © MathByte Academy y m e c Manipulating Sequences A e t y B h t a M © t h ir g y p o C 180 àmutable sequences can be modi/ied à replace elements e t y à delete elements à add elements c A d a y m e B h t a à often appended (to the end) à can also specify where in the sequence to insert © MathByte Academy C p o i r y h g © t M 181 Replacing Single Elements c A d a y m e Replace an element at index i by assigning a new element to that index e t y l = [10, 20, 30] l = 'hello' M l à [10, 'hello', 30] © MathByte Academy C p o i r y h g © t B h t a 182 Replacing an entire Slice à can also replace an entire slice e t y c A d a y m e à just assign a new collection to the slice à slice will be replaced with elements in RHS B h t a my_list = [1, 2, 3, 4, 5] M my_list [0:3] = ['a', 'b'] © t my_list [0:3] = ('a', 'b') h g my_list à ['a', 'b', 4, 5] my_list [0:3] = 'ab' i r y à Python uses the elements of the sequence in RHS when assigning to a slice (but not when assigning using a single index) © MathByte Academy C p o 183 Deleting Elements à can delete an element by index my_list = [1, 2, 3, 4, 5] del my_list my_list à [1, 3, 4, 5] à can delete an entire slice © t M e t y c A d a y m e B h t a my_list = [1, 2, 3, 4, 5] i r y h g del my_list[1:3] my_list à [1, 4, 5] © MathByte Academy C p o 184 Appending Elements à we can append one element my_list = [1, 2, 3] my_list.append(4) e t y c A d a y m e B h t a my_list à [1, 2, 3, 4] à to append multiple elements, we extend the sequence my_list = [1, 2, 3] © t M my_list.extend(['a', 'b', 'c']) my_list.extend(('a', 'b', 'c')) my_list.extend('abc') p o i r y h g does the same thing my_list à [1, 2, 3, 'a', 'b', 'c'] © MathByte Academy C 185 Inserting an Element c A d a y m e à instead of appending, we can insert at some index à use sparingly – this is much slower than appending or extending my_list = [2, 3, 4, 5] my_list.insert(0, 100) e t y B h t a my_list à [100, 2, 3, 4, 5] © t M my_list = [2, 3, 4, 5] h g my_list.insert(2, 100) i r y my_list à [2, 3, 100, 4, 5] p o à element is inserted so its position is the index - remaining elements are shifted right © MathByte Academy C 186 e t y © MathByte Academy C p o i r y h g c A d a y m e B h t Coding a M © t 187 © MathByte Academy c Copying SequencesA e t y B h t a M © t h ir g y p o C d a y m e 188 Shallow vs Deep Copies à two types of copies e t y à shallow copies à new sequence is created c A d a y m e (not same sequence object as original) B h t a à elements in new sequence reference the same elements as original à deep copies © t M à new sequence is created h g (not same sequence object as original) à each element in new sequence is a deep copy of the original i r y à totally new and independent objects © MathByte Academy C p o 189 Shallow Copy e t y obj 1 index 0 index 1 index 2 index 0 index 1 index 2 obj 2 obj 3 original © t c A d a y m e M B h t a shallow_copy original is shallow_copy à False i r y h g à original and shallow_copy are not the same containers à but the elements are referencing the same objects © MathByte Academy C p o 190 Shallow Copy obj 1 index 0 index 1 index 2 obj 2 obj 3 c A index 0 index 1 index 2 e t y d a y m e B h t a shallow_copy original à add/remove/replace element in one does not affect the other index 0 index 1 index 2 y p ri original © MathByte Academy o C © t M obj 1 h g obj 2 obj 3 obj 4 index 0 index 1 index 2 index 3 shallow_copy 191 Shallow Copy c A d a y m e à but mutating an element will affect both (since it is a shared reference) obj 1 index 0 index 1 index 2 B h t a obj 2 obj 3 original © MathByte Academy C p o i r y h g M (modi0ied) © t e t y index 0 index 1 index 2 shallow_copy 192 Creating Shallow Copies à use slicing to slice the entire sequence à use the copy method my_list = [1, 2, 3] M my_copy = my_list[:] my_copy = my_list.copy() h g del my_copy © MathByte Academy C p o i r y © t à my_list[:] e t y B h t a c A d a y m e à my_list.copy() my_copy à [1, 2, 3] my_copy à [2, 3] my_list à [1, 2, 3] 193 Mutable Elements my_list = [['a', 'b'], 2, 3] e t y my_copy = my_list.copy() c A d a y m e B h t a à my_list and my_copy are both referencing the same list ['a', 'b'] my_list is my_copy à True © t M so if we modify that element (from either sequence): h g my_copy.append('c') i r y my_copy à ['a', 'b', 'c'] p o my_list à ['a', 'b', 'c'] © MathByte Academy C 194 Creating Deep Copies à uses deepcopy function in the copy module e t y from copy import deepcopy c A d a y m e B h t a my_list = [['a', 'b'], 2, 3] my_copy = deepcopy(my_list) M my_list is my_copy à False h g © t à element has been copied too! my_copy.append('c') i r y my_copy à ['a', 'b', 'c'] p o my_list à ['a', 'b'] © MathByte Academy C 195 e t y © MathByte Academy C p o i r y h g c A d a y m e B h t Coding a M © t 196 d a © MathByte Academy c Unpacking Sequences A e t y B h t a M © t h ir g y p o C y m e 197 consider a sequence data = (1, 2, 3) d a y m e à this is a tuple with three elements c A we want to assign those values 1, 2 and 3 to some symbols a, b and c resp. à could do it this way: a = data b = data c = data M B h t a but Python has a better way of doing this! © t a, b, c = (1, 2, 3) h g e t y unpacking Since tuples don't actually need the parentheses in this case, we can write: i r y a, b, c = 1, 2, 3 © MathByte Academy C p o 198 à this works with any sequence in general a, b = [10, 20] a, b, c = 'XYZ' © MathByte Academy C p o i r y h g a à 10 b à 20 B h t a a à 'X' b à 'Y' c à 'Z' © t M e t y c A d a y m e 199 à beware! d a y m e à number of elements in sequence on RHS must match number of symbols on LHS e t y c A a, b = 1, 2, 3 à ValueError (too many values to unpack) B h t a a, b, c = 1, 2 à ValueError (not enough values to unpack) © MathByte Academy C p o i r y h g © t M 200 Swapping Two Variable Values c A d a y m e à this is a common problem given two variables a and b, swap the value of a and b Initial State: a à 10 b à 20 e t y End State: a à 20 b à 10 B h t a à typical solution uses a temporary variable temp = a a = b b = temp i r y h g © t M à 3 lines of code and an unnecessary variable © MathByte Academy C p o 201 Swapping Two Variable Values c A d a y m e à can use unpacking to our advantage à remember: in an assignment, the RHS expression is evaluated completely /irst à then the assignment takes places e t y a, b = b, a à RHS is evaluated /irst © t b, a is the tuple 20, 10 M à then the assignment is made h g B h t a a, b = 20, 10 à values of a and b have been swapped! © MathByte Academy C p o i r y 202 e t y © MathByte Academy C p o i r y h g c A d a y m e B h t Coding a M © t 203 Strings e t y © MathByte Academy C p o i r y h g © t M B h t a c A d a y m e 6 204 à strings are sequence types e t y c A d a y m e à but they are more specialized than generic sequences B h t a à they are homogeneous à each element is a single character M à we have additional functionality available © MathByte Academy C p o i r y h g © t 205 Unicode © MathByte Academy C p o i r y h g e t y © t M c A d a y m e B h t a 206 In the beginning… … there was ASCII c A d a y m e (American Standard Code for Information Interchange) à addressed the problem of a standard for assigning à numeric codes e t y à to characters à printable and non-printable B h t a à and encoding the value into binary M à using sequences of 7 bits à given a data stream filled with 0's and 1's à carve up in 7 bits and decode character i r y h g © t à fonts handle displaying the character à a bunch of pixels p o à a glyph © MathByte Academy C 207 à supported character set was limited à 128 characters à 95 printable characters c A (a-z, A-Z, 0-9, * / etc) e t y d a y m e à 33 non-printable characters (control codes, e.g. esc, newline, tab, etc) © MathByte Academy C p o i r y h g © t M B h t a 208 à attempts were made to extend the ASCII set à still far too limited à standard was poorly followed à Unicode was developed e t y B h t a à focused on assigning a code to a character c A d a y m e (code point) à does not specify how to encode the code points into a binary format M à other standards for doing that appeared à UTF-8 h g à UTF-16 à UTF-32 y p ri © t ß very popular, default in Python (utf à Unicode Transformation Format) à > 100,000 code points de/ined so far © MathByte Academy o C 209 Code Points à backward compatible with ASCII e t y c A d a y m e ASCII character code for A à 65 (decimal), 41 (hexadecimal) Unicode code point for A à 65 (decimal), 41 (hexadecimal) B h t a decimal à base 10 © MathByte Academy C p o i r y h g © t M (0 – 9) hexadecimal à base 16 (0-9, A-F) 210 What is hex anyway? Decimal system – uses powers of 10 103 102 101 100 9 0 3 4 22 21 20 1 0 1 1 e t y © MathByte Academy 162 F C C 161 p o 1 i r y 160 5 B h t a à 2 digits, 0-1 M (1011)# = 1×2! + 1×2" + 0×2# + 1×2$ = 11"! h g © t Hexadecimal – uses powers of 16 163 à 10 digits, 0-9 9034 = 4 ×10! + 3 ×10" + 0 ×10# + 9 ×10$ Binary – uses powers of 2 23 c A d a y m e à 16 digits, 0-9, A-F A à 10, Bà 11, …, F à 15 >?15 = 5×16! + 1×16" + 12×16# + 15×16$ = 64533"! 211 Unicode Character A https://www.compart.com/en/unicode/U+0041 e t y © MathByte Academy C p o i r y h g © t M B h t a c A d a y m e the character (hex) code the character name corresponding lowercase letter 212 à ord() function c A d a y m e à returns code point for a single character (in decimal) ord('A') à 65 e t y à hex() B h t a à converts decimal to hex string M hex(65) à '0x41' (0x pre/ix indicates the number after that is in hex) © MathByte Academy C p o i r y h g © t 213 https://www.compart.com/en/unicode/U+03B1 e t y © t M c A d a y m e B h t a hex(ord("α")) à '0x3b1' © MathByte Academy C p o i r y h g copy/paste the glyph from that page straight into your Python code 214 y m Other ways to specify the character in a string e d a c A e t y B h t a M © t h ir g y p o C à use escape codes à by hex code à by name "\N{Greek Small Letter Alpha}" à "α" "The letter \N{Greek Small Letter Alpha} is the first letter of the Greek alphabet." à 'The letter α is the first letter of the Greek alphabet.' "\u03b1" "\u03B1" \u must be followed by exactly 4 hex digits (0-F) "The letter \u03B1 is the first letter of the Greek alphabet." à'The © MathByte Academy letter α is the first letter of the Greek alphabet.' 215 https://www.compart.com/en/unicode/U+1F40D e t y à ' "' "\N{snake}" © t M c A d a y m e B h t a ànote how character code 1F40D has 5 digits h g à must use \U followed by exactly 8 digits i r y "\U0001F40D" p o (\u is limited to 4 digits) à ' "' pad with zeroes to make 8 digits © MathByte Academy C 216 e t y © MathByte Academy C p o i r y h g c A d a y m e B h t Coding a M © t 217 d a © MathByte Academy y m e c Common String Methods A e t y B h t a M © t h ir g y p o C 218 à Python has a ton of string methods c A d a y m e https://docs.python.org/3/library/stdtypes.html#string-methods e t y In this video we are going to look at some in these categories à case conversions B h t a à stripping start and end characters à concatenating strings M à splitting and joining strings © t à finding substrings i r y h g à methods, called using dot notation p o à remember that strings are immutable à operations never modify a string © MathByte Academy C my_string.method() à just return a new string 219 Case Mappings c A d a y m e lower() 'Hello World'.lower() à 'hello world' upper() 'python'.upper() à 'PYTHON' title() 'one two three'.title() à 'One Two Three' e t y à returns a new string h g © t M B h t a à primarily used for visual display i r y à BEWARE: may not work for caseless comparisons © MathByte Academy C p o 220 Case Folding casefold() c A à used for caseless comparisons e t y s1 = 'hello' s2 = 'HeLlo' M B h t a s1.casefold() == s2.casefold() h g © t d a y m e à True à we'll explore this vs using case mappings in the code section © MathByte Academy C p o i r y 221 Stripping c A d a y m e sometimes we want to remove leading and trailing characters à trailing commas à whitespace around a string e t y B h t a.lstrip() à strips all whitespace on left of string.rstrip() à strips all whitespace on right of string.strip() à strips all whitespace on both ends of string © t M à can specify what characters to strip i r y h g.strip(' ') à strip space characters from both ends.lstrip('abc') à strip the characters 'a', 'b', 'c' from left end p o à returns a new string © MathByte Academy C 222 Concatenation c A d a y m e combining two or more strings to form a single string is called concatenation 'Hello' + ' ' + 'World!' © t M e t y à 'Hello World!' B h t a à again, this creates a new string © MathByte Academy C p o i r y h g 223 Splitting Strings à useful for parsing data from a text file e t y data = '100, 200, 300, 400' B h t a à can easily split this on the comma data.split(',') à returns a list of strings © MathByte Academy C p o i r y h g © t M c A d a y m e ß a string containing comma delimited values ['100', ' 200', ' 300', ' 400'] note the spaces à we can strip them later 224 Joining Strings à this is the opposite of splitting strings e t y c A d a y m e suppose we want to join these strings with `, ` characters between each: 'a' 'b' 'c' 'd' we could write: B h t a 'a' + ', ' + 'b' + ', ' + 'c' + ', ' + 'd' à tedious to type out à "hardcoded" i r y h g © t M à what if we have a sequence of strings we want to concatenate p o à this approach is not general enough © MathByte Academy C 225 Joining Strings data = ('ab', 'cd', 'ef') '--'.join(data) c A d a y m e à data is a sequence of strings e t y à join the strings in data with -- in between à 'ab--cd--ef' B h t a ','.join(['10', '20', '30']) à '10,20,30' M à remember that a string is a sequence of single characters i r y h g © t '='.join('python') à 'p=y=t=h=o=n' © MathByte Academy C p o 226 Finding Substrings c A d a y m e à often just want to know if a sequence of characters is contained inside another e t y à use the in operator B h t a 'x' in 'xyz' à True 'a' in 'xyz' à False 'pyt' in 'python' à True 'pyt' in 'Python' à False i r y h g © t M à tests containment p o à but gives no indication of where the substring is © MathByte Academy C 227 à slight variation c A d a y m e à does the string start (or end) with the speci/ied characters e t y à still a containment test.startswith('…') B h t a.endswith('…') M 'python'.startswith('py') à True © t 'python'.startswith('hon') à False h g 'python'.endswith('py') à False i r y 'python'.endswith('hon') à True © MathByte Academy C p o 228 Finding the Index of a Substring c A d a y m e à used when we need to know the index of the start position of a substring e t y data = 'This is a grammatically correct sentence.' à at what index does the string 'correct' occur? B h t a data.index('correct') © t M à 24 à what if substring is not found? h g à Python raises a ValueError i r y à potentially useful once we learn how to handle exceptions © MathByte Academy C p o 229 Finding the Index of a Substring à what if we don't want an exception? à find e t y c A d a y m e à returns -1 if substring is not found B h t a data = 'This is a grammatically correct sentence.' M data.find('correct') © t data.find('DOW') i r y h g à 24 à -1 à once we know how to handle exceptions, this method is a bit redundant p o à personally, I prefer using index and using exception handling © MathByte Academy C 230 Important Note c A d a y m e à only interested in whether or not a substring is contained in another string e t y à use in B h t a à only use index or find when you need to know the index à in is much faster! © MathByte Academy C p o i r y h g © t M 231 e t y © MathByte Academy C p o i r y h g c A d a y m e B h t Coding a M © t 232 © MathByte Academy c String InterpolationA e t y B h t a M © t h ir g y p o C d a y m e 233 c A d a y m e à often we want to build strings that contain values from some variable e t y à can use concatenation B h t a à + works with two strings à cannot mix string and numeric for example 'test' + 100 à TypeError © t M 'test' + str(100) à 'test100' © MathByte Academy C p o i r y h g 234 Example Suppose we have four variables: open_ = 98 high = 100 low = 95 close=99 e t y c A d a y m e B h t a We want to build a string that looks like this for display purposes: M 'open: 98, high: 100, low: 95, close: 99' à using concatenation: i r y h g © t 'open: ' + str(open_) + ', high: ' + str(high) + ', low: ' + str(low) + ', close:' + str(close) p o à tedious and error prone! © MathByte Academy C à in fact there is an error, can you spot it?! 235 String Interpolation à multiple variants c A d a y m e à two most common techniques e t y à the format method à use {} as placeholders in our string B h t a à pass variables to format method in same order as we want them in the string à number of {} in string and arguments in format should match M à format can have more arguments, they'll just be ignored à IndexError exception if not enough arguments i r y h g © t 'open: {}, high: {}, low: {}, close: {}'.format(open_, high, low, close) p o à note how we did not have to convert the values to strings! © MathByte Academy C 236 f Strings à new to Python 3.6 e t y à prefix the string with f à use {expr} directly inside the string c A d a y m e B h t a à Python evaluates expr and interpolates the result directly inside the string f'1 + 1 = {1 + 1}' © t M à '1 + 1 = 2' value = 3.14 f'pi is approximately {value}' à 'pi is approximately 3.14' i r y h g f'open: {open_}, high: {high}, low: {low}, close: {close}' p o à 'open: 98, high: 100, low: 95, close:99' © MathByte Academy C 237 f Strings à could use do this as well open_ = 98 high = 100 low = 95 close=99 e t y M c A d a y m e B h t a f'open: {open_}, close: {close}, delta:{close – open_}' h g © t à 'open: 98, close: 99, delta: 1' © MathByte Academy C p o i r y 238 e t y © MathByte Academy C p o i r y h g c A d a y m e B h t Coding a M © t 239 Iteration e t y © MathByte Academy C p o i r y h g © t M B h t a c A d a y m e 7 240 y m e à fundamental aspect of writing programs is repetition à want to repeat the same process (code) multiple times à how many times? à known in advance à load /ile with 10,000 rows à process each row à deterministic à not known in advance © t e t y c A d a B h t a à repeat the same process 10,000 times M à get commodity tick data à analyze data until ask price falls below some level à then do something else and stop processing à process may repeat 10 times, or 100 times, we don't know in advance p o i r y h g à non deterministic © MathByte Academy C 241 à this repetition is called iteration deterministic iteration e t y c A d a y m e à we iterate over the elements of some container à e.g. sequences à more generally over objects that are iterable B h t a à not all iterables are sequences àa bag of marbles is iterable, but it is not a sequence! à for loop i r y h g © t M non-deterministic iteration à we iterate while some condition is True p o à while loop © MathByte Academy C 242 d a © MathByte Academy c The range Function A e t y B h t a M © t h ir g y p o C y m e 243 The range Object à range object is an iterable object e t y c A d a y m e à it serves up integers one by one as they are requested à but the full list of integers does not exist all at once in memory à memory efficient B h t a à it has a /inite number of integers © t M à we can iterate over that range object h g à since it exists and has a finite number of integers i r y à deterministic iteration à we can use the range() function to create range objects © MathByte Academy C p o 244 The range() Function c A d a y m e à three flavors depending on how many arguments are specified range(end) e t y (one argument) B h t a à generates integers from 0 (inclusive) to end (exclusive) range(start, end) (two arguments) M à generates integers from start (inclusive) to end (exclusive) © t range(start, end, step) h g (three arguments) à generates integers from start (inclusive) to end (exclusive) i r y à in steps of step p o à should remind you of slicing © MathByte Academy C 245 Viewing Contents of range Object r = range(5) print(r) à 'range(5)' à not what we wanted e t y c A d a y m e B h t a à can convert range object to a list or tuple print(tuple(r)) h g print(list(r)) © MathByte Academy C p o i r y M à (0, 1, 2, 3, 4) © t à [0, 1, 2, 3, 4] 246 Iteration à range object is iterable e t y c A d a y m e à we can use a for loop to iterate over the elements of this iterable (next lecture) © MathByte Academy C p o i r y h g © t M B h t a 247 e t y © MathByte Academy C p o i r y h g c A d a y m e B h t Coding a M © t 248 for Loops © MathByte Academy C p o i r y h g © t e t y M c A d a y m e B h t a 249 d a y m e à for loops are used to iterate over elements of any iterable c A à the loop mechanism retrieves elements from the iterable one at a time e t y à the body of the for loop is executed for each element retrieved B h t a à the loop terminates when all elements have been iterated © MathByte Academy C p o i r y h g © t M 250 y m e for x in ['a', 'b']: y = x + x note how the body is indented print(y) à just like if…else… code blocks print('done') unindented à not in loop body 1st iteration: 'a' is retrieved and assigned to the symbol x y is the concatenation of x and x à 'aa' e t y c A d a B h t a 'aa' is printed to the console M 2nd iteration: 'b' is retrieved and assigned to the symbol x y is the concatenation of x and x à 'bb' h g © t 'bb' is printed to the console i r y 3rd iteration: à no more elements à loop terminates à code after loop executes à 'done' © MathByte Academy C p o 251 Iterating over range Objects à range objects are iterable e t y for i in range(4): sq = i * i print(sq) output: 0 1 4 9 © MathByte Academy C p o i r y h g © t c A d a y m e range(4) à 0, 1, 2, 3 M B h t a 252 Loop Bodies (Blocks) à block can contain any valid Python code à if…else… à another loop (nested loop) B h t a for i in range(1, 4): for j in range(1, i+1): print(i, j, i*j) print('') 1 1 1 © MathByte Academy C p o i r y h g © t M e t y 2 1 2 2 2 4 3 1 3 3 2 6 3 3 9 c A d a y m e i = 1 j in range(1, 1+1) i = 2 j in range(1, 2+1) i = 3 j in range(1, 3+1) 253 data = [10, 20, 30, -10, 40, -5] c A suppose we want to replace any negative value with 0 e t y d a y m e à we can iterate over the data and test for negative numbers: for number in data: if number < 0: number = 0 print(number) i r y h g © t M B h t a 10 20 30 0 40 0 à but how do we replace -10 and -5 with 0 ? à easy if we know the index number p o à but we don't know that!! © MathByte Academy C data = 0 data = 0 254 The enumerate Function enumerate is a function that e t y à takes an iterable argument c A d a y m e à returns a new iterable whose elements are a tuple consisting of: B h t a à the index number of the original element à the original element itself M data = [10, 20, 30, -10, 40, -5] for t in enumerate(data): print(t) i r y h g © t à at each iteration t is a tuple (index, element) à it can be unpacked! © MathByte Academy C p o (0, (1, (2, (3, (4, (5, 10) 20) 30) -10) 40) -5) 255 data = [10, 20, 30, -10, 40, -5] for t in enumerate(data): index, element = t if element < 0: data[index] = 0 e t y c A d a y m e B h t a data à [10, 20, 30, 0, 40, 0] à but we can do one better h g © t M à we can unpack in the for clause itself for index, element in enumerate(data): if element < 0: data[index] = 0 © MathByte Academy C p o i r y 256 e t y © MathByte Academy C p o i r y h g c A d a y m e B h t Coding a M © t 257 while Loops © MathByte Academy C p o i r y h g © t M e t y c A d a y m e B h t a 258 à different than for d a y m e à here we want to repeat some code as long as some condition is True c A à non-deterministic à we don't necessarily know when condition becomes True e t y à maybe never! while expr: M à in/inite loop B h t a à expr is evaluated at the start of each iteration © t à if it is True, execute h g à if it is False, terminate loop immediately i r y à may never execute (if expr is False on /irst iteration) p o à may never terminate (if expr never becomes False) © MathByte Academy C 259 value = 10 increments value by 1 while value < 15: print(value) value = value + 1 output: © MathByte Academy C 10 11 12 13 14 p o i r y h g © t c A d a y m e e t y M B h t a 260 value = 100 while value < 15: print(value) value = value + 1 output: © MathByte Academy C e t y no output p o i r y h g © t M c A d a y m e B h t a 261 value = 10 while value < 15: print(value) value = value - 1 output: © MathByte Academy 10 9 8 7 6 … C p o © t c A d a y m e decrements value by 1 e t y M B h t a in/inite loop!! i r y h g 262 e t y © MathByte Academy C p o i r y h g c A d a y m e B h t Coding a M © t 263 d a © MathByte Academy y m e c continue, break,A else e t y B h t a M © t h ir g y p o C 264 Skipping an Iteration c A d a y m e à sometimes we want to skip an iteration, but without terminating the loop à continue à immediately jumps to the next iteration e t y B h t a my_list = [1, 2, 3, 100, 4, 5] for i in my_list: if i > 50: continue print(i) print('done') © MathByte Academy o C y p ri h g © t M 1 2 3 4 5 'done' à when i is 100 à continue is executed à loop jumps to next iteration 265 à continue is not used too often d a y m e à can sometimes make code difficult to read/understand for i in my_list: if i > 50: continue print(i) print('done') à equivalently: © MathByte Academy C p o i r y e t y M c A B h t a for i in my_list: if i 50: break print(i) print('done') © MathByte Academy C p o i r y h g © t 1 2 3 'done' M à when i is 100 à break is executed à loop is terminated immediately 267 Early Termination à loop terminating early using break e t y c A d a y m e à sometimes called abnormal or early termination B h t a à sometimes want to execute some code if loop terminated normally M à and different code otherwise (early/abnormal termination) © MathByte Academy C p o i r y h g © t 268 Example We are scanning through an iterable, looking for an element equal to 'Python' If we /ind the value, we want to terminate our scan immediately, and print 'found', otherwise we want to print 'not found' © MathByte Academy o C y p ri h g © t M e t y c A d a y m e found = False B h t a for el in my_list: if el == 'Python': found = True print('found') break if not found: print('not found') 269 The else Clause à Python is really confusing here… e t y for loops can have an else clause c A d a y m e à but it has nothing to do with the else clause of an if statement B h t a à the else clause of a for loop executes if and only if no break was encountered © t for i in range(5): else: # if no break © MathByte Academy C p o i r y h g M in my mind I read it as "else if no break" à executes if loop terminated normally (i.e. no break encountered) 270 Back to our Example found = False for el in my_list: if el == 'Python': found = True print('found') break if not found: print('not found') © MathByte Academy C p o i r y h g © t e t y c A d a y m e B h t a equivalently: M for el in my_list: if el == 'Python': print('found') break else: # if no break print('not found') 271 e t y © MathByte Academy C p o i r y h g c A d a y m e B h t Coding a M © t 272 Dictionaries e t y © MathByte Academy C p o i r y h g © t M B h t a c A d a y m e 8 273 d a y m e Dictionaries are one of the most important data structures in Python à we don't always see them e t y à but they're lurking in the shadows! ! c A B h t a à we saw that variables are symbols pointing to objects à some string (variable name) is associated with some object © t M objects are also dictionaries h g à properties are symbols associated to some value (object) i r y à methods are names associated to some function s.upper() l.append() © MathByte Academy C p o 274 c A à associating two things together is extremely useful d a y m e a phone book à associates a number to a name e t y DNS à associates a URL with a numeric IP address B h t a book index à associates a chunk of text with a page number à associative arrays M à sometimes called a map © t à abstract concept à can be implemented in different ways i r y h g à Dictionaries (or hash maps) are one concrete implementation © MathByte Academy C p o 275 d a y m e Associative Arrays and Dictionaries c e t y © MathByte Academy C p o i r y h g © t M A B h t a 276 Associating Things à ASCII table c A d a y m e à associates a numeric value to certain characters A à 65 B à 66 … Z à 90 a à 97 b à 98 … z à 122 We could try this: M e t y space à 32 < à 60 @ à 64 … B h t a keys = [' ', ' 0: d[widgets[i]] = sales[i] © MathByte Academy C p o i r y 340 Set Comprehensions à similar to a dictionary comprehension e t y à but elements are not key: value pairs à just the "key" portion M c A d a y m e B h t a {expr1 for item in items if expr2} © MathByte Academy C p o i r y h g © t can be any valid Python expression that calculates some value 341 Example c A d a y m e Given a list of integers, create a set that contains a unique collection of the squares of just the even integers e t y numbers = [1, 1, 2, 2, 3, 3, 4, 4, 5, 5, 6, 6] B h t a s = {number ** 2 for number in numbers if number % 2 == 0 } © MathByte Academy C p o i r y h g © t M 342 e t y © MathByte Academy C p o i r y h g c A d a y m e B h t Coding a M © t 343 Exceptions e t y © MathByte Academy C p o i r y h g © t M B h t a c A d a y m e 11 344 What are exceptions? c A d a y m e à exceptions are special events that happen when something out of the ordinary happens while our code is running e t y B h t a à an exception is generally unexpected behavior à but not always M à it may be something we expect to happen from time to time © t à we can deal with it and continue running our code i r y h g à so an exception is not necessarily an error p o à but unhandled exceptions will cause our program to terminate © MathByte Academy C 345 Terminology exception à a special type of object in Python raising à starting an exception event /low exception handling à interacting with an exception /low in some manner e t y h g unhandled exception © MathByte Academy c A d a y m e o C y p ri © t M B h t a à an exception flow that is not handled by our code à generally results in our program terminating abruptly 346 Exception Hierarchy à Python exceptions form a hierarchy c A d a y m e (we'll cover what that means precisely when we look at Object Oriented Programming - OOP) e t y B h t a https://docs.python.org/3/library/exceptions.html#exception-hierarchy à basically means that exceptions can be classes sub-divided into subexceptions that are more specific © t M à for example a broad exception might be LookupError à more speci/ically it could be an IndexError or a KeyError à both of these are categorized more broadly as a LookupError à can choose to handle IndexError specifically p o i r y h g à or LookupError more broadly © MathByte Academy C 347 Exception Hierarchy c A d a y m e this means that if an exception object is an IndexError exception à it is also a LookupError exception e t y à and it is also an Exception exception B h t a (most exceptions we work with are classi/ied as Exception types) à confused? think of it this way… © t M Say we have these classes of objects: h g College Person Student © MathByte Academy C p o i r y Staff Member Administrator à a Teacher is also Staff Member à a Staff Member is also a College Person Teacher 348 Exception Hierarchy à similarly we have an exception hierarchy Exception … LookupError IndexError KeyError OSError FileNotFoundError NotADirectoryError … h g © t M e t y c A d a y m e B h t a à can even write custom exception types à later à so to handle an IndexError, we could choose to i r y à handle IndexError exceptions very speci;ic à handle LookupError exceptions p o à handle Exception exceptions © MathByte Academy C very broad 349 Python Built-In Exceptions à Python has many built-in exception types e t y c A d a y m e https://docs.python.org/3/library/exceptions.html Common exceptions include: à SyntaxError à ZeroDivisionError à IndexError à KeyError h g à ValueError i r y © t M B h t a à TypeError p o à FileNotFoundError © MathByte Academy C à and many more… 350 EAFP vs LBYL c A d a y m e à when we think something unexpected may go wrong in our code e t y à /igure out if something is going to wrong before we do it à LBYL Look Before You Leap B h t a à just do it, and handle the exception if it occurs à EAFP generally in Python © MathByte Academy C p o i r y M Easier to Ask Forgiveness than Permission h g © t à follow EAFP à exception handling 351 Why EAFP? Something that is exceptional should be infrequent e t y c A d a y m e à if we are dividing two integers in a loop that repeats 1,000 times B h t a à out of every 1,000 times we run, we expect division by zero to occur 5 times LBYL à test that divisor is non-zero 1,000 times M EAFP à just do it, and handle the division by zero error 5 times © t à often more efficient i r y h g à also trying to fully determine if something is going to go wrong is a lot harder to write than just handling things when they do go wrong © MathByte Academy C p o 352 Exception Handling Flow à an exception occurs à an exception object is created à an exception Clow is started à we do nothing about it à program terminates M e t y c A d a y m e B h t a à we intercept the exception /low à try to handle the exception in some sense, if possible à then à resume running program uninterrupted à or, let the exception resume à or, start a new exception /low © MathByte Academy C p o i r y h g © t 353 © MathByte Academy c Raising ExceptionsA e t y B h t a M © t h ir g y p o C d a y m e 354 à often we want to start an exception /low ourselves à called raising an exception e t y c A d a y m e à an exception object is associated with an exception /low B h t a à we create a new exception object à we raise the exception object © t M à doing this is most useful when we create functions h g à we'll see this later i r y à for now, we'll just learn how to raise an exception © MathByte Academy C p o 355 Example c A d a y m e à create an exception object using one of Python's built-in exceptions e t y ex = ValueError() à usually we include a custom exception message B h t a ex = ValueError('Name must be at least 5 characters long.') M à we raise the exception, starting an exception flow raise ex i r y h g © t à often do both in one step p o raise ValueError('custom message') © MathByte Academy C 356 d a y m e à raising an exception ourselves results in the same exception flow that Python does when it raises some exception e t y à we can choose to handle the exception c A B h t a à if we don't handle the exception, program terminates © MathByte Academy C p o i r y h g © t M 357 e t y © MathByte Academy C p o i r y h g c A d a y m e B h t Coding a M © t 358 © MathByte Academy c Handling Exceptions A e t y B h t a M © t h ir g y p o C d a y m e 359 y m General Suggestions for Exception Handlinge d a c A e t y B h t a M © t h ir g y p o C à in general we do not want to just handle any exception anywhere in our code à too much work à cannot anticipate every point of failure à it's OK for program to terminate - we can figure out what went wrong and attempt to fix it later – possibly handling that case specifically à if we don't know exactly why or where the problem occurs in our code, there's not much we can do to recover from the exception à we handle exceptions that are raised by small chunks of code à we try to handle very specific exceptions, not broad ones à usually handle exceptions that we can do something about © MathByte Academy 360 try…except… c A d a y m e à wrap the code we want to implement an exception handler for inside a try block à we handle possible exception(s), using except blocks (one or more) e t y a = 1 b = 0 try: result = a / b except ZeroDivisionError as ex: print(f'Exception occurred: {ex}') result = 0 ri print(result) © MathByte Academy o C y p h g © t M B h t a this gives us the exception object that was raised – we can assign it to any symbol we want – here I just chose ex do something to handle the exception because we handled the exception, the flow was interrupted and our code continues to run normally 361 Handling and re-raising an exception c A d a y m e à sometimes we want to handle an exception, but then re-raise the same exception or a different exception e t y à often because there's nothing we can do à sometimes to create a more explicit exception B h t a to raise an exception in an except block: M raise à re-raises the same exception that caused the except block to be entered © t raise SomeException('…') à raises a new exception © MathByte Academy C p o i r y h g 362 Application c A d a y m e à one very common case for re-raising exceptions is for error logging e t y à we can view the logs after our program has terminated abnormally try: … except Exception as ex: log(ex) raise © t M B h t a à we intercept a broad range of exceptions by handling Exception i r y h g à we log the exception somewhere (console, /ile, database, etc) à we re-raise the exception and let something else either handle it or terminate the program © MathByte Academy C p o 363 à what do I mean "something else handles it"? à we'll see this more when we cover functions à try…except… can be nested c A d a y m e à usually indirectly à but directly too e t y try: try: raise ValueError('something happened') except ValueError as ex: log(ex) raise except Exception as ex: print(f'ignoring: {ex}') i r y h g © t M B h t a à here our ValueError gets handled twice! © MathByte Academy C p o 364 Handling Multiple Exception Types à not limited to a single except block try: … except IndexError as ex: … except ValueError as ex: … except Exception as ex: … h g © t M e t y c A d a y m e B h t a à Python will match the exception to the /irst type that matches in sequence of except blocks p o i r y à so write except blocks from most specific to least specific exception types à remember that exception hierarchy we looked at! © MathByte Academy C 365 The finally Clause c A d a y m e à sometimes we want some code to run after a try…except… whether an exception occurred or not, and whether it was handled or not e t y à use the finally clause B h t a try: … except ValueError as ex: … except IndexError as ex: … finally: # always runs no matter what, © MathByte Academy C p o i r y h g © t M before exception flow resumes 366 Application c A d a y m e à useful when we want a piece of code to always run à whether an exception has occurred or not à whether the exception was handled or not à whether exception was re-raised or a new one raised e t y B h t a try: open_database_connection() start_transaction() write_data() commit_transaction() except WriteException as ex: rollback_transaction() raise finally: close_database_connection() © MathByte Academy C p o i r y h g © t M 367 e t y © MathByte Academy C p o i r y h g c A d a y m e B h t Coding a M © t 368 y m e d Iterables and Iterators a c A e t y B h t a M © t h ir g y p o C 12 © MathByte Academy 369 y m e à an iterable is something that can be iterated over à i.e. we can take one element, then the next, then the next, until we've covered all elements à no speci/ic iteration order is mandated e t y c A d a à obviously a sequence type is iterable (positional ordering) B h t a à we saw dictionaries can be iterated over (insert order) à but we also so sets: iterable, but no guaranteed order of any kind M à general idea behind iteration is then: © t à start somewhere in the collection (at the beginning if that means something) i r y h g à keep requesting the next element à until there's nothing left (exhausted) © MathByte Academy C p o 370 à so we have two concepts here c A à a collection of objects that we can iterate over à an iterable e t y d a y m e à something that is able to give us the next element when we request it à an iterator © t M B h t a à we are going to look at those in this section © MathByte Academy C p o i r y h g 371 d a © MathByte Academy c Iterables and Iterators A e t y B h t a M © t h ir g y p o C y m e 372 à an iterable is something that can be iterated over à but we still need something that can e t y à give us the next item c A d a y m e B h t a à keep track of what it's given us so far (so it does not give us the same element twice) M à informs us when there's nothing left for it to give us © t à this is called an iterator i r y h g à used by Python to iterate over an iterable © MathByte Academy C p o 373 à an iterable is just a collection of objects d a à it doesn't know anything about how to iterate c A y m e à however it knows how to create and give us an iterator when we need it e t y à iterables implement a special method __iter__() that returns a new iterator B h t a à can also be called by using the iter() function à the iterator has a special method called __next__() that can be called to get the next element M à can also use the next() function h g © t à it keeps track of what it has already handed out (so iterators are kind of one time use!) i r y à it raises a StopIteration exception when next() is called if there's nothing left © MathByte Academy C p o 374 The Internal Mechanics of a for Loop c A d a y m e When we write a for loop that iterates over an iterable, what Python is actually doing this: e t y l = [1, 2, 3, 4, 5] B h t a iterator = iter(l) try: while True: # return next(iterator) – here we'll just print it print(next(iterator)) except StopIteration: # expected when we reach the end # so silence this exception pass © MathByte Academy C p o i r y h g © t M 375 d a y m e à the key thing here is that we can see the iterator has some state à it has a __next__() method c A à but there's no going back, or starting from the beginning again e t y à to do that we have to request a new iterator B h t a à and that's what a for loop does – it requests a new iterator from the iterable before it starts looping M à objects such as lists, tuples, string, dictionaries, sets, range objects are iterables © t à but some objects in Python are iterators – not iterables à iterators actually implement an __iter__ method i r y h g à but they just return themselves (with their current state), not a new iterator à they allows us to iterate over them p o à but only once © MathByte Academy C 376 e t y © MathByte Academy C p o i r y h g c A d a y m e B h t Coding a M © t 377 Generators © MathByte Academy C p o i r y h g © t e t y M c A d a y m e B h t a 378 à we've seen list, dictionary and set comprehensions à but no tuple comprehensions… e t y c A d a y m e result = [i ** 2 for i in range(5)] result = [] M B h t a for i in range(5): result.append(i ** 2) h g © t à works because list is mutable i r y à tuples are not mutable © MathByte Academy C p o à no tuple comprehension 379 so what does this (valid) expression do? (i ** 2 for i in range(5)) e t y à creates a generator object à generators are iterators c A d a y m e B h t a à next() à they calculate and hand out elements one at a time as requested M à unlike [i ** 2 for i in range(5)] © t à calculates all the elements and creates the list immediately i r y h g à generators use lazy iteration à a lazy property is one that is not calculated until it is requested © MathByte Academy C p o 380 Why use generators? à memory ef/iciency c A d a y m e à e.g. take all the rows from a /ile, and write them out, transformed to some other /ile e t y B h t a à read the entire file in memory, iterate through that and save rows à entire file in memory! M à you may not have enough memory! © t à read lines one at a time from file h g à read a row, process it, save it, discard it, request next row, … i r y à only one line in memory at any point © MathByte Academy C p o 381 Why use generators? à performance (possibly) e t y c A d a y m e à if you only need to read the /irst few elements of the iterable à why go through the computations to calculate all of them? B h t a à plus unnecessary memory usage on top of that © MathByte Academy C p o i r y h g © t M 382 What's the downside of generators? à generators are lazy iterators e t y à one-time use c A d a y m e B h t a à not good if you need to iterate through the same iterable many times M à or even just a few times if the calculations are computationally expensive or take a long time (maybe IO bound) © MathByte Academy C p o i r y h g © t 383 Creating Generators à use generator comprehension e t y c A d a y m e à use the yield keyword in functions instead of return B h t a à beyond scope of this course © MathByte Academy C p o i r y h g © t M 384 e t y © MathByte Academy C p o i r y h g c A d a y m e B h t Coding a M © t 385 Functions e t y © MathByte Academy C p o i r y h g © t M B h t a c A d a y m e 13 386 à we have used functions a lot so far print() e t y iter() next() list() math.sqrt() © t M c A d a y m e B h t a and many more… i r y h g à we can create our own, custom, functions © MathByte Academy C p o 387 Why? à easy code re-use e t y c A d a y m e à much easier to code the sqrt() function once à and then call it multiple times B h t a à breaking up complex code into easier to understand chunks à problem decomposition © MathByte Academy C p o i r y h g © t M 388 y m e à when we create a function, we may also want values to be passed into it when it is called à arguments or parameters e t y c A d a à technically not the same thing, but almost everyone uses them interchangeably à as do I L when we define a function we may define symbols for the values that will be passed to the function h g © t M when we call a function we specify values for these parameters i r y B h t a à these symbols are called parameters à these values are called arguments à so a parameter is when we de/ine the function à an argument is when we call the function © MathByte Academy C p o 389 Functions are Python Objects c A à just like everything in Python, functions are objects e t y à they have state à name (maybe!) à code à parameters à they are callable h g © t M d a y m e B h t a à and always return something when called à they can be assigned to a symbol i r y à can be passed as a parameter to another function p o à can be returned from a function call © MathByte Academy C 390 Callables à an object is callable if it can be called – using () à functions are run by calling them e t y c A d a y m e print('hello') math.sqrt(4) B h t a à but other types of objects are also callable M à not necessarily a function object my_list.copy() range(100) i r y h g © t à calling a method on the my_list object à creating a new range object à more general term is a callable © MathByte Academy C p o 391 © MathByte Academy c Custom Functions A e t y B h t a M © t h ir g y p o C d a y m e 392 à functions can be de/ined using the def keyword c A d a y m e def keyword indicates a function is being defined function's name can be any valid Python name (just like variables) def function_name(): # indented block … return © t M e t y B h t a this block is called the function body functions always return some value à function body contains any valid Python code h g à this creates a function object à the function object is associated with the symbol function_name p o i r y (in the same way a = 10 associates the integer object 10 with the symbol a) © MathByte Academy C 393 Example def say_hello(): print('Hello!') say_hello() à Hello! say_hello() à Hello! but no return? © t e t y M c A d a y m e B h t a à if a return value is not speci/ied, function will return None © MathByte Academy C p o i r y h g 394 Example def one(): return 1 result = one() © MathByte Academy C p o i r y c A d a y m e function returns the value 1 when it is called e t y B h t a à assigns the return value of calling one() to the symbol result h g © t M 395 à usually functions contain a little more complex code from datetime import datetime e t y c A d a y m e def current_time_utc(): return datetime.utcnow().isoformat() M B h t a result = current_time_utc() © t result à "2020-03-31T02:44:38.490923" © MathByte Academy C p o i r y h g 396 y m e à functions are usually more helpful when we can pass values to them len(my_iter) c A d a we are passing an argument to the len function e t y à every time we call the len function we can pass a different value B h t a à the function body (implementation) of the len function starts running à it is aware of the value that was passed to it © t à same with custom functions h g M à need to specify the parameters, by name, that will be used when we call it ri def add(a, b): return a + b © MathByte Academy o C y p add(2, 3) à 5 add(10, 1) à 11 397 def subtract(a, b): return a - b e t y subtract(10, 7) à 3 c A d a y m e à when we call subtract(10, 7), how does Python assign 10 to the symbol a, and 7 to b? à it does this by position h g © t M B h t a def my_func(a, b, c, d): … p o i r y my_func(10, 20, 30, 40) à positional arguments © MathByte Academy C 398 Namespaces à when a function is called c A d a y m e à it knows nothing about how it was called before à every time a function is called e t y B h t a à an empty dictionary is created à populated with any arguments passed in M à key = param name, value=argument à nothing else h g © t à then the function code runs i r y à this dictionary is called the (local) namespace © MathByte Academy C p o 399 d a abs_max(1, -2) def abs_max(a, b): abs_a = abs(a) abs_b = abs(b) if abs_a > abs_b: max_val = abs_a else: max_val = abs_b return max_val © t M c A y m e {'a': 1, 'b': -2} e t y {'a': 1, 'b': -2, 'abs_a': 1} {'a': 1, 'b': -2, 'abs_a': 1, 'abs_b': 2} B h t a {'a': 1, 'b': -2, 'abs_a': 1, 'abs_b': 2, 'max_val': 2} à after function return, dictionary is wiped out i r y h g à consecutive calls to the same function are independent of each other © MathByte Academy C p o 400 e t y © MathByte Academy C p o i r y h g c A d a y m e B h t Coding a M © t 401 * Arguments © MathByte Academy C p o i r y h g © t M e t y c A d a y m e B h t a 402 d a à saw how to specify positional parameters in a function def average(a, b, c, d): return (a + b + c + d)/4 e t y c A y m e B h t a à but what if we wanted to specify an arbitrary number of parameters? à we'd like to call our function with different number of args average(1) h g © t M average(1, 2, 3) i r y average(1, 2, 3, 4) © MathByte Academy C p o 403 d a y m e à could write a function to use an iterable as a single argument c A def average(iterable): return sum(iterable) / len(iterable) e t y à but this makes the calling syntax a little weird average([1, 2, 3]) average() © t M B h t a à would be nicer if we had a mechanism to accept a variable number of args © MathByte Academy C p o i r y h g 404 à Python supports a special parameter type for this à uses a * prefix on a parameter name def average(*values): # return average e t y c A d a y m e B h t a à this means we can call average with any number of arguments average(1) M average(1, 2, 3, 4, 5) © t àhow do we access these values inside the function i r y h g à use the parameter name © MathByte Academy C p o à values in this case à it will be a tuple containing all the argument values 405 def average(*values): print(type(values)) print(values) average(1, 2, 3) e t y c A d a y m e à values will be a tuple B h t a à (1, 2, 3) M def average(*values): return sum(values) / len(values) i r y h g © t à we may want to do something if someone calls this function with no arguments © MathByte Academy C p o 406 à often you will see code that uses *args à the * is the important part e t y c A d a y m e à there is nothing special about the name args à as we just saw, we can use any valid name B h t a à use a meaningful name © MathByte Academy C p o i r y h g © t M à args is often too generic 407 e t y © MathByte Academy C p o i r y h g c A d a y m e B h t Coding a M © t 408 Default Values © MathByte Academy C p o i r y h g © t M e t y c A d a y m e B h t a 409 à possible to specify optional parameters c A d a y m e à means function can be called without passing in the argument à but we still have that parameter à it needs a value e t y B h t a à we can specify a default value to use if the argument is not supplied © MathByte Academy C p o i r y h g © t M 410 def func(a=1): print(a) func() c A d a y m e a default value to use if a is not supplied when function is called e t y à 1 func(10) à 10 M B h t a à once you specify a positional parameter with a default value h g © t à all positional parameters after that must specify a default value too i r y à with the exception of a starred parameter © MathByte Academy C p o 411 à how would you interpret this? def func(a=1, b): pass e t y func(10) c A d a y m e B h t a à is 10 supposed to go into a? à in which case we're short one argument M à or use default for a and assign 10 to b? © t à don't know! © MathByte Academy C p o i r y h g 412 d a y m e à so once we have default arguments we need to specify default for all parameters after it def func(a, b, c=1, d=2): … but this is still ok: ri h g func(10, 2, 3, 4) © MathByte Academy o C

python-fundamentals.pdf

Document Details

Tags

Related

Full Transcript

Upgrade to continue