Python Modules, Namespaces and Classes

Modules and Namespaces

• dir() returns a list of an object's attributes and methods as strings, sorted lexicographically
• import module makes the module name show up
• from module import method makes the method name show up but not the module name
• Importing within a local namespace results in the imported method showing up in locals() of that method, not in globals()
• Importing twice makes the module name appear once
• Importing with from makes both the module and method names show up
• import module as name accesses the module name as "name"
• __builtins__ is a dictionary representing a module's attributes
• __name__ accesses an object's name but cannot directly change it
• globals() is a dictionary storing the current module's attributes or the global namespace
• locals() is a dictionary storing the current local namespace, depending on where the function is called

Classes and Objects

• Methods are also attributes
• Methods do not show up on a __dict__ call to an object of the class, but variable attributes and their values do
• Methods are associated with the class, not each object
• __dict__ on a class is a mappingproxy, not a dictionary
• mappingproxy can access values like a dictionary but can't write into it or change/add attributes
• Differences between a class and an object become clear when inspecting __dict__
• Variable attributes store their values in a dictionary, allowing modification of values
• __module__ is the name of the module where the class was defined; it displays __main__ when defined in the shell
• __doc__ stores the class's docstring or None if undefined
• Class attributes are defined in the class scope and appear in the dict for the class but not its objects, including class variables and methods
• Object attributes are values stored within objects
• Order of accessing an attribute of an object:
  • Checks if the attribute is defined within the object (object attribute)
  • Checks if the attribute is defined within the class (class attribute)
  • Raises an AttributeError
• Order of accessing an attribute of a class:
  • Checks if the class has the attribute
  • Raises an AttributeError
• "self" can be overridden with decorators like @staticmethod and @classmethod
  • staticmethod methods do not require an instance to be called and have no self parameter
  • classmethod methods pass the class as the first parameter (cls instead of self)
    • Factory methods create objects of a type
    • Useful for changing class attributes

Functions and Parameters

• Keyword arguments must be specified after positional arguments
  • Once a keyword argument is declared, all subsequent arguments must also be keyword arguments
  • Arguments cannot be specified more than once
• Unpacking data:
  • *args unpacks iterables like tuples and lists
  • **kwargs unpacks keyword arguments (dictionary unpacking)
• *args and **kwargs accept any arguments
• Default arguments assign a default value to an argument if it's not specified, and can be defined using keyword or positional arguments
  • Once a default argument is specified, all following arguments must also have defaults
  • Using mutable values as default parameters can be misleading
• Functions accepting any number of positional arguments use *args
  • It's illegal to define additional tuple packing parameters
  • Named arguments after * must be keyword arguments
• Dictionary packing parameters denoted by **kwargs
  • There can only be one, and it must be the last one
• Dictionary packing parameters accepts any number of keyword arguments, and collects them into a dictionary

Context Managers

• Context managers automatically close files, socket connections, and HTTP connections upon exiting
• contextlib.redirect_stdout(io.StringIO()) as output is a context manager that redirects print statements, retreiving the printed output with output.getvalue()
• Used for raising exceptions in unit tests
• Context manager as a class involves with context_manager as context
  • Has __enter__ and __exit__ dunder methods
  • __enter__(self) executes with with and returns the object to be used as context, often returning self
  • __exit__(self, exc_type, exc_value, exc_traceback) executes when an exception is raised or a block ends
    • exc_type, exc_value, and exc_traceback are None if exited normally
    • exc_type is the type of exception
    • exc_value is the error message
    • exc_traceback is the traceback information
  • Returning True suppresses the exception from being raised outside the block
  • If nothing is returned, the exception is re-raised outside of the block
  • __init__ runs before __enter__

Asymptotic Analysis

• Asymptotic analysis describes the behavior of functions as n approaches infinity
• f(n) ≤ c * g(n) is used to determine if a function f(n) is O(g(n))
• Closest fit Big O notation:
  • Does not have constant coefficients
  • Does not have lower order terms
  • Is the slowest growing
• Measuring memory:
  • Constant memory: O(1)
  • Linear memory: O(n)
  • O(1) + O(n) = O(n)

Searching

• Costs of a list include:
  • A reference to a block of memory that stores references to each of the list's elements
  • An integer indicating how many elements are stored in the list (the list's size)
  • An integer indicating how many references can be stored in the block of memory belonging to the list (the list's capacity)
• Sequential/linear search: looking through a list one by one from index 0 onward
  • Asymptotic analysis of memory: O(1)
    • given list + given search key + current index = O(1)
  • Asymptotic analysis of time: O(n)
    • determining starting index + adding 1 to current index n times + accessing an element from the list at the given index = O(1) + n * (O(1) * O(1)) = O(1) + O(n) = O(n)
• Binary search sorts through a list in ascending order, then checks if the element in the middle of the section of interest is higher or lower than the search key
  • Asymptotic analysis of time: O(log n)
    • Only need to look at log 2 n items with a list of size n
  • Asymptotic analysis of memory: O(1)
    • Only keeps track of first, last, and middle indexes = O(1) * 3 = O(1)
  • Binary search is fast but not always the best option for unsorted lists or lists that need to be changed often
• Logarithms use asymptotic analysis
  • The base is ignored
  • Logarithmic algorithms are much faster/better than linear algorithms (O(n)) in terms of time

Databases

• Database schema defines tables and their fields
• Primary key: column where all of the rows are unique
• Relationships: has a cardinality (how many elements can have a relationship with each other)
  • One-to-one: one element of A can only be related to one element of B, and vice versa
  • One-to-many: one element of A can be related to many elements of B, but each element of B can only be related to one element of A
  • Many-to-many: each element of A can be related to many elements of B, and vice versa
• Foreign key: stores a row's primary key in another row
• Connecting to SQLite database:
  • In-memory databases utilize sqlite3.connect(':memory:')
  • File-based databases use a file path instead of :memory:
• Creating a table: CREATE TABLE() SQL command
• execute() returns a cursor to access data returned by SQLite
• cursor.fetchone() returns one row of the result and is an iterator
• SQLite schema keywords:
  • Data types:
    • INTEGER: signed integer
    • TEXT: string
    • REAL: floating point
  • PRIMARY KEY: makes a column’s rows unique
  • FOREIGN KEY: reference to the primary key from another table
  • NULL/NOT NULL: like None in Python, applicable when rows/columns haven't been defined
    • Use IS NULL or IS NOT NULL instead of = NULL
    • NOT NULL restricts NULL values from specified columns
  • UNIQUE
  • CHECK <condition>
  • STRICT: enforces data types defined after creating the table
• INSERT INTO is used to add a new row to a table with specified values
• Query types:
  • SELECT <columns> FROM <table> WHERE <condition>
    • LIMIT is used to limit the number of results
    • ORDER BY (must go after WHERE)
  • INSERT INTO <table> (<columns>) VALUE (<values>)
  • UPDATE <table> SET <column> = <value> WHERE <condition>
  • DELETE FROM <table> WHERE <condition>
  • DROP TABLE <table name>
  • INNER JOIN <table> AS <nickname2> ON <nickname2>.<column> = <nickname1>.<column>
    • Joins two tables based on a condition
    • Won’t join if both columns are NULL
• WHERE conditions: <column> <operator> <value>
  • Operators: =, != or <>, <, >, <=, and >=
  • OR, and AND
  • IS NULL, and IS NOT NULL
• NULL = NULL in SQL is False
• Parameterizing a SQLite statement prevents SQL injection attacks
  • Positional parameters: use question marks as placeholders with a tuple containing the values
  • Keyword parameters: use named placeholders with a dictionary containing the values
• Transactions: making all changes visible to other connections with connection.commit(), called if connection.execute() doesn't fail

Comprehensions

• List comprehension: [action for x in iterable if condition] with O(n) complexity
  • Nested for loops: the first for loop is the outermost loop
  • Nested if statements: the first if statement is processed first
• Set comprehension: {action for x in iterable if condition} (similar to list comprehension but creates a set)
  • Set comprehensions act like sets and get rid of duplicates, only allowing immutable elements
• Dictionary comprehension: {x: action for x in iterable if condition}
  • Keys of a dictionary must be hashable
• Tuple comprehension: tuple(action for x in iterable if condition)
  • Use tuple() for a tuple comprehension
  • Applying comprehension directly to a tuple expression returns a generator; use tuple() to get the resulting tuple
• Lists are mutable, affecting comprehensions versus simply multiplying lists
• Hashable objects must have __hash__ and __eq__ methods defined
  • __hash__(self) determines an object's hash, where immutable objects must have the same hash
  • Hashing provides a unique integer identifier for an immutable object.

Iteration

• Iterable: __iter__
  • Objects that can be iterated can produce a sequence of elements one at a time
  • An object is iterable if it has either __iter__ or both __getitem__ and __len__
• Iterator: __iter__, __next__
  • An object that manages the process of producing an iterable's sequence of elements once
• __iter__(self) returns an iterator, usually either self or another class that also has a __iter__ method
• __next__(self) returns the next element from the iterator or raises a StopIteration exception if there are no more elements
• iter(iterable) creates an iterator for an iterable object
• next(i) asks an iterator for the next element, indicating that the element hasn't been seen yet and providing a means to signal that all elements have been traversed
• StopIteration is raised when all elements have been iterated and next() is called
• Mutating a list during iteration with an iterator
  • A list iterator stores a reference to the list and the index of the next element to be returned (O(n) time, O(1) memory)
  • Changing an element at the next index modifies it as iteration progresses
• __repr__(self) returns the representation of an object in the shell

Generators

• Generators have a __next__ method
• Generators yield
  • Which returns a generator object
  • Functions as an iterable, allowing next() calls
  • Can loop through for x in generator_func() and list(generator_func())
  • Often inside while or for loops
  • Runs until it hits yield: the next time next() is called, execution resumes after yield
• Calculating and yielding values one at a time is often preferred to computing and returning a whole list
• Generators are lazily evaluated (one at a time), working well with large input sizes.
• return will raise StopIteration, not be returned
  • Whatever is returned becomes the error message
• Stops when exiting the generator function, return, or exception raised
• Generator comprehension: (action for x in iterable)
• Generator functions can yield sequentially from another iterable using the yield from keyword

Data Model

• __len__(self): returns an integer specifying the length of self

• __bool__(self): returns a boolean specifying an object's truthiness

  • Truthiness is checked by determining the existence of the __bool__ method, then the existence of the __len__ method, and by returning True in all other cases unless their length is zero or they are None

• __getitem__(self, index): returns a value associated with the specified index and decides what constitutes a valid index

  • The sequence protocol dictates that using together __getitem__ + __len__ will makes a class iterable
    • Enabling reverse iteration
    • Slicing uses a slice(start, stop, step) object as a parameter
  • Access the indices with index.indices(len(self))

• __setitem__(self, index, value): assigns the specified value to the specified index

• __delitem__(self, index): deletes the value at the specified index

• __hash__: hashes

• __reversed__(self): returns a reverse iterator, intended for immutable objects to be hashed based on their memory addresses

• __contains__(self, value): returns True if self contains some value

• == vs is, == tests for equivalent meaning/value, while is tests that the object is at the same memory location

• Comparing lists: lexicographic ordering

  • Compare the first value, then the second, and so on until a difference is found

• Comparison operators:

  • __eq__(self, other): returns True if self is equivalent (==) to other, or False if not equivalent, or NotImplemented otherwise
    • Equivalent objects must have the same hash (not necessarily the other way around)
  • __ne__(self, other): returns True if self is not equivalent (!=) to other, or False if they are equivalent, or NotImplemented if inequality is not supported for the types of the arguments
  • __lt__(self, other): returns True if self is less than other, or False otherwise
  • __gt__(self, other): returns True if self is greater than other, or False otherwise
  • __le__(self, other): returns True if self is less than or equal to other, or False otherwise
  • __ge__(self, other): returns True if self is greater than or equal to other, or False otherwise
    • eq, le, and lt need to be defined to cover everything
  • These comparison operators return True, False, or NotImplemented

• __pos__(self): implements unary plus (+x)

• __neg__(self): implements unary minus (-x)

• __add__(self, other): returns the sum of self and other

  • __radd__(self, other): returns the sum of other and self when other does not support addition to self
    • Note that += modifies the object

• __sub__(self, other): returns the difference when subtracting other from self; note that -= modifies the object

• __mul__(self, other): returns the product of self and other; note that *= modifies the object

• __truediv__(self, other): returns the quotient of self / other, without taking the floor; note that /= modifies the object

• __floordiv__(self, other): returns the quotient of self / other, rounded down to an int; note that //= modifies the object

• __pow__(self, other): returns the exponent of self ^ other; note that **= modifies the object

• Reflected operators: are like other, self rather than self, other

• Augmented arithmetic: iadd, isub, imul, etc.

  • Modify self instead of returning a new object: self = self (operator) other

Inheritance

• isinstance vs type
  • isinstance(Derived(), Base) returns True
    • Because a derived object inherits from the base
  • type(Derived()) is Base) returns False
• Single inheritance (Y inherits from X)
  • Y can have an attribute that doesn't exist in X, but not the other way around
  • Y can override an attribute inherited from X
  • Y cannot remove an attribute from X
• Liskov substitution principle: objects of derived classes should be able to safely take the place of objects of base classes
• Multiple inheritance: order of base classes matter, with base1 taking precedence over base2 when they share attributes
• Method resolution order (MRO):
  • How an object will search for attributes
    • self is searched first, then the base classes in listed order, and then object
  • Linearization: if the MRO can't be decided, it's not linear
• Every class derives from the class object, and gives every class some default dunder methods
• try-except handles custom raised classes vs built-in classes
• super() allows access to a superclass method within a derived class
  • Relies on MRO and goes up it
  • Follows the initialized object's MRO

Recursion

• Recursion
  • Direct recursion: a method directly calls itself once
    • Single recursion (the once part)
  • Indirect recursion: a method calls another method that then calls the first method
• Runtime stack: holds information about each function on top of the function