CS-573 Optimization Methods Lecture 3 PDF

CS-573 Optimization Methods Lecture 3: Systems of linear equations Fall 2024 CS-573 Optimization Methods 1 Superposition and linear functions Fall 2024 CS-573 Optimization Methods 2 The inner product function Fall 2024 CS-573 Optimizati...

CS-573 Optimization Methods Lecture 3: Systems of linear equations Fall 2024 CS-573 Optimization Methods 1 Superposition and linear functions Fall 2024 CS-573 Optimization Methods 2 The inner product function Fall 2024 CS-573 Optimization Methods 3 Linear functions are inner products Fall 2024 CS-573 Optimization Methods 4 Affine functions Fall 2024 CS-573 Optimization Methods 5 Linear vs Affine functions Fall 2024 CS-573 Optimization Methods 6 Mathematical structures Fall 2024 CS-573 Optimization Methods 7  Matrix A matrix is an array of numbers with size 𝑚 ↓ by 𝑛 →, i.e. m rows and n columns. If , we say that is square. Fall 2024 CS-573 Optimization Methods 8 Examples of data matrices Fall 2024 CS-573 Optimization Methods 9  Basic Matrix Operations Addition Can only add a matrix with matching dimensions, or a scalar. Scaling Fall 2024 CS-573 Optimization Methods 10 Matrix Operations Matrix Multiplication Properties distributive associative Powers We can refer to the matrix product AA as A2, and AAA as A3, etc. Only square matrices can be multiplied that way Transpose Fall 2024 CS-573 Optimization Methods 11 Matrix Operations Trace Invariant to a lot of transformations Properties: 𝑡𝑟𝑎𝑐𝑒 𝐴 = 𝑡𝑟𝑎𝑐𝑒 𝑃−1 𝐴𝑃 , 𝑤ℎ𝑒𝑟𝑒 𝑃 𝑖𝑠 𝑖𝑛𝑣𝑒𝑟𝑡𝑖𝑏𝑙𝑒 Fall 2024 CS-573 Optimization Methods 12 Special matrices Identity matrix I Diagonal matrix Symmetric matrix Skew-symmetric matrix Fall 2024 CS-573 Optimization Methods 13 Matrix-vector product function Fall 2024 CS-573 Optimization Methods 14 Example Fall 2024 CS-573 Optimization Methods 15 Applications A y Price Quality Usefulness Durability x 0.3 Price 0.3 Quality 0.2 Usefulness 0.2 Durability Fall 2024 CS-573 Optimization Methods 16 Matrix-vector product function Fall 2024 CS-573 Optimization Methods 17 Examples Fall 2024 CS-573 Optimization Methods 18 Range, rank Fall 2024 CS-573 Optimization Methods 19 Nullspace Fall 2024 CS-573 Optimization Methods 20 Matrix Operations: Determinant Determinant returns a scalar Represents area of the parallelogram described by the vectors in the rows of the matrix For Properties: Fall 2024 CS-573 Optimization Methods 21 Determinants Fall 2024 CS-573 Optimization Methods 22 Determinants Fall 2024 CS-573 Optimization Methods 23 Determinants Fall 2024 CS-573 Optimization Methods 24 Matrices as linear maps Fall 2024 CS-573 Optimization Methods 25 Matrices as linear maps Fall 2024 CS-573 Optimization Methods 26 Inverse problems Formulation or more generic Objective, given 𝑦 and knowledge about 𝑨, recover 𝑥 Fall 2024 CS-573 Optimization Methods 27 Types of inverse problems Deblurring/deconvolution MRI reconstruction Source localization Astrophysics Climate modeling Fall 2024 CS-573 Optimization Methods 28 Types of inverse problems Fall 2024 CS-573 Optimization Methods 29 Types of inverse problems Fall 2024 CS-573 Optimization Methods 30 Types of inverse problems Fall 2024 CS-573 Optimization Methods 31 Matrix Inverse Fall 2024 CS-573 Optimization Methods 32 System of linear equations Fall 2024 CS-573 Optimization Methods 33