13 Linear Algebra and Systems of Linear Equations.pptx

LINEAR ALGEBRA AND SYSTEMS OF LINEAR EQUATIONS SETS Definition A set is a collection of objects Denoted by braces {} Examples S = {orange, apple, banana} Empty Set Contains no objects Denoted by...

LINEAR ALGEBRA AND SYSTEMS OF LINEAR EQUATIONS SETS Definition A set is a collection of objects Denoted by braces {} Examples S = {orange, apple, banana} Empty Set Contains no objects Denoted by empty braces {} Union of Sets Denoted by A ∪ B Contains all elements of A and B Intersection of Use union() or intersection() functions Sets STANDARD SETS RELATED TO NUMBERS Standard Sets and Their Symbols Set Notation Set Symbol Symbol ∈ denotes membership in a set Natural numbers N Backslash denotes set minus Whole numbers W Standard Sets Integers Z Natural numbers Rational numbers Q Whole numbers Integers Irrational numbers Q' Rational numbers Real numbers R Irrational numbers Complex numbers C Real numbers Complex numbers TABLE OF STANDARD SETS Set Name Symbol Description N = {1, 2, 3, 4,... }W = N ∪ {0}Z = W ∪ {−1, −2, −3,... }Q = p : p ∈ Z,q ∈ Z\{0} qI is the set of real Naturals N numbers not expressible as a fraction of integers2025396204848R = Q ∪ IC = a + bi : a, b ∈ R,i = √−1 Wholes W Integers Z Rationals Q Irrationals I Reals R Complex numbers C DEFINITION AND NOTATION Definition Set Example: of Rn Notation R3 Set of all (x,y, z) n-tuples of Rn(x1, x2, Set of real coordinate real x3,..., xn) triples s in 3D numbers space VECTOR Vectors in Rn NORMS n-tuple or point in Rn Can be written as row or column vectors Column vector is the default if ambiguous Vector Elements and Transpose ith element denoted by vi Transpose of column vector is a row vector Transpose of row vector is a column vector Zero Vector Vector in Rn containing all zeros Norm of a Vector Measure of vector's length VECTOR ADDITION AND MULTIPLICATION Vector Addition Vector Multiplication Pairwise addition of vector Scalar multiplication: elements product of vector and scalar Example: u = v + w with Defined as product of each elements ui = vi + wi vector element by scalar Example: u = αv with elements ui = αvi Python implements scalar multiplication similarly DOT- Definition of Dot-Product Sum of the products of respective elements PRODUCT Denoted by · and read as AND ANGLE BETWEEN VECTORS CROSS- Written as v w Definition Defined by v w v 2 w 2 sin (θ ) n PRODUCT θ is the angle between v Angle Between and w Vectors Computed from the dot- product n is perpendicular to both Perpendicular v and w Vector n has unit length (length is one) Vector perpendicular to Geometric both v and w Length equal to the area Interpretation of the parallelogram created by v and w LINEAR COMBINATIONS Example of Linear Linear Linear Linear Combination Independence Dependence Combination Defined as Σ Grocery bill as No object in αisi Σ cini the set can be αi is any real ci is the cost of written as a number item i linear si is the ith ni is the combination of object in S number of the other items i objects purchased Only considering linear independence of a set of vectors DEFINITION AND NOTATION Definition of a Matrix Norm of a Matrix An m × n matrix is Considered as a a rectangular table particular kind of of numbers vector norm Consists of m rows Elements of M are and n columns treated as m × n MATRIX NORMS Definition of p-norm for mn- dimensional vector Formula: v m [lM[lp = p n |aij |p Matrix norm calculation Uses the same norm function in NumPy as for a vector MATRIX ADDITION AND MULTIPLICATION Matrix Addition and Scalar Multiplication Work the same way as for vectors Matrix Multiplication Defined when P is m x p and Q is p x n Resulting matrix M is m x n Inner matrix dimensions must match Matrix Transpose Reversal of rows with columns Denoted by a superscript T Python Implementation Use dot method in NumPy for matrix multiplication Use T method for transpose SQUARE MATRICES AND DETERMINANTS Definition of Square n x n matrix with equal rows and columns Matrix Importance of Special number calculated from a square matrix Determinant Denoted by det in mathematics and NumPy Determinant of a 2 × 2 Formula: |M|= ·ab¸ = ad − bc Matrix Determinant of a 3 × 3 Formula: |M|= ⎣def ⎦ = a ⎣□ef ⎦ − b ⎣d□f ⎦ + c ⎣de□⎦ Matrix Expanded Formula: aei + bfg + cdh − ceg − bdi − af h Higher-Dimensional Similar approach for calculation Matrices IDENTITY MATRIX Definition of Identity A square matrix with 1s on the diagonal and 0s Matrix elsewhere Notation Usually denoted by I Analogy to Real Similar to the real number identity, 1 Number Identity Multiplication Multiplying any matrix by I (of compatible size) Property produces the same matrix MATRIX INVERSION Definition of Inverse Inverse matrix N such that M N = I Matrix Analogous to the inverse of a real number Invertibility and Matrix is invertible if it has an inverse Uniqueness Inverse is unique for an invertible matrix Analytical solution for 2x2 matrix Calculation Methods Other methods: Gaussian elimination, Newton’s method, eigendecomposition Singular and Non- Singular matrices have no inverse singular Matrices Non-singular matrices have an inverse ILL-CONDITIONED MATRICES Definition of Ill- Matrices with determinants close to zero Conditioned Matrices Have inverses but are numerically problematic Can cause overflow or underflow Numerical Issues Significant round-off errors Measure of how ill-conditioned a matrix is Condition Number Defined as the norm of the matrix times the norm of its inverse Higher condition number indicates closer to being singular Python Computation Use NumPy’s function cond from linalg Definition of Full Rank Augmented Significance Matrix Rank Matrix Matrix of Rank Number of Occurs if Matrix A Relationshi linearly rank(A) = concatenat p to independen min(m, n) ed with solutions of t columns All columns vector y linear or rows are linearly Written as equations Denoted by independen A, y Rank(A, y) rank(A) t = rank(A) +1 indicates new information RANK OF A MATRIX DEFINITION AND PROPERTIES Definition of Linear Transformation Function F is a linear transformation if F(ax + by) = aF(x) + bF(y) Matrix Multiplication as Linear Transformation Multiplying an m x n matrix A and an n x 1 vector v is a linear transformation of v A matrix is synonymous with a linear transformation function SUBSPACE Important Subspaces of a Matrix S Two subspaces: Rn and Rm ASSOCIATE Domain of A D WITH A Subspace of Rn MATRIX Set of all vectors that can be multiplied by A on the right Range of A Subspace of Rm Set of all vectors y such that y = Ax Denoted as R(A) Set of all linear combinations of the columns in A Null Space N(A) Subset of vectors x in the domain of A such that Ax = 0m MATRIX Definition of System of Linear Equations FORM OF A A set of linear equations sharing the same variables SYSTEM Example of System of Linear Equations OF LINEAR Equations with real number coefficients and EQUATION variables Matrix Form of System of Linear Equations S Represented as Ax = y A is an m x n matrix y is a vector in Rm x is an unknown vector in Rn Matrix Multiplication Carrying out matrix multiplication returns to the original system of equations DEFINITION AND NOTATION Definition of Linear Equation A linear equation is an equality It is of the form n aixi = y TYPES OF SOLUTIONS System of Linear Equations in Matrix Form A is an m x n matrix m equations and n unknowns Solution to the System x in Rn satisfies the matrix form equation Possible Solutions for x No solution One unique solution Infinitely many solutions METHODS FOR SOLVING SYSTEMS OF LINEAR EQUATIONS Introduction to solving systems of equations Focus on systems with unique solutions Common methods for solving systems Methods you will likely encounter in your work Solving systems in Python Step-by-step guide to using Python for solving systems  Gauss GAUSS Elimination Transforms matrix A into an upper-triangular form ELIMINATIO Method Used to solve systems of equations Overview N METHOD System of Four equations and four variables Equations Matrix form representation Example Solve for x4 by dividing both sides by a41,4 Backward Substitute x4 into the third equation to solve Substitution for x3 Continue substituting to solve for all x Forward Substitution Special Cases GAUSS- Gauss-Jordan elimination solves systems of JORDAN equations ELIMINATIO Procedure to turn A into a diagonal form N METHOD Matrix form of the equations ⎡10⎤⎡ x1 ⎤ ⎡ y11 ⎤ 0x2y1 ⎥ ⎥=⎢2 ⎣00⎦⎣ x3 ⎦ ⎢⎣ y31 ⎥⎦ 0 Equations become: x1+0+0+0= y11 PROCEDUR LU Decomposition Method Overview E AND Changes matrix A only, not y EXAMPLE Ideal for solving systems with same coefficient matrices A but different constant vectors y Aims to turn A into the product of two matrices L and U Solving the System Convert system of equations to LUx = y Define Ux = M Solve for M using forward substitution Solve for x using backward substitution Obtaining L and U Matrices Use Gauss elimination method Obtain upper triangular matrix U and lower triangular matrix L TABLE OF LU DECOMPOSITION LU= 3. -5.] [[ 4. [-2. -4. 5.] [ 8. 8. 0.]] INTRODUCTI Definition of ON TO Iterative Indirect methods for solving equations Start with an initial guess and improve iteratively ITERATIVE Methods METHODS Explicit Form System of linear equations of System of Expressed in explicit form for iteration Equations Initial values assumed for variables Iteration Substitute values into equation iteratively Process Continue until change is below threshold Convergence Specific conditions required for convergence Conditions GAUSS- Iterative Uses the latest estimated Method SEIDEL value for each element in x METHOD Initial Values Assume initial values for x2, x3,..., xn (except x1) Calculatio n Process Calculate x1 using initial values, then x2 using x1, and Convergen so on ce Iterations continue until values converge Applicatio n Used to solve systems of equations SOLVING Using NumPy's Provides an easy solution for SYSTEMS solve systems of linear equations function OF LINEAR Requires input matrix to be square and of full rank EQUATION LU S IN decompositi Solver performs LU decomposition on under the hood PYTHON Results match those calculated by hand Permutation matrix P Records changes in the order of equations Helps in easier calculation by switching pivot equations DEFINITIO Definition of Matrix Inversion N AND Inverse of a square matrix M is M−1 METHODS Matrix M multiplied by its inverse M−1 equals the identity matrix I Complexity of Analytical Solution High dimension matrices have complicated analytical solutions Efficient Methods for Inversion Need for efficient ways to obtain the inverse of high dimension matrices Illustration with 4 × 4 Matrix Example matrix M with elements m1,1 to m4,4 EXAMPLE Matrix Rewrite the matrix equation as four OF MATRIX Inversio separate equations n INVERSION Process Solve the four systems of equations to find the inverse Gauss- Convert the matrix equation to an Jordan identity matrix Method Obtain the solution by converting the matrix Exampl Convert the matrix to an identity e matrix Find the inverse matrix SUMMARY Foundation of Linear Matrices Engineering Independence Tables of numbers Fields Vectors not written as Characteristics: Vectors as points in linear combinations determinant, rank, of others inverse Rn Addition and multiplication defined System of Linear Equations Represented by Solving Equations Matrix Inverse matrix equation Axy PROBLEMS Matrix Multiplication Distributes Over Addition Show that A(B + C) = AB + AC Function my_is_orthogonal(v1,v2,tol) v1 and v2 are column vectors of the same size tol is a positive scalar Output is 1 if angle between v1 and v2 is within tol of π/2 Output is 0 otherwise Test Cases for my_is_orthogonal Test with vectors a and b, and tol = 0.01 Test with vectors a and b, and tol = 0.001 Test with vectors a and b, and tol = 0.01