Module 1 Complete Slides PDF

MODULE 1 Introduction to Linear Algebra and System of Linear Equations  OVERVIEW The Geometry and Algebra of Vectors Length, Angle and Projection Introduction to System of Linear Equations Solving Systems of Linear Equations  The Geome...

MODULE 1 Introduction to Linear Algebra and System of Linear Equations  OVERVIEW The Geometry and Algebra of Vectors Length, Angle and Projection Introduction to System of Linear Equations Solving Systems of Linear Equations  The Geometry and Algebra of Vectors A vector is a directed line segment that corresponds to a displacement from point 𝐴 to point 𝐵. → The vector from 𝐴 to 𝐵 is denoted by 𝐴𝐵. Set of all points in plane corresponds to all vectors whose initial point is origin 𝑂.  → → For example, 𝐚 = 𝑂𝐴 = [2,3], 𝐛 = 𝑂𝐵 = 3,1. The two vectors are said to be equal if and only if their corresponding components are equal. For example, 3,2 ≠ [2, 3]. The order of the components are important. A zero vector is a vector whose all the components are zero and is denoted by 𝟎. The two vectors are said to be equal if they have same length and direction. Vector Addition: If 𝐮 = [u1 , u2 ] and 𝐯 = v1 , v2 are two vectors then sum 𝐮 + 𝐯 is the vector 𝐮 + 𝐯 = u1 + v1 , u2 + v2. For example, if 𝐮 = 3, −1 and 𝐯 = [1,4] then 𝐮 + 𝐯 = 3 + 1, −1 + 4 = 4,3. Scalar Multiplication: If 𝑐 is a real number, then the vector c𝐯, given by c𝐯 = c v1 , v2 = [cv1 , cv2 ]. For example, if v = −2,4 , then 2v = 2 −2,4 = −4,8 , 1 1 𝐯 = −2,4 = −1,2 , and 2 2 −2v = −2 −2,4 = [4, −8]. Vector Subtraction: Similarly, 𝐮 − 𝐯 = 𝐮 + (−𝐯) denotes the vector subtraction. Algebraic Properties of Vector in ℝ𝒏 : Let 𝐮, 𝐯 and 𝐰 be vectors in ℝ𝑛 and let 𝑐 and 𝑑 be scalars. Then a) 𝐮 + 𝐯 = 𝐯 + 𝐮 (Commutativity) b) 𝐮 + 𝐯 + 𝐰 = 𝐮 + 𝐯 + 𝐰 (Associativity) c) 𝐮 + 𝟎 = 𝐮 d) 𝐮 + −𝐮 = 𝟎 e) c 𝐮 + 𝐯 = c𝐮 + c𝐯 (Distributivity) f) c + d 𝐮 = c𝐮 + d𝐮 (Distributivity) g) c d𝐮 = cd 𝐮 h) 1𝐮 = 𝐮  Length and Angle The vector version of length, distance and angle can all be described using the notion of dot product. The Dot Product: u1 v1 u2 v2 If 𝐮 = ⋮ and 𝐯 = ⋮ , then the dot product 𝐮 ∙ 𝐯 of 𝐮 and 𝐯 is defined by, un vn 𝐮 ∙ 𝐯 = u1 v1 + u2 v2 + ⋯ + un vn Example 1: 𝟏 −𝟑 Let 𝐮 = 𝟐 and 𝐯 = 𝟓 , then 𝐮 ∙ 𝐯 = 1 ∙ −3 + 2 ∙ 5 + −3 ∙ 2 = 1 −𝟑 𝟐 Properties of Dot Product: Let 𝐮, 𝐯 and 𝐰 be vectors in ℝ𝑛 and let c be a scalar. Then, a) 𝐮 ∙ 𝐯 = 𝐯. 𝐮 b) 𝐮 ∙ 𝐯 + 𝐰 = 𝐮 ∙ 𝐯 + 𝐮 ∙ 𝐰 c) 𝑐𝐮 ∙ 𝐯 = 𝑐 𝐮 ∙ 𝐯 d) 𝐮 ∙ 𝐮 ≥ 0 and 𝐮 ∙ 𝐮 = 0 if and only if 𝐮 = 0 Length (Norm) of a Vector: v1 v2 The length of a vector 𝐯 = ⋮ in ℝ𝑛 is the nonnegative scalar 𝐯 defined by vn 𝐯 = 𝐯∙𝐯 = v12 + v22 + ⋯ + vn2. Example 2: [2,3] = 22 + 32 = 13 Theorem: Let 𝐯 be a vector in ℝ𝑛 and let c be a scalar. Then, a) 𝐯 = 𝟎 if and only if 𝐯 = 0. b) c𝐯 = c 𝐯. Unit Vector: A vector of length 1 is called unit vector. Example : 1 0 In ℝ2 , let 𝑒1 = and 𝑒2 =. Then 𝑒1 and 𝑒2 are unit vectors. 0 1 1 0 0 3 In ℝ , let 𝑒1 = 0 , 𝑒2 = 1 and 𝑒3 = 0 Then 𝑒1 , 𝑒2 and 𝑒3 are unit 0 0 1 vectors. Given a nonzero vector 𝐯, we can always find a unit vector in the same direction as 𝐯 by dividing 𝐯 by its own length. 1 𝟏 1 If 𝐮 = 𝐯 , then, 𝐮 = v = 𝐯 =1 𝐯 𝐯 𝐯 Important Inequalities: For all vectors 𝒖 and 𝒗 in ℝ𝑛 , a) 𝐮 + 𝐯 ≤ 𝐮 + 𝐯 (Triangle Inequality) b) 𝐮 ∙ 𝐯 ≤ 𝐮 𝐯. (Cauchy-Schwarz Inequality) Distance between Two Vectors: The distance 𝑑(𝐮, 𝐯) between vectors 𝐮 and 𝐯 in ℝ𝑛 is defined by 𝑑 𝐮, 𝐯 = 𝐮 − 𝐯. Example 3: 2 0 The distance between 𝐮 = 1 and 𝐯 = 2 is −1 −2 2 𝑑 𝐮, 𝐯 = 𝐮 − 𝐯 = −1 =2 1 Angle between Two Vectors: For nonzero vectors 𝐮 and 𝐯 in ℝ𝑛 , 𝐮∙𝐯 cos 𝜃 = 𝐮 𝐯 , where 𝜃 is the angle between 𝐮 and 𝐯. Example 4: 2 1 Find the angle between 𝐮 and 𝐯 where 𝐮 = 1 and 𝐯 = 1 −2 1 Caculate 𝐮. 𝐯 = 2 ∙ 1 + 1 ∙ 1 + −2 ∙ 1 = 1, 𝐮 = 22 + 12 + −2 2 = 3 and 𝐯 = 12 + 12 + 12 = 3. Therefore cos 𝜃 = 1/3 3 , so 1 𝜃 = cos −1 = 1.377 radian 3 3 Orthogonal Vectors: Two vectors 𝐮 and 𝐯 in ℝ𝑛 are said to be orthogonal to each other if 𝐮 ∙ 𝐯 = 0 For all vectors 𝐮 and 𝐯 in ℝ𝑛 , 𝐮 + 𝐯 2 = 𝐮 2 + 𝐯 2 if and only if u and v are orthogonal. Projection: If 𝐮 and 𝐯 are vectors in ℝ𝑛 and 𝐮 ≠ 𝟎, then the projection of 𝐯 onto 𝐮 is the 𝐩𝐫𝐨𝐣𝐮 (𝐯) defined by 𝐮. 𝐯 𝐩𝐫𝐨𝐣𝐮 𝐯 = 𝐮 𝐮. 𝐮 −1 2 Example: Find the projection of 𝐯 onto 𝐮, where, 𝐯 = and 𝐮 =. 3 1 Sol: −1 2 2 2 We compute 𝐮. 𝐯 =. = 1 and 𝐮. 𝐮 =. = 5, so 3 1 1 1 𝐯.𝐯 1 2 2/5 proj𝐮 𝐯 = 𝐮= =. 𝐮.𝐮 5 1 1/5  Linear Equations and System of Linear Equations Linear Equation: A linear equation in the 𝑛 variables x1 , x2 … , xn is an equation that can be written in the form a1 x1 + a2 x2 + ⋯ + an xn = b where the coefficients a1 , … , an and the constant term b are constants. The following equations are linear equations, 1 𝜋 3𝑥 − 4𝑦 = −1, 2𝑥 + 𝑦 − sin 𝑧=1. 4 4 The following are not linear equations 𝑥𝑦 + 2𝑧 = 1, sin 𝑥 − 3𝑦 + 2𝑧 = 0 Solution of a Linear Equation: A solution of a linear equation a1 x1 + a2 x2 + ⋯ + an xn = b is a vector [s1 , … , sn ] whose components satisfy the equation when we substitute x1 = s1 , … , xn = sn. For example, 3,0,0 , 0,1,2 , and [6,1, −1] are the solution to 𝑥 − 𝑦 + 2𝑧 = 3 System of a Linear Equation: A system of linear equation is finite set of linear equation with same set of variables. Example: 2𝑥 − 𝑦 = 3 𝑥 + 3𝑦 = 5 forms a system of linear equation. Solution of a System of a Linear Equation: A solution of the system of linear equation is a vector that is simultaneously the solution of each equation in the system. Consistent System of Linear Equation: A system of equation with at least one solution is called consistent. Inconsistent System of Linear Equation: A system of equation with no solution is called inconsistent. A system of linear equations has either 1. A unique solution 2. Infinitely many solution 3. No solution. Example: 𝑥−𝑦= 2 2𝑥 − 2𝑦 = 4 𝑥−𝑦 = 1 𝑥+𝑦 = 3 Unique solution Infinitely many solutions 𝑥−𝑦= 1 𝑥−𝑦= 3 No solution There are two important matrices associated with a linear system: 1. Coefficient Matrix 2. Augmented Matrix For example, consider the following system, 2𝑥 + 𝑦 − 𝑧 = 3 𝑥 + 5𝑧 = 1 −𝑥 + 3𝑦 − 2𝑧 = 0 2 1 −1 | 3 2 1 −1 1 0 5 | 1 1 0 5 −1 3 −2 −1 3 −2 | 0 Coefficient Matrix Augmented Matrix  Direct Methods for Solving Linear Systems Row Echelon Form: A matrix is in row echelon form if it satisfies the following properties: 1. Any row consisting entirely of zero is at bottom. 2. In each nonzero row, the first nonzero entry (called the leading entry) is in a column to the left of any leading entries below it. 0 2 0 1 −1 3 2 4 1 0 0 −1 1 2 2 Example: 0 −1 2 and are row echelon form. 0 0 0 0 4 0 0 0 0 0 0 0 0 0 5 Elementary Row Operations: The procedure by which any matrix can be reduced to a matrix in echelon form is known as elementary row operations. The following row operations can be performed on a matrix 1. Interchange two rows. 2. Multiply a row by a nonzero constant. 3. Add a multiple of a row to another one. Remark: 1. The row echelon form of a matrix is not unique. 2. The leading entry in each row is used to create the zeros below it. 3. The pivots are not necessarily the entries that are originally in the positions eventually occupied by the leading entries. 4. Once we have pivoted and introduced zeros below the leading entry in a column, that column does not change. 5. Elementary row operations are reversible.  1 2 −4 −4 5 2 4 0 0 2 Example: Reducing to echelon form. 2 3 2 1 5 −1 1 3 6 5 1. Work column by column, from left to right and from top to bottom. 2. The strategy is to create a leading entry in a column and then use it to create zeros below it. 3. The entry chosen to become a leading entry is called a pivot, and this phase of the process is called pivoting. 𝑅2 −2𝑅1 1 2 −4 −4 5 𝑅3 −2𝑅1 1 2 −4 −4 5 1 2 −4 −4 5 2 4 0 0 2 𝑅4 +𝑅1 0 0 8 8 −8 𝑅2 ↔𝑅3 0 −1 10 9 −5 𝑅4 +3𝑅2 → → → 2 3 2 1 5 0 −1 10 9 −5 0 0 8 8 −8 −1 1 3 6 5 0 3 −1 2 10 0 3 −1 2 10 1 2 −4 −4 5 1 1 2 −4 −4 5 1 2 −4 −4 5 𝑅3 0 −1 10 9 −5 8 0 −1 10 9 −5 𝑅4 −29𝑅3 0 −1 10 9 −5 → → 0 0 8 8 −8 0 0 1 1 −1 0 0 1 1 −1 0 0 29 29 −5 0 0 29 29 −5 0 0 0 0 24 (Row echelon Form)  Row Equivalence of Two Matrices: Matrices A and B are row equivalent if there is sequence of elementary row operations that converts A to B. Matrices A and B are row equivalent if and only if they can be reduced to the same echelon form. 1 2 −4 −4 5 1 2 −4 −4 5 2 4 0 0 2 0 −1 10 9 −5 Example 9: and are row equivalent. 2 3 2 1 5 0 0 1 1 −1 −1 1 3 6 5 0 0 0 0 24  Solution of System of Linear Equations by Gaussian Elimination Gaussian elimination: The entire process of row reduction applied to the augmented matrix of a system of linear equations to create an equivalent system that can be solved by back substitution is known as Gaussian elimination. Gaussian Elimination Procedure: Step 1. Write the augmented matrix of the system of linear equations. Step 2. Use elementary row operations to reduce the augmented matrix to row echelon form. Step 3. Using back substitution, solve the equivalent system that corresponds to the row-reduced matrix.  2𝑥 + 3𝑧 = 8 Example A: Solve the system 2𝑥 + 3𝑦 + 𝑧 = 5 using Gaussian elimination procedure. 𝑥 − 𝑦 − 2𝑧 = −5 The augmented matrix for the above system is, 0 2 3 | 8 𝑅1 ↔𝑅3 1 −1 −2 | −5 1 −1 −2 | −5 1𝑅2 𝑅2 −2𝑅1 5 2 3 1 | 5 2 3 1 | 5 → 0 5 5 | 15 → 1 −1 −2 | −5 0 2 3 | 8 0 2 3 | 8 1 −1 −2 | −5 1 −1 −2 | −5 𝑅3 −2𝑅2 0 1 1 | 3 → 0 1 1 | 3 0 2 3 | 8 0 0 1 | 2 (Associated system) 𝑥 − 𝑦 − 2𝑧 = −5 𝑦+𝑧 =3 𝑧=2 The above equation can now be solved using back substitution, therefore, 𝑥 = 0, 𝑦 = 1 and 𝑧 = 2 is the solution of the above system of equation. Lets look at another example 𝑤 − 𝑥 − 𝑦 + 2𝑧 = 1 Example B: Solve the system 2𝑤 − 2𝑥 − 𝑦 + 3𝑧 = 3. −𝑤 + 𝑥 − 𝑦 = −3 The augmented matrix for the above system is 1 −1 −1 2 | 1 𝑅2 −2𝑅1 1 −1 −1 2 | 1 𝑅3 +𝑅1 𝑅3 +2𝑅2 2 2 −1 3 | 3 → 0 0 1 −1 | 1 → −1 1 −1 0 | −3 0 0 −2 2 | −2 1 −1 −1 2 | 1 0 0 1 −1 | 1 0 0 0 0 | 0 The associated system 𝑤 − 𝑥 − 𝑦 + 2𝑧 = 1 𝑦−𝑧 =1 Taking 𝑥 = 𝑠 and 𝑧 = 𝑡, we have  𝑤 2+𝑠−𝑡 𝑥 𝑠 = for all 𝑠, 𝑡 in ℝ. Hence, the system has 𝑦 1+𝑡 𝑧 𝑡 infinitely many solutions. Note: In the above example, 𝑤, 𝑦 are known as leading variable and 𝑥, 𝑧 are free variables. Rank of a Matrix: The rank of a matrix is the number of nonzero rows in its row echelon form. Let 𝐴 be the coefficient matrix of the system of linear equations with 𝑛 variables. If the system is consistent, then number of free variables := 𝑛 −rank(𝐴) For example, in Example A we have 0 free variable, whereas, in Example B we have 2 free variables  Example C: Consider the system of equation 𝑥 − 𝑦 + 2𝑧 = 3 𝑥 + 2𝑦 − 𝑧 = −3. 2𝑦 − 2𝑧 = 1 The row reduce of the augmented matrix is, 1 −1 2 | 3 1 −1 2 | 3 1𝑅2 1 −1 2 | 3 𝑅2 −𝑅1 3 𝑅3 −2𝑅2 1 2 −1 | −3 → 0 3 −3 | −6 → 0 1 −1 | −2 → 0 2 −2 | 1 0 2 −2 | 1 0 2 −2 | 1 1 −1 2 | 3 0 1 −1 | −2. 0 0 0 | 5 Solving the associated system by back substitution leads to 0 = 5 (contradiction !). Therefore, the above system has no solution. A matrix is in reduced row echelon form if it satisfies the following properties: 1. It is in row echelon form. 2. The leading entry in each nonzero row is a 1 (called a leading 1). 3. Each column containing a leading 1 has zeros everywhere else. For example, the following matrix is in reduced row echelon form : 1 2 0 0 −3 1 0 0 0 1 0 4 −1 0 0 0 0 1 3 −2 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 Gaussian-Jordan Elimination: 1. Write the augmented matrix of the system of linear equations. 2. Use elementary row operations to reduce the augmented matrix to reduced row echelon form. 3. If the resulting system is consistent, solve for the leading variables in terms of any remaining free variables Homogeneous System of Linear Equations: A system of linear equations is called homogeneous if the constant term in each equation is zero. 2𝑥 + 3𝑦 − 𝑧 = 0 For example, is a homogeneous system of equation. −𝑥 + 5𝑦 + 2𝑧 = 0 A homogeneous system either has a unique solution or infinitely many solutions. If [𝐴|𝟎] is a homogeneous system of m linear equations with n variables, where m < n, then the system has infinitely many solutions.  Spanning Sets and Linear Independence Theorem: A system of linear equations with augmented matrix [𝐴 | 𝒃] is consistent if and only if 𝒃 is a linear combination of the columns of 𝐴. Example 1: The system 𝑥−𝑦 =1 𝑥+𝑦 =3 1 −1 1 has the unique solution 𝑥 = 2, 𝑦 = 1. Thus, 2 + =. 1 1 3 Example 2: The system 𝑥−𝑦 =2 2𝑥 − 2𝑦 = 4 1 −1 2 has the unique solution 𝑥 = 2 + 𝑡, 𝑦 = 𝑡. Thus, (2 + 𝑡) +𝑡 =. for all values of t. 2 −2 4 1 −1 2 Geometrically, the vectors , and are all parallel and so all lie along the same line through the origin. 2 −2 4 Example 3: The system 𝑥 − 𝑦 = 1 𝑥−𝑦 =3 has no solutions, so there are no values of x and y that satisfy 1 −1 1 𝑥 +𝑦 =. 1 −1 3 1 −1 1 In this case, and are parallel, but does not lie along the same line through the origin. 1 −1 3 Spanning Set: If 𝑆 = {𝒗𝟏 , 𝒗𝟐 , … , 𝒗𝒌 } is a set of vectors in ℝ𝑛 , then the set of all linear combinations of 𝒗𝟏 , 𝒗𝟐 , … , 𝒗𝒌 is called the span of 𝒗𝟏 , 𝒗𝟐 , … , 𝒗𝒌 and is denoted by span(𝒗𝟏 , 𝒗𝟐 , … , 𝒗𝒌 ) or span(S). If span(S)=ℝ𝑛 , then S is called a spanning set for ℝ𝑛. 2 1 Example: Show that ℝ2 = span( , ). −1 3 𝑎 1 Solution: We need to show that an arbitrary vector can be written as a linear combination of 𝑏 3 1 and , i.e., we must show that the equation 3 2 1 𝑎 x +𝑦 = −1 3 𝑏 can always be solved for 𝑥 and 𝑦 (in terms of 𝑎 and 𝑏), regardless of the values of a and b. 3𝑎−𝑏 2 𝑎+2𝑏 1 𝑎 + =. 7 −1 7 3 𝑏 𝑥 3 Example: Let 𝑒1 , 𝑒2 and 𝑒3 be the standard unit vectors in ℝ. Then for any vector 𝑦 , we have 𝑧 𝑥 1 0 0 𝑦 = 𝑥 0 +y 1 +z 0 = x𝑒1 + y𝑒2 + z𝑒3. 𝑧 0 0 1 Thus, ℝ3 = span(𝑒1 , 𝑒2 , 𝑒3 ). In general, ℝ𝑛 = span(𝑒1 , 𝑒2 , … , 𝑒𝑛 ). Linear Dependent and Independent Sets: A set of vectors 𝒗𝟏 , 𝒗𝟐 , … , 𝒗𝒌 is linearly dependent if there are scalars 𝑐1 , 𝑐2 ,…, 𝑐𝑘 , at least one of which is not zero, such that 𝑐1 𝒗𝟏 + 𝑐2 𝒗𝟐 + ⋯ + 𝑐𝑘 𝒗𝒌 = 0. A set of vectors that is not linearly dependent is called linearly independent. Theorem: Vectors 𝒗𝟏 , 𝒗𝟐 , … , 𝒗𝒎 in ℝ𝑛 are linearly dependent if and only if at least one of the vectors can be expressed as a linear combination of the others. Proof: If one of the vectors—say, 𝒗1 —is a linear combination of the others, then there are scalars 𝑐2 ,…, 𝑐𝑘 such that 𝒗𝟏 = 𝑐2 𝒗𝟐 + ⋯ + 𝑐𝑚 𝒗𝒎. Rearranging, we obtain 𝒗𝟏 − 𝑐2 𝒗𝟐 − ⋯ − 𝑐𝑚 𝒗𝒎 = 𝟎 which implies that 𝒗𝟏 , 𝒗𝟐 , … , 𝒗𝒎 are linearly dependent, since at least one of the scalars (namely, the coefficient 1 of 𝒗𝟏 ) is nonzero. Conversely, suppose that 𝒗𝟏 , 𝒗𝟐 , … , 𝒗𝒎 are linearly dependent. Then there are scalars 𝑐1 , 𝑐2 ,…, 𝑐𝑚 , not all zero, such that 𝑐1 𝒗𝟏 + 𝑐2 𝒗𝟐 + ⋯ + 𝑐1 𝒗𝒎 = 0. Suppose 𝑐1 ≠ 0. Then 𝑐 𝑐 𝑐1 𝒗𝟏 = −𝑐2 𝒗𝟐 − ⋯ − 𝑐1 𝒗𝒎 , ⟹ 𝒗𝟏 = − 2 𝒗𝟐 − ⋯ − 𝑚 𝒗𝒎. 𝑐1 𝑐1 Example: Any set of vectors containing the zero vector is linearly dependent. For if 𝟎, 𝒗𝟐 , … , 𝒗𝒎 are in ℝ𝑛 , then we can find a nontrivial combination of the form 𝑐1 𝟎 + 𝑐2 𝒗𝟐 + ⋯ + 𝑐𝑚 𝒗𝒎 = 𝟎, by setting 𝑐1 = 1 and 𝑐2 = 𝑐3 =…=𝑐𝑚 =0. Theorem: Let 𝒗𝟏 , 𝒗𝟐 , … , 𝒗𝒎 be (column) vectors in ℝ𝑛 and let 𝐴 be the 𝑛 × 𝑚 matrix [𝒗𝟏 𝒗𝟐 … 𝒗𝒎 ] with these vectors as its columns. Then 𝒗𝟏 , 𝒗𝟐 , … , 𝒗𝒎 are linearly dependent if and only if the homogeneous linear system with augmented matrix [𝐴 | 𝟎] has a nontrivial solution. Example: The standard unit vectors 𝑒1 , 𝑒2 , and 𝑒3 are linearly independent in ℝ3 , since the system with augmented matrix [𝑒1 𝑒2 𝑒3 | 𝟎] is already in the reduced row echelon form 1 0 0 | 0 0 1 0 | 0 0 0 1 | 0 and so clearly has only the trivial solution. In general, we see that 𝑒1 , 𝑒2 , … , 𝑒𝑛 will be linearly independent in ℝ𝑛. Theorem: Let 𝒗𝟏 , 𝒗𝟐 , … , 𝒗𝒎 be (row) vectors in ℝ𝑛 and let 𝐴 be the 𝑛 × 𝑚 matrix with these vectors as its rows. Then 𝒗𝟏 , 𝒗𝟐 , … , 𝒗𝒎 are linearly dependent if and only if 𝑟𝑎𝑛𝑘 𝐴 < 𝑚. Theorem: Any set of m vectors in ℝ𝑛 is linearly dependent if 𝑚 > 𝑛.