23MAT106 Mathematics for Intelligent Systems-I Eigenvalues & Vectors (PDF)

23MAT106 MATHEMATICS FOR INTELLIGENT SYSTEMS-I 10/28/2024 10/28/2024 Sarada Jayan Geetha K N 1 1  10/28/2024 Geetha K N 2 10/28/2024 Sarada Jayan 2  Eigenvalues and Eigenvectors 10/28/2024 10...

23MAT106 MATHEMATICS FOR INTELLIGENT SYSTEMS-I 10/28/2024 10/28/2024 Sarada Jayan Geetha K N 1 1  10/28/2024 Geetha K N 2 10/28/2024 Sarada Jayan 2  Eigenvalues and Eigenvectors 10/28/2024 10/28/2024 Sarada Jayan Geetha K N 3 3  Matrix representation of functions 10/28/2024 10/28/2024 Sarada Jayan Geetha K N 4 4  Can you find a special vector x (a specific direction) that gets scaled after multiplying by a square matrix A | | Ax =  x | | 10/28/2024 Geetha K N 5 10/28/2024 Sarada Jayan 5  | | Ax =  x | | 10/28/2024 Geetha K N 6 10/28/2024 Sarada Jayan 6  How to find eigenvalues and eigenvectors of A Homogeneous Linear system of equation nth degree polynomial equation in λ which will have n roots, i.e., n eigenvalues and for each of these n eigenvalues, solution to system ഥ 𝐴 − 𝜆𝐼 𝒙 = 𝟎 will give the solution vector x, the eigenvectors 10/28/2024 Geetha K N 7 10/28/2024 Sarada Jayan 7 10/28/2024 10/28/2024 Sarada Jayan Geetha K N 8 8  How to find the eigenvalues and eigenvectors? Example 1: 10/28/2024 10/28/2024 Sarada Jayan Geetha K N 9 9  Example 1: 10/28/2024 10/28/2024 Sarada Jayan Geetha K N 10 10 Example 2: 1 0 0 A= 0 4 0 0 0 6 10/28/2024 10/28/2024 Sarada Jayan Geetha K N 11 11 Example 3: 5 4 𝐴= 1 2 10/28/2024 10/28/2024 Sarada Jayan Geetha K N 12 12 Example 4: 4 −6 −6 𝐴 = 0 −2 0 1 −1 −1 10/28/2024 10/28/2024 Sarada Jayan Geetha K N 13 13  Example 5: Find the eigenvalues and, eigenvectors of the given matrices 10/28/2024 10/28/2024 Sarada Jayan Geetha K N 14 14 Application Let A be a 2x2 matrix Let X be a set of points on a unit circle and let x  X Let Y be a set of points such that y =Ax, y  Y In general y vector is a scaled and rotated version of x But we note that there are special x's (directions) such that Ax= x For 2  2 matrix, there are two such directions(in general, except when 1 =2 ) 5 3 Example: If 𝐴 = 3 5 10/28/2024 10/28/2024 Sarada Jayan Geetha K N 15 15  MATLAB Command eig(A) (returns eigenvalues) [evec eval] = eig(A) (returns eigenvalues and eigenvectors) Matlab provide eigen vectors with unit norm 10/28/2024 10/28/2024 Sarada Jayan Geetha K N 16 16  Find the eigen values and the corresponding eigen vectors of the following matrices and write their algebraic multiplicity 2) − 2 2 − 3 3)  1 0 0 1) − 5 2   2 − 2  2 1 − 6 2 1 0    − 1 − 2 0  3 1 2  1 1 3 1 0 1 0  0 1 4  5 1  5) 0 0 1  6)    1    − 1 0  3 1  1 − 3 3 1 2  9)  0 0 8)  7)  0 1  0 0  0 0   2 4     10/28/2024 Geetha K N 17 10/28/2024 Sarada Jayan 17  Eigenvalues Eigenvectors of Special matrices And Properties of Eigenvalues 10/28/2024 10/28/2024 Sarada Jayan Geetha K N 18 18 Eigenvalues and Eigenvectors of Special Matrices Diagonal matrix First Diagonal Second Diagonal element element 3 0  A=  0 −4   Eigenvalues 1 = 3; 2 = −4; Eigenvectors 1 0 0 1     Verify  3 0  1   3  1  3 0  0   0  0  0 −4  0  =  0  = 3  0   0 −4  1  =  −4  = −4  1                  A x x A x x 10/28/2024 10/28/2024 Sarada Jayan Geetha K N 19 19 Eigenvalues and Eigenvectors of Special Matrices 10/28/2024 10/28/2024 Sarada Jayan Geetha K N 20 20 Eigenvalues and Eigenvectors of Special Matrices 10/28/2024 10/28/2024 Sarada Jayan Geetha K N 21 21 Properties of Eigenvalues 1. A square matrix of order n will have n eigenvalues. The eigenvalues can be real or complex conjugates This is because roots of an nth degree polynomial equation has n roots and also these roots are either real or if complex appears in pairs as complex conjugates. 10/28/2024 10/28/2024 Sarada Jayan Geetha K N 22 22 Properties of Eigenvalues 2. Sum of Eigenvalues of a matrix = Trace of the matrix (sum of diagonal elements of matrix) 3. Product of Eigenvalues of a matrix = Determinant of the matrix 1 + 2 = 6 = trace( A) 2 3 1 = 7; 2 = −1; A=  2 3 12 =   = −7 = det( A) 5 4 5 4 10/28/2024 10/28/2024 Sarada Jayan Geetha K N 23 23 Proof: ( A −  I ) = 0   n − (a11 + a22... + ann ) n−1 +.... + ( −1) A =  n − Tr ( A) n−1 +.... + ( −1) A = 0 n n ( A −  I ) = 0  (  − 1 )(  − 2 )....(  − n ) = 0   n − (1 + 2... + n ) n−1 + cn−2 n−2 +... Hence Trace(A)=sum of eigenvalues ( A − I ) = an  n + an−1 n−1 +.... + a1 + a0 = ( 1 −  )( 2 −  )....( n −  ) Putting  = 0 A = 12..n 10/28/2024 10/28/2024 Sarada Jayan Geetha K N 24 24 10/28/2024 10/28/2024 Sarada Jayan Geetha K N 25 25 Properties of Eigenvalues 4. If λ𝟏 and λ𝟐 are the eigenvalues of a square matrix A of order 2, then: (a) λ𝟏 𝟐 and λ𝟐 𝟐 will be the eigenvalues of the matrix 𝑨𝟐 (b) λ𝟏 𝟑 and λ𝟐 𝟑 will be the eigenvalues of the matrix 𝑨𝟑 (c) λ𝟏 𝒏 and λ𝟐 𝒏 will be the eigenvalues of the matrix 𝑨𝒏 But the unit magnitude eigenvectors will be same for all these matrices 10/28/2024 10/28/2024 Sarada Jayan Geetha K N 26 26 10/28/2024 10/28/2024 Sarada Jayan Geetha K N 27 27 Eigenvalues and Eigenvectors of Special Matrices Anti-Diagonal matrices: 0 0 2 0 2 A= ; B= 0 4 0 ; 3 0 3 0 0 Here we can use the property to find the eigenvalues as A2 will be a diagonal matrix 6 0 0 6 0 A2= ; B2= 0 16 0 ; 0 6 0 0 6 Eigenvalues of A2 are 6,6. So eigenvalues of A are + 𝟔 and - 𝟔 Eigenvalues of B2 are 6, 16 and 6. So eigenvalues of B are + 𝟔, - 𝟔 and 4 10/28/2024 10/28/2024 Sarada Jayan Geetha K N 28 28 Eigenvalues and Eigenvectors of Special Matrices 10/28/2024 10/28/2024 Sarada Jayan Geetha K N 29 29 Properties of Eigenvalues 5. If λ𝟏 and λ𝟐 are the eigenvalues of a square matrix A of order 2, then, 𝒌λ𝟏 and 𝒌λ𝟐 are the eigenvalues of the matrix 𝒌𝑨. 6. If λ𝟏 and λ𝟐 are the eigenvalues of a non-singular matrix A of 𝟏 𝟏 order 2, then, and are the eigenvalues of the matrix 𝑨−𝟏. λ𝟏 λ𝟐 (But the unit magnitude eigenvectors will be same of all these matrices) Proof: 10/28/2024 10/28/2024 Sarada Jayan Geetha K N 30 30 10/28/2024 10/28/2024 Sarada Jayan Geetha K N 31 31 Properties of Eigenvalues 7. Eigenvalues of A and AT are same 8. Eigenvalues of a symmetric matrix is real and their eigenvectors are orthogonal. 9. Eigenvalues of a skew symmetric matrix is purely imaginary or zero. 10. Eigenvalues of an orthogonal matrix has magnitude 1. 10/28/2024 10/28/2024 Sarada Jayan Geetha K N 32 32 10/28/2024 10/28/2024 Sarada Jayan Geetha K N 33 33 10/28/2024 10/28/2024 Sarada Jayan Geetha K N 34 34 Properties of Eigenvalues 11. If all row sums or column sums are same for a matrix A, then that sum is an eigenvalue of A. Example 1. 2 1 Column sum is same. It is 5. A=  Eigenvalues are 5 and 1  3 4 Example 2.  2 4 A=  Row sum is same. It is 6. Eigenvalues are 6 and -3 5 1 10/28/2024 10/28/2024 Sarada Jayan Geetha K N 35 35 Properties of Eigenvalues 12. If eigenvalues are distinct, the corresponding eigenvectors will be linearly independent. Hence the eigenvectors of an 𝒏 × 𝒏 matrix with distinct eigenvalues will form a basis for the Euclidean Space Rn. (If eigenvalues are repeated then eigenvectors may or may not form a basis of eigenvectors for Rn.) 10/28/2024 10/28/2024 Sarada Jayan Geetha K N 36 36 Home Work 1. Find the eigenvalues of at least 3 symmetric matrices, 3 skew- symmetric matrices and 3 orthogonal matrices and verify the property. 2. Using MATLAB generate the eigenvalues and eigenvectors of a symmetric matrix and verify that the eigenvectors are orthogonal. 3. The command ‘magic(3)’ in MATLAB generates a magic square matrix of order 3, with all row sum and column sum equal. Generate such a matrix and find the eigenvalues of that matrix. 10/28/2024 10/28/2024 Sarada Jayan Geetha K N 37 37 Matrix with repeated eigenvalues 10/28/2024 10/28/2024 Sarada Jayan Geetha K N 38 38 Algebraic and Geometric multiplicities of eigenvalues Algebraic multiplicity is the number of times the eigenvalue appears as a root in the characteristic equation. Geometric multiplicity is the dimension of the subspace formed by the eigenvectors of that particular eigenvalue.. And the algebraic multiplicity is always going to be greater than or equal to the geometric multiplicity. 10/28/2024 10/28/2024 Sarada Jayan Geetha K N 39 39 Algebraic and Geometric multiplicities of eigenvalues Algebraic multiplicity is the number of times the eigenvalue appears as a root in the characteristic equation. Geometric multiplicity is the dimension of the subspace formed by the eigenvectors of that particular eigenvalue.. And the algebraic multiplicity is always going to be greater than or equal to the geometric multiplicity. 10/28/2024 10/28/2024 Sarada Jayan Geetha K N 40 40 Algebraic and Geometric multiplicities of eigenvalues 10/28/2024 10/28/2024 Sarada Jayan Geetha K N 41 41 Algebraic and Geometric multiplicities of eigenvalues 10/28/2024 10/28/2024 Sarada Jayan Geetha K N 42 42 Algebraic and Geometric multiplicities of eigenvalues 10/28/2024 10/28/2024 Sarada Jayan Geetha K N 43 43 Algebraic and Geometric multiplicities of eigenvalues 10/28/2024 10/28/2024 Sarada Jayan Geetha K N 44 44 Left eigenvectors 10/28/2024 10/28/2024 Sarada Jayan Geetha K N 45 45 MATLAB Commands [R,D,L] = eig(A) % produces a diagonal matrix D of eigenvalues and a full matrix R whose columns are the corresponding right eigenvectors (eigenvectors) and also produces a full matrix L whose columns are the corresponding left eigenvectors so that L'*A = D*L’ A= [ 1 2 3 ; 2 3 4; 3 5 8]; syms x; charpoly(A,x) % Command for obtaining characteristic polynomial of a matrix A. 10/28/2024 10/28/2024 Sarada Jayan Geetha K N 46 46 Generation of a matrix with given eigenvalues and eigenvectors If D is a diagonal matrix with eigenvalues as diagonal elements and V is the matrix whose columns are the corresponding independent eigenvectors of A (as obtained using MATLAB command eig), then AV = VD → AVV-1 = VDV-1 → A = VDV-1 This way we can generate a matrix by creating D with the given eigenvalues and V with the given eigenvectors. This is possible only when the eigenvectors of 𝐴𝑛×𝑛 forms a basis for Rn. 10/28/2024 10/28/2024 Sarada Jayan Geetha K N 47 47 10/28/2024 10/28/2024 Sarada Jayan Geetha K N 48 48  Cayley-Hamilton Theorem Statement: Every matrix satisfies its own characteristic equation 10 8 Q. Verify Cayley-Hamilton Theorem for 𝐴 = −3 0 10/28/2024 10/28/2024 Sarada Jayan Geetha K N 49 49  Eigenvalues of AAT and ATA ATA and AAT have same non-zero eigenvalues. A= A= 1 2 4 5 6 2 3 2 3 4 3 4 >> eig(A'*A) >> eig(A*A') ans = ans = -0.0000 -0.0000 0.2269 0.1400 105.7731 42.8600 >> eig(A*A') >> eig(A'*A) ans = ans = 0.2269 0.1400 105.7731 42.8600 10/28/2024 10/28/2024 Sarada Jayan Geetha K N 50 50 Relation between rank of a square matrix and the number of zero eigenvalues >> r=rank(A),lambda=eig(A) r= >> A=randi([0,9],9,2)*randi([0,9],2,9) 2 A= lambda = 9 45 9 54 45 0 9 18 0 1.0e+02 * 24 15 24 9 12 6 3 12 27 2.4853 + 0.0000i 20 30 20 30 28 4 6 16 18 -0.5453 + 0.0000i 65 45 65 30 37 16 9 34 72 -0.0000 + 0.0000i 41 65 41 66 61 8 13 34 36 0.0000 + 0.0000i 72 45 72 27 36 18 9 36 81 -0.0000 + 0.0000i 15 40 15 45 39 2 8 18 9 -0.0000 - 0.0000i 24 50 24 54 48 4 10 24 18 -0.0000 + 0.0000i 16 45 16 51 44 2 9 20 9 -0.0000 - 0.0000i 0.0000 + 0.0000i 10/28/2024 10/28/2024 Sarada Jayan Geetha K N 51 51