Matrices and Vectors

Matrices and Vectors Defined

• An m x n matrix, denoted as A(m x n), is a rectangular array of numbers organized into m rows and n columns.
• The element in matrix A at row i and column j is represented as aij, where the first subscript indicates the row and the second indicates the column.
• The row dimension of A refers to the number of rows, while the column dimension refers to the number of columns; a matrix with m rows and n columns is an m x n matrix.
• A square matrix has an equal number of rows and columns.
• A row vector, or 1 x n row vector, is a matrix with one row and n columns.
• A column vector, or m x 1 column vector, is a matrix with one column and m rows.
• A vector is considered a special type of matrix.
• Uppercase letters denote matrices, while lowercase letters denote vectors.
• A prime symbol denotes a row vector; for instance, a is a column vector, and a' is a row vector.
• A vector with n elements is of order n, applying to both 1 x n row vectors and n x 1 column vectors.

Elementary Matrix Operations

Matrix Addition and Subtraction

• Two matrices, A and B, can be added or subtracted if they have the same number of rows and columns; such matrices are conformable for addition.
• Matrix addition involves adding corresponding elements of the two matrices.
• If C = A + B, then cij = aij + bij.
• Matrix addition is commutative: A + B = B + A.
• Matrix addition is associative: A + (B + C) = (A + B) + C.
• Cancellation property: A + B = A + C implies B = C.
• A - B = A + (-B).

Matrix Multiplication

• The matrix product AB is defined only if the column dimension of A (the lead matrix) equals the row dimension of B (the lag matrix).
• This condition is the conformability condition for matrix multiplication; when satisfied, A and B are conformable for multiplication.
• If A is m x n and B is p x q, then C = A(m x n)B(p x q) is defined if n = p, and D = B(p x q)A(m x n) is defined if q = m.
• When the product AB is defined, A is post-multiplied by B or B is pre-multiplied by A.
• For a 1 x m row vector a' and an m x 1 column vector b, the product c(1x1) = a'(1xm)b(mx1) is computed by multiplying corresponding elements and summing the results: c = (a11b11 + a12b21 + ... + a1mbm1) = Σ(from i=1 to m) a1i bi1.
• The product of a row vector post-multiplied by a conformable column vector is a scalar, known as the scalar product of a and b.
• For matrices A(m x n) and B(n x q), the product C(m x q) = A(m x n)B(n x q) has elements cij = a'i bj, where a'i is the ith row of A and bj is the jth column of B.
• In C(m x q) = A(m x n)B(n x q), C inherits its row dimension from A and its column dimension from B.
• Matrix multiplication is generally not commutative: AB ≠ BA.
• Matrix multiplication is associative: (AB)C = A(BC) for conformable matrices A, B, C.
• Matrix multiplication is distributive: (A + B)C = AC + BC for conformable matrices A, B, C.
• For any scalar c, the matrix cA is obtained by multiplying each element of A by c.

Matrix Transposition

• The transpose of a matrix A, denoted as A' or AT, is obtained by interchanging the rows and columns of A.
• If A is an m x n matrix, its transpose A' is an n x m matrix.
• There is no conformability condition for transposing a matrix; every matrix has a transpose.
• A symmetric matrix is a matrix where A' = A, which requires A to be a square matrix.
• (A + B)' = A' + B' for any m x n matrices A and B.
• (AB)' = B'A' for conformable matrices A and B.
• (A')' = A for any matrix A.

The Trace of a Square Matrix

• For a square matrix A(n x n), the trace of A, denoted as tr(A), is the sum of the elements on the principal diagonal of A: tr(A) = Σ(from i=1 to n) aii.
• The trace is only defined for square matrices and is always a scalar.
• tr(A + B) = tr(A) + tr(B) for conformable matrices A and B.
• tr(AB) = tr(BA) for conformable matrices A and B.
• tr(ABC) = tr(CAB) = tr(BCA) for conformable matrices A, B, and C.
• tr(k) = k for any scalar k.

Special Matrices

Identity Matrices

• An n x n matrix with ones on the diagonal and zeros elsewhere is an identity matrix of order n.
• Identity matrices are analogous to the number 1 in scalar algebra.
• ImA(mxn) = A and A(mxn)In = A; pre- or post-multiplying a matrix A by an identity matrix of the appropriate order has no effect on A.
• tr(In) = n.

Null Matrices

• A matrix with all elements equal to zero is a null matrix.
• Null matrices are analogous to zero in scalar algebra; if A and B are conformable and B is a null matrix, then AB = 0.

Diagonal Matrices

• A square, non-null matrix with all off-diagonal elements equal to zero is a diagonal matrix.
• Denoted as D = diag(a11, a22, ..., ann) for an n x n diagonal matrix.
• All identity matrices are diagonal matrices.
• Many matrix operations are easily performed on diagonal matrices.

Inner Products and Norms of Vectors

• For two n x 1 column vectors a and b, the scalar product (inner product, dot product) is c(1x1) = a'(1xn)b(nx1) = (a1b1 + a2b2 + ... + anbn).
• The scalar product is defined only for vectors of the same order and is always a scalar.
• a'b = b'a.
• If a'b = 0, vectors a and b are orthogonal.
• The norm of a vector a, denoted as ||a||, is defined as ||a|| = (a'a)^(1/2) = (a1^2 + a2^2 + ... + an^2)^(1/2).

Determinants of Square Matrices

Computing Second-Order Determinants

• A determinant is a unique scalar associated with a square matrix A, denoted as |A| or det(A).
• For a 2x2 matrix A = [[a11, a12], [a21, a22]], the determinant is |A| = (a11a22) - (a12a21).
• A determinant is defined only for square matrices and is always a scalar.

Properties of Determinants

• |A'| = |A| for any n x n matrix A.
• |AB| = |A||B| = |B||A| = |BA| for any pair of n x n matrices A and B.
• |kA| = k^n|A| for any scalar k and square matrix A.
• If A is an n x n diagonal matrix, |A| = (a11a22...ann) = Π(from i=1 to n) aii.

Inverse Matrices

Definition of an Inverse Matrix

• Matrix inversion is the matrix algebra analogue of division in scalar algebra.
• For an n x n matrix A, if there exists another n x n matrix B such that BA = In and AB = In, then B is the inverse of A (A inverse), and A is invertible.
• Inverse matrices are defined only for square matrices.
• Not all square matrices have an inverse.
• The inverse of an invertible matrix A is unique; if BA = AB = In and CA = AC = In, then C = B.
• The notation A^(-1) denotes the inverse matrix of A.
• An invertible matrix is also called a nonsingular matrix.
• A square matrix without an inverse is a singular matrix.
• One case in which it is very easy to compute the inverse of a nonsingular matrix A. Let A = diag(a11, a22,..., ann). Then it can be shown that A^(-1) = diag(a^(-1)11, a^(-1)22,..., a^(-1)nn).
• If any column or row of A is a null vector, then A^(-1) does not exist.
• If one of the columns of a matrix A can be expressed as a linear function of the remaining columns of A we say that the columns of A are linearly dependent
• A square matrix A is nonsingular (has an inverse matrix) if and only if |A| ≠ 0.

Properties of Inverse Matrices

• If A^(-1) exists, it is unique.
• (ABC)^(-1) = C^(-1)B^(-1)A^(-1) for conformable, nonsingular matrices A, B, and C.
• (A^(-1))^(-1) = A.
• (A')^(-1) = (A^(-1))'.

Partitioned Matrices

• It is often useful to subdivide or partition a matrix into a set of submatrices; a matrix subdivided that way is a partitioned matrix.
• The most basic partitioning involves constituent rows or columns.
• A(m x n) may be partitioned into constituent rows: A(m x n) = [[a'1(1xn)], [a'2(1xn)], ..., [a'm(1xn)]], where a'j denotes the jth row of A.
• A(m x n) may be partitioned into constituent columns: A(m x n) = [[a1(mx1), a2(mx1), ..., an(mx1)]], where ai denotes the ith column of A.