Intro to Vectors and Vector Operations

Vectors

• A vector constitutes an ordered list of numbers.
• An $n$-dimensional real vector exists in $\mathbb{R}^{n}$ and consists of $n$ real numbers.
• Vectors can be visualized geometrically as directed line segments.

Vector Operations

• Given vectors $\mathbf{u} = \begin{bmatrix} u_1 \ u_2 \ \vdots \ u_n \end{bmatrix}$ and $\mathbf{v} = \begin{bmatrix} v_1 \ v_2 \ \vdots \ v_n \end{bmatrix}$ in $\mathbb{R}^{n}$, and a scalar $c$:
  • Addition: $\mathbf{u} + \mathbf{v} = \begin{bmatrix} u_1 + v_1 \ u_2 + v_2 \ \vdots \ u_n + v_n \end{bmatrix}$
  • Scalar Multiplication: $c\mathbf{u} = \begin{bmatrix} cu_1 \ cu_2 \ \vdots \ cu_n \end{bmatrix}$

Properties of Vector Operations

• For vectors $\mathbf{u}$, $\mathbf{v}$, and $\mathbf{w}$ in $\mathbb{R}^{n}$, and scalars $c$ and $d$:
  • $\mathbf{u} + \mathbf{v} = \mathbf{v} + \mathbf{u}$
  • $(\mathbf{u} + \mathbf{v}) + \mathbf{w} = \mathbf{u} + (\mathbf{v} + \mathbf{w})$
  • $\mathbf{u} + \mathbf{0} = \mathbf{u}$
  • $\mathbf{u} + (-\mathbf{u}) = \mathbf{0}$
  • $c(\mathbf{u} + \mathbf{v}) = c\mathbf{u} + c\mathbf{v}$
  • $(c + d)\mathbf{u} = c\mathbf{u} + d\mathbf{u}$
  • $c(d\mathbf{u}) = (cd)\mathbf{u}$
  • $1\mathbf{u} = \mathbf{u}$

Dot Product

• The dot product of vectors $\mathbf{u} = \begin{bmatrix} u_1 \ u_2 \ \vdots \ u_n \end{bmatrix}$ and $\mathbf{v} = \begin{bmatrix} v_1 \ v_2 \ \vdots \ v_n \end{bmatrix}$ in $\mathbb{R}^{n}$ is:
  • $\mathbf{u} \cdot \mathbf{v} = u_1v_1 + u_2v_2 + \dots + u_nv_n$

Properties of Dot Product

• For vectors $\mathbf{u}$, $\mathbf{v}$, and $\mathbf{w}$ in $\mathbb{R}^{n}$, and a scalar $c$:
  • $\mathbf{u} \cdot \mathbf{v} = \mathbf{v} \cdot \mathbf{u}$
  • $(\mathbf{u} + \mathbf{v}) \cdot \mathbf{w} = \mathbf{u} \cdot \mathbf{w} + \mathbf{v} \cdot \mathbf{w}$
  • $(c\mathbf{u}) \cdot \mathbf{v} = c(\mathbf{u} \cdot \mathbf{v}) = \mathbf{u} \cdot (c\mathbf{v})$
  • $\mathbf{u} \cdot \mathbf{u} \geq 0$, and $\mathbf{u} \cdot \mathbf{u} = 0$ if and only if $\mathbf{u} = \mathbf{0}$

Length of a Vector

• The length (or norm) of a vector $\mathbf{v}$ is:
  • $| \mathbf{v} | = \sqrt{\mathbf{v} \cdot \mathbf{v}} = \sqrt{v_1^2 + v_2^2 + \dots + v_n^2}$

Unit Vector

• A unit vector has a length of 1. Normalizing a non-zero vector $\mathbf{v}$ involves dividing it by its length:
  • $\mathbf{\hat{v}} = \frac{\mathbf{v}}{| \mathbf{v} |}$

Distance Between Vectors

• The distance between two vectors $\mathbf{u}$ and $\mathbf{v}$ in $\mathbb{R}^{n}$ is:
  • $dist(\mathbf{u}, \mathbf{v}) = | \mathbf{u} - \mathbf{v} |$

Orthogonal Vectors

• Two vectors $\mathbf{u}$ and $\mathbf{v}$ are orthogonal if their dot product is zero:
  • $\mathbf{u} \cdot \mathbf{v} = 0$

Orthogonal Projection

• The orthogonal projection of a vector $\mathbf{y}$ onto a vector $\mathbf{u}$ is:
  • $proj_{\mathbf{u}}\mathbf{y} = \frac{\mathbf{y} \cdot \mathbf{u}}{\mathbf{u} \cdot \mathbf{u}}\mathbf{u}$

Matrices

• A matrix comprises a rectangular array of numbers.
• An $m \times n$ matrix has $m$ rows and $n$ columns.
• A square matrix has an equal number of rows and columns.

Matrix Operations

• Let $A$ and $B$ be matrices of the same size, and $c$ be a scalar:
  • Addition: Element-wise addition of corresponding entries.
  • Scalar Multiplication: Multiply each matrix entry by the scalar.

Matrix Multiplication

• If $A$ is an $m \times n$ matrix and $B$ is an $n \times p$ matrix, their product $AB$ is an $m \times p$ matrix.
• The $(i, j)$-th entry of $AB$ equals the dot product of the $i$-th row of $A$ and the $j$-th column of $B$.

Transpose of a Matrix

• The transpose of an $m \times n$ matrix $A$, denoted $A^{T}$, is an $n \times m$ matrix that flips the rows and columns of $A$.

Identity Matrix

• The identity matrix $I_{n}$ is an $n \times n$ matrix.
• It has 1's on the main diagonal and 0's elsewhere.

Inverse of a Matrix

• The inverse of a square matrix $A$, denoted $A^{-1}$, satisfies $AA^{-1} = A^{-1}A = I$.
• A matrix is invertible if and only if its determinant is non-zero.

Determinant of a Matrix

• The determinant of a square matrix $A$, denoted $\det(A)$ or $|A|$, is a scalar computed from its elements.
• For a $2 \times 2$ matrix $A = \begin{bmatrix} a & b \ c & d \end{bmatrix}$, $\det(A) = ad - bc$.

Properties of Determinants

• If $A$ is an $n \times n$ matrix, then $\det(A^{T}) = \det(A)$.
• If $A$ and $B$ are $n \times n$ matrices, then $\det(AB) = \det(A)\det(B)$.
• If $A$ is an $n \times n$ matrix and $c$ is a scalar, then $\det(cA) = c^{n}\det(A)$.

Linear Transformations

• A function $T: \mathbb{R}^{n} \rightarrow \mathbb{R}^{m}$ is a linear transformation if:
  • $T(\mathbf{u} + \mathbf{v}) = T(\mathbf{u}) + T(\mathbf{v})$ for all $\mathbf{u}, \mathbf{v}$ in $\mathbb{R}^{n}$
  • $T(c\mathbf{u}) = cT(\mathbf{u})$ for all $\mathbf{u}$ in $\mathbb{R}^{n}$ and all scalars $c$

Eigenvalues and Eigenvectors

• An eigenvector $\mathbf{v}$ of an $n \times n$ matrix $A$ satisfies $A\mathbf{v} = \lambda\mathbf{v}$ with eigenvalue $\lambda$.
• The scalar $\lambda$ is called an eigenvalue of $A$.

Characteristic Equation

• The characteristic equation of a matrix $A$ is:
  • $\det(A - \lambda I) = 0$
• The roots of this equation give the eigenvalues of $A$.

Eigenspace

• The eigenspace of $A$ for eigenvalue $\lambda$ includes all eigenvectors for $\lambda$ and the zero vector.
• It represents the null space of the matrix $(A - \lambda I)$.

Signal-to-Noise Ratio (SNR)

• Signal-to-noise ratio (SNR or S/N) is defined as the ratio of signal power (meaningful output) to the background noise power (meaningless or unwanted output).
  • $SNR = \frac{P_{signal}}{P_{noise}}$
• Because many signals vary over a wide range, signals are often expressed using the logarithmic decibel scale.
  • $SNR(dB) = 10 log_{10}(\frac{P_{signal}}{P_{noise}}) = P_{signal}(dB) - P_{noise}(dB)$
• The signal and noise should be measured at the same point in a system, and within the same system bandwidth.
• Sometimes, SNR is defined as the ratio of signal amplitude to noise amplitude.
  • $SNR = (\frac{A_{signal}}{A_{noise}})^2$
  • $SNR(dB) = 20 log_{10}(\frac{A_{signal}}{A_{noise}}) = A_{signal}(dB) - A_{noise}(dB)$

Example SNR Calculation

• A microphone measures a signal with an amplitude of 10 mV and noise with an amplitude of 1 μV.
  • $SNR = 20 log_{10}(\frac{1010^{-3} V}{110^{-6} V}) = 20 log_{10}(10000) = 80dB$