Orthogonal Projection Concepts

Study Notes

Orthogonal Projection Concepts

Projection of vector b onto subspace V with an orthogonal basis is the sum of projections onto each basis vector, leveraging their orthogonal nature.
Projection formula for any vector b relative to subspace V (with k orthogonal basis vectors) is expressed as:
- proj V b = proj V1 b + proj V2 b + ... + proj Vk b
- This represents the sum of k projections, where k is the count of basis vectors.

Orthogonal and Orthonormal Bases

An orthogonal set of vectors v1, v2, ... vk in R^m satisfies vi * vj = 0 for i ≠ j.
An orthogonal basis consists of vectors that are mutually orthogonal and span the subspace V.
An orthonormal basis consists of orthogonal vectors that are also unit vectors.

Linear Independence and Propositions

Proposition 2.1 states that an orthogonal set of non-zero vectors is linearly independent, as shown by applying the dot product, leading to the conclusion that all scalar coefficients must be zero.
Lemma 2.2 asserts that if vector v is in V, it can be expressed as the sum of its projections onto the orthogonal basis vectors of V.

Converting to Orthogonal Basis

The Gram-Schmidt process is utilized to transform a non-orthogonal basis of subspace V into an orthogonal basis.

Gram-Schmidt Process

The Gram-Schmidt process allows obtaining an orthogonal basis {w1, w2, ..., wk} from a basis {v1, v2, ..., vk} by:
- Setting w1 = v1
- Defining subsequent vectors as w2 = v2 - proj w1 v2 and so forth, following the outlined steps in a more detailed resource.

QR Decomposition

QR Decomposition involves factoring into two components:
- Q represents an orthonormal basis set as column vectors.
- R is an upper triangular matrix.
R values are computed as rij = q1 dot vj for i < j, and 0 when i > j.

Projection Solutions

To determine x bar using QR Decomposition, the formula is:
- x = R^-1Q^T b
Another approach for projection with QR Decomposition is represented as:
- Proj_V = QQ^T

Methods for Projection Calculation

Three methods exist for finding projections:
- Using basis vectors of a subspace along with the external vector b, the projection is determined via x = (A^TA)^{-1}A^T b, with further relations derived.
- For a one-dimensional subspace, projections can be calculated using:
  - Proj_V = (1/||a||^2) aa^T or
  - Proj_V = I3 - (1/||a||^2) aa^T if the perpendicular subspace V perp is spanned by a single vector.
- The third method involves applying QR Decomposition to obtain an orthonormal basis for V.

Description

Explore the fundamentals of orthogonal projection in vector spaces. This quiz covers key concepts such as orthogonal and orthonormal bases, linear independence, and the projection formula. Test your understanding of how these ideas interconnect in linear algebra.