Euclidean Spaces: Linear Algebra

• These are lecture notes for Linear Algebra and Analytic Geometry II, taught at the University of Porto in 2024/25.

Euclidean Spaces

• A product scalar in a real vector space E is a bilinear function that maps E x E to the real numbers, denoted as (u, v) → u | v.
• For any vectors u, v in E, the product scalar satisfies the following conditions:
  • Symmetry: u | v = v | u.
  • Positivity: u | u ≥ 0.
  • Positive Definiteness: If u ≠ 0, then u | u > 0.
• A real vector space equipped with a product scalar is called a real Euclidean space.
• The notation · | · is used for the product scalar.
• The inner product in Rⁿ is an example of a product scalar.
• For real-valued continuous functions defined on [a, b], the function f | g = ∫ₐᵇ f(t)g(t) dt is a product scalar.
• The norm associated with the product scalar is defined as ||u|| = √(u | u).
• The Cauchy-Schwarz inequality states that for a real Euclidean space E and vectors u, v ∈ E: |u | v| ≤ ||u|| ||v||, with equality if and only if u and v are linearly dependent.
• In the context of real-valued continuous functions in [a, b]: (∫ₐᵇ f(t)g(t) dt)² ≤ (∫ₐᵇ f²(t) dt)(∫ₐᵇ g²(t) dt).
• Properties of the norm for a real Euclidean space E, u, v ∈ E, and λ ∈ R include:
  • Positivity: ||u|| ≥ 0
  • Positive definiteness: ||u|| > 0, provided that u ≠ 0.
  • Homogeneity: ||λu|| = |λ| ||u||.
  • Triangle inequality: ||u + v|| ≤ ||u|| + ||v||.
• The norm can be defined independently of the dot product as a function from E to R satisfying the aforementioned conditions.
• Pythagorean Theorem: For a real Euclidean space E and u, v ∈ E, if u | v = 0, then ||u + v||² = ||u||² + ||v||².
• In Rⁿ with the usual inner product, u | v = ||u|| ||v|| cos θ, where θ is the angle between vectors u and v.
• For a real Euclidean space E and u, v ∈ E, cos θ = (u | v) / (||u|| ||v||), where θ ∈ [0, π] is the unique angle.
• Vectors u, v are orthogonal, denoted u ⊥ v, if u, v ≠ 0 and u | v = 0.

Orthogonal and Orthonormal Sets

• A set A in a real Euclidean space E is orthogonal if any two distinct vectors in A are orthogonal.
• A set A is orthonormal if it's orthogonal and ||u|| = 1 for all u ∈ A.
• A basis b = (u₁, ..., uₙ) is orthogonal/orthonormal if {u₁, ..., uₙ} is an orthogonal/orthonormal set.
• In a real Euclidean space E, an orthogonal subset A of E has linearly independent vectors.
• With coordinates relative to an orthonormal basis, a dot product is calculated exactly as the inner product in Rⁿ.

Orthonormal Basis

• If b = (o₁, ..., oₙ) is an orthonormal basis of a real Euclidean space E:
  • For any u ∈ E, u = (u | o₁) o₁ + ... + (u | oₙ) oₙ.
  • If u = (α₁, ..., αₙ)b and v = (β₁, ..., βₙ)b, then u | v = α₁β₁ + ... + αₙβₙ and ||u|| = √(α₁² + ... + αₙ²).

Orthogonal Projection

• In Rⁿ, the orthogonal projection of v onto u (pᵤv) is such that pᵤv = λu, where cos θ = λ||u|| / ||v||.
• Given u and v: v = pv + p⊥v.
• The formula is pᵤv = (cos θ ||v|| / ||u||) u = ((u | v) / ||u||²) u.
• For a real Euclidean space E, the orthogonal projection of v onto u, with u ≠ 0, is defined as pᵤv = ((v | u) / ||u||²) u.
• Note that v = pᵤv + (v – pᵤv) and v – pᵤv is orthogonal to u.

Gram-Schmidt Process

• Given a real Euclidean space E and linearly independent vectors u₁, ..., uₖ ∈ E, vectors can be defined recursively:
  • v₁ = u₁
  • v₂ = u₂ - pᵥ₁u₂
  • vₖ = uₖ - pᵥ₁uₖ - ... - pᵥₖ₋₁uₖ
• These vectors satisfy:
  • G({v₁, ..., vₖ}) = G({u₁, ..., uₖ})
  • {v₁, ..., vₖ} is an orthogonal set of non-null vectors.
• To obtain an orthonormal set with property (i), consider {v₁/||v₁||, ..., vₖ/||vₖ||}.

Basis

• Every real Euclidean space of finite dimension has an orthonormal basis.
• Given u₁ = (1, 0, 1, 0), u₂ = (3, 1, -1, 0) and u₃ = (8, -7, 0, 3), the Gram-Schmidt orthogonalization process yields:
• Orthogonal basis: {(1, 0, 1, 0), (2, 1, -2, 0), (2, -8, -2, 3)}.
• Orthonormal basis: {(1/√2, 0, 1/√2, 0), (2/3, 1/3, -2/3, 0), (2/9, -8/9, -2/9, 1/3)}.
• For a real Euclidean space E and A ⊆ E, the set A⟂ = {v ∈ E : ∀u ∈ A, u | v = 0} denotes the orthogonal complement of A in E.
• If E is a Euclidean space and A is a subset of E, then A⟂ is a subspace of E.
• For a real Euclidean space E of finite dimension and a subspace F: E = F ⊕ F⟂ and dim E = dim F + dim F⟂.
• Any vector u ∈ E is uniquely written as u = pᶠu + pᶠ⟂u, where pᶠu is a vector in F and pᶠ⟂u is a vector in F⟂ and given {e₁, ..., eₖ} an orthonormal basis of F, pᶠu = (u | e₁)e₁ + ··· + (u | eₖ)eₖ.
• The orthogonal projection a real Euclidean space E, F being the subspace of E, and u is pᶠ⟂u.
• In a real Euclidean space E of finite dimension, with F a subspace of E, for all v ∈ F, ||u – pᶠu|| ≤ ||u – v|| and ||u – pᶠu|| = ||u – v|| if and only if v = pᶠu.
• The distance from a point p = 0 + u to F is d(p, F) = ||u – pᶠu|| = ||pᶠ⟂u||.
• A k-plane in E is a set of the form π = p + F, where p ∈ E and F is a subspace of E with dimension k, passing through p with direction F.
  • If k = 1, the k-plane is a line.
  • If k = 2, the k-plane is a plane.
  • If k = n - 1, the k-plane is a hyperplane.
• Given F = G({u₁, ..., uₖ}), u₁, ..., uₖ ∈ E, u = (x₁, ..., xₙ) ∈ π = p + F, there exist λ₁, ..., λₖ ∈ R such that u = p + λ₁u₁ + ... + λₖuₖ.
• This leads to parametric equations of the k-plane.
• Cartesian equations consist of a system of linear equations in x₁, ..., xₙ whose solutions are the coordinates of points in p + F.
• If p + F is a k-plane in Rⁿ:
  • q ∈ p + F <=> (q - p) ∈ F <=> (q - p) ∈ (F⟂)⟂ <=> (q - p) | v = 0, for all v ∈ F⟂.
• To describe the proposition above, provided that dim F = k and dim F⟂ = n - k, where b = (v₁, ..., vₙ₋ₖ) is a basis of F⟂, implies that x ∈ p + F <=> (x - p) | vᵢ = 0, for all i <=> x | vᵢ = p | vᵢ, for all i.
• Lines in R² have cartesian equations of the form ax + by = c.
• Given (a, b) ≠ (0, 0), (a, b) is perpendicular to the line.
• Lines in R³ are defined by a system of two linear equations in x, y, z.
• Planes in R³ are defined by a linear equation ax + by + cz = d, where (a, b, c) ≠ (0, 0, 0) is a vector perpendicular to the plane.

Proofs

• Proof of proposition 1.9: Let E be a real Euclidean space of finite dimension and F be a vector subspace of E. Therefore, it holds that E = F ⊕ F⟂ and that dim E = dim F + dim F⟂
  • Proof: if u ∈ F ∩ F⟂, u | u = 0, so u = 0, thus F ∩ F⟂ = {0}. Given u ∈ E, {u₁, ..., uₖ} is an orthonormal basis of F, and given uF = (u | u₁) u₁ + ··· + (u | uₖ) uₖ, then uF ∈ F e u = uF + (u - uF). For i = 1, ..., k, (u - uF) | uᵢ = u | uᵢ - (u | u₁) (u₁ | uᵢ) - ··· - (u | uₖ) (uₖ | uᵢ) = u | uᵢ - u | uᵢ = 0, therefore u - uF ∈ F⟂. Given that uF ∈ F, means there is a u = uF + (u - uF) ∈ F + F⟂. It is here concluded that E = F ⊕ F⟂ and thus: dim E = dim F + dim F⟂.

Diagonalization

• A linear map between two finite-dimensional vector spaces can be described by a matrix, relative to a pair of bases.
• This section aims to investigate the possibility of representing a linear map by a diagonal matrix.
• Let E be a finite-dimensional vector space over K (R or C) and f an endomorphism of E (a linear map from E to E).
• f is diagonalizable if there exists a basis b of E such that Mb,b(f) is diagonal, that is, there exist λ₁, ..., λₙ ∈ K such that Mb,b(f) is the diagonal matrix with these values.
• If b = (e₁, ..., eₙ), then f(eᵢ) = λᵢeᵢ.

Eigenvalues and Eigenvectors

• An eigenvector of f is u ∈ E such that u ≠ 0 and there exists λ ∈ K with f(u) = λu.
• The scalar λ is the eigenvalue of f associated with the eigenvector u.
• The set E(λ) = {u ∈ E : f(u) = λu} is a subspace of E, called the eigenspace of E associated with λ.
• Given f: R² → R² where f(x, y) = (2x - 2y, 0), u = (1, 1) is an eigenvector of f, and 0 is the associated eigenvalue.
• Let E be a vector space over K of dimension n, let b = (u₁, ..., uₙ) be a basis of E and f an endomorphism of E. If Mb,b(f) is diagonal, the entries on the main diagonal are the eigenvalues of f and E(λ) = Ker(f - λ Id).
• If E is a finite-dimensional vector space over K and f is an endomorphism of E, f is diagonalizable if and only if there exists a basis of E formed by eigenvectors of f.
• Let λ₁, ..., λₘ ∈ K be distinct eigenvalues of f, and let u₁, ..., uₘ be eigenvectors of f associated with λ₁, ..., λₘ. Then the eigenvectors u₁, ..., uₘ are linearly independent.
• If E is a vector space over K, f is an endomorphism of E and dim(E) = n, where f has n distinct eigenvalues, f is diagonalizable.
• A condition for f being diagonalizable is sufficient but not necessary. Provided that f = Id, where for any base b of E, we have Mb,b(f) = I, and λ = 1 is the singular eigenvalue of f.
• To obtain a base b of E for which Mbb(f) is diagonal, it is necessary to determine the eigenvalues of f, which is done through a matrix of f.
• Vectors are defined as the vectors of an endomorphism f of Kⁿ. f(X) = AX
• A matrix A is diagonalizable if f is diagonalizable. There must exist a base of Kⁿ formed by vectors of A.
• Sendo A a matrix n x n in K, λ ∈ K is the eigenvalue of A sse det(A - λI) = 0.
• A quadratic matrix A of size n x n and entries in K has p(λ) = det(A - λI) where it is a polynomial of degree n.
• Given A “ p 0 ´1 1 0 q. For λ P K, ppλq “ detpA ´ λIq “ λ 2 ` 1, where either K = R (A has no eigenvalues) or K = C (A has two values -i and i)
• Given that E is a vector space over K with finite dimension and that f is an endomorphism of E: for any base b and b’ of E, p(x) = det(Mbb(f) – xI) = dt(Mbb’(f) – xI) exists, it can be understood that every matrix of the type Mbb (f) has the same characteristic polynomial. Moreover, given x ∈ K is an eigenvector of f, by doing p(λ) = det(A – xI) = 0, the eigenvalues f are the eigenvalues of A = Mbb (f), which are the roots of p(λ).
• Verify if f: R³ → R3, given by f(x, y, z) = (2x + y + 2z, y – z, -z), is diagonalizable.

Eigenvalues

• For a vector space E of finite dimension
• Algebraic multiplicity: the multiplicity of λ as a root of the characteristic polynomial of f.
• Geometric Multiplicity: dimension of the eigenspace E(λ).
• Thus, if f has eigenvalues, then f is diagonalizable, and we can say for an elgenvalue λ of f, the geometric multiplicity of λ = the algebraic multiplicity of λ

Similar matrices

• Two quadratic matrices A and B are similar when an invertible matrix, C exists, where B = C⁻¹AC.
• An endomorphism, f, is defined as diagonalizable, sse Mbb(f) if it's analogous to a diagonal matrix.
• A matrix n x n A is said to be diagonal as well if it's similar to a diagonal matrix.
• Steps to apply the diagonalization of a matrix:
• Determine an expression of the endomorphism f: R² -> R² that links each vector to its reflection on the ray of equation x + 2y = 0
• Given B, a base de R² constituted of two vectors signaled b = ((1,2), (2,−1)).
• Both are vectors of f because the image of the normal vector has been reflected, (-1,-2) and the image of the reay has a propre vector.
• Thus, f (1,2) = -1 * (1,2) e f (2,-1) = (2,-1), e we have a diagonal matrix Mbb(f)