Orthogonal Complement and Projection

Orthogonal Complement

• For a subspace ( W ) of a vector space ( V ), the orthogonal complement of ( W ) includes every vector in ( V ) that is orthogonal to all vectors in ( W ).
• ( W^{\perp}={v \in V \mid\langle v, w\rangle=0 \text { for all } w \in W} ) defines the orthogonal complement of ( W ).
• ( W^{\perp} ) forms a subspace.
• The orthogonal complement of the orthogonal complement of ( W ) is ( W ), denoted as ( \left(W^{\perp}\right)^{\perp}=W ).
• The intersection of ( W ) and ( W^{\perp} ) contains only the zero vector: ( W \cap W^{\perp}={0} ).

Orthogonal Projection

• Any vector ( v ) in ( V ) can be uniquely expressed as the sum of two vectors ( w ) and ( u ), where ( w ) is in ( W ) and ( u ) is in ( W^{\perp} ).
• Orthogonal projection of ( v ) onto ( W ) is vector ( w ), which is denoted as ( w=\operatorname{proj}_{W}(v) ).
• For ( V=\mathbb{R}^{2} ) and ( W=\operatorname{span}\left{\left[\begin{array}{l}1 \ 1\end{array}\right]\right} ), to find ( \operatorname{proj}_{w}\left(\left[\begin{array}{l}2 \ 4\end{array}\right]\right) ):
  • Express ( \left[\begin{array}{l}2 \ 4\end{array}\right] ) as ( w+u ), where ( w \in W ) and ( u \in W^{\perp} ).
  • Set ( w=\alpha\left[\begin{array}{l}1 \ 1\end{array}\right] ) and ( u=\left[\begin{array}{c}u_{1} \ u_{2}\end{array}\right] ).
  • Since ( \left\langle\left[\begin{array}{l}1 \ 1\end{array}\right],\left[\begin{array}{l}u_{1} \ u_{2}\end{array}\right]\right\rangle=0 ), then ( u_{1}+u_{2}=0 ), leading to ( u_{2}=-u_{1} ).
  • Solve ( \left[\begin{array}{l}2 \ 4\end{array}\right]=\left[\begin{array}{l}\alpha \ \alpha\end{array}\right]+\left[\begin{array}{c}u_{1} \ -u_{1}\end{array}\right] ) to find ( \alpha=3 ) and ( u_{1}=-1 ).
  • ( w=3\left[\begin{array}{l}1 \ 1\end{array}\right]=\left[\begin{array}{l}3 \ 3\end{array}\right] ), so ( \operatorname{proj}_{w}\left(\left[\begin{array}{l}2 \ 4\end{array}\right]\right)=\left[\begin{array}{l}3 \ 3\end{array}\right] ).

Gram-Schmidt Process

• The Gram-Schmidt process transforms a basis ( \left{v_{1}, \ldots, v_{k}\right} ) of ( W ) into an orthogonal basis ( \left{w_{1}, \ldots, w_{k}\right} ) of ( W ).
• Set ( w_{1}=v_{1} ).
• Calculate subsequent vectors using the formula ( w_{i}=v_{i}-\operatorname{proj}{\text {span }\left{w{1}, \ldots, w_{i-1}\right}}\left(v_{i}\right) ).
• For all ( i ), ( \operatorname{span}\left{w_{1}, \ldots, w_{i}\right} ) is equal to ( \operatorname{span}\left{v_{1}, \ldots, v_{i}\right} ).

Gram-Schmidt Example

• For ( V=\mathbb{R}^{3} ) and ( W=\operatorname{span}\left{\left[\begin{array}{l}1 \ 1 \ 1\end{array}\right],\left[\begin{array}{l}0 \ 1 \ 1\end{array}\right]\right} ), the orthogonal basis of ( W ) is found as follows:
  • Set ( w_{1}=v_{1}=\left[\begin{array}{l}1 \ 1 \ 1\end{array}\right] ).
  • Calculate ( w_{2}=v_{2}-\operatorname{proj}{\text {span }\left{w{1}\right}}\left(v_{2}\right) ).
  • ( \operatorname{proj}{\text {span }\left{w{1}\right}}\left(v_{2}\right)=\frac{\left\langle v_{2}, w_{1}\right\rangle}{\left\langle w_{1}, w_{1}\right\rangle} w_{1}=\frac{2}{3}\left[\begin{array}{l}1 \ 1 \ 1\end{array}\right] ).
  • ( w_{2}=\left[\begin{array}{l}0 \ 1 \ 1\end{array}\right]-\frac{2}{3}\left[\begin{array}{l}1 \ 1 \ 1\end{array}\right]=\left[\begin{array}{c}-\frac{2}{3} \ \frac{1}{3} \ \frac{1}{3}\end{array}\right] ).
  • The orthogonal basis of ( W ) is ( \left{w_{1}, w_{2}\right}=\left{\left[\begin{array}{l}1 \ 1 \ 1\end{array}\right],\left[\begin{array}{c}-\frac{2}{3} \ \frac{1}{3} \ \frac{1}{3}\end{array}\right]\right} ).