Proof of the Central Limit Theorem

Proof of the Central Limit Theorem

• Central Limit Theorem states that for i.i.d random variables $X_1, X_2,...$ with mean $\mu$ and standard deviation $\sigma$, $S_n = X_1 + X_2 +... + X_n$, then as $n \rightarrow \infty$, $\frac{S_n - n\mu}{\sigma\sqrt{n}}$ converges in distribution to the standard normal distribution $N(0,1)$.
• The proof assumes without loss of generality (WLOG) that $\mu = 0$ by subtracting $\mu$ from each $X_i$.
• $Y_i$ is defined as $\frac{X_i}{\sigma}$, with $E[Y_i] = 0$ and $Var[Y_i] = 1$.
• The theorem posits that $\frac{Y_1 + Y_2 +... + Y_n}{\sqrt{n}}$ converges in distribution to $N(0,1)$ as $n$ approaches infinity.
• Convergence in distribution can be shown by demonstrating that the CDF of the left-hand side converges to the CDF of the standard normal distribution.
• Convergence can also be shown by demonstrating that the characteristic function of the left-hand side converges to the characteristic function of the standard normal distribution.
• $\phi_Y(t)$ represents the characteristic function of $Y_i$ and is defined as $E[e^{itY_i}]$.
• $Z_n$ is defined as $\frac{Y_1 + Y_2 +... + Y_n}{\sqrt{n}}$, and its characteristic function $\phi_{Z_n}(t)$ is given by $E[e^{itZ_n}]$.
• Due to the independence of $Y_i$, $\phi_{Z_n}(t)$ simplifies to $(\phi_Y(\frac{t}{\sqrt{n}}))^n$.
• Convergence of $\phi_{Z_n}(t)$ to $e^{-\frac{t^2}{2}}$ as $n \rightarrow \infty$ needs to be demonstrated, given that $e^{-\frac{t^2}{2}}$ is the characteristic function of the standard normal distribution.
• Taylor expansion of $\phi_Y(t)$ around $t = 0$ yields $\phi_Y(t) = \phi_Y(0) + t\phi'_Y(0) + \frac{t^2}{2}\phi''Y(0) + o(t^2)$, where $o(t^2)$ represents a function such that $\lim{t \rightarrow 0} \frac{o(t^2)}{t^2} = 0$.
• $\phi_Y(0) = 1$, $\phi'_Y(0) = 0$, and $\phi''_Y(0) = -1$ are derived using the properties of characteristic functions and the definitions of $Y_i$.
• Substituting these results into the Taylor expansion, $\phi_Y(t) = 1 - \frac{t^2}{2} + o(t^2)$.
• Therefore, $\phi_Y(\frac{t}{\sqrt{n}}) = 1 - \frac{t^2}{2n} + o(\frac{t^2}{n})$.
• Recalling that $\phi_{Z_n}(t) = (\phi_Y(\frac{t}{\sqrt{n}}))^n$, it follows that $\phi_{Z_n}(t) = (1 - \frac{t^2}{2n} + o(\frac{t^2}{n}))^n$.
• Taking the limit as $n \rightarrow \infty$, $\lim_{n \rightarrow \infty} \phi_{Z_n}(t) = \lim_{n \rightarrow \infty} (1 - \frac{t^2}{2n} + o(\frac{t^2}{n}))^n = e^{-\frac{t^2}{2}}$.
• As $n \rightarrow \infty$, $\phi_{Z_n}(t)$ converges to $e^{-\frac{t^2}{2}}$, implying that $Z_n$ converges in distribution to $N(0,1)$.
• Thus, $\frac{Y_1 + Y_2 +... + Y_n}{\sqrt{n}} \xrightarrow{d} N(0,1)$, $\frac{S_n}{\sigma\sqrt{n}} \xrightarrow{d} N(0,1)$, and $\frac{S_n - n\mu}{\sigma\sqrt{n}} \xrightarrow{d} N(0,1)$.

Central Limit Theorem Example

• For i.i.d Bernoulli random variables $X_1, X_2,... , X_{100}$ with $p = 0.6$, $S_{100} = X_1 + X_2 +... + X_{100}$ can be approximated using the central limit theorem to find $P(S_{100} > 65)$.

Central Limit Theorem Solution

• Since $X_i$ is a Bernoulli random variable with $p = 0.6$, $\mu = 0.6$ and $\sigma = \sqrt{p(1-p)} = \sqrt{0.6(0.4)} = \sqrt{0.24} \approx 0.49$.

• The probability $P(S_{100} > 65)$ is approximated using the central limit theorem:

  $P(S_{100} > 65) = P(\frac{S_{100} - 100(0.6)}{0.49\sqrt{100}} > \frac{65 - 100(0.6)}{0.49\sqrt{100}}) = P(\frac{S_{100} - 60}{4.9} > \frac{65 - 60}{4.9})$

  $= P(\frac{S_{100} - 60}{4.9} > \frac{5}{4.9}) \approx P(Z > 1.02)$, where $Z \sim N(0,1)$.

• Using the standard normal table, $P(Z > 1.02) \approx 0.1539$, thus $P(S_{100} > 65) \approx 0.1539$.

The Trigonometric Functions

Radian Measure

• An angle of 1 radian is subtended at the center of the circle by an arc equal in length to the radius of the circle.
• One revolution is equal to $2\pi$ radians.
• $180^{\circ} = \pi$ radians
• 1 radian $= \frac{180^{\circ}}{\pi} \approx 57.3^{\circ}$
• $1^{\circ} = \frac{\pi}{180}$ radians $\approx 0.017$ radians

Radian Measure Example

• Conversion example:

  (a) $60^{\circ} = 60\left(\frac{\pi}{180}\right) = \frac{\pi}{3}$ rad

  (b) $\frac{\pi}{6} = \left(\frac{\pi}{6}\right)\frac{180^{\circ}}{\pi} = 30^{\circ}$

Arc Length

• $s = r\theta$
  • $s$ is the arc length
  • $\theta$ is the angle in radians
  • $r$ is the radius of the circle

Arc Length Example

• To find the length of an arc of a circle with radius $10 \text{ m}$ that subtends a central angle of $30^{\circ}$:
• $30^{\circ} = 30\left(\frac{\pi}{180}\right) = \frac{\pi}{6}$
• $s = r\theta = 10\left(\frac{\pi}{6}\right) = \frac{5\pi}{3} \text{ m}$

Area of a Sector

• $A = \frac{1}{2}r^2\theta$
  • $A$ is the area of a sector
  • $r$ is the radius of the circle
  • $\theta$ is the angle in radians

Area of a Sector Example

• To find the area of a sector of a circle with central angle $60^{\circ}$ and radius $3 \text{ m}$:
• $A = \frac{1}{2}(3)^2\left(\frac{\pi}{3}\right) = \frac{3\pi}{2} \text{ m}^2$

Angular Speed

• $\omega = \frac{\Delta \theta}{\Delta t}$
  • $\omega$ is the angular speed
  • $\Delta \theta$ is the change in angle (in radians)
  • $\Delta t$ is the change in time

Linear Speed

• $v = r\omega$
  • $v$ is the linear speed
  • $r$ is the radius
  • $\omega$ is the angular speed

Machine Learning

Topics

• Introduction
• Supervised Learning
  • Linear Regression
  • Classification
    • Logistic Regression
    • Support Vector Machines
• Unsupervised Learning
  • Clustering
    • K-means
    • Hierarchical clustering
  • Dimensionality Reduction
    • Principal Component Analysis
• Model Selection and Evaluation
  • Cross-Validation
  • Performance Metrics
• Deep Learning
  • Neural Networks
  • Convolutional Neural Networks
• Appendix: Python Libraries for Machine Learning
  • Scikit-learn
  • Tensorflow
  • Keras