Kalman & Particle Filters

Lecture 12: October 26, 2023 - Review

• Kalman Filter combines Bayes filter with linear Gaussian assumptions.
• EKF approximates non-linear functions using first-order Taylor expansion (linearization) and utilizes Jacobian matrices like $F_t = \frac{\partial f}{\partial x_{t-1}}|{x{t-1}=\mu_{t-1}, u_t}$ and $H_t = \frac{\partial h}{\partial x_{t}}|{x{t}=\bar{\mu}_{t}}$.
• UKF approximates distributions using non-parametric samples (sigma points).
• UT transform propagates sigma points through functions.

Particle Filters (Monte Carlo Localization)

• The particle filter represents the posterior $p(x_t | z_{1:t}, u_{1:t})$ using a set of $N$ samples (particles) shown as $X_t = {x_t^{(1)},..., x_t^{(N)}}$, where each particle is a state hypothesis.
• Monte Carlo Localization (MCL) is a common localization algorithm.
• Key idea: "survival of the fittest".
• Algorithm comprises prediction, weighting, and resampling steps.
• Prediction: new poses are sampled from the motion model $x_t^{(i)} \sim p(x_t | x_{t-1}^{(i)}, u_t)$.
• Weighting: particles are weighted by observation likelihood $w_t^{(i)} = p(z_t | x_t^{(i)})$.
• Resampling: samples are selected from the particle set with replacement, based on their weights, using $X_t = Resample(X_t, w_t)$.
• Bayes Filter Recap: involves prediction and correction steps:
  • Prediction: $\overline{bel}(x_t) = \int p(x_t | u_t, x_{t-1}) bel(x_{t-1}) dx_{t-1}$.
  • Correction: $bel(x_t) = \eta \cdot p(z_t | x_t) \overline{bel}(x_t)$.

Particle Filter Algorithm

• The algorithm initializes with a particle set $X_{t-1}$, control input $u_t$, and observation $z_t$.
• An empty particle set $X_t$ is created.
• For each particle $i$ from $1$ to $N$, a new state $x't[i]$ is sampled from $p(x_t | u_t, x{t-1}[i])$ and assigned a weight $w'_t[i] = p(z_t | x'_t[i])$.
• Each $x'_t[i]$ is added to $X_t$ with probability proportional to $\alpha w'_t[i]$.

Motion Model

• The motion model calculations are:
  • $x' = x + u_t^1 + \epsilon_{\alpha_1 u_t^1 + \alpha_2 u_t^2}$.
  • $y' = y + u_t^1 + \epsilon_{\alpha_1 u_t^1 + \alpha_2 u_t^2}$.
  • $\theta' = \theta + u_t^2 + \epsilon_{\alpha_3 u_t^1 + \alpha_4 u_t^2}$.
• $u_t = (u_t^1, u_t^2)$ represents control inputs.

Measurement Model

• Measurement Model: $p(z_t | x_t) = \prod_{k=1}^K p(z_t^k | x_t)$.
• $z_t = (z_t^1,..., z_t^K)$.

Resampling

• After iterations, particles become homogeneous.
• Resampling creates a new particle set from the old set with probability relative to importance weights.
• High-weight particles get sampled more; low-weight particles are discarded.
• Resampling types: low variance, residual, stratified, systematic, and multinomial, all $O(N)$.

Low Variance Resampling Algorithm

• New particle set $X'$ is created.
• Random number $r$ from $(0, 1/N)$ is generated.
• $c$ is initialized to the weight $w[0]$, and $i$ is set to 0.
• For $m$ from 1 to $N$, $U = r + (m-1)/N$ is calculated. As long as $U > c$, increment $i$ and update $c = c + w[i]$. Add $X[i]$ to $X'$.

Sensor Deprivation Problem

• Occurs when the robot is moved to a new location, and all particles are far from the true pose.
• Adding random poses resolves this issue.

Augmented MCL

• Algorithm extends MCL with parameters $\alpha_{slow}$ and $\alpha_{fast}$.
• For each particle $i$ from $1$ to $N$, a new state $x't[i]$ is sampled from $p(x_t | u_t, x{t-1}[i])$ and assigned a weight $w'_t[i] = p(z_t | x'_t[i])$.
• Weighted average $w_{avg} = 1/N * \Sigma w'_t[i]$ is calculated.
• For each particle $i$, a random pose is added to $X_t$ with probability $\max(0, 1 - w_{fast} / w_{slow})$; otherwise, $x'_t[i]$ is added with a probability proportional to $w'_t[i]$.
• $w_{slow}$ is the long-term average weight; $w_{fast}$ is the short-term average weight.

Lecture 24: The Fourier Transform - Introduction

• The Fourier transform is a fundamental tool for analyzing functions in terms of their frequency components.
• Fourier Series represent periodic functions as a sum of sines and cosines.
• Fourier Transform extends this to non-periodic functions on $\mathbb{R}$.
• The key idea is to decompose functions into their constituent frequencies.

Definition of the Fourier Transform

• For a function $f: \mathbb{R} \rightarrow \mathbb{C}$ or $\mathbb{R}$, the Fourier transform is defined as $\hat{f}(\xi) = \int_{-\infty}^{\infty} f(x) e^{-2\pi i x \xi} dx, \quad \xi \in \mathbb{R}$
• $\xi$ signifies the frequency.
• $e^{-2\pi i x \xi}$ is a complex exponential.
• $\hat{f}(\xi)$ measures amplitude and phase of frequency $\xi$ in function $f(x)$.

Examples

• Gaussian Function: For $f(x) = e^{-\pi x^2}$, the Fourier transform is $\hat{f}(\xi) = e^{-\pi \xi^2}$
• Fourier transform of a Gaussian results in another Gaussian function.
• Rectangular Function: For $f(x) = 1$ if $|x| \leq \frac{1}{2}$ else $0$, the Fourier transform is $\hat{f}(\xi) = \frac{\sin(\pi \xi)}{\pi \xi} = \text{sinc}(\xi)$.
• Fourier transform of a rectangular function is a sinc function.

Properties of the Fourier Transform

• Linearity: $\widehat{(af + bg)}(\xi) = a\hat{f}(\xi) + b\hat{g}(\xi)$
• Translation: If $g(x) = f(x - a)$, then $\hat{g}(\xi) = e^{-2\pi i a \xi}\hat{f}(\xi)$
• Modulation: If $g(x) = e^{2\pi i a x} f(x)$, then $\hat{g}(\xi) = \hat{f}(\xi - a)$
• Scaling: If $g(x) = f(ax)$, then $\hat{g}(\xi) = \frac{1}{|a|} \hat{f}\left(\frac{\xi}{a}\right)$
• Differentiation: If $g(x) = f'(x)$, then $\hat{g}(\xi) = 2\pi i \xi \hat{f}(\xi)$
• Convolution Theorem: For $(f * g)(x) = \int_{-\infty}^{\infty} f(y)g(x - y) dy$, then $\widehat{(f * g)}(\xi) = \hat{f}(\xi) \hat{g}(\xi)$

The Inverse Fourier Transform

• Defined as $f(x) = \int_{-\infty}^{\infty} \hat{f}(\xi) e^{2\pi i x \xi} d\xi$.
• Allows recovery of the original function from its Fourier transform.

Plancherel's Theorem

• States that $\int_{-\infty}^{\infty} |f(x)|^2 dx = \int_{-\infty}^{\infty} |\hat{f}(\xi)|^2 d\xi$. -Energy of the function is preserved under the Fourier transform.

Applications

• Signal Processing: noise reduction and unwanted filtering.
• Image Processing: edge detection and enhancement.
• Physics: quantum mechanics, momentum space representation, wave propagation.
• Data Analysis: spectral analysis.

Lecture 10: October 24, 2023 - Floating Point Numbers

• Intuitive representation is accomplished using one bit to represent the sign and remaining bits as the magnitude.
• Representing a number using sign and magnitude has the following equation: $(-1)^S \times M$. Where $S$ is the sign bit and $M$ is the magnitude.
• With $n$ bits, numbers from $-(2^{n-1} - 1)$ to $(2^{n-1} - 1)$ can be represented.
• Addition and subtraction require different hardware, and there are two representations of zero: +0 and -0.

Biased Exponent

• A biased exponent can represent fractions.
• The exponent represents fractions by adding a constant to it (biased) in order that the smallest exponent have a value of 0.
• To represent a number, use the equation: $(-1)^S \times M \times 2^{E-bias}$, where $S$ is the sign bit, $M$ is the mantissa, $E$ is the exponent, and $bias$ is the bias.

IEEE 754 Floating Point Standard

• The IEEE 754 standard defines how floating point numbers should be represented.
• The standard also dictates how to perform arithmetic on them.
• Single precision (32 bits) and double precision (64 bits) are the two main formats.
• Single Precision (32 bits)
  • Sign: 1 bit.
  • Exponent: 8 bits.
  • Mantissa: 23 bits.
  • Representing a number in single precision uses the equation: $(-1)^S \times (1.M) \times 2^{(E-127)}$.
• Double Precision (64 bits)
  • Sign: 1 bit.
  • Exponent: 11 bits.
  • Mantissa: 52 bits.
  • Representing a number in double precision uses the equation: $(-1)^S \times (1.M) \times 2^{(E-1023)}$.
• Special Values
  • Zero: Exponent = 0, Mantissa = 0.
  • Infinity: Exponent = all 1s, Mantissa = 0.
  • NaN: Exponent = all 1s, Mantissa != 0.

Floating Point Arithmetic

• To perform addition, you must first extract the sign, exponent and mantissa.
• Adjust mantissa of number with the smaller value for equal exponents.
• Add the mantissas.
• Normalize, round, and determine the sign of the result.

Multiplication

• To perform multiplication, you must first extract the sign, exponent and mantissa.
• Multiply the mantissas.
• Add the exponents.
• Normalize, round, and determine the sign of the result.

Capítulo 1: Introducción - ¿Qué es la economía?

• Economía studies how societies use scarce resources to produce valuable goods.
• Escasez (scarcity) is the condition where goods are limited relative to desires.
• Analiza la producción (analyzes production), distribución (distribution), y consumo de bienes y servicios (consumption of goods and services).

Dos ramas clave (Two key branches):

• Microeconomía focuses on individual entities like markets and firms.
• Macroeconomía examines the overall performance of the economy, including inflation and unemployment.

Recursos y Posibilidades (Resources and Possibilities)

• Factores de Producción (Factors of Production)
  • Tierra/Land: Natural resources such as soil and minerals.
  • Trabajo/Labor: Human effort in production.
  • Capital: Durable goods generated to produce other goods.
• Frontera de Posibilidades de Producción (FPP) / Production Possibility Frontier (PPF): Quantities of maximum goods and services can be produced using available resources and technology.
  • Costs of scarcity, efficiency, and opportunity.
  • Costo de opportunidad/Opportunity cost: Value of good or service by choosing alternative.

Mecanismos de Mercado y Sectores Económicos (Market Mechanisms and Economic Sectors)

• Economía de Mercado (Market Economy)
  • Prices guide descentralized decisions.
  • Prices coodinate the decision of consumers and producers.
• Economía Centralizada (Centralized Economy)
  • Decisions for the economy are made by the government.
• Economía Mixta (Mixed Economy)
  • Combo of elements from market economy and centralized economy.
• Sectores Económicos (Economic Sectors)
  • Sector Privado/Private Sector: Non-government companies and organizations.
  • Sector Público (Public Sector): Government and state entities.

Como Estudian los Economistas (How economists Study)

• Enfoque Científico (Scientific Approach)
  • Observe and measure economic phenomena.
  • Formulate models and theories.
  • Empirical evidence.
• Falacias Comunes (Common Fallacies)
  • Falacia Post Hoc: Causalidad because one event happens before the other one.
  • Fracaso en mantener otros factores constantes (Ceteris Paribus): Does not consider other factors influencing results.
  • Falacia de la Composicion: what happens to a part it applies to the whole.

Tipos de Declaraciones (Types of Declarations)

• Afirmaciones Positivas: Describe relationships and facts to use as evidence.
• Afirmaciones Normativas: express value and opinion.

Gráficos en Economía ( Graphs in Economy)

• Uso de Graficos ( Graphs use)
  • Representations of data.
  • Comprehension.
  • Diagramas de dispersion, series de tiempo y diagramas de pastel ( common types graphs)
• Pendiente y Elasticidad ( slope and elasticity)
  • Pendiente: measures change in variable in relation to another.
  • Elasticidad: variable sensitivity due to changes in antoher.

Bernoulli's Principle - Explanation

• An increase in fluid speed happens simultaneously with a drop in pressure or in potential energy.
• It seems like faster-moving fluids should exert more pressure, but not what happens.
• The fluid speeds up to conserver mass flow rate, but it loses pressure.

Applications of Bernoulli's Principle

• Airplanes
  • Airplanes use Bernoulli's Principle.
  • Airplane wings are designed so air moves faster above the wing than below it.
  • The result is that pressure above the wing is lower than pressure below the wing, creating a net upward force called lift.
  • $Lift = (P_{bottom} - P_{top}) \cdot Area$.
• Carburetors
  • Carburetors use this principle to mix air and fuel in an engine.
  • Intake air is forced through a narrow passage, increasing speed.
  • As air speeds up, pressure decreases, drawing fuel into the airstream.
• Chimneys
  • Chimneys work better when windy.
  • Wind blowing across the top of the chimney reduces pressure.
  • Lower pressure at the top draws combustion gases up the chimney more efficiently.

Bernoulli's Equation

• Bernoulli's equation mathematically expresses this principle relating pressure, velocity, and height of a fluid between two points in a streamline.
  • $P_1 + \frac{1}{2} \rho v_1^2 + \rho g h_1 = P_2 + \frac{1}{2} \rho v_2^2 + \rho g h_2$
  • $P$ is the fluid pressure.
  • $\rho$ is the fluid density.
  • $v$ is the fluid velocity.
  • $g$ is the acceleration due to gravity.
  • $h$ is the height of the fluid above a reference point.

Assumptions

• Fluid is incompressible (density is constant).
• Flow is steady (velocity doesn't change with time).
• Flow is inviscid (no internal friction).
• Equation is applied along a streamline.

Funciones vectoriales de variable escalar - Introducción (Vector-valued functions of a single real variable)

• Motivación:
• Descripción del movimiento de una partícula en el espacio (Description of the motion of a particle in space).
• Curvas y superficies en el espacio (Space curves and surfaces).
• Definición:
• Una función vectorial de variable escalar es una función que asigna a cada número real $$t$$ un vector en el espacio (A vector-valued function of a single real variable is a function that assigns a vector is space to each real number).
• Dominio:
• El dominio de r(t) es el conjunto de todos los valores de t para los cuales r(t) está definido (The domain of r(t) is the set of all values of t where r(t) is defined).
• Gráfica:
• La gráfica de una función vectorial de variable escalar es una curva en el espacio (The graph of a vector-valued function of a scalar variable is a curve in space).

Operaciones con funciones vectoriales

• r(t):+(t) = (f1::(t):+f2:(t), g1:(t):+g2:(t), h1:(t):+h2:(t))
• cr(t) = (cf1:(t), cg1:(t), ch1:(t))
• f(t)r(t) = (f(t)f1:(t), f(t)g1:(t), f(t)h1:(t))
• r(t) * s(t) = f1:(t)f2:(t) + g1:(t)g2:(t) + h1:(t)h2:(t)
• r(t) * s(t) = (g1:(t)h2:(t) - h1:(t)g2:(t), h1:(t)f2:(t) - f1:(t)h2:(t), f1:(t)g2:(t) - g1:(t)f2:(t))

Límite y continuidad (Limits and Continuity)

• Límite:
• El límite de una función vectorial r**(t) cuando t tiende a t0 es el vector L tal que cada una de sus componentes es el límite de la componente correspondiente de r(t) (The limit of a vector-valued function r**(t) when t tends to t0 is the vector L such that each component is the limit of the corresponding component of r(t)).
• Continuidad:
• Una función vectorial r**(t) es continua en t : t0 si (A vector-valued function r(t) is continuous at t : t**0 if):
• r**(t0) está definido (r(t**0) is defined.)
• lim(t->t:0:) r**(t) existe (lim(t->t:0:) r**(t) exists.)
• lim(t->t:0:) r**(t) = r**(t**0).

Derivada (Derivative)

• Definición:
• La derivada de una función vectorial r̄(t) se define como r̄′(t)=lim:h→0: \frac(\overrightarrow{r}(t+h) - \overrightarrow{r}(t)}{h}
• siempre y cuando este límite exista (provided that the limit exists).
• Cálculo de la derivada (Calculating Derivatives)
• Si r(t) = (f(t), g(t), h(t)), entonces
• r̄′(t)=(f′(t),g′(t),h′(t))
• siempre y cuando las derivadas de las componentes existan (provided that the limit exists).
• Interpretación geométrica (Geometric Interpretation)
• r̄′(t)** es un vector tangente a la curva descrita por r̄**(t) en el punto r̄**(t) (r̄′(t) is tangent to the curve).
• Vector tangente unitario (Unit tangent vector)
• El vector tangente unitario T**(t** se define como : T̄**(t** = r̄′(t:** / ‖r̄′(t:‖
• • Siempre que r̄′(t: ≠0**

Integral

• Definición:
• La integral de una función vectorial r**(t** se define como
• ∫r̄(t: dt= (∫f(t:dt,∫g(t:dt, h(t dt)
• donde r(t: = (f(t:, g(t:, h(t:(.
• Integral definida (Definite Integral):

Fourier Transform Properties

• Linearity: $a \cdot f(t) + b \cdot g(t) \Leftrightarrow a \cdot F(f) + b \cdot G(f)$.
• Time Scaling: $f(at) \Leftrightarrow \frac{1}{|a|}F(\frac{f}{a})$.
• Time Shifting: $f(t - t_0) \Leftrightarrow e^{-j2\pi ft_0}F(f)$.
• Frequency Shifting: $e^{j2\pi f_0t}f(t) \Leftrightarrow F(f - f_0)$.
• Multiplication: $f(t) \cdot g(t) \Leftrightarrow F(f) * G(f)$.
• Convolution: $f(t) * g(t) \Leftrightarrow F(f) \cdot G(f)$.
• Differentiation: $\frac{d}{dt}f(t) \Leftrightarrow j2\pi fF(f)$.
• Integration: $\int_{-\infty}^{t} f(\tau) d\tau \Leftrightarrow \frac{1}{j2\pi f}F(f) + \frac{1}{2}F(0)\delta(f)$.
• Area: $\int_{-\infty}^{\infty} f(t) dt = F(0)$.
• Duality: $F(t) \Leftrightarrow f(-f)$.
  • Where
    • $f(t)$: Signal in the time domain.
    • $F(f)$: Signal in the frequency domain.
    • a, b, $t_0$, $f_0$: constant.
    • *: convolution.
    • $\delta$: delta function.

PLANCK'S LAW

• Planck's law describes the spectral density of electromagnetic radiation emitted by a black body in thermal equilibrium at a given temperature $T$.
• It is a fundamental concept in quantum mechanics.
• $B(\nu, T) = \frac{2h\nu^3}{c^2} \frac{1}{e^{\frac{h\nu}{kT}}-1}$

Implications

• It laid the foundation for the development of quantum mechanics.
• It is used in various fields, including astrophysics, thermal engineering, and lighting.
• It allows scientists to determine the temperature of distant stars by analyzing the spectrum of light they emit.
• The solar spectrum is close to that of a black body with a temperature of about 5,800 K.

Lecture 16 - Channel Capacity

• The channel capacity of a continuous input, and continuous output channel's defined as $C = max[I(X; Y)]$ bits per transmission where the maximization is over all allowed probability distribution $p(x)$.

• Additive White Gaussian Noise (AWGN) Channel

  • $Y_i = X_i + Z_i$, where
    • $X_i$ is the channel input at time i.
    • $Y_i$ is the channel output at time i.
    • $Z_i$ is a zero mean Gaussian RV with variance N (noise power).
    • Assume that the signal has average power constraint P, where $E[X^2] \leq P$. and $ I(X; Y) = h(Y) - h(Y|X) = \frac{1}{2}log[2\pi e(P + N)] - \frac{1}{2}log[2\pi eN] = \frac{1}{2}log[\frac{P + N}{N}]$

• Bandwidth & Power: if the bandwidth is W Hz, then we can make 2W independent transmissions per second , $C = W log[1 + \frac{P}{N}]$ bits per second, this is the Shannon-Hartley channel capacity theorem.

• Capacity of the Bandlimited Gaussian Channel $C = \sum_{i=1}^{n}\frac{1}{2}log(1 + \frac{P_i}{N_0W})$ $\sum P_i = P \rightarrow$ Using Lagrange multipliers method $P_i = \frac{1}{2\lambda} - N_0W \rightarrow $ We should assign power to only those channels for which $\rightarrow B > N_0W$ This is called "water-filling".

Algorithmic complexity

• Algorithmic complexity measures the time (time complexity) and memory (space complexity) an algorithm needs to solve a problem.
• Big O Notation describes the upper bound of an algorithm's complexity in regards to worst-case scenarios.

Common complexities

• $O(1)$: Constant time.
• $O(log n)$: Logarithmic time.
• $O(n)$: Linear time.
• $O(n log n)$: Linearithmic time.
• $O(n^2)$: Quadratic time.
• $O(2^n)$: Exponential time.
• $O(n!)$: Factorial time.

Space complexity

Describes how much memory an algorithm needs for a given size of input data.

Factors

• Input data size
• Auxiliary space: additional memory used by the algorothim

Common space complexities

• O(1): Constant space. Doesn't depend on the size of the input data
• O(n): Linear space. Memory usage increases linearly with the size of the input data.
• O(n2): Memory usage increases quadratically with the size of the input data.