Linear Algebra & Fourier Transform

Algèbre Linéaire (Bernard Gostiaux)

• This book provides a clear and concise course in linear algebra with application exercises and problems.
• The book targets university students in scientific fields and candidates for teaching recruitment competitions.
• Numerous examples illustrate abstract concepts.
• Each chapter ends with a summary of key results.
• Topics covered include vector spaces, linear applications, matrices, determinants, linear equation systems, endomorphism reduction, Euclidean vector spaces, bilinear and quadratic forms, and affine spaces.

Fourier Transform Properties

• The Fourier Transform is a mathematical tool used to analyze the frequency components of a signal.
• $F(f)$ represents the Fourier transform of the function $f(t)$.
• These properties enable the simplification and analysis of signals in both the time and frequency domains.

Linearity

• A linear combination of functions transforms into the same linear combination of their individual transforms: $\mathcal{F}[af(t) + bg(t)] = aF(f) + bG(f)$.

Time Scaling

• Scaling in the time domain results in inverse scaling in the frequency domain: $\mathcal{F}[f(at)] = \frac{1}{|a|}F(\frac{f}{a})$.
  • Compression in time implies expansion in frequency.

Time Shifting

• A shift in time corresponds to a linear phase shift in the frequency domain: $\mathcal{F}[f(t - t_0)] = e^{-j2\pi ft_0}F(f)$.

Frequency Shifting

• Multiplying by a complex exponential in the time domain shifts the spectrum in the frequency domain: $\mathcal{F}[e^{j2\pi f_0t}f(t)] = F(f - f_0)$.
  • This is also called modulation.

Conjugation

• Taking the complex conjugate in the time domain flips and conjugates the spectrum: $\mathcal{F}[f^(t)] = F^(-f)$.

Convolution

• Convolution in one domain corresponds to multiplication in the other domain and vice-versa: $\mathcal{F}[f(t) * g(t)] = F(f)G(f)$ and $\mathcal{F}[f(t)g(t)] = F(f) * G(f)$.

Differentiation

• Differentiation in the time domain is equivalent to multiplication by $j2\pi f$ in the frequency domain: $\mathcal{F}[\frac{d}{dt}f(t)] = (j2\pi f)F(f)$ and $\mathcal{F}[\frac{d^n}{dt^n}f(t)] = (j2\pi f)^nF(f)$.

Integration

• Integration in the time domain involves division by $j2\pi f$ in the frequency domain and a delta function term: $\mathcal{F}[\int_{-\infty}^{t} f(\tau)d(\tau)] = \frac{1}{j2\pi f}F(f) + \frac{1}{2}F(0)\delta(f)$.

Chemical Kinetics

• Chemical kinetics studies reaction rates, how they change with conditions, and reaction mechanisms.
• It includes experiments to see how conditions influence reaction speed, yielding information on reaction mechanisms and transition states.
• Construction of mathematical models that describe a chemical reaction's characteristics are also used.

Factors Affecting Reaction Rate

• Reactant concentration: Higher concentrations often increase the reaction rate due to more frequent collisions between reactant molecules.
• Physical state and surface area: Reactions are faster when reactants are in the same phase or when there is a larger surface area of contact in heterogeneous reactions.
• Temperature: Higher temperature usually increases reaction rate by providing more energy and increasing collision frequency with sufficient energy.
• Presence of a catalyst: Catalysts speed up reactions by providing alternative pathways with lower activation energy.
• Pressure: For gaseous reactants, increasing pressure increases concentration and, therefore, the reaction rate.
• Light: Provides the activation energy for photochemical reactions.

Reaction Rate

• For the generic reaction $aA + bB \rightarrow cC + dD$, the rate is expressed as: Rate $= -\frac{1}{a}\frac{\Delta [A]}{\Delta t} = -\frac{1}{b}\frac{\Delta [B]}{\Delta t} = \frac{1}{c}\frac{\Delta [C]}{\Delta t} = \frac{1}{d}\frac{\Delta [D]}{\Delta t}$ where a, b, c, and d are stoichiometric coefficients.
• The rate is always positive and typically expressed in units of M/s (molar per second).

Rate Laws

• A rate law relates the rate to reactant and catalyst concentrations: Rate = $k[A]^m[B]^n$
  • k is the rate constant, which is specific to the reaction and temperature-dependent.
  • [A] and [B] are reactant concentrations.
  • m and n are reaction orders with respect to A and B, determined experimentally.
• Overall Reaction Order: The sum of the individual orders (m + n).

Reaction Order

• Defines the relationship between reactant concentration and reaction rate.
  • Zero Order: Rate = k (independent of reactant concentration).
  • First Order: Rate = k[A] (directly proportional to [A]).
  • Second Order: Rate = $k[A]^2$ or Rate = k[A][B] (proportional to the square of [A] or the product of [A] and [B]).

Teorema de Bayes (Bayes' Theorem)

• Bayes' Theorem describes the probability of an event, based on prior knowledge of conditions related to the event.
• Formula: $P(A|B) = \frac{P(B|A)P(A)}{P(B)}$.
  • $P(A|B)$: Conditional probability of A given B.
  • $P(B|A)$: Conditional probability of B given A.
  • $P(A)$ and $P(B)$: Probabilities of A and B independently.

Deduction

• From conditional probability: $P(A|B) = \frac{P(A \cap B)}{P(B)}$ and $P(B|A) = \frac{P(B \cap A)}{P(A)}$.
• Since $P(A \cap B) = P(B \cap A)$: $P(A|B)P(B) = P(B|A)P(A)$.
• Dividing by $P(B)$ yields Bayes' Theorem: $P(A|B) = \frac{P(B|A)P(A)}{P(B)}$.

Exemplo (Example)

• Assume a rare disease affects 1 in 10,000 and a test is 99% accurate.
  • $D$: Person has the disease.
  • $T$: Test is positive.
• Given probabilities:
  • $P(D) = \frac{1}{10000} = 0.0001$.
  • $P(T|D) = 0.99$.
  • $P(\neg T|\neg D) = 0.99$.

Calculation of $P(T)$

• $P(T) = P(T|D)P(D) + P(T|\neg D)P(\neg D)$.
  • $P(\neg D) = 1 - 0.0001 = 0.9999$.
  • $P(T|\neg D) = 1 - 0.99 = 0.01$.
  • $P(T) = (0.99 \times 0.0001) + (0.01 \times 0.9999) = 0.010098$.
• Result: $P(D|T) = \frac{0.99 \times 0.0001}{0.010098} \approx 0.0098039 \approx 0.98%$.
• Even with a 99% accurate test, the likelihood of having the disease after a positive test is low because of the disease's rarity, indicating many false positives.

Descriptions spectrales des couleurs (Spectral Color Descriptions)

Stimulus de couleur (Color Stimulus)

• Light that enters the eye and causes a sensation of color.
• It can be described by its spectral power distribution (SPD), denoted as $S(\lambda)$, which dictates the amount of power for each visible wavelength ($\lambda$) between 380 nm and 780 nm.

Fonction de réflectance spectrale (Spectral Reflectance Function)

• An object's spectral reflectance function, $R(\lambda)$, indicates the proportion of light that is reflected by the object at each wavelength.
• It is defined as the ratio of light reflected to incident light.

Illuminant

• A light source categorized by its spectral power distribution (SPD), $E(\lambda)$.
• Describes the amount of light emitted per wavelength.

Calcul dy stimulus de couleur (Color Stimulus Calculation)

• The color stimulus, $C(\lambda)$, from lighting an object is the product of the illuminant and the object's spectral reflectance: $C(\lambda) = E(\lambda) \cdot R(\lambda)$.

Métamérisme (Metamerism)

• Occurs when two colors appear to match under certain lighting conditions but not others.
• SPDs of the two colors might be different, but they yield the same trichromatic response in certain scenarios.

Indice de rendu des couleurs (IRC) - (Color Rendering Index (CRI))

• Gauges the ability of a light source to reproduce object colors faithfully in comparison to a reference light source.
• A high CRI suggests the light source renders colors accurately.

Espaces colorimétriques (Color Spaces)

Espace colorimétrique CIE XYZ (CIE XYZ Color Space)

• A standard developed by the CIE (International Commission on Illumination) in 1931.
• Based on color matching functions $\bar{x}(\lambda)$, $\bar{y}(\lambda)$, and $\bar{z}(\lambda)$
• These functions represent an average human observer's response to light at various wavelengths, used to calculate tristimulus values: $X = k \int_{380}^{780} C(\lambda) \bar{x}(\lambda) d\lambda$, $Y = k \int_{380}^{780} C(\lambda) \bar{y}(\lambda) d\lambda$, and $Z = k \int_{380}^{780} C(\lambda) \bar{z}(\lambda) d\lambda$, where $Y$ is luminance and $k$ is a normalization factor.

Chromaticité CIE xy (CIE xy Chromaticity)

• Derived from the XYZ tristimulus values: $x = \frac{X}{X + Y + Z}$, $y = \frac{Y}{X + Y + Z}$.
• Defines the color.
• The CIE xy chromaticity diagram is a 2D representation of all colors.

Espace colorimétrique CIE L*a*b* (CIE L*a*b* Color Space)

• A perceptually uniform color space, developed in 1976.
• Created by transforming CIE XYZ tristimulus values
• Represents color with three coordinates: $L^$ (lightness), $a^$ (green-red), and $b^$ (blue-yellow). Calculated using: $L^ = 116 \cdot f(Y/Y_n) - 16$, $a^* = 500 \cdot [f(X/X_n) - f(Y/Y_n)]$, $b^* = 200 \cdot [f(Y/Y_n) - f(Z/Z_n)]$. With reference white tristimulus, function $f(t)$ and $f(t) = \begin{cases} t^{1/3} & \text{if } t > (6/29)^3 \ (1/3)(29/6)^2 t + 4/29 & \text{otherwise} \end{cases}$

Espace colorimétrique CIE L*C*h* (CIE L*C*h* Color Space)

• A color space related to CIE L*a*b*, using lightness ($L^*$)
• Polat Coordinatees $C^$ (chroma) is given by $C^ = \sqrt{(a^)^2 + (b^)^2}$ and $h$ (hue), $h = \arctan(\frac{b^}{a^})$

Différence de couleur (Color difference)

Formule $\Delta E^*$ (Delta E Star formula)

• Measures the color difference as the Euclidean distance between the two colors in color space.
• $\Delta E^_{ab} = \sqrt{(\Delta L^)^2 + (\Delta a^)^2 + (\Delta b^)^2}$ commonly utilized in CIE L*a*b*.

Formule $\Delta E^{*}_{00}$ (Delta E Star 00 formula)

• Is a more precise and complex calculation, factoring in lightness, chroma, and hue: $\Delta E_{00}^* = \sqrt{(\frac{\Delta L'}{k_L S_L})^2 + (\frac{\Delta C'}{k_C S_C})^2 + (\frac{\Delta H'}{k_H S_H})^2 + R_T (\frac{\Delta C'}{k_C S_C}) (\frac{\Delta H'}{k_H S_H})}$
• Considers weightings and correction parameters for perceptual effects.

Comparaison des algorithmes de tri (Comparison of Sorting Algorithms)

• Various sorting algorithms have different time complexities, which affect their performance, especially on large datasets.

Complexity Table

Explanation of Algorithms

• Quicksort: A divide-and-conquer algorithm that partitions a list around a pivot element and recursively sorts the sublists.
• Merge Sort: A divide-and-conquer algorithm that divides the list into halves, sorts each half recursively, and then merges the sorted halves.
• Heap Sort: A comparison-based sorting algorithm that uses a heap data structure.
• Insertion Sort: Builds the final sorted array one item at a time by inserting elements into the correct position among those already sorted.
• Bubble Sort: Repeatedly steps through the list, compares adjacent elements, and swaps them if they are in the wrong order.
• Selection Sort: Divides the list into a sorted and unsorted part, iteratively finding the smallest element in the unsorted part and moving it to the sorted part.

Motion in Two and Three Dimensions

Position and Velocity Vector

• Position Vector: $\vec{r} = x\hat{i} + y\hat{j} + z\hat{k}$
• Displacement: $\Delta\vec{r} = \vec{r_2} - \vec{r_1} = (x_2 - x_1)\hat{i} + (y_2 - y_1)\hat{j} + (z_2 - z_1)\hat{k}$
• Average Velocity: $\vec{v}_{avg} = \frac{\Delta\vec{r}}{\Delta t} = \frac{\vec{r_2} - \vec{r_1}}{t_2 - t_1}$
• Instantaneous Velocity: $\vec{v} = \lim_{\Delta t \to 0} \frac{\Delta\vec{r}}{\Delta t} = \frac{d\vec{r}}{dt} = \frac{dx}{dt}\hat{i} + \frac{dy}{dt}\hat{j} + \frac{dz}{dt}\hat{k} = v_x\hat{i} + v_y\hat{j} + v_z\hat{k}$

Acceleration Vector

• Average Acceleration: $\vec{a}_{avg} = \frac{\vec{v_2} - \vec{v_1}}{t_2 - t_1} = \frac{\Delta\vec{v}}{\Delta t}$
• Instantaneous Acceleration: $\vec{a} = \lim_{\Delta t \to 0} \frac{\Delta\vec{v}}{\Delta t} = \frac{d\vec{v}}{dt} = \frac{dv_x}{dt}\hat{i} + \frac{dv_y}{dt}\hat{j} + \frac{dv_z}{dt}\hat{k} = a_x\hat{i} + a_y\hat{j} + a_z\hat{k}$

Projectile Motion

• Assuming $a_x = 0$ and $a_y = -g$:
  • $v_x = v_{0x}$
  • $v_y = v_{0y} - gt$
  • $x = x_0 + v_{0x}t$
  • $y = y_0 + v_{0y}t - \frac{1}{2}gt^2$
• Horizontal Range: $R = \frac{v_0^2}{g}\sin2\theta_0$

Uniform Circular Motion

• Period: $T = \frac{2\pi r}{v}$
• Radial Acceleration: $a = \frac{v^2}{r}$
• Frequency: $f = \frac{1}{T}$

Relative Motion

• Relative Positions: $\vec{r}{PA} = \vec{r}{PB} + \vec{r}_{BA}$
• Relative Velocities: $\vec{v}{PA} = \vec{v}{PB} + \vec{v}_{BA}$

Lecture 14: Channel Capacity

Channel Capacity

• The maximum rate at which information can be reliably transmitted over a channel.
• Defined as Shannon Capacity: $C = \max_{p(x)} I(X; Y)$ where the maximization is over input distributions.

Example: Noiseless Binary Channel

• Capacity is 1 bit (output is exactly the input).

Example: Noisy Channel with Non-Overlapping outputs

• Capacity is 1 bit if $p > 0$ (the receive knows the input).

Example: Noisy Typewriter

• Capacity is $log \ 26 - 1 = log \ 13$ bits (output is either the input or the next letter).

Example: Binary Symmetric Channel (BSC)

• A binary symmetric channel (BSC) channel where $C+log_{2}(1+S/N)$.
  • C is the channel capacity (bits per second).
  • B is the bandwidth of the channel (Hz).
  • S is average receive signal power (Watts).
  • N is the average noise or interference power (Watts).
  • S/N os the signal to noise ratio.

Algorithmes de tri (Sorting Algorithms)

Tri par insertion (Insertion Sort)

• Principle: Iterate through the array, inserting each element into its correct position among the sorted preceding elements.
• Complexity: $O(n^2)$ (quadratic).

Example

• The list gets sorted item by item

5 2 4 6 1 3   // initial array
2 5 4 6 1 3   // insert 2 before 5
2 4 5 6 1 3   // insert 4 before 5
2 4 5 6 1 3   // 6 is already in place
1 2 4 5 6 3   // insert 1 at the beginning
1 2 3 4 5 6   // insert 3 in the middle

Tri rapide (Quicksort)

• Principle: a divide and conquer sort where sub partitions are sorted in place
• Divide: partition the list around a pivot element.

Example

> [2, 5, 8, 1, 9, 4, 7, 6, 3]
> [2, 1, 3] 4 [8, 9, 7, 6, 5]
> [1, 2, 3] 4 [5, 6, 7, 8, 9]

• Complexity: $O(n \log n)$

Tri fusion (Merge Sort)

• Principle:
• A divide and conquer algorithm, splits to sub partitions and is merged together as sorted parts.
• Complexity: $O(n \log n)$

Example

> [8, 3, 1, 7, 0, 10, 2]
> [8, 3, 1, 7] [0, 10, 2]
> [8, 3] [1, 7] [0, 10] 
> [3, 8] [1, 7] [0, 10] 
> [1, 3, 7, 8] [0, 2, 10]
> [0, 1, 2, 3, 7, 8, 10]