Channel Capacity and Mutual Information

Channel Capacity

• Channel Capacity is the maximum rate at which information can be reliably transmitted over a communication channel.

Formulae for Mutual Information

• Mutual Information is given by:
  • (I(X;Y) = H(X) - H(X|Y))
  • (I(X;Y) = H(Y) - H(Y|X))
  • (I(X;Y) = H(X) + H(Y) - H(X,Y))
  • (I(X;Y) = \sum_{x,y} p(x,y) \log \frac{p(x,y)}{p(x)p(y)})

Formula for Channel Capacity

• Channel Capacity is the maximum of the mutual information over all possible input distributions (p(x)):
  • (C = \max_{p(x)} I(X;Y))

Properties of Mutual Information

• Mutual Information is symmetric: (I(X;Y) = I(Y;X))
• Mutual Information is non-negative: (I(X;Y) \ge 0)
  • This can be proved using the inequality (\ln(x) \le x-1).
• The mutual information of a random variable with itself is its entropy: (I(X;X) = H(X)) can be derived from the definition of mutual information.

Binary Symmetric Channel (BSC)

• A Binary Symmetric Channel (BSC) has transition probabilities: (p(y=0|x=0) = p(y=1|x=1) = 1 - p) and (p(y=0|x=1) = p(y=1|x=0) = p)
• The channel capacity is (C = 1 - H(p)) bits, achieved when the input distribution is uniform (p(x=0) = p(x=1) = \frac{1}{2}).
• (H(Y|X)) is equal to (H(p)).

Binary Erasure Channel (BEC)

• A Binary Erasure Channel (BEC) has transition probabilities:
  • (p(y=0|x=0) = 1 - \alpha)
  • (p(y=e|x=0) = \alpha),
  • (p(y=1|x=1) = 1 - \alpha)
  • (p(y=e|x=1) = \alpha)
• The channel capacity is (C = 1 - \alpha) bits, achieved when the input distribution is uniform (p(x=0) = p(x=1) = \frac{1}{2}).
• (H(Y|X)) is equal to (H(\alpha)).

Properties of Channel Capacity

• Channel Capacity is always non-negative: (C \ge 0).
• Channel Capacity is bounded by the size of input and output alphabets: (C \le \min{\log|X|, \log|Y|}).
• Channel Capacity is a convex function of the channel transition probabilities (p(y|x)).
• Channel Capacity is a concave function of the input distribution (p(x)).

Algorithmic Game Theory (AGT)

• AGT merges classical game theory (study of strategic interactions) with classical algorithm design, addressing the intersection of rationality and computational efficiency.
• Classical game theory assumes perfect rationality, while classical algorithm design often ignores incentives.
• AGT provides new perspectives on problems such as the "Price of Anarchy" and mechanism design.

Selfish Routing Model

• A network of (n) nodes and (m) links, where each link (e) has a cost function (\ell_e(x)) dependent on the traffic (x) on the link.
• There is a set of (k) user populations, where population (i) consists of (r_i) users who want to travel from (s_i) to (t_i).
• A flow (f) specifies the rate (f_p) of traffic on each path (p) in the network.
• A flow (f) is feasible if it satisfies user demands: (\sum_{p:s_i \rightarrow t_i} f_p = r_i) for each (i).
• The cost of a path (p) in a flow (f) is: (C_p(f) = \sum_{e \in p} \ell_e(f_e)) where (f_e = \sum_{p: e \in p} f_p).
• The total cost of all users is: (\sum_{p} f_p C_p(f) = \sum_{e} f_e \ell_e(f_e)).

Concepts of Equilibrium and Optimality

• Wardrop Equilibrium: a flow where all paths used have equal and minimal cost for each source-destination pair, also known as Nash equilibrium.
• Social Optimum: a flow that minimizes the total cost (\sum_{e} f_e \ell_e(f_e)), achieved by a benevolent controller.
• The Price of Anarchy (PoA) is the ratio of the cost at Wardrop equilibrium to the cost at social optimum: (PoA = \frac{\text{Cost of Wardrop Equilibrium}}{\text{Cost of Social Optimum}})

Senses Summary

• Five special senses: olfaction, gustation, vision, hearing, and equilibrium.

Olfaction (Smell)

• Olfactory epithelium contains olfactory receptor cells, supporting cells, and olfactory stem cells.
• Olfactory pathway: receptor cells synapse with mitral cells in the olfactory bulb, forming the olfactory tract to the olfactory cortex.

Gustation (Taste)

• Taste buds contain gustatory epithelial cells with gustatory hairs and basal epithelial cells.
• Five basic tastes: sweet, sour, salty, bitter, and umami.
• Gustatory pathway: cranial nerves VII and IX carry impulses to the solitary nucleus of the medulla oblongata, then to the thalamus and gustatory cortex.

Vision

• Eyeball layers: fibrous (sclera, cornea), vascular (choroid, ciliary body, iris), and inner (retina).
• Lens focuses light on the retina by changing shape.
• Retina contains photoreceptors: rods (dim light, peripheral vision) and cones (bright light, color vision).
• Visual pathway: photoreceptors synapse with bipolar cells, which synapse with ganglion cells; axons of ganglion cells form the optic nerve to the optic chiasm, thalamus, and visual cortex.

Hearing and Equilibrium

• Ear: external (auricle, auditory canal), middle (tympanic membrane, ossicles), and internal (cochlea, vestibular apparatus).
• Cochlea's spiral organ (organ of Corti) contains hair cells stimulated by vibrations for hearing.
• Vestibular apparatus monitors static and dynamic equilibrium with maculae and cristae ampullares.
• Auditory pathway: hair cells synapse with cochlear nerve, traveling to the medulla oblongata, inferior colliculus, thalamus, and auditory cortex.
• Vestibular pathway: hair cells synapse with the vestibular nerve to the vestibular nuclei in the brainstem and cerebellum.

Sensory Receptors

• Olfaction uses olfactory receptor cells in the nasal cavity.
  • The pathway goes from these cells to the olfactory bulb (mitral cells), then the olfactory tract and finally the olfactory cortex.
• Gustation uses gustatory epithelial cells in taste buds on the tongue, pharynx, and larynx.
  • The pathway goes from these cells to cranial nerves VII & IX, then the solitary nucleus in the medulla, the thalamus and finally the gustatory cortex.
• Vision uses photoreceptors (rods & cones) in the retina of the eye.
  • The pathway goes from the photoreceptors to bipolar cells, then ganglion cells (optic nerve), the optic chiasm, the thalamus and finally the visual cortex.
• Hearing uses hair cells in the spiral organ (organ of Corti) in the cochlea.
  • The pathway for this sense goes from the hair cells to the cochlear nerve, then medulla oblongata, inferior colliculus, thalamus and auditory cortex.
• Equilibrium receptors are hair cells in Maculae (utricle & saccule) and Cristae ampullares (semicircular canals)
  • the pathway goes from the hair cells to the vestibular nerve, and then the vestibular nuclei (brainstem) & cerebellum.

Cinemática (Kinematics) Formulas

• Posición (Position):
  • (\overrightarrow{r} = x\hat{i} + y\hat{j} + z\hat{k})
  • (\overrightarrow{r} = r\cos(\theta)\hat{i} + r\sin(\theta)\hat{j})
• Velocidad (Velocity):
  • (\overrightarrow{v} = \frac{d\overrightarrow{r}}{dt})
  • (\overrightarrow{v}_{avg} = \frac{\Delta \overrightarrow{r}}{\Delta t})
• Aceleración (Acceleration):
  • (\overrightarrow{a} = \frac{d\overrightarrow{v}}{dt})
  • (\overrightarrow{a}_{avg} = \frac{\Delta \overrightarrow{v}}{\Delta t})

Movimiento con Aceleración Constante (Motion with Constant Acceleration) Formulas

• Posición (Position): (x = x_0 + v_0t + \frac{1}{2}at^2)
• Velocidad (Velocity): (v = v_0 + at)
• (v^2 = v_0^2 + 2a\Delta x)

Movimiento de Proyectiles (Projectile Motion) Formulas

• Altura Máxima (Maximum Height): (H = \frac{v_0^2\sin^2\theta}{2g})
• Alcance Horizontal (Horizontal Range): (R = \frac{v_0^2\sin 2\theta}{g})
• Tiempo Total de Vuelo (Total Flight Time): (T = \frac{2v_0 \sin\theta}{g})

Movimiento Circular Uniforme (Uniform Circular Motion) Formulas

• Rapidez (Speed): (v = \frac{2\pi r}{T})
• Aceleración (Acceleration):
  • (a_c = \frac{v^2}{r})
  • (a_c = r\omega^2)
• Velocidad Angular (Angular Velocity): (\omega = \frac{d\theta}{dt})
• Aceleración Angular (Angular Acceleration): (\alpha = \frac{d\omega}{dt})
• Velocidad y Aceleración Tangencial (Tangential Velocity and Acceleration):
  • (v = r\omega)
  • (a_t = r\alpha)
• Frecuencia (Frequency): (f = \frac{1}{T})

Fuerzas (Forces) Formulas

• Primera Ley de Newton (Newton's First Law): (\sum \overrightarrow{F} = 0 \Leftrightarrow \overrightarrow{v} = \text{constante})
• Segunda Ley de Newton (Newton's Second Law): (\sum \overrightarrow{F} = m\overrightarrow{a})
• Tercera Ley de Newton (Newton's Third Law): (\overrightarrow{F}{12} = -\overrightarrow{F}{21})
• Fuerza de Gravedad (Force of Gravity): (F_g = mg)
• Fuerza Normal (Normal Force): (F_N)
• Fuerza de Fricción Estática (Static Friction Force): (f_s \le \mu_s F_N)
• Fuerza de Fricción Cinética (Kinetic Friction Force): (f_k = \mu_k F_N)
• Fuerza Elástica (Elastic Force): (F_x = -kx)

Trabajo y Energía (Work and Energy) Equations

• Trabajo (Work):
  • (W = \overrightarrow{F} \cdot \overrightarrow{d})
  • (W = Fd\cos\theta)
• Trabajo Neto (Net Work): (W_{neto} = \Delta KE)
• Energía Cinética (Kinetic Energy): (KE = \frac{1}{2}mv^2)
• Energía Potencial Gravitatoria (Gravitational Potential Energy): (U_g = mgy)
• Energía Potencial Elástica (Elastic Potential Energy): (U_s = \frac{1}{2}kx^2)
• Potencia (Power):
  • (P = \frac{dW}{dt})
  • (P = \overrightarrow{F} \cdot \overrightarrow{v})

Conservación de la Energía (Conservation of Energy) Statements

• Energía Mecánica (Mechanical Energy): (E_{mec} = KE + U)
• Conservación de la Energía Mecánica (Conservation of Mechanical Energy): (\Delta E_{mec} = 0) (Si solo actúan fuerzas conservativas - If only conservative forces act)
• Trabajo de Fuerzas No Conservativas (Work by Non-Conservative Forces): (\Delta E_{mec} = W_{nc}) (Si actúan fuerzas no conservativas - If non-conservative forces act)

Centro de Masa (Center of Mass) Equations

• Posición (Position): (\overrightarrow{r}_{CM} = \frac{\sum m_i \overrightarrow{r}_i}{\sum m_i})
• Velocidad (Velocity): (\overrightarrow{v}_{CM} = \frac{\sum m_i \overrightarrow{v}_i}{\sum m_i})
• Segunda Ley de Newton para el Centro de Masa (Newton's Second Law for Center of Mass): (\sum \overrightarrow{F}{ext} = M\overrightarrow{a}{CM})

Colisiones (Collisions) Equations

• Cantidad de Movimiento (Momentum): (\overrightarrow{p} = m\overrightarrow{v})
• Impulso (Impulse): (\overrightarrow{I} = \Delta \overrightarrow{p} = \int \overrightarrow{F} dt)
• Conservación de la Cantidad de Movimiento (Conservation of Momentum): (\sum \overrightarrow{p}_i = \sum \overrightarrow{p}_f)
• Colisión Elástica en 1D (Elastic Collision in 1D): (v_{1i} - v_{2i} = -(v_{1f} - v_{2f}))

Rotación (Rotation) Equations

• Torque:
  • (\overrightarrow{\tau} = \overrightarrow{r} \times \overrightarrow{F})
  • (\tau = rF\sin\theta)
• Momento de Inercia (Moment of Inertia): (I = \sum m_i r_i^2)
• Segunda Ley de Newton para Rotación (Newton's Second Law for Rotation): (\sum \tau = I\alpha)
• Trabajo Rotacional (Rotational Work): (W = \int \tau d\theta)
• Energía Cinética Rotacional (Rotational Kinetic Energy): (KE_{rot} = \frac{1}{2}I\omega^2)
• Momento Angular (Angular Momentum):
  • (\overrightarrow{L} = \overrightarrow{r} \times \overrightarrow{p} = I\overrightarrow{\omega})
  • (\sum \tau = \frac{d\overrightarrow{L}}{dt})
• Conservación del Momento Angular (Conservation of Angular Momentum): (\overrightarrow{L}_i = \overrightarrow{L}_f)
• Energía Cinética Total (Total Kinetic Energy): (KE = \frac{1}{2}mv_{CM}^2 + \frac{1}{2}I_{CM}\omega^2)

Gravitación (Gravitation) Equations

• Ley de Gravitación Universal (Law of Universal Gravitation): (F = G\frac{m_1m_2}{r^2})
• Energía Potencial Gravitacional (Gravitational Potential Energy): (U = -G\frac{m_1m_2}{r})
• Velocidad de Escape (Escape Velocity): (v_{esc} = \sqrt{\frac{2GM}{R}})

Leyes de Kepler (Kepler's Laws)

• Primera Ley (First Law): Las órbitas son elípticas (Orbits are elliptical)
• Segunda Ley (Second Law): La velocidad areolar es constante (The areal velocity is constant)
• Tercera Ley (Third Law): (T^2 = (\frac{4\pi^2}{GM})a^3)

Oscilaciones (Oscillations) Equations

• Movimiento Armónico Simple (MAS) - Simple Harmonic Motion (SHM):
  • (x(t) = A\cos(\omega t + \phi))
  • (v(t) = -A\omega\sin(\omega t + \phi))
  • (a(t) = -A\omega^2\cos(\omega t + \phi) = -\omega^2x(t))
• Frecuencia Angular (Angular Frequency): (\omega = \sqrt{\frac{k}{m}})
• Periodo (Period): (T = 2\pi\sqrt{\frac{m}{k}})
• Péndulo Simple (Simple Pendulum):
  • (\omega = \sqrt{\frac{g}{L}})
  • (T = 2\pi\sqrt{\frac{L}{g}})
• Péndulo Físico (Physical Pendulum):
  • (\omega = \sqrt{\frac{mgd}{I}})
  • (T = 2\pi\sqrt{\frac{I}{mgd}})
• Oscilaciones Amortiguadas (Damped Oscillations): (x(t) = Ae^{-bt/2m}\cos(\omega't + \phi))
• Oscilaciones Forzadas (Forced Oscillations): (x(t) = A\cos(\omega t + \phi)) where (A = \frac{F_0/m}{\sqrt{(\omega^2 - \omega_0^2)^2 + (b\omega/m)^2}})

Ondas (Waves) Equations

• Velocidad de Onda (Wave Speed): (v = \sqrt{\frac{F}{\mu}})
• Número de Onda (Wave Number): (k = \frac{2\pi}{\lambda})
• Función de Onda (Wave Function): (y(x, t) = A\sin(kx - \omega t + \phi))
• Velocidad (Speed): (v = f\lambda)
• Densidad Lineal (Linear Density): (\mu = \frac{m}{L})
• Potencia (Power): (P_{avg} = \frac{1}{2}\mu\omega^2A^2v)
• Superposición de Ondas (Superposition of Waves): (y(x, t) = y_1(x, t) + y_2(x, t))
• Interferencia Constructiva (Constructive Interference): (\Delta r = n\lambda)
• Interferencia Destructiva (Destructive Interference): (\Delta r = (n + \frac{1}{2})\lambda)
• Ondas Estacionarias (Standing Waves): (y(x, t) = (2A\sin kx)\cos \omega t)
• Frecuencias (Frequencies): (f_n = n\frac{v}{2L}), where (n = 1, 2, 3,...)

Termodinámica (Thermodynamics) Equations

• Ley Cero (Zeroth Law): Si A está en equilibrio térmico con C, y B está en equilibrio térmico con C, entonces A está en equilibrio térmico con B.
• Dilatación Lineal (Linear Expansion): (\Delta L = \alpha L_0 \Delta T)
• Dilatación de Área (Area Expansion): (\Delta A = 2\alpha A_0 \Delta T)
• Dilatación de Volumen (Volume Expansion): (\Delta V = \beta V_0 \Delta T)
• Calor Específico (Specific Heat): (Q = mc\Delta T)
• Calor Latente (Latent Heat): (Q = mL)
• Conducción (Conduction): (\frac{dQ}{dt} = kA\frac{T_H - T_C}{L})
• Convección (Convection): Transferencia de calor por movimiento de un fluido (Heat transfer by the movement of a fluid)
• Radiación (Radiation): (\frac{dQ}{dt} = \sigma A e T^4)
• Ley de los Gases Ideales (Ideal Gas Law): (PV = nRT)
• Trabajo (Work): (W = \int PdV)
• Energía Interna (Internal Energy):
  • (\Delta E_{int} = Q - W)
  • (E_{int} = \frac{3}{2}nRT)
• Capacidad Calorífica Molar (Molar Heat Capacity): (C_V = \frac{3}{2}R)
• (C_P = C_V + R)
• Procesos Adiabáticos (Adiabatic Processes):
  • (PV^\gamma = \text{constante})
  • (TV^{\gamma - 1} = \text{constante})
  • (W = \frac{P_fV_f - P_iV_i}{1 - \gamma})
  • (\gamma = \frac{C_P}{C_V})

Segunda Ley de la Termodinámica (Second Law of Thermodynamics) Equations

• Eficiencia (Efficiency): (e = \frac{W_{neto}}{Q_H} = 1 - \frac{Q_C}{Q_H})
• Ciclo de Carnot (Carnot Cycle): (e_C = 1 - \frac{T_C}{T_H})
• Entropía (Entropy): (\Delta S = \int \frac{dQ}{T})

Miscelánea (Miscellaneous)

• Circunferencia (Circumference): (C = 2\pi r)
• Área (Area): (A = \pi r^2)
• Área de la Superficie (Surface Area): (A = 4\pi r^2)
• Volumen (Volume): (V = \frac{4}{3}\pi r^3)

Trigonometría (Trigonometry)

• Seno (Sine): (\sin\theta = \frac{\text{opuesto}}{\text{hipotenusa}})
• Coseno (Cosine): (\cos\theta = \frac{\text{adyacente}}{\text{hipotenusa}})
• Tangente (Tangent): (\tan\theta = \frac{\text{opuesto}}{\text{adyacente}})