Algorithmic Game Theory: Networked Systems

Algorithmic Game Theory

• Explores strategic interactions between agents with differing preferences, where outcomes depend on everyone's actions.

Course Information

• Instructor: Anna Karlin ([email protected])
• TA: Aris Filos-Ratsikas ([email protected])
• Time: Tuesdays and Thursdays, 1:30 - 2:50
• Location: CSE 305
• The course webpage is available at: https://courses.cs.washington.edu/courses/cse533/23au/

Course Content

• The course examines topics in algorithmic game theory, focusing on networked systems and market design.
• Key topics include Nash equilibrium, Bayes-Nash equilibrium, correlated equilibrium, and no-regret learning.
• The course also covers mechanism design, including topics on auctions, sponsored search, and matching.
• Furthermore, the price of anarchy and fair division concepts will be explored

Prerequisites

• Knowledge of algorithms and probability is required.
• Mathematical maturity is expected.

Grading Breakdown

• 50% Problem sets
• 10% Participation
• 40% Project

Examples of Game Theory Applications

• Network routing
• Auctions
• Fair division
• Elections
• Social networks

Selfish Routing

• Focuses on n agents traveling from vertex s to vertex t in a network.
• Each edge e has a cost function l_e(x), representing the cost per agent when x agents use the edge.
• Studies and quantifies the degradation of network quality due to selfish routing behaviors.

Congestion Game Definition

• A congestion game is defined as with a tuple: $(N, R, E, {c_e})$.
• N represents the number of players
• $R_i \subseteq 2^E$ represents the set of strategies available to player $i$.
• E is the set of resources.
• $c_e$ is the cost function for resource $e$, dependent on the number of players using it.

Congestion Game Example

• A simple network connects vertices $s$ and $t$ with two edges.
• Edge 1's cost function is $\mathcal{l}_1(x) = x$.
• Edge 2's cost function is $\mathcal{l}_2(x) = 1$.

Nash Equilibrium

• A Nash Equilibrium is a strategy profile in which no player benefits from unilaterally changing their strategy.
• In the congestion game example, if all players use edge 1, any player can switch to edge 2 to reduce their cost from n to 1.
• However, if one player takes edge 2 and n-1 players take edge 1, no player has an incentive to switch, as the cost for those on edge 1 would be n-1, and the cost for the player on edge 2 is 1, representing a Nash Equilibrium.

Social Cost

• The social cost of a strategy profile equals the total cost of all players: $$ SC = \sum_{i \in N} c_i(s) $$
• In the Nash Equilibrium example, the social cost equals n since SC = (n-1) + 1 = n.
• If all players took edge 2, the social cost would be n $$ SC = n $$
• If all players took edge 1, the social cost would be $n^2$ $$ SC = n \cdot n = n^2 $$

Price of Anarchy

• Measures the ratio between the social cost of a worst-case Nash Equilibrium and the optimal social cost. $$ PoA = \frac{\text{Social cost of worst-case Nash Equilibrium}}{\text{Optimal social cost}} $$
• In the example, the Price of Anarchy is $\frac{n}{n} = 1$

Braess's Paradox

• The paradox explains that adding capacity to a network can increase congestion and travel times.

Braess's Paradox Example

• A network includes vertices A, B, C, and D.
• There is an edge from A to B with cost function $\mathcal{l}_1(x) = x$.
• There is an edge from A to C with cost function $\mathcal{l}_2(x) = 10$.
• There is an edge from B to D with cost function $\mathcal{l}_3(x) = 10$.
• There is an edge from C to D with cost function $\mathcal{l}_4(x) = x$.
• Agents traveling from A to D can take path $A \rightarrow B \rightarrow D$ with cost $x + 10$, or path $A \rightarrow C \rightarrow D$ with cost $10 + x$.
• In Nash Equilibrium, half the players take each path, incurring a cost of $\frac{n}{2} + 10$.
• The social cost is $n \cdot (\frac{n}{2} + 10)$.

Braess's Paradox with Added Edge

• Adding an edge from B to C with cost function $\mathcal{l}_5(x) = 0$ changes the equilibrium.
• Now, each player takes the path $A \rightarrow B \rightarrow C \rightarrow D$, with a cost of $x + 0 + x = 2x$.
• The social cost becomes $n \cdot 2n = 2n^2$.

Non-Atomic Routing Game

• $n$ infinitesimal players want to travel from $s$ to $t$.
• $f_e$ is the flow on edge $e$.
• $\mathcal{l}_e(f_e)$ is the latency on edge $e$.

Wardrop Equilibrium

• Wardrop Equilibrium is a flow $f$ that has all flow travels on paths of minimum latency.
• All flow travels on paths with latency $L(f)$.
• Used paths have the same latency, while unused paths have higher latency.

Chemical Kinetics

• Deals with reaction rates and reaction mechanisms.

Reaction Rate

• Defined as the change in concentration of a reactant or product with respect to time.
• $rate = \frac{\Delta [reactant \ or \ product]}{\Delta t}$

Rate Law

• An equation relating the rate of a reaction to the concentrations of reactants.
• $aA + bB \rightarrow cC + dD$
• $rate = k[A]^m [B]^n$
  • k: rate constant
  • m, n: reaction orders (determined experimentally)

Reaction Order

• The sum of the exponents in the rate law.
• Overall order = m + n

Integrated Rate Laws

• Relate the concentration of a reactant to time.

Zero-Order Reaction

• $rate = k$
• $[A]_t = -kt + [A]_0$
• $t_{1/2} = \frac{[A]_0}{2k}$

First-Order Reaction

• $rate = k[A]$
• $ln[A]_t = -kt + ln[A]_0$
• $t_{1/2} = \frac{0.693}{k}$

Second-Order Reaction

• $rate = k[A]^2$
• $\frac{1}{[A]_t} = kt + \frac{1}{[A]_0}$
• $t_{1/2} = \frac{1}{k[A]_0}$

Collision Theory

• Molecules must collide with sufficient energy (activation energy) and proper orientation for the reaction to occur.

Arrhenius Equation

• Relates the rate constant to the activation energy and temperature.
• $k = Ae^{-\frac{E_a}{RT}}$
  • $E_a$: activation energy
  • R: gas constant (8.314 J/mol·K)
  • A: frequency factor

Catalysis

• A catalyst speeds up a reaction by lowering the activation energy.
• Homogeneous catalyst: in the same phase as the reactants
• Heterogeneous catalyst: in a different phase as the reactants

Reaction Mechanisms

• A series of elementary steps describing how a reaction occurs.

Elementary Step

• A single step in a reaction mechanism.

Rate-Determining Step

• The slowest step in a reaction mechanism.

Intermediates

• Species produced in one step and consumed in a subsequent step.

Molecularity

• The number of molecules that participate in an elementary step.
  • Unimolecular: one molecule
  • Bimolecular: two molecules
  • Termolecular: three molecules

Reaction Mechanisms Example

• $NO_2(g) + CO(g) \rightarrow NO(g) + CO_2(g)$
• Mechanism:
  1. $NO_2(g) + NO_2(g) \rightarrow NO(g) + NO_3(g)$ (slow)
  2. $NO_3(g) + CO(g) \rightarrow NO_2(g) + CO_2(g)$ (fast)
• Rate-determining step: Step 1
• Intermediates: $NO_3(g)$

Bayesian Tracking

• System Model (Transition Model): $p(x_t | x_{t-1})$
• Measurement Model: $p(z_t | x_t)$
• Inference: Belief update
  • $bel(x_t) = \eta p(z_t | x_t) \int p(x_t | x_{t-1}) bel(x_{t-1}) dx_{t-1}$
  • $\eta$ is a normalization factor
  • $bel(x_t)$ is the belief about the state at time $t$ given all measurements up to time $t$

Kalman Filter

• Applicable when both system and measurement models are linear and Gaussian.
• Optimal in the sense of minimizing the mean square error.

Extended Kalman Filter (EKF)

• Linearizes the system and measurement models around the current estimate.
• Suboptimal, can diverge if the models are highly nonlinear.

Particle Filter

• Also known as Monte Carlo Localization, Sequential Monte Carlo methods.
• Represents the belief by a set of random samples (particles).
• Can represent arbitrary distributions.
• Non-parametric.

Particle Filter Algorithm

1. Initialization: Draw $N$ samples (particles) from the initial belief $p(x_0)$.
2. Prediction: For each particle, predict its new state based on the system model.
3. Weighting: Weight each particle based on the likelihood of the measurement given the particle's state.
4. Resampling: Resample particles based on their weights. Particles with higher weights are more likely to be selected.

Particle Filter Algorithm Details

1. Initialization: Draw $N$ samples (particles) from the initial belief $p(x_0)$.
   $X_0 = {x_0^i}_{i=1}^N \sim p(x_0)$
2. Prediction: For each particle, predict its new state based on the system model.
   $x_t^i \sim p(x_t | x_{t-1} = x_{t-1}^i)$
3. Weighting: Weight each particle based on the likelihood of the measurement given the particle's state.
   $w_t^i = p(z_t | x_t^i)$
4. Normalization: Normalize the weights so that they sum to 1.
   $w_t^i = \frac{w_t^i}{\sum_{j=1}^N w_t^j}$
5. Resampling: Resample particles based on their weights. Particles with higher weights are more likely to be selected.
   $X_t = {x_t^i}_{i=1}^N \sim X_t$ with probability $w_t^i$

Resampling

• Discard particles with low weights and focus on particles with high weights.
• Methods:
  • Multinomial Resampling: Draw $N$ samples from the discrete distribution defined by the weights.
  • Residual Resampling: A deterministic resampling step that guarantees that particles with high weights are selected at least once.
  • Stratified Resampling: Divides the sample space into strata and draws one sample from each stratum.