Introduction to Probability

Introduction to Probability

• A random experiment has outcomes that cannot be predicted with certainty.
• Examples include tossing a fair coin, rolling a fair die, and drawing a card from a well-shuffled deck.
• Sample space (S) refers to the set of all possible outcomes of a random experiment.
  • For tossing a coin, $S = {H, T}$.
  • For rolling a die, $S = {1, 2, 3, 4, 5, 6}$.
• An event (E) is any subset of the sample space.
  • Example: Rolling a die and getting an even number where $E = {2, 4, 6}$.
• An elementary event contains only one outcome.
  • Example: Tossing a fair coin and getting heads, $E = {H}$.

Calculating Probability

• The probability of event E is defined as $P(E) = \frac{m}{n}$ where n is the number of exhaustive outcomes and m are favorable outcomes.
• $P(E) = \frac{\text{Number of outcomes favorable to E}}{\text{Total number of possible outcomes}}$
• Probability ranges between 0 and 1: $0 \leq P(E) \leq 1$
• The probability of the sample space is 1: $P(S) = 1$
• The probability of an empty set is 0: $P(\emptyset) = 0$
• For the complement of E ($E^c$), $P(E^c) = 1 - P(E)$
• If E and F are mutually exclusive events, $P(E \cup F) = P(E) + P(F)$
• For any two events E and F, $P(E \cup F) = P(E) + P(F) - P(E \cap F)$
• Conditional probability of E given F: $P(E|F) = \frac{P(E \cap F)}{P(F)}$, provided $P(F) > 0$
• Events E and F are independent if $P(E \cap F) = P(E) \cdot P(F)$ or $P(E|F) = P(E)$ or $P(F|E) = P(F)$
• Example 1: Tossing a fair coin twice yields $P(E) = \frac{3}{4}$ for getting at least one head.
• Example 2: Probability of rolling an even number or a number greater than 4 on a die is $\frac{2}{3}$.
• Example 3: The probability of getting a king given it is a face card from a deck is $\frac{1}{3}$.

Algorithmic Game Theory

• Game theory is mathematical models of strategic interaction among rational agents.
• Agents are selfish and rational to maximize their utility.
• Strategic interaction: an agent's utility depends on what other agents do
• It is applicable in economics, political science, biology, and computer science.

Normal-Form Games

• A normal-form game is a triple $(N, A, u)$.
• $N = {1, 2,..., n}$ represents a finite set of players.
• Each player i has a finite set of actions $A_i$.
• Vector profiles are $a = (a_1,..., a_n) \in A$.
• $u = (u_1,..., u_n)$ where $u_i : A \rightarrow \mathbb{R}$ is a real-valued utility function.
• The classic Prisoner's Dilemma: two suspects, if one confesses, the confessor is freed and the other gets 10 years. If both confess, they each get 5 years. If neither confess, they each get 1 year.
• Prisoner's Dilemma: $N = {1, 2}$, $A_i = {C, D}$ where C = "cooperate", D = "defect". The variable $u_i(a)$ is the number of years in prison.
• Payoff Matrix example:
  • If both cooperate, both get -1.
  • If one cooperates and the other defects, the cooperator gets -10 and the defector gets 0.
  • If both defect, they each get -5.

Solution Concepts

• Solution concepts are used to predict played strategies, eliminate illogical strategies, and refine the options of agents.
• Pareto Optimality: minimal requirement, but multiple optimal outcomes.
• An action profile a Pareto dominates action profile a' if for all $i \in N$, $u_i(a) \ge u_i(a')$, and there exists $j \in N$ such that $u_j(a) > u_j(a')$.
• An action profile a is Pareto optimal if there does not exist a' such that a' Pareto dominates a.
• Nash Equilibrium: a stable state where no player wants to unilaterally deviate.
• Best Response: maximization of utility for the agent
  • expressed as: $BR_i(a_{-i}) = {a_i \in A_i | \forall a_i' \in A_i, u_i(a_i, a_{-i}) \ge u_i(a_i', a_{-i})}$
• Nash Equilibrium: action profile $a = (a_1,..., a_n)$ if, for all players $i$, $a_i$ is a best response to $a_{-i}$ where $a_i \in BR_i(a_{-i})$.
• Theorem: Any Nash equilibrium is guaranteed to exist in mixed strategies, but not in pure strategies.
• Theorem: Finding a Nash equilibrium is PPAD-complete.
• In Prisoner's Dilemma, the Nash equilibrium is when both players defect (D, D) because neither player benefits from unilaterally switching to cooperation

Funciones vectoriales de una variable real (Vector functions of a real variable)

• Vector functions are functions where the domain is a subset of real numbers and range is a set of vectors.
• A vector function of a real variable is a function $\overrightarrow{r}: I \subseteq \mathbb{R} \longrightarrow \mathbb{R}^{n}$. This assigns each real number $t$ from interval $I$ to a vector $\overrightarrow{r}(t)=\left(f_{1}(t), f_{2}(t), \ldots, f_{n}(t)\right)$ in $\mathbb{R}^{n}$.
• The functions $f_{1}(t), f_{2}(t), \ldots, f_{n}(t)$ are component functions of $\overrightarrow{r}(t)$.
• Example: $\overrightarrow{r}(t)=(\cos t, \operatorname{sen} t, t)$
  • This is a vector function with a domain $\mathbb{R}$ and its range is a subset of $\mathbb{R}^{3}$.
• The graph of a vector function $\overrightarrow{r}: I \subseteq \mathbb{R} \longrightarrow \mathbb{R}^{n}$ for any variable t in I is a point set in $\mathbb{R}^{n}$.
• When $n = 2$ or $n = 3$, then the function is a curve in the plane or space.
• Example: $\overrightarrow{r}(t)=(\cos t$, sen $t$ ) graph is a circle with radius 1.
• Example: $\overrightarrow{r}(t)=(\cos t$, sen $t, t)$ graph is a helix.
• The limit of a vector function $\overrightarrow{r}(t)$ as $t$ approaches $t_{0}$ is a vector $\overrightarrow{L}$ such that each one of its components is the limit of all functions of $\overrightarrow{r}(t)$.
• Given vector function $\overrightarrow{r}(t)=\left(f_{1}(t), f_{2}(t), \ldots, f_{n}(t)\right)$ and real number $t_{0}$, the limit of $\overrightarrow{r}(t)$ as $t$ approaches $t_{0}$ is vector $\vec{L}=\left(L_{1}, L_{2}, \ldots, L_{n}\right)$:
  • Which is expressed as: $\lim {t \rightarrow t{0}} \overrightarrow{r}(t)=\vec{L} \Longleftrightarrow \lim {t \rightarrow t{0}} f_{i}(t)=L_{i} \quad$ for $i=1,2, \ldots, n$

Linear Algebra

• Vector space $V$ with addition ( $u + v \in V$) and scalar multiplication ($c \cdot u \in V$).
• Vector operations that satisfy certain axioms:
  • Commutativity: $u + v = v + u$
  • Associativity: $(u + v) + w = u + (v + w)$ and $(cd)u = c(du)$
  • Additive Identity: $u + 0 = u$
  • Additive Inverse: $u + (-u) = 0$
  • Multiplicative Identity: $1 \cdot u = u$
  • Distributivity: $c(u + v) = cu + cv$ and $(c + d)u = cu + du$
• Examples of Vector Spaces:
  • $\mathbb{R}^n$: All n-tuples of real numbers.
  • $\mathbb{C}^n$: All n-tuples of complex numbers.
  • $P_n$: All polynomials of degree at most n.
  • $M_{m \times n}$: All $m \times n$ matrices.
  • $F(S, \mathbb{R})$: All functions from a set S to $\mathbb{R}$.
• A subset $W$ of $V$ is a subspace if $W$ is itself a vector space.
• For a subset $W$ of vector space $V$ to be a subspace:
  • $0 \in W$
  • $u + v \in W$ for all $u, v \in W$
  • $cu \in W$ for all $u \in W$ and $c \in \mathbb{R}$
• Linear Independence: equation $c_1v_1 + c_2v_2 + \dots + c_nv_n = 0$
• The span of a set of vectors is a linear combinations of these vectors.
• A basis for a vector space $V$ is a set of linearly independent vectors that span $V$.
• The dimension of a vector space $V$ is number of vectors in a basis for $V$.
• A linear transformation is a function $T: V \to W$ between vector spaces $V$ and $W$ such that $T(u + v) = T(u) + T(v)$ and $T(cu) = cT(u)$
• Kernel: $\text{ker}(T) = {v \in V \mid T(v) = 0}$
• Range: $\text{range}(T) = {w \in W \mid w = T(v) \text{ for some } v \in V}$
• Matrix: Every transformation can be represented by a matrix A such that $T(x) = Ax$.
• Eigenvector $v$ of matrix A such that $Av = \lambda v$ for some scalar $\lambda$.
• Characteristic Polynomial: $p(\lambda) = \det(A - \lambda I)$
• Diagonalization: $A = PDP^{-1}$
• Applications: solving systems of differential equations, principal component analysis, Markov chains, vibrational analysis, and stability analysis.

2. Derivation of the Discrete RBFN Mapping

• The section presents a derivation of the discrete Radial Basis Function Network (RBFN) mapping.
• Radial Basis Function Network (RBFN)
  • Given N dataset points {$x_i}_{i=1}^N$ in $R^n$, the RBFN approximation $\hat{f}(x)$.
  • Its defined as a linear combination of N radial basis functions: $\hat{f}(\mathbf{x}) = \sum_{i=1}^{N} w_i \phi(\left\Vert \mathbf{x} - \mathbf{x}_i \right\Vert)$
    • $\phi(.)$ : radial basis function.
    • $x_i$ : the center
    • $w_i$ : associated weight to the RBF.
  • Surface representation is when $x \in \mathbb{R}^2$.
  • Gaussian radial basis function:
  • $\phi(\left\Vert \mathbf{x} - \mathbf{x}_i \right\Vert) = e^{-\frac{\left\Vert \mathbf{x} - \mathbf{x}_i \right\Vert^2}{\sigma^2}}$ where $σ$ is the function's standard deviation.
  • With Matrixes: weights {$w_i}_{i=1}^N$ are expressed as $Aw = f$
    • A is an *N x N * matrix with elements $A_{ij} = \phi(\left\Vert \mathbf{x}_i - \mathbf{x}_j \right\Vert)$.
    • Vector is w = [$w_1, w_2,..., w_N]^T$ .
    • Vector is f = [$f_1, f_2,..., f_N]^T$ .
  • So, $w=A^{-1}f$.

Statistiques descriptives univariées (Univariate descriptive statistics)

• Population is the set of individuals or items to be studied.
• An example is a subset of population to represent a study.
• Individual/statistical unit is an element of population.
• A Variable is a trait associated with individuals.
• Modalities are different values or categories.
• The size (effectif) is amount of people with certain modalities.
• The frequency is the proportion of people with a certain modaility.

Types of Variables

• Qualitative variables.
  • Nomale - unordered categories
  • Ordinale - ordered categories.
• Quantitative variables.
  • Discrete - numerical values
  • Continuous - any value data.

Central Tendency Indicators

• The mean (average) is the sum of total values by number of total values.
  • Population expressed: $\mu = \frac{1}{N} \sum_{i=1}^{N} x_i$.
  • Sample expressed: $\bar{x} = \frac{1}{n} \sum_{i=1}^{n} x_i$.
• The median is what value divides data in to 2 equal parts.
  • if uneven its the exact middle. if even the average of 2 middle values.
• The mode is the most repeating number/value in dataset.

Dispersion Indicators

• The range is difference between the maximum and minimum value.
• The variance is the dispersion around the mean.
  • Population express: $\sigma^2 = \frac{1}{N} \sum_{i=1}^{N} (x_i - \mu)^2$.
  • Sample express: $s^2 = \frac{1}{n-1} \sum_{i=1}^{n} (x_i - \bar{x})^2$.
• Standard Variation is Square root of Variance.
• Coefficient Variation is ration of standard variety to mean.

Other Descritpotive Statistics

• Quantiles the values that split data into equal parts.
• Boxplot grapahs of quantiles, with median + outliers. -Histograms with frequency of continuous data.
• Table of Discrete data of Modal.

Understand random experiments, sample spaces, and events. Learn to calculate probability using the formula P(E) = m/n, where m is the number of favorable outcomes and n is the total number of possible outcomes. Explore probability ranges from 0 to 1.