Probability Theory Overview

Probability Theory

Definition: A branch of mathematics dealing with the analysis of random phenomena. It provides a framework for quantifying uncertainty.
Key Concepts:
- Experiment: A process that yields an outcome (e.g., rolling a die).
- Sample Space (S): The set of all possible outcomes of an experiment.
- Event: A subset of the sample space; can consist of one or more outcomes.
- Probability (P): A measure of the likelihood that an event will occur, expressed as a number between 0 and 1.
Basic Probability Rules:
- P(S) = 1: The probability of the sample space is 1.
- P(∅) = 0: The probability of the empty set is 0.
- Addition Rule: For mutually exclusive events A and B, P(A or B) = P(A) + P(B).
- Complement Rule: P(A') = 1 - P(A), where A' is the complement of A.
Conditional Probability:
- Defined as P(A | B) = P(A and B) / P(B), where P(B) > 0.
- Independence: Events A and B are independent if P(A | B) = P(A) and P(B | A) = P(B).
Bayes' Theorem:
- A way to find a conditional probability, expressed as:
  - P(A | B) = (P(B | A) * P(A)) / P(B)
Random Variables:
- Definition: A variable that takes on numerical values based on the outcome of a random phenomenon.
- Types:
  - Discrete Random Variable: Takes on a countable number of values (e.g., number of heads in coin tosses).
  - Continuous Random Variable: Takes on an infinite number of values within a range (e.g., height, weight).
Probability Distributions:
- Discrete Probability Distribution: Assigns probabilities to each possible value of a discrete random variable.
  - Example: Binomial distribution, Poisson distribution.
- Continuous Probability Distribution: Describes the probabilities of the possible values of a continuous random variable.
  - Example: Normal distribution, exponential distribution.
Expected Value (Mean):
- For discrete random variables: E(X) = Σ [x * P(x)].
- For continuous random variables: E(X) = ∫ x * f(x) dx, where f(x) is the probability density function (PDF).
Variance and Standard Deviation:
- Variance (Var(X)): Measures the spread of a random variable around its mean.
  - Var(X) = E[(X - E(X))²].
- Standard Deviation (σ): The square root of variance; indicates the average distance from the mean.
Law of Large Numbers: As the size of a sample increases, the sample mean will get closer to the expected value (population mean).
Central Limit Theorem: The distribution of the sample mean approaches a normal distribution as the sample size increases, regardless of the shape of the population distribution, given that the sample size is sufficiently large.

Probability Theory Overview

Probability theory quantifies uncertainty through mathematical analysis of random phenomena.
Fundamental to various fields, including statistics, finance, science, and engineering.

Key Concepts

Experiment: A procedure that produces outcomes, such as rolling a die.
Sample Space (S): The comprehensive set of all potential outcomes from an experiment.
Event: A specific subset of the sample space; can include one or multiple outcomes.
Probability (P): Quantifies the likelihood of an event, ranging from 0 (impossible) to 1 (certain).

Basic Probability Rules

P(S) = 1: The probability of the entire sample space equals 1.
P(∅) = 0: The probability of the empty set is always 0.
Addition Rule: For mutually exclusive events A and B, P(A or B) = P(A) + P(B).
Complement Rule: The probability of the complement of A, denoted P(A'), equals 1 - P(A).

Conditional Probability and Independence

Conditional Probability: P(A | B) = P(A and B) / P(B), provided P(B) > 0.
Independence: Events A and B are independent if P(A | B) = P(A) and P(B | A) = P(B).

Bayes' Theorem

A formula to determine conditional probabilities:
- P(A | B) = (P(B | A) * P(A)) / P(B).

Random Variables

Definition: Variables that assume numerical values based on outcomes from a random process.
Types:
- Discrete Random Variable: Countable values (e.g., number of heads in multiple coin flips).
- Continuous Random Variable: Infinite values within a range (e.g., height, weight).

Probability Distributions

Discrete Probability Distribution: Assigns probabilities to each value of a discrete random variable, examples include:
- Binomial Distribution: Counts successes in a fixed number of trials.
- Poisson Distribution: Counts the number of events in a fixed interval.
Continuous Probability Distribution: Describes probabilities for continuous variables, examples include:
- Normal Distribution: Bell-shaped curve common in statistics.
- Exponential Distribution: Models time until an event occurs.

Expected Value and Variance

Expected Value (Mean): For discrete variables: E(X) = Σ [x * P(x)]. For continuous variables: E(X) = ∫ x * f(x) dx, where f(x) is the probability density function (PDF).
Variance (Var(X)): A measure of how much a random variable deviates from its mean, calculated as Var(X) = E[(X - E(X))²].
Standard Deviation (σ): The square root of variance, indicating the average distance of values from the mean.

Theorems in Probability

Law of Large Numbers: As sample size increases, the sample mean approximates the population mean.
Central Limit Theorem: The sample mean distribution approaches a normal distribution as sample size increases, regardless of the original population distribution shape, provided the sample size is sufficiently large.