Statistics: Conditional Probability

Conditional Probability

• Definition: The probability of an event occurring given that another event has occurred.
• Notation: P(A|B) reads "the probability of A given B"
• Formula: P(A|B) = P(A ∩ B) / P(B)
• Properties:
  • P(A|B) ≥ 0 (non-negativity)
  • P(A|B) ≤ 1 (boundedness)
  • P(A|B) + P(A'|B) = 1 (completeness)
  • P(A ∩ B) = P(A|B) * P(B) (multiplication rule)
• Conditional Independence: Events A and B are conditionally independent given C if P(A|B,C) = P(A|C)
• Bayes' Theorem: P(A|B) = P(B|A) * P(A) / P(B)

Random Variables

• Definition: A variable whose possible values are determined by chance
• Types:
  • Discrete Random Variables (DRVs): take on a countable number of distinct values
  • Continuous Random Variables (CRVs): take on any value within a certain range or interval
• Probability Distribution: a function that describes the probability of each possible value of a random variable
• Notation: X for a random variable, x for a specific value of X
• Terminology:
  • Support: the set of all possible values of a random variable
  • Probability Mass Function (PMF): function describing the probability of each value of a DRV
  • Probability Density Function (PDF): function describing the probability of each value of a CRV
  • Cumulative Distribution Function (CDF): function describing the probability that a random variable takes on a value less than or equal to x
• Expectation: the long-run average value of a random variable
• Variance: the average of the squared differences between a random variable and its expectation

Conditional Probability

• The probability of an event A occurring given that event B has occurred is represented as P(A|B)
• The formula to calculate conditional probability is P(A|B) = P(A ∩ B) / P(B)
• Conditional probability has four properties: non-negativity, boundedness, completeness, and multiplication rule
• Two events A and B are conditionally independent given C if P(A|B,C) = P(A|C)
• Bayes' Theorem is P(A|B) = P(B|A) * P(A) / P(B)

Random Variables

• A random variable is a variable whose possible values are determined by chance
• There are two types of random variables: Discrete Random Variables (DRVs) and Continuous Random Variables (CRVs)
• A probability distribution is a function that describes the probability of each possible value of a random variable
• The notation X represents a random variable, and x represents a specific value of X

Random Variable Notation and Terminology

• The support of a random variable is the set of all possible values it can take
• A Probability Mass Function (PMF) is a function that describes the probability of each value of a DRV
• A Probability Density Function (PDF) is a function that describes the probability of each value of a CRV
• A Cumulative Distribution Function (CDF) is a function that describes the probability that a random variable takes on a value less than or equal to x

Expectation and Variance

• The expectation of a random variable is its long-run average value
• The variance of a random variable is the average of the squared differences between the variable and its expectation