Probability Rules and Theoretical Probability

Probability

• The theoretical probability of event A is the number of outcomes in A divided by the total number of outcomes, assuming all outcomes are equally likely.
• Example: The probability of drawing a face card from a standard deck of 52 cards is 12/52.

Probability Rules

• Rule 1: A probability of 0 means the event cannot occur, while a probability of 1 means the event always occurs, thus probability values exist on a scale from 0 to 1.
• Rule 2 (Probability Assignment Rule): The probability of the set of all possible outcomes must be 1, denoted as P(S) = 1, where S is the sample space.
• Rule 3 (Complement Rule): The probability of an event occurring is 1 minus the probability that it does not occur: P(A) = 1 - P(A^C), where A^C is the complement of A.
• Rule 4 (Multiplication Rule): For two independent events A and B, the probability that both occur is the product of their probabilities: P(A and B) = P(A) × P(B). This equation is only valid if A and B are independent.
• Rule 5 (Addition Rule): For two disjoint (or mutually exclusive) events, the probability that either occurs is the sum of their probabilities: P(A or B) = P(A) + P(B).
• Rule 6 (General Addition Rule): Calculates the probability that either of two events occurs, even if they are not disjoint: P(A or B) = P(A) + P(B) – P(A and B).

Complement of an Event

• The complement of an event A is denoted as A^C.

Disjoint Events

• Disjoint events have no outcomes in common.

Joint Probability

• Describes the probability of two events occurring together.
• Events can be placed in a contingency table.

Marginal Probability

• Depends only on the totals found in the margins of a contingency table.

Conditional Probability

• The probability of an event, given that another event has occurred.
• Denoted as P(B|A), read as "the probability of B given A," calculated as P(B|A) = P(A and B) / P(A).
• Each row or column in a contingency table shows a conditional distribution given one event.

General Multiplication Rule

• Calculates the probability that both of two events occur, even if dependent: P(A and B) = P(A) × P(B|A) = P(B) × P(A|B).

Independent Events

• Events where the occurrence of one does not influence the occurrence of the other.
• If independent, P(A and B) = P(A)P(B). Should not be confused with disjoint events.

Disjoint Events

• Two events are disjoint if only one of them can happen.

Random Variables

• A measure whose outcome is random.
• Can be discrete (countable number of distinct values) or continuous (uncountable number of distinct values).

Random Outcomes

• The outcome of a random variable is random.

Probability Distribution

• Every random variable is associated with a probability distribution.
• Captures the randomness inherent in random variables.

Discrete Probability Distribution

• Described using a probability mass function (pmf), which lists possible outcomes and their probabilities.

Properties of a Discrete Probability Distribution

• The probability of each value x is between 0 and 1, 0 ≤ P(X = x) ≤ 1.
• The sum of the probabilities equals 1, Σ P(X = x) = 1.

Cumulative Distribution Function (CDF)

• For any value x of the random variable X, the CDF is defined as P(X ≤ x).
• Demonstrates the cumulative probability of the random variable X assuming a number up to the value x.

Expected Value

• Represents the weighted average of possible outcomes of a random variable.
• Should not be confused with the most probable value.

Variance

• Describes how the values are dispersed around the expected value.
• A measure of dispersion.
• SD is square root of the variance

Discrete Uniform Distribution

• If X is a random variable with possible outcomes 1, 2, ..., n and for each i, then X has a discrete Uniform distribution U[1, ..., n].

Bernoulli Trial

• A trial with only two possible outcomes (success and failure).
• The probability of success (p) is the same for each trial, the probability of failure is q = 1 – p.
• The trials are independent.

Binomial Distribution

• Predicts the number of successes in a series of Bernoulli trials.
• n = Number of trials, p = Probability of success.
• P(X = k) = (n choose k) * p^k * (1-p)^(n-k), where X is the number of successes in n trials.

Mean and SD (Binomial Distribution)

• The mean is E(X) = np.
• SD is SD(X) = sqrt(npq).

Continuous Random Variables

• A random variable that may take on any value in some interval [a, b].
• The probabilities are shown with a curve, f(x), called a probability density function (pdf).

Continuous Probability density function (pdf)

• For a continuous r.v. X, the counterpart of a pmf.
• The PDF does not provide probabilities directly.
• The area under the PDF f(x) between two values a and b represents the probability P(a ≤ X ≤ b)

Requirements (density functions)

• They must be non-negative for every possible value and the area equals 1.

Continuous R.V. (uniform)

• For values c and d (c ≤ d) both within the interval [a, b]: P(c ≤ X ≤ d) = (d-c) / (b-a).
• E(X) = (a+b)/2
• Var(X) = (b-a)^2/12
• SD(X) = sqrt((b-a)^2 / 12)

Normal Distribution

• Familiar bell-shaped distribution.
• Extensively used continuous probability distribution in statistics.
• The normal distribution is useful in endless cases.

Characteristics (Normal Distribution )

• Completely described by two variables: mean (μ) and standard deviation (σ).
• Denoted by N(μ, σ).
• Asymptotic: the tails get closer but never touch the horizonal axis.

Probabilities calculation

• There is two options for the probability calculate:
  • Using Integrals or Using Software
• However there is an easier way: Convert to standard normal distribution.

Standard Normal Distribution

• It is denoted with letter Z to denote a random variable that is normal and has E(Z) = 0 and Var(Z) = SD(Z) = 1.
• Each value of this random variable (Z) is a z-score
• P(X ≤ x) = P(X − μ ≤ x − μ) = P((X-μ)/σ < (x-μ)/σ) = P(Z ≤ z)

Z-Score Formula

• The formula to calculate z-score: Z = (x-μ)/σ

Z-Values (empirical rule)

• Z-score 2.2 implies that the point is 2.2 standard deviations to the right of the mean.
• Z-score-1.8 implies that the point is -1.8 standard deviations to the left of the mean

Standardization

• An data value x has a corresponding value of z given by,
  • z = (x-μ)/σ

68-95-99.7 Rule

• About 68% of the values fall within one standard deviation of the mean.
• 95% within two standard deviations.
• 99.7% of the values fall within three standard deviations.

The Mean (empirical rule)

• Using the z table it can be confirmed that the empirical rule holds.