Random Experiments and Sample Spaces

Random Experiment and Sample Space

• A random experiment is a process with two or more possible outcomes, but the exact outcome is unknown beforehand
• Examples of random experiments include tossing a coin and a company receiving contract awards
• Basic outcomes are the possible results from it
• The sample space, denoted as S, is the set of all basic outcomes
• No two basic outcomes can happen simultaneously, but one basic outcome must occur after a random experiment

Event, Intersection, and Mutually Exclusive

• An event, E, is a subset of basic outcomes from the sample space
• This occurs if the experiment results in one of its basic outcomes
• The null event, denoted as Ø, represents the absence of a basic outcome
• Events include "{contract rewards are odd}" and "{contract rewards are less than 3}"
• The intersection of two events, A and B, denoted as A ∩ B, comprises basic outcomes belonging to both A and B
• A ∩ B happens only if both A and B happen
• Events A and B are mutually exclusive if they share no common basic outcomes or can’t co-occur (A ∩ B = Ø)
• Events E1, E2,..., Ek are mutually exclusive as pairwisely mutually exclusive

Complement

• The complement of A, denoted as Ā (or Aᶜ), includes basic outcomes in S but not in A

Union, Collectively Exhaustive, and Partition

• The union of two events, A and B, denoted as A ∪ B, includes basic outcomes in at least one of A or B
• A ∪ B occurs if either A or B or both occur
• If E1 ∪ E2 ∪ ... ∪ EK = S, these K events are collectively exhaustive
• A mutually exclusive and collectively exhaustive set of events {Bi}Ki=1 is a partition of sample space S
• Exactly one of the events {Bi}Ki=1 must be true
• The set of all basic outcomes is a partition of S, as are {A, Aᶜ} and {A ∩ B, A − (A ∩ B), B − (A ∩ B), Aᶜ ∩ B ᶜ}
• Any event A can also be partitioned in the same way

Classical Probability

• Three definitions of probability considered are classical, relative frequency, and subjective probability
• Classical probability is the proportion of times an event occurs, assuming equal likelihood of all outcomes in the sample space
• P(A) = NA / N, where NA is outcomes satisfying event A and N is the total outcomes in the sample space
• The number of combinations of x objects chosen from n is computed using the formula: Cnx = n! / (x! (n – x)!), with 0! = 1

With/Without Replacement and Ordered/Unordered Counting

• There are two distinctions when counting objects in a set: with and without replacement, and ordered or unordered
• Formulas for each type:-
  • Ordered without replacement = n!/(n-x)!
  • Unordered without replacement = 𝐶𝑥𝑛
  • Ordered with replacement = 𝑛𝑥
  • Unordered with replacement = 𝐶𝑥𝑛+𝑥−1
• Pxn is the number of permutations of x objects chosen from n
• Unordered counting with replacement involves finding distinct solutions to z1 + z2 + … + zn = x, where zi ∈ {0, 1, 2, ..., x}

Relative Frequency and Subjective Probability

• Relative frequency probability is the limit of the proportion of times an event occurs in many trials
• P(A) = nA / n, where nA is the A outcomes and n is the total trials or outcomes
• Probability is the limit as n approaches infinity
• Subjective probability is an individual's degree of belief about the chance an event occurs
• This probability is personal, so different individuals will have different information or views

Probability Postulates

• Properties of probability to assess and manipulate probabilities
  • For event A in S: 0 ≤ P(A) ≤ 1, where 0 means impossible and 1 means certain
  • For event A in S and basic outcomes Oi: P = ∑ P(Oi)
    • Oi∈A
  • P(S) = 1

Consequences of the Postulates

• If S consists of n equally likely basic outcomes, then 𝑃(𝑂𝑖 ) = 1/n
• If S consists of n equally likely basic outcomes and event A consists of nA of these outcomes, then 𝑃(𝐴) = 𝑛𝐴/n
• If A and B are mutually exclusive, then P(A ∪ B) = P(A) + P(B)
• For collectively exhaustive events, the union of its probabilities is 1

Complement Rule and Addition Rule

• Complement rule: for an event A and its complement Aᶜ, P(Aᶜ) = 1 − P(A) This is because 1 = P(S) = P(A ∪ Aᶜ) = P(A) + P(Aᶜ)
• Addition rule: for two events A and B, P(A ∪ B) = P(A) + P(B) − P(A ∩ B)
• P(A ∩ B) is the joint probability of A and B

Conditional Probability

• Conditional probability of event A given event B, denoted as P(A | B): P(A | B) = P(A ∩ B) / P(B), provided that P(B) > 0
• Likewise, P(B | A) = P(A ∩ B) / P(A), given P(A) > 0
• Filter or stratify the data to calculate relative frequency
• This cannot be smaller than
• Probability assigned to an event depends on the knowledge we condition on

Multiplication Rule and Statistical Independence

• Multiplication rule: For events A and B, P(A ∩ B) = P(A | B)P(B) = P(B | A)P(A)
• Events A and B are statistically independent iff P(A ∩ B) = P(A)P(B)
• Can be redefined as P(A | B) = P(A) if P(B) > 0 or P(B | A) = P(B) if P(A) > 0
• Events E1, E2, ..., Ek are mutually independent iff: P(E1 ∩ E2 ∩ ... ∩ Ek) = P(E1)P(E2)...P(Ek)

Continue (Independence)

• Independence between A and B implies knowing B occurred does not change the assessment of A's probability
• Approximately assume independence for simplicity
• P(having COVID-19) ≈ P(having COVID-19 | your friend Joe is 42 years old)
• If events A and B are not independent, they are dependent
• Dependence/independence are symmetric relations
• If A is dependent/independent on B, then B is dependent/independent on A P(A | B) = P(A) ⟹ P(B | A) = P(B)
• Independence differs from "mutually exclusive": the latter implies P(A ∩ B) = 0 and the former means P(A ∩ B) = P(A)P(B)
• A ∩ B = ∅ equals "if A occurs, then B cannot", so they are not independent unless P(A) or P(B) are zero

Examples (Cell Phone Features and Birthday Problem)

• 75% use texting (A), 80% use photo (B), 65% use both (A ∩ B)
• Then P(A ∪ B) = P(A) + P(B) − P(A ∩ B) = 0.75 + 0.80 − 0.65 = 0.90
• P(A | B) = P(A∩B)/P(B) = 0.65/0.80 = 0.8125 (probability a person has texting who has photo)
• P(B | A) = P(A∩B)/P(A) = 0.65/0.75 = 0.8667 (probability a person has photo who has texting)
• You can determine the probability that at least two people share same birthday (neglecting Feb 29) by using a probability calculation 𝑃(𝐴) = 1 − 𝑃(𝐴∁):

Bivariate Probabilities

• Events Ai and Bj are mutually exclusive and collectively exhaustive within their sets
• Interactions Ai ∩ Bj can be regarded as basic outcomes of a random experiment
• Probabilities P(Ai ∩ Bj) are called bivariate probabilities

Joint and Marginal Probabilities

• P(Ai ∩ Bj) are joint probabilities while P(Ai) or P(Bj) are marginal probabilities and put at the margin of a table
• Marginal probabilities P(Ai) (or P(Bj)) are obtained by summing probabilities for a row or column or from tree diagrams

Law of Total Probability

• Given a partition of S, {Bj}Kj=1 , it is not hard to see that {A ∩ Bj}Kj=1 is a partition of A. So we have the law of total probability
• 𝑃(𝐴) = ∑ (𝐾𝑗=1) 𝑃(𝐴 ∩ 𝐵𝑗 )
• Calculating P(A) this way is called over {Bj}Kj=j , and the resultis marginalizingprobability P(A) of course marginal probability of A.
• Because 𝑃(𝐴\ ∩ 𝐵𝑖 ) = 𝑃(𝐴 | 𝐵j)𝑃( B), then 𝑃(𝐴) = ∑(𝐾𝑗=1) |Bj)𝑃 (𝑗 )

Conditional Probabilities and Independent Events

• It is important to look at the equations ∑(𝐻 | 𝐵𝑗 ) = ( =1 , B𝑗 ) / ( B𝑗 ))= 1

Overinvolvement Ratios

• Desired conditional probabilities are hard to obtain, but alternative ones are available: Given A₁, and mutually exclusive/exhaustive B₁ and B₂, the overinvolvement ratio is ℙ(𝐴₁ B₁) /ℙ (A₁ | B₂)
• If A1 “sees advertisement”, B1 is "purchasing our products" and B₂ = 1∁ we observe ( ₁| B₁) and | B₂)
• If >1 event, increase conditional odds ratio

Understand random experiments, sample spaces, and events. Learn about basic outcomes and the concept of null events. Explore event intersections and mutually exclusive events in probability.