Probability: Definitions and Rules

• Includes definitions, methods of assigning probability, axioms, and basic probability rules.

Basic Definitions

• An experiment is a process that produces one or more outcomes.
• The sample space, denoted by ( S ), is the set of all possible outcomes of an experiment.
• An event, denoted by ( E ), is a subset of the sample space.
• An elementary event consists of only one outcome.
• Mutually exclusive events are events with no outcomes in common (disjoint).

Methods of Assigning Probability

• Classical Approach (Equally Likely Outcomes)
  • If an experiment has $n$ possible outcomes, the probability of each outcome is $\frac{1}{n}$.
  • The probability of event ( E ) is: ( P(E) = \frac{\text{Number of outcomes in } E}{\text{Total number of outcomes in } S} = \frac{n(E)}{n(S)} )
• Relative Frequency Approach
  • The probability of an event is approximated by the relative frequency of its occurrence in a large number of trials.
  • ( P(E) \approx \frac{\text{Number of times } E \text{ occurs}}{\text{Total number of trials}} )
• Subjective Approach
  • Probability is based on personal judgment or belief.

Axioms of Probability

• Axioms define the fundamental rules for probability:
  • For any event ( E ), ( 0 \leq P(E) \leq 1 ).
  • The probability of the sample space ( S ) is ( P(S) = 1 ).
  • For mutually exclusive events ( E_1, E_2, E_3, \dots ): ( P(E_1 \cup E_2 \cup E_3 \cup \dots) = P(E_1) + P(E_2) + P(E_3) + \dots )

Basic Probability Rules

• Complement Rule
  • For any event ( E ), ( P(E^c) = 1 - P(E) ), where ( E^c ) is the complement of ( E ).
• Addition Rule
  • For any two events ( A ) and ( B ): ( P(A \cup B) = P(A) + P(B) - P(A \cap B) )
• If ( A ) and ( B ) are mutually exclusive: ( P(A \cup B) = P(A) + P(B) ) since ( P(A \cap B) = 0 ).
• Conditional Probability
  • The probability of event ( A ) given that event ( B ) has occurred is: ( P(A|B) = \frac{P(A \cap B)}{P(B)} ), where ( P(B) > 0 ).
• Multiplication Rule
  • Joint probability can be calculated as: ( P(A \cap B) = P(A|B) \cdot P(B) = P(B|A) \cdot P(A) )
• Independent Events
  • Two events ( A ) and ( B ) are independent if the occurrence of one does not affect the probability of the other.
    • ( P(A|B) = P(A) ), ( P(B|A) = P(B) ), or ( P(A \cap B) = P(A) \cdot P(B) )
• Total Probability Rule
  • For mutually exclusive and exhaustive events ( B_1, B_2, \dots, B_n ), ( P(A) = \sum_{i=1}^{n} P(A|B_i) \cdot P(B_i) ) for any event ( A ).
• Bayes' Theorem
  • ( P(B_i|A) = \frac{P(A|B_i) \cdot P(B_i)}{\sum_{j=1}^{n} P(A|B_j) \cdot P(B_j)} ) for n mutually exclusive and exhaustive events ( B_1, B_2, \dots, B_n ).