Probability and Probability Distributions

• Introduction to Probability and Statistics is the first chapter, titled "Probability and Probability Distributions" by Dr. Ahmed Kafafy-FCI Menoufia University.

Probability

• If a coin is tossed 10 times and gets 10 heads in a row, it is likely an unfair coin.
• If the coin is fair, the chance of having 10 heads in a row is less than 0.1%, according to probability theory.
• Probability is a tool and foundation of statistics; to evaluate the reliability of statistical conclusions.

Basic Concepts

• An experiment is the process by which an observation (or measurement) is obtained.
• Examples of experiments include:
  • Recording an age
  • Toss a die
  • Recording an opinion (yes, no)
  • Tossing two coins
• A simple event is the outcome that is observed on a single repetition of the experiment.
  • It represents the basic element to which probability is applied.
  • Only one simple event can occur when the experiment is performed.
  • Denoted by E with a subscript
• Each simple event must be assigned a probability, measuring "how often" it occurs.
• A sample space is the set of all simple events of an experiment, usually denoted by S.
• An event is a collection of one or more simple events.
  • For a die toss example: If A is an odd number, and B is a number > 2. A = {E1, E3, E5} and B = {E3, E4, E5, E6}.
• Two events are mutually exclusive if, when one event occurs, the other cannot, and vice versa.
  • Example: Toss a die
  • A is observing an odd number
  • B is observing a number greater than 2
  • C is observing a 6
  • D is observing a 3

The Probability of an Event

• Probability of an event A measures how often we think A will occur (P(A)).
• Suppose that an experiment is performed n times. The relative frequency for an event A is the number of times A occurs divided by n.
• Where "n" gets infinitely large P(A) = lim (f/n) {as n approaches infinity}
• P(A) must be between 0 and 1.
  • If event A can never occur, P(A) = 0.
  • If event A always occurs, P(A) = 1.
• The sum of the probabilities for all simple events in S equals 1 (P(S) = 1).
• The probability of an event A can be found by adding the probabilities of all the simple events in A.
• Probabilities can be found using:
  • Estimates from empirical studies
  • Common sense estimates based on equally likely events
• Examples:
  • Toss a fair coin where P(Head) = 1/2
  • 10% of the U.S. population has red hair. If a person is selected at random, P(Red hair) = 0.10
• The probability of observing at least one head (event A) in 2 coin tosses: P(at least 1 head) = P(A) = P(E1) + P(E2) + P(E3) = 1/4 + 1/4 + 1/4 = 3/4
• The probability of observing exactly one head (event B) in 2 coin tosses: P(exactly 1 head) = P(B) = P(E2) + P(E3) = 1/4 + 1/4 = 1/2
• A bowl contains three M&Ms: two reds and one blue.
  • A child selects two M&Ms at random, the probability of observing exaclty two reds is 1/3.
• A bowl contains three M&Ms: one red, one blue and one green.
  • A child takes two M&Ms randomly one at a time.
  • The probability that at least one is red, equals 2/3.
• If a Fair coin is tossed 3 times, probabilities are:
• A = {HHH, HHT, HTH, THH}
• P(at least 2 heads) = P(A) = P(HHH)+P(HHT)+P(HTH)+P(THH) = 1/8 + 1/8 +1/8 + 1/8 = 1/2
• B={HHT,HTH,THH}
• P(Exactly 2 heads) = P(B) = P(HHT) + P(HTH) + P(THH) = 1/8 + 1/8 + 1/8 = 3/8
• A: at least two heads; B: exactly two heads; C: at least two tails; D: exactly one tail.
  • Questions: A and C mutually exclusive? B and D?
  • C={HTT,THT,TTH,TTT}, B={HHT,HTH,THH} Summary of Probability
• An event is a subset of a sample space (E ⊆ S). Both S and Ø are events as well.
• Sample spaces can be continuous or discrete.
  • What is a continuous vs. discrete sample space?
  • Example: Life in years of a component. S = ?
  • S = {t | t ≥ 0} => "all values of t such that t ≥ 0"
  • A = component fails before the end of the fifth year.
• Example: Flip a coin three times. S = ? {HHH, HHT, HTH, HTT, THH, THT, TTH, TTT}
  • Event A = 1st flip is heads. A={HHH, HHT, HTH, HTT}

Event/Set Operations

• The complement of an event A?
  • The set of all elements of S not in A. Denoted AC.
  • A = 1st flip is heads. A' = first flip is not heads.
• The intersection of two events A and B?
  • The set of all elements in both A and B. Denoted A ∩ B.
  • B = 2nd or 3rd flip, but not both, are heads.
  • B = {HHT, HTH, THT, TTH}. A ∩ B = ?
  • A ∩ B = {HHT, HTH}
  • Two events are mutually exclusive if...? A ∩ B = Ø
• The union of two events, A and B?
  • The set of elements in either A or B. A ∪ B = ?
• The Venn Diagrams visually presents events and can be helpful in showing set theory problems.
  • Standard set theory results hold:
  • A ∩ Ø = ? AUØ = ? - A ∩ A' = ? A∪A' = ?
  • S' = ?– (A')' = ?– (A∩B)' = ?– (A∪B)' = ? – (A∩B)=A' ∪ B', (A∪B)'=A'B'

Counting Rules

• If the simple events in an experiment are equally likely, we can calculate P(A) = (#A / #S)
• #A= number of simple events in A
• #S= total number of simple events

Basic Examples

• How many ways are there from A to C?
  • City A to B to C represents 3 X 2 = 6
• How many ways are there from A to D?
  • City A to B to C to D represents 3 X 2 X 2 = 12
• For a two-stage experiment of "m" ways, and "n" ways, m*n ways to accomplish the whole experiment. This is the multiplication rule.
• For a k-stage experiment, the number of ways equals to n1n2n3...nk
  • Example: Toss two coins. The total number of simple events is: 2 × 2 = 4

Practical Examples

• Toss three coins, the total number of simple events is: 2 × 2 × 2 = 8
• Toss two dice, the total number of simple events is: 6 × 6 = 36
• Two M&Ms are drawn in order from a dish containing four candies, the total number of simple events is: 4 × 3 = 12

Permutations vs Combinations

• Permutations (Order is important): How many 3-digit lock passwords can we make by using 3 different numbers among 1, 2, 3, 4 and 5? Pn! / (n-r)! where n!= n(n-1)(n-2)...(2)(1) and 0!=1` P=(5!/(5-3)!) =5(4)(3)(2)(1)2(1)= 60
• Example: A lock consists of five parts and can be assembled in any order, where the order of the choice is important: P= (5!/(5-5)!)= 5(4)(3)(2)(1) = 120
• Example: How many ways to select a student committee of 3 members: chair, vice chair, and secretary out of 8 students? , where the order of the choice is important: P8 3=(8!/(8-3)!)= (8)(7)(6)(5)(4)(3)(2)(1) / 5(4)(3)(2)(1)=8(7)(6) = 336
• Combinations (Order is not important) : n distinct objects, select r objects at a time without regard to the order where the number of different ways is: Cn! r!(n-r)!
• Example: Three members of a 5-person committee must be chosen to form a subcommittee. How many different subcommittees could be formed where there is no order `C= (5!/(3!(53)!)) = 5(4)(3)(2)1 /3(2)1(2)(1) = 10
• Example: How many ways to select a student committee of 3 members out of 8 students where you don't assign positions (chair, vice chair and secretary) where the order is NOT important: C3=8!/(3!(83)!) = (8(7)(6)(5)(4)(3)(2)(1)) / [3(2)(1)][5(4)(3)(2)(1)] = (8(7)(6))/3(2)(1) = 56/1
• Example Question: A box contains 7 M&Ms: 4 reds and 3 blues. A child selects three M&Ms at random. What is the probability that exactly one is red. The simple event is:
  • size of sample space, in solution.
• Event Relations
  • The union of two events, A and B where events are: either A or B happens, or both, where you write A∪B.
  • The intersection of two events, A and B where events are: both A and B, where you write A∩B. If two events are mutually exclusive, the P(A∩B) = 0.
  • The complement of an event A consists of all outcomes of the experiment that do not result in event A.
  • Where write AC to show "The event that event A doesn't occur".

Example Questions

• Select a student from a college:
  • A is student is colorblind
  • B is student is female
  • C is student is male
  • Mutually exclusive events occur where B = CC. AC is a student that is not colorblind, B∩C means a Student is both male and female which is illogical and BUC: Student is either male or female = all students = S.
• Toss a coin twice and name: A: At least one head {HH, HT, TH};
  • Select: B: Exact one head {HT, TH};
• C: At least one tail {HT, TH, TT}. AC is {TT} "No head"
• A∩B is {HT, TH} which is shown as"Exact one head"
• AUC is {HH, HT, TH, TT}= S as the full "Sample space"

Probabilities for Unions

• For any two events, A and B, the probability of their union, P(A∪B), is P(A∪B) = P(A) + P(B) – P(A∩B).

Examples of Additive Rule

• If there are 1000 students: A are color blind, B are male, A = 42/1000=.042, B = 510/1000=.51, P(AUB) = P(A) + P(B) – P(A∩B) `= 42/1000+ 510/1000 – 40/1000 = 512/1000 = .512 "A special case": when you have two events A and B are mutually exclusive, so P(A∩B) = 0 , and P(AUB) = P(A) + P(B)
• Another Example: A: male and colorblind is P(A) = 40/1000 B: female and colorblind is P(B) = 2/1000 - A and B are mutually exclusive so that

P(AUB) = P(A) + P(B) = 40/1000 + 2/1000 = 42?1000=.042

• In Example 4-31 of 2500 persons, there is both: 1400 that are female, has 600 as vegetarian, and 400 are female and vegetarian. So the probability that a randomly selecting person from this group is a male or vegetarian is: P(M or V) = P(M)+ P(V) ? P(M and V) = (1100/2500) + (600/2500) - (200/2500) = .44+.24-.08 = .60
• Probabilities for Complements, you know that for any event A : P(A∩Ac)=0 Since either A or Ac must occur, P(A∪Ac)=1,

Conditional Probabilities

• The probability that A occurs, given that event B has occurred can be given as: If P(B) ≠ 0, P(A|B) = P(A∩B)/P(B)
• Example: fair coin twice.
• To define: A has head on second toss, B is head on first toss
• If B occurred, the probability of A occurring means that the P(A given B occurred) = 1/2
• If B didn't occur, what is probability that A occurred where the P(A given B did not occur) = 1/2. A, B does not change if there is an independent.
• In the previous example, you found P(A∩B) with: direct from the table (impractical or impossible), however the rule of calculation to P(A∩B), is dependent of the idea of independent and dependent.

Key notes

• Two events of A and B are said to be independent if and only if, the probability that event A occurs is not changed by the occurrence of event B, or vice versa.
• A fair coin is tossed twice. Define:
  • A: head on second toss , B: head on first toss` => A∩B: head on both first and second then is: P(A|not B) = 1/2- the occurrence has been altered by first event
• A bowl contains five M&Ms, has two reds, three blue There are randomly two select two candies, and define: , B: First candy is blue where candy does change, depending on depends on whether A and B, is known as "dependent"

Additive rule for interections

• In previous examples here you have the
• Multiplicative Rule for Intersections** where for two events occurs if probability that both events occurs : PA/ B= PA =Pgiven occurrence
• Independent eventsPA/ B=!PA /!B

Example

• With 100 employees, of: "What is the probability that a randomly selected person from this group is a male or vegetarian?" if you have the info or male and vegetarian (4,31). - - Let us define the following events/ where If you have the following you can also know P(M) + P(V)– P(M and V) =.64-.2.4.6
• Key terms and definitions For any event you now have: AC is 1-p

Conditional Probs

Let: Event 1 H) . the occurrence. a fair toss, the probability formula : P(A ∩ )100 = P

Defining independence

• We can refind conditional probability we events
• two independent the. B p/ P(a (Other theyare
• conditional . P 51 has its

Law of probability

• We event that: = and they the be a

Bayes Thm is a way

• that a the:B + .B1 to likelihood
• S is - If . P is is *what happen.19

Discrete R Vs & probability & key concepts

• variable value random what it with value random events + 2

Expected value formula Discrete

• ExpectedValue: Expm * the - In game has.2 and .8 for , a

• the with value variable can

Example: finding number tosses

Computing mean with

• Find expected
• Let r a distrete randm mean. distion then ,2+p, what is

Finding variance in Example coin flips

Variance and standard the = value

Key concepts summary and probability

• The law. P0 0 P= 4 the has for event

Key concepts: IV evens relation

In, is mutually or and and is and= you know, is also P(!A

Extra notes

• KeyConcepts:54 +random variables properties
• Expected A+ for =0
• What = A/B = P for - 121 for 2 what occur? what in =P and for, The is is always a *and a : .7 =what