Probability Lecture Notes PDF

MEDITERRANEAN SCHOOL OF BUSINESS PROGRAM: Pre-MBM COURSE: Introduction to Statistics PROFESSOR: Dr. Sami Mahfoudhi TERM: Fall 2024 1 Introduction to Probability Key Terms Probability. The probability of an event can be explained as the proportion...

MEDITERRANEAN SCHOOL OF BUSINESS PROGRAM: Pre-MBM COURSE: Introduction to Statistics PROFESSOR: Dr. Sami Mahfoudhi TERM: Fall 2024 1 Introduction to Probability Key Terms Probability. The probability of an event can be explained as the proportion of times, under identical circumstances, that the event can be expected to occur. ▪ It is the event's long-run frequency of occurrence. ▪ For example, the probability of getting a head on a coin toss =.5. If you toss a coin repeatedly, for a long time, you will note that a head occurs about one half of the time. PROBABILITY 3 Key Terms Objective probabilities are long-run frequencies of occurrence, as above. ▪ Probability, in its classical (or, objective) meaning refers to a repetitive process, one which generates outcomes which are not identical and not individually predictable with certainty but which may be described in terms of relative frequencies. These processes are called stochastic processes (or, chance processes). ▪ The individual results of these processes are called events. Subjective probabilities measure the strength of personal beliefs. ▪ For example, the probability that a new product will succeed. PROBABILITY 4 Key Terms Random variable. That which is observed as the result of a stochastic process. A random variable takes on (usually numerical) values. Associated with each value is a probability that the value will occur. For example, when you toss a die: P(1) = P(2) = P(3) = P(4) = P(5) = P(6) = 1/6 ▪Simple probability. P(A). The probability that an event (say, A) will occur. ▪Joint probability. P(A and B). P(A ∩ B). The probability of events A and B occurring together. ▪Conditional probability. P(A|B), read "the probability of A given B." The probability that event A will occur given event B has occurred. PROBABILITY 5 Probability: Some Basic Rules Rule 1. The probability of an event (say, event A) cannot be less than zero or greater than one. The probability that you will pass this course cannot be 150%. So, 0 < P(A) < 1 Rule 2. The sum of the probabilities of all possible outcomes (events) of a process (or, experiment) must equal one. So, ∑Pi = 1 or P(A) + P(A') = 1 [A' means not A.] PROBABILITY 6 Probability: Some Basic Rules Rule 3. Rules of addition. a. P(A or B) = P(A U B) = P(A) + P(B) if events A and B are mutually exclusive. ▪ Two events are mutually exclusive if they cannot occur together. For example, heads or tails; defective or non-defective. b. In general, P(A or B) = P(A) + P(B) - P(A and B) ▪ This is the general formula for addition of probabilities, for any two events A and B. P(A and B) is the joint probability of A and B occurring together and is equal to zero if they are mutually exclusive (i.e., if they cannot occur together). ▪ Thus, the probability of getting a 1 or 2 when tossing a die: P (1 or 2) = P (1) + P (2) = 1/6 + 1/6 = 1/3; The probability of getting a 1 and 2 when tossing a die is 0, since they are mutually exclusive. P (1 and 2) = 0. PROBABILITY 7 Probability: Some Basic Rules MUTUALLY EXCLUSIVE EVENTS, IN VENN DIAGRAMS: P(A∩B) A B A B Events A, B are not mutually exclusive. Events A,B are mutually exclusive. P(A∩B) is the intersection of A and B. PROBABILITY 8 Probability: Some Basic Rules ▪Also, we can see from the Venn diagrams that: → P(A′) = P(not A) = 1 – P(A) →P(A′ or B′) = 1 – P(A and B) ▪Example: Suppose you go to a college where you are allowed to double major. 10% of students major in accounting (A); 15% major in business (B); 3% are double majors (both A and B). What is the probability of being an accounting or business major? [Hint: The events are not mutually exclusive.] Answer: P(A or B) = P(A ∪ B) =.10 +.15 -.03 =.22 PROBABILITY 9 Probability: Some Basic Rules Rule 4. Rules of multiplication are used for determining joint probabilities. A joint probability is the probability that two events will occur together. a. P(A and B) = P(A ∩ B) = P(A)⋅P(B) if events A and B are independent. Events A and B are independent if knowledge of the occurrence of B has no effect on the probability that A will occur. PROBABILITY 10 Probability: Some Basic Rules ▪ Independent events. Events A and B are independent if knowledge of the occurrence of B has no effect on the probability that A will occur. ✓ Events A and B are independent if: P(A|B) = P(A) OR if P(A and B) = P(A)P(B) ✓ Some Examples: 1. P(Blue eyes | Male) = P(Blue eyes) because eye color and sex are independent. 2. P(over 6 feet tall | Female) ≠ P(over 6 feet tall) 3. P(getting into car accident | under 26) ≠ P(getting into car accident) 4. P (getting head on second toss of coin I got head on first toss of coin) = ? Answer =.50 The two coin tosses are independent. What happened during the first coin toss has no effect on what will happen during the second coin toss. PROBABILITY 11 Probability: Some Basic Rules Multiplication Rule 4b. In general, P(A and B) = P(A|B) P(B) = P(B|A) P(A) ▪ This is the general formula for multiplying probabilities for any two events A and B, not necessarily independent. ▪ P(A|B) is a conditional probability. ▪ Note that if events A and B are independent, then P(A|B) = P(A) and P(A and B) reduces to P(A)P(B), as in 4a above. Rule 5. Conditional probability. From the formula in 4b, we see that we can compute the conditional probability as P(A|B) = P(A and B) / P(B) PROBABILITY 12 Probability: Some Basic Rules 6. Bayes' Theorem [OPTIONAL TOPIC] Since P(A|B) = P(A and B) / P(B) and P(A and B) = P(B|A) P(A) Therefore P(A|B) = P(B|A) P(A) / P(B) [We generally need to compute P(B)]. PROBABILITY 13 Example: Readership In a small village in upstate New York, we are looking at readership of the New York Times (T) and the Wall Street Journal (W): P(T) =.25 P(W) =.20 P(T and W) =.05 Question: What is the probability of being either a New York Times reader or a Wall Street Journal reader? Answer: P (T or W) = P(T) + P(W) – P (T and W) =.25 +.20 -.05 =.40 PROBABILITY 14 Example: Readership So, out of, say, 100 people in total: 25 people read NYT (T) Another way to solve this 20 people read WSJ (W) problem, by Venn Diagram 5 people read both 60 people read neither Thus it can be easily seen that 40 people (out of 100) read either T W the NYT or the WSJ. PROBABILITY 15 Example: Readership Let’s compute some other probabilities: P(T′ and W′) =.60 T P(T′ and W) =.15 W P(T and W′) =.20 P(T or W) = 1 – P(T′ and W′) = 1–.60 =.40 P (T′ or W′) = P(T′) + P(W′) – P(T′ and W′) =.75+.80 -.60 =.95 What does this mean? 95% of people in this town do not read at least one of the two papers; indeed, only 5% read both. We note that: P(T′ or W′) = 1- P(T and W) = 1 -.05 =.95 PROBABILITY 16 Example: Readership Another way to do solve this problem is to construct a table of joint probabilities: T T′ W.05.15.20 W′.20.60.80.25.75 1.00 PROBABILITY 17 Independence vs. Mutually Exclusive Events Important: Do not confuse mutually exclusive and independent. If two events are mutually exclusive, they cannot occur together; we only examine events for independence (or, conversely, to see if they are related) if they can occur together. Independence has to do with the effect of, say B, on A. Mutually exclusive means that If knowing about B has no effect on A, then two things cannot occur at the they are independent. same time It is very much like saying that A and B are [P (A and B) = 0] unrelated. Example: Are waist size and gender independent of each other? ▪ You cannot get a head Suppose I know that someone who is an adult has a 24-inch and tail at the same time. waist, does that give me a hint as to whether that person is male ▪ You cannot be dead and or female? How many adult men have a 24-inch waist? How alive at the same time. many women? Is P (24-inch waist/adult male) = P (24 inch waist/adult female). We suspect that the two probabilities are ▪ You cannot pass and fail a not the same, that there is a relationship between gender and course at the same time. waist size (also hand size and height for that matter). Thus, they are not independent. PROBABILITY 18 Independence vs. Mutually Exclusive Events Researchers are always testing for relationships. Sometimes we want to know whether two variables are related. ✓Is there a relationship between cigarette smoking and cancer or are they independent? We know the answer to that one. ✓Is there a relationship between your occupation and how long you will live (longevity) or are they independent? Studies show that they are not independent. Librarians and professors have relatively long life spans; coal miners have the shortest life spans. Drug dealers (is that an occupation?) also have very short life spans. PROBABILITY 19 Example: Smoking and Cancer S S′ (smoke (non- Data in a Contingency Table r) smoker) C (cancer) 100 50 150 Are smoking and cancer independent?C′ (no 300 550 850 Joint Marginal Probabilities cancer) Probabilities P(C and S) = P(C) =.15 400 600 1000.10 (150/1000) P(C` and S) = P(C`) =.85.30 (850/1000) P(C and S`) = P(S) =.40.05 (400/1000) P(C` and S`)= P(S`) =.60.55 (600/1000) PROBABILITY 20 Example: Smoking and Cancer Are smoking and cancer independent? To answer, let’s check: Are the following probabilities equal or not? P(C) ≟ P(C|S) ≟ P(C|S′) P(C|S) = P(C and S)/P(S) =.10/.40 =.25 P(C|S′) = P(C and S′)/P(S′) =.05/.60 =.083 P(C) =.15 Thus, cancer and smoking are not independent. There is a relationship between cancer and smoking. Note that P(C) is a weighted average of P(C|S) and P(C|S′): P(C) = P(S)P(C|S) + P(S′)P(C|S′) =.40(.25) +.60 (.0833) =.15 PROBABILITY 21 Example: Smoking and Cancer Are smoking and cancer independent? ▪ Alternate method of solving this problem: If C and S are independent, then P(C and S) = P(C) P(S).10 ?= (.15)(.40).10 ≠.06 ▪ Since.10 is not equal to.06, we conclude that cancer and smoking are not independent. ▪ These calculations are much easier if you set up a joint probability table. Coming up in the next two slides. PROBABILITY 22 Example: Smoking and Cancer S(smoker) S′(non- smoker) 100/100 50/1000 150/1000 C (cancer) 0 We use the frequencies in the 300/100 550/1000 850/1000 contingency table to compute the C′ (no 0 cancer) probabilities in the joint probability 400/100 600/1000 1000/100 table. 0 0 S S′ Notice that the marginal probabilities are (smoker) (non-smoker) the row and column totals. This is not an C (cancer).10.05.15 accident. The marginal probabilities are totals of the joint probabilities and C′ (no cancer).30.55.85 weighted averages of the conditional.40.60 1.00 probabilities. PROBABILITY 23 Example: Smoking and Cancer S S′ (smoker) (non-smoker) C (cancer).10.05.15 C′ (no cancer).30.55.85.40.60 1.00 Once we compute the table of joint probabilities, we can answer any question about a probability in this problem. ▪ The joint probabilities are in the center of the table; ▪ the marginal probabilities are in the margins, and ▪ the conditional probabilities are easily computed by dividing a joint probability by a marginal probability. P(C) =.15 P(S) =.40 P(C and S) =.10 P(C|S) = P(C and S) / P(S) =.10 /.40 =.25 PROBABILITY 24  Example: Gender and Black Coffee M (male) F (female) B (b coffee drinker) 450 350 800 B′ (not a b coffee 450 750 1200 drinker) 900 1100 2000 M (male) F (female) Joint Probabilities Marginal Probabilities B (b coffee drinker).225.175.40 P(B and M) =.225 P(B) =.40 P(B and F) =.175 P(B′) =.60 B′ (not a b coffee.225.375.60 Joint Probabilities Marginal Probabilities drinker) P(B and M) =P(B′.225and M) =.225 P(B) =.40 P(M) =.45 P(B and F) =P(B′.175and F) =.375 P(B′) =.60 P(F) =.55.45.55 1.00 P(B′ and M) =.225 P(M) =.45 P(B′ and F) =.375 P(F) =.55 PROBABILITY 25 Example: Gender and Black coffee M F B.225.175.40 B′.225.375.60.45.55 1.00 ▪Question. Given that an individual is Male, what is the probability that that person is a black coffee drinker? Answer. P(B|M) = P(B and M)/P(M) =.225/.45 =.50 ▪Question. Given that an individual is Female, what is the probability that that person is a black coffee drinker? Answer. P(B|F) = P(B and F)/P(F) =.175/.55 =.318 ▪Question. Are black coffee drinking and gender independent? Answer. P(B) =.40, therefore, black coffee drinking and sex are not independent. PROBABILITY 26 Example: Gender and Dove Soap Data collected from a random sample of 1,000 adults: M F (female) (male) D (use Dove soap) 80 120 200 D′ (does not use Dove 320 480 800 soap) Computed probabilities: 400 600 1,000 Joint Probability Marginal Totals P(D and M) =.08 P(D) =.20 P(D and F) =.12 P(D′) =.80 P(D′ and M) =.32 P(M) =.40 P(D′ and F) =.48 P(F) =.60 PROBABILITY 27 Example: Gender and Dove Soap Are the events Gender and Use of Dove Soap independent? Alternative method: Is P(M and D) ≟ P(M) P(D) ? P(D)= 200/1000 =.20 P(M and D) =.08 P(D|M) = 80/400 =.08/.40 =.20 P(M) = 400/1000 =.40 P(D|F) = 120/600 =.12/.60 =.20 P(D) = 200/1000 =.20 Yes, these two events are Is.08 ≟ (.40)(.20)? independent. Yes, therefore the two events M and D are independent. PROBABILITY 28