AEC301 Probability Theory 2021 PDF

AEC301 Probability Theory Omodara O.D. © 2021 1 BEFORE YOU READ: After this class, you should be able to answer these questions: What is a random probability sampling? How does dependent event different from independent event? How do we threat a mutually exclusive event separately from independent e...

AEC301 Probability Theory Omodara O.D. © 2021 1 BEFORE YOU READ: After this class, you should be able to answer these questions: What is a random probability sampling? How does dependent event different from independent event? How do we threat a mutually exclusive event separately from independent event? What are the rules of combining probability? How do we use Bayes’ rules to determine the probability of a random event? 2 Learning Objectives Define probability explain what objective and subjective probabilities are explain the rules for combining probabilities apply set rules to probability problems differentiate between a depend, independent and mutually exclusive events discuss Bayes’ theorem and how it is used and use theorems of statistics to determine probability of events 3 Definition of Probability We use probability several times in our day to day life. For instance, during the raining season, sometimes we predict that it will rain and there will be good harvest. This statement is a subjective probability because it is based on things we have learnt or experienced but it is acceptable because there may be some chances that the expected outcome will happen. The statement becomes objective probability if the tendency that the event will result in desired outcome is based on facts - concrete evidence from recorded observation or long history of collected data. This evidence is then expressed as a fraction of all possible outcomes of the event. Thus in objective probability, we use statistics, experiment and mathematical measurements rather than guesses and personal judgement to determine the outcome of any event. Probability defined Probability is the magnitude of possibility that an event will occur. It explores all position chance(s) that the expected outcome of a random situation will either happen not happen. For an event to be considered probabilistic, it must be measurable, and has a possibility of a success (favourable outcome) or failure (unfavourable outcome). Number of favourable cases to that event Probability of an event, P(E) = Total Number of equally likely case 4 Probability theory explained Consider the statement, “it may probably rain today”. Following the above definition, the total number of equally likely cases can, at the most, be two and the number of favourable cases to the event is one. Therefore, the probability that it may rain today is 1Τ2. Thus, if an event can happen in 'a' ways and fail to happen in 'b‘ ways and both options have equal chances then the probability of happening of an event is 𝑎Τ𝑎+𝑏 and probability of not happening is 𝑏Τ𝑎+𝑏. Assuming that H=𝑎Τ𝑎+𝑏 and T=𝑏Τ𝑎+𝑏 then, H+T=1…………………………...... (i). This means that probability always lies between 0 and 1. So If H=1 and T=0, the event will certainly happen. If H=0 and T= 1, the event will certainly not happen. Therefore, H/T refers to the odds in favour of the event and T/H as the odds against the event. Giving that a coin is tossed once, the possible outcome will either be “head”, symbolized by H (or 1) and “tail” symbolized by T (or 0). The elements of this set of event are {H, T} or {0, 1}. If the coin is tossed twice, its possible outcomes are four {HH, TT, HT, TH}. The probability of one of these random events occurring can then be determined. In random experiments, the study of probability becomes dynamic. 5 Random experiment explained Random experiments refers to scientific experiments where the value of certain variables that affect the outcome of the experiment can not be controlled, changed or influenced. Example of random experiment is drawing of card from a pack of cards: Assuming we are asked to determine the probability of drawing an Ace from a pack of cards, we must know that the Total No. of favourable cases for drawing an Ace is 4 and Total number of card in a pack is 52. Recall, P(E) = Number of favourable cases for drawing an Ace 4 1 then, P(E) = = or 0.077 52 13 Total Number of cards in a pack 1. In 13 other words, there is 7.7% chance that a card drawn at random from a pack of cards will be an Ace. Thus, the probability that a card drawn at random form a pack of cards will be an ace is 0.077 or The set of all possible outcomes of a random event is known as a sample space (S) and an event is a subset (A) of a sample space. In the example above, an event is the probability that if a coin is tossed twice, HH will occur. Using set operations, we can obtain other events in S, 6 Some Important Theorems in Probability Suppose we are given sample space X. If X is discrete, all subsets corresponds to events and vice versa. However, if X in not discrete, only the measurable subsets correspond to events. For every event A is in a class X, P(A)≥0. If event A is sure or certain, then, P(A)=1. if an event can never occur such event is an “empty or null set”, symbolized by () or Ø. The probability of an empty set P() =0 If “either A or B or both” event will occur, it is called the union of events A and B, (AUB) If “both A and B” events will occur, it is the intersection of A and B, (A ∩ B) A’ is the event “not A”, and it is called complement of A. If A’ is the complement of A, then, P(A’)= 1 - P(A) If “only event A but not B” will occur, such event is A-B =A ∩ B’ = X – A If the set of event A and B are disjoint, i.e. A ∩B= Ø, the even is referred to as mutually 7 exclusive Theorems on probability theory From the Venn diagram, 1. A ∩B = both A and B = A of B 2. A U B = either A or B = both A of B 3. B’ = not B but other options in the Venn diagram 4. A U B’ = A and other options in the Venn diagram except B 5. A’ U B’ = (A ∩ B)’ =Not A nor B = not A of B = any card but A of B 6. A ∩ B’ = A-B = A but not B 7. A’-B’ = not A and not B = not A of B = any options but not A and B 8. (A ∩B) U (A ∩ B’) = A of B or A and not B = A ………………………………………………………….. Fig. 1: Venn diagram showing relationship between A and B with sample space S 11. S = SUØ = Ø, this means that P(S)=P(S)+P(Ø) Then P(Ø) =0. 9. If A ≤ B, then P(A) ≤ P(B) and P(B-A)=P(B) – P(A), 12. If we have A U A’ =S, and A ∩ A’ = Ø, then If A and B are mutually exclusive, then P(A U A’) = P(S); and P(A) + P(A’) = 1 P(B) =P(A)+P(B - A). Then, P(A’) =1- P(A) This makes P(B - A) = P(B) – P(A). 13. If A U B = A U [B- (A ∩ B)]. Since P(B-A) ≥0 above, we can then say P(B) ≥ P(A) Here, sets A and B are mutually exclusive then, 10. To prove that P(A) ≤1, we must first say that A0. For any two events, A and B, P B A =𝑃 𝐴 ∩𝐵 ൗ𝑃 𝐴 ……………………………….……..5 𝑃 𝐴 ∩𝐵 = 𝑃 𝐴 ∗𝑃 𝐵 𝐴 ……………………………...6 This means that the probability that both A and B will occur is equal to the probability that A happens multiply by the probability that B will occur given that A has occurred. In the case of any three events, A, B, C, we have P(A∩B∩C) = P(A)∗P(B/A)∗P(C/A∩B) ……………..…7 so, P(C/A ∩ B) = 𝑃 𝐴∩𝐵∩𝐶 ൘𝑃 𝐴 ∗𝑃 𝐵 𝐴 …………………......... 8 14 Example 4: what is the probability that if a die is tossed once, it will result into a number less than 5 if the toss results is an even number. Solution: there are three events here Let: Event (A) = the event that a toss results into an even number =[2,4] =2 numbers Event (B) = the event that a toss results into a number less than 5 =[1,2,3,4] =4 numbers Event (B/A) =the event that a toss results into an even number less than 5. S = Sample space =[1,2,3,4,5,6] =6 numbers; Each number on a die (sample point) has equal chance to be chosen = 1Τ6 Without any condition, the probability for event B is P(B) = P(1) + P(2) + P(3) = 1Τ6+1Τ6+ 1Τ6+1Τ6 =4Τ6 =2Τ3 Then, under the condition of even number, the probability for event A is P(A) = 1Τ6+1Τ6 =2Τ6 = 1Τ3 P(A ∩ B) = 1Τ3, then P(B/A) = 𝑃(𝐴∩ 𝐵) ൗ𝑃(𝐴) = 1Τ 3 2Τ 3 = 1Τ2 Conditional probability with or without replacement Example 5: two cards are drawn form a deck of 52 playing cards. Find the probability that both cards are kings if the first card is (a) replaced (b) not replaced 15 Solution (a)First card is replaced Let 𝑋1 = event that first card is a King and 𝑋2 =event that second card is a king. Sample space = 52 cards There are 4 kings (𝑋1 ) in a pack of 52 cards so, P(𝑋1 ) = 4Τ52, if the first card is replaced, S remains 52 cards, and P(𝑋2 /𝑋1 ) =4Τ52 Then, P(𝑋1 ∩ 𝑋2 ) =P(𝑋1 )P(𝑋2 /𝑋1 ) = 4Τ52*4Τ52 = 1Τ169 b) first card is not replaced As we have in “a” above, P(𝑋1 ) = 4Τ52, If the first card is picked and not replaced, S contains 51 cards, and there are 3 kings left. So, P(𝑋2 /𝑋1 ) =3Τ51 Then, P(X 1 ∩X 2) =P(X 1 )P(X 2 /X1) = 4Τ52*3Τ51 = 1Τ221 Example 5: A box contains 4 White (W), 8 black (B) and 5 red (R) balls. If two Balls are drawn at random and the outcome were one White (W) and one Black (B): what is the 16 probability of WB (i) If the first balls is not replaced, (ii) if the first ball is replaced. Solution: there are cases here. Senerio1: first ball is not replaced Let S=sample space = 4+8+5= 17 balls (a) If the first ball is not replaced and it is White, probability of drawing white ball, P(W) =4Τ17. If the first ball is not replaced, and the remaining balls in the box are, S= 16 balls. Probability of drawing a Black ball if the first ball is White P(B/W)= 8Τ16. Being independent events, the probability of drawing 1 White and 1 Black ball is P(W∩B)= P(W)P(B/W) = 4Τ17 * 8Τ16 =32Τ272. (b) If the first ball drawn is Black, the probability of drawing a Black ball, P(B) = 8Τ17. Probability of drawing a White ball if the first ball is Black, P(W/B)=4Τ16. Being independent events, the probability of drawing 1 Black and 1 White ball P(B∩W)= 8Τ17 ∗ 4Τ16 =32Τ272. Since (a) and (b) are two mutually exclusive events, the probability of drawing 1 white and 1 black ball is 17 P(B∩W) U P(W∩B) = 32Τ272 + 32Τ272=64Τ272=4Τ17. Senerio2: first ball is replaced If the first ball drawn is white, the probability of drawing a white ball, P(W)=4Τ17. In as much as we replace the first ball, the sample space remains the same. Therefore, the probability of drawing black ball is P(B/A)= 8Τ17. Being independent events, the probability of drawing 1 White and 1 Black ball is P(W∩B)= 4Τ17 * 8Τ17= 32Τ289 If the first ball is replaced and it is Black, Probability of drawing Black ball, P(B) = 8Τ17 And probability of drawing white ball when black ball is replaced is P(W/B)= 4Τ17 Being independent events, the probability of drawing 1 Black and 1 White balls is P(W∩B) = 8Τ17 * 4Τ17 = 32Τ289 Since (a) and (b) events are mutually exclusive, the probability of drawing 1 white and 1 18 black ball is P(B∩W) U P(W∩B) = 32Τ289+32Τ289=64Τ289 Bayes’ theorem Bayes' rule is a general formula for extending conditional probability to finding the probabilities of various events that can cause an event to occur. It is often referred to as theorem on the probability of causes because it dictates how opinions should be revised in the light of new information, to be consistent with the rules of probability. Assuming there are three similar boxes. Box i contains i white balls and one black ball, i = 1,2,3, as shown in the following diagram. Suppose I mix up the boxes and then pick one at random. Then I pick a ball at random from the box and show you the ball, Which box would you guess if the ball drawn is white and what is your chance of guessing right? Suppose, more generally, that events 𝑌1 ,... , 𝑌𝑛 represent n mutually exclusive possible results of the first stage of some procedure. Which one of these results has occurred is assumed unknown. Rather, the result X of some second stage has been observed, whose chances depend on which of the 𝑋𝑖 has occurred. In the previous example 𝑋𝑖 was the event that a white ball was drawn and 𝑌𝑖 the event that it came from a box with i white balls. The general problem is to calculate the probabilities of the events 𝑌𝑖 given occurrence of X (called posterior probabilities), in 19 terms of i) the unconditional probabilities P(𝑌𝑖 ) (called prior probabilities); ii) the conditional probabilities P(Y/X i ) (called likelihoods). Here is the general calculation: (multiplication rule) 𝑌1 𝑃 𝑋∩𝑌 P( )= 𝑋 𝑃 𝑋 = 𝑃 𝑌1 𝑃 𝑋 𝑌1 𝑃 𝑋 ……………………………….…………………..9 By the rule of average conditional probability, P(X) = 𝑃 𝑌1 𝑃 Then 𝑌 𝑋 P( 1)= = 𝑃 𝑌1 𝑃 𝑃 𝑌1 𝑃 𝑋 𝑌1 + …+ 𝑃 𝑃 𝑌1 𝑃 𝑃 𝑌1 𝑃 𝑋 𝑌1 𝑋 𝑌1 𝑌𝑛 𝑃 𝑋 𝑌𝑛 𝑌𝑛 𝑃 𝑋 𝑌𝑛 𝑋 𝑌1 + …+ 𝑃 𝑋 𝑌1 + …+ 𝑃 𝑌𝑛 𝑃 𝑋 𝑌𝑛 …10 ……………………………………11 ………………………………………………….. 12 Example 6: Suppose an animal scientist performs laboratory tests on blood sample that yields one of positive or negative result. It is found that 95% of cattle with a particular disease produce a positive result. But 2% of cattle without the disease will also produce a positive result. Suppose that 1% of the population actually has the disease. What is the probability that a cattle chosen at random from the population 20 will have the disease, if the cattle's blood yields a positive result? Solution Let D =proportion of the population that has the diseases W= proportion of the population that has no disease + and - = positive and negative test results P(+ID) = probability of positive result with the disease = 0.95, P(+IW) = probability of positive result without disease =0.02, P(D) = probability of the population proportion with disease = 0.01, P(W) = probability of the population proportion without the disease = 0.99. Applying Bayes' rule with gives + P(D/+)= = = P D P D + + P D P D +P W P W (0.01)∗(0.95) (0.01)∗(0.95)…+ (0.99)∗(0.02) 95 293 = 32% Thus only 32% of those cattle that produce a positive test result actually have the disease. 21 Class Exercises 1. A box contains 4 white, 5 red and 6 black balls. Find the probability that if 2 balls are picked at random and were not replaced, they will be (a) red and red (b) white and white (c)black and white (d) red and black (e) not red (f) white but not red (g) red but not white 2. If a card is drawn at random in a pack of 52 playing cards, what is the possibility that the card is (a) a king of clubs (b) is both a king and clubs (c)a king but not clubs (d) not king nor clubs (e) king (f) not clubs (g) clubs 3. If box I contains 5 black, and 2 red marbles while box II contains 3 black and 6 red marbles. If a fair coin is tossed and turns up red, a marble is chosen from box I; if turns out tail, a marble is chosen from box II, find the probability that a black marble is chosen. 4. Suppose that in exercise 3, the man who tossed the coin did not reveal whether it turn up head or tail, but revealed that a black marble was chosen. What is the probability that ox II was chosen? 22 References Statistics for Agricultural Sciences, Second Edition-G Nageswara Rao. BS Publications, 2007. Schaum’s Outlines -Probability and Statistics Third Edition-Murray R. Spiegel, John Schiller and Alu Srinivasan. 23 NEXT CLASS Measuring probability using the counting technique Counting Permutation and Combinational analysis Probability distribution and binomial distribution probabilities The use of table of areas for a standard normal curve to determine probabilities of a normal distribution 24