Probability Theory Lecture Notes - AUM PDF

Welcome! www.aum.edu.kw 1  Probabilistic Methods in Electrical and Computer Engineering Course code: CE-302, EE-302 Textbook: Probability and Statistics for Engineering and the Sciences, 9th Edition, Jay L....

Welcome! www.aum.edu.kw 1  Probabilistic Methods in Electrical and Computer Engineering Course code: CE-302, EE-302 Textbook: Probability and Statistics for Engineering and the Sciences, 9th Edition, Jay L. Devore Lecture notes: Available on Moodle 2 www.aum.edu.kw  Chapter 2 – Probability 2.1 SAMPLE SPACES AND EVENTS (page 53-57) 2.2 AXIOMS, INTERPRETATIONS AND PROPERTIES OF PROBABILITY (page 58-66) 3 www.aum.edu.kw Outline Probability Theory Random Experiments Sample Spaces Events Set Theory Axioms and Properties of Probability 4 www.aum.edu.kw Probability The term probability refers to the study of randomness and uncertainty. 5 www.aum.edu.kw What is Probability Theory? Randomness and uncertainty exist in our daily lives and in science, engineering and technology: ○ System reliability (life expectancy of components, probability of failure, etc.) ○ Software failure ○ Communication over unreliable channels, etc. 6 www.aum.edu.kw Important Definitions Random experiment: the process of observing something uncertain (flipping a coin). Outcome: a result of a random experiment (head). Sample space: the set of all possible outcomes (S = {head, tail}) Trial: When we repeat a random experiment several times, we call each one a trial. Event: a collection of one or more outcomes. It is a subset of the sample space. 7 www.aum.edu.kw What is a Random Experiment? Example: flipping a fair coin We cannot predict whether the outcome would be head (H) or tail (T). 8 www.aum.edu.kw What is a Random Experiment? In a random experiment, the outcome varies in an unpredictable fashion when the experiment is repeated under the same conditions. We cannot get the exact outcomes; instead, we determine their probabilities. 9 www.aum.edu.kw The Sample Space of an Experiment The sample space of an experiment, denoted by 𝑆, is the set of all possible outcomes of that experiment. The simplest experiment to which probability applies is one with two possible outcomes. 10 www.aum.edu.kw The Sample Space of an Experiment Example 2.1 Find the sample space S (set of all possible outcomes) for the following random experiments : Experiment 1: Examine a single weld to see whether it is defective. (𝑁 represents not defective, 𝐷 represents defective.) The sample space for this experiment is: 𝑺 = {𝑵, 𝑫} (The braces are used to enclose the elements of a set.) 11 www.aum.edu.kw The Sample Space of an Experiment A coin is flipped twice. Find the sample space using tree representation. How many possible outcomes are there in the sample space S? 12 www.aum.edu.kw The Sample Space of an Experiment First flip Second flip Possible outcomes H HH H T HT H TH T TT T Sample space: 𝑺 = {𝑯𝑯, 𝑯𝑻, 𝑻𝑯, 𝑻𝑻} There are 22 = 2 × 2 = 𝟒 possible outcomes in the sample space S of this random experiment. 13 www.aum.edu.kw The Sample Space of an Experiment Suppose that a coin is flipped three times. Find the sample space by drawing a tree diagram. How many possible outcomes are there in the sample space S? 14 www.aum.edu.kw The Sample Space of an Experiment First flip Second flip Third flip Possible outcomes H HHH H T HHT H H HTH T T HTT H THH H T THT T H TTH T T TTT Sample space: 𝑺 = {HHH, HHT, HTH, HTT, THH, THT, TTH, TTT} There are 23 = 2 × 2 × 2 = 𝟖 possible outcomes in the sample space S of this random experiment. 15 www.aum.edu.kw The Sample Space of an Experiment Example 2.3 Two gas stations are located at a certain intersection. Each one has six gas pumps. Random experiment: count the number of pumps in use for each of the stations at a particular time of day. Possible outcomes: how many pumps are in use at the first station and how many are in use at the second station. How many possible outcomes are there in the sample space S? Find the sample space S. 16 www.aum.edu.kw The Sample Space of an Experiment number of pumps in use The 49 outcomes in 𝑆 are displayed in the following table: So, the sample space S = {(0, 0), (0, 1), (0, 2), (0, 3), (0, 4),…, (6, 4), (6, 5), (6, 6)}. 17 www.aum.edu.kw Events An event is any collection (subset ⊂) of outcomes contained in the sample space 𝑆. An event is: Simple if it consists of exactly one outcome: 𝐸1 = 0, 0 Compound if it consists of more than one outcome: 𝐶 = {(0, 0), (0, 1), (1, 0), (1, 1)} 𝐸1 = 0, 0 ⊂ S = {(0, 0), (0, 1), (0, 2), (0, 3), (0, 4),…, (6, 4), (6, 5), (6, 6)} 𝐶 = {(0, 0), (0, 1), (1, 0), (1, 1)} ⊂ S = {(0, 0), (0, 1), (0, 2), (0, 3), (0, 4),…, (6, 4), (6, 5), (6, 6)} 18 www.aum.edu.kw Events Suppose that in a random experiment: 1) 𝑆 is the set of all possible outcomes (sample space) 2) event 𝐴 is a subset of 𝑆 (𝐴 ⊂ 𝑆) 3) each event 𝐴 has been assigned a number 𝑃(𝐴) Then 𝐏(𝐀) is called the probability of occurrence of the event 𝑨. So, probability is a number assigned to an event that indicates how “likely” the event will occur when an experiment is performed. If all simple events have the same chance (i.e. equally likely events), then 𝐍𝐮𝐦𝐛𝐞𝐫 𝐨𝐟 𝐞𝐥𝐞𝐦𝐞𝐧𝐭𝐬 𝐨𝐟 𝐀 𝑷(𝑨) = 𝐍𝐮𝐦𝐛𝐞𝐫 𝐨𝐟 𝐞𝐥𝐞𝐦𝐞𝐧𝐭𝐬 𝐨𝐟 𝐒 19 www.aum.edu.kw Set Theory The union of two events 𝐴 and 𝐵, denoted by 𝐴 ∪ 𝐵 and read “𝐴 or 𝐵,” is the event consisting of all outcomes that are either in 𝑨 or in 𝑩 or in both events (so that the union includes outcomes for which both 𝐴 and 𝐵 occur as well as outcomes for which exactly one occurs)—that is, all outcomes in at least one of the events. 𝐴 = 0, 1, 2, 3, 4 𝐵 = {3, 4, 5, 6} 𝐴 ∪ 𝐵={0, 1, 2, 3, 4, 5, 6} 20 www.aum.edu.kw Set Theory The intersection of two events 𝐴 and 𝐵, denoted by 𝐴 ∩ 𝐵 and read “𝐴 and 𝐵,” is the event consisting of all outcomes that are in both 𝑨 and 𝑩. 𝐴 = 0, 1, 2, 3, 4 𝐵 = {3, 4, 5, 6} 𝐴 ∩ 𝐵 = {3, 4} The complement of an event 𝐴, denoted by 𝐴′ (𝑜𝑟 𝐴𝐶 ), is the set of all outcomes in 𝑆 that are not contained in 𝐴. 𝑆 = {0, 1, 2, 3, 4, 5, 6} 𝐴 = 0, 1, 2, 3, 4 𝐴′ = {5, 6} 21 www.aum.edu.kw Set Theory Let ∅ denote the null event (the event consisting of no outcomes). When 𝐴 ∩ 𝐵 = ∅, 𝐴 and 𝐵 are said to be mutually exclusive or disjoint events. 𝐴 = 0, 1 𝐵 = 3, 4 𝐴∩𝐵 =∅ 22 www.aum.edu.kw Axioms and Properties of Probability Axioms (basic properties) of probability: Axiom 1: 𝟎 ≤ 𝑷(𝑨) ≤ 𝟏 Axiom 2: 𝑷 𝑺 = 𝟏 Axiom 3: If 𝐴 ∩ 𝐵 = ∅, 𝑷 𝑨 ∪ 𝑩 = 𝑷 𝑨 + 𝑷(𝑩) 23 www.aum.edu.kw Axioms and Properties of Probability 𝑷 𝑺 = 𝑷 𝑨 ∪ 𝑨𝑪 = 𝑷 𝑨 + 𝑷 𝑨𝑪 = 𝟏 (where 𝑨𝑪 is the complement of 𝑨) 𝑷 𝑨𝑪 = 𝟏 − 𝑷(𝑨) 𝑷(∅) = 𝟎 (where ∅ is the null event, the event containing no outcomes) 𝑷 𝑨 ∪ 𝑩 = 𝑷 𝑨 + 𝑷 𝑩 − 𝑷(𝑨 ∩ 𝑩) 𝑷 𝑨∪𝑩∪𝑪 =𝑷 𝑨 +𝑷 𝑩 +𝑷 𝑪 −𝑷 𝑨∩𝑩 − 𝑷 𝑨 ∩ 𝑪 − 𝑷 𝑩 ∩ 𝑪 + 𝑷(𝑨 ∩ 𝑩 ∩ 𝑪) 24 www.aum.edu.kw Axioms and Properties of Probability Example 2.14 In a certain residential suburb, 60% of all households get Internet service from the local cable company, 80% get television service from that company, and 50% get both services from that company. What is the probability that that a randomly selected household gets at least one of these two services from the company? 𝑨 = 𝒈𝒆𝒕𝒔 𝑰𝒏𝒕𝒆𝒓𝒏𝒆𝒕 𝒔𝒆𝒓𝒗𝒊𝒄𝒆 𝑷 𝑨 = 𝟎. 𝟔 𝑷 𝑩 = 𝟎. 𝟖 𝑩 = 𝒈𝒆𝒕𝒔 𝑻𝑽 𝒔𝒆𝒓𝒗𝒊𝒄𝒆 𝑷 𝑨 ∩ 𝑩 = 𝟎. 𝟓 𝑷 𝒈𝒆𝒕𝒔 𝒂𝒕 𝒍𝒆𝒂𝒔𝒕 𝒐𝒏𝒆 𝒐𝒇 𝒕𝒉𝒆 𝒕𝒘𝒐 𝒔𝒆𝒓𝒗𝒊𝒄𝒆𝒔 = 𝑷 𝑨 ∪ 𝑩 = 𝑷 𝑨 + 𝑷 𝑩 − 𝑷 𝑨 ∩ 𝑩 = 0.6 + 0.8 − 0.5 = 𝟎. 𝟗 25 www.aum.edu.kw Axioms and Properties of Probability Example 2.11 Consider flipping a coin in the air. When it comes to rest on the ground, either its side on the top will be a head (the outcome 𝐻 ) or a tail (the outcome 𝑇). The sample space for this event is therefore 𝑺 = {𝑯, 𝑻}. Find the probability of the event A = {H} given that the probability of the event B = {T} is 0.75. Since 𝐴 and 𝐵 are disjoint and their union is 𝑺: 𝟏 = 𝑷(𝑺) = 𝑷(𝑨) + 𝑷(𝑩). It follows that: 𝑷(𝑨) = 𝟏 − 𝑷(𝑩). So, 𝑷 𝑨 = 𝟏 − 𝟎. 𝟕𝟓 = 𝟎. 𝟐𝟓 26 www.aum.edu.kw Determining Probabilities Systematically Consider a sample space that is finite. Let 𝐸1 , 𝐸2 , 𝐸3 , … denote the corresponding simple events, each consisting of a single outcome. Then the probability of any compound event 𝑨 is computed by adding together the probabilities of each simple event, 𝑃(𝐸𝑖 )’s for all 𝐸𝑖 ’s in 𝐴: 𝑷 𝑨 = ෍ 𝑷(𝑬𝒊 ) ∀𝑬𝒊 ∈𝑨 27 www.aum.edu.kw Determining Probabilities Systematically 𝑷 𝑨 = ෍ 𝑷(𝑬𝒊 ) ∀𝑬𝒊 ∈𝑨 Example 2.15 During off-peak hours a commuter train has five cars. Suppose that the probability that a commuter gets on train car number 𝒊 is: 𝒑𝒊 = 𝑷 𝒄𝒂𝒓 𝒊 𝒊𝒔 𝒔𝒆𝒍𝒆𝒄𝒕𝒆𝒅 = 𝑷(𝑬𝒊 ) Suppose that: 𝑝1 = 𝑃 𝑐𝑎𝑟 1 𝑖𝑠 𝑠𝑒𝑙𝑒𝑐𝑡𝑒𝑑 = 𝑃 𝐸1 = 0.1 𝑝2 = 𝑃 𝑐𝑎𝑟 2 𝑖𝑠 𝑠𝑒𝑙𝑒𝑐𝑡𝑒𝑑 = 𝑃 𝐸2 = 0.2 𝑝3 = 𝑃 𝑐𝑎𝑟 3 𝑖𝑠 𝑠𝑒𝑙𝑒𝑐𝑡𝑒𝑑 = 𝑃 𝐸3 = 0.4 𝑝4 = 𝑃 𝑐𝑎𝑟 4 𝑖𝑠 𝑠𝑒𝑙𝑒𝑐𝑡𝑒𝑑 = 𝑃 𝐸4 = 0.2 𝑝5 = 𝑃 𝑐𝑎𝑟 5 𝑖𝑠 𝑠𝑒𝑙𝑒𝑐𝑡𝑒𝑑 = 𝑃 𝐸5 = 0.1 What is the probability of the event A, that one of the three middle cars is selected? 𝑷 𝑨 = σ𝟒𝒊=𝟐 𝑷(𝑬𝒊 ) = 𝑃 𝐸2 + 𝑃 𝐸3 + 𝑃 𝐸4 = 𝑝2 + 𝑝3 + 𝑝4 = 0.2 + 0.4 + 0.2 = 𝟎. 𝟖 28 www.aum.edu.kw Equally Likely Outcomes If there are 𝑵 equally likely (equiprobable) outcomes in the sample space 𝟏 of a random experiment, the probability for each is: 𝒑 = 𝑷 𝑬𝒊 = 𝑵 for every 𝒊. Consider an event 𝐴, with 𝑁(𝐴) denoting the number of outcomes contained in 𝐴. Then: 𝟏 𝑵(𝑨) 𝑷(𝑨) = σ𝑬𝒊 ∈𝑨 𝑷 𝑬𝒊 = σ𝑬𝒊 ∈𝑨 𝑵 = 𝑵 Thus, when outcomes are equally likely, computing probabilities reduces to counting: Determine both the number of outcomes 𝑵(𝑨) in 𝑨 and the number of outcomes 𝑵 in 𝑺, and form their ratio. 29 www.aum.edu.kw Equally Likely Outcomes Example 2.16 You have six mystery books and six science fiction books on your bookshelf. The first three of each type are hardcover, and the last three are paperback. Consider randomly selecting one of the six mysteries and then randomly selecting one of the six science fiction books to take on a vacation. 1) How many possible outcomes are there in the sample space? 2) What is the probability of the event 𝐴 that both selected books are paperbacks? 30 www.aum.edu.kw Equally Likely Outcomes 𝟏 𝑵(𝑨) 𝑷(𝑨) = ෍ 𝑷 𝑬𝒊 = ෍ = 𝑵 𝑵 𝑬𝒊 ∈𝑨 𝑬𝒊 ∈𝑨 1) There are 𝑵 = 𝟔𝟐 = 𝟔 × 𝟔 = 𝟑𝟔 possible outcomes in the sample space. 2) The probability of the event 𝐴 that both selected books are paperbacks is: 𝑷 𝑨 = 𝑵 𝑨 9 = 36 = 0.25 science fiction: 𝑵 hardcover paperback hardcover mysteries: paperback 31 www.aum.edu.kw Recommended Exercises Consider the random experiment of flipping a coin twice. 1) Find the sample space S. 2) Find the event A = A head followed by a head. 3) Find the event B = A head followed by a tail. 4) Find the event C = A head occurs at the first flip. 5) Are the sets A, B and C subsets of the sample space? 6) Find two examples of a simple event. 7) Find two examples of a compound event. 8) Find the probability of event A, event B and event C. 9) Find the probability that there is at least one head in the outcome. 32 www.aum.edu.kw Exercises Let A denote the event that the next request for assistance from a statistical software consultant relates to the SPSS package, and let B be the event that the next request is for help with SAS. Suppose that P(A) = 0.30 and P(B) = 0.50. a) Calculate P(A’). b) Calculate P(A ∪ B). 33 www.aum.edu.kw Exercises Suppose that you flip a coin and then roll a die. Find the sample space by drawing a tree diagram. 34 www.aum.edu.kw