Chapter One: Basic Concepts of Probability Theory PDF

chapter one Basic Concept of Probability Theory Prepared by Biruk s.(Msc) 1.1 Why study probability? 1.2 The sample space 1.3 Events 1.4 Set theory 1.5 Axioms of probability 1.6 Conditional probability 1.7 Total probability 1.8 Independent events 1.9 Bayes’ theorem...

chapter one Basic Concept of Probability Theory Prepared by Biruk s.(Msc) 1.1 Why study probability? 1.2 The sample space 1.3 Events 1.4 Set theory 1.5 Axioms of probability 1.6 Conditional probability 1.7 Total probability 1.8 Independent events 1.9 Bayes’ theorem 1  Probability theory is the branch of mathematics that is concerned with the study of random phenomena. Experiment Not predictable yields exactly  A random phenomenon is one that, under repeated observation, yields different outcomes that are not deterministically predictable.  Examples of these random phenomena include  The number of phone calls arriving at the university’s tower over a given period  The number of A’s that a student can receive in one academic year.  Rolling die, tossing coin 2  Why study probability? Probability theory provides powerful tools to explain, model and design real world physical systems with some degree of uncertainty.  Some application areas of probability theory include:  Data communication systems  Wireless communication systems  Control systems, etc… Example  Recovery of data coming over a noisy communication channel 3  The sample space The sample space is the set of all possible outcomes of a random experiment and it is denoted by S or Ω In probability Experiment (random experiment) Collection of possible elementary outcomes is called the sample space of the experiment For example, if we toss a die, any number from 1 to 6 may appear. Therefore, in this experiment the sample space is defined by S = {1,2,3,4,5,6} If we toss a coin three times S = {HHH,HHT,HTH,HTT,THH,THT,TTH,TTT} 4  Event An event is any subset of the sample of the sample space, Ω Example Consider a random experiment of flipping a fair coin twice. i. Sample space Ω = { HH , HT, TH, TT } Some possible events An event of getting exactly one head A = { HT ,TH} An event of getting at least one tail B = { HT , TH, TT } 5  There are several ways to define probability. In this section we consider three definitions: the  axiomatic definition,  relative-frequency definition, and  classical definition. Axiomatic Definition Consider a random experiment whose sample space is S. For each event A of S we assume that a number P(A), called the probability of event A, is defined such that the following hold: 6 1. Axiom 1: 0 ≤ P(A) ≤ 1, which means that the probability of A is some number between and including 0 and 1. 2. Axiom 2: P(S) = 1, which states that with probability 1, the outcome will be a sample point in the sample space. 3. Axiom 3: For any set of n mutually exclusive events A1,A2,...,An defined on the same sample space, P(A1 ∪ A2 ∪ · · · ∪ An) = P(A1) + P(A2) + · · · + P(An) Example: A number is selected from the first 25 natural numbers. What is the probability that it would be divisible by 4 or 7 ? Solution: Let A be the event that the number selected would be divisible by 4 and B, the event that the selected number would be divisible by 7. 7 Then AuB denotes the event that the number would be divisible by 4 or 7. Next we note that S = {1, 2, 3, ……... 25} A = {4, 8, 12, 16, 20, 24} and B = {7, 14, 21} Here, AnB = Null set. The two events A and B are mutually exclusive and as such we have P(AuB) = P(A) + P(B) -----(1) Since P(A) = n(A) / n(S) = 6/25 and P(B) = n(B) / n(S) = 3/25 P(AuB) = 9/25 7  Relative-Frequency(Statistical) Definition Consider a random experiment that is performed n times. If an event A occurs n times, then the probability of A event A, P(A), is defined as follows:  The ratio nA/n is called the relative frequency of event A.  limitation is the fact that the experiment may not be repeatable  Also, the limit may not exist. This statistical definition is applicable if the above limit exists and tends to a finite value. 8  Example: The following data relate to the distribution of wages of a group of workers : If a worker is selected at random from the entire group of workers, what is the probability that (a) his wage would be less than $ 50 ? (b) his wage would be less than $ 80 ? (c) his wage would be more than $ 100 ? Solution: (a) P(A) = 0 (b) P(B) = 74/150 (c) P(C) = 17/150 8 Classical Definition In the classical definition, the probability P(A) of an event A is the ratio of the number of outcomes NA of an experiment that are favorable to A to the total number N of possible outcomes of the experiment. That is, Example: A coin is tossed once. S = { H, T } and n(S) = 2 Let A be the event of getting head. A = { H } and n(A) = 1 Then, P(A) = n(A) / n(S) P(A) = 1/2 9 Exercise: A coin is tossed three times. What is the probability of getting : (i) 2 heads (ii) at least 2 heads Demerits or limitations: (i) It is applicable only when the total no. of events is finite. (ii) It can be used only when the events are equally likely or equi-probable. This definition has only a limited field of application like coin tossing, dice throwing, drawing cards etc. where the possible events are known well in advance. 9 Elementary Set Theory A set is a collection of objects known as elements. A set can be represented in a number of ways as the following examples illustrate. Example Let A denote the set of positive integers between and including 1 and 5. Then A = {a|1 ≤ a ≤ 5} = {1,2,3,4,5} Example Let B denote the set of positive odd numbers less than 10. Then B = {1,3,5,7,9} 10 Set representation  If k is an element of the set E, we say that k belongs to E and write k ∈ E.  If k is not an element of the set E, we say that k does not belong to E and write k ∈/ E.  A set A is called a subset of set B, denoted by A ⊂ B, if every member of A is a member of B. Alternatively, we say that the set B contains the set A by writing B ⊃ A.  The set that contains all possible elements is called the universal set S.  The set that contains no elements (or is empty) is called the null set ∅ (or empty set). 11 Set Operations  Equality: Two sets A and B are defined to be equal, denoted by A = B, if and only if (iff) A is a subset of B and B is a subset of A; that is A ⊂ B, and B ⊂ A.  Complementation: The complement of A, denoted by A, Example Let S = {1,2,3,4,5,6,7,8,9,10}, A = {1,2,4,7}, and B = {1,3,4,6}. Then A = {3,5,6,8,9,10}, and B = {2,5,7,8,9,10}. Union: The union of two sets A and B, denoted by A ∪ B, is the set containing all the elements of either A or B or both A and B. A ∪ B = {1,2,3,4,6,7}. 12 Intersection: The intersection of two sets A and B, denoted by A ∩ B, is the set containing all the elements that are in both A and B. That is A = {1,2,4,7}, and B = {1,3, 4,6}. A ∩ B = {1,4} Difference: The difference of two sets A and B, denoted by A - B, is the set containing all elements of A that are not in B. That is, Note that A - B = B - A. From Example we find that A - B = {2, 7}, while B - A = {3, 6}. Disjoint Sets: Two sets A and B are called disjoint (or mutually exclusive) sets if they contain no elements in common, which means that A ∩ B = ∅. 13 Graphical representation of set: 14  The operations of forming unions, intersections, and complements of sets obey certain rules similar to the rules of algebra. Commutative law for unions: A ∪ B = B ∪ A, which states that the order of the union operation on two sets is immaterial. Commutative law for intersections: A ∩ B = B ∩ A, Associative law for unions: A ∪ (B ∪ C) = (A ∪ B) ∪ C, Associative law for intersections: A ∩ (B ∩ C) = (A ∩ B) ∩ C De Morgan’s first law: A ∪ B = A ∩ B, De Morgan’s second law: A ∩ B = A ∪ B 15  Is the probability that depends on a previous result or event.  Due to this fact, they help us understand how events are related to each other.  For example, if it rains, the probability of road accidents increases as roads have less friction.  It is represented as P(A | B) which means the probability of A when B has already happened. P(A|B) = P(A ∩ B) / P(B) Where, P(A ∩ B) represents the probability of both events A and B occurring simultaneously, and P(B) represents the probability of event B occurring. 15  How to Calculate Conditional Probability? Step 1: Identify the Events. Let’s call them Event A and Event B. Step 2: Determine the Probability of Event A i.e., P(A) Step 3: Determine the Probability of Event B i.e., P(B) Step 4: Determine the Probability of Event A and B i.e., P(A∩B). Step 5: Apply the Conditional Probability Formula and calculate the required probability. 16 Example: A fair coin was tossed two times. Given that the first toss resulted in heads, what is the probability that both tosses resulted in heads? Solution S = {HH, HT, TH, TT} Let X denote the event that both tosses came up heads; that is, X = {HH}. Let Y denote the event that the first toss came up heads; that is, Y = {HH, HT}. The probability that both tosses resulted in heads, given that the first toss resulted in heads, is given by 17 An event in probability falls under two categories, I. Dependent Events II. Independent Events Dependent Events: Dependent events are those events that are affected by the outcomes of events that had already occurred previously Example  Not paying your power bill on time and having your power cut off  Choosing a card from a deck and then choosing another without putting the first card back in the deck 18 Independent Events: Independent events are those events whose occurrence is not dependent on any other event. Examples: Tossing a Coin Sample Space(S) in a Coin Toss = {H, T} Both getting H and T are Independent Events Rolling a Die Sample Space(S) in Rolling a Die = {1, 2, 3, 4, 5, 6}, all of these events are independent too. Both of the above examples are simple events. Even compound events can be independent events. 19 Bayes’ theorem describes the probability of occurrence of an event related to any condition.  It is also considered for the case of conditional probability. P(A|B)=(P(B|A)P(A)) P(B) 20 END