Schaum's Outline of Probability, Random Variables, and Random Processes PDF

01_Hsu_Probability 8/31/19 3:55 PM Page 1 CHAPTER 1 Probability 1.1 Introduction The...

01_Hsu_Probability 8/31/19 3:55 PM Page 1 CHAPTER 1 Probability 1.1 Introduction The study of probability stems from the analysis of certain games of chance, and it has found applications in most branches of science and engineering. In this chapter the basic concepts of probability theory are presented. 1.2 Sample Space and Events A. Random Experiments: In the study of probability, any process of observation is referred to as an experiment. The results of an obser- vation are called the outcomes of the experiment. An experiment is called a random experiment if its outcome cannot be predicted. Typical examples of a random experiment are the roll of a die, the toss of a coin, drawing a card from a deck, or selecting a message signal for transmission from several messages. B. Sample Space: The set of all possible outcomes of a random experiment is called the sample space (or universal set), and it is denoted by S. An element in S is called a sample point. Each outcome of a random experiment corresponds to a sample point. EXAMPLE 1.1 Find the sample space for the experiment of tossing a coin (a) once and (b) twice. (a) There are two possible outcomes, heads or tails. Thus: S  {H, T} where H and T represent head and tail, respectively. (b) There are four possible outcomes. They are pairs of heads and tails. Thus: S  {HH, HT, TH, TT} EXAMPLE 1.2 Find the sample space for the experiment of tossing a coin repeatedly and of counting the number of tosses required until the first head appears. Clearly all possible outcomes for this experiment are the terms of the sequence 1, 2, 3, … Thus: S  {1, 2, 3, …} Note that there are an infinite number of outcomes. 1 01_Hsu_Probability 8/31/19 3:55 PM Page 2 2 CHAPTER 1 Probability EXAMPLE 1.3 Find the sample space for the experiment of measuring (in hours) the lifetime of a transistor. Clearly all possible outcomes are all nonnegative real numbers. That is, S  {τ : 0  τ  ∞} where τ represents the life of a transistor in hours. Note that any particular experiment can often have many different sample spaces depending on the observa- tion of interest (Probs. 1.1 and 1.2). A sample space S is said to be discrete if it consists of a finite number of sample points (as in Example 1.1) or countably infinite sample points (as in Example 1.2). A set is called countable if its elements can be placed in a one-to-one correspondence with the positive integers. A sample space S is said to be continuous if the sample points constitute a continuum (as in Example 1.3). C. Events: Since we have identified a sample space S as the set of all possible outcomes of a random experiment, we will review some set notations in the following. If ζ is an element of S (or belongs to S), then we write ζ僆S If S is not an element of S (or does not belong to S), then we write ζ∉S A set A is called a subset of B, denoted by A⊂B if every element of A is also an element of B. Any subset of the sample space S is called an event. A sample point of S is often referred to as an elementary event. Note that the sample space S is the subset of itself: that is, S ⊂ S. Since S is the set of all possible outcomes, it is often called the certain event. EXAMPLE 1.4 Consider the experiment of Example 1.2. Let A be the event that the number of tosses required until the first head appears is even. Let B be the event that the number of tosses required until the first head appears is odd. Let C be the event that the number of tosses required until the first head appears is less than 5. Express events A, B, and C. A  {2, 4, 6, …} B  {1, 3, 5, …} C  {1, 2, 3, 4} 1.3 Algebra of Sets A. Set Operations: 1. Equality: Two sets A and B are equal, denoted A  B, if and only if A ⊂ B and B ⊂ A. 2. Complementation: – Suppose A ⊂ S. The complement of set A, denoted A, is the set containing all elements in S but not in A. – A  {ζ : ζ ∈ S and ζ ∉ A} 01_Hsu_Probability 8/31/19 3:55 PM Page 3 CHAPTER 1 Probability 3 3. Union: The union of sets A and B, denoted A ∪ B, is the set containing all elements in either A or B or both. A ∪ B  {ζ : ζ ∈ A or ζ ∈ B} 4. Intersection: The intersection of sets A and B, denoted A ∩ B, is the set containing all elements in both A and B. A ∩ B  { ζ : ζ ∈ A and ζ ∈ B} 5. Difference: The difference of sets A and B, denoted A\ B, is the set containing all elements in A but not in B. A\ B  {ζ : ζ ∈ A and ζ ∉ B} – Note that A\ B  A ∩ B. 6. Symmetrical Difference: The symmetrical difference of sets A and B, denoted A Δ B, is the set of all elements that are in A or B but not in both. A Δ B  { ζ: ζ ∈ A or ζ ∈ B and ζ ∉ A ∩ B} – – Note that A Δ B  (A ∩ B ) ∪ (A ∩ B)  (A\ B) ∪ (B \ A). 7. Null Set: The set containing no element is called the null set, denoted ∅. Note that – ∅S 8. Disjoint Sets: Two sets A and B are called disjoint or mutually exclusive if they contain no common element, that is, if A ∩ B  ∅. The definitions of the union and intersection of two sets can be extended to any finite number of sets as follows: n ∪ Ai  A1 ∪ A2 ∪  ∪ An i 1  {ζ : ζ ∈ A1 or ζ ∈ A2 or  ζ ∈ An } n ∩ Ai  A1 ∩ A2 ∩  ∩ An i 1  {ζ : ζ ∈ A1 and ζ ∈ A2 and  ζ ∈ An } Note that these definitions can be extended to an infinite number of sets: ∞ ∪ Ai  A1 ∪ A2 ∪ A3 ∪  i 1 ∞ ∩ Ai  A1 ∩ A2 ∩ A3 ∩  i 1 01_Hsu_Probability 8/31/19 3:55 PM Page 4 4 CHAPTER 1 Probability In our definition of event, we state that every subset of S is an event, including S and the null set ∅. Then S  the certain event ∅  the impossible event If A and B are events in S, then – A  the event that A did not occur A ∪ B  the event that either A or B or both occurred A ∩ B  the event that both A and B occurred Similarly, if A1, A2, …, An are a sequence of events in S, then n ∪ Ai  the event that at least one of the Ai occurreed i 1 n ∩ Ai  the event that all of the Ai occurred i 1 9. Partition of S : k If Ai ∩ Aj  ∅ for i ≠ j and ∪ Ai  S , then the collection {Ai ; 1  i  k} is said to form a partition of S. i 1 10. Size of Set: When sets are countable, the size (or cardinality) of set A, denoted ⎪ A ⎪, is the number of elements contained in A. When sets have a finite number of elements, it is easy to see that size has the following properties: (i) If A ∩ B  ∅, then ⎪A ∪ B⎪  ⎪A⎪  ⎪B⎪. (ii) ⎪∅⎪  0. (iii) If A ⊂ B, then ⎪A⎪  ⎪B⎪. (iv) ⎪A ∪ B⎪  ⎪A ∩ B⎪  ⎪A⎪  ⎪B⎪. Note that the property (iv) can be easily seen if A and B are subsets of a line with length ⎪A⎪ and ⎪B⎪, respectively. 11. Product of Sets: The product (or Cartesian product) of sets A and B, denoted by A  B, is the set of ordered pairs of elements from A and B. C  A  B  {(a, b): a ∈ A, b ∈ B} Note that A  B ≠ B  A, and ⎪ C⎪  ⎪ A  B⎪  ⎪ A⎪  ⎪ B⎪. EXAMPLE 1.5 Let A  {a1, a2, a3} and B  {b1, b2}. Then C  A  B  {(a1, b1), (a1, b2), (a2, b1), (a2, b2), (a3, b1), (a3, b2)} D  B  A  {(b1, a1), (b1, a2), (b1, a3), (b2, a1), (b2, a2), (b2, a3)} B. Venn Diagram: A graphical representation that is very useful for illustrating set operation is the Venn diagram. For instance, in the three Venn diagrams shown in Fig. 1-1, the shaded areas represent, respectively, the events A ∪ B, A ∩ B, – – and A. The Venn diagram in Fig. 1-2(a) indicates that B 傺 A, and the event A ∩ B  A\ B is shown as the shaded area. In Fig. 1-2(b), the shaded area represents the event A Δ B. 01_Hsu_Probability 8/31/19 3:55 PM Page 5 CHAPTER 1 Probability 5 S S A B A B (a) Shaded region: A  B (b) Shaded region: A  B S A (c) Shaded region: A Fig. 1-1 S S A B B A (a) B  A (b) Shaded area: A ⌬ B Shaded area: A  B = A \ B Fig. 1-2 C. Identities: By the above set definitions or reference to Fig. 1-1, we obtain the following identities: – S∅ (1.1) — ∅ S (1.2) – AA (1.3) S∪A S (1.4) S∩A A (1.5) – A∪ A S (1.6) – A∩ A ∅ (1.7) A∪∅A (1.8) A∩∅∅ (1.9) – A\ B  A ∩ B (1.10) – S\AA (1.11) A\ ∅  A (1.12) – – A Δ B  (A ∩ B ) ∪ (A ∩ B) (1.13) The union and intersection operations also satisfy the following laws: Commutative Laws: A∪BB∪A (1.14) A∩BB∩A (1.15) 01_Hsu_Probability 8/31/19 3:55 PM Page 6 6 CHAPTER 1 Probability Associative Laws: A ∪ (B ∪ C)  (A ∪ B) ∪ C (1.16) A ∩ (B ∩ C)  (A ∩ B) ∩ C (1.17) Distributive Laws: A ∩ (B ∪ C)  (A ∩ B) ∪ (A ∩ C) (1.18) A ∪ (B ∩ C)  (A ∪ B) ∩ (A ∪ C) (1.19) De Morgan’s Laws: A ∪ B A ∩ B (1.20) A ∩ B A ∪ B (1.21) These relations are verified by showing that any element that is contained in the set on the left side of he equal- ity sign is also contained in the set on the right side, and vice versa. One way of showing this is by means of a Venn diagram (Prob. 1.14). The distributive laws can be extended as follows: ⎛ n ⎞ n A ∩ ⎜ ∪ Bi ⎟  ∪ ( A ∩ Bi ) (1.22) ⎜ ⎟ ⎝ i 1 ⎠ i 1 ⎛ n ⎞ n A ∪ ⎜ ∩ Bi ⎟  ∩ ( A ∪ Bi ) (1.23) ⎜ ⎟ ⎝ i 1 ⎠ i 1 Similarly, De Morgan’s laws also can be extended as follows (Prob. 1.21): ⎛ n ⎞ n ⎜⎜ ∪ Ai ⎟⎟  ∩ Ai (1.24) ⎝ i 1 ⎠ i 1 ⎛ n ⎞ n ⎜⎜ ∪ Ai ⎟⎟  ∩ Ai (1.25) ⎝ i 1 ⎠ i 1 1.4 Probability Space A. Event Space: We have defined that events are subsets of the sample space S. In order to be precise, we say that a subset A of S can be an event if it belongs to a collection F of subsets of S, satisfying the following conditions: (i) S ∈ F (1.26) – (ii) if A ∈ F, then A ∈ F ∞ (1.27) (iii) if Ai ∈ F for i 1, then ∪ Ai ∈ F (1.28) i 1 The collection F is called an event space. In mathematical literature, event space is known as sigma field (σ -field) or σ-algebra. Using the above conditions, we can show that if A and B are in F, then so are A ∩ B, A\ B, A Δ B (Prob. 1.22). 01_Hsu_Probability 8/31/19 3:55 PM Page 7 CHAPTER 1 Probability 7 EXAMPLE 1.6 Consider the experiment of tossing a coin once in Example 1.1. We have S  {H, T}. The set — {S, ∅}, {S, ∅, H, T } are event spaces, but {S, ∅, H } is not an event space, since H  T is not in the set. B. Probability Space: An assignment of real numbers to the events defined in an event space F is known as the probability measure P. Consider a random experiment with a sample space S, and let A be a particular event defined in F. The prob- ability of the event A is denoted by P(A). Thus, the probability measure is a function defined over F. The triplet (S, F, P) is known as the probability space. C. Probability Measure a. Classical Definition: Consider an experiment with equally likely finite outcomes. Then the classical definition of probability of event A, denoted P(A), is defined by A P( A)  (1.29) S If A and B are disjoint, i.e., A ∩ B  ∅ , then ⎪ A ∪ B⎪  ⎪ A⎪  ⎪ B⎪. Hence, in this case A∪ B A  B A B P ( A ∪ B)      P( A)  P( B) (1.30) S S S S We also have S P(S )  1 (1.31) S A S  A A P( A)   1   1  P( A) (1.32) S S S EXAMPLE 1.7 Consider an experiment of rolling a die. The outcome is S  {ζ 1, ζ 2, ζ 3, ζ4, ζ5, ζ 6}  {1, 2, 3, 4, 5, 6} Define: A: the event that outcome is even, i.e., A  {2, 4, 6} B: the event that outcome is odd, i.e., B  {1, 3, 5} C: the event that outcome is prime, i.e., C  {1, 2, 3, 5} A 3 1 B 3 1 C 4 2 Then P( A)    , P ( B)    , P(C )    S 6 2 S 6 2 S 6 3 Note that in the classical definition, P(A) is determined a priori without actual experimentation and the definition can be applied only to a limited class of problems such as only if the outcomes are finite and equally likely or equally probable. 01_Hsu_Probability 8/31/19 3:55 PM Page 8 8 CHAPTER 1 Probability b. Relative Frequency Definition: Suppose that the random experiment is repeated n times. If event A occurs n(A) times, then the probability of event A, denoted P(A), is defined as n( A) P( A)  lim (1.33) n →∞ n where n(A)/n is called the relative frequency of event A. Note that this limit may not exist, and in addition, there are many situations in which the concepts of repeatability may not be valid. It is clear that for any event A, the relative frequency of A will have the following properties: 1. 0  n(A) /n  1, where n(A)/n  0 if A occurs in none of the n repeated trials and n(A) /n = 1 if A occurs in all of the n repeated trials. 2. If A and B are mutually exclusive events, then n(A ∪ B)  n(A)  n(B) and n( A ∪ B) n( A) N ( B)   n n n (1.34) n ( A ∪ B) n( A) N ( B) P( A ∪ B)  lim  lim  lim  P( A)  P( B) n →∞ n n →∞ n n →∞ n c. Axiomatic Definition: Consider a probability space (S, F, P). Let A be an event in F. Then in the axiomatic definition, the probability P(A) of the event A is a real number assigned to A which satisfies the following three axioms: Axiom 1: P(A)  0 (1.35) Axiom 2: P(S)  1 (1.36) Axiom 3: P(A ∪ B)  P(A)  P(B) if A ∩ B  ∅ (1.37) If the sample space S is not finite, then axiom 3 must be modified as follows: Axiom 3′: If A1, A2 , … is an infinite sequence of mutually exclusive events in S (Ai ∩ Aj  ∅ for i ≠ j), then ⎛∞ ⎞ ∞ P ⎜ ∪ Ai ⎟  ∑ P( Ai ) ⎜ ⎟ (1.38) ⎝ i 1 ⎠ i 1 These axioms satisfy our intuitive notion of probability measure obtained from the notion of relative frequency. d. Elementary Properties of Probability: By using the above axioms, the following useful properties of probability can be obtained: – 1. P(A)  1 – P(A) (1.39) 2. P(∅)  0 (1.40) 3. P(A)  P(B) if A ⊂ B (1.41) 4. P(A)  1 (1.42) 5. P(A ∪ B)  P(A)  P(B) – P(A ∩ B) (1.43) 6. P(A\ B)  P(A)  P(A ∩ B) (1.44) 7. If A1, A2 , …, An are n arbitrary events in S, then ⎛ n ⎞ n P ⎜ ∪ Ai ⎟  ∑ P( Ai )  ∑ P( Ai ∩ A j )  ∑ P( Ai ∩ A j ∩ Ak ) ⎜ ⎟ ⎝ i 1 ⎠ i 1 i j i j k (1.45) n 1   (1) P( A1 ∩ A2 ∩  ∩ An ) 01_Hsu_Probability 8/31/19 3:55 PM Page 9 CHAPTER 1 Probability 9 where the sum of the second term is over all distinct pairs of events, that of the third term is over all distinct triples of events, and so forth. 8. If A1, A2, … , An is a finite sequence of mutually exclusive events in S (Ai ∩ Aj = ∅ for i ≠ j), then ⎛ n ⎞ n P ⎜ ∪ Ai ⎟  ∑ P( Ai ) (1.46) ⎜ ⎟ ⎝ i 1 ⎠ i 1 and a similar equality holds for any subcollection of the events. Note that property 4 can be easily derived from axiom 2 and property 3. Since A ⊂ S, we have P(A)  P(S) 1 Thus, combining with axiom 1, we obtain 0  P(A)  1 (1.47) Property 5 implies that P(A ∪ B)  P(A)  P(B) (1.48) since P(A ∩ B)  0 by axiom 1. Property 6 implies that P(A\ B)  P(A) – P(B) if B ⊂ A (1.49) since A ∩ B  B if B ⊂ A. 1.5 Equally Likely Events A. Finite Sample Space: Consider a finite sample space S with n finite elements S  {ζ 1, ζ 2, …, ζ n} where ζ i’s are elementary events. Let P(ζ i )  pi. Then 1. 0  pi  1 i  1, 2, …, n (1.50) n 2. ∑ pi  p1  p2   pn  1 (1.51) i 1 3. If A  ∪ ζ i , where I is a collection of subscripts, then i∈I P( A)  ∑ P(ζ i )  ∑ pi (1.52) ζi ∈ A i∈I B. Equally Likely Events: When all elementary events ζ i (i  1, 2, …, n) are equally likely, that is, p1  p2  …  pn 01_Hsu_Probability 8/31/19 3:55 PM Page 10 10 CHAPTER 1 Probability then from Eq. (1.51), we have 1 pi  i  1, 2, …, n (1.53) n n( A) and P( A)  (1.54) n where n(A) is the number of outcomes belonging to event A and n is the number of sample points in S. [See classical definition (1.29).] 1.6 Conditional Probability A. Definition: The conditional probability of an event A given event B, denoted by P(A⎪B), is defined as P( A ∩ B) P( A B)  P( B) 0 (1.55) P( B) where P(A ∩ B) is the joint probability of A and B. Similarly, P( A ∩ B) P( B A)  P( A) 0 (1.56) P( A) is the conditional probability of an event B given event A. From Eqs. (1.55) and (1.56), we have P(A ∩ B)  P(A⎪B) P(B)  P(B⎪A)P(A) (1.57) Equation (1.57) is often quite useful in computing the joint probability of events. B. Bayes’ Rule: From Eq. (1.57) we can obtain the following Bayes’ rule: P ( B A )P ( A ) P( A B)  (1.58) P( B) 1.7 Total Probability The events A1, A2, …, An are called mutually exclusive and exhaustive if n ∪ Ai  A1 ∪ A2 ∪  ∪ An  S and Ai ∩ A j  ∅ i j (1.59) i 1 Let B be any event in S. Then n n P( B )  ∑ P ( B ∩ Ai )  ∑ P ( B Ai )P( Ai ) (1.60) i 1 i 1 01_Hsu_Probability 8/31/19 3:55 PM Page 11 CHAPTER 1 Probability 11 which is known as the total probability of event B (Prob. 1.57). Let A  Ai in Eq. (1.58); then, using Eq. (1.60), we obtain P( B Ai )P( Ai ) P( Ai B)  n ∑ P( B Ai )P( Ai ) (1.61) i 1 Note that the terms on the right-hand side are all conditioned on events Ai, while the term on the left is condi- tioned on B. Equation (1.61) is sometimes referred to as Bayes’ theorem. 1.8 Independent Events Two events A and B are said to be (statistically) independent if and only if P(A ∩ B)  P(A)P(B) (1.62) It follows immediately that if A and B are independent, then by Eqs. (1.55) and (1.56), P(A ⎪ B)  P(A) and P(B ⎪ A) P(B) (1.63) – If two events A and B are independent, then it can be shown that A and B are also independent; that is (Prob. 1.63), – – P(A ∩ B )  P(A)P(B ) (1.64) P( A ∩ B ) Then P( A B )   P( A) (1.65) P( B ) Thus, if A is independent of B, then the probability of A’s occurrence is unchanged by information as to whether or not B has occurred. Three events A, B, C are said to be independent if and only if P( A ∩ B ∩ C )  P( A)P( B)P(C ) P ( A ∩ B )  P ( A )P ( B ) P( A ∩ C )  P( A)P(C ) (1.66) P( B ∩ C )  P( B)P(C ) We may also extend the definition of independence to more than three events. The events A1, A2, …, An are inde- pendent if and only if for every subset {Ai1, Ai 2, … Aik} (2  k  n) of these events, P(Ai1 ∩ Ai 2 ∩ … ∩ A ik)  p(Ai1) P(Ai 2) … P(Aik) (1.67) Finally, we define an infinite set of events to be independent if and only if every finite subset of these events is independent. To distinguish between the mutual exclusiveness (or disjointness) and independence of a collection of events, we summarize as follows: 1. {Ai, i  1, 2, …, n} is a sequence of mutually exclusive events, then ⎛ n ⎞ n P ⎜ ∪ Ai ⎟  ∑ P( Ai ) (1.68) ⎜ ⎟ ⎝ i 1 ⎠ i 1 01_Hsu_Probability 8/31/19 3:55 PM Page 12 12 CHAPTER 1 Probability 2. If {Ai, i  1, 2, …, n} is a sequence of independent events, then ⎛ n ⎞ n P ⎜ ∩ Ai ⎟  ∏ P( Ai ) ⎜ ⎟ (1.69) ⎝ i 1 ⎠ i 1 and a similar equality holds for any subcollection of the events. SOLVED PROBLEMS Sample Space and Events 1.1. Consider a random experiment of tossing a coin three times. (a) Find the sample space S1 if we wish to observe the exact sequences of heads and tails obtained. (b) Find the sample space S2 if we wish to observe the number of heads in the three tosses. (a) The sampling space S1 is given by S1  {HHH, HHT, HTH, THH, HTT, THT, TTH , TTT} where, for example, HTH indicates a head on the first and third throws and a tail on the second throw. There are eight sample points in S1. (b) The sampling space S2 is given by S2  {0, 1, 2, 3} where, for example, the outcome 2 indicates that two heads were obtained in the three tosses. The sample space S2 contains four sample points. 1.2. Consider an experiment of drawing two cards at random from a bag containing four cards marked with the integers 1 through 4. (a) Find the sample space S1 of the experiment if the first card is replaced before the second is drawn. (b) Find the sample space S2 of the experiment if the first card is not replaced. (a) The sample space S1 contains 16 ordered pairs (i, j), 1  i  4, 1  j  4, where the first number indicates the first number drawn. Thus, ⎧ (1, 1) (1, 2 ) (1, 3) (1, 4 ) ⎫ ⎪ ⎪ ⎪(2, 1) (2, 2 ) (2, 3) (2, 4 )⎪ S1  ⎨ ⎬ ⎪ ( 3, 1) ( 3, 2 ) ( 3, 3) ( 3, 4 ) ⎪ ⎪⎩( 4, 1) ( 4, 2 ) ( 4, 3) ( 4, 4 )⎪⎭ (b) The sample space S2 contains 12 ordered pairs (i, j), i ≠ j, 1  i  4, 1  j  4, where the first number indicates the first number drawn. Thus, ⎧(1, 2 ) (1, 3) (1, 4 ) ⎫ ⎪ ⎪ ⎪(2, 1) (2, 3) (2, 4 )⎪ S2  ⎨ ⎬ ⎪ ( 3, 1) ( 3, 2 ) ( 3, 4 )⎪ ⎪⎩( 4, 1) ( 4, 2 ) ( 4, 3)⎪⎭ 01_Hsu_Probability 8/31/19 3:55 PM Page 13 CHAPTER 1 Probability 13 1.3. An experiment consists of rolling a die until a 6 is obtained. (a) Find the sample space S1 if we are interested in all possibilities. (b) Find the sample space S2 if we are interested in the number of throws needed to get a 6. (a) The sample space S1 would be S1  {6, 16, 26, 36, 46, 56, 116, 126, 136, 146, 156, …} where the first line indicates that a 6 is obtained in one throw, the second line indicates that a 6 is obtained in two throws, and so forth. (b) In this case, the sample space S2 i s S2  {i: i  1}  {1, 2, 3, …} where i is an integer representing the number of throws needed to get a 6. 1.4. Find the sample space for the experiment consisting of measurement of the voltage output v from a transducer, the maximum and minimum of which are  5 and  5 volts, respectively. A suitable sample space for this experiment would be S  {v: 5  v  5} 1.5. An experiment consists of tossing two dice. (a) Find the sample space S. (b) Find the event A that the sum of the dots on the dice equals 7. (c) Find the event B that the sum of the dots on the dice is greater than 10. (d) Find the event C that the sum of the dots on the dice is greater than 12. (a) For this experiment, the sample space S consists of 36 points (Fig. 1-3): S  {(i, j) : i, j = 1, 2, 3, 4, 5, 6} where i represents the number of dots appearing on one die and j represents the number of dots appearing on the other die. (b) The event A consists of 6 points (see Fig. 1-3): A  {(1, 6), (2, 5), (3, 4), (4, 3), (5, 2), (6, 1)} (c) The event B consists of 3 points (see Fig. 1-3): B  {(5, 6), (6, 5), (6, 6)} (d) The event C is an impossible event, that is, C  ∅. 01_Hsu_Probability 8/31/19 3:55 PM Page 14 14 CHAPTER 1 Probability S (1, 6) (2, 6) (3, 6) (4, 6) (5, 6) (6, 6) B (1, 5) (2, 5) (3, 5) (4, 5) (5, 5) (6, 5) (1, 4) (2, 4) (3, 4) (4, 4) (5, 4) (6, 4) (1, 3) (2, 3) (3, 3) (4, 3) (5, 3) (6, 3) (1, 2) (2, 2) (3, 2) (4, 2) (5, 2) (6, 2) (1, 1) (2, 1) (3, 1) (4, 1) (5, 1) (6, 1) A Fig. 1-3 1.6. An automobile dealer offers vehicles with the following options: (a) With or without automatic transmission (b) With or without air-conditioning (c) With one of two choices of a stereo system (d) With one of three exterior colors If the sample space consists of the set of all possible vehicle types, what is the number of outcomes in the sample space? The tree diagram for the different types of vehicles is shown in Fig. 1-4. From Fig. 1-4 we see that the number of sample points in S is 2  2  2  3  24. Transmission Automatic Manual Air-conditioning Yes No Yes No Stereo 1 2 1 2 1 2 1 2 Color Fig. 1-4 1.7. State every possible event in the sample space S  {a, b, c, d}. There are 24  16 possible events in S. They are ∅; {a}, {b}, {c}, {d}; {a, b}, {a, c}, {a, d}, {b, c}, {b, d}, {c, d }; {a, b, c}, {a, b, d }, {a, c, d}, {b, c, d}; S  {a, b, c, d}. 1.8. How many events are there in a sample space S with n elementary events? Let S  {s1, s2, …, sn}. Let Ω be the family of all subsets of S. (Ω is sometimes referred to as the power set of S.) Let Si be the set consisting of two statements, that is, Si  {Yes, the si is in; No, the si is not in} Then Ω can be represented as the Cartesian product Ω  S1  S2   Sn  {( s1, s2 , …, sn ) : si ∈ Si for i  1, 2, …, n} 01_Hsu_Probability 8/31/19 3:55 PM Page 15 CHAPTER 1 Probability 15 Since each subset of S can be uniquely characterized by an element in the above Cartesian product, we obtain the number of elements in Ω b y n(Ω)  n( S1 ) n( S2 )  n( Sn )  2n where n(Si )  number of elements in Si 2. An alternative way of finding n(Ω) is by the following summation: n ⎛ ⎞ n n n! n(Ω)  ∑ ⎜ ⎟  ∑ i 0 ⎝ i ⎠ i 0 i ! ( n  i )! The last sum is an expansion of (1+1) n = 2n. Algebra of Sets 1.9. Consider the experiment of Example 1.2. We define the events A  {k : k is odd} B  {k : 4  k  7} C  {k : 1  k  10} – – – where k is the number of tosses required until the first H (head) appears. Determine the events A, B , C , – A ∪ B, B ∪ C, A ∩ B, A ∩ C, B ∩ C, and A ∩ B. A  {k : k is even}  {2, 4, 6, …} B  {k : k  1, 2, 3 or k  8} C  {k : k  11} A ∪ B  {k : k is odd or k  4, 6} B ∪ C C A ∩ B  {5, 7} A ∩ C  {1, 3, 5, 7, 9} B∩CB A ∩ B  {4, 6} 1.10. Consider the experiment of Example 1.7 of rolling a die. Express – – A ∪ B, A ∩ C, B , C , B \ C, C \ B, B Δ C. From Example 1.7, we have S  {1, 2, 3, 4, 5, 6}, A  {2, 4, 6}, B  {1, 3, 5}, and C  {1, 2, 3, 5}. Then – – A ∪ B  {1, 2, 3, 4, 5, 6}  S, A ∩ C  {2}, B  {2, 4, 6}  A, C  {4, 6} – – B \C  B ∩ C  {∅}, C \ B  C ∩ B  {2}, B Δ C  (B \ C) ∪ (C \ B)  {∅} ∪ {2}  {2} 01_Hsu_Probability 8/31/19 3:55 PM Page 16 16 CHAPTER 1 Probability 1.11. The sample space of an experiment is the real line express as S  {v: – ∞ v ∞} (a) Consider the events ⎧ 1⎫ A1  ⎨v : 0  v ⎬ ⎩ 2⎭ ⎧ 1 3⎫ A2  ⎨v :  v ⎬ ⎩ 2 4⎭ ⎧ 1 1⎫ Ai  ⎨v : 1  i  1  v 1 ⎬ ⎩ 2 2i ⎭ Determine the events ∞ ∞ ∪ Ai and ∩ Ai i 1 i 1 (b) Consider the events ⎧ 1⎫ B1  ⎨v : v  ⎬ ⎩ 2⎭ ⎧ 1⎫ B2  ⎨v : v  ⎬ ⎩ 4⎭  ⎧ 1⎫ Bi  ⎨v : v  i ⎬ ⎩ 2 ⎭ Determine the events ∞ ∞ ∪ Bi and ∩ Bi i 1 i 1 (a) It is clear that ∞ ∪ Ai  {v : 0  v 1} i1 Noting that the Ai’s are mutually exclusive, we have ∞ ∩ Ai  ∅ i 1 (b) Noting that B1 ⊃ B2 ⊃ … ⊃ Bi ⊃ …, we have ∞ ∞ ⎧ 1⎫ ∪ Bi  B1  ⎨⎩v : v  2 ⎬⎭ and ∩ Bi  {v : v  0} i 1 i 1 1.12. Consider the switching networks shown in Fig. 1-5. Let A1, A2, and A3 denote the events that the switches s1, s2, and s3 are closed, respectively. Let Aab denote the event that there is a closed path between terminals a and b. Express Aab in terms of A1, A2, and A3 for each of the networks shown. 01_Hsu_Probability 8/31/19 3:55 PM Page 17 CHAPTER 1 Probability 17 s2 s1 s2 s3 s1 a b a s3 b (a) (c) s1 s1 s2 s2 a b a s3 b s3 (d) (b) Fig. 1-5 (a) From Fig. 1-5(a), we see that there is a closed path between a and b only if all switches s1 , s2 , and s3 are closed. Thus, Aab  A1 ∩ A2 ∩ A3 (b) From Fig. 1-5(b), we see that there is a closed path between a and b if at least one switch is closed. Thus, Aab  A1 ∪ A2 ∪ A3 (c) From Fig. 1-5(c), we see that there is a closed path between a and b if s1 and either s2 or s3 are closed. Thus, Aab  A1 ∩ (A2 ∪ A3 ) Using the distributive law (1.18), we have Aab  (A1 ∩ A2 ) ∪ (A1 ∩ A3 ) which indicates that there is a closed path between a and b if s1 and s2 or s1 and s3 are closed. (d ) From Fig. 1-5(d ), we see that there is a closed path between a and b if either s1 and s2 are closed or s3 i s closed. Thus Aab  (A1 ∩ A2 ) ∪ A3 1.13. Verify the distributive law (1.18). Let s ∈ [A ∩ (B ∪ C )]. Then s ∈ A and s ∈ (B ∪ C). This means either that s ∈ A and s ∈ B or that s ∈ A and s ∈ C; that is, s ∈ (A ∩ B) or s ∈ (A ∩ C ). Therefore, A ∩ (B ∪ C ) ⊂ [(A ∩ B) ∪ (A ∩ C )] Next, let s ∈ [(A ∩ B) ∪ (A ∩ C )]. Then s ∈ A and s ∈ B or s ∈ A and s ∈ C. Thus, s ∈ A and (s ∈ B or s ∈ C). Thus, [(A ∩ B) ∪ (A ∩ C )] ⊂ A ∩ (B ∪ C ) Thus, by the definition of equality, we have A ∩ (B ∪ C)  (A ∩ B) ∪ (A ∩ C) 01_Hsu_Probability 8/31/19 3:55 PM Page 18 18 CHAPTER 1 Probability 1.14. Using a Venn diagram, repeat Prob. 1.13. Fig. 1-6 shows the sequence of relevant Venn diagrams. Comparing Fig. 1-6(b) and 1.6(e), we conclude that A ∩ (B ∪ C)  (A ∩ B) ∪ (A ∩ C ) A A B B C C (a) Shaded region: B  C (b) Shaded region: A  (B  C) A A B B C C (c) Shaded region: A  B (d) Shaded region: A  C A B C (e) Shaded region: (A  B)  (A  C) Fig. 1-6 1.15 Verify De Morgan’s law (1.24) A ∩ B A ∪ B ——— — Suppose that s ∈ A ∩ B , then s ∉ A ∩ B. So s ∉ {both A and B}. This means that either s ∉ A or s ∉ B or s ∉ A – – – – – – and s ∉ B. This implies that s ∈ A ∪ B. Conversely, suppose that s ∈ A ∪ B , that is either s ∈ A or s ∈ B or s ∈ – – ———— {both A and B }. Then it follows that s ∉ A or s ∉ B or s ∉ {both A and B}; that is, s ∉ A ∩ B or s ∈ A ∩ B. Thus, ———— – – we conclude that A ∩ B  A ∪ B. Note that De Morgan’s law can also be shown by using Venn diagram. 1.16. Let A and B be arbitrary events. Show that A ⊂ B if and only if A ∩ B  A. “If ” part: We show that if A ∩ B  A, then A ⊂ B. Let s ∈ A. Then s ∈ (A ∩ B), since A  A ∩ B. Then by the definition of intersection, s ∈ B. Therefore, A ⊂ B. “Only if ” part: We show that if A ⊂ B, then A ∩ B  A. Note that from the definition of the intersection, (A ∩ B) ⊂ A. Suppose s ∈ A. If A ⊂ B, then s ∈ B. So s ∈ A and s ∈ B; that is, s ∈ (A ∩ B). Therefore, it follows that A ⊂ (A ∩ B). Hence, A  A ∩ B. This completes the proof. 1.17. Let A be an arbitrary event in S and let ∅ be the null event. Verify Eqs. (1.8) and (1.9), i.e. (a) A ∪ ∅  A (1.8) (b) A ∩ ∅  ∅ (1.9) 01_Hsu_Probability 8/31/19 3:55 PM Page 19 CHAPTER 1 Probability 19 (a) A ∪ ∅  {s: s ∈ A or s ∈ ∅} But, by definition, there are no s ∈ ∅. Thus, A ∪ ∅  {s: s ∈ A}  A (b) A ∩ ∅  {s: s ∈ A and s ∈ ∅} But, since there are no s ∈ ∅, there cannot be an s such that s ∈ A and s ∈ ∅. Thus, A∩∅∅ Note that Eq. (1.9) shows that ∅ is mutually exclusive with every other event and including with itself. 1.18. Show that the null (or empty) set ∅ is a subset of every set A. From the definition of intersection, it follows that (A ∩ B) ⊂ A and (A ∩ B) ⊂ B (1.70) for any pair of events, whether they are mutually exclusive or not. If A and B are mutually exclusive events, that is, A ∩ B  ∅, then by Eq. (1.70) we obtain ∅⊂A and ∅⊂B (1.71) Therefore, for any event A, ∅⊂A (1.72) that is, ∅ is a subset of every set A. 1.19. Show that A and B are disjoint if and only if A\ B  A. – First, if A \ B  A ∩ B  A, then – – A ∩ B  (A ∩ B ) ∩ B  A ∩ (B ∩ B)  A ∩ ∅  ∅ and A and B are disjoint. – Next, if A and B are disjoint, then A ∩ B  ∅, and A \ B  A ∩ B  A. Thus, A and B are disjoint if and only if A\ B  A. 1.20. Show that there is a distribution law also for difference; that is, (A\ B) ∩ C  (A ∩ C) \ (B ∩ C) By Eq. (1.8) and applying commutative and associated laws, we have – – – (A\ B) ∩ C  (A ∩ B ) ∩ C  A ∩ (B ∩ C )  (A ∩ C ) ∩ B Next, ——— (A ∩ C ) \ (B ∩ C)  (A ∩ C) ∩ (B ∩ C ) by Eq. (1.10) – –  (A ∩ C ) ∩ (B ∪ C ) by Eq. (1.21) 01_Hsu_Probability 8/31/19 3:55 PM Page 20 20 CHAPTER 1 Probability – –  [(A ∩ C ) ∩ B ] ∪ [(A ∩ C ) ∩ C ] by Eq. (1.19) – –  [(A ∩ C ) ∩ B ] ∪ [A ∩ (C ∩ C )] by Eq. (1.17) –  [(A ∩ C ) ∩ B ] ∪ [A ∩ ∅] by Eq. (1.7) –  [(A ∩ C ) ∩ B ] ∪ ∅ by Eq. (1.9) –  (A ∩ C ) ∩ B by Eq. (1.8) Thus, we have (A\ B) ∩ C  (A ∩ C ) \ (B ∩ C ) 1.21. Verify Eqs. (1.24) and (1.25). ⎛ n ⎞ ⎛ n ⎞ (a) Suppose first that s ∈ ⎜⎜ ∪ i⎟A ⎟ ; then s ∉ ⎜ ∪ Ai ⎟. ⎜ ⎟ ⎝ i 1 ⎠ ⎝ i 1 ⎠ – That is, if s is not contained in any of the events Ai, i  1, 2, …, n, then s is contained in A i for all i  1, 2, …, n. Thus n s∈ ∩ Ai i 1 Next, we assume that n s∈ ∩ Ai i 1 – Then s is contained in A i for all i  1, 2, …, n, which means that s is not contained in Ai for any i  1, 2, …, n, implying that n s ∉ ∪ Ai i 1 ⎛ n ⎞ Thus, s ∈ ⎜ ∪ Ai ⎟ ⎜ ⎟ ⎝ i 1 ⎠ This proves Eq. (1.24). (b) Using Eqs. (1.24) and (1.3), we have ⎛ n ⎞ n ⎜ ∪ Ai ⎟  ∩ Ai ⎜ ⎟ ⎝ i 1 ⎠ i 1 Taking complements of both sides of the above yields n ⎛ n ⎞ ∪ Ai  ⎜⎜ ∩ Ai ⎟⎟ i 1 ⎝ i 1 ⎠ which is Eq. (1.25). 01_Hsu_Probability 8/31/19 3:55 PM Page 21 CHAPTER 1 Probability 21 Probability Space 1.22. Consider a probability space (S, F, P). Show that if A and B are in an event space (σ-field) F, so are A ∩ B, A\ B, and A Δ B. – – By condition (ii) of F, Eq. (1.27), if A, B ∈ F, then A , B ∈ F. Now by De Morgan’s law (1.21), we have A∩ BA∪ B ∈ F by Eq. (1.28) A∩ BA∩ B ∈ F by Eq. (1.27) Similarly, we see that – – A∩B∈F and A∩B∈F Now by Eq. (1.10), we have – A\ B  A ∩ B ∈ F Finally, by Eq. (1.13), and Eq. (1.28), we see that – – A Δ B  (A ∩ B ) ∪ (A ∩ B) ∈ F 1.23. Consider the experiment of Example 1.7 of rolling a die. Show that {S, ∅, A, B} are event spaces but {S, ∅, A} and {S, ∅, A, B, C} are not event spaces. Let F  {S, ∅, A, B}. Then we see that – — – – S ∈ F, S  ∅ ∈ F, ∅  S ∈ F, A  B ∈ F, B  A ∈ F and S ∪ ∅  S ∪ A  S ∪ B  S ∈ F, ∅ ∪ A  A ∪ ∅  A ∈ F, ∅ ∪ B  B ∪ ∅  B ∈ F, A ∪ B  B ∪ A  S ∈ F Thus, we conclude that {S, ∅, A, B} is an event space (σ-field). – Next, let F  {S, ∅, A}. Now A  B ∉ F. Thus {S, ∅, A} is not an event space. Finally, let F  {S, ∅, A, B, C}, – but C  {2, 6} ∉ F. Hence, {S, ∅, A, B, C} is not an event space. 1.24. Using the axioms of probability, prove Eq. (1.39). We have – – SA∪A and A∩A∅ Thus, by axioms 2 and 3, it follows that – P(S)  1  P(A)  P(A ) from which we obtain – P(A )  1  P(A) 01_Hsu_Probability 8/31/19 3:55 PM Page 22 22 CHAPTER 1 Probability 1.25. Verify Eq. (1.40). From Eq. (1.39), we have – P(A)  1  P(A ) – Let A  ∅. Then, by Eq. (1.2), A  ∅  S, and by axiom 2 we obtain P(∅)  1  P(S)  1  1  0 1.26. Verify Eq. (1.41). Let A ⊂ B. Then from the Venn diagram shown in Fig. 1-7, we see that – – B  A ∪ (A ∩ B) and A ∩ (A ∩ B)  ∅ (1.74) Hence, from axiom 3, – P(B)  P(A)  P(A ∩ B) – However, by axiom 1, P(A ∩ B)  0. Thus, we conclude that P(A)  P(B) if A ⊂ B S A B Shaded region: A  B Fig. 1-7 1.27. Verify Eq. (1.43). From the Venn diagram of Fig. 1-8, each of the sets A ∪ B and B can be represented, respectively, as a union of mutually exclusive sets as follows: – – A ∪ B  A ∪ (A ∩ B) and B  (A ∩ B) ∪ (A ∩ B) Thus, by axiom 3, – P(A ∪ B)  P(A)  P(A ∩ B) (1.75) – and P(B)  P(A ∩ B)  P(A ∩ B) (1.76) From Eq. (1.76), we have – P(A ∩ B)  P(B)  P(A ∩ B) (1.77) Substituting Eq. (1.77) into Eq. (1.75), we obtain P(A ∪ B)  P(A)  P(B)  P(A ∩ B) 01_Hsu_Probability 8/31/19 3:55 PM Page 23 CHAPTER 1 Probability 23 S S A B A B Shaded region: A  B Shaded region: A  B Fig. 1-8 1.28. Let P(A)  0.9 and P(B)  0.8. Show that P(A ∩ B)  0.7. From Eq. (1.43), we have P(A ∩ B)  P(A)  P(B)  P(A ∪ B) By Eq. (1.47), 0  P(A ∪ B)  1. Hence, P(A ∩ B)  P(A)  P(B)  1 (1.78) Substituting the given values of P(A) and P(B) in Eq. (1.78), we get P(A ∩ B)  0.9  0.8  1  0.7 Equation (1.77) is known as Bonferroni’s inequality. 1.29. Show that – P(A)  P(A ∩ B)  P(A ∩ B ) (1.79) From the Venn diagram of Fig. 1-9, we see that – – A  (A ∩ B) ∪ (A ∩ B ) and (A ∩ B) ∩ (A ∩ B )  ∅ (1.80) Thus, by axiom 3, we have – P(A)  P(A ∩ B)  P(A ∩ B ) S A B AB AB Fig. 1-9 – 1.30. Given that P(A)  0.9, P(B)  0.8, and P(A ∩ B)  0.75, find (a) P(A ∪ B); (b) P(A ∩ B ); and – – (c) P(A ∩ B ). (a) By Eq. (1.43), we have P(A ∪ B)  P(A)  P(B)  P(A ∩ B)  0.9  0.8  0.75  0.95 01_Hsu_Probability 8/31/19 3:55 PM Page 24 24 CHAPTER 1 Probability (b) By Eq. (1.79) (Prob. 1.29), we have – P(A ∩ B )  P(A)  P(A ∩ B)  0.9  0.75  0.15 (c) By De Morgan’s law, Eq. (1.20), and Eq. (1.39) and using the result from part (a), we get P ( A ∩ B )  P ( A ∪ B)  1  P ( A ∪ B)  1  0.95  0.05 1.31. For any three events A1, A2, and A3, show that P(A1 ∪ A2 ∪ A3)  P(A1)  P(A2)  P(A3)  P(A1 ∩ A2)  P(A1 ∩ A3)  P(A2 ∩ A3)  P(A1 ∩ A2 ∩ A3) (1.81) Let B  A2 ∪ A3. By Eq. (1.43), we have P(A1 ∪ B)  P(A1 )  P(B)  P(A1 ∩ B) (1.82) Using distributive law (1.18), we have A1 ∩ B  A1 ∩ (A2 ∪ A3 )  (A1 ∩ A2 ) ∪ (A1 ∩ A3 ) Applying Eq. (1.43) to the above event, we obtain P(A1 ∩ B)  P(A1 ∩ A2 )  P(A1 ∩ A3 )  P[(A1 ∩ A2 ) ∩ (A1 ∩ A3 )]  P(A1 ∩ A2 )  P(A1 ∩ A3 )  P(A1 ∩ A2 ∩ A3 ) (1.83) Applying Eq. (1.43) to the set B  A2 ∪ A3 , we have P(B)  P(A2 ∪ A3 )  P(A2 )  P(A3 )  P(A2 ∩ A3 ) (1.84) Substituting Eqs. (1.84) and (1.83) into Eq. (1.82), we get P(A1 ∪ A2 ∪ A3 )  P(A1 )  P(A2 )  P(A3 )  P(A1 ∩ A2 )  P(A1 ∩ A3 )  P(A2 ∩ A3 )  P(A1 ∩ A2 ∩ A3 ) 1.32. Prove that ⎛ n ⎞ n P ⎜ ∪ Ai ⎟  ∑ P( Ai ) (1.85) ⎜ ⎟ ⎝ i 1 ⎠ i 1 which is known as Boole’s inequality. We will prove Eq. (1.85) by induction. Suppose Eq. (1.85) is true for n  k. ⎛ k ⎞ k  P ⎜⎜ ∪ Ai ⎟⎟  ∑ P ( Ai ) ⎝ i 1 ⎠ i 1 ⎛k 1 ⎞ ⎡⎛ k ⎞ ⎤ Then P ⎜⎜ ∪ Ai ⎟⎟  P ⎢⎜⎜ ∪ Ai ⎟⎟ ∪ Ak  1 ⎥ ⎝ i 1 ⎠ ⎢⎣⎝ i 1 ⎠ ⎥⎦ ⎛ k ⎞  P ⎜⎜ ∪ Ai ⎟⎟  P ( Ak  1 ) [ by Eq.(1.48 )] ⎝ i 1 ⎠ k k 1  ∑ P ( Ai )  P ( Ak  1 )  ∑ P( Ai ) i 1 i 1