Schaum's Outline of Probability, Random Variables, and Random Processes PDF

 Schaum's Outline of Theory and Problems of Probability, Random Variables, and Random Processes Hwei P. Hsu, Ph.D. Professor of Electrical Engineering Fairleigh Dickinson Uni...

 Schaum's Outline of Theory and Problems of Probability, Random Variables, and Random Processes Hwei P. Hsu, Ph.D. Professor of Electrical Engineering Fairleigh Dickinson University Start of Citation[PU]McGraw-Hill Professional[/PU][DP]1997[/DP]End of Citation HWEI P. HSU is Professor of Electrical Engineering at Fairleigh Dickinson University. He received his B.S. from National Taiwan University and M.S. and Ph.D. from Case Institute of Technology. He has published several books which include Schaum's Outline of Analog and Digital Communications and Schaum's Outline of Signals and Systems. Schaum's Outline of Theory and Problems of PROBABILITY, RANDOM VARIABLES, AND RANDOM PROCESSES Copyright © 1997 by The McGraw-Hill Companies, Inc. All rights reserved. Printed in the United States of America. Except as permitted under the Copyright Act of 1976, no part of this publication may be reproduced or distributed in any form or by any means, or stored in a data base or retrieval system, without the prior written permission of the publisher. 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 PRS PRS 9 0 1 0 9 8 7 ISBN 0-07-030644-3 Sponsoring Editor: Arthur Biderman Production Supervisor: Donald F. Schmidt Editing Supervisor: Maureen Walker Library of Congress Cataloging-in-Publication Data Hsu, Hwei P. (Hwei Piao), date Schaum's outline of theory and problems of probability, random variables, and random processes / Hwei P. Hsu. p. cm. — (Schaum's outline series) Includes index. ISBN 0-07-030644-3 1. Probabilities—Problems, exercises, etc. 2. Probabilities- Outlines, syllabi, etc. 3. Stochastic processes—Problems, exercises, etc. 4. Stochastic processes—Outlines, syllabi, etc. I. Title. QA273.25.H78 1996 519.2'076—dc20 96- 18245 CIP Start of Citation[PU]McGraw-Hill Professional[/PU][DP]1997[/DP]End of Citation Preface The purpose of this book is to provide an introduction to principles of probability, random variables, and random processes and their applications. The book is designed for students in various disciplines of engineering, science, mathematics, and management. It may be used as a textbook and/or as a supplement to all current comparable texts. It should also be useful to those interested in the field for self-study. The book combines the advantages of both the textbook and the so-called review book. It provides the textual explanations of the textbook, and in the direct way characteristic of the review book, it gives hundreds of completely solved problems that use essential theory and techniques. Moreover, the solved problems are an integral part of the text. The background required to study the book is one year of calculus, elementary differential equations, matrix analysis, and some signal and system theory, including Fourier transforms. I wish to thank Dr. Gordon Silverman for his invaluable suggestions and critical review of the manuscript. I also wish to express my appreciation to the editorial staff of the McGraw-Hill Schaum Series for their care, cooperation, and attention devoted to the preparation of the book. Finally, I thank my wife, Daisy, for her patience and encouragement. HWEI P. HSU MONTVILLE, NEW JERSEY Start of Citation[PU]McGraw-Hill Professional[/PU][DP]1997[/DP]End of Citation Contents Chapter 1. Probability 1 1.1 Introduction 1 1.2 Sample Space and Events 1 1.3 Algebra of Sets 2 1.4 The Notion and Axioms of Probability 5 1.5 Equally Likely Events 7 1.6 Conditional Probability 7 1.7 Total Probability 8 1.8 Independent Events 8 Solved Problems 9 Chapter 2. Random Variables 38 2.1 Introduction 38 2.2 Random Variables 38 2.3 Distribution Functions 39 2.4 Discrete Random Variables and Probability Mass Functions 41 2.5 Continuous Random Variables and Probability Density Functions 41 2.6 Mean and Variance 42 2.7 Some Special Distributions 43 2.8 Conditional Distributions 48 Solved Problems 48 Chapter 3. Multiple Random Variables 79 3.1 Introduction 79 3.2 Bivariate Random Variables 79 3.3 Joint Distribution Functions 80 3.4 Discrete Random Variables - Joint Probability Mass Functions 81 3.5 Continuous Random Variables - Joint Probability Density Functions 82 3.6 Conditional Distributions 83 3.7 Covariance and Correlation Coefficient 84 3.8 Conditional Means and Conditional Variances 85 3.9 N-Variate Random Variables 86 3.10 Special Distributions 88 Solved Problems 89 v  vi Chapter 4. Functions of Random Variables, Expectation, Limit Theorems 122 4.1 Introduction 122 4.2 Functions of One Random Variable 122 4.3 Functions of Two Random Variables 123 4.4 Functions of n Random Variables 124 4.5 Expectation 125 4.6 Moment Generating Functions 126 4.7 Characteristic Functions 127 4.8 The Laws of Large Numbers and the Central Limit Theorem 128 Solved Problems 129 Chapter 5. Random Processes 161 5.1 Introduction 161 5.2 Random Processes 161 5.3 Characterization of Random Processes 161 5.4 Classification of Random Processes 162 5.5 Discrete-Parameter Markov Chains 165 5.6 Poisson Processes 169 5.7 Wiener Processes 172 Solved Problems 172 Chapter 6. Analysis and Processing of Random Processes 209 6.1 Introduction 209 6.2 Continuity, Differentiation, Integration 209 6.3 Power Spectral Densities 210 6.4 White Noise 213 6.5 Response of Linear Systems to Random Inputs 213 6.6 Fourier Series and Karhunen-Loéve Expansions 216 6.7 Fourier Transform of Random Processes 218 Solved Problems 219 Chapter 7. Estimation Theory 247 7.1 Introduction 247 7.2 Parameter Estimation 247 7.3 Properties of Point Estimators 247 7.4 Maximum-Likelihood Estimation 248 7.5 Bayes' Estimation 248 7.6 Mean Square Estimation 249 7.7 Linear Mean Square Estimation 249 Solved Problems 250  vii Chapter 8. Decision Theory 264 8.1 Introduction 264 8.2 Hypothesis Testing 264 8.3 Decision Tests 265 Solved Problems 268 Chapter 9. Queueing Theory 281 9.1 Introduction 281 9.2 Queueing Systems 281 9.3 Birth-Death Process 282 9.4 The M/M/1 Queueing System 283 9.5 The M/M/s Queueing System 284 9.6 The M/M/1/K Queueing System 285 9.7 The M/M/s/K Queueing System 285 Solved Problems 286 Appendix A. Normal Distribution 297 Appendix B. Fourier Transform 299 B.1 Continuous-Time Fourier Transform 299 B.2 Discrete-Time Fourier Transform 300 Index 303  Chapter 1 Probability 1.1 INTRODUCTION The study of probability stems from the analysis of certain games of chance, and it has found applications in most branches of science and engineering. In this chapter the basic concepts of prob- ability theory are presented. 1.2 SAMPLE SPACE AND EVENTS A. Random Experiments: In the study of probability, any process of observation is referred to as an experiment. The results of an observation are called the outcomes of the experiment. An experiment is called a random experi- ment if its outcome cannot be predicted. Typical examples of a random experiment are the roll of a die, the toss of a coin, drawing a card from a deck, or selecting a message signal for transmission from several messages. B. Sample Space: The set of all possible outcomes of a random experiment is called the sample space (or universal set), and it is denoted by S. An element in S is called a sample point. Each outcome of a random experiment corresponds to a sample point. EXAMPLE 1.1 Find the sample space for the experiment of tossing a coin (a) once and (b) twice. (a) There are two possible outcomes, heads or tails. Thus S = {H, T) where H and T represent head and tail, respectively. (b) There are four possible outcomes. They are pairs of heads and tails. Thus S = (HH, HT, TH, TT) EXAMPLE 1.2 Find the sample space for the experiment of tossing a coin repeatedly and of counting the number of tosses required until the first head appears. Clearly all possible outcomes for this experiment are the terms of the sequence 1,2,3,.... Thus s = (1, 2, 3,...) Note that there are an infinite number of outcomes. EXAMPLE 1.3 Find the sample space for the experiment of measuring (in hours) the lifetime of a transistor. Clearly all possible outcomes are all nonnegative real numbers. That is, S=(z:O P(A), then P(A n B) > P(A)P(B).Thus, P(B) 1.40. Consider the experiment of throwing the two fair dice of Prob. 1.31 behind you; you are then informed that the sum is not greater than 3. (a) Find the probability of the event that two faces are the same without the information given. (b) Find the probability of the same event with the information given. (a) Let A be the event that two faces are the same. Then from Fig. 1-3 (Prob. 1.5) and by Eq. (1.38), we have A = {(i, i): i = 1, 2,..., 6) and  PROBABILITY [CHAP 1 (b) Let B be the event that the sum is not greater than 3. Again from Fig. 1-3, we see that B = {(i, j): i + j 5 3) = {(I, I), (1, 21, (2, I)} and Now A n B is the event that two faces are the same and also that their sum is not greater than 3. Thus, Then by definition (1.39), we obtain Note that the probability of the event that two faces are the same doubled from 8 to 4 with the information given. Alternative Solution: There are 3 elements in B, and 1 of them belongs to A. Thus, the probability of the same event with the information given is 5. 1.41. Two manufacturing plants produce similar parts. Plant 1 produces 1,000 parts, 100 of which are defective. Plant 2 produces 2,000 parts, 150 of which are defective. A part is selected at random and found to be defective. What is the probability that it came from plant 1? Let B be the event that "the part selected is defective," and let A be the event that "the part selected came from plant 1." Then A n B is the event that the item selected is defective and came from plant 1. Since a part is selected at random, we assume equally likely events, and using Eq. (1.38), we have Similarly, since there are 3000 parts and 250 of them are defective, we have By Eq. (1.39), the probability that the part came from plant 1 is Alternative Solution : There are 250 defective parts, and 100 of these are from plant 1. Thus, the probability that the defective part came from plant 1 is # = 0.4. 1.42. A lot of 100 semiconductor chips contains 20 that are defective. Two chips are selected at random, without replacement, from the lot. (a) What is the probability that the first one selected is defective? (b) What is the probability that the second one selected is defective given that the first one was defective? (c) What is the probability that both are defective? CHAP. 1) PROBABILITY (a) Let A denote the event that the first one selected is defective. Then, by Eq. (1.38), P(A) = = 0.2 (b) Let B denote the event that the second one selected is defective. After the first one selected is defective, there are 99 chips left in the lot with 19 chips that are defective. Thus, the probability that the second one selected is defective given that the first one was defective is (c) By Eq. (l.41),the probability that both are defective is 1.43. A number is selected at random from (1, 2,..., 100). Given that the number selected is divisible by 2, find the probability that it is divisible by 3 or 5. Let A, = event that the number is divisible by 2 A, = event that the number is divisible by 3 A , = event that the number is divisible by 5 Then the desired probability is - P(A3 n A,) + P(A, n A,) - P(A3 n As n A,) C E ~(1.29)1. P(A2 ) Now A , n A, = event that the number is divisible by 6 A , n A, = event that the number is divisible by 10 A , n A , n A, = event that the number is divisible by 30 and P(A, n A,) = AS n A21 = 7% P(A, n As n A,) = &, - Z O + Ah -hi - 23 Thus, P(A3 u As I A21 = -50 - 0.46 loo 50 1.44. Let A , , A ,,..., A,beeventsinasamplespaceS. Show that P(A1 n A , n. n A,) = P(A,)P(A, 1 A,)P(A, I A, n A,). P(A, ( A , n A, n.. n A,- ,) (1.81) We prove Eq. (1.81) by induction. Suppose Eq. (1.81)is true for n = k: P(Al n A, n.. n A,) = P(Al)P(A2I A,)P(A, I A , n A:,). -.P(A, I A , n A, n -- n A,- ,) Multiplying both sides by P(A,+, I A , n A , n... n A,), we have P(Al n A, n - - - n A,)P(A,+,IA, n A, n n A,) = P(Al n A , n -..n A,,,) and P(A, n A , n -.. n A,, , ) = P(A,)P(A, 1 A,)P(A31 A , rl A,) - -.P(A,+, 1 A , n A, n -.. n A,) Thus, Eq. (1.81) is also true for n =k + 1. By Eq. ( 1 A l ) , Eq. (1.81) is true for n = 2. Thus Eq. (1.81) is true for n 2 2. 1.45. Two cards are drawn at random from a deck. Find the probability that both are aces. Let A be the event that the first card is an ace, and B be the event that the second card is an ace. The desired probability is P(B n A). Since a card is drawn at random, P(A) = A. Now if the first card is an ace, then there will be 3 aces left in the deck of 51 cards. Thus P(B I A ) = A. By Eq. ( 1.dl),  PROBABILITY [CHAP 1 Check: By counting technique, we have 1.46. There are two identical decks of cards, each possessing a distinct symbol so that the cards from each deck can be identified. One deck of cards is laid out in a fixed order, and the other deck is shufkd and the cards laid out one by one on top of the fixed deck. Whenever two cards with the same symbol occur in the same position, we say that a match has occurred. Let the number of cards in the deck be 10. Find the probability of getting a match at the first four positions. Let A,, i = 1,2,3,4, be the events that a match occurs at the ith position. The required probability is P(A, n A, n A, n A,) By Eq. (1.81), There are 10 cards that can go into position 1, only one of which matches. Thus, P(Al) = &. P(A, ( A , ) is the conditional probability of a match at position 2 given a match at position 1. Now there are 9 cards left to go into position 2, only one of which matches. Thus, P(A2 I A,) = *. In a similar fashion, we obtain P(A3 I A , n A,) = 4 and P(A, I A, n A, n A,) = 4. Thus, TOTAL PROBABILITY 1.47. Verify Eq. (1.44). Since B n S = B [and using Eq. (1.43)],we have B = B n S = B n ( A , u A, u u An) = ( B n A,) u (B n A,) u... u ( B n An) Now the events B n A,, i = 1,2,..., n, are mutually exclusive, as seen from the Venn diagram of Fig. 1-14. Then by axiom 3 of probability and Eq. (1.41),we obtain BnA, BnA, BnA, Fig. 1-14 CHAP. 1) PROBABILITY Show that for any events A and B in S , P(B) = P(B I A)P(A) + P(B I A)P(X) From Eq. (1.64) (Prob. 1.23), we have P(B) = P(B n A) + P(B n 4 Using Eq. (1.39), we obtain P(B) = P(B I A)P(A) + P(B I X)P(A) Note that Eq. (1.83) is the special case of Eq. (1.44). Suppose that a laboratory test to detect a certain disease has the following statistics. Let A = event that the tested person has the disease B = event that the test result is positive It is known that P(B I A) = 0.99 and P(B I A) = 0.005 and 0.1 percent of the population actually has the disease. What is the probability that a person has the disease given that the test result is positive? From the given statistics, we have P(A) = 0.001 then P(A) = 0.999 The desired probability is P(A ) B). Thus, using Eqs. (1.42) and (1.83), we obtain Note that in only 16.5 percent of the cases where the tests are positive will the person actually have the disease even though the test is 99 percent effective in detecting the disease when it is, in fact, present. A company producing electric relays has three manufacturing plants producing 50, 30, and 20 percent, respectively, of its product. Suppose that the probabilities that a relay manufactured by these plants is defective are 0.02,0.05, and 0.01, respectively. If a relay is selected at random from the output of the company, what is the probability that it is defective? If a relay selected at random is found to be defective, what is the probability that it was manufactured by plant 2? Let B be the event that the relay is defective, and let Ai be the event that the relay is manufactured by plant i (i = 1,2, 3). The desired probability is P(B). Using Eq. (1.44), we have  PROBABILITY [CHAP 1 (b) The desired probability is P(A2 1 B). Using Eq. (1.42) and the result from part (a), we obtain 1.51. Two numbers are chosen at random from among the numbers 1 to 10 without replacement. Find the probability that the second number chosen is 5. Let A,, i = 1, 2,..., 10 denote the event that the first number chosen is i. Let B be the event that the second number chosen is 5. Then by Eq. (1.44), Now P(A,) = A. P(B I A,) is the probability that the second number chosen is 5, given that the first is i. If i = 5, then P(B I Ai)= 0. If i # 5, then P(B I A,) = 4.Hence, 1.52. Consider the binary communication channel shown in Fig. 1-15. The channel input symbol X may assume the state 0 or the state 1, and, similarly, the channel output symbol Y may assume either the state 0 or the state 1. Because of the channel noise, an input 0 may convert to an output 1 and vice versa. The channel is characterized by the channel transition probabilities p,, 40, PI, and 91, ckfined by where x , and x, denote the events (X = 0 ) and ( X = I), respectively, and yo and y, denote the + events ( Y = 0) and (Y = I), respectively. Note that p, q , = 1 = p, + q,. Let P(xo) = 0.5, po = 0.1, and p, = 0.2. (a) Find P(yo) and P ( y l ). (b) If a 0 was observed at the output, what is the probability that a 0 was the input state? (c) If a 1 was observed at the output, what is the probability that a 1 was the input state? (d) Calculate the probability of error P,. Fig. 1-15 (a) We note that CHAP. 11 PROBABILITY Using Eq. (1.44), we obtain (b) Using Bayes' rule (1.42), we have (c) Similarly, (d) The probability of error is P, = P(yl (xo)P(xo)+ P(yo ( x l ) P ( x l =) O.l(O.5) + 0.2(0.5) = 0.15. INDEPENDENT EVENTS 1.53. Let A and B be events in a sample space S. Show that if A and B are independent, then so are (a) A and B,(b) A and B, and (c) A and B. (a) From Eq. (1.64) (Prob. 1.23),we have P(A) = P(A n B) + P(A n B) Since A and B are independent, using Eqs. (1.46) and (1. B ) , we obtain P(A n B) = P(A) - P(A n B) = P(A) - P(A)P(B) = P(A)[l - P(B)] = P(A)P(B) Thus, by definition (l.46), A and B are independent. (b) Interchanging A and B in Eq. (1.84), we obtain P(B n 3 = P(B)P(A) which indicates that A and B are independent. (c) We have P ( A n B) = P[(A u B)] [Eq. (1.1411 = 1 - P(A u B) [Eq- (1.25)1 = 1 - P(A) - P(B) + P(A n B) [Eq. (1.29)] = 1 - P(A) - P(B) + P(A)P(B) [Eq. (1.46)] = 1 - P(A) - P(B)[l - P(A)] = [l - P(A)][l - P(B)] = P(A)P(B) [Eq. (1.2511 Hence, A and B are independent. 1.54. Let A and B be events defined in a sample space S. Show that if both P(A) and P(B) are nonzero, then events A and B cannot be both mutually exclusive and independent. Let A and B be mutually exclusive events and P(A) # 01, P(B) # 0. Then P(A n B) = P(%) = 0 but P(A)P(B)# 0. Since A and B cannot be independent. 1.55. Show that if three events A, B, and C are independent, then A and (B u C) are independent.  PROBABILITY [CHAP 1 We have P [ A n ( B u C ) ] = P [ ( A n B) u ( A n C ) ] [Eq. (1.12)1 =P(AnB)+P(AnC)-P(AnBnC) [Eq.(1.29)] + = P(A)P(B) P(A)P(C)- P(A)P(B)P(C) CEq. ( 1 W I = P(A)P(B)+ P(A)P(C)- P(A)P(B n C ) [Eq. (1.50)] = P(A)[P(B)+ P(C) - P(B n C ) ] = P(A)P(B u C ) C E ~(1.2911. Thus, A and ( B u C ) are independent. 1.56. Consider the experiment of throwing two fair dice (Prob. 1.31). Let A be the event that the sum of the dice is 7, B be the event that the sum of the dice is 6, and C be the event that the first die is 4. Show that events A and C are independent, but events B and C are not independent. From Fig. 1-3 (Prob. l.5), we see that and Now and Thus, events A and C are independent. But Thus, events B and C are not independent. 1.57. In the experiment of throwing two fair dice, let A be the event that the first die is odd, B be the event that the second die is odd, and C be the event that the sum is odd. Show that events A, B, and C are pairwise independent, but A, B, and C are not independent. From Fig. 1-3 (Prob. 1.5), we see that Thus which indicates that A, B, and C are pairwise independent. However, since the sum of two odd numbers is even, ( A n B n C ) = 0and P(A n B n C ) = 0 # $ = P(A)P(B)P(C) which shows that A, B, and C are not independent. 1.58. A system consisting of n separate components is said to be a series system if it functions when all n components function (Fig. 1-16). Assume that the components fail independently and that the probability of failure of component i is pi, i = 1, 2,..., n. Find the probability that the system functions. Fig. 1-16 Series system. CHAP. I] PROBABILITY Let Ai be the event that component si functions. Then P(Ai) = 1 - P(Ai) = 1 - pi Let A be the event that the system functions. Then, since A,'s are independent, we obtain 1.59. A system consisting of n separate components is said to be a parallel system if it functions when at least one of the components functions (Fig. 1-17). Assume that the components fail indepen- dently and that the probability of failure of component i is pi, i = 1, 2,..., n. Find the probabil- ity that the system functions. Fig. 1-17 Parallel system. Let Ai be the event that component si functions. Then Let A be the event that the system functions. Then, since A,'s are independent, we obtain 1.60. Using Eqs. (1.85) and (1.86), redo Prob. 1.34. From Prob. 1.34, pi = 4,i = 1, 2, 3, 4, where pi is the probability of failure of switch si. Let A be the event that there exists a closed path between a and b. Using Eq. (1.86), the probability of failure for the parallel combination of switches 3 and 4 is P34 = P3P4 = (+)(a) a == Using Eq. (1.85), the probability of failure for the combination of switches 2, 3, and 4 is p234= 1 - (1 - 4x1 - i)1 - 38 =; =8 Again, using Eq. (1.86),we obtain 1.61. A Bernoulli experiment is a random experiment, the outcome of which can be classified in but one of two mutually exclusive and exhaustive ways, say success or failure. A sequence of Ber- noulli trials occurs when a. Bernoulli experiment is performed several independent times so that the probability of success, say p, remains the same from trial to trial. Now an infinite sequence of Bernoulli trials is performed. Find the probability that (a) at least 1 success occurs in the first n trials; (b) exactly k successes occur in the first n trials; (c) all trials result in successes. (a) In order to find the probability of at least 1 success in the first n trials, it is easier to first compute the probability of the complementary event, that of no successes in the first n trials. Let Ai denote the event  PROBABILITY [CHAP 1 of a failure on the ith trial. Then the probability of no successes is, by independence, P(A, n A, n -.. n A,) = P(Al)P(A2). -. P(A,) = (1 - p)" (1.87) Hence, the probability that at least 1 success occurs in the first n trials is 1 - (1 - p)". (b) In any particular sequence of the first n outcomes, if k successes occur, where k = 0, 1, 2,..., n, then n - k failures occur. There are (9 such sequences, and each one of these has probability pk(l - P)"-~. Thus, the probability that exactly k successes occur in the first n trials is given by -p y k. (c) Since Ai denotes the event of a success on the ith trial, the probability that all trials resulted in successes in the first n trials is, by independence, P(Al n A, n. + n An)= P(A,)P(A,).. P(A,,) = pn (1.88) Hence, using the continuity theorem of probability (1.74) (Prob. 1.28), the probability that all trials result in successes is given by 0 p x) = {C: X([) > x) (xl < X I x2) = {C: XI < X(C) l x2) CHAP. 21 RANDOM VARIABLES Fig. 2-2 One random variable associated with coin tossing. These events have probabilities that are denoted by P(X = x) = P{C: X(6) = X} P(X 5 x) = P(6: X(6) 5 x} P(X > x) = P{C: X(6) > x) P(x, < X I x,) = P { ( : x , < X(C) I x,) EXAMPLE 2.2 In the experiment of tossing a fair coin three times (Prob. 1.1), the sample space S, consists of eight equally likely sample points S , = (HHH,..., TTT). If X is the r.v. giving the number of heads obtained, find (a) P(X = 2); (b) P(X < 2). (a) Let A c S, be the event defined by X = 2. Then, from Prob. 1.1, we have A = ( X = 2) = {C: X(C) = 2 ) = {HHT, HTH, THH) Since the sample points are equally likely, we have P(X = 2) = P(A) = 3 (b) Let B c S , be the event defined by X < 2. Then B = ( X < 2) = { c : X ( ( ) < 2 ) = (HTT, THT, TTH, TTT) and P(X < 2) = P(B) = 3 = 4 2.3 DISTRIBUTION FUNCTIONS A. Definition : The distribution function [or cumulative distributionfunction (cdf)] of X is the function defined by Most of the information about a random experiment described by the r.v. X is determined by the behavior of FAX). B. Properties of FAX): Several properties of FX(x)follow directly from its definition (2.4). 2. Fx(xl)IFx(x,) if x , < x2 3. lim F,(x) = Fx(oo) = 1 x-'m 4. lim F A X )= Fx(- oo) = 0 x-r-m 5. lim F A X )= F d a + ) = Fx(a) a + = lim a +E x+a+ O 3). (a) TherangeofXisR, = {1,2, 3). (b) P(X = 1) = P[{a)] = P(a) = P(X = 2) = P[(b)] = P(b) = $ P(X = 3) = P[(c, d)] = P(c) + P(d) = $ P(X > 3) = P(%) = 0 2.4. Consider the experiment of throwing a dart onto a circular plate with unit radius. Let X be the r.v. representing the distance of the point where the dart lands from the origin of the plate. Assume that the dart always lands on the plate and that the dart is equally likely to land anywhere on the plate. (a) What is the range of X ? (b) Find (i) P(X < a) and (ii) P(a < X < b), where a < b I1. (a) The range of X is R, = (x: 0 I x < 1). (b) (i) (X < a) denotes that the point is inside the circle of radius a. Since the dart is equally likely to fall anywhere on the plate, we have (Fig. 2-10) (ii) (a < X < b) denotes the event that the point is inside the annular ring with inner radius a and outer radius b. Thus, from Fig. 2-10, we have DISTRIBUTION FUNCTION 2.5. Verify Eq. (2.6). Let x, < x,. Then (X 5 x,) is a subset of ( X Ix,); that is, (X I x,) c (X I x,). Then, by Eq. (1.27), we have  RANDOM VARIABLES [CHAP 2 Fig. 2-10 2.6. Verify (a) Eq. (2.10);(b) Eq. (2.1 1 ) ; (c) Eq. (2.12). (a) Since ( X _< b ) = ( X I a ) u (a < X _< b) and ( X I a ) n ( a < X 5 h) = @, we have P(X I h) = P(X 5 a ) + P(u < X Ib) or + F,y(b) = FX(a) P(u < X I h) Thus, P(u < X 5 b) = Fx(h) - FX(u) (b) Since ( X 5 a ) u ( X > a) = S and (X Ia) n ( X > a) = a,we have P(X S a) + P(X > a ) = P(S) = 1 Thus, P(X > a ) = 1 - P(X 5 a ) = 1 - Fx(u) (c) Now P(X < h) = P[lim X 5 h - E ] = lim P(X I b - E ) c-0 c+O c>O E>O = lim Fx(h - E ) = Fx(b - ) 8-0 >0 8: 2.7. Show that + (a) P(a i X i b ) = P(X = a) Fx(b) - Fx(a) (b) P(a < X < b ) = Fx(b) - F,(a) - P ( X = h ) + (c) P(a i X < b) = P(X = u) Fx(b) - Fx(a) - P(X = b) (a) Using Eqs. (1.23) and (2.10),we have P(a I X I h) = P[(X = u) u ( a < X I b)] + = P(X = u ) P(a < X 5 b ) = P(X = a ) + F,y(h) - FX(a) (b) We have P(a < X 5 b ) = P[(u < X c h ) u ( X = b)] = P(u < X < h) + P(X = b ) CHAP. 21 RANDOM VARIABLES Again using Eq. (2.10), we obtain P(a < X < b) = P(a < X I b) - P(X = b) = Fx(b) - Fx(a) - P(X = b) Similarly, P(a IX I b) = P[(a I X < b) u (X = b)] = P(a I X < b) + P(X = b) Using Eq. (2.64), we obtain P(a IX < b) = P(a 5 X 5 b) - P(X = b) = P(X = a) + Fx(b) - F,(a) - P(X = b) X be the r.v. defined in Prob. 2.3. Sketch the cdf FX(x)of X and specify the type of X. Find (i) P(X I I), (ii) P(l < X I 2), (iii) P(X > I), and (iv) P(l I X I 2). From the result of Prob. 2.3 and Eq. (2.18), we have which is sketched in Fig. 2-1 1. The r.v. X is a discrete r.v. (i) We see that P(X 5 1) = Fx(l) = 4 (ii) By Eq. (2.1O), P(l < X 5 2) = Fx(2) - FA1) = -4 = (iii) By Eq. (2.1I), P(X > 1) = 1 - Fx(l) = 1 - $ =$ (iv) By Eq. (2.64), P(l IX I2) = P(X = 1) + Fx(2) - Fx(l) = 3 + 3 - 3 = 3 Fig. 2-1 1 Sketch the cdf F,(x) of the r.v. X defined in Prob. 2.4 and specify the type of X. From the result of Prob. 2.4, we have 0 x 0. Since b > 0 , pro- perty 3 of F X f x )[ F x ( ~=) 1) is satisfied. It is seen that property 4 of F X ( x )[F,(-m) = O] is also satisfied. For 0 5 a I 1 and b > 0 , F ( x ) is sketched in Fig. 2-14. From Fig. 2-14, we see that F(x) is a nondecreasing function and continuous on the right, and properties 2 and 5 of t7,(x) are satisfied. Hence, we conclude that F(x) given is a valid cdf if 0 5 a 5 1 and b > 0. Note that if a = 0, then the r.v. X is a discrete r.v.; if a = 1, then X is a continuous r.v.; and if 0 < a < 1, then X is a mixed r.v. 0 Fig. 2-14 DISCRETE RANDOM VARIABLES AND PMF'S 2.12. Suppose a discrete r.v. X has the following pmfs: PXW = 4 P X =~ $ px(3) = i (a) Find and sketch the cdf F,(x) of the r.v. X. (b) Find (i) P ( X _< I), (ii) P(l < X _< 3), (iii) P ( l I X I3). (a) By Eq. (2.18), we obtain which is sketched in Fig. 2-15. (b) (i) By Eq. (2.12), we see that P(X < I ) = F x ( l - ) = 0 (ii) By Eq. (2.10), P(l < X I 3 ) = Fx(3) - F x ( l ) = - 4 =2 (iii) By Eq. (2.64), P ( l I X I 3) = P ( X = 1) + Fx(3) - F x ( l ) = 3 + 4 - 3 = 3  RANDOM VARIABLES [CHAP 2 Fig. 2-15 2.13. (a) Verify that the function p(x) defined by x =o, 1, 2,... otherwise is a pmf of a discrete r.v. X. (b) Find (i) P(X = 2), (ii) P(X I 2), (iii) P(X 2 1). (a) It is clear that 0 5 p(x) < 1 and Thus, p(x) satisfies all properties of the pmf [Eqs. (2.15) to (2.17)] of a discrete r.v. X. (b) (i) By definition (2.14), P(X = 2) = p(2) = $($)2 = (ii) By Eq. (2.18), (iii) By Eq. (l.25), 2.14. Consider the experiment of tossing an honest coin repeatedly (Prob. 1.35). Let the r.v. X denote the number of tosses required until the first head appears. (a) Find and sketch the pmf p,(x) and the cdf F,(x) of X. (b) Find (i) P(l < X s 4), (ii) P(X > 4). (a) From the result of Prob. 1.35, the pmf of X is given by Then by Eq. (2.18), CHAP. 21 RANDOM VARIABLES These functions are sketched in Fig. 2-16. (b) (9 BY Eq. ( 2. m P(l < X 1 4 ) = Fx(4)- Fx(X) = -3= (ii) By Eq. (1.Z), P(X > 4 ) = 1 - P(X 5 4) = 1 - Fx(4) = 1 - = -& Fig. 2-16 2.15. Consider a sequence of Bernoulli trials with probability p of success. This sequence is observed until the first success occurs. Let the r.v. X denote the trial number on which this first success occurs. Then the pmf of X is given by because there must be x - 1 failures before the first success occurs on trial x. The r.v. X defined by Eq. (2.67) is called a geometric r.v. with parameter p. (a) Show that px(x) given by Eq. (2.67) satisfies Eq. (2.17). (b) Find the cdf F,(x) of X. (a) Recall that for a geometric series, the sum is given by Thus, (b) Using Eq. (2.68),we obtain Thus, P(X 5 k ) = 1 - P(X > k ) = 1 - ( 1 - and Fx(x)=P(X 1100). Since (X < 900) n {X > 1100) = (21, we have Since X is a normal r.v. with p = 1000 and a 2 = 2500 (a = 50), by Eq. (2.55) and Table A (Appendix A), Fx(900)= @(900 ~ ~ o o =o @( ) - 2) = 1 - @(2) F,(1100) = @ ( 1100 - 1000 5o ) = Thus, P(A) = 2[1 - @(2)]z 0.045 2.48. The radial miss distance [in meters (m)] of the landing point of a parachuting sky diver from the center of the target area is known to be a Rayleigh r.v. X with parameter a2 = 100. (a) Find the probability that the sky diver will land within a radius of 10 m from the center of the target area. (b) Find the radius r such that the probability that X > r is e - ( x 0.368). (a) Using Eq. (2.75) of Prob. 2.23, we obtain (b) Now P(X > r) = 1 - P(X < r) = 1 - F,(r) - 1 - (1 - e-r2/200) = e - r 2 / 2 0 0 = e - l from which we obtain r2 = 200 and r = $66 = 14.142 rn. CHAP. 21 RANDOM VARIABLES CONDITIONAL DISTRIBUTIONS 2.49. Let X be a Poisson r.v. with parameter 2. Find the conditional pmf of X given B = (X is even). From Eq. (2.40), the pdf of X is ;Ik px(k) = e - A - k = 0, 1,... k! Then the probability of event B is Let A = {X is odd). Then the probability of event A is Now Ak Ak (-A)k - - k a, =1 even E- f k=odd e-Ak!=eu- k =0 7- ,-A e - -A ,-21 Hence, adding Eqs. (2.101) and (2.102), we obtain Now, by Eq. (2.62), the pmf of X given B is If k is even, ( X = k) c B and ( X = k) n B = ( X = k). If k is odd, ( X = k) n B = fZI. Hence, P(X = k) 2e-9'' k even P(B) (1+eT2")k! P*(k I B) = k odd 2.50. Show that the conditional cdf and pdf of X given the event B = (a < X Ib) are as follows: 10 x 5: a  RANDOM VARIABLES [CHAP 2 Substituting B = (a < X 5 b) in Eq. (2.59), we have P((X I x ) n (a < X I b)} FX(x(a O), From Eq. (2.52), the pdf of X = N ( 0 ; a2)is Then by Eq. (2.105), Hence, CHAP. 21 RANDOM VARIABLES Let y = x2/(2a2).Then d y =x d x / a 2 , and we get Next, = 1 - f i , -" I " x2e-x2/(2a2) d x = =2. (2.108) Then by Eq. (2.31), we obtain 2.53. A r.v. X is said to be without memory, or memoryless, if P(X~x+tlX>t)=P(Xsx) x,t>O Show that if X is a nonnegative continuous r.v. which is memoryless, then X must be an expo- nential r.v. By Eq. (1.39),the memoryless condition (2.110) is equivalent to If X is a nonnegative continuous r.v., then Eq. (2.1 11) becomes or [by Eq. (2.2511, Fx(x + t ) - Fx(t) = CFXW - FX(0)ICl - FX(O1 Noting that Fx(0) = 0 and rearranging the above equation, we get Taking the limit as t -+ 0 , we obtain F W = F>(O)[l - Fx(x)l where FX(x)denotes the derivative of FX(x).Let RX(x)= 1 - FX(x) Then Eq. (2.112) becomes The solution to this differential equation is given by Rx(x) = keRx(OIx where k is an integration constant. Noting that k = Rx(0) = 1 and letting RgO) = -FXO) = -fdO) = -1, we obtain Rx(x) = e -lx  RANDOM VARIABLES [CHAP 2 and hence by Eq. (2.113), Thus, by Eq. (2.49),we conclude that X is an exponential r.v. with parameter 1 =fx(0) (>0). Note that the memoryless property Eq. (2.110) is also known as the Markov property (see Chap. 5), and it may be equivalently expressed as Let X be the lifetime (in hours) of a component. Then Eq. (2.114) states that the probability that the component will operate for at least x + t hours given that it has been operational for t hours is the same as the initial probability that it will operate for at least x hours. In other words, the component "forgets" how long it has been operating. Note that Eq. (2.115) is satisfied when X is an exponential rev., since P(X > x) = 1 - FAX)= e-" and e-A(x+t) = e - k i -At e. Supplementary Problems 2.54. Consider the experiment of tossing a coin. Heads appear about once out of every three tosses. If this experiment is repeated, what is the probability of the event that heads appear exactly twice during the first five tosses? Ans. 0.329 2.55. Consider the experiment of tossing a fair coin three times (Prob. 1.1). Let X be the r.v. that counts the number of heads in each sample point. Find the following probabilities: (a) P(X I1); (b) P(X > 1); and (c) P(0 < X < 3). 2.56. Consider the experiment of throwing two fair dice (Prob. 1.31). Let X be the r.v. indicating the sum of the numbers that appear. (a) What is the range of X? (b) Find (i) P(X = 3); (ii) P(X 5 4); and (iii) P(3 < X 1 7). Ans. (a) Rx = (2, 3,4,..., 12) (b) (i) & ;(ii) 4 ;(iii) 4 2.57. Let X denote the number of heads obtained in the flipping of a fair coin twice. (a) Find the pmf of X. (b) Compute the mean and the variance of X. 2.58. Consider the discrete r.v. X that has the pmf px(xk)= (JP xk = 1, 2, 3,... Let A = (c: X({) = 1, 3, 5, 7,...}. Find P(A). Ans. 3 CHAP. 21 RANDOM VARIABLES Consider the function given by (0 otherwise where k is a constant. Find the value of k such that p(x) can be the pmf of a discrete r.v. X. Ans. k = 6/n2 It is known that the floppy disks produced by company A will be defective with probability 0.01. The company sells the disks in packages of 10 and offers a guarantee of replacement that at most 1 of the 10 disks is defective. Find the probability that a package purchased will have to be replaced. Ans. 0.004 Given that X is a Poisson r.v. and px(0) = 0.0498, compute E(X) and P(X 2 3). Ans. E(X) = 3, P(X 2 3) = 0.5767 A digital transmission system has an error probability of per digit. Find the probability of three or more errors in lo6 digits by using the Poisson distribution approximation. Ans. 0.08 Show that the pmf px(x) of a Poisson r.v. X with parameter 1 satisfies the following recursion formula: Hint: Use Eq. (2.40). The continuous r.v. X has the pdf - x2) 0 2).  RANDOM VARIABLES [CHAP 2 Ans. (a) k = 1. (b) Mixed r.v. (c) (i) $; (ii) ; (iii) 0 2.67. It is known that the time (in hours) between consecutive traffic accidents can be described by the exponen- tial r.v. X with parameter 1= &. Find (i) P(X I60); (ii) P(X > 120);and (iii) P(10 < X I 100). Ans. (i) 0.632; (ii) 0.135; (iii) 0.658 2.68. Binary data are transmitted over a noisy communication channel in block of 16 binary digits. The probabil- ity that a received digit is in error as a result of channel noise is 0.01. Assume that the errors occurring in various digit positions within a block are independent. (a) Find the mean and the variance of the number of errors per block. (b) Find the probability that the number of errors per block is greater than or equal to 4. Ans. (a) E(X) = 0.16, Var(X) = 0.158 (b) 0.165 x 2.69. Let the continuous r.v. X denote the weight (in pounds) of a package. The range of weight of packages is between 45 and 60 pounds. (a) Determine the probability that a package weighs more than 50 pounds. (b) Find the mean and the variance of the weight of packages. Hint: Assume that X is uniformly distributed over (45, 60). Ans. (a) 4; (b) E(X) = 52.5, Var(X) = 18.75 2.70. In the manufacturing of computer memory chips, company A produces one defective chip for every nine good chips. Let X be time to failure (in months) of chips. It is known that X is an exponential r.v. with parameter 1= f for a defective chip and A = with a good chip. Find the probability that a chip pur- chased randomly will fail before (a) six months of use; and (b) one year of use. Ans. (a) 0.501; (b) 0.729 2.71. The median of a continuous r.v. X is the value of x = x, such that P(X 2 x,) = P(X Ix,). The mode of X is the value of x = x, at which the pdf of X achieves its maximum value. (a) Find the median and mode of an exponential r.v. X with parameter 1. (b) Find the median and mode of a normal r.v. X = N(p, a2). Ans. (a) x, = (In 2)/1 = 0.69311, x, = 0 (b) x, = x, = p 2.72. Let the r.v. X denote the number of defective components in a random sample of n components, chosen without replacement from a total of N components, r of which are defective. The r.v. X is known as the hypergeometric r.v. with parameters (N, r, n). (a) Find the prnf of X. (b) Find the mean and variance of X. Hint: To find E(X), note that ( ) ( -) =x x-1 and )(: = x=O (L)(~ - n-x r, To find Var(X), first find E[X(X - I)]. CHAP. 21 RANDOM VARIABLES 2.73. A lot consisting of 100 fuses is inspected by the following procedure: Five fuses are selected randomly, and if all five "blow" at the specified amperage, the lot is accepted. Suppose that the lot contains 10 defective fuses. Find the probability of accepting the lot. Hint: Let X be a r.v. equal to the number of defective fuses in the sample of 5 and use the result of Prob. 2.72. Ans. 0.584 2.74. Consider the experiment of observing a sequence of Bernoulli trials until exactly r successes occur. Let the r.v. X denote the number of trials needed to observe the rth success. The r-v. X is known as the negative binomial r.v. with parameter p, where p is the probability of a success at each trial. (a) Find the pmf of X. (b) Find the mean and variance of X. Hint: To find E(X), use Maclaurin's series expansions of the negative binomial h(q) = (1 - 9)-' and its derivatives h'(q) and hW(q), and note that To find Var(X), first find E[(X - r)(X - r - 1)] using hU(q). Ans. (a) px(x) = x = r, r + 1,... r(l - P) (b) EIX) = r(i), Var(X) = - p2 2.75. Suppose the probability that a bit transmitted through a digital communication channel and received in error is 0.1. Assuming that the transmissions are independent events, find the probability that the third error occurs at the 10th bit. Ans. 0.017 2.76. A r.v. X is called a Laplace r.v. if its pdf is given by fx(x)=ke-'Ix1 1>0, -co 1) = 1 - P(Y I 1) = 1 - Fd1) = e-@ CHAP. 31 MULTIPLE RANDOM VARIA.BLES By De Morgan's law (1.15), we have - -- (X > x) n (Y > y) = (X > x) u (Y > y) = (X I x) u ( Y 5 y) Then by Eq. (1D ) , P[(X > X) n ( Y > y)] = P(X I x) + P(Y 5; y) - P(X s x, Y s y) = Fx(4 + FAY)- F,,(x, Y) = (1 - e-"") + (1 - e-fly)- (1 - e-ax)(l - e-fly) = 1 - e-aXe-By Finally, by Eq. (1.25), we obtain P(X > x, Y > y) = 1 - P[(X > x) n ( Y > y)] = e-""e-DY 3.9. The joint cdf of a bivariate r.v. (X, Y) is given by Find the marginal cdf's of X and Y. Find the conditions on p,, p, ,and p, for which X and Y are independent. By Eq. (3.13), the marginal cdf of X is given by By Eq. (3.1 4), the marginal cdf of Y is given by For X and Y to be independent, by Eq. (3.4), we must have FXy(x,y) = FX(x)Fr(y).Thus, for 0 x < a, 0 I y < b, we must have p , = p, p, for X and Y to be independent. DISCRETE BIVARIATE RANDOM VARIABLESJOINT PROBABILITY MASS FUNCTIONS 3.10. Verify Eq. (3.22). If X and Y are independent, then by Eq. (1.46), pxdxi, yj) = P(X = Xi ,Y = yi) = P(X = xi)P(Y = yj) = P ~ ( X ~ ) P ~ ( Y ~ ) 3.11. Two fair dice are thrown. Consider a bivariate r.v. (X, Y). Let X = 0 or 1 according to whether the first die shows an even number or an odd number of dots. Similarly, let Y = 0 or 1 according to the second die. (a) Find the range Rxy of (X, Y). (b) Find the joint pmf's of (X, Y).  MULTIPLE RANDOM VARIABLES [CHAP 3 (b) It is clear that X and Y are independent and P(X = 0) = P(X = 1) = 4 = q P(Y=O)=P(Y= I ) = $ = $ Thus pxy(i,j)=P(X=i, Y = j ) = P ( X = i ) P ( Y = j ) = $ i,j=O, 1 3.12. Consider the binary communication channel shown in Fig. 3-4 (Prob. 1.52). Let (X, Y) be a bivariate r.v., where X is the input to the channel and Y is the output of the channel. Let P(X = 0) = 0.5,P(Y = 11 X = 0)= 0.1,and P(Y = 0 1 X = 1)= 0.2. Find the joint pmf's of (X, Y). Find the marginal pmf's of X and Y. Are X and Y independent? From the results of Prob. 1.52,we found that Then by Eq. (1.41),we obtain Hence, the joint pmf's of (X, Y) are By Eq. (3.20),the marginal pmf's of X are px(0) = pxu(O,yj) = 0.45 + 0.05 = 0.5 YJ px(l) = C pxr(l, yj) = 0.1 + 0.4 = 0.5 YJ By Eq. (3.21), the marginal pmf's of Y are Fig. 3-4 Binary communication channel. CHAP. 31 MULTIPLE RANDOM VARIABLES 95 (c) Now p,(0)py(O) = OS(0.55) = 0.275 # pxy(O, 0 ) = 0.45 Hence X and Y are not independent. Consider an experiment of drawing randomly three balls from an urn containing two red, three white, and four blue balls. Let (X, Y) be a bivariate r.v. where X and Y denote, respectively, the number of red and white balls chosen. Find the range of (X, Y). Find the joint pmf's of (X, Y). Find the marginaI pmf's of X and Y. Are X and Y independent? The range of ( X , Y) is given by Rxr = {(O, 019 (0, I), (0, 2), (0, 3)9 (19 01, ( 1 , 1 1 9 (1, 3 , (2, O), (2, 1 ) ) The joint pmf's of ( X , Y ) pxy(i,j)= P ( X = i , Y = j ) i=O, 1,2 j=O, 1,2,3 are given as follows: which are expressed in tabular form as in Table 3.1. The marginal pmf's of X are obtained from Table 3.1 by computing the row sums, and the marginal pmf's of Y are obtained by computing the column sums. Thus Table 3.1 p&i, j )  MULTIPLE RANDOM VARIABLES [CHAP 3 (d) Since PXY(0, 0) = 6 Z PX(O)PY(O)= iE (%I X and Y are not independent. 3.14. The joint pmf of a bivariate r.v. (X, Y) is given by k(2xi + yj) xi = 1 , 2 ; yj = 1, 2 otherwise where k is a constant. (a) Find the value of k. (b) Find the marginal pmf's of X and Y. (c) Are X and Y independent? Thus, k = &. (b) By Eq. (3.20), the marginal pmf's of X are By Eq. (3.21), the marginal pmf's of Y are (c) Now pAxi)py(yj)# pxY(xi,y,); hence X and Y are not independent. 3.15. The joint pmf of a bivariate r.v. (X, Y) is given by kxi2yj xi=l,2;yj=l,2,3 otherwise where k is a constant. Find the value of k. Find the marginal pmf's of X and Y. Are X and Y independent? By Eq. (3.17), Thus, k = &. By Eq. (3.20), the marginal pmf's of X are CHAP. 31 MULTIPLE RANDOM VARIABLES By Eq. (3.21), the marginal pmf's of Y are (c) Now Px(xi)P~(Yj) = &ixi2yj= PxAxi Yj) Hence X and Y are independent. 3.16. Consider an experiment of tossing two coins three times. Coin A is fair, but coin B is not fair, with P(H) = and P(T) = $. Consider a bivariate r.v. (X, Y), where X denotes the number of heads resulting from coin A and Y denotes the number of heads resulting from coin B. (a) Find the range of (X, Y). (b) Find the joint pmf's of (X, Y). (c) Find P(X = Y), P(X > Y), and P(X + Y 14). (a) The range of (X, Y) is given by Rxy = {(i,j): i, j = 0, 1, 2, 3) (b) It is clear that the r.v.'s X and Y are independent, and they are both binomial r.v.'s with parameters (n, p) = (3, 4)and (n, p) = (3, $), respectively. Thus, by Eq. (2.36),we have Since X and Y are independent, the joint pmf's of (X, Y) are which are tabulated in Table 3.2. (c) From Table 3.2, we have Table 3.2 pm(i, J)  MULTIPLE RANDOM VARIABLES [CHAP 3 Thus, ~ ( ~ + y < 4 ) = 1 - ~ ( ~ + ~ > 4 ) = 1 - & = ~ CONTINUOUS BIVARIATE RANDOM VARIABLES-PROBABILITY DENSITY FUNCTIONS 3.17. The joint pdf of a bivariate r.v. (X, Y) is given by ,Ax. , = g(. + o 0, where Xn is the sample mean defined by Eq. (4.57). Equation (4.59) is known as the strong law of large numbers. CHAP. 4) FUNCTIONS O F RANDOM VARIABLES, EXPECTATION, LIMIT THEOREMS 129 Notice the important difference between Eqs. (4.58) 6nd (,4.59).Equation (4.58) tells us how a sequence of probabilities converges, and Eq. (4.59) tells us how the sequence of r.v,'s behaves in the limit. The strong law of large numbers tells us that the sequence (xn) is converging to the contant p. C. The Central Limit Theorem: The central limit theorem is one of the most remarkable results in probability theory. There are many versions of this theorem. In its simplest form, the central limit theorem is stated as follows: Let XI,... , X, be a sequence of independent, identically distributed r.v.'s each with mean p and variance a'. Let where 1,is defined by Eq. (4.57).Then the distribution of 2, tends to the standard normal as n -, oo; that is, lim 2, = N ( 0 ; 1 ) n-t m lim F,,(Z) = lim P(Z, Iz ) = @(z) fl-' aJ fl* W where @(z) is the cdf of a standard normal r.v. [Eq. (2.54)]. Thus, the central limit theorem tells us that for large n, the distribution of the sum Sn = XI +.. + X n is approximately normal regardless of the form of the distribution of the individual X,'s. Notice how much stronger this theorem is than the laws of large numbers. In practice, whenever an observed r.v. is known to be a sum of a large number of r.v.3, then the central limit theorem gives us some justification for assuming that this sum is normally distributed. Solved Problems FUNCTIONS OF ONE RANDOM VARIABLE 4.1. If X is N ( p ; a2),then show that Z = (X - p)/a is a standard normal r.v.; that is, N ( 0 ; 1). The cdf of Z is By the change of variable y = (x - p)/a (that is, x = ay + p), we obtain  FUNCTIONS O F RANDOM VARIABLES, EXPECTATION, LIMIT THEOREMS [CHAP. 4 and which indicates that Z = N(0; 1). 4.2. Verify Eq. (4.6). Assume that y = g(x) is a continuous monotonically increasing function [Fig. 4-l(a)]. Since y = g(x) is monotonically increasing, it has an inverse that we denote by x = g-'(y) = h(y). Then Applying the chain rule of differentiation to this expression yields which can be written as If y = g(x) is monotonically decreasing [Fig. 4.l(b)], then Fdy) = P( Y Iy ) = P [ X > h(y)] = 1 - Fx[h(y)] d dx Thus, =)- FAY)= -fx(x) - ~Y(Y x = h(y) (4.66) dy dy In Eq. (4.66), since y = g(x) is monotonically decreasing, dy/dx (and dxldy) is negative. Combining Eqs. (4.64) and (4.66), we obtain which is valid for any continuous monotonic (increasing or decreasing) function y = g(x). 4.3. Let X be a r.v. with cdf F,(x) and pdf f,(x). Let Y =. ax + b, where a and b are real constants and a # 0. (a) Find the cdf of Y in terms of F,(x). Fig. 4-1 CHAP. 4 ) FUNCTIONS O F RANDOM VARIABLES, EXPECTATION, LIMIT THEOREMS 131 (a) (b) Fig. 4-2 (b) Find the pdf of Y in terms of fx(x). (a) If a > 0, then [Fig. 4-2(a)] f &) = P(Y 6 y) = P(aX + b i y) = P(X 5 q) = Fx($) If a < 0, then [Fig. 4-2(b)] F&) = P(Y S y) = P(aX + b 6 y) = P(aX Iy - b) (since a < 0, note the change = P ( X z e ) in the inequality sign) -1-P(X