Real Random Variable PDF

Chapter 4. Real random variable. 1) Real Random Variable (RV) Definition 1: Let (, A , ) be a probability space. A real random variable (noted X or Y , Z ,….) is any function defined on  that takes values in IR such that, for any interval I in IR , the preimage set X 1 ( I )    ; X ( )...

Chapter 4. Real random variable. 1) Real Random Variable (RV) Definition 1: Let (, A , ) be a probability space. A real random variable (noted X or Y , Z ,….) is any function defined on  that takes values in IR such that, for any interval I in IR , the preimage set X 1 ( I )    ; X ( )  I  is a random event, this means X 1 ( I )  A (the σ-algebra A). If it’s the case, we will note the event X 1 ( I ) by ( X  I ). The set of values taken by X , known as the image of a function X , and commonly denoted by Im( X ) or X ( ) , we formulate it: X ()  X (w);    It is called the set of all observations associated with X , because the variable X indeed is the rule that assigns a numerical value to each outcome of an experiment. Once we define X , X ( ) will be essential for our future answers to any questions and calculations that may arise. Remark 1: If the σ-algebra A is the power set of  , we notice that any function X (such as DX   ) from  to IR will be a real random variable. (This is often the case in the discrete case.) Definition 2: We define the cumulative distribution function CDF or as also called, the distribution function of a random variable X defined on a probability space ( , A , ) by: FX ( x)  ( X  x) , for any x  IR. We express it F : IR 0,1 x  F ( x)  ( X  x) Where ( X  x)    , tel que X ( )  x  X 1 (]  , x]) 1.1) Discrete Real Random Variable (DRV) 1.1.1) Definition 3: A real random variable X is said to be discrete if the set X ( ) is countable. Remark 2: X ( ) can be finite, X ()  x1 , x2 ,...xn  or countably infinite X ()  x1 ,...xn ,.... Example 1: Rolling a die twice. X :  IR   (a, b)  X ()  a  b Find   ? and X ( ) = ? Example 2: Tossing a fair coin repeatedly until a heads appears. Let X be the random variable indicating the number of throws required before obtaining a tails. What is X ( ) ? 1  Chapter 4. Real random variable. Notation: For a discrete random variable X , we use the following notation: X  x     : X ( )  x  X 1 x. 1.1.2) Probability function: Definition 4: Let X be a discrete random variable. The probability function or distribution of probability of X is the function X : X () 0 ;1 defined by x  X (),   X ( x)  ( X  x)   X 1 x. (The function X satisfy that its domain DPX  X () ). Property 1: Let X be a discrete random variable with probability function X , then:  xX (  ) X ( x)  1.  Remark 3: Si X ()  x1 , x2..., xn ,... « a countably infinite set » , then  X ( xn )  1 (to prove ?) n 1 Example 3: Tossing a coin until a heads appears. Find X ( ) , X ( x), x  X () and verify that  xX (  ) X ( x)  1. Proposition 1: Any positive function f defined on a countable subset A , with real values and verifying  f ( x)  1 , can be considered as the probability function of a discrete xA ) random variable X , where the set of values taken is A. 1.1.3) Cumulative Distribution Function of a discrete random variable: If the values of X (a d.r.v) are ordered, X ()  x1  x2 ...  xn ..., n then for all x , such as xn  x  xn1 , FX ( x )   X ( x i ). i 1 2  Chapter 4. Real random variable. Property 2: let X is a discrete random variable with CDF FX then we get these following properties: i. FX is a step function. (Staircase or floor function because it is defined as a piecewise constant function) ii. FX is a non-decreasing function. iii. Specifically, a, b  IR, if a  b, then (a  X  b)  FX (b)  FX (a). iv. lim FX (a)  0 and lim FX (a )  1. a  a  v. FX is right-continuous at every point a ; a  IR, ( X  a)  FX (a)  lim FX (a   ).  0 Remark 4: The cumulative distribution function is determined based on the probability function of X , and conversely, the probability function can be determined entirely by knowledge of the CDF. Example 4: Let X be a random variable whose probability function is given as: X (1)  14 , X (2)  12 , X (3)  16 , and X (4)  1 8. Determine FX and plot it. 1.2) Continuous real random variable Definition 5: A random variable X is said to be continuous if there exists a numerical function f defined on IR such that: i. x  IR, f ( x)  0. ii. f is continuous on IR , except at a finite or countable number of points where it has finite left and right limits. x x  iii. x  IR , F ( x)   f (t )dt  and x  IR , lim F ( x)  lim x  x   f (t )dt   f (t )dt  1   f or f X is called the probability density function (PDF) of the continuous variable X. 3  Chapter 4. Real random variable. Remark 5: 1. Variables defined in this way are "absolutely continuous." 2. Contrary to discrete variables, a continuous random variable can take values in any interval of IR , a union of such intervals or IR in its entirely. 3. We have: - x  IR , ( X  x)  0 , and hence ( X  x)  ( X  x) b - a, b  IR, (a  b), (a  X  b)  F (b)  F (a)   f (t )dt a 4. A continuous random variable X can be defined by giving a function F : IR IR that satisfies these properties. a) F is a continuous and increasing on IR. b) lim F  0 and lim F  1   c) F is differentiable on IR (except perhaps at a finite or countable set I of points where it is differentiable from the left and the right of each point of I ) and F ' continuous on IR  I. We then have "almost everywhere, f  F '.   and F  FX is the CDF of X . dF  f  fX  is the PDF of X   dx  e  x x0 Example 5: Let f ( x)  e  x 1IR  ( x)   , show that f is indeed a probability density function of a 0 sin on continuous random variable and determine its cumulative distribution function F. [We say that X follows the exponential distribution with parameter   1 (Ch6: Common probability Distributions)] Remark 6: To be clearer, in discrete case, we often call the probability function given by X , the probability mass function, and in continuous case, we say the probability density function, which is given by f X. 4  Chapter 4. Real random variable. 2) Mode and percentiles: 2.1. Mode Definition 6: Let X be a discrete random variable. The mode of X , denoted M O ( X ) , is any value from IR that maximizes X (x). We have: x  X (), X ( x)  X ( M O ( X )) For the case of a continuous random variable X (x) is replaced by f X (x) (the probability density function). Example 6: (Continuation of Example 1), we obtain M O ( X )  7. Example 7: (Continuation of Example 2), we obtain M O ( X )  1. 2.2. Percentiles: Definition 7: Let X be a random variable with cumulative distribution function FX and   0;1.   A percentile of order  , or  - percentile, is any real number denoted x such that: F ( x )    ( X  x )    The median is the percentile of order 1 , the 0.5-quantile). 2  The first quartile and third quartile are the percentiles of order 1 4 and 3 4 respectively.  Similarly, the ith decile is the percentile of order i 10. 3) Mathematical Expectation Definition 8: Let X be a real random variable. The mathematical expectation of X , denoted E ( X ) , is the real number, if it exists, defined by:    x.X ( x) discrete case   x X (  ) E( X )       x. f X ( x)dx continuous case    Remark 7: If X is a discrete random variable and has an expectation, then since  xX (  ) X ( x)  1 , we can write:  x. X ( x)   X ( x )      x. x X (  ) E( X )  .  X ( x ) x X (  )   X ( x )  xX (  )  xX (  )  5  Chapter 4. Real random variable. The expectation of X looks as the barycenter of the different points x weighted by their probability X (x) , it’s the mean of all the possible values taken by X weighted by their probabilities. Remark 8: The series or integral defining the expectation may diverge. In such cases, we say that X does not have an expectation, as illustrated in the following examples: a Example 8: For the discrete random variable defined by : X ()  IN * and X (n)  ( X  n)  , n2 (a  6 2 ) , the series representing its expectation diverges. For a continuous case, the random variable with a probability law following the Cauchy distribution (common probability distribution) does not have an expectation.  Remark 9: If X ()  x1 , x2..., xn ,..., then E ( X )   xn.X ( xn ). n 1 12 Example 9: (Continuation of Example 1), we find E ( X )   x x2 X ( x)  252 36 7.  Example 10: (Continuation of Example 5), we find E ( X )   x. f ( x)dx   xe x dx  1  11 . IR 0 Theorem 1: Let X be a random variable and h a real-valued function defined on X ( ). The expectation of h ( X ) , if it exists, is given by:    h( x).X ( x) discrete case   xX (  ) E (h( X ))       h( x). f X ( x)dx continuouse case    Remark 10: This theorem is important because it allows us to calculate the expectation of the random variable h ( X ) without explicitly determining its probability function, but only using the probability function of X. Remark 11: For example, considering h( x)  x 2 :  In Example 1, we find E ( X 2 )  x xX (  ) 2.X ( x)  1974 36 ,. 6  Chapter 4. Real random variable.   In Example 5, we find, E ( X 2 )   x. f X ( x)dx   x.e dx 2 2 2 x  . 2 12 IR 0 Corollary 1: (of the theorem1) Let X be a random variable, and let a and b be real numbers. Then: E (a. X  b)  a.E ( X )  b. Definition 9: A random variable X is said to be centered if E ( X ). Property 3: For any random variable X, X  E ( X ) is centered random variable. (Exercise) 4) Moments According to Theorem 1 and considering that h( X )  X k , k  IN * is a random variable, we define what are called moments: Definition 10: The moment of order k of a random variable X , is the expectation if it exists of X k , given by:    x.X ( x) k discrete case   xX (  ) M k ( X )  E( X k )     k   x. f X ( x)dx continuous case    The centered moment of order k of X is the moment of order k of the centered random variable, i.e., the expectation, if it exists of ( X  E ( X )) k , given by:    ( x  E ( X )).X ( x) k discrete case   x X (  )  k ( X )  E[( X  E ( X )) k ]       ( x  E ( X )). f X ( x)dx continuous case k    Remark 12: 1. If k  1 , we observe that M 1 ( X )  E ( X 1 )  E ( X ) , which corresponds to the expectation of X. 2. If k  1 , we have 1 ( X )  E( X  E( X ))  0 , (refer to the definition of the centered variable). 3. If k  2 , we have  2 ( X )  E ( X  E ( X )) 2 ) , which is called the variance of the r.v. X. Definition 10 (continued from Remark 3): The variance of the random variable X , denoted by V ( X ) , if it exists, will be given by: 7  Chapter 4. Real random variable.    ( x  E ( X )).X ( x) 2 discrete case   x X (  ) V ( X ) =  2 ( X )  E ( X  E ( X )   2     ( x  E ( X )). f X ( x)dx continuous case 2    Remark 13: If the infinite series or integral above diverges, we say that X does not have a variance. Definition 11: The standard deviation of a random variable X , denoted by  ( X ) , is the square root of its variance:  (X )  V (X ) Definition 12: A random variable X is said to be reduced if V ( X )  1. Theorem 2 (Koenig's Theorem): Let X be a random variable possessing a variance. Then: V ( X )  E ( X  E ( X )   E ( X 2 )  ( E ( X )) 2. 2 Examples: Example 11: (continuation of Example 1):We have: V ( X )  E ( X 2 )  ( E ( X )) 2  54.833  7 2 . Example 12: (continuation of Example 5): We have: V ( X )  E ( X 2 )  ( E ( X )) 2  2  12  1( 112 ). Corollary 2 (from Theorem 1): Let X be a random variable, and let a and b e two real numbers, then: V (aX  b)  a 2V ( X ). X  E( X ) Property 4: For any random variable X , the random variable Y  is both centered and reduced.  (X ) In this case, Y is called standardized random variable. (Exercise) Remark14: a standardized variable has mean equal to zero and variance equal to 1. Such a variable is typically denoted Z (instead of X ) and may be referred to as “ z -score). 5) Moment-Generating Functions: In probability theory, we use three types of functions to generate the moments of a random variable. As their name suggests, they allow the determination of the moments (if they exist) of the variable through mathematical formulas, avoiding the direct application of definitions, which can sometimes lead to complicated calculations involving series or integrals. These three functions are defined as follows: for t  IR ; t  E (e t. X ) , t  E (t X ) and t  E (e i.t. X ). The third function is known as the characteristic function, which is more suitable for theoretical purposes. The second is the simplest but generates factorial moments rather than moments directly. For example: 8  Chapter 4. Real random variable. The factorial moment of order k , k  IN * of a random variable X is defined by: E ( X ( X  1)( X  2)....( X  k  1)) If we want to calculate E ( X 2 ) , by expanding E ( X ( X  1))  E ( X 2 )  E ( X ) (using the linearity of expectation), we derive that: E ( X 2 )  E ( X ( X  1))  E ( X ). E ( X ) Is already calculated, then the factorial moment of order two E ( X ( X  1)) can thus be determined from the formula related to the “second” generating function. Similarly, if we want to calculate E ( X 3 ) , by expanding E ( X ( X  1)( X  2))  E ( X 3 )  3E ( X 2 )  2 E ( X ) , we derive: E ( X 3 )  E ( X ( X  1)( X  2))  3E ( X 2 )  2 E ( X ). Where E ( X ) and E ( X 2 ) were already calculated, and the factorial moment of order three E ( X ( X  1)( X  2)) is determined in the same way (from a formula related to the “second” generating function. By extension, the moments of order k , k  4 are determined in the same manner. Our Focus: For this course, we focus on the first function, as it is more aligned with the mathematical knowledge acquired by the second year and additionally it directly generates moments without requiring factorial moments. Definition 13: For any real number t , the moment-generating function G X of a random variable X is defined as:    e.X ( x) cas discrét t.x   xX (  ) G X (t )  E (e t.X )     t. x   e. f X ( x)dx cas continu    Remark 15: The calculation of this function is given by Theorem 1, by setting h( X )  e t. X. Property 5: The moment of order k of X can be calculated by differentiating k times G X , then evaluating the derivative at t  0. d d For example: G X' (t )  E (e t. X )  E ( (e t. X )  E ( Xe t. X ) , dt dt Here, it is assumed that the interchange of differentiation and expectation (summation or integration) is valid. By evaluating at t  0 , we find: G X' (0)  E ( Xe 0. X )  E ( X ). d ' d d Similarly: G X'' (t )  G X (t )  E ( Xe t. X )  E ( ( Xe t. X )  E ( X 2 e t. X ) , dt dt dt Hence: G X'' (0)  E ( X 2 ). By induction, the expression for the k -th derivative of G X and the moment of order k of X are: G X( k ) (t )  E ( X k.e t. X ) k  1    E ( X k )  G ( k ) (0) k 1  X 9  Chapter 4. Real random variable. Examples: Below are examples of the calculation of the moment-generating function, the expectation, and the variance of a real- valued random variable following a common distribution. (Chapter 6 of our course is dedicated to common distributions). Example 13: (Bernoulli's distribution) We say that X follows the Bernoulli distribution with parameter p  0;1 , denoted by X  ( p) :  X (0)  1  p If X ()  0,1 with  , p ]0,1[ , then,  X (1)  p 1 G X (t )  E (e t. X )   e t.k X (k )  e 0.t  (1  p)  e1.t  p  [(1  p)  pe t ] , k 0 G (t )  pe hence E ( X )  G X' (0)  p , ' X t G X'' (t )  pe t hence E ( X 2 )  G X'' (0)  p , V ( X )  E ( X 2 )  ( E ( X )) 2  p  p 2  p(1  p)  E( X )  p Conclusion: to be remembred that, X  ( p) , V ( X )  p(1  p) Example 14: (Binomial distribution) We say that X follows the Binomial distribution with parameters n  IN * and p ]0,1[ , denoted by X  (n, p) , If X ()  0,1,2,..., nand X (k )  C nk p k (1  p ) n  k , then: n n G X (t )  E (e t. X )   e t.k X (k )   e t.k C nk p k (1  p) n  k   C nk ( pe t ) k (1  p) n  k k 0 k 0 k 0  ((1  p)  pe ) t n G X' (t )  n(1  p  pe t ) n1 pe t d’où E ( X )  G X' (0)  np G X'' (t )  n(n  1)(1  p  pe t ) n2 (( pe t ) 2  n(1  p  pe t ) n1 pe t hence, E ( X 2 )  G X'' (0)  n(n  1) p 2  np , and finally V ( X )  E ( X 2 )  ( E ( X )) 2  np(1  p)  E ( X )  np Conclusion: X  (n, p) , V ( X )  np(1  p) G X (t )  (GY (t )) n  Remark 16 : notice that, if X  (n, p) and Y  ( p) then  E ( X )  nE (Y )  V ( X )  nV (Y )  (It will seen and proved in chapters: 5 and 6) Example 15: (Poisson distribution) We say that X follows the Poisson distribution with parameter   IR* , denoted by X  Poisson ( ) , e  If X ()  IN and X (k )  k , then: k! 10  Chapter 4. Real random variable.  G X (t )  E (e t. X )   e t.k X (k )    e t.k e    k  e     e t k  e  e e  exp(  (e t  1) t k 0 k O k! k 0 k!  e  ( e 1) t G X' (t )  e t.e  (e 1) d’où E ( X )  G X' (0)   t GX'' (t )  (e t ) 2 e  (e 1)  e t e  (e 1) d’où E ( X 2 )  G X'' (0)  2   t t V ( X )  E ( X 2 )  ( E ( X )) 2  . E ( X )   Conclusion: X  Poisson ( ) ,  V ( X )   Example 16: (Exponential distribution) We say that X follows the Exponential distribution with parameter   IR* , denoted by X  xp( ) , If X ()  IR  and f X ( x)  e . x 1IR  ( x) , then:    G X (t )  E (e t. X )   e t. x f X ( x)dx   e t. x e x dx    e (  t ) x dx  pour t   IR 0 0  t Note that the generating function for the exponential distribution is defined in ]  , [ , not on the entire IR.  1 G X' (t )  , hence E ( X )  G X' (0)  , (  t ) 2  2 2 1 G X'' (t )  , hence E ( X 2 )  G X'' (0)  2 and V ( X )  E ( X 2 )  ( E ( X )) 2  2 (  t ) 3    1  E( X )   X  xp( ) ,  Conclusion: 1 V ( X )  2   Property 6: An important property of generating functions is that they uniquely determine the probability distribution of the corresponding random variables. If G X and its derivatives exist and are finite in a neighborhood of t  0 , then the probability distribution of X is fully determined by G X. Example 17: If G X (t )  ( 1 2)10 (e t  1)10 , we are sure that X  (10, 1 2 ). (See the example14) 11  Chapter 4. Real random variable. 6) Probability distribution of a Function of a Real Random Variable: Let X be a real random variable with a probability distribution PX (or a probability density function f X if it is continuous) and h a function defined on the domain of X ( ) with real values. We aim to determine PY , the probability distribution of the random variable Y  h( X ) , (or f Y ) based on that of X. We have: y  IR , FY ( y)  (Y  y)  (h( X )  y)  h( X )   , y We denote, D  x  X () : h( x)   , y   h 1 (]  , y]) , We have then to determine D , where D is the preimage of the interval ]  , y ] under the function h. Remark 17: To determine the cumulative distribution function (CDF) FY of Y , knowing that of X , we need to determine h 1 (]  , y] , which is the preimage of the interval ]  , y ] , (note that h may not be bijective). We will have:  In the continuous case: y  IR, FY ( y)  ( X  D)   f X ( x)dx. D  In the discrete case, we arrive at: y  Y (), Y ( y )   X ( x) , xA where A  x  X () tel que h( x)  y. 1  Bijective Case: If h is bijective (( h exists)), mathematically h 1 (]  , y] is the inverse of the interval ]  , y ] under h. In this case, if X is continuous and h is bijective (only in this case), a theorem provides the density f Y directly in terms of f X. Below are two examples (discrete and continuous cases) to show how to determine D , followed by the theorem for the case where X is continuous and h bijective. Example 18: Let X be a random variable that takes the values −1, 0, and 1 with respective probabilities : X (1)  0.2 , X (0)  0.5 et X (1)  0.3. Let Y  h( X )  X 2 , and we seek the probability mass function Y. We have: Y ()  0,1, Y (0)  (Y  0)  ( X  0)  ( X  0)  0.5 (It was possible to determine D  h 1 0  0 ) Y (1)  (Y  1)  ( X   1,1)  X (1)  X (1)  0.5 (It was possible to determine D  h 1 1   1,1) 1 x Example 19: Let f X ( x)  e , x  IR and let Y  h( X )  X 2. Determine f Y. 2  Y ()  IR , we have then:   IR , 12  Chapter 4. Real random variable. y  0, FY ( y)  0 , y  0, FY ( y)  (Y  y)  ( X 2  y)  ( X  y )  FX ( y )  FX ( y ) (it was here possible to determine D  h 1 (]  , y]  ( X  y )  ( y  X  y) Since f Y  FY' , we differentiate to find f Y , yielding: y  0, f Y ( y)  0 y  0, f Y ( y )  1 f X ( y )  f X ( y )   1 e y 2 y 2 y Theorem 3: Let X be a continuous random variable defined on a probability space (, , ) , with density f X , and let h be a numerical function such that: 1. h is defined and continuously differentiable on an interval I containing the support X ( ) of X. 2. x  I , dh dx ( x)  h , ( x)  0. 1 Then h is a bijection from I to h(I ) and admits an inverse function denoted h.  f X h 1 ( y )  y  h X ()   / 1   si  h h ( y ) The density of Y  h( X ) is given by: fY ( y )    0 si y  h X ()    Special Case: This case will be useful in the study of the normal distribution (Chapter 6: Common Distributions). Let X be a continuous random variable with density f X , and let Y  h( X ) such that x  IR, h( x)  ax  b où a  IR * et b  IR applying Theorem 3,to determine f Y in terms of f X , gives: 1  y b y  IR, f Y ( y)  fX   a  a  Remark 18: It is often not necessary to know the probability law of Y  h( X ) YYY to determine E (Y ) or V (Y ) (see Theorem 1).  Approximation Formulas for Expectation and Variance of a Function of a Random Variable: As previously noted, calculating E ( h( X )) and V ( h( X )) often involves challenging integration (continuous case) or summation (discrete case). To address this, two approximation formulas can be useful for certain asymptotic results (Chapter: Convergence). 13  Chapter 4. Real random variable. Assume h is twice differentiable near   E ( X ). From the Taylor expansion of h(x) near  :  h( x)  h( )  ( x   )h ' ( )  ( x2 ) h '' ( )  o ( x   ) 2 2    Neglecting higher-order terms o ( x   ) 2 , we obtain:  E (h( X ))  h(  )  h ' (  ) E ( X   )  12 h '' (  ) E ( X   ) 2  1 '' Hence E (h( X ))  h(  )  h (  )V ( X ) 2 And since h( x)  E h( x)   ( x   )h ' (  )  1 2 ( x   ) 2   V ( X ) h '' (  ) We will obtain 2  V (h( X ))  E h( X )  E h( X )  E ( X   ) 2 h ' ( )     h ( ).V ( X ) 2 ' 2 We conclude   V (h( X ))  h ' ( ).V ( X ) 2 14