STAT 112 Introduction to Statistics and Probability II Session 1A PDF

STAT 112 INTRODUCTION TO STATISTICS AND PROBABILITY II SESSION 1A- Basic Concepts in Probability 1 Course Writer: Dr. Gabriel Kallah-Dagadu Department of Statistics and Actuarial Science Contact Information: [email protected]...

STAT 112 INTRODUCTION TO STATISTICS AND PROBABILITY II SESSION 1A- Basic Concepts in Probability 1 Course Writer: Dr. Gabriel Kallah-Dagadu Department of Statistics and Actuarial Science Contact Information: [email protected] STAT 112 July 28, 2021 1 / 38 Session Outline 1 Random Experiment Random Variable 2 Definition of Probability 3 Axioms and Laws of Probability Axioms of Probability Properties of Probability 4 Expectation of Random Variable STAT 112 July 28, 2021 2 / 38 Session Overview The purpose of this session is to equip students with the basic knowledge and skills in probability. In this session, students will be introduced to basic concepts in probability such as random events, random variables and expectation of random variables. STAT 112 July 28, 2021 3 / 38 Learning Outcomes At the end of the session, students will be able to: 1 Define and distinguish between random events and random variable. 2 State examples of random events and random variables. 3 Define the expectation of a random variable. 4 Compute the expectation of a random variable using a given probability function. STAT 112 July 28, 2021 4 / 38 Random Experiment An experiment is a process of observation. A random experiment is any process of observation or activity in which the results can not be predicted with certainty. Alternatively, an experiment or random experiment is any specified set of actions, where the results cannot be predicted with certainty. Each repetition of an experiment is called a trial. Typical examples of random experiments are: a. the roll of a die, b. the toss of a coin, c. selecting a ball at random from a box containing 3 balls marked “a”, “b”, “c”. STAT 112 July 28, 2021 5 / 38 Definition of Some Terminologies In describing an experiment, the following terms are employed; Outcomes, Events and Sample Space. An outcome is a result of an experiment. Example: When a fair coin is tossed once, the result is either a Head or Tail, which are known as outcome. An event is any collection of outcomes, and a simple event is an event with only one possible outcome. Example: In tossing a fair die once, the possible outcomes are {1, 2, 3, 4, 5, 6}. The collection of one or more of these possible outcomes is an event, i.e. {1, 2, 3} or {4}. The sample space, S of a given experiment is a set that contains all possible outcomes of the experiment. Example: In the example of tossing a fair die, the sample space is {1, 2, 3, 4, 5, 6}. STAT 112 July 28, 2021 6 / 38 Definition of Some Terminologies A sample point, s is defined as one possible outcome of a random experiment. In the example of tossing a fair coin, ”Head” or ”Tail” is the sample point. Note that a set with one sample point, is often referred as an elementary event. A set A is called a subset of B, denoted by A ⊂ B if every element of A is also an element of B. Any subset of the sample space is called an event. Example: If S = {1, 2, 3, 4, 5, 6} and set A = {2, 3}, set B = {1, 2, 3, 4}, then we say A ⊂ B. The sample space, S is the subset of itself, that is S ⊂ S. Since S is the set of all possible outcomes of the experiment, it is often referred to as certain event. STAT 112 July 28, 2021 7 / 38 Worked Examples Example 1 Find the sample space for an experiment of tossing a coin repeatedly and counting the number of tosses required until the first head appears. Solution : It is easy to realize that all possible outcomes for this experiment are the terms of the sequence 1, 2, 3,.... Thus the sample space is given by; S = {1, 2, 3,... }. STAT 112 July 28, 2021 8 / 38 Examples Cont’d. Example 2 Consider an experiment of drawing two cards at random from a bag containing four cards marked with the integers 1, 2, 3, 4. a Find the sample space S1 of the experiment if the first card drawn is not replaced. Let the first number indicates the number of the first card drawn. b Find the sample space S2 of the experiment if the first card drawn is replaced before the second one is drawn. STAT 112 July 28, 2021 9 / 38 Solution Solution (a) The sample space S1 contains 12 ordered pairs (i , j ), where i , j, 1 ≤ i ≤ 4 and 1 ≤ j ≤ 4; the first number indicates the first card drawn. Thus:  (1, 2) (1, 3) (1, 4)     (2, 1) (2, 3) (2, 4)  S1 =    (3, 1) (3, 2) (3, 4)   (4, 1) (4, 2) (4, 3)  Solution (b) The sample space S2 contains 16 ordered pairs (i , j ), where 1 ≤ i ≤ 4 and 1 ≤ j ≤ 4, the first number indicates the first card drawn. Thus:  (1, 1) (1, 2) (1, 3) (1, 4)     (2, 1) (2, 2) (2, 3) (2, 4)  S2 =    (3, 1) (3, 2) (3, 3) (3, 4)   (4, 1) (4, 2) (4, 3) (4, 4)  STAT 112 July 28, 2021 10 / 38 Examples Cont’d. Example 3 Find the sample space for the experiment of measuring (in hours) the life time of a transistor. Solution (3): Clearly all possible outcomes are all positive real numbers, i.e. the life time can take all values on a positive number line. Thus the sample space S is given by; S = {0 ≤ X < ∞}. Note: It is useful to distinguish between two types of sample spaces: Discrete and Continuous. A sample space is discrete if it contains a finite or countable infinite set of possible outcomes. In this case the elements in the sample space can be listed or the list does not terminate as in example 1. A sample space is continuous if it contains an interval (either finite or infinite) of real numbers as seen in Example 3 above. STAT 112 July 28, 2021 11 / 38 Random Variable: Definition A random variable, usually written X , is a variable whose possible values are numerical outcomes of a random phenomenon. Alternatively: A random variable is a variable whose value is unknown or a function that assigns values to each of an experiment’s outcomes. Random variables are often designated by letters (X , Y or Z )and can be classified as: discrete, which are variables that have specific values, or continuous, which are variables that can have any values within a continuous range. STAT 112 July 28, 2021 12 / 38 Examples of Random Variables 1 A typical example of a random variable is the outcome of a coin toss. If the random variable, Y , is the number of heads we obtained from tossing two coins, then Y could be 0, 1, or 2. This means that we could have no heads, one head, or both heads on a two-coin toss. 2 Examples of discrete random variables include the number of children in a family, the Friday night attendance at a cinema, the number of patients in a doctor’s surgery, the number of defective light bulbs in a box of ten. 3 Continuous random variables are usually measurements. Examples include height, weight, the amount of sugar in an orange, the time required to run a mile. STAT 112 July 28, 2021 13 / 38 Definition of Probability: Introduction Chance and the assessment of risk play a part in everyone’s life. As a loan officer you may want to evaluate the probability that a customer defaults. As an investment officer you are interested in finding the likelihood of an investment results in a loss. Probability has found a wide range of business applications such as: calculation of risk in the banking and insurance industries, quality control and market research. Most events in life are unpredictable, i.e. events in life are non-deterministic. If the world is treated as stochastic then we can measure the chances of any particular outcome happening in a given situation. STAT 112 July 28, 2021 14 / 38 Probability Definition Probability is the measure of the likelihood of the occurrence of an event. The probability of an event can be thought of as the relative frequency of the event. The probability of an outcome can also be interpreted as a subjective probability, or degree of belief that the outcome will occur. Probabilities are numbers between 0 and 1. An event with probability 0 has no chance of occurring, and an event with probability 1 is certain to occur. Other definitions of Probability are Relative Frequency and Classical definitions. STAT 112 July 28, 2021 15 / 38 Relative Frequency Definition The probability of an outcome is interpreted as the limiting value of the proportion of times the outcome occurs in n repetitions of the random experiment as n increases beyond all bounds. For example if we assign a probability of 0.4 to the outcome that there is a defective item in a production batch, we might interpret this assignment as implying that, if we analyse many items in the production batch, approximately 40% of them will be defective. This example illustrates a relative frequency interpretation of probability. STAT 112 July 28, 2021 16 / 38 Relative Frequency Definition n(A ) P (A ) = lim , n→∞ n n(A ) where n is called the relative frequency of event A. Stated differently, the relative frequency can be stated as the ratio n(A ) n , for the probability of A , if n is a large number. When the likelihood assumption is not valid, then the relative frequency method can be used. The relative frequency technique, lead one to conduct an experiment n times and record the outcomes. The probability of event A is assigned by P (A ) = fnA , where fA denotes the number of experimental outcomes that satisfy event A. STAT 112 July 28, 2021 17 / 38 Worked Example The table below shows the frequency distribution of grades in Statistics class. Marks 60 75 78 80 83 86 88 90 92 96 98 99 Freq 1 3 4 5 7 11 13 5 7 4 2 1 The relative frequency for mark 90 is computed as: 5 = 0.079 63 Also the relative frequency for obtaining a mark less than or equal to 83 is 1+3+4+5+7 20 = = 0.31 63 63 Using the relative frequency definition, we can therefore estimate probabilities. Thus the probability of obtaining a mark 90 is 0.079 and that obtaining a mark less than or equal to 83 is 0.31. STAT 112 July 28, 2021 18 / 38 Classical Definition ( Equally Likelihood Model) In an experiment for which the sample space is S and the outcomes are equally likely (that is, each outcome has the same chance of occurring), then the probability of an event A occurring is given by the formula number of outcome in A P (A ) = total number of outcomes number of ways that A can occur = number of ways the sample space S can occur n(A ) = n(S ) In other words, P (A ) is the fraction of the total number of outcomes that cause the event A to occur. STAT 112 July 28, 2021 19 / 38 Example 1 Suppose there are 5 balls in a box, 3 balls are red and 2 are black. The box is shaken, and a ball selected without looking. What is the probability that this ball is red? Solution : The event that a ball selected is red consists of 3 outcomes out of 5 possible outcomes in all. According to the basic probability formula, the probability of selecting a red ball is 3 P (red ball) =. 5 STAT 112 July 28, 2021 20 / 38 Example 2 In a group of 400 college students, there are 170 science majors and 230 non-science majors. Of the science majors, 55 are women, and there are 120 women non-science majors. One student is selected at random from the group. Find the probability that the selected student is: a A Science major b A Woman c Neither a woman nor a science major Solution : Science Majors Non -Science Majors Total Women 55 120 175 Men 115 110 225 Total 170 230 400 STAT 112 July 28, 2021 21 / 38 Solution Cont’d Let A denotes the event that a student is a science major, i.e. n(A ) = 170; also let S denotes the total sample size, n(S ) = 400. a There are 170 science majors out of 400 students, so n(A ) P (Sciencemajor ) = P (A ) = n(S ) = 170 400. b Let W represents the event a student is a woman, n(W ) = 175, since there are 175 women out of 400 students altogether, we have n (W ) 400. 175 P (Woman) = P (W ) = n(S ) = c There are 110 students who are neither women nor science majors 110 so P (neither woman nor science major) = 400. STAT 112 July 28, 2021 22 / 38 Axioms of Probability Let S be a finite sample space, and A be an event in S. Then in the axiomatic definition, the probability P (A ) of the event A is a real number assigned to A which satisfies the axioms: Axiom 1: 0 ≤ P (A ) ≤ 1. Probability is a number between 0 and 1 (inclusive). Axiom 2: P (S ) = 1 and P (φ) = 0. In performing an experiment, we assume that the result will be one of the simple outcomes. In other words, we assume that the event S will occur. Axiom 3: (a) If A and B are mutually exclusive events, then P (A ∪ B ) = P (A ) + P (B ). STAT 112 July 28, 2021 23 / 38 Axioms of Probability Cont’d. If the sample space S is not finite (ie. infinite) then (a) must be modified as follows: (b) If A1 , A2 ,... , is an infinite sequence of mutually exclusive events in S, that is (Ai ∩ Bj = φ) for i , j, then ! ∞ P (A1 ∪ A2 ∪ A3 ∪... ) = P S Ai i =1 = P (A1 ) + P (A2 ) + P (A3 ) +... ∞ P = P (Ai ). i =1 STAT 112 July 28, 2021 24 / 38 Axioms of Probability Cont’d. This means for instance if A1 , A2 , A3 ,... , are events that can not occur simultaneously, then the probability that one among them will occur is the sum of their probabilities. For example, the probability of either a 2 or a 6 in a roll of a die is 1 1 2 + = , (Axiom 3). 6 6 6 STAT 112 July 28, 2021 25 / 38 Elementary Properties of Probability Property 1: P (Ā ) = 1 − P (A ) This means that if the probability of occurrence of an event A is P (A ), then the probability of non occurrence of the event A is P (Ā ) = 1 − P (A ). 1 For instance if the probability of obtaining a 6 in a roll of die is 6 then the probability of not obtaining a 6 is 1 − ( 16 ) = 56. Property 2: P (φ) = 0, the probability of null or empty set or an impossible event is zero. Property 3: The probability of a union (addition rule) P (A ∪ B ) = P (A ) + P (B ) − P (A ∩ B ). P (A ) ≤ P (B ), if the event A ⊆ B. STAT 112 July 28, 2021 26 / 38 Theorem Finally, if A and B are mutually exclusive, then P (A ∩ B ) = 0 and the formula becomes P (A ∪ B ) = P (A ) + P (B ). Theorem 1: If A , B and C are any three events, then P (A ∪ B ∪ C ) = P (A ) + P (B ) + P (C ) − P (A ∩ B ) − P (A ∩ C ) −P (B ∩ C ) + P (A ∩ B ∩ C ) The Proof is left as an Exercise. STAT 112 July 28, 2021 27 / 38 Worked Example Suppose events A , B and C have probabilities; P (A ) = 12 , P (B ) = 31 and P (C ) = 14. Furthermore, assume that A ∩ C = φ, B ∩ C = φ and P (A ∩ B ) = 61. Use the axioms and properties of probabilities as well as the facts concerning sets and events to find. a P [(A ∩ B )0 ] b P (A ∩ B 0 ) c P [(A ∪ B )0 ] d P (A 0 ∩ B 0 ) e P (A ∪ B ∪ C ) STAT 112 July 28, 2021 28 / 38 Solution a Applying property 1, we have P [(A ∩ B )0 ] = 1 − P (A ∩ B ) = 1 − 1 6 = 65. b We use the relation: A = (A ∩ B 0 ) ∪ (A ∩ B ) Now, P (A ) = P [(A ∩ B 0 ) ∪ (A ∩ B )] = P (A ∩ B 0 ) + P (A ∩ B ) 1 =⇒ 2 = P (A ∩ B 0 ) + 16 =⇒ P (A ∩ B 0 ) = 12 − 16 = 31. c P [(A ∪ B )0 ] = 1 − P (A ∪ B ) = 1 − [P (A ) + P (B ) − P (A ∩ B )] = 1 − [ 12 + 31 − 16 ] = 13. d According to De Morgan’s law, we have P (A 0 ∩ B 0 ) = P [(A ∪ B )0 ] = 13 STAT 112 July 28, 2021 29 / 38 Solution Cont’d e Since A ∩ C = φ and B ∩ C = φ, it follows that (A ∪ B ) ∩ C = φ. Also P (A ∪ C ) = 32 and P (C ) = 14 , then P (A ∪ B ∪ C ) = P [(A ∪ B ) ∪ C ] 2 1 = P (A ∪ B ) + P (C ) = + 3 4 11 =. 12 STAT 112 July 28, 2021 30 / 38 Example 2 In the toss of a fair die, find the probability that the result is even or divisible by 3. Solution: Let A be the event that the result is even and B the event that the result is divisible by 3. Then A = {2, 4, 6} and B = {3, 6}, it implies that A ∩ B = {6}. Now, the required probability is P (A ∪ B ). P (A ∪ B ) = P (A ) + P (B ) − P (A ∩ B ) 3 2 1 2 = + − = 6 6 6 3 STAT 112 July 28, 2021 31 / 38 Theorem Definition: Two events A and B are said to be equally likely if P (A ) = P (B ). Event A is said to be more likely than B if P (A ) > P (B ). Theorem 2 If all the outcomes of a random experiment consist of n equally likely simple events, then the probability of an event E made up of k simple events is k P (E ) =. n The proof is left as an Exercise. STAT 112 July 28, 2021 32 / 38 Trial Questions 1 A box contains 10 green balls and 8 red balls. Six balls are selected at random. Find the probability that: a all 6 balls are green. b that exactly 4 out of the 6 balls are red. c at least one of the 6 balls is red. d 3 or 4 of the 6 balls are red. 2 A bag contains two red, three green and four black balls of identical size except for colour. Three balls are drawn at random without replacement. a Show that the probability that two balls have the same colour and the third is different is 55 84. b What is the probability that at least one is red. STAT 112 July 28, 2021 33 / 38 Expectation of Random Variables An important notion in statistics is the expected value or expectation of a random variable. The expectation of a random variable is the long-term average of the random variable. For instance, imagine observing many thousands of independent random values from the random variable of interest. Take the average of these random values. The expectation is the value of this average as the sample size tends to infinity. STAT 112 July 28, 2021 34 / 38 Definition of Expectation 1 If X is a random variable distributed as p (xi ) or f (x ), then the expected value of X , which we will denote as E (X ), is defined as  P all xi xi p (xi ), where X is a discrete random variable     E (X ) =    ∞ xf (x )dx , where X is a continuous random variable    R −∞ where the summation is over the entire sample space of X , i.e., over all admissible values of X and p (xi ) is the probability mass function of X. If the random variable is continuous, then the summation is replaced by the integration and f (x ) is the density function of the random variable X. STAT 112 July 28, 2021 35 / 38 Example Consider a class of 24 students. Suppose that 5 of them are 19 years old, 7 are 20 years old , 10 are 23 years old, and 2 are 17 years old. Let us denote by µ the mean age of the class, that is : 5 × 19 + 7 × 20 + 10 × 23 + 2 × 17 µ= = 20.79 24 Let X be the random variable taking the distinct ages as values, that is X = {x1 , x2 , x3 , x4 }, with x1 = 19, x2 = 20, x3 = 23 and x4 = 17, with probability law given as: P (X = x1 ) = 24 5 , P (X = x2 ) = 24 7 , P (X = x3 ) = 10 24 , P (X = x4 ) = 24. 2 We may summarize this probability law in the following table as: x 19 20 23 17 5 7 10 2 P (X = x ) 24 24 24 24 STAT 112 July 28, 2021 36 / 38 Example Cont’d Set sample space, Ω as the class of these 24 students. The following graph X :Ω→R ω → X (ω) = age of the student ω, defines a random variable. The probability that is used on Ω, the sample space is the frequency of occurrence. For each i ∈ {1, 2, 3, 4}, P (X = xi ) is the frequency of occurrence of the age xi in the class. Therefore, through these notations, we have the following expression of the mean µ: 4 X µ= xi P (X = xi ). i =1 Note: The Expectation will be treated later in session 3B. STAT 112 July 28, 2021 37 / 38 End of Session THANK YOU!!! STAT 112 July 28, 2021 38 / 38