Probability and Statistics Lecture Notes (Fall 2024) PDF

0 What’s probability? (1/5) 0 What’s probability? A probability is a number between zero and one that measures the chance of an event happening. Unlike other areas of mathematics, probability theory didn’t emerge in any fo...

0 What’s probability? (1/5) 0 What’s probability? A probability is a number between zero and one that measures the chance of an event happening. Unlike other areas of mathematics, probability theory didn’t emerge in any form until modern times. To put this into context, calculus which is the paradigmatic example of modern mathematics, has precedents in the work of Ancient mathematicians such as Archimedes. However, probability—or better said, its mathematical treatment and formalization—didn’t appear at all until modern age, with Cardano’s book Liber de ludo aleae in the 16th century. 0.1 Why should we care about probability? Probability theory allows us to quantify uncertainty. Unfortunately, there are plenty of natural phenom- ena in which knowing the initial conditions doesn’t guarantee a fixed outcome. Even if we don’t believe that uncertainty is intrinsic to certain phenomena, it might be too expensive to control or measure the variables that determine the uncertainty. The paradigmatic example of such a phenomenon is tossing a coin. In principle, if we had access to a supercomputer with plenty of sensors we could predict with total precision the outcome of a coin toss. However, this would be so expensive that the ability to make exact predictions of the coin toss might be not worthy. Question 0.1. Have you ever used probability before? If so, in which situations? As of today, probability theory lays the foundation on which statistics is constructed. In this way, understanding probability is a must if one wants to understand the mathematical technology of nowa- days science. 0.2 Combinatorial definition of probability The following definition is the first definition we can see of probability in many books: Definition 0.a. (Combinatorial definition) The probability of an event is the number of favorable cases where the event happens divided by the number of possible cases. In other words, number of favorable cases probability =. (1) number of possible cases Unfortunately, the above definition will not be enough for us. However, this definition is a good starting point to construct intuition about many phenomena that we encounter. Example 0.1. (Fair Coin) A fair coin has two sides: heads and tails. When we toss a coin, we either get heads or tails. Hence the probability of getting heads after tossing a fair coin is 1/2 and of getting tails also 1/2. △ Example 0.2. (Fair Die) A fair die with six sides has six possible outcomes: 1, 2, 3, 4, 5 and 6. The probability of getting any of these numbers is 1/6, since there are six possibilities, but only one is favorable to getting a specific number. Now, the probability of getting an even number is 1/2, because among the six possibilities, three of them (2,4 and 6) are favorable to the event of getting an even number. △ 1/181 JHU 553.311 Intermediate Probability and Statistics Lecture Notes (Fall 2024) 0 What’s probability? (2/5) Example 0.3. (Words) Imagine you generate words uniformly at random, i.e., imagine you have a die with 26 facets—one for each letter—that you roll a certain number of times to construct a word. In this setting, you might ask about the probability of certain pattern. For example, what’s the probability of getting a palindrome? Assume that you generate words with 3 letters. The total amount of such words is 17576, but there are only 676 palindromes with three letters (why?). Hence the probability that a random three-letter word is a palindrome is 1/26. △ Example 0.4. (Draw one card) A common magic trick is to let the spectator take a card which latter on the magician would guess. Recall that the usual Poker deck has 52 cards, each one of them marked with both a suit and a value. The suit can be one of four: clubs (♣), diamonds (♢), hearths (♡) and spades (♠). The value is one of 13: Ace (A), 2, 3, 4, 5, 6, 7, 8, 9, 10, Jack (J), Queen (Q) and King (K). For example, below we can see some sample cards: A 10 r 7q q 7 10 r J Q K ♠ r r r rr q q ♣ ♣♣♣ r r r rr ♣♣ qq ♠♠ ♠ ♠ r rr r rr q q q ♣ ♣ ♣ r rr r rr ♣♣ qq ♠♠ ♣ A 10 q 7q 7♣ 10 J Q K Observe that 52 = 4 · 13, which as we will see later shows the fact that choosing a card is the same as choosing a suit and a value. Now, if the magician instead of doing a magic trick would try to guess the card at random, she would guess the card with a probability of 1/552, since there is one favorable case and 52 possible cases. △ Remark 0.1. All the above examples assume that all possibilities are equally likely. However, this is not always the case. This is the setting of fair coins, fair dices and uniformly random choices. Later on, we will learn how to handle with probability theory also when the coins are not fair and the choices are random, but not uniformly at random. ¶ Now, we have to be careful when we combine the above processes, since we need to avoid collapsing two favorable cases into a single one. Example 0.5. (Draw one ball) In an urn, where we don’t see the content, there are 5 red balls and 7 blue balls. If we pick a ball at random, what’s the probability of picking a red ball? A naive thinking might give us that both the probability of getting a red and a blue ball is the same, because there are two possibilities (red ball or blue ball) and only one of them is favorable. However, this thinking is wrong, because there are not 2 possibilities, but 12 possibilities. Assume that on top of having a color, each series of collored balls is numbered. This does not affect the probability, but it makes easier to see the 12 = 5 + 7 possibilities: 1st, 2nd, 3rd, 4th or 5th red ball or the 1st, 2nd, 3rd, 4th, 5th, 6th or 7th blue ball. In this way, the probability of getting a red ball is 5/12, because there are 5 favorable cases out of a total of 12 possible cases. △ Example 0.6. (Number of heads when tossing two coins) If we toss two fair coins, we will get either 0, 1 or 2 heads. However, not all these outputs are equally likely, not being the probability 1/3. JHU 553.311 Intermediate Probability and Statistics Lecture Notes (Fall 2024) 2/181  0 What’s probability? (3/5) To see this, assume, as in the previous example, that the coins are numbered, then we see that we get 0 heads if we have that both coins are tails, and 2 head if both coins are heads. However, we get 1 head if either the 1st coin is heads and the 2nd one tails, or the 1st coins tails and the 2nd one heads. Hence, doing the ratios between favorable and possible cases, we get that we get 0 heads with probability 1/4, 1 heads with probability 1/2 and 2 heads with probability 1/4. △ Example 0.7. (Sum of two dice) If we roll two (fair) six-sided usual dice—something very common in some board games—, we have 11 possible outcomes for the sum: 2, 3, 4, 5, 6, 7, 8, 9, 10, 11 and 12. Again, these outcomes are not equiprobable, because we have to be careful when considering favorable and possible cases. Again, assume that one die is red and the other one is blue. Then we can see that we have a total of 36 = 6 · 6 possibilities for the outcomes of the pair of dice. If we do a table with the sums, we get the following table: 1 2 3 4 5 6 1 2 3 4 5 6 7 2 3 4 5 6 7 8 3 4 5 6 7 8 9 4 5 6 7 8 9 10 5 6 7 8 9 10 11 6 7 8 9 10 11 12 Helping ourselves with the table, we can see the following: the probability of the sum being 2 is 1/12, of being 3 is 2/12 = 1/6, of being 4 is 3/12 = 1/4, of being 5 is 4/12 = 1/3, of being 6 is 5/12, of being 7 is 6/12 = 1/2, of being 8 is 5/12, of being 9 is 4/12 = 1/3, of being 10 is 3/12 = 1/4, of being 11 is 2/12 = 1/6 and of being 12 is 1/12. △ 0.3 Frequentist definition Unfortunately, not in all phenomena we have a number a finite number of possibilities. Moreover, the combinatorial definition has another flaw, what does it mean to have a certain probability? In other words, according to the combinatorial definition, the probability of a coin toss giving heads is 1/2 and the probability of two dice adding to 5 is 1/3. We can compute a number that we call probability, but what does this number mean?! Question 0.2. What does probability mean to you? How do you interpret the statement “it has a probability of 1/3” and the statement “it happens with probability 1/e”? To answer the above question is better to have in mind the repeated experiment set up: Imagine that the phenomenon repeats indefinitely under the same known conditions. When this is a deterministic phenomenon, we would get always the same result. When this is a random phenomenon, what happens will change each time. However, as we in- crease the number of repetitions, the proportion of times an event happens over the total number of trials would converge to a number: the probability of the event. 3/181 JHU 553.311 Intermediate Probability and Statistics Lecture Notes (Fall 2024) 0 What’s probability? (4/5) In the above set up, we get the so-called frequentist definition of probability. Definition 0.b. (Frequentist definition) The probability of outcome A of a random experiment is given by: number of times A happens after n repetitions of the experiment probability = lim , (2) n→∞ n where, in the sequence of repetitions, repetitions are causally independent1 of each other. The above definition allows us to cover both discrete and continuous random phenomena. How- ever, we will need to wait for later to handle properly the continuous phenomena. Example 0.8. (Number of coin tosses until getting heads) Toss a coin repeteadly until you get heads. How many tosses will you do? In particular, what’s the probability that you toss the coin more than two times to get heads? △ Example 0.9. (Number of hurricanes in a year) Each year, the number of hurricanes is not always the same. Either if you believe that this is a deterministic process or not, it makes sense to ask the following question: What’s the probability that there are more than 4 hurricanes in a year? △ Example 0.10. (Desintegration time of an isotope) The amount of time that an isotope (a radioac- tive element) takes in decomposing (dividing into simpler atoms) is a random number. Hence we can ask: what’s the probability that an isotope of uranium-238 will decompose in 4.468 × 109 years? △ Observe that the repeated experimental set-up is realistic sometimes, as we can actually run the experiment once and again and again. However, in some setting is more of a mental experiment, since we cannot repeat the sitation with the same conditions, for example, the number of hurricanes in a year. In these settings, we should think of this as if we are in Grounhog Day (1993), Run Lola Run (1998), Edge of Tomorrow (2014) or Happy Death Day (2017) movies: we go back to the past and repeat the events as if they were an experiment. However, unlike in those movies, a true probabilist would not think that the outcome would necessarily be the same if we don’t live in a fully deterministic world. Unfortunately, we will not go deeper into the philosopy of probability. As we will see later on, for working with probabilities, we don’t need a full grasp of what probability is, but of how probability behaves and is computed. 0.4 With Probability p … (Optional) How do we express that something has a probability? In the more formal setting, you will hear ex- pressions such as “[…] has/with probability p ” to express that something has probability (of happening) equal to p. However, throughout many texts you will see equivalent expressions such as “[…] has/with a chance of (a) 100p %” 1 What does this mean? In other words, the outcomes of no subset of the experiments does affect the outcome of any other experiment. JHU 553.311 Intermediate Probability and Statistics Lecture Notes (Fall 2024) 4/181  0 What’s probability? (5/5) and “the odds of […] are one in 1/p ” or “the odds of […] are a in 1/(pa )”. You should be able to switch between these kind of expressions easily and understanding in which context is each of them more prevalent. Example 0.11. (Fair Coin) When we toss a fair coin, we have half and half probabilities of getting heads and tails. Hence we will say: The probability of getting head is 1/2. The probability of getting head is 0.5. Heads have a chance of a 50%. The odd for heads are one in two. Which of these △ Example 0.12. (Fair Die) When we roll a fair 6-sided die, what are the chances of not getting six? We will say expressions such as: The probability of not getting a six is 5/6. After rolling the die, the odds of not getting a die are one in 1.2. The chance of not getting a six is of a 83.33%. Observe that the expression using odds can get complicated if we want to say it as one in x. Hence, you will see sometimes variants such as the following: “the odds of not getting a six are five in six”. △ Example 0.13. (Drawing a Card) When we draw a card from a well-shuffled poker deck, how can we express the probability of drawing a particular card such as the 3 of clubs2 out of the deck? The probability of getting the three of clubs is is 0.019.... We will get the three of clubs with a chance of 1.92% The odds of getting a three of clubs is one in fifty-two. △ As a rule of thumb, expressions with probability are the more mathematical, while expressions with chances and odds are less. The latter, the ones using odds, are pretty popular in gambling games and betting houses. 2 Of course, if Penn and Teller make you draw the card, the chances might be bigger. But let’s assume that Penn and Teller are not involved. 5/181 JHU 553.311 Intermediate Probability and Statistics Lecture Notes (Fall 2024)  29 An introduction to an statistical tank: ANOVA (3/1) About these lecture notes These lectures notes have grown out of some rough notes by Dr. Fred Torcaso, which were originally typed by Dr. Zach Pisano and then significantly improved by Dr. Taylor Jones. The current version is a full rewrite, reorganization and extension by Dr. Josué Tonelli-Cueto, with the feedback of Dr. Taylor Jones. 181/181 JHU 553.311 Intermediate Probability and Statistics Lecture Notes (Fall 2024)