IMG_0622.jpeg

# Lecture 19: Hypothesis Testing ## Goals * Introduce **Hypothesis Testing** * Discuss the concepts of **Null Hypothesis**, **Alternative Hypothesis**, **Test Statistic**, and **p-value**. * See how to use hypothesis testing in conjunction with confidence intervals. ## Motivating Example *...

# Lecture 19: Hypothesis Testing ## Goals * Introduce **Hypothesis Testing** * Discuss the concepts of **Null Hypothesis**, **Alternative Hypothesis**, **Test Statistic**, and **p-value**. * See how to use hypothesis testing in conjunction with confidence intervals. ## Motivating Example * Suppose you are consulting for a small business that sells gourmet cookies. * They have been using a particular recipe for chocolate chip cookies for the past 5 years and know that, on average, each cookie has **12 chocolate chips**. * Recently, they hired a new baker, and after a few weeks, customers have started complaining that the cookies don't have enough chocolate chips. * The business owner wants to know if the new baker is using less chocolate chips than the original recipe. * How can we approach this problem using the data we collect? ## Hypothesis Testing * Hypothesis testing is a method for testing a claim or hypothesis about a population based on sample data. * In the cookie example, the population is all the cookies baked by the new baker, and the hypothesis is that the average number of chocolate chips is less than 12. * We can test this hypothesis by collecting a sample of cookies, counting the number of chocolate chips in each cookie, and then using this data to calculate a test statistic. * The test statistic is a measure of how likely it is that the sample data would have occurred if the hypothesis were true. * If the test statistic is too extreme, we reject the hypothesis and conclude that the new baker is using less chocolate chips than the original recipe. ## Null and Alternative Hypotheses * The first step in hypothesis testing is to state the null and alternative hypotheses. * The **null hypothesis** ($H_0$) is a statement about the population that we are trying to disprove. * In the cookie example, the null hypothesis is that the average number of chocolate chips is equal to 12. * $H_0: \mu = 12$ * The **alternative hypothesis** ($H_a$) is a statement about the population that we are trying to prove. * In the cookie example, the alternative hypothesis is that the average number of chocolate chips is less than 12. * $H_a: \mu < 12$ ### One-sided vs Two-sided Tests * The alternative hypothesis can be one-sided or two-sided. * A **one-sided** alternative hypothesis is one in which the population parameter is either greater than or less than the value stated in the null hypothesis. * $H_a: \mu < 12$ or $H_a: \mu > 12$ * A **two-sided** alternative hypothesis is one in which the population parameter is not equal to the value stated in the null hypothesis. * $H_a: \mu \neq 12$ * Whether to use a one-sided or two-sided test depends on the question you are trying to answer. * If you are only interested in whether the population parameter is greater than or less than the value stated in the null hypothesis, then you should use a one-sided test. * If you are interested in whether the population parameter is different from the value stated in the null hypothesis, then you should use a two-sided test. ## Test Statistic * A test statistic is a number calculated from the sample data that is used to assess the evidence against the null hypothesis. * The formula for the test statistic depends on the type of hypothesis test you are conducting. * In the cookie example, we can use the t-statistic to test the hypothesis that the average number of chocolate chips is less than 12. * The t-statistic is calculated as follows: $t = \frac{\bar{x} - \mu_0}{s / \sqrt{n}}$ where: * $\bar{x}$ is the sample mean * $\mu_0$ is the value of the population mean under the null hypothesis * $s$ is the sample standard deviation * $n$ is the sample size * The t-statistic measures how many standard errors the sample mean is away from the value of the population mean under the null hypothesis. ## p-value * The p-value is the probability of observing a test statistic as extreme as, or more extreme than, the one observed, assuming that the null hypothesis is true. * The p-value is a measure of the evidence against the null hypothesis. * The smaller the p-value, the stronger the evidence against the null hypothesis. * In the cookie example, the p-value is the probability of observing a sample mean as small as, or smaller than, the one observed, assuming that the average number of chocolate chips is equal to 12. * We can calculate the p-value using a t-distribution with $n-1$ degrees of freedom. ## Significance Level * The significance level ($\alpha$) is the probability of rejecting the null hypothesis when it is true. * The significance level is a threshold that we set before conducting the hypothesis test. * If the p-value is less than the significance level, we reject the null hypothesis. * If the p-value is greater than the significance level, we fail to reject the null hypothesis. * The most common significance level is 0.05, which means that there is a 5% chance of rejecting the null hypothesis when it is true. ## Decision Rule * The decision rule is a rule that tells us whether to reject the null hypothesis based on the p-value and the significance level. * The decision rule is as follows: * If $p \le \alpha$, reject $H_0$ * If $p > \alpha$, fail to reject $H_0$ Where: * p is the p-value * $\alpha$ is the significance level ## Connection to Confidence Intervals * There is a close connection between hypothesis testing and confidence intervals. * In general, if the value of the population parameter under the null hypothesis is not contained in the confidence interval, then we reject the null hypothesis. * In the cookie example, if the 95% confidence interval for the average number of chocolate chips does not contain 12, then we reject the null hypothesis that the average number of chocolate chips is equal to 12 at the 5% significance level. ## Example * Let's go back to the cookie example. * Suppose we collect a sample of 30 cookies and find that the sample mean is 11 chocolate chips and the sample standard deviation is 2 chocolate chips. * We want to test the hypothesis that the average number of chocolate chips is less than 12 at the 5% significance level. 1. State the null and alternative hypotheses: * $H_0: \mu = 12$ * $H_a: \mu < 12$ 2. Calculate the test statistic: $t = \frac{11 - 12}{2 / \sqrt{30}} = -2.74$ 3. Calculate the p-value: $p = P(T < -2.74) = 0.005$ 4. Make a decision: Since the p-value (0.005) is less than the significance level (0.05), we reject the null hypothesis. 5. Conclusion: We conclude that the average number of chocolate chips is less than 12 at the 5% significance level. ## Errors in Hypothesis Testing * There are two types of errors that can occur in hypothesis testing: * Type I error: Rejecting the null hypothesis when it is true. * Type II error: Failing to reject the null hypothesis when it is false. * The probability of making a Type I error is equal to the significance level ($\alpha$). * The probability of making a Type II error is denoted by $\beta$. * The power of a test is the probability of rejecting the null hypothesis when it is false. * The power of a test is equal to $1 - \beta$. ## Summary * Hypothesis testing is a method for testing a claim or hypothesis about a population based on sample data. * The first step in hypothesis testing is to state the null and alternative hypotheses. * The null hypothesis is a statement about the population that we are trying to disprove. * The alternative hypothesis is a statement about the population that we are trying to prove. * A test statistic is a number calculated from the sample data that is used to assess the evidence against the null hypothesis. * The p-value is the probability of observing a test statistic as extreme as, or more extreme than, the one observed, assuming that the null hypothesis is true. * The significance level is the probability of rejecting the null hypothesis when it is true. * If the p-value is less than the significance level, we reject the null hypothesis. * If the p-value is greater than the significance level, we fail to reject the null hypothesis. * There is a close connection between hypothesis testing and confidence intervals. * There are two types of errors that can occur in hypothesis testing: Type I error and Type II error.