IMG_1018.jpeg

# Lecture 24: Hypothesis Testing ## 1. Statistical hypothesis testing ### Definition A statistical hypothesis is an assertion or conjecture about the distribution of one or more random variables. Hypothesis testing is the process of determining whether the evidence at hand is strong enough to rej...

# Lecture 24: Hypothesis Testing ## 1. Statistical hypothesis testing ### Definition A statistical hypothesis is an assertion or conjecture about the distribution of one or more random variables. Hypothesis testing is the process of determining whether the evidence at hand is strong enough to reject the conjecture. ### Example A researcher might conjecture that the average height of men in a certain population is 1.8 meters. In that setting: * **Null Hypothesis** $H\_0$: The average height of men in the population is 1.8 meters. * **Alternative Hypothesis** $H\_A$: The average height of men in the population is not 1.8 meters. ## 2. Elements of a hypothesis test ### Definition A hypothesis test is a rule that specifies for which sample values the decision is made to accept $H\_A$ and for which values we do not reject $H\_0$. The critical region is the set of all sample values for which $H\_0$ is rejected. ### Remarks * We only "accept" $H\_A$ when the evidence is strong enough. Otherwise, we say that we "do not reject" $H\_0$. * We can never be 100% sure that our decision is correct. There is always a chance of making a mistake. ### Types of Error in Hypothesis Testing | | **$H\_0$ is true** | **$H\_0$ is false** | | :--------------- | :------------------ | :------------------- | | **Reject $H\_0$** | Type I error | Correct Decision | | **Don't reject $H\_0$** | Correct Decision | Type II error | * **Type I error**: Rejecting $H\_0$ when it is true. The probability of making a Type I error is denoted by $\alpha$. * **Type II error**: Failing to reject $H\_0$ when it is false. The probability of making a Type II error is denoted by $\beta$. ### Definition The **power** of a test is the probability of rejecting $H\_0$ when it is false: $\qquad \text{Power} = 1 - \mathbb{P}(\text{Type II error}) = 1 - \beta$ ### Example In our previous example, a Type I error would be to conclude that the average height of men in the population is not 1.8 meters, when it actually is. A Type II error would be to fail to reject the null hypothesis when it is false, i.e., to conclude that the average height of men in the population is 1.8 meters, when it actually is not. ### The significance level * The significance level of a test, denoted by $\alpha$, is the probability of making a Type I error. * The value of $\alpha$ is chosen by the researcher before the test is conducted. * Common values for $\alpha$ are 0.01, 0.05, and 0.1. * A small value of $\alpha$ means that we require strong evidence before we reject $H\_0$. ### The $p$-value * The $p$-value is the probability of observing a test statistic as extreme as, or more extreme than, the value actually observed, assuming that $H\_0$ is true. * The $p$-value is a measure of the evidence against $H\_0$. * A small $p$-value means that the observed data is unlikely to have occurred if $H\_0$ is true. * We reject $H\_0$ if the $p$-value is less than or equal to $\alpha$. ### Example Suppose we conduct a hypothesis test and obtain a $p$-value of 0.03. If we had chosen a significance level of $\alpha = 0.05$, we would reject $H\_0$. If we had chosen a significance level of $\alpha = 0.01$, we would not reject $H\_0$. ### The $(1 - \alpha)$ confidence interval * A $(1 - \alpha)$ confidence interval is a range of values that we are $(1 - \alpha) \cdot 100\%$ confident contains the true value of the parameter being estimated. * If the null hypothesis value falls outside the $(1 - \alpha)$ confidence interval, we reject $H\_0$ at the significance level $\alpha$. * For example, a 95% confidence interval corresponds to a significance level of $\alpha = 0.05$. ### Example Suppose we are testing the null hypothesis that the average height of men in a population is 1.8 meters. We collect a sample of data and construct a 95% confidence interval for the average height of men in the population. If the confidence interval is (1.75 meters, 1.85 meters), we would not reject $H\_0$ at the 5% significance level, because 1.8 meters falls within the interval. However, if the confidence interval is (1.86 meters, 1.96 meters), we would reject $H\_0$ at the 5% significance level, because 1.8 meters falls outside the interval.