Statistical Inference and Hypothesis Testing

Statistical Inference

• Statistical inference involves using sample data to draw conclusions about a larger population.
• Proper inference leads to almost always correct conclusions about the population.

Hypothesis Testing

• Hypothesis testing, a core statistical concept, follows a process of elimination to uncover truth.
• It starts with a null hypothesis (H₀), a starting assumption.
• Evidence, often measured by a p-value, is collected against the null hypothesis.
• A small p-value (close to zero) indicates strong evidence against the null hypothesis.
• If the p-value is below the significance level (α, typically 0.05), the null hypothesis is rejected in favor of an alternative hypothesis (Hₐ).

Decision Errors

• When the p-value is near zero, an extremely rare event or a false null hypothesis are possible.
• Rejecting a true null hypothesis is a Type I error.
• Accepting a false null hypothesis is a Type II error.

Type I Error, Significance Level, Confidence

• A Type I error is rejecting a true null hypothesis.
• The significance level (α) controls the probability of a Type I error.
• A common α value is 0.05, but any value between 0 and 1 can be used.
• Higher α means increased Type I error probability and decreased Type II error probability.
• Lower α means decreased Type I error probability and increased Type II error probability.
• Confidence level = 1 - α

Type II Errors, β, and Power

• A Type II error is failing to reject a false null hypothesis.
• The probability of a Type II error is often unknown (β).
• The power of a hypothesis test is 1 - β.

Sufficient Evidence

• "Sufficient evidence" is defined statistically, typically by the p-value.
• The p-value represents the probability of observing data as extreme or more extreme than the observed data, assuming the null hypothesis is true.
• An extremely small p-value indicates strong evidence against the null hypothesis.

Evidence vs. Proof

• Hypothesis testing provides a formal way to either support the alternative or keep the null hypothesis.
• Statistical methods never prove a hypothesis; they only provide evidence for or against it.

Calculating the p-Value

• The p-value is calculated by comparing a test statistic to its sampling distribution (under the null hypothesis).
• A test statistic measures how different the observed data is from the expected data under the null hypothesis.
• The sampling distribution is the theoretical distribution of the test statistic over all possible samples, given the null hypothesis

Parametric Methods

• Parametric methods assume the test statistic follows a specific theoretical distribution under the null hypothesis.
• They are usually more powerful but assume more about the data.

The Normal Distribution

• The Normal Distribution is a fundamental distribution in statistics.
• It approximates many real-world data sets (e.g., heights, batting averages).
• The sampling distribution of the sample mean (x̄) is approximately normal if the parent population is normal or the sample size (n) is sufficiently large (often n ≥ 30).
• The normal distribution is described by two parameters: mean (μ) and standard deviation (σ).

The Chi-Squared Distribution

• Defined for x ≥ 0, and positive values of p (degrees of freedom).
• Used theoretically, often in test statistics.
• The shape becomes more normal as the degrees of freedom increase.

The t-Distribution

• Closely related to the Normal Distribution; is often the sampling distribution for several test statistics.
• Has only one parameter (degrees of freedom).
• The t-distribution becomes more normal as degrees of freedom increase

The F-Distribution

• The F-distribution is a ratio of two chi-squared random variables, each divided by their respective degrees of freedom.
• Used often in ANOVA and regression.
• As degrees of freedom increase, the F-distribution becomes more normal.

Nonparametric Methods

• Nonparametric methods make minimal assumptions about the data.
• They are generally less powerful but more widely applicable.