ECON 266: Hypothesis Testing

Introduction to Econometrics

• ECON 266 is an introduction to econometrics.
• The presentation date is February 20, 2025.

Hypothesis Testing

• Decision Rule in Hypothesis Testing depends on the alternative hypothesis.
• Reject H0 if |(b1 - βNull) / se(b1)| > critical value when HA : β1 ≠ 0.
• Reject H0 if (b1 - βNull) / se(b1) > critical value when HA : β1 > 0.
• Reject H0 if (b1 - βNull) / se(b1) < critical value when HA : β1 < 0.

Practicing t Test

• t-test practice involves writing the OLS model, outlining null and alternative hypotheses, and conducting the t-test with a 5% significance level.
• Using a sample of 42 counties, testing if charging stations increase electric vehicle adoption, the estimated coefficient is 1.2 and the standard error is 0.4.
• For a sample of 1,974 firms, assessing the impact of sex on earnings, a coefficient of 40 is estimated with a standard error of 50.

Plan for Hypothesis Testing Discussion

• The plan includes concluding the discussion on hypothesis testing, covering p-values, statistical power, and confidence intervals.

Hypothesis Testing: p Values

• Statistical inference conducted through p-values is alternative to the t-test.
• A p-value represents the probability of observing a coefficient as extreme, assuming the null hypothesis is correct.
• The null hypothesis is rejected if the p-value is less than the significance level (α).
• The smaller the p-value, the stronger one can reject the null hypothesis.

Hypothesis Testing: Statistical Power

• The significance level is the probability of making a Type I error in hypothesis testing.
• The goal is to minimize making Type I errors by a low significance level.
• There is a necessity to minimize committing Type II errors.
• Type II error in an experiment can be catastrophic, like in the efficacy of a life saving drug.
• Statistical power concept is associated with Type II errors.
• Statistical power is the likelihood of correctly rejecting a false null hypothesis.
• Statistical power = 1 – Pr(Type II error).
• The standard normal cumulative distribution function is important for this formula.

Statistical Power Deeper Look

• Type II error comprehension is key in understanding power.
• Testing H0 : β1 = 0 against HA : β1 > 0 is supposed.
• Failure to reject the null occurs when the test statistic is below the critical value.
• If a mistake occurs from failing to reject the null, another β1 ≠ 0 must be the true parameter.
• Statistical power calculates the likelihood of rejecting the null when β1True ≠ 0.

Statistical Power Examples

• An example is needed to examine how this works for a test.
• Hypotheses set at H0 : β1 = 0 and HA : β1 > 0
• A 1% significance level chosen to conduct test, resulting in t_crit = 2.32
• Calculation shows probability of rejecting null equals 1 if β1True = 1.
• Evaluate the null distribution of b₁ to demarcate the rejection region by finding the t-critical value.
• Sketch the distribution of b₁ under β1True
• Identify the region representing the probability of committing a Type II error.

Quantifying Statistical Power

• The formula for the probability of Type II error is Φ((tcrit - β1True) / se(b1)).
• If se(b1) = 1, P(Type II error) = P(z < 2.32 - 1) = P(z < 1.32) = 0.9066= 0.91
• Statistical power = 1 – 0.91 = 0.09 or 9%.
• Researchers ideally want a power of at least 0.80 (80%).
• When a real effect is present, there is an 80% chance that the study will find it statistically significant.
• With low power, skepticism is needed because of insufficient data to reject the null hypothesis.
• A high power level means there is a high probability of rejecting the null hypothesis if the true β1 is β1True.
• Higher standard error of b1 reduces the statistical power.
• Variance increase of the estimated coefficient decreases power.
• Variance of b₁ (var(b₁)) is related to σ2 / (N × var(X)).

Limitations of Hypothesis Testing

• Hypothesis testing does not tell the whole story.
• Even when statistically significant coefficients are obtained, the hypothesis testing framework still has limits.
• Tools of hypothesis testing are not useful if there is endogeneity.
• Hypothesis testing can show different conclusions for comparable test statistics.
• Consequences of a t-test do not portray the magnitude of statistical significance.
• Analyzing statistical significance can distract from real-world significance.

Statistical vs. Substantive Significance

• A substantive significant coefficient is one that is large and meaningful.
• The independent variable has a considerable impact on the dependent variable.
• With large samples, se(b1) is negligible, so the t statistic is significant even for small b1 estimates.
• Small samples can lead to large se(b1) and failure to reject the null, even when b1 is significant.

Confidence Intervals

• A confidence interval (CI) shows the range of true values of β1, consistent with the observed coefficient estimate.
• The likelihood that the true population parameter falls within some range is shown.
• Reject H0: β1 = 0 if the confidence interval for β1 does not contain zero.
• C.I. = b1 ± tcrit × se(b1)
• For a 90% confidence level, use a critical value of 1.64, so the interval is b1 ± 1.64 × se(b1).
• For a 95% confidence level, use a critical value of 1.96, so the interval is b1 ± 1.96 × se(b1).
• For a 99% confidence level, use a critical value of 2.58, so the interval is b1 ± 2.58 × se(b1).