Hypothesis Testing Explained

Introduction

• Hypothesis testing provides an objective decision making framework using probabilities, instead of subjective feelings
• It offers a uniform and consistent approach to decision-making
• In one-sample problems, hypotheses are about a single distribution
• Two-sample problems involve comparison between two different distributions

Key Hypotheses

• The null hypothesis (Ho) is the hypothesis to be tested
• The alternative hypothesis (H₁) contradicts the null hypothesis
• Only possible decisions are considered: whether Ho is true or H₁ true
• Outcomes in hypothesis testing generally refer to the null hypothesis

Outcomes and Errors in Hypothesis Testing

• If Ho is true and we decide Ho is true, we accept Ho
• If H₁ is true and we decide H₁ is true, we reject Ho
• A Type I error the probability of rejecting the null hypothesis when it is actually true, is denoted by α It's the significance level of a test
• A Type II error is the probability of accepting the null hypothesis when the alternative hypothesis H₁ is true, is denoted by ß
• The power of a test is the probability of rejecting Ho when H₁ is true, calculated as 1 - ß
• Aim: Use statistical tests to minimize α and ß, thus minimizing errors in rejecting or accepting the null hypothesis

One-Sample Test for the Mean of a Normal Distribution with One-Sided Alternatives

• Acceptance region: Values of x for which Ho is accepted
• Rejection region: Values of x for which Ho is rejected
• If x is sufficiently smaller than µ0, then Ho is rejected; otherwise, Ho is accepted.
• When Ho is true: x values cluster around µo
• When H₁ is true: x values cluster around μ₁
• A test based on sample mean has the highest power, given a type I error of α
• A one-tailed test allows values under the alternative hypothesis to be either greater or less than the null hypothesis, but not both

One-Sample z Test (One-Sided Alternative)

• To test Ho: μ = μo vs. H₁: μ < μo, with a significance level of α and known standard deviation σ, the test statistic is z = (x - μo)/(σ/√n)
• If z < zα, Ho is rejected; if z ≥ zα, Ho is accepted
• The p-value is given by p = Φ(z)
• To test Ho: μ = μo vs. H₁: μ > μo, with a significance level of α and known standard deviation σ, the test statistic is z = (x - μo)/(σ/√n)
• If z > z1-α, Ho is rejected; if z ≤ z1-α , Ho is accepted
• The p-value is given by p = 1 - Φ(z)

One-Sample z Test (Two-Sided Alternative)

• To test H₀: μ = μ₀ vs. H₁: μ ≠ μ₀ with a significance level of α and known standard deviation σ, the test statistic is z = (x - μ₀) / (σ/√n)
• If z < zα/2 or z > z1-α/2, then H₀ is rejected.
• If zα/2 ≤ z ≤ z1-α/2, then H₀ is accepted.
• To compute a two-sided p-value: p = 2Φ(z) if z ≤ 0; p = 2[1 - Φ(z)] if z > 0

One-sample t Test (Alternative Mean < Null Mean)

• To test H₀: μ = μ₀, σ unknown vs. H₁: μ < μ₀, σ unknown with a significance level of α: t = (x - μ₀) / (s/√n)
• If t < tn-1,α, then reject H₀.
• If t ≥ tn-1,α, then accept H₀.
• The value of t is called a test statistic, the value of tn-1,α is called a critical value
• The general approach is to compute a test statistic and compare it with a critical value; this is the critical-value method

P-Value Significance

• The p-value is the α level at which we would be indifferent between accepting or rejecting H₀
• It is the α level at which the given value of the test statistic (such as t) is on the borderline
• Solving for p gives p = Pr(tn-1 ≤ t)
• p-value is the probability of obtaining a test statistic as extreme as or more extreme than the one obtained, assuming the null hypothesis is true
• p-value indicates the significance of results, without repeated significance tests at different α levels
• If 0.01 ≤ p < 0.05, results are significant
• If 0.001 ≤ p < 0.01, results are highly significant
• If p < 0.001, results are very highly significant
• If p > 0.05, results are not statistically significant (NS)
• If 0.05 ≤ p < 0.1, a trend toward statistical significance may be noted

Methods to Establish Statistical Significance

• Compute the test statistic t and compare with the critical value tn-1,α at an α level of 0.05
• If Ho: μ = μo vs. H1: μ < μo is tested and t < tn-1,0.05, then Ho is rejected (statistically significant with p < 0.05); otherwise Ho is accepted (not statistically significant with p ≥ 0.05); this is the critical-value method
• Compute the exact p-value; if p < 0.05, then Ho is rejected and the results are statistically significant; otherwise Ho is accepted (results not statistically significant); this is the p-value method

One-Sample t Test (Alternative Mean > Null Mean)

• To test Ho: μ = μo vs. H₁: μ > μo with significance level α, the test statistic is based on t = (x - μo)/(s/√n)
• If t > tn-1,1-α, then Ho is rejected
• If t ≤ tn-1,1-α, then Ho is accepted
• The p-value for this test is given by p = Pr(tn-1 > t)

One-Sample Test with Two-Sided Alternatives

• A two-tailed test values under the alternative hypothesis are allowed to be either greater or less than the values under the null hypothesis (µo)
• To test for alternatives on either side of the null mean, reject Ho if t < c₁ or t > c₂ (c1, c2 are constants)
• Accept Ho if c₁ ≤ t ≤ c₂
• Pr(reject Ho | Ho true) = Pr(t < c₁ | Ho true) + Pr(t > c₂ | Ho true) = α
• Pr(t < c₁ | Ho true) = Pr(t > c₂ | Ho true) = α/2

One-sample t Test (Two-Sided Alternative)

• To test the hypothesis H₀: μ = μ₀ vs. H₁: μ ≠ μ₀ with a significance level of α, the test statistic used is t = (x̄ - μ₀) / (s/√n)
• Reject H₀ if |t| > tn-1,1-α/2
• Accept H₀ if |t| ≤ tn-1,1-α/2

P-Value (Two-Sided Alternative)

• Let t = (x - μ0) / (s/√n)
• p = 2 × Pr(tn-1 ≤ t), if t ≤ 0
• p = 2 × [1 - Pr(tn-1 ≤ t)], if t > 0
• The p-value is the probability under the null hypothesis of obtaining a test statistic as extreme as or more extreme than the observed test statistic
• Extremeness is measured by the absolute value of the test statistic

One-Sided vs. Two-Sided Tests

• The sample means falls in the expected direction from µ₀ and it's easier to reject H₀ using a one-sided test
• A two-sided test does not require guessing the appropriate side of the null hypothesis for the alternative hypothesis.
• If only alternatives on one side of the null mean are of interest or possible, a one-sided test is better because it has more power
• Decision (one-sided or two-sided test) must be made before data analysis or collection to avoid biased conclusions.
• Avoid changing from a two-sided to a one-sided test after looking at the data.

Power of a Test

• The power of a test indicates the likelihood of detecting a statistically significant difference when the alternative hypothesis is true.

Formula

• (x - μ₀) / (σ/√n) > z1-α; multiplying both sides with σ/√n and adding μ₀: x > μ₀ + z1-α σ/√n
• Alternative formula x ≤ μ₀ ± z1-α σ/√n

Power for the One-Sample z Test (One-Sided Alternative)

• For the hypothesis H₀: μ = μ₀ vs. H₁: μ = μ₁, assuming a normal distribution and known population variance (σ²): Φ(zα + |μ₀ - μ₁| √(n)/σ) = Φ(-z1-α + |μ₀ - μ₁|√(n)/σ)
• Significance Levels: Smaller significance levels (α decreases) reduces test power
• Alternative Mean: Greater difference between the alternative and null means (|μ₀ - μ₁| increases) increases test power
• Standard Deviation: Larger standard deviation in individual observations (σ increases) decreases test power
• Sample Size: Larger sample sizes (n increases) increases test power

Power for the One-Sample z Test (Two-Sided Alternative)

• Power of the two-sided test Ho: μ = μo vs. H₁: μ ≠ μo with the specific alternative μ = μ₁, where the underlying distribution is normal and the population variance (σ²) is assumed known exactly by
• Φ[-z1-α/2 + ((μo - μ₁) √n) / σ] + Φ[-z1-α/2 + ((μ1 - μo) √ n) / σ]
• Or Approximately by Φ[-z1-α/2 + (μo-μ1)√n / σ]

Sample-Size Determination: One-Sided Alternatives

• For testing Ho: μ = μo vs. H₁: μ = μ₁ with normal data with mean µ and known variance σ², the necessary sample size for a one-sided test with significance level α and the defined Power :
• n = (σ² (z1-β + z1-α)² / (μ₀ - μ₁)²)

Factors Affecting Sample Size

• The sample size increases as σ² increases
• The sample size increases as the significance level is made smaller (α decreases)
• The sample size increases as the required power increases (1- β increases)
• The sample size decreases as the absolute value of the distance increases

Sample-Size Estimation (Two-Sided Alternative)

• For testing H₀: μ = μ₀ vs. H₁: μ = μ₁ with normal data with mean µ and known variance σ², the necessary sample size for a one-sided test with significance level α and the defined power :
• n = (σ² (z1-β + z1-α/2)² / (μ₀ - μ₁)² )

Sample-Size Estimation Based on CI Width

• To estimate the mean of a normal distribution with sample variance s²: n = 4z²1-α/2 s² / L²
• The number of subjects needed for the two-sided 100% × (1 - α) Confidence Interval be no wider than L

Hypothesis Testing and Confidence Intervals Relationship

• H₀: μ = μ₀ vs. H₁: μ ≠ μ₀ is rejected with a two-sided level α test if the two-sided 100% × (1 - α) Confidence Interval for μ does not contain μ₀
• H₀: μ = μ₀ vs. H₁: μ ≠ μ₀ is accepted with a two-sided level α test if the two-sided 100% × (1 - α) Confidence Interval for μ contains μ₀
• Rejection region is a two-sided level a test

Bayesian Inference

• We need to specify a prior distribution for μ in order to use that approach
• A prior distribution generalizes the concept of a prior probability: where a probability distribution of a possible values for a specified before looking at the available sample data
• This distribution is then modified after the data is collected to find a posterior for μ
• In the absence of data, there is no knowledge of µ, so all possible values of u are equally likely This is called flat or noninformative prior distribution and is written as Pr(μ) ∝ c, where c is a constant
• If a distribution is normal and σ is known, then all the information in the sample concerning μ is contained in x, that is, Pr(μ|x1,...,xn) = Pr(μ|x) -

Bayesian Formula

• X is referred to as a sufficient statistic for μ
• distribution, we use Baye's rule. Pr(μ|x) = Pr(x | μ)Pr(μ)/Pr(x).
• the prior probability of µ, Pr(u), is the same for all values of μ
• Pr(μ|x) x Pr(x| μ)
• approximation for a continuous distribution
  • f(x|μ) =[1/(√2π σ/√n)]exp{-(1/2)[(x̄ -μ)/(σ/√n)]²} = [1/(√2π σ/√n)]exp{-(1/2)[(μ-x̄)/(σ/√n)]²]
• Pr(μ | x) ∝ [1/(√2π σ/√n)]exp{-(1/2)[(μ - x̄)/(σ/√n)]²}
• The distribution of µ given x is normally distributed with mean = x and variance σ²/n, or μ|x~ N(x, σ²/n)

Bayesian Distribution Formula

• A 100% × (1 - a) posterior predictive interval for u: Pr(μ₁ < μ < μ₂) = (1 - α) where μ₁ = x̄ - z1- α/2 σ/√n and μ₂ = x̄ + z1- α/2 σ/√n

One-Sample x² Test (Two-Sided Alternative)

• The test statistic X² = ((n − 1)s²) / σ₀²
• if X² < X²n-1,α/2 or X² > X²n-1,1- α/2, Ho is rejected
• If X²n-1,α/2 ≤ X² ≤ X²n-1,1- α/2 then Ho is accepted

P-Value for a One-Sample x² Test (Two-Sided Alternative)

• Test statistic X² = [(n-1)s²]/σ₀²
• If s² ≤ σ₀² then p-value = 2 × (area to the left of X² under a X²n-1 distribution)
• If s² > σ₀² then p-value = 2 × (area to the right of X² under a X²n-1 distribution)

One-Sample Inference for the Binomial Distribution: Normal-Theory Methods

• Test statistic z = (p^- po/√poqo/n
• if z < za/2 or z > Z1-a/2 then Ho is rejected
• if za/2 ≤ z ≤ Z1-a/2 then Ho is accepted
• Should only be used if npoqo ≥ 5

Computation of the p-value (Two-Sided Alternative)

z (P^-Po/Pogo/n

• Statistic Test
• If P^≤ Po, then value = 2xÞ(z) = twice the area to the left of z under an N(0,1) curve
• If P> Po, then z-value = 2x[1-Þ(z)] = twice the area to the right of z under an N(0,1) curve.

One Sample Poisson Distribution

• An index used to quantify a populations risk relative to the another population is the standardized mortality ration (SMR)
• It is defined by 100% x O/E = 100% * the observed number of deaths / the expected number of deaths under the assumption that the populations have the same mortality rates
• For nonfatal conditions the SMR is known as the standardized morbidity ratio

Power for the One-Sample Binomial Test (Two-Sided Alternative) Formula

• Ho: p = po vs. H₁: p ≠ po: Φ[√Polo/P191 (za/2 + |Po-P1|/√pogo)]

Formula

• • Test- 2 =(x - μ₂) for a two sided test. 2=H₂ is rejected if
• IfHo 42.²²-a is given by Pr(k²>,)
• This text should only be used if μ ₁, 10

Summary

• specification of the null and alternative hypotheses
• type I error, type II error, and the power of a hypothesis test; the p-value of a hypothesis test; distinction between on-sided and two sided tests
• methods for estimating appropriate sample size as determined by the prespecified null and alternative hypotheses and type I and type II errors
• One-sample hypothesis-testing cases
• hypothesis tests are conducted through
  • Specifying critical values to determine the acceptance and rejection regions based on a specified type I error a
  • Computing p-values.
• Bayesian inference methods
  • when no prior information exists concerning a parameter
  • when a substantial amount of prior information is available.