Hypothesis Testing Overview

Hypothesis Testing

• A hypothesis test uses sample data to evaluate a claim about a population.
• First, state a hypothesis about the population parameter.
• Next, obtain a random sample from the population.
• Compare the sample data with the prediction from the hypothesis.
• If the sample data is consistent with the prediction, the hypothesis is reasonable.
• If there's a large discrepancy between sample data and prediction, the hypothesis is likely wrong.
• The goal is to determine if the treatment has an effect on the population.

Setting Criteria for Decision

• The researcher uses sample data to evaluate the credibility of the null hypothesis.
• Data either supports or refutes the null hypothesis.
• Big discrepancy between data and hypothesis suggests a flawed hypothesis.

The Alpha Level

• The alpha level (significance level) is the probability of making a Type I error.
• It's the probability of incorrectly rejecting a true null hypothesis.
• Common values include 0.05 (5%), 0.01 (1%), or 0.10 (10%).
• A lower alpha level means a stricter criterion for rejecting the null hypothesis.
• Alpha level of 0.05 represents a 5% risk of a Type I error.

Critical Region

• Critical region is the set of values for a test statistic that leads to rejecting the null hypothesis.
• If the calculated test statistic falls within the critical region, the observed data is significantly extreme under the null hypothesis, leading to rejection.
• For example, an alpha level of 0.05 means 5% of the distribution falls in the critical region.

Type I Error (False Positive)

• Occurs when a researcher rejects a true null hypothesis.
• In a typical research situation, a Type I error means a treatment was concluded to have an effect when it actually doesn't.
• The example of a drug failing to work, though is given as a placebo.

Type II Error (False Negative)

• Occurs when a researcher fails to reject a false null hypothesis.
• A Type II error means a real treatment effect goes undetected.
• The example provided is a psychiatrist failing to recognize an effect.

Hypothesis for Directional Test

• A directional test (one-tailed test) specifies the expected direction of the effect or relationship between variables.
• One-tailed tests focus on a specific direction in a distribution, either positive or negative.

One-Tailed Test

• Used when there is a strong theoretical basis suggesting an effect in a particular direction.

Two-Tailed Test

• Used when any effect (positive or negative) is of interest.

Effect Size

• A numerical value expressing the strength of the relationship between variables or the size of group difference.
• A large effect size indicates practical significance of findings, while a small effect size suggests the finding may have limited real-world application.
• Cohen's d measures effect size. Values are described as small (0.2), medium (0.5), or large (0.8).