Power & Sample Size

Core Principles of Mental Health Research: Power & Sample Size

• Alpha Criterion & Hypothesis Test:
  • Alpha (α) = 0.05 represents a 5% chance of a Type I error (rejecting a true null hypothesis).
  • Type I error is rejecting the null hypothesis when it is actually true.
  • The red lines in the graph represent 1.96 standard deviations above and below the mean.
  • The graph is centered on 0, meaning the null hypothesis is true.

Alternative Hypothesis:

• The purple line in the graph represents the sampling distribution for a mean difference in the case where the null hypothesis is true, but the alternative hypothesis could also be true.
• The true difference between groups in the alternative hypothes is 6 points (the value is assumed, not necessarily a fixed value).
• The alternative hypothesis is centered around the point estimate of 6 points.

Power:

• Power is the probability of correctly rejecting a false null hypothesis.
• Pink area = test power.
• Green area = previous Type I error area, now contributes to power
• Yellow area = Type II error (50% chance in a study).
• Factors affecting power:
  • Significance level (alpha): Higher alpha increases power but also increases the risk of a Type I error.
  • Effect size: Larger differences between groups are easier to detect.
  • Standard deviation (SD): A smaller SD allows detection of smaller effects.
  • Sample size: Larger samples provide more power.

Continuous Outcomes – Comparison of Two Means

• Power is calculated to determine the required sample size, not the other way around.
• The sample size per group calculation is: n=(Z1-a/2+21-β)²×(2σ²)/(μ₂-μ₁)²
  • μ₂-μ₁ = true difference in means
  • σ = standard deviation
  • a = significance level
  • 1 - β = power
  • Z = Z score from a normal distribution

Binary Outcomes – Comparison of two proportions

• Sample size calculation for comparing proportions: n=(Z1-a/2+21-β)² X (p1(1-p1)+p2(1-p2))/(p2-p1)²
  • p₁ = true proportion in control
  • p₂ = true proportion in treatment

More Complicated Scenarios

• Loss to Follow Up: Follow-up loss needs to be accounted for during sample size planning; 25% loss to follow up is a common scenario.
• Unequal Treatment Group Sizes: Different group sizes may impact the required sample size.
• Baseline Adjustment: Baseline characteristics may impact the required sample size (often reducing it in correlations.)
• Hierarchical Data Structures: More complex data structures require adjustments to the sample size calculation that account for intraclass correlation.

Avoiding Post-Hoc Power Calculations

• Post-hoc power calculations are generally not meaningful; it doesn't determine why the study had a higher/lower than expected power.
• A well-planned approach should include the expected effect/group sizes/SD in power calculations before launching a study.

This quiz covers the fundamental elements of mental health research, focusing on concepts such as the alpha criterion, hypothesis testing, power, and sample size. Explore how Type I errors impact research outcomes and the significance of power in hypothesis testing. Perfect for students delving into research methodologies in psychology.