week 7 & 8

Between-subjects Methods

• Two Types of Experimental Designs:

  • Between-subjects: Different participants in each condition. Also known as "independent measures."
  • Within-subjects: Same participants in each condition. Also known as "repeated measures."

• Data in Between-subjects:

  • Only 1 score per participant.

• Statistical Analysis for Between-subjects:

  • 2 conditions: Independent samples t-test.
  • 3+ conditions: One-way ANOVA

Advantages of Between-subjects Designs

• Not influenced by time-related factors (history effects, maturation)
• Not influenced by order effects (practice, fatigue)

Disadvantages of Between-subjects Designs

• Requires larger number of participants (problem with special populations)
• Vulnerable to confounds (threats to internal validity):
  • Individual differences: Unique characteristics of each participant
  • Environmental variables: Changes in the testing environment, e.g., temperature, noise
• Difficult to avoid selection bias (creating equal groups)

Avoiding Selection Bias in Between-subjects Designs

• Restricted randomisation: Creating equal groups.
  • Hold variables constant: Restricting the range of a variable.
    • Example: Only male participants.
  • Match for a potential confound: Participants are matched based on a potential confound variable.
    • Example: Matched based on age or IQ.
• Block randomisation: Assign participants to groups randomly, but ensuring that older participants are evenly distributed across groups.
  • Example: Participants are randomly assigned to groups based on the flip of a coin. If the oldest participants are assigned to group 1 by the coin flip, the oldest remaining participants are assigned to group 2, and so on.

Reducing Environmental Threats to Internal Validity in Between-subjects Designs

• Run participants at the same time of day.
• Use the same location for testing.

Other Threats to Internal Validity in Between-subjects Designs

• Differential attrition: Differences in attrition rates between groups.
  • Example: More participants drop out of one group, leaving samples of unequal size.
• Diffusion: The treatment being spread from the experimental group to the control group.
  • Example: Participants in the control group learn about the treatment being received by the experimental group and start to use it themselves.
• Compensatory equalisation: Participants demand similar treatment in the control group as the experimental group.
  • Example: The control group demands the same treatment as the experimental group.
• Compensatory rivalry: Participants in the control group try harder to perform well to prove they are as good as the treatment group.
  • Example: Control group members might work harder to compensate and show that they can perform just as well as those in the treatment group.
• Resentful demoralisation: Control group participants become less productive and motivated when they learn about the treatment received by the experimental group.
  • Example: Members of the control group become less productive, leading to a lower difference between groups than would otherwise be expected.

Within-subjects Methods

• Also known as: Repeated measures.

Advantages of Within-subjects Designs

• Removes or reduces threats from individual differences.
• No threat from selection bias (same participants).
• Avoids increased variance.
• Fewewr participants needed, which increases statistical power.

Disadvantages of Within-subjects Designs

• Vulnerable to environmental threats (e.g., time-related factors)
• Vulnerable to order effects (e.g., practice, fatigue).

Analysis of Within-subjects Designs

• 2 conditions: Paired-samples t-test (Wilcoxon for non-parametric data).
• 3+ conditions: Repeated-measures ANOVA (Friedman’s ANOVA for non-parametric data).

Time Threats in Within-subjects Designs

• History effects: Differences in results due to events that occur between measurements.
  • Example: News reports about a new treatment could affect participants' wellbeing.
• Maturation: Changes in results due to participants getting older or developing over time.
  • Example: Participants might become more experienced or mature, which could affect their performance on a task.
• Regression to the mean: Extreme scores (high or low) may be less extreme when measured again.
  • Example: Participants who score extremely well on a test initially may improve or not perform as well when tested on that same test again. This trend is not necessarily due to the intervention.
• Instrumentation: Changes in the measurement tools or procedures between measurements.
  • Example: Changing the scoring criteria for a test could influence the results.