Within-Subjects Design & Dependent-Samples T-Tests

• Study notes for Research Methods in Psychology
• Focus is on within-subjects design and dependent-samples t-tests
• Presented March 10th, 2025

Recap from Week 5

• Variance indicates how spread out data is in statistics
• Analysis of Variance (ANOVA) compares group variation to individual variation
• Determines if groups are statistically different, on average
• Homogeneity of variance means groups' variances are similar enough to use ANOVA
• Assessed through Levene's test
• A significant difference between groups is indicated by a low p-value (p < 0.05)
• Larger F scores indicate bigger group differences

Within-Subjects Design

• Each participant acts as their own control
• This experimental setup involves the same participants undergoing different treatments
• The same participants experience different levels of a particular independent variable
• Participants may receive both Treatment A and Treatment B
• Participants may undergo the same intervention at multiple points in time, such as at 0 months and 6 months where time is the independent variable

Combining Between and Within-Subjects Designs

• Differential research design is non-experimental
• Posttest-only non-equivalent control group design is non-experimental
• Pretest-posttest non-equivalent control group design is quasi-experimental

Pretest-Posttest, Non-Equivalent, Control Group Design (Quasi-Experimental)

• Involves comparing two non-equivalent groups which are measured twice before and after a treatment

• Assignment bias is reduced using pre and post measurements.

• Differential history, instrumentation, testing, or maturation effects are potential problems

• Differential History Effects: Study-specific events might influence the outcomes

• Differential Instrumentation: Changes in instruments, observers, or scorers can alter outcomes

• Differential Testing Effects: The pre-test itself can affect the outcomes of the second test

• Differential Maturation: Maturational effects are about change over time in participants

Preventing Testing Effects

• Question-behaviour effect: Simply asking a question might change behaviour
• The Solomon four-group design addresses the question-behaviour effect (QBEs)
• Allows for controlling biases from pre-test measurement
• Useful for studies measuring pre- and post-intervention attitudes

One-Factor, Two Treatment Design

• This is the most basic form of within-subjects design
• This model includes only two levels of one independent variable
• All subjects participate in both treatment conditions
• Examples are clinical studies tracking success rates before (PRE) and after (POST) therapy
• Equal interval data might be analyzed using a parametric dependent t-test

Advantages of Within-Subjects Designs

• The design is sometimes the only feasible option
• It allows for comparing related events within the same individual
• Good for controlling subject variation
• It reduces the number of subjects needed, making it more economical

Disadvantages of Within-Subjects Designs

• Habituation can lower reactivity to a stimulus from repeated exposure which could corrupt results
• Order effects such as practice or fatigue are also a concern
• Performance may improve in the second condition due to practice
• Performance may decline in the second condition because of fatigue
• Counterbalancing can mitigate order effects

Procedure Example

• 8.45am: Participant is briefed and can ask questions
• 9am: Sensors are attached, participant practices driving
• 9.30am: First automated driving period takes place, baseline measures are recorded
• 9.40am: Non-driving related task (NDRT), drive with a 'match 2' memory game on a tablet
• 10.05am: Break
• 10.15am: Fatigue condition, prolonged automated drive without a task
• 11.05am: Break
• 11.15am: Final driving session without visible sensors
• 11.30am: Finish

T-Test for Dependent Means

• Computed when there are two sets of scores for each subject
• Used for repeated-measures, within-subjects, matched/paired samples, before-and-after, and correlated/related samples designs

Assumptions of a T-Test

• Variables should be normally distributed
• Data needs to be randomly sampled
• Data must be measured at equal intervals (NOIR)
• No check for homogeneity of variance is necessary as we are dealing with the same group

Interpreting Results of a Dependent-Samples T-Test

• Research question example: Does blood pressure change after an exercise programme?
• Look at the mean scores (M) before and after
• Look at the standard deviation scores (SD) before and after
• Need the t score (t)
• Find the degrees of freedom (df)
• Read the p-value (p)

The T Value

• The t score is a ratio between the difference between two groups and the difference within the groups
• Higher t scores indicate more difference between groups
• Lower t scores indicate more similarity between groups
• A t score of 3 means groups differ by three times as much as they do within each other
• Higher t-values in a t-test indicate a more repeatable result

Degrees of Freedom

• Degrees of freedom (df) defines the number of values in a dataset having the freedom to vary or a lack of constraint in a particular dataset
• Usually, it's the sample size minus the number of parameters you need to calculate during an analysis which is usually a positive, whole #
• Example: Choosing shirts to wear during the week, freedom decreases with each day's choice

Reporting T-Test Results

• t(7) = 3.47, p = .010 indicates a significant difference
• To explain the results, look at the mean (average) scores for the two time points to decide whether blood pressure was higher or lower after the programme
• Example summary: "Blood pressure was lower (t(7) = 3.47, p = .010) after participants completed the exercise programme (M = 137.75, SD = 12.08) compared to before (M = 147.25, SD = 14.72)."
• The above is an appropriate report of results based on APA standards