Paired Samples T-Test & Cohen's d

• Sets up to handle data in long form
• Expects two separate variables, x and y
• Requires specifying paired=TRUE
• Demands corresponding first elements of x and y, lacking an "id" variable
• Code example is provided for paired samples t test

Effect Size

• Cohen's d is a commonly used measure of effect size for a t-test
• Cohen's d is calculated by dividing the difference between means by an estimate of the standard deviation
• $d = \frac{\text{(mean 1) - (mean 2)}}{\text{std dev}}$
• Interpreting Cohen's d is aided by a rough guide as interpreted in the next section

Cohen's d from one sample measures

• Cohen's d formula: $d = \frac{X - μ_0}{\hat{\sigma}}$
• x represents a numeric vector containing the sample data
• mu represents the mean against which the mean of x is compared; default value is mu = 0
• Example R code includes data stored in grades vector to compare against a mean of 67.5
• Results indicate psychology students achieving grades (mean = 72.3%) about .5 standard deviations higher than expected (67.5%), a moderate effect size

Cohen's d from a Student t test

• Hedges' g statistic is the most used version of Cohen's d in the context of a student t-test, corresponds to method = "pooled" in the cohensD() function, and is the default
• True population effect size is calculated with the means of both populations and the standard deviation: $\delta = \frac{\mu_1 - \mu_2}{\sigma}$
• Sample Cohen's d is calculated with the means of the samples and the pooled standard deviation: $d = \frac{\bar{X_1} - \bar{X_2}}{s_p}$
• Other method options include:
  • Use only one of the groups as the basis for calculating the standard deviation
  • Omission of bias correction during the usual calculation of pooled standard deviation
  • Introducing a small correction

Cohen's d from a Welch test

• Cohen advises averaging the two populations variances with the formula $\delta' = \frac{\mu_1 - \mu_2 }{\sigma'}$
• $\sigma' = \sqrt{\frac{\sigma^2_1 + \sigma^2_2}{2}}$
• All that is done to calculate d for this version (method = "unequal") is substitution of the sample means $X_1$ and $X_2$ and the corrected sample standard deviations $\hat{\sigma}_1$ and $\hat{\sigma}_2$ into the equation for $\delta'$
• Formula for d is $d = \frac{\bar{X_1} - \bar{X_2}}{\sqrt{\frac{\hat{\sigma}^2_1 + \hat{\sigma}^2_2}{2}}}$

Cohen's d from a paired-samples test

• Calculation simply uses method = "paired"
• $d = \frac{\bar{D}}{\sigma_D}$

Checking Normality of a Sample

• All tests assume data are normally distributed
• Central Limit Theorem often ensures real-world quantities are normally distributed, especially if variables are averages of many things
• Normality can be checked using QQ plots and the Shapiro-Wilk test

QQ Plots

• Quantile-Quantile plots visually check for systematic violations of normality
• Plotted observations as single dots with x as theoretical quantile, and y as sample data quantile
• Data are normal when dots form a straight line

Shapiro-Wilk Tests

• A formal test that checks to see if a sample is normally distributed.
• Null hypothesis W tests if a set of N observations is normally distributed.
• $W = \frac{(\sum_{i=1}^{N} a_i X_i)^2}{\sum(X_i - \bar{X})^2}$

Testing non-normal Data with Wilcoxon Tests

• Runs when a t-test is undesired because data is substantively non-normal
• Tests come in one-sample and two-sample forms
• Tests assume no specific distribution, making them nonparametric
• The tests are usually less powerful than the t-test to create a higher Type II error rate

Summary of Key Points

• One sample t-tests are used to compare a single sample mean against a hypothesised value for the population mean
• Independent samples t-tests compare the means of two groups, testing the null hypothesis for that they have the same mean
  • Student t-tests assume groups have the same standard deviation
  • Welch t-tests do not
• Paired samples t-tests are used with two scores from each person, testing the null hypothesis that the two scores have the same mean; the test is equivalent to taking the difference between the two scores for each person, and then running a one sample t-test on the difference scores
• Calculation of effect size can be calculated with the Cohen's d statistic
• Check for normality of a sample using QQ plots and Shapiro-Wilk test
• Use Wilcoxon tests instead of t-tests it data are non-normal

Learn how to conduct a paired samples t-test. Understand Cohen's d for effect size, including calculation and interpretation. Includes paired data and one-sample measures.