Understanding T-Tests

Overview of T-Tests

• T-tests assess differences in means between two datasets.
• These datasets are compared using a t-distribution.
• The t-distribution is similar to a normal distribution but with thicker tails, suitable for smaller sample sizes.
• Developed by William C. Gossett, the t-test is also known as the Student's t-test, which accounts for bias in estimating the standard error of the mean with smaller samples.
• A calculated t-statistic is compared against a critical t-value to determine statistical significance.

Logic Behind T-Tests

• If two samples from the same population have means that differ significantly from what's expected, one of two conclusions can be made:.
  • The samples were not drawn randomly
  • Another factor affected one or both samples.
• Reject the null hypothesis if the second conclusion is reached, suggesting a true difference in means.

Why Use a T-Distribution Instead of a Z-Distribution?

• Z-scores are inaccurate with small sample sizes.
• T-distributions have thicker tails compared to normal distributions.
• As degrees of freedom (sample size) increase, the t-distribution approaches a normal distribution.

Interpreting T-Tables

• T-tables provide critical t-values based on degrees of freedom and probability levels.
• Degrees of freedom are listed ascending on the left side of the table.
• Probability or cumulative probability values are listed across the top.
• For a two-tailed t-test, an alpha level of 0.05 is commonly used, allowing for a 5% chance of error.
• Compare the calculated t-statistic to the critical t-value; reject the null hypothesis if the t-statistic is larger.
• Example: A sample size of 20 results in 19 degrees of freedom (n-1); at an alpha level of 0.05, the critical t-value is 2.093.

Calculating a T-Statistic

• The basic form of a t-test is a ratio of the mean difference over the standard error of the mean difference.
• The numerator is the difference between the means of the two samples being compared.
• The denominator is the standard error of that difference.
• The t-ratio can be viewed as a signal-to-noise ratio, where the signal is the mean difference and the noise is the standard error.
• A higher t-statistic results from a larger mean difference and/or a lower standard error of the mean difference.

Assumptions for Running a T-Test

• The population from which the sample is drawn should be approximately normal.
• Samples must be drawn randomly.
• There needs to be homogeneity of variance (samples having similar variances); one sample's variance should not be more than two times larger than the other.

Types of T-Tests

• Single Sample T-Test:
  • Compares a single sample against a known population mean.
  • Example: Comparing test scores of 30 students in an accelerated program to the known mean of 10,000 students at the university.
• Independent Samples T-Test:
  • Compares two independent samples against each other.
  • Example: Comparing strength values between a group of soccer players vs. a group of basketball players.
• Dependent Samples T-Test:
  • Involves a correlated sample, often with pre- and post-testing of the same subject under two different conditions.
  • Example: Testing a group's strength levels before and after a 12-week weight training program.

Single Sample T-Test in Detail

• Compares a sample mean to a known population mean by calculating the ratio of the actual mean difference to the expected mean difference.
• The expected difference is due to chance alone.
• Standard error of the mean is calculated as the standard deviation divided by the square root of the sample size.
• Example: A kinesiology department tests a new volleyball serve teaching method, comparing a class of 30 students (n=30) to the historical average score of 31 out of 60. Calculate:
  • The Sample mean is 35:
  • Degrees of freedom are 29 (n-1.)
  • Standard deviation is 8.3.

Single Sample T-Test Example Continued

• The t-statistic is calculated as (sample mean - population mean) / standard error of the mean, resulting in 2.63.
• The standard error of the mean is the standard deviation (8.3) over the square root of the sample size (30), which equals 1.52.
• If the critical t-value from the t-table is less than 2.63, the results are statistically significant at the alpha level of 0.05.
• Conclusion: This new technique is beneficial, because there is a statistically significant difference in the scores.

Confidence Intervals

• Confidence intervals can be computed around the calculated mean to determine if the population mean falls within the interval.
• For a 95% confidence interval, use the equation: mean ± (critical t-value * standard error of the mean).
• Example: With a mean of 35, a critical t-value of 2.045, and a standard error of 1.52, the 95% confidence interval is 31.89 to 38.11.
• If the confidence interval range is entirely above the previous mean of 31, then the null hypothesis can be rejected.

Independent Samples T-Test in Detail

• Tests whether two samples are drawn from the same population or different populations.
• Usually involves two different groups of people.
• Example: An athletic trainer compares two ankle sprain treatments by measuring ankle volume in milliliters between patients receiving standard care vs. a new treatment.
• Uses the standard error of the difference instead of the standard error of the mean.
• The standard error of the difference factors in standard deviation and group size from both groups.
• The t-statistic is the difference in means over the standard error of that difference.

Independent Samples T-Test Calculation Details

• To determine if the t-statistic is greater than the critical t-value, calculate degrees of freedom.
• Degrees of freedom are calculated as (n1 - 1) + (n2 - 1), which is different from the single sample t-test.
• With 15 individuals in each group, the degrees of freedom are (15-1) + (15-1) = 28.

Critical t-value and Hypothesis Testing

• For 28 degrees of freedom and a 0.05 alpha level, the critical t-value is 2.048.
• A calculated t-value of 2.15 is greater than 2.048, which means the null hypothesis is rejected.
• Rejecting the null hypothesis suggests a difference between groups.
• The t-statistic does not identify the causative factor.
• Careful controls and experimental design are needed to identify the causative factor.
• Monitor the control group closely to ensure chance is the only operating factor.
• Only permit one independent variable to influence the experimental group to identify the causative factor.
• Statistical analysis quantifies the association between the independent and dependent variables, but does not tell you exactly what the cause was.

Confidence Intervals

• A confidence interval is needed around the change (or difference) in x.
• The critical t for a two-tailed test at 0.05 with 28 degrees of freedom is 2.048.
• Calculate the difference in means, then add and subtract the critical t-value multiplied by the standard error of the difference.
• A 95% confidence interval calculation revealed a 0.09 to 3.5 milliliter difference between treatments.
• This implies the two samples represent different populations as the interval excludes zero.
• With 95% confidence the true mean difference between populations is between 0.09 and 3.5 milliliters.
• For a 99% confidence interval, use the 0.01 alpha level on the t-table, which leads to a wider confidence interval.

Unequal Sample Sizes

• Calculations change when sample sizes are unequal.
• Statistical programs like Excel, SPSS, JASP, or R are typically used for these t-tests.
• The equation has to account for the two different sample sizes.
• If n's are equal then you get a standard error of the difference.

Repeated Measures or Paired Samples T-Tests

• Used when testing subjects more than once (pre-test vs. post-test or comparing multiple treatments or conditions within the same subjects).
• These dependent samples assume a correlation exists between a subject's pre-test and post-test scores.
• The Pearson r correlation coefficient factors into the standard error of the differences.
• A larger r (correlation) results in a smaller standard error of the difference.
• Greater correlation between pre-test and post-test scores results in smaller the standard error of the difference.

Bicycle Tour Example

• A table showed the effects of a four-day bicycle tour on self-esteem.
• Self-esteem pre-test mean: 38.4, post-test mean increased slightly.
• Question: Was this statistically significant at an alpha level of 0.05 or 0.01?
• A high correlation of 0.92.
• The t statistic: -6.22.
• Calculate the t statistic by calculating the standard error of the difference and taking into account the standard error of the mean one and the standard error of the mean two.
• To calculate the degrees of freedom for a paired samples t test is not the same, there is one extra degree of freedom for each person in the group.
• Is there a statistical difference between the pre and post test results? The t value is -.622, so we can reject the null hypothesis.
• A paired sample's t statistic can also be calculated using the difference in scores over the standard error of the difference, recalling that the standard error of the difference is the standard deviation over square root of n.

Training Example

• Another paired samples t-test example with 10 weeks of training affecting VO2 max.
• Most scores are positive, indicating a statistically significant difference.

Calculating Statistical Significance

• Degrees of freedom is 7, requiring a larger t-statistic for statistical significance.
• Calculate t as the mean difference (post-tests - pre-tests) over the standard error of the mean difference (which is standard deviation over square root of n).
• The critical t-value at 0.05 with 7 degrees of freedom is 2.365.
• 2.365 is less than 2.5, suggesting that the training worked at the alpha level of 0.05.

Confidence Interval

• Take the difference in means plus or minus the critical t-value times the standard error of the difference.
• It was found the mean difference was between 0.2 to 6.4 milliliters per kilogram per minute with 95 percent confidence.
• The range shows that the null value of 0 is outside of that confidence interval.

Magnitude of Difference

• Statistical significance, as determined by the t statistic, does not determine the size or meaningfulness.
• Large sample sizes and small standard deviations (especially paired samples) can lead to statistically significant values between close means.
• A very small difference between two samples can produce a statistically significant result if there is a huge sample size, but not enough to be meaningful - also known as clinical significance.

Omega Squared

• Estimates the percentage of variance explained by the independent variable's influence.
• Analogous to R-squared.
• Calculated as t squared minus one over t squared plus both of the sample sizes minus one.

Effect size

• Similar to a z-score, so it is standardized.
• Calculated as the mean difference over the standard deviation of the control.
• If the mean from one group, minus the mean from the other group, is taken over the standard deviation we find the effect size.