IPR Term 2 Week 5 - T-tests continued: Independent and Paired samples t-tests (& the non-parametric equivalents: Mann Whitney U and Wilcoxonn signed rank test)

• A course on Independent T-Tests and Non-Parametric Equivalents with JASP software over two weeks.
• Lectures 4 and 5 cover t-tests and non-parametric equivalents.

Learning Outcomes

• Learn the purpose of t-tests
• Learn when to use t-tests.
• Understand key differences between one-sample, independent, and between-samples t-tests.
• Calculate all tests using JASP software.
• Interpret test results and write them up following APA standards.

Lesson 5 Outline

• Introduction to t-tests.
• Assumptions and the differences between parametric and non-parametric tests.
• One-sample t-test and the Wilcoxon signed-rank test.
• How to run a One-sample t-test and Wilcoxon signed-rank test in JASP.
• Independent t-test and the Mann-Whitney U test.
• How to run an Independent t-test and Mann-Whitney U test in JASP.
• Paired-samples t-test and the Wilcoxon signed-rank test.
• Running Paired-samples t-test and Wilcoxon signed-rank test in JASP.
• Focus on interpreting results and writing them up correctly.
• Red text indicates non-parametric equivalents.

Independent T-Test

• Used to determine if there is a statistically significant difference between the means of two independent groups.
• This test is part of the quantitative research approach.
• It uses an experimental design to manipulate variables and determine cause-and-effect relationships.
• A research question to explore is whether different types of exercise affect happiness.
• Compare happiness in two groups: one does aerobic exercise, the other does strength training.
• The goal is to determine if the difference in happiness scores between the two samples is significant.
• T-tests are used for two groups or conditions in your IV.
• It requires different participants in each condition.
• Parametric assumptions should be met to use the Independent t-test for between-subjects.
• If parametric assumptions are not met, the Mann-Whitney U Test may be used.

Hypothesis Testing

• Hypothesis testing steps are the same as for a One-Sample test
• Identify the IV and DV.
• Identify the hypothesis, which may be one-tailed or two-tailed.

Null Hypothesis (H0)

• There will be no difference in happiness between the groups.

Alternative Hypothesis (H1)

• There will be an effect or difference.
• In a one-tailed test, people doing aerobics will have higher happiness scores than those doing strength training.
• In a two-tailed test, there will be a difference in happiness between the two exercise groups, with no direction stated.
• The hypothesis should be either one-tailed or two-tailed, but not both.

Variables

• The independent variable is the level of exercise.
• Levels include aerobic and strength training.
• The dependent variable is the level of happiness.
• Measured using a happiness scale.

Experimental Design

• Involves different participants recruited into each condition.
• Determine if the difference in happiness scores between aerobics and strength-training groups is statistically significant.

Calculating the Independent Samples t-test

• t = (m1 - m2) / (s1 + s2) * sqrt(n1 + n2).
• m1 and m2 are the average scores for group 1 and group 2.
• s1 and s2 are the standard deviations for group 1 and group 2.
• n1 and n2 are the number of observations (participants) in groups 1 and 2.
• A 'pooled variance' formula is used when group size is unequal.

Dataset

• Includes an IV column with two levels and a DV column for happiness.
• One of the rows corresponds to one of the participants.

T-Test Outputs

• Provides the t-statistic value to measure the difference between groups relative to within-group variability.
• Provides the p-value to measure if the differences are statistically significant and the probability of a null hypothesis being true.
• Additionally, the effect size (Cohen's d) measures the magnitude of the difference.

Interpreting the Observed 't Value'

• A significant t-value has increased between-group variance and reduced within-group variance.

Non-Parametric Equivalent: Mann-Whitney U Test

• This test is used as the non-parametric equivalent, when parametric assumptions cannot be met.

What If Assumptions are Violated?

• Non-parametric tests need independent observations.
• While they deal with outliers better, issues may still arise if lots of errors are present.
• Always review the data.
• Consider a DV of happiness on a Likert Scale of 1-5.
• Numbers 1 to 5 may represent "very happy".

Mann-Whitney U Test Dataset

• Each includes one row, or one participant.
• Includes an IV column.
• The IV column consists of two levels.
• And each row includes a DV column to measure happiness.

Calculating the Mann-Whitney U Test

• Combine data from both groups.
• Sort it in ascending order.
• Rank it, by averaging ranks with tied values.
• Reorganize ranked data back into original groups.
• Tied values are people scoring 1 on the happiness scale
• The mean rank score is calculated as: Rank score = (1+2+3+4+5) / 5 = 3
• Reorganize ranked data back into original groups.
• Sum the ranks within each group.

Calculating Mann-Whitney U: Continued

• If ranks of each group are similar, there is no significant difference between the groups.
• The following can be said about the rank scores:
• In the Aerobic exercise group, PP have lower rank scores, indicating lower happiness.
• In the Strength exercise group, PP have high rank scores.
• In the Strength exercise group, PP have lower rank scores, indicating lower happiness.
• In the Aerobic exercise group, PP have high rank scores.
• Ranked scores are mixed from low to high levels.

Mann-Whitney U Value

• For Group 1: U1 = R1 - n1(n1 + 1) / 2
• For Group 2: U2 = R2 - n2(n2 + 1) / 2
• U1 and U2 are the Mann-Whitney U values for Group 1 and Group 2.
• R1 and R2 are the sum of ranks for Group 1 and Group 2 respectively.
• n1 and n2 are the number of participants.
• Afterwards, pick the smaller 'U' value.
• High U values are caused by JASP using an adjusted method for calculating results.
• The adjusted method includes corrections for tied ranks.

Paired Samples T-Test

• Used to determine happiness differences after taking an aerobics exercise class.
• Measurements are taken after one month, and after six months of taking classes.
• Quantitative research design, using experimental design, manipulates/controls variables, cause and effect relationships

Research Question

• Happiness over time, after one month compared to six months

T-Test Family Tree

• Paired T-Test only.
• When there are two groups or conditions in the IV.
• When the same people are in each condition.
• When parametric assumptions are met.
• Otherwise, use the Wilcoxon Signed-Rank Test.

Hypothesis Testing

• Similar to Independent Samples Test
• Identify the IV and the DV
• Identify the Hypothesis, either one-tailed or two-tailed.

Null Hypothesis (H0) for Paired Samples

• There will be no difference in happiness scores between 1 and 6 months of doing aerobic exercise.

Alternative Hypothesis (H1) for Paired Samples:

• There will be an effect or difference.
• One-tailed: People will show increased happiness ratings after completing exercise for 6 months compared to 1 month.
• Two-tailed: There is a difference in happiness scores between 1 month and 6 months of exercise.
• Design a hypothesis to be one-tailed, or two-tailed, but not both.

Variables - Paired Samples

• The independent variable (IV) is levels of aerobic exercise.
• There are two levels: 1 month (1M), and 6 months (6M).
• dependent variable (DV) is level of happiness, measured via a happiness scale.

Experimental Design - Paired Samples

• There is one group.
• Participants take part in the study under different conditions.
• Measure happiness.
• It is measured if there is a difference in happiness between 1 month and 6 months of exercising.

Dataset for Paired Samples t-test

• Includes two levels of the IV (1M vs 6M)
• One row equals one participant
• Happiness scores are measured twice.

Calculating Paired Samples t-test

• t = md / sd * sqrt(N)
• md: mean of the differences between the paired scores
• sd: standard deviation of the differences between each pair
• N: the number of participants (number of pairs)

What Does the T-Test Tell Us? - Paired Samples

• Test statistic: (t-statistic value) that measures the size of the difference between groups relative to within-group variability.
• p-value for the null hypothesis is whether the difference is statistically significant.
• Effect size: a measure for the magnitude of the difference.

Interpreting the Observed ‘t Value

• t = md / sd * sqrt(N)
• A significant t-test increases between-group variance while decreasing within-group variance.
• The within-group variance relates to the differences between paired observations.

Non-Parametric Equivalent: Wilcoxon Signed Rank Test

• The Wilcoxon signed rank test, when the parametric assumptions are violated.

Assumptions About Violated Data

• The data are not normally distributed
• The DV of happiness is ordinal, rated from 1 (not very happy) to 5 (very happy)
• Always review the data.
• No need for more assumptions with t-tests.

Experimental Design: - Wilcoxon Signed Ranks

• The independent variable (IV) has levels of aerobic exercise.
• It has two levels like before: 1M, and 6M.
• The dependent variable (DV) is measured with same Likert Scale of 1-5.

Calculating the Wilcoxon Signed Rank Test:

• First, calculate the difference between two conditions: exercise 1M vs 6M.
• Second, sort the differences (remove/ignore sign), then rank them (apply same tied ranks rule).
• Third, assign the signs back to the ranks (assign the original direction).
• Fourth, sum positive (W+) and negative (W-) ranks.
• T-statistic equals the smaller value (W+ or W-)

Wilcoxon Signed Rank Test: Explained

• Ranks happiness to test different levels of time.

FYI / Additional Info: The Sign-Test

• Calculate difference between two conditions: exercise after 1 month (1M) vs after 6 months (6M)

The Sign Test

• For this test, identify positive and negative signs.
• There should only be positive and negative.
• Use the p value and appropriate probability table to determine test significance.

Summary of Process

• The same types of t-tests cover research into the question: Does exercise make you happy?
• To conduct tests:
• Develop a research question.
• Design the test method: define the IV and DV, participant actions, the design, and type of data.
• Decide on the hypothesis.
• Once data is collected, ensure the data meets certain parametric assumptions about variance and normality.
• Use the T-test family decision tree to select the appropriate analysis test, then calculate test statistic.
• Obtain both P value and effect size.
• Read and interpret outputs.
• Analyze the descriptive test statistics.
• Write the interpretation.

Tests Covered

One-Sample t-test

• Non-Parametric Equivalent: Wilcoxon signed-rank test
• Compares the happiness ratings of one sample against a known, pre-existing population or value
• Feature: one IV of exercise (no experimental manipulation) and one DV, happiness

Independent t-test

• Non-Parametric Equivalent= Mann-Whitney U test
• Compares happiness ratings between two different groups of participants (aerobic vs strength exercise)
• Feature: One IV with two levels and one DV, happiness

Paired Samples t-test

• Non-Parametric Equivalent: Wilcoxon signed-rank test
• Compares happiness ratings between two conditions (time points) in one sample of study participants
• Feature: One IV with two levels compared against one DV, happiness

Reporting Analysis Results

• When reporting results from any analysis, provide the following information:
• The type of analysis and test that was used.
• What you are measuring or exploring (include key variables).
• Mention assumptions and review normality or variance test results.
• Report the line of test results.
• Clarify whether the test results are significant.
• Report any appropriate descriptive statistics.
• Interpret findings and explain how they relate to your research question and hypothesis.
• APA format is needed.

Points to Note about JASP Outputs

• Do not copy and paste full JASP output into reports.
• Write the JASP results as just a “line” item.
• While descriptive tables can be copy and pasted from JASP, they MUST be reviewed and edited first.
• Make all tables APA standard.
• Delete any redundant or unnecessary information.
• Number all tables.
• Give any tables appropriate titles, labels, and legends to match the writing or report overall.
• Do not include the full assumptions checks or normality figures.
• Report only the Shapiro-Wilk test.

How To Report the Results of Your T-Test

• Report the statistic calculated, the degrees of freedom from the JASP output, the test statistic, and the sign.
• Report to two decimal places and with a small "t."
• Report the exact p-value or report as falling under .001.
• Provide effect size with Cohen's d and report the value without the sign.
• Italicize t, p, and d.
• One-sample t-test example: t (31) = 2.06, p = .048, d = 0.37

Reporting Non-Parametric Tests

• Report the W = Wilcoxon test.
• If using MAnn-Whitney, report the U statistic.
• Report the test statistic calculated from analysis and whether the test was either positive or negative.
• p-values must be reported to two decimal places.
• The final result can have some variation to the third decimal place.
• Report effect size using the Spearman.
• There are no degrees of freedom but use the same letter italicization.
• Wilcoxon test example: W = 2.50, p = .001, rb = 0.95