IPR Term 2 Week 4 - T-tests, One Sample t-test and Wilcoxon signed-rank test

• T-tests and non-parametric equivalents will be taught over two weeks.
• Lecture 5 is the reference point for understanding the purpose and application of t-tests.
• It is important to understand the key differences between t-test types: one-sample, independent, and between samples.
• All t-tests will be calculated using JASP software.
• There will be interpretation of the results of the tests, alongside write-ups following APA standards.

Lesson 4 Outline

• Introduces t-tests, parametric versus non-parametric tests, one-sample t-tests, and how to execute t-tests in JASP software.
• Introduces the Wilcoxon signed-rank test (+ how to execute the test in JASP) used as a non-parametric alternative to the one-sample t-test.

Lesson 5 Outline

• Teaches independent t-tests, Mann-Whitney U tests, paired-samples t-tests, Wilcoxon signed-rank tests (+ how to execute the tests in JASP), and interpretation and write-up.
• Mann-Whitney U test and Wilcoxon signed-rank test serve as non-parametric equivalents.
• Covers interpretation and write-up of the presented tests.

Research Designs - Quantitative

• Experimental designs manipulate and control variables, to determine cause-and-effect relationships.
• Longitudinal designs measure data repeatedly over time.
• Correlational designs determine the relationship between variables without manipulation to find causality.
• Descriptive methods describe the characteristics of behavior using self-reporting methods like surveys and questionnaires.

Experimental Design

• Enables examination of the relationship between an independent variable (IV) and a dependent variable (DV).
• IVs are manipulated, participants are assigned to different groups or conditions, and DVs are measured.
• Measures the difference between two groups or conditions based on IV manipulation.
• Used to draw cause-and-effect conclusions by isolating the effect of the IV(s) on the DV.

Benefits of Experimental Designs

• Establishing cause-and-effect relationships.
• Control variables to avoid extraneous or confounding variables.
• Test specific hypotheses.
• Improve the replicability and reliability of research.
• Improve objectivity and precision.

Visualizing Data

• Bar graphs often compare groups by showing the magnitude of difference between groups.
• Bar graphs are used to show the mean and SD error bars.
• Box plots can show mean, median, range, and variability of data.
• Use line graphs to show changes and trends over time.
• Straightforward and intuitive visualization often leads to improved results.

Hypothesis Testing

• Involves observation, interest, or initial idea, followed by review of existing research.
• Develop a research question, formulate a hypothesis, and collect data to test the hypothesis.
• Analyze the data to draw a conclusion, then present and evaluate the findings.

Types of Hypotheses

• A well-formulated hypothesis has to be either one-tailed or two-tailed, not both.
• Null Hypothesis (H0):
  • There is no effect or relationship ("no difference").
  • There will be no difference in happiness between people who exercise and those who don't.
• Alternative Hypothesis (H1):
  • There will be an effect or difference.
  • One-tailed:
    • Direction is specified.
    • People who exercise will have higher happiness scores than people who don't.
  • Two-tailed:
    • Direction is not specified.
    • There will be an effect, but no direction is stated.

Identifying Variables

• Independent Variable (IV): variable manipulated.
• Dependent Variable (DV): variable being measured.
• A quick tip is IVs tend to be nominal/categorical and DVs are continuous data.

Variable Examples

• Independent variables: include hair color, education level or marital status.
• Dependent variables include: response time, number of chocolate bars eaten per week or anxiety score.
• T-tests should be used instead of Chi-square tests when you have one categorical IV and one continuous DV.
• Correlations are used to examine two continuous variables instead.

Experimental Designs

• Paired (repeated or within subjects) measures/design: one group of participants take part in the study under different conditions.
• Independent (or between subjects) measure/design: different participants are recruited into each condition.
• One sample measures/design: one group of participants take part in the study under one condition, compared to a known score/distribution/population.

T-Test Family Tree

• Consider "how many groups or conditions are in your IV?"
• If "two", then consider who the participants are (same/different).
• If "one", then consider it is a one-sample t-test.
• Leads to one of the following tests: Paired t-test, Independent t-test, or One-sample t-test.

T-Tests Do

• Measure the size of the difference between two groups (or a sample mean and a population mean) based on IV manipulation.
• Determine if differences are due to manipulation of the IV instead of natural variation or timing.
• Conclude if results are generalizable to a population.
• Critical t-value = expected treshold and cut-off point established from the student's t-distribution.
• Significant result = there is a statistical significance and the null hypothesis can be rejected.
• T-value can be positive or negative and are distributed around zero in a symmetrical manner.
• T-distribution depends on the sample size.
• With a two-tailed test, the study determines if the t-statistic is extreme.

Interpreting t-Statistics

• The student's t-distribution is followed under the assumption that H0 is true.
• Observed t-value falls beyond the critical value, it reveals a significant difference where the H0 should be rejected.
• Critical t-values define the boundary/threshold between non-significant and significant regions.

What Does the t-test tell us?

• The analysis gives the t-statistic value which measures the difference between groups, relative to within-group variability.
• The analysis determines the p-value, which is the probability of obtaining that result if the null hypothesis is true.
• The analysis also determines effect size; the magnitude of the practical interpretation.
• The t-test compares the average scores between groups and indicates the variability of the group/sample.
• The goal is for our experimental manipulation (IV) to cause statistically significant changes in the DV.

Achieving "Best" Results

• We want our manipulation to have a consistent effect on all participants.
• The data is explained mostly by the experimental/between-group variance.
• Increase the between-group variance and decrease the within-group variance.
• Comparisons based on a significant difference are determined by finding more 'between' groups.

Variability Calculating

• One-Sample T-Test: the mean of a group which is compared to a known reference considering the sample size and reliability of the sample.
• Independent Samples T-Test: the means of independent groups and the variations within those groups are compared.
• Paired Samples T-Test: means of related groups are compared, considering variability in the differences and the sample size.

Effect Size

• Measures the magnitude/size of the effect.
• A larger effect size= larger difference between groups.
• The typical calculation is Cohen's d = (X1 - X2) / sqrt (SD1^2 + SD2^2)/2).
• Thresholds are 0.2 = small effect, 0.5 = medium effect, and 0.8 larger effect.

T-test Summary

• Develop a research question, design a method (define independent and dependent variables).
• Decide your design and data type.
• Decide on your hypothesis, collect data, and check parametric assumptions (normality, variance, etc).
• Select analysis using the t-test family decision tree (calculate test statistic, p-value, and effect size).
• Read and interpret the output, interpret the data, then write up (APA) with descriptive and inferential results.

T-test Assumptions

• T-tests are parametric, assumptions must be met.
• These tests are appropriated to draw reliable conclusions.

Parametric Tests

• Parametric tests require data from one of the large catalogue of distributions described by statisticians.
• Parametric tests are based on the normal distribution, which require basic assumptions that must be met to be accurate.

T-test Assumptions

• Normality: data should be normally distributed and follow a Gaussian distribution (Shapiro-Wilk test).
• Homoscedasticity: variance spread should be consistent among conditions (Levene's test).
• Type of data: dependent variable should be interval or ratio data.
• Independent Observations: all observations must be independent.

Two Types of Assumptions

• The assumptions decide and meet through design of methods and measures.
• The independent observation consists of data not influenced by other participants.
• Both variable distribution, and the consistency among conditions are tested before / during analysis, in JASP.

Violated Assumptions

• Use an equivalent, non-parametric test if assumptions are violated.
• Types of data: Nominal, Ordinal, Interval or Ration.

Testing Normality

• Use histograms and boxplots.
• Use Shapiro-Wilk Test (in JASP).
  • If p < 0.05 = non-normal data.

Variance Testing in JASP

• Use Levene's test.
  • If p < 0.05 = unequal variance.
• Assumption violated: Variances not equal (called heterogeneity of variance), use Welch's adjustment option.

Non-Parametric Tests

• Do not rely on parametric test assumptions,.
• Do not assume that the sampling distribution is normally distributed.
• Data are not or cannot be normally distributed (e.g., ordinal data).
• These transform data in order to be appropriate for analysis.

Transforming Data

• Raw scores are changed to rank scores (lowest to highest).
• Tied values: calculate the mean rank score divided by the number of tied ranks.
• Raw Score and Ranked Scores are the key elements.
• For 3 tied values, they should rank as '5', '6', and '7' and the ranked score (5 + 6 + 7)/3 = 6.

Updated Family Tree

• Use the same and different metrics
• With one-sample test we have one set of data that can compare to a reference.
• The following factors are required: parametric assumptions, paired test, Wilcoxon signed-rank test, and independent test.

Non-Parametric Test Vs Parametric Test

• Less powerful that their parametric counterparts.
• If there is a true effect in our data, the parametric test may be more powerful.
• Parametric tests are sensitive to data, that said you need normality to use it.

Visualization and Descriptors

• Descriptors are important to consider with both tests
• For parametric tests report "mean and a standard deviation
• For non-parametric tests report the "median".

Parametric/Non-Parametric Tests

• Focus on Assumptions, Data Type, Use, Power, Effect Size and Descriptors.
• Fewer assumptions in Non-Parametric, but less flexible in results.

One-Sample T-Test

• A one-sample t-test is used when there is only one set of data.
• Conducted to compare to reference values.
• Imagine only having one group exercising and measured their happiness, whether it's significant to an average population.
• One row equals to one participant, one column equals the score.

Calculating, Descriptors

• One level and independent measurements.
• Used to compare your sample to a population mena, for example comparing with the average happiness in the UK.
• Important to consider null and alternative hypothesis.
• It's possible to calculate these results.

Calculating Data

• We focus on precision, and population of samples with this calculation:
• m = average score per sample
• standard deviation
• sample size
• population of relevant reference
• and, between and within group variations.

Non-parametric

• The Wilcoxon sign-rank test is used for ordinal data to measure independent properties.
• This test is similar to other tests with one set of data to compare measurements.
• Ranks differences: applies rule across tied ranks.
• Signs: assign signs to ranks relative to the direction.
• The resulting values are the output.