Statistics: Independent Groups T-Test Concepts

Hypothesis Testing: Correct Decisions and Errors

• In any hypothesis test, there are four possible outcomes
• The true situation: Either the null hypothesis is actually true or the alternative hypothesis is actually true
• The decision is either not to reject the null hypothesis or reject the null hypothesis and accept the alternative hypothesis

Hypothesis Testing: Correct Decisions and Errors

• The True Situation column represents whether the null hypothesis (H0) or the alternative hypothesis (HA) is true
• The Decision column represents the decision made in the hypothesis test (do not reject H0 or reject H0 and accept HA)
• Correct Decision: The decision matches the true situation
• Type I Error: Rejecting H0 when it is actually true
• Type II Error: Failing to reject H0 when it is actually false

Hypothesis Testing: Correct Decisions and Errors

• Probabilities for each outcome are represented as conditional probabilities
• The probabilities are conditional on the true situation
• P(Not Rejecting H0 when H0 is True) = 1 − α
• P(Rejecting H0 when H0 is True) = α
• P(Not Rejecting H0 when HA is True) = β
• P(Rejecting H0 when HA is True) = 1 − β

Significance Level

• The probability of a Type I error is represented by the symbol α and is called the significance level
• The probability of a correct rejection of the null hypothesis in favor of the alternative hypothesis is represented by 1 − β and is called the power of the test

Type II Error and the Power of the Test

• A standardized ability test has scores that are normally distributed with μ = 500 and σ = 100
• An investigator is interested in the effects of CAI (Computer Assisted Instruction) on the performance of 25 students
• The investigator predicts that CAI will lead to an improvement in performance, i.e., HA: μ = 550

Type II Error and the Power of the Test

• From the sampling distribution under the null hypothesis (with a 5% significance level)
• Critical value is 1.645
• If the observed mean is 520.4, the z-score would be 1.02 (calculated by (520.4 - 500) / 20)
• Fail to reject the null hypothesis because z-score is outside rejection region

Power of the Test: Effect Size

• As the effect size increases in an example (from μ = 550 to μ = 580), the power of the test increases, i.e., the probability of a correct rejection of H0 increases and the probability of a Type II error decreases
• By increasing sample sizes, standard error from estimating population parameter (mean) decreases, and thus the power of the test increases

The Power of the Test: Effect Size

• With reference to the CAI example, imagine that the investigator had tested 100 students (n = 100 instead of 25)
• As the sample size increases, the standard error decreases, and the distribution becomes narrower, which increases the power to detect the effect
• Small differences in means can still be statistically significant if the sample size is large enough

Hypothesis Testing Recap

• Begin with a null and alternative hypothesis
• Set the alpha level
• Perform the appropriate statistical test
• Calculate the p-value
• Compare the p-value to the alpha level
• Alternatively, find the critical and observed test statistics for a decision

Sampling Distributions

• A population of scores reflects the frequency of occurrence of every score in the distribution
• Frequency distributions are also called probability distributions
• Given the mean and standard deviation of a normal distribution, probability statements can be made about the likelihood of selecting a score from a specified area of the distribution

Sampling Distributions

• In hypothesis testing, you're usually interested in the means rather than individual scores
• To make probability statements about a randomly selected mean falling within a specified area under the normal curve, you need a normal distribution of means

Sampling Distributions

• Mean of the sampling distribution is exactly the same as the mean of the population (μ = μy)
• Standard deviation of the sampling distribution (σy) = σ/√n
• Variability of the sampling distribution is determined by: The variability of the population distribution, and the size of the samples used to establish the sampling distribution
• As sample size increases, the sampling distribution approaches a normal distribution (central limit theorem)
• A large sample (N > 30) is usually sufficient for the sampling distribution to be normally distributed, regardless of the population distribution

One-Sample t-Test

• When using a small sample, the sample standard deviation (s) is used as an estimate of the population standard deviation (σ)

Student's t Distribution

• When working with small samples, there's more error in estimating parameters based on this information
• The formula for the t-statistic (t) is t = (Υ - μy)/Sy and calculates estimated number of standard errors that the sample mean is from μ
• In a real data study, you generally have little control over the effect size, and more control over the sample sizes
• A similar value of t-statistic is observed when comparing with z-score if the sample size is larger than 40

Student's t Distribution

• The symbol v is used to represent degrees of freedom. A particular t-value always has a df associated with it: t(df)
• For a one-sample case, df = N-1
• You cannot use a z-table; must use a different table, the 't-distribution table'

z Distribution vs. t Distribution

• In a z-distribution, population standard deviation (σ) is known while in a t-distribution it is not known

Example

• Adults are recommended to exercise at least twice a week
• Sample of 35-40-year-olds consists of 30 individuals, with mean 1.84 and estimated standard deviation 1.68
• The viablity of hypothesis that μ=2 is examined using a one sample t-test

Example

• The critical values of t for a non-directional test (alpha = 0.05) and 29 degrees of freedom are ±2.045
• The test statistic for the one sample t-test is t(29) = -0.52
• Reject/Fail to reject the null hypothesis: Because the value of the t-statistic (-0.52) does not exceed the negative critical value (-2.045), we fail to reject the null hypothesis

Example (Confidence Intervals Approach)

• The critical values of t for a non-directional test (alpha = 0.05) and 29 degrees of freedom are ±2.045
• Calculated standard error = 0.31
• One-sample CI at 95% is between 1.21 and 2.47
• The null hypothesis that μ=2 is not rejected as the population mean lies in the interval

Example

• A researcher conducted a reading test on 30 students with a mean of 83 and standard deviation estimate of 17.35
• 95% confidence interval is 76.52 to 89.48
• 99% confidence interval is 74.26 to 91.94

Example: Sample Size and t-Distribution

• For a sample of size 12 (df=11), 95%CI critical t-value (or t-score) is 2.201 for a two-sided test
• With increased sample size (e.g., 26, df=25), the critical t-score becomes 2.060, illustrating an inverse relationship between sample size, standard error, and thus the confidence interval

Example: Independent samples t tests

• A researcher examines whether a new teaching method affects student math test scores compared to a traditional method
• Group 1 (traditional method): n1 = 10, X₁ = 75, ŝ1 = 8
• Group 2 (new method): n2 = 12, X2 = 82, ŝ2 = 10
• Significance level = 0.05 (two-sided test)
• Null Hypothesis: The teaching method has no effect on math scores (H0: μ1 = μ2)
• Alternative Hypothesis: The teaching method affects math scores (H1: μ1 ≠ μ2)
• Pooled Standard Deviation: 9.15
• Estimated Standard Error of Mean Difference: 3.92
• Calculated t-Statistic: -1.79
• Critical t-value: ± 2.086
• Fail to reject null hypothesis: The calculated t-value is not outside the critical t-value range

Assumptions of the Independent Groups t-test

• Samples are independently selected from their respective populations
• Scores each population are normally distributed
• Dependent variable is at least at interval level of measure
• Scores in two populations have equal variances

Using Independent Samples t-Test

• You are interested in the effects of physical exercise on health. You expect that people who regularly exercise will spend less days sick within a year than non-exercising people.
• Exercising group: n1 = 5, X1 = 24, s1 = 4.5
• Non-exercising group: n2 = 15, X2 = 30, s2 = 9.4
• Significance level: .05
• Null Hypothesis (H0): μ1 = μ2
• Alternative Hypothesis (H1): μ1 < μ2
• Degrees of Freedom: df=18
• Critical t-value: -1.734
• Calculated t-statistic: -1.36