Hypothesis Testing PDF

Hypothesis Testing  Stating hypotheses 1. Hypothesis testing Using data drawn from samples, together with probability, to determine if a treatment had a “statistically significant” effect  Stating hypotheses 1. Null Hypothesis (H0) Treatment has no effect 2. Alternative Hypothesis (...

Hypothesis Testing  Stating hypotheses 1. Hypothesis testing Using data drawn from samples, together with probability, to determine if a treatment had a “statistically significant” effect  Stating hypotheses 1. Null Hypothesis (H0) Treatment has no effect 2. Alternative Hypothesis (H1) Treatment has effect  Stating hypotheses Suppose I have a drug which I hypothesize changes IQ… H0: µtreatment = µuntreated (no effect of treatment) H0: µtreatment = 100 H1: µtreatment ≠ µuntreated (treatment has effect) H1: µtreatment ≠ 100  Stating hypotheses Suppose I have a drug which I hypothesize changes IQ… H0: µtreatment = 100 H1: µtreatment ≠ 100 Note how the hypotheses are stated: They are mutually exclusive (one or other) Exhaust all logical possibilities (no other options)  Setting criteria 1. Rejecting H0 How different does M have to be from µ… …to decide that it is unlikely that M was drawn from chance from population? …for M to be so extreme that it couldn’t arisen from chance; must be different?  Setting criteria M1? M2? M3? µ Common probability for “rejecting the null” is.05. If it’s in the extreme 5%, then we are confident the difference is not due to chance, it’s due to treatment Setting criteria Z = -1.96 Z = 1.96 2.5% 2.5% µ If M falls in the extreme.05, then we are confident it is different (α =.05) From the Z-table, we know that Z=1.96 è 2.5% in each tail Setting criteria Z = -1.96 Z = 1.96 2.5% 2.5% µ If M is more extreme than Z = +/- 1.96, then we are confident that it is different than µ +/- 1.96: cut-off criterion (“critical value”) H0 rejected è M is different from µ  Setting criteria 2. Example σ M = 106, n = 25 σ M = µ = 100 n σ = 15 M-µ 106 - 100 6 Z= = = =2 σ M 3 3 Reject H0!  Setting criteria 3. Factors affecting the Z-score Distance between M and µ Sample size (via SEM)  Setting criteria 3. Example σ M = 105, n = 25 σ M = µ = 100 n σ = 15 M-µ 105 - 100 5 Z= = = = 1.67 σ M 3 3 Fail to reject H0!  Setting criteria 3. Example σ M = 106, n = 16 σ M = µ = 100 n σ = 15 M-µ 106 - 100 6 Z= = = = 1.6 σ M 3.75 3.75 Fail to reject H0!  Errors in decision making 1. Type I and Type II errors H0 is H0 is true false Reject Type I OK H0 Error Don’t Type II Reject H0 OK Error  Errors in decision making 2. Type I errors (α) Reject the null, when you shouldn’t have Infer a difference, when there is none Decrease Type I errorsè lower α-level (.01) More extreme scores necessary to reject H0  Errors in decision making 3. Type II errors (β) Accept the null, when you shouldn’t have Infer no difference, when there is one Decrease Type II errors è bigger sample, reduce variability, better measurement  Errors in decision making 4. What if my participants were not randomly selected/assigned to the treatment condition? Would the statistical conclusions differ? No! Would affect internal validity; how the results are interpreted  Directionality 1. Stating hypotheses Non-directional = 2-tailed test Directional = 1-tailed test Z = 1.65 5.0% µ  Stating hypotheses w/ directionality Suppose I have a drug which I hypothesize changes IQ… H0: µtreatment = µuntreated (no effect of treatment) H0: µtreatment = 100 H1: µtreatment > ≠ µuntreated (treatment has effect) H1: µtreatment > ≠ 100  Effect sizes 1. Statistical significance vs. clinical (a.k.a. practical) significance Statistical– effect did not occur by chance But could be a very small difference, if n is large, SEM is small. Clinical– effect is meaningful Effect sizes Population Treatment 100 102 IQ Population Treatment 100 110 IQ  Effect sizes 2. Cohen’s d Mean difference M-µ d= = Standard deviation σ 106 - 100 6 d= = =.40 15 15  Effect sizes 2. Cohen’s d …is uninfluenced by sample size. …is independent from statistical significance.  Effect sizes 2. Cohen’s d |d| 0 -.20 “no effect”.20 -.50 “Small”.50 -.80 “Medium”.80+ “Large”