Slides for September 10th and 12th - Chapter Three, Continued PDF

Slides for September 10th and 12 th Chapter Three, Continued  Interrogating the Three Claims Using the Four Big Validities  Interrogating the Three Claims Using the Four Big Validities Interrogating frequency claims Interrogating associatio...

Slides for September 10th and 12 th Chapter Three, Continued  Interrogating the Three Claims Using the Four Big Validities  Interrogating the Three Claims Using the Four Big Validities Interrogating frequency claims Interrogating association claims Interrogating causal claims Prioritizing validities Four Validities Construct Statistical validity: validity: Quality of the measures and Statistical conclusions are manipulations appropriate and reasonable. Internal External validity: validity: No alternative causal To whom, what, or where can explanations for the outcome we generalize?  Interrogating Frequency Claims Construct validity External validity, or generalizability Statistical validity  Construct validity: the accuracy of the label being applied to the measure or manipulation. Ex) 15% of teens report feelings of depression Question: How was depression operationally defined?  External validity: whether there is generalizability of the findings to other people or contexts 74% of the world smiled yesterday Question: Which people did they survey and how did they choose their participants?  Statistical Validity: The extent to which the study’s conclusions are reasonable and accurate Point Estimate: A single estimate of some population value (percentage) based on data from a sample Question: What is the Confidence Interval (CI)? A given range indicated by a lower and upper value that is designed to capture the population value for some point estimate Question: Are there other estimates of the same percentage range? 15% of teens report feelings of depression Construct validity: How was “feelings of depression” operationally defined? External validity: Where did the teen sample come from? How obtained? Statistical validity: What is the confidence internal of the 15% point estimate? Has other research found a similar range?  Interrogating Association Claims Interrogating Association Claims Construct validity External validity Statistical validity  Association claims involve two variables that may be linked to one another. Construct validity of each variable External validity Statistical validity Construct validity and association claims construct validity is doubly important TWO labels are under scrutiny Claim: Workaholism is tied to psychiatric disorders  Construct Validity: How was workaholism operationally defined? How was psychiatric disorders operationally defined?  External validity: How was the sample obtained? Are the people in the sample representative of the larger population?  Statistical validity: E.g., What is the confidence interval of Pearson’s r ? Has other research found a similar value? Interrogating Causal Claims Interrogating Causal Claims Criterion​​ Definition​​ Covariance​​ The study’s results show that as A changes, B changes; e.g., high levels of A go with high levels of B, and low levels of A go with low levels of B.​ Temporal precedence​​ The study’s method ensures that A comes first in time, before B.​ Internal validity​​ The study’s method ensures that there are no plausible alternativeexplanations for the change in B; A is the only thing that changed.​  Be skeptical of causal claims. Must meet: Internal Validity (new) Construct Validity External Validity Statistical Conclusion Validity Internal Validity and Causal Claims Internal validity concerns the three criteria that we established earlier: Association between the supposed cause and supposed effect Temporal precedence Elimination of alternative explanations (confounds) Association Claims within Experiments must first establish that the two variables are correlated If they aren’t correlated, then one cannot possibly be causing the other Temporal Precedence By doing an experiment, a person is guaranteeing that there will be temporal precedence: one variable will come before another in time. In our case, the thing we suppose to be the cause is manipulated and therefore it comes before the thing we suppose to be the effect. This establishes temporal precedence. Experiments help to rule out alternative explanations When you are doing an experiment, it is necessary to try to get rid of different potential confounds in the design. Confounds related to the participant Confounds in the experimental design Evaluating Causal Claims Evaluating Causal Claims Construct validity of the two variables: How well was the IV manipulated? Examine each level of the IV How well was the DV measured? Evaluating Causal Claims External validity: To whom or what can we generalize this effect? How was the sample obtained? Was the sample diverse? Evaluating Causal Claims Statistical Conclusion Validity: How large was the effect of the IV on the DV? How big is the difference between two group means? Is the difference statistically significant? What was the confidence interval of the effect? Have other studies found a similar effect size? Evaluating Causal Claims Internal Validity: have confounds been eliminated? Random Assignment to Condition Other variables held constant or removed Internal Validity is the top priority when evaluating causal claims!