Defining Measuring Variables lecture_Week 2_Week 3.pptx

DEFINING & MEASURING VARIABLES Research Basics  Overview 2  Defining Variables  Types of Measures  Scales (i.e., levels) of Measurement  Reliability  Validity 3 Defining Variables  Variables & Definitions 4  Variable  Something that...

DEFINING & MEASURING VARIABLES Research Basics  Overview 2  Defining Variables  Types of Measures  Scales (i.e., levels) of Measurement  Reliability  Validity 3 Defining Variables  Variables & Definitions 4  Variable  Something that varies and can be measured  Hypothetical constructs  Attributes/mechanisms that help explain/predict behavior  May not be directly observed  But may be associated with something that can be measured  Variables & Definitions 5  Conceptual Definitions: What you’d read in a textbook  Aggression????  “A response intended to inflict pain or discomfort” (Averill, 1982, p. 30)  “Any form of behavior directed toward the goal of harming or injuring another living being who is motivated to avoid such treatment” (R.A. Baron, 1977, p. 7)  “The use of threat or force, in territory not clearly one’s own, without clear evidence that a majority of the emotionally involved people in that territory want such intervention” (R. K. White, 1984, p. 14)  Defining Variables 6  Operational definition “Operationalization”  Measurement or measurement procedure  How would you measure aggression?  Defining Variables 7  “Exercise leads to weight loss”  Exercise  Thumb twiddling 1 minute per week  10 pushups each day  ½ hour aerobic activity, 5 days a week where heart rate is greater than 140 bpm  Weight loss  Pounds  Waist Size  Body fat content  Clothes size  Multiple Outcomes 8  Converging evidence is more convincing  Exercise leads to lower scale-weight  Exercise leads to lower scale-weight, smaller waist size, and reduced body fat  More measured outcomes increases the chances we’ve tapped into all relevant parts of our construct  Let’s practice 9  Creativity  Forgiveness  Love  Extraversion  “Southern Culture of Honor” 10  “Affronts are met w/ violent retribution”  Hypotheses  Perceive insults as damaging  Emotional/physiologically upset  Prepared to respond aggressively after insult (cognitive, physiologically, & behaviorally) (Cohen, Nisbett, Bowdle, & Schwarz, 1996)  “Southern Culture of Honor” 11  Hostility assessed  2 observers measured emotional reactions  Face rating task Guess emotion being expressed in pictures  Word completion task __ight, gu__ (fight or gun) OR (light or gum)  Scenario completion task Cohen, Nisbett, Bowdle, & Schwarz, 1996)  Scenario Completion Task 12  It had only been about twenty minutes since they had arrived at the party when Jill pulled Steve aside, obviously bothered about something.  “What’s wrong?” asked Steve.  “It’s Larry, I mean, he knows that you and I are engaged, but he’s already made two passes at me tonight.”  Jill walked back into the crowd, and Steve decided to keep his eye on Larry. Sure enough, within five minutes Larry was reaching over and trying to kiss Jill. Cohen, Nisbett, Bowdle, & Schwarz, 1996)  Example: “Southern Culture of 13 Honor”  Hostility assessed  Before & after cortisol levels Hormone associated w/ stress, anxiety, & arousal  Before & after testosterone levels Hormone associated w/ aggression & dominance  Desire to demonstrate toughness Public opportunity to take shock in “electric shock stress test”  The “chicken game”  Firmness of handshake Cohen, Nisbett, Bowdle, & Schwarz, 1996)  What’s the point? 14  Many ways to operationalize vars  Didn’t use them all in one study though  Reliable measures?  Valid measures?  An operational definition should be… 15  Specific  Reliable  Valid Approaches  Quantitative  Expressed numerically  Statistical tests available  Qualitative  In-depth Grounded theory – use information to develop interpretation or theory Identify ideas, then write theoretical narrative (broad themes)  Difficulties w/ analyses  How to measure? 17  Which measure provides the best indication of the underlying construct?  Self-reports Behavioral, Cognitive, Affective  Tests Personality, Achievement, Aptitude  Behavioral measures Concern about reactivity  Physical measures  Each has advantages & disadvantages Selecting Measures Scales of Measurement  Nominal Scales  Classification  Qualitative differences  Ordinal Scales  Classify & rank Selecting Measures Scales of Measurement  Interval Scales  Classify, rank, & categories the same size Arbitrary zero  Ratio Scales  Classify, rank, categories the same size, & anchored by zero point 20 Reliability Choosing Your Measures  Reliability  Stability or consistency of measure Observed score = True Score + Measurement Error  Types of Error 22  1. Observer Error  2. Participant Changes  Stable attributes (E.g., suspicion, intelligence)  Transient states Something about person that changes regularly  3. Environmental Changes  4. Test characteristics Assessing Reliability  Test-retest reliability Participants have changed Respond in similar way Practice Effects  Parallel-forms (or alternative-forms) reliability Tests at time 1 and time 2: are they equivalent? Assessing Reliability  Internal consistency Can be measured with correlation coefficient Split-half reliability - Split items on tests into equivalent halves Assessing Reliability Judgments/Ratings by Multiple Observers  Interrater (interjudge) reliability Extent of agreement between raters  Can be measured w/ correlation or percent agreement 26 Validity Choosing Your Measures  Validity  Is the measure measuring what it is meant to? 1. Face Validity  Does the measure appear to measure what it is intended to measure? 2 + 2 = ________ 6 + 10 = ________ 7 + 3 = ________ What do you think this test measures? 1 = Strongly disagree 2 = Disagree 3 = Not sure 4 = Agree 5 = Strongly agree  I enjoy taking risks.  I prefer things that involve change and variety even if they make me feel uneasy at first.  I would enjoy fast driving.  I would do almost anything on a dare. Types of Validity 1. Face Validity  When might we NOT want a face valid measure? If we think it might influence participants’ answers 2. Content Validity  How well do the items represent the entire universe of items? Types of Validity 3. Criterion validity How well is the measure related to other variables that reflect the same construct? Concurrent validity Do scores correlate with other relevant current outcome/behavior? IQ test should correlate with current school performance Occupational aptitude test should correlate with current work performance Predictive validity Does it correlate with future outcomes? Types of Validity 4. Discriminant (Divergent) Validity  Looking for evidence that the measure is distinct from other behaviors/traits (i.e., not correlated) Types of Validity  Face  Content  Criterion  Concurrent  Predictive  Discriminant (Divergent) Relationship Between Reliability & Validity Can be reliable, without being valid. But can’t be valid, without being reliable.  Re-cap 34  Defining Variables  Types of Measures  Scales (i.e., levels) of Measurement  Reliability  Validity 35 Mini-Review 1. What is the difference between reliability and validity? 2. Provide an example on a measure that would have high content validity. 3. Imagine I wanted to measure helping behavior. How might I operationalize this variable?