PS 295 Week 3 2024 Outline PDF

Week 3 Outline Variables in research Defined: What are variables? Variables and levels of variables Types of variables Operational definition of variables Conceptual variables and operational definitions Operationalizing measured and manipulated variables The “three claims, four...

Week 3 Outline Variables in research Defined: What are variables? Variables and levels of variables Types of variables Operational definition of variables Conceptual variables and operational definitions Operationalizing measured and manipulated variables The “three claims, four validities” framework Three types of claims Frequency claims Association claims Causal claims More on causation Importance of causation Criteria for causation Four types of validity Construct External Statistical Internal Prioritizing validities Variables; Types of Claims and Validities Variables The basic building blocks of research  Variables defined  Informally:  The characteristics we study  Recall that hypotheses typically involve statements about how two things (variables) go together  Also sometimes called factors, dimensions, qualities, attributes  Definition:  Anything that can take on different values or levels across a set of cases  Must have at least two values or levels (otherwise it’s a constant)  Across people, events, organizations, etc.  E.g., self-esteem, height, gender “Set of cases”?  Human participants: Physical: height, weight, heart rate, hair colour; Psychological: opinions, personality traits, reasoning ability  Tasks carried out: importance, difficulty, complexity, time pressure, level of stress  Schools: number of students, quality of education, student-teacher ratio  Family background: number of family members, parenting style, socioeconomic status, parents’ marital status  “Change” in a variable  Note: sometimes we refer to “change” in a variable  E.g., changes in X are associated with changes in Y (or “increases/decreases” in X…)  This does not necessarily mean literal change  E.g., “Change” in self-esteem could mean different participants have different levels of self-esteem  Natural variability across cases Variables and Levels of Variables Every variable has at least two different “levels” “The study was conducted in either a hot room or room at normal temperature.” Variable: room temperature Levels: hot, normal “The study included 21 doctors, 24 nurses and 23 occupational therapists.” Variable: occupation/role Levels: doctor, nurse, occupational therapist  Exercise: Variables and Levels of Variables 1. Before completing a task, participants were assigned to the role of manager or employee. 1. Variable: 2. Levels: 2. Participants ranged in age from 18 to 22 years. 1. Variable: 2. Levels: 3. The researchers measured how many seconds it took participants to respond to the question. 1. Variable: 2. Levels: 4. The words that participants memorized were either nouns, adjectives, or verbs. 1. Variable: 2. Levels: 5. The participants rated their self-esteem on a scale from 1(extremely low) to 7 (extremely high). 1. Variable: 2. Levels:  Levels of Variables and “Experimental Conditions” In an experiment, the levels of a variable are often called the “conditions”  E.g., an experiment compares an experimental treatment and a control treatment (variable is type of treatment with two levels: experimental and control)  E.g., an experiment compares five types of diets (variable is type of diet with five levels) Thenumber of levels of the variable is the number of conditions in an experiment. Types of variables  Qualitative (AKA nominal, categorical)  Variables that classify or categorize  Levels differ in terms of quality or type, not in the amount of something  Example: religious affiliation, political party membership, marital status, animal species, type of music  Quantitative  Levels differ in terms of the quantity or amount of some underlying dimension  Can place the levels in order, from less to more  Example: degree of religiosity, income level, self- esteem, IQ score, time to complete a task Types of variables  Discrete  There are no meaningful values of the underlying variable in between the levels of the scale  Must be measured in whole units or categories  E.g., family size, type of therapy, number of friends  Continuous  There are meaningful values of the underlying variable between the levels of the scale  Not limited to certain values (e.g., whole numbers)  Can be measured in whole units or fractional units  In principle, between any two values, intermediate values are possible  Example: height, weight, rating of agreement Note: The nature of the underlying variable is distinct from how it is measured or manipulated; we are focusing here on whether the underlying variable is continuous in principle.  Exercise: Quantitative vs. Qualitative? Continuous vs. Discrete? 1. Size of food reward in grams. 2. Season (Spring, Fall, Summer, Winter). 3. Number of students in the class. 4. Time to complete a task in seconds. 5. Major in university. 6. Political affiliation (Liberal, Consverative, NDP, Green). 7. Self-esteem rating on a scale from 1 to 10. 8. Hunger level.  Types of variables  Person (or subject) variables  Qualities that the participants being studied differ on  E.g., age, marital status, extraversion, intelligence  Situation (or context) variables  Qualities of the situations that participants are in  E.g., temperature, how anxiety-provoking, privacy  Stimulus variables  Qualities of the stimuli presented to participants  Stimulus = something presented by experimenter that evokes a response (e.g., word list, video, photos)  E.g., vividness, attractiveness, abstractness  Response (or behavioural) variables  Qualities of responses made by participants  E.g., RT, test performance, number of words recalled  Types of variables  Manipulated  A variable that a researcher controls by assigning participants to different levels  A variable that is intentionally varied by the researcher  Researcher creates the different levels of the variable and exposes participants to them  E.g. (situation): room temperature, light intensity, privacy, type of music playing  E.g. (participant): mood, anxiety level, hunger, self-awareness  Measured  A variable whose levels are simply observed and recorded  Researcher takes an assessment of each participant’s level on the variable  E.g. (situation): room temperature, number of people  E.g., (participant): gender, age, IQ, personality Can ask: For each participant, did the researcher assign the participant to that level of the variable?  Did the researcher create or influence that level of the variable for the participant? Measured and Manipulated Variables Some variables can only be measured—not manipulated. Due to:  Physical constraints  Ethical constraints Some variables can be either manipulated or measured  Types of variables  Independent variable – “the cause”  The hypothesized causal variable  In an experiment, it is manipulated by the researcher  Dependent variable – “the effect”  Thehypothesized effect  Measured by the researcher  Relation of the IV/DV and Manipulated/Measured distinctions  One possible approach (usually used in Morling text)  The term IV is reserved for manipulated variables and so is only used for experimental studies  IV = the presumed cause, and is a manipulated variable  DV = the presumed effect, and is a measured variable  In non-experimental studies, a different term is used for these variables  Hypothesized cause = predictor variable, subject variable, participant variable  Hypothesized effect = criterion variable, outcome variable  Relation of the IV/DV and Manipulated/Measured distinctions  An alternative approach:  IV is sometimes defined more loosely, as the variable that is hypothesized to be the cause  It can be either manipulated or measured  In an experimental study:  IV = manipulated variable  DV = measured variable  E.g., induce different levels of anxiety in people and then measure their alcohol consumption  In a non-experimental study:  IV = measured variable  DV = measured variable  E.g., measure people’s levels of anxiety and also measure alcohol consumption  With this approach, can talk about a “measured independent variable” From Conceptual Variable to Operational Definition Conceptual Variables and Operational Definitions  Conceptual definition (AKA the construct)  Like the definition found in a dictionary  Hunger = having a desire for food  Operational definition  Specifies precisely how the concept is measured or manipulated in a study  Hunger = Number of hours of food deprivation  Hunger = Scale rating in response to the question, “How hungry are you?” (1=not at all, 5 = extremely  RESEARCH METHODS IN PSYCHOLOGY Evaluating A World of Information, Fourth Edition Copyright © 2021, W. W. Norton & Company, Inc. Examples of Operational Definitions Conceptual Variable Possible Operational Definitions Hunger Number of hours since last eating A scale rating in response to the question "how hungry are you (1=not at all, 5 = extremely) Anxiety A physiological measure such as heart rate A self report of anxiety level on a rating scale. Observers look for signs of anxiety in the person (e.g., shaking, avoiding eye contact, etc.). Depression Individual's self report of experience of depression Score on a validated psychometric scale, such as the depression scale of the MMPI Physiological measures Evaluation by a therapist Aggression in children Ratings of aggressive behavior made by the child's teacher Direct observation during play periods of the number of times a child hits, pushes, or forcibly takes toys from other children Child's rate of hitting a punching doll in an experimental situation The number of acts of aggression in stories created by participants in response to pictures Intelligence Score on standardized IQ test Judgment by others of person's ability to solve problems Grades in school  Exercise: Operationalizing Variables For each conceptual variable, try to think about how a researcher might operationalize it as (a) a measured variable, and (b) a manipulated variable. 1.Alcohol consumption 2.Current mood 3.Motivation to perform well on a task 4.Test studying strategy (distributed or massed) Three Claims & Four Validities Framework  Three Claims & Four Validities Framework What is a research “claim”? A claim is an argument someone is trying to make  Morling uses this term, primarily, to refer to an argument or conclusion that is made based on the results of a study  i.e., claims made by researchers or journalists  Also used sometimes in text to refer to the hypothesis that has been proposed Three Claims  Frequency claims  Association claims  Causal claims Frequency Claims  A frequency claim describes a particular rate or degree of a single variable  Frequency claims involve only one measured variable  Usually supported by a survey or poll Frequency claim example  National survey of American high school seniors  58% reported they texted or emailed while driving in the past month Association Claims  An association claim argues that one level of a variable is likely to be associated with a particular level of another variable  Association claims involve at least two measured variables  Variables that are associated are said to correlate  Usually supported by a correlational study Positive Association Negative Association Zero Association Causal Claims  A causal claim again argues that one level of a variable is associated with a particular level of another variable.  Causal claims again involve two variables.  But they go further – claiming that one of the variables is responsible for changing the other.  One variable affects the other variable  The association between variables is causal  What do researchers mean by causation?  A pragmatic meaning (cf. philosophical)  All else being equal, changes in one variable (x) produce changes in another variable (y)  The one variable influences the other variable  Causal language  Produces, affects, has an effect, has an impact, leads to, results in, contributes to, determines, etc.  Enhances, improves, decreases, etc.  Non-causal language  Is related to, is associated with, predicts, correlates with, covaries with, is a correlate of, etc. Association Claim Verbs​ Casual Claim Verbs​ Casual Claim Verbs​ Is linked to​ Causes​ Promotes​ Is at higher risk for ​ Affects​ Reduces​ Is associated with​ May curb​ Prevents​ Is correlated with​ Exacerbates​ Distracts​ Prefers​ Changes​ Fights​ Is more or less likely to​ May lead to​ Worsens​ May predict​ Makes​ Increases​ Is tied to​ Sometimes makes​ Trims​ Goes with​ Hurts​ Adds​ RESEARCH METHODS IN PSYCHOLOGY Evaluating A World of Information, Fourth Edition Copyright © 2021, W. W. Norton & Company, Inc.  Exercise: Which type of claim? People who work at home are more productive. Listening to classical music at a young age improves math ability. As people get older they are less likely to change their political attitudes. Most high school students have tried marijuana. Exposure to violent television increases aggressive behaviour. Your grades in high school may predict how long you live. The current approval rating for Prime Minister is below 50%. 80% of women feel dissatisfied with how their bodies look. Eating blueberries for breakfast can improve cognitive function.  What do researchers mean by causation? Criteria for Causation Three criteria for concluding that one variable causes or influences another variable: 1. Covariance – The study must show that the causal variable and the outcome variable are related. 2. Temporal precedence - The study must show that the causal variable came first in time, before the outcome variable. 3. Internal validity – The study must establish that no other explanations exist for the relationship between the two variables. The EXPERIMENT is the type of study that can best satisfy these three criteria for demonstrating cause and effect. - Causal claims are best supported by an experimental study Experimental vs. Correlational Research Experimental Correlational Presumed cause: Manipulated Measured (IV) (IV;Predictor) Presumed effect: Measured Measured (DV) (DV;Criterion)  Importance of Causation  Explaining behaviours  Goal of much research is to explain WHY a certain type of behaviour occurs  This essentially means we are trying to identify CAUSES of the behaviour  E.g.: Why do some children act aggressively? Are violent video games a cause?  Controlling behaviours  Sometimes a further goal is to be able to control or alter the behaviour  Only possible if we have first identified causes  Then we can know that changing the causal variable will change the target behaviour  Example: If violent video games cause aggression, this tells us that reducing exposure to violent video games could reduce aggressive behaviour  Four Types of Validity Construct validity – How well a variable was measured or manipulated in the study. Statistical validity (i.e., statistical conclusion validity) – The extent to which a study's statistical conclusions are accurate and reasonable. Our confidence in a finding should relate to the strength of the effect and the precision of its estimate. Also depends on whether the study has been replicated or not. Point estimate and margin of error (confidence interval) Improves with multiple estimates Related to statistical significance External validity (i.e., generalizability) – How well the results of a study generalize to, or represent, people or contexts beyond those in the original study. Internal validity – A study's ability to rule out alternative explanations for a causal relationship between two variables (applies only to causal claims).  Exercise: Which type of validity? 1. Did the items on the scale really measure participants' current self-esteem as intended? 2. How did the researchers get their sample for this study? 3. How big was the effect of the manipulation on the participants' behavior? 4. Did the study control for alternative explanations for the effect of the manipulated variable?  Exercise: Which type of validity? 5. How well did the researchers measure the two variables that were correlated? 6. The difference between scores in the control group and experimental group is very small and the margin of error is large. 7. Was the sample representative of the population? 8. Can we be sure it was the independent variable that influenced the dependent variable, or was it something else? Table 3.6: Interrogating the Three Types of Claims Using the Four Big Validities Validity of Frequency Claims  Example: 80% of university students have experienced depression during past year  Construct Validity How well was the variable measured?  Statistical Validity What is the margin of error of the estimate?  Internal Validity Not relevant because it is not a causal claim  External Validity (AKA generalizability) Would the findings generalize to other people or contexts beyond the study itself? How representative is the sample; was it a random sample? Validity of Association Claims  Example: People with higher incomes spend more time socializing  Construct Validity  How well were the two variables measured?  Statistical Validity  How strong is the association? (What is the effect size?)  How precise is the estimate of the correlation? Is the confidence interval very wide?  Are there estimates from other studies?  Internal Validity  Not relevant because it is not a causal claim  External Validity (aka generalizability)  Would the association generalize to other people or contexts beyond the study itself?  How representative is the sample; was it a random sample?  Validity of Causal Claims Example: Anxiety leads to poor task performance Construct Validity How well has the researcher manipulated or measured the two variables? Statistical Validity What is the effect size? How large is the difference between groups? How precise is the estimate? Is the confidence interval wide? Are there estimates from other studies? Internal Validity Was the study an experiment? Does the study achieve temporal precedence? Does it rule out alternative explanations by random assignment to groups? Does the study avoid common threats to internal validity? External Validity (aka generalizability) Would the effect generalize to other people or contexts beyond the study itself? How representative is the sample; was it a random sample? Criterion​ Definition​ Covariance​ The study’s results show that as A changes, B changes; e.g., high levels of A go with high levels of B, and low levels of A go with low levels of B. Temporal precedence​ The study’s method ensures that A comes first in time, before B. Internal validity​ The study’s method ensures that there are no plausible alternative explanations for the change in B; A is the only thing that changed. RESEARCH METHODS IN PSYCHOLOGY Evaluating A World of Information, Fourth Edition Copyright © 2021, W. W. Norton & Company, Inc. Experiments Can Best Support Causal Claims  Independent variable is manipulated  Dependent variable is measured  Random assignment Prioritizing Validities  Which of the four validities is the most important?  It depends on what kind of claim the researcher is making and the researcher’s priorities. Prioritizing Validities It’s impossible to find a study that satisfies all four validities at once. This means that researchers must decide on their priorities according to the goals of the study. Example: External validity is typically a top priority when making frequency claims, but may be less crucial when making association or causal claims Example: Internal validity is typically a top priority when making causal claims but is not relevant when making frequency or association claims. Table 3.6: Interrogating the Three Types of Claims Using the Four Big Validities  Prioritizing Validities Frequency Association Causal Construct Construct Construct Construct Statistical Statistical Statistical Statistical Internal ---- ---- Internal External External ---- ----