Psych 291 Basic Research Methods Lecture 1 Introduction PDF

Psych 291 Basic Research Methods Lecture 1: Introduction Annika Hillebrandt, Ph.D. Welcome! Annika Hillebrandt, PhD  Office hours: Tuesdays 11:30am – 12:30am  Office: PAS 4046  Email: [email protected] 2 Agenda Overview of t...

Psych 291 Basic Research Methods Lecture 1: Introduction Annika Hillebrandt, Ph.D. Welcome! Annika Hillebrandt, PhD  Office hours: Tuesdays 11:30am – 12:30am  Office: PAS 4046  Email: [email protected] 2 Agenda Overview of the course Introductions, expectations, evaluations, etc. Intro to research methods Producing and consuming research PAGE 3 How this course works (Online) textbook Lectures  READ before coming to class  The textbook and lectures complement each other  Material covered in the lectures may be on the exams – even if it is not also covered in the textbook. Textbook (required) Morling, B. (2020). Research Methods in Psychology, 4e. New York, NY: W.W. Norton & Company, Inc. [ISBN: 978-0-393-53626-3]  Hard cover and loose-leaf versions of the textbook are available from the campus bookstore. They include the e-book version.  The e-book version of the textbook (by itself) is available at https://www.vitalsource.com. Its content is identical to the hard cover and loose-leaf versions.  On 3-hour reserve at Dana Porter library. Evaluation  TCPS2 CORE ethics tutorial 4%  Midterm #1 30%  Midterm #2 30%  Final exam 36% PAGE 6 TCPS2 CORE ethics tutorial (4%)  Tutorial about conducting ethical research with human participants  https://tcps2core.ca  Start working on it ASAP  Completion certificate due via LEARN on September 24 by 11:59pm PAGE 7 Midterm #1 (30%) Chapters 1-4 and corresponding lectures Multiple-choice and written answer questions Midterm #2 (30%) Not cumulative Covers chapters 5-9 and corresponding lectures Multiple-choice and written answer questions Final exam (36%) Not cumulative Covers chapters 10-14 and corresponding lectures Multiple-choice and written answer questions Rules and Expectations Academic Integrity  You do not have permission to share any of my materials, whether handed out in class or posted on LEARN, with other students (future or past) or on commercial websites that share course materials (e.g., Course Hero, OneClass, etc.).  This includes, but is not limited to, the following: my course outline, my slides or the publisher’s slides, lecture notes, handouts, feedback/notes from me (e.g., feedback on assignments), exam preparation sheets, transcripts or videos of lectures, presentation materials, tests, exams, and so on.  You must ask my permission to tape or record my lectures.  You may be breaking copyright and intellectual property rules. You could be assessed for academic integrity misconduct (which has many implications).  If you are in doubt, do not hesitate to ask me or to contact the Academic Integrity Office for guidance: https://uwaterloo.ca/academic-integrity/ PAGE 12 Academic Misconduct Everything you submit for this course must reflect your own individual effort. The use of generative artificial intelligence (GenAI) trained using large language models (LLM) or other methods to produce text, images, music, or code, like ChatGPT, DALL-E, or GitHub CoPilot, is not permitted in this class. Understand what plagiarism is and how to avoid it. See UW’s Policy 71 (Student Discipline). Also see the Arts Academic Integrity webpage for common but unacceptable excuses. PAGE 13 Expectations  Attend every class  Do your readings before class  Participation is not just encouraged, it is expected  Respectfully participate  Read and abide by all the regulations in the syllabus (review re-grading, academic misconduct, deferred exams, etc.)  Raise any unresolved concerns or issues to me ASAP  Practice being uncomfortable Bonus Marks for Research Experience (3%)  Maximum of 3 BONUS credits per semester (1 research credit = 1%)  Two options (can be used in combination):  Option 1: Participation in Psychology Research, Research Experiences Group  https://uwaterloo.ca/research-experiences-group/welcome-research-experiences-group-and- sona/sona-information  0.5 credit for each in-person lab study per 30 minutes  0.25 credits for each online study per 15 minutes  Studies are posted throughout the semester. Check periodically.  Option 2: Article review  1.5-2 pages  Articles can be from scientific journals, newspapers, magazines, other printed media  Contact me for approval of your chosen article before working on your review  See course outline for details! PAGE 15 Last but not least, why are we here? Why study Psychology as a Science?  It’s fascinating! o Through research, we learn more about people (and nonhumans, too) o Helps us understand, predict, intervene Psychology as a Science  We all have knowledge and strong beliefs about human behaviour and psychological processes  … however, our intuitions and “common sense” beliefs are often wrong Example: Learning Styles  Are you a visual learner? Auditory? Tactile-kinesthetic?  Most adults and nearly all educators believe in learning styles (Newton & Salvi, 2020)  Commercial market for teaching style tests, teaching guides, workshops, etc. Example: Learning Styles  “But does scientific research really support the existence of different learning styles, or the hypothesis that people learn better when taught in a way that matches their own unique style? Unfortunately, the answer is no, according to a major report published in Psychological Science in the Public Interest, a journal of the Association for Psychological Science. The report, authored by a team of eminent researchers in the psychology of learning … reviews the existing literature on learning styles and finds that although numerous studies have purported to show the existence of different kinds of learners (such as “auditory learners” and “visual learners”), those studies have not used the type of randomized research designs that would make their findings credible.” APS, 2009 Another example: Myers-Briggs Type Indicator (MBTI)  “Despite its popularity, it has been widely regarded as pseudoscience by the scientific community. The validity … of the MBTI as a psychometric instrument has been the subject of much criticism. … Psychometric specialist Robert Hogan wrote: ‘Most personality psychologists regard the MBTI as little more than an elaborate Chinese fortune cookie...’. Nicholas Campion comments that this is ‘a fascinating example of 'disguised astrology', masquerading as science in order to claim respectability’.”  Despite controversies over validity, the instrument has demonstrated remarkable influence since its adoption by the Educational Testing Service in 1962. It is estimated that 50 million people have taken the Myers-Briggs Type Indicator and that 10,000 businesses, 2,500 colleges and universities, and 200 government agencies in the United States use the MBTI. https://en.wikipedia.org/wiki/Myers%E2%80%93Briggs_Type_Indicator#Accuracy_and_validity Psychology as a Science  The scientific approach to studying psychology puts emphasis on empiricism  Objective observation of a phenomenon Psychology as a Science  Research producers  Design, conduct, analyze, and present original research projects  Many of you are/will be research producers  Working in a lab; doing an Honours Thesis Psychology as a Science  Important to become a critical consumer of research too  Read research and apply it to work, hobbies, relationships, or personal growth  Academic articles  Popular press Empiricism separates science from pseudoscience Bad research can have serious consequences  Being a bad producer or consumer of research can have harmful consequences The Mozart Effect www.bbc.com/future/story/20130107-can-mozart-boost-brainpower The Mozart Effect  January 13, 1998. Governor Zell Miller requests $105,000 of taxpayer money to provide classical music CDs to every child born in the state of Georgia.  Commercialization Producing and Consuming Research BEFORE next class…  Read the course outline!  Read the textbook, pp. Ch. 1, pp. 10-21 PAGE 30  Comic source: XKDC Psych 291: Basic Research Methods Lec 2: How Psychological Science Advances Annika Hillebrandt, Ph.D. Textbook (required) Morling, B. (2020). Research Methods in Psychology, 4e. New York, NY: W.W. Norton & Company, Inc. [ISBN: 978-0-393-53626-3]  Hard cover and loose-leaf versions of the textbook are available from the campus bookstore. They include the e-book version.  The e-book version of the textbook (by itself) is available at https://www.vitalsource.com. Its content is identical to hard cover and loose-leaf versions.  On 3-hour reserve at Dana Porter library Evaluation  TCPS2 CORE ethics tutorial 4%  Midterm #1 30%  Midterm #2 30%  Final exam 36% PAGE 3 Sample Multiple Choice Question  To be an empiricist, one should… a. Base one’s conclusions on direct observations. b. Strive for parsimony. c. Be sure that one’s research can be applied in a real-world setting. d. Discuss one’s ideas in a public setting, such as an online chat room. Empiricism  Remember from last class? 5 Empiricism  Using evidence from the senses and/or instruments that assist the senses (e.g., questionnaires, scales, etc.)  Systematic and rigorous observations that are independently verifiable by others PAGE 6 Empiricists’ practices  How does psychological science progress?  Theory-Data Cycles  Applied and Basic Research Questions  What’s Next?  Peer Review and the Importance of Publication  From Journal to Journalism PAGE 7 The Theory-Data Cycle PAGE 8 Theory  A statement, or a sets of statements, that describes general principles about how variables relate to one another  A theory serves as a “working truth” – it is subject to revision pending the outcomes of empirical research studies PAGE 9 Example: Piaget’s Theory of Cognitive Development PAGE 10 Example: Vroom’s Expectancy Theory PAGE 11 Example: Weiner’s Attribution Theory PAGE 12 Example: Maslow’s Hierarchy of Needs WARNING! Largely unsupported by research! Research question  The question that you want to answer through research  Focuses on a specific topic or problem  (Somewhat) specific  Open-ended PAGE 14 Hypothesis development  A hypothesis is a prediction  A formal and precise statement about the relationship between variables PAGE 15 Data collection  Once you have formulated your hypothesis/hypotheses…  Measure your variables of interest  Using a suitable research method (more on this later!)  E.g., a recall study, vignette, experiment, etc. PAGE 16 Data analysis  The data may support the hypothesis  But this does not “prove” that the hypothesis (or theory) is correct!  Science relies on replication – we consider the “weight of the evidence”  The data may fail to support the hypothesis  But cannot “prove” that the hypothesis is “wrong”!  There may be other reasons you failed to find an effect (e.g., methodological issues, etc.)  It is impossible to prove the absence of a relationship or effect (just because you may never have seen a black swan, doesn’t mean black swans don’t exist!) PAGE 17 Example: Interpersonal Effects of Emotions in Negotiation Emotions in Computer- Mediated Negotiation E.g., Van Kleef, de Dreu, & Manstead, 2004; Hillebrandt & Barclay, 2017 Example: Interpersonal Effects of Emotions in Negotiation This offer makes me really angry! Example: Interpersonal Effects of Emotions in Negotiation This offer makes me really happy! Example: Interpersonal Effects of Emotions in Negotiation  What was the theory? PAGE 21 Example: Interpersonal Effects of Emotions in Negotiation  What was the theory?  Emotions as Social Information Theory (Van Kleef, 2009) PAGE 22 Research Question and Hypotheses – An Example Research question? Research Question and Hypotheses – An Example Research question? E.g., how are negotiators’ offers influenced by the emotions expressed by their opponent? Research Question and Hypotheses – An Example Hypothesis A formal statement of the expected relationship between two variables Example: “Negotiators are less demanding when an opponent expresses anger than when an opponent expresses happiness” Van Kleef, de Dreu, & Manstead, 2004 Example: Relative Deprivation versus Absolute Deprivation Two theories about income and societal unrest  Absolute deprivation: Poverty causes societal unrest  Relative deprivation: Feeling poor relative to others causes societal unrest https://www.nytimes.com/2017/06/03/opinion/sunda y/what-monkeys-can-teach-us-about-fairness.html PAGE 27 Example: Relative Deprivation versus Absolute Deprivation Two monkeys (“A” and “B”) were taught to hand over pebbles. For each pebble they handed over, they received a piece of cucumber. Monkeys don’t like cucumber very much, but they accepted this reward. The researcher then randomly selected monkey B to receive a better deal: a grape in exchange for a pebble. Monkey A continued to get cucumber. The researchers measured (observed) Monkey A’s dissatisfaction. What does absolute deprivation theory predict? How about relative deprivation theory? PAGE 28 Example: Relative Deprivation versus Absolute Deprivation PAGE 29 Example: Relative Deprivation versus Absolute Deprivation  Which theory was supported by these results?  What does this mean for the two theories? PAGE 30 Exercise What was the research question? What was the hypothesis? PAGE 31 Good Theories  Supported by data  Repeatedly and across different designs, measures, samples  Falsifiable  Lead to hypotheses that, when tested, could fail to support the theory  Parsimonious  Simplicity  Better theory is one that explains phenomenon with a minimum number of assumptions PAGE 32 Applied versus Basic Research Questions  Basic Research  Build and enhance general body of knowledge  Not intended to address a specific, practical problem  But often the basis for later applied studies PAGE 33 Applied versus Basic Research Questions  Applied Research  Focus on practical problems  Findings to be directly applied to solution of problem in real-world context  Example: Chang et al., 2019  Does diversity training in organizations reduce bias against women and minorities?  Used existing knowledge (i.e., theory) to develop an intervention  Randomly assigned employees to diversity training versus a control condition (psychological safety training)  Measured the results on employees’ attitudes and behaviors https://www.pnas.org/doi/10.1073/pnas.1816076116 https://hbr.org/2019/07/does-diversity-training-work-the-way-its-supposed-to PAGE 34 Applied vs. Basic Research Questions Theory of Behaviour Study of Accumulate Basic Body of Behaviour Knowledge = translational research Accumulate Generate Body of Applied Applied Research Knowledge Question Asking “What’s next” questions  Unanswered or new questions generated based on the results of completed studies  Why/how does this happen?  Under what circumstances is this effect most likely to occur?  Is this effect stronger/weaker for some specific population?  What are the long-term outcomes of this effect?  Etc.  Example  US born Latino high school graduates enroll in university at nearly the same rate as whites but are much less likely to earn college degrees  What’s next? PAGE 36 Peer-Review and Publication  Peer-reviewed journals  Editor sends manuscript to 2-4 experts on the subject  Experts provide feedback to authors and make recommendation to editor regarding publication  Accept  Revise & Resubmit  Reject PAGE 37 From Journal to Journalism  Scientific journals have a relatively limited readership vs. popular press journalism  A very important process  for both scientists and the public PAGE 38 From Journal to Journalism  A very important process  but one that can be dangerous PAGE 39 The Mozart Effect Rauscher, F. H., Shaw, G. L., & Ky, C. N. (1993). Music and spatial task performance. Nature, 365, 611-611. PAGE 40 The Mozart Effect  “Thirty-six college students participated in all three listening conditions … In the music condition, the subject listened to 10 min of the Mozart piece. The relaxation condition required the subject to listen to 10 min of relaxation instructions designed to lower blood pressure. The silence condition required the subject to sit in silence for 10 min. One of three abstract reasoning tasks taken from the Stanford-Binet intelligence scale was given after each of the listening conditions. The abstract/spatial reasoning tasks consisted of a pattern analysis test, a multiple-choice matrices test and a multiple-choice paper-folding and cutting test.” Rauscher et al., 1993, p. 611 What were some factors that limited the generalizability of the findings? PAGE 41 From Journal to Journalism  “The enhancing effect of the music condition is temporal, and does not extend beyond the 10-15-minute period during which subjects were engaged in each spatial task. Inclusion of a delay period (as a variable) between the music listening condition and the testing period would allow us quantitatively to determine the presence of a decay constant. It would also be interesting to vary the listening time to optimize the enhancing effect, and to examine whether other measures of general intelligence … would be similarly facilitated. Because we used only one musical sample of one composer, various other compositions and musical styles should also be examined. We predict that music lacking complexity or which is repetitive may interfere with, rather than enhance, abstract reasoning. Also, as musicians may process music in a different way from non-musicians, it would be interesting to compare these two groups.” Rauscher et al., 1993, p. 611 PAGE 42 From Journal to Journalism  Burden is on reporter to carefully and accurately present the research  Burden is on scientist to communicate clearly about what research does (and does not) show  Consumer needs to judge whether the story is important and presented accurately PAGE 43 From Journal to Journalism  Scientists can also communicate their results directly to the public  E.g., write op-eds for newspapers, give public talks, radio/TV interviews, publish in non- scientific journals  See the Chang et al. (2019) study earlier in the slides – the authors published their work in a highly reputable peer-reviewed journal (PNAS) and published a plain-language version in HBR PAGE 44 Next class…  Read Chapter 2  Research vs. Personal Experience and Intuition PAGE 45 Psych 291: Basic Research Methods Lec 3: Research vs. Personal Experience and Intuition Annika Hillebrandt, Ph.D. Comic source: 1 XKDC Sources of Information  Where do our beliefs about psychological constructs typically come from?  Experience  Intuition  Authority Why is empirical research better (i.e., more reliable) than all three of these other sources? 2 Experience as a Source of Knowledge  Our own (and our friends’ and family members’) experiences are a very powerful source of information  What are some potential problems with relying on our experiences for knowledge? PAGE 3 Experience as a Source of Knowledge  Need a comparison group  Need to have a way to compare “your experience” to a condition equivalent to “not your experience”  What happens with and without the thing you are interested in?  Historical example  Benjamin Rush and bloodletting as a disease cure  Some people got better after procedure, some did not  A challenge to his theory?  What additional information do you need to test theory? PAGE 4 Experience as a Source of Knowledge Experience as a Source of Knowledge = comparison group How does inclusion of comparison group change your interpretation? Experience as a Source of Knowledge  Some professors ban laptops in class because they believe laptops distract students and reduce students’ attention to class material. What conditions are necessary to test that theory? PAGE 7 Experience as a Source of Knowledge  Experience is confounded with many other life events Independent Variable Outcome Confounding Variable Experience as a Source of Knowledge  Experience is confounded with many other life events Getting vaccinated Covid-19 infection Mask wearing Hand washing Demographics Etc. Example #2: Emotions in Negotiation Hypothesis: “Negotiators perform worse (i.e., make larger concessions) when their opponent expresses anger” Adapted from: Van Kleef, de Dreu, & Manstead, 2004 Example #2: Emotions in Negotiation This offer makes me really angry! Example #2: Emotions in Negotiation Adapted from Van Kleef, de Dreu, & Manstead, 2004 Example #2: Emotions in Negotiation This offer makes me really happy! Example #2: Emotions in Negotiation I have received your offer. PAGE 14 Van Kleef, de Dreu, & Manstead, 2004 Sources of Information  Where do our beliefs about psychological constructs typically come from?  Experience  Intuition  Authority Intuition as a Source of Knowledge  Major problem is that intuition is biased by 1. Cognitive biases 2. Motivational biases PAGE 17 Intuition as a Source of Knowledge  Cognitive Biases a. Some explanations make for a good story  e.g., stomach ulcers caused by stress & acid; bloodletting cleanses the body b. The present/present bias  Notice when two events coincide, don’t notice it when they don’t c. The availability heuristic  Things that come easily to mind guide (and bias) our thinking PAGE 18 Availability Heuristic PAGE 19 Intuition as a Source of Knowledge  Motivational Biases (or thinking what we want)  Confirmation bias  Bias blind spot PAGE 20 Sources of Information  Where do our beliefs about psychological constructs come from?  Experience  Intuition  Authority PAGE 21 Authorities as a Source of Knowledge  Why can authorities not be sole source of information? PAGE 22 Summary: Common Sources of Knowledge PAGE 23 How to Find (Reliable) Empirical Research  Three primary sources to locate psychological research:  Journal Articles  Chapters in Edited Books  Books PAGE 24 Journal Articles  Empirical Articles vs. Review Papers  Considered the most prestigious form of publication in psychology  Empirical article presents original data  Review article presents a summary of all the research done in an area  May include some summary statistics from past studies via a meta-analysis (instead of new data)  Meta-analysis combines results of prior studies to summarize the effect size of a relationship PAGE 25 Journal Articles  To judge whether an article is empirical, look for:  A statement about the methodology being used  A statement of research questions being addressed  A definition of the group or phenomenon being studied  A description of the method of assessment (measures, tests, surveys)  Tables/Figures describing results Main question is whether you could replicate the study based on the information provided Book Chapters  Chapters published as part of an edited book  One (or more) editors who invite experts in the field to contribute chapter on set of inter- related topics  Chapters provide an overview of theory in a specific area of research and a review of findings in that area Full-Length Book  Full-length scientific books least common way primary research findings are communicated in psychology  Unlike other disciplines (e.g., humanities) Finding Scientific Sources  http://subjectguides.uwaterloo.ca/psychology Reading an Empirical Article  Abstract  Good for overview but not enough to rely on exclusively  Introduction  Focus on intro paragraph, research questions, hypotheses  Method  Focus on participants and measures  Results  Descriptive statistics and hypothesis testing. Look for Tables/Figures.  Discussion  Support for hypotheses? Limitations? Future Directions?  Reference Section  Great resource! Cautions about Popular Press  Generally not written by research producers  Although there are exceptions  Target audience  Not students/professionals trained in the field  Look carefully at sources of material  Are the opinions expressed based on data or are they based on author’s intuitions and personal experiences??  Look for inclusion of many credible sources Your turn! Design a study  Research question?  Hypothesis?  Population?  How would you test the hypothesis?  One group or comparing multiple groups? PAGE 32 Next class…  Variables; Frequency, Association, and Causal Claims  Ch. 3, pp. 55-65 PAGE 33  Comic source: XKDC Psych 291: Basic Research Methods Lec 4: Variables; Frequency, Association, and Causal Claims Annika Hillebrandt, Ph.D. 1 Sample Short Answer Question  Explain the term applied research and give an example of a topic that could be studied in an applied framework. Sample Multiple Choice Question If we use the availability heuristic, we are basing our beliefs on A. what others think. B. what we see popular people do. C. what comes to mind easily. D. bias blind spots. PAGE 3 Variables as the Foundation of Research  Variable = anything that varies  A variable has multiple “levels” (values) Var 1  e.g., Education, Depression, Gender L1 L2 L3 L4 Note: If you’re only examining a single value of a variable, you are treating it as a constant Measured vs. Manipulated Variables  Measured  E.g.,  height  weight  gender  anxiety  Manipulated  E.g.,  medication dose (0 mg vs. 2 mg vs. 4 mg vs. 6 mg)  teaching method (lecture vs. lab)  anxiety When can a variable not be manipulated? Hypotheses from last class  “Democratic leadership is associated with higher levels of employee satisfaction than authoritarian leadership.”  What are the variables? Would you measure or manipulate them? PAGE 6 Hypotheses from last class  “The longer the waitlist for AccessAbility Services (AAS), the less likely students are to seek accommodations.”  What are the variables? Would you measure or manipulate them? PAGE 7 Hypotheses from last class  “Exam review sessions improve exam performance.”  What are the variables? Would you measure or manipulate them? PAGE 8 Hypotheses from last class  Other example? PAGE 9 Constructs  Oftentimes, we want to study concepts that cannot be observed directly  For example:  Does the public trust researchers?  How fairly are employees treated by their supervisor?  We call these abstract, conceptual variables constructs  Constructs cannot be directly observed  But we can operationalize them (turn them into measured/manipulated variables)! From Conceptual Definitions to Operational Definitions Conceptual Variable Operational Definition Specific Variable to Measure Operational definition = a definition of a concept or variable in terms of precisely described operations, measures, or procedures Example: Trust Trust “The willingness to be vulnerable to the actions of another party” Multi-item scale; e.g.: “If I had my way, I wouldn’t let [person X] have any influence over issues that are important to me” (reverse-coded) Example: Motivation Motivation “The extent to which persistent effort is directed toward a goal” E.g., give participants a task and measure the amount of time they are willing to spend on it Example: Meaningfulness (Ariel’s talk) Meaningfulness “Seeing the fruits of your labours” Destroying (versus not destroying) the Bionicles that participants assembled Operationalizing Constructs Comic source: XKDC PAGE 15 Three Major Claims in Psychological Science  Frequency claim  Describes the rate or level of variable of interest  Association claim  Claims that one level of a variable is likely to be associated with a particular level of another variable  Causal claim  Change in one variable is claimed to cause change in the other Frequency Claims  Describes the rate or level of a variable  E.g., absolute numbers, percentages Frequency Claims https://uwaterloo.ca/institutional-analysis-planning/sites/ca.institutional-analysis- planning/files/uploads/files/rpt_cusc_2019_first-year_survey_executive_report_waterloo.pdf PAGE 18 Frequency Claims Average Annual Salaries of Ontario Graduates by Field of Study, 6 Months and 2 Years After Graduation in constant $2020, by Graduating Class, 2001-2018 PAGE 19 Frequency Claims PAGE 20 Frequency Claims  Defining characteristics:  Focus on a single variable  Variable is always ____________ and never _____________ (measured or manipulated?) Association Claims  Argues that one level of a variable is likely to be associated with a particular level of another variable  The same as saying two variables are correlated or that there is a relationship between two variables  e.g., height and weight are highly correlated PAGE 22 Association Claims  Defining characteristics:  At least two variables are included  Variables are always ____________ and never _____________ (measured or manipulated?) Association Claims Types of Associations a. Positive Association  A high score on one variable is associated with a high score on a second variable (and a low score on one variable is associated with a low score on the other, etc.)  e.g., Height is positively correlated with weight  e.g., Study time is positively correlated with GPA Association Claims Types of Associations (continued) b. Negative Association  High score on one variable is associated with low score on a second variable  E.g., exercise and weight are negatively correlated Association Claims Types of Associations (continued) c. Zero (non-significant) association  Score on one variable is unrelated to score on a second variable  E.g., Physician age unrelated to patient mortality  E.g., GPA and height are unrelated Scatterplot  Scatterplots show the association between two quantitative variables  Each dot represents one case (e.g., one participant)  Scatterplots display the  Form (linear or nonlinear)  Direction (positive or negative)  Strength (no, weak, strong) … of the association Scatterplot Example 6 5 Helping behavior 4 3 2 1 0 0 2 4 6 Perceived supervisor support Scatterplot Example 6 5 “Greater supervisor Helping behavior 4 support is associated with higher levels of 3 helping behavior” 2 or “Supervisor support is 1 positively associated with helping behavior” 0 0 2 4 6 Perceived supervisor support Scatterplot: Strength of the Association  As the strength of the association weakens, the points on the scatterplot move further away from the diagonal line of perfect correlation Knowing a value of x, how well can you predict the corresponding y-value? Causal Claims  Claim that one variable is responsible for changing the other  Causal claims are supported by experiments (studies that have a manipulated variable and a measured variable)  Listening to Mozart enhances IQ  Exercise ameliorates anxiety  EDI training reduces discrimination  Etc. … Causal claims  In an experiment, one variable is manipulated  The effect of this manipulation on some other variable is measured  Everything else is held constant Example: Emotions in Negotiation This offer makes me really angry! Example: Emotions in Negotiation This offer makes me really happy! Example: Emotions in Negotiation I have received your offer. PAGE 35 Van Kleef, de Dreu, & Manstead, 2004 Example: Emotions in Negotiation  Which variable was manipulated?  Which variable was measured?  Explain what how everything else was held “constant” across conditions Example: Emotions in Negotiation Manipulated Measured Opponent’s Participant’s emotion offers Correlation versus Causation  Correlation: “x is related to y” or “x and y are associated”  X and Y change together  Causal relationship: “x leads to y” or “y results from x” or “x affects y”  X changes first and then causes Y to change too Correlation ≠ Causation! 40 Hypotheses from last class  “Democratic leadership is associated with higher levels of employee satisfaction than authoritarian leadership.”  Constructs? How could you measure them?  What type of claim? PAGE 41 Hypotheses from last class  “Exam review sessions improve exam performance.”  Constructs? How could you measure them?  What type of claim? PAGE 42 Exercise: Identifying Variables and Types of Claims PAGE 43 Exercise: Identifying Variables and Types of Claims “In the screen-lit bustle of modern life, sleep is expendable. There are television shows to binge- watch, work emails to answer, homework to finish, social media posts to scroll through. We’ll catch up on shut-eye later, so the thinking goes — right after we click down one last digital rabbit hole. Brain research, which has pushed back hard against this nonchalant attitude, is now expanding rapidly, reaching beyond the laboratory and delving into exactly how sleep works in disease and in normal cognitive functions such as memory. The growing consensus is that casual disregard for sleep is wrongheaded — even downright dangerous. Preschoolers who skip naps are worse at a memory game than those who snooze, even after the children “catch up” on sleep the next night. An alarming new line of research suggests poor sleep may increase the risk of Alzheimer’s, as even a single night of sleep deprivation boosts brain levels of the proteins that form toxic clumps in Alzheimer’s patients. All-nighters push anxiety to clinical levels, and even modest sleep reductions are linked to increased feelings of social isolation and loneliness.” PAGE 44 Exercise: Identifying Variables and Types of Claims  As you read each quote, ask yourself: a) How many variables seem to be involved in this claim? b) What is/are the variable(s)? c) Is this claim frequency, association, or cause? (How do you know?) PAGE 45 Exercise: Identifying Variables and Types of Claims Quote 1:  “Preschoolers who skip naps are worse at a memory game than those who snooze.” Quote 2:  “...a single night of sleep deprivation boosts brain levels of the proteins that form toxic clumps in Alzheimer’s patients.” Quote 3:  “In 2016, the Centers for Disease Control and Prevention reported that a third of adults fail to get the recommended seven hours.” Quote 4:  “...even modest sleep reductions are linked to increased feelings of social isolation and loneliness.” PAGE 46 Exercise: Identifying Variables and Types of Claims Quote 5:  “A small study presented at the Society for Neuroscience conference last year found early evidence that playing “pink” noise — similar to white noise but with different frequencies — in a specific pattern could increase the proportion of time people spent in restorative deep sleep.” Quote 6:  “Adults over 50 with lots of insomnia symptoms were more likely to fall than those without, according to one study.” Quote 7  “Studies have linked pain to poor sleep." PAGE 47 Exercise #2: Pandemic Wellbeing https://twitter.com/SDG2030/status/1 253645596986048513 PAGE 48 Exercise #2: Pandemic Wellbeing  Tip 1: Keep social, even when socially distancing Dr. Santos states that “Happy people spend more time with other people in general. They also prioritize time with their friends and family members” a) What kind of claim is it to say that “happy people spend more time with other people in general” -- frequency, association, or cause? What are the variables in this claim? b) What kind of study was likely conducted to support this claim -- correlational or experimental? PAGE 49 Exercise #2: Pandemic Wellbeing  Tip 2: Help others To explain this tip, Dr. Santos mentions, “The research shows that doing random acts of kindness... can be incredibly powerful.” For example, happy people are giving money to charity and volunteering in their communities. d) What kind of claim is it to say that “doing random acts of kindness... can make you happy” -- frequency, association, or cause? What are the variables in this claim? e) What kind of study would need to be conducted to support this claim -- correlational or experimental? f) Can you imagine how you might manipulate the variable, “doing random acts of kindness?” PAGE 50 Exercise #2: Pandemic Wellbeing  Tip 3: Move -- even if you can't leave your house In this section, she says, “Exercise and moving your body is really correlated with improved well-being” g) The verb, “correlated” gives this one away: It’s an association claim. Dr. Santos specifies that they are “really correlated,” which seems to suggest a strong effect size. PAGE 51 Exercise #2: Pandemic Wellbeing  Tip 4: Savour everyday pleasures “Research suggests that happy people tend to be more mindful. They tend to savor the present moment; they tend to be in the present moment.” Here Dr. Santos also strongly recommends taking deep breaths to calm the vagus nerve and connect to the present. h) How might you design an experiment to test the hypothesis that deep breathing can improve people’s well-being? Whom might you recruit for such a study? How would you operationalize each of the two variables? What results would you expect to find? PAGE 52 Exercise #3: Screen Time Is Screen Time Really Bad for Kids? By Kim Tingley, Dec. 22, 2019; https://www.nytimes.com/2019/12/18/magazine/screen-time-kids-teens.html “The first iPhone was introduced in 2007; just over a decade later, in 2018, a Pew survey found that 95 percent of teenagers had access to a smartphone, and 45 percent said they were online ‘almost constantly.’ When researchers began trying to gauge the impact of all this ‘screen time’ on adolescent mental health, some reported alarming results. One widely publicized 2017 study in the journal Clinical Psychological Science found that the longer adolescents were engaged with screens, the greater their likelihood of having symptoms of depression or of attempting suicide. Conversely, the more time they spent on nonscreen activities, like playing sports or hanging out with friends, the less likely they were to experience those problems. These and other similar findings have helped stoke fears of a generation lost to smartphones. But other researchers began to worry that such dire conclusions were misrepresenting what the existing data really said. Earlier this year, Amy Orben and Andrew K. Przybylski, at Oxford University, applied an especially comprehensive statistical method to some of the same raw data that the 2017 study and others used. Their results, published this year in Nature Human Behavior, found only a tenuous relationship between adolescent well-being and the use of digital technology.” PAGE 53 Exercise #3: Screen Time a) There are two frequency claims that open this piece. What are they? b) What is the first association claim presented in this article? What are the variables in the claim? c) The second paragraph presents an association claim that contradicts the first one. What is the second association claim and how does it differ? PAGE 54  Psych 291: Basic Research Methods Lec 5: Research Claims and Their Validity Annika Hillebrandt, Ph.D. 1 Remember from last class? Conceptual Variable Operational Definition Specific Behaviors to Measure Remember from last class? Remember from last class?  Frequency Claim  Describes the rate or level of variable of interest  Association Claim  Claims that one level of a variable is likely to be associated with a particular level of another variable  Causal Claim  Change in one variable is claimed to cause change in the other Verbs for Association vs Causal Claims Association Claim Verbs​​ Casual Claim Verbs​​ Casual Claim Verbs​​ Is linked to​​ Causes​​ Promotes​​ Is at higher risk for Affects​​ Reduces​​ Is associated with​​ May curb​​ Prevents​​ Is correlated with​​ Exacerbates​​ Distracts​​ Prefers​​ Changes​​ Fights​​ Is more or less likely to​​ May lead to​​ Worsens​​ May predict​​ Makes​​ Increases​​ Is tied to​​ Sometimes makes​​ Trims​​ Goes with​​ Hurts​​ Adds​​ Identifying Claims For each statement, identify the type of claim made (frequency, association, or causal): 1. Marijuana is the most popular street drug used worldwide 2. Breastfeeding may boost children’s IQ 3. Boredom blamed for murders 4. Teen birth rate hits historic low 5. TikTok users get worse grades than non-users PAGE 6 Once Claims are Made…  How do we evaluate the validity of the claim?  Is the claim reasonable, accurate, and justifiable? The Four Big Validities PAGE 8 The Validity of Frequency Claims  Construct Validity  How well did the researchers measure their variable of interest?  Do different levels of the variable accurately correspond to true differences in the construct? Example: Construct validity of a frequency claim  “Starbucks is the most popular coffee chain in the world”  To evaluate the construct validity of this claim, you may ask yourself…  How did the researchers define their construct of interest (“popular”)?  E.g., Number of daily customers? Annual profit? The degree to which people like the coffee?  How did they measure the variable?  Was this measure appropriate, given their definition? PAGE 10 Example: Construct validity of a frequency claim  “One third of Canadian adults having sleeping problems”  To evaluate the construct validity of this claim, you may ask yourself…  How did the researchers define their construct of interest (“sleeping problems”)?  E.g., Getting < 8 hours of sleep per night? Difficulty with daily functioning due to lack of sleep?  How did they measure the variable?  Was this measure appropriate, given their definition? PAGE 11 Example: Construct validity of a frequency claim  “The University of Waterloo is Canada’s most innovative university”  To evaluate the construct validity of this claim, you may ask yourself…  How did the researchers define their construct of interest (“innovative”)?  E.g., Number of new academic programs per year? Percentage of faculty members who apply for patents?  How did they measure the variable?  Was this measure appropriate, given their definition? PAGE 12 The Validity of Frequency Claims  External Validity  How well would the claim generalize beyond the current sample?  E.g., Other people? Other contexts? Other times? Other places?  How was the sample selected?  Is the sample representative of larger population? The Validity of Frequency Claims  Statistical Validity  How accurate is the frequency claim?  Margin of error? The Validity of Frequency Claims A frequency claim with low statistical validity! PAGE 15 The Validity of Frequency Claims PAGE 16 The Validity of Association Claims  Construct Validity  How well did the researchers measure their variable of interest?  Do different levels of the variable accurately correspond to true differences in the construct? The Validity of Frequency Claims  External Validity  How well would the claim generalize beyond the current sample?  E.g., Other people? Other contexts? Other times? Other places?  How was the sample selected?  Is the sample representative of larger population? The Validity of Association Claims  Statistical Validity  How strong is the association? Is it statistically significant? The Validity of Association Claims PAGE 20 The Validity of Causal Claims  Construct Validity  Is the manipulation of the independent variable appropriate?  How well do the conditions map onto levels of the independent variable?  How well is outcome variable measured?  External Validity  How representative is the sample of the population? How well would the claim generalize to  Other people? Other contexts? Other times? Other places? The Validity of Causal Claims  Statistical Validity  Does design of study minimize probability of errors in decision making?  Is the association statistically significant?  How strong is the association?  Internal Validity  Given the design of the study, can you rule out all other alternative explanations? The Validity of Causal Claims  Three criteria for establishing causation The Validity of Causal Claims  Remember, the only type of research design that can support a causal claim is an experiment  Essential characteristics of this design? The Validity of Causal Claims  Example: music lessons and IQ in children  How would you design study to test association claim?  How would you change study design to test causal claim? PAGE 25 The Validity of Causal Claims Support for: 1. Covariance? 2. Temporal Precedence? 3. Internal Validity? From Schellenberg (2004) The Validity of Causal Claims PAGE 27 Claims and Validities – Summary Table Prioritizing some validities over others……  Depends on the question you are asking  If you want to make a frequency claim, which would you prioritize?  Why?  If you want to make a causal claim, which would you prioritize?  Why? Exercise: Does this study support a causal claim? PAGE 30 Exercise: Does this study support a causal claim? Music and risk taking What are the variables in this study? 83% Percent Accepting Risk Can this study support the claim that 57% “Happy music causes people to accept more risk”?  Covariance?  Temporal precedence?  Internal validity? Happy Sad Another one… Does this study support a causal claim? “They set up a room to be distinctively aromatic, hiding three small containers of coffee beans around the room overnight. [Next], they led 40 college students into the room one at a time to perform a tough visual-search task on a computer, finding the letter “X” or “N” in a circle of similar-looking letters (“W,” “M,” “K,” “H,” “Z,” and “V”). 40 other students completed an easier version of the same task; searching for the letter “X” or “N” among a circle of lowercase “o”s. [Students had been randomly assigned to either the difficult or easy task.] The experimenters then took the students into another room and asked them some follow up questions that grew increasingly leading : “Describe the room you just completed the task in. Try to describe it using all of your senses.” “Did you notice any odors in the room, if so what?” “Could you smell coffee in the room?” Students assigned to the difficult search task were far less likely to report having picked up the aroma (25% of participants said they noticed a coffee smell) compared to the participants assigned to the easy task (60%-70% percent of participants). When the experimenters led the students back into the test room, all of them said they could smell it. Some of them even commented that the room smelled like a café.” https://www.psychologicalscience.org/publications/observer/obsonline/ does-concentration-blunt-our-sense-of-smell.html Exercise #2: Pandemic Wellbeing https://twitter.com/SDG2030/status/1 253645596986048513 PAGE 33 Exercise #2: Pandemic Wellbeing  Tip 1: Keep social, even when socially distancing Dr. Santos states that “Happy people spend more time with other people in general. They also prioritize time with their friends and family members” a) What kind of claim is it to say that “happy people spend more time with other people in general” -- frequency, association, or cause? What are the variables in this claim? b) What kind of study was likely conducted to support this claim -- correlational or experimental? PAGE 34 Exercise #2: Pandemic Wellbeing  Tip 2: Help others To explain this tip, Dr. Santos mentions, “The research shows that doing random acts of kindness... can be incredibly powerful.” For example, happy people are giving money to charity and volunteering in their communities. d) What kind of claim is it to say that “doing random acts of kindness... can make you happy” -- frequency, association, or cause? What are the variables in this claim? e) What kind of study would need to be conducted to support this claim -- correlational or experimental? f) Can you imagine how you might manipulate the variable, “doing random acts of kindness?” PAGE 35 Exercise #2: Pandemic Wellbeing  Tip 3: Move -- even if you can't leave your house In this section, she says, “Exercise and moving your body is really correlated with improved well-being” g) The verb, “correlated” gives this one away: It’s an association claim. Dr. Santos specifies that they are “really correlated,” which seems to suggest a strong effect size. PAGE 36 Exercise #2: Pandemic Wellbeing  Tip 4: Savour everyday pleasures “Research suggests that happy people tend to be more mindful. They tend to savor the present moment; they tend to be in the present moment.” Here Dr. Santos also strongly recommends taking deep breaths to calm the vagus nerve and connect to the present. h) How might you design an experiment to test the hypothesis that deep breathing can improve people’s well-being? Whom might you recruit for such a study? How would you operationalize each of the two variables? What results would you expect to find? PAGE 37 Exercise #3: Screen Time Is Screen Time Really Bad for Kids? By Kim Tingley, Dec. 22, 2019; https://www.nytimes.com/2019/12/18/magazine/screen-time-kids-teens.html “The first iPhone was introduced in 2007; just over a decade later, in 2018, a Pew survey found that 95 percent of teenagers had access to a smartphone, and 45 percent said they were online ‘almost constantly.’ When researchers began trying to gauge the impact of all this ‘screen time’ on adolescent mental health, some reported alarming results. One widely publicized 2017 study in the journal Clinical Psychological Science found that the longer adolescents were engaged with screens, the greater their likelihood of having symptoms of depression or of attempting suicide. Conversely, the more time they spent on nonscreen activities, like playing sports or hanging out with friends, the less likely they were to experience those problems. These and other similar findings have helped stoke fears of a generation lost to smartphones. But other researchers began to worry that such dire conclusions were misrepresenting what the existing data really said. Earlier this year, Amy Orben and Andrew K. Przybylski, at Oxford University, applied an especially comprehensive statistical method to some of the same raw data that the 2017 study and others used. Their results, published this year in Nature Human Behavior, found only a tenuous relationship between adolescent well-being and the use of digital technology.” PAGE 38 Exercise #3: Screen Time a) There are two frequency claims that open this piece. What are they? b) What is the first association claim presented in this article? What are the variables in the claim? c) The second paragraph presents an association claim that contradicts the first one. What is the second association claim and how does it differ? PAGE 39 Psych 291: Basic Research Methods Lec 6: Validity continued Annika Hillebrandt, Ph.D. 1 Reminders  TCPS2 Core tutorial due TODAY by midnight  Midterm 1: October 1 during class time  45 multiple-choice questions (45 marks total)  4 short-answer questions (13 marks total) 2 Check your knowledge Three necessary criteria for causal claims are A. covariance, temporal precedence, and internal validity. B. association, construct validity, and generalizability. C. operationalization, temporal precedence, and construct validity. PAGE 3 Check your knowledge To support an association claim, a study would have to have A. one variable. B. two manipulated variables. C. two or more measured variables. PAGE 4 Check your knowledge Which of the following headlines is a causal claim? A. Holding a gun may make you think others are too. B. Younger people can’t read emotions on wrinkled faces. C. Strange but true: Babies born in the fall are more likely to live to 100. D. Check the baby! Many new moms show signs of OCD. PAGE 5 Check your knowledge Which of the following is an example of an association claim? A. 41% of people surveyed reported that they were having a good day. B. Guzzling a 20-ounce bottle of soda ramps up cellular aging. C. People who sit within two tables of the bartender have three more alcoholic drinks, on average, than those who sit three tables away. D. Viewing a recent conflict as it would look one year in the future led to increased feelings of forgiveness. PAGE 6 Check your knowledge Which of the following phrases would indicate that a researcher is making a causal claim? A. curbs B. seems to decrease C. may enhance D. All of these should be treated as a causal claim. PAGE 7 Exercise: The Effect of Divorce on Sons’ Aggression PAGE 8 Exercise: The Effect of Divorce on Sons’ Aggression  What is the authors’ main claim?  Indicate if the claim is frequency, association, or causal  Analyze the validity of the claim  Identify limitations of the study and explain how you would address them 9 PRESENTATION TITLE Limitations  How would you redesign this study? PAGE 10 Exercise: Construct Validity and External Validity 100 Humans (Netflix) Episode 2, Study 1. PAGE 11 Exercise: Construct Validity and External Validity  What are the variables?  What is the claim?  Indicate if the claim is frequency, association, or causal  Was this an experiment?  Assess construct validity  Assess external validity PAGE 12