PSYC 306 Chapter 13 Past Paper (Oct 10, 2024) - McGill PDF

PSYCH 306 Research Methods in Psychology Chapter 13: The Descriptive Research Strategy 1  Oct 10 Midterm Exam Exam format: 30 multiple choice (answers go on scantrons), 2 short answer questions (answers go on exam). There will be...

PSYCH 306 Research Methods in Psychology Chapter 13: The Descriptive Research Strategy 1  Oct 10 Midterm Exam Exam format: 30 multiple choice (answers go on scantrons), 2 short answer questions (answers go on exam). There will be multiple versions. Preparation for exam day: Bring a lead pencil and eraser, and your McGill ID. Bring as few belongings as possible. Be prepared to turn off phone. No dictionaries or notes are permitted. No questions will be answered other than "I have an illegible exam" The invigilators will have extra copies to distribute. NOTE: psychometric analysis will determine which (if any) questions are poor; those questions will be removed. No one is allowed into the room after 3:05 (30 minutes into exam). Sorry, no washroom visits (see instructor about accessibility office). No one is allowed to leave the room before 3:05. 2  Oct 10 Midterm Exam Academic Integrity Statement: McGill University values academic integrity. Therefore, all students must understand the meaning and consequences of cheating, plagiarism and other academic offences under the Code of Student Conduct and Disciplinary Procedures (see www.mcgill.ca/students/srr/honest/ for more information). By answering the questions on the Exam on Oct 10, 2024, each student agrees to the terms of the exam outlined above. Students must take the exam in one of 3 rooms - Important to report to the correct room - your exam copy will not be in another room! Last Names Go to Exam Room A - Da* Room 461 at 2001 McGill College De*- Rh* Room 304 McConnell Ri* - Z Room 11 McConnell 3  Outline Different Descriptive Research Designs Observational research designs Case study research designs Survey research designs 4 Quantitative Research Strategies grouped under three categories, according to research goal 5 Descriptive Strategy Goal: to describe a single variable or set of variables when several are involved Measure variables in natural environment to obtain a description of that variable Not interested in relationships/associations between variables Results capture interesting, naturally occurring behavior 6 Descriptive Strategy How often do viewers watch or stream movies per week? Marketing goal: Advertizers may want to target by age group Every day Several times per week About once per week Several times per month About once per month Less often than once per month Never 0% 20% 40% 60% 80% 100% 18-34 years 27% 28% 20% 9% 8% 6%2% 35-44 years 30% 30% 17% 7% 6%4%6% 45-64 years 16% 30% 18% 14% 9% 5% 8% 65 years and older 15% 21% 16% 15% 10% 10% 13% Online survey, Statistica.com 7 Descriptive Strategies 3 Types: 1) Observational study - Naturalistic, participant, and contrived observations 2) Case Study - rare and unusual cases, counterexamples 3) Survey Study- online or in person (participants choose the study, instead of study choosing participants) 8  Observational Data (from Police): Montreal Collisions - Pedestrians, Cyclists, Drivers Injury 2022 Death 2022 Green = Average (I+D) 2017-2021 What can we conclude? Late summer months (August, September, October) seem worse City of Montreal made more one-way streets to reduce collisions https://experience.arcgis.com/experience/5a9178ca 9 2ab2418bab12ed61bc229580/ Methods for quantifying Observations Frequency method How often a specific behavior occurred Duration method How much time was spent in a particular behavior Interval method Dividing observation period into a series of intervals and recording # intervals in which the behavior is observed 10  Frequency Method Leading websites in Canada, September-November 2022, by # visitors (millions) 180 171 160 Unique visits in millions 140 130 120 100 80 58.5 58.1 53.7 60 52.5 52.1 44.1 39.5 37.8 40 28.9 26 23.5 21.9 21.6 20.7 18.5 16.9 13.9 13.2 20 0 4 Conclusion: More individuals used Google than Youtube 11  Duration Method Leading websites in Canada, September - November 2022, by session duration (in minutes.seconds) 40 37.22 Time spent per visit in 35 minutes.seconds 30 24.51 23.28 25 22.33 20.19 19.09 20 17.57 17.18 16.07 15.26 14.59 15 13.02 12.52 12.07 11.51 11.45 11.17 10.5 10.29 10 7.55 5 0 People watched Youtube for the longest duration - why? 12  Interval Method Divide the entire observation time period into multiple smaller parts of equal duration, called intervals. Count the number of times the behavior of interest occurred in each interval. Requires less time and shorter concentration by experimenters Often used to generalize the rate or # behaviors per interval to other intervals. Can be applied to entire time period or to individual intervals. 13 3 Types of Observational Designs A. Naturalistic observation – No researcher intervention B. Participant observation – Researcher interacts with participants and becomes one of them. C. Contrived observation – Researcher sets up a situation likely to produce the desired behavior in participants. 14 A) Naturalistic Observation When a researcher observes in a natural setting, and is as inconspicuous as possible Provides insight into real-world behaviour – High external validity – Examines behaviours that cannot be manipulated Time consuming – Must ensure you do not influence behaviour 15  B) Participant Observation When a researcher interacts with, and becomes part of, a participant group (overt/covert) Used when natural or inconspicuous observation is impossible Example: firefighters in fire station Gives researcher a unique perspective by having same experience Can gather information that is not otherwise available High external validity 16 B) Participant Observation Ethical considerations? Extremely time-consuming, expensive Bias – Interpreted by single observer – Views influence interpretation – May lose objectivity by being part of group Reactivity: – Observer participates…may influence behaviour/what is being observed 17 C) Contrived Observation When a researcher arranges a setting to specifically facilitate or elicit a particular behaviour (HW3 reading is a good example) Can be in laboratory or public arena Get behaviour to occur in more timely fashion: – Do not have to wait for behaviours to occur, but less natural Example: At what age do children understand object constancy? Experimenter presents toys that are occluded and re-appear 18 C) Contrived Observation Two types of contrived observations: 1) In public setting people must not have an expectation of privacy in their observed behaviors Example: Grocery shoppers putting items in carts; marketer "plants" some test products on shelves and measures whether shoppers read ingredients before choosing those items Counterexample: naturalistic observation of “bathroom behavior”: people have a reasonable expectation of privacy even in a public restroom Therefore, does not meet criteria for contrived observation 19 C) Contrived Observation 2) In a more artificial environment such as a classroom Example: researchers wish to measure people’s verbal responses to motivational ads that appear on screens in a classroom 20  Observational Research Designs Strengths and Weaknesses Research Strengths Weakness Design Naturalistic Behavior observed in the real world Time-consuming observation Useful for nonmanipulated behaviors Potential for observer Actual behaviors observed and influence recorded Potential for subjective interpretation Participant When natural observation is impossible Time-consuming observation Get information not accessible Potential for loss of objectivity otherwise Increased chance for observer Participation gives unique perspective influence Contrived Do not have to wait for a behavior to Less natural observation occur 21 Issues in Observational Research Two measurement issues: 1) Controlling for experimenter's influence on participants possible risks of demand characteristics and participant reactivity 2) Controlling for experimenter’s subjective interpretations possible risks of reliability of data 22 1) Participant Influence Must ensure that participants’ behaviours are not influenced by researcher Conceal observer – Ok for public places/behaviours Habituation: Repeated exposure before measuring until presence is no longer a novel stimulus e.g., in a classroom 23 2) Subjective Interpretations Must ensure experimenter judges behaviors in a consistent way (reliability) Use preconceived and well-defined categories of behaviours Enable observers to know exactly what to look for, and provide an operational definition of the measure Use multiple, well-trained observers 24 2) Subjective Interpretations Example: aggressive behaviors in classroom 25 Types of Indirect Observational Designs Observations of behaviors in different media outlets or documents Example: number of aggressive behaviors in a tv show Types: 1. Content analysis 2. Archival research Both use INDIRECT techniques of observation 26  1) Content analysis – Applying observational techniques to media such as movies, TV, literature Often turns qualitative data into quantitative measures Example: Rate of violent behaviors shown in children’s different shows (educational tv, cartoons) 27 1) Content analysis Steps for applying a content analysis: 28  1) Content analysis Example: Public Twitter profiles in Canada were analyzed for language that reflected bias or partiality in content about vaccines AND COVID Findings: 15% of Twitter profiles contained biased language about vaccines and COVID. Themes: - Concerns over safety - Suspicion of forces driving the pandemic or vaccine development - Anti-vaccine or confusing messages from authority figures - Mistrust toward the medical community Outcomes: Leaders developed public health information to respond to people’s concerns Griffith et al (2021), J Medical Internet Research 29 1) Content analysis Another example of Content analysis: Search newspapers for language that reflects bias or partiality in content about a political party Type Strengths Weaknesses Content analysis Unobtrusive Time-consuming if done measurement, by hand; difficult to inexpensive, close to automate; can increase original source error 30  2) Archival research - Using previously compiled information to answer research questions (government records, census, news articles, sports records) - Researcher collects no new data Example: prevalence of different mental health disorders presenting in hospitals More on these methods in April lectures 31  2) Archival research 3 main types of archives: a) Census data and statistical records Public and private collections of statistical records Statistics Canada, RCMP, Center for Disease Control (US) Example: incidence of depressive disorders across Canada based on Canadian Community Health Survey (Stats Can) Statistique.Quebec.ca Statcan.gc.ca 32  2) Archival research 3 main types of archives: b) Survey Archives A repository of survey results collected across time and regions Example: 700 universities world-wide contribute to Inter-university Consortium for Political and Social Research (large collection of completed surveys) Researchers can deposit data or access data 33 2) Archival research 3 main types of archives: c) Written records and mass media Stored documents that represent cultural products. Includes political speeches; anthropological reports of other cultures; books; magazines; movies; newspapers Examples: Comparing hate speech contained in political speeches with the incidence of murder by year Analyzing Tweet content (Reddit, Facebook, etc) about Hockey Night in Canada (CBC) to capture trends in Canadian culture 34  Descriptive Strategies 3 Types: 1) Observational study - Naturalistic, participant, and contrived observations 2) Case Study - rare and unusual cases, counterexamples that break a rule or tendency 3) Survey Study- online or in person (participants choose the study, instead of study choosing participants) 35 2. Case Study Design Idiographic approach: – Intensive examination of a single person by observation, interview, self-report Usually applied to clinical psychology: – Examine outcome of individual patient History, current behaviour, changes in behavior Case studies can involve treatment manipulations 36 Case Study Design Data is usually descriptive (qualitative) in the form of a report Two applications: Rare & unusual cases Counterexamples that break a rule or general pattern 37  Rare & Unusual Cases Unusual or unique examples shed light on different processes Extremely rare forms of psychopathology example: Ex: Dissociative Identify Disorder (DID): 90% of DID cases reported physical or sexual abuse during childhood Psychotherapy can help integrate / combine the personalities 38  Rare & Unusual Cases Unusual or unique examples shed light on different processes Extremely rare forms of psychopathology example: Dissociative Identify Disorder (DID) Chris Sizemore: 55-year-old female with >2 separate personalities that controlled her behavior at different times. This case shed light on personality theory AND on successful psychotherapies for DID 39  Case Study as Counterexample Detailed description of individual to show an exception to the rule Allows researchers to refine theories or identify individual differences Multiple counterexamples can threaten the validity of a theory 40  Case Study as Counterexample Phineas Gage: railroad worker whose brain was accidentally punctured with a rod in 1848. Experienced severe cognitive & personality changes (became irritable, rude, profane) First neurological case to suggest the brain's role (frontal lobes) in determining personality, emotions, social interaction Findings overturned theories of brain function (prior theory: frontal lobes were unrelated to human behavior) J Barlow (1849) 41 Observer Bias in Case Studies Experimenter / observer may be unobjective in reporting Can horses do arithmetic? Clever Hans: said to be able to add, subtract, do word math puzzles W von Osten (1904) Hans' answers = # hoof taps "If Tuesday is the 8th day of the month, what is the date of the following Friday?" Answer = 11 taps 42 Observer Bias in Case Studies 1907: psychologist O Pfungst showed that the horse detected subtle visual cues from owner's head movements as he leaned over to count # hoof taps. W von Osten (1904) Pfungst used blinders to prevent the horse from seeing the questioner, and the horse could no longer respond correctly Today: drug-sniffing dogs detect cues from handlers (Lit et al, 2011) A protocol is established to prevent observer bias 43  Case Studies as Inspiration Case study: Kitty Genovese (1964) was murdered outdoors in Kew Gardens, New York. News reported many witnesses near the crime scene, but nobody did anything. (Truth: at least two people called the police for help). Experiment: Participants answered a questionnaire inside a room, and they were alone or with other participants Smoke came out from under the door. Response of participants was tested — who would respond and how long would it take them to report the smoke? Did it matter whether others were in the room? Latané & Darley, 1968 44  Case Studies as Inspiration Result: As the number of people present increased, the rate of responding to the smoke coming under the lab door decreased: Condition Reporting rate 1 participant 75% reported it (in 2 minutes) 1 + 2 non-reactive confeds 10% reported it 1 + 3 naïve bystanders 38% reported it (in 4 minutes) Presence of bystanders can cause "diffusion of responsibility" or "social influence": the presence of others who do not react leads participants to downplay the emergency Latané & Darley, 1968 45 Case Study Design Strengths and Weaknesses Strengths Weaknesses Not averaged over a diverse group Limited generalization Detailed description Potential for selection bias Vivid, powerful, convincing Potential for subjective interpretation Compatible with clinical work Potential for observer bias Can study rare and unusual events Can identify exceptions to the rule 46  And now for Survey Design (Descriptive Methods with Large Groups) 47  Descriptive Strategies 3 Types: 1) Observational study - Naturalistic, participant, and contrived observations 2) Case Study - rare and unusual cases, counterexamples 3) Survey Study- online or in person (participants choose the study, instead of study choosing participants) 48  Survey Research Design Researcher uses a survey to describe a particular group of individuals to obtain an accurate picture or “snapshot” More time-efficient way to gather a lot of data quickly Goal: obtain a “snapshot” of the group at a particular time Common application: companies seeking accurate descriptions of their customers 49 Common Psychology Surveys used: Anxiety surveys, Depression questionnaires, Happiness surveys, Personality surveys, Quality of life surveys, Life attitudes surveys, Mental health surveys, Couples relationship surveys 50 What Surveys are Used to Measure: – Behaviour: Exercise, eating and sleep habits – Attitudes/beliefs/opinions: Do you think Facebook facilitates communication or complicates it? Do you believe in the death penalty? – Characteristics: What is your age? Have you suffered from any major health problems in last year? – Expectations: Will you plan a vacation in the next month? Will you buy a new car this year? – Self-Classification: Are you liberal or conservative? Are you from upper, middle or lower class? 51  Types of Survey Questions Open-ended questions – Introduce a topic and allow participants to respond in their own words Restricted questions – Present a limited number of response alternatives Rating-scale questions – Require selection of a numerical value on a predetermined scale 52 Five issues in Survey Design: A) Question content B) Types of questions C) Constructing a survey D) Selecting participants E) Administration of survey 53 A) Question Content Demographic information: – What information best describes the members of your sample?  Age  Educational level  Gender  Program of study  Marital status  Profession  Ethnicity  Living situation  Income level  Languages spoken 54 Question Content For every question you ask, you will need to consider several points Avoid unnecessary questions Avoid vague questions Make sure to get only the information you need 55 Question Content 1. Is the Question Necessary/Useful? Determine what level of detail is required: Examples: Age: specific or a range? Income: specific or a range? Drinking: ounces or glasses/week? 56 Question Content 2. Are Several Questions Needed? Is the main question sufficient or are follow up questions needed? Example: Do you smoke? Have you ever smoked? How many cigarettes per day did you smoke? 57 Question Content 3. Is your Question Double-barrelled? Make sure each question asks about one topic only. Example: What do you think of the Montreal taxi service and its drivers? 58 Question Content 4. Do respondents have the needed information? Example: "What do you think of the 747 airport bus?" Respondents won't be able to answer if they are unfamiliar with the service Ask initial question: "Are you familiar with … " 59 Question Content 4. Do respondents have the needed information? Avoid unfamiliar terms or acronyms: Example: "Have you ever suffered from halitosis?" (Define terms) "What is your opinion of the new MIS program?" (spell out acronyms, Master of Information) 60 Question Content 5. Is the Question Biased or Loaded? Do not allow blind spots or personal biases to influence the wording Example: "What do you see as the benefits of increasing tuition costs?" (better: “Do you see benefits or losses to increasing tuition costs?") 61  Question Content 6. Will Respondents Answer Truthfully? Consider whether respondents will have difficulty giving a truthful answer, or whether the questions have sensitive content: Some will be sensitive about listing their exact age, income, or other personal information 62  Survey Questions: Spot the Errors Negative Too Double- Biased or Wording Complex Barrelled Loaded I enjoy studying X research methods and spending time with friends on weekends I believe teachers should X not be prevented from taking daily attendance I support Quebec's Bill X 15 I am in favor of my X teacher's whim to increase the amount of assigned reading 63 2) Types of Questions Ways in which you ask participants questions can dictate the type of answers and the types of descriptive statistics you obtain Range from complete freedom (what is your favorite type of music?) to restrictive (e.g., choose your favorite type of music from: popular; jazz; classical; dance) Restrictive questions can be transformed to rating-scale questions (for each type of music, rate your preference for it) Type of question can introduce bias into responses 64  Types of Survey Questions Open-ended questions – Introduce a topic and allow participants to respond in their own words Restricted questions – Present a limited number of response alternatives Rating-scale questions – Require selection of a numerical value on a predetermined scale 65 Types of Questions Open-ended: – Allows participants complete freedom to respond in their own words about an introduced topic “Describe your music listening behaviours over the past two weeks (duration and location during listening, type of music)" May yield different response types to same question: Subjective interpretation of answer (positive? negative?) Possible reasons for limited responses: Unwillingness, fatigue, writing level 66 Types of Questions Restrictive: – Restricts responses by giving limited number of alternatives "During the past two weeks, how many concerts have you attended or streamed?” a. None. d. Three. b. One. e. Four or more. c. Two. f. Other (enter in box) – Easy to analyze and summarize into percentages and proportions of each possible response – Blank “Other” category allows more freedom 67  Types of Questions Restrictive: – The alternatives offered may influence respondents' choices "List the type of wine you last drank" by >1200 respondents who completed one of 4 survey versions 1. No example Survey: No examples given 2. Wine Area/Region: For example, French wine, Italian wine, California wine, South American wine, etc. 3. General Category: For example: Red wine, White wine, Rose, Champagne, etc. 4. Variety Survey: For example: Cabernet sauvignon, Merlot, Chardonnay, Riesling, etc. Tournangeau et al (2016) 68 Types of Questions Restrictive: – The alternatives offered may influence respondents' choices "List the type of wine you last drank" by >1200 respondents who completed one survey version Tournangeau et al (2016) 69 Open-ended & Restricted Questions Open-ended question: “What one issue mattered most to you in deciding how you voted for president?” Restricted question: "Which of these options mattered most to you in deciding how you voted for president?" - options included "the economy" and "other (please indicate)" Those with Open-ended question: 35% responded with "economy" Those with Restricted question: 58% chose "economy" Those with Open-ended question: 43% provided a response not included in restricted options Those with Restricted question: 8% provided an "other" response 70 Pew Research Center, 2008 US election Types of Questions Rating scales: - Respondents choose numerical value on a predetermined scale (usually on horizontal line) - Easiest type to enter for numerical analysis – Often 1. IWhen called I listenatoLikert-type like music. music, I likescale, assumes to listen equal until I get distance tired. 2. between responses = interval scale measurements 3. (compare I dislike beingindividuals, describe around people whogroups, compute are listening means) to music. 4. I go to at least one concert each week. 5. I engage in behaviours that I regret when I go to concerts. – Label 6. I oftenanchors listen toand centre music point, the night and an before include positive and important negative items; easy for participants to fill out exam or event. 71 Types of Questions Rating scales: -Respondents choose numerical value on a predetermined scale (usually on horizontal line) 72 Types of Questions Rating scales: – Semantic Differential Scale: Multiple response options with opposite endpoint labels Answers can be numeric or an X placed on a line How helpful was your teacher? 73  A Likert-Type Rating Scale The opposite extremes are identified with verbal labels called anchors. Whenever questions in a series all have the same choices for responding, participants tend to use the same response to answer all (or most) of the questions. This tendency is called a response set. 74  3) Constructing a Survey Ordering the Questions 1. Which of the following best describes why you exercise: a. It relaxes me. c. Everyone else does it. b. It allows me to escape. d. I don’t exercise. Put related questions together 2. How often do you exercise? a. Rarely (0-1 times/week) c. Often (3 or more times/ week) b. Occasionally (1-2 times/week) d. Always (at least 6 times /week) 75  3) Constructing a Survey Ordering the Questions Put sensitive questions in the middle of the survey: Usimg the following scale, please rate: 15. I frequently skip exercise. 16. I have gained weight while not exercising. 76  3) Constructing a Survey Ordering the Questions Demographic questions often at beginning or end, shortened to be faster to complete: Please take a minute to fill out the following: Age: Gender: Education received (highest level): 77 4) Selecting Participants Motivated participants: Select survey participants for whom the questions are relevant. Generalizability: Do not over-restrict the survey participants Targeted representation: Make an effort to select individuals who accurately represent the group you are interested in. 78  4) Recruiting Participants Online Unique links for recruited participants: - Include link to an online survey and/or a QR code QR codes take participants to the survey automatically. QR codes can be generated in survey-creation software Avoid bots (computer programs that complete your survey): - Online survey platforms can add ‘captcha verification’ to confirm that the respondent is a human (not bot). 79  5) Administration of Survey Now you must distribute the survey to a population... how? A. Internet: – Economical (cost is low) and efficient (high response rate) – Flexibility in questions and format; can skip sections, provide many options – Sample could be biased; miss those who do not use Internet (older adults) or who do not click on weblink – Best with closed group sent via email 80 Administration of Survey If you must distribute the survey to a population... how? B. Mail: – Too large sample; complete at convenience, anonymity – 10-20% response rate; nonresponse bias (external validity) – Extremely time consuming, costly – Cover letter (importance of topic, results, responses, contact person and signature); gift (?); follow-up call/letter 81 Administration of Survey If you must distribute the survey to a population... how? C. Telephone: – Extremely time consuming: 1-to-1 – Can reach many respondents if shared among many assistants – Identify self, survey; practice and keep questions simple; beware interviewer bias 82  Administration of Survey If you must distribute the survey to a population... how? D. Interview: – For very small and specially selected individuals – Key informants; special populations Children; those with disabilities; low literacy – Time consuming and can be subject to interviewer bias 83  Administration of Survey Survey Strengths Weaknesses Type Internet Efficient to administer to a large number Expense for website surveys of participants Nonresponse bias Access to large number of individuals Sample may not be with common characteristics representative Survey can be individualized based on Cannot control composition of participant's responses the sample Mail Convenient and anonymous Can be expensive surveys Nonthreatening to participants Low response rate Easy to administer High nonresponse bias Unsure who completes the survey Telephone Can be conducted from home or office Time-consuming surveys Participants can stay at home or office Potential for interviewer bias In-person Efficient to administer with groups, 100% Time-consuming, with individual surveys response rate, and flexible (groups or interviews; risk of interviewer individual interviews) bias 84  Despite your best efforts: Reasons Given: Too long (more than 10-15 minutes) Not confidential (by the time all the background questions are answered, the person may be uniquely identified) Questions not relevant to them (make sure who sees the survey link) Survey fatigue (too many!) 85  Which Surveys are Completed? 2018 analysis of over 25,000 online surveys hosted by SurveyMonkey (not only academic research) Completion rates tended to decrease as the number of survey questions increased Completion Rate Survey Length (# questions) Liu et al (2018) 86  Which Surveys are Completed? 2018 analysis of over 25,000 online surveys hosted by SurveyMonkey (not only academic research) First question Type: Completion Rate Surveys that began with open- ended questions yielded lower completion rates open-ended = take longer Multiple Open- Other Choice Ended Liu et al (2018) 87  Despite your best efforts: 2022 meta-analysis that examined 1071 online surveys for their response rates in academic research about education. Based on published journal articles. - Average online survey response rate reported in papers was 44.1%. - Incentives did not influence response rate (gifts cards, money). Number of Surveys Meta-analysis = Statistical combination of results from multiple studies on a similar research question Wu et al (2022) 88  Who Completes Surveys? Younger participants (< =20 years old) completed surveys More students than non-students completed surveys Wu (2022) 89  Surveys, in Summary: Pros: Cons: – Convenient and usually – Potentially time consuming; easy to administer low response rate Can administer to a (nonresponse bias); unsure large number of sample in some cases – Can get a large amount of – Interviewer bias in data relatively quickly telephone and in- – If not open-ended person surveys questions: – How representative is Can summarize sample? and analyze – May not respond easily truthfully 90