Chapter 2_Research Methodology.pdf

Chapter 2: Research Methodology PSYC 200 PSYC 200 PSYC 201 Psychology as a biological science Psychology as a social science Let’s start with a simple warm-up question Suppose that you want to determine whether this fertilizer is effective in promoting pla...

Chapter 2: Research Methodology PSYC 200 PSYC 200 PSYC 201 Psychology as a biological science Psychology as a social science Let’s start with a simple warm-up question Suppose that you want to determine whether this fertilizer is effective in promoting plant growth. What will you do? First Step: Determine how you will measure plant growth Ways to measure plant growth: How do you define Increase in height growth operationally? Increase in number of leaves Increase in size of leaves Increase in number of fruits Increase in size of fruits Time to reach a particular size Use only one plant Subject Measurement Treatment Measurement Analysis One tomato Average leaf size Add fertilizer Average leaf Compare the plant before adding size 10 days change in fertilizer after adding average leaf fertilizer size before and after adding fertilizer Use one group of plants Single Group Repeated-Measures Design Group A Measurement Treatment Measurement Analysis Subjects Measurement Treatment Measurement Analysis One group of Average leaf Add fertilizer Average leaf Compare the tomato plants size before size 10 days change in adding fertilizer after adding average leaf fertilizer size before and after adding fertilizer Use two different groups of plants Two-random-samples design Random Sample A Treatment Measurement Analysis: Comparison Random Sample B Treatment Measurement between the groups Group Treatment Measurement Analysis One random sample of No fertilizer Average leaf size Compare the tomato plants (Group A) 10 days later difference in One random sample of Add fertilizer Average leaf size average leaf size tomato plants (Group B) 10 days later between the two groups Two-Groups & Repeated Measures Two-random-samples, repeated-measures design Random Sample A Measurement No Treatment Measurement Analysis Random Sample B Measurement Treatment Measurement Group Measurement Treatment Measurement Analysis One random sample Average leaf No fertilizer Average leaf Compare the of tomato plants size before size 10 days change in (Group A) adding fertilizer later average leaf One random sample Average leaf Add Average leaf size between of tomato plants size before fertilizer size 10 days the two (Group B) adding fertilizer later groups  Which of these design shall we use? When and why do we choose one Now the design over the others? question What are the advantages and is… disadvantages of each design? Let’s find out…  Three goals of The doing scientific Scientific research Approach Description to Behavior Explanation Predicting Operational Definition One goal of science (both natural and social) is to describe and define a phenomenon in such a way that the phenomenon can be measured quantitatively. Examples in physics: Velocity = Distance/ Time Density = Mass/ Volume Electric Field = Force/ Unit Charge (Force per unit charge) Examples in psychology: Short-term memory capacity = Number of chunks recalled in a free recall test IQ = (Mental Age/ Chronological Age) X 100 Stress = ? (depends on the tool that you use to measure it) Operational Definition Operational definition - A standardized meaning for an abstract concept. Example of an abstract concept: - intelligence What is intelligence? Mr. A might appear smart to some people but dumb to others. Different people have different definitions for intelligence.  An operational definition is necessary because it makes sure that different researchers are talking about the same thing (and hence measuring the same thing) when a term is used. Operational Definition An operational definition also specifies how an abstract concept is measured in a particular study. Operational Definition Example: In a study by Bandura, Ross, and Ross (1961), aggression in children was defined as and measured by the number of time a child punched or hit a Bobo doll (an inflated toy clown). Operational Definitions in Natural Sciences Not just social scientists use operational definitions, natural scientists like Einstein also operationalize their objects under study. The following is a quote from a physics textbook: “Most physicists hold that the properties of an object can only be defined by thinking of an experiment that can measure them.”  Evolutionary psychology Materialism Different ways Behaviorism (systems) to Cognitive psychology explain human Freudianism (psychodynamic theory) behaviors, Humanistic psychology thoughts, and Positive psychology learning Social psychology Cultural psychology  Hypothesis 1 Theory A System Hypothesis (school of 2 thoughts) Hypothesis Theory B 1  Workers will be more productive if they get a bonus after producing every 10 pieces of works Operant Conditioning Sales will increase by 20% if customers get a free reward after every 7 Behaviorism purchases When bell is paired with food, an animal will Classical Conditioning salivate to the bell after 10 trials The role of a theory A theory is an explanation or model of how a phenomenon works. Examples of theory in psychology Levels of processing Encoding specificity Classical conditioning Operant condition Psychodynamic theory Evolutionary theory Theory A theory explains how (mechanistic explanation) or why (functional explanation) something happens; or both how and why. Examples: Classical conditioning theory explains how a stimulus acquires the capacity to evoke a response that was originally evoked by another stimulus. Levels of processing theory explains both how and why some information is better remembered than other. A good theory is one that… A good theory is one that leads to a number of testable hypotheses. A good theory is one that is falsifiable. A good theory is one that is supported by data. The theory has to be testable in the first place. Only testable theories receive empirical support. A good theory is one that is parsimonious Each theory has its own assumptions. If a theory has too many assumptions (i.e., this theory is true under conditions A-Z), then it has very low predictability. The role of a theory A theory integrates apparently unrelated facts and principles into a coherent whole. Example: Why do some people have irrational fears of specific objects or situations (i.e., why do some people have phobias)? Classical conditioning theory provides an explanation and integrates the unrelated facts into a coherent whole. Steps in a scientific investigation 1) Pose a specific, testable research question. 2) Educate yourself about what is already known about your theory. 3) Formulate a testable hypothesis. 4) Design a study—select a research method. 5) Conduct the study—collect the data. 6) Analyze the data. 7) Report the findings. Step 1: Pose a specific, testable research question Examples of research questions: 1. How many % of people will follow the order of an authority and torture a stranger to death? 2. What is the relationship between level of arousal and level of performance? 3. When are people most likely to obey authority? 4. What is the role of the prefrontal cortex in self-control? 5. How does level of processing affect retention (demonstrates a causal relation)? 6. How does norm crystallization occur (demonstrates a mechanism)? 7. Can money make people happy? 8. Why do experts sometimes make stupid errors? Step 2: Educate yourself about what is already known about your theory Step 3: Formulate a testable hypothesis Formulate a testable hypothesis based on a theory. A theory describes a general phenomenon whereas a hypothesis predicts a specific observation in a specific study. Hypothesis = prediction (e.g., when X increases, Y decreases.) A hypothesis is a tentative statement about the relationship between two or more variables. The hypothesis is a tentative statement because the data might or might not support it. Step 3: Formulate a testable hypothesis (prediction) Levels of processing theory proposes that deeper levels of processing result in longer lasting memory codes (a general description). A hypothesis generated based on levels of processing theory in a study by Craik & Lockhart (1972): Retention of the stimulus words would increase as subjects moved from structural to phonemic to semantic encoding (a description of how two variables are related in one experiment). Step 4: Design a study and select a research method Step 5: Conduct the Study—Collect the data Common techniques of data collection: Direct observation Questionnaire Interview Psychological test Physiological recording Reading time/reaction time Examination of archival records Evaluating Measures Two major concepts 1) Reliability Does your instrument (e.g., blood pressure meter) give you the same reading every time you use it to measure the same thing (e.g., the blood pressure of the same individual on the same occasion). 2) Validity Are you measuring what you intend to measure (e.g., are you truly measuring blood pressure using this device)? How do you know for sure? Psychological Measurement Reliability: Consistency or dependability of behavioural data. A reliable result is one that will be repeated under similar conditions of testing at different times. A reliable measuring instrument yields comparable scores when employed repeatedly. There are two types of errors: random & systematic. An unreliable instrument shows random errors. The instrument is unreliable Random Errors The errors (inflation and deflation) will balance themselves and sum to zero.  The instrument is reliable but inaccurate Systematic Errors Systematic errors are in either the positive or negative direction consistently. They bias the overall measurement. Random error does not affect the average, Systematic error does affect the only the variability around the average, shifting it to the right in average(i.e., spreads the distribution out). this case. We call this a bias. Reliability versus Accuracy (or Precision) A reliable measure produces similar results when measurements are made under identical conditions. An accurate (or precise) measure produces results that agree with a known standard. A measuring instrument can be inaccurate but reliable. How do you find out whether a thermometer produces accurate readings? Psychological Measurement Construct validity In general, construct validity refers to the extent to which a measure of X truly measures X and not Y (e.g., a valid measure of intelligence measures intelligence and not something else). How much does a husband love his wife? What will be a valid measurement of love? - Measuring the amount of money that the husband gives his wife? - Measuring how frequently he has sex with her? - Measuring the amount of time that he spends with her in the mall? Psychological Measurement Internal validity An experiment with high internal validity means that the change in the dependent variable is caused by the independent variable(s) and not by the confounding variables. External validity An experiment with high external validity means that the researcher can generalize the experimental findings to broader circumstances, often from the laboratory to the real world. Step 6: Analyze the data The observations made in a study are usually converted into numbers. Even qualitative data (e.g., think-aloud protocols) can be coded in a certain way and analyzed quantitatively. Inferential statistics are used to analyzed the data and to decide whether the hypothesis is supported. Step 7: Report the findings Research findings are reported in academic meetings (oral or poster presentation) or published in a journal. The structure of an academic presentation/journal article includes: Literature review Method Participants Materials Procedure Results Discussion Case Study Also called N = 1 study A case study is the intensive observation, recording and description of an atypical person, organization, or event. Examples: Individuals with exceptional skills, rare diseases, brain damages, etc. Organizations or events that are highly successful or not successful. The case of Phineas Gage: Provides insight into the role of the prefrontal cortex in personality & self-control. Correlational Studies Correlational studies are used when the researcher wants to determine to what extent two variables, traits, or attributes are related. In a correlational study, the researcher does not control any of the variables. Participants are asked to report two aspects of their life. For examples: - amount of sleep and cumulative average - amount of sleep and amount of cigarette smoked per day - amount of sleep and monthly income  6 5 Cumulative average 4 Positive 3 Correlation 2 1 0 0 2 4 6 8 10 12 Average hours of sleep per night  14 Number of cigarettes smoked per day 12 10 8 Negative 6 Correlation 4 2 0 0 2 4 6 8 10 12 Hours of sleep per day  4000 3500 3000 Monthly income 2500 (Almost) 2000 No Relation 1500 1000 500 0 0 2 4 6 8 10 12 Average hours of sleep per night Correlation Coefficient (also called Pearson’s r) The strength of the correlation is indicated by a coefficient called Pearson’s r. The values of r range from -1 to +1 -1 means perfect negative correlation between X and Y. +1 means perfect positive correlation between X and Y. 0 means no correlation between X and Y.  Accurate Less accurate prediction prediction The best-fit line can be used to predict the value of one variable based on the value of another. The strength of the correlation coefficient (r) tells how accurate the prediction is.  Almost Perfect Correlation: An Example in Physics When a correlation is perfect, prediction is 100% accurate. Each data point represents a car being tested (the year, model, and size of the cars are not the same) The predicted stopping distance for 15 mph is 40 ft. The prediction is not very accurate.  Variables Correlation Coefficient Across many studies IQ & School Grades 0.40-0.50 IQ & Years of Education 0.60-0.80 IQ & Job Performance 0.30-0.50 IQ & Income Around 0.21 IQs of Identical Twins Around 0.86 IQs of Husband & Wife Around 0.40 Based on one study Depression & Hope -0.61 Depression & Social Support -0.45 Depression & Spirituality -0.53 Depression & Life Satisfaction -0.35 Depression & Resilience -0.48 Depression & Anxiety 0.77 Explaining the observed relationship Suppose that a researcher found a relationship between smoking and insomnia: As the number of cigarette consumption increased, the hours of sleep at night decreased (a negative relationship between the two variables). Propose some explanations for this observation. Three potential explanations: How do you find out which one is correct? When there are three potential explanations, how do you find out which one is correct? Answer: Do an experiment. In an experiment, you Manipulate the independent variable (also called the X variable). Keep the third variable (also called the extraneous variable or confounding variable) at a constant level. Measure the dependent variable (also called the Y variable). Variables Independent Variable (X Variable) The variable that the experimenter controls. E.g., # of cigarettes smoked by the subjects. Dependent Variable (Y Variable) The variable that the experimenter measures. E.g., Amount of sleep that the subjects had per day. Confounding Variable (Third Variable) Variable that affects a dependent variable and unintentionally varies between experimental conditions of a study. Level of stress experienced by the subjects.  Identify the Variables An experimenter tested the hypothesis that physical exercise improves mood (makes people happy). Fifty 20-year-old subjects participated in the experiment. Twenty-five subjects were randomly assigned to the experimental group and twenty-five to the control group. Subjects in the experimental group exercised on Saturday and Sunday and those in the control group watched a movie on Monday and Tuesday. The experimenter predicted that people who exercise twice a week will score higher on a self-report scale that measures their psychological wellbeing than people who watch movies twice a week. What is the independent variable? What is the dependent variable?  An experimenter tested the hypothesis that physical exercise improves mood (makes people happy). Fifty 20-year-old subjects participated in the experiment. Twenty-five subjects were randomly assigned to the experimental group and twenty-five to the control group. Subjects in the experimental group exercised on Saturday and Sunday and those in the control group watched a movie on Monday and Tuesday. The experimenter predicted that people who exercise twice a week will score higher on a self-report scale that measures their psychological wellbeing than people who watch movies twice a week. What is the independent variable? Exercise vs. no exercise/movie condition What is the dependent variable? Mood of the subject (the self-report scores of wellbeing)  An experimenter tested the hypothesis that physical exercise improves mood (makes people happy). Fifty 20-year-old subjects participated in the experiment. Twenty-five subjects were randomly assigned to the experimental group and twenty-five to the control group. Subjects in the experimental group exercised on Saturday and Sunday and those in the control group watched a movie on Monday and Tuesday. The experimenter predicted that people who exercise twice a week will score higher on a self-report scale that measures their psychological wellbeing than people who watch movies twice a week. What may be a confounding variable? A) Age of the subject B) Day of the week C) Both A & B  An experimenter tested the hypothesis that physical exercise improves mood (makes people happy). Fifty 20-year-old subjects participated in the experiment. Twenty-five subjects were randomly assigned to the experimental group and twenty-five to the control group. Subjects in the experimental group exercised on Saturday and Sunday and those in the control group watched a movie on Monday and Tuesday. The experimenter predicted that people who exercise twice a week will score higher on a self-report scale that measures their psychological wellbeing than people who watch movies twice a week. What may be a confounding variable? A) Age of the subject B) Day of the week C) Both A & B When there are three potential explanations, how do you find out which one is correct? Suppose that your hypothesis is: Smoking causes insomnia You would want to: Have control over the number of cigarettes your subjects smoke daily (the independent variable) Have control over the level of stress experienced by your subjects (the extraneous/third variable) But in reality, you cannot control the level of stress experienced by your subjects. What can you do then? Note: This is just a hypothetical study. An experimenter cannot ask people to start smoking in a real-life setting. Within-Subjects Design If the source of the confound is from random individual differences (e.g., some individuals are experiencing higher levels of stress than others), then the confound can be controlled by: Using a before-after within-subjects design (also called repeated- measures design) Single Group Repeated-Measures Design Group A Measurement Treatment Measurement Analysis Jane Benny Wendy Larry Susan, etc. Within-Subjects Design In this design, each individual is his/her own control (or baseline measure). We assume that an individual experiences the same level of stress throughout the course of the experiment. Single Group Repeated-Measure Design Group Measurement Treatment Measurement Analysis Group A Hours of sleep Start smoking Hours of sleep Compare the change Jane before smoking two months in amount of sleep Benny Wendy after smoking before and after Larry smoking. Susan, etc. A hypothetical experiment: Within-subjects design Before After smoking smoking One sample of healthy individuals. The same group of individuals. Before smoking, measure the daily After smoking, measure their daily amount of sleep they have over a amount of sleep again. Calculate the week. Calculate the group average. group average. Compare the group average before and after smoking. Hypothetical Results 9 Average hours of sleep per day 8 7 6 5 4 3 2 1 0 Before Smoking After Smoking Between-subjects design Alternatively, you can use a between-subjects design (or two-random- samples design) with: 1) large sample sizes 2) random assignment of participants to experimental and control (or placebo) groups Two-random-samples design Random Sample A Treatment Measurement Analysis: Comparison Jane, Benny, Wendy, between the groups Larry, Susan, etc. Random Sample B Treatment Measurement Jack, Keith, Jenny, Donald, Alice, etc. Between-subjects design Group Treatment Measurement Analysis Random sample A Given cigarettes with Hours of sleep Compare the (control group): no active ingredients per day difference in Jane, Benny, Wendy, average hours of Larry, Susan, etc. sleep per day Random sample B Given cigarettes with Hours of sleep between the two (experimental group) active ingredients per day groups Jack, Keith, Jenny, Donald, Alice, etc. Between-subjects design & random assignment of subjects A large random sample from the population Random assignment of subjects Control/Placebo group Experimental group The control group In an experimental design, the control group is treated exactly like the experimental group except that it is not exposed to the experimental treatment (use of fertilizer in the following example). The placebo group In a drug test, the control group is called the placebo group. Why is random assignment important? Confounding variable: Stress Placebo group Experimental group With large sample sizes and random assignment, we expect that the two groups would experience the same level of stress, on average, during the time of data collection. Controlling the confounding variable If the source of the confound is from some environmental factors that systematically change from one group to another, or from one time to another, then this confound cannot be eliminated with large sample sizes and random assignment of subjects. Example: The before-treatment measurement of sleep takes place at the beginning of January The after-treatment measurement of sleep takes place at the end of February. The subjects are experiencing a lower level of stress at the beginning of January than at the end of February. In this case, use a mixed design. Mixed Design Group Measurement Treatment Measurement Analysis Random sample A Hours of sleep Given Hours of sleep Compare the (placebo group): at the cigarettes at the end of change in Jane, Benny, beginning of with no February average hours Wendy, Larry, January active of sleep Susan, etc. ingredients between the Random sample B Hours of sleep Given Hours of sleep two groups. (experimental at the cigarettes at the end of group): beginning of with active February Jack, Keith, Jenny, January ingredients Donald, Alice, etc. Controlling Confounding Variable Confounding variable: Stress due to environmental factors Placebo group Experimental group 1) Report hours of sleep in the first 1) Report hours of sleep in the first week of January. week of January. 2) Smoke cigarettes with no active 2) Smoke cigarettes with active ingredients ingredients 3) Report hours of sleep at the end 3) Report hours of sleep at the end of of February. February. When the treatment has no effects Placebo and experimental groups display the same pattern of change. 12 Average Hours of Sleep/Day 10 8 6 Beginning of January 4 (Before) 2 End of February (After) 0 Placebo Experimental Group When the treatment has no effects Changes due to environmental factors 12 Average Hours of Sleep/Day 10 8 6 Beginning of January 4 (Before) 2 End of February (After) 0 Placebo Experimental Group When the treatment has an effect Placebo and experimental groups display different patterns of change. 12 Average Hours of Sleep/Day 10 8 6 Beginning of January (Before) 4 End of February (After) 2 0 Placebo Experimental Group When the treatment has an effect Changes due to environmental factors 12 Average Hours of Sleep/Day 10 Change due to smoking 8 6 Beginning of January (Before) 4 End of February (After) 2 0 Placebo Experimental Group Biased Samples Differences due to personality factor when the treatment has no effects. 12 Average hours of sleep 10 8 6 Placebo (has more type 4 B people) 2 Experimental (has more type A people) 0 Before After Group Biased Samples Differences due to personality factor 12 Average Hours of Sleep/Day 10 Difference due to treatment effect 8 6 Placebo (has more type B people) 4 Experimental (has more 2 type A people) 0 Before After Group Correlational vs. Experimental Studies Correlation does NOT imply causation. A strong correlation indicates only that two variables (A and B) are related in a systematic way. Variable A could be the cause of variable B Variable B could be the cause of variable A. A confounding variable C could be the cause of both variables A and B. Correlational Studies Three Goals of Scientific Studies Description Explanation Prediction Yes No Yes Correlational studies examine how variables are naturally related in the real world. They are used to describe and predict relationships between variables. They cannot be used to determine the causal relationship between the variables They cannot be used to explain the cause of a behavior. Experimental Studies Three Goals of Scientific Studies Description Explanation Prediction Yes Yes Yes To overcome causal ambiguity, researchers use experimental methods. In an experimental study, the researcher manipulates an independent variable to look for its effect on a dependent variable. The goal of this method is to make strong causal claims about the impact of one variable on the other. Brief Introduction to Inferential Statistics Between-Group difference is easy to observe when there is little/no error variance No error variance: The plants No error variance: The plants grow in the same rate. grow in the same rate. Poor Soil Fertilized Soil Between-Group difference is difficult to observe when there is error variance With error variance: The plants With error variance: The plants grow in different rates. grow in different rates. Poor Soil Fertilized Soil Between-Group difference is difficult to observe when there is large error variance Is the fertilizer effective? How do you find out? Poor Soil Fertilized Soil Sampling and inferencing: Draw a sample from the population, measure some attributes in the sample, generalize the results to describe the target population. If the sample is biased, the results cannot be generalized to the population. 92 Mean Standard Deviation Introduction to Inferential Statistics Population 1 Population 2 Population 3  Four Random Samples of Students from an Four Random Samples of Students from a Elementary School Middle School 20 20 15 15 Mean Age Mean Age 10 10 5 5 0 0 Sample A Sample B Sample C Sample D Sample A Sample B Sample C Sample D  Try to guess whether two samples are from the same or different populations 16 16 16 14 14 14 12 12 12 Mean Age Mean Age Mean Age 10 10 10 8 8 8 6 6 6 4 4 4 2 2 2 0 0 0 Sample X Sample Y Sample X Sample Y Sample X Sample Y  Try to guess whether two samples are from the same or different populations 16 16 16 14 14 14 12 12 12 Mean Age Mean Age Mean Age 10 10 10 8 8 8 6 6 6 4 4 4 2 2 2 0 0 0 Sample X Sample Y Sample X Sample Y Sample X Sample Y Probably the Same Probably the Same Probably Different College: The Range of Ages Is Wide Four Random Samples of Students from a College 50 45 40 35 Mean Age 30 25 20 15 10 5 0 Sample A Sample B Sample C Sample D Try to guess whether two samples are from the same or different populations (note: the mean age difference between samples X and Y is 10 years in both cases) 60 60 50 50 Mean Age 40 Mean Age 40 30 30 20 20 10 10 0 0 Sample X Sample Y Sample X Sample Y Try to guess whether two samples are from the same or different populations (note: the mean age difference between samples X and Y is 10 years in both cases) 60 60 50 50 Mean Age 40 Mean Age 40 30 30 20 20 10 10 0 0 Sample X Sample Y Sample X Sample Y Probably Different Probably the Same  M1 M2 Great Overlap Almost No Overlap  Practice Problem 1 State whether each pairwise comparison is not statistically significant (error bars completely overlap), probably not significant (error bars have a large degree of overlap), probably significant (error bars have a small degree of overlap), or significant (completely no overlap) Answers Cond_A: Significant; Cond_B: Probably not significant; Cond_C: Not significant Practice Problem 2 Answers Difference between A1 & A2: Probably not significant Difference between B1 & B2: Significant Difference between A1 & B1: Significant Difference between A2 & B2: Probably significant Use seeds from the same plant to minimize genetic differences between members in the same group. Use a strong fertilizer to maximize between-groups difference. Putting the two factors together 𝐵𝑒𝑡𝑤𝑒𝑒𝑛 𝐺𝑟𝑜𝑢𝑝𝑠 𝐷𝑖𝑓𝑓𝑒𝑟𝑒𝑛𝑐𝑒 𝑡= 𝑊𝑖𝑡ℎ𝑖𝑛 𝐺𝑟𝑜𝑢𝑝 𝑉𝑎𝑟𝑖𝑎𝑡𝑖𝑜𝑛 t statistic is a ratio A t-test tells how significant is the difference between two group means. The bigger the t ratio, the more significant is the test. We want to maximize between-groups difference and minimize within- group variation. What to read in the text The whole chapter 2!