PSYC 306 Chapter 5 Research Methods in Psychology PDF

PSYC 306 Research Methods in Psychology Chapter 5: Selecting Research Participants 1  Outline Sampling from populations Sampling biases Random sampling procedures Non-random sampling procedures...

PSYC 306 Research Methods in Psychology Chapter 5: Selecting Research Participants 1  Outline Sampling from populations Sampling biases Random sampling procedures Non-random sampling procedures 2  Sampling In order to decide who to recruit for your study (sample), you need to first determine who your population is Population: – A group sharing some common characteristic(s) – Researcher is interested in phenomenon of entire group Sample: – A subset of the population – Small set who participate in the study 3  Sampling Who do we want to generalize our results to? Usually interested in findings that apply to an entire population (for example, all university students) – but we don’t have access to an entire population Sampling refers to the process of selecting participants for a research project 4  Sampling Error Naturally occurring differences between the population and a sample Population (1000 students) Average age = 21.3 years Average IQ = 112.5 56% female, 44% male Sample 1 (5 students) Sample 2 (5 students) Average age – 19.8 Average age – 20.4 Average IQ = 104.6 Average IQ = 114.2 40% female, 60% male 80% female, 20% male 5  Sampling Researchers want to ensure that their samples are good representations of the populations they want to study The goal is to select samples that are similar to the populations from which they are taken If the sample adequately represents the population, the results of the study can generalize to the population 6 Depression Target population: Diabetes entire set of individuals sharing Sleep the characteristic of interest Disorders Accessible population: portion of TP that are accessible to be recruited This number is Sample: much smaller individuals selected than the target to be in your study population 7 Sampling Why not just include everyone in the population? – Not always possible – Population too large – Too costly – Time-consuming – Unwieldy (difficult) – Not necessary 8  Types of Population Records Census - residential population Often some population members are unknown Census- workday population Birth records Even when the population is known, can be hard to reach each member with Hospitalization records equal probability Education records (schools) Even when they can be reached, it can be expensive or time-consuming to Immigration records enroll all members Mobile phones Crime records www.statcan.gc.ca 9 Populations & Samples 10 Representativeness You want to be sure that the people in your sample are a true representation of those in the population of interest The larger and more representative the sample, the more confidence we have that the results can be generalized to the population 11  Biased Sample A biased sample occurs when participants in your sample differ from your population on a given characteristic. Can result from the way that participants were selected: selection or sampling bias Examples: Recruiting participants by cell phone - income must be large enough to support a cell phone Recruiting women with postpartum depression - those able to participate (not working) may have higher family income than total population of women with postpartum depression 12  Sampling Bias When the way we selected our participants favours the inclusion of certain people over others; no longer random e.g., university clinics: More educated? Higher SES? Older? e.g., social media: Younger? The method that we use to select our participants will affect the representativeness of our sample 13  Sampling Bias The larger the sample size, the more accurately it will represent the population - called Law of Large Numbers – But there are practical limitations Researchers must compromise between large samples and requirements needed for recruiting/testing a large sample 14 Law of Large Numbers The larger the sample size, the closer the outcomes approximate those of the population Based on mathematical probability: Discrepancy between sample and population decreases in relation to square root of N This means there is less benefit to test > 25-30 per group Accuracy increases a lot between N = 4 to 16 But less between N = 30 to 50 15 Sample Number Statistical power is a formula that identifies the minimum number of participants needed to detect an expected effect in a study – However: For survey/observational methods, you may require many (100s) of participants to get an accurate sample that is reflective of the population Topic of Psych 305 16  Sampling Procedures Probability Sampling: – Based on random sampling – Each member of a particular population has an equal chance of being selected – Good, but difficult – Selection process must be unbiased – random Nonprobability Sampling: – Not randomly selected – Each member of a particular population does not have an equal chance of being selected 17  Probability vs. Nonprobability Sampling Probability Sampling Nonprobability Sampling Exact size of population is Population size is unknown; known (can list all members). cannot list each member. Odds of selecting a certain Odds of selecting a certain individual are known; can be individual is unknown. calculated. Selection must be unbiased; a Selection is biased. Greater random process. risk of producing a biased sample. Behavioural sciences 18 Probability Sampling Estimates Estimating a population size is very difficult, so you need a best estimate of population – e.g., at any given time, people are moving away, on vacation, etc. A sampling frame is a list of cases in a population, or the best approximation of it – e.g., telephone directories, tax records, birth records This will still not be 100% accurate, but it is as close as you will get! 19 Probability Sampling Methods Simple random sampling Systematic random sampling Stratified random sampling Proportionate Stratified random sampling 20 Simple Random Sampling Entire population is represented Equality: each individual has equal chance Each selection is random, independent Define population, list all members, use random process to select individuals 21 Simple Random Sampling Assignment of numbers to participant, random process to select #s Draw #s or names out of a hat Use a random numbers table 22  Simple Random Sampling Two principal methods of random sampling – Sampling with replacement An individual selected for the sample is recorded as a sample member and then returned to the population (replaced) before the next selection – Sampling without replacement Removes each selected individual from the population before the next selection is made More likely; small change in probability Is fair, unbiased, but cannot guarantee it is representative! 23  Simple Random Sampling Two principal methods of random sampling – Sampling with replacement Example: Picking a sample of size 2 from a population of 100 probability of first person is 1 / 100 probability for second person is also 1 / 100 Considered independent odds because the outcome of previous choices does not affect odds of the current choice 24  Simple Random Sampling Two principal methods of random sampling – Sampling without replacement Example: Picking a sample of size 2 from a population of 6 probability of first choice is 1 / 6 probability for second choice is 1 / 5 Considered non-independent odds because the outcome of previous choices affects odds of the current choice 25  Simple Random Sampling Independent draws Repeating values ok? Sample with Yes Yes replacement Sample without No No, each value is replacement selected at most once If the population is not large, then sampling with replacement is best When sample size N is large, then outcomes will be similar because probability of 1 / (N-1) approaches probability of 1 / (N) 26  Simple Random Sampling Independent draws Repeating values ok? Sample with Yes Yes replacement Sample without No No, each value is replacement selected at most once If the population is not large, then sampling with replacement is best Consider N (sample size) = 100 Sampling with replacement: 1/100 Sampling without replacement: 1/99 Difference =.01 -the method matters more Consider N (sample size) = 10,000 Sampling with replacement: 1 / 10,000 Sampling without replacement: 1 / 9,999 Difference =.0001 27  Simple Random Sampling If the population is not large, then sampling with replacement is best Why is this important? Many times, researchers will use simulations to estimate likelihood of disease, psychological disorders, behaviors The simulation is only as good / accurate / predictive as the sample size from which the estimates are computed Bootstrapping datasets: sampling with replacement (resample from same dataset) Cross-validation studies: sampling without replacement (split train-test) Cameron et al (2021), Sci Reports 28  Simple Random Sampling A random sample means that that all elements of the population have equal chances to be included Selection process is fair, unbiased – Most times, a proper random sample yields results that are close to the population values – No guarantee that it is representative! Since chance determines each selection, it is possible (although unlikely) to obtain a very distorted sample – E.g.: Unrepresentative samples may result when selecting small samples Other sampling techniques impose additional restrictions on this procedure to get representative samples… 29  Systematic Sampling – Every nth participant is selected from a list containing the total population. – A random starting position is chosen. Example: you wish to sample 300 people from population = 900 people Use a sampling Interval = 3 Choose a random starting position ("200") and include every 3rd person until you reach a sample=300 people Cycle round to beginning (n=1) when you reach n=900 - Violates the principle of independence (once you have determined one value to choose, all other choices are determined) - Ensures a high degree of representativeness 30 Systematic Sampling Example: Random starting point = 4 Interval = 5 31 Systematic Sampling Type of Simple Random Sampling Systematic random sampling and systematic sampling are used interchangeably This technique is less random than simple random sampling because the principle of independence is violated However, as a probability sampling method, this method ensures a high degree of representativeness 32 Stratified Sampling Selection procedure dividing a population into subgroups, called strata, and a random sample is selected from each stratum – Guarantees that each subgroup will have adequate representation – Overall sample is usually not representative of the population Population Does not exercise Regularly Sometimes exercises Regularly Always exercises regularly 33  Stratified Sampling Example: – To ensure a sample that represented families from each of 4 income levels, choose the same number of individuals from each income bracket Choose N=25 families from each of the 4 brackets Overall sample of 100 families does NOT reflect percentage of population that falls in each bracket 34  Stratified Sampling Identify key subgroups (strata) Participants chosen by taking a simple random sample in each pre-identified subgroup Select equal numbers from within each subgroup Combine subgroups into overall sample Un Ex Tr Example: Testing memory for songs Consider 3 population strata: musically Untrained, Trained, and Expert listeners Take 3 random samples of equal size 35 Proportionate Stratified Sampling Researchers deliberately sample to ensure that the proportion of subgroups in the sample matches the proportions found in the population Determine correct proportions & randomly select from within that subgroup population = 60% men; 40% women sample = 60% men, 40% women A lot of work - need to confirm what the population percentages are! Guarantees that the composition of the sample will be representative of the 36 composition of the population Summary: Probability Sampling Methods Type of Description Strengths and Weaknesses Sampling Sample is obtained using a random The selection process is fair and process to select participants from the unbiased, but there is no Simple random total population. Each individual has an guarantee that the sample is equal and independent chance of representative. selection (sampling w/replacement). 37 Summary: Probability Sampling Methods Type of Description Strengths and Weaknesses Sampling Sample is obtained using a random The selection process is fair and process to select participants from the unbiased, but there is no Simple random total population. Each individual has an guarantee that the sample is equal and independent chance of representative. selection (sampling w/replacement). A sample is obtained by selecting every An easy method for obtaining an nth participant from a list containing essentially random sample, but Systematic the total population after a random the selections are not completely start. random or independent. 38 Summary: Probability Sampling Methods Type of Description Strengths and Weaknesses Sampling Sample is obtained using a random The selection process is fair and process to select participants from the unbiased, but there is no Simple random total population. Each individual has an guarantee that the sample is equal and independent chance of representative. selection (sampling w/replacement). A sample is obtained by selecting every An easy method for obtaining an nth participant from a list containing essentially random sample, but Systematic the total population after a random the selections are not completely start. random or independent. Guarantees that each subgroup A sample is obtained by dividing the will have adequate Stratified population into subgroups (strata) and representation, but the overall random then randomly selecting equal numbers sample is usually not from each of the subgroups. representative of the population. 39 Summary: Probability Sampling Methods Type of Description Strengths and Weaknesses Sampling Sample is obtained using a random The selection process is fair and process to select participants from the unbiased, but there is no Simple random total population. Each individual has an guarantee that the sample is equal and independent chance of representative. selection (sampling w/replacement). A sample is obtained by selecting every An easy method for obtaining an nth participant from a list containing essentially random sample, but Systematic the total population after a random the selections are not completely start. random or independent. Guarantees that each subgroup A sample is obtained by dividing the will have adequate Stratified population into subgroups (strata) and representation, but the overall random then randomly selecting equal numbers sample is usually not from each of the subgroups. representative of the population. Guarantees that the composition Same as stratified random, but number of the sample will represent the Proportionate of participants chosen so that composition of the population, stratefied proportions in the samples correspond but some strata may have limited to the proportions in the population 40 representation  Name the Sampling Method A) Simple random B) Systematic C) Stratified random D) Proportionate stratified Example: Researchers test which of 3 teaching methods best increases 5th graders’ vocabulary. They identify the number of classrooms teaching each method (Method1 = 40%; Method2 = 30%; Method 3 = 30%) and sample a proportionate number of students from classes offering each method. Example: Researchers identify age-related changes in memory. They sample every 7th person from a census list of residents after choosing a random starting point until they reach 100 participants. D; B 41  Name the Sampling Method A) Simple random B) Systematic C) Stratified random D) Proportionate stratified Example: Researchers identify age-related changes in memory. They divide a population into groups based on age (0-10; 11-20; 21-30; etc) and then choose a random sample from each of those groups. Example: There is a new mosquito-carried disease harming babies. Researchers have developed a new vaccine to test with babies, who are 15% of the local population. The researchers sample 100 babies from local communities to administer the vaccine. C; A 42  Nonprobability Sampling Used when the population is not completely known – Common in behavioural sciences Convenience sampling Quota sampling Snowball sampling Problem with these samples = no evidence that they are representative of the populations we are interested in 43  Convenience Sampling Subjects are selected on the basis of accessibility & convenience – Drawn from part of population close by Also called accidental sampling Grab whomever you can: – Participants are chosen based on availability and willingness – Readily available and convenient No attempt to know population details 44  Convenience Sampling Representativeness? (location of interviewer) Bias? (who interviewer reaches out to) Volunteer characteristics? (some walk on by) Easy / less costly / more timely How to limit the loss of validity: Use large sample with broad cross-section of individuals different income levels, education, age Provide detailed description of sample to let people draw inferences about generalizability 45  Quota Sampling Identify relevant categories of people Examples: younger/older; bilingual/monolingual Select sample size for each category based on predetermined # of participants Establish quotas (strict numbers) for the number in each subgroup Adjust quota to reflect proportions in population, OR Ensure subgroups are equally represented Example: quota = 50% younger adults, 50% older adults Example: study on sleep patterns of 1st year Medical students Population of 1st-yr Med students = 54% Female, 44% Male Set sample quotas to be 54% F, 44% M 2016 Population Census, Statistics Canada 46  Quota Sampling Often reflects proportions in population, but not randomly selected Population of 32: 10 male adults (31%) 10 female adults (31%) 6 male children (19%) 6 female children (19%) Sample of 12: 4 male adults (33%) 4 female adults (33%) 2 male children (16%) 2 female children (16%) Not randomly selected – Still selected based on convenience – Sample can be biased, but more representative 47  Quota Sampling Example: Caterer company hires researchers to find tasters who test new recipes for meals City population contains 1000 meat-eaters (66%) 400 vegetarians (27%) 100 vegan (7%) A quota sample: Researchers choose a sample of 100 eaters composed of 66 meat-eaters, 27 vegetarians, 7 vegans 48  Snowball Sampling Using a current participant to reach other potential participants, usually due to difficulty of recruitment with special populations Example: Researcher wants to measure self-esteem in teens and its relationship to belonging in a street gang. Researcher finds it easier to recruit street gang members (who may distrust researchers) by asking for referrals from current participants. Example: Researcher wants to measure performance anxiety in students whose career paths require public presentations (business school, music school). Business school majors are more plentiful than music majors. Researcher asks music-major participants to identify other music majors likely to participate in study. 49 Non-probability Sampling Methods: Summary Nonprobability Description Strengths & weaknesses Sampling Convenience A sample is obtained by An easy method for obtaining a selecting individual participants sample, but the sample is who are easy to get. probably biased. 50 Non-probability Sampling Methods: Summary Nonprobability Description Strengths & weaknesses Sampling Convenience A sample is obtained by An easy method for obtaining a selecting individual participants sample, but the sample is who are easy to get. probably biased. Quota A sample is obtained by Allows a researcher to control identifying subgroups to be the composition of a included, then establishing convenience sample, but the quotas for individuals to be sample probably is biased. selected through convenience from each subgroup. 51 Non-probability Sampling Methods: Summary Nonprobability Description Strengths & weaknesses Sampling Convenience A sample is obtained by An easy method for obtaining a selecting individual participants sample, but the sample is who are easy to get. probably biased. Quota A sample is obtained by Allows a researcher to control identifying subgroups to be the composition of a included, then establishing convenience sample, but the quotas for individuals to be sample probably is biased. selected through convenience from each subgroup. Snowball Participants are asked to refer Easy way to reach participants other people they know who (especially difficult or unique would be willing to participate samples). But sample members in the study. will know each other (less representative of target population; less anonymity) 52  Name the Sampling Method A) Convenience sampling B) Quota sampling C) Snowball sampling Example: Researchers want to know whether university students are reporting higher rates of math anxiety if they are pursuing a Bachelor of Arts or a Bachelor of Science major. They offer participation for extra credit in a psychology course that enrolls students from both programs, and students sign up to participate. Example: researchers want to know about personality differences among people who drive premium luxury cars and those who drive economical (less expensive) cars. The premium luxury car owners are harder to sample, so the researchers ask the participants in that category to refer their friends who also own luxury cars. A), C) 53  Name the Sampling Method A) Convenience sampling B) Quota sampling C) Snowball sampling Example: Researchers studied the prevalence of discrimination toward immigrant students in a university. In order to recruit more broadly from participants who may remain hidden to avoid discrimination, the researchers recruited family members of each participant. Example: A researcher wants to survey individuals with different employment status on how likely they are to purchase smartphone brands. The known employment rates in the city are 90% employed and 10% unemployed. The researcher recruits 90% of their sample from employed participants and 10% from unemployed participants. C), B) 54  Midterm Exam Structure Duration: designed to last 1 hour, 15 minutes (you will have 5 minutes extra) Format: majority = multiple choice (choose best answer) plus 3-4 short answers (answer in