Best Practices in EDI Survey Research

Best Practices in Equality, Diversity, and Inclusion (EDI) Survey Research

• EDI in survey research is crucial for ensuring accurate and inclusive representation
• Best practices for incorporating EDI principles in survey design should be outlined
• Common challenges and solutions related to hard-to-reach populations and questionnaire design must be highlighted

What is "EDI"?

• Equality: Treats all individuals and groups fairly, ensuring equal opportunities
• Diversity: Recognizes, respects, and celebrates differences in individuals and groups
• Inclusion: Fosters a welcoming and valued environment for all, recognizing and mitigating unconscious biases.

Total Survey Error Framework

• Total Survey Error is a framework for understanding the various sources of error in surveys
• Measurement error, sampling error, and nonresponse error are some error sources
• Validity and Construct represent aspects of survey accuracy

Hard-to-Reach Populations

• Hard-to-reach populations are difficult to engage with for surveys
• Population difficulty can be influenced by geographic, cultural, or social factors
• These factors might impact access

Important Issues (1)

• Some hard-to-reach populations might be impossible to fully enumerate
• Sampling frame may be incomplete for these populations
• These populations are often a small part of the overall population and might be marginalized or "invisible."

Important Issues (2)

• Social marginalization from stigmatized activities (like substance abuse, sex work) can affect participation
• Individuals from these marginalized groups may have reluctance to participate due to fear of stigma, legal repercussions, or mistrust of researchers.

Respondent-Driven Sampling

• Respondent-Driven Sampling is a non-probability sampling method
• It involves recruiting participants and having them refer other participants in their networks
• This method can be helpful for reaching hard-to-reach populations

Convenience Sampling

• Convenience sampling involves recruiting participants who are easily accessible to the researcher
• Examples include interviewing people in public places or reaching out through social media
• This method may not be generalizable to the whole population

Snowball Sampling

• Snowball sampling involves recruiting one eligible person, and having them refer peers. The process continues until the desired sample size is reached
• This method is helpful for finding participants who might not typically participate in surveys, like individuals with particular communities or identities
• It is best for qualitative or formative research

Time-Location Sampling

• This method involves creating a precise list of places visited by members of the target population
• Samples are drawn at random times and locations
• It's better to weigh survey results to correct for inclusivity

Time-Location Sampling (2)

• Strengths: Unbiased estimations are possible under certain conditions
• Generalizable to the target population
• Reproducible, particularly for multi-year or multi-city projects
• Weaknesses: Time-consuming and costly to establish the exact sampling list

Respondent-Driven Sampling (Implementation)

• Seeds (participants) are recruited non-randomly
• Seeds recruit the first wave of participants
• The subsequent waves are formed by recruiting participants known by members of the previous waves until the target sample size is reached.

Quota Samples

• Quota samples involve selecting participants within pre-defined quotas to represent characteristics of the overall population of interest

The Evidence for Equality National Survey (EVENS)

• EVENS is a large and comprehensive survey of ethnic and religious minorities in the UK.
• It has over 14,000 participants with 10,000 identifying as ethnic minorities.
• It uses innovative non-probability survey methods to improve representation.

Data Deficiencies: EDI

• Social surveys in the UK tend to represent a limited number of ethnic groups
• Survey sampling favours areas with higher concentrations of ethnic minorities
• General surveys might not have questions relevant to the concerns and experiences of minority groups
• Census/administrative data are comprehensive but often limited to specific topics

Diagrammatic Summary of Routes into EVENS Survey

• EVENS uses various pathways for survey participation, including open promotion, online registrations, referrals between participants, and different participant panels to maximize inclusivity

EVENS Questionnaire Design

• The EVENS survey is a 30-minute questionnaire available in 14 languages via online or telephone
• Eligibility is based on ethnicity and religion, residence in the UK
• It offers a £10 voucher as an incentive for completing

Responsive Recruitment

• Without a sampling frame, EVENS sets target quotas for ethnicity, age, sex and geographic region to enhance representativeness.
• A pre-survey questionnaire ensures eligibility based on GB residence and self-declared ethnic minority status.
• Recruitment efforts are responsive, adjusting to whether quota targets are being met.

Issues with Online Surveys

• Monitoring revealed spikes in survey completions, suggesting potential data manipulation
• Completions were flagged using language, timing and location indicators
• Suspicious cases and digital bots were confirmed through follow-up emails and the survey was temporarily paused

Additional Quality Assurance Measures

• Additional digital fingerprinting (e.g., reCAPTCHA) was implemented to reduce manipulation attempts
• Extra validation was applied for snowball sampling links before being supplied
• Delivery of vouchers was switched from digital to postal to mitigate fraud
• Data validation checks was performed daily

Establishing Trust

• EVENS is a collaborative partnership among various organizations

Ethnic Minorities in the UK

• EVENS data includes a range of ethnic groups in the UK

Data Adjustments: Weights

• Weights are created to address coverage and selection errors inherent in the methodology
• This allows the data collected to be representative of the overall population.
• Corrections were based on factors like voting eligibility, interest in politics, and community factors, referenced to external surveys to validate the findings.

This quiz explores the importance of Equality, Diversity, and Inclusion (EDI) in survey research. It highlights best practices for incorporating EDI principles and addresses common challenges related to hard-to-reach populations. Discover the Total Survey Error Framework and gain insights into ensuring accurate and inclusive representation in surveys.