Untitled Quiz

Survey Research I

• Survey research is an investigation that studies samples of large and small populations, taken from larger populations
• The theory behind surveys is that a sample can give as much information as studying the entire population
• Survey research is appropriate when information about a lot of people is needed instead of studying a few individuals deeply
• Surveys can be used in studies ranging from political science to social psychology and journalism, etc
• Public polls are a type of survey
• Surveys have built in checks for reliability and validity
• Qualitative and quantitative methods are both involved in a survey
• Qualitative and quantitative research are both used to describe the world as it exists
• Both use traditions of answering questions/solving problems
• Both acknowledge individuals have bias which needs to be accounted for
• Both create data based in both qual and quan thinking
• All qual data can be coded quantitatively
• All quantitative data is based on qualitative judgment
• Both use induction and deduction

Common Survey Approaches

• Face-to-face (personal interviews)
• Telephone (largely replaced by cell phones)
• Mail
• Group administration
• Online

Choosing Among Survey Types

• Important to choose the best method for the research question, budget and population
• Mixed-mode surveys are helpful when multiple approaches are needed

Web Surveys

• Web surveys are available in many sizes and purposes
• Low cost and ease of use are attractive
• Response rates are also often very low, especially without incentives
• Portions of the US population don't have access to the internet (25-30 percent)
• Recruitment through email lists, social media or consumer panels

Consumer Panels/Sampling Company

• These are panels for representative samples (non-probability quota sampling)
• This is not good for smaller areas
• Examples include Dynata, Qualtrics or Amazon Mechanical Turk
• Note reservations about using Mechanical Turk

Strengths of Survey Research

• Wide scope for information about large populations
• If a probability sample is used, it can be generalized to the population
• Economically sound (low cost per 1000)
• Realistic and non-obtrusive
• Can be done on your own time

Limitations of Survey Research

• No real depth -- limited to survey questions
• Extensive data, but not in-depth explanation
• Time-consuming, especially at the outset
• Developing questionnaires
• Training interviewers
• Recruitment, follow-ups

Problems/Prospects for Survey Research

• Total Survey Error (TSE) framework:
  • Frame: biased or incomplete sampling frame (sampling error)
  • Nonresponse: Low response raises sampling error
  • Measurement: respondent does not interpret survey as intended
  • Specification: Poor statement of what is being measured
  • Processing: Problems with data entry and tabulation

Improving Survey Reliability and Validity

• Measurement errors: Reducing social desirability bias & issues with questionnaire construction
• Sampling errors: Reporting in probability sampling, non-probability sampling, reducing self-selection bias and increasing response rates

Reporting for Probability and Nonprobability Samples

• Probability: Disclose researcher/funding, exact wording, full description of population, sample design, sample size, type of survey, data collection, response rate, reporting, confidence levels and confidence intervals
• Nonprobability: Disclose researcher/funding, exact wording, sample design, quota sample, sample size, type of survey, data collection, response rate, reporting, confidence levels and confidence intervals

Reducing Social Desirability Bias

• Social desirability bias is the tendency to portray oneself as a socially acceptable
• Survey instructions should include a more permissive frame of reference
• Method changes (ie experiment, Q-sort, content analysis) can help

Issues in Questionnaire Construction

• Incomplete wording
• Poorly defined terms
• Multiple questions/double barreled questions
• Prestige biased questions
• Biased or loaded terms
• Leading questions
• Over demanding recall questions
• Questions that cannot be answered
• Double-negative terms

Getting the Truth in Surveys

• Top 10 Guidelines for surveys include making statements clear, short and focused on the research objective, avoiding double-barreled questions and bias words, avoiding leading questions, not asking too detailed questions and avoiding embarrassing questions

Take-home Assignment

• Read and think about the differences and similarities of provided surveys
• Explain differences/similarities in question constructions, length, order of questions
• Use of open or closed ended questions
• Describe the population the surveys try to measure
• Indicate the intent of researchers based on survey questions
• Critically analyse survey introductions and instructions, self-administration possible issues with survey questions

Next Week

• Read Salkind, Chap 6 and Nardi (Chaps. 3-4), and the Millar and Dillman Article on improving Web and Mixed Mode Surveys
• Assignment due will include proposal intro/outline