Survey Methods and Issues

Surveys

• Method for asking people about themselves and studying relationships between variables.
• Can be qualitative or quantitative.
• Close-ended questions have pre-determined answers while open-ended questions allow for free responses.

Surveys: Issues

• Response sets: Respondents might answer questions in a particular manner due to social desirability.
• Tendency to answer all questions in a similar manner: More prevalent with sensitive topics.
  • Over-reporting: Respondents might overestimate their participation in socially acceptable behaviors. (e.g., exercise, charity donations)
  • Under-reporting: Respondents might underestimate their participation in socially unacceptable behaviors. (e.g., drug use)

Constructing Good Questions

• Clearly define research objectives as questions should be linked to them.
• Common questions include those about attitudes, beliefs, facts, demographics, and behaviors.
• Careful wording is crucial for clear communication.

Avoiding Poor Question Wording

• Avoid using jargon, vague terms, poor sentence structure, overwhelming phrasing, and misleading information.
• Avoid negative wording as it can be confusing.
• Avoid unnecessary complexity and words that are not easily understood.
• Avoid double-barreled questions that combine two separate questions into one.
• Avoid loaded questions that contain emotionally charged words which can create bias.

Rating Scales

• Used with close-ended questions.
• Various types of scales:
  • Graphic rating scale.
  • Semantic differential scale.
  • Non-verbal scale.

Finalizing the Survey

• Survey must be neatly typed and error-free.
• Scales should be consistent throughout the survey.
• Start with interesting questions to keep respondents engaged.
• Keep the questionnaire concise.

Administering the Survey

• Methods:
  • In-person to groups or individuals.
  • Mail surveys.
  • Internet surveys.
  • Other technologies.

Administering the Survey: Pros and Cons

• Pros:
  • Less costly than interviews.
  • Ensures anonymity.
• Cons:
  • Difficulty understanding questions.
  • Potential for boredom and distraction.
  • Ensuring all participants meet the study criteria.

Population vs Sample

• Population: The entire set of events or individuals of interest.
• Sample: A subset drawn from a population
• Parameter: Numbers summarizing a population.
• Statistic: Numbers summarizing a sample.

Population vs Sample: Applications

• Statistics are used to make assumptions about populations.
• Surveys rely on sampling as it is impossible to survey everyone in a population.
• Researchers make inferences about populations based on samples.
• Confidence intervals are used to represent the range of plausible values for the population value.

Evaluating Samples

• External validity: Degree to which research findings can be generalized to the intended population.
• Threats to external validity:
  • Sampling frame: Members of the population accessible for participation.
  • Response rate: Percentage of individuals who complete the survey.

Sampling Techniques

• Two basic techniques:
  • Probability Sampling.
  • Non-probability Sampling.

Probability Sampling

• Requires a complete list of all members of the population.
• Types:
  • Simple random sampling: Each member has an equal chance of being selected.
  • Stratified random sampling: Population divided into subgroups (strata), then random samples are taken from each strata.
  • Cluster sampling: Clusters are identified and randomly sampled from them.

Non-Probability Sampling

• No complete list of population members.
• Less time and money, but findings may not be generalizable.
• Types:
  • Convenience sampling: Samples whoever is most convenient.
  • Purposive sampling: Samples individuals meeting predetermined criteria.
  • Quota sampling: Sample reflects the numerical composition of different subgroups within the population.

Basic Experiments

• Internal validity: Confidence in the cause and effect relationship.
• High internal validity: Well-designed experiment confidently establishes the IV as the cause of the DV.
• Low internal validity: Confounding variables present, making it unclear if the IV is solely responsible for the DV.
• External validity: Deals with generalizability - ability to apply results to different populations.

Establishing Internal Validity

• Using a well-designed research design.
• Controlling for extraneous variables.

Basic Experiments: Designs

• IV with two levels, creating two groups or conditions.
• Basic steps:
  • Selecting participants (probability or non-probability sampling).
  • Assigning participants to IV levels (randomly, eliminating selection differences).
  • Introducing the IV (with an operational definition).
  • Measuring the DV (with an operational definition).
• Three basic designs:
  • Independent groups.
  • Repeated measures.
  • Matched pairs.