Survey Research Overview

Survey Research

• When to use surveys: Surveys are appropriate when little is known about a large population or when generalizable data is needed. They are efficient for large samples.
• Comparison to other methods:
  • Survey vs. Content Analysis: Surveys collect original data from respondents while content analysis examines existing media or documents.
  • Survey vs. Experiment: Surveys describe or explore relationships (correlation) without manipulation. Experiments test causality by controlling variables.
  • Survey vs. In-depth interview: Surveys prioritize breadth of coverage while in-depth interviews provide rich, qualitative insights.
• Common survey approaches:
  • Face-to-face interviews: High response rate but costly and time-consuming.
  • Telephone surveys: Fast and cost-effective but declining landline use reduces effectiveness.
  • Mail surveys: Useful for geographically dispersed populations but have lower response rates.
  • Group administration: Efficient for specific groups (e.g., classrooms).
  • Online surveys: Cost-effective and accessible but prone to low response rates and sampling bias.
• Strengths of surveys:
  • Efficient for large samples.
  • Can be generalized if probability sampling is used.
  • Economically sound.
  • Non-intrusive and convenient.
• Weaknesses of surveys:
  • Limited depth.
  • Potential sampling errors (nonresponse, self-selection bias).
  • Difficulty in ensuring honest responses due to social desirability or recall issues.

Reliability and Validity

• Reliability: Consistency of measurement; ensured by pretesting, clear questions, and training.
• Validity: Accuracy of measurement; whether it measures what it's intended to. Improved by well-constructed questions and representative sampling.

Questionnaire Construction

• Common problems: Double-barreled questions, biased or leading terms, poorly defined terms, overly complex recall, or double negatives.
• Good practices: Using clear, concise questions, avoiding leading or biased language, and avoiding absolutes like "always" or "never." Ensure a balance in statements (e.g., agree/disagree). Use Gade's Top 10 list as a guide.

Levels of Measurement

• Nominal: Categories without a logical order (e.g., gender, marital status).
• Ordinal: Ordered categories with unequal intervals (e.g., Likert scales).
• Interval: Ordered with equal intervals; no true zero (e.g., temperature in Celsius).
• Ratio: Ordered with equal intervals and a true zero (e.g., age, income).

Survey Response Rate

• Improvement strategies:
  • Personalize invitations.
  • Offer incentives.
  • Send follow-up reminders.

Statistical Analysis

• Descriptive vs. Inferential Statistics:
  • Descriptive Statistics: Summarize characteristics of the dataset.
    • Types: Central tendency (mean, median, mode); Dispersion (range, standard deviation); Frequency and proportion.
  • Inferential Statistics: Used to make inferences about a population from a sample. Tests relationships or differences between variables.
• Null Hypothesis: Suggests no relationship or effect exists between variables.
• P-value: The probability that the observed results occurred by chance.
  • If p < 0.05, reject the null hypothesis (statistically significant).
  • If p > 0.05, fail to reject the null hypothesis (not statistically significant).

Hypothesis Testing

• Variables:

  • Independent Variable (IV): Manipulated or categorized factor.
  • Dependent Variable (DV): Measured outcome.

• Statistical Tests:

  • Parametric: Require normal distribution (e.g., t-test, ANOVA).
  • Non-parametric: Require no assumption of normal distribution (e.g., Chi-square).
  • Example uses: T-test for comparing means between two groups; Chi-square for analyzing differences among categorical variables. Correlation examines relationship strength and direction; regression predicts one variable; ANOVA compares means across multiple groups.

• Hypothesis Testing Errors:

  • Type I Error (False Positive): Rejecting a true null hypothesis.
  • Type II Error (False Negative): Failing to reject a false null hypothesis.

Content Analysis

• Definition: A research technique for objective, systematic and quantitative description of manifest content of communication.

• Characteristics:

  • Objectives: Clear, defined categories to ensure different coders agree;
  • Systematic: Procedures used consistently;
  • Quantitative: Focus on numerical representation.
  • Manifest Content: Analyzes explicit content only.
  • Objectives and Purpose: Describe content, infer gatekeeping or effects, test hypotheses.

• Sampling Issues:

  • probability (generalizable) and non-probability (specific contexts)

• Measures: Content analyzed can examine factors like manifest content (length, number of sources), trends over time, framing, visual representations.

• Reliability:

  • Intracoder: Consistency of one coder over time.
  • Intercoder: Agreement amongst multiple coders. Improved via training sessions.

• Testing Methods: Krippendorff's Alpha (0.7 or above is acceptable)

Experimental Research

• Definition: Determines causality by manipulating independent variables (IVs) and measuring their effects on dependent variables (DVs), while controlling for confounds.
• When appropriate: To establish cause-and-effect relationships.
• Key tasks: Manipulate the IV and control for confounds (other variables).
• Internal Validity: Ensuring only the IV caused the effect (e.g., by randomization).
• External Validity: Generalizability to other populations or settings.
• Threats to internal validity: Selection bias, history, testing, maturation.
• Threats to external validity: Artificiality, limited sample diversity.
• Basic experimental designs: Posttest-only with control group; pretest-posttest with control group; Solomon four-group design; factorial design.

This quiz covers the fundamentals of survey research, outlining when to use surveys and how they compare to other research methods. Learn about common survey approaches and their respective advantages and disadvantages in this informative quiz.