Lecture 11 & 12 PSYCH2018

Non-Experimental Design: Observational and Archival Methods

• Chapter Objectives:
  • Differentiate naturalistic and participant observation methods.
  • Articulate problems in observational research, and how researchers address them.
  • Explain thematic analysis as a qualitative data evaluation tool.
  • Define archival research and explain why it's non-experimental.
  • Describe advantages and limitations of archival research.
  • Outline data analysis methods for observational and archival research.
  • Describe how meta-analysis evaluates research replication.

Observational Research in Anthropology

• Emic Perspective: An insider's view, understanding a phenomenon from the perspective of those directly involved (similar to participant observation).
• Etic Perspective: An outsider's view, using existing theories to describe or explain observed phenomena (similar to naturalistic observation).

Naturalistic Observation

• Focus: Describing behaviors in natural settings.
• Observer Role: Unobtrusive; habituation (observer getting used to being observed) is assumed.
• Examples:
  • Snack selection at movie theaters.
  • Gender differences in bar-setting fights.
  • Helping behaviors in preschool settings.

Participant Observation

• Focus: Experimenter joins the group being observed.
• Challenges: Data recording issues, ethical issues, and reactivity (the group changing due to the researcher's presence).
• Example: Festinger's cult study (Box 10.1).

Challenges of Observational Methods

• Absence of Control: Strong claims can be difficult to falsify due to the lack of direct control.
• Observer Bias: Minimized through the use of behavioral checklists, interobserver reliability (agreement between observers), and careful attention to time and event sampling.
• Participant Reactivity: Reduced using unobtrusive measures.
• Ethics: Addresses issues of consent and privacy (Box 10.2).

Analyzing Qualitative Data (Thematic Analysis)

• Method: Identifying patterns in qualitative data.
• Steps:
  • Familiarize yourself with the data.
  • Code the data.
  • Search for themes.
  • Review themes.
  • Define and name themes.
  • Write a report.

Archival Research

• Data Source: Data previously collected for other purposes.
• Analysis Often Involves: Content analysis, factor analysis, meta-analysis.
• Characteristics: Susceptible to missing data and bias; however, there is no reactivity issue.
• Example 31 Details: Patient recovery room, focusing on the impact of view of a park on the patient.

Analyzing Archival Data

• Methods:
  • Factor Analysis: Identifies clusters of tests/items forming factors.
  • Meta-Analysis: Analyzing effect sizes across multiple studies on a similar topic to understand consistency and magnitude of effects.
  • Research Example 31 and analysis: Includes the layout of a room with different characteristics of experimental and control rooms. Analysis determines the independent and dependent variables.

Quasi-Experimental Designs and Applied Research

• Chapter 11 Key Terms:
  • Quasi-experimental design, nonequivalent control group design, regression to the mean, interrupted time series, program evaluation, needs analysis, formative, summative evaluation, cost-effectiveness analysis.

Beyond the Laboratory

• Applied Research Functions:
  • Solves real-world problems.
  • Develops knowledge.
  • Evaluates theory.
• Research Example 33 Color-Coded Nutrition Labels: Improved non-dieters' use of color-coded labels, which influences health evaluation of food.

Applied Psychology in Historical Context

• Historical Context Issues: Pressure to apply basics to practical issues and challenges of research validity.
• Examples and Issues: Study of the effects of caffeine on sleep, the importance of careful methodology, including randomization, controls, blinding, and placebos.

Design Problems in Applied Research

• Ethical Dilemmas: Potential coercion, consent issues, violation of privacy.
• Trade-offs: Balancing internal validity (showing cause and effect within a study), and external validity (generalizing findings to other situations outside the study)
• Between-Subjects Designs: Difficulties creating equivalent groups.
• Within-Subjects Designs: Uncontrolled order effects, subject attrition.

Quasi-Experimental Designs

• Focus: Applicable when random assignment is not possible/problematic.
• Key Characteristics: Lack of complete control, no random assignment.
• Examples: Single-factor nonequivalent groups designs, ex post facto factorial designs, P x E factorial designs, correlational research.

Nonequivalent Control Group Designs

• Focus: Comparisons between groups without random assignment.
• Key Components: Pretests and posttests. Random assignment to groups is not possible.
• Potential Challenges: Groups may be different at the outset influencing results. Regression to the mean (extreme scores on the first measurement shifting towards the mean on subsequent measures).
• Example 34: Study of the effect of "play streets" program on the level of physical activity in children.

Interrupted Time Series Designs

• Focus: Trend analysis in a single group over time, before and after an intervention. Useful for evaluating the impact of a treatment (an intervention) over time.
• Basic Components: Measures before and after an intervention.
• Challenge: Isolating the intervention's impact from other factors (history effects, selection effects etc.)
• Example 36: Examining productivity effects of an incentive plan, and accounting for other factors to isolate the impact of the incentive plan.

Program Evaluation

• Focus: Assessing the effectiveness of large-scale programs.
• Types: Formative (ongoing monitoring during program implementation), summative (evaluating the overall effectiveness).
• Example: Connecticut speeding study using regression to look at effectiveness.

Needs Analysis

• Focus: Identifying program needs.
• Methods: Data collection: Census data, Surveys of resources, potential users, Key informant surveys, Focus groups, Community forums, Workplace Example
• Example: DuPont Case study- Examined employee data, surveyed existing programs and and employee knowledge of healthy behavior.

Cost-Effectiveness Analysis

• Focus: Weighing costs and benefits of different programs.
• Relevance: Useful for comparing equally effective programs that have varied costs.
• Considerations: Program cost, effective outcome, and potential costs.
• Example: Large-scale fitness programs versus more impactful social support strategies

Summary

• Applied Research: Sheds light on causes and solutions.
• Quasi-Experimental Designs: Used when random assignments are not possible.
• Program Evaluation: Used to assess the effectiveness of large programs.