Nonexperimental Research Methods

Approaches to Nonexperimental Research

• Case studies involve observation and description of a single case, often an individual
• Archival research uses existing records to study behavior, such as census data, statistical records, survey archives, and written documents
• Content analysis of written materials in archival research typically requires a coding system
• Psychological testing involves assessing psychological traits like extroversion and intelligence
• Survey research administers a survey instrument to a sample of individuals drawn from a population
• Systematic observation involves observing one or more specific behaviors in a well-defined setting
• Systematic observation is primarily quantitative and requires a coding system

Types of Observation

• Observational research encompasses studies that observe and attempt to describe the behavior of people or animals
• Types of observational research include naturalistic observation, systematic observation, contrived observation, case study, and archival research

Naturalistic Observation

• Naturalistic observation occurs when a researcher makes observations in a particular natural setting (the field)
• This method typically extends over an extended period and uses various techniques to collect information
• Naturalistic observation is often used in psychology, sociology, ethnology, and anthropology
• It identifies and measures behavior as it occurs in everyday life and can be measured at the time or recorded for later coding
• Naturalistic observation produces a rich and complex data set
• It is primarily qualitative rather than quantitative and has ecological validity, as it occurs in situations similar or identical to everyday life experiences, minimizing reactivity
• The goal is to describe and understand how people in a social or cultural setting live, work, and experience the setting without intervention
• Methods include observing, interviewing, and examining documents like newspapers or memos
• Jane Goodall's chimp studies, where she dedicated her life to studying chimps' natural behavior, is an example
• Cognition in the Wild (Hutchins, 1995) studied cultural behavior mechanics

Participant Observation

• Participant observation involves watching people or animals in their natural habitats while actively participating in their situation
• Participant observers can be disguised (not revealing their researcher status) or undisguised (being aware of the researcher)
• An example includes a researcher analyzing hate crimes entering "White racist internet chat rooms" as a newbie (Glaser, Dixit, & Green, 2002)
• Advantages include providing important insights as a member of the group
• Disadvantages include the potential to lose objectivity by identifying too much with the group

Narrative Records

• Narrative records involve full descriptions of a participant's behavior using audio and/or video recording equipment
• This method attempts to record everything, including settings, events, and behaviors
• The researcher makes interpretations and supports them with examples of observed events
• Narrative records are primarily qualitative
• Piaget's notes on everything a child did and said during a specified period exemplify this
• Advantages include providing a good start for getting new theoretical ideas about behavior
• Disadvantages include not being useful for studying specific questions because the data is not organized

Qualitative Analysis

• Naturalistic observation, along with most qualitative analyses, poses difficulties in analyzing observations
• The researcher sifts observations to develop hypotheses or accounts for the observed data
• Negative case analyses are performed to examine observations inconsistent with the working hypothesis
• Difficulties in data interpretation include experimenter bias (selective observation), subjectivity (idiosyncrasies in observations), and positive bias (failure to seek occurrences not predicted by the hypothesis)

Systematic Observation

• Systematic observation involves observing one or a few specific behaviors in a particular setting
• It relies on quantitative procedures for recording data and requires a coding system to define and measure the behaviors recorded
• Advantages include the ability to test hypotheses
• Disadvantages include potentially missing interesting behaviors not part of the checklist
• Social behavior in young children is the example of how to observe and record for each child every 15 minutes

Contrived Observation

• Contrived observation occurs in a controlled setting set up by the researcher in a lab or the field
• Advantages include allowing the researcher to control the situation and see things they might not otherwise see
• Disadvantages include the situation being less natural
• Examples include:
  • Ainsworth and Bell's (1970) "strange situation" study, in which children were put into specific scenarios in a lab
  • Simons and Levin's (1998) "change blindness" study with directions
  • Valentino, Cicchetti, Toth, & Rogosch (2006) observing interactions in set contexts
  • The Marshmallow Study where kids behaviors if they ate the marshmallow or not were observed

Observational Studies: Advantages and Disadvantages

• Primary advantage: High external validity, the data collected inherently describes real-world behavior
• A primary disadvantage is reactivity, where the presence of the investigator may change behavior
• A second major disadvantage is experimenter (observer) bias, which affects which behaviors researchers choose to observe and record
• Some things that counteract this is multiple raters and reliable scoring rubrics

Case Studies

• Case studies are in-depth descriptions of a single individual, using personal history, observation, psychological testing, and experimentation
• These are useful for studying a person with a rare condition
• Examples include "Genie" and "H.M."

Archival Research

• Archival research uses previously existing records to answer research questions
• These records include public records (census data, police records), previously existing survey data, and published records
• Statistical analyses (e.g., correlational analysis) can be performed on the data, like Gwanltney-Gibbs (1986) analysis of marriage license applications
• Advantages include answering questions that can't be answered with any other method and avoiding reactivity
• Disadvantages include being limited by the accuracy of the original data collection and not knowing if procedures for collecting data were reliable and valid

Survey Research

• Survey research takes a snapshot of a state at a particular moment in time
• Repeated measures allow the researcher to illustrate changes across time
• Potential research set biases includes responding to all questions from a particular perspective rather than responding to each question directly
• Biases include social desirability, the halo effect, yea-saying and nay-saying responses, and extreme and middling responses

Constructing Questions

• Defining the research objective is key to constraining questions to those that address the fundamental components to be addressed by the research
• Focus on attitudes and beliefs (opinions, etc.), facts and demographics (verifiable facts), and behavior (actions rather than attitudes)
• Using pre-existing questionnaires is valuable/better because of known patterns of responses and its well-normed, has higher validity and reliability

Question Wording

• When designing survey questions, use familiar terms (avoid jargon)
• Also, you want to use precise terms (avoid vague terms)
• Ensure that the questions are grammatically clean and avoid overloading working memory
• Use consistent question stems, keep the questions simple (5th-grade reading level), and avoid double-barreled and "loaded" questions
• Negative wording can be confusing and can also mess with structural validity

Types of Rating Scales

• Graphic (visual) rating scales
• Non-verbal (images, not words)
• Likert (e.g., agree-disagree)
• Semantic differential (e.g., happy-sad)
• Guttman (presuppose previous items are true)
• "High frequency" scales

Formatting, Pilot Testing, and Administration

• Examples include Formatting, professional appearance, spacing, clear instructions, logical sequence of questions, and attention checks
• Refining the questions with pilot testing and debriefing is good
• There are also "think-aloud" protocols
• Other examples include individual vs group, interview vs questionnaire, and mail vs internet vs phone/text etc