Research Methods in Science

Chapter Outline

• Empiricism: How to Know Stuff
• Methods of Observation: Discovering What People Do
• Methods of Explanation: Discovering Why People Do What They Do
• Thinking Critically About Evidence
• The Ethics of Science: Doing What's Right

Empiricism: How to Know Stuff

• Scientific method: Set of principles about the appropriate relationship between ideas using empirical evidence
• Empiricism: Belief that accurate knowledge can be acquired through observation; essential element in scientific method
• Dogmatism: Description of the tendency to cling to one's beliefs

The Scientific Method

• Scientific method: Procedure for using empirical evidence to establish facts
• Theory: Explanation of a natural phenomenon; can never be proved right
• Hypothesis: Falsifiable prediction made by a theory

The Art of Looking

• Empirical method: Set of rules and techniques for observation
• People are difficult to study because of their complexity, variability, and reactivity
• Kinds of methods: Methods of observation and methods of explanation

A World of Difference

• The history of science is replete with men.
• Until recently, women's educational and employment opportunities were limited
• Men and women have different interests and talents
• Men are often more interested in scientific topics
• Women are more variable than men in quantitative ability and visuo-spatial ability

The World's First Scientist

• In 1834, William Whewell coined the word scientist to describe a remarkable astronomer, physicist, and chemist named Mary Somerville
• Few people remember that the world's first scientist was a woman

Methods of Observation: Discovering What People Do

• Observe: Use of one's senses to learn about the properties of an event or an object
• Limitations of everyday observation: Inconsistent and incomplete
• Scientific techniques for overcoming limitations: Measurement and description

What to Measure and How to Measure It

• Operational definition: Description of property in measurable terms
• Key feature of a good operational definition: Construct validity (extent to which the thing being measured adequately characterizes the property)
• Key features of a good detector: Reliability (tendency for a measure to produce the same measurement whenever it is used to measure the same thing) and Power (ability of a measure to detect specified conditions in the operational definition)

Measurement

• There are two steps in the measurement of a property: Define the Property and Detect the Property

Power and Reliability at the Olympics

• Usain Bolt ran the 100-meter race in 9.58 seconds, and Yohan Blake ran it in 9.75 seconds
• Judges in ice dancing rate qualities of performance related to athleticism and artistry, and are consistent in their ratings

Demand Characteristics: Doing What Is Expected (part 1)

• Demand characteristics: Aspects of an observational setting that cause people to behave as they think someone else wants or expects
• These characteristics make it hard to measure behavior as it typically unfolds

Demand Characteristics: Doing What Is Expected (part 2)

• Psychologists avoid demand characteristics by observing people without their knowledge
• Naturalistic observation: Technique for gathering scientific information by unobtrusively observing people in their natural environments
• Naturalistic observation is not always practical
• Some events can only be observed through direct interaction
• Observer bias may occur

How Do Researchers Measure "Pace of Life"?

• Researchers make naturalistic observations, measuring the average walking speed of pedestrians in different cities

Demand Characteristics: Doing What Is Expected (part 3)

• Techniques for avoiding demand characteristics: Privacy, control, and unawareness

Observer Bias: Seeing What Is Expected

• Observer bias: Tendency for observers' expectations to influence what they believe they observed and what they actually observed
• Why does this occur?: Expectations can influence observations and expectations can influence reality
• How can it be avoided?: Double-blind study

An Example of Observer Bias

• Robert Parker is one of the world's foremost wine critics
• His ratings indicate how good a wine tastes, but can they also influence how good a wine tastes?
• Would knowledge of his ratings affect participants' ratings?

Description

• Population: Complete collection of people; rarely measured
• Sample: Partial collection of people drawn from a population
• Psychologists use two techniques for analyzing data: graphic representations and descriptive statistics

Graphic Representations: Picturing the Measurements

• Graphic representations: Describe data in two ways
• Frequency distribution: Graphic representation showing the number of times in which the measurement of a property takes on each of its possible values
• Normal distribution: Mathematically defined frequency distribution in which most measurements are concentrated around the middle

Frequency Distributions

• What do these distributions reveal about reported levels of happiness?

Descriptive Statistics: Summarizing the Data

• Descriptive statistics: Brief summary statements about essential information from a frequency distribution
• Central tendency (centre or midpoint): Mode, Mean, and Median
• Skewed distribution

Calculating Descriptive Statistics

• Mode = 5
• Mean = 5.95
• Median = 6

Differently Shaped Distributions

• Positively skewed distribution
• Normal distribution
• Negatively skewed distribution

Variability: How Wide Is the Distribution?

• Variability: Extent measurements differ; tells how much the measurements differ from each other or roughly how wide the distribution is
• Range: Value of the largest measurement in a frequency distribution minus the value of the smallest measurement
• Standard deviation: Statistic that describes the average difference between the measurements in a frequency distribution and the mean of that distribution

Distributions Can Differ in Variability or Central Tendency

• Distributions can differ in variability and central tendency

Methods of Explanation: Discovering Why People Do What They Do

• Correlation: Relationship between variables in which variations in the value of one variable are synchronized with variations in the value of the other
• Variable property that can have more than one value

Hypothetical Data Showing the Relationship Between Sleep and Money

Synchronized Patterns of Variation

• Synchronized patterns of variation
• Measure pair of variables
• Repeat; make a series of measurements
• Examine measurements and try to discern pattern
• When two variables are correlated, knowledge of one variable value can be used to predict the value of another variable without having to measure it

Measuring the Direction and Strength of a Correlation

• Correlation direction: Either positive or negative
• Positive correlation = more-is-more relationship
• Negative correlation = more-is-less relationship
• Correlation strength: Limited range (r ranges)
• -1.0 (perfect negative correlation)
• +1.0 (perfect positive correlation)
• 0 (no correlation)
• Correlation coefficient: Measure of the direction and strength of a correlation (r)

Graphing Correlations

• Perfect positive correlation (r = 1)
• Perfect negative correlation (r = -1)
• No correlation (r = 0)

Positive Correlations of Different Strengths

Causation

• Sometimes, we see causal relationships that do not exist
• Natural correlations: Correlations observed in the world around us
• Third-variable problem (Z): Natural correlation between two variables cannot be taken as evidence of a causal relationship between them because a third variable might be causing them both

Causes of Correlation

• Three possible reasons why X and Y are correlated: X→Y, Y→X, or Z→X & Y

Experimentation: Establishing Causation

• Experimentation: Technique for establishing the causal relationship between variables
• Elimination of differences between groups by examining two key features

Three Steps of Experimentation (part 1)

• Experimentation allows establishment of causal relationships between variables by doing three things:
• Manipulate: Manipulate independent variable; create at least two conditions. Independent variable: Variable that is manipulated in an experiment
• Measure: Measure dependent variable. Dependent variable: Variable that is measured in an experiment
• Compare: Compare measurements of conditions with each other.

Three Steps of Experimentation (part 2)

• Sample
• Violent exposure condition
• Nonviolent exposure condition

Manipulation: Making Different Conditions

Random Assignment

• Random assignment: Procedure that lets chance assign participants to the experimental or control group
• Self-selection: Problem that occurs when anything about a participant determines whether they will be included in the experimental or control group

Statistical Testing: Making Sure Conditions Don't Differ by Chance

• Calculating the odds that random assignment has failed each time an experiment is conducted
• Generally, not accepting experimental results unless the calculation suggests there is less than a 5% chance that those results would have occurred if random assignment had failed
• Statistical significance; is determined when we calculate the odds that random assignment has failed through inferential statistics
• Statistically significant results: p < .05
• Failure of random assignment: p > .05

The Real World: The Surprisingly High Likelihood of Unlikely Coincidences

• A recent survey found that roughly half of college graduates believe in ESP
• By not using probability theory, people routinely underestimate the likelihood of coincidences
• Mathematics professor John Paulos noted, "In reality, the most astonishingly incredible coincidence imaginable would be the complete absence of all coincidence" (Neimark, 2004).

Drawing Conclusions

• Internal validity: Attribute of an experiment that allows it to establish causal relationships
• External validity: Attribute of an experiment in which variables have been operationally defined in a representative way

The Generalizability Restriction: “In the People We Studied . . .”

• Case method: Method of gathering scientific knowledge by studying a single individual
• Random sampling: Technique for choosing participants to ensure that every member of a population has an equal chance of being included in the sample
• Sample is representative of population.
• Generalization from sample to population can be made.
• Nonrandom sampling: Acceptable technique if the similarity between a sample and the population doesn't matter, when direct replication is available, and if the similarity between the two is a reasonable starting assumption

Is Nonrandom Sampling a Fatal Flaw in Psychological Science?

• No, and there are two reasons why
• First, sometimes the representativeness of a sample doesn't matter
• Second, sometimes the representativeness of the sample is a reasonable starting assumption
• Learning about some people does not necessarily tell us about all people, but it can still tell us a lot

The Reliability Restriction: “It Is Likely That...”

• Replication: Experiment that uses the same procedures as a previous experiment but with a new sample from the same population
• Type I error: Error occurs when researchers conclude that there is a causal relationship between two variables when there is not
• Type II error: Error occurs when researchers conclude that there is not a causal relationship between two variables when, in fact, there is

Other Voices Psychology Is Not in Crisis

• Is psychology amid a research crisis?
• Is failure to replicate a normal part of how science works? Why? Why not?
• Do failures to replicate mean that original experiments are worthless? Why? Why not?

Thinking Critically About Evidence

• Critical thinking: Involves asking tough questions
• Has evidence been interpreted in an unbiased way?
• Does the evidence tell not just the truth, but the whole truth?

We See What We Expect and Want to See

• Why do people have so much trouble thinking critically?
• A natural and intuitive way of thinking about evidence worked better for hunter-gatherers than for today's large-scale, complex societies
• We tend to hold different kinds of evidence to different standards
• Beliefs and desires shape which evidence to consider
• We see what we expect and want to see
• We don't consider what we don't see
• The skeptical stance: Scientists constantly strive to make their observations more accurate and reasoning more rigorous

Hours Spent Partying or Studying at Some Canadian Universities

The Ethics of Science: Doing What's Right

• The Tri-Council Policy Statement (TCPS) describes the core principles, based on respect for human dignity, that all research involving human participants must follow
• First, research should show respect for persons and their right to make decisions for and about themselves without undue influence or coercion
• Second, research should show concern for welfare which means that it should attempt to maximize benefits and reduce risks to the participant
• Third, research should be just, which means that it should distribute benefits and risks equally to participants without prejudice towards particular individuals or groups

Respecting People

• Some of the most important rules that govern the conduct of psychological research
• Informed consent
• Freedom from coercion
• Protection from harm
• Risk-benefit analysis
• Deception
• Debriefing
• Confidentiality

Respecting Animals

• Canadian Council on Animal Care (CCAC) standards for the ethical use and care of animals in research
• Replacement
• Reduction
• Refinement

Respecting Truth

• Psychology, like all sciences, works on the honor system
• Results are reported truthfully on what was done and what was found
• Credit is ethically assigned
• Data are shared