Research Methods: Epistemology and Methodology

• These notes cover key concepts in research methods, focusing on epistemology, methodology, research design, validity, and more.

Epistemology and Methodology

• Epistemology is the study of knowledge itself.
• Methodology is the science of finding out or the procedures for scientific investigation.
• Empirical knowledge is derived from experience or observation.

Reality and Knowledge

• Experiential reality refers to knowledge gained through direct experiences.
• Agreement reality encompasses things accepted as knowledge due to cultural consensus.
• Tradition and authority can be problematic aspects of agreement reality, necessitating critical thinking.

Potential Errors in Personal Inquiry

• Inaccurate observation can be mitigated by specifying observations.
• Overgeneralization can be guarded against by using samples and replication.
• Selective observation can be avoided by specifying observations in advance.
• Illogical reasoning can be addressed through logical arguments.
• Ideology or politics can be avoided by maintaining neutrality and eliminating bias.

Purposes of Research

• Basic research includes exploration, description, and explanation.
• Applied research includes evaluation and policy analysis.

Theory and Hypothesis

• A theory is a systematic explanation of relationships among phenomena.
• A hypothesis is a testable statement outlining associations among variables.
• Hypotheses cannot be proven due to the existence of alternative explanations.

Reasoning

• Deductive reasoning starts with a theory and tests it against data (general to specific).
• Inductive reasoning starts with data and develops a theory (specific to general).

Data

• Qualitative data is descriptive information.
• Quantitative data is numerical information.

Questions

• Policy questions are broad issues of interest.
• Research questions are specific ways to study a policy issue.

Constructs, Variables, and Attributes

• Constructs are abstract ideas forming the basis of research and theory.
• Variables are concrete, observable representations of constructs.
• Attributes are characteristics describing something and composing a variable.

Variables

• Independent variables (IV) predict or define treatment conditions.
• Dependent variables (DV) change in response to the IV and are outcome variables.

Units of Analysis

• Units of analysis are the what or whom being studied.
• The ecological fallacy involves drawing individual-level conclusions from group-level data.
• The exception fallacy draws conclusions about groups based on a small number of people.

Types of Studies

• Cross-sectional studies examine a phenomenon at a single point in time.
• Longitudinal studies examine a phenomenon over an extended period, facing subject attrition.
  • Trend studies, cohort studies, and panel studies are types of longitudinal studies.
• Retrospective studies collect data about past outcomes and look back for causes.
• Prospective studies begin with potential causes and collect data about future outcomes.

Questions

• Descriptive questions describe a situation. Example: What percentage of violent offenders in Vancouver have a history of drug addiction?
• Causal questions explore how changes in one variable affect another. Example: Does chronic drug addiction increase the likelihood of committing violent crimes in Vancouver?

Relationships

• Correlational relationships involve synchronized movement between two variables.
• Causal relationships involve one variable causing a change in another.

Criteria for Causality

• Variables must be correlated.
• Temporal precedence: the IV must occur before the DV.
• No plausible alternative explanations.

Validity

• Validity refers to the truthfulness of statements about cause and effect.

Types of Validity

• Conclusion validity assesses the reasonableness of relationships in data and if IV is related to DV.
  • Lack of conclusion validity leads to bias, sample size affects power.
• Power is the likelihood a test will find a relationship between sample and population. A bigger sample size equals more power.
  • A priori power is determined before data collection.
  • Post hoc power is determined after data collection.
• Internal validity is the extent to which cause-and-effect inferences can be drawn from a study.
• External validity is the extent to which findings can be generalized to real-world settings.
• Construct validity is the extent to which variables measure their intended constructs.

Conceptions, Concepts, Dimensions, and Indicators

• Conceptions are mental images.
• Concepts are words and symbols representing these images.
• Dimensions are specifiable aspects of a concept.
• Indicators signify the presence or absence of a dimension.

Definitions

• Conceptual definitions assign working definitions to constructs for research agreement.
• Operationalization specifies procedures for measuring constructs.
• Operational definitions detail what will be observed, how, and what interpretations will be made.

Attributes

• Attributes need to be exhaustive to classify every observation.
• Attributes need to be mutually exclusive to avoid inflating results.

Levels of Measurement

• Nominal measures are qualitative, representing differences among groups (e.g., crime types).
• Ordinal measures rank-order attributes on a continuum with non-uniform gaps (e.g., severity of assault).
• Interval measures have meaningful distances between attributes (e.g., age).
• Ratio measures are interval measures with a true zero point (e.g., number of offenses).

Observed Scores

• An observed score consists of a true score + systematic error + random error.

Reliability

• Reliability indicates whether a measurement technique yields consistent results when applied repeatedly.
• Reliability is freedom from random error but does not ensure accuracy.
• Test-retest reliability measures consistency by using the same test on two occasions.
• Inter-rater reliability assesses how similarly two different scorers would score a test.
• Longer tests, large variation on measured factor, clear instructions and no distractions increase reliability.

Measurement Validity

• Measurement validity is the extent to which a measure reflects the construct.
• Face validity is agreement on mental images about a construct.
• Criterion-related validity compares a measure with an external criterion.
  • Convergent validity checks if a measure predicts scores on another accepted criterion.
  • Divergent validity checks if a measure does not predict scores on another accepted criterion that it shouldn't be related to.
• Content validity measures the range of meanings in a construct.

Types of Measurement

• Multiple measures compare your measure with other measures of the same construct.

Experimental Design Components

• Pretest measures the DV before the IV to establish a baseline.
• Posttest measures the DV after exposure to the IV to see the change.
• An experimental group is exposed to the IV.
• A control group is not exposed to the IV.

Bias

• Research subject bias arises from participant characteristics leading to skewed study outcomes.
• The Hawthorne effect is a change in behavior due to awareness of being studied.
• A double-blind experiment prevents both experimenters and participants from knowing treatment assignments.

Random Assignment

• Random assignment assigns participants to experimental and control conditions by chance to minimize differences between groups.

Experimental Design Essentials

• Independent and dependent variables are manipulated.
• Experimental and control groups are used for comparison.
• Random assignment of participants of the most importance.
• Pretests and posttests measure changes.
• Control of extraneous variables ensures accurate results.
• Replication verifies findings.

Randomized Controlled Trials

• Randomized controlled trials (RCTs) randomly assign individuals to experimental or control groups.

Causality Considerations

• Variables must be correlated.
• The IV must precede the DV.
• Changes in the DV should result from changes in the IV, not a third variable (random assignment helps).

Bias

• Omitted variable bias occurs when a variable affects the DV and is correlated with the IV.

Threats to Internal Validity

• History: External events affect the experimental group.
• Maturation: Natural changes over time affect participants.
• Testing: The pretest affects posttest results.
• Instrumentation: Changes in measurement instruments over time.
• Statistical regression: Extreme scores regress toward the mean upon retest.
• Attrition: Reduction in participant numbers from pretest to posttest.
• Causal time order: The IV must precede the DV temporally.
• Selection bias using nonequivalent groups, leading to differences in DV through creaming.
• Diffusion of treatment: Communication between groups contaminates results.
• Compensatory: control group deprived of something valuable. pressures to offer compensation exist
• Compensatory rivalry: The control group works harder to compensate for being deprived of the stimulus.
• Demoralization: The control group gives up due to deprivation.

Minimizing Threats to Validity

• Argue the threat is not reasonable.
• Demonstrate the threat is not a problem.
• Add a control, use random assignment, keep groups separate.
• Conduct tests to rule out alternative explanations.
• Implement increased controls or structure.

Internal and External Validity Conflict

• Greater control for internal validity may reduce the natural setting and external validity.

Threats to Statistical Conclusion Validity

• Low statistical power, such as samples that are too small.
• Weakly defined or implemented independent variable.
• Unreliable implementation of a treatment.

Experiment Notation

• R = random assignment
• O = observations
• X = the treatment (key IV)
• [ ] = no treatment (control group)
• X1 = a 2nd treatment
• T = time

Designs

• Randomized 2-group
• Before-After 2-Group Design
• Solomon 4-group design
• Latin square design
• Factorial design

Canadian Uniform Crime Reports (UCR)

• The Canadian Uniform Crime Reports (UCR) gathers crime information collected by police, the main source of crime statistics.
• Cross-sectional design enables easy annual comparability.
• Data is collected directly from police, detailing over 100 crime types.
• Data includes offense type, location, victim/accused characteristics, and case status.
• Crimes use standardized codes.
• The "most serious offense" rule records only the most severe crime in an incident.
• UCR1 reports basic incident numbers.
• UCR2 provides detailed incident reports.
• The UCR only covers reported crimes, potentially underrepresenting total criminal activity.

Sherman 1984 Study

• The Sherman 1984 study aimed to test police responses to misdemeanor domestic violence.
• The study was located Minneapolis, Minnesota.
• It employed a randomized controlled trial (RCT).
  • Arresting the suspect.
  • Separating the suspect and victim.
  • Mediation without arrest.
• Arrest was the most effective deterrent.

Limits of Sherman Study

• Sherman study's Minneapolis location limits external validity due to unique social factors.
• Uncontrolled factors like offender history limit internal validity.
• The study's focus on recidivism omits broader impacts like emotional well-being, undermining construct validity.
• Ethical issues from random assignments may put victims at risk.

Grounded Theory

• Grounded theory is an inductive approach where theories emerge from field observations.

Intersubjective Agreement

• Intersubjective agreement is when different researchers reach the same conclusion.

Paradigm

• Paradigm is a fundamental perspective shaping our worldview and research approach.

Explore research methods, epistemology, and methodology. Understand empirical knowledge, experiential reality, and agreement reality. Learn to avoid errors in personal inquiry through structured observations and logical reasoning.