Research Methods Quiz: Experiments Overview

Experiment Definition

• A type of study designed to determine if one variable causes a change in another.
• Changes in one variable (the independent variable) cause a change in another (the dependent variable).

Independent Variable

• The variable being manipulated.

Dependent Variable

• The variable being measured.

Conditions

• Different levels of the independent variable.

Extraneous Variables

• Variables other than the independent and dependent variables.
• Anything that changes in a study that's not the IV or DV.

Control

• Minimizing extraneous variables.

Independent Variable Manipulation

• A change that creates a difference in exposure level.
• Requires an active intervention.
• Not always possible to manipulate a variable.

Extraneous Variables as Confounding Variables

• Extraneous variables that differ on average across levels of the independent variable.

Control Conditions

• Participants are not given any treatment.
• Placebo control condition: Participants receive a simulated treatment that lacks an active ingredient.
• Placebo effect: Participants receiving or perceiving an effect similar to the treatment.

Between-Subjects Experiments

• Each participant is tested in only one condition.
• Participants are randomly assigned to conditions to ensure groups are similar.
• Simpler and quicker than other experiments.
• Avoids carryover effects.

Random Assignment

• Two criteria: Each participant has an equal chance of being assigned to any condition; each participant is assigned to a condition independently of other participants.

Block Randomization

• All conditions occur once in the sequence before any repeats
• Conditions occur in a random order within each block.
• New participant is assigned to the next condition in the sequence.
• Prevents unequal sample sizes.

Matched Groups

• Participants in various conditions are matched on the dependent variable and/or other extraneous variables to prevent confounding.
• Guarantees that variables aren't confounded across experimental conditions.
• Helps control for extraneous variables.

Within-Subjects Experiments

• Each participant is tested across all conditions.
• Maximizes control of extraneous participant variables.
• Removes doubts about causation.

Order Effects

• Participants' responses in conditions affected by their order of exposure.

Carryover Effects

• The effect of one condition on behavior in later conditions.

Practice Effects

• Participants perform better in later conditions due to practice.

Fatigue Effects

• Participants perform worse in later conditions due to tiredness or boredom.

Context Effects

• Being tested in one condition influences perception or task interpretation in another.

Counterbalancing

• Testing different participants in different orders.

Complete Counterbalancing

• An equal number of participants complete each possible order of conditions.

Random Counterbalancing

• Randomly determining the order of conditions for each participant.

Simultaneous Within-Subjects Designs

• Participants make multiple responses in each condition.
• Entire set of questions presented in a sequence that mixes the types.

Types of Validity

• Internal validity, external validity, construct validity, and statistical validity.

Internal Validity

• Extent to which a piece of evidence supports a cause/effect claim within a study.
• Experiments, due to their design, are typically high in internal validity.

External Validity

• Extent to which study results can be applied outside the study's context.
• Generalization across situations, people, stimuli, and times.

Construct Validity

• Degree to which a test measures what it claims to measure.

Statistical Validity

• Extent to which conclusions drawn from a statistical test are accurate and reliable.

Participant Recruitment

• Subject pool: Established group of participants.
• Participants in subject pools, and those not can be recruited via multiple methods.

Side Effects of Non-Standardized Procedures

• Introducing extraneous variables during a procedure.
• Risk is exacerbated by multiple experimenters.
• Experimenter expectancy effect: Participants conforming to what's expected of them, which may lead to bias.

Record-Keeping

• Experimental logs document unusual events and questions.
• Maintaining confidentiality of participants.

Manipulation Checks

• A separate measure to confirm the independent variable was successfully manipulated.
• Done at the procedure's end to confirm effect, avoid unnecessary attention.

Pilot Testing

• Small-scale study to confirm new procedure works as planned.

Research Types

• Retrospective research: Data collection before research questions were developed.
• Retrospective data examples include: case reports, case studies, case control and cohort studies
• Prospective research: Research questions and variables are determined before data collection.
• Prospective data examples: cohort studies and randomized controlled trials.
• Compilation research: Gathers data from diverse studies to answer clinical questions.
• Examples are systematic reviews and meta-analyses.
• Case Report/study
• Detailed description of a single case/patient
• Provides opportunity for reporting rare/unique clinical effects.
• Case series: Collection of info gathered on a topic
• Case-control: Compare patients with the condition to those without the condition.
• Prospective cohort: Group with a shared experience tracked over time.
• Retrospective cohort: Tracks groups exposed to an intervention or exposure over time.

Randomized Controlled Trials (RCT)

• Gold Standard of experimental research.
• At least two groups with the same condition (experimental and control).
• Random assignment.

Patient Randomized Treatment Order Trials.

• Provide strongest evidence.
• Patients receive all interventions (treatment order is randomized).
• Patients as their own controls.
• Minimizes responsive differences between participants.

Systematic Review

• Collects evidence from multiple studies.
• Highest standard for clinical decisions.

Meta-Analysis

• Combines statistical information from multiple studies.
• Similar to systematic reviews, but focuses on statistical combining of results.

Hierarchy of Evidence

• Levels compare the strength of an article's findings to others.

Non-Experimental Research

• Does not involve manipulating independent variables.
• Measures variables as they naturally occur.
• Can't provide proofs of causation.
• Used when manipulating IV is impossible or unethical.

Correlational Research

• Measures two variables.
• Attempts to assess relationship without control of extraneous variables.
• Non-experimental.

Data Collection in Correlational Research

• No manipulation of variables is possible, or not done
• No particular importance for how variables are measured

Measurement

• Variables can be quantitative or qualitative.
• Scatter plots, Pearson's correlation coefficient.

Correlation Coefficient

• Ranges from -1 to +1.
• 0 = no correlation.

Reasons Correlation Doesn't Imply Causation

• Directionality problem: Variable X could cause variable Y, or vice versa.
• Third-variable problem: A third variable could cause both X and Y.

Complex Correlation

• Measures multiple variables to assess their statistical relationships.
• Can be visualized using a correlation matrix.

Factor Analysis

• Studies relationships among multiple conceptually similar variables.
• Clusters variables into smaller sets based on strong correlations.

Regression

• Used to predict the value of one variable based on the value of another.
• The variable to be used to make a prediction is the predictor variable.
• The variable that is being predicted is the outcome variable or criterion variable.

Qualitative Research

• In-depth information from a small group.
• Conclusions drawn by the researcher.
• Global and exploratory in nature.

Quantitative Research

• Larger samples.
• Results based on statistical analysis.
• Specific and focused.

Data Collection

• Interviews, focus groups.

Survey Types

• Questionnaires, interviews, diaries.

Constructs

• Important for survey design; understanding differences among individuals or groups.

Survey Questions

• Demographic questions: Open-ended (no specific answer options); closed-ended (specific answer options).
• Types of scales: Categorical (nominal), Continuous (interval/ratio), Ranked (ordinal).

Summative and Cumulative Scales

• Summative scales: Each item contributes equally to the total score
• Cumulative scales: Each item represents a higher level of the attribute being measured.

Likert Scale

• Used for rating opinions and attitudes.

Scaling Methods

• Semantic differential: Scales opinions between two extremes.
• Visual Analog Scale: Measures on a line with anchored extremes.

Guttman Scaling

• Cumulative scaling, where items represent an increasing level of the aspect being measured.

Assumptions of Surveys

• Absolutes; leading questions; low inference statements; double negatives; bias wording; overlapping categories.
• Techniques to improve and reduce social desirability bias.
• Example: Catch trials and include positively and negatively keyed items.

Delphi Survey

• Consensus-building around a specific issue among experts
• Establishing clinical practice norms or guidelines.

Cognitive Model (Survey design)

• The stages people go through when interpreting questions and selecting responses, as described in a "model".

Survey Response Effects

• Context effects; item order effect; response option effect
• Open-ended items, closed-ended items.

Rating Scales

• Ordered number of response options.

BRUSO Model

• Brief, Relevant, Unambiguous, Specific, and Objective (for survey design/question wording).

One-Group Posttest-Only Designs

• Implementation of a treatment and subsequent measurement of a dependent variable.
• Weakest type of quasi-experimental design due to lack of control.

One-Group Pretest-Posttest Designs

• Measurement of a dependent variable before and after a treatment.
• Can be interpreted as a quasi-experimental design based on interpretation.
• Other factors such as history, maturation, testing, instrumentation, regression to the mean, or spontaneous remission may also create changes in the dependent variable.

Interrupted Time Series Design

• Multiple measurements of a variable over time, before and after an intervention.

Posttest-Only Nonequivalent Groups Design

• Comparing a treatment group that was exposed to an intervention, with a control group.

Multiple-Treatment Reversal Design

• Introducing multiple treatments in phases and tracking the effect of each.
• Reversing treatments to get a better understanding.

Multiple Baseline Designs

• Introducing treatments in phases and tracking the effect across different participants, behaviors or settings.

Hypothesis Testing

• Using samples to draw conclusions about a population.
• Sample statistics are estimations that are influenced by random variability.

Null Hypothesis Testing

• Formally deciding between two possibilities for a statistical relationship in a sample.
• H0 = No relationship in the population / random, sampling error is the sole reason and not a real relationship.
• Ha = A real relationship in the population is present and not sampling error.

Critical Values and Rejection Region

• Statistical values which if an observed value is beyond it, the null hypothesis is rejected in favor of the alternate hypothesis.

Directional/Non-Directional Hypotheses

• Directional hypotheses: There is a specific direction of expected difference
• Non-directional hypotheses: Only that there will be a difference is what is tested.

Sample Size and Relationships

• Higher samples generally produce results that are more likely true relationships.

t-Tests

• Comparing the means of two unrelated groups.
• Comparing the mean of a single sample with a known/hypothetical population mean.
• Comparing the means of two related scores from the same sample group.

ANOVA

• Comparing the means of three or more independent groups.

Post Hoc Tests

• Used after an ANOVA to determine which specific group means are significantly different.

Repeated Measures ANOVA

• Comparing differences among scores for the same participant over multiple measurements.

Statistical Power

• Probability of rejecting the null hypothesis if a relationship is present.

Test your knowledge on key concepts of experimental research methods, including the definitions of independent and dependent variables, control conditions, and the role of extraneous variables. Learn how manipulation and control are essential in determining causation in experiments.