Study Design and Evidence-Based Medicine

Study Design and Evidence-Based Medicine (EBM)

• Step two of EBM focuses on acquiring the best available evidence through literature searches.
• Step three involves appraising the evidence's validity, impact, and applicability to the PICOT question.

Importance of Evidence Quality

• Publication in a peer-reviewed journal does not guarantee sound or valid evidence.
• Almost all research has methodological weaknesses and flaws.
• This course provides a process for distinguishing between good and bad evidence.

Healthcare and Causal Inference

• Healthcare professionals deal with shades of gray rather than black and white answers.
• Perfect studies or definite causal conclusions (X causes Y) are rare.
• Conclusions are often conditional, and the phrase "it depends" is commonly used in EBM.

Classifying Study Designs

• Two major types of study designs exist: descriptive and analytic.
• Descriptive designs summarize information, while analytic designs make inferences.
• The best study design depends on the study's purpose.

Descriptive Study Designs

• Descriptive studies summarize information about diseases, events, or characteristics.
• They do not make causal inferences.
• They aim to answer "W" questions: who, what, why, when, and where.
• Descriptive studies help generate hypotheses for later examination.

Analytic Study Designs

• Analytic studies explore inferential hypotheses or research questions.
• Their intention is to explore cause and effect.

Types of Descriptive Study Designs

• Case reports and case series are types of descriptive studies frequently seen in drug-related medical literature.

Qualitative Studies

• Qualitative studies explore and provide deeper insights into real-world problems.
• They generate hypotheses and further investigate and understand quantitative data.
• Qualitative research gathers participants' experiences, perceptions, and behavior.
• They answer the "hows" and "whys" instead of "how many" or "how much."
• They often use open-ended questions.
• They explain processes and patterns of human behavior that can be difficult to quantify.

Cross-Sectional Studies

• Cross-sectional study designs can be descriptive or analytic, depending on the research question's purpose.

Ecological Studies

• Ecological studies analyze data at the population or group level.
• They can be descriptive or inferential.
• The SEER database conducts ecological studies on cancer rates over time.

Analytic Research Designs

• Analytic research designs focus on making causal inferences.
• Two families of analytic study designs are experimental and observational.

Experimental vs. Observational Studies

• In experimental studies, the investigator controls the exposure or assignment of an exposure.
• In observational studies, the researcher observes what is occurring without assigning treatment.

Efficacy vs. Effectiveness

• Randomized Controlled Trials (RCTs) answer questions of efficacy (and safety) under controlled circumstances.
• Observational studies answer questions of effectiveness in the real world.

Types of Experimental Studies

• Besides RCTs, quasi-experimental studies are also common, particularly in quality improvement.
• RCTs assess efficacy, quasi-experimental studies address performance, and observational studies assess effectiveness.

Upcoming Modules Preview

• Module three covers key statistical techniques for assessing evidence.
• Module four addresses experimental study designs.

Hierarchy of Evidence

• The hierarchy helps determine higher-quality versus lower-quality evidence.
• Animal and laboratory studies are below the pyramid of human evidence.
• Expert opinion is at the bottom of the human evidence pyramid.

Comparing Levels of Evidence

• "In general," cohort studies are better than case-control studies, and systematic reviews of RCTs are better than single RCTs.
• Observational studies are not necessarily useless, and RCTs are not automatically without flaws.

Levels of Evidence

• Different classifications or ranking schemes categorize levels of evidence.
• Examples include Roman numerals (I-V), a blue chart, and levels A, B, and C.
• RCTs and meta-analyses are always high quality, while expert opinion or consensus is always at the bottom.

Expert Opinion/Consensus (Bottom of Hierarchy)

• Definition: Recommendations from clinicians or stakeholders based on clinical experience.
• Example: 2022 CDC guidelines for prescribing opioids for chronic pain.
• Expert opinion is considered the lowest-quality evidence because it's based on consensus, even though the panel might have considered high-quality evidence.
• A JAMA study on ACC/AHA guidelines found that only 8.5% of recommendations were based on level A evidence (meta-analysis, RCTs), while 41.5% were based on level C evidence (expert opinion).

Case Series and Case Reports (Second Lowest Level)

• Case series and case reports are important, particularly for drug safety.
• Examples: Case series describing children dying after taking codeine after tonsillectomy led to FDA action.
• The FDA does not always wait for higher levels of evidence to take regulatory action for public safety.

Cohort Studies vs. Case-Control Studies

• In general, a cohort study is better than a case-control study design.

Observational Studies: Effectiveness and Safety

• Observational studies provide useful information about effectiveness and safety when RCTs are unavailable or impossible to conduct.
• Example: The Nurses' Health Study concluded that hormone replacement therapy (HRT) reduced the risk of coronary heart disease, which had a huge impact on clinical practice.

Contradiction by Randomized Controlled Trial

• The Women's Health Initiative RCT did not confirm the findings of the Nurses' Health Study.
• The RCT showed that HRT did not confer cardioprotection and even increased the risk of coronary heart disease.
• The RCT findings contradicted the observational study due to fewer biases in RCTs.

Importance of Higher-Level Evidence

• Higher-level evidence, like RCTs, helps make better causal inferences.

Systematic Reviews and Meta-Analysis (Top of Hierarchy)

• Systematic reviews summarize data from many other studies, often in a table.
• Meta-analysis uses data from many studies to estimate a pooled estimate of findings.

Summary and Key Take-Home Points

• The hierarchy is used to evaluate the best level of evidence.
• RCTs are generally higher than cohort studies, but not always of higher quality.
• It depends on how the researcher handled various study design issues to address potential biases.