ANM 150 Test 1

Systematic Reviews

• In 1987, Cochrane demonstrated the importance of systematic reviews in reducing premature deaths due to complications from pre-term births.

Objectives

• Recognize the important features and strengths of a systematic review or meta-analysis
• Interpret an odds ratio and a forest plot
• Design a search strategy to find a systematic review on a topic

Looking at Systematic Reviews

• Systematic reviews are conducted to answer a clear question using relevant studies, comprehensively searched, and assessed for individual study quality.
• A strong systematic review requires a clear question, relevant studies, comprehensive search, clear eligibility criteria, and assessment of individual study quality.

Methodology

• A clear question is essential, considering PICO (Population, Intervention, Comparison, and Outcome)
• Relevant studies should be identified, with an appropriate design (e.g., RCT for interventions)
• A comprehensive search involves databases, reference lists, unpublished studies, contacting experts, and non-English studies
• Clear eligibility criteria are necessary for study inclusion
• Assessment of individual study quality is crucial

Results

• Results should be combined only if studies are similar enough, with minimal heterogeneity
• Heterogeneity tests, such as Cochrane Q and I2, are used to determine if data is too heterogeneous to combine
• Publication bias can occur when only positive studies are published, and negative studies are neglected

Funnel Plots

• Funnel plots are used to analyze publication bias, with a symmetrical distribution indicating minimal bias

Cochrane Collection

• The Cochrane Collaboration provides systematic reviews in a rigorous and consistent manner
• The Cochrane Library has a strong influence on policy and standards for practice
• It includes sections on complementary medicine, with over 70 reviews
• Full access is available through the LRC website

Cochrane Caution

• Cochrane reviews are notoriously conservative in their results, often concluding that 'more research is necessary'

Mini Stats Break

• Statistical significance determines if a difference is real or due to chance
• A p-value indicates the likelihood of a difference being due to chance
• A p-value of 0.05 is generally the cut-off for significance
• Variables that influence p-value calculation include sample size, variation within and between groups
• Confidence intervals (CI) provide a range of possible results, with 95% CI indicating that 95% of the time the results would be within that range

PICO and Searching for Evidence

• PICO is a framework used to ask structured, answerable questions in evidence-based medicine.
• PICO stands for:
  • P: Population
  • I: Intervention
  • C: Comparison
  • O: Outcome

Example of PICO

• Marion's question: "Should I wear elastic stockings when I travel to reduce my risk of developing deep vein thrombosis (DVT)?"
• PICO breakdown:
  • P: Retired woman who flies frequently
  • I: Wearing elastic stockings
  • C: Not wearing elastic stockings
  • O: Reducing the risk of developing deep vein thrombosis (DVT)

Searching for Evidence

• Primary sources of scientific evidence:
  • PubMed
  • Other subscription-based databases: EBSCO, CINAHL, MEDLINE
  • Google Scholar

PubMed Searching

• Boolean operators: AND, OR, NOT (must be in capital letters)
• Examples:
  • "Cancer AND vitamin C"
  • "Cancer OR carcinoma"
  • "Cancer AND vitamin C NOT intravenous"

Truncation

• Using the asterisk () to truncate search terms: "hyperten" = hypertension, hypertensive, etc.

Field Descriptions

• PubMed can search many fields, including date, author name, journal, title, page number, abstract, etc.
• Example: "(DP) = date of publication", "(TI) = title", "(JT) = journal title"

Advanced Search Builder

• Helps to be more specific in search queries
• Especially useful when looking for a particular article

MeSH Terms

• Medical Subject Headings (MeSH) are a controlled vocabulary
• Examples: "Depression" could refer to respiratory depression, cytokine depression, or depression as a psychiatric condition

Hedges (Filters)

• Combinations of terms or standardized search strategies that can be easily used

Clinical Queries

• PubMed Clinical Queries: subsets of search terms for specific topics, such as dietary supplements or complementary and alternative medicine (CAM)

Saved Search

• Creating a saved search using a string of terms
• Example: using the dietary supplement subset: "dietsuppl [sb] AND osteoarthritis"

Botanical Medicine

• No subset for botanical/herbal medicine
• Consider saving search terms: "Plants, Medicinal"[Mesh] OR "Ethnopharmacology" [Mesh] OR "Phytotherapy"[Mesh] OR "Ethnobotany"[Mesh] OR "Plant Preparations"[Mesh]

Diet

• Diet [Mesh] includes Mediterranean, DASH, keto, vegan, etc.
• Can also look at: Food [Mesh]

Limits

• Study type
• Subjects (human/animal)
• Age of publication
• Age of subjects, gender
• Free full text

Pre-Synthesized Evidence

• Cochrane Library: highly rigorous systematic reviews and meta-analyses
• Natural Medicines: access through CCNM library and CAND member website
• Trip Database: free and upgraded version
• Others: UpToDate, Examine.com (subscription-based)

Sourcing Full Text Articles

• Google Scholar
• Research Gate
• Email the study author (corresponding author) for a pre-publication copy

Limitations of RCTs

• Sample may not be representative
• Not good for rare or distant outcomes
• Application to non-pill interventions can be challenging
• Ethics of treating only some participants

Assessing the Quality of RCTs

• Jadad scale: assesses clinical trials, score from 0 to 5
• Cochrane risk of bias: often used in systematic reviews
• CASP checklist: assesses quality of reporting
• CONSORT: assess quality of reporting, CAM-specific reporting standards (e.g., Non-pharm CONSORT, Herbal Medicine CONSORT, REDHOT, STRICTA)

Jadad Scale

• Assesses clinical trials
• Score from 0 to 5
• 2 points for randomization
• 2 points for blinding
• 1 point for dropouts

Other Kinds of Intervention Studies

Cross-Over Studies

• Everyone gets intervention and comparison
• Pro: minimizes confounding (participants are their own controls)
• Con: must be a chronic illness, treatment must wash out

Cohort Studies

• Strengths: looks at any exposure, confident that exposure came before outcome, assess multiple outcomes
• Weaknesses: assignment to comparison group is not random, time-consuming, inefficient for rare outcomes
• Example: compare exposed and un-exposed groups over time

Case-Control Studies

• Strengths: can look at rare outcomes, faster
• Weaknesses: assignment to comparison group is not random, hard to assess temporality
• Example: compare diseased and non-diseased groups

Hierarchy of Evidence

• Identify inherent strengths and weaknesses of each type of research design
• Assess quality based on design and relevance to practice, individual patients, and public health

RCTs

• Strengths: many ways to decrease risk of bias, randomized, control group, blinding, best design for confirming cause/effect
• Weaknesses: difficult to recruit, only assess one outcome, recall bias

Randomization

• For comparison to be useful, need truly random assignment to treatment/comparison
• Examples: computer-generated sequence, sequential numbered sealed opaque envelopes, 3rd party allocation

Compliance

• Should be assessed
• Simple methods: diary, pill count, phone calls, questioning
• Biological methods: blood/urine levels (costly)

Withdrawals

• How many participants enrolled, completed, dropped out (why?), analyzed
• Accounting for missing data: per-protocol analysis, intention to treat analysis

Intention to Treat Analysis

• Analyze everyone in the group randomized, even if did not complete or ended up in the other group
• Techniques: last observation carried forward, statistical approaches
• Best way to minimize bias from dropouts

RCT Limitations

• Difficult to recall bias
• Only assessing one outcome
• Recall bias)

Cross-Sectional Studies

• Strengths: can study rare outcomes, faster, no recall bias
• Weaknesses: assignment to comparison group is not random, no assessment of temporality, only assessing one outcome

Observational Studies

• Strengths: can study any question, less expensive or faster than intervention studies
• Limitations: not randomly assigned to exposure groups, investigate correlation, not necessarily causation

Appraising the Quality of Observational Studies

• Recruitment: do participants reflect the population of interest? Selection bias
• Assessment of exposure: accurate? Subjective or objective? Validated? Measurement or classification bias
• Consideration of confounding factors? Did they look for confounding factors?

Systematic Reviews

• Strengths: explicit and rigorous methods, scientific investigation, enormous effort to minimize bias, capture the big picture of evidence on a topic
• Limitations: only as good as the available studies, can’t replace good clinical reasoning

N of 1 Study

• Strengths: look at real-world use of an intervention, allows for individualization, root-cause style treatment, complex health conditions, multi-modal treatments
• Limitations: doesn’t work if the condition is curable or self-limiting, must relapse in washout, findings may not be generalizable, ethics, high cost

Preclinical Studies

• Strengths: allow for creativity and innovation, background for future research in humans, investigate mechanism of action, study possible adverse events or interactions, high level of control, ethical
• Limitations: may not be clinically applicable to humans, highly controlled, one isolated part of the story

Consider

• Is pre-clinical evidence ‘sufficient’ enough to guide clinical recommendations?

Grading the Evidence

• Grading of Recommendations, Assessment, Development, and Evaluations (GRADE) is a transparent framework for developing and presenting summaries of evidence.
• GRADE provides a systematic approach to clinical decision making.

How Does It Work?

• A clinical question is formulated in PICO (Patient, Intervention, Comparison, Outcome) format.
• A systematic review provides an estimate of the effect size of an outcome.
• The author rates the quality of the evidence and strength of recommendations.

GRADE Certainty Rating

• Certainty refers to whether an estimate of association or effect is correct or true.
• There are four levels of certainty:
  • Very low: The true effect is probably markedly different from the estimated effect.
  • Low: The true effect might be markedly different from the estimated effect.
  • Moderate: The authors believe that the true effect is probably close to the estimated effect.
  • High: The authors have a lot of confidence that the true effect is similar to the estimated effect.

What Makes Evidence Less Certain? (“Rate Down”)

• Five factors can decrease certainty:
  • Risk of bias: Inherent limitation in the design of a study that causes the results to be inaccurate.
  • Imprecision: Results may be due to chance; focus on 95% confidence interval.
  • Inconsistency: Multiple studies showing a consistent effect increase certainty.
  • Indirectness: Intervention of interest is not studied in the population of interest and reporting the outcome of interest.
  • Publication bias: Is this really all of the research evidence that exists?

What Makes Evidence More Certain? (“Rate Up”)

• Three factors can increase certainty:
  • Large magnitude of effect.
  • Dose-response gradient.
  • All residual confounding would increase our confidence in an effect.

GRADE: 2 Parts

• Certainty of Evidence: How likely is it that something works? (Consider consistency, precision, applicability, and bias.)
• Recommendation Strength: Should it be recommended for use (or not!); consider evidence, benefits/harms, equity, resources, feasibility, and acceptability.

Recommendations

• Can be in favor or against an intervention.
• Can be strong or weak.
• Weak recommendations may lead to variation in decisions made by informed individuals.
• Shared decision-making process becomes essential with weaker recommendations.

Example: Folic Acid Supplementation and Blood Pressure

• A systematic review and meta-analysis of 41,633 participants found that folic acid supplementation significantly decreased systolic BP and diastolic BP.
• Subgroup analysis showed that the results remained significant in specific populations and under certain conditions.

Mental Health Research Challenges

• Unique challenges of studying mental health include obstacles, barriers, risks, and concerns

EASe-GAD Clinical Trial at CCNM

• Trial aims to study the effect of diet on Generalized Anxiety Disorder (GAD)
• Background: mental illness is common, and many people do not benefit from medication and psychotherapy due to access issues, side effects, and inadequate relief
• Observational data suggests a link between better diet quality and better mental health, with possible mechanisms including omega-3/6 balance, vitamins/minerals, protein for neurotransmitter synthesis, and healthy blood sugar regulation

Trial Design and Participants

• Pilot study with a randomized, wait-list controlled design
• Participants: adults (18-65 years old) who identify as women, diagnosed with moderate to severe GAD
• Primary objectives: assess feasibility and acceptability of the intervention
• Secondary objectives: look for signals of change in additional outcomes
• Intervention: personalized dietary advice, motivational interviewing (MI), mindfulness, and Mediterranean diet (MED diet)
• Control group: social support
• Inclusion criteria: stable medication/supplements for 4+ weeks, baseline Omega Score

Statistics

• Define terms:
  • Sample: a subset of the population
  • Population: the entire group of interest
  • Incidence: the number of new cases of a disorder
  • Prevalence: the total number of cases of a disorder

Hypothesis Testing

• The process of deciding whether the findings of an investigation reflect chance or real effect
• Statistical significance:
  • Difference in group means (averages)
  • Difference in the likelihood of an outcome
  • P-value: the probability that the difference is due to chance (typically < 0.05)

Statistical Significance: P-Value

• P-value: the odds that the difference is due to chance
• Cut-off: 0.05 (5% likelihood)
• Interpretation:
  • P > 0.05: not significant
  • P < 0.05: significant

Statistical Significance: Confidence Interval (CI)

• 95% CI: the range within which the true result is likely to lie
• Interpretation:
  • Overlapping intervals: non-significant difference
  • Non-overlapping intervals: significant difference

Relative Risk

• Probability of a bad outcome in the intervention group divided by the probability of a bad outcome in the control group
• Interpretation:
  • RR = 1: no difference
  • RR > 1: risk of bad outcome increased with treatment
  • RR < 1: risk of bad outcome decreased with treatment

Relative Risk Reduction

• How much the treatment reduced the risk of the bad outcome
• RRR = 1 - relative risk
• Example: 40% reduction in risk with vitamin C

Absolute vs Relative Risk Reduction

• Absolute risk reduction (ARR): the absolute amount the intervention decreased the risk
• ARR = risk of outcome with placebo - risk with medication
• Example: ARR = 8% (40% reduction in risk with vitamin C)

Odds Ratio

• Looks at the relationship between an exposure and an outcome
• Often used in observational studies
• Interpretation:
  • OR = 1: exposure does not affect the likelihood of the outcome
  • OR > 1: exposure associated with increased likelihood of the outcome
  • OR < 1: exposure associated with decreased likelihood of the outcome

Outcomes

• Clinical outcomes: a clinical event (e.g. stroke, suicide, hip fracture)
• Surrogate outcomes: an indicator that can be observed sooner, at a lower cost, or less invasively (e.g. biomarkers, indicators of disease progression)

Statistical Significance vs Clinical Significance

• Statistical significance: not due to chance (p < 0.05)
• Clinical significance: the intervention has a genuine, measurable effect that makes a real difference in the lives of patients
• Power: the ability of a study to detect a real difference between two groups