Podcast
Questions and Answers
What is the main purpose of conducting a systematic review?
What is the main purpose of conducting a systematic review?
What is a key component of a comprehensive search strategy in a systematic review?
What is a key component of a comprehensive search strategy in a systematic review?
What is the purpose of a heterogeneity test in a systematic review?
What is the purpose of a heterogeneity test in a systematic review?
What is publication bias, and how is it addressed in a systematic review?
What is publication bias, and how is it addressed in a systematic review?
Signup and view all the answers
What is the Cochrane Library, and what is its significance in systematic reviews?
What is the Cochrane Library, and what is its significance in systematic reviews?
Signup and view all the answers
What is the significance of the example given in 1987 by Cochrane?
What is the significance of the example given in 1987 by Cochrane?
Signup and view all the answers
What is an essential feature of a strong systematic review?
What is an essential feature of a strong systematic review?
Signup and view all the answers
What is a characteristic of Cochrane reviews?
What is a characteristic of Cochrane reviews?
Signup and view all the answers
What is statistical significance referring to?
What is statistical significance referring to?
Signup and view all the answers
What does a p-value of 0.03 indicate?
What does a p-value of 0.03 indicate?
Signup and view all the answers
What is the main purpose of a 95% Confidence Interval?
What is the main purpose of a 95% Confidence Interval?
Signup and view all the answers
What is an advantage of a larger sample size in statistical analysis?
What is an advantage of a larger sample size in statistical analysis?
Signup and view all the answers
What is the general rule for determining statistical significance using a p-value?
What is the general rule for determining statistical significance using a p-value?
Signup and view all the answers
What does it mean if the 95% Confidence Intervals of two groups overlap?
What does it mean if the 95% Confidence Intervals of two groups overlap?
Signup and view all the answers
What is the main difference between statistical significance and practical significance?
What is the main difference between statistical significance and practical significance?
Signup and view all the answers
What is a limitation of using Google Scholar for searching?
What is a limitation of using Google Scholar for searching?
Signup and view all the answers
What is an advantage of using pre-synthesized evidence?
What is an advantage of using pre-synthesized evidence?
Signup and view all the answers
What is a common way to source full-text articles?
What is a common way to source full-text articles?
Signup and view all the answers
What is the Trip Database?
What is the Trip Database?
Signup and view all the answers
What is a consideration when searching for naturopathic topics?
What is a consideration when searching for naturopathic topics?
Signup and view all the answers
What is an advantage of using Google Scholar?
What is an advantage of using Google Scholar?
Signup and view all the answers
What is the significance of considering the age of publication?
What is the significance of considering the age of publication?
Signup and view all the answers
What is the Cochrane Library?
What is the Cochrane Library?
Signup and view all the answers
Why is it important to consider the age of subjects?
Why is it important to consider the age of subjects?
Signup and view all the answers
What is a limitation of using free full-text articles?
What is a limitation of using free full-text articles?
Signup and view all the answers
What is a major advantage of meta-analysis?
What is a major advantage of meta-analysis?
Signup and view all the answers
What is a limitation of N of 1 studies?
What is a limitation of N of 1 studies?
Signup and view all the answers
What is a strength of pre-clinical studies?
What is a strength of pre-clinical studies?
Signup and view all the answers
What is a consideration when evaluating the quality of a systematic review?
What is a consideration when evaluating the quality of a systematic review?
Signup and view all the answers
What is a limitation of pre-clinical studies?
What is a limitation of pre-clinical studies?
Signup and view all the answers
What is a strength of N of 1 studies?
What is a strength of N of 1 studies?
Signup and view all the answers
What is a consideration when conducting a systematic review?
What is a consideration when conducting a systematic review?
Signup and view all the answers
What is a limitation of systematic reviews?
What is a limitation of systematic reviews?
Signup and view all the answers
What is a consideration when evaluating the results of a systematic review?
What is a consideration when evaluating the results of a systematic review?
Signup and view all the answers
What is the main difference between a systematic review and a meta-analysis?
What is the main difference between a systematic review and a meta-analysis?
Signup and view all the answers
What is the primary goal of the GRADE framework?
What is the primary goal of the GRADE framework?
Signup and view all the answers
What does a 'very low' certainty rating in GRADE indicate?
What does a 'very low' certainty rating in GRADE indicate?
Signup and view all the answers
What is an example of 'risk of bias' in a study?
What is an example of 'risk of bias' in a study?
Signup and view all the answers
What is the focus of imprecision in GRADE?
What is the focus of imprecision in GRADE?
Signup and view all the answers
What is the consequence of 'rating down' due to imprecision in GRADE?
What is the consequence of 'rating down' due to imprecision in GRADE?
Signup and view all the answers
What does the GRADE framework assess when evaluating the certainty of evidence?
What does the GRADE framework assess when evaluating the certainty of evidence?
Signup and view all the answers
What is an example of indirectness in a study?
What is an example of indirectness in a study?
Signup and view all the answers
What is the purpose of a funnel plot in assessing the certainty of evidence?
What is the purpose of a funnel plot in assessing the certainty of evidence?
Signup and view all the answers
What is a characteristic of a strong recommendation in the GRADE framework?
What is a characteristic of a strong recommendation in the GRADE framework?
Signup and view all the answers
What is the purpose of considering the magnitude of effect in the GRADE framework?
What is the purpose of considering the magnitude of effect in the GRADE framework?
Signup and view all the answers
What is the primary outcome of folic acid supplementation on blood pressure according to the systematic review?
What is the primary outcome of folic acid supplementation on blood pressure according to the systematic review?
Signup and view all the answers
What is the subgroup of patients where the results of folic acid supplementation on systolic blood pressure remained significant?
What is the subgroup of patients where the results of folic acid supplementation on systolic blood pressure remained significant?
Signup and view all the answers
What is the effect of folic acid supplementation on diastolic blood pressure according to the systematic review?
What is the effect of folic acid supplementation on diastolic blood pressure according to the systematic review?
Signup and view all the answers
What is the duration of intervention where the results of folic acid supplementation on systolic blood pressure remained significant?
What is the duration of intervention where the results of folic acid supplementation on systolic blood pressure remained significant?
Signup and view all the answers
What is the dose of folic acid supplementation where the results of folic acid supplementation on systolic blood pressure remained significant?
What is the dose of folic acid supplementation where the results of folic acid supplementation on systolic blood pressure remained significant?
Signup and view all the answers
What is a common challenge in studying mental health?
What is a common challenge in studying mental health?
Signup and view all the answers
What is the primary objective of the EASe-GAD clinical trial?
What is the primary objective of the EASe-GAD clinical trial?
Signup and view all the answers
What is the eligibility criterion for participants in the EASe-GAD clinical trial?
What is the eligibility criterion for participants in the EASe-GAD clinical trial?
Signup and view all the answers
What is the control group in the EASe-GAD clinical trial?
What is the control group in the EASe-GAD clinical trial?
Signup and view all the answers
What is the possible mechanism by which diet may affect mental health?
What is the possible mechanism by which diet may affect mental health?
Signup and view all the answers
What is the primary outcome of the EASe-GAD clinical trial?
What is the primary outcome of the EASe-GAD clinical trial?
Signup and view all the answers
What is a characteristic of the EASe-GAD clinical trial?
What is a characteristic of the EASe-GAD clinical trial?
Signup and view all the answers
What is the population targeted in the EASe-GAD clinical trial?
What is the population targeted in the EASe-GAD clinical trial?
Signup and view all the answers
What is the primary purpose of hypothesis testing in statistical analysis?
What is the primary purpose of hypothesis testing in statistical analysis?
Signup and view all the answers
What does a p-value of 0.01 indicate?
What does a p-value of 0.01 indicate?
Signup and view all the answers
What is the main difference between incidence and prevalence?
What is the main difference between incidence and prevalence?
Signup and view all the answers
What is the purpose of a 95% Confidence Interval (CI)?
What is the purpose of a 95% Confidence Interval (CI)?
Signup and view all the answers
What is the difference between a sample and a population?
What is the difference between a sample and a population?
Signup and view all the answers
What is the primary purpose of considering the size of the sample in statistical significance?
What is the primary purpose of considering the size of the sample in statistical significance?
Signup and view all the answers
What does a relative risk of 0.6 indicate?
What does a relative risk of 0.6 indicate?
Signup and view all the answers
What is the difference between absolute risk reduction and relative risk reduction?
What is the difference between absolute risk reduction and relative risk reduction?
Signup and view all the answers
What is the limitation of using relative risk reduction when outcomes occur infrequently?
What is the limitation of using relative risk reduction when outcomes occur infrequently?
Signup and view all the answers
What is the odds ratio, according to the SR lecture?
What is the odds ratio, according to the SR lecture?
Signup and view all the answers
What does an odds ratio of 1 indicate in a study?
What does an odds ratio of 1 indicate in a study?
Signup and view all the answers
What is the main difference between a clinical outcome and a surrogate outcome?
What is the main difference between a clinical outcome and a surrogate outcome?
Signup and view all the answers
What is the primary consideration in determining clinical significance?
What is the primary consideration in determining clinical significance?
Signup and view all the answers
What is the purpose of validating a surrogate outcome?
What is the purpose of validating a surrogate outcome?
Signup and view all the answers
What is the relationship between sample size and statistical significance?
What is the relationship between sample size and statistical significance?
Signup and view all the answers
Study Notes
Systematic Reviews
- In 1987, Cochrane demonstrated the importance of systematic reviews in reducing premature deaths due to complications from pre-term births.
Objectives
- Recognize the important features and strengths of a systematic review or meta-analysis
- Interpret an odds ratio and a forest plot
- Design a search strategy to find a systematic review on a topic
Looking at Systematic Reviews
- Systematic reviews are conducted to answer a clear question using relevant studies, comprehensively searched, and assessed for individual study quality.
- A strong systematic review requires a clear question, relevant studies, comprehensive search, clear eligibility criteria, and assessment of individual study quality.
Methodology
- A clear question is essential, considering PICO (Population, Intervention, Comparison, and Outcome)
- Relevant studies should be identified, with an appropriate design (e.g., RCT for interventions)
- A comprehensive search involves databases, reference lists, unpublished studies, contacting experts, and non-English studies
- Clear eligibility criteria are necessary for study inclusion
- Assessment of individual study quality is crucial
Results
- Results should be combined only if studies are similar enough, with minimal heterogeneity
- Heterogeneity tests, such as Cochrane Q and I2, are used to determine if data is too heterogeneous to combine
- Publication bias can occur when only positive studies are published, and negative studies are neglected
Funnel Plots
- Funnel plots are used to analyze publication bias, with a symmetrical distribution indicating minimal bias
Cochrane Collection
- The Cochrane Collaboration provides systematic reviews in a rigorous and consistent manner
- The Cochrane Library has a strong influence on policy and standards for practice
- It includes sections on complementary medicine, with over 70 reviews
- Full access is available through the LRC website
Cochrane Caution
- Cochrane reviews are notoriously conservative in their results, often concluding that 'more research is necessary'
Mini Stats Break
- Statistical significance determines if a difference is real or due to chance
- A p-value indicates the likelihood of a difference being due to chance
- A p-value of 0.05 is generally the cut-off for significance
- Variables that influence p-value calculation include sample size, variation within and between groups
- Confidence intervals (CI) provide a range of possible results, with 95% CI indicating that 95% of the time the results would be within that range
PICO and Searching for Evidence
- PICO is a framework used to ask structured, answerable questions in evidence-based medicine.
- PICO stands for:
- P: Population
- I: Intervention
- C: Comparison
- O: Outcome
Example of PICO
- Marion's question: "Should I wear elastic stockings when I travel to reduce my risk of developing deep vein thrombosis (DVT)?"
- PICO breakdown:
- P: Retired woman who flies frequently
- I: Wearing elastic stockings
- C: Not wearing elastic stockings
- O: Reducing the risk of developing deep vein thrombosis (DVT)
Searching for Evidence
- Primary sources of scientific evidence:
- PubMed
- Other subscription-based databases: EBSCO, CINAHL, MEDLINE
- Google Scholar
PubMed Searching
- Boolean operators: AND, OR, NOT (must be in capital letters)
- Examples:
- "Cancer AND vitamin C"
- "Cancer OR carcinoma"
- "Cancer AND vitamin C NOT intravenous"
Truncation
- Using the asterisk () to truncate search terms: "hyperten" = hypertension, hypertensive, etc.
Field Descriptions
- PubMed can search many fields, including date, author name, journal, title, page number, abstract, etc.
- Example: "(DP) = date of publication", "(TI) = title", "(JT) = journal title"
Advanced Search Builder
- Helps to be more specific in search queries
- Especially useful when looking for a particular article
MeSH Terms
- Medical Subject Headings (MeSH) are a controlled vocabulary
- Examples: "Depression" could refer to respiratory depression, cytokine depression, or depression as a psychiatric condition
Hedges (Filters)
- Combinations of terms or standardized search strategies that can be easily used
Clinical Queries
- PubMed Clinical Queries: subsets of search terms for specific topics, such as dietary supplements or complementary and alternative medicine (CAM)
Saved Search
- Creating a saved search using a string of terms
- Example: using the dietary supplement subset: "dietsuppl [sb] AND osteoarthritis"
Botanical Medicine
- No subset for botanical/herbal medicine
- Consider saving search terms: "Plants, Medicinal"[Mesh] OR "Ethnopharmacology" [Mesh] OR "Phytotherapy"[Mesh] OR "Ethnobotany"[Mesh] OR "Plant Preparations"[Mesh]
Diet
- Diet [Mesh] includes Mediterranean, DASH, keto, vegan, etc.
- Can also look at: Food [Mesh]
Limits
- Study type
- Subjects (human/animal)
- Age of publication
- Age of subjects, gender
- Free full text
Pre-Synthesized Evidence
- Cochrane Library: highly rigorous systematic reviews and meta-analyses
- Natural Medicines: access through CCNM library and CAND member website
- Trip Database: free and upgraded version
- Others: UpToDate, Examine.com (subscription-based)
Sourcing Full Text Articles
- Google Scholar
- Research Gate
- Email the study author (corresponding author) for a pre-publication copy
Limitations of RCTs
- Sample may not be representative
- Not good for rare or distant outcomes
- Application to non-pill interventions can be challenging
- Ethics of treating only some participants
Assessing the Quality of RCTs
- Jadad scale: assesses clinical trials, score from 0 to 5
- Cochrane risk of bias: often used in systematic reviews
- CASP checklist: assesses quality of reporting
- CONSORT: assess quality of reporting, CAM-specific reporting standards (e.g., Non-pharm CONSORT, Herbal Medicine CONSORT, REDHOT, STRICTA)
Jadad Scale
- Assesses clinical trials
- Score from 0 to 5
- 2 points for randomization
- 2 points for blinding
- 1 point for dropouts
Other Kinds of Intervention Studies
Cross-Over Studies
- Everyone gets intervention and comparison
- Pro: minimizes confounding (participants are their own controls)
- Con: must be a chronic illness, treatment must wash out
Cohort Studies
- Strengths: looks at any exposure, confident that exposure came before outcome, assess multiple outcomes
- Weaknesses: assignment to comparison group is not random, time-consuming, inefficient for rare outcomes
- Example: compare exposed and un-exposed groups over time
Case-Control Studies
- Strengths: can look at rare outcomes, faster
- Weaknesses: assignment to comparison group is not random, hard to assess temporality
- Example: compare diseased and non-diseased groups
Hierarchy of Evidence
- Identify inherent strengths and weaknesses of each type of research design
- Assess quality based on design and relevance to practice, individual patients, and public health
RCTs
- Strengths: many ways to decrease risk of bias, randomized, control group, blinding, best design for confirming cause/effect
- Weaknesses: difficult to recruit, only assess one outcome, recall bias
Randomization
- For comparison to be useful, need truly random assignment to treatment/comparison
- Examples: computer-generated sequence, sequential numbered sealed opaque envelopes, 3rd party allocation
Compliance
- Should be assessed
- Simple methods: diary, pill count, phone calls, questioning
- Biological methods: blood/urine levels (costly)
Withdrawals
- How many participants enrolled, completed, dropped out (why?), analyzed
- Accounting for missing data: per-protocol analysis, intention to treat analysis
Intention to Treat Analysis
- Analyze everyone in the group randomized, even if did not complete or ended up in the other group
- Techniques: last observation carried forward, statistical approaches
- Best way to minimize bias from dropouts
RCT Limitations
- Difficult to recall bias
- Only assessing one outcome
- Recall bias)
Cross-Sectional Studies
- Strengths: can study rare outcomes, faster, no recall bias
- Weaknesses: assignment to comparison group is not random, no assessment of temporality, only assessing one outcome
Observational Studies
- Strengths: can study any question, less expensive or faster than intervention studies
- Limitations: not randomly assigned to exposure groups, investigate correlation, not necessarily causation
Appraising the Quality of Observational Studies
- Recruitment: do participants reflect the population of interest? Selection bias
- Assessment of exposure: accurate? Subjective or objective? Validated? Measurement or classification bias
- Consideration of confounding factors? Did they look for confounding factors?
Systematic Reviews
- Strengths: explicit and rigorous methods, scientific investigation, enormous effort to minimize bias, capture the big picture of evidence on a topic
- Limitations: only as good as the available studies, can’t replace good clinical reasoning
N of 1 Study
- Strengths: look at real-world use of an intervention, allows for individualization, root-cause style treatment, complex health conditions, multi-modal treatments
- Limitations: doesn’t work if the condition is curable or self-limiting, must relapse in washout, findings may not be generalizable, ethics, high cost
Preclinical Studies
- Strengths: allow for creativity and innovation, background for future research in humans, investigate mechanism of action, study possible adverse events or interactions, high level of control, ethical
- Limitations: may not be clinically applicable to humans, highly controlled, one isolated part of the story
Consider
- Is pre-clinical evidence ‘sufficient’ enough to guide clinical recommendations?
Grading the Evidence
- Grading of Recommendations, Assessment, Development, and Evaluations (GRADE) is a transparent framework for developing and presenting summaries of evidence.
- GRADE provides a systematic approach to clinical decision making.
How Does It Work?
- A clinical question is formulated in PICO (Patient, Intervention, Comparison, Outcome) format.
- A systematic review provides an estimate of the effect size of an outcome.
- The author rates the quality of the evidence and strength of recommendations.
GRADE Certainty Rating
- Certainty refers to whether an estimate of association or effect is correct or true.
- There are four levels of certainty:
- Very low: The true effect is probably markedly different from the estimated effect.
- Low: The true effect might be markedly different from the estimated effect.
- Moderate: The authors believe that the true effect is probably close to the estimated effect.
- High: The authors have a lot of confidence that the true effect is similar to the estimated effect.
What Makes Evidence Less Certain? (“Rate Down”)
- Five factors can decrease certainty:
- Risk of bias: Inherent limitation in the design of a study that causes the results to be inaccurate.
- Imprecision: Results may be due to chance; focus on 95% confidence interval.
- Inconsistency: Multiple studies showing a consistent effect increase certainty.
- Indirectness: Intervention of interest is not studied in the population of interest and reporting the outcome of interest.
- Publication bias: Is this really all of the research evidence that exists?
What Makes Evidence More Certain? (“Rate Up”)
- Three factors can increase certainty:
- Large magnitude of effect.
- Dose-response gradient.
- All residual confounding would increase our confidence in an effect.
GRADE: 2 Parts
- Certainty of Evidence: How likely is it that something works? (Consider consistency, precision, applicability, and bias.)
- Recommendation Strength: Should it be recommended for use (or not!); consider evidence, benefits/harms, equity, resources, feasibility, and acceptability.
Recommendations
- Can be in favor or against an intervention.
- Can be strong or weak.
- Weak recommendations may lead to variation in decisions made by informed individuals.
- Shared decision-making process becomes essential with weaker recommendations.
Example: Folic Acid Supplementation and Blood Pressure
- A systematic review and meta-analysis of 41,633 participants found that folic acid supplementation significantly decreased systolic BP and diastolic BP.
- Subgroup analysis showed that the results remained significant in specific populations and under certain conditions.
Mental Health Research Challenges
- Unique challenges of studying mental health include obstacles, barriers, risks, and concerns
EASe-GAD Clinical Trial at CCNM
- Trial aims to study the effect of diet on Generalized Anxiety Disorder (GAD)
- Background: mental illness is common, and many people do not benefit from medication and psychotherapy due to access issues, side effects, and inadequate relief
- Observational data suggests a link between better diet quality and better mental health, with possible mechanisms including omega-3/6 balance, vitamins/minerals, protein for neurotransmitter synthesis, and healthy blood sugar regulation
Trial Design and Participants
- Pilot study with a randomized, wait-list controlled design
- Participants: adults (18-65 years old) who identify as women, diagnosed with moderate to severe GAD
- Primary objectives: assess feasibility and acceptability of the intervention
- Secondary objectives: look for signals of change in additional outcomes
- Intervention: personalized dietary advice, motivational interviewing (MI), mindfulness, and Mediterranean diet (MED diet)
- Control group: social support
- Inclusion criteria: stable medication/supplements for 4+ weeks, baseline Omega Score
Statistics
- Define terms:
- Sample: a subset of the population
- Population: the entire group of interest
- Incidence: the number of new cases of a disorder
- Prevalence: the total number of cases of a disorder
Hypothesis Testing
- The process of deciding whether the findings of an investigation reflect chance or real effect
- Statistical significance:
- Difference in group means (averages)
- Difference in the likelihood of an outcome
- P-value: the probability that the difference is due to chance (typically < 0.05)
Statistical Significance: P-Value
- P-value: the odds that the difference is due to chance
- Cut-off: 0.05 (5% likelihood)
- Interpretation:
- P > 0.05: not significant
- P < 0.05: significant
Statistical Significance: Confidence Interval (CI)
- 95% CI: the range within which the true result is likely to lie
- Interpretation:
- Overlapping intervals: non-significant difference
- Non-overlapping intervals: significant difference
Relative Risk
- Probability of a bad outcome in the intervention group divided by the probability of a bad outcome in the control group
- Interpretation:
- RR = 1: no difference
- RR > 1: risk of bad outcome increased with treatment
- RR < 1: risk of bad outcome decreased with treatment
Relative Risk Reduction
- How much the treatment reduced the risk of the bad outcome
- RRR = 1 - relative risk
- Example: 40% reduction in risk with vitamin C
Absolute vs Relative Risk Reduction
- Absolute risk reduction (ARR): the absolute amount the intervention decreased the risk
- ARR = risk of outcome with placebo - risk with medication
- Example: ARR = 8% (40% reduction in risk with vitamin C)
Odds Ratio
- Looks at the relationship between an exposure and an outcome
- Often used in observational studies
- Interpretation:
- OR = 1: exposure does not affect the likelihood of the outcome
- OR > 1: exposure associated with increased likelihood of the outcome
- OR < 1: exposure associated with decreased likelihood of the outcome
Outcomes
- Clinical outcomes: a clinical event (e.g. stroke, suicide, hip fracture)
- Surrogate outcomes: an indicator that can be observed sooner, at a lower cost, or less invasively (e.g. biomarkers, indicators of disease progression)
Statistical Significance vs Clinical Significance
- Statistical significance: not due to chance (p < 0.05)
- Clinical significance: the intervention has a genuine, measurable effect that makes a real difference in the lives of patients
- Power: the ability of a study to detect a real difference between two groups
Studying That Suits You
Use AI to generate personalized quizzes and flashcards to suit your learning preferences.
Related Documents
Description
This quiz covers the basics of systematic reviews in healthcare, including their history and benefits. It's based on a lecture by Dr. Monique Aucoin for ANM 150 students.