Understanding Systematic Reviews in Medicine

Systematic Reviews

• In 1987, Cochrane referred to a systematic review of corticosteroid treatment in pre-term births, which showed that a short and inexpensive course of corticosteroid treatment substantially reduced the risk of premature deaths due to complications.

Objectives

• Recognize the important features and strengths of a systematic review or meta-analysis
• Interpret an odds ratio and a forest plot
• Design a search strategy to find a systematic review on a topic

Methodology

• Clear question (PICO)
• Did they look for the right type of studies (relevant to question, appropriate design, e.g., RCT if intervention)
• Comprehensive search (databases, reference lists, unpublished studies, contact experts, non-English studies)
• Clear eligibility criteria
• Assessment of individual study quality

Results

• If results were combined, was it appropriate? (were the studies similar enough?)
• Heterogeneity test (Cochrane Q, I2)
• Is the data too heterogeneous to combine and get a meaningful result?

Publication Bias

• Results of MA only valid if capturing the full range of trial results
• Attempt to analyze the potential for publication bias using individual study data (magnitude of result, statistical significance)
• Plotted on a funnel plot - Look for symmetrical distribution

Cochrane Collection

• The Cochrane Collaboration runs the Cochrane Library, providing systematic reviews in a rigorous and consistent fashion
• Strong influence on policy – standards for practice
• Sections on complementary medicine (>70 reviews)
• Full access available through LRC website

Cochrane Caution

• Cochrane reviews are notoriously conservative in results
• Frequently, it is concluded that ‘more research is necessary’
• However, this is a reputable, non-biased source of reliable, up-to-date overview of evidence on a specific topic

Mini Stats Break!

• Statistical significance: Is there a REAL difference or chance?
• Flipping a coin example: 51 H/49 T → likely chance, 99 H/1 T → likely NOT due to chance
• What about 70/30?

Statistical Significance

• P-value: what are the odds that the difference is due to chance?
• 0.05 is generally the cut-off (5% likelihood)
• p greater than 0.05 → not significant
• p less than 0.05 → significant
• Variables that influence this calculation: sample size, variation within the groups, variation between the groups

Confidence Interval (CI)

• 95% CI: we don’t know the exact result, if we repeated the experiment 100X, 95% of the time the results are within this range
• If these intervals overlap → non-significant difference
• If they don’t overall → significantly different

Odds Ratio

• Looking at the relationship between an exposure and an outcome
• Often used in observational studies (but sometimes in clinical trials too)
• OR = 1 exposure does not affect the likelihood of the outcome
• OR > 1 exposure associated with increased likelihood of the outcome

Learn about the importance and strengths of systematic reviews, interpreting odds ratios and forest plots, and designing search strategies. Test your knowledge of this essential research methodology in medicine.