Research test.pdf

ART AND PRACTICE: RESEARCH WEEK 3 DR. MONIQUE AUCOIN ND MSc ANM 150 QUESTION: WHY ARE SOME STUDY TYPES HIGHER ON THE HIERARCHY OF EVIDENCE? OUTCOMES Identify inherent strengths and weaknesses of each type of research design and how to assess quality based on their design and relevance to pract...

ART AND PRACTICE: RESEARCH WEEK 3 DR. MONIQUE AUCOIN ND MSc ANM 150 QUESTION: WHY ARE SOME STUDY TYPES HIGHER ON THE HIERARCHY OF EVIDENCE? OUTCOMES Identify inherent strengths and weaknesses of each type of research design and how to assess quality based on their design and relevance to practice/individual patients and public health. RESEARCH STUDY METHODOLOGIES HIERARCHY OF EVIDENCE QUESTION: WHAT ATTRIBUTES OF AN RCT GIVE IT A HIGH POSITION IN THE HIERARCHY? RCTS Tx A TIME Tx B RANDOMIZED CONTROLLED TRIAL: STRENGTHS Many ways to decrease the risk of bias Randomized: Equal chance of being assigned to the intervention or control group Control group: accounts for natural course of illness, placebo effect, confounding factors May have Blinding: minimize expectation effect RCT Use: **Best Design for confirming cause/effect**  Treatment COMPARISON Control Accounts for: -natural course of the disease -other factors that affect prognosis (cofounders) ex. Exercise, family history -expectation effect Placebo (inert identical tablet), other treatment, usual care, nothing -These ask different questions  Treatment Control RANDOMIZATION For comparison to be useful, need to have truly RANDOM assignment to treatment/comparison -ex. people who visit clinic on weekday vs weekend; people who visit clinic A vs clinic B; judgement of clinician; flip of a coin; alternating -better: computer generated sequence, sequential numbered sealed opaque envelopes, 3rd party allocation Should describe HOW the randomization was done Should check to see that it worked, how similar were the groups? Selection Bias TABLE 1 Sample confounding factors: Age Gender Family history Comorbidities BMI Socioeconomic status Marital status Education Exercise Diet ANYTHING, known or unknown BLINDING Remove expectation Who was blind? -Participants (‘single blind’) -person delivering the intervention (‘double blind’) -person assessing the outcome (‘triple blind’) Should describe how it was achieved Bias in measurement of outcomes CLEARLY DEFINED POPULATION Participants are a SAMPLE of some population Inclusion/exclusion criteria help define Similar enough to your patient? (“generalizability”) -more or less ill? -different ethnicity or geographic location? -pregnancy, smoking, OCPs, non-English speaking, unable to read Often excluded consent form, elderly, comorbidities, taking other medications from clinical trials RECRUITMENT What was the method? Can influence the sample Ex. newspaper ad – people who read the newspaper and motivated to respond Ex. every person who presents at a clinic with a particular condition Selection Bias REPORTING OF RESULTS Power calculation Are the things they planned to measure (methods section) reported in the results section? Any missing data? Were p values reported? Reporting Bias COMPLIANCE Should be assessed Simple methods: Diary, pill count, phone calls, questioning Biological methods: blood/urine levels (costly) WITHDRAWALS How many participants enrolled, completed, dropped out (why?), analyzed Accounting for missing data: -Per-protocol analysis (analyse people in the intervention group that they completed) -Intention to treat analysis (analyse people in the intervention group that they were assigned to)  WITHDRAWALS: RISK OF PER- PROTOCOL ANALYSIS Mean skill level High EIP Skill High EIP Skill Mean skill level Moderate EIP Skill Extremely Hard Moderate EIP Skill Course Low EIP Skill Low EIP Skill Systematic factor impacting who drops out Other ex’s: side effect, acceptability, impact on outcome INTENTION TO TREAT ANALYSIS Analyze everyone in the group randomized, even if did not complete or ended up in the other group Techniques: last observation carried forward, statistical approaches Best way to minimize bias from drop outs Cautious, under-estimate of effect -Pre-Protocol: over-estimate  INTENTION TO TREAT ANALYSIS High EIP Skill High EIP Skill Mean skill level Extremely Hard Moderate EIP Skill Moderate EIP Skill Mean skill level Course Low EIP Skill Low EIP Skill RCT LIMITATIONS Difficult! (recruitment, funding, time) Sample may not be representative Not good for rare/distant outcomes Application to non-pill interventions can be challenging (control, blinding) Ethics of treating only some participants ASSESSING THE QUALITY OF RCTS Jadad Cochrane risk of bias –often used in SRs CASP checklist - https://casp-uk.net/casp-tools-checklists/ Assess quality of REPORTING: CONSORT CAM-specific reporting standards: Non-pharm CONSORT, Herbal Medicine CONSORT, REDHOT (homeopathy), STRICTA (acupuncture) JADAD SCALE Assess clinical trials Score from 0 to 5 2 points randomization 2 points blinding 1 point drop outs Stephen, H., Halpern, M. and Joanne, D., 2005. Jadad scale for reporting randomized controlled trials. OTHER KINDS OF INTERVENTION STUDIES Cross-over: everyone gets intervention AND comparison Pro: minimize confounder (participants are their own controls) Con: must be a chronic illness, treatment must washout Image source: Cochrane COHORT STUDIES TIME Compare Exposed COHORT STUDY Incidence TIME Un-exposed COHORT STUDIES Strengths Weaknesses Look at any exposure Assignment to comparison (even harm) grp is NOT random → risk Confident that exposure of confounding came before outcome Time consuming Assess multiple Inefficient for rare outcomes outcomes CASE-CONTROL TIME Compare Diseased CASE-CONTROL Prior Exposure TIME Non-Diseased CASE CONTROL STUDIES Strengths Weaknesses Can look at rare Assignment to comparison outcomes grp is NOT random Faster (no waiting time, Hard to assess temporality minimal loss of (ex. recall bias) participants) Only assessing one outcome CROSS-SECTIONAL STUDIES CROSS-SECTIONAL STUDY Compare Current Disease Status and Current Exposure CROSS-SECTIONAL STUDIES Strengths Weaknesses Can study rare outcomes Assignment to comparison Faster grp is NOT random No recall bias No assessment of temporality (which came first?) Only assessing one outcome OBSERVATIONAL STUDIES: STRENGTHS Can study any question (harmful exposure, lack of a beneficial exposure) Can be less expensive or faster than intervention studies OBSERVATIONAL STUDIES: LIMITATIONS Not randomly assigned to exposure groups Investigate correlation, not necessarily causation APPRAISING THE QUALITY OF OBSERVATIONAL STUDIES Recruitment: Do the participants reflect the population of interest? Selection Bias Generalizability of Assessment of exposure: accurate? Subjective or Findings objective? Validated? Measurement or Consideration of confounding factors? Classification Bias Did the look for confounding factors? SYSTEMATIC REVIEWS STRENGTHS Explicit and rigorous methods to: 1. Identify (2+ databases, specific inclusion/exclusion criteria) 2. Critically appraise 3. Synthesize (combine) Scientific investigation with pre-planned methodology Enormous effort to minimize bias Capture the big picture of evidence on a topic Meta-analysis allows for the creation of a larger sample size (helps with stats: stay tuned for next semester) SYSTEMATIC REVIEW LIMITATIONS Only as good as the available (findable) studies → publication bias, lack of research on a topic Can’t replace good clinical reasoning WHAT MAKES A STRONG SR? Clear question? Did they look for the right type of studies? -relevant to question, approp design (RCT if intervention) Comprehensive search? -databases, reference lists, unpublished studies, contact experts, non-English studies Assessment of study quality? N OF 1 STUDY N OF 1 STRENGTHS Look at real world use of an intervention Allows for individualization, root-cause style treatment, complex health conditions, multi-modal treatments Can compare naturopathic and conventional treatments Could justify further research Consistent population (same person! Same genetics, family history, other risk factors) N OF 1 LIMITATIONS Doesn’t work if the condition is curable or self-limiting, must relapse in washout Findings may not be generalizable (low external validity) Ethics – need ethical approval, is it ethical to experiment in a clinical setting? High cost (manufacturing of placebo, administration of study with blinding) PRECLINICAL STRENGTHS Allow for creativity and innovation Background for future research in humans Investigate mechanism of action Study possible adverse events or interactions High level of control (ex. Exact intake of fiber in a mouse diet) Ethical (?) PRECLINICAL LIMITATION May not be clinically applicable to humans (lack generalizability) Highly controlled One isolated part of the story CONSIDER: Is pre-clinical evidence ‘sufficient’ enough to guiding clinical recommendations? Eg. In vitro anti-fungal effects of garlic are well known - does this mean that systemic fungal infections (highly dangerous) can/should be treated with garlic? At what dose? Eg. Soy phytoestrogen inhibits MCF-7 human breast cell growth in vitro - what does this mean for prevention/treatment of breast cancer? MIGHT CONSIDER Informed consent (document risks, benefits, level of evidence) Consider alternatives Monitor patient (response, safety) Alter treatment if needed WRAP UP AND QUESTIONS ANY THOUGHTS OR INSIGHTS OR NEW P E R S P E C T I V E F R O M TO D AY ’ S M AT E R I A L O R T H E R E S E A R C H S E C T I O N AS A WHOLE GRADING THE EVIDENCE CASE 5 ANM 150 Dr. Monique Aucoin ND MSc HOW DO WE WEIGH THE EVIDENCE? GRADE Grading of Recommendations, Assessment, Development and Evaluations Transparent framework for developing and presenting summaries of evidence Systematic approach to clinical decision making HOW DOES IT WORK? Clinical question (PICO format) A systematic review provides an estimate of the effect size of an outcome Author rates the quality of the evidence and strength of recommendations  GRADE CERTAINTY RATING Certainty: whether an estimate of association or effect is correct or true Certainty What it means Very low The true effect is probably markedly different from the estimated effect Low The true effect might be markedly different from the estimated effect Moderate The authors believe that the true effect is probably close to the estimated effect High The authors have a lot of confidence that the true effect is similar to the estimated effect WHAT MAKES EVIDENCE LESS CERTAIN? (“RATE DOWN”) 1. Risk of bias 2. Imprecision 3. Inconsistency 4. Indirectness 5. Publication bias 1. RISK OF BIAS Bias → inherent limitation in the design of a study cause the results to be inaccurate Ex. Studies not randomized, randomized trials not blinded when they should be, loss to follow up of participants, observational studies not adjusted for important cofounders Many tools for assessing risk of bias in individual studies (vs GRADE assess the body of evidence as a whole) 2. IMPRECISION Are the results due to chance? In GRADE, focus on 95% confidence interval Ex. Few observed events (the outcome) or few participants in the studies can decrease precision and widen the confidence interval Certainty is lower if decision is different at upper/lower end of confidence interval Imprecision may be ‘rated down’ if there were few events 3. INCONSISTENCY Many similar studies showing a consistent effect increase certainty Assessed by comparing the results of individual studies (look at a forest plot!) Ex. Confidence intervals overlapping, difference in estimate of effect, formal tests for statistical heterogeneity 4. INDIRECTNESS Certainty is down rated when the intervention of interest is not studies in the population of interest and reporting the outcome of interest Ex. surrogate vs clinical outcome, is the studied intervention the exact intervention of interest (dosing, method of administration etc) vs treatment at a highly specialized facility 5. PUBLICATION BIAS Is this really all of the research evidence that exists? Ex. Only small studies that confirm investigators’ perceptions of the effect are available but additional studies were conducted, results that have not been published There are visual and statistical methods for assessing (Funnel Plot) WHAT MAKES EVIDENCE MORE CERTAIN? (“RATE UP”) Large magnitude of effect Dose-response gradient All residual confounding would increase our confidence in an effect Ex. A very large observational or non-randomized study without other limitations GRADE: 2 PARTS 1) Certainty of Evidence How likely is it that something works? All the factors we have considered so far: are the results consistent? Precise? Apply in this setting? Bias? 2) Recommendation Strength Should it be recommended for use (or not!) Consider Evidence + benefits/harms, equity, resources, feasibility, acceptability RECOMMENDATIONS Can be in favour or against an intervention Can be strong or weak If weak: likely to be variation in the decision made by informed people The weaker the recommendation: more essential the shared decision making process becomes LET’S SEE AN EXAMPLE! Asbaghi O, Salehpour S, Rezaei Kelishadi M, Bagheri R, Ashtary-Larky D, Nazarian B, Mombaini D, Ghanavati M, Clark CC, Wong A, Naeini AA. Folic acid supplementation and blood pressure: A GRADE-assessed systematic review and dose-response meta-analysis of 41,633 participants. Critical Reviews in Food Science and Nutrition. 2021 Aug 14:1-6. Abstract: Hypertension is a predisposing factor for cardiovascular disease (CVD). The extant literature regarding the effects of folic acid supplementation on blood pressure (BP) is inconsistent. Therefore, this systematic review and meta- analysis of randomized controlled trials was conducted to summarize the effects of folic acid supplementation on BP. A systematic search was carried out in PubMed, Scopus, ISI Web of Science, and Cochrane library, from database inception to August 2021. Data were pooled using the random-effects method and were expressed as weighted mean difference (WMD) and 95% confidence intervals (CI). The pooled results of 22 studies, including 41,633 participants, showed that folic acid supplementation significantly decreased systolic BP (SBP) (WMD: −1.10 mmHg; 95% CI: −1.93 to −0.28; p = 0.008). Subgroup analysis showed that the results remained significant when baseline SBP was ≥120 mmHg, intervention duration was ≤6 weeks, intervention dose was ≥5 mg/d, in patients with CVD, males and females, and overweight participants, respectively. Furthermore, the changes observed in diastolic BP (DBP) (WMD: −0.24 mmHg; 95% CI: −0.37 to −0.10; p < 0.001) were also statistically significant. However, subgroup analysis showed that the results remained significant in subject with elevated DBP, long term duration of intervention (>6 weeks), low dose of folic acid (