Lecture 3 Appraising Research & Deciding If Therapy is Effective (RCTs) PDF

Evidence Based Practice (EBP) CLNS 415  Critically Appraising the Quality of the Research Evidence: (RCTs) By, Ms. Winnie Philip Lecturer 10/27/2019 1. 2. 3. 4. 5. Ask a focused clinical question Search for the best research evidence Appraise the quality of the research evidence Integrate t...

Evidence Based Practice (EBP) CLNS 415  Critically Appraising the Quality of the Research Evidence: (RCTs) By, Ms. Winnie Philip Lecturer 10/27/2019 1. 2. 3. 4. 5. Ask a focused clinical question Search for the best research evidence Appraise the quality of the research evidence Integrate the research evidence with information about the patient and clinical expertise Reflect on the process to improve the future 10/27/2019 Basic types of research designs  Critical appraisal and deciding if therapy is effective / Randomized Controlled Trials  What is effective? ◦ Error in clinical research  Bias  Chance/random error  History of a disease  Placebo effect and blinding  Drop-outs ◦ Intention to treat  10/27/2019 1. Experimental Research 2. Descriptive Research 3. Exploratory Research 4. Integrative Research 10/27/2019    Identifies cause and effect relationships among variables Uses comparison statistics to identify the relationships Common types ◦ Randomized controlled trials (also called true experiments) ◦ Non-randomized trials (also called quasi-experimental designs) ◦ Single-subject designs (results usually are presented in line graphs) 10/27/2019  Randomized Controlled Trial (RCT) – Participants are randomly assigned to either an intervention or control group. This allows the effect of the intervention to be studied in groups of people who are the same, except for the intervention being studied. ◦ Any differences seen in the groups at the end can be attributed to the difference in treatment alone, and not to bias or chance. ‫يمكن ألي اختالفات تظهر في املجموعات في النهاية‬ .‫ وليس إلى التحيز أو الصدفة‬،‫ويعزى ذلك إلى االختالف في املعاملة وحده‬ 10/27/2019 Randomized controlled trials are experimental studies of cause and effect relationships between treatments and outcomes. ◦ Treatment = Independent variable ◦ Outcomes = Dependent variable(s) Eg. Impact of a drug on cancer. 10/27/2019 Dependent variable- cancer independent variable- administration of drug  Randomization is the best control for bias 10/27/2019  Three ways to randomize for a study, each of which may or may not be part of the design. ◦ Random selection of a sample from the sampling frame of the population ◦ Random assignment of the sample to groups ◦ Random assignment of groups to treatment(s) and control conditions 10/27/2019 The abstract of an article gives a lot of information, but not enough to know if the study has internal validity.  The method section of an RCT, in particular, will help to answer questions related to internal validity ◦ USC Evidence Appraisal Checklist 10/27/2019 10/27/2019      Is the study purpose relevant to your clinical question? Are the study’s inclusion and exclusion criteria clearly defined and would the patient in your clinical question qualify for the study? Are the intervention and comparison/control groups receiving an intervention applicable to your clinical question? Are the outcome measures used in the study relevant to your clinical question and are they conducted in a clinically realistic manner? Is the study population sufficiently similar to the patient in your clinical question to justify expectation that the patient would respond similarly to the population? 10/27/2019 10/27/2019 10/27/2019 Describes characteristics of group of people or other phenomena  Often uses questionnaires, interviews, and/or direct observation  Usually uses descriptive statistics; may use correlation statistics  10/27/2019 Identifies relationships between variables  Usually uses correlation statistics  Common types (in no particular order)  ◦ Cohort and case control studies 10/27/2019     Rigorously integrate findings from more than one study on the same topic. Statistical methods vary Types ◦ Evidence-based clinical guidelines ◦ Meta-analyses ◦ Systematic reviews If the studies are soundly designed/conducted, these form the ‘backbone’ of evidence-based practice. 10/27/2019   Peer Review Individual/dietitian appraisal 10/27/2019  A process used to check the quality and importance of research studies. It aims to provide a wider check on the quality and interpretation of a study by having other experts in the field to review the research and conclusions. 10/27/2019 Example of Peer Review Process for Journal Publication 10/27/2019  Clinicians need to be able to answer three questions about the articles that they read. ◦ Are the design and results of the study valid? ◦ What are the results? ◦ Are the results relevant to my clinical question?  The idea is to identify the strengths and weaknesses of the study and decide to what degree (if any) it informs your practice. 10/27/2019        Studies of Interventions ‫دراسات التدخالت‬ ‫املراجعات املنهجية‬ ‫دراسات‬ ‫التشخيص‬ ‫دراسات االختبارات التشخيصية‬ Systematic Reviews ‫الدراسات النوعية‬ ‫تقييم الخبرة السريرية‬ ‫تقييم‬ ‫املبادئ التوجيهية السريرية‬ Studies of Prognosis Studies of Diagnostic Tests Qualitative Studies Appraising Clinical Expertise Appraising Clinical Guidelines 10/27/2019  Three explanations for an observed effect ◦ The treatment had an effect ◦ Chance variation between two groups ◦ Bias 10/27/2019 ‫تركز الفعالية على ما إذا كان التدخل‬ ‫يعمل في ظل ظروف مثالية )كما هو الحال في أ إعداد املختبر( وينظر في ما إذا كان التدخل‬ .‫ليس له أي تأثير على اإلطالق‬  Efficacy focuses on whether an intervention works under ideal circumstances (such as in a laboratory setting) and looks at whether the intervention has any impact at all.  Effectiveness focuses on whether a treatment works when used in the real world. ◦ An effectiveness trial is done after the intervention has been shown to have a positive effect in an efficacy trial. 10/27/2019 – Systematic error (bias) ‫أي عملية منهجية في إجراء الدراسة التي‬ ‫يؤدي إلى تحريف الحقيقة في اتجاه يمكن التنبؤ به‬ • any systematic process in the conduct of a study that causes a distortion from the truth in a predictable direction • selection bias • information bias – Random error (chance) .‫انحراف النتائج واالستدالالت عن الحقيقة يحدث كنتيجة لعملية الصدفة‬ • Deviation of results and inferences from the truth occurring as a result of operation of chance. • reduced with larger sample sizes 10/27/2019  Bias that is caused by some kind of problem in the process of selecting subjects initially or - in a longitudinal study - in the process that determines which subjects drop out of the study 10/27/2019    Inappropriate population studied Inadequate participation Selection of most ‘accessible’ subjects or of volunteers ◦ The rule of EPSEM (Equal Probability of Selection Method): Every case in the population has the same chance of being selected for the sample. 10/27/2019  If randomization is performed correctly, then selection bias on the “front-end” of the study is not possible ،‫إذا تم تنفيذ التوزيع العشوائي بشكل صحيح‬ ‫ثم انحياز االختيار على "الواجهة األمامية" لـ‬ ‫الدراسة غير ممكنة‬ 10/27/2019  Information bias refers to bias arising from measurement error. 10/27/2019  Subject variation: ◦ No two people (subjects) are exactly alike, producing different viewpoints / data ◦ Friendly subjects want to please the researcher  Observer variation: ◦ (Again) No two people (observers) are exactly alike  Deficiency of tools ◦ Invalid – doesn’t measure what you want to measure ◦ Unreliable – inconsistent measurements ◦ Not precise enough  Technical errors in measurement 10/27/2019    Specify criteria/methodology in advance Analyze directly according to criteria/methodology Reduce numbers of observers ◦ Same observer: Greater reliability than two or more  Use standardized tools for measurement ◦ Test administered in the same way to every subject 10/27/2019    Natural history of a disease Placebo effect Drop outs 10/27/2019  The way in which a disease evolves over time from initial stage > more severe > some outcome (recovery, death, disability) ◦ If a disease is staged (ie, I, II, III, etc.), each group should have approximately the same number of subjects in stage I, and/or stage II, etc. (t-test or ANOVA would reveal this) (Brookmeyer, 1990) 10/27/2019   Placebo: An inactive substance or procedure used as a control in an experiment. Average relief rate from a placebo may be as high as 35% (Beecher, 1950) 10/27/2019  Blinding: in an experimental study, refers to whether patients, clinicians providing an intervention, people assessing outcomes, and/or data analysts were aware or unaware of the group to which patients were assigned. 10/27/2019 36    Single Blinded: Researcher (Investigator) or participants (Patient) does not know treatment Double Blinded: Neither researcher nor participants knows treatment Triple blinded: Participants, researchers and data analysts (Statistician) are all unaware of the treatment the participant receives. 10/27/2019 10/27/2019   Individual meets inclusion criteria and is admitted to study The individual is unable to complete the study for any of a number of reasons that include: ◦ ◦ ◦ ◦ Might ‘withdraw’ or simply disappear Moved out of town Hospitalized Died 10/27/2019  An analyses that is conducted when people drop out or switch groups (you need at least 80% of your original subjects in their original groups to maintain validity). 10/27/2019   You are at a family get-together and you hear your 68 year-old relative complaining that “I’m so overweight I can’t even tie my shoes any more.” I’ve tried fad diets and I’ve also tried walking when the weather is good, but I don’t seem to lose much weight at all.” She turns to you and says, “Aren’t you in therapy school? Which is better, diet or exercise? What do you recommend?” 10/27/2019  Use this study to inform your response to your relative’s question. 10/27/2019        Clearly-stated purpose Demonstrates external and internal validity Appropriate participant selection and sampling Avoids/minimizes bias Outcome measures reliable and valid Appropriate analysis and interpretation of statistics Clinical bottom-line… does this study inform your practice? (Include Pros and Cons) ‫غرض محدد بوضوح‬ ‫يوضح الصالحية الخارجية والداخلية‬ ‫اختيار املشاركني‬ ‫املناسبني وأخذ العينات‬ ‫يقلل من التحيز‬/‫يتجنب‬ ‫مقاييس النتائج موثوقة وصالحة‬ ‫تحليل وتفسير مناسب لإلحصاءات‬ ‫ هل تفيدك هذه الدراسة‬... ‫النتيجة السريرية‬ (‫يمارس؟ )تشمل اإليجابيات والسلبيات‬ 10/27/2019   Complete the entire appraisal for the Villareal et al. article You can discuss this appraisal with classmates; (do it yourself first, then compare notes.) 10/27/2019 10/27/2019 10/27/2019 10/27/2019 10/27/2019 How confident are you in this study and whether you use it to inform your clinical decision?  Pros  Cons 0 9 1 10 2 3 4 5 6 7 Complete confidence / “Bet the Farm” No confidence 10/27/2019 8 There is no perfect article.    Dawes, M. (2005). Is this therapy effective? In M. Dawes, P. Davies, A. Gray, J. Mant, K. Seers, & R. Snowball, Evidence-based practice: A primer for health care professionals (2nd ed.; pp. 181200). London, UK: Elsevier Churchill Livingstone. Portney, L. G., & Watkins, M. P. (2000). Foundations of clinical research: Applications to practice (2nd ed.). Norwalk, CT: Appleton & Lange. Villareal, D.T., Chode, S., Parimi, N., Sinacore, D.R., Hilton, T., Aramamento-Villareal, R., et al. (2011). Weight loss, exercise, or both and physical function in obese older adults [Electronic version]. The New England Journal of Medicine, 364, 1218-1229. 10/27/2019    Sackett, D. L., Straus, S. E., Richardson, W. S., Rosenberg, W., & Haynes, R. B. (2000). Evidence-based medicine: How to practice and teach EBM (2nd ed.). New York: Churchill Livingstone. Straus, S. E., Richardson, W. S., Glasziou, P., & Haynes, R. B. (2005). Evidence-based medicine: How to practice and teach EBM (3rd ed.). New York: Churchill Livingstone. Wong, V. (2003). Evidence-based approach of the use of Botulinum toxin type A (BTX) in cerebral palsy. Pediatric Rehabilitation, 6, 85-96. 10/27/2019