Evaluating Randomized Controlled Trials

Evaluating Randomized Controlled Trials

• RCTs are the strongest evidence if designed and conducted correctly
• Poor design and methodology can produce misleading results, negatively impacting patient care
• Clinicians need to evaluate the impact of flaws in RCTs to apply findings to patient care effectively

Objectives

• Identify and describe the use of formal criteria to assess the quality of randomized trials
• Assess validity issues in randomized trials
• Apply general criteria to evaluate methodological rigor in randomized trials
• Evaluate common biases in randomized trials
• Interpret and apply key findings in clinical practice
• Perform relevant calculations for RCTs

Importance of Critical Evaluation

• Well-designed and conducted RCTs provide the strongest evidence
• Poor study design and methodology can lead to misleading results, impacting patient care
• Clinicians should evaluate flaws to appropriately apply findings to patient care

Criteria for Evaluating and Reporting Clinical Trials

• Consolidated Standards of Reporting Trials (CONSORT) Statement
• Provides standards for preparing RCT reports
• Enhances accuracy and clarity in medical literature

CONSORT Checklist (2010)

• The CONSORT 2010 checklist provides a comprehensive framework for the reporting of randomized trials

Validity

• Internal Validity: The degree to which a study's design and conduct accurately measure what it intends to measure
• Factors could lead to decreased internal validity:
  • History
  • Testing
  • Instrumentation
  • Maturation
  • Statistical regression
  • Selection
  • Attrition
• External Validity: The ability to generalize the findings of a study to a larger population
• Factors that could decrease external validity:
  • Interaction of selection and intervention
  • Effects of testing
  • Effects of experimental arrangements
  • Multiple treatment effects
  • Study setting (single vs. multi-center)

Validity: Empagliflozin Example

• Internal Validity:
  • History, testing, instrumentation are less likely to cause problems if a control group is used
  • Randomization makes maturation, statistical regression, and selection less likely to bias the result
  • Attrition is likely to be similar between groups, so this bias is minimized
• External Validity:
  • Multicenter studies, conducted across multiple countries, are preferable for generalization to wider populations

Baseline Characteristics

• A table showing baseline demographics and clinical characteristics of patients in the study groups
• This helps investigators critically evaluate if the groups are similar at the start of the study

Study Sample

• Recruitment:
  • Recruitment strategies and eligibility criteria are crucial
  • Comparing the studied population to the population that needs treatment
  • Small samples from a single center make it hard to discern if the results are generalisable
• Inclusion criteria: Determine the types of patients included
• Exclusion criteria: Establish reasons for excluding patients; if too many exclusions the results may not accurately reflect the general population
• Sample size and power calculation:
  • Important to find a difference when one actually exists or false negatives are present
  • Should be calculated for the primary endpoint before trial execution
• Empagliflozin Example:
  • The study utilized 6609 patients for testing

Randomization and Blinding

• Randomization Methodology:
  • Helps ensure similar groups by random assignment to treatment/control groups
  • Stratified or blocked randomization can further balance certain characteristics.
• Blinding:
  • Minimizes bias from the researchers, patients and participants, by concealing treatment assignments
  • Useful when patient or researcher expectations may affect the results or interpretations
• Empagliflozin Example:
  • Patients were randomized, using matching to the placebo

Intervention and Control Groups

• Appropriate Comparators:
  • Selecting a suitable control group, either placebo or another standard of care is vital
  • Must be an ethical choice for comparison
• Treatment Adherence:
  • Monitoring adherence to interventions can assist with determining if results are accurate

Clinical Endpoints

• Primary Outcomes:
  • Should clarify the exact clinical outcomes measured.
  • This needs to be validated with instruments and subjective /objective measures. Multiple endpoints may produce a composite endpoint, requiring careful consideration
• Empagliflozin Example: The primary outcome was kidney disease progression or cardiovascular death and was clearly defined and measured

Trial Findings

• Intention-to-Treat (ITT):
  • All participants who were enrolled in the trial and provided informed consent are analyzed
• Sample size considerations:
  • Sample size is crucial for the study to be representative of the larger population
• External Factors:
  • Factors like concurrent medications are also observed in the analysis
• Statistical tests:
  • Reporting statistical tests appropriatley, to avoid type I and type II errors is important
• Variance:
  • The variance is reflected in the standard deviation and confidence intervals
• Predefined treatment effect size:
  • Helps determine if treatments are superior
• Non-inferiority:
  • Used in determining if a new treatment is as good as an existing treatment or reference
• Empagliflozin Example: The study used a Cox proportional hazard model relating risk factors to survival time.

Trial Findings: Analysis of Subgroups

• Subgroups provide further insight, but should be planned in advance or be justified.
• Having many subgroups increases the possibility of error and could make the overall results hard to interpret

Limitations and Implications

• Bias: Assessing potential bias and limitations is crucial and this is often discussed in the article’s conclusion
• Generalizability: The study results should be considered in context of the study population. Is it safe to generalize the results to the broader population?
• Empagliflozin Example: The study reported limitations and comparisons with other relevant studies

Application to Clinical Practice

• Understanding Current Knowledge, Unknowns, and Questions: What is known and unknown, unanswered questions regarding practical use are identified
• Translate Results to Patient Care:
  • How can the study findings be applied to patient needs, care or treatment plans for individual patients?
  • Cost-effectiveness is a crucial consideration
• Empagliflozin Example: Evaluating if the treatment should be recommended to patients at risk for disease progression and considering cost-effectiveness relative to other SGLT2 inhibitors

Calculations Review

• Risk: Probability that an event will occur.
• Relative risk: Difference between treatment and control groups
• Relative risk reduction (RRR): Reduction in risk in treatment compared to control group
• Absolute risk reduction (ARR): Actual difference in risk from the control intervention
• Number needed to treat (NNT): Number of patients who need treatment intervention to have one favorable outcome.
• Number needed to harm (NNH): Number of patient interventions likely to have an adverse event.