Audit and Feedback Strategies in Clinical Practice

• Performance feedback is a strategy to improve clinical practice among students and clinicians.
• The study examines conceptual underpinnings and essential components for audit and feedback strategies.
• Aims include assessing recent audit and feedback interventions and recommending future directions.
• Assessed timeliness, individualization, lack of punitiveness, and customizability in studies published during 2009–10, based on the actionable feedback model.
• The study found wide variation in the definition, implementation, and outcomes of feedback interventions, which complicates comparisons across studies.
• Studies reviewed don't have all components of the actionable feedback model.
• Interventions often lack concepts of behavior change, which partially explains the variation in approaches and results.
• Standardization and clarity are needed in the action steps to apply behavioral theory to practice.
• Strategies to improve clinical performance and adherence to standards include educational programs, marketing, and feedback.
• Two literature reviews from 2009 showed no single approach to improving clinician prescribing practices was superior, and the effects were small.
• Auditing and feedback rely on the assumption that recipients are willing to modify behavior and agree on the goals of change.
• Feedback must be relevant, meaningful, accessible, and interpretable for the learner.
• Studies do not clearly describe the theoretical approach underlying choice of intervention or delivery.
• Aims include reviewing conceptual underpinnings and essential components for audit and feedback interventions and recommending future directions.

Conceptual Underpinnings and Essential Components for Feedback Interventions

• Variations of performance feedback have appeared in literature for many decades
• Kluger and DeNisi published an extensive discussion of feedback history in 1996
• A meta-analysis compared performance feedback with other interventions and a control group.
• The meta-analysis included 131 studies and identified 607 effect sizes, suggesting feedback has a positive effect on behavior.
• More than one-third of the interventions were associated with worse performance, indicating variability in approaches and outcomes.
• Strategies referred to as performance feedback are widely divergent in content and implementation.
• Kluger and Denisi developed the 'feedback intervention theory' (FIT) based on five assumptions:
• Behavior is regulated by comparing practice with a goal or standard
• Goals/standards are rank ordered
• Attention to gaps between feedback and goals impacts behavior
• Learner attention directs to moderate control hierarchy
• Feedback changes locus of control so the learner feels more ‘in charge’
• Central explanatory theme of FIT is how feedback focuses one’s attention.
• Components of the FIT were borrowed from control theory, people change practices when there is a gap between current behavior and desired goal.
• Behavior change theories rarely specify application
• Control theory says feedback must include clear goals and an action plan to meet them.
• If the discrepancy between practice and goal is too large, and there is no plan, the recipient may give up and cause a negative effect.
• Abraham and Michie produced a taxonomy of 26 behavior change techniques in 2008
• The change techniques came from five theoretical frameworks:
• Information-motivation-behavioral skills model
• Theory of reasoned action
• Theory of planned behavior
• Social-cognitive theory
• Control theory and operant conditioning.
• The taxonomy was used in three systematic reviews to code the presence or absence of change techniques in behavioral interventions
• Wide variation in intervention results were found, accurately categorizing behavior change interventions and establishing a common language.

Current State-of-the Art in Performance Feedback Interventions.

• One systematic review of 118 studies showed variations in effects from 16% decrease to 70% increase in adherence to guidelines
• Unclear conceptual underpinnings may explain variations in approaches and results, theoretical framework provides context for assessment.
• Authors have commented on the absence of behavioral theories in reports and reviews of educational strategies.
• Using a multivariate meta-regression of 85 interventions, 72% feedback only, Gardner reported that only 4% incorporated feedback, performance goals and an action plan
• A review of 85 randomized trials assessing feedback interventions states that variations in findings differed substantially due to the varying methods of delivery of feedback or differing contextual elements.
• Little detail is available regarding how to implement feedback, the problem is not a lack of theoretical models but a lack of understanding or effort to determine steps to put theory into practice.
• Despite the recognized need for a theoretical approach in designing feedback, it's often not used or described.
• Hysong and colleagues examined successful feedback mechanisms in six Veterans Affairs medical centers
• Four components of feedback interventions in high performing settings were described: timeliness, individualization, lack of punitiveness, and customizability.
• Characteristics have shared features with the FIT theory of Kluger and Denisi and other underlying principles of behavior change.
• Authors developed an 'actionable feedback' model, as a theoretical guide to planning and evaluating feedback interventions.
• In 2009, Hysong re-examined and updated a Cochrane Systematic Review to determine whether published research applied the components of feedback interventions proposed by Kluger and Denisi in the FIT model.
• Nineteen of 519 studies included characteristics of the feedback and content, format, and frequency theorized to be important
• A meta-analysis concluded that overall feedback interventions appeared to have a statistically significant but moderate positive impact on outcomes.
• Recently published reports use PubMed using specified search strings
• In nine studies, multiple interventions including traditional educational programs, decision support tools, simulations and various forms of reminders or alerts were included.
• 13 eligible studies were the result.
• Four studies described feedback interventions for college students; three focused on reducing heavy alcohol consumption and one reducing marijuana use.
• No effect was reported for a web-based personalized feedback intervention to reduce marijuana use.
• A modest effect on alcohol use was reported for a targeted intervention delivered electronically on a single occasion.
• Single versus biannual feedback compared in first year college heavy drinkers, gender-specific feedback related to fewer drinking problems.
• Face-to-face or computerized feedback was compared by Butler and Correia in heavy drinking students
• Interventions were more successful than control.
• Six studies used feedback interventions to influence clinician behaviors, five reported positive results.
• Naughton, et al. provided GPs written feedback on prescribing practices, a second group received feedback with academic detailing
• Feedback related to prescribing while academic detailing added no further improvement
• Clinical records of patients receiving antibiotics were reviewed by an infectious disease specialist who gave feedback to the charge physician
• Duration of antibiotic therapy was shorter during the intervention period
• Nair and colleagues tested feedback mechanisms with only the decision support alerts associated with improved antibiotic prescribing
• Another study assessed feedback in other areas, in four primary care clinics physicians were given group feedback in comparison to other clinics
• Two clinics randomized to have diabetic patients receive a letter and phone call to remind their physician to address relevant topics
• Improvements increased when pairing feedback to physicians and giving the patients supporting information versus physicians alone
• Impact assessed of performance feedback on antipsychotic medication prescribing.
• Over a period of 4.5 years, verbal and written feedback was provided at different intervals, services receiving additional feedback showed significant improvement in prescribing practices.
• Selected participants were randomized to receive feedback in the treatment condition impacted the relationship between perfectionism and emotional reactivity
• None of the recently reported studies employing feedback interventions included all the components of the actionable feedback model
• Some gave feedback infrequently, some were individualized, none were designed to be customizable, non-punitive appeared to be the case based on descriptions provided
• Recipients of feedback had minimal input into its content or delivery modality, recipients were passive
• Relevance and importance of feedback could be enhanced by more active involvement in the development of the feedback intervention
• Information should be meaningful, enhance the credibility of the information, medium convenient, easy to understand, and readily accessible to recipients

Conclusions and Recommendations

• Feedback interventions are relatively atheoretical which may explain the variation in approaches, educator and researcher face challenges attempting to replicate specific interventions
• Theory driven interventions may be more effective but the main value of theory informs generalizability and understanding of why specific applications work in certain contexts
• Standardization and clarity is needed for successful use of educational strategy.
• Features of interventions must be fully described in research reports.
• Educators and researchers should use components that improve clinical practice and behavior.
• Recipients of feedback should be actively involved from the onset of the process.