Clinical practice: Performance Feedback

• Performance feedback is a strategy to improve clinical practice among learners.
• The objectives are to examine conceptual underpinnings for audit and feedback strategies, assess published audit and feedback interventions and recommend future directions to improve feedback.

Methods

• Reviewed studies published during 2009–10 that included a feedback intervention
• Used the actionable feedback model
• Looked for timeliness, individualization, lack of punitiveness, and customizability.

Results

• There was variation in the definition, implementation and outcomes of feedback interventions.
• None of the reviewed studies included every component of the actionable feedback model.

Conclusions

• Feedback interventions often do not include concepts of behavior change, even with positive results.
• The variation in approaches and results presents challenges for replicating interventions.
• Standardization and clarity are needed in action steps to apply behavioral theory.

Strategies to Improve Clinical Performance

• Traditional educational programs
• Marketing techniques (lectures and seminars)
• Monitoring performance
• Individual or group performance feedback

Literature Reviews

• Two reviews of interventions to improve clinician prescribing practices in 2009 found no single superior approach.
• Effect sizes were generally small.
• Auditing and feedback mechanisms require recipients to modify their behavior and agree on goals.

Feedback Considerations

• Feedback should be relevant, meaningful, and accessible.
• Learner should know how to interpret and act on results.
• Approaches and components of 'audit and feedback' interventions vary widely.
• Few studies articulate a theoretical approach or how it is delivered.

Kluger and DeNisi (1996) Findings

• Meta-analysis showed feedback had a positive effect on behavior.
• More than one-third of interventions were associated with worse performance.
• The variability in approaches and outcomes is notable.

Feedback Intervention Theory (FIT) Assumptions

• Behavior is regulated by comparing practice with a goal or standard.
• Goals or standards are rank ordered by importance.
• Gaps between feedback and goals receiving attention impact behavior.
• Learner attention is normally directed to a moderate level of control hierarchy.
• Feedback changes the locus of control so the learner feels more ‘in charge'.

FIT Theme

• The central explanatory theme is how feedback focuses attention, not how it affects learning or motivation.

• Components were borrowed from control theory.

• People change practices when there is a gap between behavior and a desired goal.

• Behavior change theories rarely describe the actions to apply theoretical constructs.

• For control theory application, feedback must include articulated performance goals and an action plan.

• If the discrepancy is too large and there is no plan for bridging it, feedback can have a negative effect.

Taxonomy of Behavior Change Techniques

• Abraham and Michie (2008) produced a taxonomy of 26 behavior change techniques, including performance feedback.
• It came from 5 theoretical frameworks
  • Information-motivation-behavioral skills model
  • Theory of reasoned action
  • Theory of planned behavior
  • Social-cognitive theory
  • Control theory and operant conditioning.
• The taxonomy was used to code the presence or absence of these change techniques in behavioral interventions.
• Reports should specify and describe techniques and theoretical frameworks used.

Systematic Review of 118 Studies

• Variations in effects were noted from 16% decrease to 70% increase in adherence to guidelines.
• Unclear conceptual underpinnings of audit and feedback may explain variations in results.
• Theoretical framework provides a context to assess interventions and explain outcomes.

Meta-Regression of 85 Interventions

• Only 4% incorporated feedback, performance goals, and an action plan.
• Variations in findings differed substantially due to methods of delivery or contextual differences.
• Little detail was available about how to implement feedback.
• It is recognized a theoretical approach should be used to design feedback.

Characteristics of Successful Feedback Mechanisms

• Hysong and colleagues examined six Veterans Affairs medical centers
• The components that were common in high performing settings are
  • Timeliness
  • Individualization
  • Lack of punitiveness
  • Customizability
• These characteristics share features with FIT theory.

Hysong Re-examination (2009)

• Hysong updated a Cochrane Systematic Review to determine if published research applied the components of Kluger and DeNisi in the FIT model.
• Of 519 studies, 19 included characteristics theorized to be important.
• Overall, feedback interventions appeared to have a statistically significant but moderate positive impact on outcomes.

Performance Feedback Interventions

• Using keywords like ‘performance feedback' and 'audit and feedback' for research published in English between January 2009 and October 2010, nine studies used multiple interventions
• Those interventions used traditional educational programs, decision support tools, simulations, and reminders.
• Therefore, studies were not assessed, resulting in 13 eligible studies.

Four Studies (College Students)

• Three were efforts to reduce heavy alcohol consumption
• The fourth to reduce marijuana use.
• No effect was reported for a single web-based personalized feedback intervention to reduce marijuana use.
• A modest effect on alcohol use was reported for a targeted intervention delivered electronically on a single occasion to 263 college athletes
• Single vs. biannual feedback compared in 818 first year college heavy drinkers
  • Gender-specific feedback was associated with significantly fewer drinking problems as opposed to the controls
  • A modest effect on biannual versus annual feedback for women but not for men
• Single face-to-face or computerized feedback intervention in 84 heavy drinking students
  • Both interventions were more successful than control.

Six Studies (Clinician Behaviors)

• Five reported positive results.
• Three used feedback to improve antibiotic prescribing.
• Naughton, provided GPs written feedback and a second group received the same feedback with academic detailing
  • The feedback was associated with improved prescribing
  • The academic detailing added no further improvement
• A cross-over interventional study over two 6-month time periods on two medical units
  • The clinical records of patients receiving antibiotics were reviewed with an infectious disease specialist who provided specific feedback to the charge physician
  • Their reports included an assessment of appropriateness and recommendations for modification
  • The duration of antibiotic therapy was meaningfully shorter during the intervention period
• Nair and colleagues tested several different feedback mechanisms
  • Email regarding missed documentation, monthly report summaries, and real-time electronic decision support alerts (which is not a feedback intervention)
  • Only the decision support alerts were associated with improved antibiotic prescribing

The Impact of Feedback (Other Areas)

• Physicians in four Israeli primary care clinics received group feedback regarding their performance compared to the other clinics
  • The clinics were randomized to have diabetic patients and received a letter and phone call asking them to remind their physician at the next clinic visit to address relevant topics regarding quality diabetic care
  • Providing feedback to physicians directly and through patients improved quality diabetes care when compared to feedback to physicians alone
• Assessment to determine the impact of performance feedback on prescribing antipsychotic medication to outpatients with schizophrenia
  • Verbal and written aggregated feedback provided only once to one service and at four time intervals to two other services
  • Improvement in the groups receiving additional feedback
• A group of 60 young adults assessed to be maladaptive perfectionists were randomized to receive a single-session feedback intervention
  • The individual assessment was discussed
  • Their input and were solicited
  • Overall, there was a statistically significant effect of the treatment condition on the relationship between perfectionism and emotional reactivity

None of Reported Studies Employing Feedback Interventions

• Included all components of the actionable feedback model.
• Several provided feedback only single time or at infrequent intervals
• Some were individualized while others presented as aggregated reports
• None were designed to be customizable.
• Because the exact wording of the interventions was not published it was not possible to confirm that all feedback was non-punitive.

Another Characteristic of Described Feedback Interventions

• Recipient of the feedback had minimal input into its content or delivery modality
• The recipient was passive in the process.

Enhancement of Relevance and Importance

• Enhanced by more active involvement of recipients in the development of the feedback intervention before it is even initiated.

Enhanced Credibility

• By assuring the information is meaningful it also assures the that communication medium is convenient, easy to understand, and readily accessible to recipients.

Interventions Reported to Date

• Continue to be relatively atheoretical
• It explains the variations in approaches and results
• It creates major challenges for research who attempt to replicate the given intervention.
• There is emerging but still weak evidence that theory-driven interventions in this field are likely to be more effective
• Theory is to inform generalizability and a better understanding on why inventions might work in one context but not another.

If Feedback Processes are to be Used Successfully as Educational Strategies

• Standardization and clarity is needed in the specific action steps taken to apply behavioral theory to practice.
• Educators should develop and test audit and feedback interventions to improve clinical practice