What are the different approaches used in implementation fidelity assessments

Implementation Fidelity

• Implementation fidelity is the degree to which an intervention is delivered as it was designed or written.
• It encompasses five key elements: adherence, dosage or exposure, quality of delivery, participant responsiveness, and program differentiation.

Adherence

• Adherence refers to whether a program or intervention is being delivered as it was designed or written.
• It involves the delivery of all elements of the intervention as prescribed by its designers.

Dosage or Exposure

• Dosage or exposure refers to the amount of an intervention received by participants.
• It involves the frequency and duration of the intervention, and whether it is as full as prescribed by its designers.

Quality of Delivery

• Quality of delivery is the manner in which a program is delivered by a teacher, volunteer, or staff member.
• It can be evaluated using a benchmark, either within or beyond that stipulated by an intervention's designers.
• Quality of delivery may be viewed as a discrete aspect of fidelity or as a moderator of the relationship between an intervention and its fidelity.

Participant Responsiveness

• Participant responsiveness measures how far participants respond to, or are engaged by, an intervention.
• It involves judgments by participants or recipients about the outcomes and relevance of an intervention.

Program Differentiation

• Program differentiation is the identification of unique features of different components or programs.
• It involves identifying which elements of programs are essential for their intended effect.
• Program differentiation is more accurately described as the "Identification of an intervention's essential components".
• This element is distinct from fidelity and is concerned with determining the essential components of an intervention.

Measurement of Implementation Fidelity

• There are two distinct views on how to measure implementation fidelity:
  • One view is that each of the five elements represents an alternative way to measure fidelity.
  • The other view is that all five elements need to be measured to capture a comprehensive picture of implementation fidelity.

Implementation fidelity refers to the degree to which an intervention or programme is delivered as intended. Only by understanding and measuring whether an intervention has been implemented with fidelity can researchers and practitioners gain a better understanding of how and why an intervention works, and the extent to which outcomes can be improved.

Within this conceptualisation of implementation fidelity, adherence is defined as whether "a program service or intervention is being delivered as it was designed or written" [4]. Dosage or exposure refers to the amount of an intervention received by participants; in other words, whether the frequency and duration of the intervention is as full as prescribed by its designers [1, 4]. For example, it may be that not all elements of the intervention are delivered, or are delivered less often than required. Coverage may also be included under this element, i.e., whether all the people who should be participating in or receiving the benefits of an intervention actually do so.

Quality of delivery is defined as "the manner in which a teacher, volunteer, or staff member delivers a program" [4]. However, it is perhaps a more ambiguous element than this suggests. An evaluation of this may require using a benchmark, either within or beyond that stipulated by an intervention's designers; this element of fidelity could involve either delivering the intervention using "techniques . . . prescribed by the program" [4], or applying a benchmark from outside the programme, i.e., "the extent to which a provider approaches a theoretical ideal in terms of delivering program content" [1]. If such a clear benchmark exists then quality of delivery may be treated, along with adherence and dosage, as one of three discrete aspects required to assess the fidelity of an intervention. However, it may potentially also be viewed as a moderator of the relationship between an intervention and the fidelity with which it is implemented. This is a role that is simply not explored in the literature to date. For example, an intervention could be delivered but delivered badly; in turn, the degree of fidelity achieved by the implemented intervention could be adversely affected.

Participant responsiveness measures how far participants respond to, or are engaged by, an intervention. It involves judgments by participants or recipients about the outcomes and relevance of an intervention. In this sense, what is termed "reaction evaluation" in the evaluation literature may be considered an important part of any evaluation of an intervention [16].

Program differentiation, the fifth aspect, is defined as "identifying unique features of different components or programs", and identifying "which elements of . . . programmes are essential", without which the programme will not have its intended effect [1]. Despite being viewed as an element of implementation fidelity by the literature, programme differentiation actually measures something distinct from fidelity. It is concerned with determining those elements that are essential for its success. This exercise is an important part of any evaluation of new interventions. It enables discovery of those elements that make a difference to outcomes and whether some elements are redundant. Such so-called "essential" elements may be discovered either by canvassing the designers of the intervention or, preferably, by "component analysis", assessing the effect of the intervention on outcomes and determining which components have the most impact [17]. This element would therefore be more usefully described as the "Identification of an intervention's essential components". This process may also have implications for implementation fidelity; if, for example, these essential components are the most difficult to implement, then this may then explain a lack of success afflicting the intervention.

Despite agreeing that implementation fidelity involves measurement of these five elements, the review literature offers two distinct views on how this should be done. On the one hand, it is argued that each of these five elements represents an alternative way to measure fidelity, i.e., implementation fidelity can be measured using either adherence or dosage or quality of delivery etc [4, 5]. On the other hand, it is argued that all five elements need to be measured to capture a "comprehensive" or "more complete picture" of the process, i.e., evaluation requires the measurement of adherence, dosage, and quality of delivery, etc [1, 2]. However, relationships between the various elements are far more complex than such conceptualisations allow. This paper therefore advances a new, third conceptual framework for implementation fidelity, which not only proposes the measurement of all of these elements, but unlike all previous attempts to make sense of this concept also clarifies and explains the function of each and their relationship to one another. Two additional elements are also introduced into this new framework: intervention complexity and facilitation strategies. The potential effect of intervention complexity on implementation fidelity was suggested to the authors by literature on implementation more broadly – especially a systematic review that focused on identifying facilitators and barriers to the diffusion of innovations in organisations that found that the complexity of an idea presented a substantial barrier to its adoption [18]. The potential role of facilitation strategies was suggested by research aiming to evaluate the implementation fidelity of specific interventions that put in place strategies to optimise the level of fidelity achieved. Such strategies included the provision of manuals, guidelines, training, monitoring and feedback, capacity building, and incentives [3, 6, 8, 17].