Performance Diagnostic Checklist - Human Services Guidance PDF

Behavior Analysis in Practice (2022) 15:951–957 https://doi.org/10.1007/s40617-021-00675-4 DISCUSSION AND REVIEW PAPER The Performance Diagnostic Checklist - Human Services: Guidance for Assessment Administration Denys Brand1 · Tyra P. Sellers2 · David A. Wilder3 · James E. C...

Behavior Analysis in Practice (2022) 15:951–957 https://doi.org/10.1007/s40617-021-00675-4 DISCUSSION AND REVIEW PAPER The Performance Diagnostic Checklist - Human Services: Guidance for Assessment Administration Denys Brand1 · Tyra P. Sellers2 · David A. Wilder3 · James E. Carr2 Accepted: 17 November 2021 / Published online: 4 February 2022 © Association for Behavior Analysis International 2022 Abstract The Performance Diagnostic Checklist - Human Services (PDC-HS) is an assessment designed to assess the environmental variables contributing to employee performance concerns in human-service settings. Recent research has demonstrated that interventions indicated by the PDC-HS result in improved employee performance across several human-service settings and that the assessment has acceptable reliability and validity. Although PDC-HS-indicated interventions have been effective at increasing employee performance, there is a need for additional guidance when using the assessment given the limited nature of the original administration guidelines. Thus, the purpose of the current manuscript is to introduce additional guidance for use of the PDC-HS across a variety of situations. Key words functional assessment · human services · performance analysis · performance diagnostic checklist - human services · performance management · staff evaluation Performance analysis is the organizational equivalent of the employee performance concerns in human-service settings functional assessment of problem behavior that is often con- include poor attendance, inadequate data collection, and ducted in clinical and educational settings (Ditzian et al., failure to implement treatment protocols with sufficient 2015; Wilder et al., 2020). The goal of performance analy- integrity, among others. Given the importance of services sis is to identify environmental variables contributing to delivered in human-service settings, it is paramount that employee performance concerns and subsequently develop employee performance concerns are addressed efficiently assessment-based interventions to address them (Austin, and effectively. In addition, given the uniqueness of human- 2000). The most common performance analysis instrument service settings, adapting the PDC to those environments in the behavior-analytic literature in recent decades is the was warranted. The PDC-HS includes 20 questions, 13 of Performance Diagnostic Checklist (PDC; Austin, 2000; which are informant-based, and 7 that require direct observa- Wilder et al., 2018). The PDC is an informant-based method tion. The questions are organized into four domains: Train- for conducting a performance analysis and was designed ing; Task Clarification and Prompting; Resources, Materi- specifically for use in business and industry (Austin, 2000). als, and Processes; and Performance Consequences, Effort, In 2013, the PDC was adapted for use in human-ser- and Competition. Interventions based on results from the vice settings (e.g., clinics, residential treatment facilities, PDC-HS have been referred to as indicated interventions, schools), resulting in the Performance Diagnostic Checklist and those that are not have been referred to as nonindicated – Human Services (PDC-HS; Carr et al., 2013). Common interventions; the former should be prioritized to maximize treatment efficiency. * Denys Brand Since its publication in 2013, the PDC-HS has been [email protected] used in several empirical studies. A recent review by Wilder et al. (2020) showed that interventions indicated by 1 Department of Psychology, California State University, 6000 the PDC-HS have been successfully implemented across J Street, Sacramento, CA 95819‑6007, USA a variety of human-service settings to improve employee 2 Behavior Analyst Certification Board, Sacramento, CA, USA performance, suggesting that the assessment is a valid 3 School of Behavior Analysis, Florida Institute of Technology, and reliable tool for diagnosing the potential causes of and the Scott Center for Autism Treatment, Melbourne, FL, performance concerns (see Wilder et al., 2020 for further USA Vol.:(0123456789) 952 Behavior Analysis in Practice (2022) 15:951–957 discussion of the reliability and validity of the PDC-HS). Completing the PDC‑HS Although PDC-HS-indicated interventions have been effective at improving employee performance, the origi- The PDC-HS (Carr et al., 2013) was designed to be imple- nal administration guidelines from Carr et al. (2013) are mented in an interview format, except for the items that minimal. The original guidance appears sufficient when require direct observation. That is, the consultant or manager PDC-HS users operate under ideal or near-ideal condi- interviews the direct supervisor of the employee exhibiting tions (Wilder et al., 2019). However, PDC-HS users do the performance concern and the direct supervisor answers not always work under such conditions and may find PDC-HS questions about the employee’s performance con- themselves unsure of how to proceed when conducting cern. However, no additional guidance was provided on the assessment. For example, in real-world consultations, how to complete the assessment. Despite the authors’ inten- PDC-HS questions are often followed with additional tion that the assessment be completed by behavior analysts, clarifying questions, and the answers to these follow-up researchers have demonstrated that supervisors without questions may affect scoring and intervention selection training in behavior analysis can successfully use the PDC- (Cymbal et al., 2020). Based on our collective experience HS (e.g., Bowe & Sellers, 2018; Smith & Wilder, 2018). with the PDC-HS, it is our assertion that users may benefit In a recent review of the PDC-HS literature, Wilder et al. from additional guidance and instructions when using the (2020) proposed that it might be valuable to explore having assessment. Thus, the purpose of the present manuscript employees complete the assessment tool themselves. Given is to provide updated guidance for administering the PDC- these varied applications and recommendations, supervisors HS. We make several recommendations and suggestions and those responsible for training supervisors may benefit for how to most effectively implement the PDC-HS based from some guidance related to when and how to implement on available data, as well as our collective experience con- the PDC-HS. ducting related research, and training others to use the assessment. To that end, we discuss some considerations When to Use the PDC‑HS with respect to completing the PDC-HS, targeting perfor- mance concerns, and using data collected from PDC-HS The PDC (designed for use in business and industry) assessments. Table 1 includes a summary of our main rec- and PDC-HS (used to address performance concerns in ommendations and suggestions. human-service settings) were designed to help supervisors identify the environmental variables related to a specific Table 1  Summary of PDC-HS Assessment Guidelines User-Guidelines How to Use the PDC-HS Interview format with direct supervisor (except for direct observations) and ensure accuracy of report (e.g., others confirm, no presence of bias on part of supervisor) Where possible, use permanent product recording (e.g., videos, reports) to verify performance concern Interview target staff directly to gather information of which supervisor may not be aware or to which they may not have access Training Others to Use the PDC-HS Take a Behavioral Skills Training approach Instructions (i.e., Tailor direct instructions to the needs of individual trainees. Provide resources, such as articles) Modeling (i.e., Demonstrate how to administer the PDC-HS) Rehearsal (i.e., Give trainees the opportunity to practice using the assessment) Feedback (i.e., Trainer reviews trainee performance and provides feedback) Describe Performance Concern Accurately describe the performance concern in a way that is measurable and observable Describe performance concern as either a behavioral excess or deficit Conducting Direct Observations Conduct direct observations during typical working conditions to ensure that data are representative of employee behavior Minimize employee reactivity Collect enough data to obtain reliable information Using PDC-HS Data Every NO response is an opportunity for intervention When several NO responses are scored across more than one domain, carefully consider the performance concern, items scored as NO, and available organizational resources when selecting and designing inter- ventions Retain and aggregate data collected from the PDC-HS across employees and performance concerns to help organizations detect the need for possible systems-levels changes Behavior Analysis in Practice (2022) 15:951–957 953 performance concern and subsequently design a matched How to Implement the PDC‑HS intervention (Carr et al., 2013). Newer supervisors may have limited experience addressing staff performance In most instances, the employee’s direct supervisor will concerns in a systematic and planful manner; therefore, complete the PDC-HS (Carr et al., 2013). If the direct super- they might use the PDC-HS each time they encounter an visor is well trained in using the assessment, they may com- employee performance issue until they become proficient plete the steps – answer questions; review data; and iden- at (a) considering and identifying the relevant environ- tify, design, implement, and monitor the intervention – on mental factors that may be impacting performance and their own. In other cases, the employee’s direct supervisor (b) matching an intervention to the identified barriers. may need support from their own supervisor to complete Once proficient with this process, experienced super- the aforementioned steps (e.g., when the direct supervisor is visors may find they follow the process without using new or unfamiliar with the performance concern). In either the PDC-HS for every performance issue. Conversely, case, questions should be answered based on current, reli- consistently using the PDC-HS when addressing all per- able information about the employee. Many of the ques- formance concerns may increase the likelihood of suc- tions (e.g., question 1 in the Training and Task Clarification cessfully assessing and addressing the need early on and and Prompting domains) can be answered based on what reduce the chances that the issue will worsen or resurface the direct supervisor already knows about the employee’s in the future. Thus, for inexperienced supervisors, it may performance. However, it is important to critically evalu- be good practice to consistently implement the PDC-HS ate the degree to which the direct supervisor’s recollection for persistent staff performance concerns. represents a current, accurate, and consistent pattern of the If experienced supervisors cease using the PDC-HS performance concern. for more common staff performance concerns, they may Many supervisors may report that they accurately still consider using the PDC-HS under a few specific cir- remember details regarding an employee’s performance cumstances. First, experienced supervisors may wish to concern. Research in organizational behavior management, periodically complete the PDC-HS to evaluate their own and applied behavior analysis more broadly, has provided behavior when addressing staff performance concerns to some reason for skepticism about this claim. For example, detect any drift from carefully assessing potential envi- Iwata et al. (2013) found that informant methods of assess- ronmental barriers in a systematic manner (e.g., Godat ment of problem behavior may be accurate in only about & Brigham, 1999). Second, it may be helpful to use the 64% of cases. Researchers have also assessed the extent to PDC-HS if a supervisor did not initially use it, but the which supervisors can identify employee’s most preferred performance issue was unresponsive to typical strategies items, which requires them to remember details about for improving performance. In such cases, the supervisor employee behavior. Wilder et al. (2007) found that super- may have overlooked a critical contributing factor requir- visors are not very good at identifying preferred items for ing adjustments to the intervention to directly address the their employees. Wilder et al. (2007) had supervisors in function of the performance issue. Third, if relatively new a variety of industries list some of their employees’ most supervisors rarely encounter employee performance con- preferred items, and then had those employees rank their cerns, they may consider using the PDC-HS when those preferences. Results showed that supervisors were good issues do arise, as they are likely not fluent in taking a at identifying an employee’s most preferred item, but structured, function-matched approach to assessment and correlations between the manger-identified rank and the intervention. employee-identified rank were low. Wilder et al. (2011) All supervisors should consider using the PDC-HS in replicated these findings with 100 participants. Therefore, instances in which they may be negatively influenced by it is important to ensure that the supervisor’s account is their own perspective, history, or bias (e.g., Bernardin not influenced by a singular recent instance of poor perfor- et al., 2016; Gonsalvez & Freestone, 2007; Kaplan , 2018) mance, an overly sensitive perception of the performance with the target employee, such that they find it difficult concern, some other bias, or just a faulty recollection of to remain objective when assessing the performance con- the performance. This is especially important when the cern. Using the PDC-HS may provide sufficient structure supervisor has limited direct knowledge about the employ- to guide the supervisor through an objective evaluation ee’s performance and is relying on the employee’s direct process that minimizes attributing performance concerns supervisor to provide answers to the assessment ques- to personal characteristics (e.g., laziness, inability to tions. To address these concerns, the supervisor can ask comprehend a concept or learn a skill) and maximizes whether there is a consistent performance concern (versus critically assessing the environmental contributors. one that has only occurred once), and whether others also describe this pattern of behavior as concerning. In some cases, there may be permanent products (e.g., reports, 954 Behavior Analysis in Practice (2022) 15:951–957 work areas, video footage) that the supervisor can review Training Others to use the PDC‑HS to assist in verifying the performance issue (e.g., Ditzian et al., 2015; Merritt et al., 2019). When supervisors are training others to become supervi- Merritt et al. (2019) reported an alternative method of sors, or when they are mentoring new supervisors, they implementation in which the consultant directly interviewed should consider incorporating the PDC-HS into their train- the employees exhibiting the performance concern. These ing. It is recommended that supervisors follow a behav- researchers individually interviewed four school-based ioral skills training (BST) approach in these instances direct-care staff members themselves, as well as their direct (Miltenberger, 2003; Wurtele, 1986). Behavioral skills supervisors, using the PDC-HS to identify the variables con- training is an evidence-based teaching procedure consist- tributing to tardiness among the direct-care staff members. ing of instructions, modeling, rehearsal, and feedback Although the results obtained between the direct-care staff (Novak et al., 2019; Parsons et al., 2012). Researchers members and their supervisors were similar, 3 of the 4 par- have demonstrated that BST can be used to train others to ticipants showed some differences. This is not surprising, administer various types of assessment procedures with as responses from employees exhibiting the performance which they had no prior experience (e.g., Barnes et al., concern may be biased. That is, when asked questions about 2014; Shayne & Miltenberger, 2013). Thus, the evidence their performance, employees might not be skilled at self- suggests that BST can be an effective procedure for train- observation and evaluation and, thus, might downplay the ing supervisors to administer the PDC-HS. severity of the concern, or might provide inaccurate informa- When taking this approach, the supervisor-trainer tion to appear more capable. should first identify whether the trainee or mentee is famil- Merritt et al. (2019) deviated from PDC-HS administra- iar with the PDC-HS, and if so, to what degree. The super- tion guidelines because they were concerned that the direct- visor-trainer can then tailor the explicit instruction to the care staff members’ supervisors were unaware of some of the needs of the individual. Training activities might include variables that may have impacted tardiness, which may be any combination of the following components: providing a common occurrence. One possibility for future research articles to read, reviewing the PDC-HS in depth, discuss- might be to interview both parties simultaneously or inter- ing prior experiences using the PDC-HS, role-playing, view employees after interviewing supervisors, such that and arranging opportunities to collaboratively implement discrepancies between the two could be addressed in the the PDC-HS using actual employee performance con- interview with employees. Future research should formally cerns or case scenarios. Training activities could include evaluate PDC-HS outcomes obtained from employees exhib- the trainee or mentee observing while the supervisor- iting performance concerns relative to their supervisors. trainer completes the PDC-HS themselves or interviews the employee’s direct supervisor. It may also involve the trainee or mentee completing the PDC-HS along with the Conducting Direct Observations supervisor-trainer, and then completing it on their own and reviewing the results with the supervisor-trainer, who Three of the PDC-HS domains include items requiring may have also independently completed the assessment. direct observation of employee behavior. When conducting direct observations, supervisors should do so during typical working conditions as it is important to obtain representative samples of employee performance (Sharp et al., 2015). For Considerations for Targeting Performance example, conducting direct observations on a day in which Concerns the employee is feeling ill, has to complete new or unfa- miliar tasks, or is working a shift outside of their typical Describing the Performance Concern schedule, may not produce data that are truly representative of their performance. Thus, conducting direct observations One of the first steps when completing the PDC-HS is during times and under conditions that the employee is typi- to describe the performance concern. The original guide- cally expected to perform the task is recommended. Super- lines indicate that performance concerns should be opera- visors should be aware of employee reactivity; employees tionalized as either a behavioral excess or deficit (Carr may behave differently when they know that they are being et al., 2013). When describing employee concerns, it is observed (Kazdin, 1979; Mowery et al., 2010). Moreover, important to carefully pinpoint (i.e., accurately describ- direct observations of the employee should continue until ing behavior in a way that is measurable and observ- enough data have been collected to obtain reliable informa- able) the behavior of interest (Daniels & Bailey, 2014; tion. It is likely that the minimum number of observations DiGennaro Reed et al., 2018; Rodriguez et al., 2016). needed varies across employees and performance concerns. Behavior Analysis in Practice (2022) 15:951–957 955 Using well-crafted behavioral pinpoints is important to avoiding the use of overly technical jargon. Perhaps the skill reduce the likelihood of any misunderstandings between was explained, but not directly modeled and practiced. Or managers and consultants when discussing performance maybe the skill was practiced, but the employee was never concerns, which could lead to unreliable answers when directly observed interacting with caregivers. Some ques- completing the assessment (Wilder et al., 2019). Unreli- tions, such as those in Task Clarification and Prompting and able answers are especially concerning given that interven- Resources, Materials, and Processes domains may initially tions are selected based on the information obtained from appear irrelevant to our example performance concern. the assessment (Carr et al., 2013). Clear descriptions of However, careful consideration of the how the questions the performance concern can also help the PDC-HS user might reveal information about the relationship between the to frame questions in a way that helps them ask about the performance concern and related environmental barriers is pinpointed behavior more clearly (e.g., “have you received warranted. For example, Question 3 in the Task Clarification training on how to properly close the door following the and Prompting domain of the PDC-HS asks about the pres- completion of your shift” versus “have you received train- ence of a job-aid, which may appear irrelevant to the per- ing on the task”). formance concern in the example. It may not be so strange to think that performing a complex skill like explaining Complexity of the Performance Concern behavior-analytic interventions in everyday language could be supported by a guide that includes some examples of The PDC-HS is well suited for discrete performance con- common translations. Whereas, such a support is not techni- cerns of varying complexity that can be easily observed and cally a job-aid, reflecting on this specific question could lead measured. As noted in the review by Wilder et al. (2020), the thoughtful supervisor to create such a support and assess researchers have applied the PDC-HS to tasks related to whether it helps alleviate the performance concern. Simi- managing the environment (e.g., cleaning and securing larly, careful consideration of question 1 in the Resource, doors), implementing instructional strategies (e.g., error cor- Materials, and Processes domain (Are there sufficient num- rection and discrete-trial instruction procedures, providing bers of trained staff available in the organization to complete response opportunities), and other job-related duties (e.g., the task?) might reveal that the employee’s colleagues also pricing items, arriving on time). However, it may be the case share the same performance concern, such that speaking in that the PDC-HS could be applied to more nuanced skills, very technical terms with caregivers has become common- such as issues related to interpersonal communication (e.g., place. Therefore, application of the PDC-HS to performance being overly argumentative, failing to demonstrate good concerns that are more nuanced should not result in skip- audience control by shifting verbal behavior, or demonstrat- ping questions that may at first appear irrelevant. Instead, ing low affect), problem solving, organizational and time those questions should prompt the supervisor to engage in a management skills, or self-care (e.g., identifying stressors, thoughtful consideration of related possible environmental engaging in strategies to manage stress and maintain health). barriers relevant to the issue at hand. These types of skills may not have discrete beginnings and ends, are not performed in the same way each time, and often require in-the-moment adjustments to responding Using PDC‑HS Data based on contextual information (e.g., the way a commu- nication partner responds, the presence of risk of harm to a There are several ways the data collected from the PDC-HS client, the emergence of an acute life stressor). can be used. The most common application is to identify and Completing the questions for the skills described above implement performance improvement plans for individual may necessitate flexibility on the part of the supervisor, employees or a small group of employees. In this typical requiring them to thoughtfully consider how the questions approach, each item scored as a No on the assessment is an might support identifying related barriers or creative sup- opportunity for intervention. When several items (within or ports. For example, consider an employee who consistently across domains) have been scored as No, multiple interven- fails to use language that is easily understood by caregivers. tions can be implemented either concurrently or consecutively In such an application the supervisor will need to be flex- based on the resources that are available within an organization ible when answering the questions, but in doing so might (Carr et al., 2013). This approach has proven to be effective identify the relevant environmental variables that allow for and has resulted in improved employee performance across a a successful intervention. In our example, answering the variety of settings (Wilder et al., 2020). However, there may first question of the PDC-HS (Has the employee received be instances in which a supervisor uses the PDC-HS to derive formal training on this task?) might reveal that the employee an intervention that does not produce the desired result. For has never had high-quality instruction on how to develop a example, in two recent studies (Collier-Meek et al., 2021; Mer- positive therapeutic relationship with caregivers, including ritt et al., 2019), some participants were not responsive to the 956 Behavior Analysis in Practice (2022) 15:951–957 PDC-HS indicated interventions, requiring the researchers to effective supervisory practices within the applied behavior modify some of their procedures or interventions. analysis profession become even more pronounced when Selecting interventions when several items across multi- considering the continued large increases in the number of ple domains have been scored as No can present some dif- certified behavior-analytic practitioners (Behavior Analyst ficulties. Previously, it was recommended that priority be Certification Board, n.d.). Thus, developing and refining per- given to domains with multiple No responses (Carr et al., formance analysis methods designed to assist supervisors 2013). However, selecting interventions may not always with quickly and effectively addressing employee perfor- be that simple. Consider an example in which a supervi- mance concerns represent an important line of investigation. sor administers the PDC-HS and scores that the materials For the most part, the PDC-HS appears to be relatively sim- needed to complete the task are not readily available (ques- ple and straightforward. However, users of the PDC-HS do tion 1 in Resources, Materials, and Processes) but does not not always operate under ideal conditions and may at times mark any other items in that domain as problematic. In that find themselves unsure of how to proceed when conducting case it may appear that this domain is acceptable, whereas the assessment. We provided a discussion of several issues the reality may be that the employee cannot complete the involving the use of the PDC-HS with the intent of clarify- task without having easy access to materials. In such a case, ing areas of concern. In addition, some recommendations, an intervention would be required for that particular item particularly those related to nontraditional implementation to promote improved employee performance even if items methods of the PDC-HS, warrant investigation by research- within that domain had a smaller number of No responses ers. We hope that users of the PDC-HS find our guidance relative to other domains. Once the intervention has been and recommendations useful. Moreover, we hope that our implemented (i.e., resources needed to complete the task discussion points will stimulate more research into these have been made available) the supervisor may consider reas- areas. sessing employee performance to determine whether further action is required. Thus, we recommend that the perfor- mance concern itself, items scored as No on the PDC-HS Funding No funding was received to assist with the preparation of this manuscript. assessment, and available organizational resources be care- fully considered when selecting interventions. Disclosures The data collected from the PDC-HS can also be retained and aggregated across several employees and performance Conflict of interest The authors declare that they have no conflicts of concerns (Wilder et al., 2020). As the use of the PDC-HS interest. becomes more common within an organization, certain pat- terns within the data may begin to emerge. For example, if the Training domain is repeatedly identified as problem- atic across several employees and tasks, it may be that the References organization needs to evaluate their training procedures Austin, J. (2000). Performance analysis and performance diagnostics. and develop treatment integrity checks to ensure that high- In J. Austin & J. E. Carr (Eds.), Handbook of applied behavior quality training is maintained once provided. Thus, the data analysis (pp. 321–349). Context Press. collected across all assessments conducted within an organ- Barnes, C. S., Mellor, J. R., & Rehfeldt. (2014). Implementing the ization could indicate the need for systems-level changes verbal behavior milestones and placement program (VB-MAPP): Teaching assessment techniques. Analysis of Verbal Behavior, (Diener et al., 2009), which may assist organizations in tak- 30(1), 36–47. https://​doi.​org/​10.​1007/​s40616-​013-​0004-5 ing a more proactive approach when addressing employee Behavior Analyst Certification Board. (n.d.). BACB certificant data. performance concerns rather than always having to react https://​www.​bacb.​com/​bacb-​certi​ficant-​data/ every time an issue arises. We suggest that supervisors retain Bernardin, H. J., Thomason, S., Buckley, M. R., & Kane, J. S. (2016). Rater rating-level bias and accuracy in performance appraisals: and aggregate the data every time the assessment is con- The impact of factor rater personality, performance management ducted and update their records as new data are collected. competence, and rater accountability. Human Resource Manage- Future studies could investigate the utility of using PDC-HS ment, 55(2), 321–340. https://​doi.​org/​10.​1002/​hrm.​21678 data to inform systems-level changes. Bowe, M., & Sellers, T. P. (2018). Evaluating the performance diagnos- tic checklist - human services to assess incorrect error-correction procedures by preschool paraprofessionals. Journal of Applied Behavior Analysis, 51(1), 166–176. https://​doi.​org/​10.​1002/​jaba.​ Concluding Remarks 428 Carr, J. E., Wilder, D. A., Majdalany, L., Mathison, D., & Strain, L. A. (2013). An assessment-based solution to a human-service As the demand for effective behavior-analytic services in employee problem: An initial evaluation of the performance diag- human service settings continues to increase, so does the nostic checklist - human services. Behavior Analysis in Practice, need for effective supervision. The issues surrounding 6(1), 16–32. https://​doi.​org/​10.​1007/​BF033​91789 Behavior Analysis in Practice (2022) 15:951–957 957 Collier-Meek, M. A., Sanetti, L. M. H., Gould, K., & Pereira, B. A. Novak, M. D., DiGennaro Reed, F. D., Erath, T. G., Blackman, A. L., (2021). Using the performance diagnostic checklist to evaluate Ruby, S., & Pellegrino, A. J. (2019). Evidence-based performance and promote paraeducators’ treatment fidelity. Journal of School management: Applying behavioral science to support practition- Psychology, 86, 1–14. https://​doi.​org/​10.​1016/j.​jsp.​2021.​02.​005 ers. Perspectives on Behavior Science, 42(9), 955–972. https://​ Cymbal, D. C., Wilder, D. A., Thomas, R., & Ertel, H. (2020). Fur- doi.​org/​10.​1007/​s40614-​019-​00232-z ther evaluation of the validity and reliability of the Performance Parsons, M. B., Rollyson, J. H., & Reid, D. H. (2012). Evidence-based Diagnostic Checklist-Human Services. Journal of Organizational staff training: A guide for practitioners. Behavior Analysis in Behavior Management, 40(3-4), 249–257. https://d​ oi.o​ rg/1​ 0.1​ 080/​ Practice, 5(2), 2–11. https://​doi.​org/​10.​1007/​BF033​91819 01608​061.​2020.​17920​27 Rodriguez, M., Sundberg, D., & Biagi, S. (2016). OBM applied: A Daniels, A. C., & Bailey, J. S. (2014). Performance management: practical guide to implementing organizational behavior manage- Changing behavior that drives organizational effectiveness ((5th ment. ABA Technologies, Inc. ed.). ed.). Aubrey Daniels International, Inc.. Sharp, R. A., Mudford, O. C., & Elliffe, D. (2015). Representativeness Diener, L. H., McGee, H. M., & Miguel, C. F. (2009). An integrated of direct observations selected using a work-sampling equation. approach for conducting a behavioral systems analysis. Journal Journal of Applied Behavior Analysis, 48(1), 153–166. https://​ of Organizational Behavior Management, 29(2), 108–135. https://​ doi.​org/​10.​1002/​jaba.​193 doi.​org/​10.​1080/​01608​06090​28745​34 Shayne, R., & Miltenberger, R. G. (2013). Evaluation of behavioral DiGennaro Reed, F. D., Novak, M. D., Erath, T. G., Brand, D., & Hen- skills training for teaching functional assessment and treatment ley, A. J. (2018). Pinpointing and measuring employee behavior. selection skills to parents. Behavioral Interventions, 28(1), 4–21. In B. Wine & J. K. Pritchard (Eds.), Organizational behavior https://​doi.​org/​10.​1002/​bin.​1350 management: The essentials. Hedgehog Publishers. Smith, M., & Wilder, D. A. (2018). The use of the performance diag- Ditzian, K., Wilder, D. A., King, A., & Tanz, J. (2015). An evalua- nostic checklist - human services to assess and improve the job tion of the performance diagnostic checklist - human services to performance of individuals with intellectual disabilities. Behav- assess an employee performance problem in a center-based autism ior Analysis in Practice, 11(2), 148–153. https://​doi.​org/​10.​1007/​ treatment facility. Journal of Applied Behavior Analysis, 48(1), s40617-​018-​0213-4 199–203. https://​doi.​org/​10.​1002/​jaba.​171 Wilder, D. A., Rost, K., & McMahon, M. (2007). The accuracy of Godat, L. M., & Brigham, T. A. (1999). The effect of a self-manage- managerial prediction of employee preference. Journal of Organi- ment training program on employees of a mid-sized organization. zational Behavior Management, 27(2), 1-14. https://​doi.​org/​10.​ Journal of Organizational Behavior Management, 19(1), 65–83. 1300/​J075v​27n02_​01 https://​doi.​org/​10.​1300/​J075v​19n01_​06 Wilder, D. A., Harris, C., Casella, S., Wine, B., & Postma, N. (2011). Gonsalvez, C. J., & Freestone, J. (2007). Field supervisors’ assessments Further evaluation of the accuracy of managerial prediction of of trainee performance: Are they reliable and valid. Australian employee preference. Journal of Organizational Behavior Man- Psychologist, 42(1), 23–32. https://​doi.​org/​10.​1080/​00050​06060​ agement, 31(2), 130–139. https://d​ oi.o​ rg/1​ 0.1​ 080/0​ 16080​ 61.2​ 011.​ 08276​15 569202 Iwata, B. A., DeLeon, I. G., & Roscoe, E. M. (2013). Reliability and Wilder, D. A., Lipschultz, J., Gehrman, C., Ertel, H., & Hodges, A. validity of the functional analysis screening tool. Journal of (2019). A preliminary assessment of the validity and reliability of Applied Behavior Analysis, 46(1), 271–284. https://​doi.​org/​10.​ the performance diagnostic checklist - human services. Journal of 1002/​jaba.​31 Organizational Behavior Management, 39(3-4), 194–212. https://​ Kaplan, S. E. (2018). Further evidence on the negativity bias in perfor- doi.​org/​10.​1080/​01608​061.​2019.​16667​72 mance evaluation: When does the evaluator’s perspective matter? Wilder, D. A., Cymbal, D., & Villacorta, J. (2020). The performance Journal of Management Accounting Research, 30(1), 169–184. diagnostic checklist - human services: A brief review. Journal of https://​doi.​org/​10.​2308/​jmar-​51698 Applied Behavior Analysis, 53(2), 1170–1176. https://​doi.​org/​10.​ Kazdin. (1979). Unobtrusive measures in behavioral assessment. Jour- 1002/​jaba.​676 nal of Applied Behavior Analysis, 12(4), 713–724. https://d​ oi.o​ rg/​ Wilder, D., Lipschultz, J., King, A., Driscoll, S., & Sigurdsson, S. 10.​1901/​jaba.​1979.​12-​713 (2018). An analysis of the commonality and type of preinter- Merritt, T. A., DiGennaro Reed, F. D., & Martinez, C. E. (2019). Using vention assessment procedures in the Journal of Organizational the performance diagnostic checklist - human services to identify Behavior Management (2000–2015). Journal of Organizational an indicated intervention to decrease employee tardiness. Journal Behavior Management, 38(1), 5–17. https://​doi.​org/​10.​1080/​ of Applied Behavior Analysis, 52(4), 1034–1048. https://​doi.​org/​ 01608​061.​2017.​13258​22 10.​1002/​jaba.​643 Wurtele, S. K. (1986). Teaching young children personal body safety: Miltenberger, R. G. (2003). Behavior modification: Principles and The Behavioral Skills Training Program. Author procedures (3rd ed.).. Mowery, J. M., Miltenberger, R. G., & Weil, T. M. (2010). Evaluating Publisher’s Note Springer Nature remains neutral with regard to the effects of reactivity to supervisor presence on staff response jurisdictional claims in published maps and institutional affiliations. to tactile prompts and self-monitoring in a group home setting. Behavioral Interventions, 25(1), 21–35. https://​doi.​org/​10.​1002/​ bin.​296