Fryling and Baires (2016) The Practical Importance of the Distinction Between Open and Closed-Ended Indirect Assessments PDF

Behav Analysis Practice (2016) 9:146–151 DOI 10.1007/s40617-016-0115-2 DISCUSSION AND REVIEW PAPER The Practical Importance of the Distinction Between Open and Closed-Ended Indirect Assessments Mitch J. Fryling 1 & Natalia A. Baires 1 Published online: 17 February 2016 # Association for Behav...

Behav Analysis Practice (2016) 9:146–151 DOI 10.1007/s40617-016-0115-2 DISCUSSION AND REVIEW PAPER The Practical Importance of the Distinction Between Open and Closed-Ended Indirect Assessments Mitch J. Fryling 1 & Natalia A. Baires 1 Published online: 17 February 2016 # Association for Behavior Analysis International 2016 Abstract The identification of functional relations is a hall- developed as a primary means of understanding factors that mark of applied behavior analysis. Building upon this foun- influence the occurrence of a particular target behavior within dation, applied behavior analysts have developed and research and practice in applied behavior analysis (ABA). researched a number of practices that fall within the purview Indeed, methods and techniques associated with FBA are of Functional Behavioral Assessment, a framework used to now central to the practice of ABA. This centrality is reflected understand factors that influence a target behavior. Indeed, by content related to FBA being included in academic course there now exists a wide range of procedures that fall within sequences approved by the Behavior Analyst Certification the purview of Functional Behavioral Assessment, with dif- Board ® as well as the task lists for content on the Board ferent procedures being associated with different strengths and Certified Behavior Analyst ® and Board Certified assistant limitations. Indirect assessments are commonly featured in Behavior Analyst ® exams. most descriptions of the Functional Behavioral Assessment A wide variety of practices and tools associated with process. This paper focuses on the distinction between open conducting FBAs have been developed over the years, each and closed-ended indirect assessments specifically, highlight- of which broadly involves obtaining information about factors ing their strengths and limitations. After distinguishing be- that influence the occurrence of a target behavior. For exam- tween these two types of indirect assessments considerations ple, a number of indirect assessments have been developed, for practice are provided. each of which involve obtaining information from an infor- mant about the possible functions of a target behavior (see Keywords Interviews. Functional behavioral assessment. Kelley et al. 2011).1 A number of descriptive techniques have Indirect assessment. Behavioral assessment been developed as well, all of which involve the observation of behavior in the environment in which it naturally occurs The identification of functional relationships is central to the (e.g., Thompson and Borrero 2011). Finally, a range of varia- practice of applied behavior analysis (Baer et al. 1968). In fact, tions of the experimental functional analysis have also devel- the term functional analysis has a long history in behavior oped over the years, each of which involve the systematic analysis (e.g., Skinner 1953). Derived from this foundation, manipulation (or Btest^) of a variable to assess the extent to the Functional Behavioral Assessment (FBA) model has been which the variable is related to the occurrence of the target behavior (e.g., Beavers et al. 2013; Hanley et al. 2003). While much can be said about the FBA process (e.g., Anderson and St. Peter 2013; Hanley 2012), the current paper aims to consider some practical issues related to indirect as- * Mitch J. Fryling [email protected] sessments specifically. Importantly, our aim is not to provide 1 The extent to which verbal reports should even be considered part of 1 Division of Special Education and Counseling, California State Applied Behavior Analysis may be debated (e.g., Baer et al. 1968). The University, Los Angeles, 5151 State University Dr., Los current article focuses on the practical aspects of indirect assessments Angeles, CA 90032, USA rather than conceptual/theoretical issues. Behav Analysis Practice (2016) 9:146–151 147 an exhaustive review of various indirect assessments or of the pertaining to the target behavior in question. Examples of literature on the reliability or validity of indirect assessments, these items include BEngages in the behavior when asked to but rather, to highlight some issues to consider when and if do something (getting dressed, brush teeth, work, etc.)^ and practitioners do choose to use indirect assessments. Indeed, BDoes he/she seem to enjoy the behavior, even if no one is while indirect assessments are not without limitation (e.g., around?^ The rating scale for the QABF includes X as Cooper et al. 2007; Hanley 2012), they remain part of the Bdoesn’t apply^, 0 as Bnever^, 1 as Brarely^, 2 as Bsome^, FBA process as described in many ABA textbooks (e.g., and 3 as Boften^. After all of the items have been rated by Cooper et al.; Miltenberger 2008). The goal of the current the informant, the consultant can then proceed to score the paper is to specifically consider the strengths and limitations assessment. Each item is assigned to a category of potential of two broad categories of indirect assessments; the open and function of behavior, which includes attention, escape, non- closed-ended types. After highlighting the distinguishing fea- social, physical, and tangible. A sum for all items that corre- tures of each of these methods, considerations for practice are spond to each given category is obtained, with the highest described. scoring category(ies) considered to be potential source(s) of reinforcement for the target behavior. Indirect Assessments FAST The FAST, consisting of a 16-item questionnaire, be- gins by obtaining information on the informant-client relation- As mentioned above, indirect assessments refer to assessment ship, how long the informant has known the client, whether or methods that do not assess behavior directly, but rather, rely not the informant interacts with the client on a daily basis, and on the report of an informant (e.g., a parent, teacher, caregiver, the situations in which the informant interacts with the client. or sometimes the client themselves). Related to this, core texts Thus, this first section of the FAST provides descriptive infor- in ABA often caution that indirect assessments are subjective, mation about the perspective of the informant. The remaining and therefore that behavior analysts should always remember items on the FAST pertain to the possible functions them- that behavioral function cannot be inferred from such assess- selves, with items being answered with either a Byes^, Bno^, ments (e.g., Cooper et al. 2007, p. 510). Still, indirect assess- or BN/A^. Upon its completion, a scoring summary is yielded ments are considered a core part of the common FBA process where items are categorized by a potential source of reinforce- that proceeds from the least to most intrusive assessment ment that includes attention/preferred items, escape, sensory method, and several indirect assessments have been devel- stimulation, and pain attenuation. As with other closed-ended oped over the years (see Kelley et al. 2011). Importantly, while indirect assessments, the category(ies) with the most items all types of indirect assessments are similar in that they rely on circled Byes^ is/are hypothesized as the function/potential information from an informant, they are not all the same. One source(s) of reinforcement for the target behavior. of the core differences between the various types of indirect assessments pertains to the distinction between open and Strengths When considering the strengths of closed-ended closed-ended indirect assessments. indirect assessments, a number of benefits seem apparent. First, closed-ended indirect assessments are quick and easy Closed-Ended Indirect Assessments to administer, producing information about potential sources of reinforcement in an efficient manner. Moreover, the assess- Closed-ended indirect assessments are considered closed- ments are scored relatively easily, which means that hypothe- ended because they do not permit the informant to say just ses about possible functional relationships are easily made. anything; possible answers are prescribed a priori. In other Finally, the administration and interpretation of closed-ended words, the informant is given pre-specified options for an- indirect assessments does not require extensive training. This swers which they must choose from. A number of these in- means that closed-ended indirect assessments may be used by struments have been developed over the years, such as the many, even those without specific training in ABA. Motivation Assessment Scale (Durand and Crimmins 1992) A particular strength of closed-ended indirect assessments and the Problem Behavior Questionnaire (Lewis et al. 1994). is that they guarantee attention to behavior functions that have For the purposes of illustration, we provide a brief overview of often been associated with challenging behavior. Response two closed-ended indirect assessments; Questions About options are based on sources of reinforcement that are com- Behavioral Function (QABF; Matson and Vollmer 1995) monly identified in experimental/analog assessments (e.g., and the Functional Analysis Screening Tool (FAST; Iwata Iwata et al. 1982, 1994). This means that the opportunity for and DeLeon 1996). mentalistic and otherwise irrelevant information is removed (i.e., information that is derived from a dualistic worldview). QABF Informants who complete the QABF respond to a For example, when conducting an assessment of a child’s Likert-type rating scale and are asked to rate 25 items aggressive behavior a closed-ended assessment may prevent 148 Behav Analysis Practice (2016) 9:146–151 distracting conversation related to hypothetical inner prob- someone without adequate training in FBA may quickly ad- lems (e.g., Binternalized anger issues^), personalities (e.g., minister and score an indirect assessment, and, assume a par- Bthey have always been a very angry person^), psychiatric ticular function (despite the fact that some of these instruments explanations (e.g., Boppositional defiant disorder^), and more caution against this directly on the instrument, e.g., the FAST). (also see Skinner 1953, pp. 24–31). This is certainly not a The mere fact that functions are scored with closed-ended trivial matter, as these different ways of speaking may have indirect assessments may inadvertently encourage this. large implications for intervention planning (e.g., a child with For example, someone may use a closed-ended indirect an assumed anger problem may be asked to attend anger man- assessment in an attempt to understand factors that influence agement programs, or a child with an assumed personality a client’s challenging behavior and this may result in attention problem may be referred to psychological counseling). being scored the highest. Based upon this score, an interven- tion plan may be developed which specifically targets the Limitations Interestingly, many of the strengths of closed- attention function (such scores may also influence subsequent ended indirect assessments are also related to their limitations. assessments in important ways). As just mentioned, however, For example, while closed-ended indirect assessments force attention may or may not actually influence the target behav- an informant to choose from common behavioral factors, ior, despite the high score on the indirect assessment. It is preventing mentalistic ideas from entering the assessment in possible that the challenging behavior is related to negative this way, they may at the same time prevent the identification reinforcement, for example, and that a more detailed skills of other factors that may be functionally related to the target assessment and related intervention is required. In this case, behavior. For example, it may be that the problem behavior while the closed-ended assessment did indeed focus on behav- most often happens after an argument with a primary caregiv- ioral factors, the resulting false-positive was distracting and er, or when the client does not get adequate sleep (Kennedy led to ineffective intervention planning. To be clear, this is not and Meyer 1996), and perhaps this only happens when the necessarily a problem with closed-ended indirect assessments client is staying at her mother’s house, and not when she is themselves, but with their inappropriate use in practice. It is staying with other caregivers. Moreover, perhaps a problem is important to note that these possibilities remain speculative; developing at home because an intervention is now being future researchers should evaluate the manner in which implemented in another setting such as school, consistent with closed-ended indirect assessments are used in practice. behavioral contrast (e.g., Wahler et al. 2004). The practical importance of this issue cannot be overstated, as interventions Open-Ended Indirect Assessments that overlook critical factors may produce poor long-term results. Open-ended indirect assessments are primarily comprised of Related to the first issue is that the only information about questions that do not have a pre-specified set of answers, and, possible sources of reinforcement to be obtained from closed- in this sense, the informant is open to answer in any way. In ended indirect assessments will pertain to the sources of rein- addition, questions that open-ended assessments ask often fo- forcement that were actually assessed. Again, most closed- cus on functions in addition to the commonly assessed sources ended indirect assessments do not include questions about of reinforcement associated with the closed-ended indirect contextual information; they may not ask enough questions assessments. Interestingly, published open-ended indirect as- or probe for important information. An additional issue is that sessments seem to be much less common than closed-ended closed-ended indirect assessments may perhaps be more likely indirect assessments (for a review of indirect assessments, see to identify false-positives (this seems likely to be the case for Kelley et al. 2011). Of course, any interview that involves all closed-ended surveys), implying that a target behavior has open-ended questions about factors that may influence a target a particular function when it actually does not. For example, a behavior might be considered an open-ended indirect assess- closed-ended indirect assessment may be more likely to iden- ment. In what follows we provide a brief overview of two tify attention as a controlling variable when it is in fact not, open-ended indirect assessments, which might also be consid- simply because questions are asked about attention. (See ered structured interviews. First, we review the Functional Iwata et al. 2013 for a recent study on the validity of an Assessment Interview Form (FAIF; O’Neil et al. 1997), indirect assessment.) followed by Hanley’s (2012) Open-Ended Functional Also, as closed-ended indirect assessments may be used by Assessment Interview. many individuals without specific training in ABA, it seems possible that those without specific training may also be the FAIF Consisting of 10 sections, the FAIF is an open-ended most prone to misuse and/or misinterpret results. While it is indirect assessment which begins with inquiring about the true that closed-ended indirect assessments do encourage a target behavior and its topography. It also allows the informant focus on common behavioral factors, they do remain subjec- to provide the structure of the client’s daily schedule, such as tive and may not identify the correct factors. For example, the time, setting, and activity, as well as whether the target Behav Analysis Practice (2016) 9:146–151 149 behavior is likely to occur or not in each of these circum- Strengths There are a number of strengths associated with stances. Proceeding this section is the Functions of Problem open-ended indirect assessments. Perhaps the largest strength Behavior section, where specific questions about the common of open-ended indirect assessments pertains to the range of functions are asked, such as BIs the student usually noncom- information that can be obtained from such assessments. For pliant when asked to perform a task?^ and BDoes the student example, informants are asked to elaborate on various factors engage in this behavior when no one is around or watching?^. that may participate in the context of a target behavior, such as The informant is required to circle the item if they answer activities, preferences, and more. Thus, open-ended indirect Byes^ to the item (indeed, these particular questions are assessments may increase the chances of identifying unique, closed-ended, though may be asked in the context of a discus- context-specific factors that participate in the occurrence of a sion). The remaining four sections of the FAIF focus on re- target behavior (e.g., allergies, friendships, conflicts with par- placement behaviors, communication skills, preferences, and ents, etc.). By contrast, when focusing on the common func- previous interventions. For every target behavior listed, the tions, as in most closed-ended indirect assessments, such fac- informant is encouraged to list behaviors that could serve as tors are overlooked. In addition, open-ended indirect assess- alternatives for each respective target behavior. Indeed, infor- ments provide more of an opportunity for the consultant to mation from this section of the FAIF could prove useful when develop rapport with the informant. That is, when the indirect planning to teach functionally equivalent replacement behav- assessment is more conversational in nature, it seems that iors to the client. Finally, the section on previous interventions there is more opportunity for the consultant to develop a col- asks for descriptions of past procedures, the effectiveness on laborative relationship with the informant. This may be key the target behavior, dates in effect, and data on the target during the remaining phases of the consultation process. behavior, if available. Importantly, the broad range of contex- Importantly, these strengths remain speculative, and research tual information obtained during the FAIF is obtained in the is needed to explore these issues before more definite conclu- context of a discussion/interview, and thus, a wealth of addi- sions can be made. tional information may also be obtained with follow-up questions. Limitations As with closed-ended indirect assessments, the strengths of open-ended indirect assessments are related to Open-Ended Functional Assessment Interview Hanley their limitations. First, open-ended indirect assessments are (2012) provides an open-ended indirect assessment comprised likely to take more time to administer. Not only are open- of 20 questions, 16 of which aim to specifically inform the ended assessments more of a discussion, but open-ended in- design of a functional analysis for the target behavior. To direct assessments are more likely to provide the interviewer begin, information pertaining to the client’s age, language with opportunities to probe about additional factors (i.e., more abilities, and highly preferred items/activities is obtained. than the commonly assessed functions), and this also adds to After this, the remaining portion of the interview focuses on the time associated with the administration of open-ended obtaining information that specifically guides the consultant in indirect assessments. In addition, as a larger amount of infor- the development of an individualized functional analysis for mation about the context is obtained from open-ended indirect the client’s target behavior. In particular, questions ask about assessments, it seems likely that more irrelevant information the topography of the target behavior(s), which target behav- could also be obtained. For example, when conducting an iors are most concerning (i.e., setting priority), the intensity of open-ended assessment in a school setting it is possible that the behavior, potential precursors, antecedent events, and con- an informant will provide extensive information about the sequences, among other variables. For example, when child’s family, which may or may not be relevant to under- obtaining information related to setting priority the consultant standing the child’s challenging behavior at school. That is, asks, among other things, BWhat are the top 3 most while more information about the context is obtained during concerning problem behaviors? Are there other behaviors of an open-ended assessment, this is not to say that all of the concern?^. Similarly, when probing for information about information obtained about the context will in fact be relevant possible antecedents the consultant asks, among other ques- or functionally related to the target behavior. Added to infor- tions, BUnder what conditions or situations are the problem mation that is not functionally related to the target behavior is behaviors most likely to occur?^ and BWhat seems to trigger mentalistic or otherwise distracting theories about the target the problem behavior?^. When asking about consequences, behavior (see examples above). All of this underscores the one of the questions the consultant asks is BHow do you and need for behavior analytic skills when interpreting open- others react or respond to the problem behavior?^ As can be ended indirect assessments as different interviewers may in- seen from this brief overview, all of the questions in this in- terpret open-ended assessments in different ways. terview are open-ended and are likely to lead to a series of Lastly, while open-ended indirect assessments do provide follow-up questions within the context of a discussion type the opportunity for developing rapport with the informant, interview. some interviewing skills may be needed on behalf of the 150 Behav Analysis Practice (2016) 9:146–151 consultant to actually facilitate this. That is, conducting an questions, as with the two interviews reviewed in this paper, open-ended indirect assessment in a manner that is likely to can assure that the consultant asks questions related to impor- promote consultant-informant rapport requires some skill and tant factors, and can also set the occasion for further conver- should not be assumed to develop automatically. Interestingly, sation and questioning about specific issues. Given the impor- the topic of behavioral interviewing skills has not been ig- tance of rapport building, and for obtaining rich information nored within the behavior analytic literature (e.g., Iwata et about the context, practitioners are cautioned against al. 1982; Miltenberger and Fuqua 1985). As with the strengths attempting to sidestep this phase and quickly rushing into of open-ended indirect assessments, these limitations should other steps of the assessment process. be considered tentative as research is needed to confirm these It is also important to consider the role of reviewing a possibilities. client’s history in the early phases of an FBA. Indeed, a de- Thus far, we have provided an overview of some of the tailed file review can be a critical step in the FBA process, distinguishing features between open and closed-ended indi- regardless of the form of the assessment. Informants seem rect assessments, including their respective strengths and lim- likely to focus on information that they find to be relevant to itations (see Table 1 for a summary). In the final section of the the behavior in consideration, and thus, even when using an paper, we describe some considerations for the practice open-ended indirect assessment, it is possible that informants of ABA. will omit information that is in fact quite relevant to under- standing the target behavior. Moreover, informants may not just incidentally omit important information but may be un- Practice Considerations aware of it themselves. A detailed review of the client’s his- tory can often provide a wealth of important historical infor- Drawing attention to the distinction between open and closed- mation. In addition, a detailed file review can often lead to ended indirect assessments highlights a number of issues to several follow-up questions from the consultant, which may consider in practice. Specifically, the importance of rapport be key to understanding a particular context. building, gathering a client’s history, and considering the pop- Finally, it is important to consider the population and set- ulation and setting is underscored when thinking about indi- ting when pursuing an FBA. For example, the vast majority of rect assessments and the early phases of the assessment pro- the research on the experimental functional analysis of chal- cess more generally. lenging behavior has been conducted with individuals with Rapport building is key during the early stages of the con- developmental disabilities in specific settings (Beavers et al. sultation process, and it is important for practicing behavior 2013; Hanley et al. 2003). In addition, as highlighted by analysts to consider this as a primary goal of the indirect as- Anderson and St. Peter Pipkin (2013), a large and growing sessment phase. Open-ended indirect assessments seem espe- body of literature on assessment and intervention with typi- cially helpful when considering the development of rapport cally developing children in school-based settings is available with informants and other stakeholders. Structured interviews, and often does not involve an explicit focus on the common such as the FAIF (O’Neil et al. 1997), allow for the consultant functions targeted in the functional analysis literature. and informant to have a conversation about the larger context Depending on the circumstances, it may be the case that the and also give the informant a chance to provide information, practicing behavior analyst needs to consult with another help- possibly in depth, regarding the circumstances surrounding ing professional who has expertise with the population and the target behavior. While consultants are not necessarily re- setting (e.g., consulting with a speech-language pathologist quired to have a pre-set series of questions to ask during an to do an oral motor assessment or a school psychologist to open-ended interview, the use of a pre-specified set of conduct a detailed academic assessment). Importantly, this Table 1 Strengths and limitations of open and closed-ended indirect assessments Closed-ended indirect assessments Open-ended indirect assessments Strengths -Assures information about common controlling variables -Information about a wide range of contextual variables may with some populations. be obtained. -Quick and easy to administer. -Opportunity to develop rapport with the informant and to -Few particular skills are required to administer and interpret. listen to their experiences. Limitations -Only asks questions about pre-determined variables -Takes longer than closed-ended indirect assessments. -May suggest a non-function (i.e., false-positives). -May include information that is not relevant to the function. -May be misused by individuals without training in ABA. -Interpretation requires behavior analytic skills. -Little opportunity to develop rapport with informants. -Requires clinical interviewing skills to obtain information and develop rapport. Behav Analysis Practice (2016) 9:146–151 151 topic also relates to the ethical guidelines provided by the Durand, V. M., & Crimmins, D. B. (1992). The Motivations Assessment Scale (MAS) administration guide. Topeka: Monaco and Associates. Behavior Analyst Certification Board, which focus on Hanley, G. P. (2012). Functional assessment of problem behavior: dispel- competence (1.02a) and consultation (2.04a) (Bailey ling myths, overcoming implementation obstacles, and developing and Burch 2011). new lore. Behavior Analysis in Practice, 5, 54–72. Hanley, G. P., Iwata, B. A., & McCord, B. E. (2003). Functional analysis of problem behavior: a review. Journal of Applied Behavior Analysis, 36, 147–185. doi:10.1901/jaba.2003.36-147. Conclusions Iwata, B. A., & DeLeon, I. G. (1996). Functional Analysis Screening Tool (FAST). Gainesville: Florida Center on Self-Injury, University The current paper highlighted the distinguishing features of of Florida. open and closed-ended indirect assessment methods, includ- Iwata, B. A., Wong, S. E., Riordan, M. M., Dorsey, M. F., & Lau, M. M. (1982). Assessment and training of clinical interviewing skills: ana- ing their strengths and limitations, and highlighted some con- logue analysis and field replication. Journal of Applied Behavior siderations for practicing behavior analysts. In addition, while Analysis, 15, 191–203. doi:10.1901/jaba.1982.15-191. considering possible strengths and limitations associated with Iwata, B. A., Dorsey, M. F., Slifer, K. J., Bauman, K. E., & Richman, G. indirect assessments a number of areas for further research S. (1994). Toward a functional analysis of self-injury. Journal of were identified. Generally, practitioners should be aware of Applied Behavior Analysis, 27, 197–209. (Reprinted from Analysis and Intervention in Developmental Disabilities, 2, 3–20, 1982) doi: the possible strengths and weaknesses associated with both 10.1901/jaba.1994.27-197. open and closed-ended indirect assessments. Beyond this, Iwata, B. A., DeLeon, I. G., & Roscoe, E. M. (2013). Reliability and practitioners should carefully weigh the pros and cons associ- validity of the functional analysis screening tool. Journal of ated with using a particular assessment tool in a given situa- Applied Behavior Analysis, 46, 271–284. doi:10.1002/jaba.31. Kelley, M. E., LaRue, R. H., Roane, H. S., & Gadaire, D. M. (2011). tion. While there may be no hard and fast rules about when to Indirect behavioral assessments: interviews and rating scales. In W. use any particular tool, sensitivity to factors associated with W. Fisher, C. C. Piazza, & H. S. Roane (Eds.), Handbook of applied different practices seems likely to be beneficial. behavior analysis (pp. 182–190). New York: Guilford Press. Kennedy, C. H., & Meyer, K. A. (1996). Sleep deprivation, allergy symp- Compliance with Ethical Standards toms, and negatively reinforced problem behavior. Journal of Applied Behavior Analysis, 29, 133–135. doi:10.1901/jaba.1996. Conflict of Interest The authors declare that they have no conflict of 29-133. interest. Lewis, T. J., Scott, T. M., & Sugai, G. (1994). The problem behavior questionnaire: a teacher-based instrument to develop functional hy- Ethical Approval This article does not contain any studies with human potheses of problem behavior in general education classrooms. participants performed by any of the authors. Diagnostique, 19, 103–115. doi:10.1177/073724779401900207. Matson, J. L., & Vollmer, T. R. (1995). User’s guide: questions about behavioral function (QABF). Baton Rouge: Scientific Publishers. Miltenberger, R. G. (2008). Behavior modification: principles and procedures (4th ed.). Belmont: Thompson. Miltenberger, R. G., & Fuqua, R. W. (1985). Evaluation of a training References manual for the acquisition of behavioral assessment interviewing skills. Journal of Applied Behavior Analysis, 18, 323–328. doi:10. Anderson, C. M., & St. Peter, C. C. (2013). Functional analysis with 1901/jaba.1985.19-323. typically developing children: best practice or too early to tell?: in O’Neill, R. E., Horner, R. H., Albin, R. W., Sprague, J. R., Storey, K., & response to Hanley (2012). Behavior Analysis in Practice, 6, 62–76. Newton, J. S. (1997). Functional assessment and program develop- Baer, D. M., Wolf, M. M., & Risley, T. R. (1968). Some current dimen- ment for problem behavior. Pacific Grove: Brooks/Cole Publishing. sions of applied behavior analysis. Journal of Applied Behavior Skinner, B. F. (1953). Science and human behavior. New York: Analysis, 1, 91–97. doi:10.1901/jaba.1968.1-91. The Free Press. Bailey, J., & Burch, M. (2011). Ethical for behavior analysts (2nd ed.). Thompson, R. H., & Borrero, J. C. (2011). Direct observation. In W. W. New York: Routledge. Fisher, C. C. Piazza, & H. S. Roane (Eds.), Handbook of applied Beavers, G. A., Iwata, B. A., & Lerman, D. C. (2013). Thirty years of behavior analysis (pp. 191–205). New York: Guilford Press. research on the functional analysis of problem behavior. Journal of Wahler, R. G., Vigilante, V. A., & Strand, P. S. (2004). Generalization in a Applied Behavior Analysis, 46, 1–21. doi:10.1002/jaba.30. child’s oppositional behavior across home and school settings. Cooper, J. O., Heron, T. E., & Heward, W. L. (2007). Applied behavior Journal of Applied Behavior Analysis, 37, 43–51. doi:10.1901/ analysis (2nd ed.). Upper Saddle River: Prentice Hall. jaba.2004.37-43.