Planning Behavior Change Programs Summary PDF

Planning Behavior Change Programs Summary Purpose of Intervention Mapping – provide health promotion program planners with a systematic framework for effective decision making, planning, development, implementation, and evaluation Perspective of IM – a planning approach that is based on using theo...

Planning Behavior Change Programs Summary Purpose of Intervention Mapping – provide health promotion program planners with a systematic framework for effective decision making, planning, development, implementation, and evaluation Perspective of IM – a planning approach that is based on using theory and evidence as foundations for taking an ecological approach to assessing and intervening in health problems and engendering community participation IM is guided by three perspectives → a multi-theory approach, an ecological approach, and a participation perspective. Multi-Theory Approach Theories and Evidence enable planners to make better choices Causal Theories – help to describe the health problem and its causes Change Theories – suggest approaches to problem solutions Ecological Approach Social Ecological Model – health as a function of individuals and of the environments in which individuals live, including family, social networks, organizations, communities, and societies Social Ecological Paradigm – interrelationships between individuals and their environments Participation Perspective Inclusive – reflects concerns in local community Broad – greater breadth of skills, knowledge, and expertise → improves external validity Ethical Practice – inclusion of diverse participation, protection of rights, goals should be related to health Core Processes – used in theory and evidence 1 Posing questions – identify questions about health problems, behaviors, determinants, and change methods 2 Brainstorming answers – generate a wide range of provisional answers based on existing knowledge 3 Reviewing findings from published research – use literature to support or refute answers and to ensure quality and relevance 4 Accessing and using theory – translate answers into theoretical constructs to identify practical solutions Topic Approach – refine, add to, and discard provisional answers based on theoretical concepts Concept Approach – access theories through concepts generated during brainstorming General Theories Approach – consider potentially useful general theories 5 Identifying and addressing the need for new research – conduct qualitative and quantitative studies to validate findings for the target population 6 Formulating the working list of answers – refine the list based on relevance and changeability Theories of Behavior Theories that explain behavior Theory that help change behavior Theory of Planned Behavior – behavior is influenced by Health Belief Model – perceived severity, susceptibility, attitudes, norms, and perceived behavioral control benefits, and barriers influence behavior change Self-Determination Theory – autonomy, competence, Stages of Change Model – five stages (precontemplation, and relatedness drive behavior contemplation, preparation, action, and maintenance) for behavior change Reasoned Action Approach – behavior is influenced by Motivational Interviewing – client-centered approach with attitudes, norms, and perceived behavioral control use of empathy and reduction of ambivalence Theories that do both Social Cognitive Theory – behavior can be learned through observation, imitation, and modeling Six Steps of Intervention Mapping → Outcome – Logic Model of the Problem → Outcome – Logic Model of Change → Outcome – Designing the Program → Outcome – Materials and Pretest → Outcome – Implementation Plan → Outcome – Evaluation Plan  Step 1 – Logic Model of the Problem Task 1 – Establish and work with a planning group Planning Group – engaging community members, program implementers, members of the at-risk group, and decision makers ensures programs are relevant, builds capacity, and strengthens evaluation Involvement – consultative (working with the group) to full community-led planning (group are the planners) Include varied stakeholders (residents, service providers, leaders, academics) for diverse insights, cultural sensitivity, and to facilitate program sustainability and dissemination Planning Group ensures that program focus reflects community concerns → improves external validity Linkage System → between program developers (e.g., experts and health educators), implementers (e.g., principles and representatives) and participants (e.g., parents) Planning Group Needs o Expertise in the health problem or its causes o Diverse perspectives and community participation o Responsibility and authority o Influence o Commitment to the issue Priority Population – population at risk Key Collaboration Principles o Recognize community identity and involve stakeholders early o Balance knowledge and intervention for mutual benefit o Build from an ecological perspective, involving multiple health determinants o Ensure results are shared with and accessible to all partners o Emphasize respect, equity, learning, and sustainable outcomes Task 2 – Conduct a needs assessment to create a logic model of the problem Needs Assessment – systematic study which identifies gaps between the current state and a desired state within a community or group → use Core Processes o It is problem-focused without implying solutions o It includes an analysis of the physiological, behavioral and environmental risks to health  o Determinants can either be modifiable (e.g., beliefs, attitudes) or non-modifiable (e.g., genetics, demographics) QoL → what is the impact of the health problem on QoL on individual and societal level? Health Problem → what is the priority health problem in the population or subgroup? Behavioral Factors → what behaviors increase risk, incidence, prevalence, and burden? Personal Determinants → what theory- and evidence-based factors are causally related to the behavior? Environmental Factors → what interpersonal, organizational, community, and societal factors influence health directly or through influence on the behavior of the at-risk group? Personal Determinants → what theory- and evidence-based factors are causally related to the behavior of agents in the environment who control the environmental factors? Task 3 – Describe the context for the intervention including the population, setting, and community Develop an understanding of the context for the intervention (including population, community, and setting). 1 Identify Community Assets, Capacity, and Ability Asset-Based Approach – identifies existing community strengths (e.g., social ties, resources, skills) that can support intervention success and sustainability Conducting Asset Assessments o Social Environment – what are existing social environmental factors that could support the program? o Information Environment – what are existing communication channels (e.g., TV, radio, social media) that could be activated for the program? o Policy Environment – what are existing policies (e.g., school health policies) that could be leveraged to support the program? o Physical Environment – what are aspects of the natural or built environment that could be harnessed to support the program? 2 Identify Factors that could support the Intervention → see asset assessment 3 Identify an appropriate Setting Task 4 – State program goals Program Goals – focus on measurable changes in health, QoL, behavior, and environment SMART Goals – specific, measurable, attainable, relevant, time-based Example – after two years of intervention, the percentage of sexually active students in the intervention group who used a condom at last sexual intercourse will be 10 percent higher compared with students in the non-intervention group in Houston, TX Criteria for Setting Priorities o Magnitude between what is and what could be o Difference in burden from a problem among groups → focuses on health disparities o Practical issues (e.g., consequences of ignoring the needs vs. possible costs of implementing a solution) o Political and social factors (e.g., community values, context of priorities (local, regional, national and international), public and leader expectations, available interest and expertise, momentum, and availability of funding and human resources)  Step 2 – Program Outcomes and Objectives – Logic Model of Change Task 1 – State expected outcomes for behavior and environment Start with the Logic Model of the Problem and ask what needs to change in the behavioral and environmental risks factors to have improved health outcomes? → Do the Flip from problem-focus to solution-focus Planner will look at problematic conditions and state what needs to change: o Risk-Reduction Behaviors – goal is either elimination or reduction of harmful behavior o Health-Promoting Behaviors – goal is either primary prevention (preventing health problems before they occur) and secondary prevention (detect diseases early) o Adherence and Self-Management Behaviors – goal is to improve adherence to health care instructions Stating Behavioral Outcomes – behaviors to be accomplished as a result of the health promotion program Example – consume less than 10 percent of calories from saturated fats Stating Environmental Outcomes – environmental change o Interpersonal – families, peer groups, influential roles, support from social networks Example - parents support preschoolers to be more physically active o Organizational – norms, policies, practices, facilities Example - food service directors modify purchase for healthy foods o Community – access to health care, availability of resources, social capital Example - off-premise alcohol outlets comply with underage drinking laws o Society – legislation, enforcement, regulation, policies Example - state legislators prohibit smoking in public places Task 2 – Specify performance objectives for behavioral and environmental outcomes Performance Objectives – statements of what a program participant will do or how the individual will modify the environment → sub-behaviors Use Core Processes to write Performance Objectives. Behavior → what do the participants of this program need to do to perform the health-related behavior? Environment → what does someone in the environment need to do to accomplish the environmental outcome? Validate Performance Objectives → self-reports, observations, experts, correlations Task 3 – Select determinants for behavioral and environmental outcomes Determinants for Behavior – factors that rest within individuals and are subject to their direct control (cognitive factors and capabilities) Determinants for Environment – factors that rest outside the individual (social and structural influences) Use Core Processes to write Determinants. Importance of determinants need to be rated by relevance and changeability → make list more concise CIBER-Approach – visualization of confidence intervals for the means and correlation coefficients for all determinants simultaneously → ensures comparison Item 2 o People strongly believe that using high doses of ecstasy is bad for their health, and this belief strongly influences their attitudes and intentions o But since most people already agree with this belief, it’s less relevant to target in an intervention Item 15 o People think high doses of ecstasy make their trip more intense, but this belief doesn’t strongly affect their attitudes or intentions o This makes it a low-priority target for an intervention Item 1 o This belief is both strongly linked to attitudes and intentions and scores in the middle of the scale (some people agree, some don’t) o This makes it a good target for an intervention Task 4 – Construct matrices of change objectives Change Objectives – specify what needs to change in the determinants of behavioral or environmental outcomes to accomplish the performance objectives → stated with action verbs Use Core Processes to create Matrices of Change. Planning Question – what needs to change related to Determinant X for program participants to do PO X? Constructed by crossing performance objectives (left column) with determinants (top) Intervention levels need to be selected (at what levels of intervention is it necessary to attain the performance objectives?) → Ecological Levels Intervention populations need to be selected (are either performance objectives or determinants substantially different for subgroups?) → age and gender, geographic location, SES, education, and cultural group It is not advisable to place automaticity or habit in the matrix as a determinant → it is very difficult to formulate clear change objectives Task 5 – Create a logic model of change Logic Model of Change – illustrates pathways of program effects → read from left to right Provides the program evaluators with clearly stated behavioral and environmental outcomes, performance objectives, and change objectives that can guide the program evaluation. ABCD Matrix – Acyclic Behavior Change Diagrams This matrix combines the Logic Model of Change with the associated methods and applications which is organized by change objectives.  Step 3 – Program Design Task 1 – Generate program themes, components, scope, and sequence Generating Program Ideas o Key to having a good idea is having more than one idea → use of Core Processes o Creative Process – emphasizes brainstorming and lateral thinking without constraints o Diverse planning group fosters unique and innovative ideas o Literature reviews and participant input through focus groups refine ideas and ensure relevance Program Themes o Central organizing idea that ties program components together → creative process o Themes reflect health topics, behavior objectives, or cultural aspects o Will the program have a theme, more than one theme, and a title? o Example – Keeping It Real resonated with urban youth for promoting responsible behavior Program Components o Components – comprised of modules (e.g., teacher-facilitated activities, computer-based exercises, media campaigns) o Setting – classroom, cafeteria, workplace, outside of work o Populations/Subgroups – child component or parent component o Communication Channels and Vehicles o Selection – based on participant preferences, logistics, and budget → combines primary (e.g., school programs) and secondary (e.g., media reinforcement) delivery methods o Interpersonal Communication Channels → parents, teachers, volunteers o Mediated Communication Channels → print, radio, TV, social media, video o Multi-level/Socio-ecological Framework – policy level, social level Scope and Sequence o Defines program breadth, content, and order of delivery o Scope – breadth and amount of a program and its duration o Global scope – e.g., campaigns o Component-specific scope – e.g., classroom curriculum sessions o Sequence – order in which program components will be delivered o Example – a year-long community program combined mass media for participants and training for health care providers Theme is based on Performance Objective  Theme is based on determinant (Outcome expectation) Theme is based on an engaging narrative (motif) Task 2 – Choose theory- and evidence-based change methods Change Methods – target determinants of behavior or environmental conditions o Select methods for all change objectives → different methods are effective for different determinants o Parameters for use are important o Example – determinant like skills → methods like modeling Process 1 Reorganize objectives by determinants (e.g., knowledge, expectations) 2 Align methods with determinants, often using multiple methods for a determinant 3 Select methods by ecological level (individual, interpersonal, organizational, community, and societal) Using Core Processes – review empirical evidence and theories of change for insights → apply topic approach, construct approach, or general theories approach Taxonomy – structured system of classification that organizes items, concepts, or entities into hierarchical categories based on shared characteristics or criteria Coding Taxonomy – BCT Taxonomy v1 (Michie et al., 2013) o Coding of an existing intervention o Hierarchical classification system that organizes behavior change techniques (BCTs) into distinct, well-defined categories, ensuring they are precise, non-overlapping, and can be confidently used to describe interventions Development Taxonomy (Kok et al., 2016) o Focuses on selecting methods based on circumstances → iterative development o System categorizes behavior change methods derived from theory, aiming to guide health promotion planners in developing interventions Integration of both could enhance intervention development and meta-analysis quality. Task 3 – Select or design practical applications to deliver change methods Application – practical technique for the operationalizing methods in ways that fit the intervention group and the context → scientific groundwork on theory and evidence One change method may be accomplished by several applications and one application my deliver various change methods Designing Practical Applications o Consider change methods as "active ingredients" presented in practical formats (e.g., role models, stories) o Tailor applications to the target population and delivery the context Successful Application Examples o Modeling – use coping models demonstrating feasible skills with positive reinforcement o Active Learning – use interactive activities where participants advise or role-play to internalize behaviors o Risk Perception Information – provide cause-effect scenarios to improve personal risk understanding Application across Levels o Social Network – gather stakeholders (e.g., family) to address issues collaboratively o Organizational – conduct surveys, discussions, and action plans for employee morale o Community – host workshops to develop strategic solutions for community issues (e.g., obesity) Determinant – outcome expectation; perceived susceptibility/severity Method – fear arousal Parameter – requires self-efficacy more than only outcome expectations Practical Application – magazine ad Determinant – social norms Method – information about others approval Parameter – positive expectations available in environment Practical Application – poster Outcome – school cafeterias promote consumption of fruits and vegetables Environmental Method – systems change/structural redesign; nudging; stimulus control Practical Application – reengineering of serving line to promote FV first Outcome – promote gun control/legislation Environmental Method – media advocacy; theater of the oppressed Practical Application – Die-In in front of the White house  Step 4 – Program Production Step 4 (Program Production) is about using Step 1 - 3 to produce program materials. Two Challenges of Step 4: 1 Translation – designing the support materials so that the methods and practical applications are effectively operationalized and the change objectives accomplished 2 Fitting – ensure that the final program fits with both the populations for whom it is intended and within the context(s) or setting(s) in which it will be delivered Task 1 – Refine program structure and organization The first task in Step 3 (program themes, components, scope, and sequence) lays the foundation for this step. Plan the (co-)development of program materials and activities → e.g., videos, guidelines, protocols, slides Reality Check → check feasibility, budget and time constraints Reach the Intended Program Participants Program materials support, but don’t define, the program Ensure participants will engage with planned activities and materials Check Budget and Time Constraints Ensure program components are feasible within resource limits Prioritize key messages and materials if constraints exist Program should be culturally relevant → deep structure (values, beliefs, traditions) and surface structure (language, clothing, music) Create a Plan and Use Core Processes 1 Organize applications into program components → delivery channel, vehicle, material, length 2 Create the sequence of the program components → Gantt chart Task 2 – Prepare plans for program materials Production Planning – create detailed design documents for all program materials Pay Attention to Cultural Relevance o Integrate cultural norms, values, and behaviors into design → use community insights from early planning stages o Address both deep structures (e.g., family dynamics) and surface structures (e.g., language, visuals) Use Formative Research o Conduct research to understand participants’ needs, culture, and health literacy o Use methods like focus groups and cognitive interviews for feedback Develop a Design Documentation o Prepare project overviews for production teams (incl. audience, goals and messages) o Create detailed plans (e.g., flowcharts, storyboards) for complex or interactive materials o Align materials with program goals, ensuring feasibility within budget and timelines Create Design Documents for Community Processes o Guide non-traditional program elements (e.g., policy development, coalition building) o Detailed documents help with recruitment, meeting structure, and training programs Review Available Material o Guiding Questions Does the available material meet change objectives and deliver the intended methods? Are these culturally appropriate and fit within program scope and sequence? o Determining Sustainability (SAM) – evaluate content, literacy demand, graphics, layout, learning stimulation, and cultural relevance o Determining Availability – ensure materials are available in needed quantities and formats o Determining Reading Level – assess readability (ensure it matches the target audience's literacy level) Hiring and Working with Creative Consultants o Hiring Tips – prepare a detailed project synopsis and review portfolios and assess their understanding of the project’s intent and flexibility o Involve consultants early for better integration and allow creative freedom but ensure alignment with program goals Task 3 – Draft messages, materials, and protocols The actual development and production of materials and protocols. Consider change methods and practical applications for a particular set of change objectives → identify which will fit together for presentation in a vehicle (e.g., newspaper). Write Program Messages o Method Selection – identify methods and channels for each program component o Message Development – use community data to tailor specific behaviors and objectives o Types of Messages Focused – directly address change objectives Contextual – provide background, indirectly supporting behavior change (e.g., role modeling) o Ensure messages enhance cognitive processing → limit concepts per session, use simple, clear language, match graphics to messages, link sections explicitly and logically o Understand and adapt to cultural communication styles Producing Visual Materials o Use simple, realistic visuals to aid understanding and recall → avoid overly complex designs or distracting backgrounds o Ensure visuals are culturally appropriate and engaging Producing Print Materials o Plan design elements (e.g., lists, graphs) → consider cost, format, and distribution methods o Determine review processes and update plans Producing Video Materials o Develop contracts and budgets → use scripts and storyboards to align videos with change objectives o Pretest rough cuts with the target audience o Ensure all stakeholders approve scripts to avoid costly reshoots Producing Multimedia Materials o Involve programmers and designers to create interactive content o Allow for individualized learning experiences with branching pathways o Follow a structured review process to ensure all program objectives are met Task 4 – Pretest, refine, and produce materials Importance – prevents biases from planners or representatives and provides feedback Pretesting – tests specific messages and materials with intended participants before final production → ensures materials are appealing and understandable o Audience tries out specific components of the program (e.g., concept, readability, comprehension, usability, message, impact, cultural sensitivity, acceptability) o Methods – questionnaires, face-to-face discussions, focus group discussion, ask for feedback, experiments (showing two versions of a material) Pilot-Testing – tests the program as it will be implemented, involving both implementers and participants → identifies issues with real-world execution o Audience tries out program as it will be implemented (e.g., adoption and implementation issues, logistics, acceptability) If activities are deleted after pre-testing or pilot-testing, they must be replaced by other powerful methods. Plan for Program Evaluation o Formative Evaluation – incorporates feedback during development to improve materials o Summative Evaluation – assesses final program effectiveness and audience reception post- implementation  Step 5 – Program Implementation Plan Step 5 is about developing an implementation plan to enable adoption, implementation, and maintenance of the health promotion program. This Step is similar to Step 2, 3, and 4 but is focuses on the implementation group rather than the at-risk group. Implementation Science Task 1 – Identify potential program users (adopters, implementers, and maintainers) Impact = program effectiveness x implementation Implementation = adoption + implementation + maintenance Adoption – a decision to use a new program → Who will decide to adopt the program? Implementation – the use of the program → Who will implement the program? Maintenance – the extent to which the program is continued and becomes part of the normal practices and policies of the adopting organization → Who will ensure that the program continues as long as it is needed? Implementers deliver programs and Adopters support adoption. Example – schools → principals adopt, teachers implement Example – clinics → managers adopt, providers (nurses, dietitians) implement Impact of the intervention depends on the effectiveness of the program and the reach in the population. Selecting Organizations o Choose those with broad reach to target populations (e.g., schools, clinics) o Consider organizational structure, size, leadership, readiness for change, and fit with goals o Leaders decide on adoption and ensure proper implementation Linkage System → bridges the gap between program developers and users of the program Diffusion → passive, untargeted, unplanned, and uncontrolled spread of new interventions Dissemination → planned systematic efforts to increase program use in a target population, health care setting or social system – spread intervention in a systematic way Task 2 – State outcomes and performance objectives for program use Define who must do what to ensure a program is adopted, implemented, and maintained with completeness and fidelity. Key Concepts o Frameworks & Theories – use organizational change models to identify roles and actions o Adaptation – build flexibility while maintaining essential program elements to ensure fit across settings Adoption – decision to use a new program o Outcome – (someone) adopts the (program) as indicated by (the evidence or document to indicate adoption) o Performance Objective – what do (adopters) have to do to make the decisions to use (the program)? o Factors – program knowledge, unmet needs, dissemination o Tasks – review program details, compare current practices with program goals, gain stakeholder support, sign agreements Implementation – using the program for an evaluation period o Outcome – the (organization or individual) will implement (program) including use of (program components) o Performance Objective – what do (implementers) need to do to deliver the essential program components? o Focus Fidelity → degree of the program is implemented as it is designed to be implemented Completeness → proportion of intended program activities and components delivered Dose → amount of the program that participants receive o Tasks – follow core program components, provide staff training, monitor progress and give feedback, address structural or environmental changes needed for delivery Maintenance – integration into routine practices over time → ensure sustainability o Outcome – decide on the type of outcome to be achieved: Institutionalization – integration into organization’s routines Continuation of health effects Combination of these o Performance Objective – what do (maintainers) need to do to maintain the program? o Factor – continuous evaluation, feedback, and organizational capacity o Tasks – institutionalize processes, plan for ongoing modifications based on feedback, develop leadership and organizational structures to support longevity Example → for a mammography program o Adoption – clinic leaders review results and agree to implement o Implementation – deploy phone counselors, track participation o Maintenance – conduct regular assessments and adjust as needed Task 3 – Construct matrices of change objectives for program use Develop a matrix to guide implementation interventions by linking performance objectives and determinants for adoption, implementation, and maintenance. Determinants – factors influencing program use → using Core Processes o Adoption → e.g., awareness, perceived benefits, compatibility, social/structural factors o Implementation → e.g., skills, self-efficacy, organizational support o Maintenance → e.g., long-term commitment, resources, leadership Categories of Determinants o Characteristics of Innovation – complexity, compatibility, perceived benefits o Characteristics of Implementers – needs, skills, attitudes, social support, awareness o Characteristics of Systems – leadership, work climate, resources, funding Data Collection Methods – qualitative (focus groups, interviews) and quantitative (surveys to measure determinant impact and adoption likelihood) Determinants should be prioritized by relevance and changeability. Task 4 – Design implementation interventions Process to Design Implementation Interventions 1 Choose change methods and practical applications → similar to Step 3 2 Design the scope and sequence → similar to Step 4 3 Produce materials for an implementation intervention to influence program use → similar to Step 4 Implementation Frameworks – guide the planning, execution, and evaluation of programs to ensure effective adoption and sustainability o Interactive Systems Framework → identify key actors o RE-AIM → organize implementation outcomes o Consolidated Framework for Implementation Research → guide decisions about contextual factors that may influence adoption and implementation o Theoretical Domains Framework → identification of determinants (Task 2) and methods (Task 3) Diffusion of Innovation Model When you realize how people react to an intervention, this model gives an overview of how your intervention will be implemented. Different groups can be targeted in different ways.  Step 6 – Evaluation Plan Step 6 is about the development of an evaluation plan based on previous steps of Intervention Mapping. Reasons for Program Evaluation – to determine efficacy (does the program work under controlled conditions?), effectiveness (does the program work in the real world?), feedback, and to generate knowledge Involve Evaluation Stakeholder – sponsors, policymakers, program managers Task 1 – Write effect and process evaluation questions Evaluation Pyramid Formative Evaluation → program is feasible, acceptable, culturally relevant Process Evaluation → assesses if activities were implemented as intended Summative (Effect) Evaluation → assesses effect of program on outcomes Effect Evaluation Questions – how the program is expected to cause change Efficacy – evaluation under optimal conditions Effectiveness – evaluation under real-world settings Primary outcome (did the intervention reach the goal?) → due to time constraints, evaluators cannot determine if the goal was reached, then behavior change can be stated Secondary outcome (did the intervention reduce a certain behavior or enhance determinants?) Questions o How much did the quality-of-life problem change in the designated timeframe? o How much did the health problem change in the designated time frame? o What changes in behavior and environmental conditions occurred? o Did the participants strengthen their determinant to do a behavior? Time Frame Consideration – long-term tracking for health outcomes and short-term tracking for determinants Process Evaluation Questions – how the program is implemented Explain why program succeeded or failed by examining delivery and adherence to design → avoids black-box evaluation (assessing outcomes without knowing how a program was implemented) Context – social/environmental factors affecting implementation Reach – proportion of the intended target audience Dose Delivered – number of intended units of each program component Dose Received – extent to which participant engage Fidelity – extent to which the intervention was delivered as intended Implementation – extent to which the intervention was implemented and received Recruitment – approach used to attract program participants Questions o To what extent is the program being delivered to the persons for whom it was intended? o How well does the delivery maintain fidelity to the program’s original design? o How do aspects of implementation explain results of an effect evaluation? o Were theory- and evidence-based change methods operationalized in the program applications? Task 2 – Develop indicators and measure for assessment Indicator – construct being measured (e.g., condom use) Measure – device for quantifying or categorizing the indicator (e.g., test scores, and attendance records) Developing Measures – use existing literature and standardized tools when available or customize measures Health and Quality-of-Life Measures o Often derived from national or local data sources o Short-term indicators focus on behavior changes (e.g., reduced smoking), while long-term outcomes (e.g., reduced cancer rates) take years Behavioral and Environmental Measures o Include direct observation, self-reports, and validated tools o Examples - fat intake reduction, seatbelt use, or reduced environmental hazards Determinant Measures o Measures of determinants may need to be developed based on program-specific constructs o Use IM steps to define and validate these measures Validity – measures what it intends to measure → established through comparisons with other tools or expected outcomes Reliability – consistency across time or observers → errors can arise from procedural inconsistencies or environmental factors Task 3 – Specify the evaluation design Essential for understanding the link between program design and outcomes → combines quantitative and qualitative methods. Process Evaluation o Qualitative methods – focus groups, interviews, observations o Quantitative methods – surveys, web-analytics Effect Evaluation o Randomized Controlled Trials (RCTs) o Pre- and Post-test o Time series design Task 4 – Complete the evaluation plan Consisting of → type of evaluation, evaluation question, indicator, measure, source, and data collection timing