Program Planning, Implementation & Evaluation Class 8 PDF

Program Planning, Implementation & Evaluation PPN 302: Class 8  Conduct a Community 1 Assessment Identity Issues & Priority 2 Identify Goals and Objectives 3 Identify Strategies, Activities and 4 Resources Develop Indicators 5 Review the Program Plan 6 Im...

Program Planning, Implementation & Evaluation PPN 302: Class 8  Conduct a Community 1 Assessment Identity Issues & Priority 2 Identify Goals and Objectives 3 Identify Strategies, Activities and 4 Resources Develop Indicators 5 Review the Program Plan 6 Implement the Plan 7 Evaluation - Results/Impact 8 THCU (2001, p. 2) Program Planning in Simple Terms What am I going How do I What am to do? know I I trying have to succeede achieve? Progra d? m Planni ng After Community Assessment… Community assessment Priority setting Program planning Program implementation Evaluation  Key Considerations in Priority Setting The Stakehold Communit agency ers y you work expectatio ownership for ns & buy-in HP Values Agency / Available /principles Funder resources – Mandate “upstream ” MOHLTC Evidence Mandate on Feasibility & urgency of of plans Guidelines needs Moderating Factors in Priority Setting Power relatio ns Availabl e Social resource and s political context s Processes # of used to priorit set y priorities Who are issues the decision makers  Priority Setting: SWOT Analysis STRENGTHS Resources Capacity THREATS WEAKNESSES Political Resources environment Capacity Systemic inequity OPPORTUNITIES Community readiness Organizational After Setting Priority… Community assessment Priority setting Program planning Program implementation Evaluation Clarifying goals: Needs & objectives & priorities  What would be a more desirable situation?  What do we want to happen?  In this case you may begin with a broad goal which you may have to explore further, identifying factors that are influencing the situation.  You may find sub-goals interconnected to the larger goal will help in achieving the desired result. 9 Program Goals Goals: - Broad general statements that outline the rationale for the program & the desire outcomes Examples:  To reduce accidental injuries among young children living at St. James Town.  To improve the quality of life of new mothers & their infants.  To increase earlier intervention in the problem of alcohol & other drug use. Program Objectives  Objectives are statements that specify actions and expected outcomes  Each objective contains only one planned action and expected outcome  Objectives are long-term and short-term  Objectives should be SMART ‘SMART’ Program Objectives Specific S Unambiguous Measureable M Quantity & quality Achievable A - Political & social context Realistic R - Resources T Time frame An objective at a glance A program objective Expected Priority Action & Time frame outcome population approach Between To increase Among Using social 2017-2019 awareness of parents & marketing & childhood caregivers of health injuries by children communicati 50% under age 6 on Examples: Process objectives: *To provide 6 classes on maternal & newborn health. *To provide information to the general public about addictions Outcome objectives (Short term)  *To increase clients’ knowledge of maternal & child health by 50% by the end of the intervention.  *To increase the number of clients maintaining sobriety by 50%.... Process objectives Outcome objectives To establish...for To increase… To provide…for To maximize… To develop…for To improve… * But program planning is more than just goals, objectives 14 Choosing the Best Strategies  Identify the various ways of addressing the issue.  Assess:  Which option will have buy-in from the priority groups and other stakeholders?  Which option is most effective and efficient?  What resources are needed, and where can they be obtained?  What the roles of different Strategies for Health Promotion What are some examples? Strengthen community action Create supportive environment Build healthy public policies Develops personal skills Reorient health services Multiple HP Strategies  increased effectiveness Education: Public environment Policy: No & health; Pesticide alternative By-law pest control Intersectoral Social collaboration marketing: : Lawn care pesticide business, & health Healthy residents & environm governments ent Program Implementation: Considerations  Putting the program into action as planned  Consistency between goals, objectives, strategies being used  Keeping the program on track  Ensuring that timelines are met, program occurs as envisioned  Ensuring that we do “good” and “no harm”  Reflecting on ethical considerations; issues of equity and social justice at all phases  Dealing with the unexpected  Leaving room for contingencies  Being flexible as appropriate Resources, budgets, & flexibility  A program must have adequate resources.  Set a budget (expenses, expected income, sources) along side each action strategy.  Seek resources both within and outside the community.  There are always capacities within communities.  Community resources: people, money, infrastructure, social networks, partner agencies, etc.  CHNs are part of the program resources  Program Logic Model Goal Target group(s)/ population(s) Strategies or Components Long-term Outcome objectives (arrows show “influenced by”) Short-term Outcome objectives (arrows show “influenced by”) Program activities Citation: Source: Peel Region Public Health. Logic Model Template. Short description: The logic model is used to represent a program in a diagrammatic illustration. 20 What is a logic model?  Provides a graphic depiction of the relationship between a program’s resources, activities, and expected outcomes. It may be used during program planning, implementation, and evaluation. ▪ Public Health Ontario (PHO), 2016  Helps stakeholders understand the overall structure and function of a program (i.e., the “big picture”). ▪ TCHU, 2001 A logic model supports planning  Helps build a common understanding of what a program is all about and how the parts fit  Can be used to shape program strategies, clarify and set priorities, and illustrate program approaches  Helps to identify gaps in program logic and negotiate roles/responsibilities of partners and staff  Communication tool to gather support or funding for an initiative  Helps involve stakeholders in planning  PHO 2016; TCHU 2001 Example: Childhood Injury Prevention Middlesex-London Health Unit (THCU, 2006) Goal To reduce the number of childhood injuries and deaths in the Middlesex- London communities. Long- Increase the percentage of parents and term caregivers who provide appropriate Objecti supervision and protection by 2025. ves Increase the awareness of area residents regarding the prevention of childhood injuries in 0-6 year-olds by 2025. Strengthen the community capacity to prevent childhood injury and support the promotion of a safe community environment by 2025. Long- Number of parents/caregivers who Strategi Education & Skill-building Community es Awareness & supportive mobilizatio environments n Priority Parents of Family Child- and Populati children 0-6 physicians/ped youth- ons years of age iatricians based Caregivers Libraries services Children Child care Injury- Educators centers/caregi prevention- Residents vers oriented of the four- Community agencies/ county area events services Parents Consumer Using Multicultural parents/car multiple communities egivers HP JK to Grade 1 strategies students Concerned Teachers citizens Families in Strategi Education & Skill-building & Community es Awareness supportive mobilization environments Short- - Increase - Increase - Establish an term awareness of awareness of effective and objectiv how to modify behaviours/skills efficient es home which reduce community environments. the risk of coalition. - Increase injury. awareness of - Strengthen the skill- community building partnerships, messages of with the the media inclusion of new campaign. partners and - Increase services awareness of childhood injury issues Short- - Number of individuals able to Coalition term identify injury as a serious risk to actively take outcom children. part in at e - Number of parents/caregivers least 2 child Strateg Education & Skill-building & Community ies Awareness supportive mobilization Activiti Four-year environments Distribute safety Provide es media devices and/or leadership for campaign. economic collaborative incentives to efforts of the Participatio families in need initiative, n at (e.g. safety plugs possibly as part established and latches.) of the London community Develop a lending Safe events. resource system Communities - Educate and train Child Safety service providers Sub-Committee. (e.g. Continue PHNs and ECEs.) ongoing Produce & recruitment of distribute Safety community Never Hurts partners. bulletins. Produce & distribute home safety checklists. Strategies Education & Skill-building & Community Awareness supportive mobilization environments Process Amount of Number of indicator Developme material community s nt of a four- distributed. partners. year plan. Number of Number of Number resources meetings. of enhanced. marketing Number of Attendance messages. incentives/safe at meetings. Number ty devices of people distributed. reached. Number of Number service of sites provider where updates. messages appear. Strategi Education & Skill-building & Community es Awareness supportive mobilization environments Resour 1.2 Full Time Safety device Leadership ces Equivalent (FTE) distribution for Public ($3,000- collaborative Health Nurse for $12,000) efforts of the program Lending initiative coordination system ($5,000- ($5,000- (1.0 FTE was in- $9,000) $9,000) kind from health Professional Ongoing department) development recruitment of 0.2 FTE ($1,500-$5,000) community program Home partners. evaluator checklist ($1,500- Creative ($5,000-$6,000) $5,000) design and Safety development bulletin costs ($5,000- shared equally $15,000) amongst Expand Risk participating Watch program health ($5,000- Example: Program Logic Model Template 29 Program evaluation “The systematic examination & assessment of features of a program or other interventions in order to produce knowledge that different stakeholders can use for a variety of purposes (Rootman et al., 2001)”.  Was the program carried out as planned?  What has changed as a result of the program?  Did the results reflect the efforts?  What is the situation now? Program evaluation  Was the program carried out as planned?  What has changed as a result of the program?  Were the results worth the effort?  What is the situation now? 31 Sequence for evaluation  Choosing the questions to be answered  Creating an evaluation plan or design  Deciding the kinds of data needed to answer the questions & ways of obtaining the data  Collecting & analyzing the data  Interpreting the results 32  Sequence of evaluation activities Look at the goal/objectives of program and 1 establish the questions to be answered Create an evaluation plan or design 2 appropriate to the questions to be answered Decide what data is needed to answer the 3 questions Decide on strategies for obtaining the data 4 Collect & analyze the data 5 Interpret the results 6 Evaluation types 1) Formative Evaluation:  Takes place from the start of the program;  Provides information to the team planning and implementing the program “along the way;”  Permits improvement while activities are in progress. 2) Process Evaluation:  Focus on all the activities undertaken to meet the program goals and objectives;  Assess the extent to which the ‘implemented’ program activities have corresponded to the ‘planned’ activities;  Identity the factors that influence the program process. Evaluation types 3) Summative Evaluation:  Takes place at an endpoint;  Provides retrospective information about the performance of the program up to the point when the evaluation was completed;  More likely to be used to make decisions about whether the program should continue and if so with what changes. Impact Evaluation:  Determine the immediate effects of the program  Identify if effects were INTENDED (e.g., new mothers’ gaining knowledge about child care & self-care) or SPIN-OFFS (e.g. increased social support). Evaluation types Outcome evaluation: Assess the long-term change and whether this change was the result of the intended goals and objectives Outcome evaluation requires resources for longitudinal tracking. (e.g., quality of life of mothers & their children) 90% people increased their activity levels during the program 90%% of new mothers intended to continue breastfeeding following a breastfeeding program 85% of people continued to take their meds appropriately 3 weeks after the program Comparative evaluation: some examples Comparing two similar programs (two similar pre-natal programs & their outcomes) Comparing two different programs (e.g., early discharge & or later discharge of similar weight babies) Compare established norms across regions (e.g., condom use locally as compared to national data) Pre- and post- program assessment: before and after the program The same program over time - longitudinal assessment over time. Important considerations for evaluation  Evaluation is not an “afterthought” o Must be integrated in all stages of the planning process o Cost of evaluation must be factored into planning process  Evaluation must be realistic in terms of resources and timelines  Evaluation must reflect goals, objectives & program strategies and activities  Evaluation requires meaningful engagement of participants and stakeholders  Results must be conveyed to all participants & stakeholders in meaningful, timely, & appropriate ways ETHICAL ISSUES  Ethical codes apply in programming as they do in all other areas of nursing  Nurses must be sure that all activities can be adjusted if anyone experiences unexpected negative effects, people must give informed consent before, during and after the program - e. g., re: aspects of reporting etc.  People must be informed about and understand the program, its purposes, any possible risks and benefits, and that they can withdraw at any time without any difficulty.  Confidentiality and negligence (accepted standards of conduct etc) should be outlined.  Safety issues must be considered through out, e.g., accessibility for all, traveling to and from meetings in the evening etc. 38 Identifying appropriate, ethical strategies  What is happening in the community that might influence program success? ▪ Present & historical factors including community dynamics  What strategies are most appropriate and relevant to target group and stakeholders?  What strategies are most effective and efficient? ▪ How can synergy among strategies best occur? ▪ What strategies will be most sustainable?  What resources are needed, and where can they be obtained? ▪ What is the cost associated with resources?  What capacities already exist and what can be developed? Ethical principles must be central to all planning & evaluation activities No Harm Transparen Promote principle cy Health s Build Prevention & capacity Health protection Facilitate access & Accountabilit equity y CHN Informe competenc Build d y relationshi Consen ps t  Foundation of Program Planning Creating relevant & ethical programs - Specify goals & objectives - Identity HP strategies - Identify resources required - HP principles/values & CHN ethics - Equity, human & civil rights, social justice Remember to consider Sustainability  Longer-term viability of program interventions  Continued focus on the problem, evolution of strategies as appropriate  Social justice & equity  Are there long term costs to community members to sustain the behaviour/activity once the program is over  Has the need been met? Is there a need for the program to continue or can it be a short term program?  Application What best illustrates a short term objective for a program aimed at increasing knowledge of bicycle safety for children under 10? Explain your choice. a) Hospital data will show a 10% decrease in children admitted for bike accidents over a 2 year period. b) Parents will be able to describe strategies they can use to increase bike safety for their children under 10. c) Parents will work together to advocate for increased bike lanes in their community d) Police data will show an increase in ticketing drivers who “door” bicyclists  Application What best indicates ethical practice in planning a sexual health program for middle school children. Choose all that apply. Explain your choice(s). a) The CHN includes regular communication with parents and the school principal and teachers as part of the program b) The CHN uses her own religious beliefs and practices to guide program planning related to adolescent sexuality c) The CHN provides program leaders, teachers and the principal with guidelines for what to do should a child disclose sexual abuse during or after the program d) The CHN uses knowledge of the school’s demographics in planning the program e) The CHN teaches about condom use even though parents have asked that her not to.  Application A CHN is developing a program to help families of children with autism learn advocacy skills. Put the following in the appropriate order for effective program planning and implementation. Explain your choice of order. a) The CHN performs a needs assessment that includes families, the school board and a provincial agency that supports the families b) The CHN identifies a number of challenges to the program revises the plan accordingly c) The CHN explores the availability of similar programs in the area d) The CHN assesses the projected costs of running the program e) The CHN pilots the program with a small group of families  Application A CHN wants to share evaluation results of a safe consumption site (SCS) program for street-involved youth who are clients of the program. What would be the most appropriate strategy? a) The CHN emails the evaluation report to the youths who were involved in the program b) The CHN prints up copies of the final report and leaves them at the agency for the youths to pick up c) The CHN creates a summary of the report and invites the youths to come to a dinner at the agency to hear about the evaluation d) The CHN creates a summary of the report and gives it to agency staff to hand out to the youths