Health Program Planning and Evaluation PDF

Program Objectives and Setting Targets Learning Objectives With this chapter, readers will be able to: 1. Write clear, measurable process and effect objectives. 2. Distinguish between process and effect objectives. 3. Develop realistic and achievable target values for objectives. Presentation O...

Program Objectives and Setting Targets Learning Objectives With this chapter, readers will be able to: 1. Write clear, measurable process and effect objectives. 2. Distinguish between process and effect objectives. 3. Develop realistic and achievable target values for objectives. Presentation Outline • Key terminology • Goals vs. objectives • Program theory elements • Effect theory elements • Health domains and indicators • Criteria for selecting indicators • Effect objective examples • SMART objectives • Decision options for target setting • Data source availability and consistency for target setting • Options for calculating target values • Caveats to setting goals and objectives • Across the pyramid • Internet resources Key Terminology • Goal • Indicator • Objective Goals vs. Objectives Goals Objectives • Broad, encompassing statements • Generally, one per program • Longer time horizon • Not stated in quantifiable terms • • • • • Specific statements Shorter time horizon Measurable Process and effect “By when, who will achieve what, by how much” Program Theory Elements as the Basis for Writing Objectives Program Theory Elements and Process and Outcome Objectives Organizational plan Service utilization plan Theory outputs Format By when, who will obtain or organize how much of which types of resources in what ways By when, who will have how many interactions with participants of what type By when, how many of what types of outputs or products will be created or finalized by whom Examples By [date], the program manager will secure funding for 3 new computers with electronic clinical record software By [date], the health educator will identify 3 evidencebased interventions for improving diabetes selfmanagement By [date], staff will distribute # brochures to women receiving genetic counseling at the clinic Relationship Between Effect Theory Elements and Objectives Health Domains and Indicators Indicators to measure objectives Impact domain General measures Standardized measures Physical health Weight, dental or medical diagnoses Normal range on laboratory tests, ICD-10 Mental health Motivation, attitudes, medical diagnosis of mental illness CES-D, Daily Hassles Scale Cognitive processes Decision making, judgments, cognitive development or impairment Bayley Scales of Infant Development, IQ tests Behavior Smoking, exercise, seat belt wearing Knowledge Skill, ability, performance, recall or synthesis of facts Social health Marital status, social network, recreation activities, volunteerism Norbeck Social Support Questionnaire Resources Income, insurance coverage, housing, employment status Hollingshead Index, Dissimilarity Index Criteria for Selecting Indicators • • • • Funding agency requirements Data collection is feasible Scientifically defendable Easy to analyze Bowe County Health Problems with Indicators, Outcomes, and Impacts Problem Indicator Outcome Goal or impact Vaccination Rates of underimmunized, by age group Vaccine-preventable illness Decrease vaccine preventable hospitalizations Presence of neural tube defect Rates of neural tube defects and congenital anomalies Diagnosis of pregnancy Pregnancy rate, by age group Child abuse rate Reduced child abuse related to unwanted pregnancy Hospital admissions for gunshot wounds Rate of admissions for gunshot injuries at local hospitals, number of police reports Adolescent death rate due to gunshot wounds Reduce adolescent death rate due to gunshot wounds Diagnosis of type 2 diabetes Incidence and prevalence rates of diabetes Morbidity due to chronic illness Reduce rates of amputation and vision loss due to diabetes Maintain current rate of congenital anomalies Effect Objective Examples for Congenital Anomalies Intervention objective from intervention theory Outcome objective from causal theory Impact objective from impact theory Format By when, what proportion of recipients will have how much effect from program interventions on which causal factors that lead to the health problem By when, what proportion of recipients will have how much effect from program interventions on the immediate health problem By when, long-term or global health change or status among the target population Example By [date], [target #] women in the program will have a decrease by [target %] in exposure to teratogenic environmental hazards By [date], [target #] women in the program will have normal newborns (no neural tube defects) By [date], [target rate] of congenital anomalies among residents of Bowe County SMART Objectives • • • • • Specific Measurable Achievable Realistic Time Decision Options for Target Setting Current value of the indicator in the target audience Population indicator trend Better than longterm objective Meets long-term objective Worse than longterm objective Improving Set target to maintain current level Set target to surpass objective; continue the trend Set target to a better level; accelerate the trend No change Set target to a slightly better level Set target to surpass objective Set target to a slightly better level Deteriorating Set target to maintain current level; stop the trend Set target to maintain current level Set target to maintain current level or a slightly lower one; stop or decelerate the trend Data Source Availability and Consistency for Target Setting One or only a few sources Many sources Consistent information across sources Consider the data quality and relevance to program and objective Can use any of the data sources Inconsistent information across sources If the source is markedly different from the literature, need to either change the objective or verify the data Need to decide which source to use, given their strengths and weaknesses; consider which is most relevant to the program and objective Options for Calculating Target Values Option Description Ideal program type Pros Cons 1 Default, no change Mature, stable Doesn’t require historical data Doesn’t require improvement 2 Change based on results of statistical test Population-based with many recipients Supports argument that improvement was more than by chance Sensitive to sample size; may result in unreasonable target 3 % change based on current trend, literature, or guess Stable program and target population Easy to understand; can account for trend data Requires some statistical knowledge 4 Use existing benchmark to project for several years Program must show improvement Comparable programs can be compared Requires existing long-term objective and long-term program 5 Mean rate across geographic areas Population based Easily understood Requires data from each area Options for Calculating Target Values, Continued Option Description Ideal program type Pros Cons 6 Median rate across geographic areas Population based Easily understood Requires data from each area 7 Overall rate from best 50% across geographic areas Population based or multisite Considers all values in population and moves entire population to an achievable value Requires data from each area; harder to understand 8 Overall rate from best 75% across geographic areas Population based or multisite Considers all values in population and moves entire population to an achievable value Requires data from each area; harder to understand 9 Rate for best sociodemographic stratum Population based or diverse target audience with disparities Considers all values in population and moves entire population to an achievable value Requires data from each group; harder to understand 10 Overall rate based on different targets for strata Population based or diverse target audience with disparities Program must show improvement; more intense intervention for worst-off group Requires data from each group; harder to understand Internet Resources • Indiana Dept. of Education’s document on writing goals and objectives: http://ideanet.doe.state.in.us/sdfc/pdf/writinggos.pdf • Checklist: http://www.prm.nau.edu/prm423/goals_and_objectives_lesson. htm • Bloom’s taxonomy is especially useful for knowledge objectives; http://www.roundworldmedia.com/cvc/module4/bloomtaxx.htm l • Writing SMART objectives: http://www.rapidbi.com/created/WriteSMARTobjectives.html