1 Context of health Program.pptx

Chapter 1 The Context of Health Program Development and Evaluation Learning Objectives With this chapter, readers will be able to: 1. Articulate a definition of evaluation. 2. Understand the relevance of the public health pyramid to health program planning and evaluation. 3. List circumstances un...

Chapter 1 The Context of Health Program Development and Evaluation Learning Objectives With this chapter, readers will be able to: 1. Articulate a definition of evaluation. 2. Understand the relevance of the public health pyramid to health program planning and evaluation. 3. List circumstances under which an evaluation is not recommended. Presentation Outline • Key terminology • Concept of health • Generational milestones of evaluation • Evaluation standards • The planning and evaluation cycle • The program cycle • Evaluation types • When not to evaluate • The public health pyramid • Healthy People 2010 objectives • Internet resources Key Terminology • Aggregate • Community needs assessment • Comprehensive evaluation • Cost evaluation • Effect evaluation • Evaluation • Formative evaluation • Health • Health program • Health services • Impact evaluation • Institutionalized program • • • • • • • • • • • • Meta-evaluation Model program Outcome evaluation Pilot program Planning Process evaluation Process theory Program Project Prototype program Summative evaluation Service Health as a Concept • WHO definition (1947) – Absence of illness and presence of well-being • Health viewed as holistic – Physical, mental, developmental social, and financial aspects – “Health problems” may affect one domain while the others remain healthy • Health viewed as political – Health policy around access to care – Implications for program planning Generational Milestones of Evaluation • Technical – growth of scientific management, statistics, and research methodologies • Descriptive – use of goals and objectives as the basis of evaluation to establish program merit (outcome-focused) • Responsiveness – concern for the usefulness of the evaluation to stakeholders; participation of stakeholders in the evaluation process (utilization-focused) • Meta-evaluation – evaluation of evaluations across similar programs Outcome- vs. Utilization-Focused Evaluations Focus Purpose Audience Method Outcom e Show program effect Funders, researchers, other external audience Research methods; usually external evaluators Program people (internal audience), funders Research methods, participatory Utilizatio Get stakeholders to n use evaluation findings for decisions regarding program improvements and future program development Standards Established by the Joint Commission on Standards for Educational Evaluation Standar d Description Utility To ensure that the evaluation will meet the content needs of those involved Feasibilit To ensure that the evaluation will be y realistic, prudent, diplomatic, and frugal Propriety To ensure that the evaluation will be conducted in a legal and unbiased way Accurac To ensure that the evaluation will y communicate appropriate and accurate Adapted from the American Evaluation Association (2002) information The Planning and Evaluation Cycle The Program Life Cycle: Assessment, Implementation, and Evaluation Program Stage Community Assessment Program Implementation Program Evaluation Pilot Generic information about the health problem & target audience Few participants; strict guidelines & protocols Rigorous impact evaluation & process monitoring Model Greater information about the target audience Realistic number of participants; use previously set procedures Outcome & impact assessment; rigorous process monitoring Prototyp e Very specific information about the target audience & local variations in the health problem Some flexibility and adaptation to local needs; realistic enrollment Outcome & impact assessment; rigorous process monitoring The Program Life Cycle, Continued Program Stage Community Assessment Program Implementation Program Evaluation Organizationall y Institutionalize d More attention to assessment of organizational resources for program sustainability Use standard operating procedures; organization specific Outcome & impact assessment based on objectives; routine process monitoring Professionally institutionalize d Rarely any detail, more assessment of competitors and professional norms Standards for professional practice; may involve certification Use professionally set standards or benchmarks of outcome & impact assessment Evaluation Types • Community needs assessment • Process evaluation • Effect evaluation – Outcome evaluation – Impact evaluation • Cost evaluation • Comprehensive evaluation • Meta-evaluation When Not to Evaluate • There are no questions about the program • The program has no clear direction • Stakeholders cannot agree on the program objectives • There is not enough money to conduct a sound evaluation The Public Health Pyramid The Pyramid as an Ecological Model Healthy People 2010 Objectives in 28 Focus Areas • Access to quality health services • Arthritis, osteoporosis, and chronic back conditions • Cancer • Chronic kidney disease • Diabetes • Disability and secondary conditions • Educational and community-based programs • Environmental health • Family planning • Food safety • Health communication • Heart disease and stroke • HIV • • • • • • • • • • • • • • • Immunization and infectious disease Injury and violence prevention Maternal, infant, and child health Medical product safety Mental health and mental disorders Nutrition and overweight Occupational safety and health Oral health Physical activity and fitness Public health infrastructure Respiratory diseases Sexually transmitted diseases Substance abuse Tobacco use Vision and hearing Internet Resources • American Evaluation Association resources: http://www.eval.org/resources.asp • Centers for Disease Control and Prevention online evaluation textbook: http://www.cdc.gov/mmwr/preview/mmwrht ml/rr4811a1.htm • Western Michigan University Evaluation Center resources: http://ec.wmich.edu/resources/ • The Evaluation Exchange periodical: http://www.gse.harvard.edu/hfrp/eval.html