HPPE Lec No.19 ‫‬.pdf

Health Program Planning and Evaluation Dr. Abdelsalam Mohamed Daoud Department of Epidemiology Faculty of Public Health Jazan University 1 Steps of Evaluation ‫توضيح ما يجب تقييمه‬ ‫ تطوير تقييم قابلية التقييم مع هدف )أهداف( محددة‬:‫الغرض‬ .‫ والنتائج‬،‫خرجات‬J‫ وا‬،‫نشطة‬M‫ وا‬،‫ستراتيجيات‬O‫...

Health Program Planning and Evaluation Dr. Abdelsalam Mohamed Daoud Department of Epidemiology Faculty of Public Health Jazan University 1 Steps of Evaluation ‫توضيح ما يجب تقييمه‬ ‫ تطوير تقييم قابلية التقييم مع هدف )أهداف( محددة‬:‫الغرض‬ .‫ والنتائج‬،‫خرجات‬J‫ وا‬،‫نشطة‬M‫ وا‬،‫ستراتيجيات‬O‫ وا‬،‫ والسكان‬،‫بوضوح‬ STEP 1: CLARIFY WHAT IS TO BE EVALUATED Purpose: to develop an evaluability assessment with clearly- defined goal(s), populations, strategies, activities, outputs, and outcomes. Step 1 addresses whether an evaluation should be carried out given available information and current context. Begin with a clear description of what is to be evaluated (e.g. program, policy, or health communication initiative) so that it can be shared. Consider developing a logic model that includes goals, objectives, activities, outputs, and outcomes. ‫صلحة‬%‫إشراك أصحاب ا‬ ‫صلحة‬%‫شاركة أصحاب ا‬%‫ل ا‬3‫ بناء الدعم للتقييم من خ‬:‫الغرض‬ STEP 2: ENGAGE STAKEHOLDERS Purpose: to build support for the evaluation by engaging stakeholders This step involves identifying key stakeholders, understanding their interests and expectations, and engaging them in a review of objectives to develop evaluation questions. Consider making a list of organizations and/or people with an interest in the evaluation. Define stakeholder information needs and intended use of the evaluation. ‫وارد وقابلية التقييم‬0‫تقييم ا‬ .‫تاحة للتقييم وما إذا كان البرنامج جاهز للتقييم‬0‫وارد ا‬0‫ تقييم ا‬:‫الغرض‬ STEP 3: ASSESS RESOURCES AND EVALUABILITY Purpose: to assess available resources for evaluation and whether the program is ready to be evaluated. Evaluation can be time-consuming and expensive. Do an honest assessment of resources to avoid constraints later. Resources include funds, time, in-kind support, approval processes (e.g., ethics) and timeline for implementation and completion. When assessing evaluability consider whether: there is clarity on the program to be evaluated and why you are doing an evaluation the evaluation will be useful leadership buy-in is high and will shape programs adequate resources are available timing is right First, use this information to make a decision about whether conducting the evaluation is feasible and necessary. Then, use the information to further define your evaluation. ‫تحديد أسئلة التقييم الخاصة بك‬ .‫سئلة لتلبية أهداف التقييم‬8‫ للمساعدة في تحديد وتحديد ا‬:‫الغرض‬ STEP 4: DETERMINE YOUR EVALUATION QUESTIONS Purpose: to help identify and determine questions to meet your evaluation goals. Involve as many stakeholders as possible to ensure all needs are met. Consider: logic model contents, stage of development (e.g., planning, implementation, or completion stage), evaluations already completed, decisions needed, stakeholder interests, and resources. Consider type of evaluation needed. Formative evaluation is typically used in development and planning of a strategy. Process evaluation addresses the extent to which an intervention is implemented as planned and reaching intended populations. Outcome evaluation measures a program’s success in meeting its goals and objectives. ‫جراءات ا&ناسبة‬,‫تحديد طرق القياس وا‬ .‫ستخدامها وكيف سيتم جمع البيانات‬B ‫كثر جدوى ومصداقية‬G‫ تحديد الطرق ا‬:‫الغرض‬ STEP 5: DETERMINE APPROPRIATE METHODS OF MEASUREMENT AND PROCEDURES Purpose: to identify the most feasible and credible methods to use and how data will be collected. Consider: purpose, evaluation questions, feasibility of data access, reliability of data, who will use data, stakeholder expectations, and what is already being captured. Develop a data collection plan that includes: what to measure (indicators), when to collect data (before/after/both), how to collect data (qualitative/quantitative/both) and from whom to collect data (specific subgroups/representative sample of population of interest). Think about the ethical issues involved in data collection (e.g. anonymity of data, confidentiality, and informed consent) and take appropriate steps. ‫وضع خطة التقييم‬ .‫واعيد النهائية‬4‫ وا‬،‫وارد‬4‫ تخصيص ا‬،‫ تحديد أنشطة تقييم محددة ومهام وأدوار‬:‫الغرض‬ STEP 6: DEVELOP EVALUATION PLAN Purpose: to identify specific evaluation activities, tasks, roles, resource allocations, and deadlines. Evaluation plans detail how programs, policies, or health communication initiatives will be monitored and evaluated, and how results will be used, providing transparency to stakeholders and funders. A data collection matrix can be part of your evaluation plan. A matrix generally includes information gathered from previous steps: evaluation questions and link to logic model (if applicable), indicators, methods, data sources, timelines, roles and responsibilities, and how data will be analyzed (optional). In Step 9 you will develop your dissemination plan, which can be added to the evaluation plan once completed. ‫جمع البيانات‬ .‫ تعتمد النتائج والتوصيات على جودة البيانات‬- ‫جابة على كل سؤال تقييم‬C‫ جمع أدلة موثوقة ل‬:‫الغرض‬ STEP 7: COLLECT DATA Purpose: to collect credible evidence to answer each evaluation question - results and recommendations depend upon data quality. Develop data collection tools (survey, interview guide, etc.) and procedures and train data collectors. Consider whether incentives are appropriate and brainstorm ways to enhance response rates. To ensure validity, pilot test tools and procedures and closely monitor data gathered. If issues arise, modify tools and procedures and document changes. Computerize data collection to facilitate analysis if appropriate. ‫معالجة البيانات وتحليل النتائج‬ ‫ إدخال البيانات والتحقق من جودة البيانات واتساقها إدخال البيانات‬:‫الغرض‬ .‫وتحليلها لتحديد نتائج التقييم الخاصة بك‬ STEP 8: PROCESS DATA AND ANALYZE RESULTS Purpose: to enter data, check quality and consistency of data entry and analyze data to identify your evaluation results. Implement strategies to review data quality during and after data collection. During data collection, look closely at the first wave of responses and number of ‘no response’ or refusals, and keep data collectors and the evaluation lead connected. After data collection, enter data and double check quality and consistency of entry, sort to find missing, high or low values (quantitative), and check content by reviewing transcripts entered (qualitative). Organize data in a format that can be summarized and interpreted. Analyze by conducting statistical analysis of quantitative data; identify themes in qualitative data. This is a technical step - enlist expert support when possible. This sets the stage for interpretation. ‫تفسير النتائج وتوزيعها‬ ‫ وإشراك‬،‫ تفسير نتائج التقييم الخاصة بك ومشاركتها‬:‫الغرض‬ .‫ساعدة في تحديد توصيات‬L‫صلحة حتى يتمكنوا من ا‬L‫أصحاب ا‬ STEP 9: INTERPRET AND DISSEMINATE RESULTS Purpose: to interpret and share your evaluation findings, engaging stakeholders so that they can help identify recommendations. Anchor the interpretation to the original evaluation questions. Create a list of recommended actions that address your outcomes, and use this information to create the materials to communicate your findings. Presentation of findings can take many forms such as a written report, slide show presentation, and/or as a short informational video. Visual aids can be powerful methods for communicating evaluation results. Make results available to various stakeholders and audiences. Tailor what is disseminated to their specific interest in the evaluation and how they plan to use the results. ‫تطبيق نتائج التقييم‬ .‫ استخدام نتائج التقييم الخاصة بك‬:‫الغرض‬ STEP 10: APPLY EVALUATION FINDINGS Purpose: to use your evaluation results. Review recommendations with stakeholders to identify actionable outcomes and discuss what has been learned from conducting the evaluation and next steps to incorporate results. Prioritize actions and develop an action plan as a group. Consider evaluating your evaluation (meta-evaluation). You may ask stakeholders to reflect on the process and/or the outcomes, to improve the process moving forward. ‫ملخص خطوات التقييم‬ Summary of Steps of Evaluation 1. Assess feasibility of doing evaluation (Feasibility Assessment). 2. Develop evaluation questions. 3. Select evaluation design and methods. 4. Collect evaluation data. .(‫ تقييم جدوى إجراء التقييم )تقييم الجدوى‬.1 5. Analyze evaluation data. .‫ تطوير أسئلة التقييم‬.2 .‫ حدد تصميم التقييم وأساليبه‬.3 6. Disseminate findings. .‫ جمع بيانات التقييم‬.4 .‫ تحليل بيانات التقييم‬.5 .‫ نشر النتائج‬.6