Evaluation of Education Program PDF

Evaluation of education program Dr. Nahla Shehabelden Copyright © 2023 by Jones & Bartlett Learning, LLC an Ascend Learning Company. www.jblearning.com. Learning Outcome Define the Evaluation List the steps of evaluation Apply the Evaluation Component on Types...

Evaluation of education program Dr. Nahla Shehabelden Copyright © 2023 by Jones & Bartlett Learning, LLC an Ascend Learning Company. www.jblearning.com. Learning Outcome Define the Evaluation List the steps of evaluation Apply the Evaluation Component on Types of Evaluation  Outline - Definition - The purpose of program evaluation - Steps of program evaluation - Evaluation methods - Five Components of Evaluation Focus - Types of evaluation - Internal versus external evaluator - Barriers to evaluation - Potential problems that may block an effective evaluation  Definition of evaluation Evaluation is a process that critically examines a program. It involves collecting and analyzing information about a program’s activities, characteristics, and outcomes. 1. The purpose of evaluation *To identify ways to improve a program 2. To assessing needs of intended population 3. To improving the usefulness of program materials 4. *To determining what works, and what doesn’t works 5. To collect evidence on the effectiveness of the program 6. *To be accountable 7. *To assess efficiency  Steps of program evaluation Step 1: Clarify the evaluated program Step 2: Engage Stakeholders Step 3: Assess Resources for the Evaluation Step 4: Design the Evaluation Step 5: Determine Appropriate Methods of Measurement and Procedures  Steps of program evaluation Step 6: Develop Work Plan, Budget and Timeline for Evaluation Step 7: Collect the Data Using Agreed- upon Methods and Procedures Step 8: Process and Analyze the Data Step 9: Interpret and Disseminate the Results Step 10: Take Action  Activity Zone: Steps of program evaluation Statement Steps Collect the Data A- Step 2 Take Action B- Step 7 Engage Stakeholders C- Step 10 Design the Evaluation D- Step 1 Clarify the evaluated program E- Step 4  Activity Zone: Steps of program evaluation Statement Steps Collect the Data A- Step 2 B Take Action B- Step 7 C Engage Stakeholders A C- Step 10 Design the Evaluation E D- Step 1 Clarify the evaluated program D E- Step 4  Evaluation Methods What data to collect and from whom O Collect at least some data directly from the individuals being evaluated. O Collect data from sample, if necessary. O Use preexisting data for evaluative purposes only.  Evaluation Methods Tools used in evaluation 1. Observation 2. Pre and post tests 3. Oral questions 4. Self-report questionnaires 5. Interview questionnaires 6. Satisfaction surveys 7. Attendance sheets  Evaluation Methods O Who collects data O Learner O Educator O Evaluator (Internal& external) O Trained data collector  Five Components of Evaluation Focus on 1. Audience: Persons or groups for whom the evaluation is being conducted 2. Purpose: Why an evaluation is being conducted 3. Questions: Specific, measurable questions related to the evaluation purpose 4. Scope: The extent of what is being examined 5. Resources: Time, expertise, personnel, materials, equipment, and facilities Application of Evaluation Component on Types of Evaluation Copyright © 2023 by Jones & Bartlett Learning, LLC an Ascend Learning Company. www.jblearning.com.  Process (1-Formative) Evaluation O Purpose: To make necessary adjustments as soon as needed during educational process O Question: How can teaching improve to better facilitate learning? O Scope: Limited to specific learning experience O Resources: Usually less costly and more readily available  (2- Content) Evaluation O Purpose: To determine whether learners have acquired knowledge/skills just taught O Question: To what degree did learners achieve specified objectives? O Scope: Limited to specific learning experience and stated objectives O Resources: Funding to engage an external evaluator or evaluation team or pay for specific tasks  Outcome (3-Summative) Evaluation O Purpose: To determine effects of teaching efforts O Questions: O Was teaching appropriate? O Did the individual(s) learn? O Were behavioral objectives met? O Scope: Broader; occurs after teaching completed O Resources: Costly and complex  (4- Impact) Evaluation O Purpose: To determine long- term effects of education on the institution or community O Question: What is the effect of education on long-term changes at the organizational or community level? O Scope: Broad, complex, sophisticated, and usually long-term O Resources: Extensive, resource intensive  (5- Total) Program Evaluation O Purpose: To determine extent to which total program meets or exceeds established goals O Question: To what extent did all program activities meet annual departmental/ institutional/community goals? O Scope: Broad, long-term/complex, and strategic O Resources: Resource intensive  Activity Zone: types of evaluation Purpose Types To determine whether learners have acquired A- Formative knowledge/skills just taught To determine extent to which program meets B- Content or exceeds established goals To make necessary adjustments as soon as C- Total program needed during educational process To determine long- term effects of education on D- Summative the institution or community To determine effects of teaching efforts E- Impact  Activity Zone: types of evaluation Purpose Types To determine whether learners have acquired A- Formative knowledge/skills just taught B To determine extent to which program meets B- Content or exceeds established goals C To make necessary adjustments as soon as C- Total program needed during educational process A To determine long- term effects of education on D- Summative the institution or community E To determine effects of teaching efforts D E- Impact  Internal versus external program evaluators O Evaluators may be internal (persons associated with the program to be executed) or external (Persons not associated with any part of the execution/implementation of the program).  Internal evaluators Advantages 1. May have better overall knowledge of the program and possess informal knowledge of the program 2. Less threatening as already familiar with staff 3. Less costly  Internal evaluators Disadvantages 1. May be less objective 2. May be more preoccupied with other activities of the program and not give the evaluation complete attention 3. May not be adequately trained as an evaluator.  External evaluators Advantages 1. More objective of the process, offers new perspectives, different angles to observe and critique the process 2. May be able to dedicate greater amount of time and attention to the evaluation 3. May have greater expertise and evaluation brain  External evaluators Disadvantages 1. May be more costly and require more time for the contract, monitoring, negotiations etc. 2. May be unfamiliar with program staff and create anxiety about being evaluated 3. May be unfamiliar with organization policies, certain constraints affecting the program.  Barriers to Evaluation 1- Lack of clarity 2- Lack of ability 3- Fear of punishment or loss of self- esteem  Overcoming Barriers to Evaluation 1- Lack of clarity O Clearly describe all five evaluation focus components. O Operationally define terms. O Clearly state the purpose. 2- Lack of ability O Make necessary resources available. O Ask for help from experts.  Overcoming Barriers to Evaluation 3- Fear of punishment or loss of self-esteem O Be aware of the existence of fear. O Focus on data and results without personalizing or blaming. O Point out achievements and be supportive. O Communicate the evaluation purpose to participants. O Reassure participants of privacy and legal protection.  Potential problems that may block an effective evaluation 1. The planners failed to build evaluation into program planning. 2. Inadequate procedures( cost, time, and resources). 3. *Changes in adults come slowly. 4. *Some changes do not last. 5. *Conflict can arise between professional standards. Any Question ‹#›