Educational Evaluation - PDF

Educational Evaluation  Evaluation Methods and Techniques  What is Evaluation Method? Evaluation methods are the criteria for evaluating the success of a program or project. Evaluation methods allow the donor a way to know if you’ve achieved your goals and objectives. Methods may be...

Educational Evaluation  Evaluation Methods and Techniques  What is Evaluation Method? Evaluation methods are the criteria for evaluating the success of a program or project. Evaluation methods allow the donor a way to know if you’ve achieved your goals and objectives. Methods may be qualitative or quantitative.  Quantitative- Focus on measuring and analyzing numerical data. Methods Qualitative- Focus on exploring the "why" and "how" within the topic. Why in process: Purpose of evaluation When in process: Formative vs. Summative Style of evaluation: laboratory vs. field Choosing an Type of measures: qualitative vs. Evaluation quantitative Method Level of information: high level vs. low level Resources available: time, subjects, tools & equipment, expertise etc. techniques of evaluation  01 Teaching Dossiers 02 Student Rating Techniques 03 Peer Observations of 04 Interviews Evaluation 05 Portfolio Classroom 06 Assessment  Teaching Dossier is a portfolio that summarizes an educator's teaching experience, including their teaching philosophy, approaches, strategies, and evaluations. Teaching dossiers typically include the teaching following components: dossiers Teaching philosophy statement Teaching strategies Teaching evaluations Sample teaching materials Future teaching goals Student ratings are a fundamental element of staff evaluation processes at many educational institutions. The ability to get direct feedback from students is important to administrators. The data these ratings provide can be an excellent tool for improving education and helping an student institution grow. rating Purpose of Student Ratings: Improving Teaching Quality Managing Human Resource Decisions Maintain Enrollment Peer assessment or peer review provides a structured learning process for students to critique and provide feedback to each other on their work. It helps students develop lifelong skills in assessing and peer providing feedback to others, and also observation equips them with skills to self-assess and improve their own work. Interviews These are a good way to find out how learners feel and how their behavior has changed. interviews Portfolio assessment A holistic way to evaluate a student's growth and development by collecting a record of their work in different contexts. portfolio Portfolios can be used as formative assessments throughout a course or as summative evaluations at the end of a program. Classroom assessment and evaluation are important for improving students' learning and knowledge. They also provide teachers with valuable information to help them improve their teaching methods. classroom Summative assessment A common method of evaluation that uses a assessment rubric or standard to grade a student's performance at the end of a program or course.  Creative assessments Non-traditional assessments that challenge students to present information in creative ways. Examples of creative assessments include concept mapping, prototyping, and podcasts. classroom Diagnostic assessment assessment A systematic way to collect information to evaluate a student's learning. Diagnostic assessments can be conducted at the beginning, during, or end of the teaching- learning process. Detailed list of methods, techniques and approaches for conducting evaluation  Evaluation Methods and Techniques  1.Action Research 2. Alternative Assessment 3. Case Study Evaluation 4. Feasibility Study Methods and 5. Field Study Techniques 6. Focus Group 7. Interview 8. Observational Technique 1. Action Research Action Research is a philosophy and methodology of research generally applied in the social sciences. It seeks transformative change through the simultaneous process of taking action and doing research, which are linked together by critical reflection.  Process of Action Research 1. Focus selection 2. Clarification of theories 3. Identification of research questions 4. Data collection 5. Data Analysis 6. Result reporting 7. Informed action Data collection Data analysis Result reporting  Process of Action Research 1. Focus selection 2. Clarification of theories 3. Identification of research questions 4. Data collection 5. Data Analysis 6. Result reporting 7. Informed action 2. Alternative Assessment Alternative Assessment in eduation is in direct contrast to what is known as "traditional testing" "traditional assessment", or "standardized assessment".  A Case studies are based on an in depth 3. investigation of a single individual group or event to explore the causes of underlying principles. Case studies are commonly used in social, educational, clinical, and business research. WHAT IS THE PURPOSE OF CASE STUDY? To understand complex issues To explore how problems are addressed in the real word To extract lessons and best practices for future use.  - WHAT’S INSIDE A CASE STUDY - BACKGROUND INFORMATION - Overview of the subject PROBLEM/ISSUE - Focuses on a specific challenge or issue that needs solving. ANALYSIS - Examines the causes and impact of the problem. SOLUTIONS AND OUTCOMES - Proposes solutions and evaluates the result. LESSON LEARNED - Key takeways or insights from the case. 4. Feasibility study A feasibility study is an evaluation and analysis of a project or system that somebody has proposed. The study tries to determine whether the project is technically and financially feasible.  Is the project technically Technical possible? Can the project be afforded? Some common Economic Will it increase profit? factors that are Legal Is the project legal? considered in How will the current a feasibility Operational operations support the change? study include: Can the project be done on Scheduling time?  5. FIELD STUDY Field study is an educational technique, which makes the educational process more active and help the students to work in real situations.  What do you want to learn or Problem find out? Is it observations, interviews, Method surveys, or experiments? PARTS Collecting OF A Data Gather the information directly. FIELD Analyzing Looking at the data to find STUDY Data patterns or trends. What did you find? What can Conclusion you recomment based on the findings? 6. Focus Group A focus group is a group interview involving a small number of demographically similar people or participants who have other common traits/experiences. Their reactions to specific researcher/evaluator-posed questions are studied. 7.INTERVIEW  An interview is essentially a structured conversation where one participant asks questions, and the other provides answers. Refers to a one-on-one conversation between an interviewer and an interviewee. The interviewer asks questions to which the interviewee responds, usually providing information.  ADVANTAGES AND DISADVANTAGES  Advantages It provides flexibility to the interviewers. It has better response rate than mailed questions. The interviewer can judge the non- verbal behavior of the respondent.  Disadvantages Conducting interview is very time- consuming Interview studies provide less anonymity There is a lack of accessibility to respondents. 8. Observational techniques Observation provides the opportunity to monitor or assess a process or situation and document evidence of what is seen and heard. Seeing actions and behaviors within a natural context, or as they usually occur provides insights and understanding of the event, activity or situation being evaluated. Advantages Observation is a flexible approach to data collection, suitable for a broad range of contexts. Observation can produce a mix of qualitative and quantitative data. Non-intrusive. Disadvantages There is a vast amount of data. Must have two or more people undertake observations. Conducting observations can be labour intensive. The key to using observational data as evidence in an evaluation is to take a systematic and consistent approach as you collect, organise and analyse what is observed. Importance of evaluation methods and techniques  CIPP Evaluation model The CIPP Model’s current version (Stufflebeam, Gullickson, Wingate, 2002) reflects prolonged effort to achieve the goal of developing a sound evaluation theory which includes a coherent set of conceptual, hypothetical, pragmatic, and ethical principle’s forming a general framework to guide the evaluation. The CIPP model is an acronym for Context, Input, Process, and Product. Each element represents a distinct type of evaluation that focuses on different aspects of a program or project.  CIPP MODEL The CIPP framework was developed as a means of linking evaluation with programme decision-making. It aims to provide an analytic and rational basis for programme CIPP decision-making, based on a cycle of planning, structuring, implementing and reviewing and revising decisions, each examined through a different aspect of evaluation context, input, process and product evaluation. The CIPP model is an attempt to make evaluation directly relevant to the needs of decision-makers during the phases and activities of a programme. Stufflebeam's context, input, process, and product (CIPP) evaluation model is recommended as a framework to systematically guide the conception, design, implementation, and assessment of service- learning projects, and provide feedback and judgment of the project's effectiveness for continuous improvement.  Four Aspects of CIPP Evaluation Model 1. Context Evaluation 2. Input Evaluation 3. Process Evaluation 4. Product Evaluation  context evaluation Context in CIPP refers to the group of considerations that centers on the needs assessment, available resources, problems to be solved and background issues and the overall environment of the program. Refers to as needs assessment NEEDS - a discrepancy between an existing condition and desired condition. Assess the extent to which participants accept and carry out their roles. Data which is collected over a period of time is useful in detecting strengths and weaknesses of the program.  Goals and Objectives Needs Evaluation of Context Problems Includes: Background Environment  goals and objectives The goal of context evaluation is to ensure that the curriculum is aligned with the needs and priorities of the students and that it is responsive to the broader social and cultural context. This includes examining factors such as the social, economic, and political context, as well as the needs and goals of their studies, the resources available, and the cultural and ethical values of their community. goals and objectives Identify needs, problems, assets, and opportunities. Develop goals, priorities, and objectives. Help formulate objectives.  The needs in context evaluation Beneficiaries: The needs of the beneficiaries Problems: The problems associated with the program Constraints: The constraints associated with the program Assets: The assets associated with the program Opportunities: The opportunities associated with the program Background: The background of the program Environment: The environment of the program  problems The objective of context evaluation is to define the relevant context, identify the target population and assess its needs, identify opportunities for addressing the needs, diagnose problems underlying the needs, and judge whether project goals are sufficiently responsive to the assessed needs.  Background It assesses the background and resources of a program, including the scope of the evaluation and the program's support. It also examines the needs of the program's participants, the expectations of stakeholders, and the assets and funding available.  environment In context evaluation, the environment in which a program operates is assessed, including the program's resources and background. This helps ensure that a curriculum is responsive to the community's needs and priorities. It also helps identify problems, assets, and opportunities within the community and environmental context.  Question for Context Evaluation (What needs to be done?) What is the relation of the course to other courses? Is the time adequate? What are critical or important external factors? Should courses be integrated or separate? What are the links between the course and research/extension activities? Is there a need for the course? Is the course relevant to the job needs?  Input evaluation Concerns judgement about resources and strategies needed to accomplish program goals and objectives Refers to the ingredients of the curriculum How it should be done? This includes the quality of: Plan Research Strategies Budget Stakeholders Facilities and infrastructures Coverage Question for Input Evaluation (How should it be done?) What is the entering ability of students? What are the learning skills of students? What is the motivation of the students? What are the living conditions of students? What is the students existing knowledge? Are the aims suitable? Do the objectives derives from aim? Are the objectives 'smart'? Is the course content clearly defined? Does the content match student abilities? Is the content relevant to practical problems? What is the theory/practice balance? What resources/equipment are available?  PROCESS EVALUATION Refers to the ways and means of how the curriculum has been implemented - Involves the collection of data once the program has been designed and put into operation Monitors the project implementation process. Assess the extent to which participants accept and carry out their roles. Data which is collected over a period of time is useful in detecting strengths and weaknesses of the program.  PROCESS INCLUDING ACTIONS - Planning Stage IMPLEMENTATION - Putting the Plan into Action MONITOR - Tracking Progress FEEDBACK - Evaluation and Improvement  Question for Process Evaluation (Is it being done?) What is the workload of the students? How well/actively do students participate? Are there any problems related to teaching? Are there any problems related to learning? Is there an effective 2-way communication? Is knowledge only transferred to students, or do they use and apply it?  Question for Process Evaluation (Is it being done?) Are there any problems which students face in using/ applying/analyzing the knowledge and skills? Are the teaching and learning process continuously evaluated? Are the teaching and learning affected by practical/institutional problems? What is the level of cooperation/interpersonal relations between teachers and students? How is discipline maintained?  Product Evaluation - To determine the extent to which the goals of the program have been achieved. - Measure, interpret, and judge a project’s outcomes by assessing their merit, worth, significance, and probity. As certain the extent to which the needs of all the participants were met. - Data collected can be used to make decisions about the program and making modifications for improvement. - This can be done through feedback from former students or graduates, employers and the society. Question for Product Evaluation (Did the project succeed?) - Is there one final exam at the end or several during the course? - Is there any informal assessment? - What is the quality of the assessment? -What are the students’ KSA levels after the course? -Is the evaluation carried out for the whole process? - How do students use what they learned? -How was the overall experience for the teachers and for the students? -What are the main ‘lessons learned’? -Is there an official report? -Has the teacher’s reputation improved or been ruined as a result?  using cipp in different stages in evaluation CONTEXT: What needs to be done VS Were important needs addressed? INPUT: How it should be done? VS Was a defensible design employed? PROCESS: Is it being done? VS Was the design well executed? PRODUCT: Is it succeeding? VS Did the effort succeed?  Thank you for listening!