Curriculum Evaluation PDF - Topic 8

Topic 8 Curriculum Evaluation By the end of this topic, you should be able to do the following: 1. State what curriculum evaluation is. 2. List reasons for evaluating the curriculum. 3. Explain the characteristics of the CIPP model....

Topic 8 Curriculum Evaluation By the end of this topic, you should be able to do the following: 1. State what curriculum evaluation is. 2. List reasons for evaluating the curriculum. 3. Explain the characteristics of the CIPP model. 4. Describe the features of Stake's model of curriculum evaluation. 5. Explain the characteristics of Eisner's connoisseurship model. 6. Compare the different instruments of data collection. Hak Cipta © Open University Malaysia (OUM)  TOPIC 8 CURRICULUM EVALUATION 213 In Topic 7, we discussed the implementation of the curriculum plan. We looked at why people resist change and the role of teachers, students, administrators, and parents in ensuring the successful implementation of change. This topic will determine whether the implemented curriculum plan has achieved its goals and objectives. Various stakeholders want to know the extent to which the curriculum has been successfully implemented. In other words, the curriculum has to be evaluated to determine whether all the efforts in finance and human resources have been worthwhile. The information collected from evaluating a curriculum forms the basis for making judgements about how successfully the programme has achieved its intended outcomes and the worth or value of the programme. Hak Cipta © Open University Malaysia (OUM) 214 TOPIC 8 CURRICULUM EVALUATION ACTIVITY 8.1 Read the newspaper report „DonÊt Make Physical Education (PE) an Examination Subject‰ and answer the following questions. DonÊt Make Physical Education (PE) an Examination Subject The Ministry of Education has The sporting calendar for Term 1 asked ministry officials to look (January to April) had football, into introducing physical athletics, cross country. In term 2 education (PE) as an examination (May to August), it was athletics subject. I think PE should not be an and cricket. In term 3 (September exam subject. to December) it was hockey and rugby. As for the court games, In the 1950s, 1960s, 1970s and they were played all around the 1980s, PE was never an exam year. subject and yet the country produced world-class sportmen There were inter-house games and and sportwomen in badminton, if your school has six houses weightlifting, hockey, athletics would paly at least five matches and other events. People like for your house. There were inter- Jegathesan, Mokhtar Dahari, Tan school gamesare extinct and even Aik Huang, Rajamani, Ng Boon they do have them, it is on a knock- Bee, Nurul Huda, Marina Chin, out basis. It is the same for inter- Karu Selvaratnam, Nastar Singh, school games. Zaiton Sulaiman, Ghani Minhat, Tan Aik Mong, Dhanapal Naidu, Sporting activities have become a and many others. burden for schools. There is little organisation and the faster they We had no sports schools in are over, the better. The school those days. All schools were saves money and teachers have sport schools. How did we more time for completing the produce excellent sportmen and syllabus and revision in sportwomen? We had preparataion for national supportive parents, interested examinations. headmaster, dedicated and comitted PE teachers, coachers – Retired Physical Education and disciplined sportmen and Teacher sportwomen. Source: Letters to the Editor, New Straits Times, February 1, 2005 Hak Cipta © Open University Malaysia (OUM)  TOPIC 8 CURRICULUM EVALUATION 215 (a) Do you think physical education should be made an „examination subject‰? (b) Do you agree with the writer's opinions on the state of sports in schools? Justify your answers to your coursemates in the myINSPIRE online forum. 8.1 CURRICULUM EVALUATION What is evaluation? Evaluation is collecting data on a programme to determine its value or worth to decide whether to adopt, reject or revise it. Programmes are evaluated to answer the questions and concerns of various parties. The public wants to know whether the curriculum implemented had achieved its aims and objectives. Teachers want to know whether what they are doing in the classroom is effective, and the developer or planner wants to know how to improve the curriculum product. Table 8.1 lists some definitions of curriculum evaluation from scholars. Table 8.1: Definitions of Curriculum Evaluation Source Definition McNeil Curriculum evaluation attempts to throw light on two questions. (1977) How can the curriculum offerings best be improved? Do planned learning opportunities, programmes, courses and activities, as developed and organised, produce desired results? Ornstein and Curriculum evaluation as a process or cluster of processes people Hunkins perform to gather data. This process can allow them to decide (1998) whether to accept, change or eliminate something – the curriculum in general or an educational textbook in particular. Worthen and Curriculum evaluation as „the formal determination of the quality, Sanders effectiveness or value of a programme, product, project, process, (1987) objective or curriculum‰ (p. 22–23). Hak Cipta © Open University Malaysia (OUM) 216 TOPIC 8 CURRICULUM EVALUATION Fox and Gay Curriculum evaluation implementation aims to identify its (1995) weaknesses, strengths and problems encountered to improve the curriculum development process; to determine the effectiveness of the curriculum and the returns on finance allocated. Oliva Curriculum evaluation as delineating, obtaining and providing (1988) useful information for judging decision alternatives. Based on the evaluation results, the alternatives are to maintain the curriculum, modify it or eliminate it. Evaluation is a disciplined inquiry to determine the worth of things. „Things‰ may include programmes, procedures or objects. Generally, research and evaluation are different, even though similar data collection tools may be used. The three dimensions on which they may differ are: First, evaluation need not have as its objective the generation of knowledge. Evaluation is applied, while research tends to be basic. Second, evaluation, presumably, produces information that is used to make decisions or forms the basis of policy. Evaluation yields information that has immediate use, while research needs not. Third, evaluation is a judgement of worth. The evaluation results in value judgements while research need not, and some would say it should not. FORMATIVE AND SUMMATIVE 8.2 EVALUATION As mentioned earlier, evaluation is the process of determining the significance or worth of programmes or procedures. Scriven (1967) differentiated evaluation as formative evaluation and summative evaluation. However, they have come to mean different things to different people. Still, in this topic, ScrivenÊs original definition will be used. Hak Cipta © Open University Malaysia (OUM)  TOPIC 8 CURRICULUM EVALUATION 217 8.2.1 Formative Evaluation The term formative indicates that data is gathered during the formation or development of the curriculum so that revisions to it can be made. In education, the aim of formative evaluation is usually to obtain information to improve a programme. Formative evaluation determines who needs the programme (e.g. secondary school students). It is also to determine how great the need is (e.g. students need to be taught ICT skills to keep pace with the expansion of technology). Further, it determines how to meet the need (e.g. introducing a subject on ICT is made compulsory for all secondary school students). In formative evaluation, experts would evaluate the match between the instructional strategies and materials used and the learning outcomes or what it aims to achieve. For example, the possibility that the learning outcomes and activities in a curriculum plan do not match. You want students to develop critical thinking skills. Still, no learning activities provide opportunities for students to practise critical thinking. Formative evaluation by experts is useful before the full-scale implementation of the programme. Review by experts on the curriculum plan may provide useful information for modifying or revising selected strategies. In formative evaluation, learners may be included to review the materials to determine if they can use the new materials. Problems may be discovered in the formative reviews, such as the curriculum document may contain spelling errors, confusing sequence of content, and inappropriate examples or illustrations. The feedback obtained could be used to revise and improve instruction or to decide whether or not to adopt the programme before full implementation. Robert “When the cook tastes the soup, that's formative; Stakes when the guests taste the soup, that's summative.” 8.2.2 Summative Evaluation The term summative indicates that data is collected at the end of the implementation of the curriculum programme. Summative evaluation can occur just after the completion of new course materials (i.e. evaluate the programmeÊs effectiveness). It can also be done several months to years after the materials have been fully implemented. It is important to specify what questions you want to be answered by the evaluation and what decisions will be made due to the evaluation. You may want to know if programme learning outcomes or the course learning outcomes have been achieved. For example, Hak Cipta © Open University Malaysia (OUM) 218 TOPIC 8 CURRICULUM EVALUATION the use of specific simulation software in the teaching of geography enhanced the decision-making skills of learners. These outcomes can be determined through formal assessment tasks such as marks obtained in tests and examinations. Also of concern is whether the innovation was cost-effective. Was the innovation efficient in terms of time to completion? Were there any unexpected outcomes? Besides quantitative data to determine how well students met specified objectives, data could also include qualitative interviews, direct observations and document analyses. SELF-CHECK 8.1 1. Identify the keywords in the five definitions of curriculum evaluation. 2. Why do you need to evaluate the curriculum? 3. What is the difference between formative and summative evaluation? 8.3 CURRICULUM EVALUATION MODELS How should you go about evaluating the curriculum? Several experts have proposed different models describing how and what should be involved in evaluating a curriculum. Models are useful because they help you define the parameters of an evaluation, what concepts to study, and the procedures to extract important data. Numerous evaluation models have been proposed, but three models are discussed here. Hak Cipta © Open University Malaysia (OUM)  TOPIC 8 CURRICULUM EVALUATION 219 8.3.1 Context, Input, Process and Product Model (CIPP Model) Stufflebeam (1971), who chaired the Phi Delta Kappa National Study Committee on Evaluation, introduced a widely cited evaluation model known as the CIPP (context, input, process and product) model. When applied to education, the approach aims to determine if a particular educational effort has resulted in a positive change in school, college, university or training organisation. A major aspect of StufflebeamÊs model is centred on decision-making or making up oneÊs mind about the programme introduced. For evaluations to be done correctly and aid in the decision-making process, curriculum evaluators have to: Delineate what is to be evaluated and determine what information has to be collected (e.g. how effective has the new science programme been in enhancing the scientific thinking skills of children in the primary grades). Obtain or collect the information using selected techniques and methods (e.g. interview teachers, collect test scores of students). Provide or make available the information (in the forms of tables and graphs) to interested parties. This process decides whether to maintain, modify or eliminate the new curriculum or programme. Information is obtained by conducting the following four types of evaluation: context, input, process and product (refer to Table 8.2). Table 8.2: Types of Evaluation Type Description Context This process is the most basic evaluation process to provide (What needs to a rationale for the objectives. The evaluator defines the be done and in environment in which the curriculum is implemented: what context)? a classroom, school or training department. The evaluator determines needs that were not met and reasons why the needs are not being met. Also, the evaluators identify any shortcomings and problems in the organisation. For example, a sizable proportion of secondary school students cannot read at the desired level. Other examples include the ratio of students to computers and a sizable proportion of science teachers who are not proficient in teaching English. Goals and objectives are specified based on context evaluation. In other words, the evaluator determines the background in which the innovations are being implemented. Hak Cipta © Open University Malaysia (OUM) 220 TOPIC 8 CURRICULUM EVALUATION The data collection techniques would include observation of conditions in the school, background statistics of teachers and interviews with players involved in implementing the curriculum. Input This process provides information for determining how to (How should it utilise resources to achieve the objectives of the curriculum. be done?) The schoolÊs resources and various designs for carrying out the curriculum are considered. At this stage, the evaluator decides on the procedures to be used. Unfortunately, methods for input evaluation are lacking in education. The prevalent practices include committee deliberations, appeal to the professional literature, consultantsÊ employment and pilot experimental projects. Process This stage provides periodic feedback while the curriculum (Is it being is being implemented. done?) Product Product evaluation involves measuring the achievement of (Did it succeed?) objectives, interpreting the data and providing information that will enable them to decide whether to continue, terminate or modify the new curriculum. Data is collected to determine whether the curriculum accomplished what it set out to achieve (e.g. to what extent students have developed a more positive attitude towards science). For example, product evaluation might reveal that students have become more interested in science and are more positive after introducing the new science curriculum. Based on these findings, the decision may be made to implement the programme throughout the country. Hak Cipta © Open University Malaysia (OUM)  TOPIC 8 CURRICULUM EVALUATION 221 StufflebeamÊs evaluation model relies on both formative and summative evaluation to determine the overall effectiveness of a curriculum programme (refer to Figure 8.1). Evaluation is required at all levels of the programme implemented. Figure 8.1: StufflebeamÊs Evaluation Model SELF-CHECK 8.2 1. What is the difference between context and input evaluation according to the CIPP model? Give specific examples. 2. What is the difference between process and product evaluation according to the CIPP model? Give specific examples. 8.3.2 Case Study: Evaluation of a Programme on Technology Integration in Teaching and Learning in Secondary Schools The integration of information and communication technology (ICT) in teaching and learning is growing rapidly in many countries. The Internet and other computer software used in teaching science, mathematics and social sciences are more widespread today. To evaluate the effectiveness of such a programme using the CIPP model would involve examining the following (refer to Table 8.3): Hak Cipta © Open University Malaysia (OUM) 222 TOPIC 8 CURRICULUM EVALUATION Table 8.3: Evaluation of the Effectiveness of a Programme Type Description Context Examine the environment in which technology is used in teaching and learning. How did the real environment compare to the ideal? (E.g. The programme required five computers in each classroom, but there were only two computer labs of 40 units each for 1,000 students). What problems are hampering the success of technology integration? (E.g. technology breakdowns, not all schools had internet access). About 50% of teachers do not have basic computer skills. Input Examine what resources are put into technology integration (identify the educational strategies most likely to achieve the desired result). Is the content selected for using technology right? Have we used the right combination of media? (The Internet, video clips, etc.). Process Assess how well the implementation works (uncover implementation issues). Did technology integration run smoothly? Were there technology problems? Were teachers able to integrate technology into their lessons as planned? What are the areas of the curriculum in which most students experienced difficulty? Product This process addresses learning outcomes (gather information on the results of the educational intervention to interpret its worth and merit). Did the learners learn using technology? How do you know? Does technology integration enhance higher-order thinking? Hak Cipta © Open University Malaysia (OUM)  TOPIC 8 CURRICULUM EVALUATION 223 ACTIVITY 8.2 Regarding Case Study 8.3.2: (a) Suggest other questions you would ask regarding process evaluation (i.e. implementation issues). (b) What data collection techniques would you recommend for product evaluation to determine the teaching and learning outcomes of technology integration? Share your answers to your coursemates in the myINSPIRE online forum. 8.3.3 Stake’s Countenance Model The model proposed by Robert Stake (1991) suggests three phases of curriculum evaluation: the antecedent phase, the transaction phase and the outcome phase (refer to Figure 8.2). The antecedent phase includes conditions before the instruction that may relate to outcomes. The transaction phase constitutes the instruction process, while the outcome phase relates to the programmeÊs effects. Stake emphasises two operations: descriptions and judgements. Descriptions are divided according to whether they refer to what was intended or what was observed. Judgements are separated according to whether they refer to standards used in arriving at the judgements or the actual ones. Figure 8.2: StakeÊs Countenance Model Hak Cipta © Open University Malaysia (OUM) 224 TOPIC 8 CURRICULUM EVALUATION 8.3.4 Eisner’s Connoisseurship Model Elliot Eisner (1976), a well-known art educator, argued that learning was too complex to be broken down into objectives and measured quantitatively. He argued that teaching small manageable pieces of information prohibits students from putting the pieces back together and applying them to new situations. As long as we evaluate students based on the small bits of information, we will only learn small bits. Eisner contends that evaluation has and will always drive the curriculum. Suppose we want students to be able to solve problems and think critically. We must evaluate problem solving and critical thinking, skills that rote practice alone cannot teach. So, to evaluate a programme, we must attempt to capture the richness and complexity of classroom events. He proposed the connoisseurship model, which claimed that a knowledgeable evaluator could determine whether a curriculum programme has been successful using a combination of skills and experience. The word „connoisseurship‰ comes from the Latin word cognoscere, meaning to know. For example, to be a connoisseur of food, paintings or films, you must have knowledge about and experience with different types of food, paintings or films before you can criticise. To be a food critic, you must be a connoisseur of different kinds of foods. To be a critic, you must be aware of and appreciate the subtle differences in the phenomenon you are examining. In other words, the curriculum evaluator must seek to be an educational critic. When employing the procedure of educational criticism, the following questions may be asked: What has happened in the classrooms due to implementing the new curriculum? What are some of the events that took place? (E.g. more students are participating in fieldwork, more students are asking questions in class, and even academically weak students are talking in group activities) How did students and teachers organise themselves in these events? What were the reactions of participants in these events? (E.g. students enjoyed working collaboratively on projects) How can learnersÊ experiences be more effective as students, teachers and administrators suggested? (E.g. more resources are needed for fieldwork; more computers are needed to integrate the Internet into teaching and learning). Hak Cipta © Open University Malaysia (OUM)  TOPIC 8 CURRICULUM EVALUATION 225 You will notice that these questions emphasise the learning process and the quality of experiences by those involved in implementing the curriculum, namely, students, teachers and administrators. According to the connoisseurship model, evaluators provide a description and interpretation of the curriculum plan implemented: Description The evaluator records the actions, the features of the environment and the experiences of students, teachers, and administrators. People who read the evaluation report can visualise what the place looks like and the processes taking place. The aim here is to help the reader „see‰ the school or classroom and get a feel of what the curriculum evaluator or critic is attempting to understand and help others understand. Interpretation The evaluator explains the meaning of reported events by putting them in context. For example, why academically weak students were motivated to ask questions, why reading comprehension skills improved, why enthusiasm for doing science experiments increased and so forth. To be able to describe and interpret the implementation of a curriculum, the evaluator has to collect data, and the following are examples of activities an evaluator may engage in: The evaluator observes what is going on in the classroom and records teachers and students in action using videotapes, audiotapes, and photographs. The evaluator keeps notes of what is done, what is said and, more importantly, what is not said. The evaluator should strive to describe the tone of the curriculum in action (Ornstein & Hunkins, 1998). The evaluator interviews students, teachers, and administrators about the quality of the curriculum. The evaluator would analyse the studentÊs work. Hak Cipta © Open University Malaysia (OUM) 226 TOPIC 8 CURRICULUM EVALUATION One of the great benefits of EisnerÊs (1976) activities has been how he has made a case for concern with connoisseurship and criticism, and mediated these concerns for educators and researchers. The importance of his advocacy of these ideas cannot be underestimated, especially when rather narrow concerns with instrumental outcomes and an orientation to the technical dominant. Together they offer educators a more practical and appropriate means to approach evaluation, for example: Advocate moving beyond technocratic and behaviouristic modes of thinking and for having a concern for „expressive outcomes‰. Call to attend to fundamentals. Eisner (1976) has consistently warned against educational fads and fashion. He has criticised dominant paradigms and invited educators and others to ask questions, such as „what is basic in education?‰ Arguing that schools should help children create meaning from experience requires an education devoted to the senses, meaning-making and imagination. Eisner (1976) argues for a curriculum that fosters multiple „literacies‰ in students (especially by looking at non-verbal modes of learning and expression) and a deepening of the „artistry‰ of teachers. Over the time Eisner (1976) has been writing, there were significant shifts in the school context. However, globalisation has caused resistance to introducing school centralisation, the pedagogy process and instrumentalisation. According to Eisner, the implementation process was difficult due to the globalisation impact. Other voices have called for changes in the culture of schooling (notably by Howard Gardner in this arena). Hak Cipta © Open University Malaysia (OUM)  TOPIC 8 CURRICULUM EVALUATION 227 8.4 PHASES OF CURRICULUM EVALUATION Figure 8.3 shows the phases of curriculum evaluation. Figure 8.3: Phases of Curriculum Evaluation INSTRUMENTATION FOR CURRICULUM 8.5 EVALUATION No matter what evaluation model is used in evaluating a curriculum, the methods of data collection and the instruments used are more or less similar. The common instruments used in curriculum evaluation are interviews, observations, tests, surveys, content analysis, and portfolios (records of work or products). Hak Cipta © Open University Malaysia (OUM) 228 TOPIC 8 CURRICULUM EVALUATION 8.5.1 Questionnaires and Checklists When you need to quickly or easily get lots of information from people in a non-threatening way, questionnaires and checklists are useful data collection techniques. Questionnaires and checklists can complete anonymously and are relatively inexpensive to administer. Since the data collected is quantitative, it is easy to compare and analyse and can be administered to many people. A massive amount of data can be obtained. It is also easy to design as there are many sample questionnaires already in existence. However, the information obtained may not be accurate as it relies on how truthfully subjects respond to the questions. There is also the fear that wording can bias client responses. Questionnaires are impersonal, and since only a sample of subjects is given the instrument, we do not get the full story. 8.5.2 Interviews Interviews are usually one-on-one in which an individual asks questions to which a second individual (a teacher, principal, student, or parent) responds. The person asking the questions is called the interviewer, while the person giving answers to the questions is called the interviewee. Interviews are used when you want to fully understand someoneÊs impressions or experiences or learn more about their answers to questionnaires. There are two general types of interviews depending on the extent to which the responses required are unstructured or structured. In an unstructured interview, the interviewer does not follow a rigid script, and the responses have great flexibility. For example, „Why do you think the recommended textbook for the course is difficult for low-ability learners? The teacher responding to such a question will give a variety of reasons. Some reasons may be general, while others may be specific to certain textbook sections. This process makes the task of keeping track of responses more difficult. The open-ended questions will require the interviewer to record all responses and make sense of it later. The advantage of the unstructured interview is that it allows the evaluator to gather various information, especially about the intervieweeÊs knowledge, beliefs or feelings towards a particular situation. Hak Cipta © Open University Malaysia (OUM)  TOPIC 8 CURRICULUM EVALUATION 229 While in a structured interview, the questions asked usually require very specific responses. For example, „Is the recommended textbook difficult for low-ability learners because: (a) there is too much content? (b) the language used is beyond the comprehension of low-ability learners? or (c) there are too few examples and illustrations?‰ Regardless of the interview type, evaluators should ensure that each question is relevant for its intended purpose. Ultimately, the data must be translated into a form that can be analysed. This process has to be done carefully to preserve the accuracy and maintain the dataÊs sense. The advantage of interviews is that they can get a full range and depth of information, develop a relationship with teachers and students, and be more flexible. However, an interview can take much time, can be hard to analyse and compare, can be costly, and the interviewer can be bias against the clientÊs responses. 8.5.3 Observations Observations are about gathering accurate information about how a programme operates, in particular about the processes. These processes include viewing the operations of a programme as they are occurring that can be adapted to events as they occur. It can be not easy to interpret seen behaviours, complex to categorise observations, influence programme participantsÊ behaviours and expensive. 8.5.4 Documentation Reviews Get comprehensive and historical information when you want an impression of how the programme operates without interrupting the programme, from a review of applications, finances, memos, minutes, etc. The following are some brief descriptions of this instrument: It does not interrupt the programme or clientÊs routine in a programme; The information already exists; It has few biases in the information; Hak Cipta © Open University Malaysia (OUM) 230 TOPIC 8 CURRICULUM EVALUATION It often takes time; The information may be incomplete; It would be best if you were quite clear about what you are looking for; and It is not flexible, e.g. means to get data and data is restricted to what already exists. Table 8.4 explains the summary of instrumentation that have been used in curriculum evaluation. Table 8.4: Instrumentation in Curriculum Evaluation Method Overall Purpose Advantages Challenges Questionnaires, When need to Can complete Might not get checklists quickly or easily anonymously careful feedback get lots of information Inexpensive to Wording can from people in a administer bias clientÊs non-threatening responses Easy to compare way and analyse Impersonal Administer to In surveys, many people may need sampling expert Can get lots of data Does not get the full story Many sample questionnaires already exist Interviews When you want Get a full range Can take time to fully and depth of understand information Can be hard to someoneÊs analyse and impressions or Develops a compare experiences or relationship with the client Can be costly learn more about their An interviewer Can be flexible answers to can bias the with client questionnaires clientÊs responses Hak Cipta © Open University Malaysia (OUM)  TOPIC 8 CURRICULUM EVALUATION 231 Observation To gather View Can be accurate operations of a difficult to information programme as interpret seen about how a they are behaviours programme occurring operates, Can be particularly Can adapt to complex to about processes events as they categorise occur observations Can influence the behaviours of programme participants Can be expensive Documentation When you want Get Often takes review an impression of comprehensive time how the and historical programme information Information operates may be without Does not incomplete interrupting it, interrupt the programme or Need to be it is from a the clientÊs quite clear review of routine in the about what you applications, programme are looking for finances, memos, Not a flexible Information minutes, etc. means to get already exists data; data Few biases restricted to about what already information exists Hak Cipta © Open University Malaysia (OUM) 232 TOPIC 8 CURRICULUM EVALUATION Focus groups Explore a topic Quickly and Can be hard in depth through reliably get to analyse group discussion, common responses e.g. about impressions reactions to an Need a good experience or Can be an facilitator for suggestion, efficient way safety and common to get much closure understanding range and depth of Difficult to complaints, etc.; information in schedule 6 to useful in a short time 8 people evaluation and together marketing Can convey key information about programmes Case studies To fully Fully depicts Usually quite understand or the clientÊs time- depict clientÊs experience in consuming to experiences in a programme collect, programme and input, process organise and conduct and results- describe comprehensive powerful examination means to Represents a through cross- portray the depth of comparison of programme to information cases outsiders rather than a breadth CASE STUDY: EVALUATION OF A 8.6 MATHEMATICS CURRICULUM IN SOUTH AFRICA In this subtopic, we will discuss a case study regarding to the evaluation of a mathematics curriculum in South Africa. (a) Background The Education Initiative of the Open Society Foundation for South Africa in 1996 commissioned the Mathematics Learning and Teaching Initiative (MALATI). This initiative is to develop, pilot and disseminate alternative approaches and tools for teaching and learning mathematics. Hak Cipta © Open University Malaysia (OUM)  TOPIC 8 CURRICULUM EVALUATION 233 (b) Method Based on project workersÊ observation and written field notes made during the implementation of the MALATI curriculum, the following findings by Brodie & Pournara (2006) were obtained: (c) Findings Many teachers had not yet received the most basic communications issued to schools regarding the Curriculum 2005. Teachers had difficulty interpreting certain aspects of the official curriculum document. Lack of clarity led to confusion. The curriculum document had content errors. TeachersÊ content knowledge was inadequate to handle some of the curriculumÊs topics, such as statistics. Learners did not have the prior experience assumed in the curriculum; e.g. in Grade 9, probability teaching assumes that the learner had done some statistics in the earlier grades. Teachers continue to teach the topics they are used to and are reluctant to use the MALATI materials. The curriculum suggested that group work be used in teaching probability and data handling. Learners were not accustomed to group discussion and listening to one another. Teaching the topic took longer as teachers struggled to deal with learnersÊ everyday experiences in the teaching of probability. (d) Recommendations Teachers need workshops on selected aspects of the content. Selected parts of the curriculum documents need to be rewritten to reduce confusion. To convince teachers not to treat the teaching of probability and statistics as „new content‰ but teach it for its mathematical value. Hak Cipta © Open University Malaysia (OUM) 234 TOPIC 8 CURRICULUM EVALUATION SELF-CHECK 8.3 1. What are some of the problems identified with implementing the MALATI programme? 2. Based on the findings, list the recommendations made. ACTIVITY 8.3 1. Identify some problems in implementating the Primary School Integrated Curriculum (KSSR) and the Secondary School Integrated Curriculum (KSSM). 2. Describe how the teaching of science and mathematics in English was implemented in your school. 3. „New curriculum often fail to become established in schools because the importance and complexity of the implementation phase are not understood.‰ Do you agree with this statement? Justify. Share your answers with your coursemates in the myINSPIRE online forum. Evaluation is collecting data on a programme to determine its value or worth to decide whether to adopt, reject or revise it. The aim of formative evaluation is usually to obtain information to improve a programme. There are four types of evaluation: – Context – Input – Process – Product Hak Cipta © Open University Malaysia (OUM)  TOPIC 8 CURRICULUM EVALUATION 235 The model proposed by Robert Stake (1991) suggests three phases of curriculum evaluation: – The antecedent phase – The transaction phase – The outcome phase Eisner proposed the connoisseurship model, which claimed that a knowledgeable evaluator could determine whether a curriculum programme has been successful using a combination of skills and experience. The common instruments used in curriculum evaluation are interviews, observations, tests, surveys, content analysis and portfolios (records of work or products). CIPP model Interviews Curriculum evaluation Observations Documentation review Questionnaires EisnerÊs connoisseurship model Summative evaluation Evaluation StakeÊs countenance model Formative evaluation Hak Cipta © Open University Malaysia (OUM) 236 TOPIC 8 CURRICULUM EVALUATION Brodie, K., & Pournara, C. (2005). 2. Towards researching groupwork in mathematics. Researching Mathematics Education in South Africa: Perspectives, Practices and Possibilities, 28. Eisner, E. W. (1976). Educational connoisseurship and criticism: Their form and functions in educational evaluation. Journal of Aesthetic Education, 10(3/4), 135–150. Fox, W., & Gay, G. (1995). Integrating multicultural and curriculum principles in teacher education. Peabody Journal of Education, 70(3), 64–82. McNeil, L. M. (1977). Bibliographical Essay. Curriculum and Evaluation. Arno A. Bellack and Herbert M. Kliebard (eds.). McCutchan Publishing Corp, 627–647. Oliva, P. F., & Gordon II, W. R. (2012). Developing the curriculum. Pearson Higher Education. Ornstein, A. C., & Hunkins, F. P. (1998). Curriculum foundations, principles and issues. Allyn and Bacon. Scriven, M. (1967). The methodology of evaluation. In R. W. Tyler, R. M. Gagne, & M. Scriven (Eds.), Perspectives of curriculum evaluation (Vol. 1, pp. 39–83). Rand McNally. Stake, R. E. (1991). Chapter III: Retrospective on „The Countenance of Educational Evaluation‰. Teachers College Record, 92(6), 67–88. Stufflebeam, D. L. (1971). Critique of the Report of the Phi Delta Kappa Study Committee on Evaluation. [Paper presentation] Annual Meeting of the American Educational Research Association, New York. Worthen, B. R., Sanders, J. R., & Fitzpatrick, J. L. (1997). Program evaluation. Alternative Approaches and Practical Guidelines, 2. Hak Cipta © Open University Malaysia (OUM)