EA for MCQ (1) PDF - Roles of Assessment in Teaching and Learning

Topic  Roles of Assessment in 1 Teaching and Learning LEARNING OUTCOMES By the end of this topic, you should be able to: 1. Differentiate between tests, measu...

Topic  Roles of Assessment in 1 Teaching and Learning LEARNING OUTCOMES By the end of this topic, you should be able to: 1. Differentiate between tests, measurement, evaluation and assessment; 2. Explain the roles of assessment in teaching and learning; 3. Explain the general principles of assessment; 4. Differentiate between formative and summative assessments; and 5. Justify when norm-referenced and criterion-referenced tests are adopted.  INTRODUCTION The topic discusses the difference between tests, measurement, evaluation and assessment, the roles of assessment in teaching and learning, and some general principles of assessment. Also explored is the difference between formative and summative assessments as well as the difference between criterion and norm- referenced tests. The topic concludes with a brief discussion on the current trends in assessment. Copyright © Open University Malaysia (OUM) 2  TOPIC 1 ROLES OF ASSESSMENT IN TEACHING AND LEARNING Can you differentiate between tests, measurement, evaluation and assessment (see Figure 1.1)? Figure 1.1: Can you differentiate between tests, measurement, and evaluation and assessment? 1.1 TESTS, MEASUREMENT, EVALUATION AND ASSESSMENT It is not surprising that many people are confused with the fundamental differences between tests, measurement, evaluation and assessment as they are used in education. The following explanation on these concepts will help clarify the confusion. (a) Tests Most people are familiar with tests because all of us, at some point in our lives, have taken some form of tests. In school, tests are given to measure our academic aptitude and indirectly to evaluate whether we have gained from the teaching by the teacher. At the workplace, tests are conducted to select suitable persons for specific jobs, tests are used as the basis for job promotions and tests are used to encourage re-learning. Physicians, Copyright © Open University Malaysia (OUM)  TOPIC 1 ROLES OF ASSESSMENT IN TEACHING AND LEARNING  3 lawyers, insurance consultants, real-estate agents, engineers, civil servants and many other professions are required to take tests to demonstrate their competence in specific areas and in some cases to be licensed to practise their profession or trade. Throughout their professional careers, teachers, counsellors and school administrators are required to conduct, score and interpret a wide variety of tests. For example, school administrators rate the performance of individual teachers and school counsellors record the performance of their clients. It is possible that a teacher may construct, administer and grade thousands of tests during his or her career! Test items can be written in various formats including multiple-choice questions, matching, true or false, short answer and essay. These formats vary in strengths and weaknesses. No one format is ideal for all circumstances. According to the joint committee of the American Psychological Association (APA), the American Educational Research Association (AERA) and National Council on Measurement in Education (NCME), A test may be thought of as a set of tasks or questions intended to elicit particular types of behaviour when presented under standardised conditions and to yield scores that will have desirable psychometric properties (1974). While most people know what a test is, many have difficulty differentiating between measurement, evaluation and assessment. Some have even argued that they are similar! (b) Measurement Measurement is the act of assigning numbers to a phenomenon. In education, it is the process by which the attributes of a person are measured and assigned numbers. Remember it is a process, indicating there are certain steps involved! As educators, we frequently measure human attributes such as attitudes, academic achievements, aptitudes, interests, personality and so forth. Hence, in order to measure we have to use certain instruments so that we can conclude that Ahmad is better in mathematics than Kumar while Tajang is more positive towards science than Kong Beng. We measure to obtain information about „what is‰. Such information may or may not be Copyright © Open University Malaysia (OUM) 4  TOPIC 1 ROLES OF ASSESSMENT IN TEACHING AND LEARNING useful depending on the accuracy of the instruments we use and our skill at using them. For example, we measure temperature using a thermometer, and the thermometer is the instrument used. How do you measure performance in mathematics, for example? We use a mathematics test which is an instrument containing questions and problems to be solved by students. The number of right responses obtained is an indication of the performance of individual students in mathematics. Note that we are only collecting information. We are not evaluating! Evaluation is therefore quite different from measurement. (c) Assessment and Evaluation The literature has used the terms „assessment‰ and „evaluation‰ in education as two different concepts. Evaluation is the process of making judgments based on criteria and evidences. Assessment is the process of documenting knowledge, skills, attitudes and beliefs, usually in measurable terms. The goal of assessment is to make improvements as opposed to simply being judged. In an educational context, assessment is the process of describing, collecting, recording, scoring and interpreting information about learning. The two terms are also used interchangeably. For example, some authors used the term „formative evaluation‰ while others use the term „formative assessment‰. We will use the two terms interchangeably because there is too much overlap in the interpretations of the two concepts. Generally, assessment is viewed as the process of collecting information with the purpose of making decisions about students. We may collect information using various tests, observations of students and interviews. Rowntree (1974) views assessment as, A human encounter in which one person interacts with another directly or indirectly with the purpose of obtaining and interpreting information about the knowledge, understanding, abilities and attitudes possessed by that person. For example, based on assessment information we can determine whether Chee Keong needs special services to assist him in developing his reading skills or whether Khairul, who has been identified as a dyslexic child, needs special attention. The key words in the definition of assessment is collecting data and making decisions. Hence, in order to make decisions one has to evaluate, which is the process of making judgement, about a given situation. Copyright © Open University Malaysia (OUM)  TOPIC 1 ROLES OF ASSESSMENT IN TEACHING AND LEARNING  5 When we evaluate, we are saying that something is good, appropriate, valid or positive, for example. To make an evaluation, we need information and the information is obtained by measuring using a reliable instrument. For example, you measure the temperature in the classroom and it is 30 degrees Celsius, which is simply information. Some students may find the temperature too hot for learning while others may say that it is ideal for learning. At some point during the day, we are evaluating something or someone! Educators are constantly evaluating students. This is usually done in comparison with some standards. For example, if the objective of the lesson is for learners to apply BoyleÊs Law to solving a problem and 80 per cent of learners are able to solve the problem, then the teacher may conclude that his teaching of the principle was quite successful. Therefore, evaluation is the comparison of what is measured against some defined criteria and to determine whether it has been achieved, whether it is appropriate, whether it is good, whether it is reasonable, whether it is valid and so forth. 1.2 WHY DO WE ASSESS? Let us begin by asking the question, „Why do we, as educators, assess learners?‰ Some of us may find the question rather strange. The following may be a likely response: Question : Why do you assess? Answer : Well, I assess to find out whether my students understand what has been taught. Question : What do you mean by „understand‰? Answer : Whether they can remember what I taught them and able to solve problems. Question : What do you do with the test results? Answer : Well, I provide students the right answers and point out the mistakes made when answering the questions. The above could be the reasons educators give when asked about the purpose of assessment. In the context of education, assessment is performed to gain an understanding of an individualÊs strengths and weaknesses in order to make appropriate educational decisions. The best educational decisions are based on Copyright © Open University Malaysia (OUM) 6  TOPIC 1 ROLES OF ASSESSMENT IN TEACHING AND LEARNING information and better decisions are usually based on more information (Salvia & Ysseldyke, 1995). Based on the reasons for assessment provided by Harlen (1978) and Deale (1975), two main reasons are identified (refer to Figure 1.2). Figure 1.2: Purposes of assessment With regard to learning, assessment is aimed at providing information that will help make decisions concerning remediation, enrichment, selection, exceptionality, progress and certification. With regard to teaching, assessment provides information regarding achievement of objectives, the effectiveness of teaching methods and learning materials. (a) Assess in Order to Help Learning Let us look at the following aspects of assessment which are able to help learning: (i) Diagnosis Diagnostic evaluation or assessment is performed at the beginning of a lesson or unit for a particular subject area to assess studentsÊ readiness and background for what is about to be taught. This pre- instructional assessment is done when you need information on a particular student, group of students or a whole class before you can proceed with the most effective instructional method. For example, you could administer a Reading Test to Year One students to assess their reading level. Based on the information, you may want to assign weak readers for special intervention or remedial action. On the other hand, the test might reveal that some students are reading at an exceptionally high level and you might want to recommend that they be assigned to an enrichment programme (refer to Table 1.1). Copyright © Open University Malaysia (OUM)  TOPIC 1 ROLES OF ASSESSMENT IN TEACHING AND LEARNING  7 Table 1.1: Purpose of Assessment: To Help Learning To Help Questions to Be Answered Learning Diagnosis for  Should the student be sent for remedial classes so remedial action that difficulty in learning can be overcome? Diagnosis for  Should the student be provided with enrichment enrichment activities? Exceptionality  Does the student have special learning needs that require special education assistance? Selection  Should the student be streamed into X or Y class? Progress  To what extent is the student making progress towards specific instructional goals? Communication  How is the child doing in school and how can to parents parents help? Certification  What are the strengths and weaknesses in the overall performance of a student in specific areas assessed? Administration  How is the school performing in comparison to and counselling other schools?  Why should students be referred to counselling? (ii) Exceptionality Assessment is also conducted to make decisions on exceptionality. Based on information obtained from the assessment, teachers may make decisions as to whether a particular student needs to be assigned to a class with exceptional students. Exceptional students are students who are physically, mentally, emotionally or behaviourally different from the normal population. For example, based on the assessment information, a child who is found to be dyslexic may be assigned for special treatment or a student who has been diagnosed to be learning disabled may be assigned for special education. (iii) Certification Certification is perhaps the most important reason for assessment. For example, the Sijil Pelajaran Malaysia (SPM) is an examination aimed at providing students with a certificate. The scores obtained are converted into letter grades signifying performance level in various subject areas and used as a basis for comparison between students. The certificate obtained is further used for selecting students for further studies, scholarships or jobs. Copyright © Open University Malaysia (OUM) 8  TOPIC 1 ROLES OF ASSESSMENT IN TEACHING AND LEARNING (iv) Placement Besides certification, assessment is also conducted for the purpose of placement. Students are endowed with varying abilities and one of the tasks of the school is to place them according to their aptitude and interests. For example, performance in the Pentaksiran Tingkatan 3 (PT3) (previously Penilaian Menengah Rendah) is used as the basis for placing students in the arts or science stream. Assessment is also used to stream students according to academic performance. It has been the tradition that the „A‰ and „B‰ classes consist of high achievers based on their results in the end of semester or end of year examinations. Placement tests have even been used in preschools to stream children according their literacy levels! The practice of placing students according to academic achievement has been debated for decades with some educationists arguing against it while others supporting its merits. ACTIVITY 1.1 „Streaming students according to academic abilities should be discouraged in Malaysian schools‰. Discuss with your coursemates in the myINSPIRE online forum. (v) Communicate to Parents Parents want to know how their child is doing in school and appreciate information, particularly specific examples, of their childÊs progress. Showing examples of their childÊs work over time enables parents to personally assess the growth and progress of their child. It is essential to tell the whole story when reporting information about performance progress. Talking to parents about standards, sharing student work samples, using rubrics in conferences, and differentiating between performance and progress are some ways to ensure that they are given an accurate picture of their childÊs learning. (vi) School Administration and Counselling Schools collect assessment information in order to determine how the school is performing in relation to other schools for a particular semester or year. Assessment results are also used to compare performance over the years for the same school. Based on the results, school administrators may institute measures to remedy weaknesses such as putting in more resources to help students who perform poorly, for example, to increase the number of studentsÊ reading and writing proficiency from poor to satisfactory level. Copyright © Open University Malaysia (OUM)  TOPIC 1 ROLES OF ASSESSMENT IN TEACHING AND LEARNING  9 Assessment results (especially relating to socioemotional development) may be used by school administrators and counsellors in planning intervention strategies for at-risk students. Assessment by counsellors will enable them to identify students presenting certain socioemotional problems that require counselling services or referral to specialists such psychiatrists, legal counsellors and law enforcement authorities. (b) Assess in Order to Improve Teaching If 70 per cent of your students fail in a test, do you investigate whether your teaching-learning strategy was appropriate or do you attribute it to your students being academically weak in addition to not revising their studies? Most educators would attribute the poor performance to the latter. However, assessment information is valuable because it can indicate which of the learning outcomes have been successfully achieved and which instructional objectives students had most difficulty with. Assessment results are valuable in providing clues to the effectiveness of the teaching strategy implemented and teaching materials used. It also indicates whether students had the required prior knowledge to grasp the concepts and principles discussed (refer to Table 1.2). Assessment data may also provide insights into why some teachers are more successful in teaching a particular group of students while others are less successful. Table 1.2: Purpose of Assessment: To Improve Teaching To Improve Teaching Questions to Be Answered Objectives Were the desired learning outcomes achieved? Teaching methods Were the teaching methods employed effective? Prior knowledge Did students have relevant prior knowledge? Teaching materials Were the teaching materials used effective? Teacher differences Were particular teachers more effective than others? ACTIVITY 1.2 To what extent have you used assessment data to review your teaching- learning strategies? Discuss this with your coursemates in the myINSPIRE online forum. Copyright © Open University Malaysia (OUM) 10  TOPIC 1 ROLES OF ASSESSMENT IN TEACHING AND LEARNING 1.3 GENERAL PRINCIPLES OF ASSESSMENT Let us discuss the general principles of assessment: (a) What is to be Assessed has to be Clearly Specified The specification of the characteristics to be measured should precede the selection or development of assessment procedures. In assessing student learning, the intended learning goals or outcomes should be clearly specified. In other words, appropriate assessment procedures can only be selected if there is clear specification of the intended learning outcomes to be measured. (b) An Assessment Procedure should be Selected Based on Its Relevance to the Characteristics or Performance to be Measured When selecting an assessment procedure to measure a specific learning outcome, teachers should always ask if the procedure selected is the most effective method for measuring the learning or development to be assessed. There must be a close match between the intended learning outcomes and the types of assessment tasks to be used. For example, if the teacher would like to assess the studentsÊ ability to organise ideas, the use of multiple- choice test would be a poor choice. (c) Different Assessment Procedures are Required to Provide a Complete Picture of Student Achievement and Development No single assessment procedure can assess all the different learning outcomes in a school curriculum. Different assessment procedures (formative, summative and authentic assessments) can achieve different objectives. For example, multiple-choice questions are useful for measuring knowledge, understanding and application outcomes while essay tests are appropriate for measuring the ability to organise and express ideas. Projects that require conducting library research are needed to measure certain skills in formulating and solving problems. Observational techniques are needed to assess performance skills and various aspects of student behaviour. Using a wide variety of assessment tools allows a teacher to determine which instructional strategies are effective and which needed to be modified. In this way, assessment can be used to improve classroom teaching, plan curriculum and research oneÊs own teaching practice. Copyright © Open University Malaysia (OUM)  TOPIC 1 ROLES OF ASSESSMENT IN TEACHING AND LEARNING  11 (d) Assessment must be Aligned to Instruction In order to encourage higher-order thinking and problem-solving skills, there is an increased need to align the curriculum, instruction and assessment. Classroom assessment techniques should be focusing on aligning assessments more closely with the instructional strategies used in the classroom. In this way, what is to be assessed in the classroom is consistent with what has been taught and vice versa. For example, it would not be fair to assess students on higher-order thinking skills when what is taught is only lower level thinking skills. 1.4 TYPES OF ASSESSMENT Before we proceed to discuss more about assessment, you need to be clear about these often used concepts in assessment: (a) Formative assessment (or evaluation) and summative assessment (or evaluation); (b) Criterion-referenced assessment and norm-referenced assessment; and (c) Authentic assessment (examples of authentic assessment will be discussed in Topics 6 and 7). Let us move on and read further. 1.4.1 Formative vs Summative Assessments Teachers engage students in all aspects of their learning. There are many strategies to accomplish this ranging from informal questioning, quiz and observation to more formal monthly test and end of semester examination. In a balanced assessment system, both summative and formative assessments are an integral part of information gathering. Assessment can be done at various times throughout the school year. A comprehensive assessment plan will include both formative and summative assessments. The point at which assessment occurs and the aim of assessing distinguishes these two categories of assessment. Copyright © Open University Malaysia (OUM) 12  TOPIC 1 ROLES OF ASSESSMENT IN TEACHING AND LEARNING (a) Formative Assessment Formative assessment is often done at the beginning or during the school year, thus providing the opportunity for immediate evidence of student learning in a particular subject area or at a particular point in the programme. Classroom assessment is one of the most common formative assessment techniques used. The purpose of this technique is to improve the quality of student learning and should not be evaluative in nature or involve grading students. In formative assessment, the teacher compares the performance of a student to the performance of other students in the class and not all students in the same year. Usually, a small section of the content is tested to determine if the objectives have been met. Formative assessment is action-oriented and forms the basis for improvement of instructional methods (Scriven, 1996). For example, if a teacher observes that some students still have not grasp a concept, he may design a review activity or use a different instructional strategy. Likewise, students can monitor their progress with periodic quizzes and performance tasks. The results of formative assessments are used to modify and validate instruction. In short, formative assessments are ongoing and include reviews and observations of what is happening in the classroom. Formative assessments are generally low stakes, which means that they have minimal or no point value. (b) Summative Assessment „When the cook tastes the soup, thatÊs formative evaluation; when the guests taste the soup, thatÊs summative evaluation‰ (Robert Stakes) Summative assessment is comprehensive in nature. It provides accountability and is used to check the level of learning at the end of the programme (which may be at the end of the semester, year or after two years). For example, after five years in secondary school, students sit for the Sijil Pelajaran Menengah (SPM) examination which is summative in nature since it is based on the cumulative learning experiences of students. Copyright © Open University Malaysia (OUM)  TOPIC 1 ROLES OF ASSESSMENT IN TEACHING AND LEARNING  13 Summative assessment is also used to evaluate the effectiveness of an instructional programme at the end of an academic year or at a predetermined time. The goal of summative assessment is to make a judgment of student competency after an instructional phase is completed. For example, national examinations are administered each year in Malaysia. It is a summative assessment to determine each studentÊs acquisition of several subject areas of between two to three years coverage of content. Summative evaluations are used to determine if students have mastered the specific competencies as set out by the programme and letter grades are assigned to assess learner achievement. Summative assessments are often high stakes, which means that they have a high point value. Table 1.3 highlights the differences between formative and summative assessments. Table 1.3: Differences between Formative and Summative Assessments Criteria Formative Assessment Summative Assessment Timing  Conducted throughout the  Conducted at the end of a teaching-learning process. teaching-learning phase (for example, end of semester or year). Methods  Paper and pencil tests,  Paper and pencil tests, oral tests observations, quizzes, exercises, administered to the group. practical sessions administered to the group and individually. Aims  To assess progress and  Grading to determine if the recommend remedial action for programme was successful. non-achievement of objectives.  To certify students and improve  Remediation or enrichment or the curriculum. reteach the topic. Examples  Quizzes, essays, diagnostic tests,  Final examination, national lab reports and anecdotal examination and qualifying records. tests. Copyright © Open University Malaysia (OUM) 14  TOPIC 1 ROLES OF ASSESSMENT IN TEACHING AND LEARNING Can information from summative assessments be used formatively? Tests given at the end of a unit of work provide information about how much each student has learnt or has not learnt. If left as a summative process, then the studentsÊ test marks merely become a record of their successes or failures at a point in time and not directly helping them to improve. Moreover, in actual fact some students will know more than what is indicated in their test performance. However, when the information is used formatively, the tests results can provide an important source of detailed, individualised feedback identifying where each student needs to deepen their understanding and improve their recall of the knowledge they have learnt. The more teachers know about individual students as they engage in the learning process, the better teachers can adjust instruction to ensure that all students continue to move forward in their learning. Advantages of using information derived from summative assessments to improve future student performance are: (a) Summative data reveals how the students performed at the end of a learning programme, namely advanced, proficient, basic or below basic. For example, if a student has scored below basic in the semester exam and exhibits signs of a struggling student, the teacher may want to place the student at the front of the class so that the teacher can easily access the student when the student needs extra support; (b) Summative assessments can serve as a guide to improving teaching methods. Teachers often employ varied teaching methodologies within the classroom. Summative assessments can help teachers collaborate and improve teaching methods from year to year; and (c) Summative assessments help teachers and administrators in improving the curriculum and curriculum planning. Standards-driven instruction plays a large role in schools today. When summative assessments show consistent gaps between student knowledge and learning targets, schools may turn to the relevant authority in the Ministry of Education to suggest improving curriculum planning or new curriculum to fill those learning gaps. The data that is collected using a summative assessment can help teachers and schools make decisions based on the instruction that has already been completed. This contrasts with formative assessment, whereby formative assessment can help teachers and students during the instruction process. It is important to understand the difference between the two, as both assessments can play an important role in education. Copyright © Open University Malaysia (OUM)  TOPIC 1 ROLES OF ASSESSMENT IN TEACHING AND LEARNING  15 1.4.2 Norm-referenced vs Criterion-referenced Tests The main difference between norm-referenced tests and criterion-referenced tests depends on the purpose or aim of assessing your students, the way in which content is selected and the scoring process which defines how the test results must be interpreted. (a) Norm-referenced Tests The major reason for norm-referenced tests is to classify students. These tests are designed to highlight achievement differences between and among students to produce dependable rank order of students across a continuum of achievement from high-achievers to low achievers (Stiggins, 1994). With norm-referenced tests, a representative group of students is given the test and their scores form the norm after having gone through a complex administration and analysis. Anyone taking the norm-referenced test can compare his or her score against the norm. For example, a student who obtained a score of 70 on a norm-referenced test will not mean much until it is compared to the norm. When compared to the norm, her score is in the 80th percentile which means that she performed as well or better than 20 per cent of students in the norm group. This type of information can be useful for deciding whether or not students need remedial assistance or is a candidate for the gifted programme. However, the score gives little information about what the student actually knows or can do. A major criticism of norm-referenced tests is that they tend to focus on assessing low level, basic skills (Romberg, 1989). (b) Criterion-referenced Tests Criterion-referenced tests determine what students can or cannot do and not how they compare to others (Anastasi, 1988). Criterion-referenced tests report how well students are doing relative to a predetermined performance level on a specified set of educational goals or outcomes included in the curriculum. Criterion-referenced tests are used when teachers wish to know how well students have learnt the knowledge and skills which they are expected to have mastered. This information may be used to determine how well the student is learning the desired curriculum and how well the school is teaching that curriculum. Criterion-referenced tests give detailed information about how well a student has performed on each of the educational goals or outcomes included on that test. For instance, a criterion-referenced test score might describe which arithmetic operations a student can perform or the level of reading difficulty experienced. Copyright © Open University Malaysia (OUM) 16  TOPIC 1 ROLES OF ASSESSMENT IN TEACHING AND LEARNING Table 1.4 illustrates the differences between norm-referenced and criterion- referenced tests. Table 1.4: Differences between Norm-referenced and Criterion-referenced Tests Criteria Norm-referenced Tests Criterion-referenced Tests Aims  Compare a studentÊs  Compare a studentÊs performance with other performance against some students. criteria.  Select students for  Extent to which a student certification. has acquired the knowledge or skill.  Improve teaching and learning. Types of  Questions from simple to  Questions of nearly similar questions difficult. difficulty relating to the criteria. Reporting of  Grades are assigned.  No grades are assigned results (whether skill or knowledge is achieved). Content  Wide content coverage.  Specific aspects of the coverage content. Examples  UPSR, PT3, SPM national  Class tests, exercises and examinations, end of assignments. semester examinations and end of year examinations. SELF-CHECK 1.1 1. Explain the differences between norm-referenced and criterion- referenced tests. 2. Describe the main differences between formative and summative assessments. Copyright © Open University Malaysia (OUM)  TOPIC 1 ROLES OF ASSESSMENT IN TEACHING AND LEARNING  17 1.5 CURRENT TRENDS IN ASSESSMENT In the last two decades, there have been major changes in assessment practices in many parts of the world. Brown, Bull and Pendlebury (1997) identified the following trends in educational assessment: (a) Written examinations are gradually being replaced by more continuous assessments and coursework; (b) There is a move towards more student involvement and choice in assessments; (c) Group assessment is becoming more popular in an effort to emphasise collaborative learning between students and to reduce excessive competition; (d) Subject areas and courses state more explicitly about the expectations in assessment, more specifically the kinds of performance required from students when they are assessed. This is unlike earlier practices where assessment is so secretive and students had to figure out for themselves what was required of them; (e) An understanding of the process is now seen as, at least, equally important to the knowledge of facts. This is in line with the general shift from product-based assessment towards process-based assessment; and (f) Student-focussed „learning outcomes‰ have begun to replace teacher- oriented „objectives‰. The focus is more on what the student will learn rather than what the teacher plans to teach. Copyright © Open University Malaysia (OUM) 18  TOPIC 1 ROLES OF ASSESSMENT IN TEACHING AND LEARNING Easing up on Exams Putrajaya: Reducing the number of Among the measures proposed are: examination subjects and having a semester system are among the Reducing the number of subjects major changes being planned to in public examinations; make the education system more Emphasising skills and abilities holistic and less focussed on rather than focusing on content academic achievement. and achievement; Education Minister, Datuk Seri Encouraging personal Hishamumuddin Tun Hussein said development through subjects that these measures were in line like Art and Physical Education; with the GovernmentÊs aim to and reform the countryÊs education system. „We do not intend to abolish Improving teaching-learning public or school-level examinations methods by encouraging more totally, but we recognise that the project-based assignments. present assessment system needs to be looked at‰, he said. He said that emphasis should be on individual accomplishments rather than the schoolÊs performance in public examinations and also highlighting the individualÊs co-curricular achievements. (The Star, 21 March, 2006) ACTIVITY 1.3 Refer to the report on „Easing up on Exams‰ and discuss to what extent you agree with the measures proposed by the Ministry of Education to reduce the exam-oriented education system in Malaysia. Share your answer with your coursemates in the myINSPIRE online forum. Copyright © Open University Malaysia (OUM)  TOPIC 1 ROLES OF ASSESSMENT IN TEACHING AND LEARNING  19  A test may be thought of as a set of tasks or questions intended to elicit particular types of behaviour when presented under standardised conditions and to yield scores that will have desirable psychometric properties.  Measurement in education is the process by which the attributes of a person are measured and assigned numbers.  Assessment is viewed as the process of collecting information with the purpose of making decisions regarding students.  Assessment is aimed at helping the learner and to improve teaching.  Summative assessment is comprehensive in nature, provides accountability and is used to check the level of learning at the end of the programme.  Formative assessment is often conducted at the beginning of or during the school year, thus providing the opportunity for immediate evidence of student learning in a particular subject area or at a particular point of time in a programme.  The information from summative assessment can be used formatively. The students or schools can use the information to guide their efforts and activities in subsequent courses.  The major reason for norm-referenced tests is to classify students. These tests are designed to highlight achievement differences between and among students to produce dependable rank order of students.  Criterion-referenced tests determine what students can or cannot do and not how they compare to others. Assessment Measurement Criterion-referenced test Norm-referenced test Evaluation Summative assessment Formative assessment Test Copyright © Open University Malaysia (OUM) Topic  Foundation for Assessment: 2 What to Assess LEARNING OUTCOMES By the end of this topic, you should be able to: 1. Justify the behaviours that are to be measured to present a holistic assessment of students; 2. Describe the various types of cognitive learning outcomes to be assessed; 3. Describe the various types of affective learning outcomes to be assessed; and 4. Describe the various types of psychomotor learning outcomes to be assessed.  INTRODUCTION If you were to ask a teacher, what should be assessed in the classroom, the immediate response would be, of course, the facts and concepts taught. They are the facts and concepts found in science, history, geography, language, arts, religious education and other similar subjects. However, the Malaysian Philosophy of Education states that education should aim towards the holistic development of the individual. Hence, it is only logical that the assessment system should also seek to assess more than the acquisition of the facts and concepts of a subject area. What about assessment of physical and motor abilities? What about socioemotional behaviours such as attitudes, interests, personality and so forth? Do they not contribute to the holistic person? In this topic, you will learn the types of learning outcomes that need to be assessed in a curriculum. The topic will conclude with a brief explanation on how to plan a table of specification for a classroom test. Copyright © Open University Malaysia (OUM)  TOPIC 2 FOUNDATION FOR ASSESSMENT: WHAT TO ASSESS  21 2.1 IDENTIFYING WHAT TO ASSESS When educators are asked what should be assessed in the classroom, the majority would refer to evaluating the acquisition of the facts, concepts, principles, procedures and methods of a subject area. You might find a minority of educators who insist that skills acquired by learners should also be assessed especially in subjects such as physical education, art, drama, music, technical drawing, carpentry, automobile engineering and so forth. Even fewer educators would propose that the socioemotional behaviour of learners should also be assessed. National Philosophy of Malaysian Education Education in Malaysia is an ongoing effort towards further developing the potentials of individuals in a holistic and integrated manner so as to produce individuals who are intellectually, spiritually, emotionally and physically balanced and harmonic, based on a firm belief in and devotion to God. Such an effort is designed to produce Malaysian citizens who are knowledgeable and competent, who possess high moral standards and who are responsible and capable of achieving a high level of personal well-being as well as being able to contribute to the harmony and betterment of the family, society and the nation at large. (Curriculum Development Centre, Ministry of Education Malaysia, 1988) The National Philosophy of Malaysian Education has important implications for assessment. Theoretically, a comprehensive assessment system should seek to provide information on the extent to which the National Philosophy of Education has achieved its goal. In other words, the assessment system should seek to determine: (a) Whether our schools have developed „the potentials of individuals in a holistic and integrated manner‰; (b) Whether our students are „intellectually, spiritually, emotionally and physically balanced‰; (c) Whether our students are „knowledgeable and competent‰ and „possess high moral standards‰; (d) Whether our students have a „high level of personal well-being‰; and (e) Whether students are equipped with the abilities and attitudes that will enable them „to contribute to the harmony and betterment of the family, society and the nation at large‰. Copyright © Open University Malaysia (OUM) 22  TOPIC 2 FOUNDATION FOR ASSESSMENT: WHAT TO ASSESS On the contrary, in actual practice the assessment tends to overemphasise on intellectual competence which translates into the measurement of cognitive learning outcomes of specific subject areas. The other aspects of the holistic individual are given minimal attention because of various reasons. For example, how does a teacher assess the spiritual or emotional growth and development? These are constructs that are difficult to evaluate and extremely subjective. Hence, it is no surprise that assessment of cognitive outcomes has remained the focus of most assessment systems all over the world because it is relatively easier to observe and measure. However, in this topic we will make an attempt to present a more „holistic‰ assessment of learning, focusing on three main types of human behaviour. These are behaviours that psychometricians and psychologists have attempted to assess and are closely aligned to realising the goals of the National Philosophy of Malaysian Education. 2.2 THREE TYPES OF LEARNING OUTCOMES Few people will dispute that the purpose of schooling is the development of the holistic person. In the late 1950s and early 1960s, a group of psychologists and psychometricians proposed that schools should seek to assess three domains of learning outcomes (refer to Figure 2.1): (a) Cognitive learning outcomes (knowledge or mental skills); (b) Affective learning outcomes (feelings or emotions); and (c) Psychomotor learning outcomes (manual or physical skills). Figure 2.1: Holistic assessment of learners Copyright © Open University Malaysia (OUM)  TOPIC 2 FOUNDATION FOR ASSESSMENT: WHAT TO ASSESS  23 Domains can be thought of as categories. Educators often refer to these three domains as KSA (Knowledge, Skills and Attitude). Each domain consists of subdivisions, starting from the simplest behaviour to the most complex, thus forming taxonomy of learning outcomes. Each of the taxonomy of learning behaviour can be thought of as „the goals of the schooling process.‰ That means that after schooling, the learner should have acquired new skills, knowledge, and/or attitudes. However, the levels of each division outlined are not absolutes. While there are other systems or hierarchies that have been devised in the educational world, these three taxonomies are easily understood and are probably the most widely used today. To assess the three domains, one has to identify and isolate the behaviour that represents these domains. When we assess we evaluate some aspects of the learnerÊs behaviour, for example, his ability to compare, explain, analyse, solve, draw, pronounce, feel, reflect and so forth. The term „behaviour‰ is used broadly to include the learnerÊs ability to think (cognitive), feel (affective) and perform a skill (psychomotor). For example, you have just taught about „The Rainforest of Malaysia‰ and you would like to assess your students in their: (a) Thinking You might ask them to list the characteristics of the Malaysian rainforest and compare it with the coniferous forest of Canada; (b) Feelings (emotions, attitudes) You could ask them to design an exhibition on how students could contribute towards conserving the rainforest; and (c) Skill You could ask them to prepare satellite maps about the changing Malaysian rainforest by accessing websites from the Internet. ACTIVITY 2.1 Refer to Figure 2.1. To what extent are affective and psychomotor behaviours assessed in your institution? Discuss with your coursemates in the myINSPIRE online forum. Copyright © Open University Malaysia (OUM) 24  TOPIC 2 FOUNDATION FOR ASSESSMENT: WHAT TO ASSESS 2.3 ASSESSING COGNITIVE LEARNING OUTCOMES When we evaluate or assess a human being, we are assessing or evaluating the behaviour of a person. This might be a bit confusing to some people. Are we not assessing a personÊs understanding of the facts, concepts and principles of a subject area? Every subject, whether it is history, science, geography, economics, or mathematics, has its unique repertoire of facts, concepts, principles, generalisations, theories, laws, procedures and methods that are transmitted to learners (illustrated in Figure 2.2). Figure 2.2: Contents of a subject assessed When we assess we do not assess the learnerÊs store of the facts, concepts or principles of a subject but rather what the learner is able to do with the facts, concepts or principles of a subject area. For example, we evaluate the learnerÊs ability to compare facts, explain the concept, analyse a generalisation (or statement) or solve a problem based on a given principle. In other words, we assess the understanding or mastery of a body of knowledge based upon what the learner is able to do with the contents of the subject. Let us look at two mechanisms used to measure or assess cognitive learning, namely BloomÊs Taxonomy and The Helpful Hundred. Copyright © Open University Malaysia (OUM)  TOPIC 2 FOUNDATION FOR ASSESSMENT: WHAT TO ASSESS  25 (a) BloomÊs Taxonomy In 1956, Benjamin Bloom led a group of educational psychologists to develop a classification of levels of intellectual behaviour which are important to learning. They found that over 95 per cent of the test questions which learners encountered required them to think only at the lowest possible level, that is the recall of information. Bloom and his colleagues developed a widely accepted taxonomy (method of classification on differing levels) for cognitive objectives. This is referred to as BloomÊs Taxonomy (refer to Figure 2.3). There are six levels in BloomÊs classification with the lowest level termed knowledge. The knowledge level is followed by five increasingly difficult levels of mental abilities: comprehension, application, analysis, synthesis and evaluation. Figure 2.3: BloomÊs taxonomy of cognitive learning outcomes Now, read further to find out what each level constitutes. (i) Knowledge: The behaviours at the knowledge level require learners to recall specific information. The knowledge level is the lowest cognitive level. Examples of verbs describing behaviours at the knowledge level include the ability to list, define, name, state, recall, match, identify, tell, label, underline, locate, recognise, select and so forth. For example, learnersÊ ability to recite the factors leading to the World War II, quote formula for density and force, tell laboratory safety rules. (ii) Comprehension: The behaviours at the comprehension level which is a higher level of mental ability than the knowledge level require the understanding of the meaning of concepts and principles, translation of words and phrases into oneÊs own words, interpolation which involves filling in missing information, interpretation which involves inferring and going beyond the given information. Examples of verbs describing behaviours at the comprehension level are explain, distinguish, infer, interpret, convert, generalise, defend, estimate, extend, paraphrase, retell using own words, predict, rewrite, summarise, translate and so forth. For example, learners are able to Copyright © Open University Malaysia (OUM) 26  TOPIC 2 FOUNDATION FOR ASSESSMENT: WHAT TO ASSESS rewrite NewtonÊs three laws of motion, explain in oneÊs own words the steps for performing a complex task and translate an equation into a computer spreadsheet. (iii) Application: The behaviours at the application level require learners to apply a rule or principle learned in the classroom into novel or new situations in the workplace or unprompted use of an abstraction. Examples of verbs describing behaviours at the application level are apply, change, compute, demonstrate, discover, manipulate, modify, give an example, operate, predict, prepare, produce, relate, show, solve, use and so forth. For example, learners are able to use the formula for projectile motion to calculate the maximum distance a long jumper jumps and apply laws of statistics to evaluate the reliability of a written test. (iv) Analysis: The behaviours at the analysis level require learners to identify component parts and describe their relationship, separate material or concepts into component parts so that its organisational structure may be understood and distinguish between facts and inferences. Examples of verbs describing behaviours at the analysis level are analyse, break down, compare, contrast, diagram, deconstruct, examine, dissect, differentiate, discriminate, distinguish, identify, illustrate, infer, outline, relate, select, separate and so forth. For example, learners are able to troubleshoot a piece of equipment by using logical deduction, recognise logical fallacies in reasoning, analyse information from a company and determine needs for training. (v) Synthesis: The behaviours at the synthesis level require learners to build a structure or pattern from diverse elements and put parts together to form a whole with emphasis on creating a new meaning or structure. Examples of verbs describing behaviours at the synthesis level are categorise, combine, compile, compose, create, devise, design, explain, generate, modify, organise, plan, rearrange, reconstruct, relate, reorganise, find an unusual way, formulate, revise, rewrite, summarise, tell, write and so forth. For example, learners are able to write a creative short story, design a method to perform a specific task, integrate ideas from several sources to solve a problem, devise a new plan of action to improve the outcome. Copyright © Open University Malaysia (OUM)  TOPIC 2 FOUNDATION FOR ASSESSMENT: WHAT TO ASSESS  27 (vi) Evaluation: The behaviours at the evaluation level require learners to make judgment about materials and methods, and the value of ideas or materials. Examples of verbs describing behaviours at the evaluation level are appraise, compare, conclude, contrast, criticise, critique, defend, describe, rank, give your own opinion, discriminate, evaluate, explain, interpret, value, justify, relate, summarise, support and so forth. For example, learners are able to evaluate and decide on the most effective solution to a problem, justify the choice of a new procedure or course of action. (b) The Helpful Hundred Heinich, Molenda, Russell and Smaldino (2001) suggested 100 verbs that highlight performance or behaviours that are observable and measurable. This is not to say that these 100 verbs are the only ones but they definitely are a great reference for educators. Table 2.1 displays the verbs that would be appropriate to use when you are writing instructional objectives in each level of BloomÊs Taxonomy. Table 2.1: The Helpful Hundred add compute drill label predict state alphabetise conduct estimate locate prepare subtract analyse construct evaluate make present suggest apply contrast explain manipulate produce swing arrange convert extrapolate match pronounce tabulate assemble correct fit measure read throw attend cut generate modify reconstruct time bisect deduce graph multiply reduce translate build defend grasp name remove type cave define grind operate revise underline categorise demonstrate hit order select verbalise choose derive hold organise sketch verify classify describe identify outline ski weave colour design illustrate pack solve weigh compare designate indicate paint sort write complete diagram install plot specify compose distinguish kick position square Copyright © Open University Malaysia (OUM) 28  TOPIC 2 FOUNDATION FOR ASSESSMENT: WHAT TO ASSESS In 2001, Krathwohl and Anderson modified the original BloomÊs Taxonomy (1956). They identified and isolated the following list of behaviours that an assessment system should address (refer to Table 2.2). Table 2.2: Revised Version of BloomÊs Taxonomy Category and Cognitive Process Alternative Names 1. Remembering Recognising Identifying Recalling Retrieving 2. Understanding Interpreting Clarifying, paraphrasing, representing, Exemplifying translating Classifying Illustrating, instantiating Summarising Categorising, subsuming Inferring Abstracting, generalising Comparing Concluding, extrapolating, interpolating, predicting Explaining Contrasting, mapping, matching Constructing models 3. Applying Executing Carrying out Implementing Using 4. Analysing Differentiating Discriminating, distinguishing, focusing, Organising selecting Attributing Finding coherence, integrating, outlining, structuring Deconstructing 5. Evaluating Checking Coordinating, detecting, monitoring, testing Criticising Judging 6. Creating Generating Hypothesising Planning Designing Producing Constructing Source: Krathwohl & Anderson (2001) Copyright © Open University Malaysia (OUM)  TOPIC 2 FOUNDATION FOR ASSESSMENT: WHAT TO ASSESS  29 Note that the sequencing of some of the levels has been rearranged and renamed. The first two original levels of „knowledge‰ and „comprehension‰ were replaced with „remembering‰ and „understanding‰ respectively. The „synthesis‰ level was renamed with the term „creating‰. Note that in the original taxonomy the sequence was „synthesis‰ followed by „evaluate‰. In the modified taxonomy, the sequence was rearranged to „evaluating‰ followed by „creating‰. As you can see, the primary differences between the original and the revised taxonomy are not in the listings or rewordings from nouns to verbs, or in the renaming of some of the components, or even in the re-positioning of the last two categories. The major differences lie in the more useful and comprehensive additions of how the taxonomy intersects and acts upon different types and levels of knowledge factual, conceptual, procedural and metacognitive. (a) Factual knowledge refers to essential facts, terminologies, details or elements that learners must know or be familiar with in order to understand a discipline or solve a problem with it; (b) Conceptual knowledge is knowledge of classifications, principles, generalisations, theories, models or structures pertinent to a particular disciplinary area; (c) Procedural knowledge refers to information or knowledge that helps learners do something specific to a discipline, subject or area of study. It also refers to the methods of inquiry, very specific or finite skills, algorithms, techniques and particular methodologies; and (d) Metacognition is simply thinking about oneÊs thinking. More precisely, it refers to the processes used to plan, monitor and assess oneÊs understanding and performance. Activities such as planning how to approach a given learning task, monitoring comprehension and evaluating progress toward the completion of a task are metacognitive in nature. BloomÊs aim was to promote higher forms of thinking in education such as analysing, creating and evaluating rather than just teaching learners to remember facts (rote learning). Higher-order thinking (HOT) takes thinking to higher levels other than restating the facts and requires learners to do something with the facts such as understand them, infer from them, connect them to other facts and concepts, categorise them, manipulate them, put them together in new or novel ways, and apply them as we seek new solutions to new problems. Copyright © Open University Malaysia (OUM) 30  TOPIC 2 FOUNDATION FOR ASSESSMENT: WHAT TO ASSESS SELF-CHECK 2.1 1. Explain the differences between analysis and synthesis according to BloomÊs Taxonomy. 2. How is the revised version of BloomÊs Taxonomy different from the original version? ACTIVITY 2.2 1. Do you agree that BloomÊs Taxonomy is a hierarchy of cognitive abilities? Justify. 2. You need to be able to „analyse‰ before being able to „evaluate‰. Comment. 3. Using The Helpful Hundred, list the learning outcomes appropriate for the subject area you are teaching. Share all your answers with your coursemates in the myINSPIRE online forum. 2.4 ASSESSING AFFECTIVE OUTCOMES Affective characteristics involve the feelings or emotions of a person. Attitudes, values, self-esteem, locus of control, self-efficacy, interests, aspirations and anxiety are all examples of affective characteristics. Unfortunately, affective outcomes have not been a central part of our education system even though they are arguably as important as, or even more important than, any cognitive or psychomotor domain of learning outcomes targeted by schools. Some possible reasons for the lack of emphasis on affective outcomes include: (a) The belief that the development of appropriate feelings is the task of the family and religion. (b) The belief that appropriate feelings develop automatically from knowledge and experience with content and do not require any special pedagogical attention. Copyright © Open University Malaysia (OUM)  TOPIC 2 FOUNDATION FOR ASSESSMENT: WHAT TO ASSESS  31 (c) Attitudinal and value-oriented instructions are difficult to develop and assess because: (i) Affective goals are intangible; (ii) Affective outcomes cannot be attained in the typical periods of instruction offered in schools; (iii) Affective characteristics are considered to be private rather than public matters; and (iv) There are no sound methods to gather information about affective characteristics. However, affective goals are no more intangible than cognitive ones. Some have claimed that affective behaviours can be developed automatically when specific knowledge are taught while others argue that affective behaviours have to be explicitly developed in schools. Affective goals do not necessarily take longer to achieve in the classroom than cognitive goals. All that is required is to state a goal more concretely and behaviourally-oriented so that it can be assessed and monitored. There is also the belief that affective characteristics are private and should not be made public. While people value their privacy, the public also has the right to information. If the information gathered is needed to make a decision, then the gathering of such information is not generally considered an invasion of privacy. For example, if an assessment is used to determine whether a learner needs further attention such as special education, then gathering such information is not an invasion of privacy. On the other hand, if the information being sought-after is not relevant to the stated purpose, then gathering of such information is likely to be an invasion of privacy. Similarly, information about affective characteristics can be used for good or bad. For example, if a mathematics teacher discovers a learner has a negative attitude towards mathematics and ridicules that learner in front of the class, then the information has been misused. However, if the teacher uses the information to change his instructional methods so as to help the learner develop a more positive attitude towards mathematics, then the information has been used wisely. Krathwohl, Bloom and Bertram and their colleagues developed the affective domain in 1973 which deals with things emotionally such as feelings, values, appreciation, enthusiasm, motivation and attitudes. The five major categories which listed the simplest behaviour to the most complex behaviour are receiving, responding, valuing, organisation and characterisation (refer to Figure 2.4). Copyright © Open University Malaysia (OUM) 32  TOPIC 2 FOUNDATION FOR ASSESSMENT: WHAT TO ASSESS Figure 2.4: Krathwohl, Bloom and BertramÊs taxonomy of affective learning outcomes Let us read on to find about more about the taxonomy. (a) Receiving The behaviours at the receiving level require the learner to be aware of, willing to hear and focus his or her attention. Examples of verbs describing behaviours at the receiving level are ask, choose, describe, follow, give, hold, locate, name, point to, reply and so forth. For example, the learner: (i) Listens to others with respect; and (ii) Listens for and remembers the names of other learners. (b) Responding The behaviours at the responding level require the learner to be an active participant, attend to and react to a particular phenomenon, be willing to respond and gain satisfaction in responding (motivation). Examples of verbs describing behaviours at the responding level are answer, assist, aid, comply with, conform, discuss, greet, help, label, perform, practise, present, read, recite, report, select, tell, and write. For example, the learner: (i) Participates in class discussion; (ii) Gives a presentation; and (iii) Questions new ideals, concepts and models in order to fully understand them. Copyright © Open University Malaysia (OUM)  TOPIC 2 FOUNDATION FOR ASSESSMENT: WHAT TO ASSESS  33 (c) Valuing This level relates to the worth or value a person attaches to a particular object, phenomenon or behaviour. It ranges from simple acceptance to the more complex state of commitment. Valuing is based on the internalisation of a set of specified values while clues to these values are expressed in the learner as overt behaviours and are often identifiable. Examples of verbs describing behaviours at the valuing level are demonstrate, differentiate, explain, follow, form, initiate, invite, join, justify, propose, read, report, select, share, study and work. For example, the learner: (i) Demonstrates belief in the democratic process; (ii) Is sensitive towards individual and cultural differences (value diversity); (iii) Shows the ability to solve problems; (iv) Proposes a plan to social improvement; and (v) Follows through with commitment. (d) Organisation At this level, a person organises values into priorities by contrasting different values, resolving conflicts between them and creating a unique value system. The emphasis is on comparing, relating and synthesising values. Examples of verbs describing behaviours at the level of organisation are adhere to, alter, arrange, combine, compare, complete, defend, explain, formulate, generalise, identify, integrate, modify, order, organise, prepare, relate and synthesise. For example, the learner: (i) Recognises the need for balance between freedom and responsible behaviour; (ii) Accepts responsibility for his behaviour; (iii) Explains the role of systematic planning in solving problems; (iv) Accepts professional ethical standards; (v) Creates a life plan in harmony with abilities, interests and beliefs; and (vi) Prioritises time effectively to meet the needs of the organisation, family and self. Copyright © Open University Malaysia (OUM) 34  TOPIC 2 FOUNDATION FOR ASSESSMENT: WHAT TO ASSESS (e) Characterisation At this level, a personÊs value system controls his behaviour. The behaviour is pervasive, consistent, predictable and most importantly, characterises the learner. Examples of verbs describing behaviours at this level are act, discriminate, display, influence, listen, modify, perform, practise, propose, qualify, question, revise, serve, solve and verify. For example, the learner: (i) Shows self-reliance when working independently; (ii) Cooperates in group activities (displays teamwork); (iii) Uses an objective approach in problem-solving; (iv) Displays a professional commitment to ethical practice on a daily basis; (v) Revises judgment and changes behaviour in light of new evidence; and (vi) Values people for what they are, not how they look. Table 2.3 shows how the affective taxonomy may be applied to a value such as honesty. It traces the development of an affective attribute such as honesty from the „receiving‰ level up to the „characterisation‰ level where the value becomes a part of the individualÊs character. Table 2.3: An Affective Taxonomy for Honesty Individual Character Explanation Receiving (Attending) Aware that certain things are honest or dishonest Responding Saying that honesty is better and behaving accordingly Valuing Consistently (but not always) telling the truth Organisation Being honest in various situations Characterisation by a value or Honest in most situations, expects others to be honest value complex and interacts with others with all honesty SELF-CHECK 2.2 Explain the differences between characterisation and valuing according to the affective taxonomy of learning outcomes. Copyright © Open University Malaysia (OUM)  TOPIC 2 FOUNDATION FOR ASSESSMENT: WHAT TO ASSESS  35 ACTIVITY 2.3 1. The Role of Affect in Education „Some say schools should only be concerned with content.‰ „It is impossible to teach content without teaching affect as well.‰ „To what extent, if at all, should we be concerned with the assessment of affective outcomes?‰ Discuss the three statements in the context of the Malaysian education system. 2. Select any two values from the list of 16 universal values and design an affective taxonomy for each value as shown in Table 2.3. 3. „A student is operating at the responding level.‰ What does the statement mean? Explain. Share all your answers with your coursemates in the myINSPIRE online forum. 2.5 ASSESSING PSYCHOMOTOR LEARNING OUTCOMES The psychomotor domain includes physical movement, coordination and use of the motor-skill areas. Development of these skills requires practice and is measured in terms of speed, precision, distance, procedures or techniques in execution. The seven major categories listed from the simplest behaviour to the most complex are shown in Figure 2.5. Figure 2.5: The taxonomy of psychomotor learning outcomes Copyright © Open University Malaysia (OUM) 36  TOPIC 2 FOUNDATION FOR ASSESSMENT: WHAT TO ASSESS Let us read further to find out more about psychomotor learning. (a) Perception Perception is the ability to use sensory cues to guide motor activity. This ranges from sensory stimulation through cue selection to translation. Examples of verbs describing these types of behaviours are choose, describe, detect, differentiate, distinguish, identify, isolate, relate and select. For example, the learner: (i) Detects non-verbal communication cues from the coach; (ii) Estimates where a ball will land after it is thrown and then moving to the correct location to catch the ball; (iii) Adjusts heat of the stove to the correct temperature through smell and taste of food; and (iv) Adjusts the height of the ladder in relation to the point on the wall. (b) Set It includes mental, physical and emotional sets. These three sets are dispositions that predetermine a personÊs response to different situations (sometimes called mindset). Examples of verbs describing „set‰ are begin, display, explain, move, proceed, react, show, state and volunteer. For example, the learner: (i) Knows and acts upon a sequence of steps in a manufacturing process; (ii) Recognises his abilities and limitations; and (iii) Shows desire to learn a new process (motivation).  Note: This subdivision of the psychomotor domain is closely related with the „responding‰ subdivision of the affective domain. (c) Guided Response Guided response refers to the early stages of learning a complex skill which includes imitation and trial and error. Adequacy of performance is achieved by practicing. Examples of verbs describing „guided response‰ are copy, trace, follow, react, reproduce and respond. Copyright © Open University Malaysia (OUM)  TOPIC 2 FOUNDATION FOR ASSESSMENT: WHAT TO ASSESS  37 For example, the learner: (i) Performs a mathematical equation; (ii) Follows instructions when building a model of a kampung house; and (iii) Responds to hand signals of the coach while learning gymnastics. (d) Mechanism This is the intermediate stage in learning a complex skill. Learned responses have become habitual and the movements can be performed with some confidence and proficiency. Examples of verbs describing „mechanism‰ include assemble, calibrate, construct, dismantle, display, fasten, fix, grind, heat, manipulate, measure, mend, mix and organise. For example, the learner: (i) Uses a computer; (ii) Repairs a leaking tap; (iii) Fixes a three-pin electrical plug; and (iv) Rides a motorbike. (e) Complex Overt Response Complex overt response involves the skilful performance of motor acts that involve complex movement patterns. Proficiency is indicated by quick, accurate and highly coordinated performance requiring minimum energy. This category includes performing without hesitation and automatic performance. For example, tennis players such as Maria Sharapova and Serena Williams often utter sounds of satisfaction or expletives as soon as they hit a tennis ball. Similarly for golf players when they immediately realised that they have hit a bad shot. This is because they can tell by the feel of the act and the result that will follow. Examples of verbs describing „complex overt responses‰ are assemble, build, calibrate, construct, dismantle, display, fasten, fix, grind, heat, manipulate, measure, mend, mix, organise and sketch. For example, the learner: (i) Manoeuvres a car into a tight parallel parking spot; (ii) Operates a computer quickly and accurately; and (iii) Displays competence while playing the piano. Copyright © Open University Malaysia (OUM) 38  TOPIC 2 FOUNDATION FOR ASSESSMENT: WHAT TO ASSESS  Note that many of the verbs are the same as „mechanism‰, but will have adverbs or adjectives that indicate that the performance is quicker, better and more accurate. (f) Adaptation Skills are well-developed and the individual can modify movement patterns to fit special requirements. Examples of verbs describing „adaptation‰ are adapt, alter, change, rearrange, reorganise, revise and vary. For example, the learner: (i) Responds effectively to unexpected experiences; (ii) Modifies instructions to meet the needs of learners; (iii) Performs a task with a machine that it was originally not designed to do (assuming that the machine is not damaged and there is no danger in performing the new task). (g) Origination Origination is about creating new movements or patterns to fit a particular situation or specific problem. Learning outcomes emphasise creativity based upon highly developed skills. Examples of verbs describing „origination‰ are arrange, build, combine, compose, construct, create, design, initiate, make and originate. For example, the learner: (i) Constructs a new theory; (ii) Develops a new technique for goalkeeping; and (iii) Creates a new gymnastic routine. SELF-CHECK 2.3 Explain the differences between adaptation and guided response according the psychomotor taxonomy of learning outcomes Copyright © Open University Malaysia (OUM)  TOPIC 2 FOUNDATION FOR ASSESSMENT: WHAT TO ASSESS  39 ACTIVITY 2.4 „A student is operating at the origination level.‰ What does the statement mean? Explain. Share your answer with your coursemates in the myINSPIRE online forum. As a guide, Table 2.5 shows the allotment of time for each type of question. Table 2.5: Allotment of Time for Each Type of Question Task Approximate Time Per Item True-False 20 30 seconds Multiple-choice (factual) 40 60 seconds Multiple-choice (complex) 70 90 seconds Matching (5 stems or 6 choices) 2 4 minutes Short answers 2 4 minutes Multiple-choice (with calculations) 2 5 minutes Word problems (simple math) 5 10 minutes Short essays 15 20 minutes Data analysis or graphing 15 25 minutes Extended essays 35 50 minutes Once your questions are developed, make sure that you include clear instructions for the learners. For the objective items, specify that they should select one answer for each item and indicate the point value of each question, especially if you are allocating different weightage to different sections of the test. For essay items, indicate the point value and suggested time to be spent on the item. We will discuss different types of questions in more detail in Topics 3 and 4. If you are teaching a large class with close seating arrangements and are giving an objective test, you may want to consider administering several versions of your test to minimise the opportunities for cheating. This is done by creating versions of your test with different numberings of the items. Copyright © Open University Malaysia (OUM) 40  TOPIC 2 FOUNDATION FOR ASSESSMENT: WHAT TO ASSESS  Assessment of cognitive outcomes has remained the focus of most assessment systems all over the world because it is relatively easier to observe and measure.  Each domain of learning consists of subdivisions, starting from the simplest behaviour to the most complex thus forming taxonomy of learning outcomes.  When we evaluate or assess a human being, we are assessing or evaluating the behaviour of a person.  Every subject area has is unique repertoire of facts, concepts, principles, generalisations, theories, laws, procedures and methods that are transmitted to learners.  There are six levels in BloomÊs taxonomy of cognitive learning outcomes with the lowest level termed knowledge followed by five increasingly difficult levels of mental abilities, which are comprehension, application, analysis, synthesis and evaluation.  The six levels in the revised version of BloomÊs taxonomy are remembering, understanding, applying, analysing, evaluating and creating.  Affective characteristics involve the feelings or emotions of a person. Attitudes, values, self-esteem, locus of control, self-efficacy, interests, aspirations and anxieties are examples of affective characteristics.  The five major categories of the affective domain from the simplest behaviour to the most complex behaviour are receiving, responding, valuing, organisation and characterisation.  The psychomotor domain includes physical movement, coordination, and use of the motor-skill areas.  The seven major categories of the psychomotor domain from the simplest behaviour to the most complex are perception, set, guided response, mechanism, complex overt response, adaptation and origination. Copyright © Open University Malaysia (OUM)  TOPIC 2 FOUNDATION FOR ASSESSMENT: WHAT TO ASSESS  41 Affective learning outcomes Holistic assessment BloomÊs taxonomy Psychomotor learning outcome Cognitive learning outcomes The Helpful Hundred Copyright © Open University Malaysia (OUM) Topic  Planning the Classroom Test 3 LEARNING OUTCOMES By the end of this topic, you should be able to: 1. Plan a classroom test by deciding on its purpose; 2. Specify the learning objectives of a classroom test; 3. Develop test specifications of a classroom test; 4. Select best item types of a classroom test; 5. Decide on the rubrics or marking schemes of a classroom test; 6. Prepare the test items for a classroom test; and 7. Describe the considerations in preparing relevant test items.  INTRODUCTION In this topic we will focus on methods of planning classroom tests. Testing is part of the teaching and learning process. The importance of planning and writing a reliable, valid and fair test cannot be underestimated. Designing tests is an important part of assessing learnersÊ understanding of course content and their level of competency in applying what they have learned. Whether you use low- stake quizzes or high-stake mid and final semester examinations, careful design of the tests will help provide more calibrated results. Assessments should reveal how well learners have learnt based on what the teachers want them to learn while the instructions facilitates their learning. Thus, solely conducting a summative assessment at the end of a teaching programme is not sufficient. It is helpful to think about assessing learners at every stage of the planning process. Identifying ways in which to assess their learners help determine the most suitable learning activities. Copyright © Open University Malaysia (OUM)  TOPIC 3 PLANNING THE CLASSROOM TEST  43 In this topic we will discuss the general guidelines applicable to most assessment tools. Topics 4 and 5 will discuss in detail the objective and essay tests. Authentic assessment tools such as projects and portfolios will be discussed in the respective topics. 3.1 PURPOSE OF CLASSROOM TESTING Tests can be referred to as: (a) Traditional paper and pencil or computer-based tests in the form of multiple- choice, short answer or essay tests; and (b) Performance assessments such as projects, interviews, presentations or other alternative assessments. Tests provide teachers with an objective feedback as to how much learners have learnt and understand the subject taught. Commercially published achievement tests can provide, to some extent, evaluation of the knowledge levels of individual learners but only limited instructional guidance in assessing a wide range of skills taught in any given classroom. Teachers know their learners. Tests developed by the individual teachers for use in their own class are the most instructionally relevant. Teachers can tailor tests to emphasise the information they consider important and to match the ability levels of their learners. If carefully constructed, classroom tests can provide teachers with accurate and useful information about the knowledge retained by their learners. The key to this process is the test questions that are used to elicit evidence of learning. Test questions and tasks are not just a planning tool; they also form an essential part of the teaching sequence. Incorporating the tasks into teaching and using the evidence of the learnersÊ learning to determine what happens next in the lesson is truly an embedded formative assessment. „Sharing high quality questions may be the most significant thing we can do to improve the quality of student learning,‰ (William, D., 2011). Copyright © Open University Malaysia (OUM) 44  TOPIC 3 PLANNING THE CLASSROOM TEST 3.2 PLANNING THE CLASSROOM TEST A well-constructed test must have high quality items. A well-constructed test is an instrument that is able to provide accurate measure of the test takerÊs ability within a particular domain. It is worth spending time to write high quality items for tests. In order to produce high quality questions, the test construction has to be properly planned. The following are the six steps of test planning: (a) Deciding its purpose; (b) Specifying the learning objectives; (c) Developing test specifications; (d) Selecting best item types; (e) Deciding its rubrics or marking scheme; and (f) Preparing test items. We will look at these in detail in the following subtopics. 3.2.1 Deciding Its Purpose The first step in test planning is to decide on the purpose of the test. Tests can be used for many different purposes. If the test is to be used formatively, the test can indicate precisely what the learner needs to study and to what level. The purpose of formative tests is to assess progress and to direct the learning process. These tests will have limited sample of learning outcomes. Teachers must prepare a sufficient mix and difficulty levels of items. These items are used to make corrective prescriptions such as a review for the whole group and practice exercises for some learners. If the test is to be used in a summative manner, the test will have broad coverage of objectives. Copyright © Open University Malaysia (OUM)  TOPIC 3 PLANNING THE CLASSROOM TEST  45 Another important purpose of testing is selection. The test is used to make decisions about whether or not candidates will be admitted, pass or fail. Tests that lead to an assessment and provide evidence that a given level of knowledge or skill has been mastered (for example by awarding course credits) are referred to as summative tests. Tests can also serve a diagnostic purpose. In such cases, the test is used to provide learners with insights into gaps in their current knowledge and skill sets. Alternatively, tests can also be used to motivate learners to exhibit effective studying behaviour. 3.2.2 Specifying the Learning Objectives The next step is to consider the learning objectives and the relative importance of the learning objectives. Teachers will have to select the appropriate knowledge and skills to be assessed and include more questions with regard to the more important learning objectives. The learning objectives that the teachers would like to emphasise on will determine not only what materials to include in the test but also the specific form of the test. For example, if it is important for learners to be able to solve long division problems rapidly, consider giving a speed test. If it is important for learners to understand how historical events affect one another, short answer or essay questions might be appropriate. If it is important that learners remember dates, multiple-choice or fill-in-the-blank questions might be appropriate. 3.2.3 Developing Test Specifications Making a test blueprint or Table of Specifications is the next important step that teachers should do. The table describes the content, the behaviour of the learners and the number of questions in the test corresponding to the number of hours devoted to the learning objectives in class. In fact, exactly how many questions to include in a test is based on the importance of the objectives, the type of questions, the subject matter and the amount of time available for testing. Copyright © Open University Malaysia (OUM) 46  TOPIC 3 PLANNING THE CLASSROOM TEST Table of Specifications is a plan prepared by the classroom teacher as a basis for the test construction. It is a two-way table which describes the topics to be covered in a test and the number of items associated with each topic. The table ensures that a fair and representative sample of questions appear in the test. Teachers cannot measure every topic or objective and cannot ask every question they may wish to ask. A Table of Specifications allows the teacher to construct a test which focuses on the key areas and weightage (in percentage) given in different sections of the test based on their importance. A Table of Specifications provides the teacher with evidence that a test has content validity and that it covers what should be covered. This table also allows the teacher to view the test as a whole. A sample of the Table of Specifications shown in Table 3.1 has content on one column and cognitive levels across the top. However, teachers could also arrange the content across the top and levels down the column. In this sample, the teacher who prepared the table grouped „Remembering‰ and „Understanding‰ levels together. It is very likely that he believed a straight recall was too simple to be considered as real learning. Table 3.1: A Sample of Table of Specifications Levels Content Remembering and Applying Analysing, Evaluating Total Understanding (%) (%) and Creating (%) (%) Topic 1 15 15 30 60 Topic 2 10 20 10 40 Total 25 35 40 100 For a Table of Specifications to be useful, it normally contains more details than the sample shown in Table 3.1. The number of questions to be included in a test is based on the importance of the objectives, the type of questions, the subject matter and the amount of time available for testing. A more detailed Table of Specifications is shown in Table 3.2. Copyright © Open University Malaysia (OUM)  TOPIC 3 PLANNING THE CLASSROOM TEST  47 Copyright © Open University Malaysia (OUM) 48  TOPIC 3 PLANNING THE CLASSROOM TEST In the example shown in Table 3.2, the vertical columns on the left of the 2-way table show a list of the topics covered in class and the amount of time spent on those topics. The topics can also be further subdivided into subtopics such as „Subtract two numbers without regrouping 2-digit numbers from a 2-digit number‰ under the topic „Subtraction within the range of 1000‰. The amount of time spent in the topics as shown in the column „Hours of Interaction‰ can be used as a basis to compute the weightage or percentage and the number of questions or items for each topic. For example, the teacher has spent 20 hours teaching the three topics of which 6 hours are allotted to „Addition with the highest total of 1000‰. Thus, 6 hours from a total of 20 hours amount to 30% or 9 items from the total 30 items as planned by the teacher. The teacher might have a reason for allocating 25%, 35% and 40% for the levels „Remembering‰, „Understanding‰ and „Applying‰ respectively. Perhaps he is trying to train his Year 2 learners to pay more attention to the „thinking‰ questions. The 25% of „Remembering‰ level is actually 7.5 questions, 35% of „Understanding‰ level is 10.5 questions and 40% of „Applying‰ level is 12 questions. The total 30 is not affected as the number 7.5 and 10.5 are conveniently rounded up to 8 and 10. The cells in the # columns can be arbitrarily filled or computed using a simple formula. In the first # column, the topic „Addition...‰ under the level „Remembering‰ should be 25% 9 = 2.25, the topic „Subtraction...‰ under „Remembering‰ is 25% 9 = 2.25 and the topic „Multiplication...‰ under „Remembering‰ is 25% 12 = 3. The teacher can either round up the numbers 2.25, 2.25, 3 to 3, 2, 3 or 2, 3, 3. The teacher, especially one who is newly trained, is advised to have this Table of Specifications together with the subject syllabus reviewed by the subject matter expert or the subject Head of Department to confirm whether the test plan would actually measure what it set out to measure. When the test items have been drafted and assembled, it is advisable to once again submit the draft test paper and the Table of Specifications to the Head of Department or the recognised subject matter expert to evaluate whether the test items do, in actual fact, assess the defined content. Content validity is different from face validity. Face validity assesses whether the test „looks valid‰ to the examinees who sit for the test whereas content validity requires recognised subject matter experts to evaluate whether the test items assess the defined content. Copy