HPGD2303 Educational Assessment PDF

HPGD2303 Educational Assessment Copyright © Open University Malaysia (OUM)  HPGD2303 EDUCATIONAL ASSESSMENT Prof Dr John Arul Phillips Copyright © Open University Malaysia (OUM) Fourth edition 2021 (CS) Third edition 2020 Second edition 2017 First edition 2011...

HPGD2303 Educational Assessment Copyright © Open University Malaysia (OUM)  HPGD2303 EDUCATIONAL ASSESSMENT Prof Dr John Arul Phillips Copyright © Open University Malaysia (OUM) Fourth edition 2021 (CS) Third edition 2020 Second edition 2017 First edition 2011 Developed by Centre for Instructional Design and Technology, OUM. Copyright © Open University Malaysia (OUM), HPGD2303 All rights reserved. No part of this work may be reproduced in any form or by any means without the written permission of the President of Open University Malaysia (OUM). www.oum.edu.my Copyright © Open University Malaysia (OUM) Table of Contents Course Guide ixăxix Course Assignment Guide xxiăxxv Topic 1 Roles of Assessment in Teaching and Learning 1 1.1 Tests, Measurement, Evaluation and Assessment 2 1.2 Why Do We Assess? 5 1.3 General Principles of Assessment 10 1.4 Types of Assessment 11 1.4.1 Formative vs Summative Assessments 11 1.4.2 Norm-referenced vs Criterion-referenced 15 Tests 1.5 Current Trends in Assessment 17 Summary 19 Key Terms 19 Topic 2 Foundation for Assessment: What to Assess 20 2.1 What to Assess 21 2.2 Three Types of Learning Outcomes 22 2.3 Assessing Cognitive Learning Outcomes 24 2.4 Assessing Affective Outcomes 30 2.5 Assessing Psychomotor Learning Outcomes 35 Summary 40 Key Terms 41 Topic 3 Planning the Classroom Test 42 3.1 Purpose of Classroom Testing 43 3.2 Planning the Classroom Test 44 3.2.1 Deciding Its Purpose 44 3.2.2 Specifying the Learning Objectives 45 3.2.3 Developing Test Specifications 45 3.2.4 Selecting Best Item Types 49 3.2.5 Preparing Rubrics or Marking Schemes 50 3.2.6 Preparing Test Items 51 3.3 Assessing TeacherÊs Own Test 52 Summary 53 Key Terms 54 Copyright © Open University Malaysia (OUM) iv  TABLE OF CONTENTS Topic 4 Constructing Objective Test Items 55 4.1 What is an Objective Test? 56 4.2 Short-answer Questions 57 4.3 True-false Questions 60 4.3.1 Limitations of True-false Questions 61 4.3.2 Suggestions for Construction of True-false Questions 62 4.4 Matching Questions 64 4.4.1 Advantages of Matching Questions 65 4.4.2 Limitations of Matching Questions 66 4.4.3 Suggestions for Writing Good Matching Questions 67 4.5 Multiple-choice Questions 68 4.5.1 Parts of a Multiple-choice Question 69 4.5.2 Construction of Multiple-choice Questions 70 4.5.3 Advantages and Limitations of MCQs 75 4.5.4 Constructing MCQs to Measure Higher-order 77 Thinking (HOT) Summary 81 Key Terms 82 Topic 5 Constructing Essay Questions 83 5.1 What is an Essay Question? 84 5.2 Types of Essays 85 5.3 Why are Essay Questions Used? 86 5.4 Deciding Whether to Use Essay Or Objective Questions 87 5.5 Limitations of Essay Questions 88 5.6 Some Misconceptions about Essay Questions 89 5.7 Some Guidelines for Constructing Essay Questions 91 5.8 Verbs Describing Various Kinds of Mental Tasks 98 5.9 Marking Essay Questions 103 Summary 108 Key Terms 109 Topic 6 Authentic Assessments 110 6.1 What is Authentic Assessment? 111 6.1.1 Types of Authentic Assessments 112 6.1.2 Characteristics of Authentic Assessments 113 6.2 How to Create Authentic Assessments? 115 6.2.1 Advantages of Authentic Assessments 116 6.2.2 Criticisms of Authentic Assessments 117 6.3 Differences between Traditional Assessment and 118 Authentic Assessment Copyright © Open University Malaysia (OUM)  TABLE OF CONTENTS  v 6.4 Assessment Tools 121 6.4.1 Scoring Rubrics 122 6.4.2 Rating Scales 124 6.4.3 Checklists 126 Summary 127 Key Terms 127 Topic 7 Project and Portfolio Assessments 128 7.1 Project Assessment 129 7.1.1 Assessing Learners with Projects 132 7.1.2 Designing Effective Projects 135 7.1.3 Possible Problems with Project Work 138 7.1.4 Group Work in Projects 139 7.1.5 Assessing Project Work 140 7.1.6 Assessing Process in a Project 145 7.2 Portfolio Assessment 149 7.2.1 What is Portfolio Assessment? 150 7.2.2 Rationale for Portfolio Assessment 150 7.2.3 Types of Portfolios 151 7.2.4 How to Develop a Portfolio 152 7.2.5 Advantages and Disadvantages of Portfolio 153 Assessment 7.2.6 How and When Should a Portfolio be Assessed? 155 Summary 158 Key Terms 159 Topic 8 Test Reliability and Validity 160 8.1 What is Reliability? 160 8.2 The Reliability Coefficient 162 8.3 Methods to Estimate the Reliability of a Test 165 8.4 Inter-rater and Intra-rater Reliability 169 8.5 Types of Validity 171 8.6 Factors Affecting Reliability and Validity 175 8.7 Relationship between Reliability and Validity 176 Summary 178 Key Terms 179 Copyright © Open University Malaysia (OUM) vi  TABLE OF CONTENTS Topic 9 Appraising Classroom Tests and Item Analysis 180 9.1 What is Item Analysis? 181 9.2 Steps in Item Analysis 181 9.3 The Difficulty Index 183 9.4 The Discrimination Index 185 9.5 Application of Item Analysis on Essay-type Question 188 9.6 Relationship between Difficulty Index and Discrimination 191 Index 9.7 Distractor Analysis 193 9.8 Practical Approach to Item Analysis 194 9.9 Usefulness of Item Analysis to Teachers 196 9.10 Cautions in Interpreting Item Analysis Results 198 9.11 Item Bank 199 9.12 Psychometric Softwares 200 Summary 201 Key Terms 202 Topic 10 Analysis and Interpretation of Test Scores 203 10.1 Why Do We Use Statistics? 204 10.2 Describing Test Scores 206 10.3 Standard Scores 214 10.4 The Normal Curve 218 Summary 219 Key Terms 220 References 221 Copyright © Open University Malaysia (OUM) COURSE GUIDE Copyright © Open University Malaysia (OUM) Copyright © Open University Malaysia (OUM) Table of Contents Welcome to HPGD2303 x What will You Gain from this Course? x Description of the Course Aim of the Course Course Outcomes How can You Get the Most Out of this Course? xii Learning Package Course Synopsis Text Arrangement Guide How will You Be Assessed? xv What Support will You Get in Studying this Course? xv Seminars myINSPIRE Online Discussion Facilitator Library Resources How should You Study for this Course? xvi Time Commitment for Studying Proposed Study Strategy Copyright © Open University Malaysia (OUM) x  COURSE GUIDE WELCOME TO HPGD2303 EDUCATIONAL ASSESSMENT Welcome to HPGD2303 Educational Assessment, which is one of the required courses for the Postgraduate Diploma in Teaching programme. The course assumes no previous knowledge of educational assessment but you are encouraged to tap into your experiences as a teacher, instructor, lecturer or trainer and relate them to the principles of assessment discussed. This course is worth 3 credit hours and should be covered over 8 to 15 weeks. WHAT WILL YOU GAIN FROM THIS COURSE? Description of the Course The course discusses the differences between testing, measurement and assessment or evaluation. The focus is on the role of assessment and followed by discussion on „what to assess‰, which forms the foundation of assessment. With regard to the „what‰, emphasis is on cognitive outcomes, affective outcomes and psychomotor outcomes. Subsequent topics cover the aspect of how to assess with emphasis on higher-order thinking skills. Besides the usual objective and essay tests, other authentic assessment techniques such as projects and portfolios are presented. Also discussed are techniques to determine the effectiveness of various assessment approaches focussing on reliability, validity and item analysis. Finally, various basic statistical procedures are presented in the analysis of assessment results and their interpretation. Aim of the Course The main aim of the course is to provide learners with a foundation on the principles and theories of educational testing and assessment as well as their applications in the classroom. Copyright © Open University Malaysia (OUM)  COURSE GUIDE  xi Course Learning Outcomes By the end of this course, you should be able to do the following: 1. Apply the different principles in the development of assessment techniques for classroom setting. 2. Plan and construct different classroom tests for different purposes. 3. Perform classroom tests using statistics to analyse and interpret assessment results. STUDY SCHEDULE It is a standard OUM practice that learners accumulate 40 study hours for every credit hour. As such, for a 3 credit hour course, you are expected to spend 120 study hours. Figure 1 shows the student learning time (SLT). Figure 1: Student Learning Time Copyright © Open University Malaysia (OUM) xii  COURSE GUIDE HOW CAN YOU GET THE MOST OUT OF THIS COURSE? Learning Package In this Learning Package you are provided with three types of course materials: 1. The Course Guide you are currently reading; 2. The Course Content (consisting of 10 topics); 3. The Course Assessment Guide (which describes the assignments to be submitted and the examinations you have to sit for); and 4. The Course Online Platform i.e. myINSPIRE Online Discussion. Please ensure that you have all of these materials and the correct url. Course Synopsis To enable you to achieve the four course outcomes, the course content has been divided into 10 topics. Specific learning outcomes are stated at the start of each topic indicating what you should be able to achieve after completing each topic. Topic Title Week 1. The Role of Assessment in Teaching and Learning 1 2. Foundation for Assessment: What to Assess 2ă3 3. Planning the Classroom Test 3 4. Constructing Objective Test Items 4ă5 5. Constructing Essay Tests 6ă7 6. Authentic Assessment 8 7. Project and Portfolio Assessments 8ă9 8. Test Reliability and Validity 10 9. Appraising Classroom Tests and Item Analysis 11 10. Analysis and Interpretation of Assessment Scores 12 Copyright © Open University Malaysia (OUM)  COURSE GUIDE  xiii Topic 1 discusses the differences between testing, measurement, evaluation and assessment, the role of assessment in teaching and learning and some general principles of assessment. Also explored is the difference between formative and summative assessments as well as the difference between criterion- and norm- referenced tests. The topic concludes with a brief discussion of the current trends in assessment. Topic 2 discusses the behaviours to be tested focussing on cognitive, affective and psychomotor learning outcomes and reasons why assessments of the latter two outcomes are ignored. Topic 3 provides some useful guidelines to help teachers plan valid, reliable and useful assessments. The discussion includes determining what is to be measured and minimising measurement irrelevancies. The topic will also guide teachers to devise strategies to measure the domain well. Example of the Table of Specifications, a 2-way table is presented. Topic 4 discusses the design and development of objective tests in the assessment of various kinds of behaviours with emphasis on the limitations and advantages of using this type of assessment tool. Topic 5 examines the role of essay tests in assessing various kinds of learning outcomes as well as its limitations and strengths, and the procedures involved in the design of good essay questions. Topic 6 introduces a form of assessment in which learners are assigned to perform real-world tasks that demonstrate meaningful application of essential knowledge and skills. Teachers will be able to understand in what way Authentic Assessment is similar to or different from Traditional Assessment. Emphasis is also given to scoring rubrics. Topic 7 discusses in detail two examples of authentic assessments, namely portfolio and project assessments. Guidelines to portfolio entries and project works and evaluation criteria are discussed in detail. Topic 8 focuses on basic concepts of test reliability and validity. The topic also includes methods to estimate the reliability of a test and factors to increase reliability and validity of a test. Copyright © Open University Malaysia (OUM) xiv  COURSE GUIDE Topic 9 examines the concept of item analysis and the different procedures for establishing the effectiveness of objective and essay-type tests focussing on item difficulty and item discrimination. The topic concludes with a brief explanation of item bank. Topic 10 focuses on the analysis and interpretation of the data collected by tests. For quantitative analysis of data, various statistical procedures are used. Some of the statistical procedures used in the interpretation and analysis of assessment results are measures of central tendency and correlation coefficients. Text Arrangement Guide Before you go through this module, it is important that you note the text arrangement. Understanding the text arrangement will help you to organise your study of this course in a more objective and effective way. Generally, the text arrangement for each topic is as follows: Learning Outcomes: This section refers to what you should achieve after you have completely covered a topic. As you go through each topic, you should frequently refer to these learning outcomes. By doing this, you can continuously gauge your understanding of the topic. Self-Check: This component of the module is inserted at strategic locations throughout the module. It may be inserted after one subtopic or a few subtopics. It usually comes in the form of a question. When you come across this component, try to reflect on what you have already learnt thus far. By attempting to answer the question, you should be able to gauge how well you have understood the subtopic(s). Most of the time, the answers to the questions can be found directly from the module itself. Activity: Like Self-Check, the Activity component is also placed at various locations or junctures throughout the module. This component may require you to solve questions, explore short case studies, or conduct an observation or research. It may even require you to evaluate a given scenario. When you come across an Activity, you should try to reflect on what you have gathered from the module and apply it to real situations. You should, at the same time, engage yourself in higher order thinking where you might be required to analyse, synthesise and evaluate instead of only having to recall and define. Copyright © Open University Malaysia (OUM)  COURSE GUIDE  xv Summary: You will find this component at the end of each topic. This component helps you to recap the whole topic. By going through the summary, you should be able to gauge your knowledge retention level. Should you find points in the summary that you do not fully understand, it would be a good idea for you to revisit the details in the module. Key Terms: This component can be found at the end of each topic. You should go through this component to remind yourself of important terms or jargon used throughout the module. Should you find terms here that you are not able to explain, you should look for the terms in the module. References: The References section is where a list of relevant and useful textbooks, journals, articles, electronic contents or sources can be found. The list can appear in a few locations such as in the Course Guide (at the References section), at the end of every topic or at the back of the module. You are encouraged to read or refer to the suggested sources to obtain the additional information needed and to enhance your overall understanding of the course. HOW WILL YOU BE ASSESSED? Assessment Method Please refer to myINSPIRE. WHAT SUPPORT WILL YOU GET IN STUDYING THIS COURSE? Seminars There are 15 hours of seminars or face-to-face interactions supporting the course. This consists of four to five seminar sessions of 3 hours each. You will be notified of the dates, times and location of these seminars together with the name and phone number of your tutor as soon as you are assigned to a tutorial group. Copyright © Open University Malaysia (OUM) xvi  COURSE GUIDE MyINSPIRE Online Discussion Besides the face-to-face tutorial sessions, you have the support of online discussions. You should interact with other learners and your facilitators using myINSPIRE. Your participation and contributions to the online discussion will greatly enhance your understanding of the course content, guide you in the assignments as well as prepare for the examination. Facilitator Your facilitator will mark your assignments and assist you during the course. Do not hesitate to discuss during the tutorial sessions or online if: Ć You do not understand any part of the course content or the assigned readings; Ć You have difficulty with the self-tests and activities; or Ć You have a question or problem with the assignments. Library Resources The Digital Library has a large collection of books and journals which you can access using your learner ID. HOW SHOULD YOU STUDY FOR THIS COURSE? 1. Time Commitment for Studying You should plan to spend about six to eight hours per topic reading the notes, performing the self-tests and activities as well as referring to the suggested readings. You must also schedule your time to participate in online discussions. It is often more convenient for you to distribute the hours over a number of days rather than spend one whole day per week on your studies. Some topics may require more work than others may, therefore, it is suggested that, on average, you spend approximately three days per topic. Copyright © Open University Malaysia (OUM)  COURSE GUIDE  xvii 2. Proposed Study Strategy The following is a proposed strategy for working through the course. If you encounter any trouble, discuss it with your facilitator either online or during the seminar sessions. Remember, the facilitator is there to help you. (a) The most important step is to read the contents of this Course Guide thoroughly. (b) Organise a study schedule. Note the time you are expected to spend on each topic and the date for submission of assignments as well as seminar and examination dates. These are stated in your Course Assessment Guide. Note down all this information in one place such as your diary or a wall calendar. Jot down your own dates for working on each topic. You have some flexibility as there are 10 topics spread over a period of 14 weeks. (c) Once you have created your own study schedule, make every effort to „stick to it‰. The main reason learners are unable to cope is because they lag behind in their coursework. (d) To begin reading a topic: Ć Remember that in distance learning, much of your time will be spent READING the course content. Study the list of topics given at the beginning of each topic and examine the relationship of the topic to the other topics. Ć Read the topicÊs learning outcomes (what is expected of you). Do you already know some of the key points in the topic? What are the things you do not know? Ć Read the introduction (to see how it connects with the previous topic). Ć Work through the topic (contents of the topic are arranged to provide a sequence for you to follow through). Copyright © Open University Malaysia (OUM) xviii  COURSE GUIDE Ć As you work through the topic, you will be directed to perform the self-tests at appropriate intervals throughout the topic. This will enable you to find out if you understand what you have just read. Ć Work out the activities stated (to see if you can apply the concepts learned to real-world situations). (f) When you have completed the topic, review the learning outcomes to confirm that you have achieved them and are able to do what is required. (g) If you are confident, you can proceed to the next topic. Proceed topic by topic through the course and try to pace your study so that you keep to your planned schedule. (h) After completing all topics, review the course and prepare yourself for the final examination. Check that you have achieved all the topicsÊ learning outcomes and the course objectives (listed in this Course Guide). FINAL REMARKS Once again, welcome to the course. To maximise your gain from this course you should try at all times to relate what you are studying to the real world. Look at the environment in your organisation and ask yourself whether the ideas discussed apply. Most of the ideas, concepts and principles you learn in this course have practical applications. It is important to realise that much of what we do in education and training has to be based on sound theoretical foundations. The contents of this course merely address the basic principles and concepts of assessment in education. You are advised to go beyond the course and continue with lots of self-study to further enhance your knowledge on educational assessment. We wish you success with the course and hope that you will find it interesting, useful and relevant in your development as a professional. We hope you will enjoy your experience with OUM and we would like to end with a saying by Confucius ă „Education without thinking is labour lost‰. Copyright © Open University Malaysia (OUM) COURSE ASSIGNMENT GUIDE Copyright © Open University Malaysia (OUM) Copyright © Open University Malaysia (OUM) Table of Contents Introduction xxii Academic Writing xxii (a) Plagiarism (b) Documenting Sources (i) What is Plagiarism? (ii) How Can I Avoid Plagiarism?  Direct Citation  Indirect Citation  Third-party Citation (c) Referencing  Journal Articles  Online Journal  Webpage  Book  Article in Book  Printed Newspaper Details about Assignments xxvi Copyright © Open University Malaysia (OUM) xxii  COURSE ASSIGNMENT GUIDE INTRODUCTION This guide explains the basis in which you will be assessed in this course during the semester. It contains details of the facilitator-marked assignments, final examination and participation required for the course. One element in the assessment strategy of the course is that all learners and facilitators should be provided with the same information about how the learners will be assessed. Therefore, this guide also contains the marking criteria that facilitators will use in assessing your work. Please read through the whole guide. It should be read at the start of the course. ACADEMIC WRITING (a) Plagiarism (i) What Is Plagiarism? Any written assignment (essays, project, take-home tests and others) submitted by a learner must not be deceptive with regard to the abilities, knowledge or amount of work contributed by the learner. There are many ways that this rule can be violated. Among them are: Paraphrases A closely reasoned argument of an author is paraphrased but the learner does not acknowledge doing so. (Clearly, all our knowledge is derived from somewhere but detailed arguments from clearly identifiable sources must be acknowledged.) Outright Large sections of the paper are simply copied from other sources plagiarism and the copied parts are not acknowledged as quotations. Other These often include essays written by other students or sold by sources unscrupulous organisations. Quoting from such papers is perfectly legitimate if quotation marks are used and the source is cited. Works by Taking credit deliberately or not deliberately for works others produced by others without giving proper acknowledgement. These works include photographs, charts, graphs, drawings, statistics, video clips, audio clips, verbal exchanges such as interviews or lectures, performances on television and texts printed on the Web. Duplication The student submits the same essay for two or more courses. Copyright © Open University Malaysia (OUM)  COURSE ASSIGNMENT GUIDE  xxiii (ii) How Can I Avoid Plagiarism?  Insert quotation marks for „copy and paste‰ clauses, phrases, sentences and paragraphs, and cite the original source;  Paraphrase the clause, phrase, sentence or paragraph in your own words and cite your source;  Adhere to the APA (American Psychological Association) stylistic format, whichever applicable, when citing a source and when writing out the bibliography or reference page;  Attempt to write independently without being overly dependent on information from another personÊs original works; and  Educate yourself on what may be considered as common knowledge (no copyright necessary), public domain (copyright has expired or not protected under copyright law) or copyright (legally protected). (b) Documenting Sources Whenever you quote, paraphrase, summarise or refer to the work of another, you are required to cite its original source documentation. Offered here are some of the most commonly cited forms of material.  Direct Citation Simply having a thinking skill is no assurance that children will use it. In order for such skills to become part of day-to-day behaviour, they must be cultivated in an environment that value and sustains them. „Just as childrenÊs musical skills will likely lay fallow in an environment that doesnÊt encourage music, learnerÊs thinking skills tend to languish in a culture that doesnÊt encourage thinking‰ (Tishman, Perkins & Jay, 1995, p. 5).  Indirect Citation According to Wurman (1988), the new disease of the 21st century will be information anxiety, which has been defined as the ever-widening gap between what one understands and what one thinks one should understand. Copyright © Open University Malaysia (OUM) xxiv  COURSE ASSIGNMENT GUIDE (c) Referencing All sources that you cite in your paper should be listed in the Reference section at the end of your paper. Here is how you should list your references: Journal Article DuFour, R. (2002). The learning-centred principal: Educational Leadership, 59(8). 12ă15. Online Journal Evnine, S. J. (2001). The universality of logic: On the connection between rationality and logical ability [Electronic version]. Mind, 110, 335ă367. Webpage National Park Service. (2003, February 11). Abraham Lincoln Birthplace National Historic Site. Retrieved February 13, 2003, from http://www.nps.gov/abli/ Book Naisbitt, J., & Aburdence, M. (1989). Megatrends 2000. London, England: Pan Books. Article in a Nickerson, R. (1987). Why teach thinking? In J. B. Book Baron & R. J. Sternberg (Eds.). Teaching thinking skills: Theory and practice. New York, NY: W.H. Freeman and Company. 27ă37. Printed Holden, S. (1998, May 16). Frank Sinatra dies at 82: Newspaper Matchless stylist of pop. The New York Times, pp. A1, A22ăA23. DETAILS ABOUT ASSIGNMENTS Facilitator-Marked Assignment (FMA) You will be able to complete the assignment from the information and materials contained in your suggested readings and course content. However, it is desirable in all graduate level education to demonstrate that you have read and researched more widely than the required minimum. Using other references will give you a broader perspective and may provide a deeper understanding of the subject. When you have completed the assignment, submit it together with a FMA form to your facilitator. Make sure that your assignment reaches the facilitator on or before the deadline. Copyright © Open University Malaysia (OUM)  COURSE ASSIGNMENT GUIDE  xxv General Criteria for Assessment of FMA In general, your facilitator will expect you to write clearly, using correct spelling (please use your spell checker) and grammar. Your facilitator will look for the following:  You have critically thought about issues raised in the course;  You have considered and appreciated different points of views including those in the course;  You gave your own views and opinions;  You stated your arguments clearly with supporting evidences and proper referencing of sources; and  You have drawn from your own experiences. Copyright © Open University Malaysia (OUM) xxvi  COURSE ASSIGNMENT GUIDE Copyright © Open University Malaysia (OUM) Topic  Roles of Assessment in 1 Teaching and Learning LEARNING OUTCOMES By the end of this topic, you should be able to: 1. Differentiate between tests, measurement, evaluation and assessment; 2. Explain the roles of assessment in teaching and learning; 3. Explain the general principles of assessment; 4. Differentiate between formative and summative assessments; and 5. Justify when norm-referenced and criterion-referenced tests are adopted.  INTRODUCTION The topic discusses the difference between tests, measurement, evaluation and assessment, the roles of assessment in teaching and learning, and some general principles of assessment. Also explored is the difference between formative and summative assessments as well as the difference between criterion and norm- referenced tests. The topic concludes with a brief discussion on the current trends in assessment. Copyright © Open University Malaysia (OUM) 2  TOPIC 1 ROLES OF ASSESSMENT IN TEACHING AND LEARNING Can you differentiate between tests, measurement, evaluation and assessment (see Figure 1.1)? Figure 1.1: Can you differentiate between tests, measurement, and evaluation and assessment? 1.1 TESTS, MEASUREMENT, EVALUATION AND ASSESSMENT It is not surprising that many people are confused with the fundamental differences between tests, measurement, evaluation and assessment as they are used in education. The following explanation on these concepts will help clarify the confusion. (a) Tests Most people are familiar with tests because all of us, at some point in our lives, have taken some form of tests. In school, tests are given to measure our academic aptitude and indirectly to evaluate whether we have gained from the teaching by the teacher. At the workplace, tests are conducted to select suitable persons for specific jobs, tests are used as the basis for job promotions and tests are used to encourage re-learning. Physicians, Copyright © Open University Malaysia (OUM)  TOPIC 1 ROLES OF ASSESSMENT IN TEACHING AND LEARNING  3 lawyers, insurance consultants, real-estate agents, engineers, civil servants and many other professions are required to take tests to demonstrate their competence in specific areas and in some cases to be licensed to practise their profession or trade. Throughout their professional careers, teachers, counsellors and school administrators are required to conduct, score and interpret a wide variety of tests. For example, school administrators rate the performance of individual teachers and school counsellors record the performance of their clients. It is possible that a teacher may construct, administer and grade thousands of tests during his or her career! Test items can be written in various formats including multiple-choice questions, matching, true or false, short answer and essay. These formats vary in strengths and weaknesses. No one format is ideal for all circumstances. According to the joint committee of the American Psychological Association (APA), the American Educational Research Association (AERA) and National Council on Measurement in Education (NCME), A test may be thought of as a set of tasks or questions intended to elicit particular types of behaviour when presented under standardised conditions and to yield scores that will have desirable psychometric properties (1974). While most people know what a test is, many have difficulty differentiating between measurement, evaluation and assessment. Some have even argued that they are similar! (b) Measurement Measurement is the act of assigning numbers to a phenomenon. In education, it is the process by which the attributes of a person are measured and assigned numbers. Remember it is a process, indicating there are certain steps involved! As educators, we frequently measure human attributes such as attitudes, academic achievements, aptitudes, interests, personality and so forth. Hence, in order to measure we have to use certain instruments so that we can conclude that Ahmad is better in mathematics than Kumar while Tajang is more positive towards science than Kong Beng. We measure to obtain information about „what is‰. Such information may or may not be Copyright © Open University Malaysia (OUM) 4  TOPIC 1 ROLES OF ASSESSMENT IN TEACHING AND LEARNING useful depending on the accuracy of the instruments we use and our skill at using them. For example, we measure temperature using a thermometer, and the thermometer is the instrument used. How do you measure performance in mathematics, for example? We use a mathematics test which is an instrument containing questions and problems to be solved by students. The number of right responses obtained is an indication of the performance of individual students in mathematics. Note that we are only collecting information. We are not evaluating! Evaluation is therefore quite different from measurement. (c) Assessment and Evaluation The literature has used the terms „assessment‰ and „evaluation‰ in education as two different concepts. Evaluation is the process of making judgments based on criteria and evidences. Assessment is the process of documenting knowledge, skills, attitudes and beliefs, usually in measurable terms. The goal of assessment is to make improvements as opposed to simply being judged. In an educational context, assessment is the process of describing, collecting, recording, scoring and interpreting information about learning. The two terms are also used interchangeably. For example, some authors used the term „formative evaluation‰ while others use the term „formative assessment‰. We will use the two terms interchangeably because there is too much overlap in the interpretations of the two concepts. Generally, assessment is viewed as the process of collecting information with the purpose of making decisions about students. We may collect information using various tests, observations of students and interviews. Rowntree (1974) views assessment as, A human encounter in which one person interacts with another directly or indirectly with the purpose of obtaining and interpreting information about the knowledge, understanding, abilities and attitudes possessed by that person. For example, based on assessment information we can determine whether Chee Keong needs special services to assist him in developing his reading skills or whether Khairul, who has been identified as a dyslexic child, needs special attention. The key words in the definition of assessment is collecting data and making decisions. Hence, in order to make decisions one has to evaluate, which is the process of making judgement, about a given situation. Copyright © Open University Malaysia (OUM)  TOPIC 1 ROLES OF ASSESSMENT IN TEACHING AND LEARNING  5 When we evaluate, we are saying that something is good, appropriate, valid or positive, for example. To make an evaluation, we need information and the information is obtained by measuring using a reliable instrument. For example, you measure the temperature in the classroom and it is 30 degrees Celsius, which is simply information. Some students may find the temperature too hot for learning while others may say that it is ideal for learning. At some point during the day, we are evaluating something or someone! Educators are constantly evaluating students. This is usually done in comparison with some standards. For example, if the objective of the lesson is for learners to apply BoyleÊs Law to solving a problem and 80 per cent of learners are able to solve the problem, then the teacher may conclude that his teaching of the principle was quite successful. Therefore, evaluation is the comparison of what is measured against some defined criteria and to determine whether it has been achieved, whether it is appropriate, whether it is good, whether it is reasonable, whether it is valid and so forth. 1.2 WHY DO WE ASSESS? Let us begin by asking the question, „Why do we, as educators, assess learners?‰ Some of us may find the question rather strange. The following may be a likely response: Question : Why do you assess? Answer : Well, I assess to find out whether my students understand what has been taught. Question : What do you mean by „understand‰? Answer : Whether they can remember what I taught them and able to solve problems. Question : What do you do with the test results? Answer : Well, I provide students the right answers and point out the mistakes made when answering the questions. The above could be the reasons educators give when asked about the purpose of assessment. In the context of education, assessment is performed to gain an understanding of an individualÊs strengths and weaknesses in order to make appropriate educational decisions. The best educational decisions are based on Copyright © Open University Malaysia (OUM) 6  TOPIC 1 ROLES OF ASSESSMENT IN TEACHING AND LEARNING information and better decisions are usually based on more information (Salvia & Ysseldyke, 1995). Based on the reasons for assessment provided by Harlen (1978) and Deale (1975), two main reasons are identified (refer to Figure 1.2). Figure 1.2: Purposes of assessment With regard to learning, assessment is aimed at providing information that will help make decisions concerning remediation, enrichment, selection, exceptionality, progress and certification. With regard to teaching, assessment provides information regarding achievement of objectives, the effectiveness of teaching methods and learning materials. (a) Assess in Order to Help Learning Let us look at the following aspects of assessment which are able to help learning: (i) Diagnosis Diagnostic evaluation or assessment is performed at the beginning of a lesson or unit for a particular subject area to assess studentsÊ readiness and background for what is about to be taught. This pre- instructional assessment is done when you need information on a particular student, group of students or a whole class before you can proceed with the most effective instructional method. For example, you could administer a Reading Test to Year One students to assess their reading level. Based on the information, you may want to assign weak readers for special intervention or remedial action. On the other hand, the test might reveal that some students are reading at an exceptionally high level and you might want to recommend that they be assigned to an enrichment programme (refer to Table 1.1). Copyright © Open University Malaysia (OUM)  TOPIC 1 ROLES OF ASSESSMENT IN TEACHING AND LEARNING  7 Table 1.1: Purpose of Assessment: To Help Learning To Help Questions to Be Answered Learning Diagnosis for  Should the student be sent for remedial classes so remedial action that difficulty in learning can be overcome? Diagnosis for  Should the student be provided with enrichment enrichment activities? Exceptionality  Does the student have special learning needs that require special education assistance? Selection  Should the student be streamed into X or Y class? Progress  To what extent is the student making progress towards specific instructional goals? Communication  How is the child doing in school and how can to parents parents help? Certification  What are the strengths and weaknesses in the overall performance of a student in specific areas assessed? Administration  How is the school performing in comparison to and counselling other schools?  Why should students be referred to counselling? (ii) Exceptionality Assessment is also conducted to make decisions on exceptionality. Based on information obtained from the assessment, teachers may make decisions as to whether a particular student needs to be assigned to a class with exceptional students. Exceptional students are students who are physically, mentally, emotionally or behaviourally different from the normal population. For example, based on the assessment information, a child who is found to be dyslexic may be assigned for special treatment or a student who has been diagnosed to be learning disabled may be assigned for special education. (iii) Certification Certification is perhaps the most important reason for assessment. For example, the Sijil Pelajaran Malaysia (SPM) is an examination aimed at providing students with a certificate. The scores obtained are converted into letter grades signifying performance level in various subject areas and used as a basis for comparison between students. The certificate obtained is further used for selecting students for further studies, scholarships or jobs. Copyright © Open University Malaysia (OUM) 8  TOPIC 1 ROLES OF ASSESSMENT IN TEACHING AND LEARNING (iv) Placement Besides certification, assessment is also conducted for the purpose of placement. Students are endowed with varying abilities and one of the tasks of the school is to place them according to their aptitude and interests. For example, performance in the Pentaksiran Tingkatan 3 (PT3) (previously Penilaian Menengah Rendah) is used as the basis for placing students in the arts or science stream. Assessment is also used to stream students according to academic performance. It has been the tradition that the „A‰ and „B‰ classes consist of high achievers based on their results in the end of semester or end of year examinations. Placement tests have even been used in preschools to stream children according their literacy levels! The practice of placing students according to academic achievement has been debated for decades with some educationists arguing against it while others supporting its merits. ACTIVITY 1.1 „Streaming students according to academic abilities should be discouraged in Malaysian schools‰. Discuss with your coursemates in the myINSPIRE online forum. (v) Communicate to Parents Parents want to know how their child is doing in school and appreciate information, particularly specific examples, of their childÊs progress. Showing examples of their childÊs work over time enables parents to personally assess the growth and progress of their child. It is essential to tell the whole story when reporting information about performance progress. Talking to parents about standards, sharing student work samples, using rubrics in conferences, and differentiating between performance and progress are some ways to ensure that they are given an accurate picture of their childÊs learning. (vi) School Administration and Counselling Schools collect assessment information in order to determine how the school is performing in relation to other schools for a particular semester or year. Assessment results are also used to compare performance over the years for the same school. Based on the results, school administrators may institute measures to remedy weaknesses such as putting in more resources to help students who perform poorly, for example, to increase the number of studentsÊ reading and writing proficiency from poor to satisfactory level. Copyright © Open University Malaysia (OUM)  TOPIC 1 ROLES OF ASSESSMENT IN TEACHING AND LEARNING  9 Assessment results (especially relating to socioemotional development) may be used by school administrators and counsellors in planning intervention strategies for at-risk students. Assessment by counsellors will enable them to identify students presenting certain socioemotional problems that require counselling services or referral to specialists such psychiatrists, legal counsellors and law enforcement authorities. (b) Assess in Order to Improve Teaching If 70 per cent of your students fail in a test, do you investigate whether your teaching-learning strategy was appropriate or do you attribute it to your students being academically weak in addition to not revising their studies? Most educators would attribute the poor performance to the latter. However, assessment information is valuable because it can indicate which of the learning outcomes have been successfully achieved and which instructional objectives students had most difficulty with. Assessment results are valuable in providing clues to the effectiveness of the teaching strategy implemented and teaching materials used. It also indicates whether students had the required prior knowledge to grasp the concepts and principles discussed (refer to Table 1.2). Assessment data may also provide insights into why some teachers are more successful in teaching a particular group of students while others are less successful. Table 1.2: Purpose of Assessment: To Improve Teaching To Improve Teaching Questions to Be Answered Objectives Were the desired learning outcomes achieved? Teaching methods Were the teaching methods employed effective? Prior knowledge Did students have relevant prior knowledge? Teaching materials Were the teaching materials used effective? Teacher differences Were particular teachers more effective than others? ACTIVITY 1.2 To what extent have you used assessment data to review your teaching- learning strategies? Discuss this with your coursemates in the myINSPIRE online forum. Copyright © Open University Malaysia (OUM) 10  TOPIC 1 ROLES OF ASSESSMENT IN TEACHING AND LEARNING 1.3 GENERAL PRINCIPLES OF ASSESSMENT Let us discuss the general principles of assessment: (a) What is to be Assessed has to be Clearly Specified The specification of the characteristics to be measured should precede the selection or development of assessment procedures. In assessing student learning, the intended learning goals or outcomes should be clearly specified. In other words, appropriate assessment procedures can only be selected if there is clear specification of the intended learning outcomes to be measured. (b) An Assessment Procedure should be Selected Based on Its Relevance to the Characteristics or Performance to be Measured When selecting an assessment procedure to measure a specific learning outcome, teachers should always ask if the procedure selected is the most effective method for measuring the learning or development to be assessed. There must be a close match between the intended learning outcomes and the types of assessment tasks to be used. For example, if the teacher would like to assess the studentsÊ ability to organise ideas, the use of multiple- choice test would be a poor choice. (c) Different Assessment Procedures are Required to Provide a Complete Picture of Student Achievement and Development No single assessment procedure can assess all the different learning outcomes in a school curriculum. Different assessment procedures (formative, summative and authentic assessments) can achieve different objectives. For example, multiple-choice questions are useful for measuring knowledge, understanding and application outcomes while essay tests are appropriate for measuring the ability to organise and express ideas. Projects that require conducting library research are needed to measure certain skills in formulating and solving problems. Observational techniques are needed to assess performance skills and various aspects of student behaviour. Using a wide variety of assessment tools allows a teacher to determine which instructional strategies are effective and which needed to be modified. In this way, assessment can be used to improve classroom teaching, plan curriculum and research oneÊs own teaching practice. Copyright © Open University Malaysia (OUM)  TOPIC 1 ROLES OF ASSESSMENT IN TEACHING AND LEARNING  11 (d) Assessment must be Aligned to Instruction In order to encourage higher-order thinking and problem-solving skills, there is an increased need to align the curriculum, instruction and assessment. Classroom assessment techniques should be focusing on aligning assessments more closely with the instructional strategies used in the classroom. In this way, what is to be assessed in the classroom is consistent with what has been taught and vice versa. For example, it would not be fair to assess students on higher-order thinking skills when what is taught is only lower level thinking skills. 1.4 TYPES OF ASSESSMENT Before we proceed to discuss more about assessment, you need to be clear about these often used concepts in assessment: (a) Formative assessment (or evaluation) and summative assessment (or evaluation); (b) Criterion-referenced assessment and norm-referenced assessment; and (c) Authentic assessment (examples of authentic assessment will be discussed in Topics 6 and 7). Let us move on and read further. 1.4.1 Formative vs Summative Assessments Teachers engage students in all aspects of their learning. There are many strategies to accomplish this ranging from informal questioning, quiz and observation to more formal monthly test and end of semester examination. In a balanced assessment system, both summative and formative assessments are an integral part of information gathering. Assessment can be done at various times throughout the school year. A comprehensive assessment plan will include both formative and summative assessments. The point at which assessment occurs and the aim of assessing distinguishes these two categories of assessment. Copyright © Open University Malaysia (OUM) 12  TOPIC 1 ROLES OF ASSESSMENT IN TEACHING AND LEARNING (a) Formative Assessment Formative assessment is often done at the beginning or during the school year, thus providing the opportunity for immediate evidence of student learning in a particular subject area or at a particular point in the programme. Classroom assessment is one of the most common formative assessment techniques used. The purpose of this technique is to improve the quality of student learning and should not be evaluative in nature or involve grading students. In formative assessment, the teacher compares the performance of a student to the performance of other students in the class and not all students in the same year. Usually, a small section of the content is tested to determine if the objectives have been met. Formative assessment is action-oriented and forms the basis for improvement of instructional methods (Scriven, 1996). For example, if a teacher observes that some students still have not grasp a concept, he may design a review activity or use a different instructional strategy. Likewise, students can monitor their progress with periodic quizzes and performance tasks. The results of formative assessments are used to modify and validate instruction. In short, formative assessments are ongoing and include reviews and observations of what is happening in the classroom. Formative assessments are generally low stakes, which means that they have minimal or no point value. (b) Summative Assessment „When the cook tastes the soup, thatÊs formative evaluation; when the guests taste the soup, thatÊs summative evaluation‰ (Robert Stakes) Summative assessment is comprehensive in nature. It provides accountability and is used to check the level of learning at the end of the programme (which may be at the end of the semester, year or after two years). For example, after five years in secondary school, students sit for the Sijil Pelajaran Menengah (SPM) examination which is summative in nature since it is based on the cumulative learning experiences of students. Copyright © Open University Malaysia (OUM)  TOPIC 1 ROLES OF ASSESSMENT IN TEACHING AND LEARNING  13 Summative assessment is also used to evaluate the effectiveness of an instructional programme at the end of an academic year or at a predetermined time. The goal of summative assessment is to make a judgment of student competency after an instructional phase is completed. For example, national examinations are administered each year in Malaysia. It is a summative assessment to determine each studentÊs acquisition of several subject areas of between two to three years coverage of content. Summative evaluations are used to determine if students have mastered the specific competencies as set out by the programme and letter grades are assigned to assess learner achievement. Summative assessments are often high stakes, which means that they have a high point value. Table 1.3 highlights the differences between formative and summative assessments. Table 1.3: Differences between Formative and Summative Assessments Criteria Formative Assessment Summative Assessment Timing  Conducted throughout the  Conducted at the end of a teaching-learning process. teaching-learning phase (for example, end of semester or year). Methods  Paper and pencil tests,  Paper and pencil tests, oral tests observations, quizzes, exercises, administered to the group. practical sessions administered to the group and individually. Aims  To assess progress and  Grading to determine if the recommend remedial action for programme was successful. non-achievement of objectives.  To certify students and improve  Remediation or enrichment or the curriculum. reteach the topic. Examples  Quizzes, essays, diagnostic tests,  Final examination, national lab reports and anecdotal examination and qualifying records. tests. Copyright © Open University Malaysia (OUM) 14  TOPIC 1 ROLES OF ASSESSMENT IN TEACHING AND LEARNING Can information from summative assessments be used formatively? Tests given at the end of a unit of work provide information about how much each student has learnt or has not learnt. If left as a summative process, then the studentsÊ test marks merely become a record of their successes or failures at a point in time and not directly helping them to improve. Moreover, in actual fact some students will know more than what is indicated in their test performance. However, when the information is used formatively, the tests results can provide an important source of detailed, individualised feedback identifying where each student needs to deepen their understanding and improve their recall of the knowledge they have learnt. The more teachers know about individual students as they engage in the learning process, the better teachers can adjust instruction to ensure that all students continue to move forward in their learning. Advantages of using information derived from summative assessments to improve future student performance are: (a) Summative data reveals how the students performed at the end of a learning programme, namely advanced, proficient, basic or below basic. For example, if a student has scored below basic in the semester exam and exhibits signs of a struggling student, the teacher may want to place the student at the front of the class so that the teacher can easily access the student when the student needs extra support; (b) Summative assessments can serve as a guide to improving teaching methods. Teachers often employ varied teaching methodologies within the classroom. Summative assessments can help teachers collaborate and improve teaching methods from year to year; and (c) Summative assessments help teachers and administrators in improving the curriculum and curriculum planning. Standards-driven instruction plays a large role in schools today. When summative assessments show consistent gaps between student knowledge and learning targets, schools may turn to the relevant authority in the Ministry of Education to suggest improving curriculum planning or new curriculum to fill those learning gaps. The data that is collected using a summative assessment can help teachers and schools make decisions based on the instruction that has already been completed. This contrasts with formative assessment, whereby formative assessment can help teachers and students during the instruction process. It is important to understand the difference between the two, as both assessments can play an important role in education. Copyright © Open University Malaysia (OUM)  TOPIC 1 ROLES OF ASSESSMENT IN TEACHING AND LEARNING  15 1.4.2 Norm-referenced vs Criterion-referenced Tests The main difference between norm-referenced tests and criterion-referenced tests depends on the purpose or aim of assessing your students, the way in which content is selected and the scoring process which defines how the test results must be interpreted. (a) Norm-referenced Tests The major reason for norm-referenced tests is to classify students. These tests are designed to highlight achievement differences between and among students to produce dependable rank order of students across a continuum of achievement from high-achievers to low achievers (Stiggins, 1994). With norm-referenced tests, a representative group of students is given the test and their scores form the norm after having gone through a complex administration and analysis. Anyone taking the norm-referenced test can compare his or her score against the norm. For example, a student who obtained a score of 70 on a norm-referenced test will not mean much until it is compared to the norm. When compared to the norm, her score is in the 80th percentile which means that she performed as well or better than 20 per cent of students in the norm group. This type of information can be useful for deciding whether or not students need remedial assistance or is a candidate for the gifted programme. However, the score gives little information about what the student actually knows or can do. A major criticism of norm-referenced tests is that they tend to focus on assessing low level, basic skills (Romberg, 1989). (b) Criterion-referenced Tests Criterion-referenced tests determine what students can or cannot do and not how they compare to others (Anastasi, 1988). Criterion-referenced tests report how well students are doing relative to a predetermined performance level on a specified set of educational goals or outcomes included in the curriculum. Criterion-referenced tests are used when teachers wish to know how well students have learnt the knowledge and skills which they are expected to have mastered. This information may be used to determine how well the student is learning the desired curriculum and how well the school is teaching that curriculum. Criterion-referenced tests give detailed information about how well a student has performed on each of the educational goals or outcomes included on that test. For instance, a criterion-referenced test score might describe which arithmetic operations a student can perform or the level of reading difficulty experienced. Copyright © Open University Malaysia (OUM) 16  TOPIC 1 ROLES OF ASSESSMENT IN TEACHING AND LEARNING Table 1.4 illustrates the differences between norm-referenced and criterion- referenced tests. Table 1.4: Differences between Norm-referenced and Criterion-referenced Tests Criteria Norm-referenced Tests Criterion-referenced Tests Aims  Compare a studentÊs  Compare a studentÊs performance with other performance against some students. criteria.  Select students for  Extent to which a student certification. has acquired the knowledge or skill.  Improve teaching and learning. Types of  Questions from simple to  Questions of nearly similar questions difficult. difficulty relating to the criteria. Reporting of  Grades are assigned.  No grades are assigned results (whether skill or knowledge is achieved). Content  Wide content coverage.  Specific aspects of the coverage content. Examples  UPSR, PT3, SPM national  Class tests, exercises and examinations, end of assignments. semester examinations and end of year examinations. SELF-CHECK 1.1 1. Explain the differences between norm-referenced and criterion- referenced tests. 2. Describe the main differences between formative and summative assessments. Copyright © Open University Malaysia (OUM)  TOPIC 1 ROLES OF ASSESSMENT IN TEACHING AND LEARNING  17 1.5 CURRENT TRENDS IN ASSESSMENT In the last two decades, there have been major changes in assessment practices in many parts of the world. Brown, Bull and Pendlebury (1997) identified the following trends in educational assessment: (a) Written examinations are gradually being replaced by more continuous assessments and coursework; (b) There is a move towards more student involvement and choice in assessments; (c) Group assessment is becoming more popular in an effort to emphasise collaborative learning between students and to reduce excessive competition; (d) Subject areas and courses state more explicitly about the expectations in assessment, more specifically the kinds of performance required from students when they are assessed. This is unlike earlier practices where assessment is so secretive and students had to figure out for themselves what was required of them; (e) An understanding of the process is now seen as, at least, equally important to the knowledge of facts. This is in line with the general shift from product-based assessment towards process-based assessment; and (f) Student-focussed „learning outcomes‰ have begun to replace teacher- oriented „objectives‰. The focus is more on what the student will learn rather than what the teacher plans to teach. Copyright © Open University Malaysia (OUM) 18  TOPIC 1 ROLES OF ASSESSMENT IN TEACHING AND LEARNING Easing up on Exams Putrajaya: Reducing the number of Among the measures proposed are: examination subjects and having a semester system are among the Ć Reducing the number of subjects major changes being planned to in public examinations; make the education system more Ć Emphasising skills and abilities holistic and less focussed on rather than focusing on content academic achievement. and achievement; Education Minister, Datuk Seri Ć Encouraging personal Hishamumuddin Tun Hussein said development through subjects that these measures were in line like Art and Physical Education; with the GovernmentÊs aim to and reform the countryÊs education system. „We do not intend to abolish Ć Improving teaching-learning public or school-level examinations methods by encouraging more totally, but we recognise that the project-based assignments. present assessment system needs to be looked at‰, he said. He said that emphasis should be on individual accomplishments rather than the schoolÊs performance in public examinations and also highlighting the individualÊs co-curricular achievements. (The Star, 21 March, 2006) ACTIVITY 1.3 Refer to the report on „Easing up on Exams‰ and discuss to what extent you agree with the measures proposed by the Ministry of Education to reduce the exam-oriented education system in Malaysia. Share your answer with your coursemates in the myINSPIRE online forum. Copyright © Open University Malaysia (OUM)  TOPIC 1 ROLES OF ASSESSMENT IN TEACHING AND LEARNING  19  A test may be thought of as a set of tasks or questions intended to elicit particular types of behaviour when presented under standardised conditions and to yield scores that will have desirable psychometric properties.  Measurement in education is the process by which the attributes of a person are measured and assigned numbers.  Assessment is viewed as the process of collecting information with the purpose of making decisions regarding students.  Assessment is aimed at helping the learner and to improve teaching.  Summative assessment is comprehensive in nature, provides accountability and is used to check the level of learning at the end of the programme.  Formative assessment is often conducted at the beginning of or during the school year, thus providing the opportunity for immediate evidence of student learning in a particular subject area or at a particular point of time in a programme.  The information from summative assessment can be used formatively. The students or schools can use the information to guide their efforts and activities in subsequent courses.  The major reason for norm-referenced tests is to classify students. These tests are designed to highlight achievement differences between and among students to produce dependable rank order of students.  Criterion-referenced tests determine what students can or cannot do and not how they compare to others. Assessment Measurement Criterion-referenced test Norm-referenced test Evaluation Summative assessment Formative assessment Test Copyright © Open University Malaysia (OUM) Topic  Foundation for Assessment: 2 What to Assess LEARNING OUTCOMES By the end of this topic, you should be able to: 1. Justify the behaviours that are to be measured to present a holistic assessment of students; 2. Describe the various types of cognitive learning outcomes to be assessed; 3. Describe the various types of affective learning outcomes to be assessed; and 4. Describe the various types of psychomotor learning outcomes to be assessed.  INTRODUCTION If you were to ask a teacher, what should be assessed in the classroom, the immediate response would be, of course, the facts and concepts taught. They are the facts and concepts found in science, history, geography, language, arts, religious education and other similar subjects. However, the Malaysian Philosophy of Education states that education should aim towards the holistic development of the individual. Hence, it is only logical that the assessment system should also seek to assess more than the acquisition of the facts and concepts of a subject area. What about assessment of physical and motor abilities? What about socioemotional behaviours such as attitudes, interests, personality and so forth? Do they not contribute to the holistic person? In this topic, you will learn the types of learning outcomes that need to be assessed in a curriculum. The topic will conclude with a brief explanation on how to plan a table of specification for a classroom test. Copyright © Open University Malaysia (OUM)  TOPIC 2 FOUNDATION FOR ASSESSMENT: WHAT TO ASSESS  21 2.1 IDENTIFYING WHAT TO ASSESS When educators are asked what should be assessed in the classroom, the majority would refer to evaluating the acquisition of the facts, concepts, principles, procedures and methods of a subject area. You might find a minority of educators who insist that skills acquired by learners should also be assessed especially in subjects such as physical education, art, drama, music, technical drawing, carpentry, automobile engineering and so forth. Even fewer educators would propose that the socioemotional behaviour of learners should also be assessed. National Philosophy of Malaysian Education Education in Malaysia is an ongoing effort towards further developing the potentials of individuals in a holistic and integrated manner so as to produce individuals who are intellectually, spiritually, emotionally and physically balanced and harmonic, based on a firm belief in and devotion to God. Such an effort is designed to produce Malaysian citizens who are knowledgeable and competent, who possess high moral standards and who are responsible and capable of achieving a high level of personal well-being as well as being able to contribute to the harmony and betterment of the family, society and the nation at large. (Curriculum Development Centre, Ministry of Education Malaysia, 1988) The National Philosophy of Malaysian Education has important implications for assessment. Theoretically, a comprehensive assessment system should seek to provide information on the extent to which the National Philosophy of Education has achieved its goal. In other words, the assessment system should seek to determine: (a) Whether our schools have developed „the potentials of individuals in a holistic and integrated manner‰; (b) Whether our students are „intellectually, spiritually, emotionally and physically balanced‰; (c) Whether our students are „knowledgeable and competent‰ and „possess high moral standards‰; (d) Whether our students have a „high level of personal well-being‰; and (e) Whether students are equipped with the abilities and attitudes that will enable them „to contribute to the harmony and betterment of the family, society and the nation at large‰. Copyright © Open University Malaysia (OUM) 22  TOPIC 2 FOUNDATION FOR ASSESSMENT: WHAT TO ASSESS On the contrary, in actual practice the assessment tends to overemphasise on intellectual competence which translates into the measurement of cognitive learning outcomes of specific subject areas. The other aspects of the holistic individual are given minimal attention because of various reasons. For example, how does a teacher assess the spiritual or emotional growth and development? These are constructs that are difficult to evaluate and extremely subjective. Hence, it is no surprise that assessment of cognitive outcomes has remained the focus of most assessment systems all over the world because it is relatively easier to observe and measure. However, in this topic we will make an attempt to present a more „holistic‰ assessment of learning, focusing on three main types of human behaviour. These are behaviours that psychometricians and psychologists have attempted to assess and are closely aligned to realising the goals of the National Philosophy of Malaysian Education. 2.2 THREE TYPES OF LEARNING OUTCOMES Few people will dispute that the purpose of schooling is the development of the holistic person. In the late 1950s and early 1960s, a group of psychologists and psychometricians proposed that schools should seek to assess three domains of learning outcomes (refer to Figure 2.1): (a) Cognitive learning outcomes (knowledge or mental skills); (b) Affective learning outcomes (feelings or emotions); and (c) Psychomotor learning outcomes (manual or physical skills). Figure 2.1: Holistic assessment of learners Copyright © Open University Malaysia (OUM)  TOPIC 2 FOUNDATION FOR ASSESSMENT: WHAT TO ASSESS  23 Domains can be thought of as categories. Educators often refer to these three domains as KSA (Knowledge, Skills and Attitude). Each domain consists of subdivisions, starting from the simplest behaviour to the most complex, thus forming taxonomy of learning outcomes. Each of the taxonomy of learning behaviour can be thought of as „the goals of the schooling process.‰ That means that after schooling, the learner should have acquired new skills, knowledge, and/or attitudes. However, the levels of each division outlined are not absolutes. While there are other systems or hierarchies that have been devised in the educational world, these three taxonomies are easily understood and are probably the most widely used today. To assess the three domains, one has to identify and isolate the behaviour that represents these domains. When we assess we evaluate some aspects of the learnerÊs behaviour, for example, his ability to compare, explain, analyse, solve, draw, pronounce, feel, reflect and so forth. The term „behaviour‰ is used broadly to include the learnerÊs ability to think (cognitive), feel (affective) and perform a skill (psychomotor). For example, you have just taught about „The Rainforest of Malaysia‰ and you would like to assess your students in their: (a) Thinking ă You might ask them to list the characteristics of the Malaysian rainforest and compare it with the coniferous forest of Canada; (b) Feelings (emotions, attitudes) ă You could ask them to design an exhibition on how students could contribute towards conserving the rainforest; and (c) Skill ă You could ask them to prepare satellite maps about the changing Malaysian rainforest by accessing websites from the Internet. ACTIVITY 2.1 Refer to Figure 2.1. To what extent are affective and psychomotor behaviours assessed in your institution? Discuss with your coursemates in the myINSPIRE online forum. Copyright © Open University Malaysia (OUM) 24  TOPIC 2 FOUNDATION FOR ASSESSMENT: WHAT TO ASSESS 2.3 ASSESSING COGNITIVE LEARNING OUTCOMES When we evaluate or assess a human being, we are assessing or evaluating the behaviour of a person. This might be a bit confusing to some people. Are we not assessing a personÊs understanding of the facts, concepts and principles of a subject area? Every subject, whether it is history, science, geography, economics, or mathematics, has its unique repertoire of facts, concepts, principles, generalisations, theories, laws, procedures and methods that are transmitted to learners (illustrated in Figure 2.2). Figure 2.2: Contents of a subject assessed When we assess we do not assess the learnerÊs store of the facts, concepts or principles of a subject but rather what the learner is able to do with the facts, concepts or principles of a subject area. For example, we evaluate the learnerÊs ability to compare facts, explain the concept, analyse a generalisation (or statement) or solve a problem based on a given principle. In other words, we assess the understanding or mastery of a body of knowledge based upon what the learner is able to do with the contents of the subject. Let us look at two mechanisms used to measure or assess cognitive learning, namely BloomÊs Taxonomy and The Helpful Hundred. Copyright © Open University Malaysia (OUM)  TOPIC 2 FOUNDATION FOR ASSESSMENT: WHAT TO ASSESS  25 (a) BloomÊs Taxonomy In 1956, Benjamin Bloom led a group of educational psychologists to develop a classification of levels of intellectual behaviour which are important to learning. They found that over 95 per cent of the test questions which learners encountered required them to think only at the lowest possible level, that is the recall of information. Bloom and his colleagues developed a widely accepted taxonomy (method of classification on differing levels) for cognitive objectives. This is referred to as BloomÊs Taxonomy (refer to Figure 2.3). There are six levels in BloomÊs classification with the lowest level termed knowledge. The knowledge level is followed by five increasingly difficult levels of mental abilities: comprehension, application, analysis, synthesis and evaluation. Figure 2.3: BloomÊs taxonomy of cognitive learning outcomes Now, read further to find out what each level constitutes. (i) Knowledge: The behaviours at the knowledge level require learners to recall specific information. The knowledge level is the lowest cognitive level. Examples of verbs describing behaviours at the knowledge level include the ability to list, define, name, state, recall, match, identify, tell, label, underline, locate, recognise, select and so forth. For example, learnersÊ ability to recite the factors leading to the World War II, quote formula for density and force, tell laboratory safety rules. (ii) Comprehension: The behaviours at the comprehension level which is a higher level of mental ability than the knowledge level require the understanding of the meaning of concepts and principles, translation of words and phrases into oneÊs own words, interpolation which involves filling in missing information, interpretation which involves inferring and going beyond the given information. Examples of verbs describing behaviours at the comprehension level are explain, distinguish, infer, interpret, convert, generalise, defend, estimate, extend, paraphrase, retell using own words, predict, rewrite, summarise, translate and so forth. For example, learners are able to Copyright © Open University Malaysia (OUM) 26  TOPIC 2 FOUNDATION FOR ASSESSMENT: WHAT TO ASSESS rewrite NewtonÊs three laws of motion, explain in oneÊs own words the steps for performing a complex task and translate an equation into a computer spreadsheet. (iii) Application: The behaviours at the application level require learners to apply a rule or principle learned in the classroom into novel or new situations in the workplace or unprompted use of an abstraction. Examples of verbs describing behaviours at the application level are apply, change, compute, demonstrate, discover, manipulate, modify, give an example, operate, predict, prepare, produce, relate, show, solve, use and so forth. For example, learners are able to use the formula for projectile motion to calculate the maximum distance a long jumper jumps and apply laws of statistics to evaluate the reliability of a written test. (iv) Analysis: The behaviours at the analysis level require learners to identify component parts and describe their relationship, separate material or concepts into component parts so that its organisational structure may be understood and distinguish between facts and inferences. Examples of verbs describing behaviours at the analysis level are analyse, break down, compare, contrast, diagram, deconstruct, examine, dissect, differentiate, discriminate, distinguish, identify, illustrate, infer, outline, relate, select, separate and so forth. For example, learners are able to troubleshoot a piece of equipment by using logical deduction, recognise logical fallacies in reasoning, analyse information from a company and determine needs for training. (v) Synthesis: The behaviours at the synthesis level require learners to build a structure or pattern from diverse elements and put parts together to form a whole with emphasis on creating a new meaning or structure. Examples of verbs describing behaviours at the synthesis level are categorise, combine, compile, compose, create, devise, design, explain, generate, modify, organise, plan, rearrange, reconstruct, relate, reorganise, find an unusual way, formulate, revise, rewrite, summarise, tell, write and so forth. For example, learners are able to write a creative short story, design a method to perform a specific task, integrate ideas from several sources to solve a problem, devise a new plan of action to improve the outcome. Copyright © Open University Malaysia (OUM)  TOPIC 2 FOUNDATION FOR ASSESSMENT: WHAT TO ASSESS  27 (vi) Evaluation: The behaviours at the evaluation level require learners to make judgment about materials and methods, and the value of ideas or materials. Examples of verbs describing behaviours at the evaluation level are appraise, compare, conclude, contrast, criticise, critique, defend, describe, rank, give your own opinion, discriminate, evaluate, explain, interpret, value, justify, relate, summarise, support and so forth. For example, learners are able to evaluate and decide on the most effective solution to a problem, justify the choice of a new procedure or course of action. (b) The Helpful Hundred Heinich, Molenda, Russell and Smaldino (2001) suggested 100 verbs that highlight performance or behaviours that are observable and measurable. This is not to say that these 100 verbs are the only ones but they definitely are a great reference for educators. Table 2.1 displays the verbs that would be appropriate to use when you are writing instructional objectives in each level of BloomÊs Taxonomy. Table 2.1: The Helpful Hundred add compute drill label predict state alphabetise conduct estimate locate prepare subtract analyse construct evaluate make present suggest apply contrast explain manipulate produce swing arrange convert extrapolate match pronounce tabulate assemble correct fit measure read throw attend cut generate modify reconstruct time bisect deduce graph multiply reduce translate build defend grasp name remove type cave define grind operate revise underline categorise demonstrate hit order select verbalise choose derive hold organise sketch verify classify describe identify outline ski weave colour design illustrate pack solve weigh compare designate indicate paint sort write complete diagram install plot specify compose distinguish kick position square Copyright © Open University Malaysia (OUM) 28  TOPIC 2 FOUNDATION FOR ASSESSMENT: WHAT TO ASSESS In 2001, Krathwohl and Anderson modified the original BloomÊs Taxonomy (1956). They identified and isolated the following list of behaviours that an assessment system should address (refer to Table 2.2). Table 2.2: Revised Version of BloomÊs Taxonomy Category and Cognitive Process Alternative Names 1. Remembering Ć Recognising Ć Identifying Ć Recalling Ć Retrieving 2. Understanding Ć Interpreting Ć Clarifying, paraphrasing, representing, Ć Exemplifying translating Ć Classifying Ć Illustrating, instantiating Ć Summarising Ć Categorising, subsuming Ć Inferring Ć Abstracting, generalising Ć Comparing Ć Concluding, extrapolating, interpolating, predicting Ć Explaining Ć Contrasting, mapping, matching Ć Constructing models 3. Applying Ć Executing Ć Carrying out Ć Implementing Ć Using 4. Analysing Ć Differentiating Ć Discriminating, distinguishing, focusing, Ć Organising selecting Ć Attributing Ć Finding coherence, integrating, outlining,