Qatar University Assessment QHPE 601 PDF

Assessment QHPE 601 Evidence-based Teaching and Learning Faculty Credits The following faculty are attributed to the slides and ideas of this session (in alphabetical order) Dr Ahsan Sethi Dr Daniel Rainkie 2 Learning Objectives By the end of this...

Assessment QHPE 601 Evidence-based Teaching and Learning Faculty Credits The following faculty are attributed to the slides and ideas of this session (in alphabetical order) Dr Ahsan Sethi Dr Daniel Rainkie 2 Learning Objectives By the end of this session the learner will be able to: Describe types of assessment (for learning, of learning, formative, summative) Match assessment modalities to the levels of Miller’s pyramid Describe and calculate the assessment utility index at your institution 3 Domains of Learning Knowledge Psychomotor Attitude Skill 4 Bloom’s Taxonomy Of Cognitive Domain 5 Dave’s Taxonomy Of Psychomotor Domain P4 = Perform independently Perform LP in children with meningitis independently. P3 = Perform under supervision Perform the different steps of LP under supervision. P2 = Assist in the procedure Assist the different steps of LP in children with meningitis. P1 = Observe Observe the different steps of LP in children with meningitis. 6 Bloom’s Taxonomy Of Affective Domain Adapt behavior to the value system Organize values, determine interrelationships Accept the worth of a thing e.g. participate, continuing desire, grow, feel Respond first with compliance and then willingly e.g. engage, consider, accept Notice a particular phenomenon e.g. observe, be conscious, listen, attend 7 “As long as there are tests, there will be prayer in school” 8 What is Assessment? 9 10  Assessment is a measure of student learning. It is a systematic gathering, analyzing and interpreting evidences to determine how well student learning matches expectations. Ref. Assessing Student Learning: A common Sense Guide, Linda Suskie, Anker, 2004. 11 Why Do We Assess? 12 Why Do We Assess? (Cont. 1) We assess to: Certify attainment of course outcomes Demonstrate readiness for progression Pass or fail Grade or rank or award prizes Provide feedback Identify weak students Motivate the students and drive learning Inform career selection Award qualifications 13 Why Do We Assess? (Cont. 2) 14 What are the different Assessment tools? 15 What are the different Assessment tools? (Cont. 1) MCQ Logbook Viva EMQ. Essay OSC E Assessment Tools Portfolio 16 How to select the appropriate assessment instrument? 17 Miller’s Pyramid 18 Miller’s Pyramid (Cont. 1) 19 “The curriculum instructs teachers what to teach; The exam instructs students what to learn.” Donald Melnick, 1991 20 What are the types of Assessments? 21 What are the types of Assessments? (Cont. 1) Diagnostically? Course 22 What are the types of Assessments? (Cont. 2) Summatively? Course 23 What are the types of Assessments? (Cont. 3) Formatively? Course 24 Summative Assessments It determines how well learner has performed at fixed point on course usually the end of period of time More product-oriented and assesses the final product Purpose To determine whether the student has learned what has been taught. Cumulative, indicate whether the student is competent to move on in training. May result in accreditation/licensing/revalidation E.g. Examinations occurring at end of units, courses, semesters, or years. Or for graduation, licensure, certification, etc. 25 Formative Assessments Assessment of students during a course/programme Purpose To improve learning and student performance Assess content retention Providing feedback to students and teachers The act of assessment itself creates learning Learners need to feel ‘safe’ to expose deficiencies E.g. Classroom assessments, Technology-supported games (quick quiz), Workplace-based measures 26 Formative & Summative Assessments ‘When the chef tastes the sauce it is formative assessment; when the customer tastes, it is summative’ (UCD Teaching and Learning) 27 Assessment and Learning 28 System of Assessment 29 Low & High Stakes Assessment Low-stakes assessment has limited consequences for the learner in terms of promotion, selection or certification. High-stake assessment can have far-reaching and dramatic consequences. Low-stake decisions can be based on single data points. High-stake decisions require input from many data points. 30 Assessment & Stakeholders – Criteria 2011 31 What are the Issues with your Assessment? 32 Issues With Our Assessment Content of question not mapped to curriculum Teaching not mapped to curriculum Level of question Candidates not familiar with type of assessment Not measuring the right thing (validity) Not reliable – subjective? Assessors not adequately trained Not transparent Bad or non-existent standard setting Failure to fail No quality assurance 33 Assessment Principles 34 35 Principles of Assessment Assessing ‘knows’, ‘knows how’, ‘shows how’ 1. Competence is specific, not generic (not generalisable) 2. Objectivity is not the same as reliability (OSCE-global score) 3. What is being measured is more determined by the stimulus format than the response format (Authenticity-complexity) 4. Validity can be ‘built-in’ (peer review of exams) 5. Assessment drives learning (reinforce desirable behaviors) 6. No single method can do it all (cocktail – programmatic) 36 37 Principles of Assessment: Assessing ‘Does’ 1. A feasible sample is required to achieve reliable inferences. 2. Bias is an inherent characteristic of expert judgement. 3. Validity lies in the users of the instruments, more than in the instruments 4. Formative and summative functions are typically combined 5. Qualitative, narrative information carries a lot of weight. 6. Summative decisions can be rigorous by using non-psychometric qualitative procedures (Portfolio) 38 Utility Index 39 Usefulness of Assessment (Utility Index) The ‘utility’ or usefulness of assessment instrument has been defined as: A product of its reliability, validity, educational impact, acceptability and cost effectiveness. Later on, feasibility was also added to the index. Utility = educational impact x reliability x validity x cost effectiveness x acceptability x feasibility 40