MEDU222 Assessment Lecture 7 PDF

Assessment Tools  Roadmap 1. Overview of principles of assessment in medical education. 2. Assessment Methods/Tools 3. Assessment in Your College  Contents Overview Objectives What makes for good assessment? Assessment methods/Tools Conclusion...

Assessment Tools  Roadmap 1. Overview of principles of assessment in medical education. 2. Assessment Methods/Tools 3. Assessment in Your College  Contents Overview Objectives What makes for good assessment? Assessment methods/Tools Conclusions  Educational Triangle Needs / Objectives/Outcomes & Contents Curriculum Assessment Strategies  Overview The curriculum instructs teacher what to teach; The exam instruct students what to learn  Overview Assessment derives learning Students don’t learn what teachers expect but “Learn what teachers inspect” Assessment Formative v/s Summative  Formative vs Summative Formative Summative It is continues assessment At the end of a course/ program Feedback is an essential part Usually NO feedback session Usually NO grades Marks OR grades are essential There is no pass-fail decision There should be a pass-fail decision  Assessment Does Skills & Behaviour Shows how Knows how Cognition/ Knows Knowledge Miller GE. The assessment of clinical skills/competence/performance. Academic Medicine (Supplement) 1990; 65: S63-S7.  Assessment Action Does Shows how Performance Knows how Competence Knows Knowledge Miller GE. The assessment of clinical skills/competence/performance. Academic Medicine (Supplement) 1990; 65: S63-S7.  One method can’t do it all – One measure is no measure – We need a mixture of methods to cover the entire pyramid – We can choose different assessment tools Criteria for different assessment methods 1. Validity: 2. Reliability 3. Educational impact 4. Cost effectiveness 5. Acceptability/ Practicability  Criteria / Requirements 1. Validity: Does it measure what it is supposed to measure? 2. Reliability: Does it produce consistent results? 3. Practicability: Is it practical in term of times & resources.  Educational Impact Assessment may have facilitative or adversive effect on learning Emphasizes the important learning outcomes Provide feedback about the success of study methods  ?What Should Be Tested Important topics are weighted heavily The sample of items reflect the instructional goals and the process.  Domain of Interest Test Sample  Test Sample  Assessment Written Exam  Written Written tests can assess knowledge acquisition and reasoning ability, They cannot so easily measure skills  Progress test all students from (all classes) are… ……submitted to the same test End of course Multiple Choice assessment Open ended questions Combination MCQ Modified Essay Questions Computer assessment Assignments Construct a patient information folder Design a Poster Integration essay Prepare a forum discussion Write a research proposal  Clinical 1. Short cases 2. Long cases 3. OSCE 4. Simulated Clinics 5. Video consultation  Clinical(cont’) 6. Slides Interpretation 7. Lab Data Interpretation 8. Portfolios 9. SOE (simulated oral exam) 10.Oral  Computerized Case-based Testing Computer is used to increase effectiveness in: – cost – test administration – scoring – authenticity (e.g. by multimedia) – Written, practical, clinical,…….  "Long case" and “Short case“ The basic requirements for reliability and validity have not always been achieved in traditional long & short cases Observing candidates' interaction with patients (MINI-CEX) OSCE Observing Candidates' Interaction with Patients Oral Oral  Work Place-Based Assessment Direct The oral examination observation: is traditionally Clinical work sampling measures: Mini-CEX (CLINICAL EXAMINATION EXERCISE ) an(unstructured DOPS faceOF DIRECT OBSERVATION toPROCEDURAL face sessionSKILLS) with( PROFESIONALISM-MINI P-MEX the examiners EVALUATION EXERCISE) Global performance measures Multi-Source Feedback (MSF or 360) Aggregation and reflection measures Logbook Portfolio  PEER EVALUATION The oral examination is traditionally an unstructured face to face session Peer Assessment & Self Assessment with the examiners Assessment of professional behavior  Regular feedback of performance  For all the students  Helping students to grow as a professional  Starting from the beginning of the studies  Assessment of professional behavior Professional behaviour consists of three aspects Dealing with work Dealing with others Dealing with oneself  Portfolio the S et of resul t e ts ts of d ude m o pe rec )long asiffses nt a t s ea tudin e smren i a al or re eflt iter nt nc by repor)w co tanilto ecr re ete d tsr, ivi m aredC(ts ffe p ibe proje ttendeopet spe dion com scr ulty peer cts st gatherse n cts th e c Student asses at ce of e d fa staff evidence smenabout e m articl a ssess t en aspects es, pr of his mencompetence ts( esent t ing p apers  Portfolio assessment Mentor = coach St Role ud resul as edx ent ts o f iffper ts )w asosfes eret refle pec lty s reporr co mennt t ct as cu ti s al y fa proje stten mp asp on e r peer cts s et ec th ev b asses ta en ts se ed staff Studentsmen proves tem his ce On rib asses t e sc scompetence ment nt de s( Student discusses his competence and reflection on it with the mentor  What are the assessment tools used in your college?? 1. Contineous Assessment Exam-1 60% Exam-2 Assignment OSPE 2. End of the Course a. MCQ- 40% Assessment 40% TOTAL 100 100% Conclusion  Conclusion The oral examination Comprehensive is traditionally Assessment an unstructured face to face session Different methods to be applied with the examiners Standardized / structured Not to depend only on few methods  Does Shows how MCQ, SAQ, Short essay, etc…. Knows how Knows how Knows Knows Simple recall questions  Does Simulations requiring behavioural Shows how Shows how performance OSCE, OSPE, Simulated Knows how Knows how Clinics, Knows  Assessing performance Work- Does based assessment: Clinical work-sampling, Mini-CEX, Portfolio, practice visits, case Shows how Shows how orals…. Knows how Knows  Self-, peer, co-assessment, portfolio, multisource feedback, learning process Does evaluations…… Shows how Knows how Knows