CB Metacognition 2024 Lecture Notes PDF

University of the Year for Student Experience (The Times/Sunday Times Good University Guide 2020) Metacognition Dr Chris Benwell 2024 dundee.ac.uk...

University of the Year for Student Experience (The Times/Sunday Times Good University Guide 2020) Metacognition Dr Chris Benwell 2024 dundee.ac.uk Page 1 Metacognition - Lecture Series 1. Introduction to metacognition 2. Measuring metacognition 3. Metacognition and psychopathology 4. Metacognition and belief structures 5. Neural correlates of metacognition 6. Evolution and metacognition in other species 7. Presentations/discussions 8. Metacognition and consciousness 9. Improving metacognition 10. The limits of self-knowledge dundee.ac.uk Page 2 Metacognition – Aims of the module  To gain a detailed understanding of the field of metacognition and metacognitive deficits in both the general population and those associated with psychopathology.  To become familiar with the common methodologies and paradigms in the area.  To obtain an understanding of the psychological and biological determinants of metacognition.  To be able to critically evaluate relevant research and contrast competing theories. dundee.ac.uk Page 3 Metacognition – Assessment Level 4 Coursework: 1 x 2500 word assignment due on Wednesday November 13th. We will dedicate part of the session in week 6 (beginning 21/08)) to preparation for the assignment. Exam: Remote exams at end of semester. 2 essay questions out of 5 choices within 6 hours. Level 5 Coursework: 2 x 2500 word assignments, each worth 50% of the module grade. dundee.ac.uk Page 4 What to study? (1) Lecture notes (2) I will upload relevant papers for each lecture to the module page on MyDundee. (3) Any questions/concerns: [email protected] dundee.ac.uk Page 5 What to study? (1) Lecture notes (2) I will upload relevant papers for each lecture to the module page on MyDundee. (3) Any questions/concerns: [email protected] Please interrupt me throughout to ask as soon as any questions pop up. No need to wait until the end of the lecture. My aim is for the sessions to be transactional. dundee.ac.uk Page 6 What to study? References Bang, J. W., Shekhar, M., & Rahnev, D. (2019). Sensory noise increases metacognitive efficiency. Journal of Experimental Psychology: General, 148(3), 437. Craig, K., Hale, D., Grainger, C., & Stewart, M. E. (2020). Evaluating metacognitive self-reports: systematic reviews of the value of self-report in metacognitive research. Metacognition and Learning, 15(2), 155-213. Fleming, S. M., & Lau, H. C. (2014). How to measure metacognition. Frontiers in human neuroscience, 8, 443. Guggenmos, M. (2021). Measuring metacognitive performance: type 1 performance dependence and test-retest reliability. Neuroscience of consciousness, 2021(1), niab040. Guggenmos, M. (2022). Reverse engineering of metacognition. Elife, 11, e75420. Maniscalco, B., & Lau, H. (2012). A signal detection theoretic approach for estimating metacognitive sensitivity from confidence ratings. Consciousness and cognition, 21(1), 422-430. Shekhar, M., & Rahnev, D. (2021). The nature of metacognitive inefficiency in perceptual decision making. Psychological review, 128(1), 45. Xue, K., Shekhar, M., & Rahnev, D. (2021). Examining the robustness of the relationship between metacognitive efficiency and metacognitive bias. Consciousness and Cognition, 95, 103196. dundee.ac.uk Page 7 Measuring Metacognition Metacognition represents awareness of our own cognitive processes (“thinking about thinking”) and so it is not easily observable. dundee.ac.uk Page 8 Measuring Metacognition Metacognition represents awareness of our own cognitive processes (“thinking about thinking”) and so it is not easily observable. The earliest and most common attempts to measure metacognition used self-report questionnaires dundee.ac.uk Page 9 Measuring Metacognition Metacognition represents awareness of our own cognitive processes (“thinking about thinking”) and so it is not easily observable. The earliest and most common attempts to measure metacognition used self-report questionnaires However, no ‘gold standard’ questionnaire exists and so self-report studies have been inconsistent in how they have measured metacognition (Craig et al., 2020) dundee.ac.uk Page 10 Measuring Metacognition Different taxonomies illustrate difficulty in fully capturing different aspects of metacognition. dundee.ac.uk Page 11 Measuring metacognition Different taxonomies illustrate difficulty in fully capturing different aspects of metacognition. Also, questionnaires pre-suppose that people are able to accurately report their metacognition! Biases might be at play and insight might not always be available. dundee.ac.uk Page 12 Measuring metacognition However, one key concept that is shared by all models of metacognition is the ability to introspect on and evaluate our thoughts and abilities. dundee.ac.uk Page 13 What is Metacognition? dundee.ac.uk Page 14 What is Metacognition? dundee.ac.uk Page 15 Back to the Better-Than-Average Effect! Clever way to characterise metacognitive accuracy at the group level, but can’t say anything about individuals within the group. dundee.ac.uk Page 16 Quantifying metacognitive ability To quantify metacognitive ability at the level of individual people, researchers have investigated how closely confidence ratings dissociate correct from incorrect performance. dundee.ac.uk Page 17 Quantifying metacognitive ability To quantify metacognitive ability at the level of individual people, researchers have investigated how closely confidence ratings dissociate correct from incorrect performance. dundee.ac.uk Page 18 Quantifying metacognitive ability To quantify metacognitive ability at the level of individual people, researchers have investigated how closely confidence ratings dissociate correct from incorrect performance. Type-1 response Type-2 response dundee.ac.uk Page 19 Quantifying metacognitive ability To quantify metacognitive ability at the level of individual people, researchers have investigated how closely confidence ratings dissociate correct from incorrect performance. “While there is a positive correlation on the whole between degree of confidence and accuracy, the degree of confidence is not a reliable index of accuracy.” Henmon (1911). The relation of the time of a judgment to its accuracy. Psychol. Rev. 18, 186. dundee.ac.uk Page 20 Quantifying metacognitive ability Types of second-order report dundee.ac.uk Page 21 Quantifying metacognitive ability dundee.ac.uk Page 22 Quantifying metacognitive ability From the outset, it is important to distinguish two key contributors to metacognitive performance: sensitivity and bias. Fleming & Lau (2014) dundee.ac.uk Page 23 Quantifying metacognitive ability Fullerton and Cattell (1892) already noted that ‘different individuals place very different meanings on the degree of confidence. Some observers are nearly always quite or fairly confident, while others are seldom confident.’ Fleming & Lau (2014) dundee.ac.uk Page 24 Quantifying metacognitive ability Revisiting Signal Detection Theory (SDT) Distance between ‘signal absent’ and ‘signal present’ distributions = d-prime (d’) dundee.ac.uk Page 25 Quantifying metacognitive ability Type-1 SDT Stimulus representation Behaviour Type-2 SDT dundee.ac.uk Page 26 Quantifying metacognitive ability From the outset, it is important to distinguish two key contributors to metacognitive performance: sensitivity and bias. Fleming & Lau (2014) dundee.ac.uk Page 27  How might we expect metacognitive bias and sensitivity to relate to other cognitive processes, personality traits and real-life outcomes? dundee.ac.uk Page 28 Quantifying metacognitive ability Maniscalco & Lau (2012) formalised this extension of SDT to metacognitive (‘type-2’) judgements (see also Galvin et al., 2003). dundee.ac.uk Page 29 Quantifying metacognitive ability Maniscalco & Lau (2012) formalised this extension of SDT to metacognitive (‘type-2’) judgements (see also Galvin et al., 2003). → Meta-d’ model dundee.ac.uk Page 30 Quantifying metacognitive ability Maniscalco & Lau (2012) formalised this extension of SDT to metacognitive (‘type-2’) judgements (see also Galvin et al., 2003). → Meta-d’ model → In the same way that type-1 SDT separates criterion and sensitivity, meta-d’ separates metacognitive bias and sensitivity. dundee.ac.uk Page 31 Quantifying metacognitive ability dundee.ac.uk Page 32 Quantifying metacognitive ability One major advantage of meta-d′ is its ease of interpretation and its (theoretically) elegant control over the influence of performance on metacognitive sensitivity. dundee.ac.uk Page 33 Quantifying metacognitive ability One major advantage of meta-d′ is its ease of interpretation and its (theoretically) elegant control over the influence of performance on metacognitive sensitivity. → Specifically, because meta-d′ is in the same units as (type 1) d′, the two can be directly compared. Therefore, for a metacognitively ideal observer (a person who is rating confidence using the maximum possible metacognitive sensitivity), meta-d′ should equal d′. If meta- d′ < d′, metacognitive sensitivity is suboptimal within the SDT framework. We can therefore define metacognitive efficiency as the value of meta-d′ relative to d′, or meta-d′/d′. dundee.ac.uk Page 34 Quantifying metacognitive ability One major advantage of meta-d′ is its ease of interpretation and its (theoretically) elegant control over the influence of performance on metacognitive sensitivity. → Specifically, because meta-d′ is in the same units as (type 1) d′, the two can be directly compared. Therefore, for a metacognitively ideal observer (a person who is rating confidence using the maximum possible metacognitive sensitivity), meta-d′ should equal d′. If meta- d′ < d′, metacognitive sensitivity is suboptimal within the SDT framework. We can therefore define metacognitive efficiency as the value of meta-d′ relative to d′, or meta-d′/d′. → A meta-d′/d′ value of 1 indicates a theoretically ideal value of metacognitive efficiency. A value of 0.7 would indicate 70% metacognitive efficiency (30% of the sensory evidence available for the decision is lost when making metacognitive judgments), and so on. dundee.ac.uk Page 35 Quantifying metacognitive ability Why is it important to control for 1st order performance? dundee.ac.uk Page 36 Quantifying metacognitive ability However, the meta-d’ model has recently come under scrutiny: dundee.ac.uk Page 37 Quantifying metacognitive ability However, the meta-d’ model has recently come under scrutiny: → One limitation is that the model can only be applied to 2-alternative forced choice (2-AFC) data. So if a decision task has 3 or more response options (or open ended responses) then the meta-d’ model cannot be used. dundee.ac.uk Page 38 Quantifying metacognitive ability However, the meta-d’ model has recently come under scrutiny: → One limitation is that the model can only be applied to 2-alternative forced choice (2-AFC) data. So if a decision task has 3 or more response options (or open ended responses) then the meta-d’ model cannot be used. → In reality, measures of metacognitive efficiency based on meta-d’ have been shown to NOT be independent of type-1 performance under certain circumstances (Bang et al., 2019; Guggenmos, 2021). dundee.ac.uk Page 39 Quantifying metacognitive ability However, the meta-d’ model has recently come under scrutiny: → One limitation is that the model can only be applied to 2-alternative forced choice (2-AFC) data. So if a decision task has 3 or more response options (or open ended responses) then the meta-d’ model cannot be used. → In reality, measures of metacognitive efficiency based on meta-d’ have been shown to NOT be independent of type-1 performance under certain circumstances (Bang et al., 2019; Guggenmos, 2021). → Furthermore, overall confidence level has been shown to bias estimates of metacognitive efficiency (Xue et al., 2021; Shekhar & Rahnev, 2021). dundee.ac.uk Page 40 Quantifying metacognitive ability However, the meta-d’ model has recently come under scrutiny: → One limitation is that the model can only be applied to 2-alternative forced choice (2-AFC) data. So if a decision task has 3 or more response options (or open ended responses) then the meta-d’ model cannot be used. → In reality, measures of metacognitive efficiency based on meta-d’ have been shown to NOT be independent of type-1 performance under certain circumstances (Bang et al., 2019; Guggenmos, 2021). → Furthermore, overall confidence level has been shown to bias estimates of metacognitive efficiency (Xue et al., 2021; Shekhar & Rahnev, 2021). → Meta-d’ measures show poor test-retest reliability (Guggenmos, 2021). dundee.ac.uk Page 41 Quantifying metacognitive ability New approaches in development: → Reverse engineering of metacognition (ReMeta) (Guggenmos, 2022) dundee.ac.uk Page 42 Quantifying metacognitive ability New approaches in development: → Reverse engineering of metacognition (ReMeta) (Guggenmos, 2022) → Initial evidence promising that metacognitive measures (sensitivity and bias) are independent of type-1 performance. dundee.ac.uk Page 43 Quantifying metacognitive ability New approaches in development: → Metacognitive Information Theory (Meta-I: Dayan, 2022 ( https://europepmc.org/article/ppr/ppr547128)) dundee.ac.uk Page 44 Quantifying metacognitive ability New approaches in development: → Metacognitive Information Theory (Meta-I: Dayan, 2022 ( https://europepmc.org/article/ppr/ppr547128)) → Quantifies the amount of information about decision accuracy contained in corresponding confidence ratings. Non-parametric (i.e., does not depend on assumptions of any given model) dundee.ac.uk Page 45  Discussion/questions dundee.ac.uk Page 46  How might we expect metacognitive bias and sensitivity to relate to other cognitive processes, personality traits and real-life outcomes? Can you think of an interesting research project to investigate this? dundee.ac.uk Page 47 Next week Metacognition and Psychopathology dundee.ac.uk Page 48 dundee.ac.uk What to study? References Bang, J. W., Shekhar, M., & Rahnev, D. (2019). Sensory noise increases metacognitive efficiency. Journal of Experimental Psychology: General, 148(3), 437. Craig, K., Hale, D., Grainger, C., & Stewart, M. E. (2020). Evaluating metacognitive self-reports: systematic reviews of the value of self-report in metacognitive research. Metacognition and Learning, 15(2), 155-213. Fleming, S. M., & Lau, H. C. (2014). How to measure metacognition. Frontiers in human neuroscience, 8, 443. Guggenmos, M. (2021). Measuring metacognitive performance: type 1 performance dependence and test-retest reliability. Neuroscience of consciousness, 2021(1), niab040. Guggenmos, M. (2022). Reverse engineering of metacognition. Elife, 11, e75420. Maniscalco, B., & Lau, H. (2012). A signal detection theoretic approach for estimating metacognitive sensitivity from confidence ratings. Consciousness and cognition, 21(1), 422-430. Shekhar, M., & Rahnev, D. (2021). The nature of metacognitive inefficiency in perceptual decision making. Psychological review, 128(1), 45. Xue, K., Shekhar, M., & Rahnev, D. (2021). Examining the robustness of the relationship between metacognitive efficiency and metacognitive bias. Consciousness and Cognition, 95, 103196. dundee.ac.uk Page 50