Objective Analysis of Confidence in Decisions and Accuracy of AIs PDF
Document Details
Uploaded by emilyroseblack
Tags
Related
Summary
This document presents an overview of different studies conducted on assessing confidence in AI systems, examining methodology, findings, and limitations. It details various approaches to measuring confidence in computer decision-making. The studies focus on the role of confidence assessments in decision processes and accuracy metrics.
Full Transcript
+-----------+-----------+-----------+-----------+-----------+-----------+ | **Objecti | **Author* | **Method* | **Finding | **Interpr | **Limitat | | ve** | * | * | s** | etation** | ion** | +===========+===========+===========+===========+===========+===========+ | "C...
+-----------+-----------+-----------+-----------+-----------+-----------+ | **Objecti | **Author* | **Method* | **Finding | **Interpr | **Limitat | | ve** | * | * | s** | etation** | ion** | +===========+===========+===========+===========+===========+===========+ | "Confiden | Ais et | Made | Confidenc | Confidenc | Exact | | ce | al., 2016 | perceptua | e | e | cognitive | | in | | l | ratings | not | /neural | | decisions | | + rated | only | perfect | processes | | and its | | confidenc | partially | reflectio | behind | | accuracy" | | e | aligned | n | heuristic | | | | in those | with | of | influence | | | | judgments | decision | accuracy | s | | | | | accuracy | -- rather | remain | | | | | | heuristic | unclear | | | | | | -based | | | | | | | judgment | | +-----------+-----------+-----------+-----------+-----------+-----------+ | "Domain | Kopcanova | Measured | MC | MC shows | 2 days | | generalit | et al., | MC | performan | partial | not full | | y | 2013 | ability | ce | generalit | capture | | of MC" | | in | stable | y; | long-term | | | | perceptio | within | overarchi | patterns/ | | | | n | each | ng | variabili | | | | + | domain | strategie | ty | | | | knowledge | | s | | | | | tasks | Little | for | | | | | across 2 | overlap | self-moni | | | | | days | in MC | toring | | | | | | ability | but | | | | | Compared | between | distinct | | | | | MC | tasks = | mechanism | | | | | performan | domain-sp | s | | | | | ce | ecificity | for | | | | | across | | sensory | | | | | domains | | vs | | | | | | | cognitive | | | | | | | tasks | | +-----------+-----------+-----------+-----------+-----------+-----------+ | "Differen | Rouault | Used | Higher | Specific | Cannot | | ces | et al., | statistic | openness | traits, | establish | | in | 2018 | al | to | "openness | causation | | personali | | methods | experienc | ", | between | | ty | | to | es | may | personali | | predict | | examine | associate | facilitat | ty | | MC | | how | d | e | + MC | | sensitivi | | individua | with MC | self-moni | | | ty" | | l | ability | toring | | | | | differenc | | abilities | | | | | es | | | | | | | predict | | | | | | | MC | | | | | | | sensitivi | | | | | | | ty | | | | +-----------+-----------+-----------+-----------+-----------+-----------+ | "Training | Baird et | 2-week | Meditatio | Training | Efficienc | | programme | al., 2014 | "Samatha" | n | may | y | | " | | meditatio | improved | improve | seen for | | | | n | MC | brain's | memory | | | | on MC | efficienc | ability | but not | | | | efficienc | y; | to | perceptua | | | | y | became | evaluate/ | l | | | | in memory | better at | regulate | performan | | | | task. | aligning | cognitive | ce | | | | | their | processes | | | | | Rated | confidenc | | | | | | confidenc | e | | | | | | e | ratings | | | | | | before/af | with | | | | | | ter | memory | | | | | | training | accuracy | | | +-----------+-----------+-----------+-----------+-----------+-----------+ | "Feeback | Carpenter | Compared | Receiving | Feedback | Incentivi | | training | et al., | control | feedback | enhances | sing | | could | 2019 | group who | = sig. | self-moni | post-trai | | improve | | only | improveme | toring | ning | | MC | | received | nt | abilities | may have | | sensitivi | | feedback | in MC | , | inflated | | ty" | | on 1^st^ | sensitivi | unlike | improveme | | | | order | ty | task-focu | nt; | | | | performan | | sed | causing | | | | ce | Control = | feedback | abrupt | | | | | no | alone | increase | | | | | improveme | | in | | | | | nt | | confidenc | | | | | | | e | | | | | | | from pre | | | | | | | to post | +-----------+-----------+-----------+-----------+-----------+-----------+ | "MC | Rouy et | Replicate | Replicate | Reinforce | Findings | | feedback, | al., 2022 | d | d | s | still | | controlli | | Carpenter | the | validity | specific | | ng | | , | original | of | to | | confounds | | in | improveme | feedback | training | | " | | independe | nt | training | task; can | | | | nt | in | as | be | | | | sample; | sensitivi | effective | improved | | | | controlle | ty | method | independe | | | | d | with | | ntly | | | | incentive | training | | of 1^st^ | | | | -related | | | order | | | | confounds | | | | +-----------+-----------+-----------+-----------+-----------+-----------+ | "Effectiv | Normann & | Meta-anal | MCT = | Findings | Variabili | | eness | Morina, | ytic | highly | support | ty | | of MCT | 2018 | review | effective | MCT as | in study | | for | | covering | , | robust | designs | | treating | | condition | large | intervent | and | | psycholog | | s | effect | ion | quality | | ical | | like | sizes for | for | may | | disorders | | anxiety + | reducing | disorders | impact | | " | | depressio | symptoms; | | generalis | | | | n | greater | | ability | | | | | efficacy | | | | | | | compared | | | | | | | to other | | | | | | | approache | | | | | | | s | | | +-----------+-----------+-----------+-----------+-----------+-----------+
