Objective Analysis of Confidence in Decisions and Accuracy of AIs PDF
Document Details
Uploaded by emilyroseblack
Tags
Summary
This document presents an overview of different studies conducted on assessing confidence in AI systems, examining methodology, findings, and limitations. It details various approaches to measuring confidence in computer decision-making. The studies focus on the role of confidence assessments in decision processes and accuracy metrics.
Full Transcript
+-----------+-----------+-----------+-----------+-----------+-----------+ | **Objecti | **Author* | **Method* | **Finding | **Interpr | **Limitat | | ve** | * | * | s** | etation** | ion** | +===========+===========+===========+===========+===========+===========+ | "C...
+-----------+-----------+-----------+-----------+-----------+-----------+ | **Objecti | **Author* | **Method* | **Finding | **Interpr | **Limitat | | ve** | * | * | s** | etation** | ion** | +===========+===========+===========+===========+===========+===========+ | "Confiden | Ais et | Made | Confidenc | Confidenc | Exact | | ce | al., 2016 | perceptua | e | e | cognitive | | in | | l | ratings | not | /neural | | decisions | | + rated | only | perfect | processes | | and its | | confidenc | partially | reflectio | behind | | accuracy" | | e | aligned | n | heuristic | | | | in those | with | of | influence | | | | judgments | decision | accuracy | s | | | | | accuracy | -- rather | remain | | | | | | heuristic | unclear | | | | | | -based | | | | | | | judgment | | +-----------+-----------+-----------+-----------+-----------+-----------+ | "Domain | Kopcanova | Measured | MC | MC shows | 2 days | | generalit | et al., | MC | performan | partial | not full | | y | 2013 | ability | ce | generalit | capture | | of MC" | | in | stable | y; | long-term | | | | perceptio | within | overarchi | patterns/ | | | | n | each | ng | variabili | | | | + | domain | strategie | ty | | | | knowledge | | s | | | | | tasks | Little | for | | | | | across 2 | overlap | self-moni | | | | | days | in MC | toring | | | | | | ability | but | | | | | Compared | between | distinct | | | | | MC | tasks = | mechanism | | | | | performan | domain-sp | s | | | | | ce | ecificity | for | | | | | across | | sensory | | | | | domains | | vs | | | | | | | cognitive | | | | | | | tasks | | +-----------+-----------+-----------+-----------+-----------+-----------+ | "Differen | Rouault | Used | Higher | Specific | Cannot | | ces | et al., | statistic | openness | traits, | establish | | in | 2018 | al | to | "openness | causation | | personali | | methods | experienc | ", | between | | ty | | to | es | may | personali | | predict | | examine | associate | facilitat | ty | | MC | | how | d | e | + MC | | sensitivi | | individua | with MC | self-moni | | | ty" | | l | ability | toring | | | | | differenc | | abilities | | | | | es | | | | | | | predict | | | | | | | MC | | | | | | | sensitivi | | | | | | | ty | | | | +-----------+-----------+-----------+-----------+-----------+-----------+ | "Training | Baird et | 2-week | Meditatio | Training | Efficienc | | programme | al., 2014 | "Samatha" | n | may | y | | " | | meditatio | improved | improve | seen for | | | | n | MC | brain's | memory | | | | on MC | efficienc | ability | but not | | | | efficienc | y; | to | perceptua | | | | y | became | evaluate/ | l | | | | in memory | better at | regulate | performan | | | | task. | aligning | cognitive | ce | | | | | their | processes | | | | | Rated | confidenc | | | | | | confidenc | e | | | | | | e | ratings | | | | | | before/af | with | | | | | | ter | memory | | | | | | training | accuracy | | | +-----------+-----------+-----------+-----------+-----------+-----------+ | "Feeback | Carpenter | Compared | Receiving | Feedback | Incentivi | | training | et al., | control | feedback | enhances | sing | | could | 2019 | group who | = sig. | self-moni | post-trai | | improve | | only | improveme | toring | ning | | MC | | received | nt | abilities | may have | | sensitivi | | feedback | in MC | , | inflated | | ty" | | on 1^st^ | sensitivi | unlike | improveme | | | | order | ty | task-focu | nt; | | | | performan | | sed | causing | | | | ce | Control = | feedback | abrupt | | | | | no | alone | increase | | | | | improveme | | in | | | | | nt | | confidenc | | | | | | | e | | | | | | | from pre | | | | | | | to post | +-----------+-----------+-----------+-----------+-----------+-----------+ | "MC | Rouy et | Replicate | Replicate | Reinforce | Findings | | feedback, | al., 2022 | d | d | s | still | | controlli | | Carpenter | the | validity | specific | | ng | | , | original | of | to | | confounds | | in | improveme | feedback | training | | " | | independe | nt | training | task; can | | | | nt | in | as | be | | | | sample; | sensitivi | effective | improved | | | | controlle | ty | method | independe | | | | d | with | | ntly | | | | incentive | training | | of 1^st^ | | | | -related | | | order | | | | confounds | | | | +-----------+-----------+-----------+-----------+-----------+-----------+ | "Effectiv | Normann & | Meta-anal | MCT = | Findings | Variabili | | eness | Morina, | ytic | highly | support | ty | | of MCT | 2018 | review | effective | MCT as | in study | | for | | covering | , | robust | designs | | treating | | condition | large | intervent | and | | psycholog | | s | effect | ion | quality | | ical | | like | sizes for | for | may | | disorders | | anxiety + | reducing | disorders | impact | | " | | depressio | symptoms; | | generalis | | | | n | greater | | ability | | | | | efficacy | | | | | | | compared | | | | | | | to other | | | | | | | approache | | | | | | | s | | | +-----------+-----------+-----------+-----------+-----------+-----------+
