Signal Detection Theory: Decision Making

Objectives of the Report

• Provides a comprehensive understanding of Signal Detection Theory (SDT) and its role in decision-making under uncertainty
• Explores models and concepts of SDT like hits, misses, false positives (false alarms), and correct rejections
• Examines the applications of SDT in medical diagnostics, security, human performance, and engineering psychology
• Evaluates the impact of response bias and the Receiver Operating Characteristic (ROC) Curve in assessing the accuracy and performance of decision-making
• Introduces Fuzzy Signal Detection Theory (FSDT) as an extension of SDT
• Emphasizes the importance of SDT in optimizing alarm systems, improving recognition memory, and refining risk assessment models

Introduction to Signal Detection Theory (SDT)

• SDT offers a framework for understanding how people interpret and react to signals amid noise
• SDT categorizes observations into the presence or absence of a signal
• The theory helps analyze factors influencing detection performance and proposing error solutions
• SDT is used in psychology, medicine, and engineering to enhance detection capabilities

Binary Signal Detection

• Perception involves complex scenarios like tasks requiring absolute judgments or multidimensional categorizations
• These concepts give insights into how humans process and respond to sensory information
• Advanced concepts form a foundation for theoretical exploration and practical application in diverse fields

Signal Detection Paradigm

• SDT applies when an environment is separated into signal and noise and is applicable to detecting techniques whether by humans, by machines, or by animals.
• The goal of SDT is to distinguish whether a signal is present or absent amidst potential noise

2 x 2 Framework of SDT

• Detection decision outcomes are categorized into four possibilities in a 2x2 matrix.
• Hit: A signal is present, and the observer correctly identifies it
• Miss: A signal is present, but the observer fails to detect it
• False Alarm: A signal is absent, but the observer mistakenly identifies it as present
• Correct Rejection: A signal is absent, and the observer correctly identifies that no signal is present
• Perfect detection performance is when there are no misses or false alarms
• Problems reduce perfection, like hazy signals, fatigued operators, or external distractions
• SDT example: A lifeguard must decide if perceived movement is a swimmer in trouble (signal) or visual activity (noise)
• Factors such as fatigue, experience, and conditions affect the lifeguard's ability to detect genuine signals
• SDT uses sensitivity and decision criteria to model the decision-making process
• Outcomes include correct detection, a miss, a false alarm, or a correct rejection

Application Scenarios for SDT

• Detecting concealed weapons in airport security screenings
• Identifying targets on radar scopes
• Recognizing malignant tumors in medical imaging
• Evaluating hazardous driving conditions
• Identifying cybersecurity threats/suspicious messages
• Evaluating diagnostic tests like the COVID-19 test
• In these situations, the observer must decide between signal and noise

Key Metrics in SDT

• Performance is quantified using two probabilities
• Hit Rate (P[Hit]): Proportion of actual signals correctly identified
  • P(Hit) = Hits / (Hits + Misses)
• False Alarm Rate (P[FA]): Proportion of noise trials incorrectly identified as signals
  • P(FA) = False Alarms / (False Alarms + Correct Rejections)
• Metrics help assess the effectiveness of a detection system
• Sensitivity evaluates true positives, or hits
• Specificity (Selectivity) evaluates the correct rejection of false positives
  • Which is found by calculating 1 − P(FA)
• False alarms are called false positives, and misses are called false negatives

Practicality of SDT

• Practical uses involve a trade-off between sensitivity and specificity
• Prioritizing sensitivity maximizes hits in security but can lead to unneeded delays
• Reducing false negatives is essential in medical diagnostics since not identifying a condition could have serious repercussions
• SDT helps comprehend these trade-offs and make informed decisions

SDT Model

• SDT offers a foundation for comprehending how signals are found in surroundings with noise
• The SDT model states that in regards to the information processing process, there are two steps involved in detection:
• Aggregation of Sensory Evidence: Information is collected and combined to determine if a signal is present or absent; this combined evidence is represented by a continuously varying metric (X)
• Decision Making: Based on the value of X, a decision is made about the evidence and whether it indicates the presence of a signal.
• On average, the evidence X is greater when a signal is present than when it is absent
• Random fluctuations can occur in the environment, even with the absence of a signal

Decision Thresholds and Errors

• A critical threshold Xc is established to make a signal present decision
• Two types of errors can occur:
  • False Alarms: Fluctuations cause X to exceed Xc, leading to an incorrect "yes" response
  • Misses: Activity may keep X below Xc, resulting in an incorrect "no" response
• The probability of errors depends on how far apart the evidence distributions of signal and noise are
• The weaker the signal, the more overlap between X and noise
• Strong signals move the average level of X further from the noise distribution
• Understanding evidence, decision thresholds, and signal strength helps to design working detection systems
• Changing the decision threshold XC can be done to trade sensitivity and specificity
• This can directly be applied to medical diagnosis, security scanning, and air traffic surveillance

Signal and Noise Representations in SDT

• Signal and noise are represented as two overlapping normal distributions
• Distributions describe the probability of observing values of the evidence X
  • Noise Trial (Left Curve) occurs when Generated solely from noise,
  • Signal Trial (Right Curve) is a result of a combination of noise and signal
• The criterion value Xc represents whether the evidence indicates a signal
• Criterion placement determines decision outcomes divided into four regions
  • Hits: Correctly identifying a signal when it is present
  • Misses: Failing to detect a signal when it is present
  • False Alarms: Incorrectly identifying noise as a signal
  • Correct Rejections: Correctly identifying noise when no signal is present
• The total area under each curve equals 1
  • P(Hit) + P(Miss) = 1 and P(False Alarm) + P(Correct Rejection) = 1

Monitoring a Radar Screen

• For defense watch officers, the noise is from reflections from clouds, rain, or other non-aircraft objects, aircraft signals appears against background
• When noise levels are low, the signal stands out, making the aircraft easier to identify
• When noise levels are high, the signal may be masked, leading the officer to miss the aircraft
• Random noise spikes may resemble a signal, causing the officer to mistakenly identify it
• If the noise is recognized, the officer will conclude that no aircraft is present
• The criteria position determines the balance between hits and false alarms, depending on mission priorities

Sensitivity and Criterion Adjustment

• Choosing a Lower Xc makes operators more likely to respond, leading to more hits but more false alarms
• Choosing a Higher Xc makes operators more conservative, leading to reduced false alarms but increased misses
• Noise and sensitivity adjustment optimizes detection tasks
• The right Xc depends on priorities: avoiding potential threats or reducing unnecessary alerts
• Apply these principles across various domains such as medical diagnostics, cybersecurity, and transportation safety

Response Criterion: Optimality in Signal Detection Theory

• Observers establish a criterion that strikes a balance to prevent false alarms

• Signal Detection Theory (SDT) involves analyzing the decision-making process and involves a liberal or conservative approach

  • Liberal Strategy: Observers are prone to saying "yes," which results in many signals, but also false alarms
  • Conservative Strategy: Observers frequently say "no," reducing false alarms but increasing missed signals

• The decision criterion plays a crucial role in determining hits and false alarms, and adjusting can change the evidence required for a positive response

• An important parameter is denoted as B, and defined as:

  β=

  Where: is the probability of a signal given an evidence, and is the probability of noise given evidence.

• This ratio estimates the threshold used by observers to decide between signal and noise

Adjusting B Based on Probabilities

• Optimal value of depends on the relative probabilities of signal and noise

• Which is modeled from: =

• Rare observer signals can be conservative so as to minimize false alarms

• Environmental conditions can trade-off better hits if a signal’s are frequent

• Account costs and also benefits for the beta optimality

  • Modeled from:

  Where:Value of a correct rejection, cost of a false alarm, value of a hit, cost of a miss

• For example, an X-ray of a possible cancer might be favored to prioritizing misses so as to avoid those severe consequences

• Control inspectors instead might want to adopt a more conservative measure to minimize false alarms

• Evidence shows thresholds rely on relative signal and noise probabilities

• This optimizes decision-making by considering environmental probabilities

• Factoring in psychological and monetary payoffs into decisions involves the economic implications in settings

Understanding Beta

• Response criteria relies strongly on context, including signal and noise probabilities alongside the pay-offs of outcomes
• Adapting to situations and demands shows practical applications for utilities such as diagnostics and qualities
• Key concept for how humans make decisions under uncertainty
• This phenomenon of "sluggish beta," explores the deviation in human adjustment away from the optimal value

Performance In Setting Beta

• Beta measures data on hits and false alarms from detection
• An optimal beta is determined by the probabilities with payoffs
• Humans adjusting beta is hard, and adjustments are les pronounced than needed (aka: sluggish beta)
  • high = Humans tend to be less conservative/ optimal
  • low = Are more conservative and miss more signals

Sluggish In Action

• In adjustments beta show the ideal plots with the ideal probabilities and payoffs
• Labs show that beta is adjusted based on the probabilities than just focusing on the payoffs

Things To account For with Sluggish Behavior

• Humans adjust decision criteria to follow the probabilities rather than follow an optimal approach
• Participants rely on hits and misses, and the adjustments over time
• People overestimate/ underestimate certain odds (cognitive measure)
• Those confirm the real and labs situations while it adjust with sluggish behaviors

Practical Applications

• Setting thresholds for alarms
• Inspections, where one of each outcome is changed
• Improvments of an alarm means better beta adjustments

Importance of These Behaviors For:

• Healthcare systems
• Security systems
• Human Factors

Report on Receiver Operating Characteristic (ROC) Curve

• It is used to analyze the trade-offs between sensitivity and specificity, which represents a relationship between the possibility of a hit and the odd of of alarm (P/FA)
• You can see as how sensitive an observer has in the situation from the noise

Representation

• A point is based on an observers sensitivity
• Observers with better high true and low alarms lead to be closer to the right
  • Otherwise, closer to level performances

Interpretation

• Beta represents the tangent slopes on tangent from ROC curve
• When equal, its chance
• Empirical means that it may be wrong, and can vary within strengths

Measures

• Data cant be measured from the responses bias
• Alternative measures are out there
• Bias can be more stable using a more constant beta

Fuzzy Signal Detection Theory

• It deals with when the decisions are from uncertainty
• This mainly has to do when it detects signals the mixed in
• Usually, it has to be binary (signal or no signal)
• Because signals have fuzzy and vague, and not binary, there for there is a fuzzy SDT

Integration Of Fuzzy

• Short is, to make decisions where distinguishing is not clear
• It's the step forward while investigating performance and activity
• It works on a system so that way the signals are classified with different types of information
• It could be that sections of the world are just signals in different pieces

Functions

• This is the fuzzy logic, and works using different types of changes
• Fuzzy works better against binary
• Signals are not static, since they always depend on the changes at the timed
• Functions can be assigned with certain outputs
• As well as being applied to ranges of the environment

Applications

• Deals with operation in the industrial engineering
• It's the different models, which mean a stronger effectiveness
• Interdisciplinary and well with the engineering since its strongly support that certain are well under uncertainty

How to deal with fuzzy

• It gets well with many decisions, which lead to better performance

Weather and Risk

• Rainfall, river, soil and a team to work with
• You would like to represent s
• The team provides the chance that things will happen (predictions)

Applications of Signal Detection Theory

• Principles are vital since is able to work with signal changes

Importance Of This Concept

• It makes the comparison for which its able to detect change within noise
• Controls a base and is able to see how certain outcomes change due to signals

Two Sides

• How well one sees the signal
• Seeing how things are in real life
• Knowing what the signal means, or seeing change
• Separations to what one feels

Medical: A

• Good since, tumors come and go
• Doctors should recognize this
• All cues influence what happens, and separate data
• Important for how to prevent problems

Example

• If something is not there
• Doctors have to show the result, and how things could happen
• Doctors chose and can detect these

Recognition Memory and Eyewitness Testimony

• Signals happen at an earlier stage
• All is for this and the law (psychology)
• So if you know signal or the noice
• All comes in all the ways to look at this

Applications

• To which are certain
• See how different parts affect

SDT framework for seeing thing:

• Line up
• See the second things and what it may see
• SDT for certain factors during

Instructions

• Can affect different changes

Conclusions

• Officers do not need to understand the signal

Important Fact

• Not the the truth for an accident
• Cautions from an accidences

Signal - for if its the real deal

• This may cause:
• Miss : when it is not something
• Beta : To not see the high rate when it goes off / is not
• All is to know the signal

Automated diagnoses

• This has to do for the changes in the situation

Errors of Auto

• All is for the set of the alarms
• Most of the signal means it better to not say if it means to say things
• And have less chances the say the false one