Articles (Mundy, Waelti, Steinberg, McNally, Likhtik)

### Dopamine Signals and Reward Prediction Errors

• Dopamine signals are consistent with reward prediction errors in temporal difference reinforcement learning.
• Dopamine activity increases during unexpected rewards (positive prediction error), decreases during omitted rewards (negative prediction error), and remains unchanged during expected rewards (no prediction error).
• Monkey study: Monkeys were trained to associate visual stimuli (A, B, X, Y) with rewards.
  • Phase 1: A reliably predicted reward.
  • Phase 2: B never predicted reward.
  • Phase 3: AX combination predicted reward (learning about compound cues).
  • Phase 4: BY combination predicted reward.
• Dopamine activity in the study:
  • Increased in response to A and AX but not to X because it was not directly paired with reward.
  • Increased in response to BY and Y because they were paired with reward.
  • Decrease in response to A when the expected reward was omitted.
• Optogenetics: Artificial manipulation of dopamine neurons to mimic reward prediction errors.
  • Optogenetic inhibition: Blocking of dopamine neurons during downshift extinction (less reward than expected) impaired extinction learning.
  • Optogenetic activation: Mimicking a positive prediction error during non-rewarded trials enhanced learning of an association with reward.
  • Conclusion: Dopamine signals are causally linked to learning, with changes in their activity affecting learning from changes in reward expectations.

Opioid Receptors and Fear Extinction

• Hypothesis: Opioid receptor antagonism in the periaqueductal gray (PAG) impairs fear extinction.
• Methods: Rats were trained to associate an auditory stimulus (CS) with a foot shock (US), leading to fear conditioning.
• Fear extinction: Extinction training involved repeated presentations of the CS without the US, aiming to reduce the freezing response.
• Naloxone: An opioid receptor antagonist, its effects were tested on different PAG subregions.
• Experiment #1: Microinjections of naloxone into the ventrolateral PAG (vlPAG) impaired fear extinction during the initial phase but did not reinstate the extinguished fear response.
• Experiment #2: Naloxone in the vlPAG also impaired extinction, confirming the reliability of the impairment and demonstrating its effect on a drug-free test.
• Experiment #3: Naloxone microinjections into the dorsolateral PAG (dlPAG) did not impair fear extinction.
• Experiment #4: Naloxone's effect on fear extinction was dose-dependent, with higher doses leading to greater freezing during testing.
• Conclusion: Opioid receptors in the vlPAG are crucial for fear extinction. Disrupting them impairs the ability to extinguish conditioned fear responses.

Interconnected Brain Regions and Fear Extinction

• Fear conditioning: The amygdala (BLA) plays a critical role in fear conditioning, with projections to the central amygdala (CeA) leading to fear expression.
• Fear extinction: The infralimbic cortex (ITC) inhibits fear expression by projecting to the CeA.
• ITC disruption: Disrupting ITC neurons is hypothesized to impair fear extinction, leading to the persistence of fear responses after extinction training.

Mundy et al. Study

• Hypothesis: Hippocampus and Perirhinal Cortex contribute to distinct types of perceptual learning.
  • Hippocampus plays a selective role in scene learning.
  • Perirhinal Cortex plays a dominant role in face learning.
• Participants:
  • Two patients with brain damage (HC3, MTL3).
  • 16 healthy participants.
• Methods:
  • Combined causal (lesioned patients) and correlational (fMRI) approaches to study MTL's role in visual information encoding.
  • fMRI with BOLD signal measured brain activity.
  • Focused on regions of interest (fROI):
    • FFA: Face encoding region.
    • PPA: Scene encoding region.
    • MTL regions: Perirhinal Cortex (PrC), Anterior Hippocampus (AntHC), Posterior Hippocampus (PostHC).
• Results:
  • Experiment 1 (Lesion Study):
    • Discrimination:
      • Control participants showed equal differentiation between pre-exposed and non-exposed stimuli.
      • HC3 exhibited difficulty differentiating pre-exposed and non-exposed scenes.
      • MTL3 displayed impairment in differentiating both pre-exposed faces and scenes.
    • Reaction Time (RT):
      • Control participants showed no RT differences between pre-exposed and non-exposed stimuli.
      • HC3 took longer to respond to scene stimuli.
      • MTL3 took longer to respond to face and scene stimuli.
    • Learning:
      • Control participants showed learning across blocks for all stimulus types.
      • HC3 showed learning for dots and faces but not scenes.
      • MTL3 showed learning for dots but not faces or scenes.
  • Experiment 2 (fMRI Study):
    • 2A (Exposure History):
      • No significant differences in discrimination accuracy between stimulus types.
      • Participants performed better in discriminating pre-exposed stimuli compared to non-exposed stimuli.
    • 2B (Discrimination Accuracy):
      • FFA exhibited higher activity for pre-exposed face pairs compared to non-exposed face pairs.
      • PPA showed higher activity for pre-exposed scene pairs compared to non-exposed scene pairs.
      • No modulation observed for dots.
      • PrC showed association with discrimination accuracy for faces but not scenes.
      • PostHC showed association with discrimination accuracy for scenes but not faces.
      • AntHC exhibited no association with discrimination accuracy for either faces or scenes.

Waelti et al. Study

• Hypothesis: Dopamine signals are not always consistent with reward prediction errors.
  • Emphasizes the prediction error signal in temporal difference reinforcement learning.
• Participants: Monkeys.
• Methods:
  • Four visual stimuli (A, B, X, Y) presented during four phases.
    • Phase 1: A + juice (A > CR)
    • Phase 2: B + no juice (B)
    • Phase 3: AX + juice (A+X > CR)
    • Phase 4: BY + juice (Y > CR)
• Results:
  • Anticipatory Licking:
    • Present for A and AX but absent for X during the test phase.
    • No anticipatory licking observed for B.
    • Anticipatory licking for BY (since it was paired with juice).
    • Anticipatory licking for Y during the test phase.
  • Dopamine Activity:
    • Phase 1 & 2:
      • A: Dopamine response to CS, dopamine dip when reward not presented.
      • B: No response to CS, dopamine activity to unexpected reward.
    • Compound Phases:
      • AX: Dopamine response to A+X.
      • BY: Increasing dopamine response to B+Y.
    • Test Phase:
      • X: Response to CS (but less than Y), no change in response when not presented.
      • Y: Increased response to CS, dopamine dip when not presented.
  • Reward Prediction Errors:
    • Unexpected rewards: Prediction error signal; increase in dopamine activity.
    • Omitted reward: Negative prediction error; decrease in dopamine activity.
    • Expected reward: No prediction error; no change in dopamine activity.

Steinberg et al. Study

• Hypothesis: Dopamine activity functions as a teaching signal, influencing learning about antecedent cues.
  • Focuses on the role of reward prediction errors (RPE) in learning.
• Methods:
  • Used optogenetics to artificially mimic RPE by stimulating dopamine neurons.
  • Conducted behavioral experiments focusing on blocking and extinction tasks.
• Results:
  • Downshift Extinction Task:
    • Dopamine neurons exhibited reduced firing in response to a downshift in reward, indicating a negative prediction error.
    • Optogenetic inhibition of dopamine neurons during downshift impaired extinction of learned behavior.
  • Non-rewarded Trial:
    • Optogenetic activation of dopamine neurons during non-rewarded trials enhanced learning of a stimulus-reward association.
  • Phasic Dopamine Signals:
    • Phasic dopamine signals directly influence learning, with manipulation (inhibition or activation) affecting the ability to learn from reward expectation changes.
  • Unexpected Reward Omission:
    • Dopamine neuron activity during unexpected reward omission was crucial for behavioral adjustments in response to reduced rewards.
  • Dopamine Neuron Inhibition:
    • Animals with inhibited dopamine neurons during reward prediction errors (positive and negative) showed learning deficits in response to changing reward contingences.

McNally et al. Study

• Hypothesis: Opioid receptors within the Periaqueductal Gray (PAG) play a crucial role in fear extinction, specifically through conditioned freezing behavior.
• Methods:
  • Fear Acquisition: Rats were conditioned to associate an auditory CS with a foot shock US (tone + footshock = freezing).
  • Fear Extinction: Extinction involved repeatedly presenting the CS without the US (tone + no footshock = decreased freezing).
  • Experiments:
    • Experiment 1: Focused on the role of vlPAG opioid receptors in acquisition and expression of fear extinction, using microinjections of the opioid receptor antagonist naloxone.
    • Experiment 2: Further examined the role of vlPAG opioid receptors in extinction, using different CS exposure parameters and a drug-free test.
    • Experiment 3: Investigated the role of dPAG opioid receptors in extinction, using the same methodology as Experiment 1.
    • Experiment 4: Examined the dose-response relationship between naloxone and fear extinction in vlPAG.
• Results:
  • Experiment 1:
    • Naloxone microinjections into vlPAG impaired fear extinction but did not reinstate fear responses.
    • Naloxone microinjections into PAG disrupted the development but not the expression of extinction.
  • Experiment 2:
    • Naloxone microinjections into vlPAG hindered fear extinction and this impairment was observed in a drug-free test.
  • Experiment 3:
    • Naloxone microinjections into dPAG did not impair fear extinction, highlighting the specific role of the vlPAG in this process.
  • Experiment 4:
    • The effects of naloxone on fear extinction exhibited a dose-dependent relationship- as the naloxone dose increased, so did the amount of freezing observed.

Likthik et al. Study

• Hypothesis: Disrupting the Interpeduncular Nucleus (ITC) neurons will impair fear extinction expression, leading to persistent fear responses even after extinction training.
• Key Findings:
  • Fear Conditioning: The basolateral amygdala (BLA) projects to the central amygdala (CeA), leading to fear expression.
  • Fear Extinction: The BLA projects to the ITC, which then inhibits the CeA, suppressing fear expression.
  • ITC Neuron Disruption: Impaired fear extinction due to persistent fear responses.