Impulse Control Disorders in Parkinson's Disease (PDF)

https://doi.org/10.1093/brain/awad162 BRAIN 2023: 146; 3676–3689 | 3676 Impulse control disorder in Parkinson’s disease is associated with abnormal frontal value signalling Ceyda Sayalı,3 Rick C. Helmich1,2 and Roshan Cools1,4 See Michael Browning (https://doi.org/10.1093/brain/awad248) for a scient...

https://doi.org/10.1093/brain/awad162 BRAIN 2023: 146; 3676–3689 | 3676 Impulse control disorder in Parkinson’s disease is associated with abnormal frontal value signalling Ceyda Sayalı,3 Rick C. Helmich1,2 and Roshan Cools1,4 See Michael Browning (https://doi.org/10.1093/brain/awad248) for a scientific commentary on this article. Dopaminergic medication is well established to boost reward- versus punishment-based learning in Parkinson’s dis­ ease. However, there is tremendous variability in dopaminergic medication effects across different individuals, with some patients exhibiting much greater cognitive sensitivity to medication than others. We aimed to unravel the me­ chanisms underlying this individual variability in a large heterogeneous sample of early-stage patients with Parkinson’s disease as a function of comorbid neuropsychiatric symptomatology, in particular impulse control disor­ ders and depression. One hundred and ninety-nine patients with Parkinson’s disease (138 ON medication and 61 OFF medication) and 59 healthy controls were scanned with functional MRI while they performed an established probabilistic instrumental learning task. Reinforcement learning model-based analyses revealed medication group differences in learning from gains versus losses, but only in patients with impulse control disorders. Furthermore, expected-value related brain sig­ nalling in the ventromedial prefrontal cortex was increased in patients with impulse control disorders ON medication compared with those OFF medication, while striatal reward prediction error signalling remained unaltered. These data substantiate the hypothesis that dopamine’s effects on reinforcement learning in Parkinson’s disease vary with individual differences in comorbid impulse control disorder and suggest they reflect deficient computation of va­ lue in medial frontal cortex, rather than deficient reward prediction error signalling in striatum. 1 Radboud University Medical Centre, Donders Institute for Brain, Cognition and Behaviour, Centre for Cognitive Neuroimaging, 6525EN Nijmegen, The Netherlands 2 Radboud University Medical Center, Department of Neurology, Centre of Expertise for Parkinson and Movement Disorders, 6525GA Nijmegen, The Netherlands 3 The Johns Hopkins University School of Medicine, Center for Psychedelic and Consciousness Research, Baltimore, MD 21224, USA 4 Radboud University Medical Center, Department of Psychiatry, 6525GA Nijmegen, The Netherlands Correspondence to: Roshan Cools Radboud University Medical Center, Donders Institute for Brain, Cognition and Behaviour Kapittelweg 29, 6525 EN Nijmegen, The Netherlands E-mail: [email protected] Keywords: Parkinson’s disease; dopamine; reinforcement learning; impulse control disorder; expected value Received December 06, 2022. Revised April 18, 2023. Accepted April 26, 2023. Advance access publication May 16, 2023 © The Author(s) 2023. Published by Oxford University Press on behalf of the Guarantors of Brain. This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial License (https://creativecommons.org/licenses/bync/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited. For commercial re-use, please contact [email protected] Downloaded from https://academic.oup.com/brain/article/146/9/3676/7165416 by guest on 09 April 2024 Jorryt G. Tichelaar,1,2 Heterogeneity in cognitive response to dopamine Introduction | 3677 suffered from depression.4 Moreover, patients with PD with depres­ sion exhibited greater medication-related decreases in loss aver­ sion during risky choice on a gambling task than patients without depression.44 These findings concur with clinical evidence indicat­ ing that PD patients with more severe depressive symptoms are at increased risk for developing medication-related ICD,7,40,44-46 pos­ sibly due to deficits in dopamine autoregulatory mechanisms in fronto-striatal circuitry. This interindividual variability in dopa­ minergic medication effects may explain why some studies have failed to reveal any (dopamine-related) deficits on RL when collaps­ ing data across all patients with PD.11,47-49 Here we aimed to more definitively resolve the clinical, cognitive and neural factors that contribute to this individual variability in cognitive medication ef­ fects by studying RL in the largest sample of patients with PD to date (n = 199). A second outstanding issue follows from the first: If there is such large interindividual variability in dopamine-dependent RL impair­ ments in PD, then it is less likely that these impairments reflect the core pathology of PD, i.e. impaired dorsal striatal signalling. Thus, while some prior work with small sample sizes has suggested that RPE signals during RL are impaired in dorsolateral striatum50 (but see Cools et al.15), we ask here in a much larger sample whether depres­ sion and ICD-related RL impairments reflect dopamine-dependent changes in other nodes of the cortical-striatal-thalamo-cortical re­ ward network, particularly the prefrontal cortex, which is more vari­ ably affected across different patients.51-54 Finally, a third outstanding question concerns the computational mechanisms underlying the deficient task performance: Does it re­ flect a change in learning or a change in motivational biases of choice? While various studies implicate a role for aberrant computa­ tion of canonical RPE learning signals in the striatum,10,14-22 there is also evidence for medication-related changes in reward-based choice that cannot be attributed to changes in learning.32,47,55 Here we investigate in a much larger sample of PD patients whether medication-related changes during RL are accompanied by changes in: (i) striatal RPE learning signals during outcomes; (ii) EV-related signals at the time of choice, more pervasive in the ventromedial prefrontal cortex (vmPFC); or (iii) both.20 As such, the study also con­ tributes to the longstanding debate about dopamine’s contribution to RPE-based learning versus value-based choice.47,55-57 Based on the canonical RPE hypothesis of striatal dopamine23 and prior empirical findings,10,15,29 we hypothesize to replicate pre­ vious findings that, relative to patients OFF medication, patients ON dopaminergic medication exhibit a shift from loss towards gain learning and that this behaviour is accompanied by increased RPE signalling during gain versus loss outcomes in the (ventral) stri­ atum.10,15,16,19,21,22 We will also test the (non-exclusive) alternative hypothesis, that effects on gain versus loss trials are associated with abnormal EV-related signals at the time of choice in the vmPFC. Furthermore, we expect these valence-specific medication group effects in RL to be particularly pronounced, and perhaps pre­ sent only in patients with PD with depression11 and/or ICD,10 be­ cause of their overly dynamic ventral reward circuitry.8 Materials and methods Ethics This study was approved by the local institutional review board (Commissie Mensgebonden Onderzoek Region Arnhem-Nijmegen; reference number 2016–2934; NL59694.091.17) and was conducted in accordance with the Ethical Principles for Medical Research Downloaded from https://academic.oup.com/brain/article/146/9/3676/7165416 by guest on 09 April 2024 Parkinson’s disease (PD) is the most rapidly growing neurodegen­ erative disease, expected to affect 12 million people worldwide in 2040.1 The cardinal motor symptoms are bradykinesia, rigidity and tremor, which are related to a severe loss of dopamine in the basal ganglia. However, the disease is also accompanied by various cognitive impairments,2-4 for example in learning, memory and/or attention. These cognitive impairments are known to contribute to neuropsychiatric symptoms such as depression and impulse con­ trol disorder (ICD). Approximately 14% of patients with PD are diag­ nosed with ICD, manifesting as pathological gambling, eating, shopping and/or hypersexuality.5 Depression is also common, oc­ curring in ∼35% of patients with PD.6 Interestingly, depression is a risk-factor for developing ICDs,7 and there is considerable co­ morbidity between these symptoms, suggesting that common me­ chanisms may play a role.8 Patients on dopamine receptor agonists are particularly vulnerable to developing ICDs,9 and dopaminergic medication can alleviate depressive symptoms. However, there are large interindividual differences in the sensitivity to dopamin­ ergic medication, while certainly not all patients with PD develop these severe side effects. The origins of this variability remain poorly understood, but likely implicate abnormal reward-related ac­ tivity in cortical-striatal-thalamo-cortical circuitry.8,10,11 Here we aimed to understand the cerebral mechanisms underlying this vari­ ability. To this end, we assessed the effect of dopaminergic medica­ tion on reinforcement learning (RL) in a large (n = 199) and heterogeneous sample of early-stage patients with PD (Personalized Parkinson Project, PPP), allowing us to compare patients with and without depression as well as those with and without ICDs. RL involves the gradual, incremental learning of associations, characteristic of the formation of habits12,13 and this process is well known to implicate reward prediction error (RPE) signalling in the dopamine-rich striatum.10,14-22 In line with the canonical striatal dopamine hypothesis of RL,23-25 many studies have revealed that even mildly affected patients with PD exhibit deficits on tasks that require RL,12,26-28 and these RL deficits depend on dopaminergic medication.29-31 Specifically, as predicted by influential Go/Nogo ba­ sal ganglia pathway models of RL,22,32 dopaminergic medication in patients with PD improves performance on tasks requiring learning from gains, while impairing performance on tasks requiring learning from losses.16,17,22,30,33-37 This pattern of effects has been replicated many times across different laboratories and has been proposed to contribute to depression, dopamine dysregulation syndrome and ICD in patients with PD.38-41 However, there are three key outstand­ ing questions regarding the RL impairment in PD. First, there are large individual differences in the degree to which patients with PD exhibit dopamine-dependent RL deficits. For example, separate studies have shown greater medicationrelated shifts towards gain versus loss learning in non-tremor patients with PD than in patients with tremor,42 in patients with versus without depression11 and in those with versus without ICD.10,43 For example, Voon et al.10 have shown that dopaminergic medication boosts gain learning, but only in patients with ICDs (i.e. pathological gamblers or shoppers). This medication effect in patients with ICDs was associated with abnormal striatal RPE sig­ nalling as well as abnormal expected value (EV)-related signalling in the frontal cortex.10 Furthermore, we have shown that patients with PD with higher medication doses exhibit greater impairments in probabilistic reversal learning than patients with lower medica­ tion doses, in line with the original dopamine overdose hypoth­ esis,36 but this effect was present only in patients who also BRAIN 2023: 146; 3676–3689 3678 | BRAIN 2023: 146; 3676–3689 J. G. Tichelaar et al. Involving Human Subjects, as defined in the Declaration of Helsinki (version amended in October 2013). All participants gave written in­ formed consent. General procedure Reinforcement learning task Participants completed three blocks of a probabilistic instrumental learning paradigm.18 The first block was a training session, per­ formed outside the scanner with a keyboard. The second and third blocks were performed inside the scanner with a fMRI-compatible button box. Each block consisted of 28 gain trials pseudo-randomly interleaved with 28 loss trials and took approximately 5 min to complete. Intertrial intervals during the training sessions were ran­ domly drawn from a normal distribution with a mean of 1 s. For the fMRI blocks, intertrial intervals were optimized using the optseq2 procedure,59 also with a mean of 1 s. For each block, two unique sets (one per trial type) of two ab­ stract visual cues were randomly selected from eight cue pairs (based on Pessiglione et al.18). The side of the screen at which each cue was presented was randomized across trials. Participants were instructed to maximize their monetary payoff and to make their decisions ‘within time.’ They were told to press a key corresponding to the side of the screen where they believed Table 1 Patient characteristics ON PD ICD+ OFF PD ICD− PD ICD+ HC PD ICD− Number of participants, n 36 96 12 46 60 Age, years 57.25 (10.2) 61.08 (9.08) 63.25 (7.07) 60.46 (9.68) 60 (9.61) Gender, female/male 14/22 55/40 2/10 23/23 27/33 Depression (BDI) 13.22 (8.01) 8.52 (6.28) 12.92 (6.82) 7.39 (4.73) 4.6 (3.51) ICD (QUIP-rs) 26 (10.13) 3.68 (4.27) 23.08 (11.9) 5.57 (5.07) 21.98 (10.13) Anxiety (STAI:trait) 42.06 (9.28) 34.73 (9.91) 42.75 (10.64) 32.87 (7.97) 33.76 (6.77) Disease severity (UPDRS:ON) 28.56 (12.16) 25.73 (11.02) 32.36 (13.43) 26.7 (13.62) – Disease severity (UPDRS:OFF) 32.28 (12.52) 31.39 (11.82) 38.08 (13.61) 33.15 (13.45) – LEDD 700.64 (480.96) 499.81 (237.45) 535.06 (237.57) 400.22 (267.79) – Dopamine receptor agonist use, yes/no 20/16 28/68 6/5 15/24 – Brixton 14.33 (7.35) 13.44 (6.17) 13.92 (6.53) 15.26 (5.9) – Semantic fluency 24.19 (6.26) 25.89 (5.13) 25.08 (6.37) 25.72 (6.64) – Symbols and digits 33.81 (9.02) 38.21 (7.83) 35.58 (6.3) 38.54 (5.4) – Months since diagnosis 27.53 (18.33) 24.37 (15.44) 34.17 (17.96) 32.52 (15.73) – Resting tremor 1.08 (1.34) 1.21 (1.67) 0.83 (1.59) 1.5 (2.13) – Main Main effect effect ICD MED – n.s. ** *** *** *** n.s. n.s. ** ** n.s. n.s. * n.s. n.s. – n.s. n.s. n.s. n.s. n.s. n.s. n.s. * n.s. n.s. n.s. n.s. * n.s. MED × ICD – n.s. n.s. n.s. n.s. n.s. n.s. n.s. n.s. n.s. n.s. n.s. n.s. n.s. n.s. We subdivided the Parkinson’s disease (PD) population based on impulse control disorder (ICD) by using a clinical cut-off score for the Questionnaire for Impulsive-Compulsive Disorders in Parkinson’s disease Rating Scale (QUIP-rs). For a subdivision for depression, see Supplementary Table 5. To compare groups, we used a 2 × 2 ANOVA (Med × ICD-group). We found no interaction effects between medication (MED) and ICD-group. For gender and dopamine agonist use, we performed a chi-square test for both ICD versus non-ICD and ON versus OFF medication. *P < 0.05, **P < 0.01, ***P < 0.001. n.s. = not significant. BDI = Beck Depression Inventory; HC = healthy controls; LEDD = levodopa equivalent daily dose; UPDRS = Unified Parkinson’s disease Rating Scale; STAI = Anxiety Inventory for Adults. Downloaded from https://academic.oup.com/brain/article/146/9/3676/7165416 by guest on 09 April 2024 The current study adopted a between-subject design. All PD pa­ tients were part of the PPP, which is a single-centre, longitudinal observational study in 520 PD patients.58 Inclusion criteria were a diagnosis of idiopathic PD, a disease duration of loss) on both RPE (11 949 voxels, MNI local maximum −16, −102, 2; TFCE = 94 489.01, Pfwe < 0.001; Fig. 5C) and EV-related signalling (624 voxels, MNI local maximum −2, 49, −2; TFCE = 822.49; Pfwe < 0.001; Fig. 5D), with stronger RPE and EV coding in the gain than the loss condition. When we did not consider the presence of ICD, there were no RPE- or EV-related BOLD signal differences between any of the groups (ON versus OFF, ON versus HC or OFF versus HC; Fig. 5), also not as a function of valence or in any of the ROIs. However, whole-brain analysis revealed a significant medication by valence effect, when stratifying the effects by ICD. Specifically, there was an ICD × Medication × Valence interaction on EV-related BOLD sig­ nal in the vmPFC, encompassing the subgenual and anterior cingu­ late cortex. This effect was substantiated by both whole-brain analyses (331 voxels, MNI local maximum 12, 49, 5; TFCE = 319.06; Pfwe < 0.05; Fig. 6A and for a subthreshold map see Supplementary Fig. 5) as well as ROI analyses (vmPFC, 11 voxels, MNI local max­ imum 8, 32, −5; TFCE = 12.83; Pfwe < 0.05; ROI based on meta-analysis20; Supplementary Fig. 2) and reflects a medication group-related shift towards greater EV-related signal on gain versus loss trials in patients with ICD (ON versus OFF medication) com­ pared with patients without ICD. This pattern of cerebral effects paralleled that of the behavioural effects, showing that, compared with ICD patients OFF medication, ICD patients ON medication were more accurate, faster and showed increased WSLS behaviour on gain versus loss trials compared with patients ON versus OFF medication without ICD. This observation was substantiated by a significant brain-behaviour correlation at the whole brain [r(200) = 0.28, P ≤ 0.001; Fig. 6] and the ROI level [r(200) = 0.21, P ≤ 0.001], so that those patients who exhibited greater EV-related signals for gain versus loss trials also exhibited greater accuracy on gain versus loss trials. In contrast to EV-related BOLD signal, RPE-related BOLD signal did not vary between ICD-groups. No clusters survived familywise error correction, at the whole-brain level, or within the ROI, based on a meta-analysis by Chase et al.20 (or within the smaller ROI de­ fined by Piray et al.68; Supplementary Fig. 3). There was also no Downloaded from https://academic.oup.com/brain/article/146/9/3676/7165416 by guest on 09 April 2024 Figure 5 BOLD response for reward prediction error and expected value across all participants. Main and valence effects for expected value (EV) and reward prediction error (RPE). The variables are calculated by a simple Rescorla–Wagner model and added as parametric regressor to cue (EV) and out­ come (RPE) onsets. The whole-brain images were adjusted for family-wise error (fwe) correction with Threshold Free Cluster Enhancement (TFCE) Pfwe < 0.05. Reported images are across all participants. Beta values extracted from all clusters above, plotted separately for PD-ON medication, PD-OFF medication and healthy controls (HC). (A and D) Main effect of RPE. (B and E) Main effect of EV. (C and F) Valence effect of RPE (gain minus loss trials). (D and G) Valence effect of EV (gain minus loss). MNI = Montreal Neurological Institute; PD = Parkinson’s disease. 3684 | BRAIN 2023: 146; 3676–3689 J. G. Tichelaar et al. interaction with valence and no correlations between beta values extracted from the ROI and measures of task performance (Fig. 6F). Discussion The present findings demonstrate canonical medication group ef­ fects on gain- versus loss-based decision making in PD patients with ICDs, but not those without ICDs. This suggests that patients with ICDs are disproportionately sensitive to dopaminergic medi­ cation effects on value-based decision making. Moreover, the re­ sults show that medicated patients with ICDs also exhibit increased EV signalling during gain versus loss trials in the vmPFC, rather than changes in RPE signalling in the striatum. Together, these findings establish key clinical, computational and neural factors that contribute to the large between-subject variabil­ ity in value-based decision making in PD. The finding that medication group effects on value-based choice are observed in only a subset of PD patients, namely those with ICDs, advances our understanding of both the role of dopamine in re­ inforcement learning and decision making (RLDM) as well as the me­ chanisms of ICD in PD. Specifically, this result resolves the discrepancy between, on the one hand, the classic observation that PD medication has asymmetric effects on gain versus loss learning, an effect that has become almost canonical due its cross-lab replication,16,17,22,29,30,33-37 and on the other hand, the recent nonreplications of these asymmetric effects.11,47-49 The latter nonreplications may reflect in part failure to consider key individual variability in the presence of comorbid psychiatric disorders that implicate, among other things, unstable mesolimbic dopamine transmission, as might be the case in ICDs. The finding also has im­ pact on our understanding of ICD mechanisms by supporting the pro­ posal that medication-related increases in the weight on gains versus losses during choice might contribute to the development and/or ex­ pression of ICDs in PD. Our neuroimaging results generally concur with the hypothesis that dopaminergic medication in PD ICD acts on mesolimbic reward circuitry including the vmPFC. Specifically, compared with those OFF medication, ICD patients ON medication Downloaded from https://academic.oup.com/brain/article/146/9/3676/7165416 by guest on 09 April 2024 Figure 6 BOLD response for reward prediction error and expected value per impulse control disorder group. (A) Medication induced a shift towards greater expected value (EV)-related signal on gain versus loss trials in impulse control disorder (ICD) patients versus non-ICD patients. Here the wholebrain analysis is depicted; see Supplementary material for region of interest (ROI). (B) Beta-values from the frontal cluster in A. (C) Brain behaviour cor­ relation; increased activity in the whole-brain ventral medial prefrontal cortex (vmPFC) cluster correlated with increased accuracy during gain trials compared with loss trials across groups. (D) Striatal clusters of the reward prediction error (RPE) ROI by Chase et al.20 (E) Beta values from the ROI, for the prediction error signal during gain minus loss trials. (F) Brain behaviour correlation; increased activity in the RPE ROI did not correlate with task-accuracy. Heterogeneity in cognitive response to dopamine | 3685 well as intact autoregulatory mechanisms, rendering value coding also insensitive to dopaminergic medication. Second, the absence of abnormal striatal RPE signalling might be explained by a disproportionate reliance on a working memory strategy commonly associated with the PFC rather than the incre­ mental RPE-based learning strategy that is associated with the stri­ atum. Our task comprised only two gain and two loss cues, thus requiring the working memory of only four cue values, which is well within most people’s working memory capacity. Indeed, it is es­ tablished that performance in especially the initial learning stage of RL tasks depends on cue set-size.95-99 Thus, it is possible that PD pa­ tients relied on working memory rather than an RL strategy when completing the current task. The hypothesis that the group effects observed here might reflect dopaminergic modulation of higherorder cognitive functioning like working memory is consistent with the locus of the medication group effect in the prefrontal cor­ tex. Indeed, dopamine (receptor stimulation100,101) in the prefrontal cortex has long been implicated in a wide variety of cognitive control functions, including not just working memory,102-104 but also other higher-order cognitive functions that might be argued to contribute to performance on the current task, including set-shifting,105,106 de­ layed reward discounting,107,108 temporal control,109-112 and effortbased decision making.113,114 Third, a 12-h washout period for dopaminergic medication might not be sufficient. Hence, it is possible that persistent medica­ tion effects masked aberrant RPE signalling in the ventral striatum (Supplementary material). A medication group difference in the vmPFC in PD ICD during value-based choice is reminiscent of findings from previous neuroi­ maging studies in PD ICD,115 showing aberrant prefrontal signalling during the evaluation of future reward and punishment,116,117 dur­ ing risk taking in gambling tasks118 and during speeded decision making in a Stroop task.119 More specifically, the present observa­ tion is remarkably consistent with prior work by Voon et al.,10 who demonstrated a shift towards gain learning away from loss learning in 14 patients with impulsive shopping and/or gambling. Voon et al.,10 also found PD patients with ICD to exhibit increased EV signalling in the vmPFC, as in our study. However, in contrast to our study, this was not modulated by medication. Instead, Voon et al.10 reported a medication-related increase in striatal RPE signalling during gain trials (Supplementary material). Similarly, Piray et al.43 reported that increased probabilistic gain versus loss learning in PD ICD was best accounted for by a model that assumed abnormal RPE-based learning of values. How can we account for this apparent discrepancy with the current study showing no RPE signal changes? One factor that might play a role in the discrepancy between these studies on PD ICD is disease duration. Average dis­ ease duration of the PD ICD group in the Piray et al.43 study was 9.6 years, whereas the average disease duration of our very early-PD cohort was only 2.5 years. It is thus possible that the pres­ ence of abnormal striatal RPE signalling in their studies, but not in the current study, reflects increased degeneration of neurons pro­ jecting to the ventral striatum due to longer disease duration in the Piray et al.43 study, also leading to deficient capacity of these neurons to exhibit phasic bursting. In sum, dopaminergic medication-related deficits in ventral striatal RPE signalling might be most readily seen in clinically more advanced PD patients with ICDs, who exhibit not only reduced phasic firing capacity of cells projecting to the ventral striatum, but also reduced ability to rely on a prefrontal working memory strategy. In contrast, ICDs in the more mildly affected PD patients studied here are accompanied by biased gating of action value representations in vmPFC, leading Downloaded from https://academic.oup.com/brain/article/146/9/3676/7165416 by guest on 09 April 2024 have boosted EV-related signalling in the vmPFC at the time of choice, for gain versus loss trials, and this effect correlated with the behav­ ioural effect of medication on gain versus loss trial accuracy. The medication group effect on the vmPFC in PD patients with ICD might reflect altered ventral striatal input to the vmPFC. This is suggested by evidence from PET and SPECT studies, demonstrat­ ing that patients at risk of ICDs exhibit reduced negative feedback control over dopamine release in the ventral striatum,69 reduced D2/3 receptor availability in the ventral striatum70 and reduced dopamine transporter availability in the ventral striatum, suggest­ ing reduced dopamine clearance from the synaptic cleft.71-73 Hypersensitivity of the mesolimbic reward system has previously been demonstrated, with greater dopaminergic medication74-76 and gambling task-related dopamine release74,76 and blood flow77 in the ventral striatum in PD ICD. Deficient autoregulation of dopa­ mine transmission in mesolimbic reward circuity might reflect gen­ etic variation associated with predisposing personality characteristics such as trait impulsivity78 and novelty seeking,40 or environmental factors like neuroinflammation.79,80 Thus, the impairment on value-based choice tasks in PD ICD might follow from an unstable, hyperdynamic mesolimbic dopamine system. The observation of medication group differences in EV signal at the time of choice concurs with prior findings that dopaminergic medication in PD acts by altering the expression of learning on choice rather than learning itself.32,47,55 The suggestion that learn­ ing itself was unaffected is supported by the absence of convincing evidence for modulation of RPE signals at the time of outcome in the striatum or elsewhere. This is in contrast to our primary predic­ tion, as well as to an extensive prior literature on RL model-based neuroimaging both in healthy volunteers,18,81-83 PD patients with­ out ICDs15,16,50 and those with ICDs.10 Given that PD medication might affect a tonic rather than only a phasic mode of dopamine transmission, these findings generally support a growing literature on a key role of tonic dopamine in the impact of action value on choice and motivation56,84-88 (but see Mikhael et al.89). In this con­ text, it is at least intriguing to note that ICDs are seen more fre­ quently in PD patients using dopamine receptor agonists,9 which simulate action of tonic dopamine, than in patients using levodopa, which also promotes the phasic release of dopamine.90-92 Clearly, hypotheses regarding the effects of agonist use require future work, in which patients with agonists are compared directly with patients on levodopa. While the lack of an effect of ICD on striatal RPE signalling might reflect selective abnormality in tonic dopamine transmission, we remain particularly puzzled about the lack of an effect of PD OFF versus controls on striatal RPE signalling (and on RL performance). This contrasts with prior PD studies revealing abnormal striatal RPE signalling in PD and is hard to reconcile with the observation that PD is characterized by severe degeneration of striatal dopamine cells, which must be associated with reduced phasic dopamine re­ lease. There may be several reasons for this absence of abnormal striatal RPE signalling. First, it might be that our PD patients were in relatively early stages of the disease (average disease duration = 2.5 years), where cells projecting to the relevant ventral striatum have not yet degen­ erated.93 Thus, while the ventral striatum of PD patients with ICDs might exhibit subtle (pre-existing) autoregulatory (presynaptic D2/ D3 receptor-related) problems and aberrant tonic dopamine levels, it is less likely that the cells themselves have already degenerated completely, leaving phasic dopamine transmission relatively intact (or even upregulated94). Thus, the ventral striatum of PD patients without ICDs might exhibit both intact phasic RPE signalling, as BRAIN 2023: 146; 3676–3689 3686 | BRAIN 2023: 146; 3676–3689 Acknowledgements We would like to acknowledge the Personalized Parkinson Project team for their work on data acquisition. Furthermore, we acknow­ ledge Marcel Zwiers, Martin Johansson and the ‘PEP’ team for their work on data maintenance. Lastly, we are very grateful to the kind people who participated in this study. Funding This study was supported by the Michael J. Fox Foundation for Parkinson’s Research (grant ID #15581), Verily Life Sciences and Health ∼ Holland. The Centre of Expertise for Parkinson & Movement Disorders was supported by a centre of excellence grant of the Parkinson’s Foundation. J.T. was supported by internal funds from the Radboudumc. R.C. was supported by an Ammodo award from the Royal Netherlands Academy of Arts and Sciences and a Vici award from the Dutch Research Council (Grant No. 453-14-015). R.H. was supported by a VIDI grant from the Dutch Research Council (Grant No. 09150172010044). Supplementary material Supplementary material is available at Brain online. References 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. Competing interests The authors report no competing interests. Dorsey ER, Sherer T, Okun MS, Bloem BR. The emerging evi­ dence of the Parkinson pandemic. J Parkinsons Dis. 2018;8:S3-S8. Narayanan NS, Rodnitzky RL, Uc EY. Prefrontal dopamine sig­ naling and cognitive symptoms of Parkinson’s disease. Rev Neurosci. 2013;24:267-278. Kehagia AA, Barker RA, Robbins TW. Cognitive impairment in Parkinson’s disease: the dual syndrome hypothesis. Neurodegen Dis. 2012;11:79-92. Cools R, Tichelaar JG, Helmich RCG, et al. Role of dopamine and clinical heterogeneity in cognitive dysfunction in Parkinson’s disease. Prog Brain Res. 2022;269:309-343. Weintraub D, Koester J, Potenza MN, et al. Impulse control dis­ orders in Parkinson disease: A cross-sectional study of 3090 patients. Arch Neurol. 2010;67:589-595. Aarsland D, Påhlhagen S, Ballard CG, Ehrt U, Svenningsson P. Depression in Parkinson disease—epidemiology, mechanisms and management. Nat Rev Neurol. 2012;8:35-47. Marín-Lahoz J, Sampedro F, Martinez-Horta S, Pagonabarraga J, Kulisevsky J. Depression as a risk factor for impulse control disorders in Parkinson disease. Ann Neurol. 2019;86: 762-769. Vriend C, Pattij T, van der Werf YD, et al. Depression and im­ pulse control disorders in Parkinson’s disease: two sides of the same coin? Neurosci Biobehav Rev. 2014;38:60-71. Weintraub D, Siderowf AD, Potenza MN, et al. Association of dopamine agonist use with impulse control disorders in Parkinson disease. Arch Neurol. 2006;63:969-973. Voon V, Pessiglione M, Brezing C, et al. Mechanisms underlying dopamine-mediated reward bias in compulsive behaviors. Neuron. 2010;65:135-142. Timmer MHM, Sescousse G, van der Schaaf ME, Esselink RAJ, Cools R. Reward learning deficits in Parkinson’s disease de­ pend on depression. Psychol Med. 2017;47:2302-2311. Knowlton BJ, Mangels JA, Squire LR. A neostriatal habit learn­ ing system in humans. Science. 1996;273:1399-1402. Redgrave P, Rodriguez M, Smith Y, et al. Goal-directed and ha­ bitual control in the basal ganglia: implications for Parkinson’s disease. Nat Rev Neurosci. 2010;11:760-772. Schönberg T, Daw ND, Joel D, O’Doherty JP. Reinforcement learning signals in the human striatum distinguish learners from nonlearners during reward-based decision making. J Neurosci. 2007;27:12860-12867. Cools R, Lewis SJ, Clark L, Barker RA, Robbins TW. L-DOPA disrupts activity in the nucleus accumbens during reversal learning in Parkinson’s disease. Neuropsychopharmacology. 2007;32:180-189. McCoy B, Jahfari S, Engels G, Knapen T, Theeuwes J. Dopaminergic medication reduces striatal sensitivity to nega­ tive outcomes in Parkinson’s disease. Brain. 2019;142:3605-3620. Schmidt L, Braun EK, Wager TD, Shohamy D. Mind matters: placebo enhances reward learning in Parkinson’s disease. Nat Neurosci. 2014;17:1793-1797. Pessiglione M, Seymour B, Flandin G, Dolan RJ, Frith CD. Dopamine-dependent prediction errors underpin rewardseeking behaviour in humans. Nature. 2006;442:1042-1045. van der Schaaf ME, van Schouwenburg MR, Geurts DE, et al. Establishing the dopamine dependency of human striatal sig­ nals during reward and punishment reversal learning. Cerebral cortex (New York, NY: 1991). 2014;24:633-642. Downloaded from https://academic.oup.com/brain/article/146/9/3676/7165416 by guest on 09 April 2024 to abnormal expression of learnt values on decision making, an anomaly compounded by their tendency to rely on a working mem­ ory strategy. One might wonder whether the medication group-related in­ crease in gain- versus loss-based choice in PD patients with ICDs versus non-ICDs reflects other factors of no interest that also dif­ fered incidentally between the patient groups. Supplementary ana­ lyses revealed that the effects of ICDs survived correction for a variety of potentially confounding regressors (Supplementary material). Nevertheless, this study did not involve a withinparticipant manipulation of medication. Such within-subject de­ signs control for potential confounding factors, but also come with challenges, such as relevant learning effects on cognitive tasks that may prevent a reliable comparison between sessions.42 We also did not randomize individuals between OFF or ON medication testing, although the assignment of individuals to either of these sessions was determined by enrollment order, which we assume is a random process. Taken together, while the current data strong­ ly suggest an association between the presence of ICDs and medication-related shifts towards gain- versus loss-based choice in early PD, future placebo-controlled cross-over medication with­ drawal studies are required to firmly establish a causal link be­ tween the consequences of dopaminergic medication for RL and the presence of ICDs. In contrast to our prediction that RL and decision making vary as a function of depression, our study did not reveal evidence for an interaction between depression and medication. However, we con­ sider the evidence for no effect of depression weak and refrain from concluding that medication effects on RL do not depend on depres­ sion, particularly given that the numerical pattern of effects of de­ pression resembles that of ICD (Supplementary material). These data substantiate the hypothesis that the effect of dopa­ mine on RL in PD varies with individual differences in comorbid ICDs. Specifically, our findings suggest that the presence of ICD is associated with deficient computation of value in medial frontal cortex, rather than deficient RPE signalling in striatum. J. G. Tichelaar et al. Heterogeneity in cognitive response to dopamine | 3687 39. Potenza MN, Voon V, Weintraub D. Drug insight: impulse con­ trol disorders and dopamine therapies in Parkinson’s disease. Nature Clinical Practice Neurology. 2007;3:664-672. 40. Dagher A, Robbins TW. Personality, addiction, dopamine: in­ sights from Parkinson’s disease. Neuron. 2009;61:502-510. 41. Cools R. Dopaminergic modulation of cognitive functionimplications for L-DOPA treatment in Parkinson’s disease. Neurosci Biobehav Rev. 2006;30:1-23. 42. van Nuland AJ, Helmich RC, Dirkx MF, et al. Effects of dopamine on reinforcement learning in Parkinson’s disease depend on motor phenotype. Brain. 2020;143:3422-3434. 43. Piray P, Zeighami Y, Bahrami F, Eissa AM, Hewedi DH, Moustafa AA. Impulse control disorders in Parkinson’s disease are associated with dysfunction in stimulus valuation but not action valuation. J Neurosci. 2014;34:7814-7824. 44. Timmer MHM, Sescousse G, Esselink RAJ, Piray P, Cools R. Mechanisms underlying dopamine-induced risky choice in Parkinson’s disease with and without depression (history). Comput Psychiatr. 2020;2:11-27. 45. Voon V, Thomsen T, Miyasaki JM, et al. Factors associated with dopaminergic drug-related pathological gambling in Parkinson disease. Arch Neurol. 2007;64:212-216. 46. Evans AH, Lawrence AD, Potts J, Appel S, Lees AJ. Factors influ­ encing susceptibility to compulsive dopaminergic drug use in Parkinson disease. Neurology. 2005;65:1570-1574. 47. Shiner T, Seymour B, Wunderlich K, et al. Dopamine and per­ formance in a reinforcement learning task: evidence from Parkinson’s disease. Brain. 2012;135(Pt 6):1871-1883. 48. Coulthard EJ, Bogacz R, Javed S, et al. Distinct roles of dopamine and subthalamic nucleus in learning and probabilistic deci­ sion making. Brain. 2012;135(Pt 12):3721-3734. 49. Grogan JP, Tsivos D, Smith L, et al. Effects of dopamine on re­ inforcement learning and consolidation in Parkinson’s dis­ ease. eLife. 2017;6:e26801. 50. Schonberg T, O’Doherty JP, Joel D, Inzelberg R, Segev Y, Daw ND. Selective impairment of prediction error signaling in hu­ man dorsolateral but not ventral striatum in Parkinson’s dis­ ease patients: evidence from a model-based fMRI study. NeuroImage. 2010;49:772-781. 51. Agid Y, Ruberg M, Javoy-Agid F, et al. Are dopaminergic neu­ rons selectively vulnerable to Parkinson’s disease? Adv Neurol. 1993;60:148-164. 52. Sawamoto N, Piccini P, Hotton G, Pavese N, Thielemans K, Brooks DJ. Cognitive deficits and striato-frontal dopamine release in Parkinson’s disease. Brain. 2008;131(Pt 5):1294-1302. 53. Kaasinen V, Nurmi E, Brück A, et al. Increased frontal [18F] fluorodopa uptake in early Parkinson’s disease: sex differences in the prefrontal cortex. Brain. 2001;124(Pt 6):1125-1130. 54. Rakshi JS, Uema T, Ito K, et al. Frontal, midbrain and striatal dopaminergic function in early and advanced Parkinson’s disease: a 3D [18F]dopa-PET study. Brain. 1999;122(Pt 9): 1637-1650. 55. Smittenaar P, Chase HW, Aarts E, Nusselein B, Bloem BR, Cools R. Decomposing effects of dopaminergic medication in Parkinson’s disease on probabilistic action selection–learning or performance? Eur J Neurosci. 2012;35:1144-1151. 56. Berridge KC. The debate over dopamine’s role in reward: the case for incentive salience. Psychopharmacology (Berl). 2007;191:391-431. 57. Robbins TW, Everitt BJ. A role for mesencephalic dopamine in activation: commentary on Berridge (2006). Psychopharmacology (Berl). 2007;191:433-437. 58. Bloem BR, Marks WJ, Silva de Lima AL, et al. The personalized Parkinson project: examining disease progression through broad biomarkers in early Parkinson’s disease. BMC Neurol. 2019;19:160. Downloaded from https://academic.oup.com/brain/article/146/9/3676/7165416 by guest on 09 April 2024 20. Chase HW, Kumar P, Eickhoff SB, Dombrovski AY. Reinforcement learning models and their neural correlates: an activation likelihood estimation meta-analysis. Cogn Affect Behav Neurosci. 2015;15:435-459. 21. Verharen JPH, Adan RAH, Vanderschuren L. Differential contribu­ tions of striatal dopamine D1 and D2 receptors to component pro­ cesses of value-based decision making. Neuropsychopharmacology. 2019;44:2195-2204. 22. Frank MJ. Dynamic dopamine modulation in the basal ganglia: a neurocomputational account of cognitive deficits in medi­ cated and nonmedicated parkinsonism. J Cogn Neurosci. 2005; 17:51-72. 23. Montague P, Dayan P, Sejnowski T. A framework for mesen­ cephalic dopamine systems based on predictive Hebbian learning. J Neurosci. 1996;16:1936-1947. 24. Schultz W, Dayan P, Montague PR. A neural substrate of pre­ diction and reward. Science. 1997;275:1593-1599. 25. Schultz W. Predictive reward signal of dopamine neurons. J Neurophysiol. 1998;80:1-27. 26. Swainson R, Rogers RD, Sahakian BJ, Summers BA, Polkey CE, Robbins TW. Probabilistic learning and reversal deficits in pa­ tients with Parkinson’s disease or frontal or temporal lobe le­ sions: possible adverse effects of dopaminergic medication. Neuropsychologia. 2000;38:596-612. 27. Shohamy D, Myers CE, Onlaor S, Gluck MA. Role of the basal ganglia in category learning: how do patients with Parkinson’s disease learn? Behav Neurosci. 2004;118:676-686. 28. Moody TD, Chang GY, Vanek ZF, Knowlton BJ. Concurrent dis­ crimination learning in Parkinson’s disease. Behav Neurosci. 2010;124:1-8. 29. Frank MJ, Seeberger LC, O’Reilly RC. By carrot or by stick: cog­ nitive reinforcement learning in parkinsonism. Science. 2004; 306:1940-1943. 30. Rutledge RB, Lazzaro SC, Lau B, Myers CE, Gluck MA, Glimcher PW. Dopaminergic drugs modulate learning rates and persev­ eration in Parkinson’s patients in a dynamic foraging task. J Neurosci. 2009;29:15104-15114. 31. Schott BH, Niehaus L, Wittmann BC, et al. Ageing and early-stage Parkinson’s disease affect separable neural mechanisms of me­ solimbic reward processing. Brain. 2007;130(Pt 9):2412-2424. 32. Collins AG, Frank MJ. Opponent actor learning (OpAL): mod­ eling interactive effects of striatal dopamine on reinforce­ ment learning and choice incentive. Psychol Rev. 2014;121: 337-366. 33. Bódi N, Kéri S, Nagy H, et al. Reward-learning and the noveltyseeking personality: a between- and within-subjects study of the effects of dopamine agonists on young Parkinson’s pa­ tients. Brain. 2009;132(Pt 9):2385-2395. 34. Frank MJ, Samanta J, Moustafa AA, Sherman SJ. Hold your horses: impulsivity, deep brain stimulation, and medication in parkinsonism. Science. 2007;318:1309-1312. 35. Graef S, Biele G, Krugel LK, et al. Differential influence of levo­ dopa on reward-based learning in Parkinson’s disease. Front Hum Neurosci. 2010;4:169. 36. Cools R, Altamirano L, D’Esposito M. Reversal learning in Parkinson’s disease depends on medication status and out­ come valence. Neuropsychologia. 2006;44:1663-1673. 37. Palminteri S, Lebreton M, Worbe Y, Grabli D, Hartmann A, Pessiglione M. Pharmacological modulation of subliminal learning in Parkinson’s and Tourette’s syndromes. Proc Natl Acad Sci U S A. 2009;106:19179-19184. 38. Andrew DL, Andrew HE, Andrew JL. Compulsive use of dopa­ mine replacement therapy in Parkinson’s disease: reward sys­ tems gone awry? The Lancet Neurology. 2003;2:595-604. BRAIN 2023: 146; 3676–3689 3688 | BRAIN 2023: 146; 3676–3689 81. Deserno L, Huys QJM, Boehme R, et al. Ventral striatal dopa­ mine reflects behavioral and neural signatures of model-based control during sequential decision making. Proc Natl Acad Sci U S A. 2015;112:1595-1600. 82. Schott BH, Minuzzi L, Krebs RM, et al. Mesolimbic functional magnetic resonance imaging activations during reward antici­ pation correlate with reward-related ventral striatal dopamine release. J Neurosci. 2008;28:14311-14319. 83. Jocham G, Klein TA, Ullsperger M. Differential modulation of reinforcement learning by D2 dopamine and NMDA glutamate receptor antagonism. J Neurosci. 2014;34:13151-13162. 84. Schultz W. Behavioral dopamine signals. Trends Neurosci. 2007; 30:203-210. 85. Niv Y, Daw ND, Joel D, Dayan P. Tonic dopamine: opportunity costs and the control of response vigor. Psychopharmacology (Berl). 2007;191:507-520. 86. Howe MW, Tierney PL, Sandberg SG, Phillips PEM, Graybiel AM. Prolonged dopamine signalling in striatum signals proximity and value of distant rewards. Nature. 2013;500:575-579. 87. Hamid AA, Pettibone JR, Mabrouk OS, et al. Mesolimbic dopa­ mine signals the value of work. Nat Neurosci. 2016;19:117-126. 88. Mohebi A, Pettibone JR, Hamid AA, et al. Dissociable dopamine dynamics for learning and motivation. Nature. 2019;570:65-70. 89. Mikhael JG, Kim HR, Uchida N, Gershman SJ. The role of state uncertainty in the dynamics of dopamine. Curr Biol. 2022;32: 1077-1087.e9. 90. Breitenstein C, Korsukewitz C, Flöel A, Kretzschmar T, Diederich K, Knecht S. Tonic dopaminergic stimulation impairs associative learning in healthy subjects. Neuropsychopharmacology. 2006;31:2552-2564. 91. Jaber M, Robinson SW, Missale C, Caron MG. Dopamine receptors and brain function. Neuropharmacology. 1996;35: 1503-1519. 92. Koller WC, Rueda MG. Mechanism of action of dopaminergic agents in Parkinson’s disease. Neurology. 1998;50(Suppl 6):S11-S14. 93. Kish SJ, Shannak K, Hornykiewicz O. Uneven pattern of dopa­ mine loss in the striatum of patients with idiopathic Parkinson’s disease. Pathophysiologic and clinical implica­ tions. N Engl J Med. 1988;318:876-880. 94. Grace AA. The tonic/phasic model of dopamine system regula­ tion and its implications for understanding alcohol and psy­ chostimulant craving. Addiction. 2000;95(Suppl 2):119–1128. 95. Collins AGE, Frank MJ. How much of reinforcement learning is working memory, not reinforcement learning? A behavioral, computational, and neurogenetic analysis. Eur J Neurosci. 2012;35:1024-1035. 96. Yoo AH, Collins AGE. How working memory and reinforce­ ment learning are intertwined: a cognitive, neural, and com­ putational perspective. J Cogn Neurosci. 2022;34:551-568. 97. Collins AGE. The tortoise and the hare: interactions between reinforcement learning and working memory. J Cogn Neurosci. 2018;30:1422-1432. 98. Collins AGE, Frank MJ. Within- and across-trial dynamics of human EEG reveal cooperative interplay between reinforce­ ment learning and working memory. Proc Natl Acad Sci U S A. 2018;115:2502-2507. 99. Anne GEC, Matthew AA, James AW, James MG, Michael JF. Interactions among working memory, reinforcement learning, and effort in value-based choice: a new paradigm and selective deficits in schizophrenia. Biol Psychiatry. 2017;82:431-439. 100. Goldman-Rakic PS. The cortical dopamine system: Role in memory and cognition. In: Goldstein DS, Eisenhofer G, McCarty R, eds. Advances in pharmacology. Vol 42. Academic Press; 1997:707-711. Downloaded from https://academic.oup.com/brain/article/146/9/3676/7165416 by guest on 09 April 2024 59. Greve D. Optseq Home Page. Accessed 14 July 2022. https:// surfer.nmr.mgh.harvard.edu/optseq/ 60. R Core Team. R: a language and environment for statistical comput­ ing. R Foundation for Statistical Computing; 2019. 61. Bürkner P-C. brms: an R package for Bayesian multilevel mod­ els using Stan. J Stat Softw. 2017;80:1-28. 62. Esteban O, Markiewicz CJ, Blair RW, et al. fMRIPrep: a robust preprocessing pipeline for functional MRI. Nat Methods. 2019; 16:111-116. 63. O’Doherty JP, Hampton A, Kim H. Model-based fMRI and its ap­ plication to reward learning and decision making. Ann N Y Acad Sci. 2007;1104:35-53. 64. Garrison J, Erdeniz B, Done J. Prediction error in reinforcement learning: a meta-analysis of neuroimaging studies. Neurosci Biobehav Rev. 2013;37:1297-1310. 65. Wilson RC, Niv Y. Is model fitting necessary for model-based fMRI? PLoS Comput Biol. 2015;11:e1004237. 66. Smith SM, Nichols TE. Threshold-free cluster enhancement: ad­ dressing problems of smoothing, threshold dependence and lo­ calisation in cluster inference. NeuroImage. 2009;44:83-98. 67. Niv Y, Daniel R, Geana A, et al. Reinforcement learning in multidimensional environments relies on attention mechan­ isms. J Neurosci. 2015;35:8145. 68. Piray P, den Ouden HEM, van der Schaaf ME, Toni I, Cools R. Dopaminergic modulation of the functional ventrodorsal architecture of the human striatum. Cereb Cortex. 2015;27: 485-495. 69. Ray NJ, Miyasaki JM, Zurowski M, et al. Extrastriatal dopamin­ ergic abnormalities of DA homeostasis in Parkinson’s patients with medication-induced pathological gambling: a [11C] FLB-457 and PET study. Neurobiol Dis. 2012;48:519-525. 70. Payer DE, Guttman M, Kish SJ, et al. [¹¹C]-(+)-PHNO PET imaging of dopamine D2/3 receptors in Parkinson’s disease with im­ pulse control disorders. Mov Disord. 2015;30:160-166. 71. Vriend C, Raijmakers P, Veltman DJ, et al. Depressive symp­ toms in Parkinson’s disease are related to reduced [123I] FP-CIT binding in the caudate nucleus. J Neurol Neurosurg Psychiatry. 2014;85:159-164. 72. Cilia R, Ko JH, Cho SS, et al. Reduced dopamine transporter dens­ ity in the ventral striatum of patients with Parkinson’s disease and pathological gambling. Neurobiol Dis. 2010;39:98-104. 73. Smith KM, Xie SX, Weintraub D. Incident impulse control dis­ order symptoms and dopamine transporter imaging in Parkinson disease. J Neurol Neurosurg Psychiatry. 2016;87:864-870. 74. O’Sullivan SS, Wu K, Politis M, et al. Cue-induced striatal dopa­ mine release in Parkinson’s disease-associated impulsivecompulsive behaviours. Brain. 2011;134(Pt 4):969-978. 75. Evans AH, Pavese N, Lawrence AD, et al. Compulsive drug use linked to sensitized ventral striatal dopamine transmission. Ann Neurol. 2006;59:852-858. 76. Steeves TD, Miyasaki J, Zurowski M, et al. Increased striatal dopamine release in parkinsonian patients with pathological gambling: A [11C] raclopride PET study. Brain. 2009;132(Pt 5): 1376-1385. 77. Claassen DO, Stark AJ, Spears CA, et al. Mesocorticolimbic hemodynamic response in Parkinson’s disease patients with compulsive behaviors. Mov Disord. 2017;32:1574-1583. 78. Buckholtz JW, Treadway MT, Cowan RL, et al. Dopaminergic net­ work differences in human impulsivity. Science. 2010;329:532. 79. Hirsch EC, Hunot S. Neuroinflammation in Parkinson’s disease: a target for neuroprotection? Lancet Neurol. 2009;8:382-397. 80. Felger JC, Treadway MT. Inflammation effects on motivation and motor activity: role of dopamine. Neuropsychopharmacology. 2017; 42:216-241. J. G. Tichelaar et al. Heterogeneity in cognitive response to dopamine | 3689 111. Narayanan NS, Laubach M. Delay activity in rodent frontal cor­ tex during a simple reaction time task. J Neurophysiol. 2009;101: 2859-2871. 112. Parker K, Lamichhane D, Caetano M, Narayanan N. Executive dysfunction in Parkinson’s disease and timing deficits. J Neuropsychol. 2013;7:193-224. 113. Draper A, Koch RM, van der Meer JWM, et al. Effort but not re­ ward sensitivity is altered by acute sickness induced by experi­ mental endotoxemia in humans. Neuropsychopharmacology. 2018;43:1107-1118. 114. Floresco SB, Tse MTL, Ghods-Sharifi S. Dopaminergic and glutamatergic regulation of effort- and delaybased decision making. Neuropsychopharmacology. 2008;33: 1966-1979. 115. Marín-Lahoz J, Martinez-Horta S, Pagonabarraga J, et al. Predicting impulse control disorders in Parkinson disease through incentive biomarkers. Ann Neurol. 2022. 116. Cilia R, Cho SS, van Eimeren T, et al. Pathological gambling in patients with Parkinson’s disease is associated with fronto-striatal disconnection: a path modeling analysis. Mov Disord. 2011;26:225-233. 117. Drew DS, Muhammed K, Baig F, et al. Dopamine and reward hypersensitivity in Parkinson’s disease with impulse control disorder. Brain. 2020;143:2502-2518. 118. Reuter J, Raedler T, Rose M, Hand I, Gläscher J, Büchel C. Pathological gambling is linked to reduced activation of the mesolimbic reward system. Nat Neurosci. 2005;8:147-148. 119. Potenza MN, Leung HC, Blumberg HP, et al. An FMRI Stroop task study of ventromedial prefrontal cortical function in pathological gamblers. Am J Psychiatry. 2003;160: 1990-1994. Downloaded from https://academic.oup.com/brain/article/146/9/3676/7165416 by guest on 09 April 2024 101. Floresco S. Prefrontal dopamine and behavioral flexibility: shifting from an “inverted-U” toward a family of functions. Front Neurosci. 2013;7:62. 102. Cools R, D’Esposito M. Inverted-U-shaped dopamine actions on human working memory and cognitive control. Biol Psychiatry. 2011;69:e113-e125. 103. Floresco SB, Braaksma DN, Phillips AG. Thalamic–cortical–stri­ atal circuitry subserves working memory during delayed re­ sponding on a radial arm maze. J Neurosci. 1999;19:11061. 104. Goldman-Rakic PS, Castner SA, Svensson TH, Siever LJ, Williams GV. Targeting the dopamine D1 receptor in schizophrenia: Insights for cognitive dysfunction. Psychopharmacology (Berl). 2004;174:3-16. 105. Floresco SB, Magyar O, Ghods-Sharifi S, Vexelman C, Tse MTL. Multiple dopamine receptor subtypes in the medial prefrontal cortex of the rat regulate set-shifting. Neuropsychopharmacology. 2006;31:297-309. 106. Nagano-Saito A, Leyton M, Monchi O, Goldberg YK, He Y, Dagher A. Dopamine depletion impairs frontostriatal functional con­ nectivity during a set-shifting task. J Neurosci. 2008;28:3697. 107. Floresco SB, Magyar O. Mesocortical dopamine modulation of ex­ ecutive functions: beyond working memory. Psychopharmacology (Berl). 2006;188:567-585. 108. Kobayashi S, Schultz W. Influence of reward delays on re­ sponses of dopamine neurons. J Neurosci. 2008;28:7837. 109. Narayanan NS, Land BB, Solder JE, Deisseroth K, DiLeone RJ. Prefrontal D1 dopamine signaling is required for temporal con­ trol. Proc Natl Acad Sci USA. 2012;109:20726-20731. 110. Narayanan NS, Horst NK, Laubach M. Reversible inactivations of rat medial prefrontal cortex impair the ability to wait for a stimulus. Neuroscience. 2006;139:865-876. BRAIN 2023: 146; 3676–3689