Optimistic and Pessimistic Dopamine Signals in the Human Brain: a Mapping and Modelling Study in Health and Parkinson's Disease

The dopamine system is tasked with motivating action and driving learning and thus lies at the core of adaptive behaviour. Whereas motivation of action has consistently been found to be impaired in Parkinson's disease (PD), less is known about the potential impairments of reward-based learning in PD.

In this project, investigators use reinforcement learning (RL) as a computational framework for understanding the role of dopamine in reward-based learning in health and in PD. RL can be defined as learning based on prior experiences. A core concept of RL is the reward prediction error (RPE). An RPE is the deviation between an obtained reward and an expected reward that an agent (e.g. a person) gets in a given environment. The RPE is used to update future expectations in similar situations. Thus, RPEs are critical for the agent's ability to adapt its behavior to the environment. These RPEs are signalled by dopamine neurons in the ventral tegmental area (VTA).

Recent advancements in the study of RL in mice have fundamentally changed our understanding of dopamine's role in reward-based learning. Dabney et al. (2020) discovered with single-cell recordings how dopamine signals in the VTA systematically differ in their RPE signals. They proposed a distributional RL (distRL) model where neurons can have diverse firing patterns. Neurons can be so-called "pessimistic" neurons which expect low rewards and can be positively surprised (as expressed by increased firing rate) by rewards that are even below-average, whereas "optimistic" neurons expect high rewards and can be negatively surprised (expressed by a decreased firing rate) even by above-average rewards. It remains to be examined whether distRL also applies to the dopamine system in the healthy human brain. Furthermore, if distRL does apply to the human brain, this framework might provide mechanistic insights into the cause of a range of different symptoms in PD.

The importance of dopamine is particularly prominent in PD, a progressive neurodegenerative disease, in which the progressive loss of dopamine neurons in the midbrain causes the classical motor symptoms (referred to as parkinsonism) and contributes to non-motor disturbances, such as apathy and cognitive slowing. PD is the second most frequent age-related neurodegenerative disorder and the global burden of PD has more than doubled over the last two decades mainly as a result of increasing numbers of older people.

Dopamine neurons have been suggested to be especially vulnerable to neurotoxicity. Interactive cascades of dopamine oxidation and mitochondrial stress due to aberrant calcium signaling are thought to be main causes of neurodegeneration. In this context, distRL and its representational implementation in dopaminergic cells leads to novel hypotheses: The pessimistic dopamine neurons (which are positively "surprised" by most outcomes and increase their firing rate) are exposed to higher oxidative mitochondrial stress over their lifetime than optimistic neurons and would thus be more prone to neurodegeneration than optimistic neurons. While the prominent motor symptoms are caused by massive degeneration in the SN, neurodegeneration in the less vulnerable VTA is less extensive, but still of relevant size. Consequently, already shortly after diagnosis, reward signaling in the VTA is significantly reduced. Thus, investigators of this project hypothesize that neurodegeneration in both the SN and VTA is biased towards "pessimistic" neurons. Consequently, for most reward outcomes in an environment, there are fewer neurons responding with dopamine release to an outcome (but the same number of neurons responding with a pause in firing). This results in a reduced overall dopamine response to most events, favouring anhedonia and apathy.

Investigators of this project will use functional magnetic resonance imaging (fMRI) and derive functional maps of optimism/pessimism. During the fMRI experiment, participants will be shown different stimuli ("cards") and, upon squeezing a grip-force device, they win or lose a small amount of money each time, accumulating money as they play. The different stimuli have different probabilities of leading to a reward or loss event which participants learn through observation. Some trials are forced-choice trials with only a single stimulus available, other trials are open-choice trials where participants get to choose between stimuli and can thus affect their accumulated earnings by choosing the stimuli that they have observed to be most beneficial.

The investigators will test the following main hypotheses:

1. As shown in mice, a diversity of "optimistic" and "pessimistic" dopamine reward signals exists in the human VTA and the ventro-rostral basal ganglia circuit. 2. "Pessimistic" neurons are more severely affected by neurodegeneration in PD. Furthermore, investigators will explore how potential shifts in the pessimism-optimism balance might be related to cognitive and motor symptoms in PD.

In a second study-arm, investigators will also explore whether/how medication state affects the distribution of optimistic and pessimistic prediction error signals. Here, patients with PD are tested one day in the off-medication state, another day in the on-medication state (order counterbalanced between patients with PD). The pragmatic off-medication state implies that patients will not have taken their morning dose of antiparkinsonian medication before arrival. Control participants are tested on two days without medication challenge to test for test-retest effects.

• Movement Disorder Society - Unified Parkinson's Disease Rating Scale (MDS-UPDRS) [Time frame: Day 1 and Day 2]
• Major Depression Inventory (MDI) [Time frame: Day 1 and Day 2]
• Montreal Cognitive Assessment (MoCA) [Time frame: PD patients: ON-medication (Day 1 or Day 2, counterbalanced). Healthy controls: Day 1.]
• Apathy Scale (AS) [Time frame: Day 1 and Day 2]
• Lille Apathy Rating Scale (LARS) [Time frame: Day 1 and Day 2]
• Edinburgh Handedness Inventory (EHI) [Time frame: PD patients: ON-medication (Day 1 or Day 2, counterbalanced). Healthy controls: Day 1.]
• Questionnaire for Impulsive-Compulsive Disorders in Parkinson's Disease - Rating Scale (QUIP-RS) [Time frame: Day 1 and Day 2]
• Snaith-Hamilton Pleasure Scale (SHAPS) [Time frame: Day 1 and Day 2]
• Temporal Experience of Pleasure Scale (TEPS) [Time frame: Day 1 and Day 2]
• Paired Associates Learning (PAL) from Cambridge Neuropsychological Testing Automated Battery (CANTAB), Total errors adjusted [Time frame: Day 1 and Day 2]
• Paired Associates Learning (PAL) from Cambridge Neuropsychological Testing Automated Battery (CANTAB), Mean errors to success [Time frame: Day 1 and Day 2]
• Paired Associates Learning (PAL) from Cambridge Neuropsychological Testing Automated Battery (CANTAB), First attempt memory score [Time frame: Day 1 and Day 2]