An overview of reward prediction error and its links with dopamine
- 1 Sichuan University
* Author to whom correspondence should be addressed.
Abstract
Reward prediction error (RPE) refers to the discrepancy between expected and actual rewards received during an event, signaling the difference between what was predicted and what actually happened. Dopaminergic neuron encodes RPE signals in the brain, and is responsible for updating reward expectations and influencing decision-making processes. The relationship between RPE and dopamine has led to research in understanding reward-driven learning and its implications on cognition and behavior. In this review, I will provide an overview of the principles, and task models used to quantify RPE. I will also discuss about the neural mechanisms underlying RPE generation, with a particular focus on the role of dopamine in encoding and transmitting RPE signals and enhance learning. Overall, this review highlights the importance of RPE in reward processing, decision-making, memory and so on, emphasizing its connection to dopamine.
Keywords
reward prediction, dopamine, circuit, learning
[1]. Glimcher PW. 2011. Understanding dopamine and reinforcement learning: the dopamine reward prediction error hypothesis. Proceedings of the National Academy of Sciences of the United States of America 108 Suppl 3: 15647-54
[2]. Lerner TN, Holloway AL, Seiler JL. 2021. Dopamine, Updated: Reward Prediction Error and Beyond. Current opinion in neurobiology 67: 123-30
[3]. Montague PR, Dayan P, Sejnowski TJ. 1996. A framework for mesencephalic dopamine systems based on predictive Hebbian learning. The Journal of neuroscience : the official journal of the Society for Neuroscience 16: 1936-47
[4]. Schultz W. 2016a. Dopamine reward prediction-error signalling: a two-component response. Nature reviews. Neuroscience 17: 183-95
[5]. Schultz W. 2016b. Dopamine reward prediction error coding. Dialogues in clinical neuroscience 18: 23-32
[6]. Geugies H, Groenewold NA, Meurs M, Doornbos B, de Klerk-Sluis JM, et al. 2022. Decreased reward circuit connectivity during reward anticipation in major depression. NeuroImage. Clinical 36: 103226
[7]. Lammel S, Lim BK, Malenka RC. 2014. Reward and aversion in a heterogeneous midbrain dopamine system. Neuropharmacology 76 Pt B: 351-9
[8]. Berridge KC. 2007. The debate over dopamine’s role in reward: the case for incentive salience. Psychopharmacology 191: 391-431
[9]. Fouragnan E, Retzler C, Philiastides MG. 2018. Separate neural representations of prediction error valence and surprise: Evidence from an fMRI meta-analysis. Human brain mapping 39: 2887-906
[10]. Keiflin R, Janak PH. 2015. Dopamine Prediction Errors in Reward Learning and Addiction: From Theory to Neural Circuitry. Neuron 88: 247-63
[11]. Liebenow B, Jones R, DiMarco E, Trattner JD, Humphries J, et al. 2022. Computational reinforcement learning, reward (and punishment), and dopamine in psychiatric disorders. Frontiers in psychiatry 13: 886297
[12]. Pfabigan DM, Alexopoulos J, Bauer H, Sailer U. 2011. Manipulation of feedback expectancy and valence induces negative and positive reward prediction error signals manifest in event-related brain potentials. Psychophysiology 48: 656-64
[13]. Noritake A, Nakamura K. 2019. Encoding prediction signals during appetitive and aversive Pavlovian conditioning in the primate lateral hypothalamus. Journal of neurophysiology 121: 396-417
[14]. O’Callaghan G, Stringaris A. 2019. Reward Processing in Adolescent Depression Across Neuroimaging Modalities. Zeitschrift fur Kinder- und Jugendpsychiatrie und Psychotherapie 47: 535-41
[15]. Matsumoto M, Hikosaka O. 2009. Two types of dopamine neuron distinctly convey positive and negative motivational signals. Nature 459: 837-41
[16]. Eshel N, Bukwich M, Rao V, Hemmelder V, Tian J, Uchida N. 2015. Arithmetic and local circuitry underlying dopamine prediction errors. Nature 525: 243-6
[17]. Rothenhoefer KM, Hong T, Alikaya A, Stauffer WR. 2021. Rare rewards amplify dopamine responses. Nature neuroscience 24: 465-69
[18]. Watabe-Uchida M, Eshel N, Uchida N. 2017. Neural Circuitry of Reward Prediction Error. Annual review of neuroscience 40: 373-94
[19]. Schultz W, Dayan P, Montague PR. 1997. A neural substrate of prediction and reward. Science (New York, N.Y.) 275: 1593-9
[20]. Schultz W. 1998. Predictive reward signal of dopamine neurons. Journal of neurophysiology 80: 1-27
[21]. Solié C, Girard B, Righetti B, Tapparel M, Bellone C. 2022. VTA dopamine neuron activity encodes social interaction and promotes reinforcement learning through social prediction error. Nature neuroscience 25: 86-97
[22]. Zaghloul KA, Blanco JA, Weidemann CT, McGill K, Jaggi JL, et al. 2009. Human substantia nigra neurons encode unexpected financial rewards. Science (New York, N.Y.) 323: 1496-9
[23]. Chang CY, Esber GR, Marrero-Garcia Y, Yau HJ, Bonci A, Schoenbaum G. 2016. Brief optogenetic inhibition of dopamine neurons mimics endogenous negative reward prediction errors. Nature neuroscience 19: 111-6
[24]. Steinberg EE, Keiflin R, Boivin JR, Witten IB, Deisseroth K, Janak PH. 2013. A causal link between prediction errors, dopamine neurons and learning. Nature neuroscience 16: 966-73
[25]. Gerfen CR, Surmeier DJ. 2011. Modulation of striatal projection systems by dopamine. Annual review of neuroscience 34: 441-66
[26]. Pan WX, Coddington LT, Dudman JT. 2021. Dissociable contributions of phasic dopamine activity to reward and prediction. Cell reports 36: 109684
[27]. García-García I, Zeighami Y, Dagher A. 2017. Reward Prediction Errors in Drug Addiction and Parkinson’s Disease: from Neurophysiology to Neuroimaging. Current neurology and neuroscience reports 17: 46
[28]. Papalini S, Beckers T, Vervliet B. 2020. Dopamine: from prediction error to psychotherapy. Translational Psychiatry 10: 164
[29]. Jang AI, Nassar MR, Dillon DG, Frank MJ. 2019. Positive reward prediction errors during decision-making strengthen memory encoding. Nature Human Behaviour 3: 719-32
[30]. Mohebi A, Pettibone JR, Hamid AA, Wong JT, Vinson LT, et al. 2019. Dissociable dopamine dynamics for learning and motivation. Nature 570: 65-70
[31]. Kawato M, Cortese A. 2021. From internal models toward metacognitive AI. Biological cybernetics 115: 415-30
Cite this article
Yao,T. (2024). An overview of reward prediction error and its links with dopamine. Theoretical and Natural Science,44,24-30.
Data availability
The datasets used and/or analyzed during the current study will be available from the authors upon reasonable request.
Disclaimer/Publisher's Note
The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of EWA Publishing and/or the editor(s). EWA Publishing and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
About volume
Volume title: Proceedings of the 2nd International Conference on Modern Medicine and Global Health
© 2024 by the author(s). Licensee EWA Publishing, Oxford, UK. This article is an open access article distributed under the terms and
conditions of the Creative Commons Attribution (CC BY) license. Authors who
publish this series agree to the following terms:
1. Authors retain copyright and grant the series right of first publication with the work simultaneously licensed under a Creative Commons
Attribution License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this
series.
2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the series's published
version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial
publication in this series.
3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and
during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See
Open access policy for details).