1
|
Marcus MM, Banks ML. Mechanistic and translational insights from preclinical cocaine choice procedures on the economic substitutability of cocaine and nondrug reinforcers. Neurosci Biobehav Rev 2025; 175:106217. [PMID: 40409443 DOI: 10.1016/j.neubiorev.2025.106217] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/27/2024] [Revised: 05/08/2025] [Accepted: 05/16/2025] [Indexed: 05/25/2025]
Abstract
Substance use disorders are increasingly being conceptualized as behavioral misallocation disorders; however, the neurobiological determinants of this behavioral misallocation are poorly understood. Cocaine use disorder (CUD) develops as a result of behavior being disproportionally directed towards the procurement and use of cocaine at the expense of behaviors maintained by more adaptive, nondrug reinforcers (i.e., job, family). Preclinical cocaine-vs-nondrug reinforcer choice procedures are uniquely positioned to 1) elucidate the biological mechanisms of drug and nondrug reinforcement and 2) inform the development of effective pharmacological and behavioral CUD therapies. Accordingly, this review addresses the existing preclinical literature regarding the economic substitutability and mesolimbic dopaminergic mechanisms underlying cocaine self-administration in the context of three different concurrently available nondrug reinforcers: food, social interaction, and electric foot shock (a negative reinforcer). The manuscript focuses on how the existing cocaine-vs-nondrug reinforcer choice literature guides future research directions to facilitate advances in understanding of CUD from both a neuroscience and translational research perspective.
Collapse
Affiliation(s)
- Madison M Marcus
- Virginia Commonwealth University, Institute for Drug and Alcohol Studies, USA; Virginia Commonwealth University, Institute for Women's Health, USA; Virginia Commonwealth University, Department of Pharmacology and Toxicology, USA.
| | - Matthew L Banks
- Virginia Commonwealth University, Department of Pharmacology and Toxicology, USA
| |
Collapse
|
2
|
Proietti R, Parr T, Tessari A, Friston K, Pezzulo G. Active inference and cognitive control: Balancing deliberation and habits through precision optimization. Phys Life Rev 2025; 54:27-51. [PMID: 40424850 DOI: 10.1016/j.plrev.2025.05.008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/06/2025] [Accepted: 05/16/2025] [Indexed: 05/29/2025]
Abstract
We advance a novel formulation of cognitive control within the active inference framework. The theory proposes that cognitive control amounts to optimising a precision parameter, which acts as a control signal and balances the contributions of deliberative and habitual components of action selection. To illustrate the theory, we simulate a driving scenario in which the driver follows a well-known route, but encounters unexpected challenges. Our simulations show that a standard active inference model can form adaptive habits; i.e., can pass from deliberative to habitual control when the context is stable, but generally fails to revert to deliberative control, when the context changes. To address this failure of context-sensitivity, we introduce a novel type of hierarchical active inference, in which a lower level is responsible for behavioural control and the higher (or meta-cognitive) level observes the belief updating of the lower level below and is responsible for cognitive control. Crucially, the meta-cognitive level can both form habits and suspend them, by controlling the (precision) parameter that prioritizes deliberative choices at the behavioural level. Furthermore, we show that several processes linked to cognitive control - such as surprise detection, cognitive conflict monitoring, control signal regulation and specification, the simulation of future outcomes and the assessment of the costs of control and mental effort - stem coherently from the free energy minimization scheme that underpins active inference. Finally, we discuss the putative neurobiology of cognitive control by simulating brain dynamics in the mesolimbic and mesocortical pathways of the dopamine system, the dorsal anterior cingulate cortex and the locus coeruleus.
Collapse
Affiliation(s)
- Riccardo Proietti
- Institute of Cognitive Sciences and Technologies, National Research Council, Via San Martino della Battaglia 44, Rome 00185, Italy
| | - Thomas Parr
- Nuffield Department of Clinical Neurosciences, University of Oxford, UK
| | - Alessia Tessari
- Department of Psychology, University of Bologna, Italy; Alma Mater Research Institute for Human-Centered Artificial Intelligence, University of Bologna, Bologna, Italy
| | - Karl Friston
- Wellcome Centre for Human Neuroimaging, Queen Square Institute of Neurology, University College London, London, UK; VERSES AI Research Lab, Los Angeles, CA 90016, USA
| | - Giovanni Pezzulo
- Institute of Cognitive Sciences and Technologies, National Research Council, Via San Martino della Battaglia 44, Rome 00185, Italy.
| |
Collapse
|
3
|
Kemecsei RG, Dániel-Papp S, Balazs DB, Ghebrihiwet Tewelde E, Csillag A, Zachar G. Disrupted functional connectome in a rodent model of autism during social isolation. Front Neural Circuits 2025; 19:1525130. [PMID: 40438462 PMCID: PMC12116437 DOI: 10.3389/fncir.2025.1525130] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/08/2024] [Accepted: 04/28/2025] [Indexed: 06/01/2025] Open
Abstract
Autism spectrum disorder (ASD) is associated with disruptions in social behavior and the neural circuitry behind it. Very little data is available on the mechanisms that are responsible for the lack of motivation to reunite with conspecifics during isolation. It is as important to investigate the neural changes that reduce motivation to end social isolation, as those underlying the reactions to social stimuli. Using a rodent model of prenatal valproic acid (VPA) exposure, we investigated how social isolation affects the neural activation of key brain nuclei involved in social processing and stress regulation. Juvenile male C57BL/6 mice were treated prenatally with VPA or saline (CTR) and subjected to 24 h of social isolation from their cage mates, with neural activity assessed via c-Fos immunohistochemistry. Based on correlational activations we reconstructed and analyzed the functional connectome of the observed brain regions. Control animals exhibited elevated c-Fos expression in the regions central to the mesolimbic reward system (MRS), social brain network (SBN), and stress-related networks, with the interpeduncular nucleus (IPN) at the core, compared to VPA-treated animals. Functional network analysis revealed a more widespread but less specific pattern of connectivity in VPA-treated animals. These findings suggest that prenatal VPA exposure disrupts certain neural circuits related to social behavior and stress regulation, offering an insight into the altered perception of social isolation in ASD models, and highlighting potential therapeutic targets.
Collapse
Affiliation(s)
- Robert Gergely Kemecsei
- Department of Anatomy, Histology, and Embryology, Faculty of Medicine, Semmelweis University, Budapest, Hungary
- HUN-REN Institute of Experimental Medicine, Budapest, Hungary
| | - Szizel Dániel-Papp
- Department of Anatomy, Histology, and Embryology, Faculty of Medicine, Semmelweis University, Budapest, Hungary
| | - David Barnabas Balazs
- Department of Anatomy, Histology, and Embryology, Faculty of Medicine, Semmelweis University, Budapest, Hungary
| | | | - Andras Csillag
- Department of Anatomy, Histology, and Embryology, Faculty of Medicine, Semmelweis University, Budapest, Hungary
| | - Gergely Zachar
- Department of Anatomy, Histology, and Embryology, Faculty of Medicine, Semmelweis University, Budapest, Hungary
| |
Collapse
|
4
|
Ballion B, Bonnet ML, Brot S, Gaillard A. Electrophysiological characterisation of intranigral-grafted hiPSC-derived dopaminergic neurons in a mouse model of Parkinson's disease. Stem Cell Res Ther 2025; 16:232. [PMID: 40346597 PMCID: PMC12065326 DOI: 10.1186/s13287-025-04344-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/17/2024] [Accepted: 04/15/2025] [Indexed: 05/11/2025] Open
Abstract
BACKGROUND Parkinson's disease (PD) is a complex neurological disorder characterized by the progressive degeneration of midbrain dopaminergic (mDA) neurons in the substantia nigra (SN). This degeneration disrupts the basal ganglia loops, leading to both motor and non-motor dysfunctions. Cell therapy for PD aims to replace lost mDA neurons to restore the DA neurotransmission in the denervated forebrain targets. In clinical trials for PD, mDA neurons are implanted into the target area, the striatum, and not in the SN where they are normally located. This ectopic localisation of cells may affect the functionality of transplanted neurons due to the absence of appropriate host afferent regulation. We recently demonstrated that human induced pluripotent stem cells (hiPSCs) derived mDA progenitors grafted into the substantia nigra pars compacta (SNpc) in a mouse model of PD, differentiated into mature mDA neurons, restored the degenerated nigrostriatal pathway, and induced motor recovery. The objective of the present study was to evaluate the long-term functionality of these intranigral-grafted mDA neurons by assessing their electrophysiological properties. METHODS We performed intranigral transplantation of hiPSC-derived mDA progenitors in a 6-hydroxydopamine RAG2-KO mouse model of PD. We recorded in vivo unit extracellular activity of grafted mDA neurons in anesthetised mice from 9 to 12 months post-transplantation. Their electrophysiological properties, including firing rates, patterns and spike characteristics, were analysed and compared with those of native nigral dopaminergic neurons from control mice. RESULTS We demonstrated that these grafted mDA neurons exhibited functional characteristics similar to those of native nigral dopaminergic neurons, such as large bi- or triphasic spike waveforms, low firing rates, pacemaker-like properties, and two single-spike firing patterns. Although grafted mDA neurons also displayed low discharge frequencies below 10 Hz, their mean frequency was significantly lower than that of nigral mDA neurons, with a differential pattern distribution. CONCLUSIONS Our findings indicate that grafted mDA neurons exhibit dopaminergic-like functional properties, including intrinsic membrane potential oscillations leading to regular firing patterns. Additionally, they demonstrated irregular and burst firing patterns, suggesting they receive modulatory inputs. However, grafted mDA neurons displayed distinct properties, potentially related to their human origin or the incomplete maturation one year after transplantation.
Collapse
Affiliation(s)
- Bérengère Ballion
- Laboratoire des neurosciences expérimentales et cliniques (LNEC), Université de Poitiers- INSERM 1084, Poitiers Cedex 9, 86073, France.
| | - Marie-Laure Bonnet
- Laboratoire des neurosciences expérimentales et cliniques (LNEC), Université de Poitiers- INSERM 1084, Poitiers Cedex 9, 86073, France
- Centre hospitalier universitaire (CHU) de Poitiers, Poitiers, 86021, France
| | - Sébastien Brot
- Laboratoire des neurosciences expérimentales et cliniques (LNEC), Université de Poitiers- INSERM 1084, Poitiers Cedex 9, 86073, France
| | - Afsaneh Gaillard
- Laboratoire des neurosciences expérimentales et cliniques (LNEC), Université de Poitiers- INSERM 1084, Poitiers Cedex 9, 86073, France.
| |
Collapse
|
5
|
Pfaus JG. Orgasms, sexual pleasure, and opioid reward mechanisms. Sex Med Rev 2025:qeaf023. [PMID: 40341995 DOI: 10.1093/sxmrev/qeaf023] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/31/2024] [Revised: 03/26/2025] [Accepted: 03/28/2025] [Indexed: 05/11/2025]
Abstract
INTRODUCTION Sexual activity produces pleasure related to sexual arousal, desire, and genitosensory and erogenous stimulation. Orgasms produce a whole brain and body rush of ecstatic pleasure followed by relaxation and refractoriness. This pleasure results from the activation of neurochemical reward pathways in the brain. This is differentiated by spinal pathways that control climax, the particular motor movements of the pelvic floor and the experience of tension release. OBJECTIVES To relate the activation of key neurochemical reward and bonding systems, notably dopamine, oxytocin, and opioids, to the pleasure of sexual activity in general and orgasms in particular. METHODS A narrative review of the neurochemical and neuroanatomical mechanisms activated during sexual stimulation and orgasm in rats and humans, and how they are related overall to the generation of sexual pleasure and reward. RESULTS Appetitive sexual pleasure involves the activation of dopamine and oxytocin release in hypothalamic and mesolimbic regions that regulate sexual arousal and desire, and are reinforced by localized opioid activity. Orgasms are thought to result in part from a massive release of opioids into these regions that inhibits dopamine and oxytocin transmission, but that initiates molecular changes to sensitize both systems and induce sexually conditioned place and partner preferences. Serotonin is also activated at orgasm and contributes to feelings of satiety and refractoriness. Orgasm disorders are distressing, cause resentment and conflict in a relationship, and diminish overall sexual health and well-being. CONCLUSIONS Orgasms are an important component of sexual pleasure for humans and perhaps all vertebrates. Endogenous opioids like β-endorphin that bind to mu opioid receptors are likely responsible for sexual pleasure and reward.
Collapse
Affiliation(s)
- James G Pfaus
- Center for Sexual Health and Interventions, Czech National Institute of Mental Health, Klecany 25067, Czech Republic
- Faculty of Humanities, Department of Psychology and Life Sciences, Charles University, Prague 18200, Czech Republic
| |
Collapse
|
6
|
Horne K, de Andrade Saraiva L, de Souza LC, Irish M. Social interaction as a unique form of reward - Insights from healthy ageing and frontotemporal dementia. Neurosci Biobehav Rev 2025; 172:106128. [PMID: 40157435 DOI: 10.1016/j.neubiorev.2025.106128] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/08/2024] [Revised: 02/27/2025] [Accepted: 03/25/2025] [Indexed: 04/01/2025]
Abstract
The drive for positive social interactions, or "social rewards", is an important motivator of human behaviour, conferring several adaptive benefits. Social motivation fluctuates across the lifespan, reflecting changes in goals and priorities at different developmental stages. In older adulthood, for instance, priorities tend to shift toward maintaining emotional wellbeing and resources over seeking novel gains. Contemporary theories of social interaction must account for such motivational shifts, addressing the enhancement of social processing in ageing and its decline in dementia. Here, we propose a framework to track the evolution of social motivation across the lifespan, focusing on three mechanisms: (i) social interactions as rewards, (ii) learning from social interactions, and (iii) the effort required for social interactions. We posit that social rewards hold equivalent or increased value later in life, enhancing older adults' social connections. Conversely, social rewards become devalued in neurodegenerative disorders such as frontotemporal dementia (FTD), resulting in social withdrawal. This integrative framework serves as a foundation for understanding adaptive and maladaptive trajectories of social motivation throughout the adult lifespan.
Collapse
Affiliation(s)
- Kristina Horne
- The University of Sydney, Brain and Mind Centre, Sydney, New South Wales, Australia; The University of Sydney, School of Psychology, Sydney, New South Wales, Australia
| | - Lucas de Andrade Saraiva
- Programa de Pós-Graduação em Neurociências, Universidade Federal de Minas Gerais, Belo Horizonte, Brazil
| | - Leonardo Cruz de Souza
- Programa de Pós-Graduação em Neurociências, Universidade Federal de Minas Gerais, Belo Horizonte, Brazil
| | - Muireann Irish
- The University of Sydney, Brain and Mind Centre, Sydney, New South Wales, Australia; The University of Sydney, School of Psychology, Sydney, New South Wales, Australia.
| |
Collapse
|
7
|
Zhang Z, Costa KM, Langdon AJ, Schoenbaum G. The devilish details affecting TDRL models in dopamine research. Trends Cogn Sci 2025; 29:434-447. [PMID: 40016003 PMCID: PMC12058390 DOI: 10.1016/j.tics.2025.02.001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/25/2024] [Revised: 01/28/2025] [Accepted: 02/03/2025] [Indexed: 03/01/2025]
Abstract
Over recent decades, temporal difference reinforcement learning (TDRL) models have successfully explained much dopamine (DA) activity. This success has invited heightened scrutiny of late, with many studies challenging the validity of TDRL models of DA function. Yet, when evaluating the validity of these models, the devil is truly in the details. TDRL is a broad class of algorithms sharing core ideas but differing greatly in implementation and predictions. Thus, it is important to identify the defining aspects of the TDRL framework being tested and to use state spaces and model architectures that capture the known complexity of the behavioral representations and neural systems involved. Here, we discuss several examples that illustrate the importance of these considerations.
Collapse
Affiliation(s)
- Zhewei Zhang
- National Institute on Drug Abuse Intramural Research Program, National Institutes of Health, Baltimore, MD 21224, USA.
| | - Kauê M Costa
- Department of Psychology, University of Alabama at Birmingham, Birmingham, AL 35233, USA
| | - Angela J Langdon
- National Institute of Mental Health Intramural Research Program, National Institutes of Health, Bethesda, MD 20892, USA
| | - Geoffrey Schoenbaum
- National Institute on Drug Abuse Intramural Research Program, National Institutes of Health, Baltimore, MD 21224, USA.
| |
Collapse
|
8
|
Klinger ME, Miller RA, Komatsu N, Shiu A, Wilbrecht L, Landry MP. Optical Fibers Functionalized with Single-Walled Carbon Nanotubes for Flexible Fluorescent Catecholamine Detection. LANGMUIR : THE ACS JOURNAL OF SURFACES AND COLLOIDS 2025; 41:9654-9663. [PMID: 40223211 DOI: 10.1021/acs.langmuir.4c04910] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/15/2025]
Abstract
Despite the popularity of drugs that act on catecholamine receptors, our knowledge of catecholamine dynamics in human health and disease remains incomplete. Recent advances in fluorescent sensors have enabled unprecedented access to catecholamine dynamics in preclinical animal models, but the requirements of these technologies to use in model organisms limit their translational value for clinical diagnostics. Here, we introduce proof of principle fluorescent catecholamine detection via optical fibers functionalized with single-walled carbon nanotube (SWNT)-based near-infrared catecholamine sensors (nIRCats), a catecholamine detection form factor that has potential for more convenient and less invasive clinical translation. We show that these near-infrared functionalized (nIRF) fibers respond to dopamine in a biologically relevant concentration range (10 nM through 1 μM), with minimal responsivity loss following 16 h exposure to human blood plasma. We further demonstrate the utility of these fibers in detecting dopamine from as little as 10 μL volumes of clinically relevant biofluids up to 24 weeks after fiber synthesis. We also introduce a compact, mobile dual-near-infrared fiber photometry rig and demonstrate its success detecting dopamine in acute brain slices with nIRF fibers. Together, this fiber-based dopamine detection tool and photometry rig expand the toolset for catecholamine detection.
Collapse
Affiliation(s)
- Madeline E Klinger
- Helen Wills Neuroscience Institute, University of California Berkeley, Berkeley, California 94720, United States
| | - Rigney A Miller
- College of Chemistry, University of California Berkeley, Berkeley, California 94720, United States
| | - Natsumi Komatsu
- Department of Chemical and Biomolecular Engineering, University of California Berkeley, Berkeley, California 94720, United States
| | - Amanda Shiu
- Department of Chemical and Biomolecular Engineering, University of California Berkeley, Berkeley, California 94720, United States
| | - Linda Wilbrecht
- Helen Wills Neuroscience Institute, University of California Berkeley, Berkeley, California 94720, United States
- Department of Psychology, University of California Berkeley, Berkeley, California 94720, United States
| | - Markita P Landry
- Helen Wills Neuroscience Institute, University of California Berkeley, Berkeley, California 94720, United States
- Department of Chemical and Biomolecular Engineering, University of California Berkeley, Berkeley, California 94720, United States
- Innovative Genomics Institute, Berkeley, California 94720, United States
- California Institute for Quantitative Biosciences, University of California Berkeley, Berkeley, California 94720, United States
- Chan Zuckerberg Biohub, San Francisco, California 94063, United States
| |
Collapse
|
9
|
Lander AC, Burge MT, Thomas BG, Phillips AJK, McGlashan EM, Cain SW. Circadian photoreception influences loss aversion. Sci Rep 2025; 15:13051. [PMID: 40240563 PMCID: PMC12003914 DOI: 10.1038/s41598-025-97370-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/30/2024] [Accepted: 04/03/2025] [Indexed: 04/18/2025] Open
Abstract
Gambling behaviour is a persistent and growing societal problem. An unexplored factor that may encourage gambling behaviour is the impact of circadian photoreception on cognitive processes underlying the behaviour. We investigated the influence of circadian photoreception on loss aversion in gambling by altering the blue content of light while maintaining the same visual brightness. Fifteen participants (age 18-27 years, M = 20.40, SD = 2.03) completed an economic decision-making task under blue-enriched and blue-depleted light, of equivalent visual brightness, on separate occasions in a randomised order. The task required participants to choose between taking a risky gamble of a positive and negative outcome, or a less risky guaranteed outcome. Hierarchical Bayesian Modelling was conducted to derive individual parameter estimates for loss aversion, and trial-by-trial performance was analysed using linear mixed models. The findings demonstrated that individuals were significantly less loss averse under blue-enriched light compared to blue-depleted light (β = - .43, 95% CI [- .82, - .04], p = .03). This study shows that exposure to light that preferentially targets circadian photoreception reduces loss aversion, which may encourage gambling behaviour.
Collapse
Affiliation(s)
- Alicia C Lander
- Flinders Health and Medical Research Institute (Sleep Health), Flinders University, Bedford Park, SA, 5042, Australia
- School of Psychological Sciences, Monash University, Melbourne, VIC, 3800, Australia
| | - Malisa T Burge
- School of Psychological Sciences, Monash University, Melbourne, VIC, 3800, Australia
| | - Brianna G Thomas
- School of Psychological Sciences, Monash University, Melbourne, VIC, 3800, Australia
| | - Andrew J K Phillips
- Flinders Health and Medical Research Institute (Sleep Health), Flinders University, Bedford Park, SA, 5042, Australia
| | - Elise M McGlashan
- School of Psychological Sciences, Monash University, Melbourne, VIC, 3800, Australia
- School of Psychological Sciences, University of Melbourne, Melbourne, VIC, 3800, Australia
| | - Sean W Cain
- Flinders Health and Medical Research Institute (Sleep Health), Flinders University, Bedford Park, SA, 5042, Australia.
| |
Collapse
|
10
|
Lupacchini L, Mollinari C, Tancredi V, Garaci E, Merlo D. Impaired synaptic transmission and long-term potentiation in severe combined immunodeficient (SCID) mice. Neuroreport 2025; 36:290-296. [PMID: 40177827 PMCID: PMC11949230 DOI: 10.1097/wnr.0000000000002149] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/11/2024] [Accepted: 02/10/2025] [Indexed: 04/05/2025]
Abstract
DNA-dependent protein kinase catalytic subunit (DNA-PKcs) is one of the key enzymes involved in DNA double-strand break (DSB) repair. However, recent studies using DNA-PKcs knockout mice revealed that DNA-PKcs plays an important role in neuronal plasticity. The aim of this study was to examine the role of DNA-PKcs on synaptic plasticity in severe combined immunodeficiency disease (SCID) mice that carry a mutation resulting in a DNA-PKcs protein devoid of kinase activity but still expressed in cells, although with a small COOH-terminal truncation. To this aim, we carried out electrophysiological and molecular analysis on hippocampal slices from wild-type (WT) and SCID mice. Electrophysiological analysis showed an impairment in the basal synaptic transmission in SCID mice compared with WT, whereas paired-pulse facilitation, caused by presynaptic mechanisms, was not different in the two groups of animals. By contrast, tetanic stimulation induced long-term potentiation (LTP) with values that were approximately 43% lower in slices from SCID mice compared with WT. The same slices used for electrophysiology were analyzed to study the phosphorylation state of cAMP response element-binding protein (CREB) and extracellular signal-regulated kinases and to evaluate mRNA expression levels of CREB-target genes at different times after LTP induction. In conclusion, molecular analysis did not show significant differences between SCID and WT brain slices, thus confirming the evidence that DNA-PKcs kinase activity directly regulates neuronal functions and plays a novel role beyond DSB repair. Moreover, these results indicate that studies using SCID mice involving analysis of synaptic function need to be interpreted with caution.
Collapse
Affiliation(s)
| | - Cristiana Mollinari
- Institute of Translational Pharmacology (IFT), National Research Council
- Department of Neuroscience, Istituto Superiore di Sanità
| | - Virginia Tancredi
- Department of Systems Medicine, ‘Tor Vergata’ University of Rome
- Centre of Space Bio-Medicine, ‘Tor Vergata’ University of Rome, Rome
| | | | - Daniela Merlo
- Department of Neuroscience, Istituto Superiore di Sanità
| |
Collapse
|
11
|
Gonzalez LS, Fisher AA, Grover KE, Robinson JE. Examining the role of the photopigment melanopsin in the striatal dopamine response to light. Front Syst Neurosci 2025; 19:1568878. [PMID: 40242043 PMCID: PMC12000111 DOI: 10.3389/fnsys.2025.1568878] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/30/2025] [Accepted: 03/17/2025] [Indexed: 04/18/2025] Open
Abstract
The mesolimbic dopamine system is a set of subcortical brain circuits that plays a key role in reward processing, reinforcement, associative learning, and behavioral responses to salient environmental events. In our previous studies of the dopaminergic response to salient visual stimuli, we observed that dopamine release in the lateral nucleus accumbens (LNAc) of mice encoded information about the rate and magnitude of rapid environmental luminance changes from darkness. Light-evoked dopamine responses were rate-dependent, robust to the time of testing or stimulus novelty, and required phototransduction by rod and cone opsins. However, it is unknown if these dopaminergic responses also involve non-visual opsins, such as melanopsin, the primary photopigment expressed by intrinsically photosensitive retinal ganglion cells (ipRGCs). In the current study, we evaluated the role of melanopsin in the dopaminergic response to light in the LNAc using the genetically encoded dopamine sensor dLight1 and fiber photometry. By measuring light-evoked dopamine responses across a broad irradiance and wavelength range in constitutive melanopsin (Opn4) knockout mice, we were able to provide new insights into the ability of non-visual opsins to regulate the mesolimbic dopamine response to visual stimuli.
Collapse
Affiliation(s)
- L. Sofia Gonzalez
- Division of Experimental Hematology and Cancer Biology, Department of Pediatrics, Cincinnati Children’s Hospital Medical Center, Cincinnati, OH, United States
- Department of Pediatrics, University of Cincinnati College of Medicine, Cincinnati, OH, United States
- Neuroscience Graduate Program, University of Cincinnati College of Medicine, Cincinnati, OH, United States
| | - Austen A. Fisher
- Division of Experimental Hematology and Cancer Biology, Department of Pediatrics, Cincinnati Children’s Hospital Medical Center, Cincinnati, OH, United States
- Department of Pediatrics, University of Cincinnati College of Medicine, Cincinnati, OH, United States
| | - Kassidy E. Grover
- Division of Experimental Hematology and Cancer Biology, Department of Pediatrics, Cincinnati Children’s Hospital Medical Center, Cincinnati, OH, United States
- Department of Pediatrics, University of Cincinnati College of Medicine, Cincinnati, OH, United States
- Neuroscience Graduate Program, University of Cincinnati College of Medicine, Cincinnati, OH, United States
| | - J. Elliott Robinson
- Division of Experimental Hematology and Cancer Biology, Department of Pediatrics, Cincinnati Children’s Hospital Medical Center, Cincinnati, OH, United States
- Department of Pediatrics, University of Cincinnati College of Medicine, Cincinnati, OH, United States
| |
Collapse
|
12
|
Caie B, Endres D, Khan AZ, Blohm G. Choice anticipation as gated accumulation of sensory predictions. J Neurophysiol 2025; 133:1159-1175. [PMID: 39970908 DOI: 10.1152/jn.00041.2024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/25/2024] [Revised: 03/06/2024] [Accepted: 02/12/2025] [Indexed: 02/21/2025] Open
Abstract
Predictions are combined with sensory information when making choices. Accumulator models have conceptualized predictions as trial-by-trial updates to a baseline evidence level. These models have been successful in explaining the influence of choice history across-trials, however, they do not account for how sensory information is transformed into choice evidence. Here, we derive a gated accumulator that models the onset of evidence accumulation as a combination of delayed sensory information and a prediction of sensory timing. To test how delays interact with predictions, we designed a free-choice saccade task where participants directed eye movements to either of two targets that appeared with variable delays and asynchronies. Despite instructions not to anticipate, participants responded before target onset on some trials. We reasoned that anticipatory responses reflected a trade-off between inhibiting and facilitating the onset of evidence accumulation via a gating mechanism as target appearance became more likely. We then found that anticipatory responses were more likely following repeated choices, suggesting that the balance between anticipatory and sensory responses was driven by a prediction of sensory timing. By fitting the gated accumulator model to the data, we found that variance in within-trial fluctuations in baseline evidence best explained the joint increase of anticipatory responses and faster sensory-guided responses with longer delays. Thus, we conclude that a prediction of sensory timing is involved in balancing the costs of anticipation with lowering the amount of accumulated evidence required to trigger saccadic choice.NEW & NOTEWORTHY Evidence accumulation models are used to study how recent history impacts the processes underlying how we make choices. Biophysical evidence suggests that the accumulation of evidence is gated, however, classic accumulator models do not account for this. In this work, we show that predictions of the timing of sensory information are important in controlling how evidence accumulation is gated and that signatures of these predictions can be detected even in randomized task environments.
Collapse
|
13
|
Skałbania J, Tanajewski Ł, Furtak M, Hare TA, Wypych M. Pre-choice midbrain fluctuations affect self-control in food choice: A functional magnetic resonance imaging (fMRI) study. COGNITIVE, AFFECTIVE & BEHAVIORAL NEUROSCIENCE 2025; 25:387-401. [PMID: 39379768 PMCID: PMC11906498 DOI: 10.3758/s13415-024-01231-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 09/19/2024] [Indexed: 10/10/2024]
Abstract
Recent studies have shown that spontaneous pre-stimulus fluctuations in brain activity affect higher-order cognitive processes, including risky decision-making, cognitive flexibility, and aesthetic judgments. However, there is currently no direct evidence to suggest that pre-choice activity influences value-based decisions that require self-control. We examined the impact of fluctuations in pre-choice activity in key regions of the reward system on self-control in food choice. In the functional magnetic resonance imaging (fMRI) scanner, 49 participants made 120 food choices that required self-control in high and low working memory load conditions. The task was designed to ensure that participants were cognitively engaged and not thinking about upcoming choices. We defined self-control success as choosing a food item that was healthier over one that was tastier. The brain regions of interest (ROIs) were the ventral tegmental area (VTA), putamen, nucleus accumbens (NAc), and caudate nucleus. For each participant and condition, we calculated the mean activity in the 3-s interval preceding the presentation of food stimuli in successful and failed self-control trials. These activities were then used as predictors of self-control success in a fixed-effects logistic regression model. The results indicate that increased pre-choice VTA activity was linked to a higher probability of self-control success in a subsequent food-choice task within the low-load condition, but not in the high-load condition. We posit that pre-choice fluctuations in VTA activity change the reference point for immediate (taste) reward evaluation, which may explain our finding. This suggests that the neural context of decisions may be a key factor influencing human behavior.
Collapse
Affiliation(s)
- Jakub Skałbania
- Faculty of Psychology, University of Warsaw, Warsaw, Poland
- Department of Economics, Kozminski University, Jagiellońska 57, 03-301, Warsaw, Poland
| | - Łukasz Tanajewski
- Department of Economics, Kozminski University, Jagiellońska 57, 03-301, Warsaw, Poland.
| | - Marcin Furtak
- Institute of Cognitive Science, University of Osnabrück, Osnabrück, Germany
| | - Todd A Hare
- Zurich Center for Neuroeconomics, Department of Economics, University of Zurich, Zurich, Switzerland
| | - Marek Wypych
- Laboratory of Brain Imaging, Nencki Institute of Experimental Biology, Polish Academy of Sciences, Warsaw, Poland
| |
Collapse
|
14
|
Luo S, Zhang S, Ji D, Jiang S, Wang X, Chen B, Chen Y, Pei X, Dai C, Jiang D, Liu W, Yang Y, Song E, Wei D, Kong D, Liu Y, Wei D. A Signal-Harmonizing Hybrid Neural Pathway Enabled by Bipolar-Chemo-Synapse Spiking Interneuron. J Am Chem Soc 2025; 147:10570-10578. [PMID: 40082394 DOI: 10.1021/jacs.5c00198] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/16/2025]
Abstract
To realize human-machine fusion, a hybrid neural pathway operating in the same modality with biological systems becomes imperative, which requires an interneuron unit to encode information in biorecognizable spike sequences and tune the frequency upon excitatory and inhibitory neurotransmitters. Existing artificial interneurons cannot encode information upon different neurotransmitters, and the activation threshold and frequency responsivity do not align with those of biological counterparts, leading to limited success in constructing a signal-harmonizing hybrid neural pathway for neuroprosthetics, neurorehabilitation, and other neuroelectronic applications. Herein, we develop a bipolar-chemosynapse interneuron to encode the spike frequency in a highly bionic paradigm. Bipolar synapses dynamically respond to excitatory and inhibitory neurotransmitters and translate time-series chemical signals into the spike sequence, achieving the lowest activation threshold (6.25 μM) and the highest frequency responsivity (0.55 Hz μM-1) to date, close to the biological counterpart. A signal-harmonizing hybrid sensorimotor pathway mediated by excitatory and inhibitory neurotransmitters is constructed for the first time, which encodes upstream mechanical stimuli, modulates the downstream leg swing frequency of a mouse, and balances neural activity accordingly.
Collapse
Affiliation(s)
- Shi Luo
- State Key Laboratory of Molecular Engineering of Polymers, Department of Macromolecular Science, Fudan University, Shanghai 200433, China
- Laboratory of Molecular Materials and Devices, Fudan University, Shanghai 200433, China
| | - Shen Zhang
- State Key Laboratory of Molecular Engineering of Polymers, Department of Macromolecular Science, Fudan University, Shanghai 200433, China
- Laboratory of Molecular Materials and Devices, Fudan University, Shanghai 200433, China
| | - Daizong Ji
- State Key Laboratory of Molecular Engineering of Polymers, Department of Macromolecular Science, Fudan University, Shanghai 200433, China
- Laboratory of Molecular Materials and Devices, Fudan University, Shanghai 200433, China
| | - Shuai Jiang
- Shanghai Key Laboratory of Vascular Lesions Regulation and Remodeling, Department of Vascular Surgery, Shanghai Pudong Hospital, Fudan University Pudong Medical Center, Shanghai 201399, China
| | - Xuejun Wang
- State Key Laboratory of Molecular Engineering of Polymers, Department of Macromolecular Science, Fudan University, Shanghai 200433, China
- Laboratory of Molecular Materials and Devices, Fudan University, Shanghai 200433, China
| | - Bo Chen
- Shanghai Key Laboratory of Vascular Lesions Regulation and Remodeling, Department of Vascular Surgery, Shanghai Pudong Hospital, Fudan University Pudong Medical Center, Shanghai 201399, China
| | - Yiheng Chen
- State Key Laboratory of Molecular Engineering of Polymers, Department of Macromolecular Science, Fudan University, Shanghai 200433, China
- Laboratory of Molecular Materials and Devices, Fudan University, Shanghai 200433, China
| | - Xinjie Pei
- State Key Laboratory of Molecular Engineering of Polymers, Department of Macromolecular Science, Fudan University, Shanghai 200433, China
- Laboratory of Molecular Materials and Devices, Fudan University, Shanghai 200433, China
| | - Changhao Dai
- State Key Laboratory of Molecular Engineering of Polymers, Department of Macromolecular Science, Fudan University, Shanghai 200433, China
- Laboratory of Molecular Materials and Devices, Fudan University, Shanghai 200433, China
| | - Dingding Jiang
- State Key Laboratory of Molecular Engineering of Polymers, Department of Macromolecular Science, Fudan University, Shanghai 200433, China
- Laboratory of Molecular Materials and Devices, Fudan University, Shanghai 200433, China
| | - Wentao Liu
- State Key Laboratory of Molecular Engineering of Polymers, Department of Macromolecular Science, Fudan University, Shanghai 200433, China
- Laboratory of Molecular Materials and Devices, Fudan University, Shanghai 200433, China
| | - Yuetong Yang
- State Key Laboratory of Molecular Engineering of Polymers, Department of Macromolecular Science, Fudan University, Shanghai 200433, China
- Laboratory of Molecular Materials and Devices, Fudan University, Shanghai 200433, China
| | - Enming Song
- Shanghai Frontiers Science Research Base of Intelligent Optoelectronics and Perception, Institute of Optoelectronics, Fudan University, Shanghai 200433, China
| | - Dapeng Wei
- Chongqing Institute of Green and Intelligent Technology, Chinese Academy of Sciences, Chongqing 400714, China
| | - Derong Kong
- State Key Laboratory of Molecular Engineering of Polymers, Department of Macromolecular Science, Fudan University, Shanghai 200433, China
- Laboratory of Molecular Materials and Devices, Fudan University, Shanghai 200433, China
| | - Yunqi Liu
- Laboratory of Molecular Materials and Devices, Fudan University, Shanghai 200433, China
| | - Dacheng Wei
- State Key Laboratory of Molecular Engineering of Polymers, Department of Macromolecular Science, Fudan University, Shanghai 200433, China
- Laboratory of Molecular Materials and Devices, Fudan University, Shanghai 200433, China
| |
Collapse
|
15
|
Noritake A, Isoda M. The macaque medial prefrontal cortex simultaneously represents self and others' reward prediction error. Cell Rep 2025; 44:115368. [PMID: 40022727 DOI: 10.1016/j.celrep.2025.115368] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/21/2024] [Revised: 12/26/2024] [Accepted: 02/07/2025] [Indexed: 03/04/2025] Open
Abstract
Learning the causal structures of social environments involves predicting significant events (e.g., rewards) and detecting prediction errors for each agent. Whether the brain can simultaneously compute reward prediction errors for self (S-RPE) and others (O-RPE), and which neurons are responsible, is unclear. Here, we condition two monkeys with identical visual stimuli predicting different reward outcomes and find that dorsomedial prefrontal neurons represent both S-RPE and O-RPE simultaneously. Neuronal signatures of RPE are agent and sign specific, forming distinct populations for positive and negative S-RPE and O-RPE. A linear decoder trained on neurons encoding O-RPE, but not S-RPE, successfully discriminates RPE. Further investigation identifies coexisting actual reward and prediction confirmation signals for others. These results highlight the presence of neuronal mechanisms in the primate brain that update the value of environmental stimuli simultaneously for oneself and others, enabling primates to comprehend the causal structure of the world from the perspective of others.
Collapse
Affiliation(s)
- Atsushi Noritake
- Division of Behavioral Development, Department of System Neuroscience, National Institute for Physiological Sciences, National Institutes of Natural Sciences, Okazaki, Aichi 444-8585, Japan; Department of Physiological Sciences, School of Life Science, The Graduate University for Advanced Studies, SOKENDAI, Hayama, Kanagawa 240-0193, Japan; Department of Physiology, Kansai Medical University School of Medicine, Hirakata, Osaka 573-1010, Japan
| | - Masaki Isoda
- Division of Behavioral Development, Department of System Neuroscience, National Institute for Physiological Sciences, National Institutes of Natural Sciences, Okazaki, Aichi 444-8585, Japan; Department of Physiological Sciences, School of Life Science, The Graduate University for Advanced Studies, SOKENDAI, Hayama, Kanagawa 240-0193, Japan; Department of Physiology, Kansai Medical University School of Medicine, Hirakata, Osaka 573-1010, Japan.
| |
Collapse
|
16
|
Korbisch CC, Ahmed AA. Reaching vigor tracks learned prediction error. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2025:2025.03.24.645035. [PMID: 40196502 PMCID: PMC11974846 DOI: 10.1101/2025.03.24.645035] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Indexed: 04/09/2025]
Abstract
Movement vigor across multiple modalities increases with reward, suggesting that the neural circuits that represent value influence the control of movement. Dopaminergic neuron (DAN) activity in the basal ganglia has been suggested as the potential mediator of this response. If DAN activity is the bridge between value and vigor, then vigor should track canonical mediators of this activity, namely reward expectation and reward prediction error. Here we ask if a similar time-locked response is present in vigor of reaching movements. We explore this link by leveraging the known phasic dopaminergic response to stochastic rewards, where activity is modulated by both reward expectation at cue and the prediction error at feedback. We used probabilistic rewards to create a reaching task rich in reward expectation, reward prediction error, and learning. In one experiment, target reward probabilities were explicitly stated, and in the other, were left unknown and to be learned by the participants. We included two stochastic rewards (probabilities 33% and 66%) and two deterministic ones (probabilities 100% and 0%). Outgoing peak velocity in both experiments increased with increasing reward expectation. Furthermore, we observed a short-latency response in the vigor of the ongoing movement, that tracked reward prediction error: either invigorating or enervating velocity consistent with the sign and magnitude of the error. Reaching kinematics also revealed the value-update process in a trial-to-trial fashion, similar to the effect of prediction error signals typical in dopamine-mediated striatal phasic activity. Lastly, reach vigor increased with reward history over trials, mirroring the motivational effects often linked to fluctuating dopamine levels. Taken together, our results demonstrate and exquisite link between known short-latency reward signals and the invigoration of both discrete and ongoing movements.
Collapse
Affiliation(s)
- Colin C Korbisch
- Department of Mechanical Engineering, University of Colorado Boulder
| | - Alaa A Ahmed
- Department of Mechanical Engineering, University of Colorado Boulder
- Biomedical Engineering Program, University of Colorado Boulder
| |
Collapse
|
17
|
Zhu T, Areshenkoff CN, De Brouwer AJ, Nashed JY, Flanagan JR, Gallivan JP. Contractions in human cerebellar-cortical manifold structure underlie motor reinforcement learning. J Neurosci 2025; 45:e2158242025. [PMID: 40101964 PMCID: PMC12044045 DOI: 10.1523/jneurosci.2158-24.2025] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/12/2024] [Revised: 02/12/2025] [Accepted: 03/06/2025] [Indexed: 03/20/2025] Open
Abstract
How the brain learns new motor commands through reinforcement involves distributed neural circuits beyond known frontal-striatal pathways, yet a comprehensive understanding of this broader neural architecture remains elusive. Here, using human functional MRI (N = 46, 27 females) and manifold learning techniques, we identified a low-dimensional neural space that captured the dynamic changes in whole-brain functional organization during a reward-based trajectory learning task. By quantifying participants' learning rates through an Actor-Critic model, we discovered that periods of accelerated learning were characterized by significant manifold contractions across multiple brain regions, including areas of limbic and hippocampal cortex, as well as the cerebellum. This contraction reflected enhanced network integration, with notably stronger connectivity between several of these regions and the sensorimotor cerebellum correlating with higher learning rates. These findings challenge the traditional view of the cerebellum as solely involved in error-based learning, supporting the emerging view that it coordinates with other brain regions during reinforcement learning.Significance Statement This study reveals how distributed brain systems, including the cerebellum and hippocampus, alter their functional connectivity to support motor learning through reinforcement. Using advanced manifold learning techniques on functional MRI data, we examined changes in regional connectivity during reward-based learning and their relationship to learning rate. For several brain regions, we found that periods of heightened learning were associated with increased cerebellar connectivity, suggesting a key role for the cerebellum in reward-based motor learning. These findings challenge the traditional view of the cerebellum as solely involved in supervised (error-based) learning and add to a growing rodent literature supporting a role for cerebellar circuits in reward-driven learning.
Collapse
Affiliation(s)
- Tianyao Zhu
- Center for Neuroscience Studies, Queen's University, Kingston, Ontario, Canada.
| | - Corson N Areshenkoff
- Center for Neuroscience Studies, Queen's University, Kingston, Ontario, Canada
- Department of Psychology, Queen's University, Kingston, Ontario, Canada
| | - Anouk J De Brouwer
- Center for Neuroscience Studies, Queen's University, Kingston, Ontario, Canada
| | - Joseph Y Nashed
- Center for Neuroscience Studies, Queen's University, Kingston, Ontario, Canada
| | - J Randall Flanagan
- Center for Neuroscience Studies, Queen's University, Kingston, Ontario, Canada
- Department of Psychology, Queen's University, Kingston, Ontario, Canada
| | - Jason P Gallivan
- Center for Neuroscience Studies, Queen's University, Kingston, Ontario, Canada.
- Department of Psychology, Queen's University, Kingston, Ontario, Canada
- Department of Biomedical and Molecular Sciences, Queen's University, Kingston, Ontario, Canada
| |
Collapse
|
18
|
Hill NM, Tripp HM, Wolpert DM, Malone LA, Bastian AJ. Age-dependent predictors of effective reinforcement motor learning across childhood. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2025:2024.07.09.602665. [PMID: 39026875 PMCID: PMC11257637 DOI: 10.1101/2024.07.09.602665] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/20/2024]
Abstract
Across development, children must learn motor skills such as eating with a spoon and drawing with a crayon. Reinforcement learning, driven by success and failure, is fundamental to such sensorimotor learning. It typically requires a child to explore movement options along a continuum (grip location on a crayon) and learn from probabilistic rewards (whether the crayon draws or breaks). Here, we studied the development of reinforcement motor learning using online motor tasks to engage children aged 3 to 17 years and adults (cross-sectional sample, N=385). Participants moved a cartoon penguin across a scene and were rewarded (animated cartoon clip) based on their final movement position. Learning followed a clear developmental trajectory when participants could choose to move anywhere along a continuum and the reward probability depended on the final movement position. Learning was incomplete or absent in 3 to 8-year-olds and gradually improved to adult-like levels by adolescence. A reinforcement learning model fit to each participant identified two age-dependent factors underlying improvement across development: an increasing amount of exploration after a failed movement and a decreasing level of motor noise. We predicted, and confirmed, that switching to discrete targets and deterministic reward would improve 3 to 8-year-olds' learning to adult-like levels by increasing exploration after failed movements. Overall, we show a robust developmental trajectory of reinforcement motor learning abilities under ecologically relevant conditions i.e., continuous movement options mapped to probabilistic reward. This learning may be limited by immature spatial processing and probabilistic reasoning abilities in young children and can be rescued by reducing task demands.
Collapse
Affiliation(s)
- Nayo M. Hill
- Kennedy Krieger Institute, Baltimore, MD
- Department of Neuroscience, Johns Hopkins School of Medicine, Baltimore, MD
| | | | - Daniel M. Wolpert
- Mortimer B. Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY
- Department of Neuroscience, Columbia University, New York, NY
- Kavli Institute for Brain Science, Columbia University, New York, NY
| | - Laura A. Malone
- Kennedy Krieger Institute, Baltimore, MD
- Department of Neurology, Johns Hopkins School of Medicine, Baltimore, MD
- Department of Physical Medicine and Rehabilitation, Johns Hopkins School of Medicine, Baltimore, MD
| | - Amy J. Bastian
- Kennedy Krieger Institute, Baltimore, MD
- Department of Neuroscience, Johns Hopkins School of Medicine, Baltimore, MD
- Department of Neurology, Johns Hopkins School of Medicine, Baltimore, MD
- Department of Physical Medicine and Rehabilitation, Johns Hopkins School of Medicine, Baltimore, MD
| |
Collapse
|
19
|
Makkeh A, Graetz M, Schneider AC, Ehrlich DA, Priesemann V, Wibral M. A general framework for interpretable neural learning based on local information-theoretic goal functions. Proc Natl Acad Sci U S A 2025; 122:e2408125122. [PMID: 40042906 PMCID: PMC11912414 DOI: 10.1073/pnas.2408125122] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/26/2024] [Accepted: 12/19/2024] [Indexed: 03/19/2025] Open
Abstract
Despite the impressive performance of biological and artificial networks, an intuitive understanding of how their local learning dynamics contribute to network-level task solutions remains a challenge to this date. Efforts to bring learning to a more local scale indeed lead to valuable insights, however, a general constructive approach to describe local learning goals that is both interpretable and adaptable across diverse tasks is still missing. We have previously formulated a local information processing goal that is highly adaptable and interpretable for a model neuron with compartmental structure. Building on recent advances in Partial Information Decomposition (PID), we here derive a corresponding parametric local learning rule, which allows us to introduce "infomorphic" neural networks. We demonstrate the versatility of these networks to perform tasks from supervised, unsupervised, and memory learning. By leveraging the interpretable nature of the PID framework, infomorphic networks represent a valuable tool to advance our understanding of the intricate structure of local learning.
Collapse
Affiliation(s)
- Abdullah Makkeh
- Department of Data-driven Analysis of Biological Networks, Göttingen Campus Institute for Dynamics of Biological Networks, University of Göttingen, Göttingen37077, Germany
- Complex Systems Theory, Max Planck Institute for Dynamics and Self-Organization, Göttingen37077, Germany
| | - Marcel Graetz
- Department of Data-driven Analysis of Biological Networks, Göttingen Campus Institute for Dynamics of Biological Networks, University of Göttingen, Göttingen37077, Germany
- Department of Chemistry and Applied Biosciences, ETH Zurich, Zurich8092, Switzerland
| | - Andreas C. Schneider
- Complex Systems Theory, Max Planck Institute for Dynamics and Self-Organization, Göttingen37077, Germany
- University of Göttingen, Göttingen37073, Germany
- Cluster of Excellence “Multiscale Bioimaging: from Molecular Machines to Networks of Excitable Cells” (MBExC), University of Göttingen, Göttingen37073, Germany
| | - David A. Ehrlich
- Department of Data-driven Analysis of Biological Networks, Göttingen Campus Institute for Dynamics of Biological Networks, University of Göttingen, Göttingen37077, Germany
- Complex Systems Theory, Max Planck Institute for Dynamics and Self-Organization, Göttingen37077, Germany
| | - Viola Priesemann
- Complex Systems Theory, Max Planck Institute for Dynamics and Self-Organization, Göttingen37077, Germany
- University of Göttingen, Göttingen37073, Germany
- Cluster of Excellence “Multiscale Bioimaging: from Molecular Machines to Networks of Excitable Cells” (MBExC), University of Göttingen, Göttingen37073, Germany
| | - Michael Wibral
- Department of Data-driven Analysis of Biological Networks, Göttingen Campus Institute for Dynamics of Biological Networks, University of Göttingen, Göttingen37077, Germany
| |
Collapse
|
20
|
Vendrell-Llopis N, Read J, Boggiano S, Hetzler B, Peitsinis Z, Stanley C, Visel M, Trauner D, Donthamsetti P, Carmena J, Lammel S, Isacoff EY. Dopamine D1 receptor activation in the striatum is sufficient to drive reinforcement of anteceding cortical patterns. Neuron 2025; 113:785-794.e9. [PMID: 39814009 PMCID: PMC11886928 DOI: 10.1016/j.neuron.2024.12.013] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2024] [Revised: 07/19/2024] [Accepted: 12/13/2024] [Indexed: 01/18/2025]
Abstract
Timed dopamine signals underlie reinforcement learning, favoring neural activity patterns that drive behaviors with positive outcomes. In the striatum, dopamine activates five dopamine receptors (D1R-D5R), which are differentially expressed in striatal neurons. However, the role of specific dopamine receptors in reinforcement is poorly understood. Using our cell-specific D1R photo-agonist, we find that D1R activation in D1-expressing neurons in the dorsomedial striatum is sufficient to reinforce preceding neural firing patterns in defined ensembles of layer 5 cortico-striatal neurons of the mouse motor cortex. The reinforcement is cumulative and time dependent, with an optimal effect when D1R activation follows the selected neural pattern after a short interval. Our results show that D1R activation in striatal neurons can selectively reinforce cortical activity patterns, independent of a behavioral outcome or a reward, crucially contributing to the fundamental mechanisms that support cognitive functions like learning, memory, and decision-making.
Collapse
Affiliation(s)
- Nuria Vendrell-Llopis
- Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, CA 94720, USA; Department of Electrical and Computer Engineering, University of Alabama at Birmingham, Birmingham, AL 35233, USA.
| | - Jonathan Read
- Department of Molecular and Cell Biology, University of California, Berkeley, Berkeley, CA 94720, USA
| | - Samantha Boggiano
- Department of Pharmacology, Vanderbilt University, Nashville, TN 37235, USA
| | - Belinda Hetzler
- Department of Chemistry, New York University, New York, NY 10003, USA
| | - Zisis Peitsinis
- Department of Chemistry, New York University, New York, NY 10003, USA
| | - Cherise Stanley
- Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, CA 94720, USA
| | - Meike Visel
- Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, CA 94720, USA
| | - Dirk Trauner
- Department of Chemistry, New York University, New York, NY 10003, USA; Department of Chemistry, University of Pennsylvania, Philadelphia, PA 19104, USA
| | | | - Jose Carmena
- Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, CA 94720, USA
| | - Stephan Lammel
- Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, CA 94720, USA; Department of Molecular and Cell Biology, University of California, Berkeley, Berkeley, CA 94720, USA
| | - Ehud Y Isacoff
- Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, CA 94720, USA; Department of Molecular and Cell Biology, University of California, Berkeley, Berkeley, CA 94720, USA; Department of Neuroscience, University of California, Berkeley, Berkeley, CA 94720, USA; Weill Neurohub, University of California, Berkeley, Berkeley, CA 94720, USA; Molecular Biophysics and Integrated BioImaging Division, Lawrence Berkeley National Laboratory, Berkeley, CA 94720, USA.
| |
Collapse
|
21
|
Loh MK, Hurh SJ, Bazzino P, Donka RM, Keinath AT, Roitman JD, Roitman MF. Dopamine activity encodes the changing valence of the same stimulus in conditioned taste aversion paradigms. eLife 2025; 13:RP103260. [PMID: 40042246 PMCID: PMC11882140 DOI: 10.7554/elife.103260] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/12/2025] Open
Abstract
Mesolimbic dopamine encoding of non-contingent rewards and reward-predictive cues has been well established. Considerable debate remains over how mesolimbic dopamine responds to aversion and in the context of aversive conditioning. Inconsistencies may arise from the use of aversive stimuli that are transduced along different neural paths relative to reward or the conflation of responses to avoidance and aversion. Here, we made intraoral infusions of sucrose and measured how dopamine and behavioral responses varied to the changing valence of sucrose. Pairing intraoral sucrose with malaise via injection of lithium chloride (LiCl) caused the development of a conditioned taste aversion (CTA), which rendered the typically rewarding taste of sucrose aversive upon subsequent re-exposure. Following CTA formation, intraoral sucrose suppressed the activity of ventral tegmental area dopamine neurons (VTADA) and nucleus accumbens (NAc) dopamine release. This pattern of dopamine signaling after CTA is similar to intraoral infusions of innately aversive quinine and contrasts with responses to sucrose when it was novel or not paired with LiCl. Dopamine responses were negatively correlated with behavioral reactivity to intraoral sucrose and predicted home cage sucrose preference. Further, dopamine responses scaled with the strength of the CTA, which was increased by repeated LiCl pairings and weakened through extinction. Thus, the findings demonstrate differential dopamine encoding of the same taste stimulus according to its valence, which is aligned to distinct behavioral responses.
Collapse
Affiliation(s)
- Maxine K Loh
- Division of Endocrinology, Diabetes and Metabolism, Department of Medicine, University of Illinois at ChicagoChicagoUnited States
- Department of Psychology, University of Illinois at ChicagoChicagoUnited States
| | - Samantha J Hurh
- Department of Psychology, University of Illinois at ChicagoChicagoUnited States
| | - Paula Bazzino
- Graduate Program in Neuroscience, University of Illinois at ChicagoChicagoUnited States
| | - Rachel M Donka
- Department of Psychology, University of Illinois at ChicagoChicagoUnited States
| | - Alexandra T Keinath
- Department of Psychology, University of Illinois at ChicagoChicagoUnited States
| | - Jamie D Roitman
- Department of Psychology, University of Illinois at ChicagoChicagoUnited States
- Graduate Program in Neuroscience, University of Illinois at ChicagoChicagoUnited States
| | - Mitchell F Roitman
- Department of Psychology, University of Illinois at ChicagoChicagoUnited States
- Graduate Program in Neuroscience, University of Illinois at ChicagoChicagoUnited States
| |
Collapse
|
22
|
Prévost ED, Ward LA, Alas D, Aimale G, Ikenberry S, Fox K, Pelletier J, Ly A, Ball J, Kilpatrick ZP, Price K, Polter AM, Root DH. Untangling dopamine and glutamate in the ventral tegmental area. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2025:2025.02.25.640201. [PMID: 40060543 PMCID: PMC11888473 DOI: 10.1101/2025.02.25.640201] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 03/15/2025]
Abstract
Ventral tegmental area (VTA) dopamine neurons are of great interest for their central roles in motivation, learning, and psychiatric disorders. While hypotheses of VTA dopamine neuron function posit a homogenous role in behavior (e.g., prediction error), they do not account for molecular heterogeneity. We find that glutamate-dopamine, nonglutamate-dopamine, and glutamate-only neurons are dissociable in their signaling of reward and aversion-related stimuli, prediction error, and electrical properties. In addition, glutamate-dopamine and nonglutamate-dopamine neurons differ in dopamine release dynamics. Aversion-related recordings of all dopamine neurons (not considering glutamate co-transmission) showed a mixed response that obscured dopamine subpopulation function. Within glutamate-dopamine neurons, glutamate and dopamine release had dissociable contributions toward reward and aversion-based learning and performance. Based on our results, we propose a new hypothesis on VTA dopamine neuron function: that dopamine neuron signaling patterns and their roles in motivated behavior depend on whether or not they co-transmit dopamine with glutamate.
Collapse
Affiliation(s)
- Emily D. Prévost
- Department of Psychology and Neuroscience, University of Colorado Boulder, 2860 Wilderness Place, Boulder, CO 80301
| | - Lucy A. Ward
- Department of Psychology and Neuroscience, University of Colorado Boulder, 2860 Wilderness Place, Boulder, CO 80301
| | - Daniel Alas
- Department of Psychology and Neuroscience, University of Colorado Boulder, 2860 Wilderness Place, Boulder, CO 80301
| | - Giulia Aimale
- Department of Psychology and Neuroscience, University of Colorado Boulder, 2860 Wilderness Place, Boulder, CO 80301
| | - Sara Ikenberry
- Department of Psychology and Neuroscience, University of Colorado Boulder, 2860 Wilderness Place, Boulder, CO 80301
| | - Katie Fox
- Department of Psychology and Neuroscience, University of Colorado Boulder, 2860 Wilderness Place, Boulder, CO 80301
| | - Julianne Pelletier
- Department of Psychology and Neuroscience, University of Colorado Boulder, 2860 Wilderness Place, Boulder, CO 80301
| | - Annie Ly
- Department of Psychology and Neuroscience, University of Colorado Boulder, 2860 Wilderness Place, Boulder, CO 80301
| | - Jayson Ball
- Department of Psychology and Neuroscience, University of Colorado Boulder, 2860 Wilderness Place, Boulder, CO 80301
| | - Zachary P. Kilpatrick
- Department of Applied Mathematics, University of Colorado Boulder, 1111 Engineering Center, Boulder, CO 80309
| | - Kailyn Price
- Department of Pharmacology & Physiology, George Washington University, Washington, D.C. 20052
| | - Abigail M. Polter
- Department of Pharmacology & Physiology, George Washington University, Washington, D.C. 20052
| | - David H. Root
- Department of Psychology and Neuroscience, University of Colorado Boulder, 2860 Wilderness Place, Boulder, CO 80301
| |
Collapse
|
23
|
Jiang Q, Bakhurin KI, Hughes RN, Lu B, Ruan S, Yin HH. GABAergic neurons in the ventral tegmental area represent and regulate force vectors. Cell Rep 2025; 44:115313. [PMID: 39937645 PMCID: PMC11997961 DOI: 10.1016/j.celrep.2025.115313] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/14/2024] [Revised: 12/06/2024] [Accepted: 01/23/2025] [Indexed: 02/14/2025] Open
Abstract
The ventral tegmental area (VTA), a midbrain region associated with motivated behaviors, contains mostly dopaminergic (DA) neurons and GABAergic (GABA) neurons. Previous work has suggested that VTA GABA neurons provide a reward prediction signal, which is used in computing a reward prediction error. In this study, by using in vivo electrophysiology and continuous quantification of force exertion in head-fixed mice, we identify distinct populations of VTA GABA neurons that exhibit precise force tuning independently of learning, reward prediction, and outcome valence. Their activity usually precedes force exertion, and selective optogenetic manipulations of these neurons systematically modulate force exertion without influencing reward prediction. Together, these findings show that VTA GABA neurons can continuously regulate force vectors during motivated behavior.
Collapse
Affiliation(s)
- Qiaochu Jiang
- Department of Psychology and Neuroscience, Duke University, Durham, NC 27708, USA
| | | | - Ryan N Hughes
- Department of Psychology and Neuroscience, Duke University, Durham, NC 27708, USA
| | - Bryan Lu
- Department of Psychology and Neuroscience, Duke University, Durham, NC 27708, USA
| | - Shaolin Ruan
- Department of Neurobiology, Duke University School of Medicine, Durham, NC 27708, USA
| | - Henry H Yin
- Department of Psychology and Neuroscience, Duke University, Durham, NC 27708, USA; Department of Neurobiology, Duke University School of Medicine, Durham, NC 27708, USA.
| |
Collapse
|
24
|
Pais RC, Goldani A, Hutchison J, Mazrouei A, Khavaninzadeh M, Molina LA, Sutherland RJ, Mohajerani MH. Assessing cognitive flexibility in mice using a custom-built touchscreen chamber. Front Behav Neurosci 2025; 19:1536458. [PMID: 40017733 PMCID: PMC11865062 DOI: 10.3389/fnbeh.2025.1536458] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2024] [Accepted: 01/29/2025] [Indexed: 03/01/2025] Open
Abstract
Automated touchscreen systems have become increasingly prevalent in rodent model screening. This technology has significantly enhanced cognitive and behavioral assessments in mice and has bridged the translational gap between basic research using rodent models and human clinical research. Our study introduces a custom-built touchscreen operant conditioning chamber powered by a Raspberry Pi and a commercially available computer tablet, which effectively addresses the significant cost barriers traditionally associated with this technology. In order to test our prototype, we decided to train C57BL/6 mice on a visual discrimination serial-reversal task, and both C57BL/6 and AppNL-G-Fstrain - an Alzheimer's Disease (AD) mouse model - on a new location discrimination serial-reversal task. The results demonstrated a clear progression toward asymptotic performance, particularly in the location discrimination task, which also revealed potential genotype-specific deficits, with AppNL-G-F mice displaying an increase in the average number of errors in the first reversal as well as in perseverative errors, compared to wild-type mice. These results validate the practical utility of our touchscreen apparatus and underline its potential to provide insights into the behavioral and cognitive markers of neurobiological disorders.
Collapse
Affiliation(s)
- Rui C. Pais
- Department of Neuroscience, Canadian Centre for Behavioural Neuroscience, University of Lethbridge, Lethbridge, AB, Canada
| | - Ali Goldani
- Department of Neuroscience, Canadian Centre for Behavioural Neuroscience, University of Lethbridge, Lethbridge, AB, Canada
| | - Jayden Hutchison
- Department of Neuroscience, Canadian Centre for Behavioural Neuroscience, University of Lethbridge, Lethbridge, AB, Canada
| | - Amirhossein Mazrouei
- Department of Neuroscience, Canadian Centre for Behavioural Neuroscience, University of Lethbridge, Lethbridge, AB, Canada
| | - Mostafa Khavaninzadeh
- Department of Neuroscience, Canadian Centre for Behavioural Neuroscience, University of Lethbridge, Lethbridge, AB, Canada
| | - Leonardo A. Molina
- Cumming School of Medicine Optogenetics Core Facility, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada
| | - Robert J. Sutherland
- Department of Neuroscience, Canadian Centre for Behavioural Neuroscience, University of Lethbridge, Lethbridge, AB, Canada
| | - Majid H. Mohajerani
- Department of Neuroscience, Canadian Centre for Behavioural Neuroscience, University of Lethbridge, Lethbridge, AB, Canada
- Department of Psychiatry, Douglas Hospital Research Centre, McGill University, Montréal, QC, Canada
| |
Collapse
|
25
|
Wilt M, Magnard R, Carnicella S, Vachez YM. Zona incerta: from Parkinson's disease to addiction. Front Neural Circuits 2025; 19:1537449. [PMID: 39991498 PMCID: PMC11839659 DOI: 10.3389/fncir.2025.1537449] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/30/2024] [Accepted: 01/27/2025] [Indexed: 02/25/2025] Open
Affiliation(s)
- Mylène Wilt
- Inserm, U1216, Univ. Grenoble Alpes, Grenoble Institut Neurosciences, Grenoble, France
| | - Robin Magnard
- Department of Psychological and Brain Sciences, Krieger School of Arts and Sciences, Johns Hopkins University, Baltimore, MD, United States
| | - Sebastien Carnicella
- Inserm, U1216, Univ. Grenoble Alpes, Grenoble Institut Neurosciences, Grenoble, France
| | - Yvan M. Vachez
- Inserm, U1216, Univ. Grenoble Alpes, Grenoble Institut Neurosciences, Grenoble, France
| |
Collapse
|
26
|
Luther L, Raugh IM, Strauss GP. Probabalistic reinforcement learning impairments predict negative symptom severity and risk for conversion in youth at clinical high-risk for psychosis. Psychol Med 2025; 55:e28. [PMID: 39909851 PMCID: PMC12017368 DOI: 10.1017/s0033291724003416] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/24/2024] [Revised: 11/20/2024] [Accepted: 12/01/2024] [Indexed: 02/07/2025]
Abstract
BACKGROUND Elucidation of transphasic mechanisms (i.e., mechanisms that occur across illness phases) underlying negative symptoms could inform early intervention and prevention efforts and additionally identify treatment targets that could be effective regardless of illness stage. This study examined whether a key reinforcement learning behavioral pattern characterized by reduced difficulty learning from rewards that have been found to underlie negative symptoms in those with a schizophrenia diagnosis also contributes to negative symptoms in those at clinical high-risk (CHR) for psychosis. METHODS CHR youth (n = 46) and 51 healthy controls (CN) completed an explicit reinforcement learning task with two phases. During the acquisition phase, participants learned to select between pairs of stimuli probabilistically reinforced with feedback indicating receipt of monetary gains or avoidance of losses. Following training, the transfer phase required participants to select between pairs of previously presented stimuli during the acquisition phase and novel stimuli without receiving feedback. These test phase pairings allowed for inferences about the contributions of prediction error and value representation mechanisms to reinforcement learning deficits. RESULTS In acquisition, CHR participants displayed impaired learning from gains specifically that were associated with greater negative symptom severity. Transfer performance indicated these acquisition deficits were largely driven by value representation deficits. In addition to negative symptoms, this profile of deficits was associated with a greater risk of conversion to psychosis and lower functioning. CONCLUSIONS Impairments in positive reinforcement learning, specifically effectively representing reward value, may be an important transphasic mechanism of negative symptoms and a marker of psychosis liability.
Collapse
Affiliation(s)
- Lauren Luther
- Department of Psychology, University of Alabama at Birmingham, Birmingham, AL, USA
- Department of Psychology, University of Georgia, Athens, GA, USA
| | - Ian M. Raugh
- Department of Psychology, University of Georgia, Athens, GA, USA
- Department of Psychiatry, Douglas Mental Health Institute, McGill University, Montréal, QC, Canada
| | | |
Collapse
|
27
|
Fry BR, Russell N, Fex V, Mo B, Pence N, Beatty JA, Manfredsson FP, Toth BA, Burgess CR, Gershman S, Johnson AW. Devaluing memories of reward: a case for dopamine. Commun Biol 2025; 8:161. [PMID: 39900665 PMCID: PMC11790953 DOI: 10.1038/s42003-024-07440-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/06/2024] [Accepted: 12/23/2024] [Indexed: 02/05/2025] Open
Abstract
Midbrain dopamine cells encode differences in predictive and expected value to support learning through reward prediction error. Recent findings have questioned whether reward prediction error can fully account for dopamine function and suggest a more complex role for dopamine in encoding detailed features of the reward environment. In this series of studies, we describe a novel role for dopamine in devaluing sensory features of reward. Mesencephalic dopamine cells activated during a mediated devaluation phase were later chemogenetically reactivated. This retrieval of the devalued reward memory elicited a reduction in the hedonic evaluation of sucrose reward. Through optogenetic and chemogenetic manipulations, we confirm dopamine cells are both sufficient and necessary for mediated devaluation, and retrieval of these memories reflected dopamine release in the nucleus accumbens. Consistent with our computational modeling data, our findings indicate a critical role for dopamine in encoding predictive representations of the sensory features of reinforcement. Overall, we elucidate a novel role for dopamine function in mediated devaluation and illuminate a more elaborate framework through which dopamine encodes reinforcement signals.
Collapse
Affiliation(s)
- Benjamin R Fry
- Department of Psychology, Michigan State University, East Lansing, MI, USA
- Department of Biobehavioral Health, The Pennsylvania State University, University Park, PA, USA
| | - Nicolette Russell
- Department of Psychology, Michigan State University, East Lansing, MI, USA
| | - Victoria Fex
- Lyman Briggs College, Michigan State University, East Lansing, MI, USA
| | - Bing Mo
- Department of Psychology, Michigan State University, East Lansing, MI, USA
| | - Nathan Pence
- Department of Psychology, Michigan State University, East Lansing, MI, USA
| | - Joseph A Beatty
- Department of Physiology, Michigan State University, East Lansing, MI, USA
- Neuroscience Program, Michigan State University, East Lansing, MI, USA
| | - Fredric P Manfredsson
- Department of Translational Neuroscience, Barrow Neurological Institute, Phoenix, AZ, USA
| | - Brandon A Toth
- Michigan Neuroscience Institute, University of Michigan, Ann Arbor, MI, USA
| | | | - Samuel Gershman
- Department of Psychology, Harvard University, Cambridge, MA, USA
| | - Alexander W Johnson
- Department of Psychology, Michigan State University, East Lansing, MI, USA.
- Neuroscience Program, Michigan State University, East Lansing, MI, USA.
| |
Collapse
|
28
|
Abozaid A, Scaramuzzo M, Patel H, Gerlai R. A Novel Food Delivery Method for Learning Studies Detects Significant Differences in Food Preference in Zebrafish. Zebrafish 2025; 22:20-29. [PMID: 39909480 DOI: 10.1089/zeb.2024.0176] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/07/2025] Open
Abstract
Despite decades of research with laboratory rodents, the mechanisms underlying learning and memory, and their impairment, are still not fully understood. The zebrafish is a newcomer in this research area, but it has shown great promise. Food is often employed as a reinforcer in learning tasks with rodents. However, for zebrafish, food has been a problematic reinforcer. Controlling timing and localization of its delivery is difficult. What food types zebrafish prefer is also rarely studied? Here, we describe a novel food delivery hardware and procedure. The apparatus is simple, cheap to manufacture, and easy to employ. Using this new method, we compare how zebrafish respond to three food types, artemia nauplii, crushed tropical fish flakes, and small zebrafish pellets. In binary choice tasks, we show that zebrafish spend significantly more time near the artemia delivery cylinder, swim closer to, and visit this cylinder more frequently compared to food cylinders delivering flakes or pellets, while responses to these latter two cylinders do not differ from each other. We conclude that the newly developed method allows the quantification of food preference in zebrafish, and that it will lead to the identification of highly rewarding food types for learning studies in this species.
Collapse
Affiliation(s)
- Amira Abozaid
- Department of Psychology, University of Toronto Mississauga, Mississauga, Canada
| | - Melissa Scaramuzzo
- Department of Psychology, University of Toronto Mississauga, Mississauga, Canada
| | - Hemali Patel
- Department of Psychology, University of Toronto Mississauga, Mississauga, Canada
| | - Robert Gerlai
- Department of Psychology, University of Toronto Mississauga, Mississauga, Canada
| |
Collapse
|
29
|
Anselme P. Unconscious will as a neurobehavioral mechanism against adversity. Neurosci Biobehav Rev 2025; 169:105985. [PMID: 39709153 DOI: 10.1016/j.neubiorev.2024.105985] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/31/2024] [Revised: 12/12/2024] [Accepted: 12/16/2024] [Indexed: 12/23/2024]
Abstract
Incentive salience theory both explains the directional component of motivation (in terms of cue attraction or "wanting") and its energetic component, as a function of the strength of cue attraction. This theory characterizes cue- and reward-triggered approach behavior. But it does not tell us how behavior can show enhanced vigor under reward uncertainty, when cues are inconsistent or resources hidden. Reinforcement theory is also ineffective in explaining enhanced vigor in case reward expectation is low or nil. This paper provides a neurobehavioral interpretation of effort in situations of adversity (which always include some uncertainty about outcomes) that is complementary to the attribution of incentive salience to environmental cues. It is argued that manageable environmental challenges activate an unconscious process of self-determination to achieve "wanted" actions. This unconscious process is referred to as incentive effort, which involves the hypothalamo-pituitary-adrenal (HPA) axis, noradrenaline, as well as striatal dopamine. Concretely, HPA-induced dopamine release would have the function to make effort-or effortful actions-"wanted" in a challenging context, in which the environmental cues are poorly predictive of reward-i.e., unattractive. Stress would only emerge in the presence of unmanageable challenges. It is hypothesized that incentive effort is the core psychological basis of will-and is, for this reason, termed "willing."
Collapse
Affiliation(s)
- Patrick Anselme
- Department of Biopsychology, Faculty of Psychology, Ruhr University Bochum, Universitätsstraße 150, Bochum 44801, Germany.
| |
Collapse
|
30
|
Mehr SR, Nakhaei B, Soleimani H, Madadlou SK, Abbasi Maleky A, Abbasi-Maleki S. Addiction and stress: Exploring the reward pathways in brain affected by different drugs. PROGRESS IN BRAIN RESEARCH 2025; 291:381-404. [PMID: 40222788 DOI: 10.1016/bs.pbr.2025.01.012] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/15/2025]
Abstract
This chapter delves into the complex interplay among addiction, stress, and the reward pathways in the brain, emphasizing the ways in which various drugs affect these systems and exacerbate SUD. Drugs have physiological effects that can be both pleasurable and unpleasant. These effects change behavior through both positive and negative reinforcement. A person's genetic predisposition to addiction is mostly determined by factors such as biological sex, age of first usage, and dopamine receptor density. Drug use behaviors are also greatly influenced by environmental stressors, media exposure, and substance accessibility; nevertheless, protective variables including social support, participation in healthy activities, and preventative programs serve to reduce the dangers associated with drug use. The reinforcement of addictive behaviors is mostly dependent on the brain's reward circuits, which include the nucleus accumbens, ventral tegmental region, and prefrontal cortex, in addition to neurotransmitters such as dopamine, serotonin, and endorphins. Stress makes addiction worse by intensifying cravings and raising the possibility of relapsing. Examined are the impacts of several drug types, such as opioids, stimulants, depressants, and hallucinogens, emphasizing the long-term consequences on brain function and susceptibility to addiction. In order to create individualized interventions that target the environmental and neurological components of addiction and eventually improve treatment results, a thorough understanding of these elements is important.
Collapse
Affiliation(s)
- Samira Rostami Mehr
- Pharmaceutical Sciences Research Center, Health Institute, Kermanshah University of Medical Sciences, Kermanshah, Iran
| | - Babak Nakhaei
- Department of Pathobiology, Faculty of Veterinary Medicine, Urmia Branch, Islamic Azad University, Urmia, Iran
| | - Hossein Soleimani
- Pharmaceutical Sciences Research Center, Health Institute, Kermanshah University of Medical Sciences, Kermanshah, Iran; Department of Pathobiology, Faculty of Veterinary Medicine, Urmia Branch, Islamic Azad University, Urmia, Iran; Research and Development Unit (R&D), Daana Pharmaceutical Co, Tabriz, Iran; Department of Biological Science, North Tehran Branch, Islamic Azad University, Tehran, Iran; Department of Pharmacology & Toxicology, School of Pharmacy, Kermanshah University of Medical Science, Kermanshah, Iran
| | | | - Asghar Abbasi Maleky
- Department of Biological Science, North Tehran Branch, Islamic Azad University, Tehran, Iran
| | - Saeid Abbasi-Maleki
- Pharmaceutical Sciences Research Center, Health Institute, Kermanshah University of Medical Sciences, Kermanshah, Iran; Department of Pharmacology & Toxicology, School of Pharmacy, Kermanshah University of Medical Science, Kermanshah, Iran.
| |
Collapse
|
31
|
Zhao R, Kim SJ, Xu Y, Zhao J, Wang T, Midya R, Ganguli S, Roy AK, Dubey M, Williams RS, Yang JJ. Memristive Ion Dynamics to Enable Biorealistic Computing. Chem Rev 2025; 125:745-785. [PMID: 39729346 PMCID: PMC11759055 DOI: 10.1021/acs.chemrev.4c00587] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/08/2024] [Revised: 12/10/2024] [Accepted: 12/16/2024] [Indexed: 12/28/2024]
Abstract
Conventional artificial intelligence (AI) systems are facing bottlenecks due to the fundamental mismatches between AI models, which rely on parallel, in-memory, and dynamic computation, and traditional transistors, which have been designed and optimized for sequential logic operations. This calls for the development of novel computing units beyond transistors. Inspired by the high efficiency and adaptability of biological neural networks, computing systems mimicking the capabilities of biological structures are gaining more attention. Ion-based memristive devices (IMDs), owing to the intrinsic functional similarities to their biological counterparts, hold significant promise for implementing emerging neuromorphic learning and computing algorithms. In this article, we review the fundamental mechanisms of IMDs based on ion drift and diffusion to elucidate the origins of their diverse dynamics. We then examine how these mechanisms operate within different materials to enable IMDs with various types of switching behaviors, leading to a wide range of applications, from emulating biological components to realizing specialized computing requirements. Furthermore, we explore the potential for IMDs to be modified and tuned to achieve customized dynamics, which positions them as one of the most promising hardware candidates for executing bioinspired algorithms with unique specifications. Finally, we identify the challenges currently facing IMDs that hinder their widespread usage and highlight emerging research directions that could significantly benefit from incorporating IMDs.
Collapse
Affiliation(s)
- Ruoyu Zhao
- Ming
Hsieh Department of Electrical and Computer Engineering, University of Southern California, Los Angeles, California 90089, United States
| | - Seung Ju Kim
- Ming
Hsieh Department of Electrical and Computer Engineering, University of Southern California, Los Angeles, California 90089, United States
| | - Yichun Xu
- Ming
Hsieh Department of Electrical and Computer Engineering, University of Southern California, Los Angeles, California 90089, United States
| | - Jian Zhao
- Ming
Hsieh Department of Electrical and Computer Engineering, University of Southern California, Los Angeles, California 90089, United States
| | - Tong Wang
- Ming
Hsieh Department of Electrical and Computer Engineering, University of Southern California, Los Angeles, California 90089, United States
| | - Rivu Midya
- Sandia
National Laboratories, Livermore, California 94550, United States
- Department
of Electrical & Computer Engineering, Texas A&M University, College
Station, Texas, 77843, United States
| | - Sabyasachi Ganguli
- Air
Force Research Laboratory Materials and Manufacturing Directorate
Wright − Patterson Air Force Base Dayton, Ohio 45433, United States
| | - Ajit K. Roy
- Air
Force Research Laboratory Materials and Manufacturing Directorate
Wright − Patterson Air Force Base Dayton, Ohio 45433, United States
| | - Madan Dubey
- Sensors
and Electron Devices Directorate, U.S. Army
Research Laboratory, Adelphi, Maryland 20723, United States
| | - R. Stanley Williams
- Ming
Hsieh Department of Electrical and Computer Engineering, University of Southern California, Los Angeles, California 90089, United States
| | - J. Joshua Yang
- Ming
Hsieh Department of Electrical and Computer Engineering, University of Southern California, Los Angeles, California 90089, United States
| |
Collapse
|
32
|
Hadke S, Kang MA, Sangwan VK, Hersam MC. Two-Dimensional Materials for Brain-Inspired Computing Hardware. Chem Rev 2025; 125:835-932. [PMID: 39745782 DOI: 10.1021/acs.chemrev.4c00631] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/23/2025]
Abstract
Recent breakthroughs in brain-inspired computing promise to address a wide range of problems from security to healthcare. However, the current strategy of implementing artificial intelligence algorithms using conventional silicon hardware is leading to unsustainable energy consumption. Neuromorphic hardware based on electronic devices mimicking biological systems is emerging as a low-energy alternative, although further progress requires materials that can mimic biological function while maintaining scalability and speed. As a result of their diverse unique properties, atomically thin two-dimensional (2D) materials are promising building blocks for next-generation electronics including nonvolatile memory, in-memory and neuromorphic computing, and flexible edge-computing systems. Furthermore, 2D materials achieve biorealistic synaptic and neuronal responses that extend beyond conventional logic and memory systems. Here, we provide a comprehensive review of the growth, fabrication, and integration of 2D materials and van der Waals heterojunctions for neuromorphic electronic and optoelectronic devices, circuits, and systems. For each case, the relationship between physical properties and device responses is emphasized followed by a critical comparison of technologies for different applications. We conclude with a forward-looking perspective on the key remaining challenges and opportunities for neuromorphic applications that leverage the fundamental properties of 2D materials and heterojunctions.
Collapse
Affiliation(s)
- Shreyash Hadke
- Department of Materials Science and Engineering, Northwestern University, Evanston, Illinois 60208, United States
| | - Min-A Kang
- Department of Materials Science and Engineering, Northwestern University, Evanston, Illinois 60208, United States
| | - Vinod K Sangwan
- Department of Materials Science and Engineering, Northwestern University, Evanston, Illinois 60208, United States
| | - Mark C Hersam
- Department of Materials Science and Engineering, Northwestern University, Evanston, Illinois 60208, United States
- Department of Chemistry, Northwestern University, Evanston, Illinois 60208, United States
- Department of Electrical Engineering and Computer Science, Northwestern University, Evanston, Illinois 60208, United States
| |
Collapse
|
33
|
Wu Q, Yang X, Wang K, Zhu M, Liang J, Han Y. "PyTDL": A versatile temporal difference learning algorithm to simulate behavior process of decision making and cognitive learning. iScience 2025; 28:111600. [PMID: 39834866 PMCID: PMC11743081 DOI: 10.1016/j.isci.2024.111600] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/25/2024] [Revised: 08/13/2024] [Accepted: 12/11/2024] [Indexed: 01/22/2025] Open
Abstract
Humans and animals excel at learning complex tasks through reward-based feedback, dynamically adjusting value expectations and choices based on past experiences to optimize outcomes. However, understanding the hidden cognitive components driving these behaviors remains challenging. Neuroscientists use the Temporal Difference (TD) learning model to estimate cognitive elements like value representation and prediction error during learning and decision-making processes. However, traditional TD algorithms fall short in diverse and dynamic tasks due to their fixed patterns. We present PyTDL, a Python-based modular framework that enables customizable value updating functions and decision policies, effectively simulating dynamic, non-linear cognitive processes. PyTDL's utility was demonstrated by modeling the decision-making processes of animals in two cognitive tasks under uncertain conditions. As open-source software, PyTDL offers a user-friendly GUI and APIs, empowering researchers to tailor models for specific tasks, align computational models with empirical data, and advance the understanding of brain learning and decision-making in complex environments.
Collapse
Affiliation(s)
- Qiyun Wu
- State Key Laboratory of Intelligent Manufacturing Equipment and Technology, Huazhong University of Science and Technology, Wuhan 430074, China
| | - Xiaodan Yang
- Department of Neurobiology, School of Basic Medicine, Tongji Medical College, Huazhong University of Science and Technology, Wuhan 430030, China
| | - Kaishu Wang
- School of Civil and Hydraulic Engineering, Huazhong University of Science and Technology, Wuhan 430074, China
| | - Min Zhu
- Department of Thoracic Surgery, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan 430030, China
| | - Jiejunyi Liang
- State Key Laboratory of Intelligent Manufacturing Equipment and Technology, Huazhong University of Science and Technology, Wuhan 430074, China
| | - Yunyun Han
- Department of Neurobiology, School of Basic Medicine, Tongji Medical College, Huazhong University of Science and Technology, Wuhan 430030, China
| |
Collapse
|
34
|
Schuster BA, Lamm C. How dopamine shapes trust beliefs. Prog Neuropsychopharmacol Biol Psychiatry 2025; 136:111206. [PMID: 39586370 DOI: 10.1016/j.pnpbp.2024.111206] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/22/2024] [Revised: 11/21/2024] [Accepted: 11/21/2024] [Indexed: 11/27/2024]
Abstract
Learning whom to trust is integral for healthy relationships and social cohesion, and atypicalities in trust learning are common across a range of clinical conditions, including schizophrenia spectrum disorders, Parkinson's disease, and depression. Persecutory delusions - rigid, unfounded beliefs that others are intending to harm oneself - significantly impact affected individuals' lives as they are associated with a range of negative health outcomes, including suicidal behaviour and relapse. Recent advances in computational modelling and psychopharmacology have significantly extended our understanding of the brain bases of dynamic trust learning, and the neuromodulator dopamine has been suggested to play a key role in this. However, the specifics of this role on a computational and neurobiological level remain to be fully established. The current review article provides a comprehensive summary of novel conceptual developments and empirical findings regarding the computational role of dopamine in social learning processes. Research findings strongly suggest a conceptual shift, from dopamine as a reward mechanism to a teaching signal indicating which information is relevant for learning, and shed light on the neurocomputational mechanisms via which antipsychotics may alleviate symptoms of aberrant social learning processes such as persecutory delusions. Knowledge gaps and inconsistencies in the extant literature are examined and the most pressing issues highlighted, laying the foundation for future research that will further advance our understanding of the neuromodulation of social belief updating processes.
Collapse
Affiliation(s)
- Bianca A Schuster
- Department of Cognition, Emotion, and Methods in Psychology, University of Vienna, Liebiggasse 5, 1010 Vienna, Austria.
| | - Claus Lamm
- Department of Cognition, Emotion, and Methods in Psychology, University of Vienna, Liebiggasse 5, 1010 Vienna, Austria
| |
Collapse
|
35
|
Quan Y, Wang J, Wang Y, Kang G. The effect of reward and voluntary choice on the motor learning of serial reaction time task. Front Psychol 2025; 15:1493434. [PMID: 39839919 PMCID: PMC11747785 DOI: 10.3389/fpsyg.2024.1493434] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/09/2024] [Accepted: 12/16/2024] [Indexed: 01/23/2025] Open
Abstract
Objective Reward and voluntary choice facilitate motor skill learning through motivation. However, it remains unclear how their combination influences motor skill learning. The purpose of the present study is to investigate the effects of reward and voluntary choice on motor skill learning in a serial reaction time task (SRTT). Methods Participants completed six parts of SRTT, including pre-test, training phase, immediate post-test, a random session, delayed post-test, and retention test on the following day. During the training phase, participants were divided into four groups (reward_choice, reward_no-choice, no-reward_choice, no-reward_no-choice). In the reward condition, participants received reward for correct and faster (than a baseline) responses while those in the no-reward groups did not. For the choice manipulation, participants in the voluntary choice groups chose the color of the target, whereas in the forced choice groups, the same color was assigned by the computer. Results The results showed that the four groups did not exhibit any significant differences in reaction time and error rate in the pre-test phase. Importantly, both reward and voluntary choice significantly enhanced sequence-specific learning effects, while no interaction was found. No significant effects of reward and voluntary choice were observed in the retention test. Conclusions These findings suggest that reward and voluntary choice enhance motor skill performance and training independently, potentially at the action-selection level, which implies different mechanisms underlying the influences of reward and voluntary choice.
Collapse
Affiliation(s)
- Yanghui Quan
- School of Psychology, Beijing Sport University, Beijing, China
| | - Jiayue Wang
- School of Psychology, Beijing Sport University, Beijing, China
| | - Yandong Wang
- School of Psychology, Beijing Sport University, Beijing, China
| | - Guanlan Kang
- School of Psychology, Beijing Sport University, Beijing, China
- Laboratory of Sports Stress and Adaptation of General Administration of Sport, Beijing Sport University, Beijing, China
| |
Collapse
|
36
|
Cusin C, Dillon DG, Belleau E, Normandin MD, Petibon Y, El-Fakri G, Dhaynaut M, Hooker J, Kaptchuk T, McKee M, Hayden E, Meyer A, Jahan A, Origlio J, Ang YS, Brunner D, Kang M, Long Y, Fava M, Pizzagalli DA. Novel multi-modal methodology to investigate placebo response in major depressive disorder. J Affect Disord 2025; 368:1-7. [PMID: 39233242 DOI: 10.1016/j.jad.2024.08.226] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/06/2024] [Revised: 08/23/2024] [Accepted: 08/31/2024] [Indexed: 09/06/2024]
Abstract
The neurobiological mechanisms underlying the placebo phenomenon in patients with major depressive disorder (MDD) remain largely unknown. The progressive rise in rates of placebo responses within clinical trials over the past two decades may impede the detection of a true signal and thus present a major obstacle in new treatment development. Understanding the mechanisms would have several important implications, including (1) identifying biomarkers of placebo responders (thereby identifying those individuals who could benefit therapeutically from such interventions), (2) opening new avenues for manipulating such mechanisms to maximize symptom reduction, and (3) refining treatments with approaches that decrease (in clinical trials) or increase (in clinical practice) the placebo response. Here we investigated the research question: is the dopaminergic system one of the neurobiological underpinnings of the placebo response within MDD? Inspired by preclinical and clinical findings that have implicated dopamine in the occurrence, prediction, and expectation of reward, we hypothesized that dopaminergic activity in the mesolimbic system is a critical mediator of placebo response in MDD. To test this hypothesis, we designed a double-blind, placebo-controlled, sequential parallel comparison design clinical trial aimed at maximizing placebo antidepressant response. We integrated behavioral, imaging, and hemodynamic probes of mesocorticolimbic dopaminergic pathways within the context of manipulations of psychological constructs previously linked to placebo responses (e.g., expectation of improvement). The aim of this manuscript is to present the rationale of the study design and to demonstrate how a cross-modal methodology may be utilized to investigate the role of reward circuitry in placebo response in MDD.
Collapse
Affiliation(s)
- Cristina Cusin
- Massachusetts General Hospital, Boston, MA, United States of America.
| | | | - Emily Belleau
- McLean Hospital, Belmont, MA, United States of America
| | - Marc D Normandin
- Massachusetts General Hospital, Boston, MA, United States of America
| | - Yoann Petibon
- Massachusetts General Hospital, Boston, MA, United States of America
| | - Georges El-Fakri
- Massachusetts General Hospital, Boston, MA, United States of America
| | - Maeva Dhaynaut
- Massachusetts General Hospital, Boston, MA, United States of America
| | - Jacob Hooker
- Massachusetts General Hospital, Boston, MA, United States of America
| | - Ted Kaptchuk
- Beth Israel Deaconess Medical Center, Boston, MA, United States of America
| | - Madison McKee
- Massachusetts General Hospital, Boston, MA, United States of America
| | - Emma Hayden
- Massachusetts General Hospital, Boston, MA, United States of America
| | - Ashley Meyer
- Massachusetts General Hospital, Boston, MA, United States of America
| | - Aava Jahan
- Massachusetts General Hospital, Boston, MA, United States of America
| | - Julianne Origlio
- Massachusetts General Hospital, Boston, MA, United States of America
| | | | - Devon Brunner
- McLean Hospital, Belmont, MA, United States of America
| | - Min Kang
- McLean Hospital, Belmont, MA, United States of America
| | - Yinru Long
- McLean Hospital, Belmont, MA, United States of America
| | - Maurizio Fava
- Massachusetts General Hospital, Boston, MA, United States of America
| | | |
Collapse
|
37
|
Shi Z, Langleben DD, Rott D, Albanese M, Elman I. Blood pressure response to extended-release naltrexone in heroin and prescription opioid users and its implications for cardiovascular morbidity. J Addict Dis 2025; 43:77-87. [PMID: 38555861 PMCID: PMC11439971 DOI: 10.1080/10550887.2024.2327739] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/02/2024]
Abstract
BACKGROUND Consuming opioid agonists is a risk factor for cardiovascular disease particularly in intravenous heroin users. The monthly injectable extended-release opioid antagonist, naltrexone (XR-NTX) is an effective treatment for opioid use disorder. The impact of opioid receptor blockade through XR-NTX on blood pressure, a critical risk factor for cardiovascular morbidity, has not yet been characterized. METHODS The study evaluated the change in blood pressure during XR-NTX treatment among 14 patients who predominately used intravenous heroin and 24 patients who used prescription oral opioids, all with opioid use disorder. Blood pressure was measured in each patient immediately before the first XR-NTX injection and ∼two weeks after the first injection. The change in diastolic and systolic pressure was compared between the heroin users and the prescription opioids users using analysis of variance. RESULTS XR-NTX treatment was associated with significant decreases in diastolic blood pressure in the heroin group, but not in the prescription opioids group. Systolic blood pressure values in the heroin users showed a decline at trend level only. CONCLUSIONS Further research is warranted to replicate our findings and to determine whether XR-NTX effect is relatively specific to blood pressure or generalizes to other components of metabolic syndrome. Distinguishing between heroin and prescription opioid users could shed light on the unique clinical and pharmacological profiles of opioid drugs, particularly regarding their cardiovascular safety. This information can be useful in developing personalized therapeutic strategies based on the route of opioid administration.
Collapse
Affiliation(s)
- Zhenhao Shi
- Center for Studies of Addiction, Department of Psychiatry, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA, USA
| | - Daniel D. Langleben
- Center for Studies of Addiction, Department of Psychiatry, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA, USA
| | - David Rott
- Department of Cardiology, Sheba Medical Center, Sackler School of Medicine, Tel Aviv, Israel
| | - Mark Albanese
- Department of Psychiatry, Cambridge Health Alliance, Harvard Medical School, Boston, MA, USA
- Physician Health Services, Massachusetts Medical Society, Waltham, MA, USA
| | - Igor Elman
- Department of Psychiatry, Cambridge Health Alliance, Harvard Medical School, Boston, MA, USA
| |
Collapse
|
38
|
Jacquin-Piques A. The pleasantness of foods. Neurophysiol Clin 2025; 55:103031. [PMID: 39644807 DOI: 10.1016/j.neucli.2024.103031] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/23/2024] [Revised: 11/07/2024] [Accepted: 11/07/2024] [Indexed: 12/09/2024] Open
Abstract
Food pleasantness is largely based on the palatability of food and is linked to taste. Along with homeostatic and cognitive control, it forms part of the control of food intake (hedonic control), and does not only correspond to the pleasure that can be described of food intake. There are many factors that cause variations in eating pleasantness between individuals, such as age, sex, culture, co-morbidities, treatments, environmental factors or the specific characteristics of foods. The control of food intake is based on four determinants: conditioned satiety, the reward system, sensory specific satiety and alliesthesia. These four determinants follow one another over time, in the per-prandial and inter-prandial periods, and complement one another. There are many cerebral areas involved in the hedonic control of food intake. The most involved brain areas are the orbitofrontal and anterior cingulate cortices, which interact with deep neural structures (amygdala, striatum, substantia nigra) for the reward circuit, with the hippocampi for memorising pleasant foods, and even with the hypothalamus and insula, brain areas more recently involved in the physiology of food pleasantness. Changes in brain activity secondary to modulation of food pleasantness can be measured objectively by recording taste-evoked potentials, an electroencephalography technique with very good temporal resolution.
Collapse
Affiliation(s)
- Agnès Jacquin-Piques
- Centre des Sciences du Goût et de l'Alimentation (CSGA) - UMR CNRS 6265, INRAE 1324, University of Burgundy, L'institut Agro - 9E, Boulevard Jeanne d'Arc - 21000 DIJON, France; University Hospital of Dijon, Bourgogne - Department of Neurology - Clinical Neurophysiology - 14, rue Paul Gaffarel - 21000 DIJON, France.
| |
Collapse
|
39
|
Lee C, Park Y, Yoon S, Lee J, Cho Y, Park C. Brain-inspired learning rules for spiking neural network-based control: a tutorial. Biomed Eng Lett 2025; 15:37-55. [PMID: 39781065 PMCID: PMC11704115 DOI: 10.1007/s13534-024-00436-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/04/2024] [Revised: 09/24/2024] [Accepted: 09/28/2024] [Indexed: 01/12/2025] Open
Abstract
Robotic systems rely on spatio-temporal information to solve control tasks. With advancements in deep neural networks, reinforcement learning has significantly enhanced the performance of control tasks by leveraging deep learning techniques. However, as deep neural networks grow in complexity, they consume more energy and introduce greater latency. This complexity hampers their application in robotic systems that require real-time data processing. To address this issue, spiking neural networks, which emulate the biological brain by transmitting spatio-temporal information through spikes, have been developed alongside neuromorphic hardware that supports their operation. This paper reviews brain-inspired learning rules and examines the application of spiking neural networks in control tasks. We begin by exploring the features and implementations of biologically plausible spike-timing-dependent plasticity. Subsequently, we investigate the integration of a global third factor with spike-timing-dependent plasticity and its utilization and enhancements in both theoretical and applied research. We also discuss a method for locally applying a third factor that sophisticatedly modifies each synaptic weight through weight-based backpropagation. Finally, we review studies utilizing these learning rules to solve control tasks using spiking neural networks.
Collapse
Affiliation(s)
- Choongseop Lee
- Department of Computer Engineering, Kwangwoon University, Seoul, 01897 Republic of Korea
| | - Yuntae Park
- Department of Computer Engineering, Kwangwoon University, Seoul, 01897 Republic of Korea
| | - Sungmin Yoon
- Department of Computer Engineering, Kwangwoon University, Seoul, 01897 Republic of Korea
| | - Jiwoon Lee
- Department of Computer Engineering, Kwangwoon University, Seoul, 01897 Republic of Korea
| | - Youngho Cho
- Department of Electrical and Communication Engineering, Daelim University College, Anyang, 13916 Republic of Korea
| | - Cheolsoo Park
- Department of Computer Engineering, Kwangwoon University, Seoul, 01897 Republic of Korea
| |
Collapse
|
40
|
Ben-Zion Z, Levy I. Representation of Anticipated Rewards and Punishments in the Human Brain. Annu Rev Psychol 2025; 76:197-226. [PMID: 39418537 PMCID: PMC11930275 DOI: 10.1146/annurev-psych-022324-042614] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/19/2024]
Abstract
Subjective value is a core concept in neuroeconomics, serving as the basis for decision making. Despite the extensive literature on the neural encoding of subjective reward value in humans, the neural representation of punishment value remains relatively understudied. This review synthesizes current knowledge on the neural representation of reward value, including methodologies, involved brain regions, and the concept of a common currency representation of diverse reward types in decision-making and learning processes. We then critically examine existing research on the neural representation of punishment value, highlighting conceptual and methodological challenges in human studies and insights gained from animal research. Finally, we explore how individual differences in reward and punishment processing may be linked to various mental illnesses, with a focus on stress-related psychopathologies. This review advocates for the integration of both rewards and punishments within value-based decision-making and learning frameworks, leveraging insights from cross-species studies and utilizing ecological gamified paradigms to reflect real-life scenarios.
Collapse
Affiliation(s)
- Ziv Ben-Zion
- Department of Psychiatry, Yale School of Medicine, Yale University, New Haven, Connecticut, USA
- VA Connecticut Healthcare System, U.S. Department of Veterans Affairs, West Haven, Connecticut, USA
- Department of Comparative Medicine, Yale School of Medicine, Yale University, New Haven, Connecticut, USA;
- Clinical Neuroscience Division, National Center for PTSD, U.S. Department of Veterans Affairs, Orange, Connecticut, USA
| | - Ifat Levy
- Wu Tsai Institute, Yale University, New Haven, Connecticut, USA
- Department of Neuroscience, Yale School of Medicine, Yale University, New Haven, Connecticut, USA
- Department of Psychology, Yale University, New Haven, Connecticut, USA
- Department of Comparative Medicine, Yale School of Medicine, Yale University, New Haven, Connecticut, USA;
| |
Collapse
|
41
|
Habekost T, Ovesen J, Madsen JB. Cognition in elite soccer players: a general model. Front Psychol 2024; 15:1477262. [PMID: 39723399 PMCID: PMC11668572 DOI: 10.3389/fpsyg.2024.1477262] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/07/2024] [Accepted: 11/15/2024] [Indexed: 12/28/2024] Open
Abstract
This paper presents a general model of the cognitive processes involved in each play situation of soccer at the elite level. Theoretically the model draws on general frameworks from cognitive psychology and neuroscience, in particular the affordance competition hypothesis and the reward prediction error theory. The model includes three functional stages: situational assessment, action selection and execution, and outcome assessment. The three stages form a perception-action cycle that corresponds to a single play situation. The cognitive processes operating at each functional stage are described and related to soccer research by a review of 52 empirical studies. The review covers the main cognitive processes that have been studied in soccer research: visual orientation and attention, pattern recognition, anticipation, working memory, action selection and decision making, executive control processes, as well as behavioral and cognitive learning. The model accommodates the wide variety of findings in the empirical literature and provides a general organizing frame for cognitive soccer research at the elite level. The influence of emotional and stress-related factors on cognition are also discussed. Four general limitations of the existing soccer research are identified, and suggestions for future studies include development of more naturalistic and interventional study designs. By specifying the different cognitive processes in soccer and their dynamic interactions the model has many applied perspectives for soccer training at the professional level. Overall, the paper presents the first integrated process model of cognition in elite soccer players with implications for both research and practice.
Collapse
Affiliation(s)
- Thomas Habekost
- Department of Psychology, University of Copenhagen, Copenhagen, Denmark
| | - Jacob Ovesen
- Department of Psychology, University of Copenhagen, Copenhagen, Denmark
| | - Jes Buster Madsen
- F. C. Copenhagen, Copenhagen, Denmark
- Saudi Pro League, Riyadh, Saudi Arabia
| |
Collapse
|
42
|
Jiang Q, Bakhurin KI, Hughes RN, Lu B, Ruan S, Yin HH. GABAergic neurons from the ventral tegmental area represent and regulate force vectors. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.12.07.627361. [PMID: 39713374 PMCID: PMC11661075 DOI: 10.1101/2024.12.07.627361] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/24/2024]
Abstract
The ventral tegmental area (VTA), a midbrain region associated with motivated behaviors, consists predominantly of dopaminergic (DA) neurons and GABAergic (GABA) neurons. Previous work has suggested that VTA GABA neurons provide a reward prediction, which is used in computing a reward prediction error. In this study, using in vivo electrophysiology and continuous quantification of force exertion in head-fixed mice, we discovered distinct populations of VTA GABA neurons that exhibited precise force tuning independently of learning, reward prediction, and outcome valence. Their activity usually preceded force exertion, and selective optogenetic manipulations of these neurons systematically modulated force exertion without influencing reward prediction. Together, these findings show that VTA GABA neurons continuously regulate force vectors during motivated behavior.
Collapse
|
43
|
Cataldi S, Lacefield C, Shashaank N, Sulzer D. Direct Pathway Neurons in the Mouse Ventral Striatum Are Active During Goal-Directed Action but Not Reward Consumption During Operant Conditioning. Biomedicines 2024; 12:2755. [PMID: 39767662 PMCID: PMC11673053 DOI: 10.3390/biomedicines12122755] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/06/2024] [Revised: 11/27/2024] [Accepted: 11/29/2024] [Indexed: 01/11/2025] Open
Abstract
BACKGROUND/OBJECTIVES Learning is classically modeled to consist of an acquisition period followed by a mastery period when the skill no longer requires conscious control and becomes automatic. Dopamine neurons projecting to the ventral striatum (VS) produce a teaching signal that shifts from responding to rewarding or aversive events to anticipating cues, thus facilitating learning. However, the role of the dopamine-receptive neurons in the ventral striatum, particularly in encoding decision-making processes, remains less understood. METHODS Here, we introduce an operant conditioning paradigm using open-source microcontrollers to train mice in three sequential learning phases. Phase I employs classical conditioning, associating a 5 s sound cue (CS) with a sucrose-water reward. In Phase II, the CS is replaced by a lever press as the requirement for reward delivery, marking an operant conditioning stage. Phase III combines these elements, requiring mice to press the lever during the CS to obtain the reward. We recorded calcium signals from direct pathway spiny projection neurons (dSPNs) in the VS throughout the three phases of training. RESULTS We find that dSPNs are specifically engaged when the mouse makes a decision to perform a reward-seeking action in response to a CS but are largely inactive during actions taken outside the CS. CONCLUSIONS These findings suggest that direct pathway neurons in the VS contribute to decision-making in learned action-outcome associations, indicating a specialized role in initiating operant behaviors.
Collapse
Affiliation(s)
- Stefano Cataldi
- Department of Psychiatry, Division of Molecular Therapeutics, New York State Psychiatric Institute, Columbia University, New York, NY 10032, USA; (S.C.); (C.L.)
- Italian Academy, Columbia University, New York, NY 10027, USA
| | - Clay Lacefield
- Department of Psychiatry, Division of Molecular Therapeutics, New York State Psychiatric Institute, Columbia University, New York, NY 10032, USA; (S.C.); (C.L.)
- Division of Systems Neuroscience, New York State Psychiatric Institute, New York, NY 10032, USA
| | - N Shashaank
- Departments of Computer Science, Shapiro Center for Engineering and Physical Science Research, Columbia University, New York, NY 10027, USA;
- New York Genome Center, New York, NY 10013, USA
| | - David Sulzer
- Department of Psychiatry, Division of Molecular Therapeutics, New York State Psychiatric Institute, Columbia University, New York, NY 10032, USA; (S.C.); (C.L.)
- Departments of Neurology, Columbia University, New York, NY 10032, USA
- Departments of Pharmacology, Columbia University, New York, NY 10032, USA
| |
Collapse
|
44
|
Westwood S, Philiastides MG. Early Salience Signals Predict Interindividual Asymmetry in Decision Accuracy Across Rewarding and Punishing Contexts. Hum Brain Mapp 2024; 45:e70072. [PMID: 39584595 PMCID: PMC11586867 DOI: 10.1002/hbm.70072] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/02/2024] [Revised: 10/10/2024] [Accepted: 10/28/2024] [Indexed: 11/26/2024] Open
Abstract
Asymmetry in choice patterns across rewarding and punishing contexts has long been observed in behavioural economics. Within existing theories of reinforcement learning, the mechanistic account of these behavioural differences is still debated. We propose that motivational salience-the degree of bottom-up attention attracted by a stimulus with relation to motivational goals-offers a potential mechanism to modulate stimulus value updating and decision policy. In a probabilistic reversal learning task, we identified post-feedback signals from EEG and pupillometry that captured differential activity with respect to rewarding and punishing contexts. We show that the degree of between-context distinction in these signals predicts interindividual asymmetries in decision accuracy. Finally, we contextualise these effects in relation to the neural pathways that are currently centred in theories of reward and punishment learning, demonstrating how the motivational salience network could plausibly fit into a range of existing frameworks.
Collapse
|
45
|
Werle D, Sablottny L, Ansorge U, Biehl SC, Tuschen-Caffier B, Svaldi J. Attention to food stimuli in binge eating disorder: Electrophysiological evidence. Appetite 2024; 203:107682. [PMID: 39303828 DOI: 10.1016/j.appet.2024.107682] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2024] [Revised: 08/09/2024] [Accepted: 09/16/2024] [Indexed: 09/22/2024]
Abstract
Attentional biases towards food play an important role in the pathology of binge eating disorder (BED). Later stage electrophysiological potentials (P300, late positive potential) present promising markers of motivated attention with high temporal, albeit low spatial resolution. Complementing this, the N2pc is an earlier-latency component providing the possibility of more directly analyzing visuospatial attention. Therefore, we tested a group with BED (N = 60), as well as an overweight (OW; N = 28) and normal weight (NW; N = 30) group without BED in a Go/No-Go paradigm using food and nonfood distractor images. Only the OW group in exclusively the Go trials displayed a stronger spatial attention allocation towards nonfood distractors as evidenced by an increased N2pc amplitude. In the P300's time window, the OW group displayed no attentional bias towards food and the NW group only did so in the absence of a target. Solely the BED group allocated more motivated attention towards food distractors both in Go and No-Go trials. In the following late positive potential (LPP), the OW group exhibited a general attentional bias towards food distractors, while the BED group only did so in the absence of a target. These results are discussed in light of the incentive sensitization theory and a potential early attentional suppression of potent distractors.
Collapse
Affiliation(s)
- Dustin Werle
- University of Tuebingen, Department of Clinical Psychology and Psychotherapy, Germany.
| | - Lynn Sablottny
- University of Freiburg, Department of Clinical Psychology and Psychotherapy, Germany
| | | | - Stefanie C Biehl
- University of Tuebingen, Department of Clinical Psychology and Psychotherapy, Germany
| | | | - Jennifer Svaldi
- University of Tuebingen, Department of Clinical Psychology and Psychotherapy, Germany
| |
Collapse
|
46
|
Slifstein M, Qu W, Gil R, Weinstein JJ, Perlman G, Jaworski-Calara T, Meng J, Hu B, Moeller SJ, Horga G, Abi-Dargham A. Kappa opioid receptor availability predicts severity of anhedonia in schizophrenia. Neuropsychopharmacology 2024; 49:2087-2093. [PMID: 39217267 PMCID: PMC11480413 DOI: 10.1038/s41386-024-01975-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/10/2024] [Revised: 07/31/2024] [Accepted: 08/20/2024] [Indexed: 09/04/2024]
Abstract
The kappa opioid receptor (KOR) and its endogenous agonist dynorphin have been implicated in multiple psychiatric conditions including psychotic disorders. We tested the hypotheses that kappa expression is elevated and associated with psychotic symptoms in schizophrenia. We measured kappa expression in unmedicated patients with schizophrenia (7 female, 6 male) and matched controls (7 female, 6 male) with positron emission tomography (PET). We also acquired a measurement of cumulative dopamine activity over the life span in the same subjects using neuromelanin sensitive MRI. We hypothesized that neuromelanin accumulation would be higher in patients than controls and that in patients there would be a positive association between KOR availability and neuromelanin accumulation. Fourteen patients and thirteen controls were enrolled. Whole brain dynamic PET imaging data using the KOR selective tracer [18F]LY245998 were acquired. Distribution volume (VT) was measured with region of interest analysis in 14 brain regions. Neuromelanin accumulation in midbrain dopaminergic nuclei was assessed in the same subjects. Positive and negative symptoms were measured by a clinical psychologist. We did not observe group level differences in KOR expression, neuromelanin accumulation or relationships of these to positive symptoms. Unexpectedly, we did observe strong positive associations between KOR expression and symptoms of anhedonia in the patients (Pearson r > 0.7, uncorrected p < 0.01 in 8 cortical brain regions). We also observed moderate associations between KOR expression and neuromelanin levels in patients. In conclusion, we did not observe a relationship between kappa and symptoms of psychosis but the observed relationship to the negative symptom of anhedonia is in line with recent work testing kappa antagonism as a therapy for anhedonia in depression.
Collapse
Affiliation(s)
- Mark Slifstein
- Renaissance School of Medicine, Stony Brook University, Stony Brook, New York, USA.
| | - Wenchao Qu
- Renaissance School of Medicine, Stony Brook University, Stony Brook, New York, USA
| | - Roberto Gil
- Renaissance School of Medicine, Stony Brook University, Stony Brook, New York, USA
| | - Jodi J Weinstein
- Renaissance School of Medicine, Stony Brook University, Stony Brook, New York, USA
| | - Greg Perlman
- Renaissance School of Medicine, Stony Brook University, Stony Brook, New York, USA
| | | | - Jiayan Meng
- Renaissance School of Medicine, Stony Brook University, Stony Brook, New York, USA
| | - Bao Hu
- Renaissance School of Medicine, Stony Brook University, Stony Brook, New York, USA
| | - Scott J Moeller
- Renaissance School of Medicine, Stony Brook University, Stony Brook, New York, USA
| | - Guillermo Horga
- Vagelos College of Physicians and Surgeons, Columbia University, Stony Brook, New York, USA
- New York State Psychiatric Institute, Stony Brook, New York, USA
| | - Anissa Abi-Dargham
- Renaissance School of Medicine, Stony Brook University, Stony Brook, New York, USA
| |
Collapse
|
47
|
Zühlsdorff K, Sala‐Bayo J, Piller S, Zhukovsky P, Lamla T, Nissen W, von Heimendahl M, Deiana S, Nicholson JR, Robbins TW, Alsiö J, Dalley JW. Optogenetic activation of mesencephalic projections to the nucleus accumbens shell impairs probabilistic reversal learning by disrupting learning from negative reinforcement. Eur J Neurosci 2024; 60:6765-6778. [PMID: 39479888 PMCID: PMC11612850 DOI: 10.1111/ejn.16584] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2024] [Revised: 09/25/2024] [Accepted: 10/14/2024] [Indexed: 11/02/2024]
Abstract
Cognitive flexibility, the capacity to adapt behaviour to changes in the environment, is impaired in a range of brain disorders, including schizophrenia and Parkinson's disease. Putative neural substrates of cognitive flexibility include mesencephalic pathways to the ventral striatum (VS) and dorsomedial striatum (DMS), hypothesized to encode learning signals needed to maximize rewarded outcomes during decision-making. However, it is unclear whether mesencephalic projections to the ventral and dorsal striatum are distinct in their contribution to flexible reward-related learning. Here, rats acquired a two-choice spatial probabilistic reversal learning (PRL) task, reinforced on an 80%|20% basis (correct|incorrect responses) that assessed the flexibility of behaviour to repeated reversals of response-outcome contingencies. We report that optogenetic stimulation of projections from the ventral tegmental area (VTA) to the nucleus accumbens shell (NAcS) in the VS significantly impaired reversal learning when optical stimulation was temporally aligned with negative feedback (i.e., reward omission). VTA → NAcS stimulation during other phases of the behavioural task was without significant effect. Optogenetic stimulation of projection neurons from the substantia nigra (SN) to the DMS, aligned either with reward receipt or omission or prior to making a choice, had no significant effect on reversal learning. These findings are consistent with the notion that increased activity in the VTA → NAcS pathway disrupts behavioural flexibility by impairing learning from negative reinforcement.
Collapse
Affiliation(s)
| | | | - Sammy Piller
- Boehringer Ingelheim Pharma GmbH & Co. KG, Div. Research GermanyBiberach an der RißGermany
| | | | - Thorsten Lamla
- Boehringer Ingelheim Pharma GmbH & Co. KG, Div. Research GermanyBiberach an der RißGermany
| | - Wiebke Nissen
- Boehringer Ingelheim Pharma GmbH & Co. KG, Div. Research GermanyBiberach an der RißGermany
| | - Moritz von Heimendahl
- Boehringer Ingelheim Pharma GmbH & Co. KG, Div. Research GermanyBiberach an der RißGermany
| | - Serena Deiana
- Boehringer Ingelheim Pharma GmbH & Co. KG, Div. Research GermanyBiberach an der RißGermany
| | - Janet R. Nicholson
- Boehringer Ingelheim Pharma GmbH & Co. KG, Div. Research GermanyBiberach an der RißGermany
| | | | - Johan Alsiö
- Department of PsychologyUniversity of CambridgeCambridgeUK
- Present address:
School of Physiology, Pharmacology and NeuroscienceUniversity of BristolBristolUK
| | - Jeffrey W. Dalley
- Department of PsychologyUniversity of CambridgeCambridgeUK
- Department of Psychiatry, Herchel Smith Building for Brain & Mind SciencesUniversity of CambridgeCambridgeUK
| |
Collapse
|
48
|
Moneta N, Grossman S, Schuck NW. Representational spaces in orbitofrontal and ventromedial prefrontal cortex: task states, values, and beyond. Trends Neurosci 2024; 47:1055-1069. [PMID: 39547861 DOI: 10.1016/j.tins.2024.10.005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/04/2024] [Revised: 10/16/2024] [Accepted: 10/17/2024] [Indexed: 11/17/2024]
Abstract
The orbitofrontal cortex (OFC) and ventromedial-prefrontal cortex (vmPFC) play a key role in decision-making and encode task states in addition to expected value. We review evidence suggesting a connection between value and state representations and argue that OFC / vmPFC integrate stimulus, context, and outcome information. Comparable encoding principles emerge in late layers of deep reinforcement learning (RL) models, where single nodes exhibit similar forms of mixed-selectivity, which enables flexible readout of relevant variables by downstream neurons. Based on these lines of evidence, we suggest that outcome-maximization leads to complex representational spaces that are insufficiently characterized by linear value signals that have been the focus of most prior research on the topic. Major outstanding questions concern the role of OFC/ vmPFC in learning across tasks, in encoding of task-irrelevant aspects, and the role of hippocampus-PFC interactions.
Collapse
Affiliation(s)
- Nir Moneta
- Institute of Psychology, Universität Hamburg, 20146 Hamburg, Germany; Einstein Center for Neurosciences Berlin, Charité Universitätsmedizin Berlin, 10117, Berlin, Germany.
| | - Shany Grossman
- Institute of Psychology, Universität Hamburg, 20146 Hamburg, Germany.
| | - Nicolas W Schuck
- Institute of Psychology, Universität Hamburg, 20146 Hamburg, Germany; Max Planck UCL Centre for Computational Psychiatry and Ageing Research, Berlin, 14195 Berlin, Germany.
| |
Collapse
|
49
|
Leonard BT, Kark SM, Granger SJ, Adams JG, McMillan L, Yassa MA. Anhedonia is associated with higher functional connectivity between the nucleus accumbens and paraventricular nucleus of thalamus. J Affect Disord 2024; 366:1-7. [PMID: 39197547 DOI: 10.1016/j.jad.2024.08.113] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/14/2023] [Revised: 06/17/2024] [Accepted: 08/22/2024] [Indexed: 09/01/2024]
Abstract
BACKGROUND Anhedonia stands as a life-threatening transdiagnostic feature of many mental illnesses, most notably major depression and involves neural circuits for processing reward information. The paraventricular nucleus of the thalamus (PVT) is associated with reward-seeking behavior, however, links between the PVT circuit and anhedonia have not been investigated in humans. METHODS In a sample of adults with and without psychiatric symptoms (n = 75, 18-41 years, 55 female), we generated an anhedonia factor score for each participant using a latent factor analysis, utilizing data from depression and anxiety assessments. Functional connectivity between the PVT and the nucleus accumbens (NAc) was calculated from high-resolution (1.5 mm) resting state fMRI. RESULTS Anhedonia factor scores showed a positive relationship with functional connectivity between the PVT and the NAc, principally in males and in those with psychiatric symptoms. In males, connectivity between other midline thalamic nuclei and the NAc did not show these relationships, suggesting that this link may be specific to PVT. LIMITATIONS This cohort was originally recruited to study depression and not anhedonia per se. The distribution of male and female participants in our cohort was not equal. Partial acquisition in high-resolution fMRI scans restricted regions of interest outside of the thalamus and reward networks. CONCLUSIONS We report evidence that anhedonia is associated with enhanced functional connectivity between the PVT and the NAc, regions that are relevant to reward processing. These results offer clues as to the potential prevention and prevention and treatment of anhedonia.
Collapse
Affiliation(s)
- Bianca T Leonard
- Center for the Neurobiology of Learning and Memory, University of California, Irvine 92697, USA; Department of Neurobiology and Behavior, University of California, Irvine 92697, USA
| | - Sarah M Kark
- Center for the Neurobiology of Learning and Memory, University of California, Irvine 92697, USA; Department of Neurobiology and Behavior, University of California, Irvine 92697, USA
| | - Steven J Granger
- Division of Depression and Anxiety Disorders, McLean Hospital, Belmont, MA 02478, USA; Department of Psychiatry, Harvard Medical School, Boston, MA 02115, USA
| | - Joren G Adams
- Department of Psychiatry, Oregon Health & Science University, Portland, OR 97239, USA; VA HSR&D Center to Improve Veteran Involvement in Care, VA Portland Health Care System, Portland, OR 97239, USA
| | - Liv McMillan
- Center for the Neurobiology of Learning and Memory, University of California, Irvine 92697, USA; Department of Neurobiology and Behavior, University of California, Irvine 92697, USA
| | - Michael A Yassa
- Center for the Neurobiology of Learning and Memory, University of California, Irvine 92697, USA; Department of Neurobiology and Behavior, University of California, Irvine 92697, USA; Department of Psychiatry and Human Behavior, University of California, Irvine, CA 92697, USA.
| |
Collapse
|
50
|
Taniguchi J, Tritsch NX. A detailed look at striatal acetylcholine, dopamine, and their interactions. Proc Natl Acad Sci U S A 2024; 121:e2419752121. [PMID: 39556753 PMCID: PMC11621726 DOI: 10.1073/pnas.2419752121] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/20/2024] Open
Affiliation(s)
- James Taniguchi
- Department of Psychiatry, Douglas Hospital Research Centre, McGill University, Montreal, QCH4H 1R3, Canada
- Integrated Program in Neuroscience, Montreal Neurological Institute, McGill University, Montreal, QCH3A 2B4, Canada
| | - Nicolas X. Tritsch
- Department of Psychiatry, Douglas Hospital Research Centre, McGill University, Montreal, QCH4H 1R3, Canada
- Integrated Program in Neuroscience, Montreal Neurological Institute, McGill University, Montreal, QCH3A 2B4, Canada
| |
Collapse
|