Substance use disorders are increasingly being conceptualized as behavioral misallocation disorders; however, the neurobiological determinants of this behavioral misallocation are poorly understood. Cocaine use disorder (CUD) develops as a result of behavior being disproportionally directed towards the procurement and use of cocaine at the expense of behaviors maintained by more adaptive, nondrug reinforcers (i.e., job, family). Preclinical cocaine-vs-nondrug reinforcer choice procedures are uniquely positioned to 1) elucidate the biological mechanisms of drug and nondrug reinforcement and 2) inform the development of effective pharmacological and behavioral CUD therapies. Accordingly, this review addresses the existing preclinical literature regarding the economic substitutability and mesolimbic dopaminergic mechanisms underlying cocaine self-administration in the context of three different concurrently available nondrug reinforcers: food, social interaction, and electric foot shock (a negative reinforcer). The manuscript focuses on how the existing cocaine-vs-nondrug reinforcer choice literature guides future research directions to facilitate advances in understanding of CUD from both a neuroscience and translational research perspective.

We advance a novel formulation of cognitive control within the active inference framework. The theory proposes that cognitive control amounts to optimising a precision parameter, which acts as a control signal and balances the contributions of deliberative and habitual components of action selection. To illustrate the theory, we simulate a driving scenario in which the driver follows a well-known route, but encounters unexpected challenges. Our simulations show that a standard active inference model can form adaptive habits; i.e., can pass from deliberative to habitual control when the context is stable, but generally fails to revert to deliberative control, when the context changes. To address this failure of context-sensitivity, we introduce a novel type of hierarchical active inference, in which a lower level is responsible for behavioural control and the higher (or meta-cognitive) level observes the belief updating of the lower level below and is responsible for cognitive control. Crucially, the meta-cognitive level can both form habits and suspend them, by controlling the (precision) parameter that prioritizes deliberative choices at the behavioural level. Furthermore, we show that several processes linked to cognitive control - such as surprise detection, cognitive conflict monitoring, control signal regulation and specification, the simulation of future outcomes and the assessment of the costs of control and mental effort - stem coherently from the free energy minimization scheme that underpins active inference. Finally, we discuss the putative neurobiology of cognitive control by simulating brain dynamics in the mesolimbic and mesocortical pathways of the dopamine system, the dorsal anterior cingulate cortex and the locus coeruleus.

Autism spectrum disorder (ASD) is associated with disruptions in social behavior and the neural circuitry behind it. Very little data is available on the mechanisms that are responsible for the lack of motivation to reunite with conspecifics during isolation. It is as important to investigate the neural changes that reduce motivation to end social isolation, as those underlying the reactions to social stimuli. Using a rodent model of prenatal valproic acid (VPA) exposure, we investigated how social isolation affects the neural activation of key brain nuclei involved in social processing and stress regulation. Juvenile male C57BL/6 mice were treated prenatally with VPA or saline (CTR) and subjected to 24 h of social isolation from their cage mates, with neural activity assessed via c-Fos immunohistochemistry. Based on correlational activations we reconstructed and analyzed the functional connectome of the observed brain regions. Control animals exhibited elevated c-Fos expression in the regions central to the mesolimbic reward system (MRS), social brain network (SBN), and stress-related networks, with the interpeduncular nucleus (IPN) at the core, compared to VPA-treated animals. Functional network analysis revealed a more widespread but less specific pattern of connectivity in VPA-treated animals. These findings suggest that prenatal VPA exposure disrupts certain neural circuits related to social behavior and stress regulation, offering an insight into the altered perception of social isolation in ASD models, and highlighting potential therapeutic targets.

Parkinson's disease (PD) is a complex neurological disorder characterized by the progressive degeneration of midbrain dopaminergic (mDA) neurons in the substantia nigra (SN). This degeneration disrupts the basal ganglia loops, leading to both motor and non-motor dysfunctions. Cell therapy for PD aims to replace lost mDA neurons to restore the DA neurotransmission in the denervated forebrain targets. In clinical trials for PD, mDA neurons are implanted into the target area, the striatum, and not in the SN where they are normally located. This ectopic localisation of cells may affect the functionality of transplanted neurons due to the absence of appropriate host afferent regulation. We recently demonstrated that human induced pluripotent stem cells (hiPSCs) derived mDA progenitors grafted into the substantia nigra pars compacta (SNpc) in a mouse model of PD, differentiated into mature mDA neurons, restored the degenerated nigrostriatal pathway, and induced motor recovery. The objective of the present study was to evaluate the long-term functionality of these intranigral-grafted mDA neurons by assessing their electrophysiological properties.

We performed intranigral transplantation of hiPSC-derived mDA progenitors in a 6-hydroxydopamine RAG2-KO mouse model of PD. We recorded in vivo unit extracellular activity of grafted mDA neurons in anesthetised mice from 9 to 12 months post-transplantation. Their electrophysiological properties, including firing rates, patterns and spike characteristics, were analysed and compared with those of native nigral dopaminergic neurons from control mice.

We demonstrated that these grafted mDA neurons exhibited functional characteristics similar to those of native nigral dopaminergic neurons, such as large bi- or triphasic spike waveforms, low firing rates, pacemaker-like properties, and two single-spike firing patterns. Although grafted mDA neurons also displayed low discharge frequencies below 10 Hz, their mean frequency was significantly lower than that of nigral mDA neurons, with a differential pattern distribution.

Our findings indicate that grafted mDA neurons exhibit dopaminergic-like functional properties, including intrinsic membrane potential oscillations leading to regular firing patterns. Additionally, they demonstrated irregular and burst firing patterns, suggesting they receive modulatory inputs. However, grafted mDA neurons displayed distinct properties, potentially related to their human origin or the incomplete maturation one year after transplantation.

Sexual activity produces pleasure related to sexual arousal, desire, and genitosensory and erogenous stimulation. Orgasms produce a whole brain and body rush of ecstatic pleasure followed by relaxation and refractoriness. This pleasure results from the activation of neurochemical reward pathways in the brain. This is differentiated by spinal pathways that control climax, the particular motor movements of the pelvic floor and the experience of tension release.

To relate the activation of key neurochemical reward and bonding systems, notably dopamine, oxytocin, and opioids, to the pleasure of sexual activity in general and orgasms in particular.

A narrative review of the neurochemical and neuroanatomical mechanisms activated during sexual stimulation and orgasm in rats and humans, and how they are related overall to the generation of sexual pleasure and reward.

Appetitive sexual pleasure involves the activation of dopamine and oxytocin release in hypothalamic and mesolimbic regions that regulate sexual arousal and desire, and are reinforced by localized opioid activity. Orgasms are thought to result in part from a massive release of opioids into these regions that inhibits dopamine and oxytocin transmission, but that initiates molecular changes to sensitize both systems and induce sexually conditioned place and partner preferences. Serotonin is also activated at orgasm and contributes to feelings of satiety and refractoriness. Orgasm disorders are distressing, cause resentment and conflict in a relationship, and diminish overall sexual health and well-being.

Orgasms are an important component of sexual pleasure for humans and perhaps all vertebrates. Endogenous opioids like β-endorphin that bind to mu opioid receptors are likely responsible for sexual pleasure and reward.

The drive for positive social interactions, or "social rewards", is an important motivator of human behaviour, conferring several adaptive benefits. Social motivation fluctuates across the lifespan, reflecting changes in goals and priorities at different developmental stages. In older adulthood, for instance, priorities tend to shift toward maintaining emotional wellbeing and resources over seeking novel gains. Contemporary theories of social interaction must account for such motivational shifts, addressing the enhancement of social processing in ageing and its decline in dementia. Here, we propose a framework to track the evolution of social motivation across the lifespan, focusing on three mechanisms: (i) social interactions as rewards, (ii) learning from social interactions, and (iii) the effort required for social interactions. We posit that social rewards hold equivalent or increased value later in life, enhancing older adults' social connections. Conversely, social rewards become devalued in neurodegenerative disorders such as frontotemporal dementia (FTD), resulting in social withdrawal. This integrative framework serves as a foundation for understanding adaptive and maladaptive trajectories of social motivation throughout the adult lifespan.

Over recent decades, temporal difference reinforcement learning (TDRL) models have successfully explained much dopamine (DA) activity. This success has invited heightened scrutiny of late, with many studies challenging the validity of TDRL models of DA function. Yet, when evaluating the validity of these models, the devil is truly in the details. TDRL is a broad class of algorithms sharing core ideas but differing greatly in implementation and predictions. Thus, it is important to identify the defining aspects of the TDRL framework being tested and to use state spaces and model architectures that capture the known complexity of the behavioral representations and neural systems involved. Here, we discuss several examples that illustrate the importance of these considerations.

Despite the popularity of drugs that act on catecholamine receptors, our knowledge of catecholamine dynamics in human health and disease remains incomplete. Recent advances in fluorescent sensors have enabled unprecedented access to catecholamine dynamics in preclinical animal models, but the requirements of these technologies to use in model organisms limit their translational value for clinical diagnostics. Here, we introduce proof of principle fluorescent catecholamine detection via optical fibers functionalized with single-walled carbon nanotube (SWNT)-based near-infrared catecholamine sensors (nIRCats), a catecholamine detection form factor that has potential for more convenient and less invasive clinical translation. We show that these near-infrared functionalized (nIRF) fibers respond to dopamine in a biologically relevant concentration range (10 nM through 1 μM), with minimal responsivity loss following 16 h exposure to human blood plasma. We further demonstrate the utility of these fibers in detecting dopamine from as little as 10 μL volumes of clinically relevant biofluids up to 24 weeks after fiber synthesis. We also introduce a compact, mobile dual-near-infrared fiber photometry rig and demonstrate its success detecting dopamine in acute brain slices with nIRF fibers. Together, this fiber-based dopamine detection tool and photometry rig expand the toolset for catecholamine detection.

Gambling behaviour is a persistent and growing societal problem. An unexplored factor that may encourage gambling behaviour is the impact of circadian photoreception on cognitive processes underlying the behaviour. We investigated the influence of circadian photoreception on loss aversion in gambling by altering the blue content of light while maintaining the same visual brightness. Fifteen participants (age 18-27 years, M = 20.40, SD = 2.03) completed an economic decision-making task under blue-enriched and blue-depleted light, of equivalent visual brightness, on separate occasions in a randomised order. The task required participants to choose between taking a risky gamble of a positive and negative outcome, or a less risky guaranteed outcome. Hierarchical Bayesian Modelling was conducted to derive individual parameter estimates for loss aversion, and trial-by-trial performance was analysed using linear mixed models. The findings demonstrated that individuals were significantly less loss averse under blue-enriched light compared to blue-depleted light (β = - .43, 95% CI [- .82, - .04], p = .03). This study shows that exposure to light that preferentially targets circadian photoreception reduces loss aversion, which may encourage gambling behaviour.

DNA-dependent protein kinase catalytic subunit (DNA-PKcs) is one of the key enzymes involved in DNA double-strand break (DSB) repair. However, recent studies using DNA-PKcs knockout mice revealed that DNA-PKcs plays an important role in neuronal plasticity. The aim of this study was to examine the role of DNA-PKcs on synaptic plasticity in severe combined immunodeficiency disease (SCID) mice that carry a mutation resulting in a DNA-PKcs protein devoid of kinase activity but still expressed in cells, although with a small COOH-terminal truncation. To this aim, we carried out electrophysiological and molecular analysis on hippocampal slices from wild-type (WT) and SCID mice. Electrophysiological analysis showed an impairment in the basal synaptic transmission in SCID mice compared with WT, whereas paired-pulse facilitation, caused by presynaptic mechanisms, was not different in the two groups of animals. By contrast, tetanic stimulation induced long-term potentiation (LTP) with values that were approximately 43% lower in slices from SCID mice compared with WT. The same slices used for electrophysiology were analyzed to study the phosphorylation state of cAMP response element-binding protein (CREB) and extracellular signal-regulated kinases and to evaluate mRNA expression levels of CREB-target genes at different times after LTP induction. In conclusion, molecular analysis did not show significant differences between SCID and WT brain slices, thus confirming the evidence that DNA-PKcs kinase activity directly regulates neuronal functions and plays a novel role beyond DSB repair. Moreover, these results indicate that studies using SCID mice involving analysis of synaptic function need to be interpreted with caution.

The mesolimbic dopamine system is a set of subcortical brain circuits that plays a key role in reward processing, reinforcement, associative learning, and behavioral responses to salient environmental events. In our previous studies of the dopaminergic response to salient visual stimuli, we observed that dopamine release in the lateral nucleus accumbens (LNAc) of mice encoded information about the rate and magnitude of rapid environmental luminance changes from darkness. Light-evoked dopamine responses were rate-dependent, robust to the time of testing or stimulus novelty, and required phototransduction by rod and cone opsins. However, it is unknown if these dopaminergic responses also involve non-visual opsins, such as melanopsin, the primary photopigment expressed by intrinsically photosensitive retinal ganglion cells (ipRGCs). In the current study, we evaluated the role of melanopsin in the dopaminergic response to light in the LNAc using the genetically encoded dopamine sensor dLight1 and fiber photometry. By measuring light-evoked dopamine responses across a broad irradiance and wavelength range in constitutive melanopsin (Opn4) knockout mice, we were able to provide new insights into the ability of non-visual opsins to regulate the mesolimbic dopamine response to visual stimuli.

Predictions are combined with sensory information when making choices. Accumulator models have conceptualized predictions as trial-by-trial updates to a baseline evidence level. These models have been successful in explaining the influence of choice history across-trials, however, they do not account for how sensory information is transformed into choice evidence. Here, we derive a gated accumulator that models the onset of evidence accumulation as a combination of delayed sensory information and a prediction of sensory timing. To test how delays interact with predictions, we designed a free-choice saccade task where participants directed eye movements to either of two targets that appeared with variable delays and asynchronies. Despite instructions not to anticipate, participants responded before target onset on some trials. We reasoned that anticipatory responses reflected a trade-off between inhibiting and facilitating the onset of evidence accumulation via a gating mechanism as target appearance became more likely. We then found that anticipatory responses were more likely following repeated choices, suggesting that the balance between anticipatory and sensory responses was driven by a prediction of sensory timing. By fitting the gated accumulator model to the data, we found that variance in within-trial fluctuations in baseline evidence best explained the joint increase of anticipatory responses and faster sensory-guided responses with longer delays. Thus, we conclude that a prediction of sensory timing is involved in balancing the costs of anticipation with lowering the amount of accumulated evidence required to trigger saccadic choice.NEW & NOTEWORTHY Evidence accumulation models are used to study how recent history impacts the processes underlying how we make choices. Biophysical evidence suggests that the accumulation of evidence is gated, however, classic accumulator models do not account for this. In this work, we show that predictions of the timing of sensory information are important in controlling how evidence accumulation is gated and that signatures of these predictions can be detected even in randomized task environments.

Recent studies have shown that spontaneous pre-stimulus fluctuations in brain activity affect higher-order cognitive processes, including risky decision-making, cognitive flexibility, and aesthetic judgments. However, there is currently no direct evidence to suggest that pre-choice activity influences value-based decisions that require self-control. We examined the impact of fluctuations in pre-choice activity in key regions of the reward system on self-control in food choice. In the functional magnetic resonance imaging (fMRI) scanner, 49 participants made 120 food choices that required self-control in high and low working memory load conditions. The task was designed to ensure that participants were cognitively engaged and not thinking about upcoming choices. We defined self-control success as choosing a food item that was healthier over one that was tastier. The brain regions of interest (ROIs) were the ventral tegmental area (VTA), putamen, nucleus accumbens (NAc), and caudate nucleus. For each participant and condition, we calculated the mean activity in the 3-s interval preceding the presentation of food stimuli in successful and failed self-control trials. These activities were then used as predictors of self-control success in a fixed-effects logistic regression model. The results indicate that increased pre-choice VTA activity was linked to a higher probability of self-control success in a subsequent food-choice task within the low-load condition, but not in the high-load condition. We posit that pre-choice fluctuations in VTA activity change the reference point for immediate (taste) reward evaluation, which may explain our finding. This suggests that the neural context of decisions may be a key factor influencing human behavior.

To realize human-machine fusion, a hybrid neural pathway operating in the same modality with biological systems becomes imperative, which requires an interneuron unit to encode information in biorecognizable spike sequences and tune the frequency upon excitatory and inhibitory neurotransmitters. Existing artificial interneurons cannot encode information upon different neurotransmitters, and the activation threshold and frequency responsivity do not align with those of biological counterparts, leading to limited success in constructing a signal-harmonizing hybrid neural pathway for neuroprosthetics, neurorehabilitation, and other neuroelectronic applications. Herein, we develop a bipolar-chemosynapse interneuron to encode the spike frequency in a highly bionic paradigm. Bipolar synapses dynamically respond to excitatory and inhibitory neurotransmitters and translate time-series chemical signals into the spike sequence, achieving the lowest activation threshold (6.25 μM) and the highest frequency responsivity (0.55 Hz μM-1) to date, close to the biological counterpart. A signal-harmonizing hybrid sensorimotor pathway mediated by excitatory and inhibitory neurotransmitters is constructed for the first time, which encodes upstream mechanical stimuli, modulates the downstream leg swing frequency of a mouse, and balances neural activity accordingly.

Learning the causal structures of social environments involves predicting significant events (e.g., rewards) and detecting prediction errors for each agent. Whether the brain can simultaneously compute reward prediction errors for self (S-RPE) and others (O-RPE), and which neurons are responsible, is unclear. Here, we condition two monkeys with identical visual stimuli predicting different reward outcomes and find that dorsomedial prefrontal neurons represent both S-RPE and O-RPE simultaneously. Neuronal signatures of RPE are agent and sign specific, forming distinct populations for positive and negative S-RPE and O-RPE. A linear decoder trained on neurons encoding O-RPE, but not S-RPE, successfully discriminates RPE. Further investigation identifies coexisting actual reward and prediction confirmation signals for others. These results highlight the presence of neuronal mechanisms in the primate brain that update the value of environmental stimuli simultaneously for oneself and others, enabling primates to comprehend the causal structure of the world from the perspective of others.

Movement vigor across multiple modalities increases with reward, suggesting that the neural circuits that represent value influence the control of movement. Dopaminergic neuron (DAN) activity in the basal ganglia has been suggested as the potential mediator of this response. If DAN activity is the bridge between value and vigor, then vigor should track canonical mediators of this activity, namely reward expectation and reward prediction error. Here we ask if a similar time-locked response is present in vigor of reaching movements. We explore this link by leveraging the known phasic dopaminergic response to stochastic rewards, where activity is modulated by both reward expectation at cue and the prediction error at feedback. We used probabilistic rewards to create a reaching task rich in reward expectation, reward prediction error, and learning. In one experiment, target reward probabilities were explicitly stated, and in the other, were left unknown and to be learned by the participants. We included two stochastic rewards (probabilities 33% and 66%) and two deterministic ones (probabilities 100% and 0%). Outgoing peak velocity in both experiments increased with increasing reward expectation. Furthermore, we observed a short-latency response in the vigor of the ongoing movement, that tracked reward prediction error: either invigorating or enervating velocity consistent with the sign and magnitude of the error. Reaching kinematics also revealed the value-update process in a trial-to-trial fashion, similar to the effect of prediction error signals typical in dopamine-mediated striatal phasic activity. Lastly, reach vigor increased with reward history over trials, mirroring the motivational effects often linked to fluctuating dopamine levels. Taken together, our results demonstrate and exquisite link between known short-latency reward signals and the invigoration of both discrete and ongoing movements.

How the brain learns new motor commands through reinforcement involves distributed neural circuits beyond known frontal-striatal pathways, yet a comprehensive understanding of this broader neural architecture remains elusive. Here, using human functional MRI (N = 46, 27 females) and manifold learning techniques, we identified a low-dimensional neural space that captured the dynamic changes in whole-brain functional organization during a reward-based trajectory learning task. By quantifying participants' learning rates through an Actor-Critic model, we discovered that periods of accelerated learning were characterized by significant manifold contractions across multiple brain regions, including areas of limbic and hippocampal cortex, as well as the cerebellum. This contraction reflected enhanced network integration, with notably stronger connectivity between several of these regions and the sensorimotor cerebellum correlating with higher learning rates. These findings challenge the traditional view of the cerebellum as solely involved in error-based learning, supporting the emerging view that it coordinates with other brain regions during reinforcement learning.Significance Statement This study reveals how distributed brain systems, including the cerebellum and hippocampus, alter their functional connectivity to support motor learning through reinforcement. Using advanced manifold learning techniques on functional MRI data, we examined changes in regional connectivity during reward-based learning and their relationship to learning rate. For several brain regions, we found that periods of heightened learning were associated with increased cerebellar connectivity, suggesting a key role for the cerebellum in reward-based motor learning. These findings challenge the traditional view of the cerebellum as solely involved in supervised (error-based) learning and add to a growing rodent literature supporting a role for cerebellar circuits in reward-driven learning.

Across development, children must learn motor skills such as eating with a spoon and drawing with a crayon. Reinforcement learning, driven by success and failure, is fundamental to such sensorimotor learning. It typically requires a child to explore movement options along a continuum (grip location on a crayon) and learn from probabilistic rewards (whether the crayon draws or breaks). Here, we studied the development of reinforcement motor learning using online motor tasks to engage children aged 3 to 17 years and adults (cross-sectional sample, N=385). Participants moved a cartoon penguin across a scene and were rewarded (animated cartoon clip) based on their final movement position. Learning followed a clear developmental trajectory when participants could choose to move anywhere along a continuum and the reward probability depended on the final movement position. Learning was incomplete or absent in 3 to 8-year-olds and gradually improved to adult-like levels by adolescence. A reinforcement learning model fit to each participant identified two age-dependent factors underlying improvement across development: an increasing amount of exploration after a failed movement and a decreasing level of motor noise. We predicted, and confirmed, that switching to discrete targets and deterministic reward would improve 3 to 8-year-olds' learning to adult-like levels by increasing exploration after failed movements. Overall, we show a robust developmental trajectory of reinforcement motor learning abilities under ecologically relevant conditions i.e., continuous movement options mapped to probabilistic reward. This learning may be limited by immature spatial processing and probabilistic reasoning abilities in young children and can be rescued by reducing task demands.

Despite the impressive performance of biological and artificial networks, an intuitive understanding of how their local learning dynamics contribute to network-level task solutions remains a challenge to this date. Efforts to bring learning to a more local scale indeed lead to valuable insights, however, a general constructive approach to describe local learning goals that is both interpretable and adaptable across diverse tasks is still missing. We have previously formulated a local information processing goal that is highly adaptable and interpretable for a model neuron with compartmental structure. Building on recent advances in Partial Information Decomposition (PID), we here derive a corresponding parametric local learning rule, which allows us to introduce "infomorphic" neural networks. We demonstrate the versatility of these networks to perform tasks from supervised, unsupervised, and memory learning. By leveraging the interpretable nature of the PID framework, infomorphic networks represent a valuable tool to advance our understanding of the intricate structure of local learning.

Timed dopamine signals underlie reinforcement learning, favoring neural activity patterns that drive behaviors with positive outcomes. In the striatum, dopamine activates five dopamine receptors (D1R-D5R), which are differentially expressed in striatal neurons. However, the role of specific dopamine receptors in reinforcement is poorly understood. Using our cell-specific D1R photo-agonist, we find that D1R activation in D1-expressing neurons in the dorsomedial striatum is sufficient to reinforce preceding neural firing patterns in defined ensembles of layer 5 cortico-striatal neurons of the mouse motor cortex. The reinforcement is cumulative and time dependent, with an optimal effect when D1R activation follows the selected neural pattern after a short interval. Our results show that D1R activation in striatal neurons can selectively reinforce cortical activity patterns, independent of a behavioral outcome or a reward, crucially contributing to the fundamental mechanisms that support cognitive functions like learning, memory, and decision-making.

Mesolimbic dopamine encoding of non-contingent rewards and reward-predictive cues has been well established. Considerable debate remains over how mesolimbic dopamine responds to aversion and in the context of aversive conditioning. Inconsistencies may arise from the use of aversive stimuli that are transduced along different neural paths relative to reward or the conflation of responses to avoidance and aversion. Here, we made intraoral infusions of sucrose and measured how dopamine and behavioral responses varied to the changing valence of sucrose. Pairing intraoral sucrose with malaise via injection of lithium chloride (LiCl) caused the development of a conditioned taste aversion (CTA), which rendered the typically rewarding taste of sucrose aversive upon subsequent re-exposure. Following CTA formation, intraoral sucrose suppressed the activity of ventral tegmental area dopamine neurons (VTADA) and nucleus accumbens (NAc) dopamine release. This pattern of dopamine signaling after CTA is similar to intraoral infusions of innately aversive quinine and contrasts with responses to sucrose when it was novel or not paired with LiCl. Dopamine responses were negatively correlated with behavioral reactivity to intraoral sucrose and predicted home cage sucrose preference. Further, dopamine responses scaled with the strength of the CTA, which was increased by repeated LiCl pairings and weakened through extinction. Thus, the findings demonstrate differential dopamine encoding of the same taste stimulus according to its valence, which is aligned to distinct behavioral responses.

Ventral tegmental area (VTA) dopamine neurons are of great interest for their central roles in motivation, learning, and psychiatric disorders. While hypotheses of VTA dopamine neuron function posit a homogenous role in behavior (e.g., prediction error), they do not account for molecular heterogeneity. We find that glutamate-dopamine, nonglutamate-dopamine, and glutamate-only neurons are dissociable in their signaling of reward and aversion-related stimuli, prediction error, and electrical properties. In addition, glutamate-dopamine and nonglutamate-dopamine neurons differ in dopamine release dynamics. Aversion-related recordings of all dopamine neurons (not considering glutamate co-transmission) showed a mixed response that obscured dopamine subpopulation function. Within glutamate-dopamine neurons, glutamate and dopamine release had dissociable contributions toward reward and aversion-based learning and performance. Based on our results, we propose a new hypothesis on VTA dopamine neuron function: that dopamine neuron signaling patterns and their roles in motivated behavior depend on whether or not they co-transmit dopamine with glutamate.

The ventral tegmental area (VTA), a midbrain region associated with motivated behaviors, contains mostly dopaminergic (DA) neurons and GABAergic (GABA) neurons. Previous work has suggested that VTA GABA neurons provide a reward prediction signal, which is used in computing a reward prediction error. In this study, by using in vivo electrophysiology and continuous quantification of force exertion in head-fixed mice, we identify distinct populations of VTA GABA neurons that exhibit precise force tuning independently of learning, reward prediction, and outcome valence. Their activity usually precedes force exertion, and selective optogenetic manipulations of these neurons systematically modulate force exertion without influencing reward prediction. Together, these findings show that VTA GABA neurons can continuously regulate force vectors during motivated behavior.

Automated touchscreen systems have become increasingly prevalent in rodent model screening. This technology has significantly enhanced cognitive and behavioral assessments in mice and has bridged the translational gap between basic research using rodent models and human clinical research. Our study introduces a custom-built touchscreen operant conditioning chamber powered by a Raspberry Pi and a commercially available computer tablet, which effectively addresses the significant cost barriers traditionally associated with this technology. In order to test our prototype, we decided to train C57BL/6 mice on a visual discrimination serial-reversal task, and both C57BL/6 and AppNL-G-Fstrain - an Alzheimer's Disease (AD) mouse model - on a new location discrimination serial-reversal task. The results demonstrated a clear progression toward asymptotic performance, particularly in the location discrimination task, which also revealed potential genotype-specific deficits, with AppNL-G-F mice displaying an increase in the average number of errors in the first reversal as well as in perseverative errors, compared to wild-type mice. These results validate the practical utility of our touchscreen apparatus and underline its potential to provide insights into the behavioral and cognitive markers of neurobiological disorders.

Elucidation of transphasic mechanisms (i.e., mechanisms that occur across illness phases) underlying negative symptoms could inform early intervention and prevention efforts and additionally identify treatment targets that could be effective regardless of illness stage. This study examined whether a key reinforcement learning behavioral pattern characterized by reduced difficulty learning from rewards that have been found to underlie negative symptoms in those with a schizophrenia diagnosis also contributes to negative symptoms in those at clinical high-risk (CHR) for psychosis.

CHR youth (n = 46) and 51 healthy controls (CN) completed an explicit reinforcement learning task with two phases. During the acquisition phase, participants learned to select between pairs of stimuli probabilistically reinforced with feedback indicating receipt of monetary gains or avoidance of losses. Following training, the transfer phase required participants to select between pairs of previously presented stimuli during the acquisition phase and novel stimuli without receiving feedback. These test phase pairings allowed for inferences about the contributions of prediction error and value representation mechanisms to reinforcement learning deficits.

In acquisition, CHR participants displayed impaired learning from gains specifically that were associated with greater negative symptom severity. Transfer performance indicated these acquisition deficits were largely driven by value representation deficits. In addition to negative symptoms, this profile of deficits was associated with a greater risk of conversion to psychosis and lower functioning.

Impairments in positive reinforcement learning, specifically effectively representing reward value, may be an important transphasic mechanism of negative symptoms and a marker of psychosis liability.

Midbrain dopamine cells encode differences in predictive and expected value to support learning through reward prediction error. Recent findings have questioned whether reward prediction error can fully account for dopamine function and suggest a more complex role for dopamine in encoding detailed features of the reward environment. In this series of studies, we describe a novel role for dopamine in devaluing sensory features of reward. Mesencephalic dopamine cells activated during a mediated devaluation phase were later chemogenetically reactivated. This retrieval of the devalued reward memory elicited a reduction in the hedonic evaluation of sucrose reward. Through optogenetic and chemogenetic manipulations, we confirm dopamine cells are both sufficient and necessary for mediated devaluation, and retrieval of these memories reflected dopamine release in the nucleus accumbens. Consistent with our computational modeling data, our findings indicate a critical role for dopamine in encoding predictive representations of the sensory features of reinforcement. Overall, we elucidate a novel role for dopamine function in mediated devaluation and illuminate a more elaborate framework through which dopamine encodes reinforcement signals.

Despite decades of research with laboratory rodents, the mechanisms underlying learning and memory, and their impairment, are still not fully understood. The zebrafish is a newcomer in this research area, but it has shown great promise. Food is often employed as a reinforcer in learning tasks with rodents. However, for zebrafish, food has been a problematic reinforcer. Controlling timing and localization of its delivery is difficult. What food types zebrafish prefer is also rarely studied? Here, we describe a novel food delivery hardware and procedure. The apparatus is simple, cheap to manufacture, and easy to employ. Using this new method, we compare how zebrafish respond to three food types, artemia nauplii, crushed tropical fish flakes, and small zebrafish pellets. In binary choice tasks, we show that zebrafish spend significantly more time near the artemia delivery cylinder, swim closer to, and visit this cylinder more frequently compared to food cylinders delivering flakes or pellets, while responses to these latter two cylinders do not differ from each other. We conclude that the newly developed method allows the quantification of food preference in zebrafish, and that it will lead to the identification of highly rewarding food types for learning studies in this species.

Incentive salience theory both explains the directional component of motivation (in terms of cue attraction or "wanting") and its energetic component, as a function of the strength of cue attraction. This theory characterizes cue- and reward-triggered approach behavior. But it does not tell us how behavior can show enhanced vigor under reward uncertainty, when cues are inconsistent or resources hidden. Reinforcement theory is also ineffective in explaining enhanced vigor in case reward expectation is low or nil. This paper provides a neurobehavioral interpretation of effort in situations of adversity (which always include some uncertainty about outcomes) that is complementary to the attribution of incentive salience to environmental cues. It is argued that manageable environmental challenges activate an unconscious process of self-determination to achieve "wanted" actions. This unconscious process is referred to as incentive effort, which involves the hypothalamo-pituitary-adrenal (HPA) axis, noradrenaline, as well as striatal dopamine. Concretely, HPA-induced dopamine release would have the function to make effort-or effortful actions-"wanted" in a challenging context, in which the environmental cues are poorly predictive of reward-i.e., unattractive. Stress would only emerge in the presence of unmanageable challenges. It is hypothesized that incentive effort is the core psychological basis of will-and is, for this reason, termed "willing."

This chapter delves into the complex interplay among addiction, stress, and the reward pathways in the brain, emphasizing the ways in which various drugs affect these systems and exacerbate SUD. Drugs have physiological effects that can be both pleasurable and unpleasant. These effects change behavior through both positive and negative reinforcement. A person's genetic predisposition to addiction is mostly determined by factors such as biological sex, age of first usage, and dopamine receptor density. Drug use behaviors are also greatly influenced by environmental stressors, media exposure, and substance accessibility; nevertheless, protective variables including social support, participation in healthy activities, and preventative programs serve to reduce the dangers associated with drug use. The reinforcement of addictive behaviors is mostly dependent on the brain's reward circuits, which include the nucleus accumbens, ventral tegmental region, and prefrontal cortex, in addition to neurotransmitters such as dopamine, serotonin, and endorphins. Stress makes addiction worse by intensifying cravings and raising the possibility of relapsing. Examined are the impacts of several drug types, such as opioids, stimulants, depressants, and hallucinogens, emphasizing the long-term consequences on brain function and susceptibility to addiction. In order to create individualized interventions that target the environmental and neurological components of addiction and eventually improve treatment results, a thorough understanding of these elements is important.

Conventional artificial intelligence (AI) systems are facing bottlenecks due to the fundamental mismatches between AI models, which rely on parallel, in-memory, and dynamic computation, and traditional transistors, which have been designed and optimized for sequential logic operations. This calls for the development of novel computing units beyond transistors. Inspired by the high efficiency and adaptability of biological neural networks, computing systems mimicking the capabilities of biological structures are gaining more attention. Ion-based memristive devices (IMDs), owing to the intrinsic functional similarities to their biological counterparts, hold significant promise for implementing emerging neuromorphic learning and computing algorithms. In this article, we review the fundamental mechanisms of IMDs based on ion drift and diffusion to elucidate the origins of their diverse dynamics. We then examine how these mechanisms operate within different materials to enable IMDs with various types of switching behaviors, leading to a wide range of applications, from emulating biological components to realizing specialized computing requirements. Furthermore, we explore the potential for IMDs to be modified and tuned to achieve customized dynamics, which positions them as one of the most promising hardware candidates for executing bioinspired algorithms with unique specifications. Finally, we identify the challenges currently facing IMDs that hinder their widespread usage and highlight emerging research directions that could significantly benefit from incorporating IMDs.

Recent breakthroughs in brain-inspired computing promise to address a wide range of problems from security to healthcare. However, the current strategy of implementing artificial intelligence algorithms using conventional silicon hardware is leading to unsustainable energy consumption. Neuromorphic hardware based on electronic devices mimicking biological systems is emerging as a low-energy alternative, although further progress requires materials that can mimic biological function while maintaining scalability and speed. As a result of their diverse unique properties, atomically thin two-dimensional (2D) materials are promising building blocks for next-generation electronics including nonvolatile memory, in-memory and neuromorphic computing, and flexible edge-computing systems. Furthermore, 2D materials achieve biorealistic synaptic and neuronal responses that extend beyond conventional logic and memory systems. Here, we provide a comprehensive review of the growth, fabrication, and integration of 2D materials and van der Waals heterojunctions for neuromorphic electronic and optoelectronic devices, circuits, and systems. For each case, the relationship between physical properties and device responses is emphasized followed by a critical comparison of technologies for different applications. We conclude with a forward-looking perspective on the key remaining challenges and opportunities for neuromorphic applications that leverage the fundamental properties of 2D materials and heterojunctions.

Humans and animals excel at learning complex tasks through reward-based feedback, dynamically adjusting value expectations and choices based on past experiences to optimize outcomes. However, understanding the hidden cognitive components driving these behaviors remains challenging. Neuroscientists use the Temporal Difference (TD) learning model to estimate cognitive elements like value representation and prediction error during learning and decision-making processes. However, traditional TD algorithms fall short in diverse and dynamic tasks due to their fixed patterns. We present PyTDL, a Python-based modular framework that enables customizable value updating functions and decision policies, effectively simulating dynamic, non-linear cognitive processes. PyTDL's utility was demonstrated by modeling the decision-making processes of animals in two cognitive tasks under uncertain conditions. As open-source software, PyTDL offers a user-friendly GUI and APIs, empowering researchers to tailor models for specific tasks, align computational models with empirical data, and advance the understanding of brain learning and decision-making in complex environments.

Learning whom to trust is integral for healthy relationships and social cohesion, and atypicalities in trust learning are common across a range of clinical conditions, including schizophrenia spectrum disorders, Parkinson's disease, and depression. Persecutory delusions - rigid, unfounded beliefs that others are intending to harm oneself - significantly impact affected individuals' lives as they are associated with a range of negative health outcomes, including suicidal behaviour and relapse. Recent advances in computational modelling and psychopharmacology have significantly extended our understanding of the brain bases of dynamic trust learning, and the neuromodulator dopamine has been suggested to play a key role in this. However, the specifics of this role on a computational and neurobiological level remain to be fully established. The current review article provides a comprehensive summary of novel conceptual developments and empirical findings regarding the computational role of dopamine in social learning processes. Research findings strongly suggest a conceptual shift, from dopamine as a reward mechanism to a teaching signal indicating which information is relevant for learning, and shed light on the neurocomputational mechanisms via which antipsychotics may alleviate symptoms of aberrant social learning processes such as persecutory delusions. Knowledge gaps and inconsistencies in the extant literature are examined and the most pressing issues highlighted, laying the foundation for future research that will further advance our understanding of the neuromodulation of social belief updating processes.

Reward and voluntary choice facilitate motor skill learning through motivation. However, it remains unclear how their combination influences motor skill learning. The purpose of the present study is to investigate the effects of reward and voluntary choice on motor skill learning in a serial reaction time task (SRTT).

Participants completed six parts of SRTT, including pre-test, training phase, immediate post-test, a random session, delayed post-test, and retention test on the following day. During the training phase, participants were divided into four groups (reward_choice, reward_no-choice, no-reward_choice, no-reward_no-choice). In the reward condition, participants received reward for correct and faster (than a baseline) responses while those in the no-reward groups did not. For the choice manipulation, participants in the voluntary choice groups chose the color of the target, whereas in the forced choice groups, the same color was assigned by the computer.

The results showed that the four groups did not exhibit any significant differences in reaction time and error rate in the pre-test phase. Importantly, both reward and voluntary choice significantly enhanced sequence-specific learning effects, while no interaction was found. No significant effects of reward and voluntary choice were observed in the retention test.

These findings suggest that reward and voluntary choice enhance motor skill performance and training independently, potentially at the action-selection level, which implies different mechanisms underlying the influences of reward and voluntary choice.

The neurobiological mechanisms underlying the placebo phenomenon in patients with major depressive disorder (MDD) remain largely unknown. The progressive rise in rates of placebo responses within clinical trials over the past two decades may impede the detection of a true signal and thus present a major obstacle in new treatment development. Understanding the mechanisms would have several important implications, including (1) identifying biomarkers of placebo responders (thereby identifying those individuals who could benefit therapeutically from such interventions), (2) opening new avenues for manipulating such mechanisms to maximize symptom reduction, and (3) refining treatments with approaches that decrease (in clinical trials) or increase (in clinical practice) the placebo response. Here we investigated the research question: is the dopaminergic system one of the neurobiological underpinnings of the placebo response within MDD? Inspired by preclinical and clinical findings that have implicated dopamine in the occurrence, prediction, and expectation of reward, we hypothesized that dopaminergic activity in the mesolimbic system is a critical mediator of placebo response in MDD. To test this hypothesis, we designed a double-blind, placebo-controlled, sequential parallel comparison design clinical trial aimed at maximizing placebo antidepressant response. We integrated behavioral, imaging, and hemodynamic probes of mesocorticolimbic dopaminergic pathways within the context of manipulations of psychological constructs previously linked to placebo responses (e.g., expectation of improvement). The aim of this manuscript is to present the rationale of the study design and to demonstrate how a cross-modal methodology may be utilized to investigate the role of reward circuitry in placebo response in MDD.

Consuming opioid agonists is a risk factor for cardiovascular disease particularly in intravenous heroin users. The monthly injectable extended-release opioid antagonist, naltrexone (XR-NTX) is an effective treatment for opioid use disorder. The impact of opioid receptor blockade through XR-NTX on blood pressure, a critical risk factor for cardiovascular morbidity, has not yet been characterized.

The study evaluated the change in blood pressure during XR-NTX treatment among 14 patients who predominately used intravenous heroin and 24 patients who used prescription oral opioids, all with opioid use disorder. Blood pressure was measured in each patient immediately before the first XR-NTX injection and ∼two weeks after the first injection. The change in diastolic and systolic pressure was compared between the heroin users and the prescription opioids users using analysis of variance.

XR-NTX treatment was associated with significant decreases in diastolic blood pressure in the heroin group, but not in the prescription opioids group. Systolic blood pressure values in the heroin users showed a decline at trend level only.

Further research is warranted to replicate our findings and to determine whether XR-NTX effect is relatively specific to blood pressure or generalizes to other components of metabolic syndrome. Distinguishing between heroin and prescription opioid users could shed light on the unique clinical and pharmacological profiles of opioid drugs, particularly regarding their cardiovascular safety. This information can be useful in developing personalized therapeutic strategies based on the route of opioid administration.

Food pleasantness is largely based on the palatability of food and is linked to taste. Along with homeostatic and cognitive control, it forms part of the control of food intake (hedonic control), and does not only correspond to the pleasure that can be described of food intake. There are many factors that cause variations in eating pleasantness between individuals, such as age, sex, culture, co-morbidities, treatments, environmental factors or the specific characteristics of foods. The control of food intake is based on four determinants: conditioned satiety, the reward system, sensory specific satiety and alliesthesia. These four determinants follow one another over time, in the per-prandial and inter-prandial periods, and complement one another. There are many cerebral areas involved in the hedonic control of food intake. The most involved brain areas are the orbitofrontal and anterior cingulate cortices, which interact with deep neural structures (amygdala, striatum, substantia nigra) for the reward circuit, with the hippocampi for memorising pleasant foods, and even with the hypothalamus and insula, brain areas more recently involved in the physiology of food pleasantness. Changes in brain activity secondary to modulation of food pleasantness can be measured objectively by recording taste-evoked potentials, an electroencephalography technique with very good temporal resolution.

Robotic systems rely on spatio-temporal information to solve control tasks. With advancements in deep neural networks, reinforcement learning has significantly enhanced the performance of control tasks by leveraging deep learning techniques. However, as deep neural networks grow in complexity, they consume more energy and introduce greater latency. This complexity hampers their application in robotic systems that require real-time data processing. To address this issue, spiking neural networks, which emulate the biological brain by transmitting spatio-temporal information through spikes, have been developed alongside neuromorphic hardware that supports their operation. This paper reviews brain-inspired learning rules and examines the application of spiking neural networks in control tasks. We begin by exploring the features and implementations of biologically plausible spike-timing-dependent plasticity. Subsequently, we investigate the integration of a global third factor with spike-timing-dependent plasticity and its utilization and enhancements in both theoretical and applied research. We also discuss a method for locally applying a third factor that sophisticatedly modifies each synaptic weight through weight-based backpropagation. Finally, we review studies utilizing these learning rules to solve control tasks using spiking neural networks.

Subjective value is a core concept in neuroeconomics, serving as the basis for decision making. Despite the extensive literature on the neural encoding of subjective reward value in humans, the neural representation of punishment value remains relatively understudied. This review synthesizes current knowledge on the neural representation of reward value, including methodologies, involved brain regions, and the concept of a common currency representation of diverse reward types in decision-making and learning processes. We then critically examine existing research on the neural representation of punishment value, highlighting conceptual and methodological challenges in human studies and insights gained from animal research. Finally, we explore how individual differences in reward and punishment processing may be linked to various mental illnesses, with a focus on stress-related psychopathologies. This review advocates for the integration of both rewards and punishments within value-based decision-making and learning frameworks, leveraging insights from cross-species studies and utilizing ecological gamified paradigms to reflect real-life scenarios.

This paper presents a general model of the cognitive processes involved in each play situation of soccer at the elite level. Theoretically the model draws on general frameworks from cognitive psychology and neuroscience, in particular the affordance competition hypothesis and the reward prediction error theory. The model includes three functional stages: situational assessment, action selection and execution, and outcome assessment. The three stages form a perception-action cycle that corresponds to a single play situation. The cognitive processes operating at each functional stage are described and related to soccer research by a review of 52 empirical studies. The review covers the main cognitive processes that have been studied in soccer research: visual orientation and attention, pattern recognition, anticipation, working memory, action selection and decision making, executive control processes, as well as behavioral and cognitive learning. The model accommodates the wide variety of findings in the empirical literature and provides a general organizing frame for cognitive soccer research at the elite level. The influence of emotional and stress-related factors on cognition are also discussed. Four general limitations of the existing soccer research are identified, and suggestions for future studies include development of more naturalistic and interventional study designs. By specifying the different cognitive processes in soccer and their dynamic interactions the model has many applied perspectives for soccer training at the professional level. Overall, the paper presents the first integrated process model of cognition in elite soccer players with implications for both research and practice.

The ventral tegmental area (VTA), a midbrain region associated with motivated behaviors, consists predominantly of dopaminergic (DA) neurons and GABAergic (GABA) neurons. Previous work has suggested that VTA GABA neurons provide a reward prediction, which is used in computing a reward prediction error. In this study, using in vivo electrophysiology and continuous quantification of force exertion in head-fixed mice, we discovered distinct populations of VTA GABA neurons that exhibited precise force tuning independently of learning, reward prediction, and outcome valence. Their activity usually preceded force exertion, and selective optogenetic manipulations of these neurons systematically modulated force exertion without influencing reward prediction. Together, these findings show that VTA GABA neurons continuously regulate force vectors during motivated behavior.

Learning is classically modeled to consist of an acquisition period followed by a mastery period when the skill no longer requires conscious control and becomes automatic. Dopamine neurons projecting to the ventral striatum (VS) produce a teaching signal that shifts from responding to rewarding or aversive events to anticipating cues, thus facilitating learning. However, the role of the dopamine-receptive neurons in the ventral striatum, particularly in encoding decision-making processes, remains less understood.

Here, we introduce an operant conditioning paradigm using open-source microcontrollers to train mice in three sequential learning phases. Phase I employs classical conditioning, associating a 5 s sound cue (CS) with a sucrose-water reward. In Phase II, the CS is replaced by a lever press as the requirement for reward delivery, marking an operant conditioning stage. Phase III combines these elements, requiring mice to press the lever during the CS to obtain the reward. We recorded calcium signals from direct pathway spiny projection neurons (dSPNs) in the VS throughout the three phases of training.

We find that dSPNs are specifically engaged when the mouse makes a decision to perform a reward-seeking action in response to a CS but are largely inactive during actions taken outside the CS.

These findings suggest that direct pathway neurons in the VS contribute to decision-making in learned action-outcome associations, indicating a specialized role in initiating operant behaviors.

Asymmetry in choice patterns across rewarding and punishing contexts has long been observed in behavioural economics. Within existing theories of reinforcement learning, the mechanistic account of these behavioural differences is still debated. We propose that motivational salience-the degree of bottom-up attention attracted by a stimulus with relation to motivational goals-offers a potential mechanism to modulate stimulus value updating and decision policy. In a probabilistic reversal learning task, we identified post-feedback signals from EEG and pupillometry that captured differential activity with respect to rewarding and punishing contexts. We show that the degree of between-context distinction in these signals predicts interindividual asymmetries in decision accuracy. Finally, we contextualise these effects in relation to the neural pathways that are currently centred in theories of reward and punishment learning, demonstrating how the motivational salience network could plausibly fit into a range of existing frameworks.

Attentional biases towards food play an important role in the pathology of binge eating disorder (BED). Later stage electrophysiological potentials (P300, late positive potential) present promising markers of motivated attention with high temporal, albeit low spatial resolution. Complementing this, the N2pc is an earlier-latency component providing the possibility of more directly analyzing visuospatial attention. Therefore, we tested a group with BED (N = 60), as well as an overweight (OW; N = 28) and normal weight (NW; N = 30) group without BED in a Go/No-Go paradigm using food and nonfood distractor images. Only the OW group in exclusively the Go trials displayed a stronger spatial attention allocation towards nonfood distractors as evidenced by an increased N2pc amplitude. In the P300's time window, the OW group displayed no attentional bias towards food and the NW group only did so in the absence of a target. Solely the BED group allocated more motivated attention towards food distractors both in Go and No-Go trials. In the following late positive potential (LPP), the OW group exhibited a general attentional bias towards food distractors, while the BED group only did so in the absence of a target. These results are discussed in light of the incentive sensitization theory and a potential early attentional suppression of potent distractors.