Browsing by Subject "Reward"

Humans make decisions in highly complex physical, economic and social environments. In order to adaptively choose, the human brain has to learn about- and attend to- sensory cues that provide information about the potential outcome of different courses of action. Here I present three event-related potential (ERP) studies, in which I evaluated the role of the interactions between attention and reward learning in economic decision-making. I focused my analyses on three ERP components (Chap. 1): (1) the N2pc, an early lateralized ERP response reflecting the lateralized focus of visual; (2) the feedback-related negativity (FRN), which reflects the process by which the brain extracts utility from feedback; and (3) the P300 (P3), which reflects the amount of attention devoted to feedback-processing. I found that learned stimulus-reward associations can influence the rapid allocation of attention (N2pc) towards outcome-predicting cues, and that differences in this attention allocation process are associated with individual differences in economic decision performance (Chap. 2). Such individual differences were also linked to differences in neural responses reflecting the amount of attention devoted to processing monetary outcomes (P3) (Chap. 3). Finally, the relative amount of attention devoted to processing rewards for oneself versus others (as reflected by the P3) predicted both charitable giving and self-reported engagement in real-life altruistic behaviors across individuals (Chap. 4). Overall, these findings indicate that attention and reward processing interact and can influence each other in the brain. Moreover, they indicate that individual differences in economic choice behavior are associated both with biases in the manner in which attention is drawn towards sensory cues that inform subsequent choices, and with biases in the way that attention is allocated to learn from the outcomes of recent choices.

The brain’s dopaminergic system is critical to adaptive behaviors, and is centrally implicated in various pathologies. For decades, research has aimed at better characterizing what drives the mesolimbic dopamine system and the resulting influence on brain physiology and behavior in both humans and animals. To date, the dominant modes of research have relied on extrinsic approaches: pharmacological manipulations, direct brain stimulation, or delivering behavioral incentives in laboratory tasks. A critical open question concerns whether individuals can modulate activation within this system volitionally. That is, can individuals use self-generated thoughts and imagery to invigorate this system on their own? This process can be referred to as “cognitive neurostimulation” -- a precise and non-invasive stimulation of neural systems via cognitive and behavioral strategies. And if not, can they be taught to do so? Recent technological advances make it feasible to present human participants with information about ongoing neural activations in a fast and spatially precise manner. Such feedback signals might enable individuals to eventually learn to control neural systems via fine-tuning of behavioral strategies. The studies described herein investigate whether individuals can learn to volitionally invigorate activation within the mesolimbic reward network. We demonstrate that under the right training context, individuals can successfully learn to generate cognitive states that elicit and sustain activation in the ventral tegmental area (VTA), the source of dopamine production within the mesolimbic network. Although participants were explicitly trained to increase VTA activation, multiple mesolimbic regions exhibited increased connectivity during and after training. Together, these findings suggest new frameworks for aligning psychological and biological perspectives, and for understanding and harnessing the power of neuromodulatory systems.

Memories are not veridical representations of the environment. Rather, an individual's goals can influence how the surrounding environment is represented in long-term memory. The present dissertation aims to delineate the influence of reward and punishment motivation on human declarative memory encoding and its underlying neural circuitry. Chapter 1 provides a theoretical framework for investigating motivation's influence on declarative memory. This chapter will review the animal and human literatures on declarative memory encoding, reward and punishment motivation, and motivation's influence on learning and memory. Chapter 2 presents a study examining the behavioral effects of reward and punishment motivation on declarative memory encoding. Chapter 3 presents a study examining the neural circuitry underlying punishment-motivated declarative encoding using functional magnetic resonance imaging (fMRI), and compares these findings to previous studies of reward-motivated declarative encoding. Chapter 4 presents a study examining the influence of reward and punishment motivation on neural sensitivity to and declarative memory for unexpected events encountered during goal pursuit using fMRI. Finally, Chapter 5 synthesizes these results and proposes a model of how and why motivational valence has distinct influences on declarative memory encoding. Results indicated that behaviorally, reward motivation resulted in more enriched representations of the environment compared to punishment motivation. Neurally, these motivational states engaged distinct neuromodulatory systems and medial temporal lobe (MTL) targets during encoding. Specifically, results indicated that reward motivation supports interactions between the ventral tegmental area and the hippocampus, whereas, punishment motivation supports interactions between the amygdala and parahippocampal cortex. Together, these findings suggest that reward and punishment engage distinct systems of encoding and result in the storage of qualitatively different representations of the environment into long-term memory.

Emotions guide the way individuals interact with the world, influencing nearly every psychological process from attention, to learning, to metacognition. Constructionist models of emotion posit that emotions arise out of combinations of more general psychological ingredients. These psychological ingredients, however, also form the building blocks of other affective responses such as subjective reactions to rewarding and social stimuli. Here, I propose a domain-general account of affective functioning; I contend that subjective responses to emotional, rewarding, and social stimuli all depend on common psychological and neural mechanisms. I support this hypothesis with three independent studies using both a basic science approach and a clinical approach. In the first study (Chapter 2) I demonstrate that the ventromedial prefrontal cortex (vmPFC), which has been implicated in encoding the value of primary, monetary, and social rewards, also encodes the hedonic value of emotional stimuli. In addition to showing that the mechanisms responsible for processing affective information are shared across reward and emotional processing, I also discuss the relevance of a domain-general constructionist account of affect for clinical disorders. In particular, I hypothesize that in anorexia nervosa (AN), affective disturbances should be manifest across responses to emotional, rewarding, and social stimuli (Chapter 3). In Chapter 4, I provide empirical evidence for this conclusion by demonstrating that when viewing social stimuli, women with a history of AN show disturbances in the insula, a brain region that is responsible for interoceptive and affective processing. This suggests that the interpersonal difficulties frequently observed in patients with AN may be due to biases in domain-general affective responses. In Chapter 5, I support this conclusion by showing that individual differences in harm avoidance in healthy women, women with a current diagnosis of AN, and women who have recovered from AN explain the relationship between disordered eating and social dysfunction. Collectively, these results indicate that subjective affective responses to rewarding, emotional, and social information all rely on common mechanisms as would be suggested by a domain-general theory of affect. Furthermore, the application of a constructionist domain-general account of affect can help to explain the fundamental nature of affective disturbances in psychiatric disorders such as AN.

Background

Methodology/principal findings

Conclusions/significance

Few aspects of human cognition are more personal than the choices we make. Our decisions — from the mundane to the impossibly complex — continually shape the courses of our lives. In recent years, researchers have applied the tools of neuroscience to understand the mechanisms that underlie decision making, as part of the new discipline of decision neuroscience. A primary goal of this emerging field has been to identify the processes that underlie specific decision variables, including the value of rewards, the uncertainty associated with particular outcomes, and the consequences of social interactions. Here, across three independent studies, I focus on the neural circuitry supporting social valuation — which shapes our social interactions and interpersonal choices. In the first study (Chapter 2), I demonstrate that social valuation relies on the posterior ventromedial prefrontal cortex (pVMPFC). Extending these findings, I next show that idiosyncratic responses within pVMPFC predict individual differences in complex social decision scenarios (Chapter 3). In addition, I also demonstrate that decisions involving other people (e.g., donations to a charitable organization) produce increased activation in brain regions associated with social cognition, particularly the temporal-parietal junction (TPJ). Finally, in my last study (Chapter 4), I employ functional connectivity analyses and show that social cognition regions — including the TPJ — exhibit increased connectivity with pVMPFC during social valuation, an effect that depends upon individual differences in preferences for social stimuli. Collectively, these results demonstrate that the computation of social value relies on distributed neural circuitry, including both value regions and social cognition regions. Future research on social valuation and interpersonal choice must build upon this emerging theme by linking neural circuits and behavior.

An animal's ability to make adaptive choices is key to its fitness. Thus, the process of determining options, making a decision, evaluating outcomes, and learning from those outcomes to adjust future behavior is a central function of our nervous system. Determining the neural mechanisms of these cognitive processes is a crucial goal. One brain region, the posterior cingulate cortex (CGp), a central hub within the default mode network, is prominently dysregulated in Alzheimer's Disease and schizophrenia. Despite its clinical importance, the posterior cingulate cortex remains an enigmatic nexus of attention, memory, and motivation, all pointing to a role in decision-making. This dissertation is concerned with the role of this brain region in the learning and valuation processes involved in making adaptive choices. Specifically, I used rhesus macaques (Macaca mulatta) to examine the neural activity in posterior cingulate associated with specific learning and valuation -related variables. In the first experiment, I showed that posterior cingulate neurons track decision salience--the degree to which an option differs from a standard--but not the subjective value of a decision. To do this, I recorded the spiking activity of CGp neurons in monkeys choosing between options varying in reward-related risk, delay to reward, and social outcomes, each of which varied in level of decision salience. Firing rates were higher when monkeys chose the risky option, consistent with their risk-seeking preferences, but were also higher when monkeys chose the delayed and social options, contradicting their preferences. Thus, across decision contexts, neuronal activity was uncorrelated with how much monkeys valued a given option, as inferred from choice. Instead, neuronal activity signaled the deviation of the chosen option from the standard, independently of how it differed. The observed decision salience signals suggest a role for CGp in the flexible allocation of neural resources to motivationally significant information, akin to the role of attention in selective processing of sensory inputs. This pointed to a role for CGp in learning rather than subjective value signaling, and the second set of experiments aimed to test the role of CGp in associative learning. I recorded from single CGp neurons in monkeys performing a simple conditional motor association task while varying stimulus familiarity and motivation. CGp neurons responded phasically following commission of errors, and this error signal was modulated by motivation and stimulus novelty. Moreover, slow variations in firing rates tracked variations in learning rate over the course of sessions. Silencing these signals with muscimol impaired learning in low motivational states but spared learning in high motivational states, and spared recall of familiar associations as well. These findings endorse a role for CGp in performance and environment monitoring to regulate learning rate. Collectively, these experiments reshape our understanding of the role of posterior cingulate cortex in cognition, integrate default mode and value-based theories of CGp function, and provide a potential foundation for a circuit-level explication of Alzheimer's Disease and schizophrenia.

Humans must integrate information to make decisions. This thesis is concerned with studying neural mechanisms of decision making, and combines tools from economics, psychology, and neuroscience. I employ a neuroeconomic approach to understand the processing of reward information and motivation in the brain, utilizing neural data from functional magnetic resonance imaging (fMRI) to make connections between cognitive neuroscience and economics.

Chapter 1 lays the groundwork for the thesis and provides background on neuroscience, fMRI, and neuroeconomics. Chapter 2 sketches the central challenges of using neuroscience to address economic questions. The first half of the chapter discusses familiar arguments against the integration of neuroscience and economics: behavioral sufficiency and emergent phenomenon. The second half constructs principles for interdisciplinary research linking mechanistic (neuroscience) data to behavioral (economic) phenomena: mechanistic convergence across experiments and biological plausibility in models.

Chapters 3 and 4 employ a nonstandard analysis technique, multivariate pattern analysis (MVPA), to identify brain regions that contain information associated with different types of economic valuation. Chapter 3 uses a combinatoric approach to evaluate how brain regions uniquely contribute to the ability to predict different types of valuation (probabilistic or intertemporal). MVPA shows that early valuation phases for these rewards differ in posterior parietal cortex and suggests computational topographies for different rewards. Chapter 4 employs within- and cross-participant MVPA, which rely on potentially different sources of neural variability, to identify brain regions that contain information about monetary rewards (cash) and social rewards (images of faces). Cross-participant analyses reveal systematic changes in predictive power across multiple brain regions, and individual differences in statistical discriminability in ventromedial prefrontal cortex relate to differences in reward preferences. MVPA thus facilitates mapping behavior to both individual-specific functional organization and general organization of the brain across individuals.

Chapter 5 employs a reward anticipation task to measure variation in relative motivation without observing choices between rewards (money and candy). A reaction-time index captures individual differences in motivation, and heterogeneity in this index maps onto variability in two brain regions: nucleus accumbens and anterior insula. Further, the nucleus accumbens activation mediates the predictive effects of anterior insula. These results show that idiosyncrasies in reward efficacy persist in the absence of a choice environment.

Chapters 6 and 7 conclude the thesis. Chapter 6 complements discussions of neuroeconomics with text analysis of an exhaustive corpus from top economics journals and references from a large set of review articles. The analysis shows a mismatch between topics of importance to economics and prominent concepts in neuroeconomics. I show how neuroeconomics can grow by employing cognitive neuroscience to identify biologically plausible and generalizable models of a broader class of behaviors.

Perception arises from sensory inputs detected by peripheral organs and processed in the brain by complex neuronal circuits required for the integration of external information with internal states such as expectation and attention. Stimulus discrimination requires activation of primary sensory areas in the brain, but expectation is traditionally associated with the activation of higher-order brain areas. Sensory information obtained by tactile organs is represented along the primary areas that comprise the trigeminal thalamocortical pathway. In anesthetized animals, neuronal activity in the somatosensory system has been extensively described over the past century. However, it is still unclear how the different thalamocortical structures contribute to active tactile discrimination and represent relevant features of the stimulus. It is also unknown whether expectation modulates tactile representations in these regions. In this dissertation, I investigated neuronal ensemble activity recorded from freely behaving rats performing a whisker-based tactile discrimination t-+ask. Multielectrode arrays were chronically implanted to record simultaneously from the main stages of the trigeminal thalamocortical pathways involved in whisking: the primary somatosensory cortex (S1), the ventral posterior medial nucleus of the thalamus (VPM), the posterior medial complex (POm) and the zona incerta (ZI). In Chapter 1 I describe the behavior of rats performing the tactile discrimination task, which requires animals to associate two different tactile stimuli with two corresponding choices of spatial trajectory in order for reward to be delivered. I found that both cortical and thalamic neurons are dynamically engaged during execution of the task. The data reveal a very complex mosaic of responses comprising single or multiple periods of inhibition and excitation. Thalamocortical activity was modulated during whisker stimulation as well as after stimulus removal, up until reward delivery. To investigate whether reward expectation plays a role in tactile processing at early processing stages, I also recorded neuronal activity from rats performing a freely-rewarded version of the tactile discrimination task. Comparing data from regularly-rewarded and freely-rewarded sessions, I show in chapter 2 that the activity of single neurons in the primary somatosensory thalamocortical loop is strongly modulated by reward expectation. Stimulus-related information coded by primary thalamocortical neurons is high when a correct association between stimulus and response is crucial for reward, but decreases significantly when the association is irrelevant. These results indicate that tactile processing in primary somatosensory areas of the thalamus and cerebral cortex is directly affected by reward expectation.

Negotiating the complex decisions that we encounter daily requires coordinated neu-

ronal activity. The enormous variety of decisions we make, the intrinsic complexity

of the situations we encounter, and the extraordinary flexibility of our behaviors

suggest the existence of intricate neural mechanisms for negotiating contexts and

making choices. Further evidence for this prediction comes from the behavioral al-

terations observed in illness and after injury. Both clinical and scientific evidence

suggest that decision signals are carried by electrical neuronal activity and influenced

by neuromodulatory chemicals. This dissertation addresses the function of two puta-

tive contributors to decision-making: neuronal activity in posterior cingulate cortex

and modulatory effects of serotonin. I found that posterior cingulate neurons respond

phasically to salient events (informative cues; intentional saccades; and reward deliv-

ery) across multiple contexts. In addition, these neurons signal heuristically guided

choices across contexts in a gambling task. These observations suggest that posterior

cingulate neurons contribute to the detection and integration of salient information

necessary to transform event detection to expressed decisions. I also found that

lowering levels of the neuromodulator serotonin increased the probability of making

risky decisions in both monkeys and mice, suggesting that this neurotransmitter con-

tributes to preference formation across species. These results suggest that posterior

cingulate cortex and serotonin each contribute to decision formation. In addition, the

unique serotonergic pro jections to posterior cingulate cortex, as well as the frequent

implication of altered serotonergic and posterior cingulate function in psychiatric dis-

orders, suggest that the confluence of cingulate and serotonergic activity may offer

key insights into normal and pathological mechanisms of decision making.

Neural activities of the motor cortices have been traditionally known to represent motor information such as velocity of the movement and muscle force. Recent studies show that motor cortices, including primary motor cortex (M1), also represent non-traditional information such as observed movements of others and reward-related signal. However, how the neurons simultaneously multiplex such non-traditional information along with traditional motor parameters and whether the multiplexing leads to significant interactions are not well understood. Furthermore, understanding how the non-traditional information are encoded and they interact with motor information may help the development of more error-resistant, autonomous brain-to-machine interface and the understanding of underlying mechanism behind joint action and motor skill learning. In this dissertation, we investigate in detail how the observed movements and reward are simultaneously multiplexed along with traditional motor information and how each pair of neural representations interact with each other. First, regarding movement observation, we show that significant fraction of M1 neurons simultaneously encode the presence and direction of the movement of others along with those of self-movements. Neurons respond differently to joint action than to self-movements and show an interaction effect from the two representations of observed and executed movements rather than simple averaging of the two. Some neurons that separately encode observed and executed movements turn to suppress the representation of observed movements in joint action. In simultaneous actions, the representation of self-executed movement gets weaker, which suggests an interaction between two information and may possibly lead to behavioral interference. Preferred directions also change to be decoupled for noncongruent joint actions as to allow simultaneous multiplexing of both information with phase difference, while being synced for congruent ones. Conditional probabilities from the distribution of encoding neurons suggest a shared circuitry for movement observation, execution, and simultaneous actions. Shared circuitry with interactions between representations may explain why people can perform movements freely while watching others move; yet if the interaction between the two goes up due to simultaneous occurrence, it may result in interferences in behavior. Second, regarding the multiplexing of reward-related signal with movement signals, we show that both signals are multiplexed in individual and population neurons in M1 and S1. The activity of neural population in M1 and S1 distinguished whether the reward timing before the delivery of the reward. Furthermore, reward per se, reward anticipation, and reward prediction error (RPE) were encoded along with the motor information. The encoding of the reward-related signal interacted with the motor information in that the preferred direction changed when the reward was omitted. Change of spatial tuning of neurons due to reward prediction error signifies that there is interaction between the neural representation of reward and motor information, which may impact and underlie motor skill learning. In conclusion, both observed movements and reward are simultaneously multiplexed with traditional motor information. Co-representation of the two non-traditional information then leads to interaction between them and the motor information. Such interaction suggest that such simultaneous multiplexing may lead to behavioral interferences and motor skill learning.