Browsing by Author "Woldorff, Marty G"

Humans make decisions in highly complex physical, economic and social environments. In order to adaptively choose, the human brain has to learn about- and attend to- sensory cues that provide information about the potential outcome of different courses of action. Here I present three event-related potential (ERP) studies, in which I evaluated the role of the interactions between attention and reward learning in economic decision-making. I focused my analyses on three ERP components (Chap. 1): (1) the N2pc, an early lateralized ERP response reflecting the lateralized focus of visual; (2) the feedback-related negativity (FRN), which reflects the process by which the brain extracts utility from feedback; and (3) the P300 (P3), which reflects the amount of attention devoted to feedback-processing. I found that learned stimulus-reward associations can influence the rapid allocation of attention (N2pc) towards outcome-predicting cues, and that differences in this attention allocation process are associated with individual differences in economic decision performance (Chap. 2). Such individual differences were also linked to differences in neural responses reflecting the amount of attention devoted to processing monetary outcomes (P3) (Chap. 3). Finally, the relative amount of attention devoted to processing rewards for oneself versus others (as reflected by the P3) predicted both charitable giving and self-reported engagement in real-life altruistic behaviors across individuals (Chap. 4). Overall, these findings indicate that attention and reward processing interact and can influence each other in the brain. Moreover, they indicate that individual differences in economic choice behavior are associated both with biases in the manner in which attention is drawn towards sensory cues that inform subsequent choices, and with biases in the way that attention is allocated to learn from the outcomes of recent choices.

Attentional processes are critical aspects of the neural, cognitive, and computational mechanisms of decision-making. However, the role of such processes is often not given much focus in decision-making research, especially for studies involving economic decision-making. Here, I present three studies that evaluated the role of attention during decision making. Study 1 evaluated the role of attentional control, such as top-down and bottom-up control, in mediating conflict between internal and external demands on attention to promote optimal task performance in a discrimination decision task. Results from Study 1 provided novel neural insights into the role of attentional control in processing and resolving conflict between internal representations and external stimuli during everyday decision-making. Studies 2 and 3 evaluated the role of selective attention, namely online feature-based selective attention, underlying mechanisms of delay and effort discounting in economic decision-making. Results from these two studies demonstrated the importance of measuring (online and parametrically) and utilizing feature-based selective attention during comparative decision-making tasks to better quantify the cognitive and computational mechanisms underlying behavior and preferences. Taken together, results from all three studies provide important quantitative and qualitative implications for understanding mechanisms of decision-making through the lens of attention.

Attention can take on many forms – it can be directed externally toward sensory information or internally toward self-generated information. It can be selective or sustained, and it can be goal-directed or spontaneous. A lot of research on attention-memory interactions has focused on selective, externally-directed attention, but we are constantly shifting between internally- and externally-directed attention and we can be distracted by both external and internal sources. There are also many instances in which we must maintain attention for long periods of time. To gain a more comprehensive understanding of the mechanisms by which attention and memory interact, more research is needed on the mnemonic consequences of the less investigated types of attention, such as internally-directed and sustained attention. In Chapters 2 and 3, I describe two electroencephalography (EEG) studies that investigated the neural mechanisms by which visual mental images that were generated during an internally-directed attention task are encoded into and retrieved from memory. Just as attention can be directed externally or internally, distraction can occur in various ways. For example, while listening to a lecture, our attention may be diverted toward the movement of the person seated next to us (an external distractor). Alternatively, attention can shift internally, towards random thoughts (an internal distractor). Both types of attention lapses will negatively affect encoding of the lecture, but may do so in different ways. In Chapter 4, I describe a simultaneous pupillometry-fMRI study that investigated the fluctuations of sustained attention with the presence of both external and internal distractors as well as the impact on subsequent memory. Finally, we must consider the role that cognitive control plays in modulating interactions between attention and memory. In Chapter 5, I describe a behavioral study that investigated the cognitive control processes triggered in response to an error and their impact on subsequent memory. Taken together, these 4 studies provide a more nuanced look at the mechanisms by which attention memory interact as we process information in different ways.

At every moment in life we are receiving input from multiple sensory modalities. We are limited, however, in the amount of information we can selectively attend to and fully process at any one time. The ability to integrate the relevant corresponding multisensory inputs together and to segregate other sensory information that is conflicting or distracting is therefore fundamental to our ability to successfully navigate through our complex environment. Such multisensory integration and segregation is done on the basis of temporal, spatial, and semantic cues, often aided by selective attention to particular inputs from one or multiple modalities. The precise nature of how attention interacts with multisensory perception, and how this ramifies behaviorally and neurally, has been largely underexplored. Here, in a series of six cognitive experiments in humans using auditory and visual stimuli, along with electroencephalography (EEG) measures of brain activity and behavioral measures of task performance, I examine the interactions between attention, stimulus conflict, and multisensory processing. I demonstrate that attention can spread across modalities in a pattern that closely follows the temporal linking of multisensory stimuli, while also engendering the spatial linking of such multisensory stimuli. When stimulus inputs either within audition or across modalities conflict, I observe an electrophysiological signature of the processing of this conflict that is similar to what had been previously observed within the visual modality. Moreover, using neural measures of attentional distraction, I show that when task-irrelevant stimulus input from one modality conflicts with task-relevant input from another, attention is initially pulled toward the conflicting irrelevant modality, thereby contributing to the observed impairment in task performance. Finally, I demonstrate that there are individual differences in multisensory temporal processing in the population, in particular between those with extensive action-video-game experience versus those with little. However, everyone appears to be susceptible to multisensory distraction, a finding that should be taken into serious consideration in today's complex world of multitasking.

The overarching goal of this dissertation was to improve our understanding of the neural basis of consciousness by approaching the problem along two separate, complementary facets: examining the levels of consciousness and the contents of consciousness.

Chapter 2 examines how the level of consciousness changes under general anesthesia for surgery, and how neural (EEG) markers of this change relate to postoperative cognitive impairments afflicting many older adults. Older adult patients underwent neurocognitive testing before and after surgery, and their 32-channel EEG was recorded both before and during general anesthesia for surgery. Results showed that one of the most profound changes from the awake to the anesthetized brain—the anteriorization of alpha-band (8-12 Hz) activity—correlated with preoperative cognitive scores, which are themselves predictors for postoperative cognitive impairments. These results have added to our understanding of how manipulations of the level of consciousness under general anesthesia ramify into potentially long-lasting impairments to cognition, and how these impairments might be monitored and avoided.

Chapters 3 and 4 examined how the contents of consciousness relate to the selection mechanism of attention. Chapter 3 investigated the dissociability of these two phenomena by examining the neural mechanisms underlying the orienting of spatial attention without awareness. High-density (64-channel) EEG was recorded while subjects performed a novel task that combined classic spatial cueing with object-substitution masking to manipulate subjects’ awareness of the cues on ~half of the trials, allowing a direct comparison of orienting with and without awareness, controlled for having identical sensory stimulation. Results confirmed that attention could be oriented without awareness, leading to improved behavior (faster reaction times and better accuracy) and enhanced sensory processing (indexed by the P1 event-related potential, ERP) for validly (compared to invalidly) cued targets. Interestingly, the hallmark ERP for the orienting of attention in response to a cue, the N2pc, was only observed for conscious orienting, pointing to an alternate mechanism for unconscious orienting, such as via the subcortical retinotectal pathway.

Chapter 4 investigated the mechanisms and temporal dynamics of the attentional selection of conscious internal representations in working memory. EEG was recorded while subjects performed a modified delayed match-to-sample task where one of two sample objects, a face or a house, was retroactively cued on each trial. A multivariate classifier was trained on the pattern of alpha-band activity to determine if and when information about the selected object could be decoded from the alpha signal following the retrocue. Results showed that alpha could be used to decode the selected object, pointing to its general role as a top-down attentional control signal. This decoding was relatively transient, rather than sustained, which accords with recent proposals of “activity-silent” working memory and argues against accounts of working memory that posit sustained internal attention as the underlying mechanism. Together the results of Chapters 3 and 4 help inform our understanding of how attention operates both externally and internally to select the contents of consciousness.

Episodic memory, as a cognitive construct, exists only in relation to those other cognitive constructs that reference it. It is, as Ribot suggests: the tactile, the muscular, the auditory and so forth. And it is even more than this, extending to a breadth of cognitive operations, including, for example, attention and cognitive control, both of which are generally believed to facilitate episodic memory encoding and episodic memory retrieval. Without these types of sensory and cognitive referents, episodic memory does not exist. Accordingly, these types of referents are critical to an understanding of episodic memory. Therefore, in this dissertation I examine how different cognitive constructs serve to facilitate episodic memory.

Chapter 2 examines attention-related subsequent memory effects. Many studies of subsequent memory rely upon a reverse inference, i.e. increased activity in attention-related networks during memory encoding is related to better subsequent memory, ergo increased attention predicts better memory. However, it is only through direct manipulation of attentional states and the examination of specific neural markers that this claim can be strongly established. Additionally, attention is a multifaceted process, and claims that attention in general facilitates memory ignore the fact that attention consists of a set of rapidly enfolding processes. To address these issues, I designed a modified visual-search EEG experiment with a subsequent long-term memory test. The utilization of a visual-search paradigm has advantages, as the search process evokes a series of independent and well-established attention-related EEG markers which can be linked to subsequent memory. All of the attentional effects examined were found to also predict subsequent memory, suggesting that these attentional processes associated with visual search, aid long-term memory formation as well.

Chapter 3 examines how large-scale network dynamics affect long-term memory retrieval. Until now, all studies of long-term memory have focused on individual regions, pair-wise connections between regions, or, very rarely, complex interactions between a small subset of regions. In a pair of fMRI studies, I use mathematical concepts from network science to examine the large-scale brain networks associated with successful remembering and forgetting. In doing so, I demonstrate that the hippocampus increases its integration with the rest brain when individuals successfully remember an item as compared to when they do not.

Chapter 4 examines how individual items are represented in the brain with machine-learning techniques and fMRI data. Studies of episodic memory often focus on things that are common across a set of items, while ignoring the uniqueness of individual events. However, an event’s uniqueness is what defines it as being episodic with respect to memory. A primary reason unique events are not often studied is the difficulty of decoding brain states associated with individual events. In Chapter 4, I develop a machine-learning framework, utilizing cross-subject single-item decoding, to predict what image or word a left-out subject is viewing. This establishes a robust way to detect individual events which could be used in service of better understanding episodic memory.

By examining long-term memory from these perspectives, I provide evidence of how different cognitive constructs facilitate episodic memory. In Chapter 2, I focus on the role of attentional processes with respect to episodic memory encoding, in Chapter 3, I focus on how large-scale network interactions facilitate episodic memory retrieval, and in Chapter 4 I focus on the representational nature of unique events. In all cases, the examination centers on how diverse processes coordinate in order to facilitate episodic memory.

We live in a world with incredibly rich sensory environments. Our visual system is bombarded with objects of varying colors, shapes, and levels of brightness. Our auditory system is inundated with sounds of different pitches, timbre, and loudness. Yet, we are generally not overwhelmed with our environments because we can selectively choose information to interact with. One way of accomplishing such selection is through search; we search our environments so that we can selectively process relevant information and ignore other irrelevant stimuli. Search has been extensively studied in the visual domain, but there has been very little analogous research into search-type processes in audition. Moreover, even in vision, the research has been mostly limited to the processes of spatial orienting or focusing of attention towards relevant information. The process of search, however, involves additional steps aside from the engagement of spatial attention, including the initial detection and identification of relevant Target stimuli in our environment. Here I aimed to delineate the cascade of neural processes underlying search in both the auditory and visual domains, with particular emphasis on understanding the initial mechanisms underlying stimulus detection and identification.

I conducted four experiments using event-related potentials (i.e., time-locked averages of electroencephalogram) taking advantage of the high temporal resolution of this methodology to delineate the time course of search-related processes in both audition and vision. Participants were presented with a novel temporally distributed search paradigm in either the auditory (Studies 1-3) or visual (Study 4) modalities, where the task was to find a designated Target and make a discrimination concerning a certain feature of that Target. The temporal distribution of the stimulus presentation in the experiments enabled the selective extraction of the neural responses to the relevant Target and, separately, the irrelevant Nontarget, a separation that would not be possible with simultaneous or static presentation. The results showed, for both the auditory and visual domains, a very rapid, nonlateralized, differentiation of processing between the Target and sensory-equivalent Nontarget stimuli prior to the brain activity reflecting the spatial focusing of attention toward that Target. Based on results showing a failure of early differentiation when the Targets and Nontargets were presented in isolation, I inferred that this early differentiation is a result of a Relational Template preset for the Target stimulus relative to an ongoing environmental context. Additional results showed that the larger Target/Nontarget differentiation corresponded to faster response times and that the maintenance of multiple templates, to facilitate search for more than one Target item, resulted in significantly slower processing potentially due to a serial comparison of the incoming stimuli to each of the templates. These experiments show analogous mechanisms underlying both auditory and visual feature-based search that include an initial detection process, prior to the orienting of spatial attention.

Over the past decade there has been an explosion of interest in exploring how attention and reward value can interact with one another during cognition and behavior. This interdisciplinary work has already provided many critical findings that have revolutionized what have traditionally been isolated fields of study. However, there are still many unexplored aspects by which attention and value can interact with one another. Here I advance upon this interdisciplinary work by investigating several aspects of the dynamic interplay between attention and value. In the first three studies I look at how reward can influence attention and assess its neural impact using EEG. I first detail the neural mechanisms underlying reward-driven salience, a phenomenon that describes how reward-associated items receive higher priority during attentional orienting. These findings provide evidence that value-driven salience generates a unique increase in the strength of attentional orienting. In the second study I investigate whether associating distractors with rewards can lead to larger impairments in sustained spatial attention. Results indicate that sustained spatial attention can be resistant to a distractor’s reward-history, highlighting an important boundary condition for reward-related distraction. The objective of the third study was to investigate the neural processes underlying reward expectation and outcome processing, with a focus on how these reward expectations influence attention and attentional orienting. A core finding from this experiment is that outcome valence modulates the strength of attentional orienting, while uncertain outcomes lead to elongated attentional processing. In the fourth and final study I turn to investigating how attention can influence decision making. A burgeoning body of work has shown that attention can be highly predictive of choice, but it has not yet determined how inattention can influence decision processes. To this end I investigated if and how attentional distractors influence nutritional decision making using a combination of behavioral and eye-tracking measures. The results indicate that distractors can interrupt the decision process but that they do not reset it. Collectively, these studies demonstrate the diverse ways in which attention and reward can interact with one another, and how studying these interactions can help us better understand a number of real-world behaviors and circumstances.