Browsing by Subject "Eye-tracking"

Many augmentative and alternative communication (AAC) devices have been developed to aid individuals who have some form of severe neuromuscular disorder to communicate with the outside world, such as Amyotrophic Lateral Sclerosis (ALS). Eye-trackers are used as a primary communication device for people with ALS until they lose the ability to use their eyes. As an alternative to eye-trackers, the P300 speller brain-computer interface (BCI) is a non-invasive mode of communication that utilizes electroencephalography(EEG) data.

The P300 speller relies on eliciting and detecting event related potentials (ERPs) that occur in the EEG data when a rare or unexpected visual or auditory stimulus is presented to the user; however, only visual stimuli are used in this work. The P300 speller displays characters or symbols in a grid on a computer screen and presents (i.e. flashes) a subset of the characters simultaneously; these presentations are known as the stimuli. The presentation of the target (i.e., desired) character should elicit an ERP. After many repetitions of presentations, the speller attempts to estimate the desired character using the EEG data. This thesis is composed of two primary methods to improve the P300 speller: A novel data-driven adaptive stimulus selection paradigm based on maximizing the expected discrimination gain (EDG) metric; and fusion of an eye-gaze data stream to develop a hybrid P300 and eye-gaze speller.

Many pseudo-random stimulus presentation paradigms (i.e., patterns) have been developed to improve the accuracy and decrease the time required to communicate via the P300 speller. Few data-driven, adaptive, stimulus presentation paradigms have been developed, however, they are computationally expensive, thus have limited flexibility in the groups of characters that can be presented simultaneously. In this thesis, a novel data-driven, adaptive, stimulus selection approach based on maximizing the expected discrimination gain (EDG) is introduced. Various restrictions are set on the characters that can be presented based on system and physiological constraints. Simulations show that even with various restrictions on the proposed adaptive paradigm, the adaptive paradigm yields a higher accuracy and a decrease in time required to spell compared to the most commonly used row/column random paradigm. Online results show that the proposed paradigm decreases the time required to spell, however, there is a slight decrease in speller accuracy.

In addition to setting restrictions based on physiological effects, this thesis presents work done on explicitly modeling refractory effects, probabilistically, on a subject-specific basis. Refractory effects occur when the time between target stimulus presentations is not sufficiently long, resulting in decreased SNRs of the ERP. By modeling the refectory effects, the adaptive stimulus selection paradigm can automatically choose characters to present that minimize refractory effects, without having to explicitly set ad-hoc restrictions. Offline simulations showed that modeling refractory effects explicitly has the potential to further increase the accuracy, and decrease the time required to spell.

Beyond improving the independent P300 speller, there has been recent interest in developing a hybrid (or “fused”) BCI system. In this thesis, a probabilistic hybrid P300 and eye-tracker is developed and its effectiveness is explored. The hybrid speller collects both eye-tracking and EEG data in parallel, and the user spells the characters in the same way that they would spell them using the traditional P300 speller. Both online and offline experiments are performed to analyze the hybrid speller. Online results showed that for the fifteen non-disabled participants, the hybrid speller improved accuracy and reduced the time required to spell a character. Offline simulations showed that the system is more robust to eye-gaze abnormalities than a stand-alone eye-gaze system.

When learning a language, typically-developing infants face the daunting task of learning both the sounds and the meanings of words. In this dissertation, we focus on a source of variability that complicates the one-to-one relationship between words and their meanings: wordform variability. In Chapter 1 we make a distinction between the micro timescale, where learning and acquisition can be observed during the comprehension of individual utterances, and the macro timescale, where a longitudinal lens can reveal intuitions about the speech that infants and toddlers can learn from. In Chapter 2, a corpus analysis confirms that infants hear more irregular verbs than irregular nouns. We then compare toddlers' phonological representations of irregular nouns and verbs to their regular counterparts in an eyetracking study. Toddlers demonstrate well-specified representations for verbs 8 months later than they do for nouns, but the sounds in irregular words are represented with high fidelity at the same time as regular words in both syntactic categories. In Chapter 3, an eyetracking study with adults explores word comprehension when coarticulation cues have been manipulated. Adults take available referents into account when they hear a word whose coarticulation cues match an unfamiliar word. We also find that participants vary in whether they map an unfamiliar word to an available unfamiliar object if the unfamiliar word is sufficiently similar to a relevant known word. In Chapter 4, we return to early childhood, using annotated data from a longitudinal corpus of naturalistic recordings from the lives of 6-17-month-old infants. In this corpus, we characterize and quantify wordform variability, and find that for high-frequency words, wordform variability may aid in word learning. Theoretical implications and next steps are discussed in Chapter 5. Taken together, this work suggests that throughout the lifespan, wordform variability is no problem for learners and can in some cases facilitate learning and comprehension.