Harnessing Recent Online Data to Improve Brain-Computer Interface Operation

Abstract

Brain-computer interfaces (BCIs) have the potential to restore or replace lost neural output after injury or disease. BCIs operate by processing and interpreting brain signals such as (non-invasive) electroencephalography (EEG) data to decode user intent into commands to control external devices. An important consideration in deploying BCIs is that brain activity is constantly evolving, making long-term accurate interpretation of brain activity a challenge.This work focuses on the P300 speller, a BCI that can be used to restore communication abilities for individuals whose motor abilities have been severely compromised, such as people with amyotrophic lateral sclerosis (ALS). The P300 speller enables users to select their target character from a set of choices using their brain activity. In order to spell a character, the BCI presents a stream of stimuli to the user, anticipating that event-related potentials (ERPs) will be elicited within EEG data in response to the presentation of a rare stimulus, specifically one containing the target character. The characteristics of ERPs will change over time according to factors such as a user’s level of fatigue. The P300 speller uses a machine learning classifier to interpret the user’s brain signals; this classifier is conventionally trained in a supervised manner with EEG data during a calibration phase, with its parameters remaining static as it is applied to real-time EEG data during online operation. As this BCI is used to meet long-term communication needs, it is important to maintain the performance of the P300 speller over longitudinal use. Since the statistical properties of brain signals change over time, the current and most recently available brain signals are most suitable for understanding a user’s current cognitive state. This work focuses on developing methods to enhance the use of real-time data during online BCI operation in order to improve BCI performance.In this work, it is hypothesized that further incorporating recent online EEG data and BCI decision-making into the P300 speller framework can enhance BCI performance by providing more up-to-date information regarding the user’s intent. This research addresses methods of improving three areas of the P300 speller framework: stimulus selection, classifier learning, and representation learning. During a spelling trial, i.e., the series of stimuli used to select a user’s target character, the presentation of P300 speller stimuli directly influences how ERPs are elicited. The stimulus selection process can be optimized to improve the information content in presented stimuli based on the EEG data and stimulus presentation history from the trial. A new stimulus presentation paradigm is developed with this optimization goal, and which also focuses on designing stimulus characteristics to mitigate psychophysical effects that can negatively influence ERP elicitation. Next, as changing brain activity makes the most recent brain signals more appropriate for interpreting a user’s intent than older brain signals from offline calibration, the BCI classifier learning strategy can be modified from the conventional static approach to an adaptive approach that utilizes real-time EEG data. A semi-supervised learning strategy that relies on EEG data and ground truth labels from BCI calibration, as well as online EEG data and classifier-predicted labels, is explored. This strategy incorporates a language model in order to improve the quality of the classifier-predicted labels used for continual online learning. Finally, attention-based models afford a method to perform contextual representation learning with EEG data. The inputs to an attention-based model can be enhanced for the BCI use case with the goal of learning more robust data representations. As the stimulus presentation sequence is directly connected to the ERP elicitation patterns in a trial, providing this BCI decision-making history can improve the data context available to the model. The studies presented within this work provide evidence that these proposed methods for further integrating real-time EEG data and BCI decision-making history into the spelling framework have the potential to significantly improve BCI communication rates, including over long-term use of the BCI.