Browsing by Subject "BCI"

A series of fusion techniques are developed and applied to EEG and pupillary recording analysis in a rapid serial visual presentation (RSVP) based image triage task, in order to improve the accuracy of capturing single-trial neural/pupillary signatures (patterns) associated with visual target detection.

The brain response to visual stimuli is not a localized pulse, instead it reflects time-evolving neurophysiological activities distributed selectively in the brain. To capture the evolving spatio-temporal pattern, we divide an extended (``global") EEG data epoch, time-locked to each image stimulus onset, into multiple non-overlapping smaller (``local") temporal windows. While classifiers can be applied on EEG data located in multiple local temporal windows, outputs from local classifiers can be fused to enhance the overall detection performance.

According to the concept of induced/evoked brain rhythms, the EEG response can be decomposed into different oscillatory components and the frequency characteristics for these oscillatory components can be evaluated separately from the temporal characteristics. While the temporal-based analysis achieves fairly accurate detection performance, the frequency-based analysis can improve the overall detection accuracy and robustness further if frequency-based and temporal-based results are fused at the decision level.

Pupillary response provides another modality for a single-trial image triage task. We developed a pupillary response feature construction and selection procedure to extract/select the useful features that help to achieve the best classification performance. The classification results based on both modalities (pupillary and EEG) are further fused at the decision level. Here, the goal is to support increased classification confidence through inherent modality complementarities. The fusion results show significant improvement over classification results using any single modality.

For crucial image triage tasks, multiple image analysts could be asked to evaluate the same set of images to improve the probability of detection and reduce the probability of false positive. We observe significant performance gain by fusing the decisions drawn by multiple analysts.

To develop a practical real-time EEG-based application system, sometimes we have to work with an EEG system that has a limited number of electrodes. We present methods of ranking the channels, identifying a reduced set of EEG channels that can deliver robust classification performance.

Speech production is a unique human function that allows us to rapidly communicate on a regular basis. Characterizing the brain mechanisms of speech not only advances the scientific understanding of human cognition, but also provides the potential for therapeutic interventions to restore communication in patients with motor speech impairments. Deciphering this speech neural mechanisms requires measuring the functional interactions between neuro-anatomical regions at precise spatial and temporal scales, a feat that has remained difficult with standard non-invasive techniques for measuring brain activity. Intracranial electrode recordings provide measurements of neural activity directly from the surface of the brain and enable more accurate representations of speech neural activations with precise timings. Further, recent technological developments in increasing the number of electrodes have improved the spatial resolution, thereby revealing hidden neural features that would have been absent with low-resolution clinical electrodes. Therefore, developing analytical frameworks to effectively quantify these spatial and temporal characteristics of speech neural activations, can not only improve the ability to accurately decode speech from the brain, but also delineate the neural mechanism that enable speech production. This dissertation proposes signal analysis and neural decoding methods to quantify the spatiotemporal dynamics of speech neural features obtained from intracranial recordings. These computational approaches validate the use of high-resolution neural recordings to achieve successful speech decoding for neuroprosthetics and to elucidate the neural mechanisms of speech production in time and space.In the first section of this dissertation (chapter 2), high-resolution micro-electrocorticographic (µECoG) arrays are introduced as a novel recording tool to improve the accuracy of neural speech decoding. Using a comprehensive set of decoding algorithms and analytic metrics, µECoG is validated against standard intracranial recordings by demonstrating strong speech decoding with only ~2 minutes of training data. The decoding metrics were also applied to examine the spatial characteristics of µECoG. We demonstrated the importance of high channel count, high spatial coverage, high resolution, and micro-scale recording, to enable accurate speech decoding. The second section (chapter 3) extended these analytical techniques to identify neural mechanisms of speech for both planning and execution. Distinct neuro-anatomical networks that were identified were functionally specific to speech features. Further, the utilization of µECoG uncovered spatial mechanisms that facilitated the transition from speech planning to motor execution. These sets of results obtained from novel neural interfacing technologies and clinical recordings provide a robust analytical methodology to study neural mechanisms of human speech.