Abstract:
<p>A series of fusion techniques are developed and applied to EEG and
pupillary recording analysis in a rapid serial visual presentation
(RSVP) based image triage task, in order to improve the accuracy
of capturing single-trial neural/pupillary signatures (patterns)
associated with visual target detection.</p><p>The brain response to visual stimuli is not a localized pulse,
instead it reflects time-evolving neurophysiological activities
distributed selectively in the brain. To capture the evolving
spatio-temporal pattern, we divide an extended (``global") EEG
data epoch, time-locked to each image stimulus onset, into
multiple non-overlapping smaller (``local") temporal windows.
While classifiers can be applied on EEG data located in multiple
local temporal windows, outputs from local classifiers can be
fused to enhance the overall detection performance.</p><p>According to the concept of induced/evoked brain rhythms, the EEG
response can be decomposed into different oscillatory components
and the frequency characteristics for these oscillatory components
can be evaluated separately from the temporal characteristics.
While the temporal-based analysis achieves fairly accurate
detection performance, the frequency-based analysis can improve
the overall detection accuracy and robustness further if
frequency-based and temporal-based results are fused at the
decision level.</p><p>Pupillary response provides another modality for a single-trial
image triage task. We developed a pupillary response feature
construction and selection procedure to extract/select the useful
features that help to achieve the best classification performance.
The classification results based on both modalities (pupillary and
EEG) are further fused at the decision level. Here, the goal is to
support increased classification confidence through inherent
modality complementarities. The fusion results show significant
improvement over classification results using any single modality.</p><p>For crucial image triage tasks, multiple image analysts could be
asked to evaluate the same set of images to improve the
probability of detection and reduce the probability of false
positive. We observe significant performance gain by fusing the
decisions drawn by multiple analysts.</p><p>To develop a practical real-time EEG-based application system,
sometimes we have to work with an EEG system that has a limited
number of electrodes. We present methods of ranking the channels,
identifying a reduced set of EEG channels that can deliver robust
classification performance.</p>
