Geometric Cross-Modal Comparison of Heterogeneous Sensor Data
Abstract
In this work, we address the problem of cross-modal comparison of aerial data streams.
A variety of simulated automobile trajectories are sensed using two different modalities:
full-motion video, and radio-frequency (RF) signals received by detectors at various
locations. The information represented by the two modalities is compared using self-similarity
matrices (SSMs) corresponding to time-ordered point clouds in feature spaces of each
of these data sources; we note that these feature spaces can be of entirely different
scale and dimensionality. Several metrics for comparing SSMs are explored, including
a cutting-edge time-warping technique that can simultaneously handle local time warping
and partial matches, while also controlling for the change in geometry between feature
spaces of the two modalities. We note that this technique is quite general, and does
not depend on the choice of modalities. In this particular setting, we demonstrate
that the cross-modal distance between SSMs corresponding to the same trajectory type
is smaller than the cross-modal distance between SSMs corresponding to distinct trajectory
types, and we formalize this observation via precision-recall metrics in experiments.
Finally, we comment on promising implications of these ideas for future integration
into multiple-hypothesis tracking systems.
Type
Journal articlePermalink
https://hdl.handle.net/10161/15844Collections
More Info
Show full item recordScholars@Duke
Paul L Bendich
Associate Research Professor of Mathematics
I am a mathematician whose main research focus lies in adapting theory from ostensibly
pure areas of mathematics, such as topology, geometry, and abstract algebra, into
tools that can be broadly used in many data-centeredapplications.My initial training
was in a recently-emerging field called topological data analysis (TDA). I have beenresponsible
for several essential and widely-used elements of its theoretical toolkit, with a
particularfocus on building TDA methodology
John Harer
Professor of Mathematics
Professor Harer's primary research is in the use of geometric, combinatorial and computational
techniques to study a variety of problems in data analysis, shape recognition, image
segmentation, tracking, cyber security, ioT, biological networks and gene expression.
Christopher Tralie
Postdoctoral Associate
Alphabetical list of authors with Scholars@Duke profiles.

Articles written by Duke faculty are made available through the campus open access policy. For more information see: Duke Open Access Policy
Rights for Collection: Scholarly Articles