Geometric Cross-Modal Comparison of Heterogeneous Sensor Data
Date
2018-03
Journal Title
Journal ISSN
Volume Title
Repository Usage Stats
views
downloads
Abstract
In this work, we address the problem of cross-modal comparison of aerial data streams. A variety of simulated automobile trajectories are sensed using two different modalities: full-motion video, and radio-frequency (RF) signals received by detectors at various locations. The information represented by the two modalities is compared using self-similarity matrices (SSMs) corresponding to time-ordered point clouds in feature spaces of each of these data sources; we note that these feature spaces can be of entirely different scale and dimensionality. Several metrics for comparing SSMs are explored, including a cutting-edge time-warping technique that can simultaneously handle local time warping and partial matches, while also controlling for the change in geometry between feature spaces of the two modalities. We note that this technique is quite general, and does not depend on the choice of modalities. In this particular setting, we demonstrate that the cross-modal distance between SSMs corresponding to the same trajectory type is smaller than the cross-modal distance between SSMs corresponding to distinct trajectory types, and we formalize this observation via precision-recall metrics in experiments. Finally, we comment on promising implications of these ideas for future integration into multiple-hypothesis tracking systems.
Type
Department
Description
Provenance
Subjects
Citation
Permalink
Collections
Scholars@Duke
Paul L Bendich
I am a mathematician whose main research focus lies in adapting theory from ostensibly pure areas of mathematics, such as topology, geometry, and abstract algebra, into tools that can be broadly used in many data-centered applications.
My initial training was in a recently-emerging field called topological data analysis (TDA). I have been responsible for several essential and widely-used elements of its theoretical toolkit, with a particular focus on building TDA methodology for use on stratified spaces. Some of this work involves the creation of efficient algorithms, but much of it centers around theorem-proof mathematics, using proof techniques not only from algebraic topology, but also from computational geometry, from probability, and from abstract algebra.
Recently, I have done foundational work on TDA applications in several areas, including to neuroscience, to multi-target tracking, to multi-modal data fusion, and to a probabilistic theory of database merging. I am also becoming involved in efforts to integrate TDA within deep learning theory and practice.
I typically teach courses that connect mathematical principles to machine learning, including upper-level undergraduate courses in topological data analysis and more general high-dimensional data analysis, as well as a sophomore level course (joint between pratt and math) that serves as a broad introduction to machine learning and data analysis concepts.
John Harer
Professor Harer's primary research is in the use of geometric, combinatorial and computational techniques to study a variety of problems in data analysis, shape recognition, image segmentation, tracking, cyber security, ioT, biological networks and gene expression.
Unless otherwise indicated, scholarly articles published by Duke faculty members are made available here with a CC-BY-NC (Creative Commons Attribution Non-Commercial) license, as enabled by the Duke Open Access Policy. If you wish to use the materials in ways not already permitted under CC-BY-NC, please consult the copyright owner. Other materials are made available here through the author’s grant of a non-exclusive license to make their work openly accessible.