Data Driven Style Transfer for Remote Sensing Applications

Abstract

Recent recognition models for remote sensing data (e.g., infrared cameras) are based upon machine learning models such as deep neural networks (DNNs) and typically require large quantities of labeled training data. However, many applications in remote sensing suffer from limited quantities of training data. To address this problem, we explore style transfer methods to leverage preexisting large and diverse datasets in more data-abundant sensing modalities (e.g., color imagery) so that they can be used to train recognition models on data-scarce target tasks. We first explore the potential efficacy of style transfer in the context of Buried Threat Detection using ground penetrating radar data. Based upon this work we found that simple pre-processing of downward-looking GPR makes it suitable to train machine learning models that are effective at recognizing threats in hand-held GPR. We then explore cross modal style transfer (CMST) for color-to-infrared stylization. We evaluate six contemporary CMST methods on four publicly-available IR datasets, the first comparison of its kind. Our analysis reveals that existing data-driven methods are either too simplistic or introduce significant artifacts into the imagery. To overcome these limitations, we propose meta-learning style transfer (MLST), which learns a stylization by composing and tuning well-behaved analytic functions. We find that MLST leads to more complex stylizations without introducing significant image artifacts and achieves the best overall performance on our benchmark datasets.