From Orbit to Microscope: Using Machine Learning to Translate Pixels to Patterns for Anomaly Detection Across Environmental and Manufacturing Domains

Abstract

This dissertation showcases the capability of machine learning within the computer vision domain for applications among environmental and manufacturing domains. Improvements in the accessibility and resolution of imaging now allow vast image streams to be converted into reliable signals for decision making across surveillance, monitoring, and anomaly/defect detection. However, the evolution of image data in the age of machine learning exposes a pivotal requirement with their use: methods that remain applicable across domains, resolutions, and sensors that can scale smoothly from limited data. In practice, real-world datasets are heterogeneous in size, exhibit realistic structured missingness (i.e., cloud cover, human error), and vary in spatial resolution. This work emphasizes the ability to use machine learning among different image types and among different use cases in different domains.Many popular state-of-the-art models are trained on web-scale corpora, and excel at canonical benchmarks, yet the data they learn from often differ markedly from the imagery encountered in remote sensing-based tasks, or manufacturing microscopy tasks. This domain dependence is well known among the computer vision domain. Convolutional Neural Networks pretrained on RGB-banded images can degrade performance when used out of the box on single channel, or hyperspectral data applications, despite the use of images as inputs. It is imperative to approach the application of real-world data for real world tasks (not everything is about classification) with the use of machine learning with a careful and calculated approach. With these real-world datasets comes the reality of small, task specific datasets. Recognizing this reality, the work presented here leverages appropriate pretrained networks to translate large-model strengths into reliable performers for daily environmental monitoring and manufacturing inspection. In the era of large foundation models (LLMs, and diffusion models), memory efficient convolutional architectures remain highly effective for targeted image analysis, especially in small-sample, domain-shifted settings. This work demonstrates two complementary applications on a curated set of 613 high-resolution (3m/pixel) satellite images over Thilafushi Island, a manmade landfill in the Maldives historically known for consistent waste burning. Firstly, a pretrained CNN was finetuned to classify images as containing or not containing a waste burning plume. Secondly, a pretrained-UNet was adapted for binary semantic segmentation to localize plume pixels within the images—a task unforeseen in previous moderate and low-resolution datasets. Leveraging transfer learning and careful regularization enables robust detection despite limited images and hence limited positively class images. As of 2021, a governmental ban on open waste burning was placed on the Thilafushi Island, and this developed model on archived data from 2016-2021 provided support for image-drive compliance monitoring after the ban. A second case study applies high-resolution satellite imagery to surveil and track harmful algal blooms along the Chowan River of North Carolina (USA). We employ commercially available 3m/pixel imagery, contrasted against coarser products (~300 m/pixel), as input to a pretrained CNN for image classification. The higher spatial resolution provides finer, attuned detail of coastline and shoreline blooms, that tend to be sub-pixel or unrepresented at all at 300m resolution. Real-world riverine scenes introduce unique variability and noise via sunglint, detritus, aquatic vegetation, and seasonal changes to the watercolor, while the blooms themselves exhibit inconsistent and amorphous shapes, providing a unique challenge for state-of-the-art labeling methods. This study demonstrated the practical use of high-resolution, learning-based bloom monitoring for public-health management via support to rural sampling teams. The last use case regarding remote sensing data that this dissertation addresses regards spatiotemporal gaps in a new generation of air quality monitoring. NASA’s TEMPO instrument delivers hourly, neighborhood scale retrievals of tropospheric trace gases, enabling diurnal analysis unforeseen with legacy low-Earth-orbit sensors. However, TEMPO is not exempt of the caveats of missing data that plague real world remote sensing datasets; daylight-only sampling produces nocturnal gaps, while cloud coverage and quality-flag filtering create spatially irregular missing pixels. We treat these challenges separately. For temporal gaps, we train classical regression models to forecast overnight NO₂ from evening TEMPO snapshots and auxiliary drivers (meteorology, boundary-layer indices, emissions proxies), supported when available by lunar Pandora spectrometer observations. For spatial gaps, we develop a mask-aware Partial Convolutional U-Net that conditions on the missing data mask, which learns to preserve edges as well as learns spatial gradient trends without ground truth pixels. This adapted U-Net shows improved spatial coherence and structure when compared to baseline image inpainting techniques. This study highlights the versatile use cases of machine learning, traditional and deep learning, for use on novel, real-world datasets to produce robust data-efficient methods. On the extreme other end of the resolution spectrum, this dissertation examines the microscopic limit of the same anomaly-detection problem. In laser powder bed fusion metal additive manufacturing, microscopic pores and lack-of-fusion defects can compromise performance of parts meant for high stress and high pressure uses. This study aims to bridge the modalities of thermal tomography (TT) and scanning electron microscopy (SEM) with machine learning to enhance detection efforts. A multi-task U-Net with a shared encoder takes defect relevant information from datasets from both quality control datasets, and performs their respective tasks; segmentation on SEM images, and classification of defect parameters within TT images. Despite the disjoint nature of the datasets themselves, the shared encoder design yielded improved performance over the separate tasks over the separate datasets. This work, while distant in domain, complements the central theme of this dissertation: machine-learning-driven image analysis that translates pixels into patterns across scales—from satellite-based scenes to microstructures. The results of this dissertation show the versatility of machine learning across visual domains and datasets, emphasizing how practical, image-based machine learning methods can work among a variety of resolutions for a variety of real-world use cases. These results offer a straightforward way to utilize images and transform them into trustworthy signals to rely on for public-health decisions, and manufacturing quality checks, spotlighting the versatility of machine learning for use cases we encounter every day.