Towards Sustainable Machine Learning Systems

Abstract

This dissertation presents methods for building sustainable machine learning systems: models that remain reliable under distribution shift and evolving tasks while keeping the costs of data retention, computation, and model growth manageable in real deployments. In many practical settings, adaptation is constrained by missing historical data, limited labeling budgets, and the need to prevent performance regressions as the environment changes. The central goal of this thesis is to enable efficient reuse of prior knowledge—reusing what a system has already learned rather than repeatedly retraining or expanding it indefinitely.The thesis makes three contributions. First, it studies continual learning under recurring and partially overlapping task streams and introduces mechanisms to identify when new inputs match previously learned skills so that existing components can be repurposed, reducing unnecessary updates and slowing unbounded capacity growth while maintaining accuracy. Second, it addresses data-free and multi-source transfer, where only pretrained models are available and source data cannot be accessed, and proposes techniques to recycle and compose models through parameter-efficient reuse, enabling adaptation that scales with the number of available sources without requiring full retraining. Third, it improves the reliability of vision-language models used for classification in open or uncertain conditions by developing fallback-aware decision rules that detect when predictions are not trustworthy and defer to an ``unknown'' outcome, yielding better calibrated behavior and safer deployment.Across diverse visual recognition benchmarks and deployment-motivated evaluation protocols, these methods demonstrate that sustainable adaptation can be achieved by combining selective reuse, efficient composition, and explicit uncertainty handling, advancing the practical foundation for robust learning systems that evolve over time.