Investigating Distribution Shifts for Deep Learning Models

Abstract

Machine learning, specifically deep learning with Deep Neural Networks (DNNs), has revolutionized many fields and enabled the accurate execution of various complex tasks, including those in the field of computer vision and natural language processing, to name just a few. The great success of deep learning is often attributed to the capability of highly parameterized DNNs to capture the underlying patterns between the input and the output, given tremendous amount of training data. However, given the intrinsic complexity of the world we are living in, there are always distribution shifts between the training distribution and inference distribution. A notorious challenge for deep networks is that whenever there are distribution shifts, they will behave in unexpected and undesirable ways, rendering their normal functionalities no longer effective.In this dissertation, we study both of the two types of distribution shifts: Covariate shifts and semantic shifts. They are orthogonal to each other: Taking image classification as an example, the former shifts only change some non-semantic factors of an image (e.g., a dog photo to a blurry dog photo), while the latter shifts lead to semantic changes of the inputs (e.g., an animal classifier somehow sees a car image).More specifically, for covariate shifts we focus on adversarial perturbations/attacks which are intentionally and artificially crafted to deceive DNNs. We start by exploring how to make models more robust against adversarial perturbations, where we develop a novel ensemble-based training algorithm that can result in a robust ensemble by diversifying the vulnerable features captured by the underlying sub-models. Then, on the other side of the coin, we demonstrate that adversarial training, one of the most effective methods for training robust DNNs against adversarial noises, can silently increase the privacy risks of model. This investigation is carried out with our devised attack which can accurately reconstruct high-resolution training images from adversarial training models in a federated learning system.The other thread of our works feature the study of semantic shifts. We first propose SIO, a method that can universally enhance multiple out-of-distribution (OOD) detection methods by simply incorporating synthetic in-distribution images into the training. Then, we present MixOE, a novel training methodology that significantly improves OOD detection in the challenging (yet practical) fine-grained environments. After that, We discuss the work of OpenOOD v1.5, where we build a large-scale and unified benchmarking tool for the field of OOD detection to enable fair and straight comparison of 40+ methods. Lastly, we study the problem of pre-training data detection for large language models (LLMs), where we develop Min-K%++ which established a new baseline for the field at that time.In summary, the research efforts presented in this dissertation either deliver an innovative and effective method, provide insights or interesting findings, or lead to a useful tool that facilitates developments, experiments, and evaluations; all of which we believe have advanced the fields and contributed to the progress of the whole community.