Browsing by Subject "Interpretability"

Deep vision models have been highly successful in various computer vision applications such as image classification, segmentation, and object detection. These models encode visual data into low-dimensional representations, which are then utilized in downstream tasks. Typically, the most accurate models are fine-tuned using fully labeled data, but this approach may not generalize well to different applications. Self-supervised learning has emerged as a potential solution to this issue, where the deep vision encoder is pretrained with unlabeled data to learn more generalized representations. However, the underlying mechanism governing the generalization and specificity of representations seeks more understanding. Causality is an important concept in visual representation learning as it can help improve the generalization of models by providing a deeper understanding of the underlying relationships between features and objects in the visual world.

Through works presented in this dissertation, we provide a causal interpretation of the mechanism underlying deep vision models' ability to learn representations in both labeled and unlabeled environments and improve the generalization and the specificity of extracted representations through the interpreted causal factors. Specifically, we tackle the problem from 4 aspects: Causally Interpret Supervised Deep Vision Models; Supervised Learning with Underlabeled Data; Self-supervised Learning with Unlabeled Data; Causally Understand Unsupervised Visual Representation Learning.

Firstly, we interpret the prediction of a deep vision model by identifying causal pixels in the input images via 'inversing' the model weights. Secondly, we recognise the challenges of learning an accurate object detection model with missing labels in the dataset and we address this underlabel data issue by adapting positive-unlabeled learning approach instead of the positive-negative approach. Thirdly, we focus on improving both generalization and specificity of unsupervised representations based on prior causal relations; Finally, we enhance the stability of the unsupervised representations during the inference by intervening data variables under a well constructed causal framework.

We establish a causal relationship between deep vision models and their input/output for different applications with (partially) labeled data, and strengthen generalized representations through extensive analytical understanding of unsupervised representation learning under various hypothesized causal frameworks.

Machine learning has been increasingly deployed for myriad high-stakes decisions that deeply impact people's lives. This is concerning, because not every model can be trusted. Interpretability is crucial for making machine learning models trustworthy. It provides human-understandable reasons for each prediction. This, in turn, enables easier troubleshooting, responsible decision-making, and knowledge acquisition. However, there are two major challenges in using interpretable machine learning for high-stakes problems: (1) interpretable model optimization is often NP-hard, and (2) an inefficient feedback loop is present in the standard machine learning paradigm. My dissertation addresses these challenges and proposes a new paradigm for machine learning to advance trustworthy AI.

I first tackle the challenge of finding interpretable-yet-accurate models. This involves developing efficient optimization algorithms. Models obtained from these algorithms are inherently interpretable while maintaining accuracy comparable to that of black-box counterparts. I then discuss the interaction bottleneck in the standard machine learning paradigm and propose a new paradigm, called learning Rashomon sets, which finds and stores all machine learning models with loss that is within epsilon of the optimal loss. This allows users unprecedented ability to explore and interact with all well-performing models, enabling them to choose and modify models that are best suited for the application.

As machine learning models are playing increasingly important roles in many real-life scenarios, interpretability has become a key issue for whether we can trust the predictions made by these models, especially when we are making some high-stakes decisions. Lack of transparency has long been a concern for predictive models in criminal justice and in healthcare. There have been growing calls for building interpretable, human understandable machine learning models, and "opening the black box" has become a debated issue in the media. My dissertation research addresses precisely the demand for interpretability and transparency in machine learning models. The key problem of this dissertation is: "Can we build machine learning models that are both accurate and interpretable?"

To address this problem, I will discuss the notion of interpretability as it relates to machine learning, and present several new interpretable machine learning models and methods I developed during my dissertation research. In Chapter 1, I will discuss two types of model interpretability -- predicate-based and case-based interpretability. In Chapters 2 and 3, I will present novel predicate-based interpretable models and methods, and their applications to understanding low-dimensional structured data. In particular, Chapter 2 presents falling rule lists, which extend regular decision lists by requiring the probabilities of the desired outcome to be monotonically decreasing down the list; Chapter 3 presents two-layer additive models, which are hybrids of predicate-based additive scoring models and small neural networks. In Chapter 4, I will present case-based interpretable deep models, and their applications to computer vision. Given the empirical evidence, I conclude in Chapter 5 that, by designing novel model architectures or regularization techniques, we can build machine learning models that are both accurate and interpretable.

Machine learning algorithms are being adopted for clinical use, assisting with difficult medical tasks previously limited to highly-skilled professionals. AI (artificial intelligence) performance on isolated tasks regularly exceeds that of human clinicians, spurring excitement about AI's potential to radically change modern healthcare. However, there remain major concerns about the uninterpretable (i.e., "black box") nature of commonly-used models. Black box models are difficult to troubleshoot, cannot provide reasoning for their predictions, and lack accountability in real-world applications, leading to a lack of trust and low rate of adoption by clinicians. As a result, the European Union (through the General Data Protection Regulation) and the US Food & Drug Administration have published new requirements and guidelines calling for interpretability and explainability in AI used for medical applications.

My thesis addresses these issues by creating interpretable models for the key clinical decisions of lesion analysis in mammography (Chapters 2 and 3) and pattern identification in EEG monitoring (Chapter 4). To create models with comparable discriminative performance to their uninterpretable counterparts, I constrain neural network models using novel neural network architectures, objective functions and training regimes. The resultant models are inherently interpretable, providing explanations for each prediction that faithfully represent the underlying decision-making of the model. These models are more than just decision makers; they are decision aids capable of explaining their predictions in a way medical practitioners can readily comprehend. This human-centered approach allows a clinician to inspect the reasoning of an AI model, empowering users to better calibrate their trust in its predictions and overrule it when necessary

Even if a model is not globally sparse, it is possible for decisions made by that model to be accurately and faithfully described by a small number of features. For example, an application for a large loan might be denied to someone because they have no credit history, which overwhelms any evidence of their creditworthiness. In this paper, we introduce the Sparse Explanation Value (SEV), a new way to measure sparsity in machine learning models. In the loan denial example above, the SEV is 1 because only one factor is needed to explain why the loan was denied. SEV is a measure of \textit{decision sparsity} rather than overall model sparsity, and we can show that many machine learning models -- even if they are not sparse -- actually have low decision sparsity as measured by SEV. SEV is defined using moves over a hypercube with a predefined population commons (reference), allowing SEV to be defined consistently across model classes, with movement restrictions that reflect real-world constraints. Moreover, by allowing flexibility in this reference, and by considering how distances along the hypercube translate into distances in feature space, we can derive sparse and meaningful explanations for different types of function classes and propose three possible approaches: cluster-based SEV, SEV with flexible references and tree-based SEV. Ultimately, we propose algorithms aimed at reducing SEV without compromising model accuracy, thereby offering sparse yet fully faithful explanations, even in the absence of globally sparse models.