Bayesian Inference for Discrete Structures

Abstract

Discrete structures frequently arise in statistics and machine learning as both observed and latent variables. There is a rich literature on discrete structure modeling for a wide variety of applications, including categorical data analysis, graphical models, and clustering. However, these fields have multiple well-documented fundamental problems that are not adequately addressed by existing methods, which result from factors such as model misspecification, the curse of dimensionality, interpretability, and sparsity. This dissertation attempts to address these issues with novel methodology. Chapter 2 details methods to robustify inferences in Bayesian clustering. This field generally relies on mixture models, with each component interpreted as a different cluster. Unfortunately, if, for example, Gaussian kernels are used but the true density of data within a cluster is even slightly non-Gaussian, then clusters will be broken into multiple Gaussian components. To address this problem, we develop a Bayesian decision-theoretic clustering method that melds components together using the posterior of the kernels. In addition, the proposed methodology naturally leads to uncertainty quantification, can be easily implemented as an add-on to Markov chain Monte Carlo (MCMC) algorithms for mixtures, and favors a small number of distinct clusters. Chapter 3 presents a theoretical framework for multiview clustering: the problem of obtaining distinct but statistically dependent clusterings in a common set of entities for different data types. The complexities of the partition space make standard methods for modeling dependence, such as correlation, infeasible. In this work, we show how statistically dependent partitions can be generated by sampling from a Dirichlet process centered on the product measure of two Dirichlet process distributed random measures. We derive several appealing properties of this construction, including a finite approximation, a marginal Gibbs sampler algorithm, and closed-form expressions for the marginal and joint distributions of the clusterings. Chapter 4 focuses on the issue of interpretability in applications of Bayesian clustering to disease subtyping, or the process of deriving subgroups of patients suffering from the same medical condition. Ideally, subgroups will be used to inform treatment decisions. In this chapter, we propose a prior for Bayesian clustering that explicitly models the medical interpretation of each cluster center. This prior favors clusterings that can be summarized via meaningful feature values, leading to medically significant patient subgroups. We discuss in detail an application of our methodology to clustering sepsis patients from Moshi, Tanzania. Finally, Chapter 5 discusses discrete graphical models, i.e., when discrete variables are assumed to follow an underlying graph structure. We detail a Bayesian framework for the case in which the dependence between features is described by a directed acyclic graph (DAG). For any node in the DAG, the conditional probabilities associated with its parents are modeled from a hierarchical Dirichlet distribution. This model allows for the sharing of information between the edges in the DAG. We show that it results in a reasonable shrinkage estimator for the conditional probabilities, and leads to an efficient marginal MCMC algorithm and empirical Bayes estimation scheme.