Exact Bayesian Inference for High-dimensional Latent Variable Stochastic Models with Complex, Discrete Structures

Abstract

Stochastic compartmental models provide interpretable probabilistic descriptions of many dynamic biological phenomena, such as the spread of a contagious disease through a population or the progression of cancer in an individual. Many classical inferential tools are out of reach for fitting these models in missing data settings, however, due to the intractability of the marginal likelihood. To remedy this issue, practitioners typically rely on simplifying assumptions to make inference tractable, or on intensive simulation-based methods that do not scale to modern datasets. In this thesis, I demonstrate how long-held simplifying assumptions can be relaxed, improving model realism and yielding better data fit. These contributions are developed alongside efficient sampling algorithms to enable exact Bayesian inference in many partially observed settings. I focus on two driving applications: first, I model the spread of contagious disease through a population using a continuous-time, stochastic susceptible-infectious-removed model, and provide nonparametric temporal extensions. Next, I turn attention to modeling the natural history of cancer using a semi-Markov model. In each of these studies, I tailor a data-augmented Markov chain Monte Carlo sampling algorithm that efficiently explores its discrete, high-dimensional latent space. Taken together, these advances surmount computational and methodological challenges in a notoriously difficult setting for modern Markov chain Monte Carlo samplers, and leads to new insights in several timely applications and case studies.