Scalable Multi-Stage Bayesian Inference in Constrained and Structured Models

Abstract

This dissertation presents novel statistical methodology for fitting complex Bayesian models to constrained and structured data. The central technique used is that of cutting dependence between parameters (often only in one direction) and performing inference sequentially.We develop sequential Gibbs posteriors, which are a new class of measures that allow for direct inference on a sequence of loss minimizers or parameters defined by a sequence of likelihoods. We provide decision-theoretic justification and establish new asymptotic theory under weak assumptions, including what we believe to be the first Bernstein-von Mises theorem on manifolds. Crucially, sequential Gibbs posteriors can provide an excellent characterization of parameter uncertainty without assumptions on the underlying data-generating mechanism. We illustrate this feature through a fully developed application to principal component analysis.Next, we show how the same idea of cutting dependence can be used to scale Bayesian ecological models to very high dimensions while respecting key structures in the data. Our first example involves fitting a joint species distribution model to hundreds of thousands of rare species with computation facilitated through a two-stage approach, first fitting a model to the common species, and then fitting independent rare species models each informed by the previous common species analysis and phylogenetic similarity. The resulting model offers major improvements for predicting rare Malagasy arthropods over competitors. Our second example synthesizes citizen science data and long-term biodiversity projects to produce real-time spatiotemporal maps of bird presence/absences across all of Finland. The model accounts for variance in bird detection, migration patterns, and habitat preference. Computation is enabled by cutting dependence between these components and fitting the model sequentially. Forecasts from the resulting posterior substantially outperform those from long-term monitoring data alone, on both held-out citizen science data and a new independent test dataset collected specifically for this project.