Advances in Gaussian Process Computation, Modeling, and Optimization

Abstract

Gaussian Process (GP) models are powerful tools for flexible modeling and uncertainty quantification in scientific applications, yet they face significant challenges regarding computational scalability and sample efficiency in high dimensions. This dissertation addresses these limitations through novel methodological advances in computation, modeling, and optimization.To tackle computational bottlenecks, we improve on two families of computational techniques. We first develop Trigonometric Quadrature Fourier Features, a method that approximates GP regression via Bayesian linear regression. By deriving features from a novel trigonometric quadrature rule, TQFF reduces computational cost while achieving lower approximation error than existing Random Fourier Feature methods. We also introduce ProSpar-GP, a scalable inducing point method designed for massive non-stationary datasets. By extending a Product of Experts framework, ProSpar-GP allows varying length-scales across the input space, significantly improving the modeling of non-stationary functions compared to standard sparse GP approaches.The second part focuses on improving sample efficiency by incorporating prior structural information. We develop the Additive Multi-Index GP (AdMIn-GP) to emulate high-dimensional Quark-Gluon Plasma physics simulations. This model probabilistically models low-dimensional structures inherent in multi-physics dynamics to overcome the curse of dimensionality. We develop novel a novel variational inference approach model to fit this low dimensional structure. We also propose Concave Spline GP Bandits, which accelerate optimization by conditioning the GP prior on concavity information, a common structural property in medical and business applications. Finally, Chapter 6 explores Multivariate Linear Dynamic Models (MVDLM), which are closely related to Gaussian Process models. We present a Bayesian time series multivariate dynamic modeling framework for causal inference. We show how the MVDLM framework can be used to perform Synthetic Control analysis for causal inference.