Adaptive Experimentation and Decision-Making: From Bayesian Optimization to Multi-Armed Bandits

Abstract

This dissertation presents novel statistical methodology for the enhancement of datacollection and sequential decision-making across diverse applications.First, we consider Bayesian optimization for stochastic objectives, where the functionto be optimized depends both on control parameters and so-called "noise parameters." This setting arises in many scientific applications, including robust engineering design. We introduce Targeted Variance Reduction (TVR), a novel acquisition function that jointly selects control and noise parameters to reduce variance in the objective function within promising regions of the input space. Numerical experiments and an application to the design of automobile brake discs demonstrate TVR’s advantages over existing methods.Next, we address scenarios where the goal is not a single optimal solution, but rathera set of diverse, high-performing alternatives. This need is exemplified in real-time engine control for aviation, where stable operation requires multiple feasible control strategies. To this end, we introduce Expected Diverse Utility (EDU), a Bayesian optimization method that seeks a collection of locally optimal solutions within a given tolerance of the global optimum. We validate EDU’s effectiveness through numerical experiments and then apply it to a rover trajectory optimization task and data generation for real-time engine control.Beyond sequential experimentation via Bayesian optimization, we present a new ap-proach to static experimental design tailored to Gaussian process (GP) models, which are widely used in Bayesian optimization and scientific modeling. Cost constraints in exper- imental settings often limit the number of observations, making careful design crucial for model performance. We propose Input Warping Designs (IWDs), which use a neural net- work transformation to optimize inter-point distances, improving GP kernel parameter esti- mation while maintaining strong space-filling properties. Experiments show that IWDs can enhance prediction with GPs and serve as strong initial designs for Bayesian optimization.Last, we turn to online experimentation, where data collection and decision-making oc-cur simultaneously. We focus on recommender systems in e-commerce, where retailers must personalize recommendations based on user preferences. Traditional discrete choice models assume users consider all recommended items before purchasing, which is often unrealistic. Thus, we propose a novel modeling framework that accounts for limited customer atten- tion and uncertain preferences. We then analyze the behavior of a contextual multi-armed bandit approach for sequential batch recommendations using the proposed model.