Bayesian Decoupling: A Decision Theory-Based Approach to Bayesian Variable Selection

The spike and slab prior offers a canonical approach to Bayesian variable selection, with caveats known to be data dimension and correlation. Motivated by the pitfalls of the spike and slab prior on high dimensional correlated data, this paper introduces Bayesian decoupling (BD, proposed by Hahn and Carvalho [HC15]) as a decision theory-based approach to inducing sparsity on posterior. We formalize the decision theoretical foundation of BD, and argue that BD conducts a sparsification over the posterior mean with a tolerable degradation of the predictive ability. Moreover, the application of BD in sparse estimation motivates the notion of decoupled model fitting and variable estimation, which is an idea rooted in Bayesian decision theory stating that variable estimation should be explicitly recovered as a decision making problem after the model fitting stage. We suggest a broader use of BD in Bayesian statistics, emphasizing that it allows multiple estimation tasks to be carried out simultaneously under a single prior by using different loss functions for different estimation purposes. Our simulation results show that BD with appropriately defined loss functions leads to a desired support recovery with low MSE and FDR and offers an accurate representation of the posterior belief.