Browsing by Subject "Bayesian variable selection"

Fitting statistical models is computationally challenging when the sample size or the dimension of the dataset is huge. An attractive approach for down-scaling the problem size is to first partition the dataset into subsets and then fit using distributed algorithms. The dataset can be partitioned either horizontally (in the sample space) or vertically (in the feature space), and the challenge arise in defining an algorithm with low communication, theoretical guarantees and excellent practical performance in general settings. For sample space partitioning, I propose a MEdian Selection Subset AGgregation Estimator ({\em message}) algorithm for solving these issues. The algorithm applies feature selection in parallel for each subset using regularized regression or Bayesian variable selection method, calculates the `median' feature inclusion index, estimates coefficients for the selected features in parallel for each subset, and then averages these estimates. The algorithm is simple, involves very minimal communication, scales efficiently in sample size, and has theoretical guarantees. I provide extensive experiments to show excellent performance in feature selection, estimation, prediction, and computation time relative to usual competitors.

While sample space partitioning is useful in handling datasets with large sample size, feature space partitioning is more effective when the data dimension is high. Existing methods for partitioning features, however, are either vulnerable to high correlations or inefficient in reducing the model dimension. In the thesis, I propose a new embarrassingly parallel framework named {\em DECO} for distributed variable selection and parameter estimation. In {\em DECO}, variables are first partitioned and allocated to m distributed workers. The decorrelated subset data within each worker are then fitted via any algorithm designed for high-dimensional problems. We show that by incorporating the decorrelation step, DECO can achieve consistent variable selection and parameter estimation on each subset with (almost) no assumptions. In addition, the convergence rate is nearly minimax optimal for both sparse and weakly sparse models and does NOT depend on the partition number m. Extensive numerical experiments are provided to illustrate the performance of the new framework.

For datasets with both large sample sizes and high dimensionality, I propose a new "divided-and-conquer" framework {\em DEME} (DECO-message) by leveraging both the {\em DECO} and the {\em message} algorithm. The new framework first partitions the dataset in the sample space into row cubes using {\em message} and then partition the feature space of the cubes using {\em DECO}. This procedure is equivalent to partitioning the original data matrix into multiple small blocks, each with a feasible size that can be stored and fitted in a computer in parallel. The results are then synthezied via the {\em DECO} and {\em message} algorithm in a reverse order to produce the final output. The whole framework is extremely scalable.

Functional magnetic resonance imaging (fMRI) is a major neuroimaging methodology and have greatly facilitate basic cognitive neuroscience research. However, there are multiple statistical challenges in the analysis of fMRI data, including, dimension reduction, multiple testing and inter-dependence of the MRI responses. In this thesis, a spatial Bayesian variable selection (BVS) model is proposed for the analysis of multi-subject fMRI data. The BVS framework simultaneously account for uncertainty in model specific parameters as well as the model selection process, solving the multiple testing problem. A spatial prior incorporate the spatial relationship of the MRI response, accounting for their inter-dependence. Compared to the non-spatial BVS model, the spatial BVS model enhances the sensitivity and accuracy of identifying activated voxels.

Although there is a rich literature on scalable methods for dimensionality reduction, the focus has been on widely applicable approaches which, in certain applications, are far from optimal or not even applicable. Dimensionality reduction can improve scalability of Bayesian computation, but optimal performance needs tailoring to the model. What kind of dimensionality reduction is sensible in data applications varies by the context of the data, resulting in neglect of information contained in data that do not fit general approaches.

This dissertation introduces dimensionality reduction methods tailored to specific computational or data applications. Firstly, we scale up posterior computation in Bayesian linear regression using a dimensionality reduction approach enabled by the linearity in the model. It approximately integrates out nuisance parameters from a high-dimensional likelihood. The resulting posterior approximation scheme is competitive with state-of-the-art scalable posterior inference methods while being easier to interpret, understand, and analyze due to the explicit use of dimensionality reduction. Bayesian variable selection is considered as an example of a challenging posterior where the dimensionality reduction speeds up computation greatly and accurately.

Secondly, we show how to reduce dimensionality based on data context in varying-domain functional data, where existing methods do not apply. The data of interest are intraoperative blood pressure and heart rate measurements. The first proposed approach extracts multiple different low-dimensional features from the high-dimensional blood pressure data, which are partly predefined and partly learnt from the data. This yields insights regarding blood pressure variability new to the clinical literature since such detailed inference was not possible with existing methods. The concluding case of dimensionality reduction is quantifying coupling of blood pressure and heart rate. This reduces two time series to one measurement of the strength of coupling. The results show the utility for inference methods of dimensionality reduction that is tailored to the challenge at hand.