Understanding GPU Programming for Statistical Computation: Studies in Massively Parallel Massive Mixtures

Abstract

This article describes advances in statistical computation for large-scale data analysis in structured Bayesian mixture models via graphics processing unit (CPU) programming. The developments are partly motivated by computational challenges arising in fitting models of increasing heterogeneity to increasingly large datasets. An example context concerns common biological studies using high-throughput technologies generating many, very large datasets and requiring increasingly high-dimensional mixture models with large numbers of mixture components. We outline important strategies and processes for CPU computation in Bayesian simulation and optimization approaches, give examples of the benefits of CPU implementations in terms of processing speed and scale-up in ability to analyze large datasets, and provide a detailed, tutorial-style exposition that will benefit readers interested in developing CPU-based approaches in other statistical models. Novel, CPU-oriented approaches to modifying existing algorithms software design can lead to vast speed-up and, critically, enable statistical analyses that presently will not be performed due to compute time limitations in traditional computational environments. Supplemental materials are provided with all source code, example data, and details that will enable readers to implement and explore the CPU approach in this mixture modeling context.