# Browsing by Author "Dunson, David"

###### Results Per Page

###### Sort Options

Item Restricted Bayesian inference of the number of factors in gene-expression analysis: application to human virus challenge studies(BMC BIOINFORMATICS, 2010-11-09) Chen, Bo; Chen, Minhua; Paisley, John; Zaas, Aimee; Woods, Christopher; Ginsburg, Geoffrey S; Hero, Alfred; Lucas, Joseph; Dunson, David; Carin, LawrenceItem Open Access Bayesian Nonparametric Modeling and Theory for Complex Data(2012) Pati, DebdeepThe dissertation focuses on solving some important theoretical and methodological problems associated with Bayesian modeling of infinite dimensional `objects', popularly called nonparametric Bayes. The term `infinite dimensional object' can refer to a density, a conditional density, a regression surface or even a manifold. Although Bayesian density estimation as well as function estimation are well-justified in the existing literature, there has been little or no theory justifying the estimation of more complex objects (e.g. conditional density, manifold, etc.). Part of this dissertation focuses on exploring the structure of the spaces on which the priors for conditional densities and manifolds are supported while studying how the posterior concentrates as increasing amounts of data are collected.

With the advent of new acquisition devices, there has been a need to model complex objects associated with complex data-types e.g. millions of genes affecting a bio-marker, 2D pixelated images, a cloud of points in the 3D space, etc. A significant portion of this dissertation has been devoted to developing adaptive nonparametric Bayes approaches for learning low-dimensional structures underlying higher-dimensional objects e.g. a high-dimensional regression function supported on a lower dimensional space, closed curves representing the boundaries of shapes in 2D images and closed surfaces located on or near the point cloud data. Characterizing the distribution of these objects has a tremendous impact in several application areas ranging from tumor tracking for targeted radiation therapy, to classifying cells in the brain, to model based methods for 3D animation and so on.

The first three chapters are devoted to Bayesian nonparametric theory and modeling in unconstrained Euclidean spaces e.g. mean regression and density regression, the next two focus on Bayesian modeling of manifolds e.g. closed curves and surfaces, and the final one on nonparametric Bayes spatial point pattern data modeling when the sampling locations are informative of the outcomes.

Item Open Access General and Efficient Bayesian Computation through Hamiltonian Monte Carlo Extensions(2017) Nishimura, AkihikoHamiltonian Monte Carlo (HMC) is a state-of-the-art sampling algorithm for Bayesian computation. Popular probabilistic programming languages Stan and PyMC rely on HMC’s generality and efficiency to provide automatic Bayesian inference platforms for practitioners. Despite its wide-spread use and numerous success stories, HMC has several well known pitfalls. This thesis presents extensions of HMC that overcome its two most prominent weaknesses: inability to handle discrete parameters and slow mixing on multi-modal target distributions.

Discontinuous HMC (DHMC) presented in Chapter 2 extends HMC to discontinuous target distributions – and hence to discrete parameter distributions through embedding them into continuous spaces — using an idea of event-driven Monte Carlo from the computational physics literature. DHMC is guaranteed to outperform a Metropolis-within-Gibbs algorithm since, as it turns out, the two algorithms coincide under a specific (and sub-optimal) implementation of DHMC. The theoretical justification of DHMC extends an existing theory of non-smooth Hamiltonian mechanics and of measure-valued differential inclusions.

Geometrically tempered HMC (GTHMC) presented in Chapter 3 improves HMC’s performance on multi-modal target distributions. The efficiency improvement is achieved through differential geometric techniques, relating a target distribution to

another distribution with less severe multi-modality. We establish a geometric theory behind Riemannian manifold HMC to motivate our geometric tempering methods. We then develop an explicit variable stepsize reversible integrator for simulating

Hamiltonian dynamics to overcome a stability issue of the usual Stormer-Verlet integrator. The integrator is of independent interest, being the first of its kind designed specifically for HMC variants.

In addition to the two extensions described above, Chapter 4 describes a variable trajectory length algorithm that generalizes the acceptance and rejection procedure of HMC — and in fact of any reversible dynamics based samplers — to allow for more flexible choices of trajectory lengths. The algorithm in particular enables an effective application of a variable stepsize integrator to HMC extensions, including GTHMC. The algorithm is widely applicable and provides a recipe for constructing valid dynamics based samplers beyond the known HMC variants. Chapter 5 concludes the thesis with a simple and practical algorithm to improve computational efficiencies of HMC and related algorithms over their traditional implementations.

Item Open Access Hitting around the shift: Evaluating batted-ball trends across Major League Baseball(2020-05) Model, Michael W.The infield shift has negatively affected Major League hitters who formerly thrived on ground balls through the gaps in the infield. Nearly a quarter of plate appearances during the 2019 season featured infield shifts, up from 13.8 percent just three seasons prior. I analyzed both the evolution of shift implementation and whether batters hit differently with and without the shift using hierarchical Bayesian regression methods on both pitch-level and batter-tendency data from 2015 to 2019. Since most of the recent talk surrounding the infield shift has been related to a drastic increase in fly balls and players hitting over the shift, I looked specifically at adaptation on the ground. Not a single batter was found to have had a significant difference between their batted-ball distributions for either a given season or throughout the entire five-year period, suggesting the increase in shifting is unlikely to end in the near future.Item Open Access New tools for Bayesian clustering and factor analysis(2022) Song, HanyuTraditional model-based clustering faces challenges when applied to mixed scale multivariate data, consisting of both categorical and continuous variables. In such cases, there is a tendency for certain variables to overly influence clustering. In addition, as dimensionality increases, clustering can becomemore sensitive to kernel misspecification and less reliable. In Chapter 1, we propose a simple local-global Bayesian clustering framework designed to address both of these problems. The model assigns a separate cluster ID to each variable from each subject to define the local component of the model. These local clustering IDs are dependent on a global clustering ID for each subject through a simple hierarchical model. The proposed framework builds on previous related ideas including consensus clustering, the enriched Dirichlet process, and mixed membership models. We show its property of local-global borrowing of information and ease of handling missing data. As a canonical special case, we focus on a simple Dirichlet over-fitted local-global mixture, for which we show that the extra global components of the posterior can be emptied asymptotically. This is the first such result applicable to a broad class of over-fitted finite mixture of mixtures models. We also propose kernel and prior specification for the canonical case and show it leads to a simple Gibbs sampler for posterior computation. We illustrate the approach using simulation studies and applications, through which we see the model is able to identify relevant variables for clustering. Large data have become the norm in many modern applications; they often cannot be easily moved across computers or loaded into memory on a single computer. In such cases, model-based clustering, which typically uses the inherently serial Markov chain Monte Carlo for computation, faces challenges. Existing distributed algorithms have emphasized nonparametric Bayesian mixture models and typically require moving raw data across workers. In Chapter 2, we introduce a nearly embarrassingly parallel algorithm for clustering under a Bayesian overfitted finite mixture of Gaussian mixtures, which we term distributed Bayesian clustering (DIB-C). DIB-C can flexibly accommodate data sets with various shapes (e.g. skewed or multi-modal). With data randomly partitioned and distributed, we first run Markov chain Monte Carlo in an embarrassingly parallel manner to obtain local clustering draws and then refine across workers for a final clustering estimate based on \emph{any} loss function on the space of partitions. DIB-C can also estimate cluster densities, quickly classify new subjects and provide a posterior predictive distribution. Both simulation studies and real data applications show superior performance of DIB-C in terms of robustness and computational efficiency.

Chapter 3develops a simple factor analysis model in light of the need for new models for characterizing dependence in multivariate data. The multivariate Gaussian distribution is routinely used, but cannot characterize nonlinear relationships in the data. Most non-linear extensions tend to be highly complex; for example, involving estimation of a non-linear regression model in latent variables. We propose a relatively simple class of Ellipsoid-Gaussian multivariate distributions, which are derived by using a Gaussian linear factor model involving latent variables having a von Mises-Fisher distribution on a unit hyper-sphere. We show that the Ellipsoid-Gaussian distribution can flexibly model curved relationships among variables with lower-dimensional structures. Taking a Bayesian approach, we propose a hybrid of gradient-based geodesic Monte Carlo and adaptive Metropolis for posterior sampling. We derive basic properties and illustrate the utility of the Ellipsoid-Gaussian distribution on a variety of simulated and real data applications.

Item Open Access Item Open Access Rat intersubjective decisions are encoded by frequency-specific oscillatory contexts.(Brain Behav, 2017-06) Schaich Borg, Jana; Srivastava, Sanvesh; Lin, Lizhen; Heffner, Joseph; Dunson, David; Dzirasa, Kafui; de Lecea, LuisINTRODUCTION: It is unknown how the brain coordinates decisions to withstand personal costs in order to prevent other individuals' distress. Here we test whether local field potential (LFP) oscillations between brain regions create "neural contexts" that select specific brain functions and encode the outcomes of these types of intersubjective decisions. METHODS: Rats participated in an "Intersubjective Avoidance Test" (IAT) that tested rats' willingness to enter an innately aversive chamber to prevent another rat from getting shocked. c-Fos immunoreactivity was used to screen for brain regions involved in IAT performance. Multi-site local field potential (LFP) recordings were collected simultaneously and bilaterally from five brain regions implicated in the c-Fos studies while rats made decisions in the IAT. Local field potential recordings were analyzed using an elastic net penalized regression framework. RESULTS: Rats voluntarily entered an innately aversive chamber to prevent another rat from getting shocked, and c-Fos immunoreactivity in brain regions known to be involved in human empathy-including the anterior cingulate, insula, orbital frontal cortex, and amygdala-correlated with the magnitude of "intersubjective avoidance" each rat displayed. Local field potential recordings revealed that optimal accounts of rats' performance in the task require specific frequencies of LFP oscillations between brain regions in addition to specific frequencies of LFP oscillations within brain regions. Alpha and low gamma coherence between spatially distributed brain regions predicts more intersubjective avoidance, while theta and high gamma coherence between a separate subset of brain regions predicts less intersubjective avoidance. Phase relationship analyses indicated that choice-relevant coherence in the alpha range reflects information passed from the amygdala to cortical structures, while coherence in the theta range reflects information passed in the reverse direction. CONCLUSION: These results indicate that the frequency-specific "neural context" surrounding brain regions involved in social cognition encodes outcomes of decisions that affect others, above and beyond signals from any set of brain regions in isolation.Item Open Access Relating Traits to Electrophysiology using Factor Models(2020) Talbot, Austin BTargeted stimulation of the brain has the potential to treat mental illnesses. The objective of this work is to develop methodology that enables scientists to design stimulation methods based on the electrophysiological dynamics. We first develop several factor models that characterize aspects of the dynamics relevant to these illnesses. Using a novel approach, we can then find a single predictive factor of the trait of interest. To improve the quality of the associated loadings, we develop a method for removing concomitant variables that can dominate the observed dynamics. We also develop a novel inference technique that increases the relevance of the predictive loadings. Finally, we demonstrate the efficacy of our methodology by finding a single factor responsible for social behavior. This factor is stimulated in new subjects and modifies behavior in the new individuals. These results indicate that our methodology has high potential in developing future cures of mental illness.

Item Open Access Rethinking Nonlinear Instrumental Variables(2019) Li, ChunxiaoInstrumental variable (IV) models are widely used in the social and health sciences in situations where a researcher would like to measure a causal eect but cannot perform an experiment. Formally checking the assumptions of an IV model with a given dataset is impossible, leading many researchers to take as given a linear functional form and two-stage least squares tting procedure. In this paper, we propose a method for evaluating the validity of IV models using observed data and show that, in some cases, a more flexible nonlinear model can address violations of the IV conditions. We also develop a test that detects violations in the instrument that are present in the observed data. We introduce a new version of the validity check that is suitable for machine learning and provides optimization-based techniques to answer these questions. We demonstrate the method using both the simulated data and a real-world dataset.

Item Open Access Some Advances in Nonparametric Statistics(2023) Zhu, YichenNonparametric statistics is an important branch of statistics that utilize infinite dimensional modelsto achieve great flexibility. However, such flexibility often comes with difficulties in computations and convergent properties. One approach is to study the natural patterns for one type of datasets and summarize such patterns into mathematical assumptions that can potentially provide computational and theoretical benefits. I carried out the above idea on three different problems. The first problem is the classification trees for imbalanced datasets, where I formulate the regularity of shapes into surface-to-volumeratio and develop satisfactory theory and methodology using this ratio. The second problem is the approximation of Gaussian process, where I observe the critical role of spatial decaying covariance function in Gaussian process approximations and use such decaying properties to prove the approximation error for my proposed method. The last problem is the posterior contraction rates in Kullback-Leibler (KL) divergence, where I am motivated by the dismatch between KL divergence and Hellinger distance and develop a posterior contraction theory entirely based on KL divergence

Item Open Access Supervised Autoencoders Learn Robust Joint Factor Models of Neural Activity.(CoRR, 2020) Talbot, Austin; Dunson, David; Dzirasa, Kafui; Carlson, DavidFactor models are routinely used for dimensionality reduction in modeling of correlated, high-dimensional data. We are particularly motivated by neuroscience applications collecting high-dimensional `predictors' corresponding to brain activity in different regions along with behavioral outcomes. Joint factor models for the predictors and outcomes are natural, but maximum likelihood estimates of these models can struggle in practice when there is model misspecification. We propose an alternative inference strategy based on supervised autoencoders; rather than placing a probability distribution on the latent factors, we define them as an unknown function of the high-dimensional predictors. This mapping function, along with the loadings, can be optimized to explain variance in brain activity while simultaneously being predictive of behavior. In practice, the mapping function can range in complexity from linear to more complex forms, such as splines or neural networks, with the usual tradeoff between bias and variance. This approach yields distinct solutions from a maximum likelihood inference strategy, as we demonstrate by deriving analytic solutions for a linear Gaussian factor model. Using synthetic data, we show that this function-based approach is robust against multiple types of misspecification. We then apply this technique to a neuroscience application resulting in substantial gains in predicting behavioral tasks from electrophysiological measurements in multiple factor models.