# Browsing by Subject "variational inference"

###### Results Per Page

###### Sort Options

Item Open Access Multimodal Probabilistic Inference for Robust Uncertainty Quantification(2021) Jerfel, GhassenDeep learning models, which form the backbone of modern ML systems, generalize poorly to small changes to the data distribution. They are also bad at signalling failure, making predictions with high confidence when their training data or fragile assumptions make them unlikely to make reasonable decisions. This lack of robustness makes it difficult to trust their use in safety-critical settings. Accordingly, there is a pressing need to equip models with a notion of uncertainty to understand their failure modes and detect when their decisions cannot be used or require intervention. Uncertainty quantification is thus crucial for ML systems to work consistently on real-world data and fail loudly when they don’t.One growing line of research on uncertainty quantification is probabilistic modelling which is concerned with capturing model uncertainty by placing a distribution over the models which can be marginalized at test-time. This is especially useful in underspecified models which can have diverse near-optimal solutions, at training time, with similar population-level performance. However, probabilistic modelling approaches such as Bayesian neural networks (BNN) do not scale well in terms of memory and runtime and often underperform simple deterministic baselines in terms of accuracy. Furthermore, BNNs underperform deep ensembles as they fail to explore multiple modes, in the loss space, while being effective at capturing uncertainty within a single mode.

In this thesis, we develop multimodal representations of model uncertainty that can capture a diverse set of hypotheses. We first propose a scalable family of BNN priors (and corresponding approximate posteriors) that combine the local (i.e. within-mode) uncertainty with mode averaging to deliver robust and calibrated uncertainty estimates in addition to improving accuracy both in and out of distribution. We then leverage a multimodal representation of uncertainty to modulate the amount of information transfer between tasks in meta-learning. Our proposed framework integrates Bayesian non-parametric mixtures with deep learning to enable NNs to adapt their capacity as more data is observed which is crucial for lifelong learning. Finally, we propose to replace the reverse Kullback-Leibler divergence (RKL), known for its mode-seeking behavior and for underestimating posterior covariance, with the forward KL (FKL) divergence in a theoretically-guided novel inference procedure. This ensures the efficient combination of variational boosting with adaptive importance sampling. The proposed algorithm offers a well-defined compute-accuracy trade-off and is guaranteed to converge to the optimal multimodal variational solution as well as the optimal importance sampling proposal distribution.

Item Open Access Partition function estimation in computational protein design with continuous-label Markov random fields(2017-05-04) Mukund, AdityaProteins perform a variety of biological tasks, and drive many of the dynamic processes that make life possible. Computational structure-based protein design (CSPD) involves computing optimal sequences of amino acids with respect to particular backbones, or folds, in order to produce proteins with novel functions. In particular, it is crucial to be able to accurately model protein-protein interfaces (PPIs) in order to realize desired functionalities. Accurate modeling of PPIs raises two significant considerations. First, incorporating continuous side-chain flexibility in the design process has been shown to significantly improve the quality of designs. Second, because proteins exist as ensembles of structures, many of the properties we wish to design, including binding affinity, require the computation of ensemble properties as opposed to features of particular conformations. The bottleneck in many design algorithms that attempt to handle the ensemble nature of protein structure, including the Donald Lab’s K ∗ algorithm, is the computation of the partition function, which is the sum of the Boltzmann-weighted energies of all the conformational states of a protein or protein-ligand complex. Protein design can be formulated as an inference problem on Markov random fields (MRFs), where each residue to be designed is represented by a node in the MRF and an edge is placed between nodes corresponding to interacting residues. Label sets on each vertex correspond to allowed flexibility in the underlying design problem. The aim of this work is to extend message-passing algorithms that estimate the partition function for Markov random fields with discrete label sets to MRFs with continuous label sets in order to compute the partition function for PPIs with continuous flexibility and continuous entropy.Item Open Access Variational Inference for Nonlinear Regression Using Dimension Reduced Mixtures of Generalized Linear Models with Application to Neural Data(2015) Subramanian, Vivek AnandBrain-machine interfaces (BMIs) are devices that transform neural activity into commands executed by a robotic actuator. For paraplegics who have suffered spinal cord injury and for amputees, BMIs provide an avenue to regain lost limb mobility by providing a direct connection between the brain and an actuator. One of the most important aspects of a BMI is the decoding algorithm, which interprets patterns of neural activity and issues an appropriate kinematic action. The decoding algorithm relies heavily on a neural tuning function for each neuron which describes the response of that neuron to an external stimulus or upcoming motor action. Modern BMI decoders assume a simple parametric form for this tuning function such as cosine, linear, or quadratic, and fit parameters of the chosen function to a training data set. While this may be appropriate for some neurons, tuning curves for all neurons may not all take the same parametric form; hence, performance of BMI decoding may suffer because of an inappropriate mapping from firing rate to kinematic. In this work, we develop a non-parametric model for the identification of non-linear tuning curves with arbitrary shape. We also develop an associated variational Bayesian (VB) inference scheme which provides a fast, big data-friendly method to obtain approximate posterior distributions on model parameters. We demonstrate our model's capabilities on both simulated and experimental datasets.