Browsing by Subject "Deep generative models"

High-dimensional probability distributions are important objects in a wide variety of applications.

Generative models provide an excellent manipulation method for training from rich available unlabeled data set and sampling new data points from underlying high-dimensional probability distributions.

The recent proposed Variational auto-encoders (VAE) framework is an efficient high-dimensional inference method to modeling complicated data manifold in an approximate Bayesian way, i.e., variational inference.

We first discuss how to design fast stochastic backpropagation algorithm for the VAE based amortized variational inference method.

Particularly, we propose second order Hessian-free optimization method for Gaussian latent variable models and provide a theoretical justification to the convergence of Monte Carlo estimation in our algorithm.

Then, we apply the amortized variational inference to a dynamic modeling application in flu diffusion task.

Compared with traditional approximate Gibbs sampling algorithm, we make less assumption to the infection rate.

Differing from the maximum likelihood approach of VAE, Generative Adversarial Networks (GAN) is trying to solve the generation problem from a game theoretical way.

From this viewpoint, we design a framework VAE+GAN, by placing a discriminator on top of auto-encoders based model and introducing an extra adversarial loss.

The adversarial training induced by the classification loss is to make the discriminator believe the sample from the generative model is as real as the one from the true dataset.

This trick can practically improve the quality of generation samples, demonstrated on images and text domains with elaborately designed architectures.

Additionally, we validate the importance of generative adversarial loss with the conditional generative model in two biological applications: approximate Turing pattern PDEs generation in synthetic/system biology, and automatic cardiovascular disease detection in medical imaging processing.

Deep generative models have achieved tremendous success in recent years, with applications in various tasks involving vision and language intelligence. In this dissertation, I will mainly discuss the contributions that I have made in this field during my Ph.D. study. Specifically, the dissertation is divided into two parts.

In the first part, I will mainly focus on one specific kind of deep directed generative model, called Sigmoid Belief Network (SBN). First, I will present a fully Bayesian algorithm for efficient learning and inference of SBN. Second, since the original SBN can be only used for binary image modeling, I will also discuss the generalization of it to model spare count-valued data for topic modeling, and sequential data for motion capture synthesis, music generation and dynamic topic modeling.

In the second part, I will mainly focus on visual captioning (i.e., image-to-text generation), and conditional image synthesis. Specifically, I will first present Semantic Compositional Network for visual captioning, and emphasize interpretability and controllability revealed in the learning algorithm, via a mixture-of-experts design, and the usage of detected semantic concepts. I will then present Triangle Generative Adversarial Network, which is a general framework that can be used for joint distribution matching and learning the bidirectional mappings between two different domains. We consider the joint modeling of image-label, image-image and image-attribute pairs, with applications in semi-supervised image classification, image-to-image translation and attribute-based image editing.