Learning deep models via optimal transport distance

Abstract

Distribution matching is a core problem in modern deep learning community. Since most tasks are requiring deep models to estimate the true data distribution. For instance, GAN~\cite{goodfellow2014generative} wants to generate realistic images.In this PhD dissertation, I will discuss how to improve the distribution matching in deep learning models via optimal transport distance. This technique can be applied to variety of tasks, including computer vision, natural language processing areas. In this thesis, I will show that optimal transport can not only help to improve the existing deep models effectively, and also can reduce the scale of large and complex models. This thesis contains four parts to support the above arguments.In the first part, I show that optimal transport distance can be used for distribution matching in generative adversarial network framework.It will gain the merits from kernel methods, but still easy and robust to train. In the second parts, optimal transport is applied to improve the sequence to sequence models, by introducing the soft bipartite matching scheme. In the third part, I further extend the OT algorithm into graph matching problems by introducing Gromov-Wasserstein distance. So that OT can help to align both the content and structure within the instances. The final part is related to network embedding, OT is designed to model the relationship and content information in the social network or citation network, etc.With lots of experiment results as evidence, we can conclude that optimal transport distance can help improve deep models in both scale and performance.