Browsing by Subject "Optimal transport"
Results Per Page
Sort Options
Item Open Access Learning deep models via optimal transport distance(2021) Chen, LiqunDistribution matching is a core problem in modern deep learning community. Since most tasks are requiring deep models to estimate the true data distribution. For instance, GAN~\cite{goodfellow2014generative} wants to generate realistic images.In this PhD dissertation, I will discuss how to improve the distribution matching in deep learning models via optimal transport distance. This technique can be applied to variety of tasks, including computer vision, natural language processing areas. In this thesis, I will show that optimal transport can not only help to improve the existing deep models effectively, and also can reduce the scale of large and complex models. This thesis contains four parts to support the above arguments.
In the first part, I show that optimal transport distance can be used for distribution matching in generative adversarial network framework.It will gain the merits from kernel methods, but still easy and robust to train. In the second parts, optimal transport is applied to improve the sequence to sequence models, by introducing the soft bipartite matching scheme. In the third part, I further extend the OT algorithm into graph matching problems by introducing Gromov-Wasserstein distance. So that OT can help to align both the content and structure within the instances. The final part is related to network embedding, OT is designed to model the relationship and content information in the social network or citation network, etc.
With lots of experiment results as evidence, we can conclude that optimal transport distance can help improve deep models in both scale and performance.
Item Open Access Numerical Approximation of Gaussian-Smoothed Optimal Transport(2022) Yang, CongweiThe Optimal Transport (OT) Distance, especially the Wasserstein distance, has important applications in statistics and machine learning. Though the optimal transport distance possesses many favorable properties, it is not widely applicable, especially in high dimensions, due to its computational cost and the "curse of dimensionality". In the past few years, the Sinkhorn Algorithm [Cuturi, 2013], which uses entropy regularization to relieve the computational burden, provides an efficient approximation of the optimal transport distances. Moreover, the recently proposed Gaussian-Smoothed Optimal Transport (GOT) framework by [Goldfeld and Greenewald, 2020] provides potential solution to alleviate the "curse of dimensionality". Furthermore, [Makkuva et al., 2020] proposed a new algorithm that uses the Input Convex Neural Network (ICNN) to represent the optimal transport map with the gradient of convex functions. Inspired by previous works, we addressed the characteristics of different approximation algorithms for Optimal Transport distances and proposed a multiple sampling scheme under the Gaussian-Smoothed Optimal Transport framework. The simulation study shows that the multiple sampling essentially leads to better representation of Gaussian smoothness, and thus provides more accurate approximation, especially in high dimensions. Finally, we proposed a derivation that transforms 2-Wasserstein distance into the mean-width of a convex hull under a specific pair of distribution classes, and thus allows the analytical computation of 2-Wasserstein distances. We further verified this analytical result by Monte-Carlo simulation.