Generative Models, Normalizing Flows, and Monte Carlo Samplers

Abstract:
Contemporary generative models used in the context of unsupervised learning have primarily been designed around the construction of a map between two probability distributions that transform samples from the first into samples from the second. Advances in this domain have been governed by the introduction of algorithms or inductive biases that make learning this map, and the Jacobian of the associated change of variables, more tractable. The challenge is to choose what structure to impose on the transport to best reach a complex target distribution from a simple one used as base, while maintaining computational efficiency. In this talk, I will formalize this problem and discuss how to construct such transport maps by introducing a continuous-time normalizing flow whose velocity is the minimizer of a simple quadratic loss expressed in terms of expectations that are readily amenable to empirical estimation. The flow can be used to generate samples from either the base or target, and to estimate their likelihood. In addition, this flow can be optimized to minimize the path length in Wasserstein-2 metric, thereby paving the way for building transport maps that are optimal in the sense of Monge-Ampere. I will also also contextualize this approach in its relation to score-based diffusion models that have gained a lot of popularity lately. Finally, I will discuss how such normalizing flows can be used in the context of Monte-Carlo sampling, with applications to the calculation of free energies and Bayes factors.

Based on joint works with Michael Albergo, Marylou Gabrie, and Grant Rotskoff

Bio:
Eric Vanden-Eijnden ( https://wp.nyu.edu/courantinstituteofmathematicalsciences-eve2/ ) is a Professor of Mathematics at the Courant Institute of Mathematical Sciences, New York University. His research focuses on the mathematical and computational aspects of statistical mechanics, with applications to complex dynamical systems arising in molecular dynamics, materials science, atmosphere-ocean science, fluids dynamics, and neural networks. He is also interested in the mathematical foundations of machine learning (ML) and the applications of ML in scientific computing. He is known for the development and analysis of multiscale numerical methods for systems whose dynamics span a wide range of spatio-temporal scales. He is the winner of the Germund Dahlquist Prize and the J.D. Crawford Prize, and a recipient of the Vannevar Bush Faculty Fellowship. He was a plenary speaker at the 2015 International Congress of Industrial and Applied Mathematics (ICIAM) in Beijing and an invited speaker at the 2022 International Congress of Mathematics (ICM).