Understanding Machine Learning with Statistical Physics

Abstract:

The affinity between statistical physics and machine learning has long history, this is reflected even in the machine learning terminology that is in part adopted from physics. Current theoretical challenges and open questions about deep learning and statistical learning call for unified account of the following three ingredients: (a) the dynamics of the learning algorithm, (b) the architecture of the neural networks, and (c) the structure of the data. Most existing theories are not taking in account all of those three aspects in a satisfactory manner. In this talk I will describe some of the results stemming from statistical physics applied to machine learning and how it does include the three ingredients, although in a very simplified manner. Then I will focus on the current results improving our modelling in each of the three aspects covering recent articles [1-4].

[1] Aubin, B., Maillard, A., Krzakala, F., Macris, N., & Zdeborová, L.; The committee machine: Computational to statistical gaps in learning a two-layers neural network. NeurIPS’18.
[2] Sarao Mannelli, S., Biroli, G., Cammarota, C., Krzakala, F., & Zdeborová, L.; Who is Afraid of Big Bad Minima? Analysis of Gradient-Flow in a Spiked Matrix-Tensor Model. NeurIPS’19.
[3] Aubin, B., Loureiro, B., Maillard, A., Krzakala, F., & Zdeborová, L.; The spiked matrix model with generative priors. NeurIPS’19.
[4] Goldt, S., Mézard, M., Krzakala, F., & Zdeborová, L.; Modelling the influence of data structure on learning in neural networks. Preprint arXiv:1909.11500.

Biography:

Lenka Zdeborová is a researcher at CNRS working in the Institute of Theoretical Physics in CEA Saclay, France. She received a PhD in physics from University Paris-Sud and from Charles University in Prague in 2008. She spent two years in the Los Alamos National Laboratory as the Director’s Postdoctoral Fellow. In 2014, she was awarded the CNRS bronze medal, in 2016 Philippe Meyer prize in theoretical physics and an ERC Starting Grant, in 2018 the Irène Joliot-Curie prize. She is editorial board member for Journal of Physics A, Physical review E and Physical Review X.  Lenka’s expertise is in applications of methods developed in statistical physics, such as advanced mean field methods, replica method and related message passing algorithms, to problems in machine learning, signal processing, inference and optimization.

–
The MIT Statistics and Data Science Center hosts guest lecturers from around the world in this weekly seminar.