Stochastics and Statistics Seminar
Feature Learning and Scaling Laws in Two-layer Neural Networks: A high dimensional analysis
March 14 @ 11:00 am - 12:00 pm
Murat A. Erdogdu, University of Toronto
E18-304
Event Navigation
Abstract: This talk will focus on gradient-based optimization of
two-layer neural networks. We consider a high-dimensional setting
where the number of samples and the input dimension are both large and
show that, under different model assumptions, neural networks learn
useful features and adapt to the model more efficiently than classical
methods. Further, we derive scaling laws of the learning dynamics for
the gradient descent, highlighting the power-law dependencies on the
optimization time, and the model width.
Bio: Murat A. Erdogdu is currently an assistant professor at the
University of Toronto in departments of Computer Science and
Statistics. He is also a faculty member of the Vector Institute, and a
CIFAR Chair in AI. Before, he was a postdoctoral researcher at
Microsoft Research – New England. His research interests include
machine learning theory, statistics, and optimization. He obtained his
Ph.D. from the Department of Statistics at Stanford University and he
has an M.S. degree in Computer Science, also from Stanford.
