Abstract: This talk will focus on gradient-based optimization of two-layer neural networks. We consider a high-dimensional setting where the number of samples and the input dimension are both large and show that, under different model assumptions, neural networks learn useful features and adapt to the model more efficiently than classical methods. Further, we derive scaling laws of the learning dynamics for the gradient descent, highlighting the power-law dependencies on the optimization time, and the model width.
Bio: Murat A. Erdogdu is currently an assistant professor at the University of Toronto in departments of Computer Science and Statistics. He is also a faculty member of the Vector Institute, and a CIFAR Chair in AI. Before, he was a postdoctoral researcher at Microsoft Research – New England. His research interests include machine learning theory, statistics, and optimization. He obtained his Ph.D. from the Department of Statistics at Stanford University and he has an M.S. degree in Computer Science, also from Stanford.