Principal Components Analysis in Light of the Spiked Model

Principal components is a true workhorse of science and technology, applied everywhere from radio frequency signal processing to financial econometrics, genomics, and social network analysis. In this talk, I will review some of these applications and then describe the challenge posed by modern ‘big data asymptotics’ where there are roughly as many dimensions as observations; this setting has seemed in the past full of mysteries. Over the last ten years random matrix theory has developed a host of new tools that now can be deployed to meet this challenge; I will describe these new tools and show how I think we have recently crossed a threshold where some of the former mysteries are now cleared up. This is joint work with Matan Gavish (Hebrew Univ), Iain Johnstone and Edgar Dobriban (Stanford).