Learning the governing equations for time-varying measurement data is of great interest across different scientific fields. When such data is moreover highly corrupted, for example, due to the recording mechanism failing over unknown intervals of time, recovering the governing equations becomes quite challenging. In this work, we show that if the data exhibits chaotic behavior, it is possible to recover the underlying governing nonlinear differential equations even if a large percentage of the data is corrupted by outliers, by solving…

Latent variable models have become a key tool for the modern statistician, letting us express complex assumptions about the hidden structures that underlie our data. Latent variable models have been successfully applied in numerous fields. The central computational problem in latent variable modeling is posterior inference, the problem of approximating the conditional distribution of the latent variables given the observations. Posterior inference is central to both exploratory tasks and predictive tasks. Approximate posterior inference algorithms have revolutionized Bayesian statistics, revealing…

I will discuss trend filtering, a newly proposed tool of Steidl et al. (2006), Kim et al. (2009) for nonparametric regression. The trend filtering estimate is defined as the minimizer of a penalized least squares criterion, in which the penalty term sums the absolute kth order discrete derivatives over the input points. I will give an overview of some interesting connections between these estimates and adaptive spline estimation, and also of the provable statistical superiority of trend filtering to other…

Particle filters provide Monte Carlo approximations of intractable quantities, such as likelihood evaluations in state-space models. In many cases, the interest does not lie in the values of the estimates themselves, but in the comparison of these values for various parameters. For instance, we might want to compare the likelihood at two parameter values. Such a comparison is facilitated by introducing positive correlations between the estimators, which is a standard variance reduction technique. In the context of particle filters, this…

In this talk, I will discuss the prediction properties of techniques commonly used to scale up kernel methods and Gaussian processes. In particular, I will focus on data dependent and independent sub-sampling methods, namely Nystrom and random features, and study their generalization properties within a statistical learning theory framework. On the one hand I will show that these methods can achieve optimal learning errors while being computational efficient. On the other hand, I will show that subsampling can be seen…

Henry L. Rietz, the first president of IMS, published his book “Mathematical Statistics” in 1927. One review wrote in 1928: “Professor Rietz has developed this theory so skillfully that the ’workers in other fields’, provided only that they have a passing familiarity with the grammar of mathematics, can secure a satisfactory understanding of the points involved.” In this lecture, I would like to promote the good tradition of mathematical statistics as expressed in Rietzs book in order to gain insight…

Consider an n by n linear system Ax=b. If the right-hand side of the system is known up to a certain error, then in process of the solution, this error gets amplified by the condition number of the matrix A, i.e. by the ratio of its largest and smallest singular values. This observation led von Neumann and his collaborators to consider the condition number of a random matrix and conjecture that it should be of order n. This conjecture was…

Eigendecomposition of quadratic forms guaranteed by the spectral theorem is the foundation for many important algorithms in computer science, data analysis, and machine learning. In this talk I will discuss our recent work on generalizations from quadratic forms to a broad class of functions based on an analogue of the spectral decomposition in an orthogonal basis. We call such functions ``orthogonally decomposable". It turns out that many inferential problems of recent interest including orthogonal tensor decompositions, Independent Component Analysis (ICA),…