Latent variable models have become a key tool for the modern statistician, letting us express complex assumptions about the hidden structures that underlie our data. Latent variable models have been successfully applied in numerous fields. The central computational problem in latent variable modeling is posterior inference, the problem of approximating the conditional distribution of the latent variables given the observations. Posterior inference is central to both exploratory tasks and predictive tasks. Approximate posterior inference algorithms have revolutionized Bayesian statistics, revealing…

Learning the governing equations for time-varying measurement data is of great interest across different scientific fields. When such data is moreover highly corrupted, for example, due to the recording mechanism failing over unknown intervals of time, recovering the governing equations becomes quite challenging. In this work, we show that if the data exhibits chaotic behavior, it is possible to recover the underlying governing nonlinear differential equations even if a large percentage of the data is corrupted by outliers, by solving…

The energy of data is the value of a real function of distances between data in metric spaces. The name energy derives from Newton's gravitational potential energy which is also a function of distances between physical objects. One of the advantages of working with energy functions (energy statistics) is that even if the observations/data are complex objects, like functions or graphs, we can use their real valued distances for inference. Other advantages will be illustrated and discussed in the talk.…

Confidence sets play a fundamental role in statistical inference. In this paper, we consider confidence intervals for high dimensional linear regression with random design. We first establish the convergence rates of the minimax expected length for confidence intervals in the oracle setting where the sparsity parameter is given. The focus is then on the problem of adaptation to sparsity for the construction of confidence intervals. Ideally, an adaptive confidence interval should have its length automatically adjusted to the sparsity of…

We discuss the possibilities and limitations of estimating the mean of a real-valued random variable from independent and identically distributed observations from a non-asymptotic point of view. In particular, we define estimators with a sub-Gaussian behavior even for certain heavy-tailed distributions. We also prove various impossibility results for mean estimators. These results are in http://arxiv.org/abs/1509.05845, to appear in Ann Stat. (Joint work with L. Devroye, M. Lerasle, and G. Lugosi.)

Data in the form of pairwise comparisons between a collection of n items arises in many settings, including voting schemes, tournament play, and online search rankings. We study a flexible non-parametric model for pairwise comparisons, under which the probabilities of outcomes are required only to satisfy a natural form of stochastic transitivity (SST). The SST class includes a large family of classical parametric models as special cases, among them the Bradley-Terry-Luce and Thurstone models, but is substantially richer. We provide…

While simple supervised learning problems like binary classification and regression are fairly well understood, increasingly, many applications involve more complex learning problems: more complex label and prediction spaces, more complex loss structures, or both. The first part of the talk will discuss recent advances in our understanding of such problems, including the notion of convex calibration dimension of a loss function, unified approaches for designing convex calibrated surrogates for arbitrary losses, and connections between supervised learning and property elicitation. The…

Motivated by machine learning problems over large data sets and distributed optimization over networks, we consider the problem of minimizing the sum of a large number of convex component functions. We study incremental gradient methods for solving such problems, which process component functions sequentially one at a time. We first consider deterministic cyclic incremental gradient methods (that process the component functions in a cycle) and provide new convergence rate results under some assumptions. We then consider a randomized incremental gradient…