Likelihood-Free Frequentist Inference

Abstract: Many areas of the physical, engineering and biological sciences make extensive use of computer simulators to model complex systems. Confidence sets and hypothesis testing are the hallmarks of statistical inference, but classical methods are poorly suited for scientific applications involving complex simulators without a tractable likelihood. Recently, many techniques have been introduced that learn a surrogate likelihood using forward-simulated data, but these methods do not guarantee frequentist confidence sets and tests with nominal coverage and Type I error control, respectively.

In this talk, I will describe our recent and ongoing research on developing scalable and modular tools for constructing frequentist confidence sets with finite-sample validity. These tools apply to settings where we have access to a high-fidelity simulator but the likelihood cannot be evaluated and observed data are limited. Rather than relying on large-sample (asymptotic) theory or costly Monte Carlo samples at fixed parameter values, we leverage machine learning tools and simulated data in the neighborhood of a parameter to estimate critical values, p-values, and conditional coverage of confidence sets. We refer to our general machinery as “likelihood-free frequentist inference”. Any method that estimates a test statistic, such as the likelihood ratio, can be plugged into our framework to efficiently compute valid hypothesis tests and confidence sets, and run diagnostics.

Part of this work is joint with Niccolo Dalmasso, Rafael Izbicki and David Zhao. An earlier version of this work can be found in PMLR:

–

Biography: Dr Lee is a professor in the Department of Statistics & Data Science at Carnegie Mellon University, with a joint appointment in the Machine Learning Department. Prior to joining CMU in 2005, she was the J.W. Gibbs Assistant Professor in the department of mathematics at Yale University, and before that she served a year as a visiting research associate at the department of applied mathematics at Brown University. Dr Lee’s interests are in developing statistical methodology for the type of complex data and problems often encountered in the physical sciences. She co-directs the Statistical Methods for the Physical Sciences (STAMPS) research group at CMU, and is key personnel in the recently founded NSF AI Planning Institute for Data-Driven Discovery in Physics.