A Flexible Defense Against the Winner’s Curse
October 25 @ 11:00 am - 12:00 pm
Tijana Zrnic, Stanford University
E18-304
Event Navigation
Abstract:
Across science and policy, decision-makers often need to draw conclusions about the best candidate among competing alternatives. For instance, researchers may seek to infer the effectiveness of the most successful treatment or determine which demographic group benefits most from a specific treatment. Similarly, in machine learning, practitioners are often interested in the population performance of the model that empirically performs best. However, cherry-picking the best candidate leads to the winner’s curse: the observed performance for the winner is biased upwards, rendering conclusions based on standard measures of uncertainty invalid. We introduce a novel approach for valid inference on the winner. Our method is flexible: it handles arbitrary dependence between candidates and is entirely nonparametric. It automatically adapts to the level of selection bias; in particular, it recovers standard, uncorrected inference when the winner stands out and becomes increasingly conservative when there are multiple competitive candidates. The robust underpinnings of the method allow easy extensions to important related problems, such as inference on the top k winners, inference on the value and identity of the population winner, and inference on “near-winners.”
Tijana Zrnic is a Ram and Vijay Shriram Data Science Postdoctoral Fellow at Stanford University, where she is hosted by Emmanuel Candès in the Department of Statistics. Her research establishes foundations to ensure data-driven technologies have a positive impact. Tijana earned her PhD in Electrical Engineering and Computer Sciences from UC Berkeley in 2023, where she was advised by Moritz Hardt and Michael Jordan. Her doctoral research explored prediction and statistical inference in feedback loops, including topics such as performative prediction, prediction-powered inference, and mitigating selection bias. Before her PhD, Tijana completed a BEng in Electrical and Computer Engineering at the University of Novi Sad in Serbia.