Adaptivity in Domain Adaptation and Friends

Abstract:
Domain adaptation, transfer, multitask, meta, few-shots, representation, or lifelong learning … these are all important recent directions in ML that all touch at the core of what we might mean by ‘AI’. As these directions all concern learning in heterogeneous and ever-changing environments, they all share a central question: what information a data distribution may have about another, critically, in the context of a given estimation problem, e.g., classification, regression, bandits, etc.

Our understanding of these problems is still rather fledgling. I plan to present both some recent positive results and also some negative ones. On one hand, recent measures of discrepancy between distributions, fine-tuned to given estimation problems (classification, bandits, etc) offer a more optimistic picture than existing probability metrics (e.g. Wasserstein, TV) or divergences (KL, Renyi, etc) in terms of achievable rates. On the other hand, when considering seemingly simple extensions of basic settings, e.g., extensions to multiple choices of source datasets (as in multitask or multi source learning), or extensions to multiple prediction models to transfer (i.e., model selection under distribution shift), it turns out that minimax oracle rates are not always adaptively achievable, i.e., domain knowledge is necessary. Such negative results suggest that these problems are more structured in practice than what usual formalisms are so far able to capture.

The talk will be based on joint work with collaborators over the last few years, namely, G. Martinet, S. Hanneke, J. Suk, Y. Mahdaviyeh.

Bio:
Samory Kpotufe is an Associate Professor in Statistics at Columbia University. He works in statistical machine learning, with an emphasis on common nonparametric methods (e.g., kNN, trees, kernel averaging) and a particular interest in adaptivity (i.e., how to automatically leverage beneficial aspects of data as opposed to designing specifically for each scenario). He has previously held positions at the Max Planck Institute in Germany, the Toyota Technological Institute at Chicago, and Princeton University, and was a recent visiting member at the Institute of Advanced Study at Princeton.