IDSS Distinguished Seminars

Abstract: Causal inference is usually dichotomized into two categories, experimental (Fisher, Cox, Cochran) and observational (Neyman, Rubin, Robins, Dawid, Pearl) which, by and large, are studied separately. Understanding reality is more demanding. Experimental and observational studies are but two extremes of a rich spectrum of research designs that generate the bulk of the data available in practical, large-scale situations. In typical medical explorations, for example, data from multiple observations and experiments are collected, coming from distinct experimental setups, different sampling conditions, and heterogeneous populations.

In this talk, I will introduce the data-fusion problem, which is concerned with piecing together multiple datasets collected under heterogeneous conditions (to be defined) so as to obtain valid answers to queries of interest. The availability of multiple heterogeneous datasets presents new opportunities to causal analysts since the knowledge that can be acquired from combined data would not be possible from any individual source alone. However, the biases that emerge in heterogeneous environments require new analytical tools. Some of these biases, including confounding, sampling selection, and cross-population biases, have been addressed in isolation, largely in restricted parametric models. I will present my work on a general, non-parametric framework for handling these biases and, ultimately, a theoretical solution to the problem of fusion in causal inference tasks.

Suggested readings:

[1] E. Bareinboim and J. Pearl.   Causal inference and the Data-Fusion Problem.   Proceedings of the National Academy of Sciences, 113(27): 7345-7352, 2016.  https://www.pnas.org/content/113/27/7345

[2] E. Bareinboim, J. Correa, D. Ibeling, T. Icard. On Pearl’s Hierarchy and the Foundations of Causal Inference. In “Probabilistic and Causal Inference: The Works of Judea Pearl”, In Probabilistic and Causal Inference: The Works of Judea Pearl (ACM, Special Turing Series), pp. 507-556, 2022.  https://causalai.net/r60.pdf

[3]  K. Xia, K. Lee, Y. Bengio, E. Bareinboim. The Causal-Neural Connection: Expressiveness, Learnability, and Inference In Proceedings of the 35th Annual Conference on Neural Information Processing Systems (NeurIPS), 2021.  https://causalai.net/r80.pdf

About the speaker: Elias Bareinboim is an associate professor in the Department of Computer Science and the director of the Causal Artificial Intelligence (CausalAI) Laboratory at Columbia University. His research focuses on causal and counterfactual inference and their applications to data-driven fields in the health and social sciences as well as artificial intelligence and machine learning. His work was the first to propose a general solution to the problem of “data-fusion,” providing practical methods for combining datasets generated under different experimental conditions and plagued with various biases. More recently, Bareinboim has been exploring the intersection of causal inference with decision-making (including reinforcement learning) and explainability (including fairness analysis). Bareinboim received his Ph.D. from the University of California, Los Angeles, where he was advised by Judea Pearl. Bareinboim was named one of ” 10 to Watch” by IEEE, and is a recipient of the NSF CAREER Award, the ONR Young Investigator Award, the Dan David Prize Scholarship, the 2014 AAAI Outstanding Paper Award, and the 2019 UAI Best Paper Award.

Title: Multiple Randomization Designs

with Patrick Bajari, Brian Burdick, Lorenzo Masoero, James McQueen, Thomas Richardson, and Ido M. Rosen

Abstract: In this study we introduce a new class of experimental designs. In a classical  randomized controlled trial (RCT), or A/B test, a randomly selected subset of a population of units (e.g., individuals, plots of land, or experiences) is assigned to a treatment (treatment A), and the remainder of the population is assigned to the control treatment (treatment B). The difference in average outcome by treatment group is an estimate of the average effect of the treatment. However, motivating our study, the setting for modern experiments is often different, with the outcomes and treatment assignments indexed by multiple populations. For example, outcomes may be indexed by buyers and sellers, by content creators and subscribers, by drivers and riders, or by travelers and airlines and travel agents, with treatments potentially varying across these indices. Spillovers or interference can arise from interactions between units across populations. For example, sellers’ behavior may depend on buyers’ treatment assignment, or vice versa. This can invalidate the simple comparison of means as an estimator for the average effect of the treatment in classical RCTs.We propose new experiment designs for settings in which multiple populations interact. We show how these designs allow us to study questions about interference that cannot be answered by classical randomized experiments. Finally, we develop new statistical methods for analyzing these Multiple Randomization Designs.

About the speaker: Guido Imbens is The Applied Econometrics Professor at the Stanford Graduate School of Business and Professor of Economics in the Economics Department at Stanford University. Currently he is also the Amman Mineral Faculty Fellow at the GSB. He has held tenured positions at UCLA, UC Berkeley, and Harvard University before joining Stanford in 2012. Imbens specializes in econometrics, and in particular methods for drawing causal inferences from experimental and observational data. He has published extensively in the leading economics and statistics journals. Together with Donald Rubin he has published a book, “Causal Inference in Statistics, Social and Biomedical Sciences.” Guido Imbens is a fellow of the Econometric Society, the Royal Holland Society of Sciences and Humanities, the Royal Netherlands Academy of Sciences, the American Academy of Arts and Sciences, and the American Statistical Association. He holds an honorary doctorate from the University of St. Gallen. In 2017 he received the Horace Mann medal at Brown University. In 2021 he shared the Sveriges Riksbank Prize in Economic Sciences in Memory of Alfred Nobel with David Card and Joshua Angrist for “methodological contributions to the analysis of causal relationship.’’ Currently Imbens is Editor of Econometrica.

Register now: We are required to collect contact information for this event, including name, cell phone number and email address. This information is for contact tracing purposes only. Please provide your contact information here if you plan on attending the seminar.

Title: Neural networks: optimization, transition to linearity and deviations therefrom

Abstract: The success of deep learning is due, to a large extent, to the remarkable effectiveness of gradient-based optimization methods applied to large neural networks. I will first discuss some general mathematical principles allowing for efficient optimization in over-parameterized non-linear systems, a setting that includes deep neural networks. I will argue that optimization problems corresponding to these systems are not convex, even locally, but instead satisfy the Polyak-Lojasiewicz (PL) condition on most of the parameter space, allowing for efficient optimization by gradient descent or SGD.

As a separate but related development, I will talk about the remarkable recently discovered phenomenon of transition to linearity (constancy of NTK), when networks become linear functions of their parameters as their width increases. In particular I will talk about a quite general form of the transition to linearity for a broad class of feed-forward networks corresponding to arbitrary directed graphs. It turns out that the width of such networks is characterized by the minimum in-degree of their graphs, excluding the input layer and the first layer.

Finally, I will mention a very interesting deviation from linearity, a so-called “catapult phase”, a recently identified non-linear and, furthermore, non-perturbatative phenomenon, which persists even as neural networks become increasingly linear in the limit of the increasing width.

Based on joint work with Chaoyue Liu, Libin Zhu, Adit Radhakrishnan

About the speaker: Mikhail Belkin received his Ph.D. in 2003 from the Department of Mathematics at the University of Chicago. His research interests are in theory and applications of machine learning and data analysis. Some of his well-known work includes widely used Laplacian Eigenmaps, Graph Regularization and Manifold Regularization algorithms, which brought ideas from classical differential geometry and spectral analysis to data science. His recent work has been concerned with understanding remarkable mathematical and statistical phenomena observed in deep learning. This empirical evidence necessitated revisiting some of the basic concepts in statistics and optimization. One of his key recent findings is the “double descent” risk curve that extends the textbook U-shaped bias-variance trade-off curve beyond the point of interpolation.

Mikhail Belkin is a recipient of a NSF Career Award and a number of best paper and other awards. He has served on the editorial boards of the Journal of Machine Learning Research, IEEE Pattern Analysis and Machine Intelligence and SIAM Journal on Mathematics of Data Science.