Trustworthy Statistical Inference with Black-Box Predictions
Statistics Seminars: Fall 2025
Department of Mathematical Sciences, IU Indianapolis
Organizer: Honglang Wang (hlwang at iu dot edu)
Talk time: 12:15-1:15pm (EST), 09/02/2025, Tuesday
Zoom Meetings: We host our seminars via zoom meetings: Join from computer or mobile by clicking: Zoom to Join or use Meeting ID: 845 0989 4694 with Password: 113959 to join.
Title: Trustworthy Statistical Inference with Black-Box Predictions
Abstract: In biomedical studies involving electronic health records, manually extracting gold-standard phenotype data is labor-intensive and limited in scale. The rise of generative AI offers a systematic and significantly faster alternative through black box generated predictions. However, directly substituting gold-standard data with these predictions, without addressing their differences, can introduce biases and lead to misleading conclusions. To address this challenge, we adopt a semi-supervised learning framework that integrates both labeled data (with gold-standard annotations) and unlabeled data (without gold-standard annotations) under the covariate shift paradigm. We propose doubly robust and semiparametrically efficient estimators to infer general target parameters. Through a rigorous efficiency analysis, we compare scenarios with and without the incorporation of black box generated predictions. Furthermore, we situate our approach within existing literature, drawing connections to prediction-powered inference and its extensions, as well as some seemingly unrelated concept such as surrogacy. To validate our theoretical findings, we conduct extensive synthetic experiments and apply our method to real-world data, demonstrating its practical advantages.
Bio: Dr. Jiwei Zhao is currently an Associate Professor at the University of Wisconsin-Madison, jointly appointed by the Departments of Statistics and of Biostatistics & Medical Informatics. His research interests include semiparametric statistics, the tradeoff between efficiency and robustness, domain adaptation and transfer learning, missing data analysis and causal inference, high-dimensional statistical inference. His work has been published in top-tier statistical journals as well as in leading machine learning conferences. His research has been consistently supported by the US National Science Foundation and the National Institutes of Health. Jiwei is now Associate Editor for Annals of Applied Statistics, JRSS Series A, Scandinavian Journal of Statistics, Journal of Nonparametric Statistics, and has been the Area Chair for the annual International Conference on Artificial Intelligence and Statistics (AISTATS) since 2024.
Welcome to join us to learn more about Dr. Zhao’s research work via Zoom!