Understanding Generalization in Adaptive Data Analysis

Datasets are often reused to perform multiple statistical analyses in an adaptive way, in which each analysis may depend on the outcomes of previous analyses on the same dataset. Standard statistical guarantees do not account for these dependencies and little is known about how to provably avoid overfitting in the adaptive setting. In this talk I'll describe a new framework to address this problem, an approach based on differential privacy, and several algorithms based on this approach. Based on joint works with Dwork, Hardt, Pitassi, Reingold and Roth (STOC 2015, NIPS 2015) and with Steinke (COLT 2017)