AI Center Seminar - AI Fundamentals series - Prof. David Blei

The talk is jointly organized by the EPFL Foundations of Learning and AI Research (FLAIR) group and the EPFL AI Center.

Title
Scaling and Generalizing Approximate Bayesian Inference

Abstract
A core problem in statistics and machine learning is to approximate difficult-to-compute probability distributions. This problem is especially important in Bayesian statistics, which frames all inference about unknown quantities as a calculation about a conditional distribution. In this talk I review and discuss innovations in variational inference (VI), a method that approximates probability distributions through optimization. VI has been used in myriad applications in machine learning and Bayesian statistics.

After quickly reviewing the basics, I will discuss two lines of research in VI. I first describe stochastic variational inference, an approximate inference algorithm for handling massive datasets, and demonstrate its application to probabilistic topic models of millions of articles. Then I discuss black box variational inference, a more generic algorithm for approximating the posterior. Black box inference  applies to many models but requires minimal mathematical work to implement. I will demonstrate black box inference on deep exponential families---a method for Bayesian deep learning---and describe how it enables powerful tools for probabilistic programming. Finally, I will highlight some more recent results in variational inference, including statistical theory, score-based objective functions, and interpolating between mean-field and fully dependent variational families.

Bio
David Blei joined Columbia in Fall 2014 as a Professor of Computer Science and Statistics. His research involves probabilistic topic models, Bayesian nonparametric methods, and approximate posterior inference. He works on a variety of applications, including text, images, music, social networks, user behavior, and scientific data. Professor Blei earned his Bachelor’s degree in Computer Science and Mathematics from Brown University (1997) and his PhD in Computer Science from the University of California, Berkeley (2004). Before arriving to Columbia, he was an Associate Professor of Computer Science at Princeton University. He has received several awards for his research, including a Sloan Fellowship (2010), Office of Naval Research Young Investigator Award (2011), Presidential Early Career Award for Scientists and Engineers (2011), and Blavatnik Faculty Award (2013).