Robust Estimation via Robust Gradient Estimation

Prof. Volkan Cevher at the Laboratory for Information and Inference Systems (LIONS) invites you to the following talk: Robust Estimation via Robust Gradient Estimation by Pradeep Ravikumar,  Professor at School of Computer Science, Carnegie Mellon University.

ABSTRACT
A common assumption in the training of machine learning systems is that the data is sufficiently clean and well-behaved: there are very few or no outliers, or that the distribution of the data does not have very long tails. As machine learning finds wider usage, these assumptions are increasingly indefensible. The key question then is how to perform estimation that is robust to departure from these assumptions; and which has been of classical interest, with seminal contributions due to Box, Tukey, Huber, Hampel, and several others. Loosely, there seemed to be a computation-robustness tradeoff, practical estimators did have strong robustness guarantees, while estimators with strong robustness guarantees were computationally impractical.
 
In our work, we provide a new class of computationally-efficient class of estimators for risk minimization that are provably robust to a variety of robustness settings, such as arbitrary oblivious contamination, and heavy-tailed data, among others. Our workhorse is a novel robust variant of gradient descent, and we provide conditions under which our gradient descent variant provides accurate and robust estimators in any general convex risk minimization problem. These results provide some of the first computationally tractable and provably robust estimators for general statistical models.
 
Joint work with Adarsh Prasad, Arun Sai Suggala, Sivaraman Balakrishnan.

Bio:
Pradeep Ravikumar is an Associate Professor in the Machine Learning Department, School of Computer Science at Carnegie Mellon University. His thesis has received honorable mentions in the ACM SIGKDD Dissertation award and the CMU School of Computer Science Distinguished Dissertation award. He is a Sloan Fellow, a Siebel Scholar, a recipient of the NSF CAREER Award, and was Program Chair for the International Conference on Artificial Intelligence and Statistics (AISTATS) in 2013.