Differential Privacy to Machine Learning, and Back- Part II
Event details
| Date | 27.11.2014 |
| Hour | 10:00 › 12:00 |
| Speaker |
Abhradeep Guha Thakurta, Yahoo! Labs |
| Location |
INM 200
|
| Category | Conferences - Seminars |
In this tutorial, I will outline some of the recent developments in the area of differentially private machine learning, which exhibit strong connections between statistical data privacy and robust (stable) machine learning. The main theme of this tutorial would be to demonstrate techniques to achieve one from the other.
In the first part, I will first review some of the high-profile privacy breaches (and also outline some the underlying principles behind them) to motivate the need for a rigorous notion of statistical data privacy. Subsequently, I will move on to introduce the notion of differential privacy, and present some of the basic concepts commonly used in the design of differentially private algorithms. I will conclude part one by showing that differential privacy is a strong form of regularization (a common technique used in machine learning to control prediction error), and analyze the Follow-the-perturbed-leader (FTPL) by Kalai and Vempala' 2005 in this context.
In the second part, I will analyze the robustness (stability) properties of the some of the commonly used machine learning algorithms (e.g., gradient descent, LASSO, and, Frank-Wolfe algorithm) and bootstrap the robustness properties into obtaining optimal differentially private algorithms for empirical risk minimization, and model selection. Some of the robustness analyses are new, and can be of independent interest irrespective of privacy.
Short Bio:
Abhradeep is currently a research scientist in the Systems research group at Yahoo! Labs Sunnyvale. Prior to that he was a post-doctoral researcher at Stanford University and Microsoft Research Silicon Valley Campus. He did his PhD from Pennsylvania State University. He is primarily interested in statistical data privacy and its relation to robust machine learning and data mining. More precisely, he studies the privacy and robustness implications in problems spanning the areas of high-dimensional statistics, risk minimization, online learning and pattern mining. Recently, he been also interested in studying the interrelation between various privacy notions and their practical implications, and robust property testing.
In the first part, I will first review some of the high-profile privacy breaches (and also outline some the underlying principles behind them) to motivate the need for a rigorous notion of statistical data privacy. Subsequently, I will move on to introduce the notion of differential privacy, and present some of the basic concepts commonly used in the design of differentially private algorithms. I will conclude part one by showing that differential privacy is a strong form of regularization (a common technique used in machine learning to control prediction error), and analyze the Follow-the-perturbed-leader (FTPL) by Kalai and Vempala' 2005 in this context.
In the second part, I will analyze the robustness (stability) properties of the some of the commonly used machine learning algorithms (e.g., gradient descent, LASSO, and, Frank-Wolfe algorithm) and bootstrap the robustness properties into obtaining optimal differentially private algorithms for empirical risk minimization, and model selection. Some of the robustness analyses are new, and can be of independent interest irrespective of privacy.
Short Bio:
Abhradeep is currently a research scientist in the Systems research group at Yahoo! Labs Sunnyvale. Prior to that he was a post-doctoral researcher at Stanford University and Microsoft Research Silicon Valley Campus. He did his PhD from Pennsylvania State University. He is primarily interested in statistical data privacy and its relation to robust machine learning and data mining. More precisely, he studies the privacy and robustness implications in problems spanning the areas of high-dimensional statistics, risk minimization, online learning and pattern mining. Recently, he been also interested in studying the interrelation between various privacy notions and their practical implications, and robust property testing.
Practical information
- Informed public
- Free
Organizer
- Nisheeth Vishnoi