Data-driven Analysis of First-Order Methods via Distributionally Robust Optimization

Abstract
We consider the problem of analyzing the probabilistic performance of first-order methods when solving convex optimization problems drawn from an unknown distribution only accessible through samples. By combining performance estimation (PEP) and Wasserstein distributionally robust optimization (DRO), we formulate the analysis as a tractable semidefinite program. Our approach unifies worst-case and average-case analyses by incorporating data-driven information from the observed convergence of first-order methods on a limited number of problem instances. This yields probabilistic, data-driven performance guarantees in terms of the expectation or conditional value-at-risk of the selected performance metric. Experiments on smooth convex minimization, logistics regression, and Lasso show that our method significantly reduces the conservatism of classical worst-case bounds and narrows the gap between theoretical and empirical performance.

Biography 
Assistant Professor at the Department of Operations Research and Financial Engineering at Princeton University. He is associated faculty in the Department of Electrical and Computer Engineering and the Department of Computer Science, and an affiliated faculty in the AI at Princeton initiative, the Princeton Institute for Computational Science and Engineering, the Center for Statistics and Machine Learning, and the Robotics at Princeton initiative. Prof Stellato is also a fellow at the Princeton Whitman and Yeh colleges.
His research lies at the interface of mathematical optimization, machine learning, and optimal control. It focuses on data-driven computational tools to make decisions in highly dynamic and uncertain environments.