ML talk of Professor Panagiotis (Panos) Patrinos (KU Leuven)
Event details
| Date | 10.09.2019 |
| Hour | 11:00 › 12:00 |
| Speaker | Professor Panagiotis (Panos) Patrinos (KU Leuven) |
| Location | |
| Category | Conferences - Seminars |
TITLE "Proximal envelopes for nonconvex splitting algorithms: Block coordinate and Newton-type variants"
ABSTRACT The classical understanding of splitting algorithms hinges on monotone operator theory, and thus heavily hinges on convexity. Alternatively, when applying these schemes to nonconvex problems convergence can be shown once a suitable merit function is identified; in other words, a function that decreases along the generated iterates. The challenge of convergence analysis of splitting algorithms in the nonconvex setting is thus the identification of a suitable merit function, and so far there doesn't seem to be a clear way as to how to construct one, let alone for randomized and/or block-coordinate variants. As a result, for many algorithms it is still unclear whether or not their application to nonconvex problems is feasible. In this talk we show how proximal envelopes provide a positive answer to this challenge, as they serve as suitable merit functions for many splitting algorithms. Moreover, thanks to their regularity properties they enable the possibility to robustify splitting algorithms by means of second-order-type information, stemming for instance from quasi-Newton schemes, without affecting global convergence. Block-coordinate (BC) variants of forward-backward splitting are also investigated for the minimization of the sum of a separable smooth function and a (nonseparable) nonsmooth function, both of which allowed to be nonconvex. Differently from classical study cases where it is the nonsmooth function to be separable, the cost cannot serve as merit function and as a result this setting is only little known. Once again we show how the "forward-backward envelope" serves as the suitable merit function, providing new convergence result for a large class of BC-type algorithms for nonsmooth and nonconvex problems with rather general sampling strategies, and that include the popular Finito/MISO algorithm as a special case.
BIO Panagiotis (Panos) Patrinos is assistant professor at the Department of Electrical Engineering (ESAT) of KU Leuven, Belgium. In 2014 he was a visiting professor at Stanford University. He received his PhD in Control and Optimization, M.S. in Applied Mathematics and M.Eng. from the National Technical University of Athens in 2010, 2005 and 2003, respectively. After his PhD he held postdoc positions at the University of Trento and IMT Lucca, Italy, where he became an assistant professor in 2012. His current research interests lie in the intersection of optimization control and learning. In particular he is interested in the theory and algorithms for structured convex and nonconvex optimization as well as learning-based, risk-averse model predictive control with a wide range of applications including autonomous vehicles, smart grids, water networks, aerospace, multi-agent systems, signal processing and machine learning.
ABSTRACT The classical understanding of splitting algorithms hinges on monotone operator theory, and thus heavily hinges on convexity. Alternatively, when applying these schemes to nonconvex problems convergence can be shown once a suitable merit function is identified; in other words, a function that decreases along the generated iterates. The challenge of convergence analysis of splitting algorithms in the nonconvex setting is thus the identification of a suitable merit function, and so far there doesn't seem to be a clear way as to how to construct one, let alone for randomized and/or block-coordinate variants. As a result, for many algorithms it is still unclear whether or not their application to nonconvex problems is feasible. In this talk we show how proximal envelopes provide a positive answer to this challenge, as they serve as suitable merit functions for many splitting algorithms. Moreover, thanks to their regularity properties they enable the possibility to robustify splitting algorithms by means of second-order-type information, stemming for instance from quasi-Newton schemes, without affecting global convergence. Block-coordinate (BC) variants of forward-backward splitting are also investigated for the minimization of the sum of a separable smooth function and a (nonseparable) nonsmooth function, both of which allowed to be nonconvex. Differently from classical study cases where it is the nonsmooth function to be separable, the cost cannot serve as merit function and as a result this setting is only little known. Once again we show how the "forward-backward envelope" serves as the suitable merit function, providing new convergence result for a large class of BC-type algorithms for nonsmooth and nonconvex problems with rather general sampling strategies, and that include the popular Finito/MISO algorithm as a special case.
BIO Panagiotis (Panos) Patrinos is assistant professor at the Department of Electrical Engineering (ESAT) of KU Leuven, Belgium. In 2014 he was a visiting professor at Stanford University. He received his PhD in Control and Optimization, M.S. in Applied Mathematics and M.Eng. from the National Technical University of Athens in 2010, 2005 and 2003, respectively. After his PhD he held postdoc positions at the University of Trento and IMT Lucca, Italy, where he became an assistant professor in 2012. His current research interests lie in the intersection of optimization control and learning. In particular he is interested in the theory and algorithms for structured convex and nonconvex optimization as well as learning-based, risk-averse model predictive control with a wide range of applications including autonomous vehicles, smart grids, water networks, aerospace, multi-agent systems, signal processing and machine learning.
Practical information
- Expert
- Free
Organizer
- Professor Volkan Cevher
Contact
- Gosia Baltaian