Learning with Guarantees: Neural Feedback Controllers with Built-in Stability and Robustness Certificates

Learning-based control strategies such as deep reinforcement learning (RL) and imitation learning (IL) are powerful tools for general-purpose robotic control. They rely on simple, gradient-based optimisation schemes, and typically parameterise control policies with black-box deep neural networks, which are known to be universal approximators for nonlinear systems. However, black-box approaches to feedback control suffer from a fundamental lack of closed-loop stability and robustness. In this talk, we consider the problem of learning stabilising neural policies with built-in stability and robustness guarantees for nonlinear systems. Our approach leverages recent developments in robust neural networks, which are neural networks that automatically satisfy internal stability and robustness properties of their own. We present two novel approaches: (1) parametrising control policies with robust neural networks to improve their empirical robustness in open loop; and (2) parametrising control policies by combining robust Recurrent Equilibrium Networks (RENs) with a nonlinear version of the classical Youla-Kucera parametrisation to achieve closed-loop stability (contraction) and robustness (Lipschitz) guarantees. Our resulting "Youla-REN" policy class is parametrised to automatically satisfy these closed-loop guarantees, making it compatible with standard gradient-based optimisation pipelines deep RL/IL. We discuss our recent theoretical results, provide illustrative numerical examples, and pose directions for future research on the application of robust learning-based control in real-world robotic systems.

Nicholas Barbara is a PhD candidate at the Australian Centre for Robotics, within the University of Sydney. He received his Bachelor of Engineering (Hons 1, University Medal) and Bachelor of Science from the University of Sydney, Australia, in 2021. His current research involves developing new tools to learn control policies with stability and robustness guarantees for robotic systems. His research interests include learning-based control, robust machine learning, robotics, and spacecraft GNC.