AI Center Seminar - AI Fundamentals series - Dr. Pierre Marion

The talk is jointly organized by the EPFL AI Center and the EPFL Statistical Physics of Computation Lab (SPOC) as part of the AI Fundamentals seminar series.

Host: Prof. Lenka Zdeborova 

Title 
Edge Flow: A Tractable and Predictive Continuous-Time Model for Gradient Descent at the Edge of Stability

Abstract
Gradient descent in deep learning may operate at the edge of stability (EoS), a regime in which the largest eigenvalue of the loss Hessian hovers near the stability threshold 2/eta, where eta is the learning rate. Classical analysis tools such as gradient flow and the descent lemma do not apply here, motivating the search for a continuous-time model valid at EoS. We propose Edge Flow, a system of three coupled ordinary differential equations that provides a tractable, faithful, and predictive model of gradient descent dynamics at EoS. Discretizing Edge Flow only requires two gradient evaluations and one Hessian-vector product at each iteration. We demonstrate empirically that Edge Flow tracks the dynamics of gradient descent at least as faithfully as previously proposed continuous-time EoS models, while in addition resolving the oscillation of the sharpness at the onset of EoS, and that it provides a principled framework for understanding and mitigating instabilities in this regime. Based on https://arxiv.org/abs/2606.18080.

Bio
Pierre Marion is a research faculty at INRIA within the SIERRA team, which is a joint team between CNRS, Ecole Normale Supérieure and INRIA. He was previously in 2024-25 a postdoctoral researcher at EPFL under the supervision of Lénaïc Chizat, and before that a PhD student at Sorbonne Université under the supervision of Gérard Biau and Jean-Philippe Vert. His research interests regard the theory of deep learning with an emphasis on generative models and optimization dynamics of neural networks, as well as AI for maths and reasoning.