MechE Colloquium: Landscape and generalisation in deep learning

If you would like to attend the talk in BM 5202, please register here (on a first-come, first-served basis). This allows us to limit the number of people in the room and to satisfy contact tracing requirements.

For remote attendance: Zoom link


Abstract:
Deep learning is very powerful at a variety of tasks, including self-driving cars and playing go beyond human level. Despite these engineering successes, why deep learning works remains unclear; a question with many facets.  I will discuss two of them: (i) Deep learning is a fitting procedure, achieved by defining a loss function which is high when data are poorly fitted.  Learning corresponds to a descent in the loss landscape. Why isn’t it stuck in bad local minima, as occurs when cooling glassy systems in physics? What is the geometry of the loss landscape? (ii) in recent years it has been realised that deep learning works best in the over-parametrised regime, where the number of fitting parameters is much larger than the number of data to be fitted,  contrarily to intuition and to usual views in statistics.  I will propose a resolution of these two problems, based on both an analogy with the energy landscape of repulsive particles and an analysis of asymptotically wide nets.

Bio:
Matthieu Wyart is Associate Professor in the Institute of Physics at EPFL in Switzerland. He received his B.A. at École Polytechnique and obtained his Ph.D. degree from CEA, Saclay. He was a George Carrier Fellow at Harvard University before joining the Physics Department at NYU in 2010. He became Associate Professor in 2014 and moved to EPFL in 2015.
One focus of Wyart's work is the classification of the elementary excitations controlling the linear and the plastic response in amorphous materials. He introduced the notion that some of these excitations are marginally stable in the solid phase. Such marginality fixes key aspects of the structure, and implies that the density of excitations presents a pseudo-gap. These notions are important to describe the low-temperature properties of glasses, the elasticity near the jamming transition, the rheology of dense granular and suspension flows, the yielding transition in foams or metallic glass, and more generally glassy systems with sufficiently long-range interactions. Wyart's work has also focused on the neuronal circuit of simple organisms. He received a Sloan Fellowship in 2011 and became a Simons Investigator in 2015.