BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Memento EPFL//
BEGIN:VEVENT
SUMMARY:Theory of Neural Nets Seminar: 21st June 2021
DTSTART:20210621T163000
DTEND:20210621T173000
DTSTAMP:20260501T092724Z
UID:de6651df4cacadebe7117fc4fef2cf85a4badba396be90e8863542cb
CATEGORIES:Conferences - Seminars
DESCRIPTION:Marco Mondelli (IST Austria)\nThis seminar consists of talks a
 bout current research on the theory of neural networks. Every session last
 s one hour and comprises a talk (about 30 minutes) followed by a discussio
 n with questions from the audience.\n\nSpeaker: Marco Mondelli (IST Austr
 ia)\n\nTitle: Mode Connectivity and Convergence of Gradient Descent for (
 Not So) Over-parameterized Deep Neural Networks\n\nAbstract: Training a n
 eural network is a non-convex problem that exhibits spurious and disconnec
 ted local minima. Yet\, in practice neural networks with millions of param
 eters are successfully optimized using gradient descent methods. In this t
 alk\, I will give some theoretical insights on why this is possible. In th
 e first part\, I will focus on the problem of finding low-loss paths betwe
 en the solutions found by gradient descent. First\, using mean-field techn
 iques\, I will prove that\, as the number of neurons grows\, gradient desc
 ent solutions are approximately dropout-stable and\, hence\, connected. Th
 en\, I will present a mild condition that trades off the overparameterizat
 ion with the quality of the features. In the second part\, I will describe
  some tools to prove convergence of gradient descent to global optimality:
  the displacement convexity of a related Wasserstein gradient flow\, and b
 ounds on the smallest eigenvalue of neural tangent kernel matrices.  \n[
 Based on joint works with Pierre Brechet\, Adel Javanmard\, Andrea Montana
 ri\, Guido Montufar\, Quynh Nguyen\, and Alexander Shevchenko]
LOCATION:https://epfl.zoom.us/j/69799341099
STATUS:CONFIRMED
END:VEVENT
END:VCALENDAR