BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Memento EPFL//
BEGIN:VEVENT
SUMMARY:Theory of Neural Nets Seminar: 14th June 2021
DTSTART:20210614T163000
DTEND:20210614T173000
DTSTAMP:20260506T144120Z
UID:bed4bb1c6eb0b115c137f543b7dec61e0f7fd5c3e12770179a1b1be2
CATEGORIES:Conferences - Seminars
DESCRIPTION:Stephan Wojtowytsch (Princeton University)\nThis seminar consi
 sts of talks about current research on the theory of neural networks. Ever
 y session lasts one hour and comprises a talk (about 30 minutes) followed 
 by a discussion with questions from the audience.\n\nSpeaker: Stephan Woj
 towytsch (Princeton University)\n\nTitle: Stochastic gradient descent for
  noise with ML-type scaling\n\nAbstract: In the literature on stochastic 
 gradient descent\, there are two types of convergence results: (1) SGD fin
 ds minimizers of convex objective functions and (2) SGD finds critical poi
 nts of smooth objective functions. Classical results are obtained under th
 e assumption that the stochastic noise is L^2-bounded and that the learnin
 g rate decays to zero at a suitable speed. We show that\, if the objective
  landscape and noise possess certain properties which are reminiscent of d
 eep learning problems\, then we can obtain global convergence guarantees o
 f first type under second type assumptions for a fixed (small\, but positi
 ve) learning rate. The convergence is exponential\, but with a large rando
 m coefficient. If the learning rate exceeds a certain threshold\, we discu
 ss minimum selection by studying the invariant distribution of a continuou
 s time SGD model. We show that at a critical threshold\, SGD prefers minim
 izers where the objective function is ‘flat’ in a precise sense.
LOCATION:https://epfl.zoom.us/j/61671903666?pwd=TXNQNlZvU1F6ejVLV0VZOW92Vl
 R5dz09
STATUS:CONFIRMED
END:VEVENT
END:VCALENDAR